GB2624844A - UE measurement capability indication for AI/ML dataset construction - Google Patents

Abstract

A method of a mobile terminal for obtaining measurement data for AI/ML model training in a 5G NR communications system. The system comprises one or more network entities 104 and one or more mobile terminals UE 102. The method comprises: transmitting, from the mobile terminal to a network entity, information on one or more measurement capabilities 106 of the mobile terminal with respect to data for AI/ML model training; measuring, at the mobile terminal, one or more values associated with the data for AI/ML model training; and transmitting, from the mobile terminal to the network entity 108, the measured values. Different user equipment UE may have differing measurement capabilities which may impact training of AI/ML models used to determine network policies. Notification of UE measurement capabilities may mitigate the impact of differing measurement capabilities on the AI/ML model.

Description

UE MEASUREMENT CAPABILITY INDICATION FOR Al/ML DATASET

CONSTRUCTION

FIELD OF THE DISCLOSURE

[0001] The present disclosure relates to UE measurement capability indications for Al/ML dataset construction in 5G NR systems, and in particular methods and apparatus for taking into account UE measurement capabilities when using UE measurement data in Al/ML training datasets

BACKGROUND OF THE DISCLOSURE

[0002] Wireless or mobile (cellular) communications networks in which a mobile terminal (UE, such as a mobile handset) communicates via a radio link with a network of base stations, or other wireless access points or nodes, have undergone rapid development through a number of generations. The 3Generation Partnership Project (3GPP) design, specify and standardise technologies for mobile wireless communication networks. Fourth Generation (4G) and Fifth Generation (5G) systems are now widely deployed.

[0003] 3GPP standards for 4G systems include an Evolved Packet Core (EPC) and an Enhanced-UTRAN (E-UTRAN: an Enhanced Universal Terrestrial Radio Access Network). The E-UTRAN uses Long Term Evolution (LTE) radio technology. LTE is commonly used to refer to the whole system including both the EPC and the E-UTRAN, and LTE is used in this sense in the remainder of this document. LTE should also be taken to include LTE enhancements such as LTE Advanced and LTE Pro, which offer enhanced data rates compared to LTE.

[0004] In 5G systems a new air interface has been developed, which may be referred to as 5G New Radio (5G NR) or simply NR. NR is designed to support the wide variety of services and use case scenarios envisaged for 5G networks, though builds upon established LTE technologies. New frameworks and architectures are also being developed as part of 5G networks in order to increase the range of functionality and use cases available through 5G networks. One such new framework is the use of Artificial Intelligence/Machine Learning (Al/ML) for the optimisation of the operation of 5G networks. However, Al/ML is reliant on training data, and therefore the quality of an Al/ML model is reliant on the quality of the training data on which it is based.

[0005] More specifically, Al/ML requires a large amount of data to train the models before applying them in real time (or near real time) as solutions on the ground. The accuracy and the effectiveness of these Al/ML solutions largely depend on the quality of the training data.

[0006] In 5G systems training/learning of Al/ML can be performed at the UE and/or at the network. For example, training may be completed or mostly completed at the UE, which is termed as Federated Learning. Alternatively, training may be fully centralized in the network (including the gNBs). Hybrid models of the above two variants also exist. However, regardless of the specific training/learning model used, the quality of the training data can have a significant impact on the performance of the Al/ML model i.e. the quality/accuracy of the inferences output by the Al/ML model.

[0007] The content of the following documents is referred to below and/or their content provides useful background information that the following disclosure should be considered in the context of: -RP-213599, Study on Artificial Intelligence (AI)/Machine Learning (ML) for NR Air Interface R2-2206509, Corrections to stage 2 for NR NTN.

-3GPP IS 38.331 v17.1.0.

BRIEF SUMMARY OF THE DISCLOSURE

[0008] It is an aim of certain examples of the present disclosure to provide approaches for taking into account UE measurement capabilities that may affect training data quality when UE measurement data is used in AI/ML training datasets.

[0009] The present invention is defined in the independent claims. Further features associated with the present invention are defined in the dependent claims.

BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 provides a schematic diagram of a UE capability message and its contents in accordance with an example of the present disclosure.

[0011] Figure 2 provides a schematic diagram of a UE measurement report message and its contents in accordance with an example of the present disclosure.

[0012] Figure 3 provides a schematic diagram of a network entity in accordance with an example of the present disclosure.

DETAILED DESCRIPTION

[0013] Examples in accordance with the present disclosure will now be described in the context of a 5G wireless communication network comprising at least one or more mobile terminals (UEs), one or more base stations (gNB) or RAN, and a core network. The 5G system may also be considered to be formed from one or more mobile terminals and the network, where the network may comprise one or more network entities (e.g. gNB, AMF, CN etc.). However, it will be understood that the present disclosure is not limited to only 53 system but may be applied to other wireless communication systems in which satellite communications are available. Consequently, references to particular 30PP constructs in certain examples should not be understood as limiting the ability of examples of the present disclosure to be applied to other wireless communication networks.

[0014] The use of Artificial Intelligence/Machine Learning (Al/ML) for the New Radio (NR) air interface has been outlined in RP-213599: Study on Artificial Intelligence (AI)/Machine Learning (ML) for NR Air Interface, in which the following is set out.

Study the 3GPP framework for Al/ML for air-interface corresponding to each target use case regarding aspects such as performance, complexity, and potential specification impact Use cases to focus on: - Initial set of use cases includes: a CSI feedback enhancement, e.g., overhead reduction, improved accuracy, prediction [RANI] o Beam management, e.g., beam prediction in time, and/or spatial domain for overhead and latency reduction, beam selection accuracy improvement [RANI] a Positioning accuracy enhancements for different scenarios including, e.g., those with heavy NLOS conditions [RANI] - Finalize representative sub use cases for each use case for characterization and baseline performance evaluations by RAN#98 * The Al/ML approaches for the selected sub use cases need to be diverse enough to support various requirements on the g NB-UE collaboration levels f...1 Al/ML model, terminology and description to identify common and specific characteristics for framework investigations: - Characterize the defining stages of Al/ML related algorithms and associated complexity: ^ Model generation, e.g., model training (including input/output, pre-/post-process, online/offline as applicable), model validation, model testing, as applicable o Inference operation, e.g., input/output, pre-/post-process, as applicable -Identify various levels of collaboration between UE and g NB pertinent to the selected use cases, e.g., o No collaboration: implementation-based only Al/ML algorithms without information exchange [for comparison purposes] o Various levels of UE/gNB collaboration targeting at separate or joint ML operation.

-Characterize lifecycle management of Al/ML model: e.g., model training, model deployment, model inference, model monitoring, model updating - Dataset(s) for training, validation, testing, and inference - Identify common notation and terminology for Al/ML related functions, procedures and interfaces -Note: Consider the work done for FS NR ENDC data_collect when appropriate For the use cases under consideration: 1) Evaluate performance benefits of Al/ML based algorithms for the agreed use cases in the final representative set: o Methodology based on statistical models (from TR 38.901 and TR 38.857 [positioning]), for link and system level simulations.

* Extensions of 3GPP evaluation methodology for better suitability to Al/ML based techniques should be considered as needed.

* Whether field data are optionally needed to further assess the performance and robustness in real-world environments should be discussed as part of the study * Need for common assumptions in dataset construction for training, validation and test for the selected use cases.

* Consider adequate model training strategy collaboration levels and associated implications * Consider agreed-upon base Al model(s) for calibration * Al model description and training methodology used for evaluation should be reported for information and cross-checking purposes o KPls: Determine the common KPIs and corresponding requirements for the Al/ML operations. Determine the use-case specific KPls and benchmarks of the selected use-cases.

* Performance, inference latency and computational complexity of Al/ML based algorithms should be compared to that of a state-of-the-art baseline * Overhead, power consumption (including computational), memory storage, and hardware requirements (including for given processing delays) associated with enabling respective Al/ML scheme, as well as generalization capability should be considered.

2) Assess potential specification impact, specifically for the agreed use cases in the final representative set and fora common framework: * PHY layer aspects, e.g., (RANI) * Consider aspects related to, e.g., the potential specification of the Al Model lifecycle management, and dataset construction for training, validation and test for the selected use cases * Use case and collaboration level specific specification impact, such as new signalling, means for training and validation data assistance, assistance information, measurement, and feedback o Protocol aspects, e.g., (RAN2) -RAN2 only starts the work after there is sufficient progress on the use case study in RANI Consider aspects related to, e.g., capability indication, configuration and control procedures (training/inference), and management of data and Al/ML model, per RANI input * Collaboration level specific specification impact per use case * Interoperability and testability aspects, e.g., (RAN4) -RAN4 only starts the work after there is sufficient progress on use case study in RANI and RAN2 * Requirements and testing frameworks to validate Al/ML based performance enhancements and ensuring that UE and g NB with Al/ML meet or exceed the existing minimum requirements if applicable * Consider the need and implications for Al/ML processing capabilities definition [0015] From this study, the present disclose considers issues relevant to at least the following aspects in the context of UE measurements and Al/ML.

- Positioning accuracy enhancements for different scenarios including, e.g., those with heavy NLOS conditions - Dataset(s) for training, validation, testing, and inference - Need for common assumptions in dataset construction for training, validation and test for the selected use cases.

[0016] More specifically, currently in mobile and wireless communications, UE (or device/terminal/mobile device) measurement reports are used in determining network procedures that impact the given UE only, so the quality of the UE measurement is of little importance outside of the UE. However, when Al/ML is used, a vast amount of UE measurements, collected by different type of UEs, which may have different measurement capabilities and thus produce different quality measurements or measurements with different parameters, will be used to determine overall network policies and behaviours via their use in the training of the Al/ML models. Therefore, these differing measurement capabilities will impact other entities in the network, such as other UEs of network that make use of the Al/ML model trained on the UE measurement data. For example, Al/ML models trained using UE measurement data may be used for handover control and NLOS path detection and thus shortcomings in Al/ML models with have adverse effects on network performance. In the present disclosure, measurement capabilities or indications of measurement quality may include indications of measurement quality, accuracy, validity, and periodicity for example, where the measurements themselves may relate to one or more of signal strength (absolute or relative), signal quality, signal timings, UE location or any other measurement that may be made by a UE and potentially used for Al/ML training. For example, different UEs may have different radio-frequency (RF) reception chain qualities that will influence the accuracy of measurements made on signal strength/quality for example. Therefore, mixing measurement data from UEs with different measurement capabilities may lead to inconsistent Al/ML training datasets and thus potentially reduced quality inferences output by the Al/ML model.

[0017] A UE may collect relative or absolute measurements in certain scenarios and the measurement quality can impact Al/ML models using both these measurement types. For example, 'relative' signal strengths or signal quality (Reference Signal Received Power (RSRP) or Reference Signal Received Quality RSRQ measurements) from the serving and neighbour gNBs is considered when deciding on handover. Consequently, the actual handover point may differ based on the quality of the measurements provided by the relevant UEs, where different types of UEs may provide different measurement qualities. Therefore, when an Al/ML model is developed to predict the handover point, for example, having the training data set 'biased' towards a certain type of UE due to their measurement capabilities (where the mix of UEs will be different in the actual implementation) will impact the model negatively and thus the output of the model when used to control handover i.e. non-optimal or less optimal handover points. Similarly, 'Absolute' measurements of signal strength can be used, along with the UE location, to determine if a UE is receiving LOS or NLOS signals from the serving and neighbour gNBs. However, when such measurements are used in an Al/ML model, low quality signal strength measurements from some UEs will impact the model negatively, thus leading to poorly NLOS detection. Therefore, in an Al/ML model, high-quality signal strength measurements from some UEs will have to be taken account of and other measurements 'calibrated up' to reflect the actual measurement values in the training data. This will ensure that the Al/ML model training is not 'polluted' by certain low-quality measurements, in situations where the absolute value of the measurements are needed.

[0018] In summary, dataset(s) for training Al/ML models preferably need to be constructed based on highly accurate/ high quality measurement data in order to achieve accurate Al/ML operation (e.g. final inference). It is also desirable for the measurements used to form such datasets to have a consistent quality. However, a problem occurs when the measurements used to form the training dataset(s) are of different qualifies/have different parameters due to the capabilities or other behaviour of the UEs that collected them.

[0019] At present, there is no way for the network entity constructing the training dataset(s) for a given Al/ML model, to know whether the UE or group of UEs that belong to a given training session is/are capable of measuring the desired Al/ML data at a given measurement quality and/or accuracy level/threshold. Likewise, it is not currently possible for the network to determine the quality of measurement reports it receives. Consequently, it is not currently possible for the network to ensure a minimum quality level for training data and/or to account for or compensate for different measurement capabilities/qualities of UEs from which data for forming datasets is obtained. In other words, when an Al/ML relies on UE measurement data and different UEs are expected to provide such data, the impacts of different levels of measurement qualifies provided by each U E type needs to be accounted for.

[0020] Accordingly, the present disclosure is directed towards the problem of accounting for the impact that UE measurement capabilities have on the quality of training data for Al/ML, and in particular Al/ML models used in the air interface of 50 NR systems. More specifically, the present disclosure provides several solutions/methods to address the problem of possible training data pollution due to mixing measurements from a different UE types (i.e. UEs with different measurement capabilities/parameters).

Al/ML Training Dataset Calibration [0021] To take account of the impact of UE measurement capabilities and the quality/characteristics of the measurements they produce, the present disclosure focusses on the quality of the measurement data a UE (or a wireless device including sensor devices) provides for Artificial Intelligence (Al) or Machine Learning (ML) based solutions, and the use of dataset calibration using information on the data quality. For example, different makes (or categories) of devices will have different capabilities to measure signals (e.g. reference signals) transmitted from the gNBs and hence will report back data with different qualities. Different qualities may also result from different measurement parameters (e.g. accuracy, time period etc.) unrelated to the type of UE. This information on the different qualifies can be used to control UEs to perform measurements of a consistent quality and/or calibrate the resulting data (e.g. to a common level) so that adverse effects on the output of the Al/ML model due to inconsistent training data can be reduced. In particular, it is proposed that measurement quality/measurement capabilities are taken into account by the network by configuring UEs to report back their measurement capability or measurement qualifies to the network, and the network doing one or more of: configuring measurement rules/parameters to control the characteristics of the measurements performed by the UEs, adjusting received measurements or taking account of their differences before forming a training dataset or when the dataset is being used, and/or adjusting the outputs of the Al/ML model to account for the varying measurement qualifies. In other words, an indication of measurement capabilities/quality associated with UEs is provided to the network so that the network can carry out some form of calibration before measurement takes place and/or before or after the data is used to train Al/IVI models. In the context of the present application, training data may relate to one or more of a training a model/training session/model updates, such that training data may or may not fully train a model and/or may be partial training data. For example, pre-defined training sessions may be defined by the network in order to train or update Al/ML models. Furthermore, the term network refers to one or more entities on the network side of the 5G systems, such as the gNB, AMF, etc. [0022] More specifically, the proposed approaches are centred around reporting UE measurement capability to the network, before or along with the set of UE measurement data or training outcomes to be used in an Al/ML algorithm. The Al/ML algorithm can be Centralized, Federated, or hybrid, but reporting the UE measurement capability will be useful in all these cases. The AI/ML algorithm/model may be implemented by any suitable network entity and the collection of measurement capabilities, the configuring of measurement rules, the calibration of measurement data, and the aggregation of measurement data may be performed by any suitable network entity (e.g. gNB, AMF, other etc.). In a centralized algorithm where the training is done in the network, this UE measurement capability can be used to control measurement collection and/or calibrate the measurement datasets before using it in the training stage. In Federated or hybrid algorithms, where part or all of the training can be done in the UE/device, reporting the UE measurement capability can help the network to calibrate the training datasets(s) and/or training outcomes. In some cases the network may decide to discard the reported data set(s) considering the UE capability/measurement quality.

[00231 In accordance with an example of the present disclosure, the network ( including the gNB) may configure Al/ML related data measurements and/or measurements rules (e.g. measurement accuracy-level, reporting periodicity, number of attempts at a certain measurement etc. ) at a UE taking account of a UE's type (i.e. UE's Al/ML measurement capability). For example, the network may configure the: - Signal strength measurements of a serving cell and neighbour cells provided by UEs in Handover initiation scenarios.

- Signal strength measurements of a serving cell and neighbour cells provided by UEs at any location of a controlled network (like Industry loT). These measurements can be used for identifying the prevailing LOS/NLOS links at a given location.

[0024] In this example, the configuring of one or more UEs that are providing Al/ML measurements is performed prior to the collection of the actual measurements. Therefore, UEs can be configured to provide measurements of a predefined accuracy and at predefined times for example, so that the acquired measurement data is consistent or more consistent across the UEs from which measurements are gathered by the gNB. In other words, the measurement data received by the network has to at least some extent already been calibrated. However, approaches of the present disclosure alternatively or additionally encompass post-processing of measurement data in order to improve consistency between acquired measurement data before its use as training data. For example, such an approach may involve scaling of measurement data taking into account characteristics of the UE or otherwise appropriately weighting data based on UE characteristics.

[0025] The network may configure Al/ML measurements, measurement rules, and reporting at the UE and/or adjust received measurements based on the following: - UE's subscriber information retrieved, e.g. from UDM, that include information on UE's Al/ML measurement capability, if available.

- Network analytics, e.g. NWDAF, assistance information on statistics and/or predictions on UE's measurement trustability, accuracy (e.g. locations calculation accuracy), and/or other.

-Information on UE Al/ML measurement capability obtained directly from the UE or from any other NW internal or external entity or NW function.

[0026] UE measurement capabilities and indications thereof may take any appropriate form, for example, measurement capabilities may be broadly equated to UE type, measurement capabilities may be classified into predefined bands, or specific details (e.g. receiver sensitivity, interference suppression abilities etc., location accuracy) may be provided as measurement capabilities.

[0027] Configuration of UE measurements/measurement rules may be based on a desired measurement quality of the network, the measurement capabilities of the relevant UEs or a combination of these. For example, UE measurements may be configured based on the highest achievable quality given the UEs that are performing the measurements. Alternatively, measurements may be configured based on parameters such as power consumption or the quantity/quality of measurements required by the network. In yet another alternative, measurements may be configured to achieve a desired accuracy output from the Al/ML model. In other examples, if UEs with varying capabilities are present, the network may only configure those UEs which can provide higher quality measurements to perform measurements/provide measurement data for Al/ML purposes.

[0028] The network may also take a number of other approaches to the configuration of UE measurements/measurement rules. For example, the network may configure one or more measurement rules depending on the training models/sessions that a UE is part of; the network may configure/assign one or more measurement rules for a UE or group of UEs that have the same or similar Al/ML measurement capabilities; the network may configure the UE with different measurement rules depending on the required training data, training session, training model, training model updates, or other factors such as time of day, UE location, etc. (i.e. measurement characteristics required by the network).

[0029] Similarly, when post-measurement calibration is used either alone or in combination with pre-measurement configuration, the network may calibrate measurements (e.g. adjust, weight, discard etc.) based on the factors set out above, such as the desired accuracy of the output from the Al/ML models or a required consistency between measurements used to train the Al/ML models.

[0030] The appropriate configuration/calibration of UE measurements requires knowledge at the network of UE capabilities. Such knowledge may be obtained by the network (e.g. gNB, AMF, other network entity) when a UE is first registered with the network or may be obtained at any subsequent point in time, such as when measurements are provided to the network or when Al/ML models/parameters are provided to the network (i.e. in the case of federated or hybrid models). For example, the network may request information related to the UE Al/ML measurement capability from the UE, using a UE CAPABILITY TRANSFER procedure. In particular, if the UE supports Al/ML measurements and reporting, the UE may include the information on UE Al/ML measurement capability in a newly defined 1E, e.g. UECapabilityAl/ML-Meas IF or any other existing 1E, where this IF may be included in a UECapabilitylnformation message or other suitable message.

[0031] Figure 1 provides a message flow diagram illustrating example capability transfer messaging where the UE 102 sends information 108 on a UE Al/ML measurement capability to the network 104 or entity thereof (e.g. gNB, AM F, other etc.) in a UECapabilitylnformation message in response to a UECapabilityEnquity message 106. Such messaging may be performed periodically, when a UE first registers, when Al/ML measurements are requested by the network, or when measurements/Al/ML data is provided by the UE to the network when the UE is RRC_connected.

[0032] As noted above, UE capability information is required to be received at the network in order for some form of configuration to take place, whether this be pre or post measurement. In one example, a UE informs the network that it is capable of performing Al/ML measurements and provides information describing these measurements (e.g. measurement quality, accuracy, validity, periodicity, other). If the network has received such a UE capability, the network may configure the UE with suitable measurements (and/or measurements rules) or calibrate received measurements. With respect to the provision of capability information from a UE to the network in the context of Al/ML datasets, the following update may be made to 3GPP TS 38.331 v17.1.0 UE-NR-Capability information element ASNiSTART TAG UE NR CAFABILI7Y START UE NR Capability v1800:- SEQUENCE { ai/m1MeasAndReport-rle ENUMERATED supported} CRTIONAL, ai/ml-MeasParameters-r18 AI/ML-MeasParameters-r18 OPTIONAL, UE-NR-Capability information element ai/m/MeasAndReport-r18 The field indicates that the UE supports Al/ML data measurements and Al/ML data measurement reporting. ailml-MeasParameters-r18 The field provides information related to the type and features of Al/ML measurements (e.g. measurements quality, accuracy, validity time and location, other). The field is optionally present. Otherwise, it is absent.

[0033] In some examples, a UE may send to the network information on existing measurements rules (e.g. previously preconfigured). The network may check and update measurement rules if needed.

[0034] Alternatively, the UE may send to the network assistance information (i.e. measurement parameters or characteristics of measurements) on performed Al/ML measurements together (or in a separate newly defined or existing signaling /messages). For example, the UE may send ai/ml-MeasParameters-r18 IF in a MeasurementReport message.

[0035] Figure 2 provides an illustration of this message exchange where a UE 202 sends to the network 204 or entity thereof (e.g. gNB, AMF, other etc.) a MeasurementReport message 206 including information on the parameters related to the measurements included in the reports, such as quality, validity, periodicity etc. [0036] A noted above, the nature of the UE measurement capability or quality may take many forms, for example, a limited number of categories for the UE measurement quality may be defined, and one of the category numbers (e.g. 1 to 6 or 1 to 10) reported back to the network. The categories may apply to all measurement types of a UE or different categories for different measurement types (e.g. RSRP or RSRQ). Alternatively, specific measurement parameters may be provided to the network, such as receiver sensitivity or interference suppression characteristics. In other examples, merely an indication of a UE category/type may be provided to the network and measurement characteristics of the UE inferred from the category/type.

[0037] With respect to measurement reporting from a UE to the network in the context of Al/ML datasets, the following highlighted fields from 3GPP IS 38.331 v17.1.0 may be used.

MeasurementReport The MeasurementReport message is used for the indication of measurement results.

Signalling radio bearer: SRB I SRB3 RLC-SAP: AM Logical channel: DCCH Direction: UE to Network MeasurementReport message MeasurementJeport criticalExtensions measuremencReporc criticalExonsionsFuturc MeasurementReporcilEs, Measurementaeport-:Es measResulcs MeasResulcs, Is eNcriCriLicalEx Lension nonCrtticalExtension The MeasurementReportAppLayer message is used for sending application layer measurement report.

Signalling radio bearer: SRB4 RLC-SAP: AM Logical channel: DCCH Direction: UE to Network MeasurementReportAppLayer message MeasurementaeportAppLayer-r17 criLdcalExLensions CNG.LC.:E: { maasuremencJepocAppLayer-r17 MeasuremeneportAppLayer-r17-IEs, criLicalEx_ensionsFuLure MeasuremenLReporpLayer-r17-IEs::= PNCE { measurerentRepertAppLayerLtsc r17 MeasurementReporcAppLayerList r17, _a_eNcuCiLLicalExLension nonCrticalExtension MeasurementReportAppLaverList-r17::= U..maxNrotAppLayerMeas-r17;) MeasReporuAppLayer-r17 appLaverSessionStanas-r17 LI fsarted, stopped] RAN-VIsIbleMeasurements-r17 appLayerBulkerLeve LisL-r17 _ppLayerBulLerLevel-r17 playoutDelavForMed±aStartup r17 IN (0..30000) pdu-SessionI L'sL-J,17 (1..maxNrciPDU-Sessicris-,,17)) 07 PDU-SessionID AppL::= 03..30000)

MeasurementReportAppLayer field descriptions

appLayerBufferLeyelList The field indicates a list of application layer buffer levels, and each AppLayerBufferLevel indicates the application layer buffer level in ms. Value 0 corresponds to Oms, value 1 corresponds to 10ms, value 2 corresponds to 20 ms and so on. If the buffer level is larger than the maximum value of 30000(5 minutes), the UE reports 30000.

appLayerSessionStatus Indicates that an application layer measurement session in the application layer starts or ends.

playoutDelayForMediaStartup Indicates the application layer playout delay for media start-up in ms. Value 0 corresponds to Urns, value 1 corresponds to 1ms, value 2 corresponds to 2 ms and so on. If the playout delay for media start-up is larger than the maximum value of 30000ms, the UE reports 30000.

measRepottAppLayerContainer The field contains application layer measurement report, see Annex L (normative) in TS 26.247 [68], clause 16.5 in TS 26.114 [69] and TS 26.118 [70].

pdu-SessionldList Contains the identity of the PDU session, or the identities of the PDU sessions, used for application data flows subject to the RAN visible application layer measurements.

[0038] VVith respect to measurements that have been obtained by UEs with differing capabilities and thus have different associated qualifies (either due to no configuration of the measurement rules or configuration of the measurement rules has only partially reduced differences in measurement quality), a number of approaches may be taken account for these variability between the measurements. For example, measurements from each UE and type of UE may be adjusted/weighted/extrapolated etc. based on their measurement capabilities/parameters under which the measurements were collected.

[0039] The network may also take other approaches to handling measurement data/values from UEs, for example, the network may only consider reported measurements from one or more UEs (either pre-determined UEs or determined on measurement characteristics); after receiving UE Al/ML measurement data/reports and/or UE Al/ML measurement capability, the network may indicate (e.g. via further measurement configuration information) to the UE (or group of UEs) to stop measurement (e.g. if the network has collected enough measurement data for the desired training model/session/model updates) and/or stop reporting measurement and/or repeat measurements (e.g. at a different quality, accuracy, or using a different measurement method(s)); the network may discard the reported UE or UEs Al/ML measurements if not provided according to the preconfigured measurement rules or if the network has collected enough measurement data for the desired training model/session/model updates; and the network may only consider Al/ML measurements reported from a given UE or set of UEs and ignore /discard measurements from other UEs (e.g. UEs in a given training session). The network may also implement any combination of these examples.

[0040] Alternatively, or additionally, Al/ML models constructed from such variable measurements may be considered to be only fully applicable to UEs with a corresponding measurement capability. For example, in a situation like handover optimization, where 'relative' signal strengths are the key, the problem happens when the mix of differently capable UEs providing the training data is different to the mix of UEs in the implementation of the solution. As an example, considering 2 broad categories of reported measurements from UEs of category A and B -if the training data contains 50% each of A and B type UEs and then in a given situation of actual implementation of the Al/ML solution there is a mix of 80% of A and 20% of B type UEs -there will be a mismatch. However, such drawbacks may be addressed by the network by identifying the % mix of different UE categories in the training data set and in the actual implementation, and then applying appropriate weightings to data from different types of UEs in the training data set. For this to happen, the UE measurement capability or type may be reported back to the network, and the UEs categorized (possibly to bands) based on their measurement capability. However, approaches to the post-collection adjustment of measurements may be configured to have a higher complexity or lower complexity dependent on the desired level of consistency between the resulting adjusted measurements.

[0041] In some situations, like the NLOS path identification, the absolute values of signal strengths matter. If two types of UEs (A and B) report different signal strengths and A value is much closer to the actual measurement (identified through the measurement quality of the UE type), the network can calibrate (up) the measurements received through type B UEs and then use this calibration both in training and in actual implementation. For this solution also, the measurement category/characteristics has to be reported back to the network and the network has to make an assessment on the level of calibration to apply, depending on the best quality UE measurements it will receive.

Handover Optimisation [0042] The traditional handover procedure can be termed as reactive, the serving gNB responds to decreasing measurement reports from a UE and when this falls below a threshold, the handover procedures are initiated. The Reference Secondary Synchronisation Signal Received Quality (SS-RSRQ) or Channel State Information (CSI)-RSRQ measurements in 5G can be used to initiate the handover procedures. In a single UE (non Al/ML) scenario, the handover for the UE needs to be determined as per the measurements reported by the said UE, so quality of the signal measurements does not significantly impact the decision. However in an Al/ML based predictive scenario, much finer details like sudden signal blockages or spikes of interference can be identified at a particular cell border and the handovers can be optimized considering these effects. In order to train these algorithms, RSRQ data from UEs/devices crossing a particular cell boarder can be accumulated, and thus the quality of such measurements is important.

[0043] The measurements of signal strengths and particularly the interference levels will depend on the quality of the RF chains in a particular device. In particular, interference can occur as co-channel and cross-channel and the quality of the out-of-band RE filters in the UE will determine how much cross-channel interference is captured by the UE/device. Hence some calibration of this measurements, when they are used as input data to train Al/ML algorithms, will be needed and for this reporting the quality of the UE measurements (as part of the UE capability) will be useful. Such calibration can be based on information on parameters of the UE (i.e. measurement capabilities) that have been provided to the network from the UE as described above. Alternatively, via the setting of measurement parameters/rules by the network based on knowledge of the UE capabilities, consistent interference level measurements may be obtained, reducing the need for further calibration of measurements received from multiple UEs.

NLOS Path Detection [0044] Dense and ultra-dense gNB deployments will be common for 5G-adv networks, in particular to support industry loT sites and urban hotspots. Precise localization will be a requirement in some of these networks, for example in Indoor factory (Industry loT) sites to locate moving objects and workers. One of the key requirements for precise localization is to have a sufficient number of Line of Sight (LOS) links to a particular UE under localization/tracking. In a dynamic environment, the LOS/Non LOS (NLOS) links at a given location will vary. Hence it will be very useful to have an Al/ML based solution to identify NLOS links at a given location, so they can be excluded from the usage in localization/tracking algorithms.

[0045] Such Al/ML solutions can use the Reference Signal Received Power (RSRP) measured and reported by a UE for the different gNBs (serving and neighbour) from a given location as training data. Again, the quality Al/ML model will be dependent on the quality of the reported measurements and the quality of the reported measurements will depend on the quality of the RE chains in a particular device. When data from multiple UEs/devices or sensors are used in such Al/ML algorithms, reporting the quality of the UE measurements (as part of the UE capability) will be useful and measurements configured/calibrated as set out above in order to improve the quality of training data for the Al/ML model.

[0046] Although the approaches set out above have specified particular types of measurements for specific purposes, the approaches of this disclosure are not limited to these. For example, the approaches may be applied to any form of data that is provided by a UE for Al/ML training purposes and which may be affected by characteristics of the UE, so that variability in a wide range of UE data can be compensated for and resulting datasets appropriately calibrated. Furthermore, the approaches set out above may be used alone or in combination with each other, for example, any combination of UE capability reporting, measurement rule adjustment, and measurement calibration may be used.

[0047] Further examples in accordance with the present disclosure are set out in the following numbered clauses, where these examples may be combined with one or more of the approaches set out above unless stated otherwise.

1. A method of a mobile terminal for obtaining measurement data for Al/ML model training in a 5G NR communications system comprising a base station and one or more mobile terminals, the method comprising: transmitting, from the mobile terminal to the base station, information on one or more measurement capabilities of the mobile terminal with respect to data for Al/ML model training; measuring, at the mobile terminal, one or more values associated with the data for Al/ML model training; and transmitting, from the mobile terminal to the base station, the measured values.

2. The method of clause 1, wherein the method further comprises receiving, at the mobile terminal from the base station, a measurement capability enquiry, and transmitting the information on one or more measurement capabilities of the mobile terminal with respect to data for Al/ML model training in response to the measurement capability enquiry.

3. The method of clauses 1 or 2, wherein the method further comprises receiving measurement configuration information from the base station, and the measuring of the one or more values associated with the data for Al/ML model training is based on the measurement configuration information.

4. The method of clause 3, wherein the measurement configuration information is based on the information on one or more measurement capabilities of the mobile terminal.

5. The method of clauses 3 or 4, wherein the measurement configuration information includes information on one or more of measurement quality, measurement accuracy, measurement validity, and measurement periodicity.

6. The method of clause 1, wherein the information on one or more measurement capabilities of the mobile terminal with respect to data for Al/ML model training is transmitted to the base station along with the measured values.

7. The method of any preceding clause, wherein the information on one or more measurement capabilities of the mobile terminal with respect to data for Al/ML model training includes information on one or more of mobile terminal type, mobile terminal category, measurement quality, measurement accuracy, measurement validity, measurement periodicity, and a number of attempts to measure a value (e.g. a reference signal).

8. The method of any preceding clause, wherein the one or more values include one or more of a location of the mobile terminal, a received signal strength, a received signal quality, and signal timing information.

9. A method of a base station for obtaining measurement data for Al/ML model training in a 5G NR communications system comprising the base station and one or more mobile terminals, the method comprising: receiving, from the mobile terminal at the base station, information on one or more measurement capabilities of the mobile terminal with respect to data for Al/ML model training; and receiving, from the mobile terminal at the base station, one or more values associated with the data for Al/ML model training measured by the mobile terminal.

10. The method of clause 9, wherein the method further comprises transmitting from the base station to the mobile terminal, measurement configuration information based on the information on one or more measurement capabilities of the mobile terminal.

11. The method of clause 10, wherein the measurement configuration information includes information on one or more of measurement quality, measurement accuracy, measurement validity, and measurement periodicity 12. The method of clauses 9 or 10, wherein the measurement configuration information is based on one or more of an Al/ML model or training session associated with the mobile terminal, a measurement characteristic required by the base station or other network entity, and measurement capabilities of at least one other mobile terminal served by the base station.

13. The method of clause 9, wherein the information on one or more measurement capabilities of the mobile terminal with respect to data for Al/ML model training is received by the base station or other network entity along with the measured values.

14. The method of any of clauses 9 to 13, wherein the information on one or more measurement capabilities of the mobile terminal with respect to data for Al/ML model training includes information on one or more of mobile terminal type, mobile terminal category, measurement quality, measurement accuracy, measurement validity, and measurement periodicity.

15. The method of clause 14, wherein the base station or other network entity adapts the received measured values based on the information on one or more measurement capabilities of the mobile terminal with respect to data for Al/ML model training 16. The method of any of clauses 9 to 15, wherein the one or more values include one or more of a location of the mobile terminal, a received signal strength, a received signal quality, and signal timing information.

17. The method of any of clauses 1 to 16, wherein the base station or other network entity trains an Al/ML model based on the received one or more values.

18. The method of any of clauses 9 to 17, wherein the base station or other network entity trains an Al/ML model based on the received one or more values if the mobile terminal is included in a predefined group of mobile terminals.

19. The method of clause 10, wherein base station or other network entity discards one or more of the received values if the one or more of the received values are not in accordance with the measurement configuration information.

20. The method of any of clauses 9 to 19, wherein the method further comprises, in response to receiving the one or more values, transmitting from the base station to the mobile terminal, an indication of one or more of: the mobile terminal should stop further measurements of values associated with the data for Al/ML model training, the mobile terminal should stop reporting measured values associated with the data for Al/ML model training, the mobile terminal should repeat the measurement of one or more values associated with the data for Al/ML model training, and the mobile should perform further measurement of one or more values associated with the data for Al/ML model training based on updated measurement configuration information.

21. A mobile terminal for a 5G communications systems, wherein the mobile terminal is configured to implement the method of any of clauses 1 to 8.

22. A base station for a 5G communications system, wherein the base station is configured to implement the method of any of clauses 9 to 20.

23. A 5G communications system comprising one or more mobile terminals, a base station, and other core network elements, wherein the 5G communications system is configured to implement the method of any of clauses 1 to 22.

[0048] A UE which is arranged to operate in accordance with any of the examples of the present disclosure described above includes a transmitter arranged to transmit signals to one or more RANs, including but not limited to a satellite network and a 3GPP RAN such as 5G NR network; a receiver arranged to receive signals from one or more RANs and other UEs; and a controller arranged to control the transmitter and receiver and to perform processing in accordance with the above described methods. The transmitter, receiver, and controller may be separate elements, but any single element or plurality of elements which provide equivalent functionality may be used to implement the examples of the present disclosure described above.

[0049] Figure 3 is a block diagram of an exemplary network entity that may be used in the implementation of the examples of the present disclosure. For example, the UE, entities of the network-side, core network or RAN (e.g. eNB, gNB or satellite) may be provided in the form of the network entity illustrated in Figure 3. The skilled person will appreciate that a network entity may be implemented, for example, as a network element on a dedicated hardware, as a software instance running on a dedicated hardware, and/or as a virtualised function instantiated on an appropriate platform, e.g. on a cloud infrastructure.

[0050] The entity 300 comprises a processor (or controller) 301, a transmitter 303 and a receiver 305. The receiver 305 is configured for receiving one or more messages from one or more other network entities, for example as described above. The transmitter 303 is configured for transmitting one or more messages to one or more other network entities, for example as described above. The processor 301 is configured for performing one or more operations, for example according to the operations as described above.

[0051] The techniques described herein may be implemented using any suitably configured apparatus and/or system. Such an apparatus and/or system may be configured to perform a method according to any aspect, embodiment, example or claim disclosed herein. Such an apparatus may comprise one or more elements, for example one or more of receivers, transmitters, transceivers, processors, controllers, modules, units, and the like, each element configured to perform one or more corresponding processes, operations and/or method steps for implementing the techniques described herein. For example, an operation/function of X may be performed by a module configured to perform X (or an X-module). The one or more elements may be implemented in the form of hardware, software, or any combination of hardware and software.

[0052] A particular network entity may be implemented as a network element on a dedicated hardware, as a software instance running on a dedicated hardware, and/or as a virtualised function instantiated on an appropriate platform, e.g. on a cloud infrastructure.

[0053] It will be appreciated that examples of the present disclosure may be implemented in the form of hardware, software or any combination of hardware and software. Any such software may be stored in the form of volatile or non-volatile storage, for example a storage device like a ROM, whether erasable or rewritable or not, or in the form of memory such as, for example, RAM, memory chips, device or integrated circuits or on an optically or magnetically readable medium such as, for example, a CD, DVD, magnetic disk or magnetic tape or the like.

[0054] It will be appreciated that the storage devices and storage media are embodiments of machine-readable storage that are suitable for storing a program or programs comprising instructions that, when executed, implement certain examples of the present disclosure. Accordingly, certain examples provide a program comprising code for implementing a method, apparatus or system according to any example, embodiment, aspect and/or claim disclosed herein, and/or a machine-readable storage storing such a program. Still further, such programs may be conveyed electronically via any medium, for example a communication signal carried over a wired or wireless connection.

[0055] Throughout the description and claims of this specification, the words "comprise" and "contain" and variations of them mean "including but not limited to", and they are not intended to (and do not) exclude other components, integers or steps. Throughout the description and claims of this specification, the singular encompasses the plural unless the context otherwise requires. In particular, where the indefinite article is used, the specification is to be understood as contemplating plurality as well as singularity, unless the context requires otherwise.

[0056] Features, integers or characteristics described in conjunction with a particular aspect, embodiment or example of the present disclosure are to be understood to be applicable to any other aspect, embodiment or example described herein unless incompatible therewith. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and/or all of the steps of any method or process so disclosed, may be combined in any combination, except combinations where at least some of such features and/or steps are mutually exclusive. The disclosure is not restricted to the details of any foregoing embodiments. Examples of the present disclosure extend to any novel one, or any novel combination, of the features disclosed in this specification (including any accompanying claims, abstract and drawings), or to any novel one, or any novel combination, of the steps of any method or process so disclosed.

[0057] The reader's attention is directed to all papers and documents which are filed concurrently with or previous to this specification in connection with this application and which are open to public inspection with this specification, and the contents of all such papers and documents are incorporated herein by reference.

[0058] The above embodiments are to be understood as illustrative examples of the present disclosure. Further embodiments are envisaged. It is to be understood that any feature described in relation to any one embodiment may be used alone, or in combination with other features described, and may also be used in combination with one or more features of any other of the embodiments, or any combination of any other of the embodiments. Furthermore, equivalents and modifications not described above may also be used without departing from the scope of the invention, which is defined in any accompanying claims.

Claims (14)

1. CLAIMS1. A method of a mobile terminal for obtaining measurement data for Al/ML model training in a 5G NR communications system comprising one or more network entities and one or more mobile terminals, the method comprising: transmitting, from the mobile terminal to a network entity, information on one or more measurement capabilities of the mobile terminal with respect to data for Al/ML model training; measuring, at the mobile terminal, one or more values associated with the data for Al/ML model training; and transmitting, from the mobile terminal to the network entity, the measured values.
2. 2. The method of claim 1, wherein the method further comprises receiving, at the mobile terminal from the network entity, a measurement capability enquiry, and transmitting the information on one or more measurement capabilities of the mobile terminal with respect to data for Al/ML model training in response to the measurement capability enquiry.
3. 3. The method of claims 1 or 2, wherein the method further comprises receiving measurement configuration information from the network entity, and the measuring of the one or more values associated with the data for Al/ML model training is based on the measurement configuration information.
4. 4. The method of claim 3, wherein the measurement configuration information is based on the information on one or more measurement capabilities of the mobile terminal. 25
5. 5. The method of any preceding claim, wherein the information on one or more measurement capabilities of the mobile terminal with respect to data forAl/ML model training includes information on one or more of mobile terminal type, mobile terminal category, measurement quality, measurement accuracy, measurement validity, measurement periodicity, and a number of attempts to measure a value (e.g. a reference signal).
6. 6. A method of a network entity for obtaining measurement data for Al/ML model training in a 5G NR communications system comprising the network entity and one or more mobile terminals, the method comprising: receiving, from the mobile terminal at the network entity, information on one or more measurement capabilities of the mobile terminal with respect to data for Al/ML model training; and receiving, from the mobile terminal at the network entity, one or more values associated with the data for Al/ML model training measured by the mobile terminal.
7. 7. The method of claim 6, wherein the method further comprises transmitting from the network entity to the mobile terminal, measurement configuration information based on the information on one or more measurement capabilities of the mobile terminal.
8. 8. The method of claim 7, wherein the measurement configuration information includes information on one or more of measurement quality, measurement accuracy, measurement validity, and measurement periodicity
9. 9. The method of claim 6, wherein the information on one or more measurement capabilities of the mobile terminal with respect to data for Al/ML model training is received by the network entity along with the measured values.
10. 10. The method of claim 6, wherein the network entity adapts the received measured values based on the information on one or more measurement capabilities of the mobile terminal with respect to data for Al/ML model training.
11. 11. The method of any of claims 1 to 10, wherein the network entity trains an Al/ML model based on the received one or more values.
12. 12. A mobile terminal for a 5G communications systems, wherein the mobile terminal is configured to implement the method of any of claims 1 to 5
13. 13. A network entity for a 5G communications system, wherein the network entity is configured to implement the method of any of claims 6 to 11.
14. 14. A 5G communications system comprising one or more mobile terminals and one or more network entities, wherein the 5G communications system is configured to implement the method of any of claims 1 to 11.