WO2025231785A1

WO2025231785A1 - Model performance monitoring for ue-based ai/ml positioning

Info

Publication number: WO2025231785A1
Application number: PCT/CN2024/092154
Authority: WO
Inventors: Oghenekome Oteri; Haitong Sun; Wei Zeng; Dawei Zhang; Peng Cheng; Weidong Yang; Huaning Niu
Original assignee: Apple Inc
Current assignee: Apple Inc
Priority date: 2024-05-10
Filing date: 2024-05-10
Publication date: 2025-11-13
Anticipated expiration: 2026-11-10

Abstract

A positioning model monitoring technique includes operations of obtaining an inference result output by an artificial intelligence/machine learning (AI/ML) model for determining a position of a user equipment (UE); obtaining a ground truth label for model monitoring corresponding to the inference result; and performing a monitoring function by comparing the inference result to the ground truth label.

Description

MODEL PERFORMANCE MONITORING FOR UE-BASED AI/ML POSITIONING

FIELD

This disclosure relates to wireless communication networks including techniques for AI/ML positioning.

BACKGROUND

As telecommunication network technology advances, emerging using cases require even more refined positioning performance by telecommunication networks in various environments where satellite-based systems have limited coverage. For example, for industrial internet of things (IIoT) indoor scenarios, accurate positioning down to centimeter-level may be required by logistics and manufacturing needs. Such accurate positioning also plays an important role in modern user cases such as smart cities, enhanced emergency response, asset tracking, or immersive augmented reality experiences.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure will be readily understood and enabled by the detailed description and accompanying figures of the drawings. Like reference numerals may designate like features and structural elements. Figures and corresponding descriptions are provided as non-limiting examples of aspects, implementations, etc., of the present disclosure, and references to "an" or “one” aspect, implementation, etc., may not necessarily refer to the same aspect, implementation, etc., and may mean at least one, one or more, etc.

Fig. 1 illustrates an overview functional diagram illustrating operations for a UE-based AI/ML positioning in accordance with various aspects described herein.

Fig. 2 illustrates a block diagram illustrating a wireless communication network in accordance with various aspects described herein.

Fig. 3 illustrates a functional block diagram illustrating an example AI/ML system for generating a UE-based AI/ML positioning in accordance with various aspects.

Figs. 4-9 illustrate signal flow diagrams illustrating a monitoring process for UE-based AI/ML positioning in accordance with various aspects.

Figs. 10-11 illustrate operation flow diagrams showing a monitoring process for UE-based AI/ML positioning in accordance with various aspects.

Fig. 12 is a block diagram of a wireless communication network in accordance with various aspects described herein.

Fig. 13 is a diagram of example components of a device in accordance with various aspects described herein.

Fig. 14 is a diagram of example interfaces of baseband circuitry in accordance with various aspects described herein.

DETAILED DESCRIPTION

The following detailed description refers to the accompanying drawings. Like reference numbers in different drawings may identify the same or similar features, elements, operations, etc. Additionally, the present disclosure is not limited to the following description as other implementations may be utilized, and structural or logical changes made, without departing from the scope of the present disclosure.

A wireless network may include user equipments (UEs) capable of communicating with base stations and/or other network access nodes. The base stations may provide the UEs with access to a core network (CN) and additional external networks, such as the Internet. Various techniques and standards can be used for positioning of the UEs, including in various environments where satellite-based systems have limited coverages. For example, a location server in the wireless network may be used to collect and distribute information related to positioning to other entities involved in the positioning procedures. Examples of the information related to positioning may include device capabilities, assistance data, measurements, and position estimates.

The techniques described herein enable the positioning of a UE with greater accuracy by applying artificial intelligence (AI) , machine learning (ML) , and neural networks (NN) to UE positioning procedures. These techniques may include applying NN models for UE positioning, and model performance monitoring including generating information on ground truth label for monitoring metric calculation. By specifying signal flows for the ground truth label generation and monitoring metric calculation, various options are provided to realize a label-based model performance monitoring, such that positioning accuracy is further enhanced for the UE-based AI/ML positioning. These and many other processes, operations, and features are described below with reference to the figures.

Fig. 1 illustrates an overview functional diagram illustrating operations for a UE-based AI/ML positioning in accordance with various aspects. As shown in Fig. 1, monitoring function 170 performs monitoring metric calculation by comparing inference result 161 to ground truth label 146. Inference result 161 may be or be composed of the output of an AI/ML model. The AI/ML model is hosted by UE 110 and outputs the position of the UE in direct AI/ML technique. Ground truth label 146 may be generated and provided by GTL generation function 145. If the calculated monitoring metrics indicate a model degradation, monitoring action 180 may provide feedback to update the AI/ML model.

For model monitoring purpose, inference result 161 comprises position information, such as a position of UE 110 (signaling examples described in more detail associated with Figs. 4, 5, 8) or a position of a positioning reference unit (PRU) (signaling examples described in more detail associated with Figs. 6, 7, 9) . In some aspects, the PRU is identified for monitoring model performance. The PRU may be identified based on various criteria, such as sharing the same network conditions with UE 110, locating within a proximity range of UE 110, or applying a common inference measurement with UE 110 for monitoring.

Still for model monitoring purpose, ground truth label 146 corresponds to inference result 161. Ground truth label 146 indicates a ground truth value estimation or a ground truth value to be compared with inference result 161. In some aspects, ground truth label 146 may be a ground truth value estimation of a position estimation of UE 110 derived by a positioning method other than the AL/ML model (e.g., Figs. 4, 5, 8) . In some aspects, the positioning method other than the AL/ML model may be performed by a positioning server of a network device (e.g., Figs. 4, 8) . The positioning server may be a location management function (LMF) in a core network, for example, LMF 133 of core network 130 shown and described associated with various figures. Alternatively, the positioning method other than the AL/ML model may be performed by UE 110 based on positioning assistance information received from the positioning server (e.g., Fig. 5) . In some alternative aspects, ground truth label 146 may be an actual ground truth value of the known position of the PRU (e.g., Figs. 6, 7, 9) . In one aspect, the actual ground truth value of the known position of the PRU may be communicated to UE 110 via the positioning server using a specific positioning protocol between the positioning server and UEs, such as LTE positioning protocol (LPP) for the LMF to configure UEs (e.g., Fig. 6) . In another aspect, the actual ground truth value of the known position of the PRU may be communicated to UE 110 directly from the PRU using sidelink, proprietary, and/or high layer protocol (e.g., Fig. 7) . The position information of UE 110 or the PRU used for generating the ground truth label may be generated by a positioning method other than the AL/ML model, such as by a positioning server within the core network or communicated with the core network, or by a non-radio access positioning method, such as a positioning method involving wireless local-area network (WLAN) , global positioning system (GPS) or other global navigation satellite systems (GNSS) .

As described in detailed examples below, monitoring function 170 is performed by a monitoring entity, which may be a system or device such as UE 110 (signaling examples described in more detail associated with Figs. 4-7) , or the positioning server that handles positioning (signaling examples described in more detail associated with Figs. 8-9) . As discussed, GTL generation function 145 may be performed by UE, LMF or PRU.

Fig. 2 is a block diagram of a wireless communication network 200 in accordance with various aspects. As shown in Fig. 2, the wireless communication network 200 may include UE 110, base station 120, and CN 130. CN 130 may include access and mobility management function (AMF) 131, LMF 133, and/or one or more other types of functions or entities 135. Examples of such functions or entities may include a session management function (SMF) , a unified data management (UDM) function, a gateway mobile location center (GMLC) , and more. AMF 131, LMF 133, and the one or more other types of functions or entities 135 may be implemented by one or more servers in a centralized or distributed networking environment.

AMF 131 may communicate with base station 120 via an N2 interface and UE 110 via an N1 interface. The AMF 131 may be configured to manage authentication, registration, and other functionalities for UE 110 to securely access the wireless communication network 200. The AMF 131 may also be configured to handle handovers, paging, and other functionality for the mobility of UE 110 within the wireless communication network 200.

LMF 133 may be configured to provide positioning functionality, such that the geographic or relative location, also referred as a position of UE 110, can be determined based on downlink (DL) and uplink (UL) location measuring signals. LMF 133 may receive measurement information from base station 120 and/or UE 110 via the AMF 131 and generate assistance information for UE positioning. LMF 133 may provide the positioning assistance information to UE 110. Examples of the positioning assistance information may include information regarding signals to be measured (e.g., expected signal timing, signal power, signal coding, signal frequencies, signal Doppler, etc. ) , locations and identities of terrestrial transmitters (e.g., base station 120) and/or signal, timing and orbital information for non-terrestrial transmitters, such as satellites and satellite systems. The positioning functionality of LMF 133 may be used to facilitate positioning techniques such as Assisted GNSS (A-GNSS) , Advanced Forward Link Trilateration (AFLT) , Observed Time Difference of Arrival (OTDOA) , Enhanced Cell Identity (ECID) , and so on. The positioning assistance information from LMF 133 may improve signal acquisition and measurement accuracy of UE 110 and, in some cases, enable UE 110 for enhanced positioning. Alternatively or additionally, LMF 133 may use the measurement and/or the assistance information to compute the position of UE 110. For model monitoring of AI/ML positioning, the positioning by LMF 133 may be used for deriving position estimation or providing position assistance information to a requesting application or service (e.g., associated with the UE or a lawful external entity) for position estimation during a positioning session.

In some aspects, LMF 133 configures UE 110 using a positioning protocol 137 between positioning server and UE, for example, the LTE positioning protocol (LPP) , via the AMF 131. Another positioning protocol 139 for communication of positioning server and base station, for example, the NR positioning protocol A (NRPPa) protocol may be used to carry the positioning information between base station 120 and LMF 133 over next generation control plane interface (NG-C) ) . Base station 120 configures UE 110 using radio resource control (RRC) protocol over LTE-Uu and/or NR-Uu interface.

Fig. 3 is a functional block diagram illustrating an AI/ML system 300 for generating a UE-based AI/ML positioning in accordance with various aspects. The AI/ML system 300 applies AI/ML models to enhance positioning accuracy for different scenarios, including those with heavy non-line-of-sight (NLOS) conditions. In some implementations, an AI/ML model may include a trained neural network. The AI/ML system 300 model may include a framework of features, vectors, and/or functions capable of processing input data and producing an output. As shown in the, the AI/ML system 300 may include a data collection function 310, a model training function 320, a model inference function 330, and a model monitoring and management function 340. The model monitoring and management function may be or be comprised of the monitoring function 170 as described associated with Fig. 1, or other monitoring functions described herein. In some aspects, various functions of the AI/ML system 300 may be implemented in various entities of the wireless communication network 200, such as UE 110, base station 120, and/or one or more elements of CN 130, such as LMF 133 as described throughout this disclosure.

As an example, the data collection function 310 may collect input data to provide to other functions. For example, the data collection function 310 may collect reference signal measurements and feedback information and provide training data to the model training function 320 and inference data to the model inference function 330.

Training data may include input for the AI/ML model training function 320. The model training function 320 may perform AI/ML model training, validation, and testing which may generate model performance metrics as part of the model testing procedure. The model training function 320 may also be responsible for data preparation (e.g., data pre-processing and cleaning, formatting, and transformation) based on training data delivered by the data collection function 310.

Inference data may include input for the model inference function 330. A model deployment/update may be used to initially deploy a trained, validated, and tested AI/ML model to the model inference function 330 or to deliver an updated model to the model inference function 330. The model inference function 330 may provide AI/ML model inference output (e.g., predictions or decisions) . The model inference function 330 may also be responsible for data preparation (e.g., data pre-processing and cleaning, formatting, and transformation) based on inference data delivered by the data collection function 310. The model inference function 330 may provide model performance feedback to the model training function 320 when applicable. The model performance feedback may be used to derive training data, inference data or to monitor the performance of the AI/ML model and its impact to the network through updating of performance indicators and performance counters. The model inference function 330 may provide inference output, such as UE location or assistance information for determining the UE location. Details of inference output may be use case specific.

The model inference function 330 may also provide inference output to the monitoring and management function 340. The model monitoring and management function 340 may be used for monitoring the performance of the AI/ML model. The model monitoring can be performed to the model input, inference output, and/or other metrics that suggest performance of the AI/ML model. The model monitoring can be label-based or label-less. As an example of label-based monitoring, an inference output may be compared with a ground truth value/label to calculate monitoring metrics. As another example of label-less monitoring, an inference output may be compared with an average of past inference outputs to determine deviation. The model monitoring and management function 340 may trigger or perform corresponding actions when criteria is met. The model monitoring and management function 340 may provide model performance feedback to the model training function 320 and data feedback to the data collection function 310.

Fig. 4 is a signal flow diagram illustrating a model monitoring process 400 for UE-based AI/ML positioning in accordance with various aspects. In Fig. 4, UE 110 serves the monitoring function, and information on the ground truth label of UE 110 is generated by LMF 133 using LMF support positioning method and then provided to UE 110. The model monitoring process 400 may be referred as Option A-1. Since the ground truth label is generated based on positioning estimation of UE 110, the model monitoring process 400 may not work well in NLOS environment, and the quality of the ground truth label depends upon estimation accuracy.

The model monitoring process 400, and all other processes shown in the figures and described in the specification, may include one or more fewer, additional, differently ordered and/or arranged operations than those shown or described. In some implementations, some or all of the operations of a process may be performed independently, successively, simultaneously, etc., of one or more of the other operations of the same process. As such, techniques described herein are not limited to a number, sequence, arrangement, timing, etc., of the operations or process depicted in the figures. In addition, various signaling, processes and methods described in the specification may include one or more same or similar actions or steps. These similar actions or steps may not be described repeatedly for simplicity reason, and features described in one signaling, process or method should be amenable to others when applicable.

At act 410, a UE capability message is communicated from UE 110 to LMF 133. In some aspects, support of a label-based model monitoring procedure and a method of generating and communicating a ground truth label and an input needed by a AI/ML positioning model may be included in the UE capability message. UE 110 may also indicate permission for LMF 133 to estimate its position.

In some aspects, the UE capability message may indicate positioning support information such as UE support of AI/ML positioning and one or more other positioning methods supported by UE 110 and/or LMF 133, such as downlink time-difference of arrival (DL-TDOA) , downlink angle of departure (DL-AOD) , etc. The positioning support information may additionally or alternatively indicate an input, label, and/or monitoring requirements for AI/ML modeling training. The UE capability message may also include monitoring conditions that indicate conditions the model monitoring should be triggered. For example, the UE capability message may indicate network conditions under which the AI/ML model is applicable. When there is a change in the network conditions, a signal is communicated to UE for initiating a UE model monitoring procedure, or a LMF model monitoring procedure is initiated in response to the monitoring conditions met. In some aspects, a quality of the indicated positioning method may also be communicated to LMF 133, for example, line of sight (LOS) /NLOS probability or a location of the target UE. For example, LOS/NLOS probability of AI/ML positioning method and/or positioning methods other than the AI/ML positioning may be used to determine or indicate reliability of the ground truth label to be generated.

In some aspects, one or more base stations are identified for providing reference signals for generating positioning estimate using positioning methods other than AI/ML positioning for generating ground truth label. For example, three LOS base stations may be needed for DL-TDOA based positioning, while a single base station may be sufficient for DL-AOD or a range estimation positioning method. In one aspects, the identities of the base stations may be explicitly indicated to LMF 133 via the capability message or another message. The term “base station” is used herein to represent sites, units, or entities that transmit reference signals and should be understood as any applicable form of reference signal sources or transmission devices, such as gNodeBs, eNodeBs, Transmit/Receive Points (TRPs) .

LOS/NLOS probability information may be used to identify the base stations for providing the reference signals used for ground truth label and also to generate information about quality of the ground truth label. As an example, a link of reference signal communication may be chosen to be used for ground truth label if determined as LOS. A LOS/NLOS probability of the link may indicate the quality of a ground truth label determined using the link.

LOS/NLOS probability may be determined by, for example, one or more methods of: 1) the relative strength of a signal received from the direction (e.g., receiver tap tuned to receive the signal) as compared to signals received from other directions (other receiver taps) by the link endpoint; 2) statistical/distribution based analysis of the received signal, 3) estimation based on parameters like Kurtosis, Peak to Lead delay, Mean Excess delay, or root mean square (RMS) delay spread; 4) polarization based methods.

A LOS/NLOS probability of the link may be expressed as a soft value that indicates a level of confidence or quality associated with the LOS probability. For example, a LOS/NLOS probability expressed as 1.0 may indicate LOS with the highest confidence or quality while LOS/NLOS probability expressed as 0.8 indicates LOS with less confidence or quality. Similarly, a LOS/NLOS probability of 0.0 may indicate NLOS with the highest confidence or quality and a LOS/NLOS probability expressed as 0.2 may indicate NLOS with a lower confidence or quality. If the LOS/NLOS probability meets a threshold, LOS is determined or assumed. When more than one methods are used to determine LOS/NLOS probability of the link, the output LOS/NLOS probabilities of the more than one methods may be combined. For example, the ground truth label may be determined as LOS if all of the LOS probabilities exceed a threshold or based on a combination (e.g., average) of the LOS probabilities.

At act 420, a reference signal triggering is communicated from LMF 133. For example, a triggering of downlink positioning reference signal, such as a PRS, may be communicated to one or more base stations 120. In one aspect, the triggering of downlink positioning reference signal may be communicated using NRPPa, or another specific positioning protocol between LMF and base station. Optionally, a downlink reference signal notification may be communicated from LMF 133 to UE 110, such that UE 110 could anticipate arrival of the downlink reference signal. The downlink reference signal notification may include scheduling of the downlink reference signal. Alternatively, an uplink positioning reference signal, such as a sounding reference signal (SRS) may be triggered from UE 110 to base stations 120. The measurement of the uplink positioning reference signal may also be used for generating the reference result by the AI/ML model and/or ground truth label for monitoring purpose. The signaling for using uplink positioning reference signals may be in a similar manner as described here.

At act 430, the one or more base stations 120 transmit the downlink reference signals (e.g. sets of PRS respectively from the one or more base stations 120) to UE 110. In some aspects, each set of PRS resources is configured to one base station by LMF 133 and also provided to UE 110, such that LMF 133 and UE 110 obtain knowledge about the identities of the one or more base stations 120 for reference signal measurement and further processing.

At act 435, UE 110 performs measurements on the received downlink reference signals for determining the position of UE 110 using the AI/ML model.

At act 440, UE 110 communicates the measurements of the downlink reference signals (e.g. PRS) to LMF 133 for determining a ground truth label using another positioning method other than the AI/ML model.

At act 445, LMF 133 generates the ground truth label. In some aspects, the ground truth label is an estimation of the position of UE 110, derived by a positioning method other than the AL/ML model. The estimation of the position of UE 110 may be based on the measurement of the downlink reference signal, measurement of other reference signals, or even using other non-radio technology such as WLAN or GNSS.

At act 450, the ground truth label is communicated to UE 110 for monitoring. In some aspects, the ground truth label is communicated using LPP, or another specific positioning protocol between LMF and UE.

At act 460, UE 110 performs inference using the AI/ML model. The measurements of the downlink reference signals may be used as an input to generate an inference result indicating a UE position for monitoring purpose.

At act 470, UE 110 performs a model monitoring. As an example, UE 110 may perform monitoring metric calculation by comparing the ground truth label of the estimated position of UE 110 derived by another positioning method with the position of UE 110 inferred by the AI/ML model.

Fig. 5 is a signal flow diagram illustrating a monitoring process 500 for UE-based AI/ML positioning in accordance with various aspects. In Fig. 5, UE 110 serves the monitoring function and derives the ground truth label by estimating UE position using another positioning method other than the AI/ML model. LMF 133 provides UE 110 positioning assistance data for the derivation of the ground truth label. The model monitoring process 500 may be referred as Option A-2. Since both label generation and model monitoring are performed at UE 110 and thus no need to be communicated from another entity, less signaling is required. Similar as the model monitoring process 400, since the ground truth label is generated based on positioning estimation of UE 110, the model monitoring process 500 may not work well in NLOS environment, and the quality of the ground truth label depends upon estimation accuracy.

Acts 510, 520, 530, 535, 560, 570 are similar as acts 410, 420, 430, 435, 460, 470 described above and not repeated for simplicity reason. Comparing to the model monitoring process 400, since UE 110 derives the ground truth label by estimating the UE position, LMF 133 does not need to receive measurements of the downlink reference signals and is not aware of UE position or UE position estimation. Thus, no privacy check is needed, and UE 110 is not required a permission for LMF 133 to estimate its position. The model monitoring process 500 has a better privacy protection than the model monitoring process 400.

At act 540, positioning assistance information may be communicated from LMF 133 to UE 110. As an example, the positioning assistance information may include information indicating locations and/or identities of base station 120 including terrestrial and/or non-terrestrial transmitters. Examples of the positioning assistance information may also include information regarding signals to be measured (e.g., expected signal timing, signal power, signal coding, signal frequencies, signal Doppler, etc. ) . The positioning assistance information from LMF 133 may improve signal acquisition and measurement accuracy of UE 110 and, in some cases, enable UE 110 for a more accurate and reliable ground truth label through an enhanced positioning method other than AI/ML modeling. The positioning assistance information may also broadcast by base station 120.

At act 545, UE 110 generates the ground truth label based on the measurements of the downlink reference signals and the positioning assistance information. In some aspects, the ground truth label is an estimation of the position of UE 110, derived by a positioning method other than the AL/ML model based on the measurement of the downlink reference signal and the positioning assistance information.

Fig. 6 is a signal flow diagram illustrating a monitoring process 600 for UE-based AI/ML positioning in accordance with various aspects. In Fig. 6, UE 110 serves the monitoring function and receives the ground truth label from PRU 111 with its position known via LMF 133. The model monitoring process 600 may be referred as Option A-3. PRU 111 is selected with better positioning capabilities. For example, the positioning of PRU 111 may be pre-known or easier to acquire by being equipped with GNSS or WLAN and/or with better location (e.g. better LOS condition) . Since PRU 111 is used for generating the ground truth label, the quality of the ground truth label is typically higher than the model monitoring process 400 or 500. On the other hand, the monitoring reliability depends on the selection and condition change of PRU 111.

At act 610, a UE capability message is communicated from UE 110 to LMF 133. In some aspects, support of a label-based model monitoring procedure and a method of selecting and communicating a ground truth label and an input needed by an AI/ML positioning model may be included in the UE capability message and used to configure one or more PRUs for model monitoring. Information on UE 110 may be communicated to LMF 133 to facilitate selection of the one or more PRUs. Such information may include network condition of UE 110, position estimation of UE 110, and/or inference measurement used by UE 110 for positioning. In some aspects, UE 110 may identify a number of base stations needed for providing reference signals to the one or more PRUs for AI/ML positioning.

At act 615, the one or more PRUs are identified by UE 110 and LMF 133. For example, PRU 111 may be identified based on various criteria so that monitoring PRU 111 could suggest model performance of UE 110. The various criteria may include one or more of sharing the same network conditions with UE 110, locating within a proximity range of UE 110, or applying a common inference measurement with UE 110 for monitoring.

In some aspects, as shown by acts 620, 630, 635, 640, PRU 111 with a known position (also referred to as “ground truth” location or position) can act as a measurement entity and perform measurements of reference signals for position-related measurement (e.g., positioning reference signals (PRS) from a base station) and report these measurements for use in training or monitoring the AI/ML model.

At acts 640 and 650, LMF 133 may be used to forward the ground truth label and measurements of the downlink positioning reference signal (e.g. PRS) from PRU 111 to UE 110 using LPP, or another specific positioning protocol between LMF and UE. At act 660, UE 110 uses received measurements of the downlink positioning reference signal to generate an inference result. At act 670, UE 110 performs model monitoring by comparing the inference result with the received ground truth label.

Acts 610, 620, 630, 635, 660, 670 are similar as acts 410, 420, 430, 435, 460, 470 described above and not repeated for simplicity reason. Comparing to the model monitoring process 400, since UE 110 receives the ground truth label of the actual PRU position, LMF 133 is not aware of UE position. Thus, no privacy check is needed, and UE 110 is not required a permission for LMF 133 to estimate its position. The model monitoring process 600 has a better privacy protection than the model monitoring process 400. By using PRU 111 with known position, the reliability of the ground truth label is improved.

Also referring back to Fig. 2, PRU 111 may be included in the wireless communication network 200. In addition PRU 111 can act as a reference signal entity and transmit sounding reference signals (SRS) to enable base station 120 to measure and report uplink (UL) positioning measurements for PRU 111 (at its known location) for use in training or monitoring the AI/ML model. Functionality of PRU 111 is realized as a UE (either stationary or mobile) with a known location. The ground truth location for PRU 111 may be programmed and stored (for a stationary PRU) or determined during a positioning session that uses a method that does not include use of the present AI/ML model-enhanced techniques. For example, the position of PRU 111 may be determined by a global navigation satellite systems (GNSS) available to PRU 111.

Fig. 7 is a signal flow diagram illustrating a monitoring process 700 for UE-based AI/ML positioning in accordance with various aspects. In Fig. 7, UE 110 serves the monitoring function and receives the ground truth label directly from PRU 111 with known position. The model monitoring process 700 may be referred as Option A-4. Similar as the model monitoring process 600, PRU 111 is selected with better positioning capabilities. For example, the positioning of PRU 111 may be pre-known or easier to acquire by being equipped with GNSS or WLAN and/or with better location (e.g. better LOS condition) . Since PRU 111 is used for generating the ground truth label, the quality of the ground truth label is typically higher than the model monitoring process 400 or 500. On the other hand, the monitoring reliability depends on the selection and condition change of PRU 111.

Acts 710, 720, 730, 735, 760, 770 are similar as acts 610, 620, 630, 635, 660, 670 described above and not repeated for simplicity reason. Comparing to the model monitoring process 600, at act 750, monitoring information is communicated from PRU 111 to UE 110 directly. The monitoring information may include the ground truth label of the actual PRU position of PRU 111 and measurements of the downlink positioning reference signal (e.g. PRS) serving as AI/ML input for model monitoring. As examples, the monitoring information may be communicated using sidelink, proprietary method, and/or high layer protocol. In one aspect, the monitoring information may be communicated using sidelink mode 1, where base station 120 configures time and frequency resources for the sidelink communication. In another aspect, the monitoring information may be communicated using sidelink mode 2, where PRU 111 autonomously communicates the monitoring information to UE 110. In a further aspect, the monitoring information is exchanged using a proprietary method such as airdrop. In a further aspect, higher layer protocol is involved for communicating monitoring information. For example, a sidelink positioning protocol (SLPP) may be used for configuring and communicating the monitoring information to UE 110.

Fig. 8 is a signal flow diagram illustrating a monitoring process 800 for UE-based AI/ML positioning in accordance with various aspects. In Fig. 8, LMF 133 generates ground truth label of UE 110 using LMF support positioning method. LMF 133 also serves the monitoring function and compares model output received from UE 110 with the generated ground truth label.

Acts 810, 820, 830, 835, 840, 860 are similar as acts 410, 420, 430, 435, 440, 460 described above and not repeated for simplicity reason. As described, UE 110 performs measurements of downlink reference signals (act 835) and then communicates measurements to LMF 133 (act 840) . LMF 133 generates the ground truth label of the estimated position of UE 110 based on the measurements of downlink reference signals using a positioning method other than the AI/ML model (act 845) . UE 110 generates the inference result based on the measurements of downlink reference signals using the AI/ML model (act 860) .

Comparing to the model monitoring process 400, at act 865, UE 110 then communicates the inference result to the LMF 133 for modeling monitoring purpose, for example, using LPP. The purpose of modeling monitoring may be transparent to UE 110. At act 870, a model monitoring action is performed by LMF 133. As an example, LMF 133 may perform monitoring metric calculation by comparing the ground truth label with the inference result. At act 880, optionally, a monitoring action may be performed. For example, if the calculated monitoring metrics indicate a model degradation (e.g., meeting degradation criteria) , LMF 133 may provide feedback to UE 110, such as to indicate a need to update the AI/ML model.

Fig. 9 is a signal flow diagram illustrating a monitoring process 900 for UE-based AI/ML positioning in accordance with various aspects. In Fig. 9, LMF 133 receives the ground truth label from PRU 111 with a known position. LMF 133 also serves the monitoring function and compares the inference result received from UE 110 with the ground truth label received from PRU 111.

Acts 910, 915, 920, 930, 935, 940, 960 are similar as acts 410, 415, 420, 430, 435, 440, 460 described above and not repeated for simplicity reason. As described, PRU 111 performs measurements of downlink reference signals (act 935) and then communicates the measurements of downlink reference signals to LMF 133. PRU 111 may also communicate the ground truth label to LMF 133. In one aspect, the measurements of downlink reference signals and the ground truth label may be communicated together, for example, using LPP (act 940) .

Comparing to the model monitoring process 600, at act 950, the measurements of downlink reference signals is forwarded and communicated to UE 110. UE 110 then generates the inference result based on the measurements of downlink reference signals received from PRU 111 using the AI/ML model (act 960) . The ground truth label may not need to be forwarded to UE 110, since modeling monitoring is performed by LMF 133. Acts 965, 970, 980 are similar as acts 865, 870, 880 described above and not repeated for simplicity reason.

Figs. 10-11 illustrate operation flow diagrams showing various aspects of a monitoring process for UE-based AI/ML positioning as disclosed by description and figures herein. Many details of the monitoring process are already discussed above associated with other figures and are not repeated here for simplicity.

As shown by the example in flow diagram 1000 of Fig. 10, the monitoring process involves a monitoring function performed by a model monitoring entity as disclosed herein. The model monitoring entity may be or compose a UE or a LMF. In some aspects, an inference result is obtained for model monitoring (act 1010) . The inference result is output by an AI/ML model which is trained and validated for determining a position of the UE. The AI/ML model is hosted by the UE. In some aspects in which the UE positioning information is used for monitoring, the inference result may be comprised of the position of the UE. The ground truth label may be derived by the LMF or the UE based on positioning assistance information received from the LMF.

In some aspects in which a PRU is used for monitoring, the inference result may be comprised of the position of the PRU. The PRU is identified for monitoring model performance. Identification of the PRU may be based on criteria including one or more of: sharing the same network conditions with the UE, locating within a proximity range of the UE, or applying a common inference measurement with the UE for monitoring.

A ground truth label is also obtained for model monitoring (act 1020) . The ground truth label is an actual or estimated positioning information corresponding to the inference result. In some aspects in which the UE positioning information is used for monitoring, the ground truth label is an estimated position of the UE derived by a positioning method other than the AI/ML model. The estimated position may be determined by LMF or by UE based on assistance information provided by LMF. The estimated position may be determined using other non-RAT methods, such as involving WLAN or GNSS. The ground truth label may also be a known position of the PRU communicated by the PRU. In one aspect, the known position of the PRU is communicated to the UE via a location management function (LMF) (e.g, using LPP) . In another aspect, the known position of the PRU is communicated to the UE directly by the PRU (e.g., using sidelink, proprietary, and/or high layer protocol) . In some aspect, the known position of the PRU is communicated to the LMF using LPP.

To perform the monitoring function, the inference result is compared to the ground truth label (act 1030) . Optionally, the monitoring function may further comprise performing a monitoring action (act 1040) , responsive to the comparison meeting a pre-determined criteria, which indicates that the UE positioning generated by the AI/ML model is not reliable anymore. The monitoring action may trigger a data or model updating.

As shown by the example in flow diagram 1100 of Fig. 11, the monitoring process involves operations performed by a UE or a component (e.g., a processor or processing unit) of the UE. At act 1110, The UE communicates UE capability information to LMF. The UE capability information indicates support of a label-based model monitoring procedure. The UE capability information may further indicate a method of generating the ground truth label and an input needed by the AI/ML model. At act 1120, the UE generates an inference result for model monitoring. The inference result is output by an AI/ML mode based on a measurement of a downlink reference signal. At act 1130, the UE obtains a ground truth label for model monitoring. The ground truth label corresponds to the inference result as discussed above. The ground truth label may be a UE position estimation corresponds to a UE position generated as the inference result. The UE position estimation may be generated by the UE based on assistance information provided by the LMF. The UE position estimation may be generated by LMF based on the measurements of reference signals. Alternatively, the ground truth label may be a PRU position information communicated by PRU directly or via LMF. At act 1140, the UE compares the inference result to the ground truth label and may trigger actions to update model input or model itself when the comparison metrics meet monitoring criteria. More operations of the monitoring process are discussed above associated with other figures, for example, figures 4-7.

Fig. 12 is a further example of wireless communication network 200 according to one or more implementations described herein. As discussed, wireless communication network 200 may include UE 110, base station 120, which represents a node in a radio access network that may also referred as RAN, CN 130, and may also include application servers 140, and external networks 150.

The systems and devices of wireless communication network 200 may operate in accordance with one or more communication standards, such as 2nd generation (2G) , 3rd generation (3G) , 4th generation (4G) (e.g., long-term evolution (LTE) ) , and/or 5th generation (5G) (e.g., new radio (NR) ) communication standards of the 3rd generation partnership project (3GPP) . Additionally, or alternatively, one or more of the systems and devices of example network 1700 may operate in accordance with other communication standards and protocols discussed herein, including future versions or generations of 3GPP standards (e.g., sixth generation (6G) standards, seventh generation (7G) standards, etc. ) , institute of electrical and electronics engineers (IEEE) standards (e.g., wireless metropolitan area network (WMAN) , worldwide interoperability for microwave access (WiMAX) , etc. ) , and more.

As shown, UE 110 may include mobile or non-mobile computing devices capable of wireless communications, such as personal data assistants (PDAs) , pagers, laptop computers, desktop computers, wireless handsets, smartphones, etc. In some implementations, UE 110 may include internet of things (IoT) devices (or IoT UEs) that may comprise a network access layer designed for low-power IoT applications utilizing short-lived UE connections. Additionally, or alternatively, an IoT UE may utilize one or more types of technologies, such as machine-to-machine (M2M) communications or machine-type communications (MTC) (e.g., to exchanging data with an MTC server or other device via a public land mobile network (PLMN) ) , proximity-based service (ProSe) or device-to-device (D2D) communications, sensor networks, IoT networks, and more. Depending on the scenario, an M2M or MTC exchange of data may be a machine-initiated exchange, and an IoT network may include interconnecting IoT UEs (which may include uniquely identifiable embedded computing devices within an Internet infrastructure) with short-lived connections. In some scenarios, IoT UEs may execute background applications (e.g., keep-alive messages, status updates, etc. ) to facilitate the connections of the IoT network. UE 110 may be understood as one or more devices that need a positioning operation and may be within or not always within coverage of the RAN network.

The wireless communication network 200 may include PRU 111. Functionality of PRU 111 is realized as a UE (either stationary or mobile) with a known location. In some aspects, PRU 111 has a known location (also referred to as “ground truth” location or position) can act as a measurement entity and perform measurements of reference signals for position-related measurement (e.g., positioning reference signals (PRS) from a base station) and report these measurements for use in training or monitoring the AI/ML model. In addition, PRU 111 can act as a reference signal entity and transmit sounding reference signals (SRS) to enable base station 120 to measure and report uplink (UL) positioning measurements for PRU 111 (at its known location) for use in training or monitoring the AI/ML model. The ground truth location for PRU 111 may be programmed and stored (for a stationary PRU) or determined during a positioning session that uses a method that does not include use of the present AI/ML model- enhanced techniques. For example, the position of PRU 111 may be determined by a global navigation satellite systems (GNSS) and/or communicated by WLAN available to PRU 111.

UE 110 and PRU 111 may communicate and establish a connection with (e.g., be communicatively coupled) with base station 120, which may involve one or more wireless channels 114-1 and 114-2, each of which may comprise a physical communications interface /layer. In some implementations, a UE may be configured with dual connectivity (DC) as a multi-radio access technology (multi-RAT) or multi-radio dual connectivity (MR-DC) , where a multiple receive and transmit (Rx/Tx) capable UE may use resources provided by different network nodes (e.g., 120-1 and 120-2) that may be connected via non-ideal backhaul (e.g., where one network node provides NR access and the other network node provides either E-UTRA for LTE or NR access for 5G) . In such a scenario, one network node may operate as a master node (MN) and the other as the secondary node (SN) . The MN and SN may be connected via a network interface, and at least the MN may be connected to CN 130. Additionally, at least one of the MN or the SN may be operated with shared spectrum channel access, and functions specified for UE 110 can be used for an integrated access and backhaul mobile termination (IAB-MT) . In some implementations, a base station (as described herein) may be an example of network node 120 or an exchangeable term of network node.

RAN may include one or more base stations 120-1 and 120-2 (referred to collectively as base station (s) 120) that enable channels 114-1 and 114-2 to be established with UE 110. Base station 120 may include network access points configured to provide radio baseband functions for data and/or voice connectivity between users and the network based on one or more of the communication technologies described herein (e.g., 2G, 3G, 4G, 5G, WiFi, etc. ) . As examples therefore, a RAN node may be an E-UTRAN Node B (e.g., an enhanced Node B, eNodeB, eNB, 4G base station, etc. ) , a next generation base station (e.g., a 5G base station, NR base station, next generation eNBs (gNB) , etc. ) . Base station 120 may include a roadside unit (RSU) , a transmission reception point (TRxP or TRP) , and one or more other types of ground stations (e.g., terrestrial access points) . In some scenarios, base station 120 may be a dedicated physical device, such as a macrocell base station, and/or a low power (LP) base station for providing femtocells, picocells or other like having smaller coverage areas, smaller user capacity, or higher bandwidth compared to macrocells. References herein to base station 120, the RAN node, etc., may involve implementations where base station 120, the RAN node, etc., is a terrestrial network node and also to implementation where base station 120, the RAN node, etc., is a non-terrestrial network node (e.g., satellite) .

Some or all of base station 120 may be implemented as one or more software entities running on server computers as part of a virtual network, which may be referred to as a centralized RAN (CRAN) and/or a virtual baseband unit pool (vBBUP) . In these implementations, the CRAN or vBBUP may implement a RAN function split, such as a packet data convergence protocol (PDCP) split wherein radio resource control (RRC) and PDCP layers may be operated by the CRAN/vBBUP and other Layer 2 (L2) protocol entities may be operated by individual base stations 120; a media access control (MAC) /physical (PHY) layer split wherein RRC, PDCP, radio link control (RLC) , and MAC layers may be operated by the CRAN/vBBUP and the PHY layer may be operated by individual base station 120; or a “lower PHY” split wherein RRC, PDCP, RLC, MAC layers and upper portions of the PHY layer may be operated by the CRAN/vBBUP and lower portions of the PHY layer may be operated by individual base station 120. This virtualized framework may allow freed-up processor cores of base station 120 to perform or execute other virtualized applications.

In some implementations, an individual base station 120 may represent individual gNB-distributed units (DUs) connected to a gNB-control unit (CU) via individual F1 interfaces. In such implementations, the gNB-DUs may include one or more remote radio heads or radio frequency (RF) front end modules (RFEMs) , and the gNB-CU may be operated by a server (not shown) located in RAN or by a server pool (e.g., a group of servers configured to share resources) in a similar manner as the CRAN/vBBUP. Additionally, or alternatively, one or more of base stations 120 may be next generation eNBs (i.e., gNBs) that may provide evolved universal terrestrial radio access (E-UTRA) user plane and control plane protocol terminations toward UE 110, and that may be connected to a 5G core network (5GC) 130 via an NG interface.

Any of the base stations 120 may terminate an air interface protocol and may be the first point of contact for UE 110. In some implementations, any of the base stations 120 may fulfill various logical functions for the RAN including, but not limited to, radio network controller (RNC) functions such as radio bearer management, uplink and downlink dynamic radio resource management and data packet scheduling, and mobility management. UE 110 may be configured to communicate using orthogonal frequency-division multiplexing (OFDM) communication signals with each other or with any of the base stations 120 over a multicarrier communication channel in accordance with various communication techniques, such as, but not limited to, an OFDMA communication technique (e.g., for downlink communications) or a single carrier frequency-division multiple access (SC-FDMA) communication technique (e.g., for uplink and ProSe or sidelink (SL) communications) , although the scope of such implementations may not be limited in this regard. The OFDM signals may comprise a plurality of orthogonal subcarriers.

The base stations 120 may be configured to communicate with one another via interface 123. In implementations where the system is an LTE system, interface 123 may be an X2 interface. The X2 interface may be defined between two or more base stations 120 (e.g., two or more eNBs /gNBs or a combination thereof) that connect to evolved packet core (EPC) or CN 130, or between two eNBs connecting to an EPC. In some implementations, the X2 interface may include an X2 user plane interface (X2-U) and an X2 control plane interface (X2-C) . The X2-U may provide flow control mechanisms for user data packets transferred over the X2 interface and may be used to communicate information about the delivery of user data between eNBs or gNBs. For example, the X2-U may provide specific sequence number information for user data transferred from a master eNB (MeNB) to a secondary eNB (SeNB) ; information about successful in sequence delivery of PDCP packet data units (PDUs) to a UE 110 from an SeNB for user data; information of PDCP PDUs that were not delivered to a UE 110; information about a current minimum desired buffer size at the SeNB for transmitting to the UE user data; and the like. The X2-C may provide intra-LTE access mobility functionality (e.g., including context transfers from source to target eNBs, user plane transport control, etc. ) , load management functionality, and inter-cell interference coordination functionality.

As shown, RAN may be connected (e.g., communicatively coupled) to CN 130. CN 130 may comprise a plurality of network elements configured to offer various data and telecommunications services to customers/subscribers (e.g., users of UE 110) who are connected to CN 130 via the RAN.

As discussed above, the CN may comprises the AMF 131 and LMF 133. LMF 133 may receive measurements and assistance information from base station 120 and UE 110, via the AMF 131 over an interface 132 (e.g., the NLs interface) to compute the position of UE 110. LMF 133 configures UE 110 using the LTE positioning protocol (LPP) via the AMF 131. In some aspects, an NR positioning protocol A (NRPPa) protocol is used to carry the positioning information between base station 120 and LMF 133 over an interface 128 (e.g. next generation control plane interface (NG-C) ) and the interface 132. Base station 120 configures UE 110 using radio resource control (RRC) protocol over LTE-Uu and/or NR-Uu interface.

LMF 133 provides position estimation or position-related information to a requesting application or service (e.g., associated with the UE or a lawful external entity) during a positioning session. In one aspect, LMF 133 provides positioning assistance data to UE 110 including, for example, information regarding signals to be measured (e.g., expected signal timing, signal coding, signal frequencies, and so on) , locations and identities of terrestrial transmitters (e.g., base stations 120) and/or signal timing or orbital information for Global Navigation Satellite System (GNSS) satellites (not shown) . This positioning information may be used to facilitate positioning techniques such as Assisted GNSS (A-GNSS) , Advanced Forward Link Trilateration (AFLT) , Observed Time Difference of Arrival (OTDOA) , Enhanced Cell Identity (ECID) , and so on.

To support AI/ML model-enhanced positioning, CN 130 may include an AI/ML management function which is implemented on a server (s) or other entity within CN 130 or associated with CN 130. In one example, the AI/ML-MF management function may be implemented as part of the functions performed by LMF 133. However, in other examples, the AI/ML management function may be implemented as a separate or independent function with respect to LMF 133 within CN 130 or at least communicated with CN 130. The AI/ML management function may maintain and deploy completed or partial positioning-related AI/ML models within the network. For simplicity reason, LMF 133 is described as the entity configured to perform AI/ML model management functions of the core network side. However, as discussed, such AI/ML model management functions may be performed by the AI/ML management function as a function/entity/server within LMF 133, implemented separately from LMF 133 within CN 130, or even discrete from and communicated with CN 130.

In some implementations, CN 130 may include an evolved packet core (EPC) , a 5G CN, and/or one or more additional or alternative types of CNs. The components of CN 130 may be implemented in one physical node or separate physical nodes including components to read and execute instructions from a machine-readable or computer-readable medium (e.g., a non-transitory machine-readable storage medium) . In some implementations, network function virtualization (NFV) may be utilized to virtualize any or all the above-described network node roles or functions via executable instructions stored in one or more computer-readable storage mediums (described in further detail below) . A logical instantiation of CN 130 may be referred to as a network slice, and a logical instantiation of a portion of CN 130 may be referred to as a network sub-slice. Network Function Virtualization (NFV) architectures and infrastructures may be used to virtualize one or more network functions, alternatively performed by proprietary hardware, onto physical resources comprising a combination of industry-standard server hardware, storage hardware, or switches. In other words, NFV systems may be used to execute virtual or reconfigurable implementations of one or more EPC components/functions.

As shown, CN 130, application servers 140, and external networks 150 may be connected to one another via interfaces 134, 136, and 138, which may include IP network interfaces. Application servers 140 may include one or more server devices or network elements (e.g., virtual network functions (VNFs) offering applications that use IP bearer resources with CN 330 (e.g., universal mobile telecommunications system packet services (UMTS PS) domain, LTE PS data services, etc. ) . Application servers 140 may also, or alternatively, be configured to support one or more communication services (e.g., voice over IP (VoIP sessions, push-to-talk (PTT) sessions, group communication sessions, social networking services, etc. ) for UE 110 via CN 130. Similarly, external networks 150 may include one or more of a variety of networks, including the Internet, thereby providing the mobile communication network and UE 110 of the network access to a variety of additional services, information, interconnectivity, and other network features.

Fig. 13 is a diagram of an example of components of a device 1300 according to one or more implementations described herein. In some implementations, the device 1300 can include application circuitry 1302, baseband circuitry 1304, RF circuitry 1306, front-end module (FEM) circuitry 1308, one or more antennas 1310, and power management circuitry (PMC) 1312 coupled together at least as shown. The components of the illustrated device 1300 can be included in a UE or a base station (RAN node) , such as UE 110 or base station 120 as described throughout the specification. In some implementations, the device 1300 can include fewer elements (e.g., a RAN node may not utilize application circuitry 1302, and instead include a processor/controller to process IP data received from a CN such as a 5GC or an Evolved Packet Core (EPC) ) . In some implementations, the device 1300 can include additional elements such as, for example, memory/storage, display, camera, sensor (including one or more temperature sensors, such as a single temperature sensor, a plurality of temperature sensors at different locations in device 1300, etc. ) , or input/output (I/O) interface. In other implementations, the components described below can be included in more than one device (e.g., said circuitries can be separately included in more than one device for Cloud-RAN (C-RAN) implementations) .

The application circuitry 1302 can include one or more application processors. For example, the application circuitry 1302 can include circuitry such as, but not limited to, one or more single-core or multi-core processors. The processor (s) can include any combination of general-purpose processors and dedicated processors (e.g., graphics processors, application processors, etc. ) . The processors can be coupled with or can include memory/storage and can be configured to execute instructions stored in the memory/storage to enable various applications or operating systems to run on the device 1300. In some implementations, processors of application circuitry 1302 can process IP data packets received from an EPC.

The baseband circuitry 1304 can include circuitry such as, but not limited to, one or more single-core or multi-core processors. The baseband circuitry 1304 can include one or more baseband processors or control logic to process baseband signals received from a receive signal path of the RF circuitry 1306 and to generate baseband signals for a transmit signal path of the RF circuitry 1306. Baseband circuitry 1304 can interface with the application circuitry 1302 for generation and processing of the baseband signals and for controlling operations of the RF circuitry 1306. For example, in some implementations, the baseband circuitry 1304 can include a 3G baseband processor 1304A, a 4G baseband processor 1304B, a 5G baseband processor 1304C, or other baseband processor (s) 1304D for other existing generations, generations in development or to be developed in the future (e.g., 2G, 6G, etc. ) . The baseband circuitry 1304 (e.g., one or more of baseband processors 1304A-D) can handle various radio control functions that enable communication with one or more radio networks via the RF circuitry 1306. In other implementations, some or all of the functionality of baseband processors 1304A-D can be included in modules stored in the memory 1304G and executed via a Central Processing Unit (CPU) 1304E. The radio control functions can include, but are not limited to, signal modulation/demodulation, encoding/decoding, radio frequency shifting, etc. In some implementations, modulation/demodulation circuitry of the baseband circuitry 1304 can include Fast-Fourier Transform (FFT) , precoding, or constellation mapping/de-mapping functionality. In some implementations, encoding/decoding circuitry of the baseband circuitry 1304 can include convolution, tail-biting convolution, turbo, Viterbi, or Low-Density Parity Check (LDPC) encoder/decoder functionality. Implementations of modulation/demodulation and encoder/decoder functionality are not limited to these examples and can include other suitable functionality in other implementations.

In some implementations, the baseband circuitry 1304 can include one or more audio digital signal processor (s) (DSP) 1304F. The audio DSPs 1304F can include elements for compression/decompression and echo cancellation and can include other suitable processing elements in other implementations. Components of the baseband circuitry can be suitably combined in a single chip, a single chipset, or disposed on a same circuit board in some implementations. In some implementations, some or all of the constituent components of the baseband circuitry 1304 and the application circuitry 1302 can be implemented together such as, for example, on a system on a chip (SOC) .

In some implementations, the baseband circuitry 1304 can provide for communication compatible with one or more radio technologies. For example, in some implementations, the baseband circuitry 1304 can support communication with a NG-RAN, an evolved universal terrestrial radio access network (EUTRAN) or other wireless metropolitan area networks (WMAN) , a wireless local area network (WLAN) , a wireless personal area network (WPAN) , etc. Implementations in which the baseband circuitry 1304 is configured to support radio communications of more than one wireless protocol can be referred to as multi-mode baseband circuitry.

RF circuitry 1306 can enable communication with wireless networks using modulated electromagnetic radiation through a non-solid medium. In various implementations, the RF circuitry 1306 can include switches, filters, amplifiers, etc. to facilitate the communication with the wireless network. RF circuitry 1306 can include a receive signal path which can include circuitry to down-convert RF signals received from the FEM circuitry 1308 and provide baseband signals to the baseband circuitry 1304. RF circuitry 1306 can also include a transmit signal path which can include circuitry to up-convert baseband signals provided by the baseband circuitry 1304 and provide RF output signals to the FEM circuitry 1308 for transmission.

In some implementations, the receive signal path of the RF circuitry 1306 can include mixer circuitry 1306A, amplifier circuitry 1306B and filter circuitry 1306C. In some implementations, the transmit signal path of the RF circuitry 1306 can include filter circuitry 1306C and mixer circuitry 1306A. RF circuitry 1306 can also include synthesizer circuitry 1306D for synthesizing a frequency for use by the mixer circuitry 1306A of the receive signal path and the transmit signal path. In some implementations, the mixer circuitry 1306A of the receive signal path can be configured to down-convert RF signals received from the FEM circuitry 1308 based on the synthesized frequency provided by synthesizer circuitry 1306D. The amplifier circuitry 1306B can be configured to amplify the down-converted signals and the filter circuitry 1306C can be a low-pass filter (LPF) or band-pass filter (BPF) configured to remove unwanted signals from the down-converted signals to generate output baseband signals. Output baseband signals can be provided to the baseband circuitry 1304 for further processing. In some implementations, the output baseband signals can be zero-frequency baseband signals, although this is not a requirement. In some implementations, mixer circuitry 1306A of the receive signal path can comprise passive mixers, although the scope of the implementations is not limited in this respect.

In some implementations, the mixer circuitry 1306A of the transmit signal path can be configured to up-convert input baseband signals based on the synthesized frequency provided by the synthesizer circuitry 1306D to generate RF output signals for the FEM circuitry 1308. The baseband signals can be provided by the baseband circuitry 1304 and can be filtered by filter circuitry 1306C.

In some implementations, the mixer circuitry 1306A of the receive signal path and the mixer circuitry 1306A of the transmit signal path can include two or more mixers and can be arranged for quadrature down conversion and up conversion, respectively. In some implementations, the mixer circuitry 1306A of the receive signal path and the mixer circuitry 1306A of the transmit signal path can include two or more mixers and can be arranged for image rejection (e.g., Hartley image rejection) . In some implementations, the mixer circuitry 1306A of the receive signal path and the mixer circuitry 1306A can be arranged for direct down conversion and direct up conversion, respectively. In some implementations, the mixer circuitry 1306A of the receive signal path and the mixer circuitry 1306A of the transmit signal path can be configured for super-heterodyne operation.

In some implementations, the output baseband signals, and the input baseband signals, can be analog baseband signals, although the scope of the implementations is not limited in this respect. In some alternate implementations, the output baseband signals, and the input baseband signals, can be digital baseband signals. In these alternate implementations, the RF circuitry 1306 can include analog-to-digital converter (ADC) and digital-to-analog converter (DAC) circuitry and the baseband circuitry 1304 can include a digital baseband interface to communicate with the RF circuitry 1306.

In some dual-mode implementations, a separate radio IC circuitry can be provided for processing signals for each spectrum, although the scope of the implementations is not limited in this respect.

In some implementations, the synthesizer circuitry 1306D can be a fractional-N synthesizer or a fractional N/N+1 synthesizer, although the scope of the implementations is not limited in this respect as other types of frequency synthesizers can be suitable. For example, synthesizer circuitry 1306D can be a delta-sigma synthesizer, a frequency multiplier, or a synthesizer comprising a phase-locked loop with a frequency divider.

The synthesizer circuitry 1306D can be configured to synthesize an output frequency for use by the mixer circuitry 1306A of the RF circuitry 1306 based on a frequency input and a divider control input. In some implementations, the synthesizer circuitry 1306D can be a fractional N/N+1 synthesizer.

In some implementations, frequency input can be provided by a voltage-controlled oscillator (VCO) , although that is not a requirement. Divider control input can be provided by either the baseband circuitry 1304 or the applications circuitry 1302 depending on the desired output frequency. In some implementations, a divider control input (e.g., N) can be determined from a look-up table based on a channel indicated by the applications circuitry 1302.

Synthesizer circuitry 1306D of the RF circuitry 1306 can include a divider, a delay-locked loop (DLL) , a multiplexer and a phase accumulator. In some implementations, the divider can be a dual modulus divider (DMD) and the phase accumulator can be a digital phase accumulator (DPA) . In some implementations, the DMD can be configured to divide the input signal by either N or N+1 (e.g., based on a carry out) to provide a fractional division ratio. In some example implementations, the DLL can include a set of cascaded, tunable, delay elements, a phase detector, a charge pump and a D-type flip-flop. In these implementations, the delay elements can be configured to break a VCO period up into Nd equal packets of phase, where Nd is the number of delay elements in the delay line. In this way, the DLL provides negative feedback to help ensure that the total delay through the delay line is one VCO cycle.

In some implementations, synthesizer circuitry 1306D can be configured to generate a carrier frequency as the output frequency, while in other implementations, the output frequency can be a multiple of the carrier frequency (e.g., twice the carrier frequency, four times the carrier frequency) and used in conjunction with quadrature generator and divider circuitry to generate multiple signals at the carrier frequency with multiple different phases with respect to each other. In some implementations, the output frequency can be a LO frequency (fLO) . In some implementations, the RF circuitry 1306 can include an IQ/polar converter.

FEM circuitry 1308 can include a receive signal path which can include circuitry configured to operate on RF signals received from one or more antennas 1310, amplify the received signals and provide the amplified versions of the received signals to the RF circuitry 1306 for further processing. FEM circuitry 1308 can also include a transmit signal path which can include circuitry configured to amplify signals for transmission provided by the RF circuitry 1306 for transmission by one or more of the one or more antennas 1310. In various implementations, the amplification through the transmit or receive signal paths can be done solely in the RF circuitry 1306, solely in the FEM circuitry 1308, or in both the RF circuitry 1306 and the FEM circuitry 1308.

In some implementations, the FEM circuitry 1308 can include a TX/RX switch to switch between transmit mode and receive mode operation. The FEM circuitry can include a receive signal path and a transmit signal path. The receive signal path of the FEM circuitry can include an LNA to amplify received RF signals and provide the amplified received RF signals as an output (e.g., to the RF circuitry 1306) . The transmit signal path of the FEM circuitry 1308 can include a power amplifier (PA) to amplify input RF signals (e.g., provided by RF circuitry 1306) , and one or more filters to generate RF signals for subsequent transmission (e.g., by one or more of the one or more antennas 1310) .

In some implementations, the PMC 1312 can manage power provided to the baseband circuitry 1304. In particular, the PMC 1312 can control power-source selection, voltage scaling, battery charging, or DC-to-DC conversion. The PMC 1312 can often be included when the device 1300 is capable of being powered by a battery, for example, when the device is included in a UE. The PMC 1312 can increase the power conversion efficiency while providing desirable implementation size and heat dissipation characteristics.

While Fig. 13 shows the PMC 1312 coupled only with the baseband circuitry 1304. However, in other implementations, the PMC 1312 may be additionally or alternatively coupled with, and perform similar power management operations for, other components such as, but not limited to, application circuitry 1302, RF circuitry 1306, or FEM circuitry 1308.

In some implementations, the PMC 1312 can control, or otherwise be part of, various power saving mechanisms of the device 1300. For example, if the device 1300 is in an RRC_Connected state, where it is still connected to the RAN node as it expects to receive traffic shortly, then it can enter a state known as Discontinuous Reception Mode (DRX) after a period of inactivity. During this state, the device 1300 can power down for brief intervals of time and thus save power.

If there is no data traffic activity for an extended period of time, then the device 1300 can transition off to an RRC_Idle state, where it disconnects from the network and does not perform operations such as channel quality feedback, handover, etc. The device 1300 goes into a very low power state and it performs paging where again it periodically wakes up to listen to the network and then powers down again. The device 1300 may not receive data in this state; to receive data, it can transition back to RRC_Connected state.

An additional power saving mode can allow a device to be unavailable to the network for periods longer than a paging interval (ranging from seconds to a few hours) . During this time, the device is totally unreachable to the network and can power down completely. Any data sent during this time incurs a large delay and it is assumed the delay is acceptable.

Processors of the application circuitry 1302 and processors of the baseband circuitry 1304 can be used to execute elements of one or more instances of a protocol stack. For example, processors of the baseband circuitry 1304, alone or in combination, can be used execute Layer 3, Layer 2, or Layer 1 functionality, while processors of the baseband circuitry 1304 can utilize data (e.g., packet data) received from these layers and further execute Layer 4 functionality (e.g., transmission communication protocol (TCP) and user datagram protocol (UDP) layers) . As referred to herein, Layer 3 can comprise a RRC layer, described in further detail below. As referred to herein, Layer 2 can comprise a medium access control (MAC) layer, a radio link control (RLC) layer, and a packet data convergence protocol (PDCP) layer, described in further detail below. As referred to herein, Layer 1 can comprise a physical (PHY) layer of a UE/RAN node, described in further detail below.

Fig. 14 is a diagram of example interfaces of baseband circuitry according to one or more implementations described herein. As discussed above, the baseband circuitry 1304 of Fig. 13 can comprise processors 1304A-1304E and a memory 1304G utilized by said processors. Each of the processors 1304A-1304E can include a memory interface, 1404A-1404E, respectively, to send/receive data to/from the memory 1304G.

The baseband circuitry 1304 can further include one or more interfaces to communicatively couple to other circuitries/devices, such as a memory interface 1412 (e.g., an interface to send/receive data to/from memory external to the baseband circuitry 1304) , an application circuitry interface 1414 (e.g., an interface to send/receive data to/from the application circuitry 1302 of Fig. 13) , an RF circuitry interface 1416 (e.g., an interface to send/receive data to/from RF circuitry 1306 of Fig. 11) , a wireless hardware connectivity interface 1418 (e.g., an interface to send/receive data to/from Near Field Communication (NFC) components, Bluetooth components, Wi-Fi components, and other communication components) , and a power management interface 1420 (e.g., an interface to send/receive power or control signals to/from the PMC 1312) .

Examples herein can include subject matter such as a method, means for performing acts or blocks of the method, at least one machine-readable medium including executable instructions that, when performed by a machine (e.g., a processor (e.g., processor , etc. ) with memory, an application-specific integrated circuit (ASIC) , a field programmable gate array (FPGA) , or the like) cause the machine to perform acts of the method or of an apparatus or system for concurrent communication using multiple communication technologies according to implementations and examples described.

In Example 1, which may also include one or more of the examples described herein, a device for positioning model monitoring comprises a memory and a processor, coupled to the memory and when executing instructions stored in the memory, configured to perform operations comprising obtaining an inference result for model monitoring, wherein the inference result is output by an artificial intelligence/machine learning (AI/ML) model for determining a position of a user equipment (UE) ; obtaining a ground truth label for model monitoring corresponding to the inference result; and performing a monitoring function by comparing the inference result to the ground truth label.

Example 2 includes the subject matter of example 1, including or omitting optional elements, wherein the AI/ML model is hosted by the UE, and the inference result is output by the UE and comprised of the position of the UE.

Example 3 includes the subject matter of example 1, including or omitting optional elements, wherein the inference result is a position of the UE output by the AL/ML model, and the ground truth label is a position estimation of the UE derived by a positioning method other than the AL/ML model.

Example 4 includes the subject matter of example 3, including or omitting optional elements, wherein the ground truth label is derived by a location management function (LMF) instantiated on a network device.

Example 5 includes the subject matter of example 3, including or omitting optional elements, wherein the ground truth label is derived by the UE based on positioning assistance information received from a location management function (LMF) .

Example 6 includes the subject matter of example 1, including or omitting optional elements,

Example 6 includes the subject matter of example 1, including or omitting optional elements, wherein the inference result is a position of a positioning reference unit (PRU)

Example 7 includes the subject matter of example 6, including or omitting optional elements, wherein the device is or composes the UE, and wherein the known position of the PRU is communicated to the UE via a location management function (LMF) using LTE positioning protocol (LPP) .

Example 8 includes the subject matter of example 6, including or omitting optional elements, wherein the device is or composes the UE, and wherein the known position of the PRU is communicated to the UE directly by the PRU using sidelink, proprietary, and/or high layer protocol.

Example 9 includes the subject matter of example 1, including or omitting optional elements, wherein the PRU is identified for monitoring model performance, based on criteria including one or more of: sharing the same network conditions with the UE, locating within a proximity range of the UE, or applying a common inference measurement with the UE for monitoring.

Example 10 includes the subject matter of example 6, including or omitting optional elements, wherein the device is or composes a location management function (LMF) instantiated on a network device, and wherein the known position of the PRU is communicated to the LMF using LPP.

Example 11 includes the subject matter of example 1, including or omitting optional elements, wherein the device is or composes the UE.

Example 12 includes the subject matter of example 1, including or omitting optional elements, wherein the device is or composes a location management function (LMF) instantiated on a network device.

Example 13 includes the subject matter of example 1, including or omitting optional elements, wherein the operations further comprise performing a monitoring action responsive to the comparison meeting a pre-determined criteria.

In Example 14, which may also include one or more of the examples described herein, a method of model monitoring for artificial intelligence/machine learning (AI/ML) positioning, comprising: communicating, to a location management function (LMF) , user equipment (UE) capability information indicating support of a label-based model monitoring procedure; generating an inference result for model monitoring, wherein the inference result is output by an AI/ML mode based on a measurement of a downlink reference signal; obtaining a ground truth label for model monitoring corresponding to the inference result; and performing a monitoring function by comparing the inference result to the ground truth label.

Example 15 includes the subject matter of example 14, including or omitting optional elements, wherein the UE capability information further indicates a method of generating the ground truth label and an input needed by the AI/ML model.

Example 16 includes the subject matter of example 14, including or omitting optional elements, further comprising: receiving, from the LMF, the downlink reference signal; performing the measurement of the downlink reference signal; communicating, to the LMF, the measurement of the downlink reference signal; and receiving, from the LMF, the ground truth label derived by a positioning method other than the AL/ML model based on the measurement of the downlink reference signal.

Example 17 includes the subject matter of example 14, including or omitting optional elements, further comprising receiving, from the LMF, positioning assistance information; and generating the ground truth label based on the measurement of the downlink reference signal and the positioning assistance information by a positioning method other than the AL/ML model.

Example 18 includes the subject matter of example 14, including or omitting optional elements, further comprising receiving, from the LMF, the measurement of the downlink reference signal performed by a position of a reference unit (PRU) ; and receiving, from the LMF, the ground truth label based on a known position of the PRU.

Example 19 includes the subject matter of example 14, including or omitting optional elements, further comprising receiving, from a reference unit (PRU) , the measurement of the downlink reference signal and the ground truth label based on a known position of the PRU.

Example 20 includes the subject matter of example 14, including or omitting optional elements, further comprising performing a monitoring action responsive to the comparison meeting pre-determined monitoring criteria.

In Example 21, which may also include one or more of the examples described herein, a baseband processor, when executing instructions stored in a memory coupled to the baseband processor, configured to perform operations comprising: providing, for communicating to a location management function (LMF) , UE capability information indicating support of a label-based model monitoring procedure; generating an inference result for model monitoring, wherein the inference result is output by an artificial intelligence/machine learning (AI/ML) model; obtaining a ground truth label for model monitoring corresponding to the inference result; and performing a monitoring function by comparing the inference result to the ground truth label.

Example 22 includes the subject matter of example 14, including or omitting optional elements, wherein the inference result is a position of a UE, and the ground truth label is an estimate position of the UE derived by a positioning method independent from the AL/ML model and based on a positioning assistance information communicated by the LMF.

Example 23 includes the subject matter of example 14, including or omitting optional elements, wherein the inference result is a position of a reference unit (PRU) , and the ground truth label is a known position of the PRU communicated by the PRU via the LMF.

Example 24 is an apparatus that includes means for performing functions corresponding to the operations performed by the baseband processor or one or more processors or devices of examples 1-13 and 21-23.

Example 25 is a UE including the baseband processor of examples 21-23.

Example 26 is a method that includes any action or combination of actions as substantially described herein in the Detailed Description.

Example 27 is a method as substantially described herein with reference to each or any combination of the Figures included herein or with reference to each or any combination of paragraphs in the Detailed Description.

Example 28 is a user equipment configured to perform any action or combination of actions as substantially described herein in the Detailed Description as included in the user equipment.

Example 29 is a network node configured to perform any action or combination of actions as substantially described herein in the Detailed Description as included in the network node.

Example 30 is a non-volatile computer-readable medium that stores instructions that, when executed, cause the performance of any action or combination of actions as substantially described herein in the Detailed Description.

Example 31 is a baseband processor of a user equipment configured to perform any action or combination of actions as substantially described herein in the Detailed Description as included in the user equipment.

Example 32 is a baseband processor of a network node configured to perform any action or combination of actions as substantially described herein in the Detailed Description as included in the user equipment.

Example 33 is a method that includes functions corresponding to the operations performed by the baseband processor or one or more processors of examples 1-13 and 21-23.

Example 34 is an apparatus that includes means for performing functions corresponding to the operations performed by the baseband processor or one or more processors of examples 1-13 and 21-23.

Example 35 is a UE configured to perform operations of examples 14-20.

Other examples may include a method (e.g., a process) and/or a computer-readable medium implementation of any of the foregoing examples or combinations thereof. The above description of illustrated examples, implementations, aspects, etc., of the subject disclosure, including what is described in the Abstract, is not intended to be exhaustive or to limit the disclosed aspects to the precise forms disclosed. While specific examples, implementations, aspects, etc., are described herein for illustrative purposes, various modifications are possible that are considered within the scope of such examples, implementations, aspects, etc., as those skilled in the relevant art can recognize.

In this regard, while the disclosed subject matter has been described in connection with various examples, implementations, aspects, etc., and corresponding Figures, where applicable, it is to be understood that other similar aspects can be used or modifications and additions can be made to the disclosed subject matter for performing the same, similar, alternative, or substitute function of the subject matter without deviating therefrom. Therefore, the disclosed subject matter should not be limited to any single example, implementation, or aspect described herein, but rather should be construed in breadth and scope in accordance with the appended claims below.

In particular regard to the various functions performed by the above described components or structures (assemblies, devices, circuits, systems, etc. ) , the terms (including a reference to a “means” ) used to describe such components are intended to correspond, unless otherwise indicated, to any component or structure which performs the specified function of the described component (e.g., that is functionally equivalent) , even though not structurally equivalent to the disclosed structure which performs the function in the herein illustrated exemplary implementations. In addition, while a particular feature may have been disclosed with respect to only one of several implementations, such feature may be combined with one or more other features of the other implementations as may be desired and advantageous for any given, or particular, application.

As used herein, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or” . That is, unless specified otherwise, or clear from context, “X employs A or B” is intended to mean any of the natural inclusive permutations. That is, if X employs A; X employs B; or X employs both A and B, then “X employs A or B” is satisfied under any of the foregoing instances. In addition, the articles “a” and “an” as used in this application and the appended claims should generally be construed to mean “one or more” unless specified otherwise or clear from context to be directed to a singular form. Furthermore, to the extent that the terms “including” , “includes” , “having” , “has” , “with” , or variants thereof are used in either the detailed description or the claims, such terms are intended to be inclusive in a manner similar to the term “comprising. ” Additionally, in situations wherein one or more numbered items are discussed (e.g., a “first X” , a “second X” , etc. ) , in general the one or more numbered items can be distinct, or they can be the same, although in some situations the context may indicate that they are distinct or that they are the same.

It is well understood that the use of personally identifiable information should follow privacy policies and practices that are generally recognized as meeting or exceeding industry or governmental requirements for maintaining the privacy of users. In particular, personally identifiable information data should be managed and handled so as to minimize risks of unintentional or unauthorized access or use, and the nature of authorized use should be clearly indicated to users.

Claims

A device for positioning model monitoring, comprising:

a memory; and

a processor, coupled to the memory and when executing instructions stored in the memory, configured to perform operations comprising:

obtaining an inference result for model monitoring, wherein the inference result is output by an artificial intelligence/machine learning (AI/ML) model for determining a position of a user equipment (UE) ;

obtaining a ground truth label for model monitoring corresponding to the inference result; and

performing a monitoring function by comparing the inference result to the ground truth label.
The device of claim 1, wherein the AI/ML model is hosted by the UE, and the inference result is output by the UE and comprised of the position of the UE.
The device of claim 1, wherein the inference result is a position of the UE output by the AL/ML model, and the ground truth label is a position estimation of the UE derived by a positioning method other than the AL/ML model.
The device of claim 3, wherein the ground truth label is derived by a location management function (LMF) instantiated on a network device.
The device of claim 3, wherein the ground truth label is derived by the UE based on positioning assistance information received from a location management function (LMF) .
The device of claim 1, wherein the inference result is a position of a positioning reference unit (PRU) output by the AL/ML model, and the ground truth label is a known position of the PRU communicated by the PRU.
The device of claim 6,

wherein the device is or composes the UE, and

wherein the known position of the PRU is communicated to the UE via a location management function (LMF) using LTE positioning protocol (LPP) .
The device of claim 6,

wherein the device is or composes the UE, and

wherein the known position of the PRU is communicated to the UE directly by the PRU using sidelink, proprietary, and/or high layer protocol.
The device of claim 6, wherein the PRU is identified for monitoring model performance, based on criteria including one or more of: sharing the same network conditions with the UE, locating within a proximity range of the UE, or applying a common inference measurement with the UE for monitoring.
The device of claim 6,

wherein the device is or composes a location management function (LMF) instantiated on a network device, and

wherein the known position of the PRU is communicated to the LMF using LPP.
The device of claim 1, wherein the device is or composes the UE.
The device of claim 1, wherein the device is or composes a location management function (LMF) instantiated on a network device.
The device of claim 1, wherein the operations further comprise performing a monitoring action responsive to the comparison meeting a pre-determined criteria.
A method of model monitoring for artificial intelligence/machine learning (AI/ML) positioning, comprising:

communicating, to a location management function (LMF) , user equipment (UE) capability information indicating support of a label-based model monitoring procedure;

generating an inference result for model monitoring, wherein the inference result is output by an AI/ML mode based on a measurement of a downlink reference signal;

obtaining a ground truth label for model monitoring corresponding to the inference result; and

performing a monitoring function by comparing the inference result to the ground truth label.
The method of claim 14, wherein the UE capability information further indicates a method of generating the ground truth label and an input needed by the AI/ML model.
The method of claim 14, further comprising:

receiving, from the LMF, the downlink reference signal;

performing the measurement of the downlink reference signal;

communicating, to the LMF, the measurement of the downlink reference signal; and

receiving, from the LMF, the ground truth label derived by a positioning method other than the AL/ML model based on the measurement of the downlink reference signal.
The method of claim 14, further comprising:

receiving, from the LMF, positioning assistance information; and

generating the ground truth label based on the measurement of the downlink reference signal and the positioning assistance information by a positioning method other than the AL/ML model.
The method of claim 14, further comprising:

receiving, from the LMF, the measurement of the downlink reference signal performed by a position of a reference unit (PRU) ; and

receiving, from the LMF, the ground truth label based on a known position of the PRU.
The method of claim 14, further comprising:

receiving, from a reference unit (PRU) , the measurement of the downlink reference signal and the ground truth label based on a known position of the PRU.
The method of claim 14, further comprising performing a monitoring action responsive to the comparison meeting pre-determined monitoring criteria.
A baseband processor, when executing instructions stored in a memory coupled to the baseband processor, configured to perform operations comprising:

providing, for communicating to a location management function (LMF) , UE capability information indicating support of a label-based model monitoring procedure;

generating an inference result for model monitoring, wherein the inference result is output by an artificial intelligence/machine learning (AI/ML) model;

obtaining a ground truth label for model monitoring corresponding to the inference result; and

performing a monitoring function by comparing the inference result to the ground truth label.
The baseband processor of claim 21, wherein the inference result is a position of a UE, and the ground truth label is an estimate position of the UE derived by a positioning method independent from the AL/ML model and based on a positioning assistance information communicated by the LMF.
The baseband processor of claim 21, wherein the inference result is a position of a reference unit (PRU) , and the ground truth label is a known position of the PRU communicated by the PRU via the LMF.