[go: up one dir, main page]

WO2024176170A1 - Surveillance de dérive de modèle pour positionnement reposant sur ia/ml par distribution statistique conjointe et métrique de performance - Google Patents

Surveillance de dérive de modèle pour positionnement reposant sur ia/ml par distribution statistique conjointe et métrique de performance Download PDF

Info

Publication number
WO2024176170A1
WO2024176170A1 PCT/IB2024/051730 IB2024051730W WO2024176170A1 WO 2024176170 A1 WO2024176170 A1 WO 2024176170A1 IB 2024051730 W IB2024051730 W IB 2024051730W WO 2024176170 A1 WO2024176170 A1 WO 2024176170A1
Authority
WO
WIPO (PCT)
Prior art keywords
model
stage
processing circuitry
host
threshold
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/IB2024/051730
Other languages
English (en)
Inventor
Zhan Zhang
Yufei Blankenship
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Telefonaktiebolaget LM Ericsson AB
Original Assignee
Telefonaktiebolaget LM Ericsson AB
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Telefonaktiebolaget LM Ericsson AB filed Critical Telefonaktiebolaget LM Ericsson AB
Publication of WO2024176170A1 publication Critical patent/WO2024176170A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S5/00—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations
    • G01S5/02—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations using radio waves
    • G01S5/0205—Details
    • G01S5/0244—Accuracy or reliability of position solution or of measurements contributing thereto
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S5/00—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations
    • G01S5/02—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations using radio waves
    • G01S5/0278—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations using radio waves involving statistical or probabilistic considerations
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S5/00—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations
    • G01S5/02—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations using radio waves
    • G01S5/0252—Radio frequency fingerprinting
    • G01S5/02521—Radio frequency fingerprinting using a radio-map
    • G01S5/02524—Creating or updating the radio-map
    • G01S5/02527—Detecting or resolving anomalies in the radio frequency fingerprints of the radio-map

Definitions

  • Embodiments of the invention relate to the field of networking; and more specifically, to machine learning model monitoring with autoencoder used in a network.
  • Al/machine learning (ML) enabled solutions essentially employ data-driven learning approaches where the models learn the underlying data distribution and relationship between the inputs and outputs without the need for understanding the underlying complex processes.
  • ML has been found to be an effective tool in radio positioning, for instance, 3GPP has now been investigating an AI/ML based positioning method that can include channel state information or time of arrival measurements based on a so-called fingerprint method for positioning, especially for indoor.
  • a 3GPP study item on fingerprint-based machine learning method for indoor position has been under progress.
  • Figure 1 shows an example scenario of radio propagation.
  • Different radio propagations can result in different channel features, such as channel coherent bandwidth, channel variation over time and space.
  • One important feature is that the channel becomes rich multipath at indoor, especially when the indoor is fully occupied with many so-called clutters, such as machines and storages.
  • the line of sight (LOS) between the network node or transmission reception point (TRP) and the User-terminal (UE) is seldomly available.
  • the actual radio propagation environment shapes the channel states to be spatial selective, hence, a direct triangularization method assuming all radio paths being LOS have an unsatisfactory performance. Due to constant time-varying environment change, all positioning estimation models including machine learning (ML) model face a possibility of performance drift.
  • ML machine learning
  • a general embodiment includes a method performed by a user equipment (UE) for a multi-stage drift monitoring of an AI/ML based radio positioning model.
  • the method includes collecting a model inference dataset of the radio positioning model, performing (at a first stage of the multi-stage drift monitoring) a distance metric calculation of the model inference dataset against a training dataset of the radio positioning model to obtain a distance metric, comparing the distance metric against a first threshold, determining (based on the comparison) that the distance metric exceeds the first threshold, and responsive to this determination, performing a second stage of the multi-stage drift monitoring.
  • the method further includes performing (at the second stage of the multi-stage drift monitoring) a performance-based metric calculation that measures a quality of model output to obtain a performance metric, comparing the performance threshold against a second threshold, and determining (based on the comparison) that the performance metric exceeds the second threshold, and responsive to this determination, transmitting a message that indicates that a drift of the radio positioning model is detected.
  • the model inference dataset is model input data.
  • the model inference dataset is model output data.
  • the first stage is performed without a ground truth label or performance metric. Determining that the distance metric exceeds the first threshold is an indication of possible model drift.
  • Performing the performance-based metric calculation includes receiving an actual position of a user equipment (UE) and checking it against an estimated position of the UE.
  • Performing the distance metric calculation includes Kullback-Leibler (KL) divergence of probability density functions (PDF) or a derivative of cumulative density function (CDF) of the training dataset and the model inference dataset.
  • Embodiments of the method may further include obtaining user data and forwarding the user data to a host or a user equipment.
  • Another general embodiment includes a method performed by a network node for a multi-stage drift monitoring of an AI/ML based radio positioning model.
  • the method includes collecting a model inference dataset of the radio positioning model, performing, at a first stage of the multi-stage drift monitoring, a distance metric calculation of the model inference dataset against a training dataset of the radio positioning model to obtain a distance metric, comparing the distance metric against a first threshold, and determining, based on the comparison, that the distance metric exceeds the first threshold, and responsive to this determination, performing a second stage of the multi-stage drift monitoring.
  • the method further includes performing, at the second stage of the multi-stage drift monitoring, a performance-based metric calculation that measures a quality of model output to obtain a performance metric, comparing the performance threshold against a second threshold, and determining, based on the comparison, that the performance metric exceeds the second threshold, and responsive to this determination, transmitting a message that indicates that a drift of the radio positioning model is detected.
  • the model inference dataset is model input data.
  • the model inference dataset is model output data.
  • the first stage is performed without a ground truth label or performance metric. Determining that the distance metric exceeds the first threshold is an indication of possible model drift.
  • Performing the performance-based metric calculation includes receiving actual position of a user equipment (UE) and checking against an estimated position of the UE.
  • Performing the distance metric calculation includes Kullback-Leibler (KL) divergence of probability density functions (PDF) or a derivative of cumulative density function (CDF) of the training dataset and the model inference dataset.
  • Embodiments may further include operations of obtaining user data and forwarding the user data to a host or a user equipment.
  • Another general embodiment includes a user equipment for a multi-stage drift monitoring of an AI/ML based radio positioning model.
  • the user equipment includes processing circuitry configured to perform any of the steps of any of the methods described herein and power supply circuitry configured to supply power to the processing circuitry.
  • Another general embodiment includes a network node for a multi-stage drift monitoring of an AI/ML based radio positioning model.
  • the network node includes processing circuitry configured to perform any of operations described herein and power supply circuitry configured to supply power to the processing circuitry.
  • a user equipment (UE) for a multi-stage drift monitoring of an AI/ML based radio positioning model is described.
  • the UE includes an antenna configured to send and receive wireless signals, radio front-end circuitry connected to the antenna and to processing circuitry, and configured to condition signals communicated between the antenna and the processing circuitry.
  • the processing circuitry is configured to perform any of the methods described herein.
  • the UE further includes an input interface connected to the processing circuitry and configured to allow input of information into the UE to be processed by the processing circuitry, an output interface connected to the processing circuitry and configured to output information from the UE that has been processed by the processing circuitry, and a battery connected to the processing circuitry and configured to supply power to the UE.
  • Figure 1 shows an example scenario of radio propagation.
  • Figure 2 illustrates an exemplary two-stage detection of model drift, according to an embodiment.
  • Figure 3 is a flow diagram that illustrates exemplary operations for performing a multi-stage drift monitoring of an AI/ML based radio positioning model, according to an embodiment.
  • Figure 4 shows an example of a communication system, in accordance with some embodiments.
  • Figure 5 shows a UE, in accordance with some embodiments.
  • Figure 6 shows a network node, in accordance with some embodiments.
  • FIG. 7 is a block diagram of a host, in accordance with various aspects described herein.
  • Figure 9 shows a communication diagram of a host communicating via a network node with a UE over a partially wireless connection, in accordance with some embodiments.
  • Positioning integrity is a measure of trust in the accuracy of the position-related estimation/data provided by the positioning system and the ability to provide timely and valid warnings to the Location Services (LCS) client when the positioning system does not fulfil the condition for intended operation. Integrity is focused on the tail of the positioning error distribution (i.e., the rare events), and aims to keep the probability of hazardous events extremely low. For example, ⁇ 10" 7 /hr Target Integrity Risk (TIR) translates to one failure permitted every 10 million hours (equivalent to 1142 years approximately).
  • TIR Target Integrity Risk
  • embodiments monitor the pretrained ML model for positioning on its possible drifts in performance, due to either radio environment changes or device failure/interference resultant measurement distortions on radio signals.
  • the precondition is the high quality of measurement data of radio signal features (time delay or RSRP, channel impulse responses, etc.) with an accurate label of its positioning.
  • radio signal features time delay or RSRP, channel impulse responses, etc.
  • these features may be time varying due to many factors such as indoor radio environment changes by indoor equipment location changes.
  • the radio signals may suffer time-varying or constant/occasional interferences or device malfunctions in the reality. Therefore, this gives a rise of issue on monitoring the AI/ML models on its possible positioning drifts.
  • ground truth label the true value of a variable or parameter that is estimated at a model with some input data
  • a two-stage drift monitoring method can be applied.
  • the ground truth label is typically the actual UE position.
  • the first stage uses a distribution-based metric, which can be run without the need of the ground truth label or its approximation. If a warning is triggered by the first-stage monitoring, then the monitoring is escalated to the second-stage monitoring. If a warning is not triggered, then the monitoring does not proceed with the second-stage monitoring.
  • the node hosting the ML model can make a request to one or more node(s), such as one or more positioning reference units (PRUs), to have higher-accuracy positioning info.
  • the node hosting the ML model may be at the UE or a network node.
  • the request is for ground truth label (e.g., the actual position of the UE) or a performance metric.
  • the node with the ML model can trigger timely denser periodic sounding signal transmissions and estimations on the positions of the UEs and then track the estimation data within a duration.
  • the UE makes a request to the location management function (LMF) to have the estimated UE location so that the UE can perform second-stage monitoring of UE-side AI/ML model or the (PRUs) UEs send their measurements to the LMF for processing.
  • LMF location management function
  • the estimated UE location/reported location can be used for model monitoring at the second stage.
  • Certain embodiments may provide one or more of the following technical advantage(s).
  • the two-stage solution to manage the model monitoring service by AI/ML methods of radio access technology (RAT) network reduces cost.
  • the first stage may be performed without a ground truth label or its approximation (which may be costly to obtain such as incurred signaling overhead, assistance information from a separate entity, and potentially extra reference signal transmission).
  • the second stage is performed only if the first stage indicates a possible model drift.
  • the second stage of model drift monitoring is invoked only when it is worthwhile to have a more accurate diagnosis.
  • the first stage uses a distribution-based metric that can be run without the need of the ground truth label or its approximation. If a warning is triggered by the first-stage monitoring, then it is escalated to the second-stage monitoring.
  • the node with the ML model can make a request to one or more nodes, such as PRUs, to have higher-accuracy positioning info.
  • the request is for ground truth label or performance metric.
  • the node with the ML model can trigger timely denser periodic sounding signal transmissions and estimations on the positions of the UEs and then track the estimation data within a duration.
  • the UE makes a request to the LMF to have the estimated UE location so that the UE can perform second-stage monitoring of UE-side AI/ML model or the (PRUs) UEs send their measurements to the LMF for processing.
  • the estimated UE location/reported location can be used for model monitoring at the second stage.
  • a positioning monitoring function is established and operates to secure integrity requirement.
  • Distribution based metrics includes relative-entropy or so-called Kullback-Leibler (KL) divergence of probability density functions (PDF) or derivative of cumulative density function (CDF) of the training input datapoints and operational inputs datapoints.
  • KL Kullback-Leibler
  • PDF probability density functions
  • CDF cumulative density function
  • the indoor radio propagation environment continually changes.
  • a change in the physical environment can change indoor radio propagation.
  • a machine can move or leave an area.
  • the indoor furniture and/or decoration can change. These changes could change the measurement data set and the relationship of datapoints and their corresponding labels (positions). These changes can cause drift of an AI/ML model.
  • KL divergence has a difficulty handling highly dimensional features in a data-point.
  • the KL divergence calculation is easier built to test the probability function p(x) against probability function q(x) for lower dimension data. If x is of a higher dimension, usually, p(x) or q(x) is not so easy to obtain in a strict mathematically defined sense. If there are a long list of K features X o , X lt ... , X ⁇ _ 17 it is difficult to directly calculate KL divergence for the joint probability function p x 0 , X lt ... , X K _ ⁇ ). Thus, there are some approximation solutions to handle this.
  • PCA Principal component analysis
  • Another approximation solution is to employ domain knowledge to scale the input data dimension. Domain knowledge is used to select a small number of most significant features to focus on, rather than all features (all input elements) at the model input. For example, for the use case of UE positioning with a network-side model, the important features to focus on may be one of the following:
  • TRPs transmission reception points
  • n selected TRPs are those with the highest uplink (UL) sounding reference signal (SRS) received power (UL SRS-RSRP) or highest reference signal received UL SRS path power (UL SRS-RSRPP) of the first path.
  • SRS sounding reference signal
  • UL SRS-RSRP highest uplink
  • UL SRS-RSRPP highest reference signal received UL SRS path power
  • n + n 2 is limited to a small number
  • different weights can be applied to each, with higher weight Wj to more important feature(s), and lower weight w ⁇ - to less important features.
  • Another approximation solution is to approximate the probability function by assuming the independence among the features.
  • the joint probability function of feature vectors becomes a cascading multiplication of probabilities of each feature. In such a way, an approximate joint probability function would be available for calculating the KL divergence between the training set of feature vectors and online data set.
  • KL divergence in a discrete format is as follows:
  • the first step obtains the probability functions of p(x): the function of training data set, and q(x): the function for online operational data set, as discussed above.
  • KL divergence is a non-negative real number. The larger the number, the bigger difference between the distributions, and the heavier shifts in data sets.
  • a threshold is set to make the calculated divergence value as a metric for detecting input data set distribution shift.
  • One example is that a Gaussian distribution (or Dirichlet Distribution) is assumed as an approximate distribution type for a metric of stage 1. Then the means and deviation are used to specify the concrete PDF or CDF. Or the means or deviation matrixes are directly used to calculate the relative entropy of the assumed PDF function.
  • Gaussian distribution or Dirichlet Distribution
  • the model drift monitoring method can be extended to include the following metric in the model drift detection.
  • metric for example, statistical-moments-based metric, different statistical moments are n-order statistics, such as E (X), E (X 2 ), E (X 3 ), E (X 4 ) or E((X-M) 2 ) could be used as inputs to build metric for stage 1 detection of model drift.
  • one type of data-set distribution distance could be defined as: where the element division calculation of stochastic means of datapoint matrixes/vectors is done first, and then a Frobenius norm is calculated over dimensions of the datapoint.
  • P(x) stands for training data set PDF
  • q(x) represents the PDF for online operational data set at model monitoring.
  • Affine Invariant Riemannian Metric a distance metric can be defined as follows:
  • the monitoring is escalated to the second stage.
  • the second stage tends to be more costly. For example, the second stage may incur signaling overhead, assistance information from a separate entity, and potentially extra reference signal transmission. Thus, the second stage of monitoring is invoked when it is worthwhile to have a more accurate diagnose.
  • a performance-based metric (e.g., an error statistic) is computed.
  • the ground truth label is needed, which corresponds to the model input during model inference stage.
  • the ground truth label may be sent to the model drift monitoring (MDM) entity.
  • MDM model drift monitoring
  • the ground truth label is compared with the model output to obtain an estimate of its quality (i.e., performance metric). Without losing generality, the methods and signaling are explained below using the exemplary use case of AI/ML based UE positioning.
  • the ground truth label is typically the actual UE position. That is, actual UE position(s) are sent to the MDM entity to analyze. Sending actual UE position requires data label reporting, such as requiring positioning reference units (PRUs) to report position. For instance, the PRUs report their labels (positions if the AI/ML model monitored is a position service providing model) and the monitoring function checks the errors of the estimated results by the AI/ML model monitored against the reported data labels (positions). To say it another way, the MDM can compare the actual position of the UEs with the estimated position of the UEs (estimated by the AI/ML model) to obtain a performance-based metric (e.g., a performance error statistic). This performance error statistic provides a measure of quality of the AI/ML model output. This performance error statistic is used as a metric for the second stage model drift detection.
  • a performance-based metric e.g., a performance error statistic
  • Key signaling with PRUs include one or more of the following: system-info-message, PRU registration message, PRU registration confirmation-message, and configurations of PRU measurement and reporting.
  • the system-info-message may include the MDM broadcasting its reporting identifier (address/port to receive the reports) and reporting channels.
  • the PRU registration message may include a PRU sending a request according to an MDM (according to the system-info-message) to register at the MDM.
  • the PRU registration confirmation-message may include the MDM checking the PRU registration message and sending a registration confirmation-message to PRU.
  • the registration confirmation-message may include (along with configurations of PRU measurement and reporting) the PRU ID and attached TRPs.
  • the configurations of PRU measurement and reporting may include: format/periodicity; thresholds; thresholds for event-triggered reporting if PRU takes some predefined detection of drift-relevant metric; PRU capability of obtaining “ground-truth” values, its stationarity/mobility state, and the measurement accuracy; PRU capability of computing/reasoning; signaling radio bearer ID for reporting PDU from PRU to eNBs to MDM; and PRU preprocessing on measurements: algorithm/parameters, PRU-activating and deactivating message.
  • the format/periodicity includes the measurement report format and in what period the measurement is done and reported.
  • the PRU capabilities include determining how fast a PRU can complete a certain computing task, how large its data memory size is, etc.
  • the key signaling with TRPs of positioning functions include one or more of the following: MDM requests on TRPs/eNBs to share measurement data; MDM configures TRPs/eNBs on its measurement report: contents and formats such as TRP clustering/coordination on synchronized measurements.
  • Figure 2 illustrates an exemplary two-stage detection of model drift according to an embodiment.
  • Figure 2 shows the operational states of the model drift monitoring function according to an embodiment.
  • MDM model drift monitoring
  • the MDM entity may be a UE or a network node depending on the implementation.
  • the model inference dataset 260 is collected.
  • the model inference dataset 260 is the dataset that the model experiences during model deployment.
  • the model inference dataset 260 may be the model input data or model output data.
  • the model inference dataset 260 may be collected from a data collection unit. As shown in Figure 2, the model inference dataset 260 is collected from the model tuning completion/running 255, which represents the model being monitored (and re-tuning the model).
  • stage 1 215 the MDM entity performs a distance metric calculation of the model inference dataset 260 against the training dataset for the model to obtain a distance metric. If the training dataset is not available at the MDM, the MDM requests the training dataset from an entity that stores the training dataset. The distance metric calculation may be performed like as described earlier herein. The distance metric calculation is periodically carried out and the distance metric is compared to a threshold. If the distance metric of the datasets (the model inference dataset 260 and the training dataset) exceeds a predefined threshold, then it triggers a state change to enter the second stage 225 of detection. Otherwise, there is no state change, and the monitoring stays at stage 1 215. As shown in Figure 2, the state transitions to stage 2 225 when the distance metric (Ml) is greater than the threshold_l 220.
  • the MDM entity performs a performance-based model monitoring.
  • the MDM entity performs a performance-based metric calculation to obtain a performance metric.
  • the performance metric can be any metric that measures the quality of model output.
  • One typical performance metric is the error rate (or accuracy) of the model output. Error rate/residual error of estimation of AI/ML model monitored as a metric is included as a type of performance metric.
  • Performing the performance-based metric calculation may be done like as described earlier herein.
  • the MDM entity may receive the actual positions of UE(s) and for each compare that actual position with the estimated position of that UE to generate the performance metric.
  • Stage 2225 may trigger an alarm message and/or a performance metric reporting (e.g., error rate of model output).
  • stage 2 225 includes multiple observation time windows that are used in separate sub stages. For instance, in sub stage 1 230, in an observation time window T, if an alarm triggering criterion is satisfied, then the monitoring function may send out an alarm message to indicate that a model drift is detected.
  • various types of alarm triggering criterion can be used.
  • Examples include: (a) the error rate of the model output is larger than a second threshold (i.e., error rate surpasses a predefined alarm triggering threshold), (b) the statistical mean of estimation errors exceeds a predefined threshold, and (c) Euclidean distance between the model output and the ground truth label is greater than a predefined threshold.
  • a second threshold i.e., error rate surpasses a predefined alarm triggering threshold
  • the statistical mean of estimation errors exceeds a predefined threshold
  • Euclidean distance between the model output and the ground truth label is greater than a predefined threshold.
  • the performance metric collected over a period can be reported also. As shown in Figure 2, the state transitions from the sub stage 1 230 of the stage 2 225 to the suspend/alarm state 250 when the performance metric (M2) is greater than the threshold_2 during an observation time window T 240.
  • the state transitions to sub stage 2 235 of the stage 2 225.
  • the state transitions from the sub stage 1 230 to the sub stage 2 235 when the performance metric (M2) is not greater than the threshold_2 and the observation time window T has elapsed 242.
  • the MDM entity keeps monitoring the performance metric in a sliding window to determine whether the alarm triggering criterion is satisfied. If the continuous monitoring does not result in an alarm triggering criterion being satisfied within an extended time period (time window W), the monitoring function will re-enter stage 1 215 and follow the monitoring procedure at stage 1.
  • the sliding window can take the form of [x(t), x(t+l), ... x(t+s)] where s is the window size. If t+s ⁇ W and the performance metric is greater than the threshold_2, then an alarm is triggered. However, if t+s > W and the performance metric is not greater than the threshold_2, the monitoring function re-enters stage 1 215. As shown in Figure 2, the state transitions from the sub stage 2 235 to the suspend/alarm state 250 when the performance metric (M2) is greater than the threshold_2 during the sliding window of the observation time window W 245 ((T2 + S) ⁇ W). Also, the state transitions from the sub stage 2 235 back to stage 1 215 when the performance metric (M2) is not greater than the threshold_2 and the sliding window of the observation time window W has elapsed 247 ((T2 + S) > W).
  • the MDM entity may send out an alarm message to indicate that a model drift is detected.
  • the model tuning completion/running 255 may then start a fine tuning or retraining of the Al model.
  • Figure 2 shows multiple observation windows in stage 2, in an embodiment only a single observation window is used. In such an embodiment, if the performance metric exceeds the threshold within the single observation window, then the state transitions to the suspend/alarm state; otherwise, the state transitions back to stage 1.
  • the model monitoring function continuously monitors either or both the distribution distance metric and/or performance metric and shifts its monitoring stages.
  • the stage changes according to whether the metric values surpassing certain thresholds at certain time windows. If the model drifting happens, it will detect it in a high likelihood and send out an alarm message to the AI/ML model monitored for it to start a fine tuning or retraining.
  • Figure 3 is a flow diagram that illustrates exemplary operations for performing a multi-stage drift monitoring of an AI/ML based radio positioning model according to an embodiment.
  • the operations of Figure 3 are described as being performed by an MDM entity.
  • the MDM entity may be a UE or a network node depending on the implementation.
  • the MDM entity collects a model inference dataset of a radio positioning model.
  • the model inference dataset may be model input data or model output data.
  • the MDM entity performs, in a first stage of the multi-stage drift monitoring, a distance metric calculation of the model inference dataset against a training dataset of the radio positioning model to obtain a distance metric. If the training dataset is not available at the MDM entity, the MDM entity requests and receives the training dataset from an entity that stores the training dataset.
  • Performing the distance metric calculation may include Kullback- Leibler (KL) divergence of probability density functions (PDF) or derivative of cumulative density function (CDF) of the training dataset and the model inference dataset.
  • KL Kullback- Leibler
  • PDF probability density functions
  • CDF cumulative density function
  • the first stage may be performed without a ground truth label (e.g., the actual position of the radio) or a performance metric.
  • the MDM entity compares the distance metric against a first threshold.
  • the MDM entity determines, based on the comparison, whether the distance metric exceeds the first threshold. Exceeding this threshold is an indication of possible model drift. If the distance metric exceeds the first threshold, then operation 330 is performed (the multi-stage drift monitoring proceeds to the second stage of drift monitoring). If the distance metric does not exceed the first threshold (which is an indication that there is not model drift), then flow moves back to operation 310.
  • the MDM entity performs, at the second stage of the multi-stage drift monitoring, a performance-based metric calculation that measures a quality of model output to obtain a performance metric.
  • the performance metric can be any metric that measures the quality of model output.
  • One typical performance metric is the error rate (or accuracy) of the model output. Error rate/residual error of estimation of AI/ML model monitored as a metric is included as a type of performance metric.
  • Performing the performance-based metric calculation may include receiving actual positions of user equipments (UEs) and checking against the estimated positions of the UEs respectively.
  • UEs user equipments
  • the MDM entity compares the performance threshold against a second threshold.
  • the MDM entity determines, based on the comparison, whether the performance metric exceeds the second threshold. The determination may be performed over an observation window. Exceeding this second threshold is an indication that the model has drifted. If the performance metric exceeds the second threshold (which is an indication that the model has drifted), then operation 345 is performed. If the performance metric does not exceed the second threshold (which is an indication that the model has not drifted), then flow moves back to operation 310.
  • Figure 4 shows an example of a communication system 400 in accordance with some embodiments.
  • the communication system 400 includes a telecommunication network 402 that includes an access network 404, such as a radio access network (RAN), and a core network 406, which includes one or more core network nodes 408.
  • the access network 404 includes one or more access network nodes, such as network nodes 410a and 410b (one or more of which may be generally referred to as network nodes 410), or any other similar 3 rd Generation Partnership Project (3GPP) access node or non-3GPP access point.
  • 3GPP 3 rd Generation Partnership Project
  • the network nodes 410 facilitate direct or indirect connection of user equipment (UE), such as by connecting UEs 412a, 412b, 412c, and 412d (one or more of which may be generally referred to as UEs 412) to the core network 406 over one or more wireless connections.
  • UE user equipment
  • Example wireless communications over a wireless connection include transmitting and/or receiving wireless signals using electromagnetic waves, radio waves, infrared waves, and/or other types of signals suitable for conveying information without the use of wires, cables, or other material conductors.
  • the communication system 400 may include any number of wired or wireless networks, network nodes, UEs, and/or any other components or systems that may facilitate or participate in the communication of data and/or signals whether via wired or wireless connections.
  • the communication system 400 may include and/or interface with any type of communication, telecommunication, data, cellular, radio network, and/or other similar type of system.
  • the UEs 412 may be any of a wide variety of communication devices, including wireless devices arranged, configured, and/or operable to communicate wirelessly with the network nodes 410 and other communication devices.
  • the network nodes 410 are arranged, capable, configured, and/or operable to communicate directly or indirectly with the UEs 412 and/or with other network nodes or equipment in the telecommunication network 402 to enable and/or provide network access, such as wireless network access, and/or to perform other functions, such as administration in the telecommunication network 402.
  • the UEs 412 may perform, either alone or in cooperation with the network, the multi-stage drift monitoring described herein.
  • the core network 406 connects the network nodes 410 to one or more hosts, such as host 416. These connections may be direct or indirect via one or more intermediary networks or devices. In other examples, network nodes may be directly coupled to hosts.
  • the core network 406 includes one more core network nodes (e.g., core network node 408) that are structured with hardware and software components. Features of these components may be substantially similar to those described with respect to the UEs, network nodes, and/or hosts, such that the descriptions thereof are generally applicable to the corresponding components of the core network node 408.
  • Example core network nodes include functions of one or more of a Mobile Switching Center (MSC), Mobility Management Entity (MME), Home Subscriber Server (HSS), Access and Mobility Management Function (AMF), Session Management Function (SMF), Authentication Server Function (AUSF), Subscription Identifier De-concealing function (SIDE), Unified Data Management (UDM), Security Edge Protection Proxy (SEPP), Network Exposure Function (NEF), and/or a User Plane Function (UPF).
  • the core network node 408 may perform, either alone or in cooperation with the UEs and/or host 416, the multi-stage drift monitoring described herein.
  • the host 416 may be under the ownership or control of a service provider other than an operator or provider of the access network 404 and/or the telecommunication network 402, and may be operated by the service provider or on behalf of the service provider.
  • the host 416 may host a variety of applications to provide one or more service. Examples of such applications include live and pre-recorded audio/video content, data collection services such as retrieving and compiling data on various ambient conditions detected by a plurality of UEs, analytics functionality, social media, functions for controlling or otherwise interacting with remote devices, functions for an alarm and surveillance center, or any other such function performed by a server.
  • the host 416 may perform, either alone or in cooperation with network nodes (e.g., core network node 408) and/or the UEs, the multi-stage drift monitoring described herein.
  • the communication system 400 of Figure 4 enables connectivity between the UEs, network nodes, and hosts.
  • the communication system may be configured to operate according to predefined rules or procedures, such as specific standards that include, but are not limited to: Global System for Mobile Communications (GSM); Universal Mobile Telecommunications System (UMTS); Long Term Evolution (LTE), and/or other suitable 2G, 3G, 4G, 5G standards, or any applicable future generation standard (e.g., 6G); wireless local area network (WLAN) standards, such as the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standards (WiFi); and/or any other appropriate wireless communication standard, such as the Worldwide Interoperability for Microwave Access (WiMax), Bluetooth, Z-Wave, Near Field Communication (NFC) ZigBee, LiFi, and/or any low-power wide-area network (LPWAN) standards such as LoRa and Sigfox.
  • GSM Global System for Mobile Communications
  • UMTS Universal Mobile Telecommunications System
  • LTE Long Term Evolution
  • the telecommunication network 402 is a cellular network that implements 3GPP standardized features. Accordingly, the telecommunications network 402 may support network slicing to provide different logical networks to different devices that are connected to the telecommunication network 402. For example, the telecommunications network 402 may provide Ultra Reliable Low Latency Communication (URLLC) services to some UEs, while providing Enhanced Mobile Broadband (eMBB) services to other UEs, and/or Massive Machine Type Communication (mMTC)/Massive loT services to yet further UEs.
  • URLLC Ultra Reliable Low Latency Communication
  • eMBB Enhanced Mobile Broadband
  • mMTC Massive Machine Type Communication
  • the UEs 412 are configured to transmit and/or receive information without direct human interaction.
  • a UE may be designed to transmit information to the access network 404 on a predetermined schedule, when triggered by an internal or external event, or in response to requests from the access network 404.
  • a UE may be configured for operating in single- or multi-RAT or multi-standard mode.
  • a UE may operate with any one or combination of Wi-Fi, NR (New Radio) and LTE, i.e. being configured for multi-radio dual connectivity (MR-DC), such as E-UTRAN (Evolved-UMTS Terrestrial Radio Access Network) New Radio - Dual Connectivity (EN-DC).
  • MR-DC multi-radio dual connectivity
  • the hub 414 communicates with the access network 404 to facilitate indirect communication between one or more UEs (e.g., UE 412c and/or 412d) and network nodes (e.g., network node 410b).
  • the hub 414 may be a controller, router, content source and analytics, or any of the other communication devices described herein regarding UEs.
  • the hub 414 may be a broadband router enabling access to the core network 406 for the UEs.
  • the hub 414 may be a controller that sends commands or instructions to one or more actuators in the UEs.
  • the hub 414 may be a data collector that acts as temporary storage for UE data and, in some embodiments, may perform analysis or other processing of the data.
  • the hub 414 may be a content source. For example, for a UE that is a VR headset, display, loudspeaker or other media delivery device, the hub 414 may retrieve VR assets, video, audio, or other media or data related to sensory information via a network node, which the hub 414 then provides to the UE either directly, after performing local processing, and/or after adding additional local content.
  • the hub 414 acts as a proxy server or orchestrator for the UEs, in particular in if one or more of the UEs are low energy loT devices.
  • the hub 414 may have a constant/persistent or intermittent connection to the network node 410b.
  • the hub 414 may also allow for a different communication scheme and/or schedule between the hub 414 and UEs (e.g., UE 412c and/or 412d), and between the hub 414 and the core network 406.
  • the hub 414 is connected to the core network 406 and/or one or more UEs via a wired connection.
  • the hub 414 may be configured to connect to an M2M service provider over the access network 404 and/or to another UE over a direct connection.
  • UEs may establish a wireless connection with the network nodes 410 while still connected via the hub 414 via a wired or wireless connection.
  • the hub 414 may be a dedicated hub - that is, a hub whose primary function is to route communications to/from the UEs from/to the network node 410b.
  • the hub 414 may be a nondedicated hub - that is, a device which is capable of operating to route communications between the UEs and network node 410b, but which is additionally capable of operating as a communication start and/or end point for certain data channels.
  • FIG. 5 shows a UE 500 in accordance with some embodiments.
  • a UE refers to a device capable, configured, arranged and/or operable to communicate wirelessly with network nodes and/or other UEs.
  • Examples of a UE include, but are not limited to, a smart phone, mobile phone, cell phone, voice over IP (VoIP) phone, wireless local loop phone, desktop computer, personal digital assistant (PDA), wireless cameras, gaming console or device, music storage device, playback appliance, wearable terminal device, wireless endpoint, mobile station, tablet, laptop, laptop-embedded equipment (LEE), laptop-mounted equipment (LME), smart device, wireless customer-premise equipment (CPE), vehicle-mounted or vehicle embedded/integrated wireless device, etc.
  • VoIP voice over IP
  • LME laptop-embedded equipment
  • LME laptop-mounted equipment
  • CPE wireless customer-premise equipment
  • UEs identified by the 3 rd Generation Partnership Project (3GPP), including a narrow band internet of things (NB-IoT) UE, a machine type communication (MTC) UE, and/or an enhanced MTC (eMTC) UE.
  • 3GPP 3 rd Generation Partnership Project
  • NB-IoT narrow band internet of things
  • MTC machine type communication
  • eMTC enhanced MTC
  • a UE may support device-to-device (D2D) communication, for example by implementing a 3GPP standard for sidelink communication, Dedicated Short-Range Communication (DSRC), vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), or vehicle-to- everything (V2X).
  • a UE may not necessarily have a user in the sense of a human user who owns and/or operates the relevant device.
  • a UE may represent a device that is intended for sale to, or operation by, a human user but which may not, or which may not initially, be associated with a specific human user (e.g., a smart sprinkler controller).
  • a UE may represent a device that is not intended for sale to, or operation by, an end user but which may be associated with or operated for the benefit of a user (e.g., a smart power meter).
  • the UE 500 includes processing circuitry 502 that is operatively coupled via a bus 504 to an input/output interface 506, a power source 508, a memory 510, a communication interface 512, and/or any other component, or any combination thereof.
  • Certain UEs may utilize all or a subset of the components shown in Figure 5. The level of integration between the components may vary from one UE to another UE. Further, certain UEs may contain multiple instances of a component, such as multiple processors, memories, transceivers, transmitters, receivers, etc.
  • the processing circuitry 502 is configured to process instructions and data and may be configured to implement any sequential state machine operative to execute instructions stored as machine-readable computer programs in the memory 510 such as instructions for performing the multi-stage drift monitoring described herein.
  • the processing circuitry 502 may be implemented as one or more hardware-implemented state machines (e.g., in discrete logic, field-programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), etc.); programmable logic together with appropriate firmware; one or more stored computer programs, general-purpose processors, such as a microprocessor or digital signal processor (DSP), together with appropriate software; or any combination of the above.
  • the processing circuitry 502 may include multiple central processing units (CPUs).
  • the input/output interface 506 may be configured to provide an interface or interfaces to an input device, output device, or one or more input and/or output devices.
  • Examples of an output device include a speaker, a sound card, a video card, a display, a monitor, a printer, an actuator, an emitter, a smartcard, another output device, or any combination thereof.
  • An input device may allow a user to capture information into the UE 500.
  • Examples of an input device include a touch-sensitive or presence- sensitive display, a camera (e.g., a digital camera, a digital video camera, a web camera, etc.), a microphone, a sensor, a mouse, a trackball, a directional pad, a trackpad, a scroll wheel, a smartcard, and the like.
  • the presence-sensitive display may include a capacitive or resistive touch sensor to sense input from a user.
  • a sensor may be, for instance, an accelerometer, a gyroscope, a tilt sensor, a force sensor, a magnetometer, an optical sensor, a proximity sensor, a biometric sensor, etc., or any combination thereof.
  • An output device may use the same type of interface port as an input device. For example, a Universal Serial Bus (USB) port may be used to provide an input device and an output device.
  • USB Universal Serial Bus
  • the power source 508 is structured as a battery or battery pack. Other types of power sources, such as an external power source (e.g., an electricity outlet), photovoltaic device, or power cell, may be used.
  • the power source 508 may further include power circuitry for delivering power from the power source 508 itself, and/or an external power source, to the various parts of the UE 500 via input circuitry or an interface such as an electrical power cable. Delivering power may be, for example, for charging of the power source 508.
  • Power circuitry may perform any formatting, converting, or other modification to the power from the power source 508 to make the power suitable for the respective components of the UE 500 to which power is supplied.
  • the memory 510 may be or be configured to include memory such as random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic disks, optical disks, hard disks, removable cartridges, flash drives, and so forth.
  • the memory 510 includes one or more application programs 514, such as an operating system, web browser application, a widget, gadget engine, or other application, and corresponding data 516.
  • the memory 510 may store, for use by the UE 500, any of a variety of various operating systems or combinations of operating systems.
  • the memory 510 may be configured to include a number of physical drive units, such as redundant array of independent disks (RAID), flash memory, USB flash drive, external hard disk drive, thumb drive, pen drive, key drive, high-density digital versatile disc (HD-DVD) optical disc drive, internal hard disk drive, Blu-Ray optical disc drive, holographic digital data storage (HDDS) optical disc drive, external mini-dual in-line memory module (DIMM), synchronous dynamic random access memory (SDRAM), external micro-DIMM SDRAM, smartcard memory such as tamper resistant module in the form of a universal integrated circuit card (UICC) including one or more subscriber identity modules (SIMs), such as a USIM and/or ISIM, other memory, or any combination thereof.
  • RAID redundant array of independent disks
  • HD-DVD high-density digital versatile disc
  • HDDS holographic digital data storage
  • DIMM external mini-dual in-line memory module
  • SDRAM synchronous dynamic random access memory
  • SDRAM synchronous dynamic random access memory
  • the UICC may for example be an embedded UICC (eUICC), integrated UICC (iUICC) or a removable UICC commonly known as ‘SIM card.’
  • the memory 510 may allow the UE 500 to access instructions, application programs and the like, stored on transitory or non-transitory memory media, to off-load data, or to upload data.
  • An article of manufacture, such as one utilizing a communication system may be tangibly embodied as or in the memory 510, which may be or comprise a device-readable storage medium.
  • the processing circuitry 502 may be configured to communicate with an access network or other network using the communication interface 512.
  • the communication interface 512 may comprise one or more communication subsystems and may include or be communicatively coupled to an antenna 522.
  • the communication interface 512 may include one or more transceivers used to communicate, such as by communicating with one or more remote transceivers of another device capable of wireless communication (e.g., another UE or a network node in an access network).
  • Each transceiver may include a transmitter 518 and/or a receiver 520 appropriate to provide network communications (e.g., optical, electrical, frequency allocations, and so forth).
  • the transmitter 518 and receiver 520 may be coupled to one or more antennas (e.g., antenna 522) and may share circuit components, software or firmware, or alternatively be implemented separately.
  • communication functions of the communication interface 512 may include cellular communication, Wi-Fi communication, LPWAN communication, data communication, voice communication, multimedia communication, short- range communications such as Bluetooth, near-field communication, location-based communication such as the use of the global positioning system (GPS) to determine a location, another like communication function, or any combination thereof.
  • GPS global positioning system
  • Communications may be implemented in according to one or more communication protocols and/or standards, such as IEEE 802.11, Code Division Multiplexing Access (CDMA), Wideband Code Division Multiple Access (WCDMA), GSM, LTE, New Radio (NR), UMTS, WiMax, Ethernet, transmission control protocol/internet protocol (TCP/IP), synchronous optical networking (SONET), Asynchronous Transfer Mode (ATM), QUIC, Hypertext Transfer Protocol (HTTP), and so forth.
  • CDMA Code Division Multiplexing Access
  • WCDMA Wideband Code Division Multiple Access
  • WCDMA Wideband Code Division Multiple Access
  • GSM Global System for Mobile communications
  • LTE Long Term Evolution
  • NR New Radio
  • UMTS Worldwide Interoperability for Microwave Access
  • WiMax Ethernet
  • TCP/IP transmission control protocol/internet protocol
  • SONET synchronous optical networking
  • ATM Asynchronous Transfer Mode
  • QUIC Hypertext Transfer Protocol
  • HTTP Hypertext Transfer Protocol
  • a UE may provide an output of data captured by its sensors, through its communication interface 512, via a wireless connection to a network node.
  • Data captured by sensors of a UE can be communicated through a wireless connection to a network node via another UE.
  • the output may be periodic (e.g., once every 15 minutes if it reports the sensed temperature), random (e.g., to even out the load from reporting from several sensors), in response to a triggering event (e.g., when moisture is detected, an alert is sent), in response to a request (e.g., a user initiated request), or a continuous stream (e.g., a live video feed of a patient).
  • a UE comprises an actuator, a motor, or a switch, related to a communication interface configured to receive wireless input from a network node via a wireless connection.
  • the states of the actuator, the motor, or the switch may change.
  • the UE may comprise a motor that adjusts the control surfaces or rotors of a drone in flight according to the received input or to a robotic arm performing a medical procedure according to the received input.
  • a UE when in the form of an Internet of Things (loT) device, may be a device for use in one or more application domains, these domains comprising, but not limited to, city wearable technology, extended industrial application and healthcare.
  • loT device are a device which is or which is embedded in: a connected refrigerator or freezer, a TV, a connected lighting device, an electricity meter, a robot vacuum cleaner, a voice controlled smart speaker, a home security camera, a motion detector, a thermostat, a smoke detector, a door/window sensor, a flood/moisture sensor, an electrical door lock, a connected doorbell, an air conditioning system like a heat pump, an autonomous vehicle, a surveillance system, a weather monitoring device, a vehicle parking monitoring device, an electric vehicle charging station, a smart watch, a fitness tracker, a head-mounted display for Augmented Reality (AR) or Virtual Reality (VR), a wearable for tactile augmentation or sensory enhancement, a water sprinkler, an animal-
  • AR Augmented Reality
  • VR
  • a UE may represent a machine or other device that performs monitoring and/or measurements, and transmits the results of such monitoring and/or measurements to another UE and/or a network node.
  • the UE may in this case be an M2M device, which may in a 3GPP context be referred to as an MTC device.
  • the UE may implement the 3GPP NB-IoT standard.
  • a UE may represent a vehicle, such as a car, a bus, a truck, a ship and an airplane, or other equipment that is capable of monitoring and/or reporting on its operational status or other functions associated with its operation.
  • a first UE might be or be integrated in a drone and provide the drone’s speed information (obtained through a speed sensor) to a second UE that is a remote controller operating the drone.
  • the first UE may adjust the throttle on the drone (e.g. by controlling an actuator) to increase or decrease the drone’s speed.
  • the first and/or the second UE can also include more than one of the functionalities described above.
  • a UE might comprise the sensor and the actuator, and handle communication of data for both the speed sensor and the actuators.
  • FIG. 6 shows a network node 600 in accordance with some embodiments.
  • network node refers to equipment capable, configured, arranged and/or operable to communicate directly or indirectly with a UE and/or with other network nodes or equipment, in a telecommunication network.
  • network nodes include, but are not limited to, access points (APs) (e.g., radio access points), base stations (BSs) (e.g., radio base stations, Node Bs, evolved Node Bs (eNBs) and NR NodeBs (gNBs)).
  • APs access points
  • BSs base stations
  • Node Bs Node Bs
  • eNBs evolved Node Bs
  • gNBs NR NodeBs
  • Base stations may be categorized based on the amount of coverage they provide (or, stated differently, their transmit power level) and so, depending on the provided amount of coverage, may be referred to as femto base stations, pico base stations, micro base stations, or macro base stations.
  • a base station may be a relay node or a relay donor node controlling a relay.
  • a network node may also include one or more (or all) parts of a distributed radio base station such as centralized digital units and/or remote radio units (RRUs), sometimes referred to as Remote Radio Heads (RRHs). Such remote radio units may or may not be integrated with an antenna as an antenna integrated radio.
  • RRUs remote radio units
  • RRHs Remote Radio Heads
  • Such remote radio units may or may not be integrated with an antenna as an antenna integrated radio.
  • Parts of a distributed radio base station may also be referred to as nodes in a distributed antenna system (DAS).
  • DAS distributed antenna system
  • network nodes include multiple transmission point (multi-TRP) 5G access nodes, multi-standard radio (MSR) equipment such as MSR BSs, network controllers such as radio network controllers (RNCs) or base station controllers (BSCs), base transceiver stations (BTSs), transmission points, transmission nodes, multi-cell/multicast coordination entities (MCEs), Operation and Maintenance (O&M) nodes, Operations Support System (OSS) nodes, Self-Organizing Network (SON) nodes, positioning nodes (e.g., Evolved Serving Mobile Location Centers (E-SMLCs)), and/or Minimization of Drive Tests (MDTs).
  • MSR multi-standard radio
  • RNCs radio network controllers
  • BSCs base station controllers
  • BTSs base transceiver stations
  • OFDM Operation and Maintenance
  • OSS Operations Support System
  • SON Self-Organizing Network
  • positioning nodes e.g., Evolved Serving Mobile Location Centers (E-SMLCs)
  • the network node 600 includes a processing circuitry 602, a memory 604, a communication interface 606, and a power source 608.
  • the network node 600 may be composed of multiple physically separate components (e.g., a NodeB component and a RNC component, or a BTS component and a BSC component, etc.), which may each have their own respective components.
  • the network node 600 comprises multiple separate components (e.g., BTS and BSC components)
  • one or more of the separate components may be shared among several network nodes.
  • a single RNC may control multiple NodeBs.
  • each unique NodeB and RNC pair may in some instances be considered a single separate network node.
  • the network node 600 may be configured to support multiple radio access technologies (RATs). In such embodiments, some components may be duplicated (e.g., separate memory 604 for different RATs) and some components may be reused (e.g., a same antenna 610 may be shared by different RATs).
  • the network node 600 may also include multiple sets of the various illustrated components for different wireless technologies integrated into network node 600, for example GSM, WCDMA, LTE, NR, WiFi, Zigbee, Z-wave, LoRaWAN, Radio Frequency Identification (RFID) or Bluetooth wireless technologies. These wireless technologies may be integrated into the same or different chip or set of chips and other components within network node 600.
  • RFID Radio Frequency Identification
  • the processing circuitry 602 may comprise a combination of one or more of a microprocessor, controller, microcontroller, central processing unit, digital signal processor, application- specific integrated circuit, field programmable gate array, or any other suitable computing device, resource, or combination of hardware, software and/or encoded logic operable to provide, either alone or in conjunction with other network node 600 components, such as the memory 604, to provide network node 600 functionality.
  • the processing circuitry 602 includes a system on a chip (SOC). In some embodiments, the processing circuitry 602 includes one or more of radio frequency (RF) transceiver circuitry 612 and baseband processing circuitry 614. In some embodiments, the radio frequency (RF) transceiver circuitry 612 and the baseband processing circuitry 614 may be on separate chips (or sets of chips), boards, or units, such as radio units and digital units. In alternative embodiments, part or all of RF transceiver circuitry 612 and baseband processing circuitry 614 may be on the same chip or set of chips, boards, or units.
  • SOC system on a chip
  • the processing circuitry 602 includes one or more of radio frequency (RF) transceiver circuitry 612 and baseband processing circuitry 614.
  • the radio frequency (RF) transceiver circuitry 612 and the baseband processing circuitry 614 may be on separate chips (or sets of chips), boards, or units, such as radio units and digital units. In alternative embodiments, part or all of
  • the memory 604 may comprise any form of volatile or non-volatile computer-readable memory including, without limitation, persistent storage, solid-state memory, remotely mounted memory, magnetic media, optical media, random access memory (RAM), read-only memory (ROM), mass storage media (for example, a hard disk), removable storage media (for example, a flash drive, a Compact Disk (CD) or a Digital Video Disk (DVD)), and/or any other volatile or non-volatile, non-transitory device-readable and/or computer-executable memory devices that store information, data, and/or instructions that may be used by the processing circuitry 602.
  • volatile or non-volatile computer-readable memory including, without limitation, persistent storage, solid-state memory, remotely mounted memory, magnetic media, optical media, random access memory (RAM), read-only memory (ROM), mass storage media (for example, a hard disk), removable storage media (for example, a flash drive, a Compact Disk (CD) or a Digital Video Disk (DVD)), and/or any other volatile or non-
  • the memory 604 may store any suitable instructions, data, or information, including a computer program, software, an application including one or more of logic, rules, code, tables, and/or other instructions capable of being executed by the processing circuitry 602 and utilized by the network node 600, such as for performing the multi-stage drift monitoring described herein.
  • the memory 604 may be used to store any calculations made by the processing circuitry 602 and/or any data received via the communication interface 606.
  • the processing circuitry 602 and memory 604 is integrated.
  • the communication interface 606 is used in wired or wireless communication of signaling and/or data between a network node, access network, and/or UE. As illustrated, the communication interface 606 comprises port(s)/terminal(s) 616 to send and receive data, for example to and from a network over a wired connection.
  • the communication interface 606 also includes radio front-end circuitry 618 that may be coupled to, or in certain embodiments a part of, the antenna 610. Radio front-end circuitry 618 comprises filters 620 and amplifiers 622. The radio front-end circuitry 618 may be connected to an antenna 610 and processing circuitry 602. The radio front-end circuitry may be configured to condition signals communicated between antenna 610 and processing circuitry 602.
  • the radio front-end circuitry 618 may receive digital data that is to be sent out to other network nodes or UEs via a wireless connection.
  • the radio front-end circuitry 618 may convert the digital data into a radio signal having the appropriate channel and bandwidth parameters using a combination of filters 620 and/or amplifiers 622.
  • the radio signal may then be transmitted via the antenna 610.
  • the antenna 610 may collect radio signals which are then converted into digital data by the radio front-end circuitry 618.
  • the digital data may be passed to the processing circuitry 602.
  • the communication interface may comprise different components and/or different combinations of components.
  • the network node 600 does not include separate radio front-end circuitry 618, instead, the processing circuitry 602 includes radio front-end circuitry and is connected to the antenna 610. Similarly, in some embodiments, all or some of the RF transceiver circuitry 612 is part of the communication interface 606. In still other embodiments, the communication interface 606 includes one or more ports or terminals 616, the radio front-end circuitry 618, and the RF transceiver circuitry 612, as part of a radio unit (not shown), and the communication interface 606 communicates with the baseband processing circuitry 614, which is part of a digital unit (not shown).
  • the antenna 610 may include one or more antennas, or antenna arrays, configured to send and/or receive wireless signals.
  • the antenna 610 may be coupled to the radio front-end circuitry 618 and may be any type of antenna capable of transmitting and receiving data and/or signals wirelessly.
  • the antenna 610 is separate from the network node 600 and connectable to the network node 600 through an interface or port.
  • the antenna 610, communication interface 606, and/or the processing circuitry 602 may be configured to perform any receiving operations and/or certain obtaining operations described herein as being performed by the network node. Any information, data and/or signals may be received from a UE, another network node and/or any other network equipment. Similarly, the antenna 610, the communication interface 606, and/or the processing circuitry 602 may be configured to perform any transmitting operations described herein as being performed by the network node. Any information, data and/or signals may be transmitted to a UE, another network node and/or any other network equipment.
  • the power source 608 provides power to the various components of network node 600 in a form suitable for the respective components (e.g., at a voltage and current level needed for each respective component).
  • the power source 608 may further comprise, or be coupled to, power management circuitry to supply the components of the network node 600 with power for performing the functionality described herein.
  • the network node 600 may be connectable to an external power source (e.g., the power grid, an electricity outlet) via an input circuitry or interface such as an electrical cable, whereby the external power source supplies power to power circuitry of the power source 608.
  • the power source 608 may comprise a source of power in the form of a battery or battery pack which is connected to, or integrated in, power circuitry. The battery may provide backup power should the external power source fail.
  • Embodiments of the network node 600 may include additional components beyond those shown in Figure 6 for providing certain aspects of the network node’s functionality, including any of the functionality described herein and/or any functionality necessary to support the subject matter described herein.
  • the network node 600 may include user interface equipment to allow input of information into the network node 600 and to allow output of information from the network node 600. This may allow a user to perform diagnostic, maintenance, repair, and other administrative functions for the network node 600.
  • FIG. 7 is a block diagram of a host 700, which may be an embodiment of the host 416 of Figure 4, in accordance with various aspects described herein.
  • the host 700 may be or comprise various combinations hardware and/or software, including a standalone server, a blade server, a cloud-implemented server, a distributed server, a virtual machine, container, or processing resources in a server farm.
  • the host 700 may provide one or more services to one or more UEs.
  • the host 700 includes processing circuitry 702 that is operatively coupled via a bus 704 to an input/output interface 706, a network interface 708, a power source 710, and a memory 712.
  • processing circuitry 702 that is operatively coupled via a bus 704 to an input/output interface 706, a network interface 708, a power source 710, and a memory 712.
  • Other components may be included in other embodiments. Features of these components may be substantially similar to those described with respect to the devices of previous figures, such as Figures 5 and 6, such that the descriptions thereof are generally applicable to the corresponding components of host 700.
  • the memory 712 may include one or more computer programs including one or more host application programs 714 and data 716, which may include user data, e.g., data generated by a UE for the host 700 or data generated by the host 700 for a UE.
  • Embodiments of the host 700 may utilize only a subset or all of the components shown.
  • the host application programs 714 may be implemented in a container-based architecture and may provide support for video codecs (e.g., Versatile Video Coding (VVC), High Efficiency Video Coding (HEVC), Advanced Video Coding (AVC), MPEG, VP9) and audio codecs (e.g., FLAC, Advanced Audio Coding (AAC), MPEG, G.711), including transcoding for multiple different classes, types, or implementations of UEs (e.g., handsets, desktop computers, wearable display systems, heads-up display systems).
  • the host application programs 714 may also provide for user authentication and licensing checks and may periodically report health, routes, and content availability to a central node, such as a device in or on the edge of a core network.
  • the host 700 may select and/or indicate a different host for over-the-top services for a UE.
  • the host application programs 714 may support various protocols, such as the HTTP Live Streaming (HLS) protocol, Real-Time Messaging Protocol (RTMP), Real-Time Streaming Protocol (RTSP), Dynamic Adaptive Streaming over HTTP (MPEG-DASH), etc.
  • HLS HTTP Live Streaming
  • RTMP Real-Time Messaging Protocol
  • RTSP Real-Time Streaming Protocol
  • MPEG-DASH Dynamic Adaptive Streaming over HTTP
  • FIG. 8 is a block diagram illustrating a virtualization environment 800 in which functions implemented by some embodiments may be virtualized.
  • virtualizing means creating virtual versions of apparatuses or devices which may include virtualizing hardware platforms, storage devices and networking resources.
  • virtualization can be applied to any device described herein, or components thereof, and relates to an implementation in which at least a portion of the functionality is implemented as one or more virtual components.
  • Some or all of the functions described herein may be implemented as virtual components executed by one or more virtual machines (VMs) implemented in one or more virtual environments 800 hosted by one or more of hardware nodes, such as a hardware computing device that operates as a network node, UE, core network node, or host.
  • VMs virtual machines
  • the virtual node does not require radio connectivity (e.g., a core network node or host)
  • the node may be entirely virtualized.
  • Applications 802 (which may alternatively be called software instances, virtual appliances, network functions, virtual nodes, virtual network functions, etc.) are run in the virtualization environment Q400 to implement some of the features, functions, and/or benefits of some of the embodiments disclosed herein.
  • Hardware 804 includes processing circuitry, memory that stores software and/or instructions executable by hardware processing circuitry, and/or other hardware devices as described herein, such as a network interface, input/output interface, and so forth.
  • Software may be executed by the processing circuitry to instantiate one or more virtualization layers 806 (also referred to as hypervisors or virtual machine monitors (VMMs)), provide VMs 808a and 808b (one or more of which may be generally referred to as VMs 808), and/or perform any of the functions, features and/or benefits described in relation with some embodiments described herein.
  • the virtualization layer 806 may present a virtual operating platform that appears like networking hardware to the VMs 808.
  • the VMs 808 comprise virtual processing, virtual memory, virtual networking or interface and virtual storage, and may be run by a corresponding virtualization layer 806.
  • a virtualization layer 806 Different embodiments of the instance of a virtual appliance 802 may be implemented on one or more of VMs 808, and the implementations may be made in different ways.
  • Virtualization of the hardware is in some contexts referred to as network function virtualization (NFV). NFV may be used to consolidate many network equipment types onto industry standard high volume server hardware, physical switches, and physical storage, which can be located in data centers, and customer premise equipment.
  • NFV network function virtualization
  • a VM 808 may be a software implementation of a physical machine that runs programs as if they were executing on a physical, non-virtualized machine.
  • Each of the VMs 808, and that part of hardware 804 that executes that VM be it hardware dedicated to that VM and/or hardware shared by that VM with others of the VMs, forms separate virtual network elements.
  • a virtual network function is responsible for handling specific network functions that run in one or more VMs 808 on top of the hardware 804 and corresponds to the application 802.
  • Hardware 804 may be implemented in a standalone network node with generic or specific components. Hardware 804 may implement some functions via virtualization. Alternatively, hardware 804 may be part of a larger cluster of hardware (e.g. such as in a data center or CPE) where many hardware nodes work together and are managed via management and orchestration 810, which, among others, oversees lifecycle management of applications 802.
  • hardware 804 is coupled to one or more radio units that each include one or more transmitters and one or more receivers that may be coupled to one or more antennas. Radio units may communicate directly with other hardware nodes via one or more appropriate network interfaces and may be used in combination with the virtual components to provide a virtual node with radio capabilities, such as a radio access node or a base station.
  • some signaling can be provided with the use of a control system 812 which may alternatively be used for communication between hardware nodes and radio units.
  • Figure 9 shows a communication diagram of a host 902 communicating via a network node 904 with a UE 906 over a partially wireless connection in accordance with some embodiments.
  • Eike host 700 embodiments of host 902 include hardware, such as a communication interface, processing circuitry, and memory.
  • the host 902 also includes software, which is stored in or accessible by the host 902 and executable by the processing circuitry.
  • the software includes a host application that may be operable to provide a service to a remote user, such as the UE 906 connecting via an over-the-top (OTT) connection 950 extending between the UE 906 and host 902.
  • OTT over-the-top
  • a host application may provide user data which is transmitted using the OTT connection 950.
  • the network node 904 includes hardware enabling it to communicate with the host 902 and UE 906.
  • the connection 960 may be direct or pass through a core network (like core network 406 of Figure 4) and/or one or more other intermediate networks, such as one or more public, private, or hosted networks.
  • a core network like core network 406 of Figure 4
  • one or more other intermediate networks such as one or more public, private, or hosted networks.
  • an intermediate network may be a backbone network or the Internet.
  • the UE 906 includes hardware and software, which is stored in or accessible by UE 906 and executable by the UE’s processing circuitry.
  • the software includes a client application, such as a web browser or operator-specific “app” that may be operable to provide a service to a human or non-human user via UE 906 with the support of the host 902.
  • a client application such as a web browser or operator-specific “app” that may be operable to provide a service to a human or non-human user via UE 906 with the support of the host 902.
  • an executing host application may communicate with the executing client application via the OTT connection 950 terminating at the UE 906 and host 902.
  • the UE’s client application may receive request data from the host's host application and provide user data in response to the request data.
  • the OTT connection 950 may transfer both the request data and the user data.
  • the UE’s client application may interact with the user to generate the user data that it provides to the host application through the OTT
  • the OTT connection 950 may extend via a connection 960 between the host 902 and the network node 904 and via a wireless connection 970 between the network node 904 and the UE 906 to provide the connection between the host 902 and the UE 906.
  • the connection 960 and wireless connection 970, over which the OTT connection 950 may be provided, have been drawn abstractly to illustrate the communication between the host 902 and the UE 906 via the network node 904, without explicit reference to any intermediary devices and the precise routing of messages via these devices.
  • the host 902 provides user data, which may be performed by executing a host application.
  • the user data is associated with a particular human user interacting with the UE 906.
  • the user data is associated with a UE 906 that shares data with the host 902 without explicit human interaction.
  • the host 902 initiates a transmission carrying the user data towards the UE 906.
  • the host 902 may initiate the transmission responsive to a request transmitted by the UE 906.
  • the request may be caused by human interaction with the UE 906 or by operation of the client application executing on the UE 906.
  • the transmission may pass via the network node 904, in accordance with the teachings of the embodiments described throughout this disclosure. Accordingly, in step 912, the network node 904 transmits to the UE 906 the user data that was carried in the transmission that the host 902 initiated, in accordance with the teachings of the embodiments described throughout this disclosure. In step 914, the UE 906 receives the user data carried in the transmission, which may be performed by a client application executed on the UE 906 associated with the host application executed by the host 902.
  • the UE 906 executes a client application which provides user data to the host 902.
  • the user data may be provided in reaction or response to the data received from the host 902.
  • the UE 906 may provide user data, which may be performed by executing the client application.
  • the client application may further consider user input received from the user via an input/output interface of the UE 906. Regardless of the specific manner in which the user data was provided, the UE 906 initiates, in step 918, transmission of the user data towards the host 902 via the network node 904.
  • the network node 904 receives user data from the UE 906 and initiates transmission of the received user data towards the host 902.
  • the host 902 receives the user data carried in the transmission initiated by the UE 906.
  • One or more of the various embodiments improve the performance of OTT services provided to the UE 906 using the OTT connection 950, in which the wireless connection 970 forms the last segment. More precisely, the teachings of these embodiments may improve the positioning determination of UEs.
  • factory status information may be collected and analyzed by the host 902.
  • the host 902 may process audio and video data which may have been retrieved from a UE for use in creating maps.
  • the host 902 may collect and analyze real-time data to assist in controlling vehicle congestion (e.g., controlling traffic lights).
  • the host 902 may store surveillance video uploaded by a UE.
  • the host 902 may store or control access to media content such as video, audio, VR or AR which it can broadcast, multicast or unicast to UEs.
  • the host 902 may be used for energy pricing, remote control of non-time critical electrical load to balance power generation needs, location services, presentation services (such as compiling diagrams etc. from data collected from remote devices), or any other function of collecting, retrieving, storing, analyzing and/or transmitting data.
  • a measurement procedure may be provided for the purpose of monitoring data rate, latency and other factors on which the one or more embodiments improve.
  • the measurement procedure and/or the network functionality for reconfiguring the OTT connection may be implemented in software and hardware of the host 902 and/or UE 906.
  • sensors (not shown) may be deployed in or in association with other devices through which the OTT connection 950 passes; the sensors may participate in the measurement procedure by supplying values of the monitored quantities exemplified above, or supplying values of other physical quantities from which software may compute or estimate the monitored quantities.
  • the reconfiguring of the OTT connection 950 may include message format, retransmission settings, preferred routing etc. ; the reconfiguring need not directly alter the operation of the network node 904. Such procedures and functionalities may be known and practiced in the art.
  • measurements may involve proprietary UE signaling that facilitates measurements of throughput, propagation times, latency and the like, by the host 902.
  • the measurements may be implemented in that software causes messages to be transmitted, in particular empty or ‘dummy’ messages, using the OTT connection 950 while monitoring propagation times, errors, etc.
  • computing devices described herein may include the illustrated combination of hardware components, other embodiments may comprise computing devices with different combinations of components. It is to be understood that these computing devices may comprise any suitable combination of hardware and/or software needed to perform the tasks, features, functions and methods disclosed herein. Determining, calculating, obtaining or similar operations described herein may be performed by processing circuitry, which may process information by, for example, converting the obtained information into other information, comparing the obtained information or converted information to information stored in the network node, and/or performing one or more operations based on the obtained information or converted information, and as a result of said processing making a determination.
  • processing circuitry may process information by, for example, converting the obtained information into other information, comparing the obtained information or converted information to information stored in the network node, and/or performing one or more operations based on the obtained information or converted information, and as a result of said processing making a determination.
  • computing devices may comprise multiple different physical components that make up a single illustrated component, and functionality may be partitioned between separate components.
  • a communication interface may be configured to include any of the components described herein, and/or the functionality of the components may be partitioned between the processing circuitry and the communication interface.
  • non-computationally intensive functions of any of such components may be implemented in software or firmware and computationally intensive functions may be implemented in hardware.
  • processing circuitry executing instructions stored on in memory, which in certain embodiments may be a computer program product in the form of a non-transitory computer- readable storage medium.
  • some or all of the functionality may be provided by the processing circuitry without executing instructions stored on a separate or discrete device-readable storage medium, such as in a hard-wired manner.
  • the processing circuitry can be configured to perform the described functionality. The benefits provided by such functionality are not limited to the processing circuitry alone or to other components of the computing device, but are enjoyed by the computing device as a whole, and/or by end users and a wireless network generally.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Probability & Statistics with Applications (AREA)
  • Mobile Radio Communication Systems (AREA)

Abstract

Un procédé est mis en œuvre par un équipement utilisateur (UE) pour la surveillance de dérive en plusieurs étapes d'un modèle de positionnement radio reposant sur IA/ML. Le procédé consiste à recueillir un ensemble de données d'inférence de modèle, à réaliser, au niveau d'une première étape, un calcul de métrique de distance de l'ensemble de données d'inférence par rapport à un ensemble de données d'entraînement du modèle pour obtenir une métrique de distance, à comparer la métrique de distance et un premier seuil, à déterminer, sur la base de la comparaison, que la métrique de distance dépasse le premier seuil, et en réponse à cette détermination, à réaliser une seconde étape. Le procédé consiste à réaliser, au niveau d'une seconde étape, un calcul de métrique reposant sur les performances mesurant une qualité de sortie de modèle pour obtenir une métrique de performance, à comparer le seuil de performance et un second seuil, et à déterminer, sur la base de la comparaison, que la métrique de performance dépasse le second seuil, et à transmettre en réponse un message indiquant une détection d'une dérive du modèle.
PCT/IB2024/051730 2023-02-22 2024-02-22 Surveillance de dérive de modèle pour positionnement reposant sur ia/ml par distribution statistique conjointe et métrique de performance Ceased WO2024176170A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN2023077606 2023-02-22
CNPCT/CN2023/077606 2023-02-22

Publications (1)

Publication Number Publication Date
WO2024176170A1 true WO2024176170A1 (fr) 2024-08-29

Family

ID=90059458

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2024/051730 Ceased WO2024176170A1 (fr) 2023-02-22 2024-02-22 Surveillance de dérive de modèle pour positionnement reposant sur ia/ml par distribution statistique conjointe et métrique de performance

Country Status (1)

Country Link
WO (1) WO2024176170A1 (fr)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170343639A1 (en) * 2014-12-04 2017-11-30 Here Global B.V. Supporting radio model quality assurance
US20220264514A1 (en) * 2021-02-18 2022-08-18 Nokia Technologies Oy Rf-fingerprinting map update
US20230035737A1 (en) * 2021-07-23 2023-02-02 Stichting Imec Nederland Method for Estimating a Location of At Least One Mobile Network Node and Respective Network

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170343639A1 (en) * 2014-12-04 2017-11-30 Here Global B.V. Supporting radio model quality assurance
US20220264514A1 (en) * 2021-02-18 2022-08-18 Nokia Technologies Oy Rf-fingerprinting map update
US20230035737A1 (en) * 2021-07-23 2023-02-02 Stichting Imec Nederland Method for Estimating a Location of At Least One Mobile Network Node and Respective Network

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
FUTUREWEI: "RAN Intelligence Framework - Further Discussion on Model Testing and Model Performance Evaluation", vol. RAN WG3, no. E-meeting; 20211101 - 20211111, 21 October 2021 (2021-10-21), XP052067735, Retrieved from the Internet <URL:https://ftp.3gpp.org/tsg_ran/WG3_Iu/TSGR3_114-e/Docs/R3-214733.zip R3-214733.docx> [retrieved on 20211021] *
XUEMING PAN ET AL: "Evaluation on AI/ML for positioning accuracy enhancement", vol. 3GPP RAN 1, no. Athens, GR; 20230227 - 20230303, 17 February 2023 (2023-02-17), XP052247591, Retrieved from the Internet <URL:https://www.3gpp.org/ftp/TSG_RAN/WG1_RL1/TSGR1_112/Docs/R1-2300448.zip R1-2300448 Evaluation on AIML for positioning accuracy enhancement.docx> [retrieved on 20230217] *

Similar Documents

Publication Publication Date Title
US20250219898A1 (en) :user equipment report of machine learning model performance
EP4384947A1 (fr) Systèmes et procédés pour optimiser l&#39;entraînement de modèles et d&#39;algorithmes d&#39;ia/aa
US20240296342A1 (en) Selection of Global Machine Learning Models for Collaborative Machine Learning in a Communication Network
WO2024242612A1 (fr) Configuration et test d&#39;ue rapportant des résultats de surveillance des performances d&#39;un modèle ia/ml
WO2024176165A1 (fr) Surveillance de modèle d&#39;apprentissage automatique avec autocodeur
EP4616222A1 (fr) Collecte de données pour positionnement
WO2023232743A1 (fr) Systèmes et procédés pour une rétroaction d&#39;estimation de corrélation de caractéristiques assistée par un équipement utilisateur
US12317165B2 (en) Determining a path of movement of a terminal device
WO2025038021A1 (fr) Surveillance de performance d&#39;un modèle d&#39;intelligence artificielle/apprentissage automatique à deux côtés au côté équipement utilisateur
WO2023187678A1 (fr) Gestion de modèle de machine d&#39;équipement utilisateur assistée par réseau d&#39;apprentissage
WO2024176170A1 (fr) Surveillance de dérive de modèle pour positionnement reposant sur ia/ml par distribution statistique conjointe et métrique de performance
WO2024171099A1 (fr) Surveillance de modèle d&#39;apprentissage machine à l&#39;aide d&#39;inférences multiples
EP4666219A1 (fr) Surveillance de modèle d&#39;apprentissage machine à l&#39;aide d&#39;inférences multiples
US20250234219A1 (en) Network assisted user equipment machine learning model handling
WO2025172969A1 (fr) Détermination d&#39;un temps de référence de rtoa pour positionnement d&#39;équipement utilisateur basé sur l&#39;ia/aa
WO2024171159A1 (fr) Positionnement d&#39;ue sensible à l&#39;environnement
EP4666086A1 (fr) Positionnement d&#39;ue sensible à l&#39;environnement
WO2025165273A1 (fr) Dispositif sans fil, nœud de réseau et procédés de surveillance de performance pour de multiples schémas de prédiction de csi
WO2025172948A1 (fr) Autosurveillance d&#39;un modèle d&#39;intelligence artificielle/apprentissage automatique
WO2025172939A1 (fr) Aide à la signalisation pour validation de modèle d&#39;intelligence artificielle/apprentissage automatique
WO2025172964A1 (fr) Données à valeurs complexes pour ia/ml
WO2025172968A1 (fr) Rapport de mesure pour positionnement assisté par ia/ml
WO2025038023A1 (fr) Surveillance efficace en temps réel de modèles côté équipement utilisateur à l&#39;aide de la validité de résultats de surveillance
WO2025212031A1 (fr) Surveillance de modèle externe d&#39;un modèle ia/aa à l&#39;aide d&#39;informations de liaison de ligne de visée
WO2025172947A1 (fr) Surveillance de modèle externe de modèle d&#39;intelligence artificielle/apprentissage automatique

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24707955

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2024707955

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2024707955

Country of ref document: EP

Effective date: 20250922