WO2024229840A1

WO2024229840A1 - Signaling framework for ai/ml based semi-supervised learning

Info

Publication number: WO2024229840A1
Application number: PCT/CN2023/093679
Authority: WO
Inventors: Afef Feki; Dick CARRILLO MELGAREJO; Muhammad Ikram ASHRAF; Yijia Feng
Original assignee: Nokia Shanghai Bell Co Ltd; Nokia Solutions and Networks Oy; Nokia Technologies Oy
Current assignee: Nokia Shanghai Bell Co Ltd; Nokia Solutions and Networks Oy; Nokia Technologies Oy
Priority date: 2023-05-11
Filing date: 2023-05-11
Publication date: 2024-11-14
Anticipated expiration: 2025-11-11
Also published as: CN121080066A

Abstract

Example embodiments of the present disclosure provide a signaling framework for artificial intelligence (AI) /machine learning (ML) based semi-supervised learning (SSL). In an example method, a first device receives, from a second device, an indication of triggering SSL training for an AI/ML model at the first device and at least one SSL training rule. The first device collects unlabeled data based on the at least one SSL training rule and train the AI/ML model based on a labeled dataset obtained from the second device and an unlabeled dataset including the collected unlabeled data. In this way, the performance of the AI/ML model at UE side can be enhanced by triggering or switching to SSL under network assistance.

Description

SIGNALING FRAMEWORK FOR AI/ML BASED SEMI-SUPERVISED LEARNING

FIELD

Example embodiments of the present disclosure generally relate to the field of communication, and in particular, to devices, methods, apparatuses, and a computer readable medium for a signaling framework for artificial intelligence (AI) /machine learning (ML) based semi-supervised learning (SSL) .

BACKGROUND

3GPP radio access network (RAN) 1 has initiated a study item on AI/ML for air interface, where one of the key use cases being considered is positioning accuracy enhancements with the use of AI/ML. It has been agreed that the performance impact from ground truth label availability on AI/ML positioning cases needs to be evaluated.

However, in real-world deployment, the ground truth labels are difficult or even not available to attain. Therefore, studying the standardization impact and the corresponding signaling on semi-supervised-learning (SSL) -based AI/ML positioning approach is a necessary and timely task.

SUMMARY

In general, example embodiments of the present disclosure provide a solution for a signaling framework for AI/ML based semi-supervised learning (SSL) .

In a first aspect, there is provided a first device. The first device comprises at least one processor and at least one memory storing instructions that, when executed by the at least one processor, cause the first device at least to: receive, from a second device, an indication of triggering semi-supervised learning (SSL) training for an artificial intelligence/machine learning (AI/ML) model at the first device and at least one SSL training rule; collect unlabeled data based on the at least one SSL training rule; and train the AI/ML model based on a labeled dataset obtained from the second device and an unlabeled dataset including the collected unlabeled data.

In a second aspect, there is provided second device. The second device comprises at least one processor and at least one memory storing instructions that, when executed by the at least one processor, cause the second device at least to: determine triggering semi-supervised learning (SSL) training for an artificial intelligence/machine learning (AI/ML) model at a first device; and transmit an indication of triggering SSL training and at least one SSL training rule to the first device.

In a third aspect, there is provided a method. The method comprises: receiving, at a first device from a second device, an indication of triggering semi-supervised learning (SSL) training for an artificial intelligence/machine learning (AI/ML) model at a first device and at least one SSL training rule; collecting unlabeled data based on the at least one SSL training rule; and training the AI/ML model based on a labeled dataset obtained from the second device and an unlabeled dataset including the collected unlabeled data.

In a fourth aspect, there is provided a method. The method comprises: determining, at a second device, triggering semi-supervised learning (SSL) training for an artificial intelligence/machine learning (AI/ML) model at a first device; and transmitting an indication of triggering SSL training and at least one SSL training rule to the first device.

In a fifth aspect, there is provided an apparatus. The apparatus comprises: means for receiving, at a first device from a second device, an indication of triggering semi-supervised learning (SSL) training for an artificial intelligence/machine learning (AI/ML) model at a first device and at least one SSL training rule; means for collecting unlabeled data based on the at least one SSL training rule; and means for training the AI/ML model based on a labeled dataset obtained from the second device and an unlabeled dataset including the collected unlabeled data.

In a sixth aspect, there is provided an apparatus. The apparatus comprises: means for determining, at a second device, triggering semi-supervised learning (SSL) training for an artificial intelligence/machine learning (AI/ML) model at a first device; and means for transmitting an indication of triggering SSL training and at least one SSL training rule to the first device.

In a seventh aspect, there is provided a non-transitory computer-readable storage medium comprising program instructions. The program instructions, when executed by an apparatus, cause the apparatus to perform at least the following: receiving, from a second device, an indication of triggering semi-supervised learning (SSL) training for an artificial intelligence/machine learning (AI/ML) model at a first device and at least one SSL training rule; collecting unlabeled data based on the at least one SSL training rule; and training the AI/ML model based on a labeled dataset obtained from the second device and an unlabeled dataset including the collected unlabeled data.

In an eighth aspect, there is provided a non-transitory computer-readable storage medium comprising program instructions. The program instructions, when executed by an apparatus, cause the apparatus to perform at least the following: determining triggering semi-supervised learning (SSL) training for an artificial intelligence/machine learning (AI/ML) model at a first device; and transmitting an indication of triggering SSL training and at least one SSL training rule to the first device.

In a ninth aspect, there is provided a computer program comprising instructions, which, when executed by an apparatus, cause the apparatus at least to: receive, from a second device, an indication of triggering semi-supervised learning (SSL) training for an artificial intelligence/machine learning (AI/ML) model at a first device and at least one SSL training rule; collect unlabeled data based on the at least one SSL training rule; and train the AI/ML model based on a labeled dataset obtained from the second device and an unlabeled dataset including the collected unlabeled data.

In a tenth aspect, there is provided a computer program comprising instructions, which, when executed by an apparatus, cause the apparatus at least to: determine triggering semi-supervised learning (SSL) training for an artificial intelligence/machine learning (AI/ML) model at a first device; and transmit an indication of triggering SSL training and at least one SSL training rule to the first device.

In an eleventh aspect, there is provided a terminal device. The terminal device comprises: a receiving circuitry configured to: receive, from a second device, an indication of triggering semi-supervised learning (SSL) training for an artificial intelligence/machine learning (AI/ML) model at the first device and at least one SSL training rule; collect unlabeled data based on the at least one SSL training rule; and train the AI/ML model based on a labeled dataset obtained from the second device and an unlabeled dataset including the collected unlabeled data.

In a twelfth aspect, there is provided a network device. The network device comprises: a transmitting circuitry configured to: determine triggering semi-supervised learning (SSL) training for an artificial intelligence/machine learning (AI/ML) model at a first device; and transmit an indication of triggering SSL training and at least one SSL training rule to the first device.

It is to be understood that the summary section is not intended to identify key or essential features of embodiments of the present disclosure, nor is it intended to be used to limit the scope of the present disclosure. Other features of the present disclosure will become easily comprehensible through the following description.

BRIEF DESCRIPTION OF THE DRAWINGS

Some example embodiments will now be described with reference to the accompanying drawings, in which:

FIG. 1 illustrates an example communication network in which embodiments of the present disclosure may be implemented;

FIG. 2 illustrates an example of a process flow in accordance with some example embodiments of the present disclosure;

FIG. 3 illustrates an example framework for AI/ML based positioning in accordance with some example embodiments of the present disclosure;

FIG. 4 illustrates another example of a process flow in accordance with some example embodiments of the present disclosure;

FIG. 5 illustrates a flowchart of an example method implemented at a first device in accordance with some embodiments of the present disclosure;

FIG. 6 illustrates another flowchart of an example method implemented at a second device in accordance with some embodiments of the present disclosure;

FIG. 7 illustrates a simplified block diagram of a device that is suitable for implementing some example embodiments of the present disclosure; and

FIG. 8 illustrates a block diagram of an example of a computer-readable medium in accordance with some example embodiments of the present disclosure.

Throughout the drawings, the same or similar reference numerals represent the same or similar elements.

DETAILED DESCRIPTION

Principles of the present disclosure will now be described with reference to some example embodiments. It is to be understood that these embodiments are described only for the purpose of illustration and help those skilled in the art to understand and implement the present disclosure, without suggesting any limitation as to the scope of the disclosure. The disclosure described herein can be implemented in various manners other than the ones described below.

In the following description and claims, unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skills in the art to which this disclosure belongs.

References in the present disclosure to “one embodiment, ” “an embodiment, ” “an example embodiment, ” and the like indicate that the embodiment described may include a particular feature, structure, or characteristic, but it is not necessary that every embodiment includes the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.

It shall be understood that although the terms “first” and “second” etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first element could be termed a second element, and similarly, a second element could be termed a first element, without departing from the scope of example embodiments. As used herein, the term “and/or” includes any and all combinations of one or more of the listed terms.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments. As used herein, the singular forms “a” , “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” , “comprising” , “has” , “having” , “includes” and/or “including” , when used herein, specify the presence of stated features, elements, and/or components etc., but do not preclude the presence or addition of one or more other features, elements, components and/or combinations thereof. As used herein, “at least one of the following: <a list of two or more elements>” and “at least one of <a list of two or more elements>” and similar wording, where the list of two or more elements are joined by “and” or “or” , mean at least any one of the elements, or at least any two or more of the elements, or at least all the elements.

As used in this application, the term “circuitry” may refer to one or more or all of the following:

(a) hardware-only circuit implementations (such as implementations in only analog and/or digital circuitry) and

(b) combinations of hardware circuits and software, such as (as applicable) :

(i) a combination of analog and/or digital hardware circuit (s) with software/firmware and

(ii) any portions of hardware processor (s) with software (including digital signal processor (s) ) , software, and memory (ies) that work together to cause an apparatus, such as a mobile phone or server, to perform various functions) and

(c) hardware circuit (s) and or processor (s) , such as a microprocessor (s) or a portion of a microprocessor (s) , that requires software (for example, firmware) for operation, but the software may not be present when it is not needed for operation.

This definition of circuitry applies to all uses of this term in this application, including in any claims. As a further example, as used in this application, the term circuitry also covers an implementation of merely a hardware circuit or processor (or multiple processors) or portion of a hardware circuit or processor and its (or their) accompanying software and/or firmware. The term circuitry also covers, for example and if applicable to the particular claim element, a baseband integrated circuit or processor integrated circuit for a mobile device or a similar integrated circuit in server, a cellular network device, or other computing or network device.

As used herein, the term “network” , “communication network” or “data network” refers to a network following any suitable communication standards, such as long term evolution (LTE) , LTE-advanced (LTE-A) , wideband code division multiple access (WCDMA) , high-speed packet access (HSPA) , narrow band Internet of things (NB-IoT) , wireless fidelity (Wi-Fi) and so on. Furthermore, the communications between a terminal device and a network device/element in the communication network may be performed according to any suitable generation communication protocols, including, but not limited to, the fourth generation (4G) , 4.5G, the future fifth generation (5G) , IEEE 802.11 communication protocols, and/or any other protocols either currently known or to be developed in the future. Embodiments of the present disclosure may be applied in various communication systems. Given the rapid development in communications, there will of course also be future type communication technologies and systems with which the present disclosure may be embodied. It should not be seen as limiting the scope of the present disclosure to only the aforementioned system.

As used herein, the term “network device” refers to a node in a communication network via which a terminal device receives services (e.g., positioning services) therefrom. The network device may refer to a core network device or access network device, such as base station (BS) or an access point (AP) or a transmission and reception point (TRP) , for example, a node B (NodeB or NB) , an evolved NodeB (eNodeB or eNB) , a NR NB (also referred to as a gNB) , a remote radio unit (RRU) , a radio header (RH) , a remote radio head (RRH) , a WiFi device, a relay, a low power node such as a femto, a pico, and so forth, depending on the applied terminology and technology. In the following description, the terms “network device” , “AP device” , “AP” and “access point” may be used interchangeably.

The term “terminal device” refers to any end device that may be capable of wireless communication. By way of example rather than limitation, a terminal device may also be referred to as a communication device, user equipment (UE) , a Subscriber Station (SS) , a Portable Subscriber Station, a Mobile Station (MS) , a station (STA) or station device, or an Access Terminal (AT) . The terminal device may include, but not limited to, a mobile phone, a cellular phone, a smart phone, voice over IP (VoIP) phones, wireless local loop phones, a tablet, a wearable terminal device, a personal digital assistant (PDA) , portable computers, desktop computer, image capture terminal devices such as digital cameras, gaming terminal devices, music storage and playback appliances, vehicle-mounted wireless terminal devices, wireless endpoints, mobile stations, laptop-embedded equipment (LEE) , laptop-mounted equipment (LME) , USB dongles, smart devices, wireless customer-premises equipment (CPE) , an Internet of Things (IoT) device, a watch or other wearable, a head-mounted display (HMD) , a vehicle, a drone, a medical device and applications (for example, remote surgery) , an industrial device and applications (for example, a robot and/or other wireless devices operating in an industrial and/or an automated processing chain contexts) , a consumer electronics device, a device operating on commercial and/or industrial wireless networks, and the like. In the following description, the terms “station” , “station device” , “STA” , “terminal device” , “communication device” , “terminal” , “user equipment” and “UE” may be used interchangeably.

As used herein, the term “location management function” (LMF) refers to an entity defined in a network (e.g. 5G network) to provide positioning functionality by means to determine the geographic position of a terminal device based on downlink and uplink location radio measurements. The LMF may be provided in a network device in the core network or in the access network, or provided in a terminal device moving around in a communication environment. The LMF may receive measurements and assistance information from the next generation radio access network (NG-RAN) and the terminal device via e.g. the access and mobility management function (AMF) to compute the position of the terminal device.

As used herein, the term “positioning reference unit” (PRU) refers to an entity which provide its accurate position information as ground truth to other entities in the communication network, e.g. a terminal device and LMF. The PRU may be geographically fixed or movable in the environment.

3GPP RAN1 has initiated a study item on AI/ML for air interface, where one of the key use case being considered is positioning accuracy enhancements with the use of AI/ML. It was agreed that the performance impact from ground truth label availability on AI/ML positioning cases should be evaluated. In real-world deployment, the ground truth labels are expensive or even not available to attain. Therefore, studying the standardization impact and the corresponding signaling on semi-supervised-learning-based AI/ML positioning approach should be a necessary and timely task.

Supervised, self-supervised and semi-supervised learning for AI/ML models are studied and applied in variety of fields. Fitting a model to a dataset is considerably easier when labeled data is plentiful. In many circumstances, however, access to labeled data is limited -meaning that learning a performant model is much harder, with high risk of overfitting the limited quantity of data. In such situations, therefore, using unlabeled data effectively is of critical importance.

The common characteristic of semi-supervised learning is that the dataset is composed by a small set of labeled data and a large unlabeled data. The expectation is that there is a gain on using semi-supervised learning in scenarios with small, labeled datasets compared to scenarios that used supervised learning with the same small labeled dataset.

As mentioned, UE position may be estimated using AI/ML method. Generally supervised learning (SL) -based approach may be applied to infer UE position based on preselected input (e.g., radio measurements) for the direct positioning case. Similarly, for the assisted positioning, AI/ML with a supervised learning approach may be used to estimate an intermediate feature (e.g., line of sight (LOS) /non line of sight (NLOS) flag) which in turn is used afterwards to estimate the UE position.

To use a supervised learning model for inference, it is mandatory to go through a training phase. This relates to the estimation of the model parameters using a set of labeled (ground truth) data (input and corresponding desired outputs of the model) . The performance of the trained model may be highly dependent on the used/available labeled data.

In practice, labeled data (e.g., radio measurements with UE position as label) may be costly to gather, and it is more probable to have much more unlabeled data available (e.g., radio measurements without UE position) . In this regard, semi supervised learning (SSL) is a good option. Specifically, for training phase in addition to the labeled data, unlabeled data can also be utilized to enhance the performance of the model training. Therefore, the overhead of data collection can be significantly reduced by utilizing the unlabeled data. Particularly, to enable the semi-supervised learning different entities with their capability (PRU and regular UEs) can improve the data collection phase. PRUs may be utilized to provide the labeled data, whereas the regular UEs by default collect the unlabeled data samples.

Semi supervised learning takes benefit of unlabeled data in addition to the labeled one in order to further enhance the performance of the trained model. However, the way forward to enable the use of such semi supervised approach for ML based positioning needs to be clearly defined to ensure the targeted performance.

Embodiments of the present disclosure relate to a signaling framework to enable the use of semi supervised learning (SSL) for AI/ML based positioning. In a case where AI/ML model is trained at UE side, a location management function (LMF) may provide labeled data to the UE. The labeled data may comprise radio measurements and corresponding geographical positions e.g., assisted by PRUs. At the UE side, AI/ML model inputs can be radio measurements measured by the UE as unlabeled data but the UE position is not an available information all the time.

Embodiments of the present disclosure provide the framework to benefit from the existing unlabeled data at UE side through the establishment of specific rules by the LMF. In other terms, the LMF may provide indication to the UE when and in which conditions it can proceed with semi supervised learning training instead of supervised learning training. In some embodiments, the rules may be set for SSL model triggering or/and switching with regards to the supervised learning model. In addition, the rules may be set for SSL model preparation and training accounting for predefined requirements such as the minimum size/proportion of labeled/unlabeled data as well as filtering rules for unlabeled data selection to include for the semi supervised learning training. Detailed description of the signaling framework tailored for SSL operation as well as a description of SSL model training with unlabeled data selection is provided below.

For illustrative purposes, principles and example embodiments of the present disclosure will be described below with reference to FIG. 1 to FIG. 8. However, it is to be noted that these embodiments are given to enable the skilled in the art to understand inventive concepts of the present disclosure and implement the solution as proposed herein, and not intended to limit scope of the present application in any way.

FIG. 1 illustrates an example of an application scenario 100 in which some example embodiments of the present disclosure may be implemented. The application scenario 100, which is a part of a communication network, includes terminal devices and network devices.

In the descriptions of the example embodiments of the present disclosure, the network environment 100 may also be referred to as a communication system 100 (for example, a portion of a communication network) . For illustrative purposes only, various aspects of example embodiments will be described in the context of one or more terminal devices and network devices that communicate with one another. It should be appreciated, however, that the description herein may be applicable to other types of apparatus or other similar apparatuses that are referenced using other terminology.

A first device 110 may receive services (e.g., positioning services) from a second device 120, and the first device 110 and the second device 120 may communicate data and control information with each other via a network 102. In some embodiments, the first device 110 and the second device 120 may communicate with direct links/channels. The first device 110 may be terminal device or UE movable in the network environment 100. The second device may be a network device located at an access network or a core network. In some embodiments, the second device may be or comprise an LMF.

In the communication system 100, a link from the second device 120 to the first device 110 is referred to as a downlink (DL) , while a link from the first device 110 to the second device 120 is referred to as an uplink (UL) . In downlink, the second device 120 is a transmitting (TX) device (or a transmitter) and the first device 110 is a receiving (RX) device (or a receiver) . In uplink, the first device 110 is a transmitting (TX) device (or a transmitter) and the second device 120 is a RX device (or a receiver) .

The network 102 may be implemented according to any proper wireless or wired communication protocol (s) , comprising, but not limited to, cellular communication protocols and core network communication protocols of the fourth generation (4G) and the fifth generation (5G) and the like, wireless local network communication protocols such as Institute for Electrical and Electronics Engineers (IEEE) 802.11 and the like, and/or any other protocols currently known or to be developed in the future. Moreover, the communication may utilize any proper wireless communication technology, comprising but not limited to: Code Division Multiple Access (CDMA) , Frequency Division Multiple Access (FDMA) , Time Division Multiple Access (TDMA) , Frequency Division Duplex (FDD) , Time Division Duplex (TDD) , Multiple-Input Multiple-Output (MIMO) , Orthogonal Frequency Division Multiple (OFDM) , Discrete Fourier Transform spread OFDM (DFT-s-OFDM) and/or any other technologies currently known or to be developed in the future.

The first device 110 may run an AI/ML model to infer its position based on radio measurements. The AI/ML model may be a trained model based on supervised learning, for example, using a labeled dataset including e.g., UE positions as ground truth. The accuracy of the AI/ML may be checked by the first device 110 or the second device 120 periodically or by events. When the accuracy is running low, the first device 110 may be triggered by the second device 120 to perform a semi-supervised learning for a model update. In some embodiments, the first device 110 may receive from the second device 120 an indication of triggering SSL training for an AI/ML model and a set of SSL training rule (s) . The first device 110 may collect unlabeled data, for example, radio measurements, based on the SSL training rule (s) . The first device 110 may then train the AI/ML model based on a labeled dataset obtained from the second device 120 and an unlabeled dataset including the collected unlabeled data.

It is to be understood that the number of devices and their connection relationships and types shown in FIG. 1 are for illustrative purposes without suggesting any limitation. The communication system 100 may comprise any suitable number of devices adapted for implementing embodiments of the present disclosure.

FIG. 2 illustrates an example of a process flow 200 in accordance with some example embodiments of the present disclosure. For ease of understanding, the process flow 200 will be described with reference to FIG. 1. It would be appreciated that although the process flow 200 has been described referring to the application scenario 100 of FIG. 1, this process flow 200 may be likewise applied to other similar communication scenarios.

The second device 120 determines (202) triggering semi-supervised learning (SSL) training for an AI/ML model at a first device 110. In some embodiments, if the second device 120 detects that the accuracy of an AI/ML model trained based on supervised learning (SL) is lower than a threshold, it may determine triggering SSL training for the AI/ML model. Alternatively or additionally, if the second device 120 determines that there is no sufficient labeled data or it will take some time to reach the needs size of labeled dataset for SL training, the second device 120 may decide to trigger the first device 110 to perform SSL training for the model update. Alternatively or additionally, the second device 120 may decide to trigger the SSL training at the first device 110 if it receives a request for SSL training from the first device 110.

In some embodiments, the second device 120 may determine SSL training rules for the AI/ML model. The SSL training rules may comprise, for example, a measurement collection period within which the second device 120 may collect radio measurements as unlabeled data. In some embodiments, the SSL rules may comprise a minimum size of unlabeled dataset for SSL training.

Alternatively or additionally, the SSL rules may comprise rules for unlabeled data selection. The selection of the unlabeled data is important to ensure optimal performance of the SSL training. The rules for unlabeled data selection may be, for example, random sampling of unlabeled samples. This relates to a random selection of the samples to include for the SSL training. Alternatively or additionally, the rules for unlabeled data selection may be cluster based sampling. In this approach, unlabeled data are clustered to form groups with similar characteristics and then representative samples are extracted from each cluster which allows to form heterogeneous/diverse unlabeled samples. Alternatively or additionally, the SSL rules may comprise a balance (for example, a proportion) between the labeled data and the unlabeled data for SSL training. The balance may be referred as meta information for SSL training in the present disclosure.

The second device 120 transmits (203) an indication of triggering SSL training and the SSL training rule (s) (204) to the first device 110. Accordingly, the first device 110 receives (205) the indication of triggering SSL training and the SSL training rule (s) (204) .

Upon reception (205) of the indication and rules (204) , the first device 110 collects (207) unlabeled data based on the SSL rule (s) . In some embodiments, the first device 110 may perform and collect radio measurements in the collection period indicated in the SSL rule (s) .

In some embodiments, the first device 110 may further receive at least part of the unlabeled data from other devices (for example, other UEs) to improve the performance of SSL training with more diverse unlabeled dataset. The first device 110 may receive unlabeled data from other UEs (which probably correspond to other conditions and regions not seen) . For example, the second device 120 may determine the related UEs with different conditions and regions and indicate the related UEs to the first device 110. In some embodiments, the first device 110 may receive unlabeled data from those devices via sidelink communications (for example, with PC5 interfaces) or via a network device between the first device and the other UE (athird device) (for example, with uplinks and downlinks) .

In some embodiments, the first device 110 may apply the rules for unlabeled data selection of samples to include for SSL training. As mentioned, a random sampling approach and/or a cluster based sampling approach may be applied. In the meantime, the first device 110 may obtain labeled dataset from the second device 120. Alternatively, at least part of the labeled dataset may be received in advance before the SSL training is triggered, or even during or after pre-training of the AI/ML model with the unlabeled dataset.

The first device 110 may combine the unlabeled data and the labeled data as a training dataset following the balance indicated in the SSL training rule (s) (204) . Then, the first device 110 may train (209) the AI/ML model based on a labeled dataset obtained from the second device and an unlabeled dataset including the collected unlabeled data.

In some embodiments, a two-step training process may be adopted. First, the first device 110 may train the AI/ML model with the unlabeled dataset (self-supervised learning) ; next the insights learned on the unlabeled dataset are transferred to a smaller labeled dataset, and the AI/ML model is fine-tuned using the limited quality labeled data. Once the model is pre-trained using unlabeled data, the first device 110 may request labeled data for fine tuning of existing pre-trained model. Alternatively, the first device 110 may train the AI/ML model with the labeled dataset (supervised learning) and the AI/ML model is fine-tuned using the unlabeled dataset.

Once the SSL training is completed, the first device 110 may apply the trained SSL model for position inference. In some embodiments, the first device 110 may decide a final model to run from the SL model and the newly obtained SSL model based on a comparison of their accuracies. The SSL model accuracy could be better compared to the SL model in case of use of limited set of labeled data. In addition, the decision may be made by the second device 120 and indicated to the first device 110.

In view of the above, embodiments of the disclosure provide a signaling framework in which a SSL model at UE side can be triggered or/and switched with regards to a supervised learning (SL) model. In some embodiments, the SSL model preparation and training is under control of a controlling device (e.g., LMF) based on the SSL training rules, such as the minimum size/proportion of labeled/unlabeled data as well as filtering rules for unlabeled data selection to include for the semi supervised learning training. In this way, the performance of AI/ML model at UE side can be improved with limited expensive labeled data.

FIG. 3 illustrates an example framework 300 for AI/ML based positioning in accordance with some example embodiments of the present disclosure. In FIG. 3, the functioning of the SSL based positioning and its interworking with supervised learning (SL) method is depicted. The framework 300 may be implemented by interworking between a UE and a LMF which respectively may be an example of the first device 110 and the second device 120 in FIGS. 1 and 2.

At block 301, the UE runs a SL method to train an AI/ML model (also referred as “SL model” ) . In fact, the model is trained with labeled data for AI/ML based positioning (typically radio measurements and corresponding geographical position) . The performance of the trained SL model may be monitored, and the accuracy of the model checked by either of the UE or the LMF.

At block 302, if the accuracy is satisfactory, e.g., higher than a predefined threshold, then the procedure can be return to block 301 and keep using the same SL model. However, if the model monitoring indicates that the accuracy is not satisfactory (e.g., below the predefined threshold) , then there is a need to perform a model update. In this case, at block 303, it is needed to check first if it is possible to collect labeled data. If yes, then at block 304 the SL model can be updated/retrained using the collected labeled data.

However, if it is difficult or costly to get the labeled data, then the procedure may proceed with block 305 to trigger SSL model update. Thus, at block 306 the unlabeled data are firstly collected and filtered following pre-established rules set by the LMF. Thereafter, at block 307 the SSL training can be performed using the unlabeled data combined with any available labeled data (if any) . Finally, at block 308, the newly trained model (either through SL or SSL) is used for inference.

FIG. 4 illustrates another example of a process flow 400 in accordance with some example embodiments of the present disclosure. In FIG. 4, the signaling between the UE 41 and the LMF 42 is depicted, where the UE 41 and the LMF 42 respectively may be an example of the first device 110 and the second device 120 in FIG. 1 and FIG. 2.

In the following model id1 refers to the SL model which is trained with labeled data. The SL Model id1 is run at UE side. During inference with model id1 at 401, the monitoring is performed at 402 to check the accuracy of the model upon predefined conditions (such as periodically every T time) . If the accuracy is not satisfactory, then at 403 it identifies a need to realize a model update.

In this case, at 404, the UE 41 may transmit a request to the LMF 42 for model update and obtention of labeled data. At 405, the LMF 42 may check labeled data availability. If the LMF 42 can get additional labeled data (e.g., from PRU) , then at 406 it can send the labeled data to the UE 41.

Optionally, at 407 the UE 41 may initiate the SSL training. In an example embodiment, at 408 the UE 41 may determine the UE capabilities and a type of the model. The capability may refer to the ability of the device to perform a new round of training which accounts for example on its battery level and memory size (to store collected unlabeled data) . And the type of the model may comprise supervised learning (SL) , self-supervised learning, or semi-supervised learning (SSL) . The UE may check whether SSL training is applicable to the model, if applicable, the UE may transmits a request for SSL. Based on the UE capabilities and the type of the model, the UE 41 may request for SSL training from the LMF 42 at 409 to perform SSL training and request the related indications and corresponding SSL training rules. Accordingly, at 410 the LMF 42 may transmit an indication of triggering or switching to SSL training together with the SSL training rules to the UE 41.

Alternatively or additionally, for the LMF case, if the gathered labeled data size is not sufficient to ensure efficient model retraining/refinement (and possibly it will take some time to reach the needed size) , then at 411 the LMF 42 may decide to request the UE 41 to proceed with SSL training following specific rules. Alternatively or additionally, the LMF 42 may determine triggering or switching to SSL training in a case that the accuracy of the SL model is lower than the predefined threshold.

At 412, the UE 41 collects the unlabeled data following the LMF rules (such as measurement period, filtering rules to ensure enough diverse samples) and meta information about the balance between labeled/unlabeled samples. In some embodiments, at 413 the UE 41 may calculate a difference or discrepancy between the statistical distributions of the collected unlabeled dataset and the labeled dataset with some mathematical approaches (e.g., Kullback-Leibler divergence, maximum mean divergence) . If the difference is beyond a predefined threshold, it indicates that the collected unlabeled data cannot represent very well the environment where the UE 41 is located, and the model performance may not meet the requirement if this unlabeled dataset is used for model training. Therefore, at 414, the UE 41 may request the LMF 42 to adjust SSL training rules. At 415, the LMF 42 may send new SSL training rules to the UE 41. The UE 41 may collect the unlabeled following the new rules.

At 416, the unlabeled and labeled samples may be combined and prepared. Note that the preparation may depend on the selected SSL training method. In some embodiments, the preparation of SSL unlabeled samples corresponds to the addition of pseudo or dump values as labels for the unlabeled samples. Thereafter at 417 the SSL model is trained using the prepared combined data set.

At 418, the performance of the new SSL model id2 may be compared with the previous SL model id1. In some embodiments, at 419 the LMF 42 may be informed with both model accuracies to let it at 420 check the model accuracies and select the final model to run at UE side. At 421, the LMF 42 indicates selected model id for inference to the UE 41. Optionally, the decision can be directly made by the UE 41 on the final model to use (either SL or SSL based) . Note that the SSL model accuracy could be better compared to the SL model in case of use of limited set of labeled data. Finally, at 422 the UE 41 may run the final model and discard the other one.

FIG. 5 illustrates a flowchart of an example method 500 implemented at a first device in accordance with some other embodiments of the present disclosure. For ease of understanding, the method 500 will be described from the perspective of the first device 110 with reference to FIG. 1.

At block 510, the first device 110 receives, from a second device 120, an indication of triggering semi-supervised learning (SSL) training for an artificial intelligence/machine learning (AI/ML) model at the first device and at least one SSL training rule. In some embodiments, the first device 110 may transmit, based on capability of the first device and a type of the AI/ML model, a request for SSL training of the AI/ML model to the second device 120.

In some embodiments, the first device 110 may determine whether the labeled data is sufficient for supervised learning (SL) training. If the labeled data is insufficient for SL training, the first device 110 may transmit the request for SSL training to the second device 120.

In some embodiments, the training SSL rule may comprise, but not limited to, one or more of the following a measurement collection period, a minimum size of unlabeled dataset, an approach for unlabeled data selection, or a balance between the labeled data and the unlabeled data for SSL training. In some embodiments, the approach for unlabeled data selection may comprise at least one of random sampling or cluster-based sampling of unlabeled samples.

At block 520, the first device 110 collects unlabeled data based on the at least one SSL training rule. The first device 110 may perform radio measurements as the unlabeled data following the SSL training rules. In some embodiments, the first device 110 may receive at least part of the unlabeled data from a third device via sidelink communications or via a network device between the first device 110 and the third device.

In some embodiments, the first device 110 may calculate a statistical distribution difference between the labeled data and the unlabeled data. If the statistical distribution difference exceeds a threshold, the first device 110 may transmit a request for adjustments of the at least one SSL training rule to the second device 120, and accordingly receive an adjusted SSL training rule from the second device 120.

At block 530, the first device 110 trains the AI/ML model based on a labeled dataset obtained from the second device and an unlabeled dataset including the collected unlabeled data. In some embodiments, the first device 110 may combine the unlabeled data and the labeled data as a training dataset, and perform SSL training of the AI/ML model with the combined training dataset. Pseudo value (s) may be added as a label for the unlabeled data.

In some embodiments, the first device 110 may train the AI/ML model in two steps. The first device 110 may pre-train the AI/ML model based on the unlabeled dataset, and then perform fine tuning of the pre-trained ML model based on the labeled dataset. Alternatively, the first device 110 may pre-train the AI/ML model based on the labeled dataset, and then perform fine tuning of the pre-trained ML model based on the unlabeled dataset.

AI/ML model may be a first AI/ML model. In some embodiments the first device 110 may determine whether a second AI/ML model trained based on supervised learning (SL) has an accuracy lower than a predefined threshold. If the accuracy of the second AI/ML model is lower than the predefined threshold, the first device 110 may request labeled data from the second device 120 for retraining the second AI/ML model.

In some embodiments, the first device 110 may determine a first accuracy of the trained first AI/ML model and a second accuracy of the trained second AI/ML model, and may select one of the trained first AI/ML model and the trained second AI/ML model to be used at the first device 110 based on the first accuracy and the second accuracy.

FIG. 6 illustrates another flowchart of an example method implemented at a second device in accordance with some embodiments of the present disclosure. For ease of understanding, the method 600 will be described from the perspective of the second device 120 with reference to FIG. 1.

At 610, the second device 120 determines triggering semi-supervised learning (SSL) training for an artificial intelligence/machine learning (AI/ML) model at a first device 110. In some embodiments, the second device 120 may receive a request for SSL training of the AI/ML model from the first device, and based on reception of the request for SSL training, it determines triggering SSL training for the AI/ML model.

In some embodiments, the second device 120 may comprise a location management function (LMF) capable of generating labeled data. In some embodiments, the second device 120 may determine whether a second AI/ML model trained based on supervised learning (SL) has an accuracy lower than a predefined threshold. Based on determining that the accuracy is lower than the predefined threshold, the second device 120 may determine triggering SSL training for a first AI/ML model.

At 620, the second device 120 transmits an indication of triggering SSL training and at least one SSL training rule to the first device 110. In some embodiments, the training SSL rule may comprise, but not limited to, one or more of the following a measurement collection period, a minimum size of unlabeled dataset, an approach for unlabeled data selection, or a balance between the labeled data and the unlabeled data for SSL training. In some embodiments, the approach for unlabeled data selection may comprise at least one of random sampling or cluster-based sampling of unlabeled samples. The approach for unlabeled data selection may comprise at least one of random sampling or cluster-based sampling of unlabeled samples.

In some embodiments, the second device 120 may determine a first accuracy of the trained first AI/ML model and a second accuracy of the trained second AI/ML model, and select one of the trained first AI/ML model and the trained second AI/ML model to be used at the first device 110 based on the first accuracy and the second accuracy; and transmit an indication of the selection to first device 110.

In some embodiments, the second device 120 may determine a third device for collecting at least part of the unlabeled data, and transmit to the first device 110 an indication indicating the first device 110 to receive the at least part of the unlabeled data from the third device via sidelink communications or via a network device between the first device 110 and the third device.

In some embodiments, the second device 120 may receive a request for adjustments of the at least one training SSL rule from the first device 110. Upon reception of the request, the second device 120 may transmit an indication of an adjusted SSL training rule to the first device 110.

In some embodiments, an apparatus capable of performing the method 500 (for example, the first device 110) may comprise means for performing the respective steps of the method 500. The means may be implemented in any suitable form. For example, the means may be implemented in a circuitry or software module.

In some example embodiments, the apparatus comprises: means for receiving, at a first device from a second device, an indication of triggering semi-supervised learning (SSL) training for an artificial intelligence/machine learning (AI/ML) model at the first device and at least one SSL training rule; means for collecting unlabeled data based on the at least one SSL training rule; and means for training the AI/ML model based on a labeled dataset obtained from the second device and an unlabeled dataset including the collected unlabeled data.

In some embodiments, means for receiving an indication of triggering SSL training for an AI/ML model and at least one SSL training rule may comprise means for transmitting, based on capability of the first device and a type of the AI/ML model, a request for SSL training of the AI/ML model to the second device; and means for receiving, from the second device, the indication of triggering SSL training for the AI/ML model and the at least one SSL training rule.

In some embodiments, means for transmitting a request for SSL training of the AI/ML model may comprise means for determining whether the labeled data is sufficient for supervised learning (SL) training; and means for transmitting, based on determining that the labeled data is insufficient for SL training, the request for SSL training to the second device.

In some embodiments, the at least one training SSL rule may comprise at least one of the following: a measurement collection period; a minimum size of unlabeled dataset; an approach for unlabeled data selection; or a balance between the labeled data and the unlabeled data for SSL training.

In some embodiments, the approach for unlabeled data selection may comprise at least one of random sampling or cluster-based sampling of unlabeled samples.

In some embodiments, means for collecting unlabeled data may comprise means for receiving at least part of the unlabeled data from a third device via sidelink communications or via a network device between the first device and the third device.

In some embodiments, means for training the AI/ML model may comprise means for combing the unlabeled data and the labeled data as a training dataset, wherein at least one pseudo value is added as a label for the unlabeled data; and means for performing SSL training of the AI/ML model with the combined training dataset.

In some embodiments, the apparatus may further comprise: means for calculating a statistical distribution difference between the labeled data and the unlabeled data; means for transmitting, based on determining that the statistical distribution difference exceeds a threshold, a request for adjustments of the at least one SSL training rule to the second device; and means for receiving an adjusted SSL training rule from the second device.

In some embodiments, means for training the AI/ML model may comprise means for pre-training the AI/ML model based on one of the unlabeled dataset and the labeled dataset; and means for performing fine tuning of the pre-trained ML model based on the other of the unlabeled dataset and the labeled dataset.

In some embodiments, the AI/ML model is a first AI/ML model, and the apparatus may further comprise: means for determine whether a second AI/ML model trained based on supervised learning (SL) has an accuracy lower than a predefined threshold; and means for requesting, based on determining that the accuracy of the second AI/ML model is lower than the predefined threshold, labeled data from the second device for retraining the second AI/ML model.

In some embodiments, the apparatus may further comprise: means for determining a first accuracy of the trained first AI/ML model and a second accuracy of the second AI/ML model; and means for selecting one of the trained first AI/ML model and the second AI/ML model to be used at the first device based on the first accuracy and the second accuracy.

In some embodiments, the apparatus further comprises means for performing other steps in some embodiments of the method 500. In some embodiments, the means comprises at least one processor and at least one memory including computer program code, the at least one memory and computer program code configured to, with the at least one processor, cause the performance of the apparatus.

In some embodiments, an apparatus capable of performing the method 600 (for example, the second device 120) may comprise means for performing the respective steps of the method 600. The means may be implemented in any suitable form. For example, the means may be implemented in a circuitry or software module.

In some example embodiments, the apparatus comprises: means for determining, at a second device, triggering semi-supervised learning (SSL) training for an artificial intelligence/machine learning (AI/ML) model at a first device; and means for transmitting an indication of triggering SSL training and at least one SSL training rule to the first device.

In some embodiments, means for determining triggering SSL training for the AI/ML model at the first device may comprise: means for receiving a request for SSL training of the AI/ML model from the first device; and means for determining, based on reception of the request for SSL training, triggering SSL training for the AI/ML model.

In some embodiments, means for determining triggering SSL training for the AI/ML model at the first device may comprise: means for receiving, from the first device, a request for labeled data for supervised learning (SL) training; and means for determining, based on determining that the labeled data is insufficient for SL training, triggering SSL training for the AI/ML model.

In some embodiments, the AI/ML model is a first AI/ML model, and means for determining triggering SSL training for the AI/ML model at the first device may comprise means for determining, based on determining that a second AI/ML model trained based on supervised learning (SL) has an accuracy lower than a predefined threshold, triggering SSL training for the first AI/ML model.

In some example embodiments, the apparatus may further comprise: means for determining a first accuracy of the trained first AI/ML model and a second accuracy of the second AI/ML model; means for selecting one of the trained first AI/ML model and the second AI/ML model to be used at the first device based on the first accuracy and the second accuracy; and means for transmitting an indication of the selection to first device.

In some example embodiments, the at least one training SSL rule may comprise at least one of the following: a measurement collection period; a minimum size of unlabeled dataset; an approach for unlabeled data selection; or a balance between labeled data and the unlabeled data for SSL training.

In some example embodiments, the approach for unlabeled data selection may comprise at least one of random sampling or cluster-based sampling of unlabeled samples.

In some example embodiments, the apparatus may further comprise: means for determining a third device for collecting at least part of the unlabeled data; and means for transmitting, to the first device, an indication indicating the first device to receive the at least part of the unlabeled data from the third device via sidelink communications or via a network device between the first device and the third device.

In some example embodiments, the apparatus may further comprise: means for receiving a request for adjustments of the at least one training SSL rule from the first device; and means for transmitting an indication of an adjusted SSL training rule to the first device.

In some example embodiments, the second device may comprise a location management function (LMF) .

In some embodiments, the apparatus further comprises means for performing other steps in some embodiments of the method 600. In some embodiments, the means comprises at least one processor and at least one memory including computer program code, the at least one memory and computer program code configured to, with the at least one processor, cause the performance of the apparatus.

FIG. 7 illustrates a simplified block diagram of a device 700 that is suitable for implementing some example embodiments of the present disclosure. The device 700 may be provided to implement a communication device, for example, the first device 110 or the second device 120 as shown in FIG. 1. As shown, the device 700 includes one or more processors 710, one or more memories 720 coupled to the processor 710, and one or more communication modules 740 coupled to the processor 710.

The communication module 740 is for bidirectional communications. The communication module 740 has at least one antenna to facilitate communication. The communication interface may represent any interface that is necessary for communication with other network elements.

The processor 710 may be of any type suitable to the local technical network and may include one or more of the following: general purpose computers, special purpose computers, microprocessors, digital signal processors (DSPs) and processors based on multicore processor architecture, as non-limiting examples. The device 700 may have multiple processors, such as an application specific integrated circuit chip that is slaved in time to a clock which synchronizes the main processor.

The memory 720 may include one or more non-volatile memories and one or more volatile memories. Examples of the non-volatile memories include, but are not limited to, a Read Only Memory (ROM) 724, an electrically programmable read only memory (EPROM) , a flash memory, a hard disk, a compact disc (CD) , a digital video disk (DVD) , and other magnetic storage and/or optical storage. Examples of the volatile memories include, but are not limited to, a random access memory (RAM) 722 and other volatile memories that will not last in the power-down duration.

A computer program 730 includes computer executable instructions that are executed by the associated processor 710. The program 730 may be stored in the ROM 724. The processor 710 may perform any suitable actions and processing by loading the program 730 into the RAM 722.

The embodiments of the present disclosure may be implemented by means of the program 730 so that the device 700 may perform any process of the disclosure as discussed with reference to FIGS. 5 and 6. The embodiments of the present disclosure may also be implemented by hardware or by a combination of software and hardware.

In some example embodiments, the program 730 may be tangibly contained in a computer-readable medium which may be included in the device 700 (such as in the memory 720) or other storage devices that are accessible by the device 700. The device 700 may load the program 730 from the computer-readable medium to the RAM 722 for execution. The computer-readable medium may include any types of tangible non-volatile storage, such as ROM, EPROM, a flash memory, a hard disk, CD, DVD, and the like.

FIG. 8 illustrates a block diagram of an example of a computer-readable medium 800 in accordance with some example embodiments of the present disclosure. The computer-readable medium 800 has the program 730 stored thereon. It is noted that although the computer-readable medium 800 is depicted in form of CD or DVD in FIG. 8, the computer-readable medium 800 may be in any other form suitable for carry or hold the program 730.

Generally, various embodiments of the present disclosure may be implemented in hardware or special purpose circuits, software, logic or any combination thereof. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software which may be executed by a controller, microprocessor or other computing device. While various aspects of embodiments of the present disclosure are illustrated and described as block diagrams, flowcharts, or using some other pictorial representations, it is to be understood that the block, apparatus, system, technique or method described herein may be implemented in, as non-limiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.

The present disclosure also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as those included in program modules, being executed in a device on a target real or virtual processor, to carry out the method 500 or 600 as described above with reference to FIG. 5 or 6. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, or the like that perform particular tasks or implement particular abstract data types. The functionality of the program modules may be combined or split between program modules as desired in various embodiments. Machine-executable instructions for program modules may be executed within a local or distributed device. In a distributed device, program modules may be located in both local and remote storage media.

Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowcharts and/or block diagrams to be implemented. The program code may execute entirely on a machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of the present disclosure, the computer program codes or related data may be carried by any suitable carrier to enable the device, apparatus or processor to perform various processes and operations as described above. Examples of the carrier include a signal, computer-readable medium, and the like.

The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable medium may include but not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the computer-readable storage medium would include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM) , a read-only memory (ROM) , an erasable programmable read-only memory (EPROM or Flash memory) , an optical fiber, a portable compact disc read-only memory (CD-ROM) , an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. The term “non-transitory, ” as used herein, is a limitation of the medium itself (i.e., tangible, not a signal) as opposed to a limitation on data storage persistency (e.g., RAM vs. ROM) .

Further, while operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are contained in the above discussions, these should not be construed as limitations on the scope of the present disclosure, but rather as descriptions of features that may be specific to particular embodiments. Certain features that are described in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment may also be implemented in multiple embodiments separately or in any suitable sub-combination.

Although the present disclosure has been described in languages specific to structural features and/or methodological acts, it is to be understood that the present disclosure defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

A first device comprising:

at least one processor; and

at least one memory storing instructions that, when executed by the at least one processor, cause the first device at least to:

receive, from a second device, an indication of triggering semi-supervised learning (SSL) training for an artificial intelligence/machine learning (AI/ML) model at the first device and at least one SSL training rule;

collect unlabeled data based on the at least one SSL training rule; and

train the AI/ML model based on a labeled dataset obtained from the second device and an unlabeled dataset including the collected unlabeled data.
The first device of claim 1, wherein the first device is caused to receive the indication of triggering SSL training for the AI/ML model and the at least one SSL training rule by:

transmitting, based on capability of the first device and a type of the AI/ML model, a request for SSL training of the AI/ML model to the second device; and

receiving, from the second device, the indication of triggering SSL training for the AI/ML model and the at least one SSL training rule.
The first device of claim 2, wherein the first device is caused to transmit the request for SSL training to the second device by:

determining whether the labeled data is sufficient for supervised learning (SL) training; and

based on determining that the labeled data is insufficient for SL training, transmitting the request for SSL training to the second device.
The first device of any of claims 1 to 3, wherein the at least one training SSL rule comprises at least one of the following:

a measurement collection period;

a minimum size of unlabeled dataset;

an approach for unlabeled data selection; or

a balance between the labeled data and the unlabeled data for SSL training.
The first device of claim 4, wherein the approach for unlabeled data selection comprises at least one of random sampling or cluster-based sampling of unlabeled samples.
The first device of any of claims 1 to 5, wherein the first device is caused to collect the unlabeled data by:

receiving at least part of the unlabeled data from a third device via sidelink communications or via a network device between the first device and the third device.
The first device of any of claims 1 to 6, wherein the first device is caused to train the AI/ML model by:

combing the unlabeled data and the labeled data as a training dataset, wherein at least one pseudo value is added as a label for the unlabeled data; and

performing SSL training of the AI/ML model with the combined training dataset.
The first device of any of claims 1 to 7, wherein the first device is further caused to:

calculate a statistical distribution difference between the labeled data and the unlabeled data;

based on determining that the statistical distribution difference exceeds a threshold; transmit a request for adjustments of the at least one SSL training rule to the second device; and

receive an adjusted SSL training rule from the second device.
The first device of any of claims 1 to 8, wherein the first device is caused to train the AI/ML model by:

pre-training the AI/ML model based on one of the unlabeled dataset and the labeled dataset; and

performing fine tuning of the pre-trained ML model based on the other of the unlabeled dataset and the labeled dataset.
The first device of any of claims 1 to 9, wherein the AI/ML model is a first AI/ML model, and wherein the first device is further caused to:

determine whether a second AI/ML model trained based on supervised learning (SL) has an accuracy lower than a predefined threshold; and

based on determining that the accuracy of the second AI/ML model is lower than the predefined threshold, request labeled data from the second device for retraining the second AI/ML model.
The first device of claim 10, wherein the first device is further caused to:

determine a first accuracy of the trained first AI/ML model and a second accuracy of the second AI/ML model; and

select one of the trained first AI/ML model and the second AI/ML model to be used at the first device based on the first accuracy and the second accuracy.
A second device comprising:

at least one processor; and

at least one memory storing instructions that, when executed by the at least one processor, cause the second device at least to:

determine triggering semi-supervised learning (SSL) training for an artificial intelligence/machine learning (AI/ML) model at a first device; and

transmit an indication of triggering SSL training and at least one SSL training rule to the first device.
The second device of claim 12, wherein the second device is caused to determine triggering SSL training for the AI/ML model at the first device by:

receiving a request for SSL training of the AI/ML model from the first device; and

based on reception of the request for SSL training, determining triggering SSL training for the AI/ML model.
The second device of claim 12, wherein the second device is caused to determine triggering SSL training for the AI/ML model at the first device by:

receiving, from the first device, a request for labeled data for supervised learning (SL) training; and

based on determining that the labeled data is insufficient for SL training, determining triggering SSL training for the AI/ML model.
The second device of claim 12, wherein the AI/ML model is a first AI/ML model, and wherein the second device is caused to determine triggering SSL training for the AI/ML model by:

based on determining that a second AI/ML model trained based on supervised learning (SL) has an accuracy lower than a predefined threshold, determining triggering SSL training for the first AI/ML model.
The second device of claim 15, wherein the second device is further caused to:

determine a first accuracy of the trained first AI/ML model and a second accuracy of the second AI/ML model;

select one of the trained first AI/ML model and the second AI/ML model to be used at the first device based on the first accuracy and the second accuracy; and

transmit an indication of the selection to first device.
The second device of any of claims 12 to 16, wherein the at least one training SSL rule comprises at least one of the following:

a measurement collection period;

a minimum size of unlabeled dataset;

an approach for unlabeled data selection; or

a balance between labeled data and the unlabeled data for SSL training.
The second device of claim 17, wherein the approach for unlabeled data selection comprises at least one of random sampling or cluster-based sampling of unlabeled samples.
The second device of any of claims 12 to 18, wherein the second device is further caused to:

determine a third device for collecting at least part of the unlabeled data; and

transmit, to the first device, an indication indicating the first device to receive the at least part of the unlabeled data from the third device via sidelink communications or via a network device between the first device and the third device.
The second device any of claims 12 to 19, wherein the second device is further caused to:

receive a request for adjustments of the at least one training SSL rule from the first device; and

transmit an indication of an adjusted SSL training rule to the first device.
The second device of any of claims 12 to 20, wherein the second device comprises a location management function (LMF) .
A method comprising:

receiving, at a first device from a second device, an indication of triggering semi-supervised learning (SSL) training for an artificial intelligence/machine learning (AI/ML) model at the first device and at least one SSL training rule;

collecting unlabeled data based on the at least one SSL training rule; and

training the AI/ML model based on a labeled dataset obtained from the second device and an unlabeled dataset including the collected unlabeled data.
A method comprising:

determining, at a second device, triggering semi-supervised learning (SSL) training for an artificial intelligence/machine learning (AI/ML) model at a first device; and

transmitting an indication of triggering SSL training and at least one SSL training rule to the first device.
An apparatus comprising:

means for receiving, at a first device from a second device, an indication of triggering semi-supervised learning (SSL) training for an artificial intelligence/machine learning (AI/ML) model at the first device and at least one SSL training rule;

means for collecting unlabeled data based on the at least one SSL training rule; and

means for training the AI/ML model based on a labeled dataset obtained from the second device and an unlabeled dataset including the collected unlabeled data.
An apparatus comprising:

means for determining, at a second device, triggering semi-supervised learning (SSL) training for an artificial intelligence/machine learning (AI/ML) model at a first device; and

means for transmitting an indication of triggering SSL training and at least one SSL training rule to the first device.
A non-transitory computer readable medium comprising program instructions that, when executed by an apparatus, cause the apparatus to perform at least:

receiving, at a first device from a second device, an indication of triggering semi-supervised learning (SSL) training for an artificial intelligence/machine learning (AI/ML) model at the first device and at least one SSL training rule;

collecting unlabeled data based on the at least one SSL training rule; and

training the AI/ML model based on a labeled dataset obtained from the second device and an unlabeled dataset including the collected unlabeled data.
A non-transitory computer readable medium comprising program instructions that, when executed by an apparatus, cause the apparatus to perform at least:

determining, at a second device, triggering semi-supervised learning (SSL) training for an artificial intelligence/machine learning (AI/ML) model at a first device; and

transmitting an indication of triggering SSL training and at least one SSL training rule to the first device.