WO2025065259A1 - Performance evaluation method for artificial intelligence prediction model, terminal, and medium - Google Patents

Abstract

The present disclosure relates to a performance evaluation method for an artificial intelligence prediction model, a terminal, and a storage medium. The performance evaluation method comprises: predicting a first signal quality by means of a first model, wherein the first signal quality is a predicted value of a first reference signal transmitted by means of a first beam; obtaining first difference information, wherein the first difference information is used for representing a difference between the first signal quality and a second signal quality, and the second signal quality is a reference value of the signal quality of the first reference signal; and determining prediction performance of the first model on the basis of the first difference information. According to the method, the performance of the first model can be accurately evaluated.

Description

Performance evaluation methods, terminals and media for artificial intelligence prediction models Technical Field

The present disclosure relates to the field of communication technology, and in particular to a performance evaluation method, terminal, and storage medium for an artificial intelligence prediction model.

Background Art

The terminal can predict the optimal beam through the artificial intelligence prediction model, and can also predict the signal quality of the reference signal transmitted through a certain beam. Therefore, it is necessary to evaluate the accuracy of the prediction and evaluate the performance of the artificial intelligence prediction model based on the accuracy.

Summary of the invention

In order to evaluate the performance of an artificial intelligence prediction model, the embodiments of the present disclosure provide a performance evaluation method, a terminal, and a storage medium for an artificial intelligence prediction model.

According to a first aspect of an embodiment of the present disclosure, a performance evaluation method for an artificial intelligence prediction model is provided, which is applied to a terminal, and the method includes:

Predicting a first signal quality by using a first model, where the first signal quality is a predicted value of a first reference signal transmitted by a first beam;

Acquire first difference information, where the first difference information is used to indicate a difference between the first signal quality and a second signal quality, where the second signal quality is a reference value of a signal quality of the first reference signal;

The prediction performance of the first model is determined according to the first difference information.

According to a second aspect of an embodiment of the present disclosure, a performance evaluation method for an artificial intelligence prediction model is provided, which is applied to a terminal, and the method includes:

Predicting a first beam by using a first model, wherein the first beam is a beam with the best signal quality in the first set predicted by the terminal;

determining a third signal quality, where the third signal quality is a signal quality of a first reference signal transmitted through the first beam;

Acquire second difference information, where the second difference information is used to indicate a difference between the third signal quality and the fourth signal quality, where the fourth signal quality is a reference value of the signal quality of the first reference signal transmitted through the second beam, wherein the second beam is a beam with the best signal quality in the first set measured by the terminal, or the second beam is a best beam determined from the first set based on actual signal quality, or the second beam is a default beam;

The prediction performance of the first model is determined according to the second difference information.

According to a third aspect of an embodiment of the present disclosure, a terminal is provided, including:

The processing module is configured as follows:

Predicting a first signal quality by using a first model, where the first signal quality is a predicted value of a first reference signal transmitted by a first beam;

Acquire first difference information, where the first difference information is used to indicate a difference between the first signal quality and a second signal quality, where the second signal quality is a reference value of a signal quality of the first reference signal;

The prediction performance of the first model is determined according to the first difference information.

According to a fourth aspect of an embodiment of the present disclosure, a terminal is provided, including:

The processing module is configured as follows:

Predicting a first beam by using a first model, wherein the first beam is a beam with the best signal quality in the first set predicted by the terminal;

determining a third signal quality, where the third signal quality is a signal quality of a first reference signal transmitted through the first beam;

Acquire second difference information, where the second difference information is used to indicate a difference between the third signal quality and the fourth signal quality, where the fourth signal quality is a reference value of the signal quality of the first reference signal transmitted through the second beam, wherein the second beam is a beam with the best signal quality in the first set measured by the terminal, or the second beam is a best beam determined from the first set based on actual signal quality, or the second beam is a default beam;

The prediction performance of the first model is determined according to the second difference information.

According to a fifth aspect of an embodiment of the present disclosure, a terminal is provided, including:

one or more processors;

The terminal is used to execute the method described in the first aspect or the second aspect.

According to a sixth aspect of an embodiment of the present disclosure, a storage medium is provided, wherein the storage medium stores instructions, and when the instructions are executed on a communication device, the communication device executes the method described in the first aspect or the second aspect.

The method provided in the embodiment of the present disclosure compares the predicted or measured signal quality with the signal quality of the reference standard to obtain the difference information between the two, and evaluates the prediction performance of the model based on the difference information, which can be used in different application scenarios or at different times. At different points in time, the performance of the first model is evaluated using different beams to obtain more accurate evaluation results.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings required for describing the embodiments are introduced below. The following drawings are only some embodiments of the present disclosure and do not impose specific limitations on the protection scope of the present disclosure.

FIG1 is a schematic diagram of a communication system architecture provided according to an embodiment of the present disclosure.

FIG2 is an interactive schematic diagram of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure.

FIG3 is an interactive schematic diagram of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure.

FIG4 is an interactive schematic diagram of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure.

FIG5 is a flowchart of a method for evaluating the performance of an artificial intelligence prediction model according to an embodiment of the present disclosure.

FIG6 is an interactive schematic diagram of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure.

FIG7 is an interactive schematic diagram of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure.

FIG8 is an interactive schematic diagram of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure.

FIG9 is an interactive schematic diagram of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure.

FIG10 is an interactive schematic diagram of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure.

FIG11 is a flowchart of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure.

FIG12 is a flowchart of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure.

FIG13 is a structural diagram of a performance evaluation device for an artificial intelligence prediction model provided according to an embodiment of the present disclosure.

FIG14 is a structural diagram of a performance evaluation device for an artificial intelligence prediction model provided according to an embodiment of the present disclosure.

DETAILED DESCRIPTION

The embodiments of the present disclosure provide a performance evaluation method, a terminal, and a storage medium for an artificial intelligence prediction model.

In a first aspect, an embodiment of the present disclosure provides a performance evaluation method for an artificial intelligence prediction model, which is applied to a terminal, and the method includes:

Predicting a first signal quality by using a first model, where the first signal quality is a predicted value of a first reference signal transmitted by a first beam;

Acquire first difference information, where the first difference information is used to indicate a difference between the first signal quality and a second signal quality, where the second signal quality is a reference value of a signal quality of the first reference signal;

The prediction performance of the first model is determined according to the first difference information.

In the above embodiment, the predicted or measured signal quality is compared with the signal quality of the reference standard to obtain difference information between the two. The prediction performance of the model is evaluated based on the difference information. Different beams can be used for evaluation in different application scenarios or at different time points to obtain more accurate evaluation results.

In conjunction with some embodiments of the first aspect, in some embodiments, predicting the first signal quality by using the first model includes:

Measuring, in a first time domain unit, a signal quality of the first reference signal transmitted through the second beam to obtain a first measurement value of the first reference signal;

Taking a first measurement value of the first reference signal as input, predicting the first signal quality in a second time domain unit by using the first model;

The time domain unit is one of: time slot, half time slot, symbol.

In the above embodiment, the current signal quality of the target beam is predicted based on the historical signal quality of other beams, and the correlation between beams, such as the carrier aggregation relationship, can be used to obtain a reasonable prediction result.

In combination with some embodiments of the first aspect, in some embodiments, the method further includes:

The signal quality of the first reference signal transmitted through the first beam in the second time domain unit is measured to obtain the second signal quality.

In the above embodiment, the measured signal quality is used as the reference quality, so that the difference information represents the relative difference, which can better reflect the impact of the application scenario on the signal quality in different application scenarios.

In combination with some embodiments of the first aspect, in some embodiments, the second signal quality is a true value of the first reference signal transmitted through the first beam in the second time domain unit.

In the above embodiment, the real signal quality is used as the reference signal quality, so that the first difference information represents the absolute difference, and the first difference information is not affected by the application scenario, and can better reflect the real difference.

In combination with some embodiments of the first aspect, in some embodiments, the method further includes:

obtaining the second signal quality;

sending the first signal quality and the second signal quality to a test device;

The obtaining of the first difference information includes:

The first difference information sent by the test device is received, where the first difference information is determined by the test device according to the first signal quality and the second signal quality.

In the above embodiment, the first signal quality and the second signal quality are provided to the test device, which is suitable for an application scenario in which the terminal has the ability to calculate the second signal quality.

In combination with some embodiments of the first aspect, in some embodiments, the method further includes:

sending the first signal quality to a test device;

The obtaining of the first difference information includes:

The first difference information sent by the test device is received, where the first difference information is determined by the test device according to the first signal quality and the second signal quality, and the second signal quality is determined by the test device.

In the above embodiment, only the first signal quality is provided to the test device, which is suitable for an application scenario in which the test device has the ability to calculate the second signal quality.

In combination with some embodiments of the first aspect, in some embodiments, the method further includes:

Predicting a first beam by using a first model, wherein the first beam is a beam with the best signal quality in the first set predicted by the terminal;

determining a third signal quality, where the third signal quality is a signal quality of a first reference signal transmitted through the first beam;

Acquire second difference information, where the second difference information is used to indicate a difference between the third signal quality and the fourth signal quality, where the fourth signal quality is a reference value of the signal quality of the first reference signal transmitted through the second beam, wherein the second beam is a beam with the best signal quality in the first set measured by the terminal, or the second beam is a best beam determined from the first set based on actual signal quality, or the second beam is a default beam;

The prediction performance of the first model is determined according to the second difference information.

In the above embodiment, the optimal beam is predicted, and the first signal quality is determined according to the optimal beam, so that the performance of the model in predicting the signal quality and the performance of the predicted beam can be evaluated simultaneously.

In conjunction with some embodiments of the first aspect, in some embodiments, determining the third signal quality includes:

The third signal quality is predicted by using the first model.

In combination with some embodiments of the first aspect, in some embodiments, the determining the third signal quality, the method further includes:

Measuring a signal quality of the first reference signal corresponding to each beam in the first set to obtain a measurement result;

According to the identification information of the first beam, it is determined that the third signal quality is a measurement result in the measurement results corresponding to the identification information of the first beam.

In the above embodiments, the measurement results may be used for reasonable processing required in the evaluation process.

In combination with some embodiments of the first aspect, in some embodiments, the method further includes:

sending the third signal quality to a test device;

The obtaining of the second difference information includes:

receiving the second difference information sent by the test device, wherein the fourth signal quality is a true value of the signal quality of the first reference signal transmitted through the second beam determined by the test device;

In the above embodiment, only the first signal quality is provided to the test device, which is suitable for an application scenario in which the test device has the ability to obtain the second beam and the ability to determine the fourth signal quality.

In combination with some embodiments of the first aspect, in some embodiments, the method further includes:

Predicting the third signal quality by using the first model;

sending the third signal quality and the measurement result to a test device;

The obtaining of the second difference information includes:

The second difference information sent by the test device is received, wherein the fourth signal quality is the signal quality corresponding to the second beam in the measurement result.

In the above embodiment, providing the first signal quality and the measurement result to the test device is applicable to an application scenario in which the test device has the ability to obtain the second beam but does not have the ability to determine the fourth signal quality.

In combination with some embodiments of the first aspect, in some embodiments, the method further includes:

Sending identification information of the first beam and the measurement result to a test device;

The obtaining of the second difference information includes:

The second difference information sent by the test device is received, wherein the first signal quality is the signal quality corresponding to the first beam in the measurement result, and the fourth signal quality is the signal quality corresponding to the second beam in the measurement result.

The above embodiment is applicable to the application field where the test device has the ability to obtain the second beam but does not have the ability to calculate the fourth signal quality. scene.

In combination with some embodiments of the first aspect, in some embodiments, the method further includes:

Predicting the third signal quality by using the first model;

Determine the fourth signal quality according to the measurement result, where the fourth signal quality is the signal quality corresponding to the second beam in the measurement result;

sending the third signal quality and the fourth signal quality to a test device;

The obtaining of the second difference information includes:

The second difference information sent by the test device is received.

The above embodiment is applicable to applications where the test device does not have the ability to obtain the second beam and does not have the ability to calculate the fourth signal quality.

scene.

In combination with some embodiments of the first aspect, in some embodiments, the method further includes:

determining a third signal quality and a fourth signal quality according to the measurement result, the third signal quality being a measurement result corresponding to the identification information of the first beam in the measurement result, and the fourth signal quality being a signal quality corresponding to the second beam in the measurement result;

sending the third signal quality and the fourth signal quality to a test device;

The obtaining of the second difference information includes:

The second difference information sent by the test device is received.

The above embodiment is applicable to applications where the test device does not have the ability to obtain the second beam and does not have the ability to calculate the fourth signal quality.

scene.

In a second aspect, an embodiment of the present disclosure provides a terminal, including:

The processing module is configured as follows:

Predicting a first signal quality by using a first model, where the first signal quality is a predicted value of a first reference signal transmitted by a first beam;

Acquire first difference information, where the first difference information is used to indicate a difference between the first signal quality and a second signal quality, where the second signal quality is a reference value of a signal quality of the first reference signal;

The prediction performance of the first model is determined according to the first difference information.

In a third aspect, an embodiment of the present disclosure provides a terminal, including:

The processing module is configured as follows:

Predicting a first beam by using a first model, wherein the first beam is a beam with the best signal quality in the first set predicted by the terminal;

determining a third signal quality, where the third signal quality is a signal quality of a first reference signal transmitted through the first beam;

Acquire second difference information, where the second difference information is used to indicate a difference between the third signal quality and the fourth signal quality, where the fourth signal quality is a reference value of the signal quality of the first reference signal transmitted through the second beam, wherein the second beam is a beam with the best signal quality in the first set measured by the terminal, or the second beam is a best beam determined from the first set based on actual signal quality, or the second beam is a default beam;

The prediction performance of the first model is determined according to the second difference information.

In a fourth aspect, an embodiment of the present disclosure provides a terminal, comprising: one or more processors; wherein the above-mentioned terminal is used to execute the method described in the first aspect or the second aspect.

In a fifth aspect, an embodiment of the present disclosure provides a storage medium, wherein the storage medium stores instructions, and when the instructions are executed on a communication device, the communication device executes the method described in the first aspect or the second aspect.

In a sixth aspect, an embodiment of the present disclosure provides a program product. When the program product is executed by a communication device, the communication device executes the method described in the first aspect or the second aspect.

In a seventh aspect, an embodiment of the present disclosure provides a computer program, which, when executed on a computer, enables the computer to execute the method described in the first aspect or the second aspect.

In an eighth aspect, an embodiment of the present disclosure provides a chip or a chip system, wherein the chip or the chip system comprises a processing circuit configured to execute the method according to the first aspect or the second aspect.

It is understandable that the above-mentioned terminal, storage medium, program product, computer program, chip or chip system are all used to execute the method proposed in the embodiment of the present disclosure. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, which will not be repeated here.

The embodiments of the present disclosure are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and cannot be understood as limiting the present disclosure.

As shown in FIG1 , the method provided in the embodiment of the present disclosure may be applied to a wireless communication system 100, which may include a terminal 101 and a test device 102. It should be noted that the wireless communication system 100 may also include other devices. The equipment included is not limited.

It should be understood that the above wireless communication system 100 can be applied to both low-frequency scenarios and high-frequency scenarios. Application scenarios of the wireless communication system 100 include, but are not limited to, long-term evolution (LTE) systems, LTE frequency division duplex (FDD) systems, LTE time division duplex (TDD) systems, worldwide interoperability for microwave access (WiMAX) communication systems, cloud radio access network (CRAN) systems, future fifth-generation (5G) systems, new radio (NR) communication systems or future evolved public land mobile network (PLMN) systems, Internet of Things systems, etc.

The terminal 101 shown above may be a terminal, an access terminal, a terminal unit, a terminal station, a mobile station (MS), a remote station, a remote terminal, a mobile terminal, a wireless communication device, a terminal agent, an Internet of Things terminal, etc. The terminal 101 may have a wireless transceiver function, and it can communicate with one or more network devices of one or more communication systems (such as wireless communication) and receive network services provided by the network devices.

Among them, terminal 101 can be a cellular phone, a cordless phone, a session initiation protocol (SIP) phone, a wireless local loop (WLL) station, a personal digital assistant (PDA) device, a handheld device with wireless communication function, a computing device or other processing device connected to a wireless modem, a vehicle-mounted device, a wearable device, a terminal in a future 5G network, or a terminal in a future evolved PLMN network, etc.

Among them, the terminal 101 can also be an Internet of Things terminal. Among them, the complexity, manufacturing cost and maintenance cost of the Internet of Things terminal are lower than those of ordinary terminals. The Internet of Things terminal can be powered by receiving electromagnetic signals without a battery. The Internet of Things terminal can also have a battery with a small amount of electrical storage function, which does not need to be manually charged, but obtains battery energy from the outside, for example, by obtaining external electromagnetic waves, heat energy, kinetic energy, etc.

When applying the artificial intelligence prediction model, the terminal can predict the RSRP of some beams based on the RSRP of some measured beams. The artificial intelligence prediction model may introduce additional prediction errors. In order to determine the prediction performance of the artificial intelligence prediction model, it is necessary to calculate the difference information, which can be the difference between the predicted RSRP and the reference RSRP, thereby indicating the difference between the predicted value and the actual value.

Then the prediction accuracy includes measurement error and prediction error, so it is necessary to consider both prediction error and measurement error.

As shown in Tables 1 and 2 below, Table 1 shows the accuracy requirements of SSB based on L1-RSRP absolute accuracy in FR1, and Table 2 shows the accuracy requirements of SSB based on L1-RSRP absolute accuracy in FR2.

Table 1

Table 2

In the following embodiments, the signal quality may be a reference signal receiving power (RSRP), such as L1-RSRP.

FIG2 is an interactive schematic diagram of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure. As shown in FIG2 , the method includes the following steps:

Step S2101: The terminal predicts a first signal quality through a first model.

In some embodiments, the first model is an artificial intelligence model.

In some embodiments, the first model is an artificial intelligence predictive model.

In some embodiments, the first signal quality is a predicted value of a first reference signal transmitted via the first beam.

In some embodiments, the first reference signal is a downlink reference signal.

In some embodiments, the first beam is any beam in the first set.

In some embodiments, the first set is a beam set.

In some embodiments, the first set is a preset set, and the beams included in the first set are pre-set.

In some embodiments, predicting the first signal quality by using the first model includes:

The signal quality of the first reference signal transmitted through the second beam is measured in the first time domain unit to obtain a first measurement value of the first reference signal; the first measurement value of the first reference signal is used as input to predict the first signal quality in the second time domain unit through a first model; wherein the time domain unit is one of: time slot, half time slot, symbol.

In one example, the signal quality of the first reference signal transmitted through the first beam (beam1) is measured in the first time slot (slot1) to obtain a first measurement value of the first reference signal, and the first measurement value of the first reference signal is used as input to predict the first signal quality of the second beam (beam2) in the second time slot (slot2) through the first model.

In another example, the signal quality of the first reference signal transmitted through the first beam (beam3) is measured in the first time slot (slot1) to obtain a first measurement value of the first reference signal, and the first measurement value of the first reference signal is used as input to predict the first signal quality of the second beam (beam4) in the second time slot (slot2) through the first model.

Step S2102: The terminal determines a second signal quality.

In some embodiments, the terminal measures a signal quality of a first reference signal to obtain a measured second signal quality.

In some embodiments, the second signal quality is a reference value of a first reference signal transmitted via the first beam.

In some embodiments, the terminal obtains a known second signal quality.

In some embodiments, the second signal quality is a true signal quality.

Optionally, the real signal quality is an accurate signal quality.

Alternatively, the true signal quality may be referred to as genie-aided signal quality.

Step S2103: The testing device sends a second signal quality to the terminal.

In some embodiments, the second signal quality is a true signal quality.

Optionally, the real signal quality is an accurate signal quality.

Alternatively, the true signal quality may be referred to as genie-aided signal quality.

Step S2104: The terminal determines first difference information according to the first signal quality and the second signal quality.

In some embodiments, the first difference information indicates a difference between the first signal quality and the second signal quality.

In some embodiments, the first difference information is a difference between the predicted RSRP and the actual RSRP.

Step S2105: The terminal determines the prediction performance of the first model according to the first difference information.

The method involved in the embodiment of the present disclosure may include step S2101, step S2103, step S2105, and step S2106. In this case, step S2102 is omitted.

The method involved in the embodiment of the present disclosure may also include step S2101, step S2102, step S2105, and step S2106. In this case, step S2103 is omitted.

FIG3 is an interactive schematic diagram of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure. As shown in FIG3 , the method includes the following steps:

Step S3101: The terminal predicts a first signal quality through a first model.

The optional implementation of step S3101 can refer to the optional implementation of step S2101 in FIG. 2 and other related parts in the embodiment involved in FIG. 2 , which will not be described in detail here.

Step S3102: The terminal obtains the second signal quality.

For optional implementations of step S3102, see the optional implementations of step S2102 or step S2103 in FIG. 2 .

Step S3103: The terminal sends the first signal quality and the second signal quality to the test device.

Step S3104: The testing device determines first difference information according to the first signal quality and the second signal quality.

When step S2102 of FIG. 2 is used in step S3102, the first difference information is relative difference information.

When step S2103 of FIG. 2 is used in step S3102, the first difference information is absolute difference information.

Step S3105: The testing device sends first difference information to the terminal.

In some embodiments, the first difference information is a difference between the predicted RSRP and the actual RSRP.

Step S3106: The terminal determines the prediction performance of the first model according to the first difference information.

FIG4 is an interactive schematic diagram of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure. As shown in FIG4 , the method includes the following steps:

Step S4101: The terminal predicts a first signal quality through a first model.

The optional implementation of step S4101 can refer to the optional implementation of step S2101 in FIG. 2 and other related parts in the embodiment involved in FIG. 2 , which will not be described in detail here.

Step S4102: The terminal sends a first signal quality to the test device.

Step S4103: The testing device obtains the second signal quality.

In some embodiments, the test device simulates the behavior of the base station, that is, sends the first reference signal to the terminal through different beams, so that the test device can calculate the second signal quality.

In some embodiments, the second signal quality is a true signal quality.

Optionally, the real signal quality is an accurate signal quality.

Alternatively, the true signal quality may be referred to as genie-aided signal quality.

Step S4104: The testing device determines first difference information according to the first signal quality and the second signal quality.

In some embodiments, the first difference information is absolute difference information.

Step S4105: The testing device sends first difference information to the terminal.

Step S4106: The terminal determines the prediction performance of the first model according to the first difference information.

FIG5 is an interactive schematic diagram of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure, which is applied to a terminal, as shown in FIG5 , and includes the following steps:

Step S5101: The terminal predicts a first beam through a first model.

In some embodiments, the first beam is a beam with the best signal quality in the first set predicted by the terminal.

In some embodiments, the signal quality of the first reference signal transmitted via the first beam is optimal in the first set.

In some embodiments, the first beam is the best beam in the first set.

In some embodiments, the first set is a preset set, and the beams included in the first set are pre-set.

In some embodiments, the index of the predicted first beam is i.

Step S5102: The terminal predicts the third signal quality by using the first model.

In some embodiments, the third signal quality is a signal quality of a first reference signal transmitted via the first beam.

In some embodiments, the predicted quality of the third signal is recorded as Predicted L1-RSRP _i .

Step S5103: The terminal measures the signal quality of the first reference signal corresponding to each beam in the first set to obtain a measurement result.

In some embodiments, the measurement result includes a signal quality of a first reference signal corresponding to each beam in the first set.

Step S5104: The terminal determines a third signal quality according to the measurement result.

In some embodiments, based on the identification information of the first beam, the third signal quality is determined to be a measurement result corresponding to the identification information of the first beam in the measurement results.

Step S5104: the terminal obtains second difference information.

In some embodiments, the second difference information is used to indicate a difference between the third signal quality and the fourth signal quality, where the fourth signal quality is a reference value of the signal quality of the first reference signal transmitted via the second beam.

In some embodiments, the second beam is a beam with the best signal quality in the first set measured by the terminal.

In some embodiments, the second beam is the best beam determined from the first set based on actual signal quality.

In some embodiments, the second beam is a default beam. For example, the index of the second beam is q, where q is a default value. The index of the second beam can be called the best genie-aided beam index.

Step S5105: The terminal determines the prediction performance of the first model according to the second difference information.

In the method involved in the embodiments of the present disclosure, step S5102 or step S5103 may be omitted.

FIG6 is an interactive schematic diagram of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure. As shown in FIG6 , the method includes the following steps:

Step S6101: The terminal predicts a first beam according to a first model.

The optional implementation of step S6101 can refer to the optional implementation of step S5101 in Figure 5 and other related parts in the embodiment involved in Figure 5, which will not be repeated here.

Step S6102: The terminal predicts the third signal quality according to the first model.

The optional implementation of step S6102 can refer to the optional implementation of step S5102 in FIG. 5 and other related parts in the embodiment involved in FIG. 5 , which will not be described in detail here.

Step S6103: The terminal sends a third signal quality to the test device.

In some embodiments, the terminal sends the third signal quality and identification information of the first beam to the test device.

Step S6104: The testing device determines the second beam and the fourth signal quality.

In some embodiments, the test equipment determines the second beam.

In some embodiments, the second beam is the best beam determined from the first set based on actual signal quality.

In some embodiments, the test device simulates the behavior of the base station, that is, sends the first reference signal to the terminal through different beams, so that the test device can calculate the fourth signal quality.

In some embodiments, the second beam is a default beam. For example, the index of the second beam is q, where q is a default value. The index of the second beam can be called the best genie-aided beam index.

In some embodiments, the three signal qualities are recorded as Predicted L1-RSRP _i .

In some embodiments, the fourth signal quality is denoted as Genie L1-RSRP _q .

Step S6105: The testing device determines the second difference information.

In some embodiments, the testing device determines that the second difference information is the difference between the third signal quality and the fourth signal quality. The second difference information is the difference between the third signal quality predicted by the terminal and the fourth signal quality calculated by the first reference signal corresponding to the actual optimal beam (i.e., the second beam), which can be considered as an absolute difference information.

In some embodiments, the second difference information is: Predicted L1-RSRP _i −Genie L1-RSRP _q .

Step S6106: The testing device sends second difference information to the terminal.

Step S6107: The terminal determines the prediction performance of the first model according to the second difference information.

FIG. 7 is an interactive schematic diagram of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure. As shown in FIG. 7 , the method includes the following steps:

Step S7101: The terminal predicts a first beam through a first model.

The optional implementation of step S7101 can refer to the optional implementation of step S5101 in Figure 5 and other related parts in the embodiment involved in Figure 5, which will not be repeated here.

Step S7102: The terminal predicts the third signal quality through the first model.

For the optional implementation of step S7102, reference may be made to the optional implementation of step S5102 in FIG5 and other related parts in the embodiment involved in FIG5, which will not be described in detail here.

Step S7103: The terminal measures the signal quality of the first reference signal corresponding to each beam in the first set to obtain a measurement result.

The optional implementation of step S7103 can refer to the optional implementation of step S5103 in FIG5 and other related parts in the embodiment involved in FIG5 , which will not be described in detail here.

Step S7104: The terminal sends the third signal quality and the measurement result to the test device.

In some embodiments, the terminal sends the third signal quality, the measurement result, and identification information of the first beam to the test device.

Step S7105: The testing device determines a second beam, and determines a fourth signal quality according to the second beam and the measurement result.

In some embodiments, the second beam is the best beam determined from the first set based on actual signal quality.

In some embodiments, the second beam is a default beam. For example, the index of the second beam is q, where q is a default value. The index of the second beam can be called the best genie-aided beam index.

In some embodiments, the test device determines that the fourth signal quality is a measured signal quality corresponding to the second beam in the measurement result.

In some embodiments, the third signal quality is recorded as Predicted L1-RSRP _i .

In some embodiments, the fourth signal quality is recorded as Measured L1-RSRP _q .

Step S7106: The testing device determines second difference information.

In some embodiments, the testing device determines that the second difference information is the difference between the third signal quality and the fourth signal quality. The second difference information is the difference between the predicted third signal quality and the measured fourth signal quality corresponding to the actual optimal beam (i.e., the second beam), which can be considered as an absolute difference information.

In some embodiments, the second difference information is: Predicted L1-RSRP _i −Measured L1-RSRP _q .

Step S7107: The testing device sends second difference information to the terminal.

Step S7108: The terminal determines the prediction performance of the first model based on the second difference information.

FIG8 is an interactive schematic diagram of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure. As shown in FIG8 , the method includes the following steps:

Step S8101: The terminal predicts a first beam through a first model.

The optional implementation of step S8101 can refer to the optional implementation of step S5101 in FIG5 and other related parts in the embodiment involved in FIG5 , which will not be described in detail here.

Step S8102: The terminal measures the signal quality of the first reference signal corresponding to each beam in the first set to obtain a measurement result.

The optional implementation of step S8102 can refer to the optional implementation of step S5103 in FIG. 5 and other related parts in the embodiment involved in FIG. 5 , which will not be described in detail here.

Step S8103: The terminal determines a third signal quality according to the measurement result.

In some embodiments, the third signal quality is determined to be a measurement result corresponding to the identification information of the first beam in the measurement results.

Step S8104: The terminal sends the measurement result to the test device.

In some embodiments, the terminal sends the measurement result and the identification of the first beam to the test device.

Step S8105: The testing device determines a second beam, and determines a fourth signal quality according to the second beam and the measurement result.

In some embodiments, the second beam is the best beam determined from the first set based on the actual signal quality. For example, the index of the second beam is q, and the index of the second beam can be called the best genie-aided beam index.

In some embodiments, the second beam is a default beam.

In some embodiments, the test device determines that the fourth signal quality is a measured signal quality corresponding to the second beam in the measurement result.

In some embodiments, the third signal quality is recorded as Measured L1-RSRP _i .

In some embodiments, the fourth signal quality is recorded as Measured L1-RSRP _q .

Step S8106: The testing device determines the second difference information.

In some embodiments, when the testing device receives the measurement result and the identifier of the first beam, it determines the third signal quality based on the identifier of the first beam and the measurement result, and determines the second difference information based on the third signal quality and the fourth signal quality. The second difference information is the difference between the measured third signal quality and the measured fourth signal quality corresponding to the actual optimal beam (i.e., the second beam), which can be considered as absolute difference information.

In some embodiments, the second difference information is: Measured L1-RSRP _i −Measured L1-RSRP _q .

Step S8107: The testing device sends second difference information to the terminal.

Step S8108: The terminal determines the prediction performance of the first model based on the second difference information.

Step S8103 may be omitted in the method according to the embodiment of the present disclosure.

FIG9 is an interactive schematic diagram of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure. As shown in FIG9 , the method includes the following steps:

Step S9101: The terminal predicts a first beam according to a first model.

The optional implementation of step S9101 can refer to the optional implementation of step S5101 in Figure 5 and other related parts in the embodiment involved in Figure 5, which will not be repeated here.

Step S9102: The terminal predicts a third signal quality according to the first model.

For the optional implementation of step S9102, reference may be made to the optional implementation of step S5102 in FIG5 and other related parts in the embodiment involved in FIG5, which will not be described in detail here.

Step S9103: The terminal measures the signal quality of the first reference signal corresponding to each beam in the first set to obtain a measurement result.

The optional implementation of step S9103 can refer to the optional implementation of step S5103 in FIG5 and other related parts in the embodiment involved in FIG5 , which will not be described in detail here.

Step S9104: The terminal determines a fourth signal quality according to the measurement result.

In some embodiments, the fourth signal quality is the signal quality with the best signal quality in the measurement results. The index of the second beam corresponding to the fourth signal quality is k.

In an example, the third signal quality is recorded as Predicted L1-RSRPi, and the fourth signal quality is recorded as Measured L1-RSRP _k .

In some embodiments, the fourth signal quality is the signal quality corresponding to the second beam in the measurement result. The second beam is the best beam determined from the first set based on the actual signal quality. For example, the index of the second beam is q, and the index of the second beam can be called the best genie-aided beam index.

In one example, the third signal quality is recorded as Predicted L1-RSRPi, and the fourth signal quality is recorded as Measured L1-RSRPq.

Step S9105: The terminal sends the third signal quality and the fourth signal quality to the test device.

In some embodiments, the terminal sends the third signal quality, the fourth signal quality, and identification information of the first beam to the test device.

Step S9106: The testing device determines the second difference information.

In some embodiments, the test device determines the second difference information as the difference between the received third signal quality and the received fourth signal quality. The second difference information is the difference between the predicted third signal quality and the measured optimal signal quality, i.e., the fourth signal quality, and can be considered as relative difference information.

In some embodiments, the second difference information is: Predicted L1-RSRP _i −Measured L1-RSRP _q .

Step S9107: The testing device sends second difference information to the terminal.

Step S9108: The terminal determines the prediction performance of the first model based on the second difference information.

FIG10 is an interactive schematic diagram of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure. As shown in FIG10 , the method includes the following steps:

Step S10101: The terminal predicts a first beam according to a first model.

The optional implementation of step S10101 can refer to the optional implementation of step S5101 in FIG5 and other related parts in the embodiment involved in FIG5 , which will not be described in detail here.

Step S10102: The terminal measures the signal quality of the first reference signal corresponding to each beam in the first set to obtain a measurement result.

The optional implementation of step S10103 can refer to the optional implementation of step S5103 in FIG5 , and other related parts in the embodiment involved in FIG5 , which will not be described in detail here.

Step S10103: The terminal determines the third signal quality and the fourth signal quality according to the measurement result.

In some embodiments, the third signal quality is the measurement quality corresponding to the first beam in the measurement result, and the fourth signal quality is the signal quality with the best signal quality in the measurement result. The index of the second beam corresponding to the fourth signal quality is k.

In an example, the third signal quality is recorded as Measured L1-RSRP _i , and the fourth signal quality is recorded as Measured L1-RSRP _k .

In some embodiments, the fourth signal quality is the signal quality corresponding to the second beam in the measurement result. The second beam is the best beam determined from the first set based on the actual signal quality. For example, the index of the second beam is q, and the index of the second beam can be called the best genie-aided beam index.

In one example, the third signal quality is recorded as Measured L1-RSRPi, and the fourth signal quality is recorded as Measured L1-RSRPq.

Step S10104: The terminal sends the third signal quality and the fourth signal quality to the test device.

In some embodiments, the terminal sends the third signal quality, the fourth signal quality, and identification information of the first beam to the test device.

Step S10105: the testing device determines the second difference information.

In some embodiments, the testing device determines the second difference information as the difference between the received third signal quality and the received fourth signal quality. The second difference information is the difference between the measured third signal quality and the measured optimal signal quality, i.e., the fourth signal quality, and can be considered as a relative difference information.

In some embodiments, the second difference information is: Measured L1-RSRP _i −Measured L1-RSRP _q .

Step S10106: The testing device sends second difference information to the terminal.

Step S10107: The terminal determines the prediction performance of the first model according to the second difference information.

FIG11 is a flow chart of a method for evaluating the performance of an artificial intelligence prediction model according to an embodiment of the present disclosure, which is applied to a terminal, as shown in FIG11 , and includes the following steps:

Step S11101: predicting a first signal quality using a first model.

The first signal quality is a predicted value of a first reference signal transmitted through the first beam.

In some embodiments, predicting the first signal quality by using the first model includes:

Measuring, in a first time domain unit, a signal quality of the first reference signal transmitted through the second beam to obtain a first measurement value of the first reference signal;

Taking a first measurement value of the first reference signal as input, predicting the first signal quality in a second time domain unit by using the first model;

The time domain unit is one of: time slot, half time slot, symbol.

In some embodiments, the historical quality is a measured signal quality of a first reference signal transmitted via at least one beam other than the first beam.

Step S11102, determine the first difference information.

The first difference information is used to indicate the difference between the first signal quality and the second signal quality, and the second signal quality is a reference value of the signal quality of the first reference signal.

In some embodiments, the reference quality is a measured signal quality; the method further comprises: measuring a signal quality of the first reference signal transmitted via the first beam in the second time domain unit to obtain the second signal quality.

In some embodiments, the reference quality is a real signal quality; the method further comprises: determining the real quality of the first reference signal; and obtaining a second signal quality.

In some embodiments, the method further comprises: sending the first signal quality and the second signal quality to the test device;

Determining first difference information includes:

First difference information sent by the test device is received, where the first difference information is determined by the test device according to the first signal quality and the second signal quality.

In some embodiments, the method further comprises:

sending the first signal quality to a test device;

Determining first difference information includes:

First difference information sent by a test device is received, where the first difference information is determined by the test device according to the first signal quality and the second signal quality, and the second signal quality is determined by the test device.

Step S11103: determine the prediction performance of the first model according to the first difference information.

FIG12 is a flow chart of a method for evaluating the performance of an artificial intelligence prediction model according to an embodiment of the present disclosure, which is applied to a terminal, as shown in FIG12 , and includes the following steps:

Step S12101: predict a first beam using a first model.

The first beam is a beam with the best signal quality in the first set predicted by the terminal.

Step S12102: determine the third signal quality.

The third signal quality is the signal quality of the first reference signal transmitted through the first beam.

In some embodiments, determining the third signal quality includes: predicting the third signal quality by using a first model.

In some embodiments, determining the third signal quality includes: measuring a signal quality of the first reference signal corresponding to each beam in the first set to obtain a measurement result;

According to the identification information of the first beam, it is determined that the third signal quality is a measurement result in the measurement results corresponding to the identification information of the first beam.

Step S12102, determine the second difference information.

In some embodiments, the second difference information is used to indicate a difference between a third signal quality and a fourth signal quality, where the fourth signal quality is a signal quality of the first reference signal transmitted via the second beam.

In some embodiments, the second beam is the beam with the best signal quality in the first set measured by the terminal, or the second beam is the best beam determined from the first set based on actual signal quality, or the second beam is a default beam.

In some embodiments, before step S12102, the process further includes:

sending the third signal quality to a test device;

The obtaining of the second difference information includes:

The second difference information sent by the test device is received, wherein the fourth signal quality is a true value of the signal quality of the first reference signal transmitted through the second beam determined by the test device.

In some embodiments, before step S12102, the process further includes:

Predicting the third signal quality by using the first model;

sending the third signal quality and the measurement result to a test device;

The obtaining of the second difference information includes:

The second difference information sent by the test device is received, wherein the fourth signal quality is the signal quality corresponding to the second beam in the measurement result.

In some embodiments, before step S12102, the process further includes:

Sending identification information of the first beam and the measurement result to a test device;

The obtaining of the second difference information includes:

The second difference information sent by the test device is received, wherein the first signal quality is the signal quality corresponding to the first beam in the measurement result, and the fourth signal quality is the signal quality corresponding to the second beam in the measurement result.

In some embodiments, before step S12102, the process further includes:

Predicting the third signal quality by using the first model;

Determine the fourth signal quality according to the measurement result, where the fourth signal quality is the signal quality with the best signal quality in the measurement result;

sending the third signal quality and the fourth signal quality to a test device;

The obtaining of the second difference information includes:

The second difference information sent by the test device is received.

In some embodiments, before step S12102, the process further includes:

determining a third signal quality and a fourth signal quality according to the measurement result, the third signal quality being a measurement result corresponding to the identification information of the first beam in the measurement result, and the fourth signal quality being a signal quality with the best signal quality in the measurement result;

sending the third signal quality and the fourth signal quality to a test device;

The obtaining of the second difference information includes:

The second difference information sent by the test device is received.

Step S12103: determine the prediction performance of the first model based on the second difference information.

The second difference information sent by the test device is received.

The embodiments of the present disclosure also propose a device for implementing any of the above methods, for example, a device is proposed, the above device includes a unit or module for implementing each step performed by the terminal in any of the above methods. For another example, another device is also proposed, including a unit or module for implementing each step performed by a network device (such as an access network device, a core network function node, a core network device, etc.) in any of the above methods.

It should be understood that the division of the units or modules in the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. In addition, the units or modules in the device can be called by the processor. : For example, the device includes a processor, the processor is connected to a memory, the memory stores instructions, and the processor calls the instructions stored in the memory to implement any of the above methods or realize the functions of each unit or module of the above device, wherein the processor is, for example, a general-purpose processor, such as a central processing unit (CPU) or a microprocessor, and the memory is a memory in the device or a memory outside the device. Alternatively, the unit or module in the device can be implemented in the form of a hardware circuit, and the functions of some or all of the units or modules can be realized by designing the hardware circuit. The above hardware circuit can be understood as one or more processors; for example, in one implementation, the above hardware circuit is an application-specific integrated circuit (ASIC), and the functions of some or all of the above units or modules are realized by designing the logical relationship of the components in the circuit; for another example, in another implementation, the above hardware circuit can be realized by a programmable logic device (PLD), taking a field programmable gate array (FPGA) as an example, which can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured through a configuration file, so as to realize the functions of some or all of the above units or modules. All units or modules of the above devices may be implemented entirely in the form of a processor calling software, or entirely in the form of a hardware circuit, or partially in the form of a processor calling software and the rest in the form of a hardware circuit.

In the disclosed embodiments, the processor is a circuit with signal processing capability. In one implementation, the processor may be a circuit with instruction reading and running capability, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which may be understood as a microprocessor), or a digital signal processor (DSP); in another implementation, the processor may implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the above hardware circuit may be fixed or reconfigurable, such as a hardware circuit implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the hardware circuit configuration may be understood as the process of the processor loading instructions to implement the functions of some or all of the above units or modules. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as ASIC, such as Neural Network Processing Unit (NPU), Tensor Processing Unit (TPU), Deep Learning Processing Unit (DPU), etc.

FIG. 13 is a schematic diagram of the structure of a communication device according to an embodiment of the present disclosure, which is applied to a terminal. As shown in FIG. 13 , the communication device 13 may include: a processing module 1301 and a transceiver module 1302 .

In some embodiments, the processing module 1301 is configured to:

Predicting a first signal quality by using a first model, where the first signal quality is a predicted value of a first reference signal transmitted by a first beam;

Acquire first difference information, where the first difference information is used to indicate a difference between the first signal quality and a second signal quality, where the second signal quality is a reference value of a signal quality of the first reference signal;

The prediction performance of the first model is determined according to the first difference information.

In some embodiments, the processing module 1301 is further configured to: measure, in a first time domain unit, a signal quality of the first reference signal transmitted through the second beam to obtain a first measurement value of the first reference signal;

Taking a first measurement value of the first reference signal as input, predicting the first signal quality in a second time domain unit by using the first model;

The time domain unit is one of: time slot, half time slot, symbol.

In some embodiments, the processing module 1301 is further configured to: measure the signal quality of the first reference signal transmitted through the first beam in the second time domain unit to obtain the second signal quality. The second signal quality is the true value of the first reference signal transmitted through the first beam in the second time domain unit.

In some embodiments, the processing module 1301 is further configured to: obtain the second signal quality;

The transceiver module 1302 is configured to: send the first signal quality and the second signal quality to the test device;

First difference information sent by the test device is received, where the first difference information is determined by the test device according to the first signal quality and the second signal quality.

In some embodiments, the transceiver module 1302 is further configured to: send the first signal quality to a test device; receive the first difference information sent by the test device, wherein the first difference information is determined by the test device based on the first signal quality and the second signal quality, and the second signal quality is determined by the test device.

In some embodiments, the processing module 1301 is configured to:

Predicting a first beam by using a first model, wherein the first beam is a beam with the best signal quality in the first set predicted by the terminal;

determining a third signal quality, where the third signal quality is a signal quality of a first reference signal transmitted through the first beam;

obtaining second difference information, where the second difference information is used to indicate a difference between the third signal quality and the fourth signal quality, where the fourth The signal quality is a reference value of the signal quality of the first reference signal transmitted through the second beam, wherein the second beam is a beam with the best signal quality in the first set measured by the terminal, or the second beam is a best beam determined from the first set based on actual signal quality, or the second beam is a default beam;

The prediction performance of the first model is determined according to the second difference information.

In some embodiments, the processing module 1301 is further configured to: predict the third signal quality by using the first model.

In some embodiments, the processing module 1301 is further configured to: measure a signal quality of the first reference signal corresponding to each beam in the first set to obtain a measurement result;

According to the identification information of the first beam, it is determined that the third signal quality is a measurement result in the measurement results corresponding to the identification information of the first beam.

In some embodiments, the transceiver module 1302 is further configured to: send the third signal quality to the test device;

The second difference information sent by the test device is received, wherein the fourth signal quality is a true value of the signal quality of the first reference signal transmitted through the second beam determined by the test device.

In some embodiments, the processing module 1301 is further configured to: predict the third signal quality by using the first model;

In some embodiments, the transceiver module 1302 is further configured to: send the third signal quality and the measurement result to the test device;

The second difference information sent by the test device is received, wherein the fourth signal quality is the signal quality corresponding to the second beam in the measurement result.

In some embodiments, the transceiver module 1302 is further configured to: send identification information of the first beam and the measurement result to a test device;

The second difference information sent by the test device is received, wherein the first signal quality is the signal quality corresponding to the first beam in the measurement result, and the fourth signal quality is the signal quality corresponding to the second beam in the measurement result.

In some embodiments, the processing module 1301 is further configured to: predict the third signal quality by using the first model;

Determine the fourth signal quality according to the measurement result, where the fourth signal quality is the signal quality corresponding to the second beam in the measurement result;

The transceiver module 1302 is further configured to: send the third signal quality and the fourth signal quality to the test device; and receive the second difference information sent by the test device.

In some embodiments, the processing module 1301 is further configured to: determine a third signal quality and a fourth signal quality according to the measurement result, wherein the third signal quality is a measurement result corresponding to the identification information of the first beam in the measurement result, and the fourth signal quality is a signal quality corresponding to the second beam in the measurement result;

The transceiver module 1302 is further configured to: send the third signal quality and the fourth signal quality to the test device; and receive the second difference information sent by the test device.

FIG8A is a schematic diagram of the structure of a communication device 8100 according to an embodiment of the present disclosure. The communication device 8100 may be a network device (e.g., an access network device, a core network device, etc.), or a terminal (e.g., a user device, etc.), or a chip, a chip system, or a processor, etc. having a network device to implement any of the above methods, or a chip, a chip system, or a processor, etc. having a terminal to implement any of the above methods. The communication device 8100 may be used to implement the method described in the above method embodiment, and the details may refer to the description in the above method embodiment.

As shown in FIG. 14 , the communication device 8100 includes one or more processors 8101. The processor 8101 may be a general-purpose processor or a dedicated processor, for example, a baseband processor or a central processing unit. The baseband processor may be used to process the communication protocol and the communication data, and the central processing unit may be used to control the communication device (such as a base station, a baseband chip, a terminal device, a terminal device chip, a DU or a CU, etc.), execute a program, and process the data of the program. The communication device 8100 is used to execute any of the above methods.

In some embodiments, the communication device 8100 further includes one or more memories 8102 for storing instructions. Optionally, all or part of the memory 8102 may also be outside the communication device 8100.

In some embodiments, the communication device 8100 further includes one or more transceivers 8103. When the communication device 8100 includes one or more transceivers 8103, the transceiver 8103 performs at least one of the communication steps such as sending and/or receiving in the above method (for example, step S2102, step S2103, but not limited thereto), and the processor 8101 performs at least one of the other steps (for example, step S2101, but not limited thereto).

In some embodiments, the transceiver may include a receiver and/or a transmitter, and the receiver and the transmitter may be separate or integrated. Optionally, the terms such as transceiver, transceiver unit, transceiver, transceiver circuit, etc. may be replaced with each other, the terms such as transmitter, transmission unit, transmitter, transmission circuit, etc. may be replaced with each other, and the terms such as receiver, receiving unit, receiver, receiving circuit, etc. may be replaced with each other.

In some embodiments, the communication device 8100 may include one or more interface circuits 8104. Optionally, the interface circuit 8104 is connected to the memory 8102, and the interface circuit 8104 may be used to receive signals from the memory 8102 or other devices, and may be used to send signals to the memory 8102. For example, the interface circuit 8104 can read the instruction stored in the memory 8102 and send the instruction to the processor 8101.

The communication device 8100 described in the above embodiments may be a network device or a terminal, but the scope of the communication device 8100 described in the present disclosure is not limited thereto, and the structure of the communication device 8100 may not be limited by FIG. 14. The communication device may be an independent device or may be part of a larger device. For example, the communication device may be: 1) an independent integrated circuit IC, or a chip, or a chip system or subsystem; (2) a collection of one or more ICs, optionally, the above IC collection may also include a storage component for storing data and programs; (3) an ASIC, such as a modem; (4) a module that can be embedded in other devices; (5) a receiver, a terminal device, an intelligent terminal device, a cellular phone, a wireless device, a handheld device, a mobile unit, a vehicle-mounted device, a network device, a cloud device, an artificial intelligence device, etc.; (8) others, etc.

Industrial Applicability

The performance of the first model can be evaluated using different beams in different application scenarios or at different time points to obtain more accurate evaluation results.

Claims (18)

A performance evaluation method for an artificial intelligence prediction model, applied to a terminal, comprising: Predicting a first signal quality by using a first model, where the first signal quality is a predicted value of a first reference signal transmitted by a first beam; Acquire first difference information, where the first difference information is used to indicate a difference between the first signal quality and a second signal quality, where the second signal quality is a reference value of a signal quality of the first reference signal; The prediction performance of the first model is determined according to the first difference information. The method of claim 1, wherein predicting the first signal quality by using the first model comprises: Measuring, in a first time domain unit, a signal quality of the first reference signal transmitted through the second beam to obtain a first measurement value of the first reference signal; Taking a first measurement value of the first reference signal as input, predicting the first signal quality in a second time domain unit by using the first model; The time domain unit is one of: time slot, half time slot, symbol. The method according to claim 2, wherein the method further comprises: The signal quality of the first reference signal transmitted through the first beam in the second time domain unit is measured to obtain the second signal quality. The method of claim 2, wherein the second signal quality is a true value of the first reference signal transmitted through the first beam in the second time domain unit. The method according to any one of claims 1 to 4, wherein the method further comprises: obtaining the second signal quality; sending the first signal quality and the second signal quality to a test device; The obtaining of the first difference information includes: The first difference information sent by the test device is received, where the first difference information is determined by the test device according to the first signal quality and the second signal quality. The method according to any one of claims 1 to 4, wherein the method further comprises: sending the first signal quality to a test device; The obtaining of the first difference information includes: The first difference information sent by the test device is received, where the first difference information is determined by the test device according to the first signal quality and the second signal quality, and the second signal quality is determined by the test device. A performance evaluation method for an artificial intelligence prediction model, applied to a terminal, comprising: Predicting a first beam by using a first model, wherein the first beam is a beam with the best signal quality in the first set predicted by the terminal; determining a third signal quality, where the third signal quality is a signal quality of a first reference signal transmitted through the first beam; Acquire second difference information, where the second difference information is used to indicate a difference between the third signal quality and the fourth signal quality, where the fourth signal quality is a reference value of the signal quality of the first reference signal transmitted through the second beam, wherein the second beam is a beam with the best signal quality in the first set measured by the terminal, or the second beam is a best beam determined from the first set based on actual signal quality, or the second beam is a default beam; The prediction performance of the first model is determined according to the second difference information. The method of claim 7, wherein determining the third signal quality comprises: The third signal quality is predicted by using the first model. The method of claim 7, wherein the determining the third signal quality further comprises: Measuring a signal quality of the first reference signal corresponding to each beam in the first set to obtain a measurement result; According to the identification information of the first beam, it is determined that the third signal quality is a measurement result in the measurement results corresponding to the identification information of the first beam. The method according to claim 8, wherein the method further comprises: sending the third signal quality to a test device; The obtaining of the second difference information includes: The second difference information sent by the test device is received, wherein the fourth signal quality is a true value of the signal quality of the first reference signal transmitted through the second beam determined by the test device. The method according to claim 9, wherein the method further comprises: Predicting the third signal quality by using the first model; sending the third signal quality and the measurement result to a test device; The obtaining of the second difference information includes: The second difference information sent by the test device is received, wherein the fourth signal quality is the signal quality corresponding to the second beam in the measurement result. The method according to claim 9, wherein the method further comprises: Sending identification information of the first beam and the measurement result to a test device; The obtaining of the second difference information includes: The second difference information sent by the test device is received, wherein the first signal quality is the signal quality corresponding to the first beam in the measurement result, and the fourth signal quality is the signal quality corresponding to the second beam in the measurement result. The method according to claim 9, wherein the method further comprises: Predicting the third signal quality by using the first model; Determine the fourth signal quality according to the measurement result, where the fourth signal quality is the signal quality corresponding to the second beam in the measurement result; sending the third signal quality and the fourth signal quality to a test device; The obtaining of the second difference information includes: The second difference information sent by the test device is received. The method according to claim 9, wherein the method further comprises: determining a third signal quality and a fourth signal quality according to the measurement result, wherein the third signal quality is a measurement result corresponding to the identification information of the first beam in the measurement result, and the fourth signal quality is a signal quality corresponding to the second beam in the measurement result; sending the third signal quality and the fourth signal quality to a test device; The obtaining of the second difference information includes: The second difference information sent by the test device is received. A terminal, comprising: The processing module is configured as follows: Predicting a first signal quality by using a first model, where the first signal quality is a predicted value of a first reference signal transmitted by a first beam; Acquire first difference information, where the first difference information is used to indicate a difference between the first signal quality and a second signal quality, where the second signal quality is a reference value of a signal quality of the first reference signal; The prediction performance of the first model is determined according to the first difference information. A terminal, comprising: The processing module is configured as follows: Predicting a first beam by using a first model, wherein the first beam is a beam with the best signal quality in the first set predicted by the terminal; determining a third signal quality, where the third signal quality is a signal quality of a first reference signal transmitted through the first beam; Acquire second difference information, where the second difference information is used to indicate a difference between the third signal quality and the fourth signal quality, where the fourth signal quality is a reference value of the signal quality of the first reference signal transmitted through the second beam, wherein the second beam is a beam with the best signal quality in the first set measured by the terminal, or the second beam is a best beam determined from the first set based on actual signal quality, or the second beam is a default beam; The prediction performance of the first model is determined according to the second difference information. A terminal, comprising: one or more processors; The terminal is used to execute the method described in any one of claims 1 to 6 or 7 to 14. A storage medium storing instructions, which, when executed on a communication device, causes the communication device to execute the method as described in any one of claims 1 to 6 or 7 to 14.