WO2025000209A1

WO2025000209A1 - Model performance monitoring method and apparatus

Info

Publication number: WO2025000209A1
Application number: PCT/CN2023/102531
Authority: WO
Inventors: 刘正宣; 刘敏
Original assignee: Beijing Xiaomi Mobile Software Co Ltd
Current assignee: Beijing Xiaomi Mobile Software Co Ltd
Priority date: 2023-06-26
Filing date: 2023-06-26
Publication date: 2025-01-02
Anticipated expiration: 2025-12-26
Also published as: CN117063510A

Abstract

The embodiments of the present disclosure can be applied to the technical field of communications. Disclosed are a model performance monitoring method and apparatus. The method executed by a terminal device comprises: monitoring a model, and acquiring channel status information (CSI) currently associated with the model, wherein the CSI comprises any one of the following: first information, the current first input of the model and the first information of the model, the first information is the current first output of the model or associated information of the first output, and the model is a CSI feedback model; according to the CSI currently associated with the model, determining current performance information of the model; and sending first indication information, wherein the first indication information includes the current performance information of the model. Thus, the terminal device can accurately determine the current performance information of the model by means of the CSI, thereby improving the accuracy and reliability of model performance monitoring.

Description

Model performance monitoring method and device

Technical Field

本公开涉及通信技术领域，尤其涉及一种模型性能的监测方法和装置。The present disclosure relates to the field of communication technology, and in particular to a method and device for monitoring model performance.

Background Art

相关技术中，终端设备可以通过信道状态信息(Channel Status Information，CSI)反馈模型将CSI反馈给网络设备。即终端设备通过CSI生成部分模型把信道状态信息压缩后通过量化为二进制比特流发送给网络设备，网络设备通过CSI恢复部分模型恢复出信道状态信息。但是，若CSI反馈模型的性能较差，可能会导致网络设备恢复的信号状态信息准确性较低。In the related art, the terminal device can feed back the channel status information (CSI) to the network device through the CSI feedback model. That is, the terminal device compresses the channel status information through the CSI generation part model and sends it to the network device by quantizing it into a binary bit stream, and the network device recovers the channel status information through the CSI recovery part model. However, if the performance of the CSI feedback model is poor, the accuracy of the signal status information recovered by the network device may be low.

发明内容Summary of the invention

本公开实施例提供一种模型性能的监测方法和装置。在本公开实施例中，终端设备通过对模型进行监测，获取所述模型当前关联的信道状态信息CSI，进而根据所述模型当前关联的CSI，确定所述模型当前的性能信息，最后向网络设备发送包含模型当前的性能信息的第一指示信息。由此，终端设备可以通过CSI，准确地确定模型当前的性能信息，提高了对模型性能监测的准确性及可靠性。The disclosed embodiment provides a method and device for monitoring model performance. In the disclosed embodiment, the terminal device monitors the model to obtain the channel state information CSI currently associated with the model, and then determines the current performance information of the model according to the CSI currently associated with the model, and finally sends the first indication information containing the current performance information of the model to the network device. As a result, the terminal device can accurately determine the current performance information of the model through the CSI, thereby improving the accuracy and reliability of the model performance monitoring.

第一方面，本公开实施例提供一种模型性能的监测方法，该方法由终端设备执行，该方法包括：对模型进行监测，获取所述模型当前关联的信道状态信息CSI，其中，所述CSI包括以下任一项：第一信息，所述模型当前的第一输入及所述第一信息；其中，所述第一信息为所述模型当前的第一输出或所述第一输出的关联信息，所述模型为CSI反馈模型；根据所述模型当前关联的CSI，确定所述模型当前的性能信息；发送第一指示信息，其中，所述第一指示信息中包含所述模型当前的性能信息。In a first aspect, an embodiment of the present disclosure provides a method for monitoring model performance, which is executed by a terminal device, and the method includes: monitoring the model, obtaining channel state information CSI currently associated with the model, wherein the CSI includes any one of the following items: first information, the current first input of the model and the first information; wherein the first information is the current first output of the model or associated information of the first output, and the model is a CSI feedback model; determining the current performance information of the model based on the CSI currently associated with the model; and sending first indication information, wherein the first indication information includes the current performance information of the model.

第二方面，本公开实施例还提供另一种模型性能的监测方法，该方法由网络设备执行，该方法包括：接收第一指示信息，其中，所述第一指示信息中包含信道状态信息CSI反馈模型当前的性能信息。In a second aspect, an embodiment of the present disclosure also provides another method for monitoring model performance, which is executed by a network device and includes: receiving first indication information, wherein the first indication information includes current performance information of a channel state information CSI feedback model.

第三方面，本公开实施例提供一种终端设备，包括：In a third aspect, an embodiment of the present disclosure provides a terminal device, including:

处理模块，用于对模型进行监测，获取所述模型当前关联的信道状态信息CSI，其中，所述CSI包括以下任一项：第一信息，所述模型当前的第一输入及所述第一信息；其中，所述第一信息为所述模型当前的第一输出或所述第一输出的关联信息，所述模型为CSI反馈模型；A processing module, configured to monitor the model and obtain channel state information CSI currently associated with the model, wherein the CSI includes any one of the following: first information, a current first input of the model and the first information; wherein the first information is a current first output of the model or associated information of the first output, and the model is a CSI feedback model;

所述处理模块，还用于根据所述模型当前关联的CSI，确定所述模型当前的性能信息；The processing module is further configured to determine current performance information of the model according to the CSI currently associated with the model;

收发模块，还用于发送第一指示信息，其中，所述第一指示信息中包含所述模型当前的性能信息。The transceiver module is also used to send first indication information, wherein the first indication information includes current performance information of the model.

第四方面，本公开实施例还提供另一种网络设备，包括：In a fourth aspect, the embodiment of the present disclosure also provides another network device, including:

收发模块，用于接收第一指示信息，其中，所述第一指示信息中包含信道状态信息CSI反馈模型当前的性能信息。The transceiver module is used to receive first indication information, wherein the first indication information includes current performance information of a channel state information CSI feedback model.

第五方面，本公开实施例提供一种通信装置，该通信装置包括处理器，当该处理器调用存储器中的计算机程序时，执行上述第一方面所述的方法。In a fifth aspect, an embodiment of the present disclosure provides a communication device, which includes a processor. When the processor calls a computer program in a memory, the method described in the first aspect is executed.

第六方面，本公开实施例提供一种通信装置，该通信装置包括处理器，当该处理器调用存储器中的计算机程序时，执行上述第二方面所述的方法。In a sixth aspect, an embodiment of the present disclosure provides a communication device, which includes a processor. When the processor calls a computer program in a memory, the method described in the second aspect is executed.

第七方面，本公开实施例提供一种通信装置，该通信装置包括处理器和存储器，该存储器中存储有计算机程序；所述处理器执行该存储器所存储的计算机程序，以使该通信装置执行上述第一方面所述的方法。In a seventh aspect, an embodiment of the present disclosure provides a communication device, which includes a processor and a memory, in which a computer program is stored; the processor executes the computer program stored in the memory so that the communication device executes the method described in the first aspect above.

第八方面，本公开实施例提供一种通信装置，该通信装置包括处理器和存储器，该存储器中存储有计算机程序；所述处理器执行该存储器所存储的计算机程序，以使该通信装置执行上述第二方面所述的方法。In an eighth aspect, an embodiment of the present disclosure provides a communication device, which includes a processor and a memory, in which a computer program is stored; the processor executes the computer program stored in the memory so that the communication device executes the method described in the second aspect above.

第九方面，本公开实施例提供一种通信装置，该装置包括处理器和接口电路，该接口电路用于接收代码指令并传输至该处理器，该处理器用于运行所述代码指令以使该装置执行上述第一方面所述的方法。In a ninth aspect, an embodiment of the present disclosure provides a communication device, which includes a processor and an interface circuit, wherein the interface circuit is used to receive code instructions and transmit them to the processor, and the processor is used to run the code instructions to enable the device to execute the method described in the first aspect above.

第十方面，本公开实施例提供一种通信装置，该装置包括处理器和接口电路，该接口电路用于接收代码指令并传输至该处理器，该处理器用于运行所述代码指令以使该装置执行上述第二方面所述的方法。In a tenth aspect, an embodiment of the present disclosure provides a communication device, which includes a processor and an interface circuit, wherein the interface circuit is used to receive code instructions and transmit them to the processor, and the processor is used to run the code instructions to enable the device to execute the method described in the second aspect above.

第十一方面，本公开实施例提供一种通信系统，该系统包括第三方面所述的终端设备、第四方面所述的网络设备，或者，该系统包括第五方面至第六方面所述的通信装置，或者，该系统包括第七方面至第八方面所述的通信装置，或者，该系统包括第九方面至第十方面所述的通信装置。In the eleventh aspect, an embodiment of the present disclosure provides a communication system, which includes the terminal device described in the third aspect, the network device described in the fourth aspect, or the system includes the communication device described in the fifth to sixth aspects, or the system includes the communication device described in the seventh to eighth aspects, or the system includes the communication device described in the ninth to tenth aspects.

第十二方面，本发明实施例提供一种计算机可读存储介质，用于储存为上述终端设备所用的指令，当所述指令被执行时，使所述终端设备执行上述第一方面所述的方法。In a twelfth aspect, an embodiment of the present invention provides a computer-readable storage medium for storing instructions used by the above-mentioned terminal device, When the instruction is executed, the terminal device executes the method described in the first aspect above.

第十三方面，本发明实施例提供一种计算机可读存储介质，用于储存为上述网络设备所用的指令，当所述指令被执行时，使所述网络设备执行上述第二方面所述的方法。In a thirteenth aspect, an embodiment of the present invention provides a computer-readable storage medium for storing instructions used by the above-mentioned network device, and when the instructions are executed, the network device executes the method described in the above-mentioned second aspect.

第十四方面，本公开还提供一种包括计算机程序的计算机程序产品，当其在计算机上运行时，使得计算机执行上述第一方面所述的方法。In a fourteenth aspect, the present disclosure further provides a computer program product comprising a computer program, which, when executed on a computer, enables the computer to execute the method described in the first aspect above.

第十五方面，本公开还提供一种包括计算机程序的计算机程序产品，当其在计算机上运行时，使得计算机执行上述第二方面所述的方法。In a fifteenth aspect, the present disclosure further provides a computer program product comprising a computer program, which, when executed on a computer, enables the computer to execute the method described in the second aspect above.

第十六方面，本公开提供一种芯片系统，该芯片系统包括至少一个处理器和接口，用于支持终端设备实现第一方面所涉及的功能，例如，确定或处理上述方法中所涉及的数据和信息中的至少一种。在一种可能的设计中，所述芯片系统还包括存储器，所述存储器，用于保存终端设备必要的计算机程序和数据。该芯片系统，可以由芯片构成，也可以包括芯片和其他分立器件。In a sixteenth aspect, the present disclosure provides a chip system, which includes at least one processor and an interface, for supporting a terminal device to implement the functions involved in the first aspect, for example, determining or processing at least one of the data and information involved in the above method. In a possible design, the chip system also includes a memory, which is used to store computer programs and data necessary for the terminal device. The chip system can be composed of a chip, or it can include a chip and other discrete devices.

第十七方面，本公开提供一种芯片系统，该芯片系统包括至少一个处理器和接口，用于支持网络设备实现第二方面所涉及的功能，例如，确定或处理上述方法中所涉及的数据和信息中的至少一种。在一种可能的设计中，所述芯片系统还包括存储器，所述存储器，用于保存网络设备必要的计算机程序和数据。该芯片系统，可以由芯片构成，也可以包括芯片和其他分立器件。In a seventeenth aspect, the present disclosure provides a chip system, which includes at least one processor and an interface, and is used to support a network device to implement the functions involved in the second aspect, for example, determining or processing at least one of the data and information involved in the above method. In a possible design, the chip system also includes a memory, and the memory is used to store computer programs and data necessary for the network device. The chip system can be composed of a chip, or it can include a chip and other discrete devices.

第十八方面，本公开提供一种计算机程序，当其在计算机上运行时，使得计算机执行上述第一方面所述的方法。In an eighteenth aspect, the present disclosure provides a computer program, which, when executed on a computer, enables the computer to execute the method described in the first aspect.

第十九方面，本公开提供一种计算机程序，当其在计算机上运行时，使得计算机执行上述第二方面所述的方法。In a nineteenth aspect, the present disclosure provides a computer program which, when executed on a computer, enables the computer to execute the method described in the second aspect.

BRIEF DESCRIPTION OF THE DRAWINGS

为了更清楚地说明本公开实施例或背景技术中的技术方案，下面将对本公开实施例或背景技术中所需要使用的附图进行说明。In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the background technology, the drawings required for use in the embodiments of the present disclosure or the background technology will be described below.

图1是本公开提供的一种通信系统的架构示意图；FIG1 is a schematic diagram of the architecture of a communication system provided by the present disclosure;

图2是本公开实施例提供的一种模型性能的监测方法的流程图；FIG2 is a flow chart of a method for monitoring model performance provided by an embodiment of the present disclosure;

图3是本公开实施例提供的另一种模型性能的监测方法的流程图；FIG3 is a flow chart of another method for monitoring model performance provided by an embodiment of the present disclosure;

图4是本公开实施例提供的另一种模型性能的监测方法的流程图；FIG4 is a flow chart of another method for monitoring model performance provided by an embodiment of the present disclosure;

图5是本公开实施例提供的另一种模型性能的监测方法的流程图；FIG5 is a flow chart of another method for monitoring model performance provided by an embodiment of the present disclosure;

图5A为本公开一实施例提供的一种CSI测量和上报时序图；FIG5A is a CSI measurement and reporting timing diagram provided by an embodiment of the present disclosure;

图6是本公开实施例提供的又一种模型性能的监测方法的流程图；FIG6 is a flow chart of another method for monitoring model performance provided by an embodiment of the present disclosure;

图7是本公开实施例提供的一种模型性能的监测方法的流程图；FIG7 is a flow chart of a method for monitoring model performance provided by an embodiment of the present disclosure;

图8是本公开实施例提供的另一种模型性能的监测方法的流程图；FIG8 is a flow chart of another method for monitoring model performance provided by an embodiment of the present disclosure;

图9是本公开实施例提供的又一种模型性能的监测方法的流程图；FIG9 is a flow chart of another method for monitoring model performance provided by an embodiment of the present disclosure;

图10是本公开实施例提供的又一种模型性能的监测方法的流程图；FIG10 is a flow chart of another method for monitoring model performance provided by an embodiment of the present disclosure;

图11是本公开实施例提供的一种模型性能的监测方法的信令交互图；FIG11 is a signaling interaction diagram of a method for monitoring model performance provided by an embodiment of the present disclosure;

图12是本公开实施例提供的一种通信装置的结构图；FIG12 is a structural diagram of a communication device provided in an embodiment of the present disclosure;

图13是本公开实施例提供的另一种通信装置的结构图；FIG13 is a structural diagram of another communication device provided in an embodiment of the present disclosure;

图14是本公开实施例提供的一种芯片的结构示意图。FIG. 14 is a schematic diagram of the structure of a chip provided in an embodiment of the present disclosure.

DETAILED DESCRIPTION

为了便于理解本公开的技术方案，下面简单介绍本公开实施例涉及的一些术语。In order to facilitate understanding of the technical solution of the present disclosure, some terms involved in the embodiments of the present disclosure are briefly introduced below.

1、参考信号(reference signal，RS)1. Reference signal (RS)

参考信号就是“导频”信号，是由发射端提供给接收端用于信道估计或信道探测的一种已知信号。可以用于终端设备的相干检测和解调、波束测量或网络设备的相干检测和监测、或信道质量测量等。The reference signal is a "pilot" signal, which is a known signal provided by the transmitter to the receiver for channel estimation or channel detection. It can be used for coherent detection and demodulation of terminal equipment, beam measurement or coherent detection and monitoring of network equipment, or channel quality measurement.

2、信道状态信息参考信号(channel state information reference signal，CSI-RS)2. Channel state information reference signal (CSI-RS)

CSI-RS主要存在的意义就是测量下行信号的信息。其为由发射端提供给接收端用于信道估计或信道探测的一种已知信号。可以用于终端设备的信道状态信息测量、波束管理、时频跟踪、移动性管理等。The main purpose of CSI-RS is to measure the information of downlink signals. It is a known signal provided by the transmitter to the receiver for channel estimation or channel detection. It can be used for channel state information measurement, beam management, time-frequency tracking, mobility management, etc. of terminal devices.

3、CSI反馈模型3. CSI feedback model

CSI反馈模型也可称为CSI双边模型，CSI反馈模型包括CSI生成部分模型和CSI恢复部分模型，其中，CSI生成部分模型用于对信道状态信息CSI进行压缩，CSI恢复部分模型用于对压缩后的信道状态信息CSI进行恢复。CSI生成部分模型和CSI恢复部分模型需要通过收集的数据集训练得到。 The CSI feedback model can also be called a CSI bilateral model. The CSI feedback model includes a CSI generation model and a CSI recovery model. The CSI generation model is used to compress the channel state information CSI, and the CSI recovery model is used to recover the compressed channel state information CSI. The CSI generation model and the CSI recovery model need to be trained through the collected data set.

CSI生成部分模型可以定义为Encoder，CSI恢复部分模型可以定义为Decoder。一个可输出不同CSI压缩信息长度或者不同CSI反馈负载的Encoder可以称之为可扩展Encoder。The CSI generation model can be defined as an encoder, and the CSI recovery model can be defined as a decoder. An encoder that can output different CSI compression information lengths or different CSI feedback payloads can be called an extensible encoder.

4、无线资源控制(radio resource control，RRC)4. Radio resource control (RRC)

无线资源控制(radio resource control，RRC)，又称为无线资源管理或者无线资源分配，是指通过一定的策略和手段进行无线资源管理、控制和调度，在满足服务质量的要求下，尽可能地充分利用有限的无线网络资源，确保到达规划的覆盖区域，尽可能地提高业务容量和资源利用率。Radio resource control (RRC), also known as wireless resource management or wireless resource allocation, refers to the management, control and scheduling of wireless resources through certain strategies and means. It makes full use of limited wireless network resources as much as possible while meeting the requirements of service quality, ensures that the planned coverage area is reached, and maximizes service capacity and resource utilization.

5、媒体接入控制(medium access control,MAC)控制单元(Control Element，CE)5. Media access control (MAC) control element (CE)

MAC CE是在无线资源控制(radio resource control,RRC)消息和非接入层(non access stratum，NAS)消息之外，终端设备和网络之间的交换控制信息的一个途径，它交换的是关于MAC层的控制信息。MAC CE is a way to exchange control information between terminal devices and the network, in addition to radio resource control (RRC) messages and non-access stratum (NAS) messages. It exchanges control information about the MAC layer.

6、下行控制信息(Downlink Control Information，DCI)6. Downlink Control Information (DCI)

DCI为在物理下行控制信道(Physical Downlink Control Channel，PDCCH)传输的、与物理上下行共享信道相关的控制信息，这些DCI信息包含了诸如资源块(resource block，RB)分配信息、调制方式等等若干相关内容。DCI is the control information related to the physical uplink and downlink shared channel and transmitted on the Physical Downlink Control Channel (PDCCH). The DCI information includes several related contents such as resource block (RB) allocation information and modulation method.

为了更好的理解本公开实施例公开的一种模型性能的监测方法和装置，下面首先对本公开实施例适用的通信系统进行描述。In order to better understand the method and device for monitoring model performance disclosed in the embodiment of the present disclosure, the communication system to which the embodiment of the present disclosure is applicable is first described below.

请参见图1，图1为本公开实施例提供的一种通信系统的架构示意图。该通信系统可包括但不限于一个网络设备、一个终端设备，图1所示的设备数量和形态仅用于举例并不构成对本公开实施例的限定，实际应用中可以包括两个或两个以上的网络设备，两个或两个以上的终端设备。图1所示的通信系统以包括一个网络设备101和一个终端设备102。Please refer to FIG. 1, which is a schematic diagram of the architecture of a communication system provided by an embodiment of the present disclosure. The communication system may include, but is not limited to, a network device and a terminal device. The number and form of devices shown in FIG. 1 are only used as examples and do not constitute a limitation on the embodiment of the present disclosure. In actual applications, two or more network devices and two or more terminal devices may be included. The communication system shown in FIG. 1 includes a network device 101 and a terminal device 102.

需要说明的是，本公开实施例的技术方案可以应用于各种通信系统。例如：长期演进(long term evolution，LTE)系统、第五代(5th generation，5G)移动通信系统、5G新空口(new radio，NR)系统，或者其他未来的新型移动通信系统等。It should be noted that the technical solutions of the embodiments of the present disclosure can be applied to various communication systems, such as long term evolution (LTE) system, fifth generation (5G) mobile communication system, 5G new radio (NR) system, or other future new mobile communication systems.

本公开实施例中的网络设备101是网络侧的一种用于发射或接收信号的实体，用于与终端设备通信的设备，该网络设备101可以是全球移动通信(global system for mobile communications，GSM)系统或码分多址(code division multiple access，CDMA)中的基站(base transceiver station，BTS)，也可以是宽带码分多址(wideband code division multiple access，WCDMA)系统中的基站(nodeB，NB)，还可以是LTE系统中的演进型基站(evolved nodeB，eNB或eNodeB)，还可以是云无线接入网络(cloud radio access network，CRAN)场景下的无线控制器，还可以是无线网络控制器(radio network controller，RNC)、基站控制器(base station controller，BSC)、家庭基站(例如，home evolved nodeB，或home nodeB，HNB)、基带单元(baseband unit，BBU)，或者该网络设备可以为中继站、接入点、车载设备、可穿戴设备以及5G网络中的网络设备或者未来演进的PLMN网络中的网络设备等，可以是WLAN中的接入点(access point，AP)、无线中继节点、无线回传节点、传输点(transmission point，TP)或者发送接收点(transmission and reception point，TRP)等,可以是新型无线系统(new radio，NR)系统中的gNB或传输点(TRP或TP)，或者，5G系统中的基站的一个或一组(包括多个天线面板)天线面板，或者，还可以为构成gNB或传输点的网络节点，如基带单元(BBU)，或，分布式单元(distributed unit，DU)等，本公开的实施例对网络设备所采用的具体技术和具体设备形态不做限定。本公开实施例提供的网络设备可以是由集中单元(central unit，CU)与分布式单元(distributed unit，DU)组成的，其中，CU也可以称为控制单元(control unit)，采用CU-DU的结构可以将网络设备，例如网络设备的协议层拆分开，部分协议层的功能放在CU集中控制，剩下部分或全部协议层的功能分布在DU中，由CU集中控制DU。The network device 101 in the embodiment of the present disclosure is an entity on the network side for transmitting or receiving signals, and is a device for communicating with a terminal device. The network device 101 can be a base station (base transceiver station, BTS) in a global system for mobile communications (GSM) system or code division multiple access (CDMA), a base station (nodeB, NB) in a wideband code division multiple access (WCDMA) system, an evolved nodeB (eNB or eNodeB) in an LTE system, a wireless controller in a cloud radio access network (CRAN) scenario, a radio network controller (RNC), a base station controller (BSC) or a LTE system. The network device may be a base station (BSC), a home base station (for example, a home evolved nodeB, or a home nodeB, HNB), a baseband unit (BBU), or the network device may be a relay station, an access point, a vehicle-mounted device, a wearable device, a network device in a 5G network, or a network device in a future evolved PLMN network, etc., and may be an access point (AP) in a WLAN, a wireless relay node, a wireless backhaul node, a transmission point (TP) or a transmission and reception point (TRP), etc., and may be a gNB or a transmission point (TRP or TP) in a new wireless system (NR) system, or one or a group of (including multiple antenna panels) antenna panels of a base station in a 5G system, or may also be a network node constituting a gNB or a transmission point, such as a baseband unit (BBU), or a distributed unit (DU), etc. The embodiments of the present disclosure do not limit the specific technology and specific device form adopted by the network device. The network device provided by the embodiments of the present disclosure may be composed of a centralized unit (central unit, CU) and a distributed unit (distributed unit, DU), wherein the CU may also be called a control unit (control unit). The CU-DU structure may be used to split the network device, such as the protocol layer of the network device, with the functions of some protocol layers being centrally controlled by the CU, and the functions of the remaining part or all of the protocol layers being distributed in the DU, and the DU being centrally controlled by the CU.

本公开实施例中的终端设备102可以是一种向用户提供语音/数据连通性的设备，例如，具有无线连接功能的手持式设备、车载设备等。一些终端设备的举例为：手机(mobile phone)、平板电脑、笔记本电脑、掌上电脑、移动互联网设备(mobile internet device，MID)、可穿戴设备，虚拟现实(virtual reality，VR)设备、增强现实(augmented reality，AR)设备、工业控制(industrial control)中的无线终端、无人驾驶(self driving)中的无线终端、远程手术(remote medical surgery)中的无线终端、智能电网(smart grid)中的无线终端、运输安全(transportation safety)中的无线终端、智慧城市(smart city)中的无线终端、智慧家庭(smart home)中的无线终端、蜂窝电话、无绳电话、会话启动协议(session initiation protocol，SIP)电话、无线本地环路(wireless local loop，WLL)站、个人数字助理(personal digital assistant，PDA)、具有无线通信功能的手持设备、计算设备或连接到无线调制解调器的其它处理设备、车载设备、可穿戴设备，5G网络中的终端设备或者未来演进的公用陆地移动通信网络(public land mobile network，PLMN)中的终端设备和/或用于在无线通信系统上通信的任意其它适合设备，本公开实施例对此并不限定。The terminal device 102 in the embodiment of the present disclosure may be a device that provides voice/data connectivity to a user, for example, a handheld device or a vehicle-mounted device with a wireless connection function. Some examples of terminal devices are: mobile phones, tablet computers, laptop computers, PDAs, mobile internet devices (MIDs), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication capabilities, computing devices or other processing devices connected to wireless modems, vehicle-mounted devices, wearable devices, terminal devices in 5G networks or future The embodiments of the present disclosure are not limited to terminal devices in an evolved public land mobile network (PLMN) and/or any other suitable devices for communicating on a wireless communication system.

其中，可穿戴设备也可以称为穿戴式智能设备，是应用穿戴式技术对日常穿戴进行智能化设计、开发出可以穿戴的设备的总称，如眼镜、手环、手表、服饰及鞋等。可穿戴设备即直接穿在身上，或是整合到用户的衣服或配件的一种便携式设备。可穿戴设备不仅仅是一种硬件设备，更是通过软件支持以及数据交互、云端交互来实现强大的功能。广义穿戴式智能设备包括功能全、尺寸大、可不依赖智能手机实现完整或者部分的功能，例如：智能手表或智能眼镜等，以及只专注于某一类应用功能，需要和其它设备如智能手机配合使用，如各类进行体征监测的智能手环、智能首饰等。Among them, wearable devices can also be called wearable smart devices, which are a general term for the intelligent design and development of wearable devices for daily wear using wearable technology, such as glasses, bracelets, watches, clothing and shoes. Wearable devices are portable devices that are worn directly on the body or integrated into the user's clothes or accessories. Wearable devices are not only hardware devices, but also realize powerful functions through software support, data interaction, and cloud interaction. Broadly speaking, wearable smart devices include full-featured, large-sized, and independent of smartphones to achieve complete or partial functions, such as smart watches or smart glasses, as well as those that only focus on a certain type of application function and need to be used in conjunction with other devices such as smartphones, such as various smart bracelets and smart jewelry for vital sign monitoring.

此外，在本公开实施例中，终端设备102还可以是物联网系统中的终端设备，IoT(Internet of Things，物联网)是未来信息技术发展的重要组成部分，其主要技术特点是将物品通过通信技术与网络连接，从而实现人机互连，物物互连的智能化网络。In addition, in the disclosed embodiment, the terminal device 102 can also be a terminal device in an Internet of Things system. IoT (Internet of Things) is an important part of the future development of information technology. Its main technical feature is to connect objects to the network through communication technology, thereby realizing an intelligent network that interconnects people and machines and things.

此外，在本公开实施例中，终端设备102还可以包括智能打印机、火车探测器、加油站等传感器，主要功能包括收集数据(部分终端设备)、接收网络设备的控制信息与下行数据，并发送电磁波，向网络设备传输上行数据。In addition, in the embodiment of the present disclosure, the terminal device 102 may also include sensors such as smart printers, train detectors, and gas stations. Its main functions include collecting data (partial terminal devices), receiving control information and downlink data from network devices, and sending electromagnetic waves to transmit uplink data to network devices.

其中，本公开实施例提供的终端设备可以执行如图2至图6实施例所示的方法，本公开实施例提供的网络设备可以执行图7至图10实施例所示的方法。Among them, the terminal device provided by the embodiment of the present disclosure can execute the methods shown in the embodiments of Figures 2 to 6, and the network device provided by the embodiment of the present disclosure can execute the methods shown in the embodiments of Figures 7 to 10.

可以理解的是，本公开实施例描述的通信系统是为了更加清楚的说明本公开实施例的技术方案，并不构成对于本公开实施例提供的技术方案的限定，本领域普通技术人员可知，随着系统架构的演变和新业务场景的出现，本公开实施例提供的技术方案对于类似的技术问题，同样适用。It can be understood that the communication system described in the embodiment of the present disclosure is for the purpose of more clearly illustrating the technical solution of the embodiment of the present disclosure, and does not constitute a limitation on the technical solution provided by the embodiment of the present disclosure. A person skilled in the art can know that with the evolution of the system architecture and the emergence of new business scenarios, the technical solution provided by the embodiment of the present disclosure is also applicable to similar technical problems.

此外，为了便于理解本公开实施例，做出以下几点说明。In addition, in order to facilitate understanding of the embodiments of the present disclosure, the following points are explained.

第一，在本公开中第一、第二以及各种数字编号仅为描述方便进行的区分，并不用来限制本公开实施例的范围。例如，区分不同的信息、区分不同的终端设备等。First, the first, second and various digital numbers in the present disclosure are only used for the convenience of description and are not used to limit the scope of the embodiments of the present disclosure, for example, to distinguish different information, to distinguish different terminal devices, etc.

第二，本公开实施例中涉及的“协议”可以是指通信领域的标准协议，例如可以包括LTE协议、NR协议以及应用于未来的通信系统中的相关协议，本公开对此不做限定。Second, the “protocol” involved in the embodiments of the present disclosure may refer to a standard protocol in the communication field, for example, it may include an LTE protocol, an NR protocol, and related protocols used in future communication systems, and the present disclosure does not limit this.

第三，本公开实施例列举了多个实施方式以对本公开实施例的技术方案进行清晰地说明。当然，本领域内技术人员可以理解，本公开实施例提供的多个实施例，可以被单独执行，也可以与本公开实施例中其他实施例的方法结合后一起被执行，还可以单独或结合后与其他相关技术中的一些方法一起被执行；本公开实施例并不对此进行限定。Third, the embodiments of the present disclosure list multiple implementation methods to clearly illustrate the technical solutions of the embodiments of the present disclosure. Of course, those skilled in the art can understand that the multiple embodiments provided by the embodiments of the present disclosure can be executed separately, or can be executed together with the methods of other embodiments of the embodiments of the present disclosure, or can be executed together with some methods in other related technologies separately or in combination; the embodiments of the present disclosure do not limit this.

下面结合附图对本公开所提供的一种模型性能的监测方法和装置进行详细地介绍。A method and device for monitoring model performance provided by the present disclosure are described in detail below in conjunction with the accompanying drawings.

请参见图2，图2是本公开实施例提供的一种模型性能的监测方法的流程图。如图2所示，方法应用于终端设备。该方法可以包括但不限于如下步骤：Please refer to Figure 2, which is a flow chart of a method for monitoring model performance provided by an embodiment of the present disclosure. As shown in Figure 2, the method is applied to a terminal device. The method may include but is not limited to the following steps:

步骤201，对模型进行监测，获取模型当前关联的信道状态信息CSI。Step 201: monitor the model and obtain channel state information CSI currently associated with the model.

其中，模型为CSI反馈模型。Among them, the model is a CSI feedback model.

其中，CSI可以包括以下任一项：第一信息，模型当前的第一输入及第一信息。Among them, CSI may include any of the following items: first information, the current first input and first information of the model.

其中，第一信息为模型当前的第一输出或第一输出的关联信息。The first information is the current first output of the model or associated information of the first output.

其中，第一输入可以为CSI生成部分模型的输入。第一输出可以为CSI恢复部分模型的输出。The first input may be an input of the CSI generation part model, and the first output may be an output of the CSI recovery part model.

在一些可能的实现方式中，可以根据接收的第二CSI-RS，确定第一输入。第二CSI-RS可以为网络设备发送的，用于进行信道测量及信道估计的信号。In some possible implementations, the first input may be determined according to the received second CSI-RS. The second CSI-RS may be a signal sent by a network device and used for channel measurement and channel estimation.

具体的，终端设备可以根据接收的第二CSI-RS，估计不同频域单元的下行信道信息H，对第i个频域单元的下行信道信息H做奇异值分解得到特征向量v_i，将特征向量v_i作为第一输入。Specifically, the terminal device may estimate downlink channel information H of different frequency domain units according to the received second CSI-RS, perform singular value decomposition on the downlink channel information H of the i-th frequency domain unit to obtain a eigenvector _vi , and use the eigenvector _vi as the first input.

在一些可能的实现方式中，在终端设备侧包含CSI恢复部分模型的情况下，将CSI恢复部分模型的输出，确定为第一输出，其中，终端设备侧的CSI恢复部分模型与网络设备侧的CSI恢复部分模型相同或不同。In some possible implementations, when a CSI recovery partial model is included on the terminal device side, an output of the CSI recovery partial model is determined as a first output, wherein the CSI recovery partial model on the terminal device side is the same as or different from the CSI recovery partial model on the network device side.

具体的，将经过CSI生成部分模型压缩后的CSI经过量化后发送到终端设备侧的CSI恢复部分模型，以使终端设备侧的CSI恢复部分模型输出不同频域单元的向量信息为e_i，即第一输出。Specifically, the CSI compressed by the CSI generation partial model is quantized and sent to the CSI recovery partial model on the terminal device side, so that the CSI recovery partial model on the terminal device side outputs vector information of different frequency domain units as e _i , ie, the first output.

若终端设备侧包含的CSI恢复部分模型与网络设备侧的CSI恢复部分模型不相同，可以将终端设备侧的CSI恢复部分模型定义为代理模型或参考模型。在一些可能的实现方式中，可以基于网络设备的CSI恢复部分模型生成终端设备侧的CSI恢复部分模型。 If the CSI recovery partial model included in the terminal device side is different from the CSI recovery partial model on the network device side, the CSI recovery partial model on the terminal device side can be defined as a proxy model or a reference model. In some possible implementations, the CSI recovery partial model on the terminal device side can be generated based on the CSI recovery partial model of the network device.

或者，在终端设备侧未包含CSI恢复部分模型的情况下，接收网络设备发送的第一输出。Alternatively, when the terminal device side does not include the CSI recovery partial model, the first output sent by the receiving network device.

本公开中，若终端设备侧未包含CSI恢复部分模型，则需要获取网络设备侧的CSI恢复部分模型的输出。具体的，终端设备将经过CSI生成部分模型压缩后的CSI量化后上报给网络设备，网络设备将其输入至CSI恢复部分模型，经过CSI恢复部分模型恢复得到各频域单元的特征向量e_i，即第一输出。进而，网络设备可以对e_i做标量量化或以码本参数的方式发送给终端设备。In the present disclosure, if the terminal device side does not include a CSI recovery partial model, it is necessary to obtain the output of the CSI recovery partial model on the network device side. Specifically, the terminal device quantizes the CSI compressed by the CSI generation partial model and reports it to the network device, and the network device inputs it into the CSI recovery partial model, and recovers the characteristic vector e _i of each frequency domain unit through the CSI recovery partial model, that is, the first output. Then, the network device can perform scalar quantization on e _i or send it to the terminal device in the form of codebook parameters.

或者，在终端设备侧未包含CSI恢复部分模型的情况下，根据接收的第一信道状态信息参考信号CSI-RS，确定关联信息，其中，第一CSI-RS为由第一输出对应的波束赋形后的CSI-RS。Alternatively, when the terminal device side does not include a CSI recovery partial model, the associated information is determined based on the received first channel state information reference signal CSI-RS, wherein the first CSI-RS is the CSI-RS after beamforming corresponding to the first output.

本公开中，网络设备在通过CSI恢复部分模型恢复得到各频域单元的特征向量e_i之后，可以对特征向量e_i进行波束赋形，并将波束赋形的第一CSI-RS发送给终端设备，终端设备根据接收的波束赋形的第一CSI-RS估计第一输出的关联信息。其中，所用的波束为第一输出，即特征向量e_i。In the present disclosure, after the network device recovers the eigenvector e _i of each frequency domain unit through the CSI recovery partial model, it can beamform the eigenvector e _i and send the beamformed first CSI-RS to the terminal device, and the terminal device estimates the association information of the first output according to the received beamformed first CSI-RS. The beam used is the first output, that is, the eigenvector e _i .

在一些可能的实现方式中，在终端设备侧未包含CSI恢复部分模型的情况下，向网络设备发送第一信息获取请求。即终端设备向网络设备发送第一信息获取请求，以请求网络设备将CSI恢复部分模型的第一输出或第一CSI-RS发送给终端设备。In some possible implementations, when the terminal device side does not include the CSI recovery partial model, a first information acquisition request is sent to the network device. That is, the terminal device sends the first information acquisition request to the network device to request the network device to send the first output of the CSI recovery partial model or the first CSI-RS to the terminal device.

在一些可能的实现方式中，在终端设备侧未包含CSI恢复部分模型的情况下，网络设备也可以主动将CSI恢复部分模型的第一输出或第一CSI-RS发送给终端设备。In some possible implementations, when the terminal device side does not include the CSI recovery partial model, the network device may also actively send the first output of the CSI recovery partial model or the first CSI-RS to the terminal device.

步骤202，根据模型当前关联的CSI，确定模型当前的性能信息。Step 202: Determine the current performance information of the model according to the CSI currently associated with the model.

在一些可能的实现方式中，模型当前的性能信息，可以包括中间KPI、最终KPI、模型的第一输入、第一输出与训练数据的输入数据和输出数据之间的分布偏移量、及根据中间KPI、最终KPI、分布偏移量确定的性能结果等中的一项或者多项。In some possible implementations, the current performance information of the model may include one or more of the intermediate KPI, the final KPI, the first input of the model, the distribution offset between the first output and the input data and output data of the training data, and the performance results determined based on the intermediate KPI, the final KPI, and the distribution offset.

其中，中间KPI可以为余弦相似度的平方，归一化的最小均方误差等。本公开对此不做限定。The intermediate KPI may be the square of cosine similarity, normalized minimum mean square error, etc. This disclosure does not limit this.

其中，最终KPI也可以成为传输参数，最终KPI可以包括以下至少一项：系统吞吐量，网络误块率(Block Error Rate，BLER)，假定的BLER，确认字符ACK，非确认字符NACK。Among them, the final KPI can also become a transmission parameter, and the final KPI can include at least one of the following: system throughput, network block error rate (Block Error Rate, BLER), assumed BLER, confirmation character ACK, non-confirmation character NACK.

其中，系统吞吐量，网络误块率是数据传输的性能，数据传输是基于预编码传输的，预编码是根据第一输出确定的。ACK、NACK可根据BLER确定，当BLER大于对应的门限值，将恢复至NACK，否则，为ACK。Among them, system throughput and network block error rate are the performance of data transmission. Data transmission is based on precoding transmission, and precoding is determined according to the first output. ACK and NACK can be determined according to BLER. When BLER is greater than the corresponding threshold value, it will be restored to NACK, otherwise, it will be ACK.

步骤203，发送第一指示信息，其中，第一指示信息中包含模型当前的性能信息。Step 203: Send first indication information, where the first indication information includes current performance information of the model.

可以理解的是，终端设备在确定了模型当前的性能信息之后，即可将性能信息发送给网络设备，以使网络设备可以根据CSI反馈模型当前的性能信息，对CSI反馈模型执行以下操作中的任一项：激活，去激活，更新，切换。It can be understood that after the terminal device determines the current performance information of the model, it can send the performance information to the network device so that the network device can perform any of the following operations on the CSI feedback model based on the current performance information of the CSI feedback model: activation, deactivation, update, and switching.

综上，终端设备对模型进行监测，获取模型当前关联的信道状态信息CSI，进而根据模型当前关联的CSI，确定模型当前的性能信息，最后向网络设备发送包含模型当前的性能信息的第一指示信息。由此，终端设备可以通过CSI，准确地确定模型当前的性能信息，提高了对模型性能监测的准确性及可靠性。In summary, the terminal device monitors the model, obtains the channel state information CSI currently associated with the model, and then determines the current performance information of the model based on the CSI currently associated with the model, and finally sends the first indication information containing the current performance information of the model to the network device. As a result, the terminal device can accurately determine the current performance information of the model through the CSI, thereby improving the accuracy and reliability of model performance monitoring.

请参见图3，图3是本公开实施例提供的一种模型性能的监测方法的流程图。如图3所示，方法应用于终端设备。该方法可以包括但不限于如下步骤：Please refer to Figure 3, which is a flow chart of a method for monitoring model performance provided by an embodiment of the present disclosure. As shown in Figure 3, the method is applied to a terminal device. The method may include but is not limited to the following steps:

步骤301，接收第二指示信息，其中，第二指示信息中包含模型关联的监测参数。Step 301: Receive second indication information, wherein the second indication information includes monitoring parameters associated with the model.

在一些可能的实现方式中，监测参数可以包括以下一项或多项：In some possible implementations, the monitoring parameters may include one or more of the following:

监测启动时刻，上报监测结果的次数，监测时间信息及监测周期。Monitoring start time, number of monitoring results reported, monitoring time information and monitoring cycle.

其中，监测启动时刻可以为终端设备对模型进行监测的启动时刻。The monitoring start time may be the start time when the terminal device monitors the model.

上报检测结果的次数，可以为终端设备在对模型监测的过程中，一共向网络设备上报监测结果的次数。The number of times the detection result is reported may be the total number of times the terminal device reports the monitoring result to the network device during the process of monitoring the model.

监测时间信息，可以为终端设备对模型进行监测的时间长度。The monitoring time information may be the length of time that the terminal device monitors the model.

监测周期可以为终端设备对模型进行监测的周期。比如，多长时间对模型进行一次监测。The monitoring cycle can be the period during which the terminal device monitors the model, for example, how often the model is monitored.

在一些可能的实现方式中，可以通过以下一项或多项接收网络设备发送的第二指示信息：In some possible implementations, the second indication information sent by the network device may be received by one or more of the following:

无线资源控制RRC消息，媒体接入控制控制单元MAC-CE,下行控制信息DCI。Radio resource control RRC message, media access control control element MAC-CE, downlink control information DCI.

比如，终端设备可以通过RRC和MAC-CE的结合接收第二指示信息，或者通过RRC和DCI的结合和接收第二指示信息。本公开对此不做具体限定。For example, the terminal device may receive the second indication information through a combination of RRC and MAC-CE, or may receive the second indication information through a combination of RRC and DCI. This disclosure does not make specific limitations on this.

在一些可能的实现方式中，终端设备也可以基于协议约定，确定模型关联的监测参数。In some possible implementations, the terminal device may also determine the monitoring parameters associated with the model based on protocol agreements.

在一些可能的实现方式中，终端设备在确定监测参数之后，还可以向网络设备发送模型关联的监测参数。以便网络设备可以基于监测参数向终端设备发送第二CSI-RS。 In some possible implementations, after determining the monitoring parameters, the terminal device may also send the monitoring parameters associated with the model to the network device, so that the network device may send the second CSI-RS to the terminal device based on the monitoring parameters.

在一些可能的实现方式中，终端设备可以基于预设的事件触发，向网络设备发送模型关联的监测参数。In some possible implementations, the terminal device may send model-associated monitoring parameters to the network device based on a preset event trigger.

其中，预设的事件可以为终端设备预先设置的，用于触发将模型关联的监测参数发送给网络设备的事件。The preset event may be pre-set by the terminal device and is used to trigger an event in which the monitoring parameters associated with the model are sent to the network device.

步骤302，对模型进行监测，获取模型当前关联的信道状态信息CSI。Step 302: monitor the model and obtain channel state information CSI currently associated with the model.

步骤303，根据模型当前关联的CSI，确定模型当前的性能信息。Step 303: Determine the current performance information of the model according to the CSI currently associated with the model.

步骤304，发送第一指示信息，其中，第一指示信息中包含模型当前的性能信息。Step 304: Send first indication information, where the first indication information includes current performance information of the model.

本公开中，终端设备先接收包含模型关联的监测参数的第二指示信息，之后基于监测参数对模型进行监测，获取模型当前关联的信道状态信息CSI，进而根据模型当前关联的CSI，确定模型当前的性能信息，最后向网络设备发送包含模型当前的性能信息的第一指示信息。由此，可以基于监测参数、触发对模型的监测，进而通过确定的CSI，准确地确定模型当前的性能信息，进一步提高了对模型性能监测的准确性及可靠性。In the present disclosure, the terminal device first receives the second indication information including the monitoring parameters associated with the model, then monitors the model based on the monitoring parameters, obtains the channel state information CSI currently associated with the model, and then determines the current performance information of the model according to the CSI currently associated with the model, and finally sends the first indication information including the current performance information of the model to the network device. Thus, the monitoring of the model can be triggered based on the monitoring parameters, and then the current performance information of the model can be accurately determined through the determined CSI, further improving the accuracy and reliability of the model performance monitoring.

请参见图4，图4是本公开实施例提供的一种模型性能的监测方法的流程图。如图4所示，方法应用于终端设备。该方法可以包括但不限于如下步骤：Please refer to Figure 4, which is a flow chart of a method for monitoring model performance provided by an embodiment of the present disclosure. As shown in Figure 4, the method is applied to a terminal device. The method may include but is not limited to the following steps:

步骤401，对模型进行监测，获取模型当前关联的信道状态信息CSI。Step 401: monitor the model and obtain channel state information CSI currently associated with the model.

步骤402，在数据传输秩大于1的情况下，确定模型在每个数据传输层对应的CSI。Step 402: When the data transmission rank is greater than 1, determine the CSI corresponding to the model in each data transmission layer.

可以理解的是，在数据传输秩大于1的情况下，网络设备可以通过每个通道(数据传输层)分别向终端设备发送不同的CSI-RS，从而终端设备可以通过接收到的每个CSI-RS，确定每个数据传输层对应的CSI。It can be understood that when the data transmission rank is greater than 1, the network device can send different CSI-RS to the terminal device through each channel (data transmission layer), so that the terminal device can determine the CSI corresponding to each data transmission layer through each received CSI-RS.

其中，确定每个数据传输层的对应的CSI的具体实现方式，可以参照本公开中其他各实施例中的详细描述，此处不再具体赘述。Among them, the specific implementation method of determining the corresponding CSI of each data transmission layer can refer to the detailed description in other embodiments of the present disclosure, and will not be repeated here.

步骤403，根据每个数据传输层对应的CSI，确定模型在每个数据传输层的性能信息，和/或，确定模型在所有数据传输层的性能信息。Step 403: Determine the performance information of the model at each data transmission layer according to the CSI corresponding to each data transmission layer, and/or determine the performance information of the model at all data transmission layers.

可以理解的是，在确定了每个数据传输层对应的CSI之后，即可确定模型在每个数据传输层的性能信息，和/或，确定模型在所有数据传输层的性能信息。It can be understood that after determining the CSI corresponding to each data transmission layer, the performance information of the model in each data transmission layer can be determined, and/or the performance information of the model in all data transmission layers can be determined.

需要说明的是，确定模型在每个数据传输层的性能信息，还是确定模型在所有数据传输层的性能信息，与模型的设计相关。本公开对此不做具体限定。It should be noted that whether to determine the performance information of the model at each data transmission layer or to determine the performance information of the model at all data transmission layers is related to the design of the model, and the present disclosure does not make any specific limitation on this.

步骤404，发送第一指示信息，其中，第一指示信息中包含模型当前的性能信息。Step 404: Send first indication information, where the first indication information includes current performance information of the model.

本公开中，终端设备先基于监测参数对模型进行监测，获取模型当前关联的信道状态信息CSI，在数据传输秩大于1的情况下，确定模型在每个数据传输层对应的CSI，并根据每个数据传输层对应的CSI，确定模型在每个数据传输层的性能信息，和/或，确定模型在所有数据传输层的性能信息，最后向网络设备发送包含模型当前的性能信息的第一指示信息。由此，可以基于模型在每个数据传输层对应的CSI，准确地确定模型在每个数据传输层的性能信息，提高了对模型性能监测的准确性、可靠性和全面性。In the present disclosure, the terminal device first monitors the model based on the monitoring parameters, obtains the channel state information CSI currently associated with the model, and when the data transmission rank is greater than 1, determines the CSI corresponding to the model at each data transmission layer, and determines the performance information of the model at each data transmission layer based on the CSI corresponding to each data transmission layer, and/or determines the performance information of the model at all data transmission layers, and finally sends the first indication information containing the current performance information of the model to the network device. In this way, the performance information of the model at each data transmission layer can be accurately determined based on the CSI corresponding to the model at each data transmission layer, thereby improving the accuracy, reliability and comprehensiveness of the model performance monitoring.

请参见图5，图5是本公开实施例提供的一种模型性能的监测方法的流程图。如图5所示，方法应用于终端设备。该方法可以包括但不限于如下步骤：Please refer to Figure 5, which is a flow chart of a method for monitoring model performance provided by an embodiment of the present disclosure. As shown in Figure 5, the method is applied to a terminal device. The method may include but is not limited to the following steps:

步骤501，对模型进行监测，获取模型当前关联的信道状态信息CSI。Step 501: monitor the model and obtain the channel state information CSI currently associated with the model.

步骤502，根据模型当前关联的CSI，确定模型当前的性能信息。Step 502: Determine the current performance information of the model according to the CSI currently associated with the model.

在一些可能的实现方式中，根据第一输入及第一信息，确定模型当前的第一性能参数。In some possible implementations, a current first performance parameter of the model is determined based on the first input and the first information.

其中，第一性能参数，也可以称为中间KPI。第一性能参数可以为第一输入与第一信息之间的余弦相似度的平方，或者也可以为第一输入与第一信息之间的归一化的最小均方误差等。本公开对此不做限定。The first performance parameter may also be referred to as an intermediate KPI. The first performance parameter may be the square of the cosine similarity between the first input and the first information, or may be the normalized minimum mean square error between the first input and the first information, etc. This disclosure does not limit this.

其中，确定余弦相似度的平方的公式可以为：
Among them, the formula for determining the square of cosine similarity can be:

其中，K₁为余弦相似度的平方，H为终端设备根据第二CSI-RS，估计的不同频域单元的下行信道信息H，v_i为第一输入，e_i为第一信息，N₃表示频域单元的个数。Among them, _K1 is the square of cosine similarity, H is the downlink channel information H of different frequency domain units estimated by the terminal device according to the second CSI-RS, _vi is the first input, e _i is the first information, and _N3 represents the number of frequency domain units.

在一些可能的实现方式中，计算KPI所用的下行信道信息:不早于CSI上报时刻且离该CSI上报时刻最近一次估计的下行信道信息。如果存在多个CSI report，不早于第一个CSI上报时刻且离该CSI上报时刻最近一次估计的下行信道信息。In some possible implementations, the downlink channel information used to calculate the KPI is: not earlier than the CSI reporting time and If there are multiple CSI reports, the downlink channel information estimated at the time not earlier than the first CSI reporting time and the most recent time to the CSI reporting time.

图5A为本公开一实施例提供的一种CSI测量和上报时序图，如图5A所示，存在CSI report1和CSI report2，终端设备根据CSI-RS1、CSI-RS2、CSI-RS3估计了三个不同时刻的下行信道信息，但是，终端设备是利用与CSI-RS3估计的下行信道信息计算第一性能参数。Figure 5A is a CSI measurement and reporting timing diagram provided by an embodiment of the present disclosure. As shown in Figure 5A, there are CSI report1 and CSI report2. The terminal device estimates the downlink channel information at three different times based on CSI-RS1, CSI-RS2, and CSI-RS3. However, the terminal device calculates the first performance parameter using the downlink channel information estimated by CSI-RS3.

在一些可能的实现方式中，根据第一输入及第一信息，确定模型当前的第一性能参数，之后将当前的第一性能参数与第一门限值间的比较结果，确定为模型当前的第二性能参数。In some possible implementations, a current first performance parameter of the model is determined based on the first input and the first information, and then a comparison result between the current first performance parameter and the first threshold value is determined as the current second performance parameter of the model.

其中，第一性能参数与第一门限值间的比较结果可以为第一性能参数与第一门限值之间的大小关系，也可以为第一性能参数与第一门限值之间的差值。The comparison result between the first performance parameter and the first threshold value may be a magnitude relationship between the first performance parameter and the first threshold value, or may be a difference between the first performance parameter and the first threshold value.

其中，第一门限值可用于对第一性能参数进行判断，以确定模型的性能结果。比如，在第一性能参数小于或等于第一门限值情况下，确定模型的性能差，在第一性能参数大于第一门限值的情况下，确定模型的性能好。The first threshold value may be used to judge the first performance parameter to determine the performance result of the model. For example, when the first performance parameter is less than or equal to the first threshold value, the performance of the model is determined to be poor, and when the first performance parameter is greater than the first threshold value, the performance of the model is determined to be good.

本公开中，终端设备可以通过1比特(bit)指示第二性能参数。比如，当第一性能参数小于或等于第一门限值时，该bit的值可以为0，指示模型的性能差；否则为1，指示模型的性能好。In the present disclosure, the terminal device may indicate the second performance parameter by 1 bit. For example, when the first performance parameter is less than or equal to the first threshold value, the value of the bit may be 0, indicating that the performance of the model is poor; otherwise, it may be 1, indicating that the performance of the model is good.

其中，第一门限值可以是网络设备指示的；也可以是预先设置的；还可以是根据协议约定的。本公开对此不做限定。The first threshold value may be indicated by the network device, may be preset, or may be agreed upon in a protocol, which is not limited in the present disclosure.

在一些可能的实现方式中，也可以根据第一输入及第一信息，确定模型当前的第一性能参数，之后基于接收的码本参数信息及第一输入，确定参考参数，再将参考参数及第一性能参数，或者将参考参数与第一性能参数的比较结果，确定为模型当前的第三性能参数。In some possible implementations, the current first performance parameter of the model can also be determined based on the first input and the first information, and then the reference parameter can be determined based on the received codebook parameter information and the first input, and then the reference parameter and the first performance parameter, or the comparison result of the reference parameter and the first performance parameter, can be determined as the current third performance parameter of the model.

其中，码本参数信息可以包括网络设备为终端设备配置的码本参数组合。码本参数信息中可以包含一个或多个码本参数组合。本公开对此不做限定。The codebook parameter information may include a codebook parameter combination configured by the network device for the terminal device. The codebook parameter information may include one or more codebook parameter combinations. This disclosure does not limit this.

其中，码本参数组合中可以包括码本参数和码本类型。其中，码本类型可以包括类型Type I和类型Type II，其中，Type II码本又分为Rel-15/16 Type II码本或者Rel-17 Type II端口选择码本。The codebook parameter combination may include a codebook parameter and a codebook type. The codebook type may include a Type I codebook and a Type II codebook, wherein the Type II codebook is further divided into a Rel-15/16 Type II codebook or a Rel-17 Type II port selection codebook.

其中，参考参数可以包括参考余弦相似度的平方，或者参考归一化的最小均方误差等。The reference parameter may include the square of the reference cosine similarity, or the reference normalized minimum mean square error, etc.

其中，参考参数与第一性能参数的比较结果，可以为参考参数的第一性能参数的差值，也可以为第一性能参数与参考参数的比值。The comparison result between the reference parameter and the first performance parameter may be a difference between the reference parameter and the first performance parameter, or may be a ratio of the first performance parameter to the reference parameter.

可选的，终端设备可以确定模型对应的压缩后的第一CSI反馈负载大小及码本参数信息确定的第二CSI反馈负载大小，之后确定第二CSI反馈负载大小与第一CSI反馈负载大小间的差值的绝对值，进而基于对应绝对值最小的码本参数组合及第一输入，确定参考参数。Optionally, the terminal device can determine the compressed first CSI feedback load size corresponding to the model and the second CSI feedback load size determined by the codebook parameter information, and then determine the absolute value of the difference between the second CSI feedback load size and the first CSI feedback load size, and then determine the reference parameter based on the codebook parameter combination with the smallest absolute value and the first input.

其中，CSI反馈负载大小是指每个传输层或所有传输层对应的反馈负载。CSI反馈负载大小可以为进行CSI反馈所需的信道资源或数据容量的量度。它可以表示为用于传输CSI的比特数、频带宽度或时间片的数量。The CSI feedback load size refers to the feedback load corresponding to each transmission layer or all transmission layers. The CSI feedback load size can be a measure of the channel resources or data capacity required for CSI feedback. It can be expressed as the number of bits, bandwidth or time slices used to transmit CSI.

其中，第一CSI反馈负载大小可以为CSI生成模型压缩后的CSI对应的反馈负载大小。第二CSI反馈负载大小可以为基于码本参数信息确定的CSI反馈负载大小。The first CSI feedback payload size may be a feedback payload size corresponding to the CSI compressed by the CSI generation model, and the second CSI feedback payload size may be a CSI feedback payload size determined based on codebook parameter information.

具体的，基于绝对值最小的码本参数组合对根据第二CSI-RS估计的不同频域单元的下行信道信息H进行预编码，以得到不同频域单元的预编码向量b，基于预编码向量b及第一输入，确定参考参数。Specifically, the downlink channel information H of different frequency domain units estimated according to the second CSI-RS is precoded based on the codebook parameter combination with the smallest absolute value to obtain the precoding vector b of the different frequency domain units, and the reference parameter is determined based on the precoding vector b and the first input.

其中，参考参数可以包括参考余弦相似度的平方(即预编码向量b与第一输入之间的余弦相似度的平方)，或者参考归一化的最小均方误差(即预编码向量b与第一输入之间的归一化的最小均方误差)等。Among them, the reference parameters may include the square of the reference cosine similarity (i.e., the square of the cosine similarity between the precoding vector b and the first input), or the reference normalized minimum mean square error (i.e., the normalized minimum mean square error between the precoding vector b and the first input), etc.

举例来说，码本参数信息中配置了三个码本参数组合，分别为码本参数组合1、码本参数组合2、码本参数组合3；码本参数组合1对应的第二CSI反馈负载大小是50，码本参数组合2对应的第二CSI反馈负载大小是100，码本参数组合3对应的第二CSI反馈负载大小是150，第一CSI反馈负载大小为90。则确定码本参数组合2为绝对值最小的码本参数组合，进而基于码本参数组合2，确定参考参数。For example, three codebook parameter combinations are configured in the codebook parameter information, namely, codebook parameter combination 1, codebook parameter combination 2, and codebook parameter combination 3; the second CSI feedback load size corresponding to codebook parameter combination 1 is 50, the second CSI feedback load size corresponding to codebook parameter combination 2 is 100, the second CSI feedback load size corresponding to codebook parameter combination 3 is 150, and the first CSI feedback load size is 90. Codebook parameter combination 2 is determined to be the codebook parameter combination with the smallest absolute value, and then based on codebook parameter combination 2, the reference parameter is determined.

在一些可能的实现方式中，终端设备还可以确定第一输入相对模型对应的训练数据中的第二输入的第一分布偏移量，或第一输出相对训练数据中的第二输出的第二分布偏移量，之后将第一分布偏移量或第二分布偏移量，确定为模型当前的第四性能参数。In some possible implementations, the terminal device may also determine a first distribution offset of the first input relative to the second input in the training data corresponding to the model, or a second distribution offset of the first output relative to the second output in the training data, and then determine the first distribution offset or the second distribution offset as the current fourth performance parameter of the model.

在一些可能的实现方式中，终端设备还可以确定第一输入相对模型对应的训练数据中的第二输入的第一分布偏移量，及第一输出相对训练数据中的第二输出的第二分布偏移量，之后将第一分布偏移量与第二门限值的比较结果，或者将第二分布偏移量与第三门限值的比较结果，确定为模型当前的第五性能参数。In some possible implementations, the terminal device may also determine the first input relative to the second input in the training data corresponding to the model. The first distribution offset of the first output relative to the second output in the training data is determined, and the second distribution offset of the first output relative to the second output in the training data is then compared with the second threshold value, or the comparison result of the second distribution offset with the third threshold value is determined as the current fifth performance parameter of the model.

其中，第一分布偏移量与第二门限值间的比较结果可以为第一分布偏移量与第二门限值之间的大小关系，也可以为第一分布偏移量与第二门限值之间的差值。同理，第二分布偏移量与第三门限值间的比较结果可以为第二分布偏移量与第三门限值之间的大小关系，也可以为第二分布偏移量与第三门限值之间的差值。The comparison result between the first distribution offset and the second threshold value may be the magnitude relationship between the first distribution offset and the second threshold value, or the difference between the first distribution offset and the second threshold value. Similarly, the comparison result between the second distribution offset and the third threshold value may be the magnitude relationship between the second distribution offset and the third threshold value, or the difference between the second distribution offset and the third threshold value.

其中，第二门限值可用于对第一分布偏移量进行判断，以确定模型的性能结果。比如，在第一分布偏移量小于或等于第二门限值情况下，确定模型的性能差，在第一分布偏移量大于第二门限值的情况下，确定模型的性能好。The second threshold value may be used to judge the first distribution offset to determine the performance result of the model. For example, when the first distribution offset is less than or equal to the second threshold value, the performance of the model is determined to be poor, and when the first distribution offset is greater than the second threshold value, the performance of the model is determined to be good.

其中，第三门限值可用于对第二分布偏移量进行判断，以确定模型的性能结果。比如，在第二分布偏移量小于或等于第三门限值情况下，确定模型的性能差，在第二分布偏移量大于第三门限值的情况下，确定模型的性能好。The third threshold value may be used to judge the second distribution offset to determine the performance result of the model. For example, when the second distribution offset is less than or equal to the third threshold value, the performance of the model is determined to be poor, and when the second distribution offset is greater than the third threshold value, the performance of the model is determined to be good.

本公开中，终端设备可以通过1比特(bit)指示第五性能参数。比如，当第二分布偏移量小于或等于第三门限值时，该bit的值可以为0，指示模型的性能差；否则为1，指示模型的性能好。In the present disclosure, the terminal device may indicate the fifth performance parameter by 1 bit. For example, when the second distribution offset is less than or equal to the third threshold value, the value of the bit may be 0, indicating that the performance of the model is poor; otherwise, it may be 1, indicating that the performance of the model is good.

其中，第二门限值、第三门限值可以是网络设备指示的；也可以是预先设置的；还可以是根据协议约定的。本公开对此不做限定。The second threshold value and the third threshold value may be indicated by the network device, may be preset, or may be agreed upon in a protocol, which is not limited in the present disclosure.

在一些可能的实现方式中，终端设备还可以根据第一信息，确定系统当前的传输参数，之后，根据当前的传输参数，确定模型对应的第六性能参数。In some possible implementations, the terminal device may also determine a current transmission parameter of the system based on the first information, and then determine a sixth performance parameter corresponding to the model based on the current transmission parameter.

其中，当前的传输参数也可以成为最终KPI，传输参数包括以下至少一项：系统吞吐量，网络误块率BLER，假定的BLER，确认字符ACK，非确认字符NACK。The current transmission parameter may also become the final KPI, and the transmission parameter includes at least one of the following: system throughput, network block error rate BLER, assumed BLER, acknowledgment character ACK, and non-acknowledgment character NACK.

在一些可能的实现方式中，终端设备还可以根据第一信息，确定系统当前的传输参数，之后根据当前的传输参数与第四门限值的比较结果，确定模型对应的第七性能参数。In some possible implementations, the terminal device may also determine a current transmission parameter of the system based on the first information, and then determine a seventh performance parameter corresponding to the model based on a comparison result between the current transmission parameter and the fourth threshold value.

其中，传输参数与第四门限值间的比较结果可以为传输参数与第四门限值之间的大小关系，也可以为传输参数与第四门限值之间的差值。The comparison result between the transmission parameter and the fourth threshold value may be a magnitude relationship between the transmission parameter and the fourth threshold value, or may be a difference between the transmission parameter and the fourth threshold value.

其中，第四门限值可用于对传输参数进行判断，以确定模型的性能结果。比如，在传输参数小于或等于第四门限值的情况下，确定模型的性能差，在传输参数大于第四门限值的情况下，确定模型的性能好。The fourth threshold value may be used to judge the transmission parameter to determine the performance result of the model. For example, when the transmission parameter is less than or equal to the fourth threshold value, the performance of the model is determined to be poor, and when the transmission parameter is greater than the fourth threshold value, the performance of the model is determined to be good.

本公开中，终端设备可以通过1比特(bit)指示第七性能参数。比如，当传输参数小于或等于第四门限值时，该bit的值可以为0，指示模型的性能差；否则为1，指示模型的性能好。In the present disclosure, the terminal device may indicate the seventh performance parameter by 1 bit. For example, when the transmission parameter is less than or equal to the fourth threshold value, the value of the bit may be 0, indicating that the performance of the model is poor; otherwise, it may be 1, indicating that the performance of the model is good.

其中，第四门限值可以是网络设备指示的；也可以是预先设置的；还可以是根据协议约定的。本公开对此不做限定。The fourth threshold value may be indicated by the network device, may be preset, or may be agreed upon in a protocol, which is not limited in the present disclosure.

在一些可能的实现方式中，终端设备还可以确定当前的性能信息与第五门限值的第一比较结果，之后根据模型连续的K次第一比较结果，确定模型对应的第八性能参数，其中，k为自然数。In some possible implementations, the terminal device may also determine a first comparison result between the current performance information and the fifth threshold value, and then determine an eighth performance parameter corresponding to the model based on K consecutive first comparison results of the model, where k is a natural number.

其中，第五门限值、K可以是网络设备指示的，也可以是预先设置的；还可以是根据协议约定的。本公开对此不做限定。The fifth threshold value, K, may be indicated by the network device, or may be preset, or may be agreed upon in a protocol, which is not limited in the present disclosure.

其中，第八性能参数可以包括K次第一比较结果中，性能信息大于第五门限值的次数，或者还可以包括K次第一比较结果中，性能信息大于第五门限值的比例。The eighth performance parameter may include the number of times the performance information is greater than the fifth threshold value in the K first comparison results, or may also include the proportion of the performance information greater than the fifth threshold value in the K first comparison results.

在一些可能的实现方式中，终端设备还可以确定当前的性能信息与第五门限值的第一比较结果，之后根据模型连续的K次第一比较结果与第六门限值的第二比较结果，确定模型对应的第九性能参数，其中，K为自然数。In some possible implementations, the terminal device may also determine a first comparison result between the current performance information and the fifth threshold value, and then determine a ninth performance parameter corresponding to the model based on K consecutive first comparison results of the model and a second comparison result of the sixth threshold value, where K is a natural number.

其中，第六门限值可以是网络设备指示的，也可以是预先设置的；还可以是根据协议约定的。本公开对此不做限定。The sixth threshold value may be indicated by the network device, or may be preset, or may be agreed upon in a protocol, which is not limited in the present disclosure.

其中，第六门限值可以为次数阈值，也可以为比例阈值。The sixth threshold value may be a times threshold value or a ratio threshold value.

其中，第九性能参数可以包括K次第一比较结果中，性能信息大于第五门限值的次数与第六门限值(次数阈值)的关系；或者还可以包括K次第一比较结果中，性能信息大于第五门限值的比例与第六门限值(比例阈值)的关系。 Among them, the ninth performance parameter may include the relationship between the number of times the performance information is greater than the fifth threshold value in the K first comparison results and the sixth threshold value (number threshold); or it may also include the relationship between the proportion of the performance information greater than the fifth threshold value in the K first comparison results and the sixth threshold value (proportion threshold).

步骤503，发送第一指示信息，其中，第一指示信息中包含模型当前的性能信息。Step 503: Send first indication information, where the first indication information includes current performance information of the model.

终端设备可以通过第一指示信息上报一次测试的性能参数，也可以上报多次测试的性能参数。本公开对此不做限定。The terminal device may report the performance parameters of a single test or the performance parameters of multiple tests through the first indication information, which is not limited in the present disclosure.

本公开中，终端设备对模型进行监测，获取模型当前关联的信道状态信息CSI，进而根据模型当前关联的CSI，确定模型当前的性能信息，最后向网络设备发送包含模型当前的性能信息的第一指示信息。由此，终端设备可以通过CSI，准确地确定模型当前的性能信息，提高了对模型性能监测的准确性及可靠性。In the present disclosure, the terminal device monitors the model, obtains the channel state information CSI currently associated with the model, and then determines the current performance information of the model according to the CSI currently associated with the model, and finally sends the first indication information containing the current performance information of the model to the network device. Thus, the terminal device can accurately determine the current performance information of the model through the CSI, thereby improving the accuracy and reliability of the model performance monitoring.

请参见图6，图6是本公开实施例提供的一种模型性能的监测方法的流程图。如图6所示，方法应用于终端设备。该方法可以包括但不限于如下步骤：Please refer to Figure 6, which is a flow chart of a method for monitoring model performance provided by an embodiment of the present disclosure. As shown in Figure 6, the method is applied to a terminal device. The method may include but is not limited to the following steps:

步骤601，确定选择的码本参数组合确定的第二CSI反馈负载大小，及N个模型中每个模型对应的压缩后的第一CSI反馈负载大小。Step 601: determine a second CSI feedback payload size determined by a selected codebook parameter combination and a compressed first CSI feedback payload size corresponding to each model in N models.

其中，选择的码本参数组合可以为终端设备从网络设备配置的多个码本参数组合中选择的，也可以为网络设备配置的。The selected codebook parameter combination may be selected by the terminal device from a plurality of codebook parameter combinations configured by the network device, or may be configured by the network device.

步骤602，将N个模型中对应的第一CSI反馈负载大小小于或等于第二CSI反馈负载大小的模型，或者与第一CSI反馈负载大小间的差值的绝对值最小的模型，确定为待监测的模型。Step 602: Determine, as the model to be monitored, the model whose corresponding first CSI feedback payload size is less than or equal to the second CSI feedback payload size among the N models, or the model whose absolute value of the difference between the model and the first CSI feedback payload size is the smallest.

举例来说，假设终端设备部署了N＝3个CSI生成部分模型，每个CSI生成部分模型对应不同的第一CSI反馈负载大小。此外，还确定选择的码本参数组合确定的第二CSI反馈负载大小。如果CSI生成部分模型1和CSI生成部分模型2的CSI压缩反馈负载均小于第二CSI反馈负载大小，则终端设备可以选择CSI生成部分模1和CSI生成部分模型2，作为待监测的模型。For example, assume that the terminal device deploys N=3 CSI generation part models, and each CSI generation part model corresponds to a different first CSI feedback load size. In addition, the second CSI feedback load size determined by the selected codebook parameter combination is also determined. If the CSI compression feedback load of CSI generation part model 1 and CSI generation part model 2 are both smaller than the second CSI feedback load size, the terminal device can select CSI generation part model 1 and CSI generation part model 2 as the models to be monitored.

在一些可能的实现方式中，终端设备也可以对N个模型中的至少M个模型进行监测，其中，N为终端设备中部署的模型数量，M为小于或等于N的值。即终端设备也可以随机选择M个模型进行监测。In some possible implementations, the terminal device may also monitor at least M models out of N models, where N is the number of models deployed in the terminal device and M is a value less than or equal to N. That is, the terminal device may also randomly select M models for monitoring.

步骤603，对模型进行监测，获取模型当前关联的信道状态信息CSI。Step 603: monitor the model and obtain channel state information CSI currently associated with the model.

步骤604，根据模型当前关联的CSI，确定模型当前的性能信息。Step 604, determining the current performance information of the model according to the CSI currently associated with the model.

步骤605，确定第一指示信息的发送模式。Step 605: determine a sending mode of the first indication information.

其中，发送模式包括以下至少一项：发送频率，发送次数，发送内容。The sending mode includes at least one of the following: sending frequency, sending times, and sending content.

其中，发送频率可以为终端设备多久发送一次第一指示信息。The sending frequency may be how often the terminal device sends the first indication information.

发送次数可以为终端设备在对模型进行监测的过程中，共发送第一指示信息的次数。The number of times of sending may be the total number of times the terminal device sends the first indication information during the process of monitoring the model.

发送内容可以为终端通过第一指示信息发送的性能信息具体为哪几个性能参数。即发送内容可以为第一性能参数、第二性能参数、第三性能参数、第四性能参数、第五性能参数、第六性能参数、第七性能参数、第八性能参数、第九性能参数中的一个或多个。The sent content may be the specific performance parameters of the performance information sent by the terminal through the first indication information, that is, the sent content may be one or more of the first performance parameter, the second performance parameter, the third performance parameter, the fourth performance parameter, the fifth performance parameter, the sixth performance parameter, the seventh performance parameter, the eighth performance parameter, and the ninth performance parameter.

在一些可能的实现方式中，终端设备可以根据网络设备的指示，确定发送模式；或者，根据协议约定，确定发送模式。In some possible implementations, the terminal device may determine the sending mode according to an instruction of the network device; or determine the sending mode according to a protocol agreement.

或者，根据模型当前的性能信息，确定发送模式。比如，性能信息指示模型的性能比较好，就可以降低发送频次，或次数；性能信息指示模型的性能比较好差，可以增加发送频次或次数等。Alternatively, the sending mode is determined based on the current performance information of the model. For example, if the performance information indicates that the performance of the model is relatively good, the sending frequency or number of times can be reduced; if the performance information indicates that the performance of the model is relatively poor, the sending frequency or number of times can be increased.

步骤606，基于发送模式，发送第一指示信息，其中，第一指示信息中包含模型当前的性能信息。Step 606: Send first indication information based on the sending mode, wherein the first indication information includes current performance information of the model.

本公开中，在确定了发送模式之后，即可基于发送模式，发送第一指示信息。In the present disclosure, after the sending mode is determined, the first indication information can be sent based on the sending mode.

本公开中，终端设备可以从部署的N个模型中选择待检测的模型，进而对待检测的模型进行监测，获取模型当前关联的信道状态信息CSI，进而根据模型当前关联的CSI，确定模型当前的性能信息，最后基于确定的发送模式，向网络设备发送包含模型当前的性能信息的第一指示信息。由此，终端设备可以先确定待监测的模型，进而可以通过待监测的模型对应的CSI，准确地确定待监测的模型当前的性能信息，提高了对模型性能监测的准确性及可靠性。In the present disclosure, the terminal device can select a model to be detected from the deployed N models, and then monitor the model to be detected, obtain the channel state information CSI currently associated with the model, and then determine the current performance information of the model according to the CSI currently associated with the model, and finally send the first indication information containing the current performance information of the model to the network device based on the determined sending mode. Thus, the terminal device can first determine the model to be monitored, and then accurately determine the current performance information of the model to be monitored through the CSI corresponding to the model to be monitored, thereby improving the accuracy and reliability of model performance monitoring.

请参见图7，图7是本公开实施例提供的一种模型性能的监测方法的流程图。如图7所示，方法应用于网络设备。该方法可以包括但不限于如下步骤：Please refer to Figure 7, which is a flow chart of a method for monitoring model performance provided by an embodiment of the present disclosure. As shown in Figure 7, the method is applied to a network device. The method may include but is not limited to the following steps:

步骤701，接收第一指示信息，其中，第一指示信息中包含信道状态信息CSI反馈模型当前的性能信息。Step 701: Receive first indication information, where the first indication information includes current performance information of a channel state information (CSI) feedback model.

其中，第一指示信息为终端设备发送给网络设备的。终端设备可以通过第一指示信息上报一次测试的性能参数，也可以上报多次测试的性能参数。本公开对此不做具体限定。The first indication information is sent by the terminal device to the network device. The terminal device can report the performance parameters of a single test or the performance parameters of multiple tests through the first indication information. This disclosure does not make specific limitations on this.

在一些可能的实现方式中，性能信息可以包括以下至少一项：In some possible implementations, the performance information may include at least one of the following:

第一性能参数，其中，第一性能参数为根据模型当前的第一输入与第一信息确定的； A first performance parameter, wherein the first performance parameter is determined according to a current first input and first information of the model;

第一性能参数与第一门限值的比较结果；A comparison result of the first performance parameter and the first threshold value;

参考参数及第一性能参数，其中，参考参数为终端设备基于接收的码本参数信息及第一输入确定的参数；A reference parameter and a first performance parameter, wherein the reference parameter is a parameter determined by the terminal device based on the received codebook parameter information and the first input;

参考参数与第一性能参数的比较结果；a comparison result of the reference parameter and the first performance parameter;

第一分布偏移量，其中，第一分布偏移量为第一输入相对模型对应的训练数据中的第二输入间的偏移量；A first distribution offset, wherein the first distribution offset is an offset between the first input and a second input in the training data corresponding to the model;

第二分布偏移量，其中，第二分布偏移量为第一输出相对模型对应的训练数据中的第二输出间的偏移量；A second distribution offset, wherein the second distribution offset is an offset between the first output and the second output in the training data corresponding to the model;

第五性能参数，其中，第五性能参数为第一分布偏移量与第二门限值的比较结果，或者为第二分布偏移量与第三门限值的比较结果；a fifth performance parameter, wherein the fifth performance parameter is a comparison result of the first distribution offset and the second threshold value, or a comparison result of the second distribution offset and the third threshold value;

第六性能参数，其中，第六性能参数为终端设备基于系统当前的传输参数确定的，其中，传输参数包括以下至少一项：系统吞吐量，网络误块率BLER，假定的BLER，确认字符ACK，非确认字符NACK；A sixth performance parameter, wherein the sixth performance parameter is determined by the terminal device based on a current transmission parameter of the system, wherein the transmission parameter includes at least one of the following: system throughput, network block error rate BLER, assumed BLER, confirmation character ACK, non-confirmation character NACK;

第六性能参数与第四门限值的比较结果；a comparison result of the sixth performance parameter and the fourth threshold value;

第八性能参数，其中，第八性能参数为模型连续的K次性能信息分别与第五门限值的第一比较结果；an eighth performance parameter, wherein the eighth performance parameter is a first comparison result of K consecutive performance information of the model and the fifth threshold value;

第九性能参数，其中，第九性能参数为模型连续的K次第一比较结果与第六门限值的第二比较结果，其中，第一比较结果为模型连续每次性能信息与第五门限值的比较结果。The ninth performance parameter is the second comparison result of K consecutive first comparison results of the model with the sixth threshold value, wherein the first comparison result is the comparison result of each consecutive performance information of the model with the fifth threshold value.

其中，第一输入可以为终端设备根据接收的第二CSI-RS，估计不同频域单元的下行信道信息H，进而对第i个频域单元的下行信道信息H做奇异值分解得到特征向量v_i。The first input may be that the terminal device estimates the downlink channel information H of different frequency domain units according to the received second CSI-RS, and then performs singular value decomposition on the downlink channel information H of the i-th frequency domain unit to obtain the eigenvector v _i .

其中，第一性能参数与第一门限值的比较结果可以为第一性能参数与第一门限值之间的大小关系，也可以为第一性能参数与第一门限值之间的差值。第一性能参数与第一门限值的比较结果即为第二性能参数。The comparison result between the first performance parameter and the first threshold value may be the magnitude relationship between the first performance parameter and the first threshold value, or may be the difference between the first performance parameter and the first threshold value. The comparison result between the first performance parameter and the first threshold value is the second performance parameter.

其中，参考参数与第一性能参数的比较结果,可以为参考参数的第一性能参数的差值，也可以为第一性能参数与参考参数的比值。参考参数与第一性能参数的比较结果即为第三性能参数。The comparison result between the reference parameter and the first performance parameter may be a difference between the reference parameter and the first performance parameter, or a ratio between the first performance parameter and the reference parameter. The comparison result between the reference parameter and the first performance parameter is the third performance parameter.

其中，第一分布偏移量或第二分布偏移量也可称为第四性能参数。The first distribution offset or the second distribution offset may also be referred to as a fourth performance parameter.

其中，第六性能参数与第四门限值间的比较结果可以为传输参数与第四门限值之间的大小关系，也可以为传输参数与第四门限值之间的差值。The comparison result between the sixth performance parameter and the fourth threshold value may be the magnitude relationship between the sixth performance parameter and the fourth threshold value, or may be the difference between the sixth performance parameter and the fourth threshold value.

其中，第六性能参数与第四门限值的比较结果也可称为第七性能参数。The comparison result between the sixth performance parameter and the fourth threshold value may also be referred to as the seventh performance parameter.

其中，第九性能参数可以包括K次第一比较结果中，性能信息大于第五门限值的次数与第六门限值(次数阈值)的关系；或者还可以包括K次第一比较结果中，性能信息大于第五门限值的比例与第六门限值(比例阈值)的关系。Among them, the ninth performance parameter may include the relationship between the number of times the performance information is greater than the fifth threshold value in the K first comparison results and the sixth threshold value (number threshold); or it may also include the relationship between the proportion of the performance information greater than the fifth threshold value in the K first comparison results and the sixth threshold value (proportion threshold).

在一些可能的实现方式中，第一门限值、第二门限值、第三门限值、第四门限值、第五门限值、第六门限值可以是网络设备指示的；也可以是预先设置的；还可以是根据协议约定的。本公开对此不做限定。In some possible implementations, the first threshold, the second threshold, the third threshold, the fourth threshold, the fifth threshold, and the sixth threshold may be indicated by the network device, may be preset, or may be agreed upon in a protocol, which is not limited in the present disclosure.

由此，网络设备在接收到第一指示信息之后，即可确定CSI反馈模型当前的性能信息。Therefore, after receiving the first indication information, the network device can determine the current performance information of the CSI feedback model.

本公开中，网络设备可以通过接收包含信道状态信息CSI反馈模型当前的性能信息对应的第一指示信息，准确地获取CSI反馈模型当前的性能。 In the present disclosure, the network device can accurately obtain the current performance of the CSI feedback model by receiving the first indication information corresponding to the current performance information of the channel state information CSI feedback model.

请参见图8，图8是本公开实施例提供的一种模型性能的监测方法的流程图。如图8所示，方法应用于网络设备。该方法可以包括但不限于如下步骤：Please refer to Figure 8, which is a flow chart of a method for monitoring model performance provided by an embodiment of the present disclosure. As shown in Figure 8, the method is applied to a network device. The method may include but is not limited to the following steps:

步骤801，向终端设备发送模型当前对应的第一输出；或者，向终端设备发送第一信道状态信息参考信号CSI-RS，其中，第一CSI-RS为由第一输出对应的波束赋形后的CSI-RS。Step 801, sending a first output currently corresponding to the model to the terminal device; or, sending a first channel state information reference signal CSI-RS to the terminal device, wherein the first CSI-RS is a CSI-RS after beamforming corresponding to the first output.

本公开中，若终端设备侧未包含CSI恢复部分模型，则需要获取网络设备侧的CSI恢复部分模型的第一输出。具体的，终端设备将经过CSI生成部分模型压缩后的CSI量化后上报给网络设备，网络设备将其输入至CSI恢复部分模型，经过CSI恢复部分模型恢复得到各频域单元的特征向量e_i，即第一输出。进而，网络设备可以对e_i做标量量化或以码本参数的方式发送给终端设备。In the present disclosure, if the terminal device side does not include a CSI recovery partial model, it is necessary to obtain the first output of the CSI recovery partial model on the network device side. Specifically, the terminal device quantizes the CSI compressed by the CSI generation partial model and reports it to the network device, and the network device inputs it into the CSI recovery partial model, and recovers the characteristic vector e _i of each frequency domain unit through the CSI recovery partial model, that is, the first output. Then, the network device can perform scalar quantization on e _i or send it to the terminal device in the form of codebook parameters.

在一些可能的实现方式中，网络设备可以接收终端设备发送的第一信息获取请求，其中，第一信息为模型的第一输出或第一输出的关联信息。In some possible implementations, the network device may receive a first information acquisition request sent by a terminal device, wherein the first information is a first output of the model or associated information of the first output.

步骤802，接收第一指示信息，其中，第一指示信息中包含信道状态信息CSI反馈模型当前的性能信息。Step 802: Receive first indication information, where the first indication information includes current performance information of a channel state information (CSI) feedback model.

在一些可能的实现方式中，在数据传输秩大于1的情况下，确定模型在每个数据传输层对应的性能信息，和/或确定模型在所有传输层的性能信息。In some possible implementations, when the data transmission rank is greater than 1, the performance information corresponding to the model in each data transmission layer is determined, and/or the performance information of the model in all transmission layers is determined.

可以理解的是，在数据传输秩大于1的情况下，网络设备可以通过每个通道(数据传输层)分别向终端设备发送不同的CSI-RS，从而终端设备可以通过接收到的每个CSI-RS，确定每个数据传输层对应的CSI。进而根据每个数据传输层对应的CSI，确定模型在每个数据传输层的性能信息，和/或，确定模型在所有数据传输层的性能信息。It can be understood that when the data transmission rank is greater than 1, the network device can send different CSI-RS to the terminal device through each channel (data transmission layer), so that the terminal device can determine the CSI corresponding to each data transmission layer through each received CSI-RS. Then, based on the CSI corresponding to each data transmission layer, the performance information of the model at each data transmission layer is determined, and/or the performance information of the model at all data transmission layers is determined.

本公开中，网络设备先向终端发送模型当前对应的第一输出、或第一CSI-RS，之后可以接收包含信道状态信息CSI反馈模型当前的性能信息对应的第一指示信息。由此，可以使终端设备结合第一输出、或第一CSI-RS，准确地确定CSI反馈模型当前的性能信息，进而可以准确地获取CSI反馈模型当前的性能。In the present disclosure, the network device first sends the first output or the first CSI-RS currently corresponding to the model to the terminal, and then can receive the first indication information corresponding to the current performance information of the channel state information CSI feedback model. As a result, the terminal device can accurately determine the current performance information of the CSI feedback model in combination with the first output or the first CSI-RS, and then accurately obtain the current performance of the CSI feedback model.

请参见图9，图9是本公开实施例提供的一种模型性能的监测方法的流程图。如图9所示，方法应用于网络设备。该方法可以包括但不限于如下步骤：Please refer to Figure 9, which is a flow chart of a method for monitoring model performance provided by an embodiment of the present disclosure. As shown in Figure 9, the method is applied to a network device. The method may include but is not limited to the following steps:

步骤901，发送第二指示信息，其中，第二指示信息中包含模型关联的监测参数。Step 901: Send second indication information, wherein the second indication information includes monitoring parameters associated with the model.

在一些可能的实现方式中，监测参数包括以下一项或多项：In some possible implementations, the monitoring parameters include one or more of the following:

在一些可能的实现方式中，可以通过以下一项或多项发送的第二指示信息：In some possible implementations, the second indication information may be sent through one or more of the following:

比如，可以通过RRC和MAC-CE相结合发送第二指示信息，或者，通过RRC和DCI的结合发送第二指示信息。本公开对此不做具体限定。For example, the second indication information may be sent through a combination of RRC and MAC-CE, or through a combination of RRC and DCI. This disclosure does not make specific limitations on this.

在一些可能的实现方式中，网络设备可以基于协议约定，确定模型关联的监测参数。In some possible implementations, the network device may determine the monitoring parameters associated with the model based on protocol agreements.

在一些可能的实现方式中，终端设备也可以基于协议约定，确定模型关联的监测参数，并在确定监测参数之后，向网络设备发送模型关联的监测参数。从而网络设备可以接收终端设备发送的模型关联的监测参数，进而基于监测参数向终端设备发送第二CSI-RS。In some possible implementations, the terminal device may also determine the monitoring parameters associated with the model based on the protocol agreement, and after determining the monitoring parameters, send the monitoring parameters associated with the model to the network device. Thus, the network device can receive the monitoring parameters associated with the model sent by the terminal device, and then send the second CSI-RS to the terminal device based on the monitoring parameters.

步骤902，接收第一指示信息，其中，第一指示信息中包含信道状态信息CSI反馈模型当前的性能信息。 Step 902: Receive first indication information, where the first indication information includes current performance information of a channel state information (CSI) feedback model.

本公开中，网络设备可以先向终端发送模型关联的监测参数，之后接收终端发送的包含信道状态信息CSI反馈模型当前的性能信息对应的指示信息。由此，可以基于监测参数触发终端对模型的监测，进而接收CSI反馈模型当前的性能信息，从而准确地获取CSI反馈模型当前的性能。In the present disclosure, the network device may first send the monitoring parameters associated with the model to the terminal, and then receive the indication information corresponding to the current performance information of the channel state information CSI feedback model sent by the terminal. Thus, the terminal may be triggered to monitor the model based on the monitoring parameters, and then receive the current performance information of the CSI feedback model, so as to accurately obtain the current performance of the CSI feedback model.

请参见图10，图10是本公开实施例提供的一种模型性能的监测方法的流程图。如图10所示，方法应用于网络设备。该方法可以包括但不限于如下步骤：Please refer to Figure 10, which is a flow chart of a method for monitoring model performance provided by an embodiment of the present disclosure. As shown in Figure 10, the method is applied to a network device. The method may include but is not limited to the following steps:

步骤1001，确定第一指示信息的发送模式。Step 1001: determine a sending mode of first indication information.

其中，发送模式包括以下至少一项：发送频率，发送次数，发送内容；The sending mode includes at least one of the following: sending frequency, sending times, and sending content;

一些可能的实现方式中，网络设备可以根据协议约定，确定发送模式。In some possible implementations, the network device may determine the sending mode according to a protocol agreement.

或者，终端设备在确定发送模式之后，向终端设备指示发送模式，以使终端设备可以基于发送模式，发送第一指示信息。Alternatively, after determining the sending mode, the terminal device indicates the sending mode to the terminal device, so that the terminal device can send the first indication information based on the sending mode.

或者，网络设备也可以接收终端设备发送的发送模式。Alternatively, the network device may also receive the transmission mode sent by the terminal device.

步骤1002，基于发送模式，接收第一指示信息。Step 1002: Receive first indication information based on the sending mode.

步骤1003，根据CSI反馈模型当前的性能信息，对CSI反馈模型执行以下操作中的任一项：激活，去激活，更新，切换。Step 1003: perform any one of the following operations on the CSI feedback model according to the current performance information of the CSI feedback model: activate, deactivate, update, switch.

在一些可能的实现方式中，还可以将CSI反馈方法由基于CSI反馈模型实现CSI反馈，切换至基于Type I和TypeII码本方式实现CSI反馈。In some possible implementations, the CSI feedback method may be switched from implementing CSI feedback based on a CSI feedback model to implementing CSI feedback based on Type I and Type II codebooks.

本公开中，在确定了CSI反馈模型当前的性能信息之后，即可CSI反馈模型执行以下操作中的任一项：激活，去激活，更新，切换。以提高CSI反馈模型的性能，或切换CSI反馈方法，以提高获取的CSI反馈信息的准确性。In the present disclosure, after determining the current performance information of the CSI feedback model, the CSI feedback model can perform any of the following operations: activation, deactivation, updating, switching, so as to improve the performance of the CSI feedback model, or switch the CSI feedback method, so as to improve the accuracy of the acquired CSI feedback information.

本公开中，网络设备可以先确定第一指示信息的发送模式，进而基于发送模式，接收终端发送的包含信道状态信息CSI反馈模型当前的性能信息对应的指示信息。由此，可以基于发送模式，准确地获取终端设备发送的CSI反馈模型当前的性能信息。In the present disclosure, the network device may first determine the transmission mode of the first indication information, and then based on the transmission mode, receive the indication information corresponding to the current performance information of the channel state information CSI feedback model sent by the terminal. Thus, the current performance information of the CSI feedback model sent by the terminal device may be accurately obtained based on the transmission mode.

请参见图11，图11是本公开实施例提供的又一种模型性能的监测方法的信令交互图。如图11所示，该方法可以包括但不限于如下步骤：Please refer to Figure 11, which is a signaling interaction diagram of another model performance monitoring method provided by an embodiment of the present disclosure. As shown in Figure 11, the method may include but is not limited to the following steps:

步骤1101，终端设备对模型进行监测，获取模型当前关联的信道状态信息CSI。Step 1101: The terminal device monitors the model and obtains the channel state information CSI currently associated with the model.

步骤1102，终端设备根据模型当前关联的CSI，确定模型当前的性能信息。Step 1102: The terminal device determines the current performance information of the model based on the CSI currently associated with the model.

步骤1103，终端设备向网络设备发送第一指示信息，其中，第一指示信息中包含模型当前的性能信息。Step 1103: The terminal device sends first indication information to the network device, wherein the first indication information includes current performance information of the model.

步骤1104，网络设备在接收到第一指示信息的情况下，根据CSI反馈模型当前的性能信息，对CSI反馈模型执行以下操作中的任一项：激活，去激活，更新，切换。Step 1104: upon receiving the first indication information, the network device performs any one of the following operations on the CSI feedback model according to the current performance information of the CSI feedback model: activation, deactivation, update, and switching.

由此，终端设备对模型进行监测，获取模型当前关联的信道状态信息CSI，进而根据模型当前关联的CSI，确定模型当前的性能信息，最后向网络设备发送包含模型当前的性能信息的第一指示信息。由此，终端设备可以通过CSI，准确地确定模型当前的性能信息，提高了对模型性能监测的准确性及可靠性。Thus, the terminal device monitors the model, obtains the channel state information CSI currently associated with the model, and then determines the current performance information of the model according to the CSI currently associated with the model, and finally sends the first indication information including the current performance information of the model to the network device. Thus, the terminal device can accurately determine the current performance information of the model through the CSI, thereby improving the accuracy and reliability of the model performance monitoring.

请参见图12，为本公开实施例提供的一种通信装置的结构示意图。图12所示的通信装置1200可包括收发模块1201及处理模块1202。收发模块可包括发送模块和/或接收模块，发送模块用于实现发送功能，接收模块用于实现接收功能，收发模块可以实现发送功能和/或接收功能。Please refer to Figure 12, which is a schematic diagram of the structure of a communication device provided in an embodiment of the present disclosure. The communication device 1200 shown in Figure 12 may include a transceiver module 1201 and a processing module 1202. The transceiver module may include a sending module and/or a receiving module, the sending module is used to implement a sending function, the receiving module is used to implement a receiving function, and the transceiver module may implement a sending function and/or a receiving function.

通信装置1200可以是终端设备，也可以是终端设备中的装置，还可以是能够与终端设备匹配使用的装置。The communication device 1200 may be a terminal device, or a device in a terminal device, or a device that can be used in conjunction with the terminal device. device.

通信装置1200在终端设备侧，其中：The communication device 1200 is on the terminal device side, wherein:

处理模块1202，用于对模型进行监测，获取模型当前关联的信道状态信息CSI，其中，CSI包括以下任一项：第一信息，模型当前的第一输入及第一信息；其中，第一信息为模型当前的第一输出或第一输出的关联信息，模型为CSI反馈模型；The processing module 1202 is used to monitor the model and obtain the channel state information CSI currently associated with the model, wherein the CSI includes any one of the following: first information, the current first input and first information of the model; wherein the first information is the current first output of the model or the associated information of the first output, and the model is a CSI feedback model;

处理模块1202，还用于根据模型当前关联的CSI，确定模型当前的性能信息；The processing module 1202 is further configured to determine current performance information of the model according to the CSI currently associated with the model;

收发模块1201，还用于发送第一指示信息，其中，第一指示信息中包含模型当前的性能信息。The transceiver module 1201 is further used to send first indication information, wherein the first indication information includes current performance information of the model.

在一些可能的实现方式中，处理模块1202，还用于：In some possible implementations, the processing module 1202 is further configured to:

在终端设备侧包含CSI恢复部分模型的情况下，将CSI恢复部分模型的输出，确定为第一输出，其中，终端设备侧的CSI恢复部分模型与网络设备侧的CSI恢复部分模型相同或不同；或者，In the case where the terminal device side includes a CSI recovery partial model, determining an output of the CSI recovery partial model as a first output, wherein the CSI recovery partial model on the terminal device side is the same as or different from the CSI recovery partial model on the network device side; or,

在终端设备侧未包含CSI恢复部分模型的情况下，接收网络设备发送的第一输出；或者，In the case where the terminal device side does not include the CSI recovery partial model, receiving the first output sent by the network device; or,

在终端设备侧未包含CSI恢复部分模型的情况下，根据接收的第一信道状态信息参考信号CSI-RS，确定关联信息，其中，第一CSI-RS为由第一输出对应的波束赋形后的CSI-RS。When the terminal device side does not include a CSI recovery partial model, the associated information is determined based on the received first channel state information reference signal CSI-RS, wherein the first CSI-RS is the CSI-RS after beamforming corresponding to the first output.

在一些可能的实现方式中，收发模块1201，还用于：In some possible implementations, the transceiver module 1201 is further configured to:

在终端设备侧未包含CSI恢复部分模型的情况下，发送第一信息获取请求。When the terminal device side does not include the CSI recovery partial model, a first information acquisition request is sent.

根据接收的第二CSI-RS，确定第一输入。A first input is determined according to the received second CSI-RS.

在一些可能的实现方式中，In some possible implementations,

收发模块1201，还用于接收第二指示信息，其中，第二指示信息中包含模型关联的监测参数；或者，The transceiver module 1201 is further configured to receive second indication information, wherein the second indication information includes monitoring parameters associated with the model; or,

处理模块1202，还用于基于协议约定，确定模型关联的监测参数；或者，The processing module 1202 is further configured to determine monitoring parameters associated with the model based on the protocol agreement; or,

收发模块1201，还用于向网络设备发送模型关联的监测参数。The transceiver module 1201 is also used to send the monitoring parameters associated with the model to the network device.

基于预设的事件触发，向网络设备发送模型关联的监测参数。Based on preset event triggers, model-associated monitoring parameters are sent to network devices.

在一些可能的实现方式中，通过以下一项或多项接收第二指示信息：无线资源控制RRC消息，媒体接入控制控制单元MAC-CE,下行控制信息DCI。In some possible implementations, the second indication information is received through one or more of the following: a radio resource control RRC message, a media access control control element MAC-CE, and downlink control information DCI.

在数据传输秩大于1的情况下，确定模型在每个数据传输层对应的CSI；When the data transmission rank is greater than 1, determine the CSI corresponding to the model in each data transmission layer;

根据每个传输层对应的CSI，确定模型在每个传输层的性能信息，和/或，确定模型在所有传输层的性能信息。According to the CSI corresponding to each transmission layer, the performance information of the model at each transmission layer is determined, and/or the performance information of the model at all transmission layers is determined.

根据第一输入及第一信息，确定模型当前的第一性能参数。A current first performance parameter of the model is determined according to the first input and the first information.

根据第一输入及第一信息，确定模型当前的第一性能参数；Determine a current first performance parameter of the model according to the first input and the first information;

将当前的第一性能参数与第一门限值间的比较结果，确定为模型当前的第二性能参数。A comparison result between the current first performance parameter and the first threshold value is determined as the current second performance parameter of the model.

基于接收的码本参数信息及第一输入，确定参考参数；Determining a reference parameter based on the received codebook parameter information and the first input;

将参考参数及第一性能参数，或者将参考参数与第一性能参数的比较结果，确定为模型当前的第三性能参数。The reference parameter and the first performance parameter, or a comparison result between the reference parameter and the first performance parameter, is determined as a current third performance parameter of the model.

确定模型对应的压缩后的第一CSI反馈负载大小及码本参数信息确定的第二CSI反馈负载大小；Determine a compressed first CSI feedback payload size corresponding to the model and a second CSI feedback payload size determined by codebook parameter information;

确定第二CSI反馈负载大小与第一CSI反馈负载大小间的差值的绝对值；Determining an absolute value of a difference between a second CSI feedback payload size and a first CSI feedback payload size;

基于对应绝对值最小的码本参数组合及第一输入，确定参考参数。A reference parameter is determined based on a codebook parameter combination with a minimum corresponding absolute value and the first input.

确定第一输入相对模型对应的训练数据中的第二输入的第一分布偏移量，或determining a first distribution offset of the first input relative to a second input in the training data corresponding to the model, or

第一输出相对训练数据中的第二输出的第二分布偏移量；a second distribution offset of the first output relative to the second output in the training data;

将第一分布偏移量或第二分布偏移量，确定为模型当前的第四性能参数。The first distribution offset or the second distribution offset is determined as the fourth current performance parameter of the model.

在一些可能的实现方式中，处理模块1202，还用于： In some possible implementations, the processing module 1202 is further configured to:

确定第一输入相对模型对应的训练数据中的第二输入的第一分布偏移量，及所Determine a first distribution offset of the first input relative to the second input in the training data corresponding to the model, and

述第一输出相对训练数据中的第二输出的第二分布偏移量；a second distribution offset of the first output relative to the second output in the training data;

将第一分布偏移量与第二门限值的比较结果，或者将第二分布偏移量与第三门The comparison result of the first distribution offset and the second threshold value, or the comparison result of the second distribution offset and the third threshold value

限值的比较结果，确定为模型当前的第五性能参数。The comparison result of the limit values is determined as the current fifth performance parameter of the model.

根据第一信息，确定系统当前的传输参数，其中，传输参数包括以下至少一项：Determine a current transmission parameter of the system according to the first information, wherein the transmission parameter includes at least one of the following:

系统吞吐量，网络误块率BLER，假定的BLER，确认字符ACK，非确认字符NACK；System throughput, network block error rate BLER, assumed BLER, acknowledgment character ACK, non-acknowledgment character NACK;

根据当前的传输参数，确定模型对应的第六性能参数。A sixth performance parameter corresponding to the model is determined according to the current transmission parameter.

根据第一信息，确定系统当前的传输参数，其中，传输参数包括以下至少一项：系统吞吐量，网络误块率BLER，假定的BLER，确认字符ACK，非确认字符NACK；Determine the current transmission parameters of the system according to the first information, wherein the transmission parameters include at least one of the following: system throughput, network block error rate BLER, assumed BLER, confirmation character ACK, non-confirmation character NACK;

根据当前的传输参数与第四门限值的比较结果，确定模型对应的第七性能参数。A seventh performance parameter corresponding to the model is determined according to a comparison result between the current transmission parameter and the fourth threshold value.

确定当前的性能信息与第五门限值的第一比较结果；Determine a first comparison result between the current performance information and a fifth threshold value;

根据模型连续的K次第一比较结果，确定模型对应的第八性能参数，其中，K为自然数。An eighth performance parameter corresponding to the model is determined according to K consecutive first comparison results of the model, where K is a natural number.

根据模型连续的K次第一比较结果与第六门限值的第二比较结果，确定模型对应的第九性能参数，其中，K为自然数。A ninth performance parameter corresponding to the model is determined according to K consecutive first comparison results of the model and a second comparison result of the sixth threshold value, wherein K is a natural number.

对N个模型中的至少M个模型进行监测，其中，N为终端设备中部署的模型数量，M为小于或等于N的值。At least M models out of N models are monitored, where N is the number of models deployed in the terminal device and M is a value less than or equal to N.

确定选择的码本参数组合确定的第二CSI反馈负载大小，及N个模型中每个模型对应的压缩后的第一CSI反馈负载大小；Determine a second CSI feedback payload size determined by the selected codebook parameter combination and a compressed first CSI feedback payload size corresponding to each model in the N models;

将N个模型中对应的第一CSI反馈负载大小小于或等于第二CSI反馈负载大小的模型，或者与第一CSI反馈负载大小间的差值的绝对值最小的模型，确定为待监测的模型。The model whose corresponding first CSI feedback load size is less than or equal to the second CSI feedback load size among the N models, or the model whose absolute value of the difference with the first CSI feedback load size is the smallest, is determined as the model to be monitored.

确定第一指示信息的发送模式，其中，发送模式包括以下至少一项：发送频率，发送次数，发送内容；Determine a sending mode of the first indication information, wherein the sending mode includes at least one of the following: sending frequency, sending times, and sending content;

基于发送模式，发送第一指示信息。Based on the sending mode, first indication information is sent.

根据网络设备的指示，确定发送模式；或者，Determine the sending mode according to the instruction of the network device; or,

根据协议约定，确定发送模式；或者，Determine the sending mode according to the agreement; or,

根据模型当前的性能信息，确定发送模式。Determine the sending mode based on the current performance information of the model.

通信装置1200还可以是网络设备，也可以是网络设备中的装置，还可以是能够与网络设备匹配使用的装置。The communication device 1200 may also be a network device, a device in a network device, or a device that can be used in conjunction with a network device.

通信装置1200在网络设备侧，其中：The communication device 1200 is on the network device side, wherein:

收发模块1201，用于接收第一指示信息，其中，第一指示信息中包含信道状态信息CSI反馈模型当前的性能信息。The transceiver module 1201 is configured to receive first indication information, wherein the first indication information includes current performance information of a channel state information CSI feedback model.

向终端设备发送模型当前对应的第一输出；或者，Sending the first output currently corresponding to the model to the terminal device; or,

向终端设备发送第一信道状态信息参考信号CSI-RS，其中，第一CSI-RS为由第一输出对应的波束赋形后的CSI-RS。A first channel state information reference signal CSI-RS is sent to a terminal device, wherein the first CSI-RS is a CSI-RS after beamforming corresponding to the first output.

接收终端设备发送的第一信息获取请求，其中，第一信息为模型的第一输出或第一输出的关联信息。Receive a first information acquisition request sent by a terminal device, wherein the first information is a first output of the model or associated information of the first output interest.

在一些可能的实现方式中，In some possible implementations,

收发模块1201，还用于发送第二指示信息，其中，第二指示信息中包含模型关联的监测参数；或者，The transceiver module 1201 is further configured to send a second indication message, wherein the second indication message includes a monitoring parameter associated with the model; or

收发模块1201，还用于接收终端设备发送的模型关联的监测参数。The transceiver module 1201 is also used to receive model-associated monitoring parameters sent by the terminal device.

在一些可能的实现方式中，可以通过以下一项或多项发送第二指示信息：无线资源控制RRC消息，媒体接入控制控制单元MAC-CE,下行控制信息DCI。In some possible implementations, the second indication information may be sent via one or more of the following: a radio resource control RRC message, a media access control control element MAC-CE, and downlink control information DCI.

在数据传输秩大于1的情况下，确定模型在每个数据传输层对应的性能信息，和/或确定模型在所有传输层的性能信息。When the data transmission rank is greater than 1, the performance information corresponding to the model at each data transmission layer is determined, and/or the performance information of the model at all transmission layers is determined.

在一些可能的实现方式中，性能信息包括以下至少一项：In some possible implementations, the performance information includes at least one of the following:

第一性能参数，其中，第一性能参数为终端设备根据模型当前的第一输入与第一信息确定的；A first performance parameter, wherein the first performance parameter is determined by the terminal device according to a current first input and first information of the model;

在一些可能的实现方式中，处理模块1202，还用于确定第一指示信息的发送模式，其中，发送模式包括以下至少一项：发送频率，发送次数，发送内容；In some possible implementations, the processing module 1202 is further configured to determine a sending mode of the first indication information, wherein the sending mode includes at least one of the following: sending frequency, sending times, and sending content;

收发模块1201，还用于基于发送模式，接收第一指示信息。The transceiver module 1201 is further configured to receive first indication information based on the sending mode.

在一些可能的实现方式中，In some possible implementations,

处理模块1202，还用于根据协议约定，确定发送模式；或者，The processing module 1202 is further configured to determine a sending mode according to a protocol agreement; or,

收发模块1201，还用于向终端设备指示发送模式；或者，The transceiver module 1201 is further configured to indicate a sending mode to the terminal device; or,

接收终端设备发送的发送模式。Receive the transmission mode sent by the terminal device.

在一些可能的实现方式中，处理模块1202，还用于In some possible implementations, the processing module 1202 is further configured to:

根据CSI反馈模型当前的性能信息，对CSI反馈模型执行以下操作中的任一项：激活，去激活，更新，切换。According to the current performance information of the CSI feedback model, any one of the following operations is performed on the CSI feedback model: activation, deactivation, update, switching.

综上，网络设备可以通过接收包含信道状态信息CSI反馈模型当前的性能信息对应的第一指示信息，准确地获取CSI反馈模型当前的性能。In summary, the network device can accurately obtain the current performance of the CSI feedback model by receiving the first indication information corresponding to the current performance information of the channel state information CSI feedback model.

请参见图13，图13是本公开实施例提供的另一种通信装置1300的结构示意图。通信装置1300可以是终端设备，也可以是网络设备，还可以是支持终端设备实现上述方法的芯片、芯片系统、或处理器等，也可以是支持网络设备实现上述方法的芯片、芯片系统、或处理器等。该通信装置1300可用于实现上述方法实施例中描述的方法，具体可以参见上述方法实施例中的说明。Please refer to Figure 13, which is a schematic diagram of the structure of another communication device 1300 provided in an embodiment of the present disclosure. The communication device 1300 can be a terminal device, or a network device, or a chip, a chip system, or a processor that supports the terminal device to implement the above method, or a chip, a chip system, or a processor that supports the network device to implement the above method. The communication device 1300 can be used to implement the method described in the above method embodiment, and the details can be referred to the description in the above method embodiment.

通信装置1300可以包括一个或多个处理器1301。处理器1301可以是通用处理器或者专用处理器等。例如可以是基带处理器或中央处理器。基带处理器可以用于对通信协议以及通信数据进行处理，中央处理器可以用于对通信装置(如，网络设备、基带芯片，终端设备、终端设备芯片，DU或CU等)进行控制，执行计算机程序，处理计算机程序的数据。The communication device 1300 may include one or more processors 1301. The processor 1301 may be a general-purpose processor or a dedicated processor. For example, it may be a baseband processor or a central processing unit. The baseband processor may be used to process the communication protocol and the communication data. The central processing unit can be used to control communication devices (such as network equipment, baseband chips, terminal equipment, terminal equipment chips, DU or CU, etc.), execute computer programs, and process data of computer programs.

在一些可能的实现形式中，通信装置1300中还可以包括一个或多个存储器1302，其上可以存有计算机程序1304，存储器1302执行计算机程序1304，以使得通信装置1300执行上述方法实施例中描述的方法。在一些可能的实现形式中，存储器1302中还可以存储有数据。通信装置1300和存储器1302可以单独设置，也可以集成在一起。In some possible implementation forms, the communication device 1300 may further include one or more memories 1302, on which a computer program 1304 may be stored, and the memory 1302 executes the computer program 1304 so that the communication device 1300 performs the method described in the above method embodiment. In some possible implementation forms, data may also be stored in the memory 1302. The communication device 1300 and the memory 1302 may be provided separately or integrated together.

在一些可能的实现形式中，通信装置1300还可以包括收发器1305、天线1306。收发器1305可以称为收发单元、收发机、或收发电路等，用于实现收发功能。收发器1305可以包括接收器和发送器，接收器可以称为接收机或接收电路等，用于实现接收功能；发送器可以称为发送机或发送电路等，用于实现发送功能。In some possible implementation forms, the communication device 1300 may further include a transceiver 1305 and an antenna 1306. The transceiver 1305 may be referred to as a transceiver unit, a transceiver, or a transceiver circuit, etc., and is used to implement a transceiver function. The transceiver 1305 may include a receiver and a transmitter, the receiver may be referred to as a receiver or a receiving circuit, etc., and is used to implement a receiving function; the transmitter may be referred to as a transmitter or a transmitting circuit, etc., and is used to implement a transmitting function.

在一些可能的实现形式中，通信装置1300中还可以包括一个或多个接口电路1307。接口电路1307用于接收代码指令并传输至处理器1301。处理器1301运行代码指令以使通信装置1300执行上述方法实施例中描述的方法。In some possible implementation forms, the communication device 1300 may further include one or more interface circuits 1307. The interface circuit 1307 is used to receive code instructions and transmit them to the processor 1301. The processor 1301 executes the code instructions to enable the communication device 1300 to perform the method described in the above method embodiment.

在一种实现方式中，处理器1301中可以包括用于实现接收和发送功能的收发器。例如该收发器可以是收发电路，或者是接口，或者是接口电路。用于实现接收和发送功能的收发电路、接口或接口电路可以是分开的，也可以集成在一起。上述收发电路、接口或接口电路可以用于代码/数据的读写，或者，上述收发电路、接口或接口电路可以用于信号的传输或传递。In one implementation, the processor 1301 may include a transceiver for implementing the receiving and sending functions. For example, the transceiver may be a transceiver circuit, an interface, or an interface circuit. The transceiver circuit, interface, or interface circuit for implementing the receiving and sending functions may be separate or integrated. The above-mentioned transceiver circuit, interface, or interface circuit may be used for reading and writing code/data, or the above-mentioned transceiver circuit, interface, or interface circuit may be used for transmitting or delivering signals.

在一种实现方式中，处理器1301可以存有计算机程序1303，计算机程序1303在处理器1301上运行，可使得通信装置1300执行上述方法实施例中描述的方法。计算机程序1303可能固化在处理器1301中，该种情况下，处理器1301可能由硬件实现。In one implementation, the processor 1301 may store a computer program 1303, which runs on the processor 1301 and enables the communication device 1300 to perform the method described in the above method embodiment. The computer program 1303 may be fixed in the processor 1301, in which case the processor 1301 may be implemented by hardware.

在一种实现方式中，通信装置800可以包括电路，电路可以实现前述方法实施例中发送或接收或者通信的功能。本公开中描述的处理器和收发器可实现在集成电路(integrated circuit，IC)、模拟IC、射频集成电路RFIC、混合信号IC、专用集成电路(application specific integrated circuit，ASIC)、印刷电路板(printed circuit board，PCB)、电子设备等上。该处理器和收发器也可以用各种IC工艺技术来制造，例如互补金属氧化物半导体(complementary metal oxide semiconductor，CMOS)、N型金属氧化物半导体(nMetal-oxide-semiconductor，NMOS)、P型金属氧化物半导体(positive channel metal oxide semiconductor，PMOS)、双极结型晶体管(bipolar junction transistor，BJT)、双极CMOS(BiCMOS)、硅锗(SiGe)、砷化镓(GaAs)等。In one implementation, the communication device 800 may include a circuit that can implement the functions of sending or receiving or communicating in the aforementioned method embodiments. The processor and transceiver described in the present disclosure may be implemented in an integrated circuit (IC), an analog IC, a radio frequency integrated circuit RFIC, a mixed signal IC, an application specific integrated circuit (ASIC), a printed circuit board (PCB), an electronic device, etc. The processor and transceiver may also be manufactured using various IC process technologies, such as complementary metal oxide semiconductor (CMOS), N-type metal oxide semiconductor (NMOS), P-type metal oxide semiconductor (positive channel metal oxide semiconductor, PMOS), bipolar junction transistor (BJT), bipolar CMOS (BiCMOS), silicon germanium (SiGe), gallium arsenide (GaAs), etc.

以上实施例描述中的通信装置可以是终端设备，但本公开中描述的通信装置的范围并不限于此，而且通信装置的结构可以不受图13的限制。通信装置可以是独立的设备或者可以是较大设备的一部分。例如通信装置可以是：The communication device described in the above embodiments may be a terminal device, but the scope of the communication device described in the present disclosure is not limited thereto, and the structure of the communication device may not be limited by FIG. 13. The communication device may be an independent device or may be part of a larger device. For example, the communication device may be:

(1)独立的集成电路IC，或芯片，或，芯片系统或子系统；(1) Independent integrated circuit IC, or chip, or chip system or subsystem;

(2)具有一个或多个IC的集合，在一些可能的实现形式中，该IC集合也可以包括用于存储数据，计算机程序的存储部件；(2) having a set of one or more ICs, which, in some possible implementations, may also include a storage component for storing data and computer programs;

(3)ASIC，例如调制解调器(Modem)；(3) ASIC, such as modem;

(4)可嵌入在其他设备内的模块；(4) Modules that can be embedded in other devices;

(5)接收机、终端设备、智能终端设备、蜂窝电话、无线设备、手持机、移动单元、车载设备、网络设备、云设备、人工智能设备等等；(5) Receivers, terminal devices, intelligent terminal devices, cellular phones, wireless devices, handheld devices, mobile units, vehicle-mounted devices, network devices, cloud devices, artificial intelligence devices, etc.;

(6)其他等等。(6)Others

对于通信装置可以是芯片或芯片系统的情况，请参见图14，为本公开实施例中提供的一种芯片的结构图。For the case where the communication device may be a chip or a chip system, please refer to FIG. 14 , which is a structural diagram of a chip provided in an embodiment of the present disclosure.

芯片1400包括处理器1401和接口1403。其中，处理器1401的数量可以是一个或多个，接口1403的数量可以是多个。The chip 1400 includes a processor 1401 and an interface 1403. The number of the processor 1401 may be one or more, and the number of the interface 1403 may be multiple.

对于芯片用于实现本公开实施例中终端设备、网络设备、BSF实体或UDM实体的功能的情况：For the case where the chip is used to implement the functions of the terminal device, network device, BSF entity or UDM entity in the embodiments of the present disclosure:

接口1403，用于接收代码指令并传输至处理器。Interface 1403 is used to receive code instructions and transmit them to the processor.

处理器1401，用于运行代码指令以执行如上面一些实施例的模型性能的监测方法。Processor 1401 is used to run code instructions to execute the model performance monitoring method as described in some of the above embodiments.

在一些可能的实现形式中，芯片1400还包括存储器1402，存储器1402用于存储必要的计算机程序和数据。In some possible implementation forms, the chip 1400 further includes a memory 1402 , and the memory 1402 is used to store necessary computer programs and data.

本领域技术人员还可以了解到本公开实施例列出的各种说明性逻辑块(illustrative logical block)和步骤(step)可以通过电子硬件、电脑软件，或两者的结合进行实现。这样的功能是通过硬件还是软件来实现取决于特定的应用和整个系统的设计要求。本领域技术人员可以对于每种特定的应用，可以使用各种方法实现的功能，但这种实现不应被理解为超出本公开实施例保护的范围。Those skilled in the art can also understand that the various illustrative logical blocks and steps listed in the embodiments of the present disclosure can be implemented by electronic hardware, computer software, or a combination of both. Whether such functions are implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art can The functions can be implemented using various methods, but such implementation should not be understood as exceeding the scope of protection of the embodiments of the present disclosure.

本公开实施例还提供一种通信系统，该系统包括前述图13实施例中作为终端设备的通信装置、作为网络设备的通信装置的通信装置，或者，该系统包括前述图14实施例中作为终端设备的通信装置、作为网络设备的通信装置的通信装置。An embodiment of the present disclosure also provides a communication system, which includes the communication device in the embodiment of Figure 13 as a terminal device and the communication device in the embodiment of network equipment, or the system includes the communication device in the embodiment of Figure 14 as a terminal device and the communication device in the embodiment of network equipment.

本公开还提供一种可读存储介质，其上存储有指令，该指令被计算机执行时实现上述任一方法实施例的功能。The present disclosure also provides a readable storage medium having instructions stored thereon, which implement the functions of any of the above method embodiments when executed by a computer.

本公开还提供一种计算机程序产品，该计算机程序产品被计算机执行时实现上述任一方法实施例的功能。The present disclosure also provides a computer program product, which implements the functions of any of the above method embodiments when executed by a computer.

在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。计算机程序产品包括一个或多个计算机程序。在计算机上加载和执行计算机程序时，全部或部分地产生按照本公开实施例的流程或功能。计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。计算机程序可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，计算机程序可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(digital subscriber line，DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。可用介质可以是磁性介质(例如，软盘、硬盘、磁带)、光介质(例如，高密度数字视频光盘(digital video disc，DVD))、或者半导体介质(例如，固态硬盘(solid state disk，SSD))等。In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs. When the computer program is loaded and executed on a computer, the process or function according to the embodiment of the present disclosure is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer program can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer program can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated. Available media can be magnetic media (e.g., floppy disks, hard disks, tapes), optical media (e.g., high-density digital video discs (DVD)), or semiconductor media (e.g., solid state disks (SSD)), etc.

本领域普通技术人员可以理解：本公开中涉及的第一、第二等各种数字编号仅为描述方便进行的区分，并不用来限制本公开实施例的范围，也表示先后顺序。Those skilled in the art can understand that the various numerical numbers such as first and second involved in the present disclosure are only used for distinction for convenience of description and are not used to limit the scope of the embodiments of the present disclosure, and also indicate the order of precedence.

本公开中的至少一个还可以描述为一个或多个，多个可以是两个、三个、四个或者更多个，“和/或”，描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B的情况，其中A，B可以是单数或者复数。字符“/”一般表示前后关联对象是一种“或”的关系。“以下至少一项(个)”或其类似表达，是指的这些项中的任意组合，包括单项(个)或复数项(个)的任意组合。例如，a、b和c中的至少一项(个)，可以表示：a，或，b，或，c，或，a和b，或，a和c，或，b和c，或，a、b和c。其中a、b和c分别可以是单个，也可以是多个。本公开不做限制。在本公开实施例中，对于一种技术特征，通过“第一”、“第二”、“第三”、“A”、“B”、“C”和“D”等区分该种技术特征中的技术特征，该“第一”、“第二”、“第三”、“A”、“B”、“C”和“D”描述的技术特征间无先后顺序或者大小顺序。At least one in the present disclosure can also be described as one or more, and multiple can be two, three, four or more. "And/or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and/or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character "/" generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b and c can represent: a, or b, or c, or a and b, or a and c, or b and c, or a, b and c. Where a, b and c can be single or multiple, respectively. The present disclosure is not limited. In the embodiments of the present disclosure, for a technical feature, the technical features in the technical feature are distinguished by "first", "second", "third", "A", "B", "C" and "D", and there is no order of precedence or order of size between the technical features described by the "first", "second", "third", "A", "B", "C" and "D".

本公开中各表所示的对应关系可以被配置，也可以是预定义的。各表中的信息的取值仅仅是举例，可以配置为其他值，本公开并不限定。在配置信息与各参数的对应关系时，并不一定要求必须配置各表中示意出的所有对应关系。例如，本公开中的表格中，某些行示出的对应关系也可以不配置。又例如，可以基于上述表格做适当的变形调整，例如，拆分，合并等等。上述各表中标题示出参数的名称也可以采用通信装置可理解的其他名称，其参数的取值或表示方式也可以通信装置可理解的其他取值或表示方式。上述各表在实现时，也可以采用其他的数据结构，例如可以采用数组、队列、容器、栈、线性表、指针、链表、树、图、结构体、类、堆、散列表或哈希表等。The corresponding relationships shown in the tables in the present disclosure can be configured or predefined. The values of the information in each table are only examples and can be configured as other values, which are not limited by the present disclosure. When configuring the corresponding relationship between the information and each parameter, it is not necessarily required to configure all the corresponding relationships illustrated in each table. For example, in the table in the present disclosure, the corresponding relationships shown in some rows may not be configured. For another example, appropriate deformation adjustments can be made based on the above table, such as splitting, merging, etc. The names of the parameters shown in the titles of the above tables can also use other names that can be understood by the communication device, and the values or representations of the parameters can also be other values or representations that can be understood by the communication device. When implementing the above tables, other data structures can also be used, such as arrays, queues, containers, stacks, linear lists, pointers, linked lists, trees, graphs, structures, classes, heaps, hash tables or hash tables.

本公开中的预定义可以理解为定义、预先定义、存储、预存储、预协商、预配置、固化、或预烧制。The predefined in the present disclosure may be understood as defined, predefined, stored, pre-stored, pre-negotiated, pre-configured, solidified, or pre-burned.

本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本公开的范围。Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this disclosure.

所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

以上，仅为本公开的具体实施方式，但本公开的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本公开揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本公开的保护范围之内。因此，本公开的保护范围应以所述权利要求的保护范围为准。 The above are only specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

A method for monitoring model performance, characterized in that it is executed by a terminal device, and the method comprises:

Monitor the model to obtain channel state information CSI currently associated with the model, wherein the CSI includes any one of the following: first information, a current first input of the model and the first information; wherein the first information is a current first output of the model or associated information of the first output, and the model is a CSI feedback model;

Determining current performance information of the model according to the CSI currently associated with the model;

Sending first indication information, wherein the first indication information includes current performance information of the model.

The method according to claim 1, characterized in that the obtaining of the channel state information CSI currently associated with the model comprises:

In a case where the terminal device side includes a CSI recovery partial model, determining an output of the CSI recovery partial model as the first output, wherein the CSI recovery partial model on the terminal device side is the same as or different from the CSI recovery partial model on the network device side; or

In a case where the terminal device side does not include a CSI recovery partial model, receiving the first output sent by the network device; or,

In a case where the terminal device side does not include a CSI recovery partial model, the associated information is determined based on a received first channel state information reference signal CSI-RS, wherein the first CSI-RS is a CSI-RS after beamforming corresponding to the first output.

The method according to claim 2, further comprising:

When the terminal device side does not include the CSI recovery partial model, a first information acquisition request is sent.

The method according to claim 2, further comprising:

The first input is determined according to the received second CSI-RS.

The method according to claim 1, further comprising:

receiving second indication information, wherein the second indication information includes monitoring parameters associated with the model; or

Determine monitoring parameters associated with the model based on protocol agreement; or,

The monitoring parameters associated with the model are sent to the network device.

The method according to claim 5, characterized in that the sending of the monitoring parameters associated with the model to the network device comprises:

Based on a preset event trigger, the monitoring parameters associated with the model are sent to the network device.

The method according to claim 5 or 6, characterized in that the monitoring parameters include one or more of the following:

Monitoring start time, number of monitoring results reported, monitoring time information and monitoring cycle.

The method according to any one of claims 5 or 7, characterized in that the receiving the second indication information comprises:

The second indication information is received through one or more of the following: a radio resource control RRC message, a media access control control element MAC-CE, and downlink control information DCI.

The method according to claim 1, wherein determining the current performance information of the model according to the CSI currently associated with the model comprises:

When the data transmission rank is greater than 1, determining the CSI corresponding to the model in each data transmission layer;

According to the CSI corresponding to each transmission layer, the performance information of the model in each transmission layer is determined, and/or the performance information of the model in all transmission layers is determined.

The method according to any one of claims 1 to 9, wherein determining the current performance information of the model according to the CSI currently associated with the model comprises:

A current first performance parameter of the model is determined according to the first input and the first information.

Determining a current first performance parameter of the model according to the first input and the first information;

A comparison result between the current first performance parameter and the first threshold value is determined as the current second performance parameter of the model.

Determining a reference parameter based on the received codebook parameter information and the first input;

The reference parameter and the first performance parameter, or a comparison result between the reference parameter and the first performance parameter, is determined as a current third performance parameter of the model.

The method according to claim 12, wherein determining the reference parameter based on the received codebook parameter information and the first input comprises:

Determine a compressed first CSI feedback payload size corresponding to the model and a second CSI feedback payload size determined by the codebook parameter information;

determining an absolute value of a difference between the second CSI feedback payload size and the first CSI feedback payload size;

The reference parameter is determined based on a codebook parameter combination with a minimum corresponding absolute value and the first input.

determining a first distribution offset of the first input relative to a second input in training data corresponding to the model, or

a second distribution offset of the first output relative to a second output in the training data;

The first distribution offset or the second distribution offset is determined as a current fourth performance parameter of the model.

Determine a first distribution offset of the first input relative to a second input in the training data corresponding to the model, and

a second distribution offset of the first output relative to the second output in the training data;

The comparison result of the first distribution offset and the second threshold value, or the comparison result of the second distribution offset and the third threshold value is

The comparison result of the limit values is determined as the current fifth performance parameter of the model.

Determine a current transmission parameter of the system according to the first information, wherein the transmission parameter includes at least one of the following:

System throughput, network block error rate BLER, assumed BLER, acknowledgment character ACK, non-acknowledgment character NACK;

A sixth performance parameter corresponding to the model is determined according to the current transmission parameter.

Determine the current transmission parameters of the system according to the first information, wherein the transmission parameters include at least one of the following: system throughput, network block error rate BLER, assumed BLER, confirmation character ACK, non-confirmation character NACK;

A seventh performance parameter corresponding to the model is determined according to a comparison result between the current transmission parameter and the fourth threshold value.

Determine a first comparison result between the current performance information and a fifth threshold value;

An eighth performance parameter corresponding to the model is determined according to K consecutive first comparison results of the model, where K is a natural number.

Determine the ninth threshold value corresponding to the model according to the K consecutive first comparison results of the model and the second comparison result of the sixth threshold value. Performance parameters, where K is a natural number.

The method according to any one of claims 1 to 19, wherein monitoring the model comprises:

At least M models out of N models are monitored, where N is the number of models deployed in the terminal device and M is a value less than or equal to N.

The method according to any one of claims 1 to 19, further comprising:

Determine a second CSI feedback payload size determined by the selected codebook parameter combination and a compressed first CSI feedback payload size corresponding to each model in the N models;

A model whose corresponding first CSI feedback load size among the N models is less than or equal to the second CSI feedback load size, or a model whose absolute value of the difference between the model and the first CSI feedback load size is the smallest, is determined as the model to be monitored.

The method according to any one of claims 1 to 21, wherein sending the first indication information comprises:

Determine a sending mode of the first indication information, wherein the sending mode includes at least one of the following: sending frequency, sending times, and sending content;

Based on the sending mode, the first indication information is sent.

The method according to claim 22, wherein determining a sending mode of the first indication information comprises:

determining the sending mode according to an instruction of the network device; or,

Determine the sending mode according to the agreement; or,

The sending mode is determined according to the current performance information of the model.

A method for monitoring model performance, characterized in that the method is executed by a network device, and the method comprises:

First indication information is received, wherein the first indication information includes current performance information of a channel state information (CSI) feedback model.

The method of claim 24, further comprising:

Sending the first output currently corresponding to the model to the terminal device; or,

A first channel state information reference signal CSI-RS is sent to the terminal device, wherein the first CSI-RS is a CSI-RS after beamforming corresponding to the first output.

The method of claim 24, further comprising:

A first information acquisition request sent by the terminal device is received, wherein the first information is a first output of the model or associated information of the first output.

The method of claim 24, further comprising:

Sending second indication information, wherein the second indication information includes monitoring parameters associated with the model; or,

Receive monitoring parameters associated with the model sent by the terminal device.

The method according to claim 27, wherein sending the second indication information comprises:

The second indication information is sent through one or more of the following: a radio resource control RRC message, a media access control control element MAC-CE, and downlink control information DCI.

The method according to claim 27 or 28, characterized in that the monitoring parameters include one or more of the following:

The method according to any one of claims 24 to 29, further comprising:

When the data transmission rank is greater than 1, the performance information corresponding to the model in each data transmission layer is determined, and/or the performance information of the model in all transmission layers is determined.

The method according to any one of claims 24 to 30, wherein the performance information includes at least one of the following:

A first performance parameter, wherein the first performance parameter is determined by the terminal device according to a current first input and first information of the model;

A comparison result of the first performance parameter and a first threshold value;

A reference parameter and the first performance parameter, wherein the reference parameter is a parameter determined by the terminal device based on the received codebook parameter information and the first input;

a comparison result between the reference parameter and the first performance parameter;

A first distribution offset, wherein the first distribution offset is an offset between the first input and a second input in the training data corresponding to the model;

A second distribution offset, wherein the second distribution offset is an offset between the first output and a second output in the training data corresponding to the model;

a fifth performance parameter, wherein the fifth performance parameter is a comparison result of the first distribution offset and a second threshold value, or a comparison result of the second distribution offset and a third threshold value;

A sixth performance parameter, wherein the sixth performance parameter is determined by the terminal device based on a current transmission parameter of the system, wherein the transmission parameter includes at least one of the following: system throughput, network block error rate BLER, assumed BLER, confirmation character ACK, non-confirmation character NACK;

A comparison result of the sixth performance parameter and the fourth threshold value;

an eighth performance parameter, wherein the eighth performance parameter is a first comparison result of K consecutive performance information of the model and the fifth threshold value respectively;

A ninth performance parameter, wherein the ninth performance parameter is a second comparison result of K consecutive first comparison results of the model with a sixth threshold value, wherein the first comparison result is a comparison result of each consecutive performance information of the model with a fifth threshold value.

The method according to any one of claims 24 to 31, wherein receiving the first indication information comprises:

Based on the sending mode, the first indication information is received.

The method according to claim 32, wherein determining a sending mode of the first indication information comprises:

Determine the sending mode according to the agreement; or,

indicating the sending mode to the terminal device; or,

Receive the sending mode sent by the terminal device.

The method according to any one of claims 24 to 33, further comprising:

According to the current performance information of the CSI feedback model, any one of the following operations is performed on the CSI feedback model: activation, deactivation, updating, and switching.

A terminal device, characterized by comprising:

A processing module, configured to monitor the model and obtain channel state information CSI currently associated with the model, wherein the CSI includes any one of the following: first information, a current first input of the model and the first information; wherein the first information is a current first output of the model or associated information of the first output, and the model is a CSI feedback model;

The processing module is further configured to determine current performance information of the model according to the CSI currently associated with the model;

The transceiver module is also used to send first indication information, wherein the first indication information includes current performance information of the model.

A network device, comprising:

The transceiver module is used to receive first indication information, wherein the first indication information includes current performance information of a channel state information CSI feedback model.

A communication device, characterized in that the device comprises a processor and a memory, the memory stores a computer program, the processor executes the computer program stored in the memory so that the device performs the method as described in any one of claims 1 to 23, or the processor executes the computer program stored in the memory so that the device performs the method as described in any one of claims 24 to 34.

A communication system, characterized in that it comprises a terminal device and a network device;

The terminal device is used to execute the method as claimed in any one of claims 1 to 23, and the network device is used to execute the method as claimed in any one of claims 24 to 34.

A computer-readable storage medium for storing instructions, which, when executed, enables the method according to any one of claims 1 to 23 to be implemented, or, when executed, enables the method according to any one of claims 24 to 34 to be implemented.