TWI828324B - Parameters adjustment system and method in wireless communication based on online reinforcement learning - Google Patents

Abstract

A parameters adjustment system and method in wireless communication based on online reinforcement learning are provided, the system includes a transceiver, a storage medium and a processor. The processor is coupled to the storage medium and the transceiver, accesses and executes mobile network data collection module, feature construction module, neural network training module. The mobile network data collection module obtains mobile network operation status data of a mobile network system. The feature construction module extracts the mobile network operation status data to construct an original signal strength distribution and a distribution of user clusters. The neural network training module allows each base station to actually transmit according to an acquired wireless communication parameters of each base station , update the original signal strength distribution and obtain user quality feedback to correct neural network parameters of the neural network training module.

Description

Wireless communication parameter adjustment system and method based on online reinforcement learning

The present invention relates to a wireless communication parameter adjustment system and a method thereof, and in particular, to a wireless communication parameter adjustment system and method based on online reinforcement learning.

Due to the increasing user demand for mobile network data, mobile communication networks have developed towards Heterogeneous Network (HetNet) networks. Multi-level and high-density deployment methods are a development trend for operators to improve spectrum efficiency per unit area. . However, in an environment where high-density base stations are densely deployed, the problem of interference between base stations is one of the important issues that operators need to face. When the interference between base stations increases, it will lead to a reduction in the transmission quality and spectrum efficiency of the system, and thus Decrease the overall user experience. For example, when a user watches a video using a mobile communication network, there may be a delay (lag) and the video cannot be played smoothly, or there may be sound but no picture, etc., thus affecting the user's experience.

In addition, in a high-density base station deployment environment, small cell base stations (Small Cell) are one of the options for operators. They are characterized by low power, plug-and-play, easy deployment, etc. However, this is also why operators are interested in Its management and control is difficult, and because the functions of miniaturized base stations are relatively simple, although they have plug-and-play functions, they are also prone to causing interference to other surrounding base stations.

The present invention provides a wireless communication parameter adjustment system and method based on online reinforcement learning, which not only improves wireless communication user speed, but also improves network system performance.

The invention provides a wireless communication parameter adjustment system based on online reinforcement learning, which is suitable for mobile network systems. The mobile network system includes multiple base stations, and the wireless communication parameter adjustment system includes a transceiver, a storage medium and a processor. Storage media stores multiple modules. The processor is coupled to the storage medium and the transceiver, and accesses and executes the modules, where the modules include a mobile network data collection module, a feature construction module, and a neural network training module. The mobile network data collection module is used to obtain the mobile network operation status data of the mobile network system. The feature construction module is electrically connected to the mobile network data collection module, and extracts mobile network operation status data to construct the original signal strength distribution and user gathering location distribution at a point in time. The neural network training module is electrically connected to the feature construction module. The neural network training module is used to obtain the wireless communication parameter configuration probability distribution of each base station based on the original signal strength distribution and user gathering location distribution, so that each base station can be configured based on The wireless communication parameter configuration probability distribution is actually transmitted, the original signal strength distribution is updated and user quality feedback is obtained to correct the neural network parameters of the neural network training module.

In an embodiment of the present invention, in the above-mentioned wireless communication parameter adjustment system based on online reinforcement learning, the neural network training module is further used to adjust the parameters based on the updated original signal strength distribution, user quality feedback, original signal strength distribution and Users gather to perform neural network training in a distributed manner to output the optimal wireless communication parameter configuration of each base station to the mobile network system for deployment optimization.

In an embodiment of the present invention, in the wireless communication parameter adjustment system based on online reinforcement learning, the mobile network operation status data at least includes base station configuration management information, performance management information and user terminal measurement report information.

In an embodiment of the present invention, the above-mentioned wireless communication parameter adjustment system based on online reinforcement learning, wherein the wireless communication parameter adjustment system further includes a Queue collection module electrically connected to the mobile network data collection module, and the Queue collection module Groups are used to organize mobile network operation status data into multi-dimensional arrays.

In an embodiment of the present invention, in the above-mentioned wireless communication parameter adjustment system based on online reinforcement learning, the neural network training module is used to obtain the wireless communication parameter configuration of each base station based on the original signal strength distribution and user gathering location distribution. Probability distribution, allowing each base station to actually transmit according to the probability distribution configured by wireless communication parameters, updating the original signal strength distribution and obtaining user quality feedback to correct the neural network parameters of the neural network training module, including the neural network The training module is also used to organize the mobile network operating status data into a multi-dimensional array through the Queue collection module, and then converts it into a three-dimensional array after convolutional neural network operations and pooling operations. The neural network training module is also used to The arg max operator is used to obtain the wireless communication parameter configuration probability distribution of each base station for actual transmission, update the original signal strength distribution and obtain user quality feedback. The neural network training module is also used to pass the updated original signal strength distribution through Another set of convolutional neural network operations and pooling operations on the original signal intensity distribution are converted into a three-dimensional array. The neural network training module is also used to take the max operator to obtain the optimal wireless communication parameter configuration probability distribution, and the optimal wireless The communication parameter configuration probability distribution and user quality feedback are nonlinearly combined, and then combined with the original signal strength distribution to form an array for mean square error calculation. The neural network training module is also used to output the beam direction corresponding to the probability matrix and then execute To obtain scalar entropy, the neural network training module is further used to shrink the mean square error to obtain the probability function and sampling weight, and the neural network training module is further used to repeat the above steps to collect training neural network data for correction. Neural network parameters of the neural network training module.

A wireless communication parameter adjustment method based on online reinforcement learning of the present invention is suitable for mobile network systems. The mobile network system includes multiple base stations. The wireless communication parameter adjustment method includes: obtaining the mobile network operating status of the mobile network system. Data; extract the mobile network operation status data to construct the original signal strength distribution and user gathering place distribution at a point in time; and obtain the wireless communication parameter configuration probability distribution of each base station based on the original signal strength distribution and user gathering place distribution. Let each base station actually transmit according to the probability distribution configured by wireless communication parameters, update the original signal strength distribution and obtain user quality feedback to correct the neural network parameters.

In an embodiment of the present invention, the above-mentioned wireless communication parameter adjustment method based on online reinforcement learning further includes: executing based on the updated original signal strength distribution, user quality feedback, original signal strength distribution and user gathering place distribution. Neural network training is used to output the optimal wireless communication parameter configuration of each base station to the mobile network system for deployment optimization.

In an embodiment of the present invention, in the wireless communication parameter adjustment method based on online reinforcement learning, the mobile network operation status data at least includes base station configuration management information, performance management information and user terminal measurement report information.

In an embodiment of the present invention, the above-mentioned wireless communication parameter adjustment method based on online reinforcement learning further includes: organizing mobile network operation status data into a multi-dimensional array.

In an embodiment of the present invention, the above-mentioned wireless communication parameter adjustment method based on online reinforcement learning obtains the wireless communication parameter configuration probability distribution of each base station based on the original signal strength distribution and user gathering location distribution, so that each base station is configured according to The steps of configuring the probability distribution of wireless communication parameters for actual transmission, updating the original signal strength distribution and obtaining user quality feedback to correct the neural network parameters of the neural network training module also include: organizing the mobile network operation status data into multi-dimensional Array, and converted into a three-dimensional array after convolutional neural network operation and pooling operation; take the arg max operator to obtain the probability distribution of wireless communication parameter configuration of each base station for actual transmission, update the original signal strength distribution and obtain user quality Feedback; convert the updated original signal intensity distribution into a three-dimensional array after undergoing another set of convolutional neural network operations and pooling operations on the original signal intensity distribution; use the max operator to obtain the optimal wireless communication parameter configuration probability distribution, and finally The optimal wireless communication parameter configuration probability distribution is nonlinearly combined with the user quality feedback, and then combined with the original signal strength distribution to form an array for mean square error calculation; the beam direction corresponding to the probability matrix is output and then executed To obtain scalar entropy; shrink the mean square error to obtain the probability function and sampling weight; and repeat the above steps to collect training neural network data to train the neural network training module.

Based on the above, the present invention provides a wireless communication parameter adjustment system and method based on online reinforcement learning, which not only alleviates the GPU (Graphical Processing Unit) memory caused by inputting multi-dimensional data by using a low-cost orthogonal coding method of multi-base station frequency reuse In order to solve the problem of insufficient space and need to reduce the resolution, we collect relevant information about base stations and user terminals in a periodic and centralized manner, and consider the integrity of the system to optimize the network performance of multiple base stations in the same area. , it also introduces an online learning mechanism with experience transfer, allowing the neural network to learn more quickly and modify the neural network parameters to adapt to environmental changes, improving wireless communication user speeds, and at the same time automatically collecting data on base station operations to use artificial intelligence This method instantly selects the optimal wireless communication parameter configuration to reduce operator maintenance costs and improve network system performance.

In order to make the above-mentioned features and advantages of the present invention more obvious and easy to understand, embodiments are given below and described in detail with reference to the accompanying drawings.

Some embodiments of the present invention will be described in detail with reference to the accompanying drawings. The component symbols cited in the following description will be regarded as the same or similar components when the same component symbols appear in different drawings. These embodiments are only part of the present invention and do not disclose all possible implementations of the present invention. Rather, these embodiments are only examples of methods, electronic devices, and computer-readable storage media within the scope of the patent application of the present invention.

Figure 1 is a schematic diagram of a wireless communication parameter adjustment system based on online reinforcement learning according to the present invention.

Referring to FIG. 1 , the wireless communication parameter adjustment system 10 based on online reinforcement learning includes a transceiver 110 , a storage device 120 and a processor 130 . The wireless communication parameter adjustment system 10 is suitable for a mobile network system 20. The mobile network system 20 includes multiple base stations.

The transceiver 110 transmits and receives signals in a wireless or wired manner. Transceiver 110 may also perform, for example, low noise amplification, impedance matching, mixing, up or down frequency conversion, filtering, amplification, and similar operations.

The storage medium 120 is, for example, any type of fixed or removable random access memory (RAM), read-only memory (ROM), or flash memory. , hard disk drive (HDD), solid state drive (SSD) or similar components or a combination of the above components, the storage device 102 is used to record multiple instructions that can be executed by the processor 130, and is also used to store Multiple modules or various applications that can be executed by the processor 130 . In one embodiment, the storage medium 120 can store multiple modules such as a mobile network data collection module 1201, a Queue collection module 1202, a feature construction module 1203, and a neural network training module 1204. Their functions will be discussed later. instruction.

The processor 130 is, for example, a central processing unit (CPU), or other programmable general-purpose or special-purpose micro control unit (MCU), microprocessor, digital signal processing Digital signal processor (DSP), programmable controller, application specific integrated circuit (ASIC), graphics processing unit (GPU), arithmetic logic unit (ALU) , complex programmable logic device (CPLD), field programmable gate array (FPGA) or other similar components or a combination of the above components. The processor 130 can be coupled to the storage medium 120 and the transceiver 110, and access and execute multiple modules and various applications stored in the storage medium 120 to control the overall operation of the wireless communication parameter adjustment system 10.

The mobile network data collection module 1201 is used to obtain mobile network operation status data of the mobile network system, where the mobile network operation status data may include base station configuration management information, performance management information and user terminal measurement report information. In one embodiment, the mobile network operation status data may include mobile network operation status (such as CM, PM, FM, KPI...), user identification (such as location, service type, traffic volume, movement path...), user identification Terminal measurement reports (such as MR), RF signals, Radio Signal related information (such as Scanner measurements, mobile phone measurements) and other base station related data. The present invention is not limited thereto.

The Queue collection module 1202 is electrically connected to the mobile network data collection module 1201, and is used to organize the above mobile network operation status data into a multi-dimensional array.

The feature construction module 1203 is electrically connected to the mobile network data collection module 1201, and extracts the mobile network operation status data to construct the original signal strength distribution and user gathering place distribution at a point in time.

The neural network training module 1204 is electrically connected to the feature construction module 1203. The neural network training module 1204 is used to obtain the wireless communication parameter configuration probability distribution of each base station based on the original signal strength distribution and user gathering location distribution, so that each The base station actually transmits according to the wireless communication parameter configuration probability distribution, updates the original signal strength distribution and obtains user quality feedback to correct the neural network parameters of the neural network training module 1204.

In the above operation, the neural network training module 1204 is further used to organize the mobile network operation status data into a multi-dimensional array through the Queue collection module 1202, and convert it into a three-dimensional array after the convolutional neural network operation and pooling operation. , take the arg max operator to obtain the wireless communication parameter configuration probability distribution of each base station for actual transmission, update the original signal strength distribution and obtain user quality feedback, and pass the updated original signal strength distribution through another set of original signal strength distributions After the convolutional neural network operation and pooling operation, it is converted into a three-dimensional array. The max operator is used to obtain the probability distribution of the optimal wireless communication parameter configuration. The probability distribution of the optimal wireless communication parameter configuration is nonlinearly combined with the user quality feedback, and then combined with the original The signal intensity distribution is formed into an array for mean square error calculation, and the beam direction corresponding to the probability matrix is output and then executed. To obtain scalar entropy, shrink the mean square error to obtain the probability function and sampling weight, and repeat the above steps to collect training neural network data to train the neural network training module 1204.

In one embodiment, the neural network training module 1204 is further used to perform neural network training based on the updated original signal strength distribution, user quality feedback, original signal strength distribution, and user gathering place distribution to output the optimal signal strength of each base station. The best wireless communication parameters are configured to the mobile network system 20 for deployment optimization.

Figure 2 is a schematic diagram of a wireless communication parameter adjustment method based on online reinforcement learning according to the present invention.

Referring to Figure 2, in step S201, the mobile network data collection module 1201 obtains mobile network operation status data of the mobile network system.

In step S202, the feature construction module 1203 extracts the mobile network operation status data to construct the original signal strength distribution and user gathering place distribution at a point in time.

In step S203, the neural network training module 1204 obtains the wireless communication parameter configuration probability distribution of each base station based on the original signal strength distribution and user gathering place distribution, so that each base station actually transmits according to the wireless communication parameter configuration probability distribution, and updates the original Signal strength distribution and user quality feedback are obtained to modify neural network parameters.

In step S204, the neural network training module 1204 performs neural network training based on the updated original signal strength distribution, user quality feedback, original signal strength distribution, and user gathering location distribution to output the optimal wireless communication parameters of each base station. Configured to the mobile network system 20 for deployment optimization.

Figure 3 is a schematic diagram of training neural network parameters according to a first embodiment of the present invention.

Please refer to Figure 3. In step S301, the current field signal distribution, base station frequency sparse coding and user location are extracted and input into the neural network training module 1204.

In step S302, the wireless communication parameter configuration probability distribution of each base station is obtained for actual transmission, the field signal distribution is updated and user quality feedback is obtained.

In step S303, the updated field signal distribution is input into another neural network to obtain a comprehensive score of signal distribution.

In step S304, a nonlinear weighted score is calculated based on the signal distribution comprehensive score and user quality feedback.

In step S305, the mean square error is calculated based on the nonlinear weighted score and the original score and averaged with the scalar entropy.

In step S306, the mean square error is converted into a probability format and into a sampling weight to train neural network parameters.

Figure 4 is a schematic diagram of training neural network parameters according to a second embodiment of the present invention.

Please refer to Figure 4. In step S401, the mobile network operation status data is organized into a multi-dimensional array through the Queue collection module 1202, and converted into a three-dimensional array after a convolutional neural network operation and a pooling operation.

In step S402, the arg max operator is used to obtain the wireless communication parameter configuration probability distribution of each base station for actual transmission, update the original signal strength distribution and obtain user quality feedback.

Specifically, multiple subprograms (Sub Process) are cut on the GPU (Graphical Processing Unit), and each Sub Process executes the following:

At present, the RF Map formed by the antenna directions of each base station constitutes the "state", and its dimension is B x L x W, where B is the number of base stations, L and W are the field length and width respectively;

Incorporate frequency orthogonal coding and user location between each base station;

Use Epsilon Greedy to perform antenna direction setting: perform Random Action at a low ratio; perform Random Action at a low ratio; Next State is opened as [Batch x B x L x W], where B is the number of base stations, and L and W are the fields respectively. Length and width (removing frequency orthogonal coding and user location information);

The above Sub Process results are composed into a Batch, and its content is [Batch Size x State x Action x Reward x Next State].

In step S403, the updated original signal intensity distribution is converted into a three-dimensional array after undergoing another set of convolutional neural network operations and pooling operations on the original signal intensity distribution.

Specifically, open the previously obtained [Batch Size x Next State] as [Batch x B x L x W], where B is the number of base stations, L and W are the field length and width respectively;

Input [Batch x B x L x W] into the neural network NN and make its output format [Batch x B x Antenna].

In step S404, the max operator is used to obtain the optimal wireless communication parameter configuration probability distribution. The optimal wireless communication parameter configuration probability distribution is nonlinearly combined with the user quality feedback, and then combined with the original signal strength distribution to form an array for mean square error calculation.

Specifically, take the max operation of the respective B elements of [Batch x B x Antenna] to obtain [Batch x B];

Multiply [Batch x B] by alpha and Reward to get [Batch x B]; input State into NN and then take arg max, take out the corresponding element of Action to get [Batch x B]; these two [Batch x B] The matrix takes MSE, and the output dimension is [Batch].

In step S405, output the beam direction corresponding to the probability matrix and then execute to obtain scalar entropy.

Specifically, [Batch x B x Antenna] is obtained in the above step S403. That is to say, for each Batch, [B x Antenna] is a probability distribution. Execute To obtain scalar entropy, its output dimension is [Batch].

In step S406, the mean square error is subjected to probability shrinkage to obtain the probability function and sampling weight.

Specifically, the MSE of [Batch] obtained in step S404 and the scalar entropy obtained in step S405 are added to obtain [Batch]. The probability function is obtained through probability contraction, and the probability function is obtained through Eq. Get the sampling weight.

In step S407, the above steps are repeated to collect training neural network data to train the neural network training module 1204.

Figure 5 is a schematic diagram of training neural network parameters according to a third embodiment of the present invention.

Please refer to Figure 5. In step S501, the original signal intensity distribution and user gathering place distribution are extracted from the distributed Queue and organized into a four-dimensional array. The four-dimensional array is converted into a three-dimensional array after a convolutional neural network operation and a pooling operation. array.

In step S502, the arg max operator is used to obtain the wireless communication parameter configuration probability distribution of each base station for actual transmission, update the original signal strength distribution and obtain user quality feedback.

In step S503, the updated original signal intensity distribution is converted into a three-dimensional array after undergoing another set of convolutional neural network operations and pooling operations on the original signal intensity distribution.

In step S504, the max operator is used to obtain the optimal wireless communication parameter configuration probability distribution. The optimal wireless communication parameter configuration probability distribution is nonlinearly combined with the user quality feedback, and then combined with the original signal strength distribution to form an array for mean square error calculation.

In step S505, the mean square error is subjected to probability shrinkage to obtain a probability function. Get weight.

In step S506, the above steps are repeated to collect training neural network data to train the neural network training module 1204.

Figure 6 is a schematic diagram of training neural network parameters according to a fourth embodiment of the present invention.

Please refer to Figure 6. In step S601, the original signal intensity distribution and user gathering place distribution are extracted from the multi-thread Queue and organized into a four-dimensional array. The four-dimensional array is converted into a three-dimensional array after a convolutional neural network operation and a pooling operation. array.

In step S602, the arg max operator is obtained to obtain the wireless communication parameter configuration probability distribution of each base station for actual transmission, update the original signal strength distribution and obtain user quality feedback.

In step S603, the updated original signal intensity distribution is converted into a three-dimensional array after undergoing another set of convolutional neural network operations and pooling operations on the original signal intensity distribution.

In step S604, the max operator is used to obtain the optimal wireless communication parameter configuration probability distribution. The optimal wireless communication parameter configuration probability distribution is nonlinearly combined with the user quality feedback, and then combined with the original signal strength distribution to form an array for mean square error calculation.

In step S605, output the wireless communication parameter configuration corresponding to the probability matrix and then execute to obtain scalar entropy.

In step S606, the results of steps S604 and S605 are added.

In step S607, the probability function is obtained from the mean square error through probability shrinkage. Get weight.

In step S608, the above steps are repeated to collect training neural network data to train the neural network training module 1204.

It should be noted that in the three-dimensional array described in the above embodiments, the first dimension may be the number of base stations in the same area, and the second and third dimensions may be the length and width of the area, that is, The size of the area.

After the operations of training neural network parameters in the first to fourth embodiments described above in FIGS. 3 to 6 , the neural network parameters of the neural network training module 1204 can be modified, so that the neural network training module 1204 performs neural network training based on the updated original signal strength distribution, user quality feedback, original signal strength distribution, and user gathering location distribution to output the optimal wireless communication parameter configuration of each base station to the mobile network system 20 for deployment. Department optimization.

Based on the above, the present invention provides a wireless communication parameter adjustment system and method based on online reinforcement learning, which not only alleviates the GPU (Graphical Processing Unit) memory caused by inputting multi-dimensional data by using a low-cost orthogonal coding method of multi-base station frequency reuse In order to solve the problem of insufficient space and need to reduce the resolution, we collect relevant information about base stations and user terminals in a periodic and centralized manner, and consider the integrity of the system to optimize the network performance of multiple base stations in the same area. , it also introduces an online learning mechanism with experience transfer, allowing the neural network to learn more quickly and modify the neural network parameters to adapt to environmental changes, improving wireless communication user speeds, and at the same time automatically collecting data on base station operations to use artificial intelligence This method instantly selects the optimal wireless communication parameter configuration to reduce operator maintenance costs and improve network system performance.

Although the disclosure has been disclosed above through embodiments, they are not intended to limit the disclosure. Anyone with ordinary knowledge in the technical field may make slight changes and modifications without departing from the spirit and scope of the disclosure. Therefore, The scope of protection of this disclosure shall be determined by the scope of the appended patent application.

10: Wireless communication parameter adjustment system

110:Transceiver

120:Storage device

130: Processor

20:Mobile network system

1201:Mobile network data collection module

1202:Queue collection module

1203: Feature construction module

1204:Neural network training module

S201, S202, S203, S204, S301, S302, S303, S304, S305, S306, S401, S402, S403, S404, S405, S406, S407, S501, S502, S503, S504, S505, S506, S601, S602 , S603, S604, S605, S606, S607, S608: steps

Figure 1 is a schematic diagram of a wireless communication parameter adjustment system based on online reinforcement learning according to the present invention. Figure 2 is a schematic diagram of a wireless communication parameter adjustment method based on online reinforcement learning according to the present invention. Figure 3 is a schematic diagram of training neural network parameters according to a first embodiment of the present invention. Figure 4 is a schematic diagram of training neural network parameters according to a second embodiment of the present invention. Figure 5 is a schematic diagram of training neural network parameters according to a third embodiment of the present invention. Figure 6 is a schematic diagram of training neural network parameters according to a fourth embodiment of the present invention.

S201, S202, S203, S204: steps

Claims (8)

A wireless communication parameter adjustment system based on online reinforcement learning, suitable for mobile network systems. The mobile network system includes multiple base stations. The wireless communication parameter adjustment system includes: a transceiver; a storage medium that stores multiple modules; and a processor, coupling the storage medium and the transceiver, and accessing and executing the modules, wherein the modules include: a mobile network data collection module to obtain the mobile network of the mobile network system The road operation status data; the feature construction module is electrically connected to the mobile network data collection module, and extracts the mobile network operation status data to construct the original signal strength distribution and user gathering place distribution at a point in time; and The neural network training module is electrically connected to the feature construction module. The neural network training module is used to obtain the wireless communication parameter configuration probability distribution of each base station based on the original signal strength distribution and the user gathering place distribution. , causing each base station to actually transmit according to the wireless communication parameter configuration probability distribution, updating the original signal strength distribution and obtaining user quality feedback to correct the neural network parameters of the neural network training module, where the neural network training The module is further used to perform neural network training based on the updated original signal strength distribution, the user quality feedback, the original signal strength distribution and the user gathering place distribution to output the best wireless communication of each base station. Parameters are configured to the mobile network system for deployment optimization. The wireless communication parameter adjustment system based on online reinforcement learning as described in claim 1, wherein the mobile network operation status data at least includes base station configuration management information, performance management information and user terminal measurement report information. The wireless communication parameter adjustment system based on online reinforcement learning as described in claim 1, wherein the wireless communication parameter adjustment system further includes a Queue collection module electrically connected to the mobile network data collection module, and the Queue collection module Used to organize the mobile network operation status data into a multi-dimensional array. The wireless communication parameter adjustment system based on online reinforcement learning as described in claim 3, wherein the neural network training module is used to obtain the wireless communication parameters of each base station based on the original signal strength distribution and the user gathering place distribution Configure a probability distribution so that each base station actually transmits according to the wireless communication parameter configuration probability distribution, update the original signal strength distribution and obtain the user quality feedback to correct the operation of the neural network parameters of the neural network training module Among them, the neural network training module is further used to organize the mobile network operation status data into the multi-dimensional array through the Queue collection module, and convert it into a three-dimensional array after a convolutional neural network operation and a pooling operation. Array, the neural network training module is further used to obtain the arg max operator to obtain the wireless communication parameter configuration probability distribution of each base station for actual transmission, update the original signal strength distribution and obtain the user quality feedback, The neural network training module is further used to convert the updated original signal intensity distribution into a three-dimensional array through the convolutional neural network operation and the pooling operation of another set of original signal intensity distributions. The neural network The training module is further used to obtain the optimal wireless communication parameter configuration probability distribution by taking the max operator. The optimal wireless communication parameter configuration probability distribution is nonlinearly combined with the user quality feedback, and then combined with the original signal strength distribution to form an array for averaging. Square error calculation, the neural network training module is also used to output the beam direction corresponding to the probability matrix and then execute -sum(Probability*log(Probability)) to obtain the scalar entropy. The neural network training module is also used to The mean square error is probabilistically shrunk to obtain a probability function and a sampling weight to correct the neural network parameters of the neural network training module. A wireless communication parameter adjustment method based on online reinforcement learning, suitable for mobile network systems. The mobile network system includes multiple base stations. The wireless communication parameter adjustment method includes: obtaining the mobile network operation status of the mobile network system. data; extract the mobile network operation status data to construct the original signal strength distribution and user gathering place distribution at a point in time; obtain the wireless communication parameter configuration of each base station based on the original signal strength distribution and the user gathering place distribution Probability distribution, causing each base station to actually transmit according to the probability distribution configured by the wireless communication parameters, updating the original signal strength distribution and obtaining user quality feedback to correct the neural network parameters; and Perform neural network training based on the updated original signal strength distribution, the user quality feedback, the original signal strength distribution and the user gathering place distribution to output the optimal wireless communication parameter configuration of each base station to the mobile network system for deployment optimization. The wireless communication parameter adjustment method based on online reinforcement learning as described in claim 5, wherein the mobile network operation status data at least includes base station configuration management information, performance management information and user terminal measurement report information. The wireless communication parameter adjustment method based on online reinforcement learning as described in claim 5, wherein the method further includes: organizing the mobile network operation status data into a multi-dimensional array. The wireless communication parameter adjustment method based on online reinforcement learning as described in claim 7, wherein the wireless communication parameter configuration probability distribution of each base station is obtained based on the original signal strength distribution and the user gathering place distribution, so that each base station The station actually transmits based on the wireless communication parameter configuration probability distribution, updates the original signal strength distribution and obtains the user quality feedback to correct the neural network parameters. The step further includes: organizing the mobile network operation status data into the The multi-dimensional array is converted into a three-dimensional array after convolutional neural network operation and pooling operation; the arg max operator is used to obtain the probability distribution of the wireless communication parameter configuration of each base station for actual transmission, and the original signal strength distribution is updated. and obtain quality feedback from the user; The updated original signal intensity distribution is converted into a three-dimensional array after undergoing the convolutional neural network operation and the pooling operation of another set of original signal intensity distribution; the max operator is used to obtain the optimal wireless communication parameter configuration probability distribution, The optimal wireless communication parameter configuration probability distribution is nonlinearly combined with the user quality feedback, and then combined with the original signal strength distribution to form an array for mean square error calculation; output the beam direction corresponding to the probability matrix and then execute -sum(Probability*log( Probability)) to obtain scalar entropy; the mean square error is probabilistically shrunk to obtain a probability function and sampling weight to train the neural network parameters.