WO2022242175A1 - Data processing method and apparatus, and terminal - Google Patents

Abstract

A data processing method and apparatus, and a terminal, which relate to the field of artificial intelligence. The method comprises: once a terminal acquires application data to be processed and scene perception data, according to the scene perception data which influences the processing of the application data and a preset condition, selecting a first neural network suitable for processing the application data, and using the first neural network to obtain a processing result of the application data. Thus, scene requirements for the speed and accuracy of data processing are ensured, and the user experience is improved. The scene perception data is used to indicate factors that influence the terminal processing the application data. The preset condition is used to indicate a correspondence between scene data that influences operations of a neural network and the neural network.

Description

Data processing method, device and terminal

This application claims the priority of the Chinese patent application with the application number 202110558861.1 and the application name "data processing method, device and terminal" submitted to the State Intellectual Property Office on May 21, 2021, the entire content of which is incorporated by reference in this application middle.

technical field

The present application relates to the field of artificial intelligence, and in particular to a data processing method, device and terminal.

Background technique

Artificial Intelligence (AI) is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. Research in the field of artificial intelligence includes machine learning (Machine Learning, ML) natural language processing, computer vision, decision-making and reasoning, human-computer interaction, recommendation and search, AI basic theory, etc. Among them, the neural network is a method of machine learning.

At present, terminals (such as smart phones, self-driving cars, etc.) can use the configured neural network to process the acquired data (such as images, voice, etc.) to realize application functions such as face recognition and voice recognition. However, the terminal usually processes data obtained in different scenarios based on a neural network. Since the data in different scenarios has different characteristics, sampling a neural network to process data in different scenarios cannot ensure the speed and accuracy of data processing required by the scenario. Therefore, how to provide a data processing method that ensures the speed and accuracy of data processing required by the scene has become an urgent problem to be solved.

Contents of the invention

The present application provides a data processing method, device, and terminal, thereby ensuring the speed and accuracy of data processing to meet the scenario requirements.

In the first aspect, the present application provides a data processing method, which can be executed by a terminal, and specifically includes the following steps: after the terminal obtains the application data and scene perception data to be processed, it determines the processing according to the scene perception data and preset conditions The first neural network of the application data is used to obtain the processing result of the application data by using the first neural network. The scene awareness data is used to indicate factors influencing the terminal to process the application data. The preset condition is used to indicate the corresponding relationship between the scene data that affects the operation of the neural network and the neural network.

In this way, since the terminal considers the scene perception data that affects the processing of application data when selecting the first neural network for processing application data, and selects the first neural network suitable for processing application data according to the scene perception data and preset conditions, thereby ensuring The speed and accuracy of data processing meet the scene requirements and improve user experience.

Wherein, the application data includes data in at least one form among image, voice and text. Terminals can obtain application data through sensors. The sensor includes at least one of a camera, an audio sensor, and a lidar.

Understandably, the scene awareness data is used to indicate factors affecting speed and precision when the terminal processes application data. Wherein, the scene perception data includes at least one of external influencing factors and internal influencing factors; the external influencing factors are used to describe the characteristics of the application scenario in which the terminal obtains the application data, and the internal influencing factors are used to describe the characteristics of the operating scenario of the hardware resources for the terminal to run the application data . Exemplarily, the external influencing factors include at least one of temperature data, humidity data, illumination data and time data. Exemplarily, the internal influencing factors include at least one of computing capability of the processor, available storage capacity and available remaining power.

Since the terminal selects the first neural network suitable for processing application data according to the scene perception data and preset conditions, under the premise of ensuring the speed requirement, the first neural network is a large-scale network that meets the accuracy requirement; or, in Under the premise of ensuring the accuracy requirement, the first neural network is a smaller-scale network that meets the speed requirement. In this way, the accuracy and speed of processing application data are balanced, and the user experience is improved.

In a possible implementation manner, determining the first neural network for processing the application data according to the scene awareness data and the preset conditions includes: the terminal determining the parameters of the first neural network corresponding to the scene data including the scene awareness data in the preset conditions , determining the first neural network from the supernetwork according to the parameters of the first neural network. Wherein, the parameters of the first neural network include the number of channels and the number of network layers. The super network is used to determine the first neural network corresponding to the scene data. The first neural network is a subnetwork in the supernetwork. The number of network layers included in the sub-network is smaller than the number of network layers included in the super-network, or the number of channels included in the sub-network is smaller than the number of channels included in the super-network, and each network layer includes at least one neuron.

In another possible implementation manner, the terminal stores the parameters of the supernetwork and at least one subnetwork included in the supernetwork; determining the first neural network from the supernetwork according to the parameters of the first neural network includes: according to the first neural network The parameters of determine the weights of the first neural network from the weights of the supernetwork. Since the sub-networks share the parameters of the super-network, compared with storing multiple sub-networks in the terminal, the storage space of the terminal is effectively reduced.

In another possible implementation manner, after the first neural network for processing application data is determined according to the scene perception data and preset conditions, the method further includes: the terminal determines a second neural network from the super network, and uses the second neural network to obtain The result of processing the application data. The second neural network is a sub-network in the supernetwork, the number of network layers included in the second neural network is greater than or less than the number of network layers included in the first neural network, or the number of channels included in the second neural network is greater than or less than The number of channels included in the first neural network. If the processing result of the first neural network does not meet the speed and precision requirements of the user, the terminal may also adjust the neural network so that the processing result of the application data obtained by the terminal using the second neural network meets the speed and precision requirements of the user.

In another possible implementation, the method further includes: if the speed and accuracy of the first neural network do not meet user requirements, adjusting the scene data corresponding to the first neural network, so that the terminal uses the first neural network to process the application data again to obtain The processing results meet the user's speed and accuracy requirements.

Optionally, the method further includes: the terminal also displays the correspondence between the scene data and the first neural network that affect the speed and accuracy of the first neural network, and the processing results. The user can see the processing result intuitively, so that the user can judge whether the running time meets the user's speed requirement, and whether the accuracy of the processing result meets the user's precision requirement.

In a second aspect, the present application provides a data processing device, the device including various modules for executing the data processing method in the first aspect or any possible design of the first aspect.

In a third aspect, the present application provides a terminal, the terminal includes at least one processor and a memory, and the memory is used to store a set of computer instructions; When the device executes the set of computer instructions, it executes the first aspect or the operation steps of the data processing method in any possible implementation manner of the first aspect.

In a fourth aspect, the present application provides a computer-readable storage medium, including: computer software instructions; when the computer software instructions are run in the terminal, the terminal is made to execute the first aspect or any one of the possible implementation manners of the first aspect. Operational steps of the method.

In a fifth aspect, the present application provides a computer program product. When the computer program product runs on a computer, the terminal executes the operation steps of the method described in the first aspect or any possible implementation manner of the first aspect.

On the basis of the implementation manners provided in the foregoing aspects, the present application may further be combined to provide more implementation manners.

Description of drawings

Fig. 1 is the structural representation of a kind of neural network provided by the present application;

Fig. 2 is a schematic structural diagram of a convolutional neural network provided by the present application;

FIG. 3 is a schematic diagram of the architecture of a data processing system provided by the present application;

FIG. 4 is a flowchart of a method for generating preset conditions provided by the present application;

FIG. 5 is a schematic structural diagram of a supernetwork and a subnetwork provided by the present application;

FIG. 6 is a schematic diagram of a Pareto boundary provided by the present application;

FIG. 7 is a flowchart of a data processing method provided by the present application;

FIG. 8 is a flowchart of another data processing method provided by the present application;

FIG. 9 is a flowchart of another data processing method provided by the present application;

FIG. 10 is a schematic diagram of an interface for adjusting scene data provided by the present application;

FIG. 11 is a schematic structural diagram of a system provided by the present application;

FIG. 12 is a schematic structural diagram of a data processing device provided by the present application;

FIG. 13 is a schematic structural diagram of a terminal provided by the present application.

Detailed ways

For ease of understanding, the following first introduces related terms and neural network related concepts involved in the embodiments of the present application.

(1) neural network

A neural network can be made up of neurons, which can refer to operational units that take x _s and intercept 1 as input. The output of the arithmetic unit satisfies the following formula (1).

Among them, s=1, 2, ... n, n is a natural number greater than 1, W _s is the weight of x _s , and b is the bias of the neuron. f is the activation function of the neuron, which is used to introduce nonlinear characteristics into the neural network to convert the input signal in the neuron into an output signal. The output signal of the activation function can be used as the input of the next layer, and the activation function can be a sigmoid function. A neural network is a network formed by connecting multiple above-mentioned single neurons, that is, the output of one neuron can be the input of another neuron. The input of each neuron can be connected with the local receptive field of the previous layer to extract the features of the local receptive field, and the local receptive field can be an area composed of several neurons. Weights characterize the strength of connections between different neurons. The weight determines the influence of the input on the output. A weight close to 0 means that changing the input does not change the output. Negative weights mean that increasing the input decreases the output.

As shown in FIG. 1 , it is a schematic structural diagram of a neural network provided in the embodiment of the present application. The neural network 100 includes N processing layers, where N is an integer greater than or equal to 3. The first layer of the neural network 100 is the input layer 110, which is responsible for receiving input signals, and the last layer of the neural network 100 is the output layer 130, which is responsible for outputting the processing results of the neural network. The other layers except the first layer and the last layer are intermediate layers 140, and these intermediate layers 140 together form a hidden layer 120, and each intermediate layer 140 in the hidden layer 120 can receive input signals and output signals. The hidden layer 120 is responsible for the processing of the input signal. Each layer represents a logical level of signal processing, and through multiple layers, data signals can be processed by multi-level logic.

In some feasible embodiments, the input signals of the neural network may be signals in various forms such as video signals, voice signals, text signals, image signals, and temperature signals. The image signal can be various sensor signals such as the landscape signal captured by the camera (image sensor), the image signal of the community environment captured by the display monitoring equipment, and the face signal obtained by the access control system. The input signals of the neural network also include various other computer-processable engineering signals, which will not be listed here. If the neural network is used to carry out deep learning on the image signal, the image quality can be improved.

(2) Deep Neural Network

Deep Neural Network (DNN), also known as a multi-layer neural network, can be understood as a neural network with multiple hidden layers. The deep neural network is divided according to the position of different layers. The neural network inside the deep neural network can be divided into three categories: input layer, hidden layer and output layer. Generally speaking, the first layer is the input layer, the last layer is the output layer, and the middle layers are all hidden layers. The layers are fully connected, that is, any neuron in the i-th layer is connected to any neuron in the i+1-th layer.

其中

是输入向量，

是输出向量，

是偏移向量，W是权重矩阵(也称系数)，α()是激活函数。每一层仅仅是对输入向量

经过如此简单的操作得到输出向量

由于深度神经网络的层数多，系数W和偏移向量

的数量也比较多。这些参数在深度神经网络中的定义如下所述：以系数W为例：假设在一个三层的深度神经网络中，第二层的第4个神经元到第三层的第2个神经元的线性系数定义为

其中，上标3代表系数W所在的层数，而下标对应的是输出的第三层索引2和输入的第二层索引4。 Although the deep neural network looks complicated, it is actually not complicated in terms of the work of each layer. Simply put, it is the following linear relationship expression:

in

is the input vector,

is the output vector,

Is the offset vector, W is the weight matrix (also called coefficient), and α() is the activation function. Each layer is just the input vector

After such a simple operation to get the output vector

Due to the large number of layers in the deep neural network, the coefficient W and the offset vector

The number is also higher. The definition of these parameters in the deep neural network is as follows: Take the coefficient W as an example: Assume that in a three-layer deep neural network, the 4th neuron of the second layer to the 2nd neuron of the third layer The linear coefficients are defined as

Among them, the superscript 3 represents the number of layers where the coefficient W is located, and the subscript corresponds to the index 2 of the third layer output and the index 4 of the second layer input.

In summary, the coefficient from the kth neuron of the L-1 layer to the jth neuron of the L layer is defined as

It should be noted that the input layer has no W parameter. In deep neural networks, more hidden layers make the network more capable of describing complex situations in the real world. Theoretically speaking, a model with more parameters has a higher complexity and a greater "capacity", which means that it can complete more complex learning tasks. Training the deep neural network is also the process of learning the weight matrix, and its ultimate goal is to obtain the weight matrix of all layers of the trained deep neural network (the weight matrix formed by the vector W of many layers).

(3) Convolutional neural network

Convolutional Neuron Network (CNN) is a deep neural network with a convolutional structure. A convolutional neural network consists of a feature extractor consisting of a convolutional layer and a subsampling layer. The feature extractor can be seen as a filter, and the convolution process can be seen as using a trainable filter to convolve with an input image or feature map. The convolutional layer refers to the neuron layer that performs convolution processing on the input signal in the convolutional neural network. In the convolutional layer of a convolutional neural network, a neuron can only be connected to some adjacent neurons. A convolutional layer can output several feature maps, and the feature map can refer to the intermediate results during the operation of the convolutional neural network. Neurons in the same feature map share weights, and the shared weights here are convolution kernels. Shared weights can be understood as a way to extract image information that is independent of position. That is, the statistics for one part of the image are the same as for other parts. That means that the image information learned in one part can also be used in another part. So for all positions on the image, the same learned image information can be used. In the same convolution layer, multiple convolution kernels can be used to extract different image information. Generally, the more the number of convolution kernels, the richer the image information reflected by the convolution operation.

The convolution kernel can be initialized in the form of a matrix of random size, and the convolution kernel can obtain reasonable weights through learning during the training process of the convolutional neural network. In addition, the direct benefit of sharing weights is to reduce the connections between the layers of the convolutional neural network, while reducing the risk of overfitting.

For example, as shown in FIG. 2 , it is a schematic structural diagram of a convolutional neural network provided by the embodiment of the present application. The convolutional neural network 200 may include an input layer 210 , a convolutional/pooling layer 220 (where the pooling layer is optional) and a neural network layer 230 .

The convolutional layer/pooling layer 220 may include layers 221 to 226, for example. In one example, layer 221 may be a convolutional layer, layer 222 may be a pooling layer, layer 223 may be a convolutional layer, layer 224 may be a pooling layer, and layer 225 may be a convolutional layer, for example. , the layer 226 may be, for example, a pooling layer. In another example, layers 221 and 222 may be, for example, convolutional layers, layer 223 may be, for example, a pooling layer, layers 224 and 225 may be, for example, convolutional layers, and layer 226 may be, for example, a pooling layer. The output of a convolutional layer can be used as input to a subsequent pooling layer, or as input to another convolutional layer to continue the convolution operation.

Taking the convolutional layer 221 as an example, the inner working principle of one convolutional layer will be introduced.

The convolution layer 221 may include many convolution operators, and the convolution operators may also be called kernels. The role of the convolution operator in image processing is equivalent to a filter that extracts specific information from the input image matrix. The convolution operator can essentially be a weight matrix, which is usually predefined. During the convolution operation on the image, the weight matrix is usually one pixel by one pixel (or two pixels by two pixels, depending on the value of the stride) along the horizontal direction on the input image. processing to complete the work of extracting specific features from the image. The size of this weight matrix is related to the size of the image. It should be noted that the depth dimension of the weight matrix is the same as the depth dimension of the input image. During the convolution operation, the weight matrix is extended to the full depth of the input image. Therefore, convolution with a single weight matrix will produce a convolutional output with a single depth dimension, but in most cases instead of using a single weight matrix, multiple weight matrices of the same size (row×column) are applied, That is, multiple matrices of the same shape. The output of each weight matrix is stacked to form the depth dimension of the convolved image. Different weight matrices can be used to extract different features in the image. For example, one weight matrix is used to extract image edge information, another weight matrix is used to extract specific colors of the image, and another weight matrix is used to filter unwanted noise in the image. Do blurring etc. The multiple weight matrices have the same size (row×column), and the feature maps extracted by the multiple weight matrices of the same size are also of the same size, and then the extracted multiple feature maps of the same size are combined to form the convolution operation. output.

The weight values in these weight matrices need to be obtained through a lot of training in practical applications, and each weight matrix formed by the weight values obtained through training can be used to extract information from the input image, so that the convolutional neural network 200 can make correct predictions .

When the convolutional neural network 200 has multiple convolutional layers, the initial convolutional layer (such as layer 221 ) often extracts more general features, which can also be called low-level features. As the depth of the convolutional neural network 200 deepens, the features extracted by the later convolutional layers (for example, layer 226) become more and more complex, such as high-level semantic features, and the higher the semantic features, the more suitable for unresolved issues.

Since it is often necessary to reduce the number of training parameters, it is often necessary to periodically introduce pooling layers after convolutional layers. Each layer from layer 221 to layer 226 as shown in the convolutional layer/pooling layer 220 in Figure 2 can be a layer of convolutional layer followed by a layer of pooling layer, or a multi-layer convolutional layer followed by a layer or multiple pooling layers. In image processing, the sole purpose of pooling layers is to reduce the spatial size of the image. The pooling layer may include an average pooling operator and/or a maximum pooling operator for sampling an input image to obtain an image of a smaller size. The average pooling operator can calculate the pixel values in the image within a specific range to generate an average value as the result of average pooling. The maximum pooling operator can take the pixel with the largest value within a specific range as the result of maximum pooling. Also, just like the size of the weight matrix used in the convolutional layer should be related to the size of the image, the operators in the pooling layer should also be related to the size of the image. The size of the image output after being processed by the pooling layer may be smaller than the size of the image input to the pooling layer, and each pixel in the image output by the pooling layer represents the average or maximum value of the corresponding sub-region of the image input to the pooling layer.

After being processed by the convolutional layer/pooling layer 220, the convolutional neural network 200 is not enough to output the required output information. Because as mentioned earlier, the convolutional layer/pooling layer 220 extracts features and reduces the parameters brought by the input image. However, in order to generate the final output information (required class information or other relevant information), the convolutional neural network 200 needs to use the neural network layer 230 to generate one or a group of outputs with the required number of classes. Therefore, the neural network layer 230 may include a multi-layer hidden layer (layer 231, layer 232 to layer 23n as shown in FIG. 2 ) and an output layer 240, and the parameters contained in the multi-layer hidden layer may be determined according to specific tasks. Types of related training data are pre-trained, for example, the task type can include image recognition, image classification, image super-resolution reconstruction, and so on.

After the multi-layer hidden layer in the neural network layer 230, that is, the last layer of the entire convolutional neural network 200 is the output layer 240, which has a loss function similar to the classification cross entropy, and is specifically used to calculate the prediction error. Once The forward propagation of the entire convolutional neural network 200 (as shown in Figure 2 is forward propagation from layer 210 to layer 240 direction) is completed, and the reverse propagation (as shown in Figure 2 is backward propagation from layer 240 to layer 210 direction) ) will begin to update the weight values and deviations of the above-mentioned layers, so as to reduce the loss of the convolutional neural network 200, and the error between the output result of the convolutional neural network 200 through the output layer and the ideal result.

It should be noted that the convolutional neural network 200 shown in FIG. 2 is only an example of a convolutional neural network, and in specific applications, the convolutional neural network may also exist in the form of other network models.

(4) Loss function

In the process of training the deep neural network, because it is hoped that the output of the deep neural network is as close as possible to the value that is really wanted to be predicted, it is possible to compare the predicted value of the network with the target value that is really wanted, and then based on the relationship between the two Different situations to update the weight vector of each layer of neural network (of course, there is usually an initialization process before the first update, which is to pre-configure parameters for each layer in the deep neural network), for example, if the network's prediction value is high , just adjust the weight vector to make it predict lower, and keep adjusting until the deep neural network can predict the real desired target value or a value very close to the real desired target value. Therefore, it is necessary to pre-define "how to compare the difference between the predicted value and the target value", which is the loss function (loss function) or objective function (objective function), which is used to measure the difference between the predicted value and the target value important equation. Among them, taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference. Then the training of the deep neural network becomes a process of reducing the loss as much as possible.

(5) Back propagation algorithm

The convolutional neural network can use the error back propagation (back propagation, BP) algorithm to correct the size of the parameters in the initial super-resolution model during the training process, so that the reconstruction error loss of the super-resolution model becomes smaller and smaller. Specifically, passing the input signal forward until the output will generate an error loss, and updating the parameters in the initial super-resolution model by backpropagating the error loss information, so that the error loss converges. The backpropagation algorithm is a backpropagation movement dominated by error loss, aiming to obtain the parameters of the optimal super-resolution model, such as the weight matrix.

The implementation of the embodiment of the present application will be described in detail below with reference to the accompanying drawings.

FIG. 3 is a schematic structural diagram of a data processing system provided by an embodiment of the present application. As shown in FIG. 3 , the system 300 includes an execution device 310 , a training device 320 , a database 330 , a terminal device 340 , a data storage system 350 and a data collection device 360 .

The execution device 310 may be a terminal, such as a mobile phone terminal, a tablet computer, a notebook computer, a virtual reality (virtual reality, VR)/augmented reality (augmented reality, AR) device, a vehicle terminal, etc., and may also be an edge device (for example, carrying a processing power chip box), etc.

The training device 320 may be a server or a cloud device or the like. The training device 320 has strong computing power, and can run neural networks, perform calculations such as training neural networks.

As a possible embodiment, the executing device 310 and the training device 320 are different processors deployed on different physical devices (such as servers or servers in a cluster). For example, the execution device 310 may be a central processing unit (central processing unit, CPU), other general-purpose processors, a digital signal processor (digital signal processing, DSP), an application-specific integrated circuit (application-specific integrated circuit, ASIC), on-site Programmable gate array (field-programmable gate array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general purpose processor may be a microprocessor or any conventional processor or the like. The training device 320 may be a graphics processing unit (graphics processing unit, GPU), a neural network processor (neural network processing unit, NPU), a microprocessor, a specific application integrated circuit (application-specific integrated circuit, ASIC), or one or A plurality of integrated circuits used to control the program execution of the program of this application.

The data collection device 360 is used to collect training data and test data, and store the training data and test data into the database 330 . Both the training data and the testing data may be data in at least one form among image, voice and text. For example, training data includes training images and objects in the training images. The test data includes test images and objects in the test images. Both the features of the training image and the features of the test image are features similar to the features of the same application scene. Application scene characteristics include at least environmental characteristics (such as: temperature, humidity, light, etc.) and time characteristics (such as: day time, night time, traffic peak hours, traffic low peak hours, etc.), and so on.

The training device 320 is used to realize the function of searching N subnetworks (subnetwork) from the supernetwork 301 (supernet) based on the training data and test data maintained in the database 330, and generating the preset condition 302. The preset condition 302 is used to indicate the corresponding relationship between the scene data that affects the operation of the neural network and the neural network. Wherein, the neural network refers to a sub-network searched from the super-network. N is an integer greater than or equal to 2.

Exemplarily, as shown in FIG. 4 , the specific process for generating preset conditions is explained.

Step 410 , the training device 320 trains the supernetwork and the M subnetworks sampled in the supernetwork based on the training data maintained in the database 330 .

The training device 320 may sample M sub-networks from the super-network according to preset rules. The default rule is, for example, to select M sub-networks of different sizes from the first neuron of the first network layer of the supernetwork. Optionally, the training device 320 randomly samples M sub-networks from the super-network. M is an integer greater than or equal to 2.

The training device 320 uses the training data to train the super-network until the loss function in the super-network converges, and the super-network training is completed if the loss function value is less than a specific threshold, so that the super-network can reach a certain accuracy. Alternatively, all the training data in the training set is used for training, and the supernetwork training is completed.

The training device 320 uses the training data to train the sub-network until the loss function in the sub-network converges, and the training of the sub-network is completed if the value of the loss function is less than a specific threshold, so that the sub-network can reach a certain accuracy. Alternatively, all the training data in the training set is used for training, and the sub-network training is completed.

Understandably, the supernetwork can be used as the basic network for subnetwork search. A hypernetwork is a very large neural network that includes a large number of network layers, and each network layer contains a large number of neurons. Exemplarily, (a) in FIG. 5 illustrates a structure of a hypernetwork. A subnetwork is a partial network within a supernetwork. A subnet is a small neural network. As shown in (b) in FIG. 5 , the subnetwork includes three network layers, and the number of network layers included in the subnetwork is smaller than the number of network layers included in the supernetwork shown in (a) in FIG. 5 . Each network layer in the supernetwork and subnetwork includes at least one neuron. As shown in (c) in FIG. 5 , the number of neurons included in the sub-network is smaller than the number of neurons included in the super-network shown in (a) in FIG. 5 . The number of channels included in the sub-network may also be smaller than the number of channels included in the super-network shown in (a) in FIG. 5 . The weights of the subnetworks share the weights of the supernetwork. For example, the sub-network includes a first network layer and a second network layer in the supernetwork, and a first neuron in the first network layer is connected to a first neuron in the second network layer. The weight of the connection between the first neuron in the first network layer and the first neuron in the second network layer reuses the first neuron in the first network layer and the first neuron in the second network layer in the super network The weight of the connection. The neural network described in this embodiment may be a deep neural network. Specifically, the neural network may be a convolutional neural network.

It should be noted that the training device 320 can perform multiple rounds of training on the supernetwork and the subnetwork. For example, the training device 320 may re-sample a sub-network randomly from the super-network in each round, and train the super-network and the randomly-sampled sub-network. After executing all the rounds, the training device 320 randomly samples M sub-networks and completes the training on the M sub-networks. The training device 320 may randomly sample different numbers of sub-networks in each round; or, the training device 320 may randomly sample sub-networks of different network sizes in each round.

After the training device 320 completes the training of the supernetwork and the subnetworks in the supernetwork, it can search the subnetworks from the supernetwork, for example, the training device 320 can also execute steps 420 to 460 to obtain multiple subnetworks.

Step 420, the training device 320 searches for Q sub-networks from the super-network.

Step 430 , the training device 320 tests the accuracy of the Q sub-networks based on the test data maintained in the database 330 .

The test data used to test the accuracy of the subnet includes a large number of images. For each of the Q sub-networks, the training device 320 inputs a large number of images into the sub-network to obtain prediction results. Comparing the predicted result with the original image, the accuracy of the sub-network is obtained.

Since the training device 320 has strong computing power, it can run the neural network quickly, thereby effectively reducing the time for training and testing the neural network.

Step 440 , the execution device 310 tests the running durations of the Q sub-networks based on the test data maintained in the database 330 .

Since the computing power of the execution device 310 is less than that of the training device 320, the amount of test data used by the test execution device 310 for running the subnet is much smaller than the test data used by the training device 320 for the accuracy of the subnetwork. The data volume of the test data. For example, the test execution device 310 may run the test data with a long running time of the subnetwork as an image.

For each of the Q sub-networks, the executing device 310 inputs test data into the sub-network to obtain a prediction result and a running time of the sub-network. The running time of the subnetwork may refer to the time period from when the executing device 310 runs the subnetwork to process test data to when the test result is obtained. Understandably, the longer the running time of the sub-network, the slower the data processing speed; the shorter the running time of the sub-network, the faster the data processing speed.

It should be noted that since different types of execution devices are configured with different hardware resources (such as processors, memories, and batteries), the speeds at which the execution devices are configured with different hardware resources to run the neural network are also different. Therefore, the speeds of the Q sub-networks are respectively tested on execution devices configured with different hardware resources, so that the training device 320 can select a neural network that runs as fast as possible for the execution devices configured with different hardware resources.

Step 450 , the executing device 310 transmits the running durations of the Q sub-networks to the training device 320 .

It should be noted that since the training device 320 has trained the supernetwork and a large number of randomly sampled subnetworks according to the above step 410, the Q subnetworks searched by the training device 320 from the supernetwork are subnetworks with certain precision. The training device 320 performs step 460 . The Q sub-networks may be part of the M sub-networks trained by the training device 320 according to the above step 410 . Alternatively, the Q sub-networks may be sub-networks randomly sampled by the training device 320 .

Optionally, the training device 320 can randomly search for Q sub-networks from the supernetwork for the first time, update the Q sub-networks according to the accuracy of the Q sub-networks and the running time of the Q sub-networks, and perform steps 430 to 450 in a loop, after multiple rounds of iterations After updating the Q sub-networks to obtain Q sub-networks with high accuracy, the training device 320 executes step 460 .

Step 460, the training device 320 searches for N sub-networks from the Q sub-networks according to the accuracy of the Q sub-networks and the running time of the Q sub-networks.

In an example, the training device 320 may search for N sub-networks from the Q sub-networks according to evolutionary algorithm, reinforcement learning or greedy algorithm. Both Q and N are integers greater than or equal to 2, and Q is greater than N.

In another example, the training device 320 may also perform statistics on the accuracies of the Q sub-networks and the running time of the Q sub-networks to generate a Pareto boundary as shown in FIG. 6 . The horizontal axis represents the error rate, and the vertical axis represents the network size. The network size can also be referred to as the scale of the neural network. The larger the size of the neural network, the higher the accuracy of the neural network, the lower the error rate, and the longer the running time; conversely, the smaller the size of the neural network, the lower the accuracy of the neural network, the higher the error rate, and the shorter the running time. Therefore, the accuracy and speed of neural networks are inversely proportional.

From the perspective of the accuracy of the neural network, each point on the Pareto frontier represents the fastest neural network at an accuracy. From the perspective of the speed of the neural network, each point on the Pareto frontier represents the most accurate neural network at a speed.

The training device 320 can select N sub-networks from the Q sub-networks according to the accuracy and speed of the Pareto boundary trade-off neural network, and the training device 320 selects a neural network that meets the accuracy requirements and speed requirements for execution devices with different hardware resource configurations.

In another example, the training device 320 can also search Q sub-networks from the super network according to the evolutionary algorithm, reinforcement learning or greedy algorithm, and the training device 320 generates the Q sub-networks according to the accuracy of the Q sub-networks and the running time of the Q sub-networks as shown in Figure 6. As shown in the Pareto boundary, the training device 320 may select N sub-networks from the Q sub-networks according to the Pareto boundary to balance the accuracy and speed of the neural network.

In another example, the training device 320 may also search for N sub-networks from the super-network according to evolutionary algorithm, reinforcement learning or greedy algorithm.

Optionally, the training device 320 can also use the training data to train the N sub-networks, adjust the weights of the N sub-networks, and improve the accuracy of the N sub-networks.

In step 470, the training device 320 generates preset conditions according to the N sub-networks and scene data associated with each of the N sub-networks.

Since the application data processed by a neural network may be obtained in different application scenarios, and different application scenarios have different characteristics of the application scenarios, the accuracy of processing the application data in different application scenarios by the neural network is different.

For example, in an autonomous driving scenario, when an autonomous vehicle passes through a barrier in a clear sky, the image of the license plate is clear enough, and the execution device uses a small-scale neural network to recognize the license plate with high accuracy, which can Meet the accuracy requirements of license plate recognition. Application scene characteristics include sufficient light and sunny weather. As another example, when a self-driving car passes through a barrier gate in a heavy rainy day, the license plate image may be blurred due to the unclear line of sight in the rainy day. Accuracy requirements for license plate recognition. Application scene characteristics include low light and rainy weather.

As another example, for the gate gate scene, the flow of people is relatively large during the rush hour, and the execution equipment uses a large-scale neural network to identify the flow of people at a slow speed, which cannot meet the speed requirements for flow recognition. Application scenario characteristics can include rush hour hours and human traffic.

Therefore, the training device 320 can generate preset conditions according to a large number of application scene features and N sub-networks. The preset conditions may include the association relationship between a large number of application scene features and the N sub-networks. For example, the application scene features include sunny days, cloudy days, and rainy days. The preset conditions include an association relationship between sunny days and the first subnetwork, an association relationship between cloudy days and the second subnetwork, and an association relationship between rainy days and the third subnetwork. The size of the first sub-network is smaller than the size of the second sub-network. The size of the second sub-network is smaller than the size of the third sub-network.

The application scene features may be obtained by the training device 320 by analyzing test data and training data. Alternatively, the database 330 maintains the characteristics of the application scenario under the application scenario where the test data is collected and the characteristics of the application scenario under the application scenario where the training data is collected.

In addition, since the hardware resources of the execution device change dynamically, the execution time of the neural network may also be different under different hardware resources of the execution device. For example, taking a mobile phone as an example, when different users use the same brand of mobile phone, the available storage capacity and the available remaining power are different due to the different usage habits of different users. Even for the same mobile phone, the applications, available storage capacity, and available remaining power of the mobile phone are different at different time periods. When the mobile phone has a low battery, it may take a long time to run the neural network. Therefore, when the mobile phone has a low battery, a smaller neural network can be selected to process application data.

Therefore, the training device 320 may also receive the running scenario characteristics of the hardware resources when testing the N subnetworks sent by the execution device 310 . The training device 320 generates preset conditions according to a large number of application scene features, running scene features and N sub-networks.

The running scenario feature may include at least one of computing capability of the processor, available storage capacity, and available remaining power. The computing capability of the processor can also be understood as the occupancy rate of computing resources of the processor. For example, operating scene characteristics include low battery, medium battery, and high battery. The preset conditions include the association relationship between the low battery level and the fourth subnetwork, the association relationship between the medium battery level and the fifth subnetwork, and the association relationship between the high battery level and the sixth subnetwork. The size of the fourth sub-network is smaller than the size of the fifth sub-network. The size of the fifth sub-network is smaller than the size of the sixth sub-network.

After the training device 320 generates the precondition 302 according to the above steps 410 to 470 , it can configure the hypernetwork 301 and the precondition 302 to the execution device 310 .

The execution device 310 is configured to realize the function of determining the first neural network for processing application data from the hypernetwork 301 according to the scene awareness data and the preset condition 302 .

Since the execution device 310 selects the first neural network suitable for processing application data from the super network 301 according to the scene perception data and the preset condition 302, the first neural network is of a scale that meets the accuracy requirement under the premise of ensuring the speed requirement A larger network; or, under the premise of ensuring the accuracy requirement, the first neural network is a smaller-scale network that meets the speed requirement. In this way, the accuracy and speed of processing application data are balanced, and the user experience is improved. The first neural network is a subnetwork in the supernetwork 301 .

Regarding the specific method for the execution device 310 to determine the first neural network for processing application data, reference may be made to the illustration in FIG. 7 below.

It should be noted that, in practical applications, the training data and test data maintained in the database 330 do not necessarily all come from the data collection device 360, and may also be received from other devices. In addition, the training device 320 does not necessarily train the sub-network entirely based on the training data maintained by the database 330, and it is also possible to obtain training data from the cloud or other places to train the super-network. The execution device 310 does not necessarily test the sub-network entirely based on the test data maintained by the database 330, and may also obtain test data from the cloud or other places, so as to test the processing speed of the sub-network based on the test data. The above description should not be used as a limitation to the embodiments of the present application.

Further, according to the functions performed by the execution device 310, the execution device 310 can be further subdivided into the architecture shown in FIG. Preprocessing module 313 .

The I/O interface 312 is used for data interaction with external devices. A user can input data to the I/O interface 312 through the terminal device 340 . Input data can include images or video. In addition, the input data can also come from the database 330 .

The preprocessing module 313 is configured to perform preprocessing according to the input data received by the I/O interface 312 . In the embodiment of the present application, the preprocessing module 313 may be used to identify the application scenario characteristics of the application data received from the I/O interface 312 .

When the execution device 310 preprocesses the input data, or in the execution device 310 computing module 311 performs calculation and other related processing, the execution device 310 can call the data, codes, etc. in the data storage system 350 for corresponding processing , and the correspondingly processed data and instructions may also be stored in the data storage system 350 .

For example, the hypernetwork 301 and preset conditions 302 stored in the execution device 310 may be applied to the execution device 310 . After the execution device 310 obtains the application data, the calculation module 311 searches the first neural network from the super network 301 according to the scene awareness data and the preset condition 302, and uses the first neural network to process the application data. Since the first neural network is determined by the execution device 310 based on the scene perception data, processing the application data by using the first neural network can meet the speed and precision requirements of the user for data processing.

Finally, the I/O interface 312 returns the processing result to the terminal device 340, thereby providing it to the user, so that the user can view the processing result.

In the situation shown in FIG. 3 , the user can manually specify the input data, and the manual setting can be operated through the interface provided by the I/O interface 312. In another case, the terminal device 340 can automatically send input data to the I/O interface 312. If the terminal device 340 is required to automatically send the input data to obtain authorization from the user, the user can set corresponding permissions in the terminal device 340. The user can view the processing results output by the execution device 310 on the terminal device 340, and the specific presentation form may be specific ways such as display, sound, and action. The terminal device 340 can also be used as a data collection terminal, collecting input data input to the I/O interface 312 as shown in the figure and output processing results of the I/O interface 312 as new sample data, and storing them in the database 330 . Of course, the terminal device 340 may not be used for collection, but the I/O interface 312 will use the input data input to the I/O interface 312 as shown in the figure and the processing results of the output I/O interface 312 as new sample data Stored in database 330 .

Fig. 3 is only a schematic diagram of a system architecture provided by the embodiment of the present application, and the positional relationship between devices, devices, modules, etc. shown in Fig. 3 does not constitute any limitation. For example, in Fig. 3, the data storage system 350 is The execution device 310 is an external memory, and in other cases, the data storage system 350 may also be placed in the execution device 310 .

Next, the data processing method provided by the embodiment of the present application will be described in detail with reference to FIG. 7 to FIG. 11 . FIG. 7 is a schematic flowchart of a data processing method provided by an embodiment of the present application. Here, the execution device 310 in FIG. 3 is taken as an example for illustration. As shown in Fig. 7, the method includes the following steps.

Step 710, the executing device 310 obtains application data to be processed.

The application data includes data in at least one form among image, voice and text. The execution device 310 may acquire application data to be processed through a sensor. The sensor includes at least one of a camera, an audio sensor, and a lidar. For the application data of different applications, the sensors for collecting the application data may be different.

For example, if the executing device 310 is a smart terminal, the application scenario is that the smart terminal uses a face recognition function to recognize a user's face to realize the function of unlocking the smart terminal. The smart terminal can use the camera to capture the face to obtain a face image, and the application data to be processed is the face image. For another example, the application scenario is that the smart terminal uses the voice assistant function to recognize the user's voice, and realize the function of unlocking the smart terminal. The smart terminal can use the audio sensor to obtain the user's voice to obtain audio data, and the application data to be processed is audio data.

As another example, if the execution device 310 is a barrier gate monitor. The application scenario is that when the self-driving car passes through the barrier gate, the barrier gate monitor performs license plate recognition. The barrier gate monitor can use the camera to take pictures of the license plate of the self-driving car to obtain the license plate image, and the application data to be processed is the license plate image.

Step 720, the executing device 310 acquires scene perception data.

From the description of step 460 above, it can be seen that the factors affecting the speed and accuracy of the neural network processing application data include the application scenario characteristics of the application scenario where the application data is acquired and the operating scenario characteristics of the hardware resources that run the application data. Therefore, after acquiring the application data to be processed, the execution device 310 acquires the scene awareness data. The scene perception data is used to indicate the influencing factors when the executing device 310 processes the application data.

It can be understood that the scene awareness data is used to indicate the factors affecting the speed and accuracy when the execution device 310 uses the neural network to process the application data. The scene awareness data includes at least one of external influencing factors and internal influencing factors. The external influencing factors are used to describe the characteristics of the application scenario when the executing device 310 obtains the application data. The external influencing factors include at least one of temperature data, humidity data, illumination data and time data. The internal influencing factors are used to describe the characteristics of the running scenario of the hardware resources on which the execution device 310 runs the application data. The internal influencing factors include at least one of computing power of the processor, available storage capacity and available remaining power.

If the scene awareness data includes external influencing factors, in a possible implementation manner, the executing device 310 may acquire the scene awareness data from other devices. For example, if the scene perception data is temperature data, the executing device 310 may acquire the temperature data from a temperature sensor. For another example, if the scene perception data is humidity data, the executing device 310 may acquire temperature data from a humidity sensor. In another possible implementation manner, the executing device 310 may acquire scene awareness data according to application data to be processed. For example, the application data is an image, the scene perception data may be brightness or light intensity, and the execution device 310 may analyze the image to obtain the brightness or light intensity.

If the scene perception data includes internal influencing factors, the controller in the execution device 310 can monitor the processor, storage and power in the execution device 310 to obtain at least one of the processor's computing power, available storage capacity and available remaining power. The controller may be a central processing unit in the execution device 310 .

Step 730, the execution device 310 determines the first neural network for processing application data according to the scene awareness data and preset conditions.

The preset condition is used to indicate the corresponding relationship between the scene data that affects the operation of the neural network and the neural network.

In the first possible situation, the scenario data includes an application scenario identifier and an application scenario feature. The application scenario identifier is used to indicate the application scenario. An application scenario can have one or more application scenario characteristics. Understandably, one application scenario identifier can be associated with one application scenario feature, or one application scenario identifier can also be associated with multiple application scenario features. For example, the application scenario is a turnstile scenario in a residential garage, and the characteristics of the application scenario include sunny days, cloudy days, and rainy days.

The preset condition is used to indicate the corresponding relationship between the characteristics of the application scene that affect the operation of the neural network and the neural network. Each application scene feature is associated with a neural network. The preset conditions may include correspondences between multiple application scene features and neural networks. In an example, the corresponding relationship between the application scene features and the neural network may be presented in the form of a table, as shown in Table 1.

Table 1

It can be seen from Table 1 that the application scene features associated with the application scene indicated by the identifier 1 include sunny days, cloudy days, and rainy days. Sunny days have a binding relationship with neural network 1, cloudy days have a binding relationship with neural network 2, and rainy days have a binding relationship with neural network 3. The characteristics of the application scene associated with the application scene indicated by the mark 2 include strong light, medium light and weak light. The strong light has a binding relationship with the neural network 4, the medium light has a binding relationship with the neural network 5, and the weak light has a binding relationship with the neural network 6. The application scenario characteristics associated with the application scenario indicated by the identifier 3 include the usage interval of the application. The usage interval of the application has a binding relationship with the neural network 7 .

Assume that the application scenario indicated by the identifier 1 is the turnstile scenario in the garage of the community. When the car passes through the barrier in a sunny environment, the scene perception data acquired by the executing device 310 includes the car passing through the barrier and sunny weather. The executing device 310 looks up Table 1 according to the scene sensing data, and determines that the application scene is the scene indicated by the identifier 1. The execution device 310 further determines that the application scene characteristic of the scene indicated by the identifier 1 is sunny, and determines the neural network 1 corresponding to the sunny day as the first neural network.

Since the sky is clear, the image of the license plate captured by the executing device 310 is clear enough. The first neural network determined by the execution device 310 according to the characteristics of the application scenario and preset conditions may be a smaller-scale neural network. The execution device 310 uses a small-scale neural network to recognize the license plate with a high accuracy rate, which can meet the accuracy requirement of license plate recognition. Moreover, the running time of the execution device 310 using a small-scale neural network to recognize the license plate is relatively short, which can meet the speed requirement of license plate recognition.

As another example, when a car passes through a barrier in a heavy rain environment, the scene perception data obtained by the executing device 310 includes the car passing through a barrier and rainy weather, and the executing device 310 looks up Table 1 according to the scene sensing data, and determines that the application scene is indicated by the sign 1 The execution device 310 then determines that the application scene feature of the scene indicated by the identifier 1 is rainy, and determines the neural network 3 corresponding to the rainy day as the first neural network. The size of neural network 3 is larger than that of neural network 1.

Since the line of sight is unclear in rainy days, the image of the license plate captured by the executing device 310 may be blurred. If the execution device 310 continues to use a small-scale neural network to recognize the license plate, the accuracy rate is low, which cannot meet the accuracy requirement of license plate recognition. Therefore, the first neural network determined by the execution device 310 according to the characteristics of the application scenario and preset conditions may be a relatively large-scale neural network. The execution device 310 uses a large-scale neural network to recognize the license plate with high accuracy, which can meet the accuracy requirement of license plate recognition under the premise of ensuring the speed requirement.

Therefore, the execution device 310 dynamically selects the first neural network for processing application data according to the characteristics of the application scene, and makes the speed as fast as possible under the premise of meeting the precision requirement, or makes the precision as high as possible under the premise of meeting the speed requirement , balance the accuracy requirements and speed requirements when processing application data, and improve user experience.

It should be noted that Table 1 only shows the storage form of the corresponding relationship in the storage device in the form of a table, and does not limit the storage form of the corresponding relationship in the storage device. Of course, the storage form of the corresponding relationship in the storage device The format may also be stored in other formats, which is not limited in this embodiment.

In addition, the application scenario feature may be expressed in the form of a specific value or value range. For example, if the feature of the application scene is light intensity, the value range of light intensity may include three value ranges of strong light, medium light and weak light. For another example, the feature of the application scenario is the duration of the usage interval of the application, and the duration of the usage interval of the application may be 3 seconds. The more value ranges corresponding to an application scenario feature, the more accurate the execution device 310 determines the first neural network for processing application data.

In the second possible situation, the scenario data includes the identifier of the running scenario and the characteristics of the running scenario. The running scenario identifier is used to indicate the running scenario. A running scenario can have one or more running scenario characteristics. Understandably, one running scenario identifier can be associated with one running scenario feature, or one running scenario identifier can also be associated with multiple running scenario features. For example, the running scenario is available remaining power, and the characteristics of the running scenario include high power, medium power and low power.

The preset condition is used to indicate the corresponding relationship between the characteristics of the operating scene that affect the operation of the neural network and the neural network. Each running scene feature is associated with a neural network. The preset conditions may include correspondences between multiple running scene features and neural networks. In an example, the corresponding relationship between the running scene features and the neural network may be presented in the form of a table, as shown in Table 2.

Table 2

It can be known from Table 2 that the running scenario indicated by the identifier 4 is the available remaining power scenario of the executing device 310 . The running scenario characteristics associated with the available remaining power scenarios include high available remaining power, medium available remaining power and low available remaining power. The high available remaining power has a binding relationship with the neural network 8 , the available remaining power has a binding relationship with the neural network 9 , and the low available remaining power has a binding relationship with the neural network 10 .

The characteristics of the running scenario can be expressed in the form of a specific value or value range. For example, if the running scene is characterized by available remaining power. The value range of available remaining power is 100% to 60%. The range of values in Available Remaining Battery is 60% to 30%. The value range for the low battery remaining is 30% to 5%. The more value ranges corresponding to an operating scenario feature, the more accurate the execution device 310 determines the first neural network for processing the operating data.

Assume that when the execution device 31 needs to start the voice assistant function, the execution device 310 determines that the power is lower than 20%, then the execution device 310 determines that the scene perception data is that the power is lower than 20%. The execution device 310 determines that the running scenario is the scenario indicated by the identifier 4 according to the scene perception data lookup table 2, and then the execution device 310 determines that the running scenario characteristic of the scenario indicated by the identifier 4 is low available remaining power, and the neural network corresponding to the low available remaining power 10 is determined as the first neural network.

Since the available remaining power of the execution device 310 is low, if the execution device 310 continues to use a large-scale neural network for speech recognition, the speed requirement for license plate recognition cannot be met. At this time, the first neural network determined by the execution device 310 according to the characteristics of the running scenario and preset conditions may be a smaller-scale neural network. The execution device 310 can recognize the user's voice by using a small-scale neural network, which can meet the speed requirement of voice recognition under the premise of ensuring the accuracy requirement, and can save power.

For another example, if the execution device 310 determines that the power level is higher than 80%, then the execution device 310 determines that the scene awareness data indicates that the power level is higher than 80%. The execution device 310 determines that the running scenario is the scenario indicated by the identifier 4 according to the scene perception data query table 2, and the execution device 310 then determines that the running scenario characteristic of the scenario indicated by the identifier 4 is high available remaining power, and the neural network corresponding to the high available remaining power 8 is determined as the first neural network.

Since the available remaining power of the execution device 310 is high, the first neural network determined by the execution device 310 according to the characteristics of the running scenario and preset conditions may be a large-scale neural network. The execution device 310 can recognize the user's voice by using a large-scale neural network, and can meet the accuracy requirement of voice recognition under the premise of ensuring the speed requirement.

Therefore, the execution device 310 dynamically selects the first neural network for processing application data according to the characteristics of the running scene, and makes the speed as fast as possible under the premise of meeting the precision requirement, or makes the precision as high as possible under the premise of meeting the speed requirement , balance the accuracy requirements and speed requirements when processing application data, and improve user experience.

It should be noted that Table 2 only shows the storage form of the corresponding relationship in the storage device in the form of a table, and does not limit the storage form of the corresponding relationship in the storage device. Of course, the storage form of the corresponding relationship in the storage device The format may also be stored in other formats, which is not limited in this embodiment.

In the third possible situation, the scenario data includes application scenario features and running scenario features. For the explanation of the features of the application scenario and the features of the running scenario, reference may be made to the explanations of the first possible scenario and the second possible scenario above. An application scenario feature can correspond to one or more running scenario features. A running scenario feature can correspond to one or more application scenario features. The embodiment of the present application does not limit the association relationship between the application scenario feature and the running scenario feature, and the specific association relationship can be set according to the application scenario.

In an example, the corresponding relationship between the scene data and the neural network may be presented in the form of a table, as shown in Table 3.

table 3

It can be known from Table 3 that the application scene characteristics associated with the application scene indicated by the identifier 7 include sunny weather. The running scenario characteristics associated with the application scenario indicated by the identifier 7 include high remaining available power and low remaining available power. Both the sunny day and the high remaining power available have a binding relationship with the neural network 17 . Sunny days and low available remaining power both have a binding relationship with the neural network 18 .

Assume that the application scenario indicated by the mark 7 is the scenario where the self-driving car passes through the barrier gate. When the self-driving car passes the barrier in a sunny environment, the scene perception data acquired by the execution device 310 includes the vehicle passing the barrier and the sunny day, and the scene perception data determined by the execution device 310 also includes the high remaining power available, the execution device 310 according to Scene-aware data lookup table 3, determine that the application scene is the scene indicated by the mark 7, and the execution device 310 determines that the application scene characteristic of the scene indicated by the mark 7 is sunny, and the running scene is characterized by high available remaining power, and the sunny day and the high available remaining power The corresponding neural network 17 is defined as the first neural network.

Since the sky is clear, the image of the license plate captured by the execution device 310 is clear enough, and the execution device 310 can use a small-scale neural network to recognize the license plate to meet the speed requirement of license plate recognition while ensuring the accuracy requirement. However, if the available remaining power of the executing device 310 is high, the first neural network determined by the executing device 310 according to the characteristics of the application scenario, the characteristics of the operating scenario, and the preset conditions can further improve the license plate recognition on the premise that it meets the speed requirements of the license plate recognition. The accuracy of the neural network to meet the accuracy requirements of license plate recognition.

If the available remaining power of the execution device 310 is low, since the execution device 310 may affect the running time of the neural network when the power is low, the execution device 310 may select the neural network 18 corresponding to the sunny day and the low available remaining power to be determined as the first neural network 18. a neural network. On the premise that the accuracy requirement can be ensured, the license plate is recognized by using the neural network 18 which is smaller in scale than the neural network 17, so as to meet the speed requirement of the license plate recognition.

As another example, when the self-driving car passes the barrier in a rainy environment, the scene perception data obtained by the execution device 310 includes the car passing the barrier and rainy weather, and the scene perception data determined by the execution device 310 also includes the high available remaining power, the execution device 310 310, according to the scene perception data query table 3, determine that the application scene is the scene indicated by the mark 8, and the execution device 310 determines that the application scene feature of the scene indicated by the mark 8 is rainy, and the running scene feature is that the available remaining power is high, and the rainy day and the available remaining power The neural network 19 corresponding to the high battery level is determined as the first neural network.

Due to the unclear line of sight in rainy days, the image of the license plate captured by the execution device 310 may be blurred. The execution device 310 uses a large-scale neural network to recognize the license plate with a high accuracy rate, which can meet the requirements of license plate recognition under the premise of ensuring the speed requirement. Accuracy needs. However, the available remaining power of the execution device 310 is high, and the first neural network determined by the execution device 310 according to the characteristics of the application scene, the characteristics of the operating scene and the preset conditions can further improve the accuracy of the license plate recognition under the premise of meeting the speed requirements of the license plate recognition A neural network to meet the accuracy requirements of license plate recognition.

If the available remaining power of the execution device 310 is low, the running time of the execution device 310 using a large-scale neural network to recognize the license plate is relatively long, which cannot meet the speed requirement for license plate recognition. Therefore, the execution device 310 may select the neural network 20 corresponding to the rainy day and the low remaining available power to be determined as the first neural network. On the premise that the accuracy requirement can be ensured, using the neural network 20 smaller in scale than the neural network 19 to recognize the license plate can meet the speed requirement of the license plate recognition under the premise that the accuracy requirement can be ensured.

Therefore, the execution device 310 dynamically selects the first neural network for processing application data according to the characteristics of the application scene and the characteristics of the running scene, and makes the speed as fast as possible under the premise of meeting the precision requirement, or uses the first neural network to process the application data under the premise of meeting the speed requirement The precision is as high as possible, balancing the precision requirements and speed requirements when processing application data, and improving user experience.

The method flow described in FIG. 8 is an illustration of the specific operation process included in step 730 in FIG. 7 .

Step 7301, the execution device 310 determines the parameters of the first neural network corresponding to the scene data including the scene awareness data in the preset conditions.

It can be understood that the scene data is predetermined data for generating preset conditions. The scene awareness data is the data acquired by the executing device 310 in real time according to the application data.

In some embodiments, if the execution device 310 inquires the application scene characteristics identified by the scene awareness data in the preset conditions (as explained in the first possible situation above), or if the execution device 310 inquires in the preset conditions The running scene feature identified by the scene-aware data (as described in the second possible situation above), or, if the execution device 310 inquires the application scene feature and the running scene feature identified by the scene-aware data in the preset condition (as described above Explanation of the third possible situation), the execution device 310 processes the application data according to the first neural network associated with the scene data, which balances the accuracy and speed of processing application data, meets user needs, and improves user experience.

Step 7302, the execution device 310 determines the first neural network from the supernetwork according to the parameters of the first neural network.

The preset condition may also include an identifier of the first neural network and parameters of the first neural network. The parameters of the first neural network include the number of channels and the number of network layers. The first neural network may be a sub-network in the super network, and the super network is used to determine the first neural network corresponding to the scene data. A channel can refer to the input result of a convolutional layer in a convolutional neural network, that is, a channel can also be called a feature map. The network layer may be a convolutional layer in a convolutional neural network.

The execution device 310 may determine the first neural network from the supernetwork according to the parameters of the first neural network. Specifically, the execution device 310 determines the weight of the first neural network from the weights of the supernet according to the parameters of the first neural network. Since the subnetworks share the parameters of the supernetwork, storing multiple subnetworks relative to the execution device 310 effectively reduces the storage space of the execution device 310.

In step 740, the execution device 310 obtains a processing result of the application data by using the first neural network.

The execution device 310 inputs the application data into the first neural network, and uses the first neural network to obtain a processing result of the application data. For example, an application scenario is that a smart terminal uses a face recognition function to recognize a user's face to realize the function of unlocking the smart terminal. The application data is a face image, and the executing device 310 inputs the face image into the first neural network to obtain a processing result. If the processing result is that the face is recognized successfully, the smart terminal is successfully unlocked; if the processing result is that the face recognition fails, the smart terminal fails to be unlocked.

Further, after the execution device 310 determines the first neural network, it may further adjust the first neural network. The method flow shown in FIG. 9 is a supplementary description of the method shown in FIG. 7 .

In step 750, the execution device 310 displays the corresponding relationship between the scene data and the first neural network that affect the running speed and accuracy of the first neural network, and the processing results.

Processing results can include runtime and accuracy. The running time may refer to the time from when the executing device 310 runs the first neural network to process the application data to when the processing result is obtained. The execution device 310 starts timing after inputting the application data into the first neural network, and ends timing until the first neural network outputs the processing result to obtain the running time. The precision of the processing result is used to indicate whether the processing result of using the first neural network to process the application data satisfies the accuracy requirement of the user. The execution device 310 may obtain the accuracy of the processing result through user feedback information. The execution device 310 displays the corresponding relationship between the scene data and the first neural network that affect the speed and accuracy of the first neural network, as well as the processing results, so that the user can intuitively see the processing results, so that the user can judge whether the running time meets the user's requirements. Speed requirements, and judging whether the accuracy of the processing results meets the user's accuracy requirements.

If the processing result of the first neural network does not meet the speed and accuracy requirements of the user, the execution device 310 may also execute step 760 and step 770, or the execution device 310 may also execute step 780.

In step 760, the executing device 310 determines the second neural network from the hypernetwork.

The second neural network may be a user-specified neural network, for example, the second neural network may be a subnetwork in the supernetwork. The rules for specifying the neural network by the user are as follows: if the user needs to improve the accuracy of processing application data by the execution device 310, the number of network layers included in the second neural network is greater than the number of network layers included in the first neural network, or the second neural network includes The number of channels is greater than the number of channels included in the first neural network. If the user needs to increase the processing speed of the execution device 310 to process application data, that is, to reduce the running time of the execution device 310 to process application data, the number of network layers included in the second neural network is smaller than the number of network layers included in the first neural network, or, the first neural network includes The number of channels included in the second neural network is smaller than the number of channels included in the first neural network.

In step 770, the execution device 310 obtains the processing result of the application data by using the second neural network.

Step 780, the execution device 310 adjusts the scene data corresponding to the first neural network.

If the execution device 310 uses the first neural network to process the application data, the accuracy of the processing results cannot meet the user's precision requirements, or the running time for the execution device 310 to use the first neural network to process the application data to obtain the processing results cannot meet the user's speed requirements. If there are many application scenario features and running scenario features associated with the first neural network, the executing device 310 may modify at least one of the application scenario features and running scenario features associated with the first neural network.

For example, the application scenario is to identify the flow of people during rush hours. Since there are a lot of people during rush hours, if the execution device 310 continues to use a small-scale neural network to identify the flow of people, the accuracy rate is low, and it cannot Meet the accuracy requirements of people flow recognition. Therefore, the execution device 310 divides the rush hour period into multiple periods according to the flow of people, and each period is associated with a neural network, so that the execution device 310 uses a different neural network for each period to identify the flow of people. For example, during a time period when the flow of people is large, the execution device 310 uses a small-scale neural network to identify the flow of people, which can meet the speed requirement for the recognition of the flow of people while ensuring the accuracy requirements. For another example, during a time period when the flow of people is small, the execution device 310 uses a large-scale neural network to identify the flow of people, which can meet the accuracy requirements of the recognition of the flow of people under the premise of ensuring the speed requirement.

For another example, the application scenario is that the smart terminal uses the face recognition function to recognize the user's face to realize the function of unlocking the smart terminal. The user performs face recognition twice in a short period of time to unlock the smart terminal. If the smart terminal continuously uses a large-scale neural network to recognize faces, although the accuracy rate is high and meets the accuracy requirements of face recognition, the running time of face recognition is long, which cannot meet the speed of face recognition by users. need. Therefore, the smart terminal can adjust the correspondence between the preset interval of face recognition and the neural network. If the smart terminal detects that the time interval between two face recognitions is within the preset interval, the smart terminal determines that the first neural network associated with the preset interval may be a smaller-scale neural network. The smart terminal uses a small-scale neural network to recognize faces, which can meet the speed requirements of face recognition under the premise of ensuring the accuracy requirements.

By way of example, FIG. 10 is a schematic diagram of an interface for adjusting scene data provided in an embodiment of the present application. Assuming that the execution device 310 is a smart terminal, as shown in (a) in FIG. 10 , the smart terminal displays a schematic diagram of a face recognition result. The smart terminal shows that there are two face recognitions within ten minutes, and the duration of the two face recognitions is 4 minutes. Optionally, the smart terminal may also display a button 1010 of "whether to update". If the user clicks the button 1010 of "whether to update". The smart terminal can display an interface as shown in (b) in FIG. 10 . As shown in (b) in Figure 10, the smart terminal displays two face recognitions within two minutes, the first face recognition lasts for 4 minutes, the second face recognition lasts for 2 minutes, Whether to execute. The interface may display a "Yes" button 1020 and a "No" button 1030 . The user may click on the “Yes” button 1020 . If the smart terminal performs face recognition twice within two minutes, the time for the first face recognition is 4 minutes, and the time for the second face recognition is 2 minutes.

It should be noted that the application scenarios described in the embodiments of the present application may include target detection scenarios, monitoring scenarios, speech recognition scenarios, commodity recommendation scenarios, and so on.

Object detection is an important part of computer vision. Computer vision is an integral part of various intelligent/autonomous systems in various application fields, such as manufacturing, inspection, document analysis, and medical diagnosis. The subject's data and information knowledge. To put it figuratively, it is to install eyes (cameras/video cameras) and brains (algorithms) on computers to replace human eyes to identify and measure targets, so that computers can perceive the environment. Because perception can be thought of as extracting information from sensory signals, computer vision can also be thought of as the science of how to make artificial systems "perceive" from images or multidimensional data. In general, computer vision is to use various imaging systems to replace the visual organs to obtain input information, and then use the computer to replace the brain to complete the processing and interpretation of these input information. The ultimate research goal of computer vision is to enable computers to observe and understand the world through vision like humans, and have the ability to adapt to the environment autonomously.

Object detection methods can be applied in scenarios such as face detection, vehicle detection, pedestrian counting, automatic driving, security systems, and medical fields. For example, in an autonomous driving scenario, an autonomous vehicle recognizes objects in the surrounding environment during driving to adjust the speed and direction of the autonomous vehicle so that the autonomous vehicle can drive safely and avoid traffic accidents. Objects may be other vehicles, traffic control devices, or other types of objects. As another example, in the security system, a large number of users are identified to assist the staff to determine the target person as soon as possible. Usually, the input data (such as image or video) is input to the neural network with target detection function, the neural network performs feature extraction on the input data, and the target detection is performed based on the extracted features, and the detection result is obtained.

In addition, the execution device 310 may have stored the hypernetwork and the preset conditions before executing step 730, that is, the execution device 310 determines the first neural network for processing application data according to the scene awareness data and the preset conditions. Therefore, the execution device 310 may The hypernetwork and preset conditions are read from the memory, and the first neural network for processing application data is determined according to the scene perception data and preset conditions.

Optionally, the execution device 310 does not store the hypernetwork and preset conditions, and needs to download the hypernetwork and preset conditions from the server. The server may refer to a cloud server.

As an example, FIG. 11 is a schematic structural diagram of a system 1100 provided in the present application. As shown in FIG. 11 , the system 1100 may be an entity that provides cloud services to users by using basic resources. System 1100 includes cloud data center 1110 . The cloud data center 1110 includes a device resource pool (including computing resources 1111 , storage resources 1112 and network resources 1113 ) and a cloud service platform 1120 . The computing resource 1111 included in the cloud data center 1110 may be a computing device (such as a server).

The interaction device 1210 may be deployed on the execution device 1200 . The interaction means 1210 may be a browser or an application capable of message interaction with the cloud service platform 1120 . The user can access the cloud service platform 1120 through the interaction device 1210 , upload a request to the cloud data center 1110 , and request to obtain the supernetwork 301 and the preset condition 302 . After receiving the request uploaded by the execution device 1200 , the cloud data center 1110 feeds back the supernetwork 301 and the preset condition 302 to the execution device 1200 . The execution device 1200 may be a smart terminal or an edge station. The edge station can process the application data of the self-driving car and transmit the processing results to the self-driving car. The processing results are used to instruct the autonomous vehicle to operate.

It can be understood that, in order to implement the functions in the foregoing embodiments, the terminal includes hardware structures and/or software modules corresponding to each function. Those skilled in the art should easily realize that the present application can be implemented in the form of hardware or a combination of hardware and computer software with reference to the units and method steps of the examples described in the embodiments disclosed in the present application. Whether a certain function is executed by hardware or computer software drives the hardware depends on the specific application scenario and design constraints of the technical solution.

The data processing method provided according to this embodiment is described in detail above with reference to FIG. 1 to FIG. 11 , and the data processing device provided according to this embodiment will be described below in conjunction with FIG. 12 .

FIG. 12 is a schematic structural diagram of a possible data processing device provided by this embodiment. These data processing apparatuses can be used to implement the functions of the execution devices in the above method embodiments, and thus can also achieve the beneficial effects of the above method embodiments. In this embodiment, the data processing apparatus may be the execution device 310 shown in FIG. 3 , or may be a module (such as a chip) applied to a server.

As shown in FIG. 12 , the data processing device 1200 includes a communication module 1210 , a selection module 1220 , a detection module 1230 and a storage module 1240 . The data processing apparatus 1200 is configured to implement the functions of the executing device 310 in the method embodiment shown in FIG. 7 , FIG. 8 , or FIG. 9 .

The communication module 1210 is used to obtain application data and scene perception data to be processed. The scene awareness data is used to instruct the execution device 310 to process the application data influencing factors. The scene perception data is used to indicate factors affecting speed and precision when the executing device 310 processes the application data. Wherein, the scene perception data includes at least one of external influencing factors and internal influencing factors. The external influencing factors are used to describe the characteristics of the application scenario in which the terminal obtains the application data. The internal influencing factors are used to describe the running scenario characteristics of the hardware resources of the terminal running the application data. Exemplarily, the external influencing factors include at least one of temperature data, humidity data, illumination data and time data. The internal influencing factors include at least one of computing capability of the processor, available storage capacity and available remaining power. For example, the communication module 1210 is used to execute step 710 and step 720 in FIG. 7 , FIG. 8 and FIG. 9 .

The selection module 1220 is configured to determine a first neural network for processing the application data according to the scene perception data and a preset condition, the preset condition being used to indicate the corresponding relationship between the scene data that affects the operation of the neural network and the neural network. For example, the selection module 1220 is used to execute step 730 in FIG. 7 , FIG. 8 and FIG. 9 .

The detection module 1230 is configured to use the first neural network to obtain the processing result of the application data. For example, the detection module 1230 is used to execute step 740 in FIG. 7 , FIG. 8 and FIG. 9 .

When the selection module 1220 determines the first neural network for processing the application data according to the scene awareness data and preset conditions, it is specifically used to: determine the first neural network corresponding to the scene data including the scene awareness data in the preset conditions. The parameters of the neural network, the first neural network is determined from the super network according to the parameters of the first neural network. The parameters of the first neural network include the number of channels and the number of network layers. The first neural network is a subnetwork in the supernetwork, and the supernetwork is used to determine the first neural network corresponding to the scene data. Wherein, the number of network layers included in the sub-network is less than the number of network layers included in the super-network, or, the number of channels included in the sub-network is smaller than the number of channels included in the super-network, each network layer Include at least one neuron.

The storage module 1240 is configured to store preset conditions, parameters of the supernetwork and at least one subnetwork included in the supernetwork. When the selection module 1220 determines the first neural network from the supernet according to the parameters of the first neural network, it is specifically used to: determine the first neural network from the weight of the supernet according to the parameters of the first neural network The weight of the network.

Optionally, the selection module 1220 is also configured to determine a second neural network from the supernetwork, the second neural network is a subnetwork in the supernetwork, and the number of network layers included in the second neural network is greater than or is less than the number of network layers included in the first neural network, or, the number of channels included in the second neural network is greater than or smaller than the number of channels included in the first neural network.

Optionally, the detection module 1230 is further configured to use the second neural network to obtain a processing result of the application data. For example, the detection module 1230 is configured to execute step 770 in FIG. 9 .

The data processing device 1200 also includes an update module 1250 and a display module 1260 .

The update module 1250 is used to adjust the scene data corresponding to the first neural network if the speed and accuracy of the first neural network do not meet the user's requirements, obtain the adjusted scene data, and store the adjusted scene data in the storage module 1240. For example, the update module 1250 is used to execute step 760 and step 780 in FIG. 9 .

The display module 1260 is used to display the corresponding relationship between the scene data and the first neural network that affect the speed and accuracy of the first neural network, and the processing results. For example, the display module 1260 is used to execute step 750 in FIG. 9 .

It should be understood that the data processing device 1200 in the embodiment of the present application can be realized by an ASIC, or a programmable logic device (programmable logic device, PLD), and the above-mentioned PLD can be a complex programmable logic device (complex programmable logical device, CPLD), Field-programmable gate array (field-programmable gate array, FPGA), general array logic (generic array logic, GAL) or any combination thereof. When the data processing methods shown in Fig. 7, Fig. 8 and Fig. 9 can also be implemented by software, the data processing device 1200 and its modules can also be software modules.

The data processing device 1200 according to the embodiment of the present application may correspond to execute the method described in the embodiment of the present application, and the above-mentioned and other operations and/or functions of the various units in the data processing device 1200 are respectively in order to realize Fig. 7, Fig. 8 and For the sake of brevity, the corresponding flow of each method in FIG. 9 will not be repeated here.

FIG. 13 is a schematic structural diagram of a terminal 1300 provided in this embodiment. As shown in the figure, the terminal 1300 includes a processor 1310 , a bus 1320 , a memory 1330 and a communication interface 1340 .

It should be understood that in this embodiment, the processor 1310 may be a CPU, and the processor 1310 may also be other general-purpose processors, DSP, ASIC, FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components Wait. A general purpose processor may be a microprocessor or any conventional processor or the like.

The processor can also be a GPU, NPU, microprocessor, ASIC, or one or more integrated circuits used to control the program execution of the solution of this application.

The communication interface 1340 is used to implement communication between the terminal 1300 and external devices or devices. In this embodiment, the communication interface 1340 is used to receive application data and scene perception data to be processed.

Bus 1320 may include a path for communicating information between the components described above (eg, processor 1310 and memory 1330). In addition to the data bus, the bus 1320 may also include a power bus, a control bus, a status signal bus, and the like. However, for clarity of illustration, the various buses are labeled as bus 1320 in the figure.

As one example, the terminal 1300 may include multiple processors. The processor can be a multi-core

(multi-CPU) processor. A processor herein may refer to one or more devices, circuits, and/or computing units for processing data (eg, computer program instructions). The processor 1310 may call the hypernetwork and the preset conditions stored in the memory 1330 to determine a first neural network for processing application data according to the scene perception data and the preset conditions, and use the first neural network to obtain the information of the application data. process result.

It is worth noting that in FIG. 13, the terminal 1300 includes only one processor 1310 and one memory 1330 as an example. Here, the processor 1310 and the memory 1330 are respectively used to indicate a type of device or device. In a specific embodiment, The quantity of each type of device or equipment can be determined according to business needs.

The memory 1330 may correspond to a storage medium used to store information such as a hypernetwork and preset conditions in the above method embodiments, for example, a magnetic disk, such as a mechanical hard disk or a solid state hard disk.

The aforementioned terminal 1300 may be a general-purpose device or a dedicated device. For example, the terminal 1300 may be a mobile phone terminal, a tablet computer, a notebook computer, a VR device, an AR device, a mixed reality (Mixed Reality, MR) device or an extended reality (Extended Reality, ER) device, a vehicle terminal, etc., and may also be an edge device (e.g., a box carrying a chip with processing power), etc. Optionally, terminal 1330 may also be a server or other devices with computing capabilities.

It should be understood that the terminal 1300 according to this embodiment may correspond to the data processing device 1200 in this embodiment, and may correspond to a corresponding subject performing any method in FIG. 7 , FIG. 8 and FIG. 9 , and the data processing device The above-mentioned and other operations and/or functions of each module in 1200 are respectively for realizing the corresponding flow of each method in FIG. 7 , FIG. 8 and FIG. 9 , and for the sake of brevity, details are not repeated here.

The method steps in this embodiment may be implemented by means of hardware, and may also be implemented by means of a processor executing software instructions. Software instructions can be composed of corresponding software modules, and software modules can be stored in random access memory (random access memory, RAM), flash memory, read-only memory (read-only memory, ROM), programmable read-only memory (programmable ROM) , PROM), erasable programmable read-only memory (erasable PROM, EPROM), electrically erasable programmable read-only memory (electrically EPROM, EEPROM), register, hard disk, mobile hard disk, CD-ROM or known in the art any other form of storage medium. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. Of course, the storage medium may also be a component of the processor. The processor and storage medium can be located in the ASIC. In addition, the ASIC can be located in a network device or a terminal device. Of course, the processor and the storage medium may also exist in the network device or the terminal device as discrete components.

In the above embodiments, all or part of them may be implemented by software, hardware, firmware or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer programs or instructions. When the computer program or instructions are loaded and executed on the computer, the processes or functions described in the embodiments of the present application are executed in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, network equipment, user equipment, or other programmable devices. The computer program or instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer program or instructions may be downloaded from a website, computer, A server or data center transmits to another website site, computer, server or data center by wired or wireless means. The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server or a data center integrating one or more available media. Described usable medium can be magnetic medium, for example, floppy disk, hard disk, magnetic tape; It can also be optical medium, for example, digital video disc (digital video disc, DVD); It can also be semiconductor medium, for example, solid state drive (solid state drive) , SSD).

The above is only a specific embodiment of the application, but the scope of protection of the application is not limited thereto. Any person familiar with the technical field can easily think of various equivalents within the scope of the technology disclosed in the application. Modifications or replacements, these modifications or replacements shall be covered within the scope of protection of this application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims (21)

A data processing method, characterized in that the method is executed by a terminal, and the method includes: Obtain pending application data; Acquiring scene awareness data, where the scene awareness data is used to indicate factors influencing the terminal to process the application data; determining a first neural network for processing the application data according to the scene perception data and preset conditions, where the preset conditions are used to indicate the corresponding relationship between the scene data affecting the operation of the neural network and the neural network; A processing result of the application data is obtained by using the first neural network. The method according to claim 1, wherein the scene awareness data is used to indicate factors affecting speed and accuracy when the terminal processes the application data. The method according to claim 2, wherein the scene awareness data includes at least one of an external influence factor and an internal influence factor; the external influence factor is used to describe an application scenario in which the terminal acquires the application data Features, the internal influencing factors are used to describe the characteristics of the running scenario of the hardware resources of the terminal running the application data. The method according to claim 3, wherein the external influencing factors include at least one of temperature data, humidity data, illumination data and time data. The method according to claim 3, wherein the internal influencing factors include at least one of computing capability of the processor, available storage capacity and available remaining power. The method according to any one of claims 1 to 5, wherein the determining the first neural network for processing the application data according to the scene perception data and preset conditions comprises: determining parameters of the first neural network corresponding to the scene data including the scene perception data in the preset condition, where the parameters of the first neural network include the number of channels and the number of network layers; Determine the first neural network from a super network according to the parameters of the first neural network, the first neural network is a sub-network in the super network, and the super network is used to determine the scene data corresponding first neural network; Wherein, the number of network layers included in the sub-network is less than the number of network layers included in the super-network, or, the number of channels included in the sub-network is smaller than the number of channels included in the super-network, each network layer Include at least one neuron. The method according to claim 6, wherein the terminal stores parameters of the supernetwork and at least one subnetwork included in the supernetwork; then the parameters of the first neural network are obtained from the supernetwork Determining the first neural network includes: The weight of the first neural network is determined from the weights of the super network according to the parameters of the first neural network. The method according to any one of claims 1 to 7, wherein after the first neural network for processing the application data is determined according to the scene perception data and preset conditions, the method further comprises: Determining a second neural network from the supernetwork, the second neural network is a subnetwork in the supernetwork, the number of network layers included in the second neural network is greater or smaller than the network included in the first neural network the number of layers, or the number of channels included in the second neural network is greater or smaller than the number of channels included in the first neural network; A processing result of the application data is obtained by using the second neural network. The method according to claim 1, further comprising: If the speed and accuracy of the first neural network do not meet user requirements, adjust the scene data corresponding to the first neural network. The method according to any one of claims 1 to 9, further comprising: Displaying the corresponding relationship between the scene data that affects the speed and accuracy of the first neural network and the first neural network, and the processing results. A data processing device, characterized in that the device comprises: A communication module, configured to obtain application data to be processed; The communication module is further configured to acquire scene awareness data, where the scene awareness data is used to indicate factors influencing the terminal to process the application data; A selection module, configured to determine a first neural network for processing the application data according to the scene perception data and a preset condition, the preset condition being used to indicate the corresponding relationship between the scene data that affects the operation of the neural network and the neural network; A detection module, configured to use the first neural network to obtain a processing result of the application data. The device according to claim 11, wherein the scene awareness data is used to indicate factors affecting speed and accuracy when the terminal processes the application data. The device according to claim 12, wherein the scene awareness data includes at least one of an external influence factor and an internal influence factor; the external influence factor is used to describe an application scenario in which the terminal acquires the application data Features, the internal influencing factors are used to describe the characteristics of the running scenario of the hardware resources of the terminal running the application data. The device according to claim 13, wherein the external influencing factors include at least one of temperature data, humidity data, illumination data and time data. The device according to claim 13, wherein the internal influencing factors include at least one of computing capability of the processor, available storage capacity and available remaining power. The device according to any one of claims 11 to 15, wherein when the selection module determines the first neural network for processing the application data according to the scene perception data and preset conditions, it is specifically used for: determining parameters of the first neural network corresponding to the scene data including the scene perception data in the preset condition, where the parameters of the first neural network include the number of channels and the number of network layers; Determine the first neural network from a super network according to the parameters of the first neural network, the first neural network is a sub-network in the super network, and the super network is used to determine the scene data corresponding first neural network; Wherein, the number of network layers included in the sub-network is less than the number of network layers included in the super-network, or, the number of channels included in the sub-network is smaller than the number of channels included in the super-network, each network layer Include at least one neuron. The device according to claim 16, wherein the terminal stores parameters of the supernetwork and at least one subnetwork included in the supernetwork; then the selection module selects from When determining the first neural network in the super network, it is specifically used for: The weight of the first neural network is determined from the weights of the super network according to the parameters of the first neural network. The device according to any one of claims 11 to 17, wherein the selection module is also used for: Determining a second neural network from the supernetwork, the second neural network is a subnetwork in the supernetwork, the number of network layers included in the second neural network is greater or smaller than the network included in the first neural network the number of layers, or the number of channels included in the second neural network is greater or smaller than the number of channels included in the first neural network; The detection module is further configured to use the second neural network to obtain a processing result of the application data. The device according to claim 11, further comprising an update module; The update module is configured to adjust the scene data corresponding to the first neural network if the speed and accuracy of the first neural network do not meet user requirements. The device according to any one of claims 11 to 19, further comprising a display module; The display module is used to display the correspondence between scene data and the first neural network that affect the speed and accuracy of the first neural network, and the processing results. A terminal, characterized in that the terminal includes a memory and at least one processor, the memory is used to store a set of computer instructions; when the processor executes the set of computer instructions, the above-mentioned claims 1 to 1 are executed. The operation steps of the method described in any one of 10.

Images