CN116777814A - Image processing methods, devices, computer equipment, storage media and program products - Google Patents

Abstract

This application provides an image processing method, device, computer equipment, storage medium and program product, relating to the field of image processing. The image processing method includes: obtaining an image to be recognized, and determining weight coefficients of at least two image recognition models respectively for the image to be recognized based on the weight prediction model; wherein the weight coefficient of each image recognition model is consistent with the recognition of the image to be recognized by the image recognition model. The accuracy is directly proportional; the image to be recognized is recognized based on at least two image recognition models respectively, and the predicted label of each image recognition model is obtained; the weight coefficient of the image to be recognized and the weight coefficient of each image recognition model based on at least two image recognition models are obtained. Predict labels to determine the recognition results of the image to be recognized. Embodiments of the present invention can be applied to the map field, and can emphasize the prediction labels of image recognition models with higher recognition accuracy and ignore the prediction labels of image recognition models with lower recognition accuracy, thereby improving the accuracy of the final recognition result.

Description

Image processing methods, devices, computer equipment, storage media and program products

Technical field

This application relates to the technical field of image processing. This application relates to an image processing method, device, computer equipment, storage medium and program product.

Background technique

With the rapid development of information technology, there are usually multiple tools for data samples, such as images, such as multiple different models to process these images. Due to different features extracted, different models have different performance capabilities on different data. Based on a specific fusion strategy, selecting the best-performing model can result in an image processing result with better overall performance.

Currently, for the fusion of image processing models, you can first set the positive and negative samples, calculate the confidence based on the angle between the feature vectors of the positive and negative samples, and then set the weights of the two image processing models. For different images to be processed, the two fused The accuracy of image processing models can be unstable.

Contents of the invention

This application provides an image processing method, device, computer equipment, storage medium and program product, which can solve the problem in related technologies that the accuracy of the fused model cannot be ensured for different images to be processed. The technical solutions are as follows:

On the one hand, an image processing method is provided, and the method includes:

Obtain the image to be recognized, and determine the weight coefficients of at least two image recognition models for the image to be recognized based on the weight prediction model; wherein the weight coefficient of each image recognition model is proportional to the recognition accuracy of the image recognition model for the image to be recognized;

Respectively identify images to be recognized based on at least two image recognition models, and obtain predicted labels for each image recognition model;

The recognition result of the image to be recognized is determined based on the weight coefficients of the at least two image recognition models for the image to be recognized and the predicted labels of each image recognition model.

In a possible implementation, the weight prediction model is trained based on the following method:

Acquire multiple sample images; each sample image is set with a corresponding sample standard annotation;

For each sample image, input the sample image into at least two image recognition models respectively to obtain sample prediction annotations for each image recognition model;

Based on the sample standard annotation and the sample prediction annotation of each image recognition model, determine the recognition accuracy of each image recognition model for the sample image;

A weight prediction model is obtained based on the recognition accuracy of each image recognition model for the sample image.

In one possible implementation, based on the sample standard annotation and the sample prediction annotation of each image recognition model, the recognition accuracy of each image recognition model for the sample image is determined, including:

Match the sample prediction annotation of each image recognition model with the sample standard annotation to obtain the matching degree of the predicted annotation of each sample;

The matching degree of the predicted annotation of each sample is set to the recognition accuracy of the corresponding image recognition model.

In one possible implementation, the weight prediction model is obtained based on the recognition accuracy of each image recognition model for the sample image, including:

Determine the image recognition model with the highest recognition accuracy for the sample image based on the recognition accuracy of each image recognition model;

Based on each sample image, a weight prediction model is obtained for the image recognition model with the highest recognition accuracy for each sample image.

In one possible implementation, the weight prediction model is obtained based on each sample image and the image recognition model with the highest recognition accuracy for each sample image, including:

For each sample image, set the model category of the image recognition model with the highest recognition accuracy of the sample image as the sample classification label;

The initial weight prediction model is trained based on each sample image and the sample classification label corresponding to each sample image to obtain a weight prediction model.

In one possible implementation, the weight coefficients of at least two image recognition models for the image to be recognized are determined based on the weight prediction model, including:

Input the image to be recognized into the weight prediction model to obtain the probability that the image to be recognized belongs to the classification label of at least two image recognition models;

The weight coefficient of each image recognition model is determined based on the probability that the images to be recognized respectively belong to the classification labels of at least two image recognition models.

In one possible implementation, for each image recognition model, the probability that the image to be recognized belongs to the classification label of the image recognition model is proportional to the weight coefficient of the image recognition model.

In one possible implementation, the recognition result of the image to be recognized is determined based on the weight coefficient of the image to be recognized and the predicted label of each image recognition model based on at least two image recognition models, including:

Based on the weight coefficients of at least two image recognition models respectively for the image to be recognized, a weighted sum of predicted labels of each image recognition model is determined to obtain a recognition result.

In one possible implementation, the recognition result of the image to be recognized is determined based on the weight coefficient of the image to be recognized and the predicted label of each image recognition model based on at least two image recognition models, including:

Determine an image recognition model whose weight coefficient is greater than a preset coefficient from at least two image recognition models;

The predicted labels of the image recognition models whose weight coefficients are greater than the preset coefficients are fused to obtain the recognition results.

In one possible implementation, the recognition result of the image to be recognized is determined based on the weight coefficient of the image to be recognized and the predicted label of each image recognition model based on at least two image recognition models, including:

Determine a preset number of image recognition models with the highest weight coefficients from at least two image recognition models;

The predicted labels of the preset number of image recognition models with the highest weight coefficients are fused to obtain the recognition results.

On the other hand, an image processing device is provided, and the device includes:

The first determination module is used to obtain the image to be recognized, and determine the weight coefficients of at least two image recognition models for the image to be recognized based on the weight prediction model; wherein the weight coefficient of each image recognition model and the image recognition model for the image to be recognized are is directly proportional to the recognition accuracy;

A recognition module, configured to respectively identify images to be recognized based on at least two image recognition models, and obtain predicted labels for each image recognition model;

The second determination module is configured to determine the recognition result of the image to be recognized based on the weight coefficient of the image to be recognized and the predicted label of each image recognition model based on at least two image recognition models.

In a possible implementation, a training module is also included for:

Acquire multiple sample images; each sample image is set with a corresponding sample standard annotation;

For each sample image, input the sample image into at least two image recognition models respectively to obtain sample prediction annotations for each image recognition model;

Based on the sample standard annotation and the sample prediction annotation of each image recognition model, determine the recognition accuracy of each image recognition model for the sample image;

A weight prediction model is obtained based on the recognition accuracy of each image recognition model for the sample image.

In one possible implementation, the training module is specifically used to determine the recognition accuracy of each image recognition model for the sample image based on the sample standard annotation and the sample prediction annotation of each image recognition model:

Match the sample prediction annotation of each image recognition model with the sample standard annotation to obtain the matching degree of the predicted annotation of each sample;

The matching degree of the predicted annotation of each sample is set to the recognition accuracy of the corresponding image recognition model.

In one possible implementation, when the training module obtains the weight prediction model based on the recognition accuracy of each image recognition model for the sample image, it is specifically used to:

Determine the image recognition model with the highest recognition accuracy for the sample image based on the recognition accuracy of each image recognition model;

Based on each sample image, a weight prediction model is obtained for the image recognition model with the highest recognition accuracy for each sample image.

In a possible implementation, when the training module obtains the weight prediction model based on each sample image and the image recognition model with the highest recognition accuracy for each sample image, it is specifically used to:

For each sample image, set the model category of the image recognition model with the highest recognition accuracy of the sample image as the sample classification label;

The initial weight prediction model is trained based on each sample image and the sample classification label corresponding to each sample image to obtain a weight prediction model.

In a possible implementation, when the first determination module determines the weight coefficients of at least two image recognition models respectively for the image to be recognized based on the weight prediction model, it is specifically used to:

Input the image to be recognized into the weight prediction model to obtain the probability that the image to be recognized belongs to the classification label of at least two image recognition models;

The weight coefficient of each image recognition model is determined based on the probability that the images to be recognized respectively belong to the classification labels of at least two image recognition models.

In one possible implementation, for each image recognition model, the probability that the image to be recognized belongs to the classification label of the image recognition model is proportional to the weight coefficient of the image recognition model.

In one possible implementation, when determining the recognition result of the image to be recognized based on at least two image recognition models for the weight coefficient of the image to be recognized and the prediction label of each image recognition model, the second determination module is specifically used to:

Based on the weight coefficients of at least two image recognition models respectively for the image to be recognized, a weighted sum of predicted labels of each image recognition model is determined to obtain a recognition result.

In one possible implementation, when determining the recognition result of the image to be recognized based on at least two image recognition models for the weight coefficient of the image to be recognized and the prediction label of each image recognition model, the second determination module is specifically used to:

Determine an image recognition model whose weight coefficient is greater than a preset coefficient from at least two image recognition models;

The predicted labels of the image recognition models whose weight coefficients are greater than the preset coefficients are fused to obtain the recognition results.

In one possible implementation, when determining the recognition result of the image to be recognized based on at least two image recognition models for the weight coefficient of the image to be recognized and the prediction label of each image recognition model, the second determination module is specifically used to:

Determine a preset number of image recognition models with the highest weight coefficients from at least two image recognition models;

The predicted labels of the preset number of image recognition models with the highest weight coefficients are fused to obtain the recognition results.

In one possible implementation, when determining the recognition result of the image to be recognized based on at least two image recognition models for the weight coefficient of the image to be recognized and the prediction label of each image recognition model, the second determination module is specifically used to:

Determine a target recognition model with a weight coefficient greater than a preset coefficient from at least two image recognition models;

Based on the determined weight coefficient of the target recognition model, the predicted labels of the target recognition model are fused to obtain the recognition result.

On the other hand, a computer device is provided, including a memory, a processor, and a computer program stored on the memory. The processor executes the computer program to implement the above image processing method.

On the other hand, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the above image processing method is implemented.

On the other hand, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the above image processing method is implemented.

The beneficial effects brought by the technical solution provided by this application are:

A weight prediction model is used to predict the weight coefficients of at least two image recognition models for the image to be recognized, and the weight coefficient of each image recognition model is related to the recognition accuracy of the image recognition model for the image to be recognized. Proportional to each other, the final recognition result is determined based on the weight coefficients of different image recognition models and the predicted labels of each image recognition model. The predicted labels of the image recognition models with higher recognition accuracy can be emphasized and the prediction labels with lower recognition accuracy can be ignored. The predicted labels of the image recognition model, thereby improving the accuracy of the final recognition results.

Further, the initial weight prediction model is trained through each sample image and the image recognition model with the highest recognition accuracy for each sample image, so that the weight prediction model can output the same recognition as the image recognition model for the image to be recognized. The weight coefficient is proportional to the accuracy, thereby improving the accuracy of the final recognition result.

Description of drawings

In order to explain the technical solutions in the embodiments of the present application more clearly, the drawings needed to be used in the description of the embodiments of the present application will be briefly introduced below.

Figure 1 is a schematic diagram of the implementation environment of an image processing method provided by an embodiment of the present application;

Figure 2 is a schematic flow chart of an image processing method provided by an embodiment of the present application;

Figure 3 is a schematic diagram of the acquisition scheme of the weight prediction model provided by the embodiment of the present application;

Figure 4 is a schematic diagram of a scheme for obtaining recognition results provided by the example of this application;

Figure 5 is a schematic diagram of a scheme for obtaining recognition results provided by the example of this application;

Figure 6 is a schematic diagram of the expression processing solution provided by this application example;

Figure 7 is a schematic structural diagram of an image processing device provided by an embodiment of the present application;

FIG. 8 is a schematic structural diagram of a computer device provided by an embodiment of the present application.

Detailed ways

The embodiments of the present application are described below with reference to the drawings in the present application. It should be understood that the embodiments described below in conjunction with the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application, and do not limit the technical solutions of the embodiments of the present application.

Those skilled in the art will understand that, unless expressly stated otherwise, the singular forms "a", "an", "the" and "the" used herein may also include the plural form. It should be further understood that the terms "comprising" and "including" used in the embodiments of this application mean that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements and/or components, but do not exclude Implementation is other features, information, data, steps, operations, elements, components and/or their combinations supported by the technical field. It should be understood that when we refer to an element being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element, or one element and the other element may be connected to the other element through intervening elements. Establish connections. Additionally, "connected" or "coupled" as used herein may include wireless connections or wireless couplings. The term "and/or" as used herein indicates at least one of the items defined by the term, for example, "A and/or B" indicates implemented as "A", or implemented as "A", or implemented as "A and B" ".

In order to make the purpose, technical solutions and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

With the rapid development of information technology, there are a large number of data samples, and there are usually multiple tools (such as models) to process these data. Due to different extracted features, different models have different performance capabilities on different data. Taking this into account, based on a specific fusion strategy and dynamically selecting the best performing model in the local space, a fusion model with better overall performance can be obtained. Generally speaking, as the number of individual models in the ensemble increases, the error rate of the ensemble will decrease exponentially. However, when the performance gap between different models is too large, the result of model fusion will be lower than the accuracy of a single model.

The related technology determines a classification threshold for classifying multiple training samples for each of the multiple classification models through the classification threshold; for each space in the multiple classification models, outputs of the classification model with respect to the multiple training samples are The score is divided into multiple subspaces according to the probability density of each output score, thereby determining the confidence of each unit in the multiple subspaces. The confidence indicates the confidence level of the output score of each unit, and then based on each of the multiple classification models The predetermined weights and the classification thresholds of each classification model are used to fuse the classification thresholds of multiple classification models.

This method simply calculates the classification threshold based on positive and negative sample indicators. This threshold is not directly related to the results predicted by the final model, but only indirectly improves the model fusion strategy through the final results.

In some related technologies, the feature vector of each face image in the positive sample and the negative sample is obtained; according to the feature vector, the two face recognition models are respectively obtained for the positive sample and the negative sample. The angle between the feature vectors of the face image in can be used to obtain the confidence of the corresponding face recognition model; compare the confidence results of the two face recognition models, and obtain the weight values of the two face recognition models based on the comparison results. The combination; according to the combination of the weight values, calculate the weight of the two face recognition models and determine the face recognition fusion model to ensure that the face recognition fusion model generated by using the two face recognition models The face recognition accuracy is higher than the accuracy of a single face recognition model.

This method calculates the confidence based on the angle of the feature vector, thereby indirectly setting the weight, and the number of models is limited. It cannot ensure the accuracy of the recognition results for different images to be recognized.

This application uses the weight prediction model to emphasize models with better performance and ignore models with poorer performance. Even if the performance gap between different models is large, a better fusion effect can be achieved, and the accuracy of the obtained recognition results is higher.

The embodiments of this application can be applied to various scenarios, including but not limited to artificial intelligence. For example, the image processing method provided by this application can be applied to water body recognition. Multiple image semantic segmentation models are obtained through training on existing data sets. Each model has different performance effects in different regions. Through image recognition based on weight prediction The model fusion strategy can maximize strengths and avoid weaknesses, give full play to the advantages of multiple image recognition models, and obtain accurate recognition results.

Artificial Intelligence (AI) is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is the study of the design principles and implementation methods of various intelligent machines, so that the machines have the functions of perception, reasoning and decision-making.

Artificial intelligence technology is a comprehensive subject that covers a wide range of fields, including both hardware-level technology and software-level technology. Basic artificial intelligence technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation/interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, machine learning/deep learning, autonomous driving, smart transportation and other major directions.

Machine Learning (ML) is a multi-field interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers can simulate or implement human learning behavior to acquire new knowledge or skills, and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications cover all fields of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, teaching learning and other technologies.

Figure 1 is a schematic diagram of the implementation environment of an image processing method provided by an embodiment of the present invention. Refer to Figure 1. Specifically, sample images are respectively identified according to at least two image recognition models, and sample prediction annotations are obtained, thereby determining each image recognition The model aims at the recognition accuracy of the sample image, and then uses the recognition accuracy of each image recognition model and the sample image to train the initial weight prediction model to obtain the weight prediction model. The weight prediction model is used to determine at least two image recognition models that are respectively suitable for the target. The weight coefficient of the image to be recognized is described, and then the final recognition result is determined based on the predicted label and weight coefficient of each image recognition model for the image to be recognized.

It can be understood that Figure 1 represents an application scenario in an example and does not limit the application scenarios of the image processing method of the present application.

FIG. 2 is a schematic flowchart of an image processing method provided by an embodiment of the present application. The execution subject of this method may be a computer device. As shown in Figure 2, the method may include the following steps:

Step 201: Obtain the image to be recognized, and determine the weight coefficients of at least two image recognition models for the image to be recognized based on the weight prediction model.

Among them, the weight coefficient of each image recognition model is proportional to the recognition accuracy of the image recognition model for the image to be recognized.

Among them, the image recognition model can be a semantic segmentation network, such as a fully convolution network (FCN), UNet network, UNet++ network, HRNet network (high-resolution network); among them, UNet network and UNet++ network are both full-volume Machine neural network is a pixel-level classification, and the output is the category of each pixel.

Specifically, the process of image recognition by the image recognition model can be the process of predicting the image pixel by pixel by the semantic segmentation network.

Taking Unet++ as an example, Unet++ introduces a built-in variable-depth UNet collection that provides improved segmentation performance for objects of different sizes, and redesigns jumpers in UNet++ to enable flexible features in the decoder. Fusion, by designing a scheme to prune the trained UNet++ to speed up its inference while maintaining its performance, while training multi-depth UNet embedded in the UNet++ architecture can stimulate collaborative learning among the component UNet, and It can lead to better performance than training isolated UNet with the same architecture separately.

Taking HRNet as an example, HRnet changes the link between high and low resolutions from series to parallel, maintains high-resolution representation throughout the network structure, and introduces interaction between high and low resolutions to improve model performance.

The weight prediction model may be a convolutional neural network. The input of the weight prediction model is an image, and the output image is a probability that it is suitable for recognition by at least two image recognition models, that is, the probability that the output is a different image recognition model.

Take the weight prediction model BagNet as an example. BagNet is a convolutional neural network for image classification. BagNet does not consider spatial sorting, but classifies images based on small local features of the image. Its constraints on local features can be directly Analyze how each part of the image affects classification. This method makes it focus more on the overall information of the image, and can better learn useful spatial information and eliminate the influence of noisy data. The brief process is as follows:

1) Cut the input image into image blocks of a certain size of pixels;

2) After intercepting the image blocks, use a 1×1 convolutional deep network on each image block to obtain the category vector;

3) Sum all output category vectors according to space;

4) Predict the classification category through the largest element count of the category vector, and output the predicted probability of each category. The probability obtained for each category is the probability that the image is suitable for recognition by at least two image recognition models.

Specifically, the sample images can be recognized based on at least two image recognition models, and the recognition accuracy of each image recognition model for the sample images can be determined, so as to determine what kind of sample images are suitable for different image recognition models, and then based on different The recognition accuracy of the image recognition model and the sample image training are used to obtain the weight prediction model. The specific acquisition process of the weight prediction model will be further elaborated below.

Step 202: Recognize the image to be recognized based on at least two image recognition models, and obtain the predicted label of each image recognition model.

Among them, the image recognition model is trained based on the training samples and the training labels corresponding to the training samples.

Specifically, the images to be recognized are input into different image recognition models respectively, and the predicted labels of the images to be recognized by different image recognition models are obtained.

Step 203: Determine the recognition result of the image to be recognized based on the weight coefficients of the at least two image recognition models for the image to be recognized and the predicted labels of each image recognition model.

In some implementations, based on the weight coefficient, the predicted label of the image recognition model with the largest weight coefficient can be selected to determine the recognition result.

In other embodiments, based on the weight coefficients of different image recognition models, an image recognition model with a weight coefficient greater than a preset threshold can be selected from multiple image recognition models, and the recognition can be determined based on the predicted label of the selected image recognition model. result.

In some embodiments, the prediction labels of different image recognition models can also be fused based on the weight coefficients of different image recognition models to obtain recognition results. The specific process of obtaining the recognition results will be further elaborated below.

In the above embodiment, the weight prediction model is used to predict the weight coefficients of at least two image recognition models for the image to be recognized, and the weight coefficient of each image recognition model is proportional to the recognition accuracy of the image recognition model for the image to be recognized, The final recognition result is then determined based on the weight coefficients of different image recognition models and the predicted labels of each image recognition model. The predicted labels of the image recognition models with higher recognition accuracy can be emphasized and the image recognition models with lower recognition accuracy can be ignored. predicted labels, thereby improving the accuracy of the final recognition result.

The process of obtaining the weight prediction model will be further explained below with reference to embodiments.

In a possible implementation, as shown in Figure 3, the weight prediction model can be trained based on the following method:

Step S301: Obtain multiple sample images.

Among them, each sample image is set with a corresponding sample standard annotation, and the sample standard annotation can be close to or ideally equivalent to the real annotation of the sample.

Step S302: For each sample image, input the sample image into at least two image recognition models respectively to obtain sample prediction annotations for each image recognition model.

Among them, the image recognition model is a model that has been trained.

It can be understood that the purpose of inputting the sample images here to the image recognition model is not to train the image recognition model, but to judge the recognition accuracy of the trained image recognition model for the sample images.

Step S303: Determine the recognition accuracy of each image recognition model for the sample image based on the sample standard annotation and the sample prediction annotation of each image recognition model.

Specifically, step S303 determines the recognition accuracy of each image recognition model for the sample image based on the sample standard annotation and the sample prediction annotation of each image recognition model, which may include:

(1) Match the sample prediction annotation of each image recognition model with the sample standard annotation to obtain the matching degree of the predicted annotation of each sample;

(2) Set the matching degree of the predicted annotation of each sample to the recognition accuracy of the corresponding image recognition model.

Specifically, the matching degree between the sample prediction annotation and the sample standard annotation can be calculated by calculating the similarity. The coincidence degree between the sample prediction annotation and the sample standard annotation can also be calculated by, for example, calculating the intersection ratio. That is, the matching degree. The specific method of determining the matching degree is not limited here.

Step S304: Obtain a weight prediction model based on the recognition accuracy of each image recognition model for the sample image.

Specifically, step S304 obtains a weight prediction model based on the recognition accuracy of each image recognition model for the sample image, which may include:

(1) Based on the recognition accuracy of each image recognition model, determine the image recognition model with the highest accuracy for sample image recognition;

(2) Based on each sample image, obtain the weight prediction model for the image recognition model with the highest recognition accuracy for each sample image.

In the specific implementation process, the weight prediction model is obtained based on each sample image and the image recognition model with the highest recognition accuracy for each sample image, which may include:

a. For each sample image, set the model category of the image recognition model with the highest recognition accuracy of the sample image as the sample classification label;

b. Train the initial weight prediction model based on each sample image and the sample classification label corresponding to each sample image to obtain a weight prediction model.

During the specific implementation process, the initial weight prediction model is trained through each sample image and the sample classification label corresponding to each sample image, so that the sample classification label with the highest probability output by the initial weight prediction model is the one with the highest recognition accuracy as much as possible. Image recognition model.

Specifically, when the sample image is input to the weight prediction model, the probability of candidate classification labels of different image recognition models will be obtained, and the sample classification label of the image recognition model with the highest probability will be further output.

In the above embodiment, the initial weight prediction model is trained through each sample image and the image recognition model with the highest recognition accuracy for each sample image, so that the weight prediction model can output the same recognition accuracy as the image recognition model for the image to be recognized. Proportional weight coefficient, thereby improving the accuracy of the final recognition result.

The above embodiments illustrate the specific acquisition process of obtaining the weight prediction model. The process of obtaining the weight coefficient will be further explained below with reference to the accompanying drawings and embodiments.

In one possible implementation, step S201 determines the weight coefficients of at least two image recognition models respectively for the image to be recognized based on the weight prediction model, which may include:

(1) Input the image to be recognized into the weight prediction model and obtain the probability that the image to be recognized belongs to the classification label of at least two image recognition models;

(2) Determine the weight coefficient of each image recognition model based on the probability that the image to be recognized belongs to the classification labels of at least two image recognition models.

Specifically, for each image recognition model, the higher the probability of the classification label, the more likely the image recognition model corresponding to the classification label will achieve better recognition results for the image to be recognized, that is, the image to be recognized belongs to the classification label of the image recognition model The probability is proportional to the weight coefficient of the image recognition model.

In the specific implementation process, the probability that the image to be recognized belongs to the classification label of the image recognition model can be directly set as the weight coefficient of the image recognition model.

The above embodiments illustrate the determination process of the weight coefficient, and the specific acquisition process of the image recognition result will be further elaborated below with reference to the accompanying drawings and embodiments.

In some possible implementations, the predicted labels of all image recognition models can be fused to obtain the recognition result.

Specifically, step S203 determines the recognition result of the image to be recognized based on the weight coefficient of the image to be recognized and the prediction label of each image recognition model based on at least two image recognition models, which may include:

Based on the weight coefficients of at least two image recognition models respectively for the image to be recognized, a weighted sum of predicted labels of each image recognition model is determined to obtain a recognition result.

Specifically, as shown in Figure 4, the weighted sum of the predicted labels of all image recognition models is calculated based on their respective weight coefficients to obtain the recognition results.

In some possible implementations, the recognition result may be determined based on a preset number of image recognition models with the highest weight coefficients.

Specifically, step S203 determines the recognition result of the image to be recognized based on the weight coefficient of the image to be recognized and the prediction label of each image recognition model based on at least two image recognition models, which may include:

Based on the weight coefficients of at least two image recognition models for the image to be recognized, determine a preset number of image recognition models with the highest weight coefficients;

The predicted labels of the preset number of image recognition models with the highest weight coefficients are fused to obtain the recognition results.

Specifically, after a preset number of image recognition models with the highest weight coefficients are determined, the weight coefficients of these image recognition models can be processed proportionally.

As shown in Figure 5, the weight coefficients of the two image recognition models with the highest weight coefficients are 0.5 and 0.3 respectively. These weight coefficients can be adjusted proportionally to 0.625 and 0.375.

In some possible implementations, prediction labels of image recognition models with larger weight coefficients can be fused to obtain recognition results.

Specifically, step S203 determines the recognition result of the image to be recognized based on the weight coefficient of the image to be recognized and the prediction label of each image recognition model based on at least two image recognition models, which may include:

Determine an image recognition model whose weight coefficient is greater than a preset coefficient from at least two image recognition models;

The predicted labels of the image recognition models whose weight coefficients are greater than the preset coefficients are fused to obtain the recognition results.

Specifically, the image recognition models can be screened based on the weight coefficients to determine whether the weight coefficients are greater than the preset coefficients. After determining the image recognition models whose weight coefficients are greater than the preset coefficients, the weight coefficients of these image recognition models can be proportionally calculated. Processing, the specific process can be found in the above processing method, and will not be repeated here.

In order to explain the image processing method of the present application more clearly, the image processing method of the present application will be further described below with examples.

As shown in Figure 6, in one example, the image processing method of this application may include the following steps:

1) Acquire multiple sample images, that is, the source data test set images shown in the figure; each sample image is set with a corresponding sample standard annotation, that is, the source data test set annotation shown in the figure;

2) For each sample image, input the sample image into at least two image recognition models respectively to obtain the sample prediction annotation of each image recognition model; wherein, the image recognition model is trained based on the initial sample and initial annotation, that is, The source data training set images shown in the figure and the source data training set annotation training are obtained;

3) Match the sample standard annotation with the sample prediction annotation of each image recognition model, that is, the evaluation index calculation shown in the figure;

4) Determine the recognition accuracy of each image recognition model for the sample image, expressed by the model score corresponding to each image recognition model in the figure;

5) Select the image recognition model with the highest model score, and set the model category of the image recognition model to the sample classification label, that is, the annotation category shown in the figure;

6) Train the initial weight prediction model based on each sample image and the sample classification label corresponding to each sample image to obtain the weight prediction model, that is, obtain the neural network shown in the figure;

7) Input the image to be recognized into the weight prediction model, and obtain the probability that the image to be recognized belongs to the classification label of at least two image recognition models, that is, the probability of category 1, the probability of category 2, and the probability of category n shown in the figure;

8) Determine the weight coefficient of each image recognition model based on the probability that the image to be recognized belongs to the classification label of at least two image recognition models;

9) Respectively identify the image to be recognized based on at least two image recognition models, and obtain the predicted label of each image recognition model; based on the weight coefficient of the image to be recognized and the predicted label of each image recognition model based on at least two image recognition models, Determine the recognition result of the image to be recognized.

The above image processing method uses a weight prediction model to predict the weight coefficients of at least two image recognition models for the image to be recognized, and the weight coefficient of each image recognition model is proportional to the recognition accuracy of the image recognition model for the image to be recognized, The final recognition result is then determined based on the weight coefficients of different image recognition models and the predicted labels of each image recognition model. The predicted labels of the image recognition models with higher recognition accuracy can be emphasized and the image recognition models with lower recognition accuracy can be ignored. predicted labels, thereby improving the accuracy of the final recognition result.

Further, the initial weight prediction model is trained through each sample image and the image recognition model with the highest recognition accuracy for each sample image, so that the weight prediction model can output an output proportional to the image recognition model's recognition accuracy for the image to be recognized. weight coefficient, thereby improving the accuracy of the final recognition result.

FIG. 7 is a schematic structural diagram of an image processing device provided by an embodiment of the present application. As shown in Figure 7, the device includes:

The first determination module 701 is used to obtain the image to be recognized, and determine the weight coefficients of at least two image recognition models for the image to be recognized based on the weight prediction model; wherein the weight coefficient of each image recognition model and the image recognition model for the image to be recognized are The recognition accuracy of the image is directly proportional;

The identification module 702 is used to identify images to be recognized based on at least two image recognition models, and obtain predicted labels for each image recognition model;

The second determination module 703 is configured to determine the recognition result of the image to be recognized based on the weight coefficient of the image to be recognized and the prediction label of each image recognition model of at least two image recognition models.

In a possible implementation, a training module is also included for:

Acquire multiple sample images; each sample image is set with a corresponding sample standard annotation;

For each sample image, input the sample image into at least two image recognition models respectively to obtain sample prediction annotations for each image recognition model;

Based on the sample standard annotation and the sample prediction annotation of each image recognition model, determine the recognition accuracy of each image recognition model for the sample image;

A weight prediction model is obtained based on the recognition accuracy of each image recognition model for the sample image.

In one possible implementation, the training module is specifically used to determine the recognition accuracy of each image recognition model for the sample image based on the sample standard annotation and the sample prediction annotation of each image recognition model:

Match the sample prediction annotation of each image recognition model with the sample standard annotation to obtain the matching degree of the predicted annotation of each sample;

The matching degree of the predicted annotation of each sample is set to the recognition accuracy of the corresponding image recognition model.

In one possible implementation, when the training module obtains the weight prediction model based on the recognition accuracy of each image recognition model for the sample image, it is specifically used to:

Determine the image recognition model with the highest recognition accuracy for the sample image based on the recognition accuracy of each image recognition model;

Based on each sample image, a weight prediction model is obtained for the image recognition model with the highest recognition accuracy for each sample image.

In a possible implementation, when the training module obtains the weight prediction model based on each sample image and the image recognition model with the highest recognition accuracy for each sample image, it is specifically used to:

For each sample image, set the model category of the image recognition model with the highest recognition accuracy of the sample image as the sample classification label;

The initial weight prediction model is trained based on each sample image and the sample classification label corresponding to each sample image to obtain a weight prediction model.

In one possible implementation, when the first determination module 701 determines the weight coefficients of at least two image recognition models respectively for the image to be recognized based on the weight prediction model, it is specifically used to:

Input the image to be recognized into the weight prediction model to obtain the probability that the image to be recognized belongs to the classification label of at least two image recognition models;

The weight coefficient of each image recognition model is determined based on the probability that the images to be recognized respectively belong to the classification labels of at least two image recognition models.

In one possible implementation, for each image recognition model, the probability that the image to be recognized belongs to the classification label of the image recognition model is proportional to the weight coefficient of the image recognition model.

In one possible implementation, when the second determination module 703 determines the recognition result of the image to be recognized based on the weight coefficient of the image to be recognized and the prediction label of each image recognition model based on at least two image recognition models, it is specifically used to: :

Based on the weight coefficients of at least two image recognition models respectively for the image to be recognized, a weighted sum of predicted labels of each image recognition model is determined to obtain a recognition result.

In one possible implementation, when the second determination module 703 determines the recognition result of the image to be recognized based on the weight coefficient of the image to be recognized and the prediction label of each image recognition model based on at least two image recognition models, it is specifically used to: :

Determine an image recognition model whose weight coefficient is greater than a preset coefficient from at least two image recognition models;

The predicted labels of the image recognition models whose weight coefficients are greater than the preset coefficients are fused to obtain the recognition results.

In one possible implementation, when the second determination module 703 determines the recognition result of the image to be recognized based on the weight coefficient of the image to be recognized and the prediction label of each image recognition model based on at least two image recognition models, it is specifically used to: :

Determine a preset number of image recognition models with the highest weight coefficients from at least two image recognition models;

The predicted labels of the preset number of image recognition models with the highest weight coefficients are fused to obtain the recognition results.

The above image processing device uses a weight prediction model to predict the weight coefficients of at least two image recognition models for the image to be recognized, and the weight coefficient of each image recognition model is proportional to the recognition accuracy of the image recognition model for the image to be recognized. , and then determine the final recognition result based on the weight coefficients of different image recognition models and the predicted labels of each image recognition model. The predicted labels of the image recognition models with higher recognition accuracy can be emphasized and the image recognition with lower recognition accuracy can be ignored. model’s predicted labels, thereby improving the accuracy of the final recognition result.

Further, the initial weight prediction model is trained through each sample image and the image recognition model with the highest recognition accuracy for each sample image, so that the weight prediction model can output an output proportional to the image recognition model's recognition accuracy for the image to be recognized. weight coefficient, thereby improving the accuracy of the final recognition result.

On the other hand, a computer device is provided, including a memory, a processor, and a computer program stored on the memory. The processor executes the computer program to implement the above image processing method.

Figure 8 is a schematic structural diagram of a computer device provided in an embodiment of the present application. As shown in Figure 8, the computer device includes: a memory and a processor; at least one program is stored in the memory and used for execution by the processor. Compared with the existing technology, it can achieve:

A weight prediction model is used to predict the weight coefficients of at least two image recognition models for the image to be recognized, and the weight coefficient of each image recognition model is related to the recognition accuracy of the image recognition model for the image to be recognized. Proportional to each other, the final recognition result is determined based on the weight coefficients of different image recognition models and the predicted labels of each image recognition model. The predicted labels of the image recognition models with higher recognition accuracy can be emphasized and the prediction labels with lower recognition accuracy can be ignored. The predicted labels of the image recognition model, thereby improving the accuracy of the final recognition results.

In an optional embodiment, a computer device is provided, as shown in Figure 8. The computer device 800 shown in Figure 8 includes: a processor 801 and a memory 803. Among them, the processor 801 is connected to the memory 803, such as through a bus 802. Optionally, the computer device 800 may also include a transceiver 804, which may be used for data interaction between the computer device and other computer devices, such as data transmission and/or data reception. It should be noted that in practical applications, the number of transceivers 804 is not limited to one, and the structure of the computer device 800 does not constitute a limitation on the embodiments of the present application.

The processor 801 may be a CPU (Central Processing Unit, central processing unit), a general-purpose processor, a DSP (Digital Signal Processor, data signal processor), ASIC (Application Specific Integrated Circuit, application specific integrated circuit), FPGA (Field Programmable Gate Array, field programmable gate array) or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various illustrative logical blocks, modules, and circuits described in connection with this disclosure. The processor 801 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

Bus 802 may include a path that carries information between the above-mentioned components. The bus 802 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industrial Standard Architecture) bus, etc. The bus 802 can be divided into an address bus, a data bus, a control bus, etc. For ease of presentation, only one thick line is used in Figure 8, but it does not mean that there is only one bus or one type of bus.

The memory 803 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, RAM (Random Access Memory, random access memory) or other types that can store information and instructions. Dynamic storage devices can also be EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact DiscRead Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, Laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), magnetic disk storage medium or other magnetic storage device, or any other device capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer medium, but not limited to this.

The memory 803 is used to store application code (computer program) for executing the solution of the present application, and is controlled by the processor 801 for execution. The processor 801 is used to execute the application program code stored in the memory 803 to implement the contents shown in the foregoing method embodiments.

Among them, computer equipment includes but is not limited to: virtualized computer equipment, virtual machines, servers, service clusters, user terminals, etc.

Embodiments of the present application provide a computer-readable storage medium. The computer-readable storage medium stores a computer program. When run on a computer, the computer can execute the corresponding content of the image processing method in the foregoing method embodiment.

Embodiments of the present application provide a computer program product or computer program. The computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the above image processing method.

It should be understood that although various steps in the flowchart of the accompanying drawings are shown in sequence as indicated by arrows, these steps are not necessarily performed in the order indicated by arrows. Unless explicitly stated in this article, the execution of these steps is not strictly limited in order, and they can be executed in other orders. Moreover, at least some of the steps in the flow chart of the accompanying drawings may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but may be executed at different times, and their execution order is also It does not necessarily need to be performed sequentially, but may be performed in turn or alternately with other steps or sub-steps of other steps or at least part of the stages.

It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two. The computer-readable storage medium may be, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination thereof. More specific examples of computer readable storage media may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard drive, random access memory (RAM), read only memory (ROM), removable Programmed read-only memory (EPROM or flash memory), fiber optics, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In this disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device . Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wire, optical cable, RF (radio frequency), etc., or any suitable combination of the above.

The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; it may also exist independently without being assembled into the electronic device.

The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device performs the method shown in the above embodiment.

Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional Procedural programming language—such as "C" or a similar programming language. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In situations involving remote computers, the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as an Internet service provider through Internet connection).

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of code that contains one or more logic functions that implement the specified executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown one after another may actually execute substantially in parallel, or they may sometimes execute in the reverse order, depending on the functionality involved. It will also be noted that each block of the block diagram and/or flowchart illustration, and combinations of blocks in the block diagram and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or operations. , or can be implemented using a combination of specialized hardware and computer instructions.

The modules involved in the embodiments of the present disclosure can be implemented in software or hardware. The name of a module does not constitute a limitation on the module itself under certain circumstances. For example, the first determination module can also be described as a "module for determining weight coefficients."

The above description is only a description of the preferred embodiments of the present disclosure and the technical principles applied. Those skilled in the art should understand that the disclosure scope involved in the present disclosure is not limited to technical solutions composed of specific combinations of the above technical features, but should also cover solutions composed of the above technical features or without departing from the above disclosed concept. Other technical solutions formed by any combination of equivalent features. For example, a technical solution is formed by replacing the above features with technical features with similar functions disclosed in this disclosure (but not limited to).

Claims (14)

1. An image processing method, characterized in that the method includes: Obtain the image to be recognized, and determine the weight coefficients of at least two image recognition models respectively for the image to be recognized based on the weight prediction model; Wherein, the weight coefficient of each image recognition model is proportional to the recognition accuracy of the image recognition model for the image to be recognized; Respectively identify the image to be recognized based on the at least two image recognition models, and obtain the predicted label of each image recognition model; The recognition result of the image to be recognized is determined based on the weight coefficients of the at least two image recognition models for the image to be recognized and the predicted labels of each image recognition model. 2. The image processing method according to claim 1, characterized in that the weight prediction model is trained based on the following method: Acquire multiple sample images; each sample image is provided with a corresponding sample standard annotation; For each sample image, input the sample image into the at least two image recognition models respectively to obtain sample prediction annotations for each of the image recognition models; Determine the recognition accuracy of each of the image recognition models for the sample image based on the sample standard annotation and the sample predictive annotation of each of the image recognition models; The weight prediction model is obtained based on the recognition accuracy of each of the image recognition models for the sample image. 3. The image processing method according to claim 2, characterized in that, based on the sample standard annotation and the sample prediction annotation of each of the image recognition models, it is determined that each of the image recognition models is The recognition accuracy of sample images includes: Match the sample prediction annotation of each image recognition model with the sample standard annotation to obtain the matching degree of each sample prediction annotation; The matching degree of the predicted annotation of each sample is set as the recognition accuracy of the corresponding image recognition model. 4. The image processing method according to claim 2, wherein obtaining the weight prediction model based on the recognition accuracy of each of the image recognition models for the sample image includes: Determine the image recognition model with the highest recognition accuracy for the sample image based on the recognition accuracy of each image recognition model; The weight prediction model is obtained based on each sample image and the image recognition model with the highest recognition accuracy for each sample image. 5. The image processing method according to claim 4, characterized in that, based on each sample image, obtaining the weight prediction model for the image recognition model with the highest recognition accuracy for each sample image includes: For each sample image, set the model category of the image recognition model with the highest recognition accuracy of the sample image as the sample classification label; The initial weight prediction model is trained based on each sample image and the sample classification label corresponding to each sample image to obtain the weight prediction model. 6. The image processing method according to claim 5, wherein the determining based on the weight prediction model of at least two image recognition models respectively for the weight coefficients of the image to be recognized includes: Input the image to be recognized into the weight prediction model to obtain the probability that the image to be recognized respectively belongs to the classification label of the at least two image recognition models; The weight coefficient of each of the image recognition models is determined based on the probability that the images to be recognized respectively belong to the classification labels of the at least two image recognition models. 7. The image processing method according to claim 6, characterized in that, for each of the image recognition models, the probability that the image to be recognized belongs to the classification label of the image recognition model is equal to the weight of the image recognition model. The coefficient is proportional. 8. The image processing method according to claim 1, wherein the weight coefficient of the image to be recognized and the prediction label of each image recognition model are determined based on the at least two image recognition models. Describe the recognition results of the image to be recognized, including: Based on the weight coefficients of the at least two image recognition models respectively for the image to be recognized, a weighted sum of predicted labels of each of the image recognition models is determined to obtain the recognition result. 9. The image processing method according to claim 1, wherein the weight coefficient of the image to be recognized and the prediction label of each image recognition model are determined based on the at least two image recognition models. Describe the recognition results of the image to be recognized, including: Determine an image recognition model whose weight coefficient is greater than a preset coefficient from at least two image recognition models; The predicted labels of the image recognition model whose weight coefficients are greater than the preset coefficient are fused to obtain the recognition result. 10. The image processing method according to claim 1, wherein the weight coefficient of the image to be recognized and the prediction label of each image recognition model are determined based on the at least two image recognition models. Describe the recognition results of the image to be recognized, including: Determine a preset number of image recognition models with the highest weight coefficients from at least two image recognition models; The predicted labels of the image recognition models of the preset number with the highest weight coefficients are fused to obtain the recognition result. 11. An image processing device, characterized in that the device includes: The first determination module is used to obtain the image to be recognized, and determine the weight coefficients of at least two image recognition models respectively for the image to be recognized based on the weight prediction model; wherein the weight coefficient of each image recognition model is consistent with the image The recognition accuracy of the recognition model for the image to be recognized is proportional to the recognition accuracy; A recognition module, configured to respectively recognize the image to be recognized based on the at least two image recognition models, and obtain the predicted label of each image recognition model; The second determination module is configured to determine the recognition result of the image to be recognized based on the weight coefficient of the image to be recognized and the prediction label of each image recognition model respectively based on the at least two image recognition models. 12. A computer device, comprising a memory, a processor and a computer program stored on the memory, characterized in that the processor executes the computer program to implement the image processing method according to any one of claims 1 to 10 . 13. A computer-readable storage medium with a computer program stored thereon, characterized in that when the computer program is executed by a processor, the image processing method according to any one of claims 1 to 10 is implemented. 14. A computer program product, comprising a computer program, characterized in that when the computer program is executed by a processor, the image processing method according to any one of claims 1 to 10 is implemented.