CN115880263A - Image quality scoring method, device, system, storage medium and electronic equipment - Google Patents

Abstract

The application discloses an image quality scoring method, which comprises the following steps: inputting an original image to be scored and a label corresponding to each current service type into a pre-trained state recognition model, marking an image area related to each current service type, and performing feature extraction based on the image area to obtain a first state feature vector corresponding to each current service type; for each first state feature vector, calculating the similarity of the first state feature vector and the state feature vectors of the R x P standard images corresponding to the corresponding current service type respectively to obtain R x P similarity results; and for each first state feature vector, determining a quality scoring result of the original image to be scored under the corresponding current service type based on the R x P similarity result. By applying the method and the device, reasonable image quality judgment can be performed in a targeted manner corresponding to different service types, so that the quality judgment result is more accurate.

Description

Image quality scoring method, device, system, storage medium and electronic equipment

technical field

The present application relates to image processing technology, in particular to a method, device, system, storage medium and electronic equipment for evaluating the quality of images used for state recognition.

Background technique

With the advancement of image processing technology, the technology of state recognition using images has been more and more widely used.

When the image quality is poor, it will affect the performance of the state recognition system and lead to state recognition errors. For example, judging illegal behavior in the cab is a typical application of state recognition using images. When the image quality of the collected cab is poor, it will affect the performance of the cab illegal behavior judgment model, resulting in false alarms of illegal behavior, requiring a lot of manpower for review and reducing revenue. Based on this, it is hoped that the quality scoring process will be introduced before the state recognition process to filter out images with poor quality and improve the accuracy of state recognition.

Contents of the invention

The present application provides an image quality scoring method, device, system, storage medium and electronic equipment for state recognition, which can reasonably score image quality for different business types, thereby improving the accuracy of state recognition.

In order to achieve the above object, the application adopts the following technical solutions:

This application provides an image quality scoring method, including:

Input the original image to be scored and the label corresponding to each current business type into a pre-trained state recognition model, mark the image area related to each current business type, and perform feature extraction based on the image area, Obtaining a first state feature vector corresponding to each current service type;

For each of the first state feature vectors, calculate the similarity between the first state feature vector and the state feature vectors of R*P standard images corresponding to the corresponding current service type, and obtain R*P similarity results;

For each of the first state feature vectors, based on the R*P similarity results, determine the quality scoring results of the original image to be scored under the corresponding current service type;

Wherein, the state feature vector of the standard image is: the state feature vector obtained after the standard image and the label of the corresponding current business type are input into the state recognition model for feature extraction; R*P standard images are corresponding to the corresponding current business type Typical images of various status settings that can be unambiguously identified status categories, the R is the total number of all status categories under the corresponding current service type, and the P is a positive integer.

Preferably, the state recognition model includes a first-level network for semantic segmentation and a second-level network for state recognition;

The input of the first-level network is the original image to be scored and the label, and the output is a semantic feature map corresponding to each of the labels, which is used to mark the image area related to each of the current business types ;

The second-level network includes M sub-networks corresponding to M service types; wherein, the M is the total number of all service types;

The input of each sub-network is the original image to be scored and the corrected semantic feature map corresponding to each label of the first current business type corresponding to the sub-network, and each of the sub-networks combines the The original image to be scored and the corrected semantic feature map are connected in the channel dimension, and the connected image is input into the classification network for state recognition to obtain the first state corresponding to the first current business type A feature vector; wherein, the corrected semantic feature map is a semantic feature map upsampled to the same size as the original image to be scored.

Preferably, said inputting the connected images into the classification network for state recognition includes:

Divide the connected image into several two-dimensional slice patches, and add the two-dimensional position information corresponding to the two-dimensional slice in each of the two-dimensional slices;

Input all the two-dimensional slices with two-dimensional position information into the classification network for state recognition;

Wherein, through the training of the state recognition model, it is ensured that the weight of the two-dimensional position information corresponding to the two-dimensional slice located in the image area related to the current business type is greater than that of the two-dimensional position information located outside the image area related to the current business type. The weight of the two-dimensional position information corresponding to the slice.

Preferably, an attention mechanism is added to the classification network, and a weight is assigned to each pixel input to the classification network, and the weight of pixels located in the image area related to the current business type is greater than that of pixels located in the current business type The relative weights of pixels outside the image region.

Preferably, when training the state recognition model, the training of the first-level network is completed using training sample images, and the second-level network is trained based on the trained first-level network; or,

When training the state recognition model, jointly train the first-level network and the second-level network; or,

When training the state recognition model, use training sample images to perform initial training on the first-level network, and based on the first-level network obtained after the initial training, perform initial training on the second-level network; After the initial training of the second-level network, joint training is performed on the first-level network and the second-level network;

Wherein, when performing the joint training, the loss function is a weighted sum of the first loss function of the first-level network and the second loss function of the second-level network.

Preferably, when performing the joint training, the first loss function corresponding to each input business type is calculated based on the loss weights of all input tags; wherein, the input business type is the business type to which the input tag belongs;

When calculating the first loss function corresponding to any input service type, the loss weight of the input tags of the any input service type is greater than the loss weight of the input tags not belonging to the any input service type.

Preferably, the first loss function is a dice loss function, and the second loss function is a focal loss function.

Preferably, each state category under each current business type includes P standard images, and the P standard images include original images of different scene brightnesses, different shooting angles, and/or different postures under each state category. image.

Preferably, the determination of the quality scoring result of the original image to be scored under the corresponding test service type includes:

Calculate a weighted mean value for the R*P similarity results, and use the calculation result as the quality scoring result of the original image to be scored under the corresponding current service type.

The application also provides an image quality scoring method, including:

Input the original image to be scored and the identification of each current business type into a pre-trained quality scoring regression model to obtain the quality scoring result of the original image to be scored under each current business type;

Wherein, the quality scoring regression model is a neural network model obtained by using the test sample image, the identification of each test service type, and the quality score result of the test sample image under each test service type;

The determination process of the quality scoring result of the test sample image under each test service type includes:

Input the test sample image and the label corresponding to each test service type into a pre-trained state recognition model, mark the image area related to each test service type, and perform feature based on the image area Extracting to obtain the first state feature vector corresponding to each test service type;

For each of the first state feature vectors, calculate the similarity between the first state feature vector and the state feature vectors of R*P standard images corresponding to the corresponding test service type, and obtain R*P similarity results;

For each of the first state feature vectors, based on the R*P similarity results, determine the quality scoring results of the test sample images under the corresponding test service types;

Wherein, the state feature vector of the standard image is: the state feature vector obtained after the standard image and the label of the corresponding test service type are input into the state recognition model for feature extraction; R*P standard images are corresponding to the corresponding test service type Typical images of various status settings that can be distinguished without ambiguity. The R is the total number of all status categories under the corresponding test service type, and the P is a positive integer.

Preferably, the state recognition model includes a first-level network for semantic segmentation and a second-level network for state recognition;

The input of the first-level network is the test sample image and the label, and the output is a semantic feature map corresponding to each of the labels, which is used to mark the image area related to each test service type;

The second-level network includes M sub-networks corresponding to M service types; wherein, the M is the total number of all service types;

The input of each sub-network is the test sample image and the corrected semantic feature map corresponding to each label of the first test service type corresponding to the sub-network, and each of the sub-networks takes the test sample The image and the corrected semantic feature map are connected in the channel dimension, and the connected image is input into the classification network for state recognition to obtain the first state feature vector corresponding to the first test service type; wherein , the corrected semantic feature map is a semantic feature map upsampled to the same size as the test sample image.

Preferably, said inputting the connected images into the classification network for state recognition includes:

Divide the connected image into several two-dimensional slice patches, and add the two-dimensional position information corresponding to the two-dimensional slice in each of the two-dimensional slices;

Input all the two-dimensional slices with two-dimensional position information into the classification network for state recognition;

Wherein, through the training of the state recognition model, it is ensured that the weight of the two-dimensional position information corresponding to the two-dimensional slice located in the image area related to the current business type is greater than that of the two-dimensional position information located outside the image area related to the current business type. The weight of the two-dimensional position information corresponding to the slice.

Preferably, an attention mechanism is added to the classification network, and a weight is assigned to each pixel input to the classification network, and the weight of pixels located in the image area related to the test service type is greater than that of pixels located in the test service type The relative weights of pixels outside the image region.

Preferably, when training the state recognition model, the training of the first-level network is completed using training sample images, and the second-level network is trained based on the trained first-level network; or,

When training the state recognition model, jointly train the first-level network and the second-level network; or,

When training the state recognition model, use training sample images to perform initial training on the first-level network, and based on the first-level network obtained after the initial training, perform initial training on the second-level network; After the initial training of the second-level network, joint training is performed on the first-level network and the second-level network;

When performing the joint training, the loss function is a weighted sum of the first loss function of the first-level network and the second loss function of the second-level network.

Preferably, when performing the joint training, the first loss function corresponding to each input business type is calculated based on the loss weights of all input tags; wherein, the input business type is the business type to which the input tag belongs;

When calculating the first loss function corresponding to any input service type, the loss weight of the input tags of the any input service type is greater than the loss weight of the input tags not belonging to the any input service type.

Preferably, the determination of the quality scoring result of the test sample image under the corresponding test service type includes:

Calculate a weighted mean value for the R*P similarity results, and use the calculation result as the quality scoring result of the test sample image under the corresponding test service type.

The present application provides an image quality scoring device, including: a state identification unit and a scoring unit;

The state recognition unit is used to input the original image to be scored and the label corresponding to each current business type into a pre-trained state recognition model, mark the image area related to each current business type, and based on performing feature extraction on the image area to obtain a first state feature vector corresponding to each current service type;

The scoring unit is configured to, for each of the first state feature vectors, calculate the similarity between the first state feature vector and the state feature vectors of R*P standard images corresponding to the corresponding test service type, to obtain R* P similarity results; for each of the first state feature vectors, based on the R*P similarity results, determine the quality scoring results of the original image to be scored under the corresponding current service type;

Wherein, the state feature vector of the standard image is: the state feature vector obtained after the standard image and the label of the corresponding current business type are input into the state recognition model for feature extraction; R*P standard images are corresponding to the corresponding current business type Typical images of various status settings that can be unambiguously identified status categories, the R is the total number of all status categories under the corresponding current service type, and the P is a positive integer.

Preferably, in the state recognition unit, the state recognition model includes a first-level network for semantic segmentation and a second-level network for state recognition;

The input of the first-level network is the original image to be scored and the label, and the output is a semantic feature map corresponding to each of the labels, which is used to mark the image area related to each of the current business types ;

The second-level network includes M sub-networks corresponding to M service types; wherein, the M is the total number of all service types;

The input of each sub-network is the original image to be scored and the corrected semantic feature map corresponding to each label of the first current business type corresponding to the sub-network, and each of the sub-networks combines the The original image to be scored and the corrected semantic feature map are connected in the channel dimension, and the connected image is input into the classification network for state recognition to obtain the first state corresponding to the first current business type A feature vector; wherein, the corrected semantic feature map is a semantic feature map upsampled to the same size as the original image to be scored.

Preferably, in the state identification unit, the input of the connected image into the classification network for state identification includes:

Divide the connected image into several two-dimensional slice patches, and add the two-dimensional position information corresponding to the two-dimensional slice in each of the two-dimensional slices;

Input all the two-dimensional slices with two-dimensional position information into the classification network for state recognition;

Wherein, through the training of the state recognition model, it is ensured that the weight of the two-dimensional position information corresponding to the two-dimensional slice located in the image area related to the current business type is greater than that of the two-dimensional position information located outside the image area related to the current business type. The weight of the two-dimensional position information corresponding to the slice.

Preferably, in the state identification unit, an attention mechanism is added to the classification network to assign a weight to each pixel input to the classification network, and the weight of the pixel located in the image area related to the current business type Greater than the weight of pixels located outside the image area related to the current service type.

Preferably, the device further includes a training unit for training and generating the state recognition model;

Wherein, when the training unit trains the state recognition model, the training sample image is used to complete the training of the first-level network, and the second-level network is trained based on the trained first-level network; or,

When the training unit trains the state recognition model, jointly train the first-level network and the second-level network; or,

When the training unit trains the state recognition model, the training sample image is used to initially train the first-level network, and based on the first-level network obtained after the initial training, the second-level network is initially trained. training; after the initial training of the second-level network, jointly train the first-level network and the second-level network;

When performing the joint training, the loss function is a weighted sum of the first loss function of the first-level network and the second loss function of the second-level network.

Preferably, when performing the joint training, the first loss function corresponding to each input business type is calculated based on the loss weights of all input tags; wherein, the input business type is the business type to which the input tag belongs;

When calculating the first loss function corresponding to any input service type, the loss weight of the input tags of the any input service type is greater than the loss weight of the input tags not belonging to the any input service type.

Preferably, in the state identification unit, the determination of the quality scoring result of the original image to be scored under the corresponding current business type includes:

Calculate a weighted mean value for the R*P similarity results, and use the calculation result as the quality scoring result of the original image to be scored under the corresponding current service type.

The application also provides an image quality scoring system, including a training device and an image quality scoring device;

The training device includes a state recognition unit, a scoring unit and a first training unit;

The state recognition unit is used to input the test sample image and the label corresponding to each test service type into the pre-trained state recognition model, mark the image area related to each test service type, and based on the performing feature extraction on the image area to obtain a first state feature vector corresponding to each test service type;

The scoring unit is configured to, for each of the first state feature vectors, calculate the similarity between the first state feature vector and the state feature vectors of R*P standard images corresponding to the corresponding test service type, to obtain R* P similarity results; for each of the first state feature vectors, based on the R*P similarity results, determine the quality scoring result of the test sample image under the corresponding test service type;

The first training unit is configured to use the test sample image, the identification of each current service type, and the quality score result of the test sample image under each current service type to train and generate a quality score regression model;

The image quality scoring device includes an input unit, a quality scoring unit and an output unit;

The input unit is used to receive the original image to be scored and the identification of each current service type, and send them to the quality scoring unit;

The quality scoring unit is configured to input the received original image to be scored and the identification of each current business type into the quality scoring regression model to obtain the original image to be scored under each current business type quality rating results for

The output unit is configured to output a quality scoring result of the original image to be scored under each current service type;

Wherein, the state feature vector of the standard image is: the state feature vector obtained after the standard image and the label of the corresponding test service type are input into the state recognition model for feature extraction; R*P standard images are corresponding to the corresponding test service type Typical images of various status settings that can be distinguished without ambiguity. The R is the total number of all status categories under the corresponding test service type, and the P is a positive integer.

Preferably, in the state recognition unit, the state recognition model includes a first-level network for semantic segmentation and a second-level network for state recognition;

The input of the first-level network is the test sample image and the label, and the output is a semantic feature map corresponding to each of the labels, which is used to mark the image area related to each test service type;

The second-level network includes M sub-networks corresponding to M service types; wherein, the M is the total number of all service types;

The input of each sub-network is the test sample image and the corrected semantic feature map corresponding to each label of the first test service type corresponding to the sub-network, and each of the sub-networks takes the test sample The image and the corrected semantic feature map are connected in the channel dimension, and the connected image is input into the classification network for state recognition to obtain the first state feature vector corresponding to the first test service type; wherein , the corrected semantic feature map is a semantic feature map upsampled to the same size as the test sample image.

Preferably, in the state identification unit, the input of the connected image into the classification network for state identification includes:

Divide the connected image into several two-dimensional slice patches, and add the two-dimensional position information corresponding to the two-dimensional slice in each of the two-dimensional slices;

Input all the two-dimensional slices with two-dimensional position information into the classification network for state recognition;

Wherein, through the training of the state recognition model, it is ensured that the weight of the two-dimensional position information corresponding to the two-dimensional slice located in the image area related to the current business type is greater than that of the two-dimensional position information located outside the image area related to the current business type. The weight of the two-dimensional position information corresponding to the slice.

Preferably, in the state recognition unit, an attention mechanism is added to the classification network, and a weight is assigned to each pixel input to the classification network, and the weight of the pixel located in the image area related to the test service type Greater than the weight of pixels located outside the image area related to the test service type.

Preferably, the training device further includes a second training unit for training and generating the state recognition model;

Wherein, when the second training unit trains the state recognition model, the training sample image is used to complete the training of the first-level network, and based on the trained first-level network, the second-level network is training; or,

When the second training unit trains the state recognition model, jointly train the first-level network and the second-level network; or,

When the second training unit trains the state recognition model, the training sample image is used to initially train the first-level network, and based on the first-level network obtained after the initial training, the second-level network is performing initial training; after performing initial training on the second-level network, performing joint training on the first-level network and the second-level network;

When performing the joint training, the loss function is a weighted sum of the first loss function of the first-level network and the second loss function of the second-level network.

Preferably, when the second training unit performs the joint training, the first loss function corresponding to each input business type is calculated based on the loss weights of all input tags; wherein, the input business type is an input tag the type of business it belongs to;

When calculating the first loss function corresponding to any input service type, the loss weight of the input tags of the any input service type is greater than the loss weight of the input tags not belonging to the any input service type.

Preferably, in the state identification unit, the determination of the quality scoring result of the test sample image under the corresponding test service type includes:

Calculate a weighted mean value for the R*P similarity results, and use the calculation result as the quality scoring result of the test sample image under the corresponding test service type.

An image quality scoring device, comprising: an input unit, a quality scoring unit and an output unit;

The input unit is used to receive the original image to be scored and the identification of each current service type, and send them to the quality scoring unit;

The quality scoring unit is configured to input the received original image to be scored and the identification of each current service type into a quality scoring regression model pre-trained by a training device, and obtain the original image to be scored in each The quality scoring results under the current business type;

The output unit is configured to output a quality scoring result of the original image to be scored under each current service type;

The training device includes a state recognition unit, a scoring unit and a first training unit;

The state recognition unit is used to input the test sample image and the label corresponding to each test service type into the pre-trained state recognition model, mark the image area related to each test service type, and based on the performing feature extraction on the image area to obtain a first state feature vector corresponding to each test service type;

The scoring unit is configured to, for each of the first state feature vectors, calculate the similarity between the first state feature vector and the state feature vectors of R*P standard images corresponding to the corresponding test service type, to obtain R* P similarity results; for each of the first state feature vectors, based on the R*P similarity results, determine the quality scoring result of the test sample image under the corresponding test service type;

The first training unit is configured to train and generate the quality score regression model;

Wherein, the state feature vector of the standard image is: the state feature vector obtained after the standard image and the label of the corresponding test service type are input into the state recognition model for feature extraction; R*P standard images are corresponding to the corresponding test service type Typical images of various status settings that can be distinguished without ambiguity. The R is the total number of all status categories under the corresponding test service type, and the P is a positive integer.

Preferably, in the state recognition unit, the state recognition model includes a first-level network for semantic segmentation and a second-level network for state recognition;

The input of the first-level network is the test sample image and the label, and the output is a semantic feature map corresponding to each of the labels, which is used to mark the image area related to each test service type;

The second-level network includes M sub-networks corresponding to M service types; wherein, the M is the total number of all service types;

The input of each sub-network is the test sample image and the corrected semantic feature map corresponding to each label of the first test service type corresponding to the sub-network, and each of the sub-networks takes the test sample The image and the corrected semantic feature map are connected in the channel dimension, and the connected image is input into the classification network for state recognition to obtain the first state feature vector corresponding to the first test service type; wherein , the corrected semantic feature map is a semantic feature map upsampled to the same size as the test sample image.

Preferably, in the state identification unit, the input of the connected image into the classification network for state identification includes:

Divide the connected image into several two-dimensional slice patches, and add the two-dimensional position information corresponding to the two-dimensional slice in each of the two-dimensional slices;

Input all the two-dimensional slices with two-dimensional position information into the classification network for state recognition;

Wherein, through the training of the state recognition model, it is ensured that the weight of the two-dimensional position information corresponding to the two-dimensional slice located in the image area related to the current business type is greater than that of the two-dimensional position information located outside the image area related to the current business type. The weight of the two-dimensional position information corresponding to the slice.

Preferably, in the state recognition unit, an attention mechanism is added to the classification network, and a weight is assigned to each pixel input to the classification network, and the weight of the pixel located in the image area related to the test service type Greater than the weight of pixels located outside the image area related to the test service type.

Preferably, the training device further includes a second training unit for training and generating the state recognition model;

Wherein, when the second training unit trains the state recognition model, the training sample image is used to complete the training of the first-level network, and based on the trained first-level network, the second-level network is training; or,

When the second training unit trains the state recognition model, jointly train the first-level network and the second-level network; or,

When the second training unit trains the state recognition model, the training sample image is used to initially train the first-level network, and based on the first-level network obtained after the initial training, the second-level network is performing initial training; after performing initial training on the second-level network, performing joint training on the first-level network and the second-level network;

When performing the joint training, the loss function is a weighted sum of the first loss function of the first-level network and the second loss function of the second-level network.

Preferably, when the second training unit performs the joint training, the first loss function corresponding to each input business type is calculated based on the loss weights of all input tags; wherein, the input business type is an input tag the type of business it belongs to;

When calculating the first loss function corresponding to any input service type, the loss weight of the input tags of the any input service type is greater than the loss weight of the input tags not belonging to the any input service type.

Preferably, in the state identification unit, the determination of the quality scoring result of the test sample image under the corresponding test service type includes:

Calculate a weighted mean value for the R*P similarity results, and use the calculation result as the quality scoring result of the test sample image under the corresponding test service type.

The present application also provides a training device, including: a state recognition unit, a scoring unit and a first training unit;

The state recognition unit is used to input the test sample image and the label corresponding to each test service type into the pre-trained state recognition model, mark the image area related to each test service type, and based on the performing feature extraction on the image area to obtain a first state feature vector corresponding to each test service type;

The scoring unit is configured to, for each of the first state feature vectors, calculate the similarity between the first state feature vector and the state feature vectors of R*P standard images corresponding to the corresponding test service type, to obtain R* P similarity results; for each of the first state feature vectors, based on the R*P similarity results, determine the quality scoring result of the test sample image under the corresponding test service type;

The first training unit is configured to use the test sample image, the identification of each current service type, and the quality score result of the test sample image under each current service type to train and generate a quality score regression model;

Wherein, the quality scoring model is used to process the input original image to be scored and the identification of each current business type to obtain the quality scoring result of the original image to be scored under each current business type; The state feature vector of the standard image is: the state feature vector obtained after the standard image and the label of the corresponding test service type are input into the state recognition model for feature extraction; the R*P standard images are each corresponding to the corresponding test service type. A typical image of a status category that can be distinguished without ambiguity for a status setting, the R is the total number of all status categories under the corresponding test service type, and the P is a positive integer.

Preferably, in the state recognition unit, the state recognition model includes a first-level network for semantic segmentation and a second-level network for state recognition;

The input of the first-level network is the test sample image and the label, and the output is a semantic feature map corresponding to each of the labels, which is used to mark the image area related to each test service type;

The second-level network includes M sub-networks corresponding to M service types; wherein, the M is the total number of all service types;

The input of each sub-network is the test sample image and the corrected semantic feature map corresponding to each label of the first test service type corresponding to the sub-network, and each of the sub-networks takes the test sample The image and the corrected semantic feature map are connected in the channel dimension, and the connected image is input into the classification network for state recognition to obtain the first state feature vector corresponding to the first test service type; wherein , the corrected semantic feature map is a semantic feature map upsampled to the same size as the test sample image.

Preferably, in the state identification unit, the input of the connected image into the classification network for state identification includes:

Divide the connected image into several two-dimensional slice patches, and add the two-dimensional position information corresponding to the two-dimensional slice in each of the two-dimensional slices;

Input all the two-dimensional slices with two-dimensional position information into the classification network for state recognition;

Wherein, through the training of the state recognition model, it is ensured that the weight of the two-dimensional position information corresponding to the two-dimensional slice located in the image area related to the current business type is greater than that of the two-dimensional position information located outside the image area related to the current business type. The weight of the two-dimensional position information corresponding to the slice.

Preferably, in the state recognition unit, an attention mechanism is added to the classification network, and a weight is assigned to each pixel input to the classification network, and the weight of the pixel located in the image area related to the test service type Greater than the weight of pixels located outside the image area related to the test service type.

Preferably, the training device further includes a second training unit for training and generating the state recognition model;

Wherein, when the second training unit trains the state recognition model, the training sample image is used to complete the training of the first-level network, and based on the trained first-level network, the second-level network is training; or,

When the second training unit trains the state recognition model, jointly train the first-level network and the second-level network; or,

When the second training unit trains the state recognition model, the training sample image is used to initially train the first-level network, and based on the first-level network obtained after the initial training, the second-level network is performing initial training; after performing initial training on the second-level network, performing joint training on the first-level network and the second-level network;

When performing the joint training, the loss function is a weighted sum of the first loss function of the first-level network and the second loss function of the second-level network.

Preferably, when the second training unit performs the joint training, the first loss function corresponding to each input business type is calculated based on the loss weights of all input tags; wherein, the input business type is an input tag the type of business it belongs to;

When calculating the first loss function corresponding to any input service type, the loss weight of the input tags of the any input service type is greater than the loss weight of the input tags not belonging to the any input service type.

Preferably, in the state identification unit, the determination of the quality scoring result of the test sample image under the corresponding test service type includes:

Calculate a weighted mean value for the R*P similarity results, and use the calculation result as the quality scoring result of the test sample image under the corresponding test service type.

The present application provides a computer-readable storage medium, on which computer instructions are stored, and when the instructions are executed by a processor, the image quality scoring method described in any one of the above-mentioned methods can be realized.

The present application provides an electronic device, the electronic device at least includes a computer-readable storage medium, and also includes a processor;

The processor is configured to read the executable instructions from the computer-readable storage medium, and execute the instructions to implement the image quality scoring method described in any one of the above.

It can be seen from the above technical solution that the pre-trained state recognition model can be used to process the input original image and the label corresponding to the business type, mark the image area related to each business type, and extract features based on the image area to obtain the first A state feature vector, the first state feature vector obtained in this way mainly reflects the feature information of the area related to the business type; next, input the R*P standard images corresponding to the business type into the state recognition model to obtain R*P standard The state feature vector of the image, by calculating the similarity between the first state feature vector and the R*P standard image state feature vectors, determines the quality scoring result of the original image under a certain service type. When performing image quality scoring, the quality scoring results of images under business types can be obtained directly through the above image quality scoring process, or a fractional regression network can be used to fit the above image quality scoring process. When actually performing image scoring, Input the image to be scored and the business type identifier into the trained score regression network to obtain the quality scoring result of the image under the corresponding business type.

Through the above processing, using the current recognized business type as supervisory information, the first state feature vector used for similarity comparison mainly reflects the feature information of the area related to the business type, and the image quality judgment thus carried out pays more attention to business-related Instead of judging the quality of the entire image, it can correspond to different business types and make reasonable image quality judgments in a targeted manner, making the quality judgment results more accurate.

Description of drawings

Fig. 1 is a schematic flow chart of the first image quality scoring method in the present application;

FIG. 2 is a schematic flow chart of the image quality scoring method in Embodiment 1 of the present application;

FIG. 3 is a schematic diagram of the overall framework of the illegal business judgment model in Embodiment 1;

FIG. 4 is a schematic structural diagram of each sub-network in the second-level network of Embodiment 1;

FIG. 5 is a schematic flow chart of the image quality scoring method in Embodiment 2;

FIG. 6 is a schematic diagram of the basic structure of the first image quality scoring device in the present application;

7 is a schematic diagram of the basic structure of the image quality scoring system corresponding to the second image quality scoring method in the present application;

FIG. 8 is a schematic diagram of the basic structure of the second image quality scoring device in the present application;

9 is a schematic diagram of the basic structure of the training device in the image quality scoring system of the present application;

FIG. 10 is a schematic diagram of the basic structure of the electronic device provided by the present application.

Detailed ways

In order to make the purpose, technical means and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings.

As mentioned in the background technology section, before state recognition, it is usually necessary to score the image quality, filter out images with poor quality, and improve the accuracy of state recognition. However, in application scenarios where there are many different recognition services, some of the existing image quality scoring methods require manual calibration and scoring, and some cannot perform targeted image quality scoring for different recognition services, so that the image quality scoring results are accurate Sex is not high.

For example, in the application of judging illegal behavior in the cab, illegal behavior is involved in a wide range, and there are many different illegal behavior judging services (such as judging whether to wear a seat belt, judging whether to make a phone call, judging whether to smoke, etc.) Judging whether to hold a baby, etc.); at the same time, in the application of judging illegal behavior in the cab, there are many areas that need to be paid attention to (such as the main driver, co-pilot, rear seat passengers, etc.); different judging businesses focus on different areas, and the image quality If the scoring is generalized, it will affect the accuracy of the scoring results; if judging businesses for different illegal activities and maintaining different image quality scoring methods, there will be many versions and difficult maintenance.

Based on the above analysis, this application provides an image quality scoring method suitable for a variety of different recognition services. The original image is semantically segmented, the image area related to the current business type is marked, and image features are extracted based on the image area for image quality scoring. In this way, the state recognition model is driven by the business to focus on the area related to the current identification business type, and the image quality is judged based on the relevant area instead of the whole picture, so as to correspond to different recognition business types and reasonably judge the image quality. Make image quality scoring results more accurate.

Based on the above basic idea of the present application, the present application provides two image quality scoring methods. Among them, the basic process of the first method is shown in Figure 1, including:

Step 101, input the original image to be scored and the label corresponding to each current business type into the pre-trained state recognition model, mark the image area related to each current business type, and perform feature extraction based on the image area to obtain A first state feature vector corresponding to each current service type.

The state recognition model in this application is a pre-trained model that can recognize the current state based on the original image. For example, in the application of cab illegal behavior judgment, the state recognition model can be a cab behavior judgment model, which judges the current state as not wearing a seat belt, making a phone call, or holding a baby according to the input original image.

In this application, the state recognition model is trained and processed with the business type as the supervisory information. Therefore, it is necessary to correspond to various business types to be identified in advance, and set corresponding tags, and one or more tags may be corresponding to a business type. For different service types, the number of corresponding tags may be the same or different. Preferably, the tags may correspond to areas related to service types. For example, in the application of judging illegal behavior in the cab, the type of illegal business (that is, the type of business to be identified) may include: seat belt business, phone call business, embrace baby business, etc., corresponding to the seat belt business, you can set two tags , are the torso and the belt, respectively, corresponding to the torso-related region and the belt-related region.

In the processing of this step, the original image to be scored and the label corresponding to each current business type are used as the input of the state recognition model, and the pre-trained state recognition model is used to perform semantic segmentation on the original image to obtain the input label The corresponding image area, that is, the image area related to each current business type, may be, for example, a mask map corresponding to the input label.

Using the state recognition model, after determining the image area related to each current business type, corresponding to each current business type, image classification feature extraction is performed based on the image of the relevant area, and the state feature vector is obtained, and the state feature vector is the corresponding for a current business type. The obtained state eigenvector is called the first state eigenvector. Since the first state feature vector is obtained based on an image area related to a certain current service type, the first state feature vector mainly reflects feature information of an area related to a certain current service type.

Among them, the current business type is also the business type that needs to be identified in this processing. For example, the current processing of the state recognition model inputs an original image and a label corresponding to the seat belt business, so the processing of the state recognition model can Obtaining the first state feature vector corresponding to the seat belt service can be used to determine which state of the seat belt service is currently in. In addition, the current service type can be one or more. For example, in one process, an original image and the label corresponding to the seat belt service and the phone service can be input, and the corresponding seat belt service can be obtained through the processing of the state recognition model. A first state feature vector of the service and a first state feature vector corresponding to the call service can be used to determine which state of the seat belt service and which state of the call service are currently in, respectively.

Step 102, for each first state feature vector, calculate the similarity between the first state feature vector and the state feature vectors of R*P standard images corresponding to the current service type, and obtain R*P similarity results.

As mentioned above, the first state feature vector is a feature vector obtained after processing the input original image by using the state recognition model.

In this application, in order to score the image quality corresponding to different business types, R*P standard images are set for each business type, and the R*P standard images are input into the state recognition model to obtain the R*P standard images respectively. state feature vector. Among them, the R*P standard images are typical images corresponding to various state settings of a certain service type, which can distinguish the state categories without ambiguity, R is the total number of all state categories under a certain service type, and P is a positive integer.

Preferably, the P standard images set corresponding to each status category of the business type include original images of different scene brightnesses, different shooting angles and/or different poses under the corresponding status categories.

R*P standard images can unambiguously judge the state category, and the feature information of the standard image obtained by using the state recognition model represents the characteristics of the image that can unambiguously judge the state category; for the first state feature corresponding to each current business type B Information, respectively calculate the similarity between the first state characteristic information and the state characteristic information of these standard images corresponding to the service type B, and obtain R*P similarity results, then these similarity results can reflect the first state characteristic information and The similarity between image features of state categories can be judged without ambiguity.

Step 103, for each first state feature vector, based on the R*P similarity results, determine the quality scoring result of the original image to be scored under the corresponding current service type.

Since the R*P similarity results determined in step 102 can reflect the similarity between the first state feature information and the image features that can judge the state category without ambiguity, the original image quality can be scored by using these similarity results, To reflect the quality status of the original image relative to the standard image, so as to effectively give a reasonable image quality score for each current business type.

So far, the flow of the method shown in FIG. 1 ends.

In addition, in order to further optimize the processing delay and resources occupied by image quality scoring, the above first method can also be regarded as a complete image quality scoring process, and a quality scoring regression model is trained to fit the first method. The image quality scoring process, that is to say, execute the image quality scoring process given by the first method on a series of test sample images and test service types, and obtain the quality scoring results of the test sample images under the test service type, and then use the test The sample images and corresponding quality scoring results are used as training sample pairs to train the generated quality scoring regression model to fit the image quality scoring process given by the first method. The second image quality scoring method provided in this application is to perform image quality scoring based on a trained quality scoring regression model. Specifically, the second image quality scoring method includes:

Input the original image to be scored and the identification of each current business type into the pre-trained quality scoring regression model, and obtain the quality scoring result of the original image to be scored under each current business type;

Wherein, the quality scoring regression model is a neural network model obtained by using the test sample image, the identification of each test service type and the quality score result training of the test sample image under each test service type;

The process of determining the quality scoring results of test sample images under each test business type includes:

Input the test sample image and the label corresponding to each test business type into the pre-trained state recognition model, mark the image area related to each test business type, and perform feature extraction based on the image area to obtain the The first state eigenvector corresponding to the type;

For each first state feature vector, calculate the similarity between the first state feature vector and the state feature vectors of R*P standard images corresponding to the corresponding test service type, and obtain R*P similarity results;

For each first state feature vector, based on the R*P similarity results, determine the quality scoring result of the test sample image under the corresponding test service type.

Both of the above two image quality scoring methods introduce the state feature vector of the standard image, which is used to compare with the first state feature vector of the input image to obtain the similarity, so as to determine the image quality score by using the similarity. Among them, the state feature vector of the standard image refers to: the state feature vector obtained after feature extraction of the standard image and the label input state recognition model of the corresponding test service type; R*P standard images are provided corresponding to each service type, and these standard images It is a typical image set corresponding to various states of a business type that can distinguish the state category without ambiguity. R is the total number of all state categories under a certain business type, and P is a positive integer, that is, the number of typical images in each state . For example, in the application scenario of judging illegal behavior in the cab, the business type is to judge whether to make a phone call, and R can be the total number of status categories under this business. For each state category, P typical images are provided, such as taking into account the different brightness of the scene, various typical postures of the cab personnel, and different shooting angles.

In the above two image quality scoring methods, the first state feature vector used for similarity comparison mainly reflects the feature information of the area related to the business type, and the image quality judgment thus carried out pays more attention to the area related to the business, while It does not judge the quality of the whole picture, so that it can correspond to different business types, and carry out reasonable image quality judgment in a targeted manner, so that the quality judgment result is more accurate.

The two image quality scoring methods of the present application will be described in detail below through two specific embodiments.

Embodiment one:

This embodiment is used to introduce the first image quality scoring method in detail.

Fig. 2 is a schematic flowchart of a specific embodiment of the first image quality scoring method in the present application. For the convenience of description, in this specific embodiment, the application scenario of judging illegal behavior in the cab is taken as an example to illustrate the image quality scoring method, and at the same time, the training process of the state recognition model is also described. As shown in Figure 2, the method includes:

Step 201, using training sample images to train and generate an illegal business judgment model.

The illegal business judgment model in this embodiment is also the aforementioned state recognition model.

The illegal business judging model in this embodiment is trained and processed with the type of illegal business as supervision information. Therefore, it is necessary to correspond to various illegal business types in advance and set corresponding tags. One or more tags may be corresponding to one illegal business type. For different illegal business types, the corresponding number of tags may be the same or different. Preferably, the tags may correspond to areas associated with illegal business types. For example, for the seat belt business, two labels can be set, which are the torso and the seat belt, corresponding to the torso-related area and the seat belt-related area.

The input of the illegal business judgment model includes the original image used for quality scoring and labels corresponding to one or more illegal business types (hereinafter referred to as input illegal types), and the illegal business judgment model is used to mark the original image with each input illegal Type-related image regions, and then perform classification feature extraction based on the marked image regions, and obtain the state feature vector corresponding to each input violation type for output.

The above-mentioned illegal business judgment model can be realized by using various network structures. This embodiment provides a specific implementation structure.

The illegal service judging model in this embodiment is a neural network model including two hierarchical networks, the first hierarchical network is a semantic segmentation network, and the second hierarchical network is a semantic fusion and service identification network. Specifically, the overall framework of the illegal business judgment model in this embodiment may be shown in FIG. 3 .

First introduce the first level network:

1. In the first-level semantic segmentation network, the choice of semantics is determined by the type of illegal business. In fact, the semantics can directly correspond to the input labels, and semantic segmentation is processed according to the input labels;

2. The first-level network can be various neural network models for processing images, such as the convolutional neural network shown in Figure 3, namely the CNN network;

3. When performing semantic segmentation on the input original image, firstly mark the areas corresponding to each input label in the input original image to obtain a Mask map;

Since the input label may correspond to one or more input illegal business types, in the Mask diagram, the label areas corresponding to all input illegal business types are marked, that is, the labels corresponding to each input illegal business type are marked Type-dependent regions. These may include areas of imagery associated with one or more types of illegal business. For example, assuming that the input labels include the labels of seat belt business and phone business (specifically including human head, arm and palm), that is, including torso, seat belt, human head, arm and palm, in the semantic segmentation network, firstly, the torso, safety Mark the areas corresponding to the belt, head, arm, and palm to obtain the Mask map;

4. Based on the Mask map obtained in point 3, corresponding to each input label A, only keep the mark of the corresponding area of the input label, and generate a Mask map corresponding to label A; in this way, corresponding to each input label, a corresponding Mask is obtained The Mask map corresponding to all input labels is the output of the semantic segmentation network, and the Mask map corresponding to the input label is also called a semantic feature map.

It can be seen from the above that through the processing of the semantic segmentation network, the semantic feature map corresponding to each input label is obtained, and the semantic feature map corresponding to the label belonging to the same illegal business type marks the image area related to the illegal business type.

Next, introduce the second level network:

1. As shown in Figure 3, the second-level network includes M sub-networks corresponding to M service types one by one, and each sub-network has the same structure; M is the total number of all service types to be identified. The first-level network outputs the semantic feature map corresponding to each input label. After entering the second-level network, the semantic feature map and original image corresponding to the label belonging to the same input business type (that is, the original image input to the first-level network) Enter the subnetwork corresponding to the corresponding input business type. For example, the second-level network includes three sub-networks, which correspond to the seat belt business, the phone business, and the baby-holding business; the semantic feature map output by the first-level network corresponds to the label of the seat belt business and the label of the phone business , then when entering the second-level network, input all the semantic feature maps and original images corresponding to the labels of the seat belt service into the sub-network corresponding to the seat belt service, and input all the semantic feature maps and the original images corresponding to the labels of the phone call service The original image is input into the sub-network corresponding to the call service. The following descriptions take a subnetwork and the service type X corresponding to the subnetwork as an example for illustration;

2. The processing of the sub-network includes two parts: fusion processing and classification network, as shown in Figure 4;

3. Fusion processing:

Correct all the semantic feature maps of the input sub-network, that is, upsample to the same size as the original image; perform concat operation on all the corrected semantic feature maps and the original image, that is, connect in the channel dimension, as the fusion processing result output to the classification network;

4. Classification network:

In the simplest way, the connected images obtained through the fusion process can be input into the classification network for image classification feature extraction and classification result determination to obtain the state feature vector and state classification result corresponding to the service type X. For example, for the sub-network corresponding to the seat belt service, the state feature vector corresponding to the seat belt service and the state classification result of the seat belt service (such as the state of not wearing a seat belt) are finally obtained. The network structure of the classification network can adopt various existing neural network models for realizing classification, such as CNN network and the like.

In addition, in the classification network processing, in order to further accelerate the convergence of the classification task and improve the robustness of the classification network, optionally, the connected image output by the fusion processing can be divided into several two-dimensional slice patches, assuming that the connected image The size is H*W*C, H and W are the height and width of the image respectively, C is the number of connected channels, and the size of each two-dimensional slice is P*P, then the connected image is divided into two-dimensional slices After that, the number of two-dimensional slices is H*W*C/P ² . Add the two-dimensional position information corresponding to the two-dimensional slice to each two-dimensional slice; then input all the two-dimensional slices after adding the two-dimensional position information into the classification network for state identification;

Wherein, through the training of the state recognition model, it is ensured that the weight of the two-dimensional position information corresponding to the two-dimensional slice located in the image area related to the business type X is greater than the two-dimensional position corresponding to the two-dimensional slice located outside the image area related to the business type X information weight. Here, the process of adding the two-dimensional position information to the two-dimensional slice may be performing a dot product operation on the two-dimensional slice and the weight of the two-dimensional position information. For example, when the service type X is a seat belt service, it is guaranteed through training that the two-dimensional location information weight (for example, 2) corresponding to the two-dimensional slice located in the seat belt-related area is greater than the two-dimensional position information weight (for example, 2) located in the seat belt-related area. The weight of the two-dimensional position information corresponding to the dimension slice (for example, it may be 1).

In the classification network processing, in order to further improve the classification accuracy of the classification network, optionally, an attention mechanism can also be introduced into the classification network to assign a weight to each pixel input into the classification network, and ensure that it is in the business type X through training. The weight of pixels in the relevant image area is greater than the weight of pixels located outside the image area related to service type X. In this way, more critical and important feature information can be extracted, so that the classification network can make more accurate classification result judgments, and at the same time, it will not bring more consumption to the calculation and storage of the state recognition model.

The above is the exemplary architecture and specific processing of the illegal business judgment model given in this embodiment.

When the illegal business judgment model (that is, the state recognition model) is trained, the training sample image and one or more labels corresponding to the training business type are input, and after processing, the processing result is compared with the state category of the known training sample image The loss function is obtained by comparison, and the parameters of the illegal business judgment model are adjusted according to the loss function.

Specific to the above-mentioned illegal business judgment model composed of two hierarchical networks, the two hierarchical networks can be trained separately during training, or the two hierarchical networks can be jointly trained, or staged training can also be performed In the initial stage, the two-layer networks are trained separately, and after the initial stage, the two-layer networks are jointly trained.

Specifically, when training the two-level networks separately, a part of the training sample images can be used to train the first-level network. According to the loss function of the first-level network (hereinafter referred to as the first loss function), the first-level network Adjust the parameters of the first-level network, and then return to the first-level network after the updated parameters to process the training sample images until the training end condition of the first-level network is reached; when the first-level network training is completed, use another part of the training sample images for For the training of the second-level network, the first-level network needs to be used when the second-level network is trained. The first-level network used here has been trained and the parameter values have been fixed. After the second-level network is processed, Adjust the parameters of the second-level network according to the loss function of the second-level network (hereinafter referred to as the second loss function), and then return to the training sample image using the trained first-level network and the second-level network after updating parameters Process until the end-of-training condition for the second-level network is reached.

Wherein, when the first-level network is trained separately, the input label can be one or more labels corresponding to the business type, including all N labels corresponding to all business types with the input label (N is all labels corresponding to all business types total) as an example, the first loss function can be the Mask image obtained by semantic segmentation and the standard Mask image of the preset training sample image (where the N label regions are accurately marked) The result of the comparison, generally, the same label areas can be compared one by one, and the final Mask map comparison result is obtained according to the comparison results of each area as the first loss function. For different label areas, the comparison results participate in the final result calculation The weights are usually the same.

When training the second-level network separately, since the second-level network includes M sub-networks, the second loss function corresponding to the sub-network is determined according to the processing results of each sub-network during training, and used to adjust the parameters of the corresponding sub-network . Specifically, the training sample images are first input into the trained first-level network, and after being processed by the first-level network, the semantic feature maps and training sample images corresponding to each input label are input into the second-level network; the second-level network will belong to the same The semantic feature map corresponding to the input label of the business type X and the training sample image are input into the sub-network Y corresponding to the business type X, and the sub-network Y determines the classification of the training sample image under the business type X after processing, and compares it with the training sample image in The standard classification under the service type is compared to determine the second loss function, and the parameters of the subnetwork Y are adjusted according to the second loss function.

When performing joint training on the two-level network, the training sample image can be used to input the first-level network, after being processed by the first-level network, the first loss function is obtained, and then the processing result of the first-level network is calculated according to the second-level The input of the network requires input into the second-level network, and the second loss function is obtained after processing by the second-level network, and then the joint loss function is obtained according to the weighted sum of the first loss function and the second loss function, and the first loss function is calculated according to the joint loss function. Adjust the parameters of the hierarchical network and the second-level network, and then return to the updated first-level network and second-level network to process the input training sample images until the training end condition of the illegal business judgment model is reached.

Among them, in the joint training, the input label can be one or more business type labels. Due to the joint training, the weighted sum of the first loss function and the second loss function is required to determine the final joint loss function, and the second The loss function corresponds to a single business type, so the first loss function also needs to be calculated for a single business type. Specifically, the training sample image and one or more business type labels are input into the first-level network for processing, and a Mask map is obtained that marks the regions corresponding to all input labels, and the Mask map and the corresponding regions of the input labels in the training sample image are obtained. The marked standard Mask graphs are compared, corresponding to each business type, and the first loss function corresponding to the single business type is calculated. The following describes how to determine the first loss function corresponding to a single business type.

Assuming that the input label corresponds to two business types, namely the seat belt service and the phone call service, the Mask graph can be obtained in the manner described in the previous paragraph. The Mask graph marks the label areas of the seat belt service and the phone call service, and the Mask Compare the image with the standard Mask image. Specifically, you can compare the area corresponding to the same input label in the Mask image with the area corresponding to the same input label in the standard Mask image, and set weights for each input label. According to the comparison results of each input label area The first loss function S1 corresponding to the seat belt service and the first loss function S2 corresponding to the phone call service are determined with the weight of the input label. When calculating the first loss function corresponding to the seat belt service, the weight of the label corresponding to the seat belt service (i.e., torso and seat belt) is greater than the weight of the corresponding label (i.e., head, arm, and palm) of the call service; When the first loss function is used, the weight of the label corresponding to the phone call service (ie, head, arm, and palm) is greater than the weight of the label corresponding to the seat belt service (ie, torso and seat belt). That is to say, during joint training, the first loss function of each input business type is calculated based on the loss weights of all input tags. When calculating the first loss function corresponding to an input business type, the labels belonging to the input business type The loss weight of is greater than the loss weight of tags that do not belong to the input business type. The above processing can ensure that the label area matching the target business has a greater impact on the first loss function, so that the model can pay more attention to the area related to the target business.

When the two-level network is trained in stages, the training sample image can be used to initially train the first-level network, and based on the first-level network obtained after the initial training, the second-level network is initially trained; in the first level After the network and the second-level network complete the initial training, the first-level network and the second-level network are jointly trained.

Among them, the first loss function can be various loss functions used to measure distance loss, such as Dice loss function, and the second loss function can be various loss functions used to measure classification loss, such as Focal loss function or cross-entropy loss function wait.

As above, the illegal business judgment model can be trained. Next, through steps 202 and 203, the trained illegal business judgment model is used to process the original image to be scored.

Step 202, input the original image to be scored and the label corresponding to each current business type into the pre-trained illegal business judgment model, and mark the image area related to each current business type.

It can be seen from the description of the illegal business judgment model in step 201 above that the trained illegal business judgment model can judge the state of one or more business types for an original image.

When it is actually necessary to perform image quality scoring under one or more business types on an original image, the original image to be scored can be corresponding to each target business type (hereinafter referred to as the current business type) that requires quality scoring The label input of the illegal business model marks the image regions related to each current business type.

When the above-mentioned two-level network is used to realize the illegal business model, the processing in this step is also the processing of the first-level network, that is, the original image is semantically segmented to obtain a Mask map corresponding to each input label.

Step 203, in the illegal business judgment model, perform feature extraction based on the image area related to each current business type, and obtain a first state feature vector corresponding to each current business type.

For each current business type, after the image areas related to the current business type are marked in step 202, image classification feature extraction is performed on the original image based on these marked image areas to obtain the first state feature vector corresponding to the corresponding business type.

When the aforementioned two-level network is used to implement the illegal business model, the processing of this step is also the processing of the second-level network, that is, for the i-th current business type, based on the Mask map and the original image corresponding to the label corresponding to the business type , performing fusion processing and feature extraction processing of the classification network to obtain the classification feature vector of the fused image, that is, the first state feature vector corresponding to the i-th current service type. Wherein, i is the index of the current service type.

Next, the processing of steps 204 and 205 is performed for each first state feature vector to obtain the quality scoring result of the original image under the business type corresponding to the first state feature vector.

Step 204, for each first state feature vector, calculate the similarity between the first state feature vector and the state feature vectors of R*P standard images corresponding to the current service type, and obtain R*P similarity results.

The processing of each first state feature vector in this step is the same. For the convenience of description, the processing of a first state feature vector corresponding to a current service type X is used as an example to illustrate the specific processing of this step.

As mentioned above, in this application, in order to score the image quality corresponding to different business types, corresponding R*P standard images are set for each business type. In this step, R*P standard images corresponding to the current service type X and state feature vectors of the R*P standard images are determined. Among them, each standard image and the label of the current business type X are input into the illegal business judgment model, and the output classification feature vector is obtained after being processed by the illegal business judgment model, which is the state feature vector of the standard image. The process of processing the standard image to obtain the state feature vector of the standard image may be performed in this step or in advance. In this step, the state feature vector of the standard image obtained after the prediction process is obtained.

Comparing the first state feature vector with the state feature vectors of R*P standard images corresponding to the current service type X to obtain R*P similarity results corresponding to the current service type X.

Step 205, for each first state feature vector, based on the R*P similarity results, determine the quality scoring result of the original image to be scored under the corresponding current service type.

The processing of each first state feature vector in this step is the same. For the convenience of description, the processing of a first state feature vector corresponding to a current service type X is used as an example to illustrate the specific processing of this step.

For the R*P similarity results corresponding to the current service type X, the R*P similarity results are used to calculate the quality scoring result of the original image to be scored under the current service type X. The specific calculation method may be to calculate the weighted average of R*P similarity results. Of course, the method of calculating the weighted average is just a general example in the future, and the actual application is not limited to this calculation method.

Since the R*P similarity results determined in step 204 can reflect the similarity between the first state feature information and the image features of the standard image (that is, an image that can distinguish states without ambiguity), these similarity results can be used for The original image quality is scored to reflect the quality status of the original image relative to the standard image, so as to effectively give a reasonable image quality score for each current business type.

So far, the flow of the specific embodiment of the first image quality scoring method shown in FIG. 2 ends.

Embodiment two:

This embodiment is used to introduce the second image quality scoring method in detail.

Fig. 5 is a schematic flowchart of a specific embodiment of the second image quality scoring method in the present application. For the convenience of description, in this specific embodiment, the application scenario of judging illegal behavior in the cab is still taken as an example to illustrate the image quality scoring method, and at the same time, the training process of the illegal business judging model and the quality scoring regression model are described together. As shown in Figure 5, the method includes:

Step 501, using training sample images to train and generate an illegal business judgment model.

The processing of this step is the same as that of step 201, and will not be repeated here.

Step 502, input the test sample image and the label corresponding to each test business type into the pre-trained illegal business judgment model, and mark the image area related to each test business type.

The test sample image and test business type in this embodiment are sample images used to train the quality scoring regression model. In the processing of the illegal business judgment model, the processing of the test sample image and test business type is the same as in Embodiment 1, respectively. The original image to be scored is treated the same as the current business type. Only in this embodiment, the test sample image is processed to obtain the corresponding image quality score, the purpose of which is to use the test sample image and its image quality score result as a training sample for training the quality score regression model.

Since the processing of the test sample image and the test service type is the same as the processing of the original image to be scored and the current service type in Embodiment 1, the detailed processing of steps 502-505 will not be repeated.

Step 503, in the illegal business judgment model, perform feature extraction based on the image area related to each test business type, and obtain a first state feature vector corresponding to each test business type.

Step 504, for each first state feature vector, calculate the similarity between the first state feature vector and the state feature vectors of R*P standard images of the corresponding test service type, and obtain R*P similarity results.

Step 505, for each first state feature vector, based on the R*P similarity results, determine the quality scoring result of the test sample image under the corresponding test service type.

By repeatedly performing steps 502-505 on multiple test sample images, multiple test sample images and their corresponding quality scoring results are obtained.

Step 506, using a plurality of test sample images and their corresponding quality scoring results as training samples to train and generate a quality scoring regression model.

There may be one or more quality scoring results corresponding to the test sample image, corresponding to one or more test service types. The network structure of the quality scoring regression model can adopt an existing neural network model, such as a CNN network.

When training the quality scoring regression model, the test sample image and the identification of the test service type are used as input, and the quality score result of the test sample image under the test service type is output, and the quality score result is compared with the quality score result obtained in step 505. Compare to get the loss function, and then adjust the parameters of the quality score regression model according to the loss function. Wherein, the input test service type identifier may be one test service type identifier or multiple test service type identifiers. That is to say, the trained quality score regression model can obtain image quality scores under one or more business types for an original image. The quality scoring regression model generated by training in the above manner can fit the image quality scoring process in steps 502-505 as much as possible.

Step 507, when it is necessary to perform image quality scoring on the original image to be scored, input the original image to be scored and the identification of each current business type into the pre-trained quality scoring regression model, and obtain the original image to be scored in each The quality scoring result under the current business type.

The processing of the aforementioned steps 501-506 can be classified as the training processing of the quality scoring regression model. Since this part of the processing is time-consuming and uses a lot of resources, it can be completed in advance. When the original image to be scored is actually required to perform image quality scoring, step 507 is directly executed to complete real-time image quality scoring. In this way, on the one hand, through the pre-completed training process, the time and resource consumption during real-time scoring are greatly saved; on the other hand, through the fitting of the quality scoring regression model to the image quality scoring process in steps 502-506, the image quality scoring results are guaranteed It can be applied to various business types, improve the accuracy of image quality scoring, and can also realize image quality scoring under multiple business types at the same time.

So far, the flow of the specific embodiment of the second image quality scoring method shown in FIG. 5 ends.

The above is the specific implementation of the two image quality scoring methods provided in this application. In this application, the image quality score is generated through the weak supervision of the business type, instead of using similar appearance, template comparison, and human perception, but driven by the business, by training the state recognition model, the original image Calculate the average similarity with the features of multiple standard samples with unambiguous and identifiable states, and use it as the quality score to fully serve the needs of the state recognition business and improve the accuracy of image quality scoring.

This application corresponds to the above two image quality scoring methods, and also provides two kinds of image quality scoring devices. Fig. 6 is a first image quality scoring device, which can be used to realize the first image quality scoring method. As shown in Figure 6, the first image quality scoring device includes: a state recognition unit and a scoring unit;

Among them, the state recognition unit is used to input the original image to be scored and the label corresponding to each current business type into the pre-trained state recognition model, mark the image area related to each current business type, and based on the image area Perform feature extraction to obtain a first state feature vector corresponding to each current service type;

Scoring unit, for each first state feature vector, calculate the similarity between the first state feature vector and the state feature vectors of R*P standard images corresponding to the corresponding test service type, and obtain R*P similarities Result; also used for each first state feature vector, based on the corresponding R*P similarity results, to determine the quality scoring result of the original image to be scored under the corresponding current service type;

Among them, the state feature vector of the standard image is: the state feature vector obtained after feature extraction of the standard image and the label input state recognition model corresponding to the current business type; R*P standard images are corresponding to various states of the current business type A typical image of the set status category that can be distinguished without ambiguity, R is the total number of all status categories under the corresponding current service type, and P is a positive integer.

Optionally, in the state recognition unit, the state recognition model includes a first-level network for semantic segmentation and a second-level network for state recognition;

The input of the first-level network is the original image and label to be scored, and the output is a semantic feature map corresponding to each label, which is used to mark the image area related to each current business type;

The second-level network includes M sub-networks corresponding to M service types; wherein, M is the total number of all service types;

The input of each sub-network is the original image to be scored and the corrected semantic feature map corresponding to each label of the first current business type corresponding to the sub-network, and each sub-network combines the original image to be scored and the corrected The final semantic feature map is connected in the channel dimension, and the connected image is input into the classification network for state recognition, and the first state feature vector corresponding to the first current business type is obtained; wherein, the corrected semantic feature map is the above Sampling to a semantic feature map of the same size as the original image to be scored.

Optionally, in order to further accelerate the classification convergence and improve the robustness of the network, position information can be provided for the connected images in the state recognition unit. Specifically, the processing of inputting the connected image into the classification network for state recognition may include:

Divide the connected image into several two-dimensional slice patches, and add the two-dimensional position information corresponding to the two-dimensional slice in each of the two-dimensional slices;

Input all the two-dimensional slices with two-dimensional position information into the classification network for state recognition;

Wherein, through the training of the state recognition model, it is ensured that the weight of the two-dimensional position information corresponding to the two-dimensional slice located in the image area related to the current business type is greater than that of the two-dimensional position information located outside the image area related to the current business type. The weight of the two-dimensional position information corresponding to the slice.

Optionally, in order to further improve the classification accuracy, in the state recognition unit, an attention mechanism can be added to the classification network to assign a weight to each pixel input to the classification network, and the weight of the pixel located in the image area related to the current business type Greater than the weight of pixels located outside the image area related to the current business type.

Optionally, the image quality scoring device shown in FIG. 6 may further include a training unit for training and generating the state recognition model;

Wherein, when the training unit trains the state recognition model, the training sample image can be used to complete the training of the first-level network, and based on the trained first-level network, the second-level network is trained; or,

When the training unit trains the state recognition model, the first-level network and the second-level network can be jointly trained; or,

When training the state recognition model in the training unit, use the training sample image to perform initial training on the first-level network, based on the first-level network obtained after the initial training, perform initial training on the second-level network; perform initial training on the second-level network After that, jointly train the first-level network and the second-level network;

During joint training, the loss function is a weighted sum of the first loss function of the first-level network and the second loss function of the second-level network.

Optionally, when performing joint training, the first loss function corresponding to each input business type may be calculated based on the loss weights of all input tags; wherein, the input business type is the business type to which the input tag belongs;

When calculating the first loss function corresponding to any input service type, the loss weight of the input tag of any input service type is greater than the loss weight of the input tags not belonging to any input service type.

Optionally, in the state identification unit, the process of determining the quality scoring result of the original image to be scored under the corresponding current business type may specifically include:

Calculate the weighted average of the R*P similarity results, and use the calculation result as the quality scoring result of the original image to be scored under the corresponding current business type.

Corresponding to the second image quality scoring method, the present application provides an image quality scoring system that can be used to implement the second image quality scoring method. The basic structure of the system is shown in Figure 7, specifically including an image quality scoring device and The training device, wherein the basic structures of the image quality scoring device and the training device in the image quality scoring system are shown in Figure 8 and Figure 9 respectively.

Referring to Fig. 7, Fig. 8 and Fig. 9, the training device comprises a state recognition unit, a scoring unit and a first training unit;

Among them, the state recognition unit is used to input the test sample image and the label corresponding to each test business type into the pre-trained state recognition model, mark the image area related to each test business type, and perform feature based on the image area Extract to obtain the first state feature vector corresponding to each test service type;

Scoring unit, for each first state feature vector, calculate the similarity between the first state feature vector and the state feature vectors of R*P standard images corresponding to the corresponding test service type, and obtain R*P similarities Result; also used for each first state feature vector, based on R*P similarity results, to determine the quality scoring result of the test sample image under the corresponding test service type;

The first training unit is used to use the test sample image, the identification of each current business type, and the quality scoring results of the test sample image under each current business type to train and generate a quality scoring regression model;

Wherein, the state feature vector of the standard image is: the state feature vector obtained after the standard image and the label of the corresponding test service type are input into the state recognition model for feature extraction; R*P standard images are each corresponding to the corresponding test service type A typical image of the unambiguously identifiable status category of a status setting, R is the total number of all status categories under the corresponding test service type, and P is a positive integer.

The image quality scoring device includes: an input unit, a quality scoring unit and an output unit;

Wherein, the input unit is used to receive the original image to be scored and the identification of each current business type, and send it to the quality scoring unit;

The quality scoring unit is used to input the received original image to be scored and the identification of each current business type into the quality scoring regression model pre-trained by the training device to obtain the original image to be scored under each current business type quality rating results for

The output unit is used to output the quality scoring result of the original image to be scored under each current service type.

Optionally, in the state recognition unit of the training device, the state recognition model may include a first-level network for semantic segmentation and a second-level network for state recognition;

The input of the first-level network is the test sample image and the label, and the output is a semantic feature map corresponding to each of the labels, which is used to mark the image area related to each test service type;

The second-level network includes M sub-networks corresponding to M service types; wherein, M is the total number of all service types;

The input of each sub-network is the test sample image and the corrected semantic feature map corresponding to each label of the first test service type corresponding to the sub-network, and each sub-network will test the sample image and the corrected semantic feature map in the channel Dimensional connection is performed, and the connected image is input into the classification network for state recognition, and the first state feature vector corresponding to the first test service type is obtained; wherein, the corrected semantic feature map is up-sampled to the Semantic feature maps with the same size as test sample images.

Optionally, in the state recognition unit of the training device, the connected image is input into the classification network for state recognition, including:

Divide the connected image into several two-dimensional slice patches, and add the two-dimensional position information corresponding to the two-dimensional slice in each of the two-dimensional slices;

Input all the two-dimensional slices with two-dimensional position information into the classification network for state recognition;

Among them, through the training of the state recognition model, it is ensured that the weight of the two-dimensional position information corresponding to the two-dimensional slice located in the image area related to the current business type is greater than the two-dimensional position corresponding to the two-dimensional slice located outside the image area related to the current business type information weight.

Optionally, in the state identification unit of the training device, an attention mechanism is added to the classification network, and a weight is assigned to each pixel input into the classification network, and the weight of the pixel located in the image area related to the test service type is greater than that located in the pixel located in the classification network. The weight of pixels outside the image area related to the test service type described above.

Optionally, the training device may further include a second training unit for training the generated state recognition model;

Wherein, when the state recognition model is trained in the second training unit, the training sample image is used to complete the training of the first-level network, and the second-level network is trained based on the trained first-level network; or,

When the second training unit trains the state recognition model, jointly train the first-level network and the second-level network; or,

When the second training unit trains the state recognition model, the first-level network is initially trained using the training sample image, and the second-level network is initially trained based on the first-level network obtained after the initial training; at the second level After the network is initially trained, the first-level network and the second-level network are jointly trained;

During joint training, the loss function is a weighted sum of the first loss function of the first-level network and the second loss function of the second-level network.

Optionally, when the second training unit performs joint training, the first loss function corresponding to each input business type is calculated based on the loss weights of all input tags; wherein, the input business type is the business type to which the input tag belongs;

When calculating the first loss function corresponding to any input service type, the loss weight of the input tag of any input service type is greater than the loss weight of the input tags not belonging to any input service type.

Optionally, in the state identification unit of the training device, the process of determining the quality scoring result of the test sample image under the corresponding test service type may specifically include:

Calculate a weighted mean value for the R*P similarity results, and use the calculation result as the quality scoring result of the test sample image under the corresponding test service type.

The present application also provides a computer-readable storage medium, where the computer-readable storage medium stores instructions, and the instructions, when executed by a processor, can perform the steps in the method for scoring image quality as described above. In practical applications, the computer-readable medium may be included in each device/device/system of the above-mentioned embodiments, or may exist independently without being assembled into the device/device/system. Wherein, instructions are stored in the computer-readable storage medium, and the instructions stored in the computer-readable storage medium can execute the steps in the above-mentioned image quality scoring method when executed by the processor.

According to the embodiments disclosed in the present application, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as may include but not limited to: portable computer disk, hard disk, random access memory (RAM), read-only memory (ROM) ), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above, but not used to limit this application scope of protection. In the embodiments disclosed in the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, and the program may be used by or in combination with an instruction execution system, apparatus or device.

Fig. 10 is an electronic device further provided by the present application. As shown in Figure 10, it shows a schematic structural diagram of the electronic device involved in the embodiment of the present application, specifically:

The electronic device may include a processor 1001 of one or more processing cores, a memory 1002 of one or more computer-readable storage media, and a computer program stored on the memory and executable on the processor. When the programs in the memory 1002 are executed, the image quality scoring method can be implemented.

Specifically, in practical applications, the electronic device may further include components such as a power supply 1003 and an input and output unit 1004 . Those skilled in the art can understand that the structure of the electronic device shown in Figure 10 does not constitute a limitation to the electronic device, and may include more or less components than shown in the illustration, or combine some components, or different components layout. in:

The processor 1001 is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device, by running or executing software programs and/or modules stored in the memory 1002, and calling the data, perform various functions of the server and process data, thereby exercising overall control of the electronic device.

The memory 1002 can be used to store software programs and modules, that is, the above-mentioned computer-readable storage medium. The processor 1001 executes various functional applications and data processing by executing software programs and modules stored in the memory 1002 . The memory 1002 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, at least one application required by a function, etc.; the data storage area may store data created according to the use of the server, etc. In addition, the memory 1002 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage devices. Correspondingly, the memory 1002 may further include a memory controller to provide the processor 1001 with access to the memory 1002 .

The electronic device also includes a power supply 1003 for supplying power to various components, which can be logically connected to the processor 1001 through the power management system, so as to implement functions such as managing charge, discharge, and power consumption through the power management system. The power supply 1003 may also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.

The electronic device may also include an input/output unit 1004, the input unit output 1004 may be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical signal input related to user settings and function control. The input unit output 1004 can also be used to display information input by the user or provided to the user, as well as various graphical user interfaces. These graphical user interfaces can be composed of graphics, text, icons, videos and any combination thereof.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the present invention. within the scope of protection.

Claims (22)

1. An image quality scoring method, characterized in that, comprising: Input the original image to be scored and the label corresponding to each current business type into a pre-trained state recognition model, mark the image area related to each current business type, and perform feature extraction based on the image area, Obtaining a first state feature vector corresponding to each current service type; For each of the first state feature vectors, calculate the similarity between the first state feature vector and the state feature vectors of R*P standard images corresponding to the corresponding current service type, and obtain R*P similarity results; For each of the first state feature vectors, based on the R*P similarity results, determine the quality scoring results of the original image to be scored under the corresponding current service type; Wherein, the state feature vector of the standard image is: the state feature vector obtained after the standard image and the label of the corresponding current business type are input into the state recognition model for feature extraction; R*P standard images are corresponding to the corresponding current business type Typical images of various status settings that can be unambiguously identified status categories, the R is the total number of all status categories under the corresponding current service type, and the P is a positive integer. 2. The method according to claim 1, wherein the state recognition model comprises a first-level network for semantic segmentation and a second-level network for state recognition; The input of the first-level network is the original image to be scored and the label, and the output is a semantic feature map corresponding to each of the labels, which is used to mark the image area related to each of the current business types ; The second-level network includes M sub-networks corresponding to M service types; wherein, the M is the total number of all service types; The input of each sub-network is the original image to be scored and the corrected semantic feature map corresponding to each label of the first current business type corresponding to the sub-network, and each of the sub-networks combines the The original image to be scored and the corrected semantic feature map are connected in the channel dimension, and the connected image is input into the classification network for state recognition to obtain the first state corresponding to the first current business type A feature vector; wherein, the corrected semantic feature map is a semantic feature map upsampled to the same size as the original image to be scored. 3. The method according to claim 2, wherein the state recognition of the connected image input classification network includes: Divide the connected image into several two-dimensional slice patches, and add the two-dimensional position information corresponding to the two-dimensional slice in each of the two-dimensional slices; Input all the two-dimensional slices with two-dimensional position information into the classification network for state recognition; Wherein, through the training of the state recognition model, it is ensured that the weight of the two-dimensional position information corresponding to the two-dimensional slice located in the image area related to the current business type is greater than that of the two-dimensional position information located outside the image area related to the current business type. The weight of the two-dimensional position information corresponding to the slice. 4. The method according to claim 2 or 3, wherein an attention mechanism is added to the classification network, a weight is assigned to each pixel input into the classification network, and it is located at the relevant pixel of the current business type. The weight of pixels in the image area is greater than the weight of pixels outside the image area related to the current service type. 5. The method according to claim 2, wherein, when training the state recognition model, the training sample image is used to complete the training of the first-level network, based on the trained first-level network, to The second level network is trained; or, When training the state recognition model, jointly train the first-level network and the second-level network; or, When training the state recognition model, use training sample images to perform initial training on the first-level network, and based on the first-level network obtained after the initial training, perform initial training on the second-level network; After the initial training of the second-level network, joint training is performed on the first-level network and the second-level network; Wherein, when performing the joint training, the loss function is a weighted sum of the first loss function of the first-level network and the second loss function of the second-level network. 6. The method according to claim 5, wherein when performing the joint training, the first loss function corresponding to each input business type is calculated based on the loss weights of all input tags; wherein, the input The business type is the business type to which the input tag belongs; When calculating the first loss function corresponding to any input service type, the loss weight of the input tags of the any input service type is greater than the loss weight of the input tags not belonging to the any input service type. 7. The method according to claim 5 or 6, wherein the first loss function is a dice loss function, and the second loss function is a focal loss function. 8. The method according to claim 1, wherein each status category under each current business type includes P standard images, and the P standard images include different scene brightnesses, different shooting angles and/or Raw images of different poses under each state category. 9. The method according to claim 1, wherein the determining the quality scoring result of the original image to be scored under the corresponding test service type comprises: Calculate a weighted mean value for the R*P similarity results, and use the calculation result as the quality scoring result of the original image to be scored under the corresponding current service type. 10. An image quality scoring method, characterized in that, comprising: Input the original image to be scored and the identification of each current business type into a pre-trained quality scoring regression model to obtain the quality scoring result of the original image to be scored under each current business type; Wherein, the quality scoring regression model is a neural network model obtained by using the test sample image, the identification of each test service type, and the quality score result of the test sample image under each test service type; The determination process of the quality scoring result of the test sample image under each test service type includes: Input the test sample image and the label corresponding to each test service type into a pre-trained state recognition model, mark the image area related to each test service type, and perform feature based on the image area Extracting to obtain the first state feature vector corresponding to each test service type; For each of the first state feature vectors, calculate the similarity between the first state feature vector and the state feature vectors of R*P standard images corresponding to the corresponding test service type, and obtain R*P similarity results; For each of the first state feature vectors, based on the R*P similarity results, determine the quality scoring results of the test sample images under the corresponding test service types; Wherein, the state feature vector of the standard image is: the state feature vector obtained after the standard image and the label of the corresponding test service type are input into the state recognition model for feature extraction; R*P standard images are corresponding to the corresponding test service type Typical images of various status settings that can be distinguished without ambiguity. The R is the total number of all status categories under the corresponding test service type, and the P is a positive integer. 11. An image quality scoring device, comprising: a state recognition unit and a scoring unit; The state recognition unit is used to input the original image to be scored and the label corresponding to each current business type into a pre-trained state recognition model, mark the image area related to each current business type, and based on performing feature extraction on the image area to obtain a first state feature vector corresponding to each current service type; The scoring unit is configured to, for each of the first state feature vectors, calculate the similarity between the first state feature vector and the state feature vectors of R*P standard images corresponding to the corresponding test service type, to obtain R* P similarity results; for each of the first state feature vectors, based on the R*P similarity results, determine the quality scoring results of the original image to be scored under the corresponding current service type; Wherein, the state feature vector of the standard image is: the state feature vector obtained after the standard image and the label of the corresponding current business type are input into the state recognition model for feature extraction; R*P standard images are corresponding to the corresponding current business type Typical images of various status settings that can be unambiguously identified status categories, the R is the total number of all status categories under the corresponding current service type, and the P is a positive integer. 12. The device according to claim 11, wherein, in the state recognition unit, the state recognition model includes a first-level network for semantic segmentation and a second-level network for state recognition ; The input of the first-level network is the original image to be scored and the label, and the output is a semantic feature map corresponding to each of the labels, which is used to mark the image area related to each of the current business types ; The second-level network includes M sub-networks corresponding to M service types; wherein, the M is the total number of all service types; The input of each sub-network is the original image to be scored and the corrected semantic feature map corresponding to each label of the first current business type corresponding to the sub-network, and each of the sub-networks combines the The original image to be scored and the corrected semantic feature map are connected in the channel dimension, and the connected image is input into the classification network for state recognition to obtain the first state corresponding to the first current business type A feature vector; wherein, the corrected semantic feature map is a semantic feature map upsampled to the same size as the original image to be scored. 13. The device according to claim 12, wherein, in the state identification unit, the input of the connected image into the classification network for state identification includes: Divide the connected image into several two-dimensional slice patches, and add the two-dimensional position information corresponding to the two-dimensional slice in each of the two-dimensional slices; Input all the two-dimensional slices with two-dimensional position information into the classification network for state recognition; Wherein, through the training of the state recognition model, it is ensured that the weight of the two-dimensional position information corresponding to the two-dimensional slice located in the image area related to the current business type is greater than that of the two-dimensional position information located outside the image area related to the current business type. The weight of the two-dimensional position information corresponding to the slice. 14. The device according to claim 12 or 13, wherein in the state recognition unit, an attention mechanism is added to the classification network, and a weight is assigned to each pixel input to the classification network, and is located at The weight of pixels in the image area related to the current service type is greater than the weight of pixels outside the image area related to the current service type. 15. The device according to claim 12, characterized in that the device further comprises a training unit for training and generating the state recognition model; Wherein, when the training unit trains the state recognition model, the training sample image is used to complete the training of the first-level network, and the second-level network is trained based on the trained first-level network; or, When the training unit trains the state recognition model, jointly train the first-level network and the second-level network; or, When the training unit trains the state recognition model, the training sample image is used to initially train the first-level network, and based on the first-level network obtained after the initial training, the second-level network is initially trained. training; after the initial training of the second-level network, jointly train the first-level network and the second-level network; When performing the joint training, the loss function is a weighted sum of the first loss function of the first-level network and the second loss function of the second-level network. 16. The device according to claim 15, wherein when performing the joint training, the first loss function corresponding to each input service type is calculated based on the loss weights of all input tags; wherein, the input The business type is the business type to which the input tag belongs; When calculating the first loss function corresponding to any input service type, the loss weight of the input tags of the any input service type is greater than the loss weight of the input tags not belonging to the any input service type. 17. The device according to claim 11, wherein, in the state identification unit, the determination of the quality scoring result of the original image to be scored under the corresponding current service type comprises: Calculate a weighted mean value for the R*P similarity results, and use the calculation result as the quality scoring result of the original image to be scored under the corresponding current service type. 18. An image quality scoring system, comprising a training device and an image quality scoring device; The training device includes a state recognition unit, a scoring unit and a first training unit; The state recognition unit is used to input the test sample image and the label corresponding to each test service type into the pre-trained state recognition model, mark the image area related to each test service type, and based on the performing feature extraction on the image area to obtain a first state feature vector corresponding to each test service type; The scoring unit is configured to, for each of the first state feature vectors, calculate the similarity between the first state feature vector and the state feature vectors of R*P standard images corresponding to the corresponding test service type, to obtain R* P similarity results; for each of the first state feature vectors, based on the R*P similarity results, determine the quality scoring result of the test sample image under the corresponding test service type; The first training unit is configured to use the test sample image, the identification of each current service type, and the quality score result of the test sample image under each current service type to train and generate a quality score regression model; The image quality scoring device includes an input unit, a quality scoring unit and an output unit; The input unit is used to receive the original image to be scored and the identification of each current service type, and send them to the quality scoring unit; The quality scoring unit is configured to input the received original image to be scored and the identification of each current business type into the quality scoring regression model to obtain the original image to be scored under each current business type quality rating results for The output unit is configured to output a quality scoring result of the original image to be scored under each current service type; Wherein, the state feature vector of the standard image is: the state feature vector obtained after the standard image and the label of the corresponding test service type are input into the state recognition model for feature extraction; R*P standard images are corresponding to the corresponding test service type Typical images of various status settings that can be distinguished without ambiguity. The R is the total number of all status categories under the corresponding test service type, and the P is a positive integer. 19. An image quality scoring device, comprising: an input unit, a quality scoring unit, and an output unit; The input unit is used to receive the original image to be scored and the identification of each current service type, and send them to the quality scoring unit; The quality scoring unit is configured to input the received original image to be scored and the identification of each current service type into a quality scoring regression model pre-trained by a training device, and obtain the original image to be scored in each The quality scoring results under the current business type; The output unit is configured to output a quality scoring result of the original image to be scored under each current service type; The training device includes a state recognition unit, a scoring unit and a first training unit; The state recognition unit is used to input the test sample image and the label corresponding to each test service type into the pre-trained state recognition model, mark the image area related to each test service type, and based on the performing feature extraction on the image area to obtain a first state feature vector corresponding to each test service type; The scoring unit is configured to, for each of the first state feature vectors, calculate the similarity between the first state feature vector and the state feature vectors of R*P standard images corresponding to the corresponding test service type, to obtain R* P similarity results; for each of the first state feature vectors, based on the R*P similarity results, determine the quality scoring result of the test sample image under the corresponding test service type; The first training unit is configured to train and generate the Quality score regression model; Wherein, the state feature vector of the standard image is: the state feature vector obtained after the standard image and the label of the corresponding test service type are input into the state recognition model for feature extraction; R*P standard images are corresponding to the corresponding test service type Typical images of various status settings that can be distinguished without ambiguity. The R is the total number of all status categories under the corresponding test service type, and the P is a positive integer. 20. A training device, characterized in that, comprising: a state recognition unit, a scoring unit and a first training unit; The state recognition unit is used to input the test sample image and the label corresponding to each test service type into the pre-trained state recognition model, mark the image area related to each test service type, and based on the performing feature extraction on the image area to obtain a first state feature vector corresponding to each test service type; The scoring unit is configured to, for each of the first state feature vectors, calculate the similarity between the first state feature vector and the state feature vectors of R*P standard images corresponding to the corresponding test service type, to obtain R* P similarity results; for each of the first state feature vectors, based on the R*P similarity results, determine the quality scoring result of the test sample image under the corresponding test service type; The first training unit is configured to use the test sample image, the identification of each current service type, and the quality score result of the test sample image under each current service type to train and generate a quality score regression model; Wherein, the quality scoring model is used to process the input original image to be scored and the identification of each current business type to obtain the quality scoring result of the original image to be scored under each current business type; The state feature vector of the standard image is: the state feature vector obtained after the standard image and the label of the corresponding test service type are input into the state recognition model for feature extraction; the R*P standard images are each corresponding to the corresponding test service type. A typical image of a status category that can be distinguished without ambiguity for a status setting, the R is the total number of all status categories under the corresponding test service type, and the P is a positive integer. 21. A computer-readable storage medium, on which computer instructions are stored, wherein the image quality scoring method according to any one of claims 1-10 can be implemented when the instructions are executed by a processor. 22. An electronic device, characterized in that the electronic device includes at least a computer-readable storage medium, and also includes a processor; The processor is configured to read the executable instructions from the computer-readable storage medium, and execute the instructions to implement the image quality scoring method according to any one of claims 1-10.

Images