CN116310605A - Image detection method, device, equipment and computer storage medium - Google Patents

Abstract

The embodiment of the invention relates to the technical field of computer data processing, and discloses an image detection method, which comprises the following steps: inputting an image to be detected into a target detection model to obtain a target detection result corresponding to the image to be detected; the target detection model includes a modified YOLOv3 model; the improved YOLOv3 model comprises a high-dimensional convolution module; the high-dimensional convolution module is used for carrying out depth convolution on the image to be detected in a high-dimensional space to obtain image characteristics of the image to be detected. By the mode, the embodiment of the invention realizes the balance of the precision and the detection speed when the target detection is carried out on the high-resolution images such as the remote sensing image and the like.

Description

Image detection method, device, equipment and computer storage medium

technical field

Embodiments of the present invention relate to the technical field of computer data processing, and in particular to an image detection method, device, equipment, and computer storage medium.

Background technique

Remote sensing images have the advantages of ultra-long distance, all-day operation and strong anti-interference ability. Image detection and recognition of remote sensing images is a research hotspot in computer vision. In the prior art, the detection tasks of high-resolution remote sensing images are mainly based on the YOLO (You only look once) series. The YOLO algorithm first divides an image into SxS grids. By determining that the center of the object overlaps with a certain grid, the prediction of the object will be in charge of the network. On the basis of v1, YOLOv3 analyzes the data set labels through K-means clustering, and upgrades the feature extraction network from Darknet19 to Darknet53.

The inventor found in the process of implementing the embodiment of the present invention that the YOLOv3 model mainly extracts features through the RES and DBL structures. Among them, a RES operation is mainly composed of a 3x3 convolution with a step size of 2 and n ResUnits, and a ResUnit is composed of a 1x1 convolution and a 3x3 convolution operation. However, a large number of 3x3 convolution operations have caused a sharp increase in model parameters. Considering the high resolution of the remote sensing image itself, YOLOv3 cannot take into account both detection speed and accuracy.

Contents of the invention

In view of the above problems, an embodiment of the present invention provides an image detection method, which is used to solve the problem that the existing YOLOv3 cannot balance detection speed and accuracy when performing target detection on high-resolution images.

According to an aspect of an embodiment of the present invention, an image detection method is provided, the method comprising:

The image to be detected is input into the target detection model to obtain the target detection result corresponding to the image to be detected; the target detection model includes an improved YOLOv3 model; the improved YOLOv3 model includes a high-dimensional convolution module; the high-dimensional The convolution module is used to perform deep convolution on the image to be detected in a high-dimensional space to obtain image features of the image to be detected.

In an optional manner, the high-dimensional convolution module includes a first depth convolution layer, a dimension adjustment convolution layer, and a second depth convolution layer connected in sequence; the first depth convolution layer and the The second depth convolution layer is used to perform depth convolution on the input data; the dimension adjustment convolution layer is used to perform dimension adjustment on the input data, so that the data input to the second depth convolution layer is high-dimensional data .

In an optional manner, the improved YOLOv3 model includes: a multi-scale convolution module; the multi-scale convolution module is used to convolve the input data under a multi-scale convolution kernel to obtain a multi-scale The image features; wherein, the convolution kernel includes the high-dimensional convolution module.

In an optional manner, the multi-scale convolution module includes multiple layers of sequentially connected convolution kernels; the size of the convolution kernels decreases sequentially, and the depth of the convolution kernels increases sequentially; multiple The convolution kernels use group convolution, so that each of the convolution kernels outputs the image features with the same number of channels.

In an optional manner, the improved YOLOv3 model further includes a feature enhancement module: the feature enhancement module is configured to perform feature enhancement on the image features according to a channel attention mechanism.

In an optional manner, the feature enhancement module includes sequentially connected pooling layers, multiple fully connected layers, and weighted calculation layers; the pooling layer is used to extract image features of multiple channels; the full The connection layer is used to determine inter-channel dependencies according to the image features of each of the channels; the weighted calculation layer is used to perform weighted calculations on the image features of the multiple channels according to the inter-channel dependencies to obtain enhanced image features .

In an optional manner, the image to be detected includes a high-resolution image; the improved YOLOv3 model is improved based on the original YOLOv3 model; the original YOLOv3 model includes an original DBL module and an original RES module, and the The original DBL module includes sequentially connected original convolutional layers, normalization layers, and activation function layers, and the original RES module includes sequentially connected zero-fill layers, the original DBL module, and the original residual structure; wherein, in the In the improved YOLOv3 model, the original convolution layer is replaced by a high-dimensional convolution module, a feature enhancement module is added after the activation function layer, and a multi-scale convolution module is added after the original residual structure; the The high-dimensional convolution module is used to perform deep convolution on the input data in a high-dimensional space; the feature enhancement module is used to perform feature enhancement on the image features according to the channel attention mechanism; the multi-scale convolution module is used for It is used to perform convolution on input data under a multi-scale convolution kernel; the convolution kernel includes the high-dimensional convolution module.

According to another aspect of the embodiments of the present invention, an image detection device is provided, including:

The detection module is used to input the image to be detected into the target detection model to obtain the target detection result corresponding to the image to be detected; the target detection model includes an improved YOLOv3 model; the improved YOLOv3 model includes a high-dimensional convolution module ; The high-dimensional convolution module is used to perform depth convolution on the image to be detected in a high-dimensional space to obtain image features of the image to be detected.

According to another aspect of the embodiments of the present invention, an image detection device is provided, including:

A processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface complete mutual communication through the communication bus;

The memory is used to store at least one executable instruction, and the executable instruction causes the processor to execute the operation of the image detection method embodiment described in any one of the foregoing.

According to still another aspect of the embodiments of the present invention, a computer-readable storage medium is provided, and at least one executable instruction is stored in the storage medium, and the executable instruction causes the image detection device to perform the above-mentioned any one of the above-mentioned The operation of the image detection method embodiment.

In the embodiment of the present invention, the target detection result corresponding to the image to be detected is obtained by inputting the image to be detected into the target detection model; the target detection model includes an improved YOLOv3 model; the improved YOLOv3 model includes a high-dimensional convolution module ; Wherein, the high-dimensional convolution module is used to perform deep convolution on the image to be detected in a high-dimensional space to obtain the image features of the image to be detected, so as to be different from the existing YOLOv3 model that adopts a large number of 3x3 volumes feature extraction by product operation, which leads to the problem that the parameter amount of the model is too large and the detection speed is not good. In the embodiment of the present invention, the depth convolution is placed at the beginning and end of the entire convolution structure, so that the depth convolution operation acts on the High-dimensional space, from which information-rich feature representations can be extracted, and identity mapping and spatial transformation can be performed in a higher dimension, which reduces the risk of information loss and gradient confusion caused by the traditional mobile network inverted residual structure, so that it can Effectively improve the processing speed of the model and achieve a balance between detection accuracy and detection speed.

The above description is only an overview of the technical solutions of the embodiments of the present invention. In order to better understand the technical means of the embodiments of the present invention, it can be implemented according to the contents of the description, and in order to make the above and other purposes, features and The advantages can be more obvious and understandable, and the specific embodiments of the present invention are enumerated below.

Description of drawings

The drawings are only for illustrating the embodiments and are not to be considered as limiting the invention. Also throughout the drawings, the same reference numerals are used to designate the same components. In the attached picture:

Figure 1 shows a schematic diagram of the existing YOLOv3 model structure;

FIG. 2 shows a schematic flowchart of an image detection method provided by an embodiment of the present invention;

FIG. 3 shows a schematic structural diagram of a high-dimensional convolution model in an image detection method provided by an embodiment of the present invention;

FIG. 4 shows a schematic structural diagram of a multi-scale convolution module in an image detection method provided by an embodiment of the present invention;

FIG. 5 shows a schematic structural diagram of a feature enhancement module in an image detection method provided by an embodiment of the present invention;

FIG. 6 shows a schematic structural diagram of an improved DBL module in an image detection method provided by an embodiment of the present invention;

FIG. 7 shows a schematic structural diagram of an improved NRES module in an image detection method provided by an embodiment of the present invention;

Fig. 8 shows a comparison diagram of the detection results of the improved YOLOv3 in the image detection method provided by the embodiment of the present invention and the prior art;

FIG. 9 shows a schematic structural diagram of an image detection device provided by an embodiment of the present invention;

FIG. 10 shows a schematic structural diagram of an image detection device provided by an embodiment of the present invention.

Detailed ways

Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the invention may be embodied in various forms and should not be limited to the embodiments set forth herein.

Explain related nouns:

IoU value: Intersection over Union, also known as the intersection over union ratio, is usually used in the evaluation of current target detection algorithms. The higher the IoU value, the higher the prediction accuracy of the algorithm for the target. The value range of IoU is [0, 1]. The larger the IoU value, the better the coincidence of the two frames.

Pooling operation: use a matrix window to scan the input tensor, and reduce the number of elements by taking the value in each matrix window by taking the maximum value, average value, etc. (reducing the number of elements is equivalent to extracting important features of elements) Because the purpose of convolution is to extract features, after all the features are extracted, the key features must be further screened out. Therefore, the feature mechanical energy pooling operation can effectively reduce the size of the matrix, such as reducing the length and width of the matrix.

Group convolution: For ordinary convolution, if the input feature map size is CxHxW, the number of convolution kernels is N, and the size of each convolution kernel is CxKxK, then the output feature map size is CxHxN, and the total parameter amount is : NxCxKxK. If group convolution is performed, it is assumed to be divided into G groups, the number of feature maps input by each group is CG, the number of feature maps output by each group is NG, the size of each convolution kernel is CGxKxK, and the number of convolution kernels in each group is NG, the convolution kernel is only convolved with the input of the same group, then the total parameter amount is NxCGxKxK, and the total parameter amount is reduced to the previous 1G. When the number of groups is equal to the number of input images, the number of output images is also equal to the number of input images, that is, G=N=C, and each convolution kernel size is 1xKxK, it becomes Depthwiseconvolution (depth separable convolution).

Channel attention mechanism: used to explicitly model the interdependence between feature channels, by adopting a new "feature recalibration" strategy - adaptively recalibrating the feature response of the channel, so that the network pays more attention The target to be detected improves the detection effect.

Pyramid Convolution (PyConv): Processes input at multiple filter scales. PyConv consists of a kernel pyramid, and each layer contains different types of filters (the size and depth of the filters are variable, so that details at different scales can be extracted). In addition to extracting multi-scale information, PyConv is efficient compared to standard convolution, that is, it does not increase the amount of additional calculations and parameters. Furthermore, it is more flexible and scalable, providing a larger architecture design space for different applications.

NWPU VHR-10 Dataset: A dataset of 10 types of geospatial object detection for research and public use. These 10 types of objects are airplanes, ships, storage tanks, baseballs, tennis courts, basketball courts, ground runways, ports, bridges, and vehicles. . This dataset contains a total of 800 very high-resolution (VHR) remote sensing images, cropped from Google Earth and Vaihingen datasets, and manually annotated by experts.

Before carrying out the description of the embodiment of the present invention, the prior art and its existing problems are further described:

Remote sensing images have the advantages of ultra-long distance, all-day operation and strong anti-interference ability. Target detection and recognition of remote sensing images is a research hotspot in computer vision. At present, target detection in optical remote sensing images has attracted much attention, and it has been widely used in urban security, land planning and other fields.

For the detection tasks of high-resolution remote sensing images, the YOLO series is mainly used. The YOLO algorithm first divides an image into SxS grids. By determining that the center of the object overlaps with a certain grid, the prediction of the object will be in charge of the network. Each grid in YOLO predicts two pieces of information about the bounding box: position and confidence (Confidence). The location information contains four parameters, namely: the coordinates of the center point and the width and height of the bounding box. Confidence has two meanings, one is whether there is an object in the predicted bounding box, and the other is the accuracy of the predicted bounding box. It is defined as follows:

表示预测边界框和groundtruth(标准值)之间的IoU值。每个边界框要预测(x，y，w，h)和置信度共5个值。每个网格还要预测一个类别信息，记为C类。则SxS个网格，每个网格要预测B个边界框和C个类别。最终网络输出一个张量SxSx(5xB+C)。Among them, P _r (object) indicates whether there is an object in the grid, and it is 1 if it exists, otherwise it is 0;

Indicates the IoU value between the predicted bounding box and groundtruth (standard value). Each bounding box has 5 values to predict (x, y, w, h) and confidence. Each grid also predicts a category information, denoted as C category. Then there are SxS grids, and each grid needs to predict B bounding boxes and C categories. The final network outputs a tensor SxSx(5xB+C).

On the basis of v1, YOLOv3 analyzes the data set labels through K-means clustering, and upgrades the feature extraction network from Darknet19 to Darknet53. Among them, the darknet53 network is the backbone network of YOLOV3, which is used to extract the features of 8, 16, and 32 times downsampling respectively. The network part uses a lot of 1x1 convolution and 3x3 convolution, of which 1x1 is mainly used for channel expansion and reduction. The overall convolutional network adopts the Conv+BN+LeakyReLU structure. The residual network first uses a 1x1 convolution kernel to shrink the channel, and then uses a 3x3 convolution kernel to restore the channel. Its essence is the idea of matrix decomposition , used to reduce the number of parameters.

Specifically, the structure of the existing YOLOv3 model is shown in Figure 1. First, the three basic components of YOLOv3 are explained:

DBL: The smallest component in the YOLOv3 network structure, consisting of Conv+BN+Leaky_relu. Among them, Conv is the convolution layer, BN is Batch Normalization, normalization layer, Leaky_relu is the activation function layer; among them, BN will directly normalize the same channel of different samples in the dimension of batch size, and get C Mean and variance, and C γ, β (γ, β are the learning parameters of the BN layer). The role of the BN layer includes speeding up the training and convergence of the network, controlling the gradient explosion to prevent the gradient from disappearing, and preventing overfitting.

Res unit: Learn from the residual structure in the Resnet network, so that the network can be built deeper.

ResX: Consisting of a DBL and X residual components, it is a large component in YOLOv3. The DBL in front of each Res module plays the role of downsampling, so after 5 Res modules, the obtained feature map is 608->304->152->76->38->19 in size.

Other basic operations in YOLOv3:

Concat: Tensor splicing, splicing the upsampling of the middle layer of darknet and a certain layer behind. The operation of splicing is different from that of add in the residual layer. Splicing will expand the dimension of the tensor, while add will not change the dimension of the tensor.

add: Tensor addition, tensors are added directly, and the dimension will not be expanded. For example, if 104x104x128 and 104x104x128 are added, the result is still 104x104x128. add has the same function as the shortcut in the cfg file.

The number of convolutional layers in Backbone: each ResX contains 1+2xX convolutional layers, so the entire backbone network Backbone contains a total of 1+(1+2x1)+(1+2x2)+(1+2x8)+( 1+2x8)+(1+2x4)=52, plus a FC fully connected layer, can form a Darknet53 classification network. However, in the image detection YOLOv3, the FC layer is removed, but the backbone network of YOLOv3 is still called the Darknet53 structure for convenience.

As shown in Figure 1, YOLO_v3 uses the first 52 layers of darknet-53 (without a fully connected layer). The network of YOLO_v3 is a fully convolutional network, which uses a large number of residual layer skip connections, and in order to reduce the pooling. The negative effect of the gradient directly abandons POOLing (pooling), and uses conv (convolution) stride (that is, after a convolution, the feature map slides several grids, the default is 1, that is, slides one grid) to achieve downsampling . In this network structure, a convolution with a step size of 2 is used for downsampling.

In order to enhance the accuracy of the algorithm for small image detection, YOLO v3 adopts FPN-like upsample (upsampling) and fusion methods, and finally integrates 3 scales (scales), and the sizes of the other two scales are 26x26 and 52x52), respectively. Perform detection on feature maps (feature maps) of multiple scales.

The three prediction branches also adopt a full convolution structure, in which the number of convolution kernels in the last convolution layer is 255, which is for the 80 categories of the COCO dataset: 3x(80+4+1)=255, 3 indicates that a grid cell (grid) contains 3 bounding boxes (detection boxes), 4 indicates the 4 coordinate information of the detection box, and 1 indicates the objectness score (target existence possibility score). The multi-scale image feature detection results come from these three prediction paths. The depths of y1, y2, and y3 are all 255 (that is, 3x(5+80)), and the rule of side length is 13:26:52. YOLO v3 sets that each grid unit predicts 3 bounding boxes, so each box needs to have five basic parameters (x, y, w, h, confidence (confidence)), and then there are 80 categories The probability.

Three detections were performed in the network, which were detected at 32 times downsampling, 16 times downsampling, and 8 times downsampling, so that the detection on the multi-scale feature map is a bit like SSD. The reason for using up-sample (up-sampling) in the network is that the deeper the network, the better the feature expression effect. For example, when performing 16-fold down-sampling detection, if you directly use the fourth down-sampled feature to detect, then use If shallow features are removed, the effect is generally not good. If you want to use the features after 32 times downsampling, but the size of the deep features is too small, so YOLO_v3 uses up-sample (upsampling) with a step size of 2 to double the size of the feature map obtained by 32 times downsampling , which becomes the dimension after 16 times downsampling. Similarly, 8-fold sampling is also an upsampling with a step size of 2 on the features of 16-fold downsampling, so that deep features can be used for detection.

Finally, YOLOv3 extracts deep features by upsampling, and its dimension is the same as the dimension of the feature layer to be fused (but the channels are different). As shown in Figure 1, the 85th layer upsamples the features of 13x13x256 to get 26x26x256, and then stitches it with the features of the 61st layer to get 26x26x768. In order to obtain channel255, a series of 3x3 and 1x1 convolution operations are required, which can not only improve the degree of nonlinearity, increase generalization performance, improve network accuracy, but also reduce parameters and improve real-time performance. The characterization of 52x52x255 is also a similar process.

In summary, for an input image, YOLOv3 maps it to an output tensor of 3 scales, representing the probability of various objects in each position of the image. Based on the feature map obtained by Darknet-53, the first feature map is obtained through six DBL structures and the last convolutional layer, and the first prediction is made on this feature map. On the Y1 branch, the output of the third convolutional layer from the back to the front, after a DBL structure and an upsampling, connects the upsampling feature with the convolutional feature tensor output by the second Res8 structure, and passes through six The DBL structure and the last convolutional layer get the second feature map, and make a second prediction on this feature map. On the Y2 branch, the output of the third convolutional layer from the back to the front, after a DBL structure and an upsampling, connect the upsampling feature with the convolutional feature tensor output by the first Res8 structure, and go through six DBLs The structure and the last convolutional layer obtain the third feature map, and then perform the third prediction on the third feature map, thus obtaining the feature maps on three scales as the image feature detection results of the input image.

The inventors found that: the existing YOLOv3 model mainly extracts features through the RES and DBL structures in Figure 1. A RES operation is mainly composed of 3x3 convolution with a step size of 2 and n ResUnits, and ResUnit is composed of 1x1 convolution and 3x3 convolution operations. A large number of 3x3 convolution operations make the model parameters increase dramatically, and considering the high resolution of the remote sensing image itself, the existing YOLOv3 model cannot take into account the detection speed and accuracy. And when detection targets of different scales appear in the detection task at the same time, it is difficult for the network to make accurate judgments on the size changes of larger targets.

Therefore, there is a need for a multi-scale target detection method for high-resolution images such as remote sensing images that can take into account both detection speed and accuracy.

Fig. 2 shows a flowchart of an image detection method provided by an embodiment of the present invention, and the method is executed by a computer processing device. The computer processing device may include a cell phone, a laptop, and the like. As shown in Figure 2, the method includes the following steps:

Step 10: Input the image to be detected into the target detection model to obtain the target detection result corresponding to the image to be detected; the target detection model includes an improved YOLOv3 model; the improved YOLOv3 model includes a high-dimensional convolution module; The high-dimensional convolution module is used to perform depth convolution on the image to be detected in a high-dimensional space to obtain image features of the image to be detected.

In one embodiment of the present invention, considering that the purpose of convolution is to extract features, the depth of the output feature map=the number of convolution kernels=the number of channels. The cumulative convolution of four 3x3 convolution kernels can reach the perceptual field corresponding to a 9x9 convolution kernel, but requires fewer parameters and produces more features. In the face of a fixed-size picture, if a 9x9 image is used, features are extracted within the 9x9 range. If a picture is just right with a step size of 9, then the number of features extracted is definite data. Therefore, the smaller the convolution, the more and finer the features extracted. As mentioned earlier, the existing YOLOv3 model uses a large number of 3x3 convolutions for feature extraction of pictures, resulting in too many parameters of the model. The model when performing target detection on high-resolution images such as remote sensing images The detection speed drops. Therefore, the embodiment of the present invention replaces the 3x3 convolution in the YOLOv3 model with a high-dimensional convolution module, and performs deep convolution in a high-dimensional space through the high-dimensional convolution module, thereby extracting information-rich feature representations At the same time, the parameter amount of the model can be reduced, and the high-dimensional convolution module can perform identity mapping and spatial transformation in a higher dimension, which reduces the information loss and gradient confusion caused by the traditional target detection network inverted residual structure. risk.

Specifically, the high-dimensional convolution module can be hourglass-shaped, that is, the depth convolution structure is set at the start position and the end position of the high-dimensional convolution module, so as to realize the convolution of the input and output high-dimensional data, and realize The extraction of picture features in high-dimensional space, correspondingly, one or more data dimension adjustment structures can be set between the start position and the end position, so as to adjust the dimension of the data of the depth convolution structure at the input end position to high-dimensional space. Wherein, the deep convolution structure may be a convolution kernel of a preset size, such as a 3x3 convolution kernel.

Therefore, in another embodiment of the present invention, the structure of the high-dimensional convolution module can refer to FIG. 3 .

As shown in Figure 3, the high-dimensional convolution module includes a first depth convolution layer, a dimension adjustment convolution layer, and a second depth convolution layer connected in sequence; the first depth convolution layer and the second depth convolution layer The depth convolution layer is used to perform depth convolution on the input data; the dimension adjustment convolution layer is used to perform dimension adjustment on the input data, so that the data input to the second depth convolution layer is high-dimensional data.

Wherein, the first depth convolution layer may include a convolution kernel of a preset size, such as a 3x3 convolution kernel, that is, Dwise 3x3 (depth separable 3x3 convolution kernel) in FIG. 3 . The structure of the second depthwise convolutional layer may be the same as that of the first depthwise convolutional layer. The dimension adjustment convolutional layer can include two sequentially connected 1x1 convolutional layers, that is, Conv 1x1 in Figure 3. The 1x1 convolutional layer is used to adjust the number of channels, linearly combine the pixels on different channels, and then perform nonlinear Linearization operation, so that the functions of dimensionality enhancement and dimensionality reduction can be completed. Among them, the 1x1 convolutional layer connected to the first deep convolutional layer is used to reduce the dimension of the data, and the 1x1 convolutional layer connected to the second deep convolutional layer is used to expand the data. operate. Optionally, the input of the high-dimensional convolution module can be connected with the activation function in the existing YOLOv3 model.

Further, considering that the YOLOv3 model is used to extract the multi-scale target detection results of the input image, and the existing YOLOv3 model uses the aforementioned existing RES module when performing multi-scale feature extraction, and the RES module includes DBL Module, wherein in the DBL module, multiple 3x3 convolution kernels are used to extract the features of the picture, which has the problems of too many parameters and poor model processing speed. Therefore, in another embodiment of the present invention, the existing The feature extraction in the RES module in the YOLOv3 model is realized by the high-dimensional convolution module in the aforementioned embodiments, thereby performing identity mapping and spatial transformation in a higher dimension, reducing the traditional target detection network inverted residual structure. The risks of information loss and gradient confusion are achieved, and the balance between the processing speed and processing accuracy of the model is achieved.

Therefore, in another embodiment of the present invention, the improved YOLOv3 model includes: a multi-scale convolution module; the multi-scale convolution module is used to convolve the input data under a multi-scale convolution kernel to obtain multiple The image features of the scale; wherein, the convolution kernel includes the high-dimensional convolution module.

Specifically, the multi-scale convolution module may include sequentially connected convolution kernels of multiple scales and depths, wherein the multi-scale convolution module may have a pyramid structure, and the size of the convolution kernel from the top to the bottom of the pyramid structure decreases sequentially. At the same time, in the channel dimension, the number of channels of the convolution kernel from the top to the bottom of the pyramid structure increases sequentially. Each convolution kernel in the multi-scale convolution module finally stitches the obtained feature maps together. When splicing the feature maps output by convolution kernels of different sizes, in order to achieve the same number of output channels, the method of group convolution can be used. Furthermore, in order to improve the processing and detection speed of the network on the basis of obtaining the multi-scale feature map, in the convolution of each layer and group convolution in the multi-scale convolution module, the high dimensional convolution module to extract features.

Therefore, in another embodiment of the present invention, with reference to FIG. 4, the multi-scale convolution module includes multiple layers of convolution kernels connected in sequence; the size of the convolution kernels decreases successively, and the convolution kernels The depth increases sequentially; multiple convolution kernels use group convolution, so that each convolution kernel outputs the image features with the same number of channels.

As shown in Figure 4, the feature map of the input multi-scale convolution module (Input Feature Maps in Figure 4) will pass through convolution kernels of different sizes (Pyramidal Convolution Kernels in Figure 4, including Level1-n PyConv), and then the Each feature map is connected by channel to get output (Input Feature Maps in Figure 4). As the size of the convolution kernel continues to increase, the depth of the convolution kernel continues to decrease. In order to be able to use convolution kernels of different depths, Grouped Convolution is used to obtain feature maps with the same number of channels. Among them, in order to further improve the performance of feature extraction, when performing convolution processing in the pyramid structure, the aforementioned high-dimensional convolution module (Sandglass Block, hourglass block convolution layer) can be used to extract features.

Further, considering that the image accuracy of high-resolution images such as remote sensing images is high, and there are relatively many features that can be extracted, in order to screen a large number of extracted features, enhance the target features that can better characterize the input image, thereby improving the current performance. The accuracy rate of feature extraction in some YOLOv3 models, in another embodiment of the present invention, the improved YOLOv3 model also includes a feature enhancement module: the feature enhancement module is used to process the image features according to the channel attention mechanism Perform feature enhancements.

Specifically, the feature enhancement module is connected after the aforementioned high-dimensional convolution module, and the feature enhancement module performs feature enhancement on the image features extracted by the aforementioned high-dimensional convolution module, suppressing unimportant redundant features, thereby enhancing image detection. target features. Among them, the feature enhancement module can use the channel attention mechanism to perform feature enhancement on the image features extracted by the high-dimensional convolution module. Specifically, the feature enhancement module can first use average pooling to obtain the feature vector of the channel direction, and then capture the dependencies between different channels through the fully connected layer. Finally, the weights corresponding to the obtained dependencies between different channels are mapped, so that the weights are multiplied by the original features to obtain the final enhanced features.

In one embodiment of the present invention, a schematic structural diagram of the feature enhancement module is shown in Figure 5. Referring to Figure 5, the feature enhancement module includes sequentially connected pooling layers (Global Pooling), multiple full connections (FC, FullConection ) layer and a weighted calculation layer; the pooling layer is used to extract the image features of multiple channels; the fully connected layer is used to determine the inter-channel dependency according to the image features of each of the channels; the weighted calculation layer is used for Perform weighted calculation on the image features of the multiple channels according to the inter-channel dependencies to obtain enhanced image features.

Wherein, the number of fully connected layers may be two, and the inter-channel dependence is determined according to the image features of each of the channels through the two connected fully connected layers. The weighted calculation layer can map the weight corresponding to the inter-channel dependence to [0, 1] through the Sigmoid function, and multiply the weight with the original input image feature to obtain the enhanced feature.

In yet another embodiment of the present invention, for the scene of target detection of high-resolution images such as remote sensing images, when applying the YOLOv3 model, in order to further improve the processing performance of the model and the accuracy of multi-scale feature extraction, the aforementioned implementation can be The high-dimensional convolution module, feature enhancement module and multi-scale feature extraction module in the example are integrated into the existing YOLOv3 model to obtain an improved YOLOv3 model.

Specifically, the image to be detected includes a high-resolution image; the improved YOLOv3 model is improved based on the original YOLOv3 model; the original YOLOv3 model includes an original DBL module and an original RES module, and the original DBL module includes sequentially connected The original convolution layer, normalization layer and activation function layer, the original RES module includes a zero-fill layer connected in sequence, the original DBL module and the original residual structure; wherein, in the improved YOLOv3 model, The original convolution layer is replaced by a high-dimensional convolution module, a feature enhancement module is added after the activation function layer, and a multi-scale convolution module is added after the original residual structure; the high-dimensional convolution module is used It is used to perform deep convolution on the input data in a high-dimensional space; the feature enhancement module is used to perform feature enhancement on the image features according to the channel attention mechanism; the multi-scale convolution module is used to perform feature enhancement on the input data in Convolution is performed under a multi-scale convolution kernel; the convolution kernel includes the high-dimensional convolution module.

As shown in Figure 1, in the original YOLOv3 model, the original convolution layer includes the conv layer, the normalization layer includes the BN layer, the activation function layer includes the Leaky Relu layer, and the zero padding layer includes the zero padding layer; the original residual structure includes ResUnit *n.

For the original YOLOv3 model shown in Figure 1, in one embodiment of the present invention, as shown in Figure 6, the first is the improvement of the original DBL module, replacing the conv layer in the DBL module with a high-dimensional convolution module , that is, the sandglass block in the figure. At the same time, the feature enhancement module is added after the activation function layer, that is, the SE (Squeeze and excitation) module in the figure, and the improved DBL module (denoted as NDBL) is obtained.

Furthermore, for the improvement of the original RES module, as shown in Figure 7, first replace the original DBL module in the original RES module with the improved DBL module, and then add a multi-scale convolution module after the original residual structure, That is, the hourglass-pyconv in the figure, the improved RES module (denoted as NRES) is obtained.

After the above-mentioned improvements to the existing YOLOv3 original DBL module and the original RES module, the improved YOLOv3 model is obtained. Its overall network structure is the same as that of YOLOv3, and NDBL and NRES are used to extract features.

The embodiment of the present invention tests the improved YOLOv3 model on the NWPU VHR-10 dataset. The comparison results of the improved YOLOv3 model of the embodiment of the present invention and the detection results of the existing YOLOv3 model can refer to FIG. 8 .

In one embodiment of the present invention, the training process of the improved YOLOv3 model can be as follows:

Step 1: Perform data augmentation on the remote sensing images in the dataset.

Wherein, the process of data enhancement includes: intercepting a series of blocks with a size of 1024×1024 from the remote sensing image in the data set according to the stride 512 .

And, if the number of target categories is small, the image size is adjusted according to the ratio of 1:0.5, and random flip and rotation are used to deal with the problem of unbalanced dataset categories.

Step 2: Label the enhanced data to obtain training samples.

Specifically, it includes determining the prior box through the K-means clustering algorithm, setting up 9 clusters and evenly distributing them on 3 scales, and obtaining the prior box in NWPU VHR-10.

Step 3: Train the improved YOLOv3 model based on the training samples.

Input the processed picture into the improved YOLOv3 model, and after the third and fourth NRES in the improved YOLOv3 model, two branches are independently extracted, and the features are extracted to output the other two features, and finally three target feature maps are obtained.

The image detection device provided by the embodiment of the present invention obtains the target detection result corresponding to the image to be detected by inputting the image to be detected into the target detection model; the target detection model includes an improved YOLOv3 model; the improved YOLOv3 model includes A high-dimensional convolution module; wherein, the high-dimensional convolution module is used to perform deep convolution on the image to be detected in a high-dimensional space to obtain image features of the image to be detected, thereby distinguishing it from the existing YOLOv3 model A large number of 3x3 convolution operations are used for feature extraction, which leads to the problem that the parameter amount of the model is too large and the detection speed is not good. The embodiment of the present invention places the depth convolution at the beginning and end of the entire convolution structure, so that the depth convolution Product operations all act on high-dimensional space, so that feature representations with rich information can be extracted, and identity mapping and space transformation are performed in higher dimensions, which reduces the information loss and gradient confusion caused by the traditional mobile network inverted residual structure. Therefore, the processing speed of the model can be effectively improved, and the balance between detection accuracy and detection speed can be achieved.

FIG. 9 shows a schematic structural diagram of an image detection device provided by an embodiment of the present invention. As shown in FIG. 9 , the device 20 includes: a detection module 201 . Wherein, the detection module 201 is used to input the image to be detected into the target detection model to obtain the target detection result corresponding to the image to be detected; the target detection model includes an improved YOLOv3 model; the improved YOLOv3 model includes a high-dimensional Convolution module; the high-dimensional convolution module is used to perform deep convolution on the image to be detected in a high-dimensional space to obtain image features of the image to be detected.

The operation process of the image detection device provided by the embodiment of the present invention is substantially the same as that of the foregoing method embodiment, and will not be repeated here.

The image detection device provided by the embodiment of the present invention obtains the target detection result corresponding to the image to be detected by inputting the image to be detected into the target detection model; the target detection model includes an improved YOLOv3 model; the improved YOLOv3 model includes A high-dimensional convolution module; wherein, the high-dimensional convolution module is used to perform deep convolution on the image to be detected in a high-dimensional space to obtain image features of the image to be detected, thereby distinguishing it from the existing YOLOv3 model A large number of 3x3 convolution operations are used for feature extraction, which leads to the problem that the parameter amount of the model is too large and the detection speed is not good. The embodiment of the present invention places the depth convolution at the beginning and end of the entire convolution structure, so that the depth convolution Product operations all act on high-dimensional space, so that feature representations with rich information can be extracted, and identity mapping and space transformation are performed in higher dimensions, which reduces the information loss and gradient confusion caused by the traditional mobile network inverted residual structure. Therefore, the processing speed of the model can be effectively improved, and the balance between detection accuracy and detection speed can be achieved.

FIG. 10 shows a schematic structural diagram of an image detection device provided by an embodiment of the present invention. The specific embodiment of the present invention does not limit the specific implementation of the image detection device.

As shown in FIG. 10 , the image detection device may include: a processor (processor) 302 , a communication interface (Communications Interface) 304 , a memory (memory) 306 , and a communication bus 308 .

Wherein: the processor 302 , the communication interface 304 , and the memory 306 communicate with each other through the communication bus 308 . The communication interface 304 is used to communicate with network elements of other devices such as clients or other servers. The processor 302 is configured to execute the program 310, specifically, may execute the above-mentioned relevant steps in the embodiment of the image detection method.

Specifically, the program 310 may include program codes including computer-executable instructions.

The processor 302 may be a central processing unit CPU, or an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. The one or more processors included in the image detection device may be of the same type, such as one or more CPUs, or may be different types of processors, such as one or more CPUs and one or more ASICs.

The memory 306 is used to store the program 310 . The memory 306 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

Specifically, the program 310 can be called by the processor 302 to make the image detection device perform the following operations:

The image to be detected is input into the target detection model to obtain the target detection result corresponding to the image to be detected; the target detection model includes an improved YOLOv3 model; the improved YOLOv3 model includes a high-dimensional convolution module; the high-dimensional The convolution module is used to perform deep convolution on the image to be detected in a high-dimensional space to obtain image features of the image to be detected.

The operation process of the image detection device provided by the embodiment of the present invention is substantially the same as that of the foregoing method embodiment, and will not be repeated here.

The image detection device provided by the embodiment of the present invention obtains the target detection result corresponding to the image to be detected by inputting the image to be detected into the target detection model; the target detection model includes an improved YOLOv3 model; the improved YOLOv3 model includes A high-dimensional convolution module; wherein, the high-dimensional convolution module is used to perform deep convolution on the image to be detected in a high-dimensional space to obtain image features of the image to be detected, thereby distinguishing it from the existing YOLOv3 model A large number of 3x3 convolution operations are used for feature extraction, which leads to the problem that the parameter amount of the model is too large and the detection speed is not good. The embodiment of the present invention places the depth convolution at the beginning and end of the entire convolution structure, so that the depth convolution Product operations all act on high-dimensional space, so that feature representations with rich information can be extracted, and identity mapping and space transformation are performed in higher dimensions, which reduces the information loss and gradient confusion caused by the traditional mobile network inverted residual structure. Therefore, the processing speed of the model can be effectively improved, and the balance between detection accuracy and detection speed can be achieved.

An embodiment of the present invention provides a computer-readable storage medium, the storage medium stores at least one executable instruction, and when the executable instruction is run on the image detection device, the image detection device executes any of the above method embodiments Image detection methods in .

The executable instructions can specifically be used to make the image detection device perform the following operations:

The image to be detected is input into the target detection model to obtain the target detection result corresponding to the image to be detected; the target detection model includes an improved YOLOv3 model; the improved YOLOv3 model includes a high-dimensional convolution module; the high-dimensional The convolution module is used to perform deep convolution on the image to be detected in a high-dimensional space to obtain image features of the image to be detected.

The operation process of the executable instruction stored in the computer-readable storage medium provided by the embodiment of the present invention is substantially the same as that of the foregoing method embodiment, and will not be repeated here.

The executable instruction stored in the computer-readable storage medium provided by the embodiment of the present invention obtains the target detection result corresponding to the image to be detected by inputting the image to be detected into the target detection model; the target detection model includes an improved YOLOv3 model; The improved YOLOv3 model includes a high-dimensional convolution module; wherein, the high-dimensional convolution module is used to perform deep convolution on the image to be detected in a high-dimensional space to obtain image features of the image to be detected, thereby distinguishing In the existing YOLOv3 model, a large number of 3x3 convolution operations are used for feature extraction, which leads to the problem that the parameter amount of the model is too large and the detection speed is not good. The embodiment of the present invention puts the depth convolution at the beginning of the entire convolution structure and the end position, so that the deep convolution operation acts on the high-dimensional space, so that the feature representation with rich information can be extracted, and the identity mapping and space transformation are performed in a higher dimension, which reduces the traditional mobile network inverted residual structure. The risks of information loss and gradient confusion can effectively improve the processing speed of the model and achieve a balance between detection accuracy and detection speed.

An embodiment of the present invention provides an image detection device configured to execute the above image detection method.

An embodiment of the present invention provides a computer program, and the computer program can be invoked by a processor to cause an image detection device to execute the image detection method in any method embodiment above.

An embodiment of the present invention provides a computer program product. The computer program product includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions. The image detection method in the method embodiment.

The algorithms or displays presented herein are not inherently related to any particular computer, virtual system, or other device. Various generic systems can also be used with the teachings based on this. The structure required to construct such a system is apparent from the above description. Furthermore, embodiments of the present invention are not directed to any particular programming language. It should be understood that various programming languages can be used to implement the content of the present invention described herein, and the above description of specific languages is for disclosing the best mode of the present invention.

In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure the understanding of this description.

Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, in order to streamline the present disclosure and to facilitate an understanding of one or more of the various inventive aspects, various features of the embodiments of the invention are sometimes grouped together into a single implementation examples, figures, or descriptions thereof. This method of disclosure, however, is not to be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim.

Those skilled in the art can understand that the modules in the device in the embodiment can be adaptively changed and arranged in one or more devices different from the embodiment. The modules or units or components in the embodiments can be combined into one module or unit or component, and they can be divided into a plurality of sub-modules or sub-units or sub-components. All features disclosed in this specification (including accompanying claims, abstract and drawings) and any method or method so disclosed may be used in any combination, except that at least some of such features and/or processes or units are mutually exclusive. All processes or units of equipment are combined. Each feature disclosed in this specification (including accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means can be embodied by one and the same item of hardware. The use of the words first, second, and third, etc. does not indicate any order. These words can be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the execution order.

Claims (10)

1. An image detection method, the method comprising:

inputting an image to be detected into a target detection model to obtain a target detection result corresponding to the image to be detected; the target detection model includes a modified YOLOv3 model; the improved YOLOv3 model comprises a high-dimensional convolution module; the high-dimensional convolution module is used for carrying out depth convolution on the image to be detected in a high-dimensional space to obtain image characteristics of the image to be detected.

2. The method of claim 1, wherein the high-dimensional convolution module comprises a first depth convolution layer, a dimension adjustment convolution layer, and a second depth convolution layer connected in sequence; the first depth convolution layer and the second depth convolution layer are used for performing depth convolution on input data; the dimension adjustment convolution layer is used for performing dimension adjustment on input data so that the data input into the second depth convolution layer are high-dimensional data.

3. The method of claim 1, wherein the modified YOLOv3 model comprises: a multi-scale convolution module; the multi-scale convolution module is used for convolving the input data under a multi-scale convolution kernel to obtain the multi-scale image characteristics; wherein the convolution kernel comprises the high-dimensional convolution module.

4. A method according to claim 3, wherein the multi-scale convolution module comprises a plurality of layers of sequentially connected convolution kernels; the size of the convolution kernel is sequentially reduced, and the depth of the convolution kernel is sequentially increased; a plurality of the convolution kernels are convolved with a grouping such that each of the convolution kernels outputs the image feature for the same number of channels.

5. The method of claim 1, wherein the improved YOLOv3 model further comprises a feature enhancement module: the feature enhancement module is used for carrying out feature enhancement on the image features according to a channel attention mechanism.

6. The method of claim 5, wherein the feature enhancement module comprises a pooling layer, a plurality of fully connected layers, and a weight calculation layer connected in sequence; the pooling layer is used for extracting image features of a plurality of channels; the full connection layer is used for determining inter-channel dependence according to the image characteristics of each channel; the weighting calculation layer is used for carrying out weighting calculation on the image characteristics of the channels according to the inter-channel dependence to obtain enhanced image characteristics.

7. The method of claim 1, wherein the image to be detected comprises a high resolution image; the improved YOLOv3 model is improved based on the original YOLOv3 model; the original YOLOv3 model comprises an original DBL module and an original RES module, wherein the original DBL module comprises an original convolution layer, a normalization layer and an activation function layer which are sequentially connected, and the original RES module comprises a zero filling layer, the original DBL module and an original residual error structure which are sequentially connected; in the improved YOLOv3 model, the original convolution layer is replaced by a high-dimensional convolution module, a feature enhancement module is added after the function layer is activated, and a multi-scale convolution module is added after the original residual structure; the high-dimensional convolution module is used for carrying out depth convolution on input data in a high-dimensional space; the characteristic enhancement module is used for carrying out characteristic enhancement on the image characteristics according to a channel attention mechanism; the multi-scale convolution module is used for convolving the input data under a multi-scale convolution kernel; the convolution kernel includes the high-dimensional convolution module.

8. An image detection apparatus, the apparatus comprising:

the detection module is used for inputting the image to be detected into the target detection model to obtain a target detection result corresponding to the image to be detected; the target detection model includes a modified YOLOv3 model; the improved YOLOv3 model comprises a high-dimensional convolution module; the high-dimensional convolution module is used for carrying out depth convolution on the image to be detected in a high-dimensional space to obtain image characteristics of the image to be detected.

9. An image detection apparatus, characterized by comprising: the device comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete communication with each other through the communication bus;

the memory is configured to store at least one executable instruction that causes the processor to perform the operations of the image detection method according to any one of claims 1-7.

10. A computer readable storage medium having stored therein at least one executable instruction that, when executed on an image detection device, causes the image detection device to perform the operations of the image detection method of any one of claims 1-7.

Images