WO2020258077A1 - Pedestrian detection method and device - Google Patents

Abstract

The present application is applicable for the technical field of computer applications, and provides a pedestrian detection method and device. The method comprises: obtaining an image to be detected in real time; inputting said image to a pedestrian detection model obtained by pre-training, and identifying pedestrian data comprised in said image; and performing non-maximum suppression processing on the pedestrian data and determining a portrait frame corresponding to the pedestrian data in said image. In this embodiment, the pedestrian detection model is obtained by training on the basis of a depthwise separable convolution mode, the obtained image to be detected is identified according to the pedestrian detection model, and the portrait frame corresponding to the pedestrian comprised in said image is determined, which not only improves the efficiency of portrait detection, so that a user can determine a corresponding processing mode according to the detected portrait frame at the first time, but also improves the accuracy of portrait detection, ensuring that the current pedestrian situation can be clearly detected even in the low-visibility environment such as haze.

Description

Pedestrian detection method and device Technical field

This application belongs to the field of computer application technology, and in particular relates to a pedestrian detection method and device.

Background technique

Walking is one of the basic modes of transportation. According to surveys, in Europe, more than 7,000 pedestrians die each year, accounting for 27% of all deaths. Therefore, effective detection of pedestrians in various environments will Significantly improve the driving safety of autonomous vehicles. However, due to the diversity and complexity of pedestrian postures, positioning, clothing and weather conditions, pedestrian detection problems still exist.

Most of the detection models in the prior art are only tested under sufficient light conditions. Generally speaking, they are not capable of detecting pedestrians under insufficient light conditions, such as foggy days, because the visibility is reduced due to bad weather. Insufficient color reflection and blurring of pedestrian outline and appearance make it difficult to distinguish it from the background. Therefore, in the prior art, when the environment is blurry, it is difficult to distinguish pedestrians from the background when detecting pedestrians, which causes the problem of inaccurate pedestrian detection results.

technical problem

In view of this, the embodiments of the present application provide a pedestrian detection method and device to solve the problem of inaccurate pedestrian detection results in the prior art.

Technical solutions

The first aspect of the embodiments of the present application provides a pedestrian detection method, including:

Obtain the image to be inspected in real time;

Inputting the image to be detected into a pre-trained pedestrian detection model to identify pedestrian data contained in the image to be detected; the pedestrian detection model is trained according to a preset depth separable convolution method;

Non-maximum value suppression processing is performed on the pedestrian data, and a portrait frame corresponding to the pedestrian data in the image to be detected is determined.

A second aspect of the embodiments of the present application provides a pedestrian detection device, including:

The acquiring unit is used to acquire the image to be detected in real time;

A recognition unit, configured to input the image to be detected into a pre-trained pedestrian detection model to identify pedestrian data contained in the image to be detected; the pedestrian detection model is trained according to a preset depth separable convolution method;

The determining unit is configured to perform non-maximum value suppression processing on the pedestrian data, and determine a portrait frame corresponding to the pedestrian data in the image to be detected.

The third aspect of the embodiments of the present application provides a pedestrian detection device, including a processor, an input device, an output device, and a memory. The processor, input device, output device, and memory are connected to each other, wherein the memory is A computer program that executes the above method in a storage support device, the computer program includes program instructions, and the processor is configured to invoke the program instructions to execute the method of the above first aspect.

The fourth aspect of the embodiments of the present application provides a computer-readable storage medium, the computer storage medium stores a computer program, and the computer program includes program instructions that, when executed by a processor, cause the processing The device executes the method of the first aspect described above.

Beneficial effect

Compared with the prior art, the embodiments of the present application have the following beneficial effects: acquiring the image to be detected in real time; inputting the image to be detected into a pre-trained pedestrian detection model, and identifying the pedestrian data contained in the image to be detected; The pedestrian data is subjected to non-maximum value suppression processing to determine the portrait frame corresponding to the pedestrian data in the image to be detected. In this embodiment, the pedestrian detection model is obtained by training based on the depth separable convolution method, the acquired image to be detected is recognized according to the pedestrian detection model, and the portrait frame corresponding to the pedestrian contained therein is determined, which not only improves the efficiency of portrait detection, but also This allows the user to determine the corresponding processing method based on the detected portrait frame in the first time, and also improves the accuracy of portrait detection, ensuring that the current pedestrian situation can be clearly detected in the environment with low visibility such as haze.

Description of the drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only of the present application. For some embodiments, for those of ordinary skill in the art, other drawings can be obtained from these drawings without creative labor.

FIG. 1 is a flowchart of a pedestrian detection method provided in Embodiment 1 of the present application;

FIG. 2 is a flowchart of a pedestrian detection method provided in Embodiment 2 of the present application;

Fig. 3 is an application example of the image enhancement technology provided in the second embodiment of the present application;

4 is a schematic diagram of the training process and application process of the training model provided in Embodiment 2 of the present application;

FIG. 5 is a schematic structural diagram of a pedestrian detection method provided in Embodiment 2 of the present application;

6 is a schematic diagram of comparison between standard convolution and depth separable convolution provided in the second embodiment of the present application;

7 is a schematic diagram of the structure of the bottleneck layer in the pedestrian detection method provided in the second embodiment of the present application;

FIG. 8 is a schematic structural diagram of a weight connection layer provided in Embodiment 2 of the present application;

FIG. 9 is a schematic diagram of the compression incentive mechanism provided in the second embodiment of the present application;

FIG. 10 is a distribution diagram of a priori frames of historical images provided in Embodiment 2 of the present application;

FIG. 11 is a schematic diagram of the MNPB-YOLO label provided in Embodiment 2 of the present application;

FIG. 12 is an example of the detection result provided in Embodiment 2 of the present application;

FIG. 13 is a schematic diagram of a pedestrian detection device provided in Embodiment 3 of the present application;

FIG. 14 is a schematic diagram of a pedestrian detection device provided in Embodiment 4 of the present application.

Specific embodiments of the invention

In the following description, for the purpose of illustration rather than limitation, specific details such as a specific system structure and technology are proposed for a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from obstructing the description of this application.

In order to illustrate the technical solutions described in the present application, specific embodiments are used for description below.

Example 1:

Refer to FIG. 1, which is a flowchart of a pedestrian detection method according to Embodiment 1 of the present application. The execution subject of the pedestrian detection method in this embodiment is a device with a pedestrian detection function, including but not limited to devices such as computers, servers, tablets, or terminals. The pedestrian detection method shown in the figure may include the following steps:

S101: Acquire images to be detected in real time.

Walking is one of the basic modes of transportation. According to surveys, in Europe, more than 7,000 pedestrians die each year, accounting for 27% of all deaths. Therefore, effective detection of pedestrians in various environments will Significantly improve the driving safety of autonomous vehicles. However, due to the diversity and complexity of pedestrian postures, positioning, clothing and weather conditions, pedestrian detection problems still exist. With the continuous improvement of computer performance, detection methods based on deep architecture have also been widely used. However, most detection models are only tested under sufficient light conditions. Generally speaking, they are not capable of detecting pedestrians under insufficient light conditions, such as foggy days. According to the survey, the incidence of traffic accidents in haze weather is much higher than in sunny conditions, because in haze weather, the field of vision that the human eye can feel becomes smaller, making it difficult for drivers to see road signs and pedestrians, which increases The probability of a traffic accident. Generally speaking, the more severe the haze, the higher the accident rate. However, detecting pedestrians in fog and haze is a particularly challenging task, because bad weather reduces visibility, insufficient color reflection, and blurs the outline and appearance of pedestrians, making it difficult to distinguish them from the background. These algorithms are mainly used in daytime scenes with uniform atmospheric light. However, more severe haze conditions usually occur in dim light, which makes the existing defogging algorithms unable to adapt to more severe haze scenes.

In this embodiment, in the process of detecting pedestrians, the image to be detected is first acquired in real time. The image to be detected in this embodiment can be in the form of a single image or a piece of video. After the video is obtained, the video can be sampled within a preset period to obtain a single frame of image as the Check the image. At the same time, the image to be detected in this embodiment may be a color image, a black and white image, an infrared image, etc., which is not limited here.

Exemplarily, during the traveling of the vehicle, an image or video can be captured in real time through a camera device installed at the front end of the vehicle, such as a driving recorder, to use the image as the image to be detected, and the video is sampled to obtain a single frame The image screen as the image to be detected.

S102: Input the image to be detected into a pre-trained pedestrian detection model, and identify pedestrian data included in the image to be detected; the pedestrian detection model is trained according to a preset depth separable convolution method.

In this embodiment, a pedestrian detection model is pre-trained, and this embodiment uses deep separable convolution and linear bottleneck layer technology to reduce the amount of calculation and the number of parameters, and improve the operating efficiency of the network. In addition, we innovatively combine multi-scale feature fusion with compression incentive mechanism, and propose a new feature fusion method—weight connection layer. Using the above method, we propose an efficient pedestrian detection method MNPB-YOLO pedestrian detection model in haze weather.

After the pedestrian detection model is trained, the acquired image to be detected is input into the pedestrian detection model, and the pedestrian data contained in the image to be detected is identified. The pedestrian data in this embodiment may include the position of the pedestrian in the image to be processed, the position of the corresponding pixel, and the pedestrian detection model to find a bunch of boxes corresponding to the pedestrian, etc., which are not limited here.

S103: Perform non-maximum suppression processing on the pedestrian data, and determine a portrait frame corresponding to the pedestrian data in the image to be detected.

After the pedestrian data in the image to be processed is obtained through the pedestrian detection model, non-maximum suppression processing is performed on the pedestrian data to determine the portrait frame corresponding to the pedestrian data in the image to be detected. In this embodiment, non-maximum value suppression is used to suppress elements that are not maximum value. For example, the edge of the corresponding area in the image is determined by means of local maximum search. This part is used to represent a neighborhood, and the neighborhood has two variable parameters, one is the dimension of the neighborhood, and the other is the size of the neighborhood. In the pedestrian detection in this embodiment, the feature is extracted through the sliding window, and after classification and recognition by the classifier, each window will get a score. However, sliding windows will cause many windows to contain or mostly overlap with other windows. At this time, it is necessary to use non-maximum value suppression processing to select those neighborhoods with the highest scores, that is, the probability of pedestrians is the largest, and the probability of not being pedestrians is small, and the windows with low scores are suppressed.

Specifically, for the identified bounding boxes corresponding to each pedestrian data in the image to be detected, we need to determine which rectangular boxes are useless. In this embodiment, it is assumed that the set of bounding boxes corresponding to each pedestrian is B, and the detection frame M with the largest bounding box is selected, which is removed from the set B and added to the final detection result, and each remaining set in B is calculated Intersection over Union (IOU) between the bounding box and the detection frame M, remove the bounding box with IOU greater than or equal to the preset overlap threshold from B, repeat this process until B is empty, and finally The retained bounding box is used as the portrait bounding box corresponding to the pedestrian data in the image to be detected.

Exemplarily, when a pedestrian is located in the image to be detected, a bunch of boxes are found through the pedestrian detection model, and we need to determine which rectangular boxes are useless. The method of suppressing non-maximum values in this embodiment is as follows: first assume that there are a preset number of rectangular boxes, that is, bounding boxes, and sort according to the classification probability of the classifier. Assuming that the number of rectangular boxes is 6, the probabilities of pedestrians from small to large are A, B, C, D, E, and F respectively. Starting from the maximum probability rectangle F, judge whether the IOUs of A to E and F are greater than a set overlap threshold. If the overlap of B, D and F exceeds the overlap threshold, then discard B and D; And mark the first rectangular frame F, which we kept. From the remaining rectangular boxes A, C, and E, select the E with the highest probability, and then judge the overlap between E and A, C. If the overlap is greater than a certain overlap threshold, then throw it away; and mark E as we keep The second rectangular frame that comes down is repeated in this way, finding all the remaining rectangular frames as the portrait frame corresponding to the pedestrian data in the image to be detected.

In the above solution, the image to be detected is acquired in real time; the image to be detected is input into a pre-trained pedestrian detection model to identify the pedestrian data contained in the image to be detected; the pedestrian detection model can be separated according to a preset depth Obtained by training in a product manner; non-maximum suppression processing is performed on the pedestrian data, and a portrait frame corresponding to the pedestrian data in the image to be detected is determined. In this embodiment, the pedestrian detection model is obtained by training based on the depth separable convolution method, the acquired image to be detected is recognized according to the pedestrian detection model, and the portrait frame corresponding to the pedestrian contained therein is determined, which not only improves the efficiency of portrait detection, but also This allows the user to determine the corresponding processing method based on the detected portrait frame in the first time, and also improves the accuracy of portrait detection, ensuring that the current pedestrian situation can be clearly detected in the environment with low visibility such as haze.

Example 2:

Refer to FIG. 2, which is a flowchart of a pedestrian detection method according to Embodiment 2 of the present application. The execution subject of the pedestrian detection method in this embodiment is a device with a pedestrian detection function, including but not limited to devices such as computers, servers, tablets, or terminals. The pedestrian detection method shown in the figure may include the following steps:

S201: Acquire an image to be detected in real time.

In this embodiment, in the process of detecting pedestrians, the image to be detected is first acquired in real time. The image to be detected in this embodiment can be in the form of a single image or a piece of video. After the video is obtained, the video can be sampled within a preset period to obtain a single frame of image as the Check the image. At the same time, the image to be detected in this embodiment may be a color image, a black and white image, an infrared image, etc., which is not limited here. Exemplarily, during the traveling of the vehicle, an image or video can be captured in real time through a camera device installed at the front end of the vehicle, such as a driving recorder, to use the image as the image to be detected, and the video is sampled to obtain a single frame The image screen as the image to be detected.

S202: Acquire historical images containing pedestrians.

In this embodiment, when the image to be detected is recognized, the recognition is performed based on a pre-trained pedestrian detection model. Therefore, before recognizing the acquired image to be detected, a historical image containing a pedestrian is acquired first, so as to train a pedestrian detection model based on the historical image.

The form of the historical image in this embodiment is the same as the form of the image to be detected. The historical image in this embodiment may be a color image, a black and white image, an infrared image, etc., which are not limited here. The historical image in this embodiment can be in the form of a single image or a piece of video. After the video is obtained, the video can be sampled within a preset period to obtain a single frame of image as the historical image .

S203: Construct a training model according to a preset weight connection layer, and train the training model according to the historical image to obtain the pedestrian detection model.

After acquiring the historical image, a training model is constructed according to the preset weight connection layer, and the training model is trained according to the historical image, and a pedestrian detection model is obtained to detect the pedestrian in the image to be detected. It should be noted that, in order to distinguish the training models corresponding to different processing methods, the training model in this embodiment includes a first training model and a second training model, where the first training model is used to represent the result obtained after the historical image is expanded Training model, the second training model is used to represent S2031～S2034 trained through steps S2031～S2034. The two training models can realize training and image recognition separately, or they can be combined for training and image recognition.

Further, step S203 may specifically include:

Performing image enhancement processing on the historical image to obtain at least two expanded images corresponding to the historical image;

A first training model is constructed according to a preset weight connection layer, and the first training model is trained according to the historical image and the corresponding extended image to obtain the pedestrian detection model.

Specifically, please refer to Figure 3 together. Figure 3 shows some application examples of image enhancement technology, where (a) ~ (f) are the original image of the historical image, the image after random flip, and the random contrast change. The image after, the image after random cropping, the image after random color change, and the image after random affine transformation. In order to make the model have better generalization performance, we added image enhancement technology during training, through random cropping, flipping, color change, affine transformation, Gaussian noise and other operations to expand the original data set to ensure With a strong data set foundation, increase the number of model training and improve the accuracy of model training. The first training model is constructed according to the preset weight connection layer, and the first training model is trained according to the historical image and its corresponding expanded image to obtain a pedestrian detection model.

Please also participate in Figure 4, which is a schematic diagram of the training process and application process of the training model provided in this embodiment. In the training, the historical image in the form of Red Green Blue (RGB) is first obtained, namely RGB image, and then perform image expansion on the RGB image, for example, image cropping, image enhancement, etc., and then through the pre-involved algorithm architecture, namely the MNPB-YOLO pedestrian detection method to recognize the historical image and the expanded image, and Calculate the loss function between the recognition result and the original image according to the preset truth label determination method, and finally use the loss function to update the model parameters in the MNPB-YOLO pedestrian detection method, and finally obtain the fixed parameters and weights in the training model , Get an efficient and accurate training model.

Further, step S203 may specifically include steps S2031 to S2034:

S2031: Construct a second training model based on a preset weight connection layer, a preset depth separable convolution method and a preset linear bottleneck layer technology.

Please also participate in FIG. 5. FIG. 5 is a schematic diagram of the structure of the pedestrian detection method provided by this embodiment, where the multiplication calculation formula between the numbers in FIG. 5 is used to indicate the amount of data currently participating in the calculation. In order to effectively realize pedestrian detection in haze weather, this embodiment proposes a new YOLO-based deep learning method, which includes a basic convolution model part, a weight connection layer, a detection module, and a classification module. Using deep separable convolution and linear bottleneck layer technology to form a basic convolution model, the amount of calculation and the number of parameters are reduced, and the operating efficiency of the network is improved. In addition, we also combine the multi-scale feature fusion of space-to-depth transformation with the compression incentive mechanism, and propose a new feature fusion method-weight connection layer. Finally, the detection module and the classification module are used to complete the middle of the image to be processed. Pedestrian detection.

Using the above method, we propose an efficient pedestrian detection method under haze weather-MNPB-YOLO. Please also participate in Figure 6. Figure 6 shows the standard convolution and depth separable convolution provided by this embodiment. The comparison diagram of, where H, W are used to represent the height and width of the convolution kernel, M is used to represent the number of channels of the input feature map or the number of channels of the convolution kernel, and N is used to represent the number of convolution kernels. The difference between the effect of deep separable convolution and ordinary convolution is that it can effectively reduce the amount of network parameters and calculations. In order to allow MNPB-YOLO to run on a general processor at a higher speed, we use deep separable convolution Used to build the overall MNPB-YOLO model. The comparison between its convolution method and ordinary convolution is shown in Figure 6. It can be seen from Figure 6 that the depthwise separable convolution consists of two parts, vertical depthwise convolution and pointwise convolution, depthwise convolution in different channels The feature map is convolved above, and then the pointwise convolution is convolved on all channels of the feature map.

Please also participate in FIG. 7, which is a schematic diagram of the structure of the bottleneck layer in the pedestrian detection method provided by this embodiment. Depth separable convolution and Relu activation function will cause a certain amount of information loss. In order to reduce this information loss, a bottleneck layer technology is needed. Specifically, the implementation process of the bottleneck layer technology in this solution is shown in Table 1. The image first undergoes a 3×3 convolution to output a feature map, which is used as the Input in the bottleneck layer structure, as shown in Figure 6. According to the convolution step size of 1 or 2, different methods are used to perform convolution. When the convolution step size is 1, the 1×1 convolution is performed first, and the activation function is Relu6 to increase the dimension, that is, Conv in Figure 7 1×1, Relu6; the multiple of the ascending dimension is reflected in the "expansion coefficient" column in Table 1. Through this step, the information can be spread more widely in the feature map to prevent depthwise convolution (when the activation function is Relu6) Information loss caused by); use 3×3 Depthwise convolution to reduce the dimension of the feature map after dimensionality, that is, Dwise3×3, Relu6 in Figure 7; then through 1×1 ordinary convolution, the activation function is a linear function To fuse the information of different channels, that is, Conv 1×1, Linear in Figure 7; then the input and this part of the output are added element-wise, that is, ADD in Figure 7, and finally this feature map is output to the next Layer, as the input of the next layer. When the convolution stride is 2, that is, Stride=2 in Figure 7, the difference from convolution stride 1 is that there is no element-wise addition operation. This is because when the convolution stride is 2, the main purpose is It is to down-sample the height and width of the feature map, and the height and width are reduced by one time. The reduced feature map cannot be added element-wise with the original feature map.

The detailed network structure parameters in the pedestrian detection method are shown in Table 1. It should be noted that Conv in the text and drawings in this embodiment is used to represent convolution; the characters in the format of "number×number×number" are used It represents the height×width×the number of channels of the feature map, which will not be repeated hereafter.

Table 1: MNPB-YOLO network configuration parameters

Weighted connection layer

Please also participate in FIG. 8, which is a schematic diagram of the structure of the weight connection layer provided by this embodiment. The weight connection layer can automatically filter the importance of feature maps from different feature scales, and then filter out those unimportant information to improve the performance of the network. First, collect information from multiple feature maps with different layers and different scales. These feature maps are inconsistent in size, that is, the feature maps are different in height × width × number of channels, such as 28x28x16, 14x14x48 or 7x7x320 in the figure, which cannot be directly Splicing, so the method of segmentation and splicing is adopted. The feature maps of different sizes are adjusted to the same size, that is, the space-to-depth transformation, and then all the adjusted feature maps are spliced together to form multi-scale feature information, such as image In the 7x7x256+7x7x192+7x7x320, the important features are finally screened out by the compressed incentive mechanism model.

X用于表示输入的特征图。在本实施例中，X用于表示多尺度特征信息，

为筛选后的信息(请一并参阅图8)。 Please also participate in Figure 9. Figure 9 is a schematic diagram of the compression incentive mechanism provided by this embodiment. The compression incentive mechanism is actually a channel attention mechanism. The important ones are selected by assigning different weights ω to each channel. Characteristic, the weight ω is obtained by learning, and the direction of its update is the direction of loss reduction. As shown in the figure, the dimensionality reduction process is performed through 1×1 convolution first, and then the feature value of the channel is obtained through global pooling, and then the weight ω is calculated through the fully connected layer, and then the weight ω is multiplied by the compressed feature map. Finally, the re-calibrated features are obtained. As shown in Figure 9, H, W, and C are used to represent the height, width, and number of channels of a feature map, and the superscript'has the same meaning; F _{tr is} used to represent the drop of a 1×1 convolution The purpose of dimensional operation is to reduce the number of channels by way of dimensionality reduction, so as to reduce the amount of calculation required in the following steps, and to improve the efficiency of the method in this embodiment to detect pedestrians; F _sq (·) is used to represent a The compression operation is replaced by global average pooling in practice; F _ex (·,W) represents the feature information of the shape of 1×1×C being mapped to another feature information of the shape of 1×1×C, that is, after mapping The feature information of is used to represent the importance coefficient of the channel, and the mapping scheme uses a multi-layer perceptron; W is used to represent the weight of the multi-layer perceptron, and the update direction of this weight is the direction of the loss gradient; F _scala (·,·) It is used to express the feature map U and the mapped feature information, that is, the channel importance coefficient, and do the multiplication on the channel; finally, the re-calibrated feature is obtained

X is used to represent the input feature map. In this embodiment, X is used to represent multi-scale feature information,

This is the filtered information (please refer to Figure 8 together).

S2032: Input the historical image into the second training model, detect pedestrian images in the historical image, and determine the truth value label corresponding to each pedestrian image; the truth value label is used to indicate the recognized history The a priori box of the image is based on the transform coefficient of the truth box.

The MNPB-YOLO detection idea is inspired by YOLO. It divides the image into N×N grids, each grid predicts B detection frames, and each detection frame is composed of the parameters describing the location of the detection frame and the code of the object type. In YOLOv1 (the first version) and YOLOv2 (in the second version), which grid the center of the object is located in, the detection frame in which grid is used for prediction, but in MNPB-YOLO, we first calculate the grid The IOU values of the prior box and the truth box in the grid are then sorted in descending order of the IOU value, and the first k prior boxes are selected to predict the size of the object in a complex manner.

Please also participate in FIG. 10, which is a distribution diagram of a priori frames of historical images provided in this embodiment. We use the clustering method to obtain the cluster centers of the pedestrian size in the data set in advance. We divide the pedestrian size into two categories, use the parameters of the cluster centers as the height and width of the prior frame, and then evenly distribute them on the N×N net In the grid, the thick black boxes are two a priori boxes located at the center of the grid. The black grid lines are used to represent the N×N grid, where N is taken as 7.

其中，w _a，h _a用于表示先验框的宽和高，x _a，y _a用于表示先验框距离图像左边缘和上边缘的距离。w _g，h _g用于表示真值框的宽和高，x _g，y _g用于表示真值框距离图像左边缘和上边缘的距离。否则，则其标签为

其中a、b、c、d用于表示此处的值随意设置。 Please also participate in Figure 11. Figure 11 is a schematic diagram of the MNPB-YOLO label provided in this embodiment. In MNPB-YOLO, each target person corresponds to a truth label, which is described by 6 parameters, which are used to represent the prior The offset of the box in the x-direction and the y-direction from the box to the truth box, the transformation coefficients of the height and width of the prior box to the truth box, and the feature vector used to distinguish between people and background, expressed in one-hot encoding . In the left image of Figure 11, the box represented by the thin line is a priori box, and the box represented by the thick line is the truth box. Assuming that the box represented by the thin line is responsible for predicting the truth box, then its label is

Wherein, w _a, h _a priori to represent the width and height of the block, x _a, y _a prior frame for indicating the distance from the image of the left edge and upper edge. w _g , h _{g are} used to represent the width and height of the truth value box, and x _g , y _{g are} used to represent the distance of the truth value box from the left and upper edges of the image. Otherwise, its label is

Among them, a, b, c, d are used to indicate that the value here is set at will.

S2033: Determine the loss function corresponding to the second training model according to the truth label.

For the detection task (regression of the detection frame), we use smooth L1 to evaluate the loss of the network output and the truth label. For the classification task (to distinguish pedestrians from the background), we use cross entropy as the loss, while YOLOV1 and V2 both use L2 as the loss Loss of detection and classification.

The YOLO loss function is as follows:

L _total ＝L _center +L _size +L _socre +L _class

分别用于表示网络预测的网格I,j处检测框k内有无物体的置信度及其真值，x _i,j,k,y _i,j,k,w _i,j,k,h _i,j,k用于表示网络预测的网格I,j处的检测框k的位置及大小，

用于表示与之对应的真值，p _i,j,k和

分别用于表示网络预测的网格I,j处的检测框k内物体所属类别的概率及其真值，

用来指示网格I,j处的检测框k是否用来负责检测物体，

用来指示网格I,j处的检测框k是否用来负责检测背景。 Among them, L _center , L _size, and L _{score are} used to indicate the confidence loss of the center, size, and whether there is an object in the box, and L _{class is} used to indicate the classification loss, c _i,j,k ,

Respectively used to represent the confidence and true value of whether there is an object in the detection frame k at the grid I and j of the network prediction, x _i,j,k ,y _i,j,k ,wi _,j,k ,h _{i,j,k are} used to represent the position and size of the detection frame k at the grid I,j predicted by the network,

Used to represent the corresponding truth value, p _{i, j, k} and

They are used to represent the probability and its true value of the category of the object in the detection frame k at the grid I, j predicted by the network,

Used to indicate whether the detection frame k at grid I, j is used to detect objects,

It is used to indicate whether the detection frame k at grid I, j is used to detect the background.

In this embodiment, the MNPB-YOLO loss is as follows:

L _total =λ _class L _class +λ _box L _box

用于表示物体种类的真值，

用于表示网络预测的先验框向真值框变换的值，

用于表示先验框向真值框变换的真值。λ _class和λ _box是两个常数，用以平衡不同类型的损失，CrossEntropy用于表示交叉熵函数；

用于表示网格I,j处的检测框k是否用来负责检测物体；x用于表示函数的输入。 Among them, class _{i, j, k are} used to represent the one-hot encoding of the object type predicted by the network,

Used to represent the true value of the object type,

The value used to represent the transformation of the prior box of network prediction to the truth box,

It is used to represent the true value of the transformation from the prior box to the truth box. λ _class and λ _box are two constants used to balance different types of losses, and CrossEntropy is used to represent the cross entropy function;

It is used to indicate whether the detection box k at grid I, j is used to detect objects; x is used to indicate the input of the function.

S2034: Optimize the second training model according to the loss function to obtain the pedestrian detection model.

The features filtered by the weight connection layer need to be further extracted, and finally the specific tensor shape can be output, which is compared with the truth label to constitute a loss. We use 3×3 convolution and 1×1 convolution to form our detection and classification module.

As shown in Table 2, the meaning of the final output shape of the tensor is: divide the image into 7×7 grids, each grid predicts 2 detection frames, and each detection frame requires 6 parameter descriptions, where Including 4 parameters describing the position and size of the detection frame and 2 one-hot codes for distinguishing background and characters.

From the technical comparison of different methods, it can be seen that the method parameter amount of this embodiment is much less than that of the original YOLO, which means that the method of this embodiment can run more efficiently. Compared with YOLOv1 and YOLOv2, MNPB-YOLO has made major changes in the network structure, using deep separable convolution and bottleneck layer technology to greatly reduce the amount of parameters and calculations, and the proposed weight connection layer can improve network performance , Which gives our network the advantages of speed and accuracy at the same time.

Table 2: Classification and detection modules

The technical comparison of different methods is shown in Table 3. Among them, the network structure of YOLOv1 is inspired by GoogleLenet, the input size is 224×224×3 image, the output grid is divided into 7×7, and each grid predicts 2 detection frames , The detection idea is to use the mapping ability of the convolutional neural network to directly construct the mapping process from the image to the parameters of the detection frame. The detection frame is composed of four parameters: the object center position (x, y) and the object height and width (h, w). In addition to the confidence level c of whether there is an object in a certain detection frame in the grid and the one-hot encoding of the object type together constitute the output of the network, the entire network is equivalent to learning how to label the object from scratch. In fact, for some objects in specific scenes, we have some prior knowledge that can help the network to better identify the objects, such as the aspect ratio information of the target object, which is generally around 3:1 for pedestrians. YOLOv2 incorporates this prior knowledge into network prediction. This is what we call a priori box technology. For YOLOv2, the input size is adjusted to 448×448×3, and the grid is divided into 13×13. Two detection frames are predicted. The size of the prior frame is obtained from the label size in the training set through the clustering method. In addition, the network structure has been greatly changed, and the darknet backbone is proposed, and the parameter amount is about half of the original. The comparison of YOLOv1 and YOLOv2 can be seen in Table 3. From Table 3, it can be seen that the method of MNPB-YOLO in this embodiment includes batch standardization, a priori box technology, convolution method, weight connection layer and parameter amount. obvious advantage:

Table 3: Technical comparison of different methods

S204: Input the image to be detected into a pre-trained pedestrian detection model, and identify pedestrian data contained in the image to be detected; the pedestrian detection model is trained according to a preset depth separable convolution method;

The implementation of S204 in this embodiment is exactly the same as that of S102 in the embodiment corresponding to FIG. 1. For details, reference may be made to the related description of S102 in the embodiment corresponding to FIG. 1, which will not be repeated here.

S205: Perform non-maximum value suppression processing on the pedestrian data, and determine a portrait frame corresponding to the pedestrian data in the image to be detected.

Please refer to the previously introduced Figure 4 together. Figure 4 is a schematic diagram of the training process and application process of the training model provided in this embodiment. After the parameters and weights in the training model are determined according to the historical images and their expanded images, the training can be The model is put into application, that is, pedestrian detection is performed on the image to be detected by training the model. In the detection process, the real-time image, that is, the image to be detected, is acquired through the vehicle-mounted camera. The acquired real-time image is input into the trained training model, and at least one frame corresponding to each pedestrian in the real-time image is detected. The large-value suppression method calculates and judges each frame, determines the pedestrian frame corresponding to the pedestrian in the image to be detected, and obtains an accurate detection result.

In the simulation experiment, the original YOLO is a multi-target detection model. In order to compare the single-target detection of YOLO and MNPB-YOLO, we have made some changes. The modified model is called S-YOLOV1 and S-YOLOV2, in addition, we also compared two traditional methods: pedestrian detection algorithm based on HAAR feature and Adaboost classifier, and pedestrian detection algorithm based on HOG feature and SVM classifier. The test results of different methods are shown in Table 4. Among them, AP is used to indicate the average accuracy, specifically the ratio of the area enclosed by the PR curve and the XY axis, P is used to indicate the accuracy, R is used to indicate the recall rate, and FPS is used to indicate the processing speed, that is, how many frames are processed per second . It can be seen from Table 4 that the method of this embodiment is not only more accurate than other methods, but also has a much faster speed. The experiment is measured on the Intel-i7 6700K and GTX 1080 computer platforms.

Table 4: Comparison of test results of different methods

Please refer to FIG. 12 together. FIG. 12 is an example of some detection results in this embodiment. It can be seen from FIG. 12 that the method in this embodiment can detect in the image in the case of haze weather or low environmental visibility. Pedestrians, and can accurately determine the pedestrian frame.

S206: Detect the orientation of the pedestrian corresponding to the portrait frame relative to the current vehicle, and the distance between the current vehicle and the pedestrian corresponding to the portrait frame.

After determining the acquired portrait frame in the image to be detected, the pedestrian in the current field of view can be determined according to the portrait frame, and corresponding control operations can be performed.

Exemplarily, when the portrait frame corresponding to the pedestrian in front of the vehicle is detected, that is, it is determined that there is someone in front of the vehicle, the position of the pedestrian relative to the vehicle and the current distance between the vehicle and the pedestrian can be detected again. Specifically, the position can be determined according to the area of the pedestrian in the image to be processed. For example, according to the lower right of the portrait frame in the image to be processed, it is determined that the pedestrian is in front of the right of the current vehicle. The distance can be determined by infrared ranging, and the distance between the pedestrian's position and the current vehicle can be determined by an infrared ranging device.

S207: Generate reminder information according to the direction and the distance, and broadcast it; the reminder information is used to remind the driver of the current vehicle to pay attention to the pedestrian corresponding to the portrait frame.

After determining the position and distance of the pedestrian relative to the current vehicle, a reminder message is generated and broadcasted to remind the driver of the current vehicle to pay attention to the pedestrian corresponding to the portrait frame.

Further, if the method in this embodiment is used in an unmanned environment, after determining the position and distance of the pedestrian relative to the current vehicle, a vehicle control instruction can be generated according to the position and distance of the pedestrian relative to the current vehicle. , To control the speed and direction of the vehicle through vehicle control commands to avoid crashing into pedestrians.

In the above solution, the image to be detected is acquired in real time; the historical image containing the pedestrian is acquired; the training model is constructed according to the preset weight connection layer, and the training model is trained according to the historical image to obtain the pedestrian detection model. Input the image to be detected into a pre-trained pedestrian detection model to identify the pedestrian data contained in the image to be detected; the pedestrian detection model is trained according to a preset depth separable convolution method; for the pedestrian data Performing non-maximum value suppression processing to determine the portrait frame corresponding to the pedestrian data in the image to be detected. Increase the base of image training by processing historical images, construct a training model based on the weighted connection layer and perform training to obtain the pedestrian detection model, detect the image to be detected, and perform corresponding processing based on the detection result after the detection result is obtained , Not only improves the detection efficiency and detection accuracy of the image to be detected, but also improves the safety of vehicle driving and passersby.

Example 3:

Refer to FIG. 13, which is a schematic diagram of a pedestrian detection device provided in Embodiment 3 of the present application. The pedestrian detection device 1300 may be a mobile terminal such as a smart phone or a tablet computer. Each unit included in the pedestrian detection device 1300 of this embodiment is used to execute the steps in the embodiment corresponding to FIG. 1. For details, please refer to the related descriptions in the embodiment corresponding to FIG. 1 and FIG. 1, which will not be repeated here. The pedestrian detection device 1300 of this embodiment includes:

The acquiring unit 1301 is configured to acquire the image to be detected in real time;

The recognition unit 1302 is configured to input the image to be detected into a pre-trained pedestrian detection model to identify pedestrian data contained in the image to be detected; the pedestrian detection model is trained according to a preset depth separable convolution method ；

The determining unit 1303 is configured to perform non-maximum value suppression processing on the pedestrian data, and determine a portrait frame corresponding to the pedestrian data in the image to be detected.

Further, the pedestrian detection device further includes:

The historical acquisition unit is used to acquire historical images containing pedestrians;

The training unit is configured to construct a training model according to a preset weight connection layer, and train the training model according to the historical image to obtain the pedestrian detection model.

Further, the training unit includes:

An expansion unit, configured to perform image enhancement processing on the historical image to obtain at least two expanded images corresponding to the historical image;

The first training unit is configured to construct a first training model according to a preset weight connection layer, and train the first training model according to the historical image and the corresponding extended image to obtain the pedestrian detection model.

Further, the training unit includes:

The second training unit is used for constructing a second training model based on the preset weight connection layer, the preset depth separable convolution method and the preset linear bottleneck layer technology;

The truth value unit is used to input the historical image into the second training model, detect pedestrian images in the historical image, and determine the truth value label corresponding to each pedestrian image; the truth value label is used to indicate The a priori box of the recognized historical image is based on the transform coefficient of the truth box;

A loss function unit, configured to determine a loss function corresponding to the second training model according to the truth label;

The optimization unit is configured to optimize the second training model according to the loss function to obtain the pedestrian detection model.

Further, the pedestrian detection device further includes:

A positioning unit for detecting the orientation of the pedestrian corresponding to the portrait frame relative to the current vehicle, and the distance between the current vehicle and the pedestrian corresponding to the portrait frame;

The reminding unit is used to generate and broadcast reminding information according to the orientation and the distance; the reminding information is used to remind the driver of the current vehicle to pay attention to the pedestrian corresponding to the portrait frame.

In the above solution, the image to be detected is acquired in real time; the historical image containing the pedestrian is acquired; the training model is constructed according to the preset weight connection layer, and the training model is trained according to the historical image to obtain the pedestrian detection model. Input the image to be detected into a pre-trained pedestrian detection model to identify the pedestrian data contained in the image to be detected; the pedestrian detection model is trained according to a preset depth separable convolution method; for the pedestrian data Performing non-maximum value suppression processing to determine the portrait frame corresponding to the pedestrian data in the image to be detected. Increase the base of image training by processing historical images, construct a training model based on the weighted connection layer and perform training to obtain the pedestrian detection model, detect the image to be detected, and perform corresponding processing based on the detection result after the detection result is obtained , Not only improves the detection efficiency and detection accuracy of the image to be detected, but also improves the safety of vehicle driving and passersby.

It should be understood that the size of the sequence number of each step in the foregoing embodiment does not mean the order of execution. The execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiment of the present application.

Example 4:

Refer to FIG. 14, which is a schematic diagram of a pedestrian detection device provided in Embodiment 5 of the present application. The pedestrian detection device 1400 in this embodiment as shown in FIG. 14 may include a processor 1401, a memory 1402, and a computer program 1403 that is stored in the memory 1402 and can run on the processor 1401. When the processor 1401 executes the computer program 1403, the steps in the foregoing embodiments of the pedestrian detection method are implemented. The memory 1402 is used to store a computer program, and the computer program includes program instructions. The processor 1401 is configured to execute program instructions stored in the memory 1402. Wherein, the processor 1401 is configured to call the program instructions to perform the following operations:

The processor 1401 is used to:

Obtain the image to be inspected in real time;

Inputting the image to be detected into a pre-trained pedestrian detection model to identify pedestrian data contained in the image to be detected; the pedestrian detection model is trained according to a preset depth separable convolution method;

Non-maximum value suppression processing is performed on the pedestrian data, and a portrait frame corresponding to the pedestrian data in the image to be detected is determined.

Further, the processor 1401 is specifically configured to:

Obtain historical images containing pedestrians;

A training model is constructed according to a preset weight connection layer, and the training model is trained according to the historical images to obtain the pedestrian detection model.

Further, the processor 1401 is specifically configured to:

Performing image enhancement processing on the historical image to obtain at least two expanded images corresponding to the historical image;

A first training model is constructed according to a preset weight connection layer, and the first training model is trained according to the historical image and the corresponding extended image to obtain the pedestrian detection model.

Further, the processor 1401 is specifically configured to:

Construct a second training model based on the preset weight connection layer, based on the preset depth separable convolution method and the preset linear bottleneck layer technology;

Input the historical image into the second training model, detect pedestrian images in the historical image, and determine the truth value label corresponding to each pedestrian image; the truth value label is used to indicate the identity of the recognized historical image The a priori box is based on the transform coefficient of the truth box;

Determine the loss function corresponding to the second training model according to the truth label;

The second training model is optimized according to the loss function to obtain the pedestrian detection model.

Further, the processor 1401 is specifically configured to:

Detecting the orientation of the pedestrian corresponding to the portrait frame relative to the current vehicle, and the distance between the current vehicle and the pedestrian corresponding to the portrait frame;

According to the azimuth and the distance, a reminder message is generated and broadcast; the reminder message is used to remind the driver of the current vehicle to pay attention to the pedestrian corresponding to the portrait frame.

In the above solution, the image to be detected is acquired in real time; the historical image containing the pedestrian is acquired; the training model is constructed according to the preset weight connection layer, and the training model is trained according to the historical image to obtain the pedestrian detection model. Input the image to be detected into a pre-trained pedestrian detection model to identify the pedestrian data contained in the image to be detected; the pedestrian detection model is trained according to a preset depth separable convolution method; for the pedestrian data Performing non-maximum value suppression processing to determine the portrait frame corresponding to the pedestrian data in the image to be detected. Increase the base of image training by processing historical images, construct a training model based on the weighted connection layer and perform training to obtain the pedestrian detection model, detect the image to be detected, and perform corresponding processing based on the detection result after the detection result is obtained , Not only improves the detection efficiency and detection accuracy of the image to be detected, but also improves the safety of vehicle driving and passersby.

It should be understood that in the embodiments of the present application, the processor 1401 may be a central processing unit (Central Processing Unit, CPU), and the processor may also be other general-purpose processors or digital signal processors (Digital Signal Processors, DSP). , Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.

The memory 1402 may include a read-only memory and a random access memory, and provides instructions and data to the processor 1401. A part of the memory 1402 may also include a non-volatile random access memory. For example, the memory 1402 may also store device type information.

In specific implementation, the processor 1401, the memory 1402, and the computer program 1403 described in the embodiments of the present application can execute the implementations described in the first and second embodiments of the pedestrian detection method provided in the embodiments of the present application. The terminal implementation described in the embodiments of the present application can also be implemented, and details are not described herein again.

In another embodiment of the present application, a computer-readable storage medium is provided, the computer-readable storage medium stores a computer program, the computer program includes program instructions, and the program instructions are implemented when executed by a processor:

Obtain the image to be inspected in real time;

Inputting the image to be detected into a pre-trained pedestrian detection model to identify pedestrian data contained in the image to be detected; the pedestrian detection model is trained according to a preset depth separable convolution method;

Non-maximum value suppression processing is performed on the pedestrian data, and a portrait frame corresponding to the pedestrian data in the image to be detected is determined.

Further, when the computer program is executed by the processor, it also implements:

Obtain historical images containing pedestrians;

A training model is constructed according to a preset weight connection layer, and the training model is trained according to the historical images to obtain the pedestrian detection model.

Further, when the computer program is executed by the processor, it also implements:

Performing image enhancement processing on the historical image to obtain at least two expanded images corresponding to the historical image;

A first training model is constructed according to a preset weight connection layer, and the first training model is trained according to the historical image and the corresponding extended image to obtain the pedestrian detection model.

Further, when the computer program is executed by the processor, it also implements:

Construct a second training model based on the preset weight connection layer, based on the preset depth separable convolution method and the preset linear bottleneck layer technology;

Input the historical image into the second training model, detect pedestrian images in the historical image, and determine the truth value label corresponding to each pedestrian image; the truth value label is used to indicate the identity of the recognized historical image The a priori box is based on the transform coefficient of the truth box;

Determine the loss function corresponding to the second training model according to the truth label;

The second training model is optimized according to the loss function to obtain the pedestrian detection model.

Further, when the computer program is executed by the processor, it also implements:

Detecting the orientation of the pedestrian corresponding to the portrait frame relative to the current vehicle, and the distance between the current vehicle and the pedestrian corresponding to the portrait frame;

According to the azimuth and the distance, a reminder message is generated and broadcast; the reminder message is used to remind the driver of the current vehicle to pay attention to the pedestrian corresponding to the portrait frame.

In the above solution, the image to be detected is acquired in real time; the historical image containing the pedestrian is acquired; the training model is constructed according to the preset weight connection layer, and the training model is trained according to the historical image to obtain the pedestrian detection model. Input the image to be detected into a pre-trained pedestrian detection model to identify the pedestrian data contained in the image to be detected; the pedestrian detection model is trained according to a preset depth separable convolution method; for the pedestrian data Performing non-maximum value suppression processing to determine the portrait frame corresponding to the pedestrian data in the image to be detected. Increase the base of image training by processing historical images, construct a training model based on the weighted connection layer and perform training to obtain the pedestrian detection model, detect the image to be detected, and perform corresponding processing based on the detection result after the detection result is obtained , Not only improves the detection efficiency and detection accuracy of the image to be detected, but also improves the safety of vehicle driving and passersby.

The computer-readable storage medium may be the internal storage unit of the terminal described in any of the foregoing embodiments, such as the hard disk or memory of the terminal. The computer-readable storage medium may also be an external storage device of the terminal, for example, a plug-in hard disk equipped on the terminal, a smart memory card (Smart Media Card, SMC), or a Secure Digital (SD) card , Flash Card, etc. Further, the computer-readable storage medium may also include both an internal storage unit of the terminal and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the terminal. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

A person of ordinary skill in the art may realize that the units and algorithm steps of the examples described in the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two, in order to clearly illustrate the hardware and software Interchangeability. In the above description, the composition and steps of each example have been generally described in terms of function. Whether these functions are executed by hardware or software depends on the specific application and design constraint conditions of the technical solution. Professionals and technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered beyond the scope of this application.

Those skilled in the art can clearly understand that, for the convenience and conciseness of description, the specific working process of the terminal and unit described above can refer to the corresponding process in the foregoing method embodiment, which is not repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed terminal and method may be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components can be combined or It can be integrated into another system, or some features can be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may also be electrical, mechanical or other forms of connection.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments of the present application.

In addition, the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solution of this application is essentially or the part that contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium It includes several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program code .

The above are only specific implementations of this application, but the protection scope of this application is not limited to this. Anyone familiar with the technical field can easily think of various equivalents within the technical scope disclosed in this application. Modifications or replacements, these modifications or replacements shall be covered within the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.

Claims (10)

A pedestrian detection method, characterized in that it comprises: Obtain the image to be inspected in real time; Inputting the image to be detected into a pre-trained pedestrian detection model to identify pedestrian data contained in the image to be detected; the pedestrian detection model is trained according to a preset depth separable convolution method; Non-maximum value suppression processing is performed on the pedestrian data, and a portrait frame corresponding to the pedestrian data in the image to be detected is determined. The pedestrian detection method according to claim 1, wherein before inputting the image to be detected into a pre-trained pedestrian detection model, and identifying the pedestrian data contained in the image to be detected, the method comprises: Obtain historical images containing pedestrians; A training model is constructed according to a preset weight connection layer, and the training model is trained according to the historical images to obtain the pedestrian detection model. The pedestrian detection method according to claim 2, wherein the constructing a training model according to a preset weight connection layer, and training the training model according to the historical image to obtain the pedestrian detection model, comprises: Performing image enhancement processing on the historical image to obtain at least two expanded images corresponding to the historical image; A first training model is constructed according to a preset weight connection layer, and the first training model is trained according to the historical image and the corresponding extended image to obtain the pedestrian detection model. The pedestrian detection method according to claim 2, wherein the constructing a training model according to a preset weight connection layer, and training the training model according to the historical image to obtain the pedestrian detection model, comprises: Construct a second training model based on the preset weight connection layer, based on the preset depth separable convolution method and the preset linear bottleneck layer technology; Input the historical image into the second training model, detect pedestrian images in the historical image, and determine the truth value label corresponding to each pedestrian image; the truth value label is used to indicate the identity of the recognized historical image The a priori box is based on the transform coefficient of the truth box; Determine the loss function corresponding to the second training model according to the truth label; The second training model is optimized according to the loss function to obtain the pedestrian detection model. The pedestrian detection method according to any one of claims 1 to 4, wherein the non-maximum value suppression processing is performed on the pedestrian data to determine the portrait frame corresponding to the pedestrian data in the image to be detected After that, it also includes: Detecting the orientation of the pedestrian corresponding to the portrait frame relative to the current vehicle, and the distance between the current vehicle and the pedestrian corresponding to the portrait frame; According to the direction and the distance, a reminder message is generated and broadcast; the reminder message is used to remind the driver of the current vehicle to pay attention to the pedestrian corresponding to the portrait frame. A pedestrian detection device, characterized in that it comprises: The acquiring unit is used to acquire the image to be detected in real time; A recognition unit, configured to input the image to be detected into a pre-trained pedestrian detection model to identify pedestrian data contained in the image to be detected; the pedestrian detection model is trained according to a preset depth separable convolution method; The determining unit is configured to perform non-maximum value suppression processing on the pedestrian data, and determine a portrait frame corresponding to the pedestrian data in the image to be detected. 8. The pedestrian detection device of claim 6, wherein the pedestrian detection device further comprises: The historical acquisition unit is used to acquire historical images containing pedestrians; The training unit is configured to construct a training model according to a preset weight connection layer, and train the training model according to the historical image to obtain the pedestrian detection model. The pedestrian detection device according to claim 6, wherein the training unit comprises: An expansion unit, configured to perform image enhancement processing on the historical image to obtain at least two expanded images corresponding to the historical image; The first training unit is configured to construct a first training model according to a preset weight connection layer, and train the first training model according to the historical image and the corresponding extended image to obtain the pedestrian detection model. A pedestrian detection device, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program as claimed in claim 1. Steps of any one of the methods to 5. A computer-readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the steps of the method according to any one of claims 1 to 5.

Images