CN116206276A - Linear object recognition method and device, vehicle and equipment - Google Patents

Abstract

The application relates to a linear target identification method, a linear target identification device, a linear target identification vehicle and linear target identification equipment. The method comprises the following steps: obtaining a perceived image, wherein the perceived image comprises a linear target object to be identified; performing discrete sampling on the perceived image to obtain a sampled image containing discrete sampling results; identifying a characteristic region of the sampling image through a convolutional neural network to obtain position information of sampling points corresponding to the linear target object; and determining the identification result of the linear object according to the position information of the sampling point corresponding to the linear object. The invention can solve the problem of limited output number of linear targets by discrete sampling of the perceived image; the linear target recognition method is small in calculated amount, wide in applicability and capable of being conveniently expanded to detection of other linear objects.

Description

Recognition method, device, vehicle and equipment for linear targets

technical field

The present application relates to the technical field of automatic driving perception recognition, and in particular to a method, device, vehicle and equipment for recognition of linear objects.

Background technique

Neural networks are a tool for large-scale, multi-parameter optimization. Relying on a large amount of training data, the neural network can learn hidden features that are difficult to summarize in the data, thereby completing multiple complex tasks, such as face detection, image semantic segmentation, object detection, action tracking, natural language translation, etc. Neural networks have been widely used in the field of artificial intelligence. In recent years, Convolutional Neural Networks (CNN) have achieved rapid development in the field of autonomous driving. For the autonomous driving system, the perception system is equivalent to the eyes of the autonomous vehicle, which can quickly identify dynamic and static obstacles around the vehicle body and the surrounding road structure environment. With the rapid embedding of CNN, the current perception system in self-driving vehicles uses neural networks to recognize the corresponding road environment. Compared with traditional methods, convolutional neural networks often have higher recognition accuracy and robustness. . However, when the image recognition method of the general convolutional neural network is applied to the recognition of linear targets, it will cause problems such as inaccurate recognition and complicated calculation process.

Contents of the invention

Based on this, it is necessary to provide a neural network-based linear target recognition method, device, vehicle, and computer equipment for the above-mentioned technical problems.

In a first aspect, the present application provides a neural network-based linear target recognition method, the method comprising:

Acquiring a perceptual image, the perceptual image including a linear target;

Perform discrete sampling and processing on the pixels of the perceptual image to obtain a perceptual image containing discrete sampling results, that is, a sampled image;

Identifying the feature region of the perceptual image containing discrete sampling results through a convolutional neural network to obtain position information of the sampling points corresponding to the linear target;

The identification result is determined according to the position information of the sampling point.

In one of the embodiments, the convolutional neural network includes a backbone network and a branch network, and the backbone network is used to identify and extract feature regions in the sampled image, and output a feature image of a preset size to the branch network.

In one of the embodiments, the branch network includes at least a first branch network and a second branch network, the first branch network and the second branch network generate feature images with different scales, and/or generate images corresponding to the perceptual Feature images of different regions of .

In one of the embodiments, the first branch network classifies the perceptual image, perceives whether each sampling point in the feature region contains a linear target, and perceives whether the linear target is contained through the second branch network. location information of the sampling points.

In one of the embodiments, after acquiring the perceptual image, it further includes: performing preprocessing on the perceptual image.

In one of the embodiments, the discrete sampling of the perceived image to obtain the sampled image includes:

sampling the pixels of the perceptual image at intervals of a preset number of points;

Establishing a coordinate system based on the perceived image;

Obtain the coordinates of the sampling point;

Based on the coordinates of the sampling points, a sampling image corresponding to the perceived image is obtained.

In one of the embodiments, the offset value of the sampling point coordinates relative to the center point of the feature area is the position information of the sampling point.

In one of the embodiments, the determination of the recognition result of the linear target according to the position information of the sampling point corresponding to the linear target includes:

determining the offset of the linear target in the sampling direction according to the position information of the sampling point corresponding to the linear target;

The identification result of the linear target is determined according to the offset of the linear target in the sampling direction.

In one of the embodiments, the coordinates of the sampling points are ( _xi , y _i ), then _xi = si-1, the sampling point data input into the neural network is the value of y _i , where _xi is the sampling point The abscissa of , y _i is the ordinate of the sampling point, s is the preset value of the pixel sampling interval of the perceptual image, i=0,1,2,3....

In a second aspect, the present application also provides a neural network-based linear target recognition device, the device comprising:

An acquisition module, configured to acquire a perceptual image, the perceptual image including a linear target to be identified;

The processing module is preset with a convolutional neural network; the processing module performs discrete sampling on the pixels of the perceptual image to obtain a sampled image containing discrete sampling results; the processing module recognizes the sampled image to obtain the corresponding linear target The position information of the sampling point; according to the position information of the sampling point corresponding to the linear target, determine the recognition result of the linear target;

The transmission module is used for transmitting data and identifying results obtained by the processing module.

In the third aspect, the present application also provides a vehicle, which adopts the above-mentioned identification device, and further includes:

The control device is connected to the recognition device, and the control device performs calculation according to the result of the processing module, and judges the traffic condition of the vehicle, and if the judgment is yes, controls the vehicle to move forward.

In one of the embodiments, the control device performs the following steps:

Select or set the reference system, fit the recognition results, and obtain the linear target equation;

Based on the linear target equation, calculate the relative position of the linear target and the reference system, and judge the state of the target according to the relative position;

The state of the target is used as a basis for judging whether the vehicle is moving forward.

In a fourth aspect, the present application also provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

Acquiring a perceptual image, the perceptual image includes a linear target to be identified;

Perform discrete sampling on the pixels of the perceptual image to obtain a sampled image containing discrete sampling results;

The convolutional neural network identifies the feature area of the sampled image, and obtains the position information of the sampling point corresponding to the linear target object;

The recognition result of the linear target is obtained according to the position information of the sampling point corresponding to the linear target.

In one of the embodiments, the convolutional neural network includes a backbone network and a branch network, and the branch network includes a first branch network and a second branch network; the sampling image is identified by the convolutional neural network to obtain a The position information of the sampling point corresponding to the linear target in the perceptual image includes:

extracting feature regions in the sampled image through the backbone network, and transmitting the identified feature regions to the first branch network and the second branch network respectively;

The first branch network is used to sense whether each sampling point in the feature area contains a linear target, and the second branch network is used to sense the position information of the sampling point containing the linear target.

In one of the embodiments, after acquiring the perceptual image, it further includes: performing preprocessing on the perceptual image.

In one of the embodiments, the discrete sampling of the perceived image to obtain the sampled image includes:

Sampling the pixels of the perceptual image at intervals of preset points;

Establishing a coordinate system based on the perceived image;

Obtain the coordinates of the sampling point;

Based on the coordinates of the sampling points, a sampling image corresponding to the perceived image is obtained.

In one of the embodiments, the offset value of the sampling point coordinates relative to the central point of the feature image is the position information of the sampling point.

In one of the embodiments, the determination of the recognition result of the linear target according to the position information of the sampling point corresponding to the linear target includes:

determining the offset of the linear target in the sampling direction according to the position information of the sampling point corresponding to the linear target;

The identification result of the linear target is determined according to the offset of the linear target in the sampling direction.

In one of the embodiments, the coordinates of the sampling points are ( _xi , y _i ), then _xi = si-1, the sampling point data input into the neural network is the value of y _i , where _xi is the sampling point The abscissa of , y _i is the ordinate of the sampling point, s is the preset value of the pixel sampling interval of the perceptual image, i=0,1,2,3....

The above-mentioned linear target recognition method, device, vehicle, and equipment can solve the problem of a limited number of linear target outputs by discretely sampling perceptual images; the linear target recognition method of the present application has a small amount of calculation, wide applicability, and It is convenient to extend to the detection of other linear objects.

Description of drawings

Fig. 1 is the flowchart of the linear object recognition method in an embodiment;

Fig. 2 is a schematic structural diagram of a convolutional neural network in an embodiment;

Fig. 3 is a schematic diagram of a sampling image in an embodiment;

Fig. 4 is a schematic structural diagram of a neural network-based linear object recognition device in another embodiment.

Detailed ways

In order to make the purpose, technical solution and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application, and are not intended to limit the present application.

First, the application scenarios of the embodiments of the present application and some of the vocabulary involved are explained.

The linear object recognition method, device, and equipment provided in the embodiments of the present application can be applied to the application scenarios of image recognition for driving vehicles, and are not limited to the field of fully automatic driving technology, and should be regarded as including similar image recognition fields such as assisted driving.

The executing subject of the linear object recognition method provided in the embodiment of the present application may be a linear object recognition device, a computer device, a storage medium, and a computer program product device. Exemplarily, the device can be implemented by software and/or hardware.

The image of the linear object involved in the embodiment of the present application and/or the perceptual image containing the linear object is mainly the image of the lever of the gate, but in other embodiments, it may also include but not limited to at least one of the following: wires pole image, zebra crossing image or lane line image.

Deep learning is a technology that performs nonlinear transformation or representation learning on the input by a neural network with no less than two hidden layers. By constructing a deep neural network, various analysis activities are performed. Neural networks include a variety of different categories according to different neurons. The common ones are feedforward neural network (FNN), convolutional neural network (CNN) - encoding spatial correlation, cyclic neural network (RNN) - encoding Time correlation, and confrontational neural network (GAN), etc. The generalized deep neural network includes any of the above and its combination. The narrowly defined deep neural network refers to the feedforward neural network. The simple feedforward neural network can even include only the input layer and the output layer, while the more complex ones have hidden layers. A neural network is also called a multilayer perceptron (MLP), and typically has three parts: an input layer, an output layer, and at least one hidden layer. Compared with other image classification algorithms, CNN uses relatively less preprocessing. This means the network will learn hand-crafted filters found in traditional algorithms. This independence from prior knowledge and human effort in feature design is a major advantage. CNNs have recognition applications as described in this paper in image and video recognition through detection and classification.

For the scene where the self-driving vehicle passes the gate, the key information is to identify whether the status of the gate lever allows passage. In the prior art, the above images are usually sent to the neural network to directly train a classification network. However, because the critical point is not easy to distinguish, it often leads to classification errors. In the visual detection tasks of some deep learning neural networks, a rectangular box is usually used to represent a target.

For the brake lever of the gate machine in this embodiment, if it is represented by a rectangular frame, when the brake lever of the gate machine is closed, the width of the rectangular frame will be much larger than its height due to the size ratio of the brake lever itself. At the same time, when the gate lever of the gate is lifted, the height of the rectangular frame is much larger than the width. During the whole gate lever closing to lifting process, the width-to-height ratio of the rectangular frame changes greatly, which is not conducive to the learning of neural network model features. .

In order to solve this technical problem, this application provides a linear target recognition method, which uses the inherent linear characteristics of the gate lever to abstract it into a linear target representation.

Therefore, an embodiment of the present application provides a linear object recognition method, and FIG. 1 is a schematic flowchart of the above recognition method. As shown in Figure 1, the linear target recognition method provided in this embodiment may include:

S100. Acquire a perception image, where the perception image includes a linear target to be identified.

In this step, a perception image is acquired through an image perception device (for example, a camera), and the perception image includes an image of a linear target. Further, the acquisition process in this step may be an image acquired by the device in real time, or an image acquired and stored in advance, and includes an acquisition process from a stored module.

After the perceptual image is acquired, the perceptual image can be preprocessed before the discrete sampling process on the perceptual image. Further, the preprocessing includes compressing or cropping the image to a preset initial image size WxH, where W represents the initial image width, and H represents the initial image height. In a preprocessing manner of this embodiment, the perceptual image is compressed to an initial image size whose WxH is 512x288 pixels. The perceptual image may be an image in RGB format, which this embodiment includes but is not limited to.

S200. Perform discrete sampling on pixels of the perceived image to obtain a sampled image including discrete sampling results.

In this embodiment, the perceptual image in step S200 may also represent the preprocessed initial image. In other embodiments, the perceptual image may not be preprocessed, and step S200 may be performed directly.

Specifically, step S200 also includes the following steps:

S201. Sampling the pixels of the perceptual image or the initial image at intervals of s points, s=0, 1, 2, 3...; further, selecting one or both of the horizontal and vertical directions for interval sampling , where the horizontal direction is parallel to the image width W, and the vertical direction is parallel to the image height H.

In this embodiment, the initial image size WxH is 512x288 pixels, and the value of s is 4. Sample every fourth pixel across the width of the image.

S202. Establish a coordinate system based on the perceived image;

In this embodiment, the bottom edge of the perceptual image is used as the x-axis, the left side of the perceptual image is the y-axis, and the intersection point between the bottom edge of the perceptual image and the left side of the perceptual image is used as the origin to establish a sampling point for describing The coordinate system for the coordinates.

S203. Obtain the coordinates of the sampling point as a discrete sampling result;

In this embodiment, the target object is the gear lever of the gate. The sampling point coordinates are expressed as (xi _, y _i ), then x _i =si-1, the sampling point data input into the neural network is the value of y _i , where x _i is the abscissa of the sampling point, and y _i is the sampling point The ordinate of the point, i=1,2,3....

Specifically, this embodiment needs to sequentially sample each row of the perceptual image or the initial image, and obtain a set of sampling point coordinates for each row. For example, if the sampling interval is s and the size of the image is 512x288 pixels, then a total of 128 sampling points containing linear objects are collected for discrete sampling of the row where the linear object is located in the perception image or initial image, including linear objects The mathematical formula description of the sampling point is expressed as:

pole＝[(x ₁ ,y ₁ ),...(x _i ,y _i ),...(x ₁₂₈ ,y ₁₂₈ )] (1)

Further, if the sampling interval is 4, s=4 is brought into x _i =4i-1, then the above mathematical formula (1) can be simplified to the following form:

pole＝[(3,y ₁ ),...(4i-1,y _i ),...(511,y ₁₂₈ )] (2)

It should be noted that, as shown in Figure 3, the vertical line in the figure is the sampling line, since the linear target (such as the brake lever) only occupies the middle section of all 128 sampling points, at this time the linear target and the sampling line are not Where they intersect, their vertical coordinates are zero. The above formulas (1) and (2) only represent the set of sampling point coordinates for the perceptual image or the initial image containing the linear target, since this embodiment needs to sample each row of the perceptual image or the initial image, so it can be obtained Multiple sets of sampling point sets corresponding to formula (1) and formula (2). And the x coordinate value of each sampling point in each sampling set is known, and the y coordinate value is unknown. For example, the x coordinates of each sampling point in each sampling set are all 3,...,4i-1,... .,511.

S204. Determine a sampling image corresponding to the perceived image based on the coordinates of the sampling point;

In this embodiment, based on the position coordinates of each sampling point in each group of sampling sets collected in S203, the perception image is divided into grids to obtain the sampling image corresponding to the perception image. For example, what is shown in FIG. 3 is the perception Schematic diagram of the sampled image obtained after the image is discretely sampled. In this embodiment, the sampling result is specifically a coordinate matrix formed by coordinates ( _xi , y _i ) of all discrete sampling points. Mark the coordinates of all the sampling points obtained in the image, that is, each perceptual image or initial image corresponds to an annotation text, and the annotation text contains the coordinate matrix of the sampling points. For sampling points that do not contain linear targets, the coordinates of the sampling results are known, and for sampling points that contain linear targets, the sampling results are shown in formula (2), which contains unknown y-coordinate information. In this embodiment, the value of the abscissa has been determined for each sampling point, so when using the convolutional neural network in subsequent steps, it is only necessary to predict the corresponding y. Further, it is particularly selected that the offset value of the sampling point coordinates relative to the center point of the feature image is the position information of the sampling point.

S300. Recognize the sampled image through a convolutional neural network to obtain position information of the sampling point.

Convolutional neural network is mainly composed of input layer, convolutional layer, pooling (Pooling) layer and fully connected layer. The input layer is to input raw data or data preprocessed by other algorithms into the convolutional neural network.

As shown in FIG. 2 , the convolutional neural network 1 of this embodiment includes a backbone network 20 and a branch network 30 , and the branch network 30 includes a first branch network 31 and a second branch network 32 . The backbone network 20 is used to identify feature regions in the sampled images, and output feature images of preset sizes to the branch networks. Further, the backbone network 20 transmits the identified feature regions to the first branch network 31 and the second branch network respectively. 32. The backbone network 20 is mainly composed of multiple convolutional layers and pooling layers. In this embodiment, each of the convolutional layers and pooling layers has at least two layers. The convolutional layer of the convolutional neural network 1 extracts features from the input data, and abstracts the implicit correlation in the original data through the convolutional kernel matrix. Convolutional neural networks 1 usually periodically insert a pooling layer between successive convolutional layers. The function of the pooling layer is to gradually reduce the spatial size of the data body, so that the number of parameters in the network can be reduced, and the computing resource consumption is reduced.

Convolutional neural networks identify feature regions in perceptual images. Further, after the sampled image 10 is input into the backbone network 20 of the convolutional neural network 1, feature extraction is performed through the backbone network 20 to obtain an intermediate feature image 21 (ie, a feature region). In this embodiment, as shown in FIG. 2 , the size of the intermediate feature image 21 output by the backbone network 20 is wxh=W/32xH/32=16x9 pixels.

The backbone network 20 is connected with at least two branch networks 30, including a first branch network 31 and a second branch network 32, and the first branch network 31 and the second branch network 32 generate feature maps with different scales, and/or generate Feature maps corresponding to different regions of the perceptual image. The first branch network 31 senses whether each sampling point in the feature area contains a linear target, and senses the position information of the sampling point containing the linear target through the second branch network 32 . It should be noted that in this embodiment, "connection" may indicate that the output of the previous network is used as the input of the next network in the direction of signal transmission. For example, “the backbone network 20 is connected to at least two branch networks 30 ” may indicate that the output of the backbone network 20 is used as the common input of the two branch networks 30 . The first branch network 31 classifies the intermediate feature image 21, which is specifically used to judge whether each sampling point contains a linear object, that is, to classify each sampling point into a category that contains a linear object, or does not contain a linear object. a class of things. The second branch network 32 supervises the first branch network 31 . Further, the first branch network 31 judges all the intermediate feature images 21 according to the training, and obtains which category the intermediate feature images 21 belong to by calculating probability values and other methods. The first branch network 31 has two channels, respectively outputting a type of sampling point containing a linear object and a type of sampling point not containing a linear object. In this embodiment, the linear object is a gate lever. Further, the second branch network 32 is a regression network, and the second branch network 32 continuously calculates the coordinate values and difference values of the data to determine the final recognition result. In the present embodiment, the second branch network 32 has 128 channels, and the intermediate feature image 21 in each channel contains corresponding sampling points, and the second branch network 32 is used to output the image of the sampling points containing the linear target. location information. Optionally, for each sampling point that does not contain a linear target, its position information may not be output, or its position information may be set to 0 by default.

Before the convolutional neural network can make actual predictions, it needs to be trained according to the preset training set. The training process of the convolutional neural network 1 includes: obtaining or inputting sample sampling images, constructing a training set, and the sample sampling images include linear objects; inputting the training set into the convolutional neural network, and using the loss function to obtain the final neural network Model. Specifically, when constructing the training set, it is necessary to sample the sample-aware image, and input the indexed sample sampled image to the neural network for learning. The specific sampling method is as described above. During training, a corresponding loss function needs to be constructed. The loss function designed for linear targets is as follows:

In formula (3). The calculation formulas of the loss function loss _cls of the first branch network 31 and the loss function loss _offset of the second branch network 32 are respectively:

loss _cls ＝-[c ^t lnc ^p +(1-c ^t )ln(1-c ^p )] (4)

l^obj表示当第(i,j)网格内有挡杆通过时为1，否则为0；N_cls表示最后特征图的大小，也就是网格的数目N_cls＝wxh。c^t表示第(i,j)的网格里的真值，c^p表示第(i,j)的网格里的预测值，y_k ^p表示神经网络预测偏移量的值，y_k ^gt表示偏移量的真值。待训练的卷积神经网络的训练目标是最小化系统损失值，通过训练结果不断调整卷积神经网络的各参数，最终构建所需要的卷积神经网络1。For the classification branch, the loss function used in this embodiment is a cross-entropy loss function; for the coordinate offset regression branch, the loss function used in this embodiment is a square difference loss function. in,

l ^obj means 1 when there is a blocking rod passing through the (i, j)th grid, otherwise it is 0; N _cls means the size of the final feature map, that is, the number of grids N _cls =wxh. c ^t represents the true value in the (i,j)th grid, c ^p represents the predicted value in the (i,j)th grid, y _k ^p represents the value of the neural network prediction offset, y _k ^gt The truth value representing the offset. The training goal of the convolutional neural network to be trained is to minimize the system loss value, continuously adjust the parameters of the convolutional neural network through the training results, and finally construct the required convolutional neural network 1 .

Further, the backbone network 20 of the convolutional neural network 1 performs convolution and pooling on the sampled image to obtain an intermediate feature image 21 (ie, a feature area); the branch network classifies and returns the intermediate feature image 21 to obtain the perceptual image The position information of the sampling point corresponding to the linear target in .

Optionally, in this embodiment, in this step, after classification by the first branch network 31 and supervision calculation by the second branch network 32, the position of the sampling point corresponding to the linear target in the perceptual image The information may be an offset Δy _{ci =} y ci −y center of the c discrete sampling points y _ci of _the linear target (brake bar) relative to the grid center point y _center in the y direction.

S400. Obtain a recognition result according to the position information of the sampling point corresponding to the linear target.

Wherein, the recognition result of the linear target in this embodiment may be the position of the linear target, the state of the linear target, or the offset angle of the linear target relative to the sampling direction.

Further, if the recognition result of the linear target is the state of the linear target, S400 includes the following steps:

S401. Select or set a reference system; in this embodiment, select the bottom edge and the side edge of the perceptual image as the x-axis and the y-axis of the coordinate axes respectively.

S402. Process the position information of the corresponding sampling points of the linear target, and obtain the equation of the linear target by fitting;

In step S302, the convolutional neural network has obtained the position information of the sampling point corresponding to the linear target, such as the coordinate information of the x-axis and y-axis of the sampling point, or the offset of the y-axis, that is, △ y _ci . If the position information is an offset, then according to certain calculations, the coordinates of the sampling points corresponding to the linear targets, that is, the coordinates of the gate lever pixels in the feature area, can be obtained. According to the position coordinates of the sampling points corresponding to the linear target objects, the equation of the brake lever can be obtained after straight line fitting using the least square method in this embodiment. If the position information is the position coordinates of the sampling points corresponding to the linear target, the equation of the linear target (such as a brake lever) can be obtained by direct fitting.

S403. Based on the equation of the linear target, calculate the relative position of the linear target and the reference frame, judge the state of the target according to the relative position, and output the recognition result.

Further, in this embodiment, according to the equation and coordinate system of the above brake lever, the slope of the brake lever equation can be calculated, and then the tilt angle of the brake lever can be calculated, and the state performance of the gate can be judged according to the change of the tilt angle of the brake lever. No to allow the vehicle to pass, and output the recognition result.

Further, if the recognition result of the linear target is the inclination angle of the linear target, S400 may also determine the offset of the linear target in the sampling direction according to the position information of the sampling point corresponding to the linear target ; Determine the recognition result of the linear target according to the offset of the linear target in the sampling direction. Specifically, the difference between the sampling point corresponding to the linear target and the center point of the sampling grid area where the sampling point is located can be calculated as the offset of the linear target in the sampling direction, and based on the The offset of the sampling direction determines the inclination angle of the linear target relative to a certain direction, and takes the inclination angle as the final recognition result of the linear target. If the linear target is a gate, the inclination angle between the gate and the horizontal direction can be determined as the identification result of the gate state.

In this embodiment, the linear target is taken as the gate lever as an example. At the same time, by detecting the sampling point of the gate lever, the corresponding angle change during the lifting process of the gear lever can be obtained. Finally, the status of whether the gate is allowed to pass can be effectively obtained through threshold control. It is convenient to grasp the state of the barrier bar of the gate in real time; through the change of the angle, the critical value of the gate bar from closing to lifting can be obtained more accurately, and the state of the gate given is more accurate; the invention directly classifies the image The method is robust, and at the same time can additionally obtain the angle of the gear lever, which is convenient for informing the driver of the current gear lever status.

It should be understood that although the steps in the flow charts involved in the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least some of the steps in the flow charts involved in the above-mentioned embodiments may include multiple steps or stages, and these steps or stages are not necessarily executed at the same time, but may be performed at different times For execution, the execution order of these steps or stages is not necessarily performed sequentially, but may be executed in turn or alternately with other steps or at least a part of steps or stages in other steps.

Based on the same inventive concept, an embodiment of the present invention also provides a linear object recognition device for realizing the neural network-based linear object recognition method mentioned above. The solution to the problem provided by this device is similar to the implementation described in the above method, so the specific limitations in one or more embodiments of the linear target recognition device provided below can be referred to above for the linear target recognition The limitation of the method will not be repeated here.

As shown in FIG. 4 , in this embodiment, the linear object recognition device 100 includes a collection module 101 , a data storage module 103 and an output module 104 , and the linear target recognition device 100 is set on a terminal 200 . Wherein, the terminal 102 communicates with the linear object recognition device 100 through the transmission module 104 . The data storage module 103 may store or cache data that the processing module 102 needs to process. The data storage module 103 can be integrated on the processing module 102, or placed on the cloud or other servers, and the server can be realized by an independent server or a server cluster composed of multiple servers. The acquisition module 101 is used to acquire a perception image, and the perception image includes a linear target object. Further, the acquisition module is a camera, and the perception image is a picture or a video. The processing module 102 is preset with the above-mentioned trained convolutional neural network. The processing module 102 performs discrete sampling and processing on the pixels of the perceptual image; identifies the feature region of the perceptual image to obtain a feature image containing sampling results; and obtains the recognition result according to the feature image. The processing module 102 transmits the identification result to the terminal 200 through the transmission module 104 . The terminal 200 can be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, IoT devices and portable wearable devices, and the IoT devices can be smart speakers, smart TVs, smart air conditioners, smart vehicle devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, and the like. In this embodiment, the terminal 200 is a vehicle.

Each module in the above-mentioned linear object recognition device can be fully or partially realized by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, and can also be stored in the memory of the computer device in the form of software, so that the processor can invoke and execute the corresponding operations of the above-mentioned modules.

The embodiment of the present invention also provides an application scenario for realizing the neural network-based linear object recognition method involved above. This embodiment provides a vehicle, the vehicle is provided with a linear object recognition device 100, and the vehicle It also includes a control device connected to the linear object recognition device 100, the control device calculates according to the recognition result of the processing module 102, and judges the traffic situation of the vehicle, and if the judgment is yes, then controls the vehicle to move forward. The control device performs the following steps: select or set the reference system, fit the recognition results to obtain the target object equation; calculate the relative position of the target object and the reference system, and judge the state of the target object according to the relative position; use the state of the target object as the judgment The basis for whether the vehicle is moving forward. Further, in this embodiment, the control device executes step S400 of the neural network-based linear object recognition method above, and judges whether the state of the gate can allow vehicles to pass according to the change of the tilt angle of the brake lever, and compares the angle with the preset Threshold comparison, after meeting the passing requirements, the control device controls the vehicle to move forward or gives a forward signal. Further, when the vehicle in this embodiment is a vehicle using automatic driving technology or the vehicle is set to automatic driving mode, the control device automatically controls the vehicle to advance; when the vehicle in this embodiment is an ordinary vehicle, the control device gives Signal.

In one embodiment, a computer device is provided, which may be a server. The computer device includes a processor, memory and a network interface connected by a system bus. Wherein, the processor of the computer device is used to provide calculation and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs and databases. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer device is used to store images and calculate process data. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, the above-mentioned neural network-based linear object recognition method can be realized.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above-mentioned embodiments can be completed by instructing related hardware through computer programs, and the computer programs can be stored in a non-volatile computer-readable memory In the medium, when the computer program is executed, it may include the processes of the embodiments of the above-mentioned methods. Wherein, any reference to storage, database or other media used in the various embodiments provided in the present application may include at least one of non-volatile and volatile storage. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive variable memory (ReRAM), magnetic variable memory (Magnetoresistive Random Access Memory, MRAM), Ferroelectric Random Access Memory (FRAM), Phase Change Memory (Phase Change Memory, PCM), graphene memory, etc. The volatile memory may include random access memory (Random Access Memory, RAM) or external cache memory. As an illustration and not a limitation, RAM can be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (Dynamic Random Access Memory, DRAM). The databases involved in the various embodiments provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a blockchain-based distributed database, etc., but is not limited thereto. The processors involved in the various embodiments provided by this application can be general-purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, data processing logic devices based on quantum computing, etc., and are not limited to this.

The technical features of the above embodiments can be combined arbitrarily. To make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be It is considered to be within the range described in this specification.

The above-mentioned embodiments only express several implementation modes of the present application, and the description thereof is relatively specific and detailed, but should not be construed as limiting the patent scope of the present application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be determined by the appended claims.

Claims (10)

1. A linear target recognition method, characterized in that the method comprises: Acquiring a perceptual image, the perceptual image includes a linear target to be identified; Perform discrete sampling on the perceptual image to obtain a sampled image containing discrete sampling results; Identifying the feature region of the sampled image through a convolutional neural network to obtain the position information of the sampling point corresponding to the linear target; The recognition result of the linear target is determined according to the position information of the sampling point corresponding to the linear target. 2. linear target recognition method according to claim 1, is characterized in that, described convolutional neural network comprises backbone network and branch network, and described backbone network is used for identifying and extracting the characteristic area in described sampling image, And output a feature image of a preset size to the branch network; the branch network includes at least a first branch network and a second branch network, the first branch network and the second branch network generate feature images with different scales, or generate feature images corresponding to the The feature images of different regions of the sampled image; the first branch network classifies the sampled image; the second branch network perceives the position information of the sampling point containing the linear target. 3 . The linear target recognition method according to claim 1 , further comprising: preprocessing the perceptual image after acquiring the perceptual image. 4 . 4. The linear target recognition method according to any one of claims 1 to 3, wherein said performing discrete sampling on the perceptual image to obtain a sampled image comprises: Sampling the pixels of the perceptual image at intervals of preset points; Establishing a coordinate system based on the perceived image; Obtain the coordinates of the sampling point; Based on the coordinates of the sampling points, a sampling image corresponding to the perceived image is obtained. 5. The linear target recognition method according to any one of claims 1 to 3, wherein the offset value of the sampling point coordinates relative to the center point of the feature area is the position information of the sampling point . 6. linear target recognition method according to claim 4, is characterized in that, described sampling point coordinate is ( _xi , y _i ), then x _i =si-1, input the sampling point data of described neural network is the value of y _i , wherein, x _i is the abscissa of the sampling point, y _i is the ordinate of the sampling point, s is the preset value of the pixel sampling interval of the perceptual image, i=0,1,2,3... … 7. A linear target recognition device, characterized in that it comprises: An acquisition module, configured to acquire a perceptual image, the perceptual image including a linear target to be identified; The processing module is preset with a convolutional neural network; the processing module performs discrete sampling on the pixels of the perceptual image to obtain a sampled image containing discrete sampling results; the processing module recognizes the sampled image to obtain the corresponding linear target The position information of the sampling point; according to the position information of the sampling point corresponding to the linear target, the recognition result of the linear target is determined. 8. A vehicle using the identification device according to claim 7, further comprising: The control device is connected to the recognition device, and the control device performs calculation according to the result of the processing module, and judges the traffic condition of the vehicle, and if the judgment is yes, controls the vehicle to move forward. 9. The vehicle of claim 8, wherein the control device performs the following steps: Select or set the reference system, fit the recognition results, and obtain the linear target equation; Based on the linear target equation, calculate the relative position of the linear target and the reference system, and judge the state of the target according to the relative position; The state of the target is used as a basis for judging whether the vehicle is moving forward. 10. A computer device, comprising a memory and a processor, the memory stores a computer program, wherein the processor implements the method according to any one of claims 1 to 6 when executing the computer program step.

Images