CN107886036B - Vehicle control method and device and vehicle - Google Patents

Abstract

The invention discloses a vehicle control method, a vehicle control device and a vehicle, which can reduce the manufacturing cost of an adaptive cruise system. The method comprises the following steps: acquiring a first image and a second image, wherein the first image is a color image or a brightness image, and the second image is a depth image; identifying a target vehicle in the second image to acquire distance information of the target vehicle; acquiring an azimuth angle of the target vehicle according to the first image or the second image; determining the relative speed of the target vehicle according to the azimuth angle of the target vehicle and the intermediate frequency signal acquired by the constant carrier frequency radar; and controlling the motion parameters of the main vehicle according to the distance information and the relative speed.

Description

Vehicle control method, device and vehicle

technical field

The present invention relates to the technical field of vehicles, in particular to a vehicle control method, device and vehicle.

Background technique

With the continuous development of science and technology, people's travel has become more and more convenient, and all kinds of cars, electric vehicles, etc. have become indispensable means of transportation in people's lives. Some existing vehicles already have adaptive cruise capabilities.

At present, the vehicle's adaptive cruise system can use millimeter-wave radar, lidar, or stereo cameras as ranging sensors. By installing these types of ranging sensors, the vehicle can simultaneously sense multiple target vehicles in front of the vehicle, and then automatically Adaptively adjust the motion parameters of the cruise system.

However, the ranging algorithm using stereo cameras is more complicated, which may lead to an increase in the power consumption of computer chips, while using a single ordinary camera with millimeter-wave radar or lidar requires a larger installation space in the vehicle and is more expensive .

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a vehicle control method, device and vehicle, which can reduce the manufacturing cost of the adaptive cruise system.

According to a first aspect of the embodiments of the present invention, a vehicle control method is provided, including:

acquiring a first image and a second image, wherein the first image is a color image or a luminance image, and the second image is a depth image;

Identifying a target vehicle in the second image to obtain distance information of the target vehicle;

obtaining the azimuth angle of the target vehicle according to the first image or the second image;

Determine the relative speed of the target vehicle according to the azimuth angle of the target vehicle and the intermediate frequency signal obtained by the constant carrier frequency radar;

According to the distance information and the relative speed, the motion parameters of the subject vehicle are controlled.

Optionally, the method further includes:

Identify highway lane lines according to the first image;

According to the mapping relationship between the first image and the second image, the highway lane line is mapped to the second image, so as to determine at least one vehicle identification range in the second image, wherein each Two adjacent highway lane lines create a vehicle identification range;

Identifying the target vehicle in the second image includes:

The target vehicle is identified in the at least one vehicle identification range.

Optionally, the method further includes:

obtaining the slope of the initial straight line mapped to each highway lane line in the second image;

Mark the vehicle identification range created by the highway lane lines corresponding to the two initial straight lines with the largest slopes as this lane, and mark the rest of the vehicle identification ranges as non-own lanes;

Identifying a target vehicle in the at least one vehicle identification range includes:

Recognition of the target vehicle in the own lane in the vehicle recognition range marked as the own lane, recognition of the target vehicle in the non-own lane in the vehicle recognition range marked as the non-own lane, and vehicle recognition in the combination of two adjacent vehicle recognition ranges Identify the target vehicle changing lanes in range.

Optionally, the method further includes:

determining a target vehicle area in the second image by identifying the target vehicle;

mapping the target vehicle area to the first image according to the mapping relationship between the first image and the second image, so as to generate a vehicle lamp recognition area in the first image;

identifying the turn signal of the target vehicle in the vehicle lamp identification area;

According to the distance information and the relative speed, the motion parameters of the subject vehicle are controlled, including:

The motion parameters of the subject vehicle are controlled according to the distance information, the relative speed, and the identified turn signals of the target vehicle.

Optionally, acquiring the azimuth angle of the target vehicle according to the first image or the second image, including:

Obtain the azimuth angle of the target vehicle according to the position of the target vehicle area in the second image; or,

Obtain the azimuth angle of the target vehicle according to the position of the vehicle lamp identification area in the first image.

Optionally, the method further includes:

According to the identified azimuth angle of the target vehicle, the constant carrier frequency radar is automatically calibrated.

According to a first aspect of the embodiments of the present invention, there is provided a vehicle control device, comprising:

an image acquisition module, configured to acquire a first image and a second image, wherein the first image is a color image or a luminance image, and the second image is a depth image;

a first identification module for identifying a target vehicle in the second image to obtain distance information of the target vehicle;

a first acquisition module, configured to acquire the azimuth angle of the target vehicle according to the first image or the second image;

a first determining module, configured to determine the relative speed of the target vehicle according to the azimuth angle of the target vehicle and the intermediate frequency signal obtained by the constant carrier frequency radar;

The control module is configured to control the motion parameters of the subject vehicle according to the distance information and the relative speed.

Optionally, the device further includes:

a second identification module, configured to identify highway lane lines according to the first image;

a first mapping module, configured to map the highway lane lines to the second image according to the mapping relationship between the first image and the second image, so as to determine at least one vehicle recognition range, where every two adjacent highway lane lines create a vehicle recognition range;

The first identification module is used for:

The target vehicle is identified in the at least one vehicle identification range.

Optionally, the device further includes:

a second acquiring module, configured to acquire the slope of the initial straight line mapped to each highway lane line in the second image;

The creation module is used to mark the vehicle identification range created by the highway lane lines corresponding to the two initial straight lines with the largest slopes as this lane, and mark the rest of the vehicle identification ranges as non-own lanes;

The first identification module is used for:

Recognition of the target vehicle in the own lane in the vehicle recognition range marked as the own lane, recognition of the target vehicle in the non-own lane in the vehicle recognition range marked as the non-own lane, and vehicle recognition in the combination of two adjacent vehicle recognition ranges Identify the target vehicle changing lanes in range.

Optionally, the device further includes:

a second determination module, configured to determine a target vehicle area in the second image by identifying the target vehicle;

A second mapping module, configured to map the target vehicle area to the first image according to the mapping relationship between the first image and the second image, so as to generate a vehicle in the first image Light identification area;

a third identification module, configured to identify the turn signal of the target vehicle in the vehicle lamp identification area;

The control module is used for:

The motion parameters of the subject vehicle are controlled according to the distance information, the relative speed, and the identified turn signals of the target vehicle.

Optionally, the first obtaining module is used for:

Obtain the azimuth angle of the target vehicle according to the position of the target vehicle area in the second image; or,

Obtain the azimuth angle of the target vehicle according to the position of the vehicle lamp identification area in the first image.

Optionally, the device further includes:

The calibration module is configured to automatically calibrate the constant carrier frequency radar according to the identified azimuth angle of the target vehicle.

According to a first aspect of the embodiments of the present invention, there is provided a vehicle, comprising:

An image acquisition device for acquiring a first image and a second image, wherein the first image is a color image or a luminance image, and the second image is a depth image; and the vehicle control device according to the second aspect.

In the embodiment of the present invention, the azimuth angle of the target vehicle can be obtained by identifying the image, the relative speed of the target vehicle can be obtained through the constant carrier frequency radar, and the distance information of the target vehicle can be obtained in combination with the depth image. The combination of cameras can more accurately sense the target vehicle near the subject vehicle, and then better perform adaptive cruise. At the same time, because the transmitter of the constant carrier frequency radar works on an almost constant electromagnetic wave frequency, the electromagnetic wave bandwidth occupied by the constant carrier frequency radar is very small compared with the FM radar for ranging, so the constant carrier frequency radar can reduce the use of components. Reduced cost of adaptive cruise system.

Other features and advantages of the present invention will be described in detail in the detailed description that follows.

Description of drawings

The accompanying drawings are used to provide a further understanding of the present invention, and constitute a part of the specification, and together with the following specific embodiments, are used to explain the present invention, but do not constitute a limitation to the present invention. In the attached image:

FIG. 1 is a flow chart of a vehicle control method according to an exemplary embodiment.

FIG. 2 is a flow chart of another vehicle control method according to an exemplary embodiment.

FIG. 3 is a flowchart of another vehicle control method according to an exemplary embodiment.

FIG. 4 is a flowchart illustrating another vehicle control method according to an exemplary embodiment.

FIG. 5 is a schematic diagram illustrating a target vehicle area and a vehicle lamp identification area according to an exemplary embodiment.

FIG. 6 is a schematic diagram of a temporal micromolecule image according to an exemplary embodiment.

FIG. 7 is a block diagram of a vehicle control device according to an exemplary embodiment.

Fig. 8 is a block diagram of a vehicle according to an exemplary embodiment.

Detailed ways

The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present invention, but not to limit the present invention.

FIG. 1 is a flowchart of a vehicle control method according to an exemplary embodiment. As shown in FIG. 1 , the vehicle control method can be applied to a body vehicle, and includes the following steps.

Step S11: Acquire a first image and a second image, wherein the first image is a color image or a luminance image, and the second image is a depth image.

Step S12: Identify the target vehicle in the second image to obtain distance information of the target vehicle.

Step S13: Acquire the azimuth angle of the target vehicle according to the first image or the second image.

Step S14: Determine the relative speed of the target vehicle according to the azimuth angle of the target vehicle and the intermediate frequency signal obtained by the constant carrier frequency radar.

Step S15: Control the motion parameters of the subject vehicle according to the distance information and the relative speed.

The first image may be a color image or a luminance image, the second image may be a depth image, and the first image and the second image may be acquired by the same image acquisition device installed on the subject vehicle. For example, the first image is acquired by an image sensor of the image acquisition device, and the second image is acquired by a TOF (Time of Flight, time of flight) sensor of the image acquisition device.

In the embodiment of the present invention, the pixels of the color or brightness image and the pixels of the depth image may be interleaved and arranged in a certain ratio, and the embodiment of the present invention does not limit the ratio. For example, both image sensors and TOF sensors can be fabricated using a complementary metal-oxide-semiconductor (CMOS) process, and luminance pixels and TOF pixels can be fabricated on the same substrate at scale, such as 8 luminance pixels fabricated at an 8:1 ratio and 1 TOF pixel to form a large interleaved pixel, wherein the photosensitive area of 1 TOF pixel can be equal to the photosensitive area of 8 luminance pixels, of which 8 luminance pixels can be arranged in an array of 2 rows and 4 columns. For example, an array of active interleaved pixels with 360 rows and 480 columns can be fabricated on a substrate with a 1-inch optical target surface, and an active luminance pixel array with 720 rows and 1920 columns, and an active TOF pixel array with 360 rows and 480 columns can be obtained. The same image acquisition device composed of this image sensor and TOF sensor can simultaneously acquire color or brightness images and depth images.

Optionally, please refer to FIG. 2 , which is a flowchart of another vehicle control method. After acquiring the first image and the second image, it may further include step S16 to identify the highway lane line according to the first image; step S17 to map the highway lane line to the second image according to the mapping relationship between the first image and the second image , to determine at least one vehicle identification range in the second image, wherein every two adjacent highway lane lines can create a vehicle identification range. In this case, step S12 may be to identify at least one target vehicle in a vehicle identification range to obtain distance information of the target vehicle.

Since the first image is a color or brightness image, and identifying the position of the highway lane line only needs to use the brightness difference between the highway lane line and the road surface, only the brightness information of the first image is needed to obtain the highway lane line. Then, when the first image is a luminance image, the road lane line can be identified directly according to the luminance information of the first image. When the first image is a color image, the road lane line can be identified after the first image is converted into a luminance image.

Every two adjacent highway lane lines create a vehicle identification range, that is, the vehicle identification range corresponds to the actual lane, then the target vehicle is identified within the vehicle identification range, that is, the target vehicle on the lane is identified. In this way, the range of the recognized target vehicle can be determined to the lane, to ensure that the recognized object is a vehicle driving in the lane, to avoid the interference caused by other non-vehicle objects in the image, and to improve the accuracy of the recognition of the target vehicle.

Optionally, since the highway lane lines have both solid line lane lines and dashed line lane lines, identifying the highway lane lines in the first image may be to obtain all the solid line lane lines included in the highway lane line according to the first image. Edge pixel positions, and obtain all edge pixel positions of each dashed lane line included in the highway lane line. Only in this way can the solid lane line and the dashed lane line be completely recognized, thereby improving the accuracy of identifying the target vehicle.

Optionally, obtain all edge pixel positions of each solid lane line included in the highway lane line, create a binary image corresponding to the first image, and then detect all edges of each solid lane line in the binary image. pixel location.

How to create the binary image corresponding to the first image is not limited in this embodiment of the present invention, and several possible manners are exemplified below.

For example, using the brightness difference between the road lane line and the road surface, some brightness thresholds can be obtained by searching. The brightness threshold can be found by using the "histogram statistics-double peak" algorithm, and the brightness threshold and the brightness image can be used to create prominent highway lane lines. the binary image.

Or, for example, it is also possible to divide the luminance image into multiple luminance sub-images, perform the "histogram statistics-bimodal" algorithm on each luminance sub-image to find multiple luminance thresholds, and use each luminance threshold and corresponding luminance sub-images. Create a binary sub-image highlighting the highway lane lines, and use the binary sub-image to create a complete binary image highlighting the highway lane lines, which can cope with changes in the brightness of the road surface or lane lines.

After the binary image corresponding to the first image is created, all edge pixel positions of each solid line lane line may be detected in the binary image, and the detection method is also not limited in this embodiment of the present invention.

For example, since the curvature radius of the highway lane line cannot be too small, and due to the camera projection principle, the near lane lines have more imaging pixels than the far lane lines, so that the solid lane lines of the curve are arranged in a straight line in the luminance image. The pixels of the solid line also account for most of the imaging pixels of the solid lane line, so straight line detection algorithms such as the Hough transform algorithm can be used to detect all the edge pixel positions of the solid line lane line of the straight road in the binary image highlighting the highway lane line or Most of the initial straight edge pixel locations for the solid lane line of the curve are detected.

The line detection may also detect most of the line edge pixel positions of the isolation belt and the utility pole in the binary image. Then, for example, the slope range of the lane line in the binary image can be set according to the aspect ratio of the image sensor, the focal length of the camera lens, the road width range of the road design specification and the installation position of the image sensor in the subject vehicle, so that according to the slope range, the range of the slope of the lane line in the binary image can be set. Straight line filtering for non-lane lines is excluded.

Since the edge pixel positions of the solid lane line of the curve always change continuously, the connected pixel positions of the edge pixel positions at both ends of the initial straight line detected above are searched, and the connected pixel positions are merged into the initial straight line edge pixel set , repeat the above search and merge the connected pixel position, and finally determine all edge pixel positions of the solid line lane line of the curve uniquely.

All edge pixel positions of solid highway lane lines can be detected in the above manner.

Optionally, the first dashed highway lane line can be any dashed highway lane line included in the highway lane line, and the edge pixel position of the first dashed line lane line can be obtained, the first solid line highway lane line can be identified according to the first image, and then All edge pixel positions of the first solid line highway lane line are projected to the edge pixel positions of the initial straight line of the first dashed line lane line to obtain all edge pixel positions of the first dashed line lane line. The first solid-line highway lane line may be any solid-line highway lane line included in the highway lane line.

In the embodiment of the present invention, all edge pixel positions of the first solid line lane line can be projected to the first solid line lane line according to the prior knowledge of the solid line lane line, the principle that the lane lines are parallel to each other in reality, and the projection parameters of the image sensor and the camera. The initial straight line edge pixel position of the dashed lane line is to connect the initial straight line edge pixel position of the first dashed line lane line and the edge pixel positions of other shorter lane lines belonging to the first dashed line lane line, so as to obtain all the edge pixels of the dashed line lane line Location.

Optionally, the first dashed highway lane line is any dashed highway lane line included in the highway lane line, and the edge pixel positions of the first dashed line lane line are obtained, and the binary images corresponding to the plurality of consecutively obtained first images can be obtained respectively. Superposition is performed to superimpose the first dashed lane line into a solid line lane line, and then all edge pixel positions of the superimposed solid line lane line are obtained.

In the embodiment of the present invention, it is not necessary to obtain the prior knowledge of the straight road or the curve. Since the vehicle is cruising on a straight road or cruising on a curve with a constant steering angle, the lateral offset of the dashed lane line can almost be reduced in a short continuous time. It is ignored, but the longitudinal offset is relatively large, so the dotted lane line can be superimposed into a solid lane line in several consecutive binary images of prominent highway lane lines at different times, and then use the above-mentioned solid lane line identification method All edge pixel positions of the dashed lane line can be obtained.

Since the longitudinal offset of the dashed lane line is affected by the speed of the subject vehicle, when identifying the first dashed lane line, the binary value of the continuous protruding highway lane line at different times can be dynamically determined according to the vehicle speed obtained from the wheel speed sensor. The minimum number of images is to superimpose the first dashed lane line into a solid line lane line, so as to obtain all edge pixel positions of the first dashed line lane line.

Optionally, please refer to FIG. 3 , which is a flowchart of another vehicle control method in an embodiment of the present invention, which may further include step S18 : obtaining the initial straight line mapped to each highway lane line in the second image Slope; Step S19 : mark the vehicle identification range created by the highway lane lines corresponding to the two initial straight lines with the largest slope as the current lane, and mark the remaining vehicle identification ranges as non-own lanes. Then step S12 may be to identify the target vehicle in the own lane in the vehicle identification range marked as the own lane, identify the target vehicle in the non-own lane in the vehicle identification range marked as the non-own lane, and identify the target vehicle in the two adjacent vehicle recognition ranges In the combined vehicle identification range, identify the target vehicle changing lanes to obtain the distance information of the target vehicle.

Due to the interleaved mapping relationship between the first image and the second image, the row and column coordinates of each pixel in the first image can be adjusted in equal proportions to determine at least one pixel's row and column coordinates in the second image. At least one pixel position of each edge pixel position of the highway lane line can be determined in the second image, so that the equal-scale adjusted highway lane line is obtained in the second image. In the second image, a vehicle recognition range is created for every two adjacent highway lane lines.

According to the equal-scale highway lane lines obtained in the second image, compare the number of rows and columns occupied by the initial straight line part of each highway lane line to obtain the slope of the initial straight line of the highway lane line. The vehicle identification range created by the highway lane line where the initial straight line of the two highway lane lines is located is marked as the own lane, and the other created vehicle identification ranges are marked as the non-own lane.

After marking the lane, the target vehicle in this lane can be identified in the vehicle recognition range marked as the lane, the target vehicle in the non-own lane can be identified in the vehicle recognition range marked as non-own lane, and the two adjacent vehicles can be identified. The target vehicle for lane change is identified in the vehicle recognition range formed by the range combination.

The manner of identifying the target vehicle is not limited in the embodiment of the present invention, and several possible manners are described below.

The first way:

Since the distance and position of the target vehicle relative to the TOF sensor always change with time, the distance and position of the road surface and the isolation belt relative to the TOF sensor are approximately unchanged with time. Therefore, a temporal differential depth image can be created by using two depth images acquired at different times, so as to identify the position of the target vehicle in the second image, or the distance between the target vehicle and the subject vehicle, and so on.

The second way:

In the second image, that is, the depth image, the light reflected from the back of the same target vehicle and the depth sub-image formed by the TOF sensor contain consistent distance information, so as long as the depth sub-image formed by the target vehicle is identified in The position in the depth image can obtain the distance information of the target vehicle.

The light from the back of the same target vehicle is reflected to the TOF sensor to form a depth sub-image that contains consistent distance information, while the light from the road surface is reflected to the TOF sensor to form a depth sub-image that contains continuously changing distance information, so it contains consistent distance information. The junction of the depth sub-image and the depth sub-image containing continuously changing distance information must form abrupt differences, and the junction of these sudden differences forms the target boundary of the target vehicle in the depth image.

For example, various boundary detection methods such as Canny, Sobel, and Laplace for boundary detection in image processing algorithms can be used to detect the target boundary of the target vehicle.

Further, the vehicle recognition range is determined by all the pixel positions of the lane line, so detecting the target boundary of the target vehicle within the vehicle recognition range will reduce the boundary interference formed by road facilities such as isolation belts, street light poles, and guardrails.

In practical applications, there may be multiple target vehicles. Therefore, the target boundary detected in each vehicle identification range can be projected onto the row coordinate axis of the image, and a one-dimensional search can be performed on the row coordinate axis. Determine the row number and row coordinate range occupied by the longitudinal target boundary of all target vehicles within the vehicle identification range, and determine the column number and row coordinate position occupied by the horizontal target boundary. The target boundary with a small number, the horizontal target boundary refers to the target boundary with a small number of pixel rows and a large number of columns. According to the number of columns and row coordinate positions occupied by all the horizontal target boundaries within the vehicle identification range, find the column coordinate positions of all vertical target boundaries within the vehicle identification range (that is, the column coordinate starting positions and The target boundary of different target vehicles is distinguished according to the principle that the target boundary contains consistent distance information, so as to determine the position and distance information of all target vehicles within the vehicle identification range.

Therefore, by detecting and acquiring the target boundary of the target vehicle, the position of the depth sub-image formed by the target vehicle in the depth image can be uniquely determined, thereby uniquely determining the distance information of the target vehicle.

Of course, the target vehicle may also be identified in other ways, which is not limited in the embodiment of the present invention, as long as the target vehicle can be identified.

Optionally, please refer to FIG. 4 , which is a flowchart of another vehicle control method in an embodiment of the present invention, which may further include step S20 : determining the target vehicle area in the second image by identifying the target vehicle; step S21 : map the target vehicle area to the first image according to the mapping relationship between the first image and the second image, so as to generate a vehicle lamp identification area in the first image; step S22 : identify the target vehicle in the vehicle lamp identification area turn signal. Of course, FIG. 4 shows one execution sequence, and the execution sequence of each step can also be other. For example, step S20-step S22 is executed after step S14, etc., the embodiment of the present invention is for step S20-step S20-step The execution order of S22 is not limited. In this case, step S15 may be to control the motion parameters of the subject vehicle according to the distance information, the relative speed, and the identified turn signals of the target vehicle.

After the target vehicle has been identified, the target vehicle area can be determined in the second image. The target vehicle area, that is, the area where the target vehicle is located in the second image, may be a closed area enclosed by the boundary of the identified target vehicle, or may also be a closed area enclosed by an extension of the identified boundary of the target vehicle, Or it can also be a closed area enclosed by several pixel positions of the target vehicle, and so on. The embodiment of the present invention does not limit what kind of area the target vehicle area is, as long as it is an area including the target vehicle.

Due to the interlaced mapping relationship between the first image and the second image, the row and column coordinates of at least one pixel in the first image can be determined by adjusting the row and column coordinates of each pixel of the target vehicle area in the second image in equal proportions. Referring to FIG. 5 , after the target vehicle area in the second image is mapped to the first image, a vehicle lamp recognition area can be generated at the corresponding position of the first image, since the image of the vehicle lamp of the target vehicle is included in the target vehicle area , the turn signals of the target vehicle can therefore be recognized in the light recognition area generated in the first image.

Optionally, the manner of identifying the turn signal of the target vehicle in the headlight recognition area is not limited in this embodiment of the present invention, and time differential processing may be performed on the headlight recognition areas in the plurality of first images obtained continuously to create a Corresponding to the temporal micro-molecular image of the target vehicle, and then identifying the turn signal of the target vehicle according to the temporal micro-molecular image.

For example, rear turn signals can be identified based on the color, flashing frequency or flashing sequence of the taillights in the light identification area.

The longitudinal and lateral displacements of the target vehicle at the initial stage of lane change are small, which means that the size of the recognition area of the target vehicle's headlights changes little, and only the brightness of the image at the rear turn signal changes greatly due to flickering. Therefore, a temporal micromolecular image of the target vehicle is created by continuously acquiring a plurality of first images at different times, that is, color or luminance images, and performing temporal differentiation processing on the headlight identification area of the target vehicle. The temporal micromolecular image will highlight the continuous flashing taillight sub-image of the target vehicle. Then, the temporal microscopic image can be projected onto the column coordinate axis, and a one-dimensional search can be performed to obtain the starting and ending column coordinate positions of the taillight sub-image of the target vehicle, and these starting and ending column coordinate positions can be projected to the temporal micromolecule image. And find the row coordinate positions of the start and end points of the taillight sub-image, and project the row and column coordinates of the start and end points of the taillight sub-image into the above-mentioned multiple color or brightness images at different times to confirm the target vehicle. The color, flashing frequency, or flashing sequence of the taillights determines the row and column coordinate positions of the flashing taillight sub-images.

Further, when the row and column coordinates of the flashing taillight sub-image are only on the left side of the headlight recognition area of the target vehicle, it can be determined that the target vehicle is turning on the left turn signal, and the row and column coordinates of the flashing taillight sub-image can be determined. Only when the position is on the right side of the light recognition area of the target vehicle, it can be determined that the target vehicle is turning on the right turn signal, and the row and column coordinates of the flashing rear light sub-image are on both sides of the light recognition area of the target vehicle. Make sure that the target vehicle is flashing double flashing warning lights.

In addition, when the target vehicle changes lanes in the process of its large longitudinal displacement or lateral displacement, the size of the light recognition area of the target vehicle changes greatly. The recognition area is compensated for longitudinal displacement or lateral displacement and scaled into a light recognition area of the same size, and then time differential processing is performed on the adjusted light recognition area of the target vehicle to create a time micro-molecular image of the target vehicle. The micro-molecule image is projected to the column coordinate axis, and a one-dimensional search is performed to obtain the starting and ending column coordinate positions of the taillight sub-image of the target vehicle, and these starting and ending column coordinate positions are projected to the time-differentiated headlight recognition area sub-image and then obtained. Find the starting and ending row coordinate positions of the taillight sub-image, and project the row and column coordinate positions of the starting and ending point of the taillight sub-image into the above-mentioned multiple color or brightness images at different times to confirm the target vehicle The color, flashing frequency or flashing sequence of the taillights determines the row and column coordinate positions of the flashing taillight sub-images, and finally completes the identification of the left turn signal, right turn signal or double flashing warning light.

For example, as shown in Figure 6, the temporal micro-molecular image corresponding to the recognition area of the headlight, in this temporal micro-molecular image, there are continuously flashing rear light sub-images, and by identifying the coordinates, it is determined that the rear light sub-image is located in the headlight recognition area. To the left of the area, the flashing frequency is 1 time per second, so for example, it can be determined that the target vehicle is currently turning on the left turn signal.

In the above manner, the turn signal of the target vehicle can be better identified, so as to know in advance whether the target vehicle is going to turn and how to turn, so that adaptive cruise can be performed better and more safely.

Optionally, how to obtain the azimuth of the target vehicle according to the first image or the second image is not limited in this embodiment of the present invention. For example, the azimuth of the target vehicle may be obtained according to the position of the target vehicle area in the second image. or, obtaining the azimuth angle of the target vehicle according to the position of the identification area of the headlight in the first image.

Since the lens parameters and installation position of the camera that obtains the first image or the second image can be obtained through the camera calibration technology in advance, the relationship between the coordinates of the road scene with the camera as the origin and the pixel coordinates of the first image or the second image can be established lookup table.

Through the above relationship lookup table, the pixel coordinates contained in the target vehicle range or the headlight recognition area can be converted into the target vehicle coordinates with the camera as the origin, so as to calculate the target vehicle with the camera as the origin according to the converted coordinates of the target vehicle with the camera as the origin Azimuth.

When there is a relative speed between the target vehicle and the main vehicle, the reflected signal of the target vehicle received by the constant carrier frequency radar can pass through the phase shifter to generate a quadrature reflected signal, and the quadrature reflected signal and the transmitted signal of the constant carrier frequency radar pass through The mixer generates a quadrature intermediate frequency signal, the quadrature intermediate frequency signal contains the Doppler frequency about the relative velocity, the magnitude of the Doppler frequency is proportional to the magnitude of the relative velocity, the sign of the Doppler frequency Identical to the sign of the relative velocity.

The spectrum of the quadrature intermediate frequency signal highlighting the Doppler frequency can be created by using an analog-to-digital converter and a complex fast Fourier algorithm; the Doppler frequency of the spectrum of the quadrature intermediate frequency signal can be obtained by using a peak detection algorithm The size and sign; according to the obtained size and sign of the Doppler frequency, the size and sign of the relative velocity can be determined by using the Doppler velocity measurement formula.

The constant carrier frequency radar can contain more than two receivers to obtain the azimuth angle of the radar target. The mutual position difference of each receiver of the constant carrier frequency radar causes the phase difference of the quadrature intermediate frequency signal acquired by each receiver at the same Doppler frequency.

According to the phase difference of each receiver at the same Doppler frequency and the mutual position relationship of each receiver obtained from the spectrum of the quadrature intermediate frequency signal, the azimuth angle of the radar target can be obtained by using the phase method angle measurement formula. That is, the relative speed and azimuth angle of the target sensed by the constant carrier frequency radar can be known by the intermediate frequency signal obtained by the constant carrier frequency radar.

When there are multiple target vehicles, the azimuth angles of the multiple target vehicles can be obtained through step S13, and the relative speeds and azimuth angles of the multiple radar targets can be obtained according to the intermediate frequency signal of the constant carrier frequency radar. Using the azimuth angle of a single target vehicle and the The principle that the azimuth angles of a certain radar target are approximately equal can determine the relative speed of the radar target as the relative speed of the target vehicle.

When the installation positions of the camera and the constant carrier frequency radar are far apart, the above principle of approximately equal azimuth angle may lead to errors, as long as the azimuth angle coordinates of the different origins of the two are calibrated according to the installation position relationship between the camera and the constant carrier frequency radar. The azimuth coordinates of the same origin can eliminate the above errors.

After obtaining the distance information and relative speed of the target vehicle, during the adaptive cruise process, the motion parameters of the subject vehicle can be controlled according to the obtained information. The embodiment of the present invention does not limit how to control. For example, it is recognized that the target vehicle is driving at a speed of -10 m/s relative to the subject vehicle at a position 100 meters in front of the subject vehicle, then in order to prevent a rear-end collision, the subject vehicle can be controlled to slow down, etc.,

Of course, if the turn signal of the target vehicle is also identified, during the adaptive cruise process, the motion parameters of the subject vehicle can also be controlled according to the distance information, relative speed, and turn signal of the target vehicle. For example, if it is recognized that the target vehicle is located in the lane to the left of the subject vehicle, driving at a speed of -10 m/s relative to the subject vehicle, and the right turn signal is turned on at the same time, it can be considered that the target vehicle may change to the lane of the subject vehicle, Therefore, the subject vehicle can be controlled to decelerate, and so on.

Optionally, the constant carrier frequency radar can also be automatically calibrated according to the azimuth of the identified target vehicle.

Since the installation position of the constant carrier frequency radar is outside the cab, the measurement results of its azimuth angle may be affected by vibration, temperature changes, rain, snow, mud and dirt coverage, and automatic calibration is required. For example, when multiple target vehicles with different azimuth angles are identified in front of the subject vehicle according to the present invention, it is possible to compare whether the azimuth angles of the identified multiple target vehicles and the azimuth angle of the radar target have consistent deviations, and if there is a consistent deviation , record the deviation into the storage of the constant carrier frequency radar, and the constant carrier frequency radar reads out the deviation in the subsequent azimuth measurement for automatic calibration and compensation. Of course, if there are inconsistent deviations, a warning that the constant carrier frequency radar is unavailable can be issued to the driver of the main vehicle, and the driver of the main vehicle can be reminded to check or clean the constant carrier frequency radar.

Referring to FIG. 7, based on the same inventive concept, an embodiment of the present invention provides a vehicle identification device 100. The device 100 may include:

an image acquisition module 101, configured to acquire a first image and a second image, wherein the first image is a color image or a luminance image, and the second image is a depth image;

a first identification module 102, configured to identify the target vehicle in the second image to obtain distance information of the target vehicle;

a first acquisition module 103, configured to acquire the azimuth angle of the target vehicle according to the first image or the second image;

The first determination module 104 is configured to determine the relative speed of the target vehicle according to the azimuth angle of the target vehicle and the intermediate frequency signal obtained by the constant carrier frequency radar;

The control module 105 is configured to control the motion parameters of the subject vehicle according to the distance information and the relative speed.

Optionally, the apparatus 100 further includes:

a second identification module, configured to identify highway lane lines according to the first image;

The first mapping module is configured to map the road lane line to the second image according to the mapping relationship between the first image and the second image, so as to determine at least one vehicle identification range in the second image, wherein every two phases are Adjacent highway lane lines create a vehicle identification range;

The first identification module 102 is used for:

A target vehicle is identified in at least one vehicle identification range.

Optionally, the apparatus 100 further includes:

a second acquiring module, configured to acquire the slope of the initial straight line mapped to each highway lane line in the second image;

The creation module is used to mark the vehicle identification range created by the highway lane lines corresponding to the two initial straight lines with the largest slopes as this lane, and mark the rest of the vehicle identification ranges as non-own lanes;

The first identification module 102 is used for:

Recognition of the target vehicle in the own lane in the vehicle recognition range marked as the own lane, recognition of the target vehicle in the non-own lane in the vehicle recognition range marked as the non-own lane, and vehicle recognition in the combination of two adjacent vehicle recognition ranges Identify the target vehicle changing lanes in range.

Optionally, the apparatus 100 further includes:

a second determining module, configured to determine the target vehicle area in the second image by identifying the target vehicle;

a second mapping module, configured to map the target vehicle area to the first image according to the mapping relationship between the first image and the second image, so as to generate a vehicle lamp recognition area in the first image;

The third identification module is used to identify the turn signal of the target vehicle in the headlight identification area;

The control module 105 is used to:

The motion parameters of the subject vehicle are controlled according to the distance information, relative speed, and turn signals of the identified target vehicle.

Optionally, the first obtaining module 103 is used for:

Obtain the azimuth angle of the target vehicle according to the position of the target vehicle area in the second image; or,

The azimuth angle of the target vehicle is obtained according to the position of the identification area of the headlight in the first image.

Optionally, the apparatus 100 further includes:

The calibration module is used to automatically calibrate the constant carrier frequency radar according to the azimuth angle of the identified target vehicle.

Referring to FIG. 8, based on the same inventive concept, an embodiment of the present invention provides a vehicle 200. The vehicle 200 may include an image acquisition device for acquiring a first image and a second image, wherein the first image is a color image or a luminance image , the second image is a depth image; and, the vehicle identification device 100 of FIG. 7 .

In the embodiments provided by the present invention, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the device embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be Incorporation may either be integrated into another system, or some features may be omitted, or not implemented.

Each functional module in each embodiment of the present application may be integrated in one processing unit, or each module may exist physically alone, or two or more modules may be integrated in one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

The integrated unit, if implemented in the form of a software functional unit and sold or used as an independent product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application can be embodied in the form of software products in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, and the computer software products are stored in a storage medium , including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, ROM (Read-Only Memory, read-only memory), RAM (Random Access Memory, random access memory), magnetic disk or optical disk and other media that can store program codes .

As mentioned above, the above embodiments are only used to describe the technical solutions of the present invention in detail, but the descriptions of the above embodiments are only used to help understand the method and the core idea of the present invention, and should not be construed as a limitation of the present invention. Those skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should all be included within the protection scope of the present invention.

Claims (7)

1. a vehicle control method, is characterized in that, comprises: acquiring a first image and a second image, wherein the first image is a color image or a luminance image, and the second image is a depth image; Identifying a target vehicle in the second image to obtain distance information of the target vehicle; obtaining the azimuth angle of the target vehicle according to the first image or the second image; Determine the relative speed of the target vehicle according to the azimuth angle of the target vehicle and the intermediate frequency signal obtained by the constant carrier frequency radar; determining a target vehicle area in the second image by identifying the target vehicle; mapping the target vehicle area to the first image according to the mapping relationship between the first image and the second image, so as to generate a vehicle lamp recognition area in the first image; identifying the turn signal of the target vehicle in the vehicle lamp identification area; controlling the motion parameters of the subject vehicle according to the distance information, the relative speed, and the identified turn signals of the target vehicle; Perform automatic calibration on the constant carrier frequency radar according to the identified azimuth of the target vehicle; Obtaining the azimuth angle of the target vehicle according to the first image or the second image includes: Obtain the azimuth angle of the target vehicle according to the position of the target vehicle area in the second image; or obtain the azimuth angle of the target vehicle according to the position of the lamp recognition area in the first image Azimuth. 2. The method according to claim 1, wherein the method further comprises: Identify highway lane lines according to the first image; According to the mapping relationship between the first image and the second image, the highway lane line is mapped to the second image, so as to determine at least one vehicle identification range in the second image, wherein each Two adjacent highway lane lines create a vehicle identification range; Identifying the target vehicle in the second image includes: A target vehicle is identified in the at least one vehicle identification range. 3. The method according to claim 2, wherein the method further comprises: obtaining the slope of the initial straight line mapped to each highway lane line in the second image; Mark the vehicle identification range created by the highway lane lines corresponding to the two initial straight lines with the largest slopes as this lane, and mark the rest of the vehicle identification ranges as non-own lanes; Identifying a target vehicle in the at least one vehicle identification range includes: Recognition of the target vehicle in the own lane in the vehicle recognition range marked as the own lane, recognition of the target vehicle in the non-own lane in the vehicle recognition range marked as the non-own lane, and vehicle recognition in the combination of two adjacent vehicle recognition ranges Identify the target vehicle changing lanes in range. 4. A vehicle control device, comprising: an image acquisition module, configured to acquire a first image and a second image, wherein the first image is a color image or a luminance image, and the second image is a depth image; a first identification module for identifying a target vehicle in the second image to obtain distance information of the target vehicle; a first acquisition module, configured to acquire the azimuth angle of the target vehicle according to the first image or the second image; a first determining module, configured to determine the relative speed of the target vehicle according to the azimuth angle of the target vehicle and the intermediate frequency signal obtained by the constant carrier frequency radar; a second determination module, configured to determine a target vehicle area in the second image by identifying the target vehicle; A second mapping module, configured to map the target vehicle area to the first image according to the mapping relationship between the first image and the second image, so as to generate a vehicle in the first image Light identification area; a third identification module, configured to identify the turn signal of the target vehicle in the vehicle lamp identification area; a control module, configured to control the motion parameters of the subject vehicle according to the distance information, the relative speed, and the identified turn signal of the target vehicle; a calibration module for automatically calibrating the constant carrier frequency radar according to the identified azimuth of the target vehicle; The first obtaining module is specifically configured to obtain the azimuth angle of the target vehicle according to the position of the target vehicle area in the second image; or, according to the identification area of the vehicle lights in the first image to obtain the azimuth of the target vehicle. 5. The apparatus according to claim 4, wherein the apparatus further comprises: a second identification module, configured to identify highway lane lines according to the first image; a first mapping module, configured to map the highway lane lines to the second image according to the mapping relationship between the first image and the second image, so as to determine at least one vehicle recognition range, where every two adjacent highway lane lines create a vehicle recognition range; The first identification module is used for: The target vehicle is identified in the at least one vehicle identification range. 6. The apparatus according to claim 5, wherein the apparatus further comprises: a second acquiring module, configured to acquire the slope of the initial straight line mapped to each highway lane line in the second image; The creation module is used to mark the vehicle identification range created by the highway lane lines corresponding to the two initial straight lines with the largest slopes as this lane, and mark the rest of the vehicle identification ranges as non-own lanes; The first identification module is used for: Recognition of the target vehicle in the own lane in the vehicle recognition range marked as the own lane, recognition of the target vehicle in the non-own lane in the vehicle recognition range marked as the non-own lane, and vehicle recognition in the combination of two adjacent vehicle recognition ranges Identify the target vehicle changing lanes in range. 7. A vehicle, characterized in that, comprising: The vehicle control device according to any one of claims 4-6.

Images