JP7084223B2 - Image processing equipment and vehicle lighting equipment - Google Patents

Description

The present invention relates to an image processing device used in an automobile or the like.

In recent years, various attempts have been made to discriminate the surrounding environment and objects based on the surrounding information acquired by cameras and sensors mounted on the vehicle, and to control the vehicle according to the environment and objects. Examples of vehicle control include various controls such as braking control, drive control, operation control, and light distribution control.

For example, when controlling the light distribution according to the situation in front of the vehicle, it is difficult to perform accurate light distribution control unless the road shape, which is one of the environments in which the vehicle travels, is known. Therefore, various methods for estimating the road shape itself have been devised. For example, a partition line recognition device capable of estimating the other white line with high accuracy based on one white line detected from the acquired image has been devised (see Patent Document 1).

Japanese Patent Application Laid-Open No. 2003-44836

In the light distribution control, it is relatively easy to determine the attributes of the light spots (light emitters such as vehicles and street lights, and reflectors such as delineators and signs) in the vicinity of the own vehicle from the acquired image. However, in the acquired image, the distant light spots tend to be dark, and since the light spots are often close to each other, it is difficult to determine the attributes of the distant light spots. Therefore, if a high-precision camera or an arithmetic unit having high image processing capability is used, it is relatively easy to determine the attributes of distant light spots, but the cost of the entire light distribution control system is increased.

The present invention has been made in view of such a situation, and an object of the present invention is to provide a new technique for discriminating the situation in front of a vehicle by a relatively simple method.

In order to solve the above-mentioned problems, the image processing apparatus according to an embodiment of the present invention defines a region including a road outer line from an image captured in front of the vehicle, and estimates the road shape based on the shape of the region. It includes a shape estimation unit, a reference point calculation unit that calculates a distant reference point based on an intersection of a straight line extending a diagonal line of a region and a horizontal line in an captured image.

According to this aspect, a distant reference point, for example, a vanishing point can be calculated by using the shape of the region where the road shape is estimated. Therefore, the situation in front of the vehicle can be determined by a relatively simple method from the captured image obtained by capturing the front of the vehicle.

The road shape estimation unit may define a rectangular area as an area and estimate the road shape based on the information correlated with the aspect ratio of the rectangular area. As a result, the road shape can be estimated based on the shape of the rectangular region including the outer line of the road.

A discriminant unit for discriminating the situation in front of the vehicle using a distant reference point may be further provided. The discriminating unit may discriminate the light spot that moves diagonally downward from the distant reference point as an oncoming vehicle. As a result, the oncoming vehicle in front of the vehicle can be easily identified.

A discriminant unit for discriminating the situation in front of the vehicle using a distant reference point may be further provided. The discriminating unit may determine that the light spot that remains between the distant reference point and the own vehicle is the preceding vehicle. As a result, the preceding vehicle in front of the vehicle can be easily identified.

An image processing device, a headlight unit that illuminates the front of the vehicle, and a light distribution control unit that controls the light distribution of the headlight unit according to the attribute of the light spot determined by the image processing device may be provided. .. As a result, appropriate light distribution can be realized for the oncoming vehicle and the preceding vehicle in front of the vehicle.

It should be noted that any combination of the above components and the conversion of the expression of the present invention between methods, devices, systems and the like are also effective as aspects of the present invention.

According to the present invention, the situation in front of the vehicle can be determined by a relatively simple method.

It is a schematic diagram which shows the appearance of the vehicle to which the lamp for vehicle which concerns on this embodiment is applied. It is a block diagram which shows the schematic structure of the lamp for vehicles which concerns on this embodiment. It is a flowchart which shows the light distribution control method including the situation determination processing in front of a vehicle which concerns on this embodiment. It is a flowchart which shows the detail of step S10 shown in FIG. FIG. 5A schematically shows image data when the road shape in front of the vehicle is straight, and FIG. 5B schematically shows image data when the road shape in front of the vehicle is a left curve. FIG. 5 (c) is a diagram schematically showing image data when the road shape in front of the vehicle is a right curve. It is a schematic diagram which shows the situation in which a large number of light emitting objects exist on the road of a left curve at night seen from the forward surveillance camera. FIG. 7A is a diagram schematically showing image data when the road shape in front of the vehicle is a straight line with no slope, and FIG. 7B is an image when the road shape in front of the vehicle is a straight line uphill. FIG. 7C, which schematically shows the data, is a diagram schematically showing the image data when the road shape in front of the vehicle is a downhill.

Hereinafter, the present invention will be described with reference to the drawings based on the embodiments. The same or equivalent components, members, and processes shown in the drawings shall be designated by the same reference numerals, and duplicate description thereof will be omitted as appropriate. Further, the embodiment is not limited to the invention but is an example, and all the features and combinations thereof described in the embodiment are not necessarily essential to the invention.

(Vehicle lamps)
FIG. 1 is a schematic view showing the appearance of a vehicle to which a vehicle lamp according to the present embodiment is applied. As shown in FIG. 1, the vehicle 10 according to the present embodiment includes a headlight unit 12 that illuminates the front of the vehicle, a control system 14 that controls light irradiation by the headlight unit 12, and a traveling vehicle 10. Various sensors that detect information indicating the situation and output the detection signal to the control system 14, a front monitoring camera 16 that monitors the front of the vehicle, and an antenna that receives the orbit signal from the GPS satellite and outputs it to the control system 14. 18 and.

As various sensors, for example, a steering sensor 22 for detecting the steering angle of the steering wheel 20, a vehicle speed sensor 24 for detecting the vehicle speed of the vehicle 10, and an illuminance sensor 26 for detecting the illuminance around the own vehicle are provided. These sensors 22, 24, 26 are connected to the above-mentioned control system 14.

In order to use the front surveillance camera 16 as the light distribution control of the headlight unit (headlight), it is required to be able to identify an object in front of the vehicle at night. However, there are various objects in front of the vehicle. There are also objects for which the optimum light distribution control for the own vehicle can be performed without doing so.

In order to realize such light distribution control of the headlight unit, a light emitter such as a front-running vehicle (oncoming vehicle or a preceding vehicle) traveling in front of the own vehicle, a road lighting, or a light such as a delineator. It is preferable to use a camera that detects the reflector. In addition, it is more preferable to have a function of discriminating the situation in front of the vehicle such as the attributes of the light emitting body and the light reflecting body detected as an object and the road shape. Here, the attribute distinguishes, for example, whether the light emitting body or the light reflecting body in front is a vehicle in front or a facility attached to a road. More specifically, if the light emitter or the like is a vehicle, it is a preceding vehicle or an oncoming vehicle, and if the light emitter or the like is a facility attached to a road, it is a road lighting, a delineator, or another light emitting facility. It distinguishes whether it is a traffic signal (for example, store lighting, advertisement, etc.). Regarding the road shape, it is determined whether the road is going straight, right curve, left curve, uphill, or downhill. In addition, the curvature of the curve and the slope of the slope can be discriminated in several stages.

The headlight unit applicable to the present embodiment is not particularly limited as long as it has a configuration in which the light distribution of the emitted light can be changed according to the attributes of an object existing in front. For example, a halogen lamp, a gas discharge headlamp, or a headlamp using a semiconductor light emitting element (LED, LD, EL) can be adopted. In the present embodiment, a headlight unit having a configuration in which a part of the light distribution pattern can be non-irradiated so as not to give glare to the vehicle in front will be described as an example. The configuration in which a part of the light distribution pattern can be non-irradiated includes a configuration in which a shade is driven to partially block the light of the light source and a configuration in which a part of a plurality of light emitting portions is not lit. Is done.

The headlight unit 12 has a pair of left and right headlight units 12R and 12L. The headlight units 12R and 12L have the same configuration as each other except that the internal structure is symmetrical. The low beam lamp unit 28R and the high beam lamp unit 30R are in the lamp housing on the right side, and the low beam is in the lamp housing on the left side. The lamp unit 28L and the high beam lamp unit 30L are arranged respectively.

The control system 14 illuminates a part of the headlight units 12R and 12L, that is, a part of the light distribution pattern, which are mounted on the left and right sides of the front part of the vehicle based on each output of various input sensors. Controls the headlight unit 12 whose light distribution characteristics can be changed.

Next, a vehicle lamp according to the present embodiment will be described. FIG. 2 is a block diagram showing a schematic configuration of a vehicle lamp 110 according to the present embodiment. The vehicle lighting equipment 110 includes a headlight unit 12R, 12L and a control system 14 for controlling the irradiation of light by the headlight unit 12R, 12L. Then, the vehicle lighting tool 110 determines the situation in front of the vehicle such as the attribute of the object existing in front of the vehicle and the road shape in the control system 14, determines the light distribution control condition based on the situation, and determines the light distribution. The irradiation of light by the headlight units 12R and 12L is controlled based on the control conditions.

Therefore, the control system 14 according to the present embodiment is connected to the front surveillance camera 16 for acquiring the captured image of the front of the vehicle including the driver's visual target. Further, a steering sensor 22, a vehicle speed sensor 24, and an illuminance sensor 26 for detecting steering information and vehicle speed, which are referred to when determining the traveling state of the vehicle, are connected.

(Control system)
The control system 14 includes an image processing ECU 32, a light distribution control ECU 34, and a GPS navigation ECU 36. Various ECUs and various in-vehicle sensors are connected by an in-vehicle LAN bus to enable transmission and reception of data. The image processing ECU 32 determines the attributes of the object existing in front based on the data of the captured image acquired by the front surveillance camera 16 and various in-vehicle sensors. The light distribution control ECU 34 determines light distribution control conditions suitable for the driving environment in which the vehicle is placed based on the information of the image processing ECU 32 and various in-vehicle sensors, and transmits the control signals to the headlight units 12R and 12L. Output.

The headlight units 12R and 12L control the light distribution by inputting the control signal output from the light distribution control ECU 34 to the driving device of the optical component and the lighting control circuit of the light source. The forward monitoring camera 16 is a monocular zoom camera equipped with an image sensor such as a CCD or CMOS, and from the image data, road alignment information necessary for driving, road attachment facilities, and information on the existence status and position of oncoming vehicles / preceding vehicles. And so on.

(Situation discrimination process in front of the vehicle)
It is difficult to instantly distinguish the light spots of vehicles (headlamps and tail lamp lights) and other light spots (street lights, reflectors, advertisement displays, store lighting, etc.) from nighttime images. In particular, the light spot of a distant vehicle has a dark brightness and is more difficult to distinguish. Further, in order to reduce undetected light spots in a distant place, it is also an idea to detect a predetermined distant region for performing distant detection and separately perform a light spot discriminating process suitable for the distant region. For that purpose, it is necessary to detect a distant region with high accuracy. However, the position of the distant region changes depending on the shape of the road (whether it is a curve or not, whether it is a slope or not).

Therefore, the inventors of the present application have devised a new image processing device that estimates the road shape and calculates a reference point in a distant region from the road shape. This image processing device calculates a distant region including a distant reference point and a distant reference point by the method shown below, and can perform image processing with a relatively small amount of calculation. The image processing apparatus according to the present embodiment can determine the situation in front of the vehicle such as the type of light spot and the shape of the road by using such a simple image processing.

First, the outline of the situation determination process in front of the vehicle in the image processing ECU according to the present embodiment will be described. In the vehicle front situation determination process according to the present embodiment, a distant reference point is calculated based on the shape of the region including the road outer line included in the captured image of the vehicle front, and the distant reference point is used. It determines the situation in front of the vehicle. Here, the lane outside line is a so-called white line indicating the boundary between the traveling lane and the sidewalk, and when the vehicle is traveling on the left side, it means the white line on the left side of the traveling lane.

FIG. 3 is a flowchart showing a light distribution control method including a situation determination process in front of the vehicle according to the present embodiment. FIG. 4 is a flowchart showing the details of step S10 shown in FIG.

(Estimation of road shape)
The determination of the situation in front of the vehicle is mainly executed by the image processing ECU 32 shown in FIG. 2, and the light distribution control is mainly executed by the light distribution control ECU 34. The image processing ECU 32 according to the present embodiment estimates the road shape corresponding to the light spot based on the feature information such as the shape and position of the road outer line included in the captured image information (S10 in FIG. 3). ..

The road shape estimation shown in step S10 is performed, for example, in the process shown in FIG. First, the image information acquisition unit 38 acquires the data of the captured image obtained by capturing the front of the vehicle with the front surveillance camera 16 (S100), and binarizes the image data (S102). FIG. 5A schematically shows image data when the road shape in front of the vehicle is straight, and FIG. 5B schematically shows image data when the road shape in front of the vehicle is a left curve. FIG. 5 (c) is a diagram schematically showing image data when the road shape in front of the vehicle is a right curve. In the following description, the case where the road shape is immediately before will be described.

Next, the white line detection unit 40 determines whether or not there is a white line (hereinafter, appropriately referred to as “left white line”) drawn on the left side of the traveling lane of the own vehicle from the binarized image data. Detect (S104). At this time, in order to prevent erroneous detection of the left white line, information such as the size, position, and width of the left white line is used as the detection condition. If the left white line is not detected (No in S104), the process is terminated once and the captured image data is acquired again.

On the other hand, for example, when the left white line L1 is detected in the image data shown in FIG. 5A (Yes in S104), the road shape estimation unit 42 defines a region including a part of the left white line (S106). , The road shape is estimated based on the shape of the region (S108). Specifically, a rectangular region R1 including a part of the left white line L1 is defined, and the road shape is estimated based on the aspect ratio A (shape of the rectangular region R1) calculated by vertical Y / horizontal X. For example, the road shape estimation unit 42 estimates that the road shape is straight when the aspect ratio A is 0.8 or more and 1.2 or less.

Further, the road shape estimation unit 42 defines a rectangular region R2 including a part of the left white line L2 in the image data shown in FIG. 5B, and the road is based on the aspect ratio A calculated by vertical Y / horizontal X. Estimate the shape. For example, the road shape estimation unit 42 estimates that the road shape is a left curve when the aspect ratio A is larger than 1.2.

Further, the road shape estimation unit 42 defines a rectangular region R3 including a part of the left white line L3 in the image data shown in FIG. 5 (c), and the road is based on the aspect ratio A calculated by vertical Y / horizontal X. Estimate the shape. For example, the road shape estimation unit 42 estimates that the road shape is a right curve when the aspect ratio A is smaller than 0.8.

In this way, after the road shape is estimated by the flowchart shown in FIG. 4, step S12 shown in FIG. 3 is executed. When the road shape is a straight line, the reference point calculation unit 44 includes a straight line L1'(which substantially coincides with the left white line L1), which is an extension of the diagonal line of the rectangular region R1 shown in FIG. 5 (a), and a horizontal line H1 in the captured image P1. A distant reference point (vanishing point V1) is calculated based on the intersection C1 of the above (S12 in FIG. 4). The horizon according to the present embodiment is, for example, a boundary between the ground and the sky, and is a boundary where shades and colors change rapidly in image data.

Further, when the road shape is a left curve, the reference point calculation unit 44 is based on the intersection C2 of the straight line L2'that extends the diagonal line of the rectangular region R2 shown in FIG. 5B and the horizontal line H2 in the captured image P2. And calculate the distant reference point. The straight line L2'is the diagonal line of the two diagonal lines of the rectangular region R2 that intersects the left white line L2. Since the intersection C2 is shifted toward the center of the image from the vanishing point V2 in the road shape shown in FIG. 5B, for example, the intersection C2 is corrected in consideration of the aspect ratio A that changes depending on the curvature of the curve, and the vanishing point C2 is corrected. V2 may be calculated.

Further, when the road shape is a left curve, the reference point calculation unit 44 is based on the intersection C3 of the straight line L3'that extends the diagonal line of the rectangular region R3 shown in FIG. 5 (c) and the horizontal line H3 in the captured image P3. And calculate the distant reference point. Since the intersection C3 is shifted toward the center of the image from the vanishing point V3 in the road shape shown in FIG. 5C, the vanishing point C3 is corrected in consideration of the aspect ratio A that changes depending on the curvature of the curve, for example. V3 may be calculated.

As described above, the image processing ECU 32 according to the present embodiment can calculate a distant reference point, for example, a vanishing point by using the shape of the region where the road shape is estimated. Therefore, the situation in front of the vehicle can be determined by a relatively simple method from the captured image obtained by capturing the front of the vehicle.

When the distant reference point (or vanishing point) is calculated by the reference point calculation unit 44, the image processing ECU 32 determines the situation in front of the vehicle using the distant reference point (S14 in FIG. 3). For example, when the distant reference point is substantially in the center region of the image, the left region of the image, and the right region of the image, it is determined that the road on which the own vehicle travels is a straight line, a left curve, or a right curve.

Next, the determination of the preceding vehicle and the oncoming vehicle traveling in front of the own vehicle will be described. FIG. 6 is a schematic diagram showing a situation in which a large number of light emitting objects exist on a road with a left curve at night as seen from a forward surveillance camera.

When the own vehicle travels, the light spots L10 and L11 indicating the oncoming vehicle 50 shown in FIG. 6 move diagonally downward from the distant reference point (vanishing point V2). Therefore, the discriminating unit 48 determines that the light spot that moves diagonally downward from the distant reference point is an oncoming vehicle. As a result, the oncoming vehicle 50 in front of the vehicle can be easily identified. At that time, the image processing ECU 32 may determine the oncoming vehicle in consideration of conditions such as the brightness, area, and color of the light spot (headlamp).

Further, when the own vehicle travels, the intersections L12 and L13 indicating the preceding vehicle 52 shown in FIG. 6 move closer to or further away from the distant reference point and the own vehicle. Therefore, the discriminating unit 48 determines that the light spot that remains between the distant reference point and the own vehicle is the preceding vehicle 52. As a result, the preceding vehicle in front of the vehicle can be easily identified. At that time, the image processing ECU 32 may determine the preceding vehicle in consideration of conditions such as the brightness, area, and color of the light spot (tail lamp).

(Light distribution control)
The vehicle lamp 110 according to the present embodiment includes an image processing ECU 32 and a light distribution control ECU 34 that controls the light distribution of the headlight unit 12 according to the attribute of the light spot determined by the image processing ECU 32. There is. As a result, appropriate light distribution can be realized for the oncoming vehicle and the preceding vehicle in front of the vehicle.

Specifically, the image processing ECU 32 is based on characteristic information such as the position, movement, size, brightness, and locus of a plurality of light spots included in the captured image information, and the attributes of the object corresponding to the light spots. To determine. The light distribution control ECU 34 targets the light spot determined by the image processing ECU 32 as an object that requires light distribution control in consideration of glare, such as an oncoming vehicle or a preceding vehicle, as the target of the light distribution control of the headlight unit 12. The headlight unit 12 determines that the light spot determined by the image processing ECU 32 to be an object other than the vehicle in front (road lighting, delineator, store lighting, advertisement, or other light emitting body attached to the road) that does not require light distribution control in consideration of glare. Is excluded from the target of light distribution control (S16 in FIG. 3).

That is, the light distribution control ECU 34 controls the headlight unit 12 so as to irradiate a range including a light spot determined that the vehicle is not a vehicle that needs to consider the influence of glare, thereby visually recognizing the front of the vehicle. It is possible to control the light distribution to improve the performance. As described above, the vehicle lamp 110 according to the present embodiment enables appropriate light distribution control according to the attributes of the object in front of the vehicle without imposing a special operational burden on the driver.

(Estimation of slope)
The image processing ECU 32 described above estimates whether or not the road shape is a curve, but it is also possible to estimate whether or not the road shape is a slope.

FIG. 7A is a diagram schematically showing image data when the road shape in front of the vehicle is a straight line with no slope, and FIG. 7B is an image when the road shape in front of the vehicle is a straight line uphill. FIG. 7C, which schematically shows the data, is a diagram schematically showing the image data when the road shape in front of the vehicle is a downhill.

The image processing ECU 32 estimates whether or not the road is a slope by using the detected left white line in the same manner as when estimating whether or not the road shape is a curve. When a straight left white line L4 is detected in the image data shown in FIG. 7A, a rectangular area R4 including a part of the left white line L4 is defined, and an aspect ratio A (rectangular) calculated by vertical Y / horizontal X is defined. The road shape is estimated based on the shape of the region R4). For example, the road shape estimation unit 42 estimates that the road shape is flat without a gradient when the left white line is a straight line and the aspect ratio A is 0.8 or more and 1.0 or less.

Further, the road shape estimation unit 42 defines a rectangular region R5 including a part of the straight left white line L5 in the image data shown in FIG. 7B, and is based on the aspect ratio A calculated by vertical Y / horizontal X. To estimate the road shape. For example, the road shape estimation unit 42 estimates that the road shape is an uphill when the aspect ratio A is larger than 1.0.

Further, the road shape estimation unit 42 defines a rectangular region R6 including a part of the straight left white line L6 in the image data shown in FIG. 7 (c), and is based on the aspect ratio A calculated by vertical Y / horizontal X. To estimate the road shape. For example, the road shape estimation unit 42 estimates that the road shape is a downhill when the aspect ratio A is smaller than 0.8.

Conventionally, it is possible to estimate the curvature of the road and the slope of the slope based on the yaw rate using the gyro sensor and the inclination of the vehicle using the acceleration sensor, but in both cases, after the own vehicle enters a curve or slope. Otherwise, accurate estimation was impossible. However, the image processing ECU 32 according to the present embodiment estimates whether or not the vehicle is a curve or a slope before entering a curve or a slope where the posture of the vehicle has not changed without using a sensor. can.

Although the present invention has been described above with reference to the above-described embodiment, the present invention is not limited to the above-described embodiment, and the present invention is not limited to the above-described embodiment, and the present invention may be a combination or substitution of the configurations of the embodiments as appropriate. It is included in the present invention. Further, it is also possible to appropriately rearrange the combinations and the order of processing in the embodiment based on the knowledge of those skilled in the art, and to add modifications such as various design changes to the embodiments, and such modifications are added. The embodiments described above may also be included in the scope of the present invention.

C1 intersection, H1 horizontal line, L1 left white line, L1'straight line, R1 rectangular area, C2 intersection, H2 horizontal line, L2 left white line, L2'straight line, R2 rectangular area, C3 intersection, H3 horizontal line, L3 left white line, L3'straight line, R3 rectangular area, 10 vehicles, 12 headlight unit, 14 control system, 16 forward monitoring camera, 32 image processing ECU, 34 light distribution control ECU, 38 image information acquisition unit, 40 white line detection unit, 42 road shape estimation unit, 44 Reference point calculation unit, 48 discrimination unit, 50 oncoming vehicle, 52 preceding vehicle, 110 vehicle lighting equipment.

Claims (5)

A road shape estimation unit that determines a region including the road outer line from an image captured in front of the vehicle and estimates the road shape based on the shape of the region.
A reference point calculation unit that calculates a distant reference point based on the intersection of a straight line extending the diagonal line of the region and a horizontal line in a captured image.
An image processing device comprising. The image processing apparatus according to claim 1, wherein the road shape estimation unit defines a rectangular area as the area and estimates the road shape based on information correlating with the aspect ratio of the rectangular area. Further provided with a discriminating unit for discriminating the situation in front of the vehicle using the distant reference point.
The image processing apparatus according to claim 1 or 2, wherein the discrimination unit discriminates a light spot that moves diagonally downward from the distant reference point as an oncoming vehicle. Further provided with a discriminating unit for discriminating the situation in front of the vehicle using the distant reference point.
The image processing device according to claim 1 or 2, wherein the discriminating unit discriminates a light spot that remains between the distant reference point and the own vehicle as a preceding vehicle. The image processing apparatus according to claim 3 or 4,
A headlight unit that illuminates the front of the vehicle and
A light distribution control unit that controls the light distribution of the headlight unit according to the attribute of the light spot determined by the image processing device.
A vehicle lamp that is characterized by being equipped.

Images