JP2003067752A - Vehicle periphery monitoring device - Google Patents

Abstract

(57) [Problem] To provide a vehicle periphery monitoring device capable of immediately and easily calculating an optical flow of an entire image and monitoring a situation around the vehicle based on the calculated flow. . SOLUTION: A continuous image acquiring unit 2 for photographing a surrounding image of a vehicle, a traveling lane recognizing unit 3 for recognizing a traveling lane of an own vehicle based on the photographed image, and traveling obtained by the traveling lane recognizing unit 3. An optical flow calculator that calculates an optical flow of a moving object existing around the own vehicle based on the lane data and the speed data of the vehicle, and extracts the moving object based on the optical flow. Then, the optical flow calculation unit 5 determines the luminance value of the desired point in the image at the time of the current frame shooting and the luminance value at a point corresponding to the desired point in the image of the next frame shooting, and The optical flow is calculated using an energy functional created on condition that the speed change is smooth.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a vehicular surveillance device for photographing a video image around a vehicle and notifying a driver of the photographed information.

[0002]

2. Description of the Related Art In recent years, a vehicle periphery monitoring device has been proposed and put into practical use for the purpose of recognizing a vehicle ahead or a road ahead of the vehicle when the vehicle is traveling. By using such a vehicle periphery monitoring device, it is possible to recognize the driving lanes (white lines or yellow lines) such as the partition lines on the left and right sides of the road, the center line, and the like, and thus to recognize the positional relationship between the vehicle and the driving lane. In addition, since it is possible to recognize the inter-vehicle distance to the vehicle ahead, it is possible to improve the safety when the vehicle is traveling.

In such a vehicle periphery monitoring device, an image of the front of the vehicle is captured by a camera, and image processing technology is used.
The driving lane included in the captured image is extracted, the partition line,
Or it recognizes the center line. Furthermore, the image of the vehicle ahead is extracted.

An optical flow analysis method is known as one of moving image processing. The optical flow expresses the movement of the target object by a vector, and the most typical expression is to display the movement direction of a certain point and its velocity at the same time with the length of the arrow. is there.

As a method for detecting an optical flow, conventionally known methods include a correlation method, a concentration gradient method, a Fourier transform method, a method using a spatiotemporal filter, and a spatiotemporal correlation method. Here, among the above five optical flow detection methods, the correlation method and the concentration gradient method, which have reached the stage of practical application, will be described.

<Correlation method> One image is divided into small areas,
A correlation method that finds a similar distribution of brightness in other images is used in moving images. Specifically, the image is divided into windows of a certain size, and 2 at consecutive times.
For two images gi-1 (x, y) and gi (x, y), g
The movement amount (ξ, η) is determined by shifting i (x, y) and obtaining the minimum value by the following equation (1) or equation (2).

MinΣΣ {gi (x−ξ, y−η) −gi−1 (x, y)} ² (1) minΣΣ | gi (x−ξ, y−η) −gi−1 (x , Y) | (2) The correlation method has a drawback in that it is weak against changes in the shape and brightness distribution, so it is difficult to apply it to objects with a rapid change rate such that these change greatly between screens. . However, when two images are input for a very short time, the change between adjacent images is not so large in many cases, and the correlation is easily processed by hardware, so it is often used practically. Has been.

A feature matching method is also used in which feature points are found on each screen from two images and correspondence is made by the matching method. Edges, corners of objects, and the like are often used as the feature points. When the characteristic points are obtained, next, the association is performed. At this time, there is no strong constraint condition as in the case of multiview.

However, assuming that the object is a rigid body, it is considered that similar movements occur at points in the vicinity of a certain point. Therefore, the correspondence of the points of interest should be restricted from the correspondence of the points in the vicinity. You can For example, since it is considered that a plurality of feature points move in the same manner, it is possible to select a point having a similar movement from several candidates of corresponding points as a candidate of corresponding points.

In such a correlation method, there is a problem that the calculation cost is high because the number of combinations of the processing for obtaining the correlation of the movements of a plurality of characteristic points and searching for the correspondence becomes huge.

<Density Gradient Method> In the method using the correlation method described above, when applied to a moving image, only characteristic points can be dealt with. Therefore, the space-time differential method is known as a method for obtaining a more dense correspondence between points. This is a method that makes the best use of the characteristics of moving images. In moving images, the fact that "the gradient of spatial brightness and the temporal change of brightness are related to each other" is used.

That is, the x and y components of the movement vector between the two screens at a certain point (x, y) are (u, v), the spatial brightness gradient is (Ex, Ey), and the two screens are Letting Et be the change in the brightness of, the following expression (3) is established.

Ex · u + Ey · v + Et = 0 (3) Here, Ex, Ey, and Et can be calculated from the image, and u and v are numerical values desired to be obtained. This equation was discovered in a study of band compression of television images, and theoretically can be derived as follows.

It is now assumed that the brightness of the point (x, y) on the image at time t is E (x, y, t) and the object has moved by Δx, Δy after a minute time Δt. Here, assuming that the brightness of the point on the object does not change after the movement,
The following expression (4) is established.

E (x, y, t) = E (x + Δx, y + Δy, t + Δt) (4) Here, if the right side of the equation (4) is Taylor-expanded, the following equation (5) can be obtained. it can.

[0016]

[Equation 2] Here, e is a higher-order term of Δx, Δy, and Δt, but if this is ignored and both sides are divided by Δt, and Δt approaches 0, the following equation (6) can be obtained. .

[0017]

[Equation 3] Then, the expression (3) can be derived from the expression (6).

The equation (3) gives one constraint condition on (u, v), but the movement vectors u, v cannot be immediately obtained from this formula, and further constraint conditions are required.

As the constraint condition, the global optimization method (Global Optimization) using the assumption that "the optical flow field changes smoothly on the screen" or "the optical flow field is uniform in a local small area" is used. Local optimization using the assumption "Are"
Etc. are known.

This method has an advantage that it is possible to measure the motion to an accuracy of one pixel or less over the entire image by a relatively simple calculation without performing a correspondence search which requires a huge calculation cost. However, it is necessary to satisfy the assumption that the change in the luminance of the image is smooth temporally and spatially, and if this is not the case, the detection error becomes large.

[0021]

Now, let us consider a case where the optical flow in the entire image is calculated in order to detect the stable characteristics of the forward vehicle. When the optical flow is calculated for all points of the image using the correlation method in which a correlation window is set around the point of interest and the correlation value of the luminance within the correlation window between frames is calculated, the amount of calculation becomes enormous. , Not practical.

Further, when the density gradient method is used, the optical flow of the entire image can be calculated by a relatively simple calculation, but the brightness change of the image is temporally and spatially changed as in the front image of the vehicle. It is not easy to meet the constraint condition of the density gradient method under the condition that the assumption that it is smooth is difficult to apply and the object in the image makes a complicated motion.

Therefore, the image is roughly smoothed,
There is a problem that a good optical flow cannot be calculated unless an appropriate processing such as detecting edge points of an image in which continuity of brightness change is difficult to be maintained and adding various constraint conditions in the vicinity thereof is added. Occurs.

The present invention has been made in order to solve such a conventional problem, and its object is to quickly calculate the optical flow of the entire image of a video image in front of the vehicle, Based on the above, it is to provide a vehicle periphery monitoring device capable of monitoring the situation around the vehicle.

[0025]

In order to achieve the above object, the invention according to claim 1 of the present application sequentially captures peripheral images while the vehicle is traveling, and monitors the condition around the vehicle based on the captured images. In the vehicle periphery monitoring device, the image acquisition means for capturing an image around the vehicle, the traveling lane recognition means for recognizing the traveling lane from the image captured by the image acquisition means, and the traveling lane recognition means are obtained. The traveling lane data, the speed data of the vehicle, and the optical flow calculation means for extracting the moving object from the image captured by the image acquisition means, and the status of the moving object extracted by the optical flow calculation means And a notification means for notifying the occupant of the vehicle, wherein the optical flow calculation means is a brightness value at a desired point in the image at the time of the current frame shooting, and An energy pan created based on the brightness value at the point corresponding to the desired point in the image at the time of frame imaging and on the condition that the speed change of the moving object imaged by the image acquisition unit is smooth. The feature is that the optical flow is calculated using a function.

According to the second aspect of the present invention, the optical flow calculating means calculates the energy functional E (u, v) shown in the equation (13) as (u, v) which minimizes the energy functional E (u, v). Characterize.

According to a third aspect of the present invention, the optical flow calculating means obtains a first averaged frame from an average value of pixel data of a plurality of frames including a frame at the time of current photographing, and a frame at the time of next photographing. A second averaging frame is obtained from an average value of pixel data of a plurality of frames including, and an optical flow is calculated based on the first averaging frame and the second averaging frame. To do.

According to a fourth aspect of the present invention, the optical flow calculating means regards a plurality of pixels included in each of the first averaged frame and the second averaged frame as one pixel. As a result, the image size is reduced, one reduced image is generated, or a plurality of reduced images are generated at different reduction ratios.
And the second averaged frame, the optical flow is calculated based on the calculated averaged frame and the second averaged frame. It is characterized in that the calculation process is repeated to calculate a full-scale optical flow.

According to a fifth aspect of the present invention, the traveling lane recognition means determines an edge point which is a boundary between the road surface and the partition line by obtaining a change in brightness in the image captured by the image acquisition means. It is characterized by detecting a plurality of lines and performing a linear approximation process based on the plurality of edge points to extract a right side partition line and a left side partition line of the traveling lane of the own vehicle.

According to a sixth aspect of the present invention, the traveling lane recognition means determines that the images captured by the image acquisition means are in a first area near the own vehicle and a distance from the first area. Divided into a second region consisting of
Linear approximation is performed based on a plurality of edge points, and further, left and right partition lines are extracted based on at least one of the following conditions (1-1) to (1-3), (1-1) The deviation from the FOE point of the image one frame before is small (1-2) The deviation from the angle of the partition line of the image one frame before is small (1-3) In conformity with the theoretical angle calculated For the second area, the linear approximation is performed based on a plurality of edge points, and the left and right partition lines are further based on at least one of the following conditions (2-1) and (2-2). Is extracted.

(2-1) The deviation between the Y coordinate of the FOE point and the Y coordinate of the FOE point of the partition line extracted in the first area is small. (2-2) The lower end of the approximated straight line is The invention according to claim 7 existing in the vicinity of the partition line extracted in the region 1 extracts the line portion and the line portion of the broken line included in the image captured by the image acquisition unit, It is characterized by further comprising a speed information calculation unit that obtains the speed data based on the length of the line portion and the length of the interline portion that are set.

In the invention according to claim 8, the notifying means indicates the degree of danger of the host vehicle according to the distance between the moving object and the host vehicle or the speed at which the moving object approaches the host vehicle. Is characterized by.

According to a ninth aspect of the present invention, the image acquisition means captures a front image of the host vehicle, and the moving object is a vehicle traveling in front of the host vehicle.

[0034]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing the configuration of a vehicle periphery monitoring device according to the first embodiment of the present invention.

As shown in the figure, the vehicle periphery monitoring device 1
Is a continuous image acquisition unit 2 that acquires continuous images of the front view of the vehicle, a lane recognition unit 3, a speed information calculation unit 4, an optical flow calculation unit 5, a separation processing unit 6 of the front vehicle region, and a danger. The evaluation unit 7 is used.

In the present embodiment, the continuous image acquisition unit 2 captures a front image of the host vehicle, and the traveling lane recognition unit 3 determines the traveling lane ahead of the host vehicle based on the captured image. recognize. Further, the traveling speed of the host vehicle is calculated by the speed information calculation unit 4 based on the captured image. Then, the optical flow calculation unit 5 detects the presence of a vehicle traveling in front of the host vehicle, and the separation processing unit 6 for the front vehicle region performs a process of separating the image of the front vehicle from the entire image. .

Further, the inter-vehicle distance between the own vehicle and the front vehicle or the approaching speed between the own vehicle and the front vehicle is calculated based on the image of the front vehicle obtained by the front vehicle region separation processing section 6, and the degree of danger is calculated. The evaluation unit 7 evaluates the degree of danger from the inter-vehicle distance and the approaching speed, and notifies the driver of the own vehicle of the degree of danger. Hereinafter, each component will be described in detail.

<Continuous image acquisition unit 2> In the continuous image acquisition unit 2, one camera (color digital video camera or the like) is installed at the front of the vehicle and the vehicle is traveling (in this case, traveling on a highway). ), The front image of the vehicle is captured by the digital video camera. That is, the image of the area "S1" in front of the vehicle shown in FIG. 2 is captured. Then, a series of captured moving images are taken out as frames at regular intervals and continuous processing is performed.

<Driving Lane Recognizing Section 3> The traveling lane recognizing section 3 recognizes the lane in which the host vehicle is traveling in the continuous traveling image of the vehicle front image.

In general highways, the following (1) to (3) are premised.

(1) The white line has a sufficiently high brightness as compared with other road surface portions.

(2) The white line can be approximated by a plurality of straight line groups.

(3) The detection position of the white line changes continuously in time.

By using these assumptions, the left and right white lines of the lane in which the vehicle is traveling are detected by two-step linear approximation. In the present embodiment, the partition line (side line or center line) drawn on the road surface is described as a white line, but the present invention is not limited to this. That is, a yellow line or a line of another color may be used.

Most of the conventional researches on the monitoring of the front of the vehicle assume a straight road or a very gentle curved road among the expressways, and the monitoring area is set in advance or the vicinity of the own vehicle. Approximate the white line to a straight line, and part 2
In many cases, the area surrounded by a straight line is the monitoring area.

However, when traveling on a straight road as shown in FIG. 3 (a), there is no problem in the above method, but as shown in FIG. 3 (b), in the case of a relatively sharp curve road. The vehicle in front that should be detected does not enter the monitoring area,
There is a problem that a side wall of a road that should not be detected or a vehicle in an adjacent lane enters the monitoring area.

On the other hand, as a study assuming a curved road, a curved road on a highway is assumed to be a part of an arc having a certain curvature in a three-dimensional space, and the arc is used on the image plane by using a camera parameter. A method of fitting a curve that has been projectively transformed to is proposed. However, an actual highway does not have long arcs having a certain type of curvature, but is configured as a combination of a plurality of arcs having different curvatures. Therefore, even if a curve obtained by projective transformation of one type of arc is applied to the image plane, it is not easy to exactly apply a certain curve to all images.

Therefore, in the present embodiment, the traveling lane in the vicinity of the own vehicle is simply detected, and at the same time, the traveling lane is divided into two stages so as not to erroneously recognize the adjacent lane even near the tip of the curve away from the own vehicle. Recognize by linear approximation of. By making the linear approximation into two stages, it becomes possible to deal with a curve road that is steep to some extent. As the monitoring range, the first stage was set to about 25 m from the host vehicle, and the second stage was set to about 70 m.

The details of the lane recognition method in two stages will be described below.

(A-1) Edge Extraction of White Line First, the boundary between the white line (side line on the road or center line) existing in the image and the road surface, that is, the edge point of the white line is extracted. In this method, as shown in FIG. 4, the white line edge search range is set in each of the first and second steps.

The set position is determined in advance by the parameter of the camera (digital video camera included in the continuous image acquisition section 2). Then, by performing a differentiation process on the brightness in the left and right directions within each search range, the boundary between the white line and the other road surface portion having a large difference in brightness, that is, the edge point of the white line is extracted. Hereinafter, the white line extraction processing of the first stage will be specifically described as an example. The white line edge extraction in the second step is performed based on the white line recognition result in the first step.

First, 1 × 6 for each line (horizontal line) in the search range in the image captured by the camera.
Differentiating processing is performed by scanning the operator of (indicated by "X" in FIG. 4). As an operator, use [1,1,1, -1, -1, -1,] on the left side of the image center and [-1, -1, -1,1,1,1] on the right side. . This corresponds to an operation in which the brightness of a pixel is checked from the center of the image in the left-right direction and a positive value is obtained when the pixel has a low brightness and a pixel with a high brightness changes.

FIG. 5 is a characteristic diagram showing a differential value when one line is scanned. As shown in FIG. 5, a point at which a road surface with low brightness changes to a white line with high brightness, that is, the left and right white lines. Large positive differential values a1 and a2 at the inner edge points
Is calculated. Further, large negative differential values a3 and a4 are calculated at the outer edge points of the left and right white lines. Then, a predetermined threshold value is set, and only points equal to or higher than the threshold value are extracted to obtain candidates for the edge point of the white line. In this case, candidate points are b1, b2, b3, and b4 shown in FIG.

However, in the actual road image, as shown in FIG. 4, the road surface other than the white line does not maintain constant brightness, and dust, scratches, buildings, shadows of vehicles running in parallel, etc. Due to the influence, unnecessary edges other than the white line may be extracted. Therefore, in the following processing, when performing linear approximation of the white lines in the first step and the second step, a device is provided so that these unnecessary edges are not erroneously recognized as white line edges.

(A-2) First-step white line recognition Based on the white-line candidate points obtained from the white-line edge extraction within the first-step search range (area S2 in FIG. 4), the first-step white line is recognized. Performs linear approximation processing. Here, as a method of creating an approximate straight line from a set of a plurality of points, the Hough transform (H
OUGH conversion) is used.

Further, as described above, the edge points obtained from the acquired image also include dust and shadows on the road surface, and it is necessary to remove their influence. Therefore, linear approximation is performed by using Hough transform that utilizes the characteristics of continuous images.

First, a normal Hough transform is performed on the image of the first frame to obtain straight lines for each of the left and right. Then, from the second frame, the FOE point (Fo
cusof Expansion) Hough transform using points. Here, the FOE point is the intersection of the left and right white lines that are linearly approximated, and represents a point in the image at the same height as the camera and located at infinity. On a road with good visibility such as an expressway, the white line is considered to extend to infinity, so the approximate straight line passes through the FOE point.

In general, in a continuous image, the point at infinity is considered not to change within a minute time. Therefore,
A condition that the straight line passes near the FOE point one frame before is added to the straight line approximated by the Hough transform.

As a result, it is possible to avoid erroneously recognizing edge points continuously arranged in a direction different from infinity as white line candidate points. In addition, although only one straight line is calculated in the straight line approximation by the general Hough transform, there are many points that are continuous in the direction of the FOE point in the road image, and thus there are many candidate points used for the Hough transform in this embodiment. Several right and left approximate straight lines are extracted as white line approximate straight line candidates.

The optimum combination is selected from the approximate straight lines obtained by the Hough transform and the right and left lines are approximated.
The white line model of the first stage is determined. The criteria for selection are (1) the deviation from the FOE point one frame before,
Consider (3) a deviation from the angle of the white line one frame before, and (3) a theoretical combination of angles calculated.

The above (1) is based on the assumption that the FOE point, which is a point at infinity, is at a substantially constant position within a minute time as described above. Also, regarding (2), 1
This is because the angle of the white line does not change so much within the minute time frame.

For (3), the lane width is constant,
When traveling on a flat road, the angle between the left and right white lines has a certain relationship.

More specifically, as shown in FIG. 6, when a plurality of first-stage edge points p1 are extracted in the area S2, based on the extracted edge points p1 as shown in FIG.
Obtain an approximate straight line by Hough transform. Furthermore, by adding a condition that the area passes near the FOE point p2 one frame before, the influence of the edge points created by dust on the road surface is avoided, and the approximate straight lines indicated by the broken lines L1 and L2 in FIG. Can be excluded.

Then, as shown in FIG. 7, a few straight line candidates (two in the figure) passing through the vicinity of the first-stage FOE point one frame before are obtained. Then, as shown in FIG. 8, a combination of white lines having a small deviation from the FOE point p2 of the first stage one frame before is selected. Therefore, if there is a straight line L3 passing through a position distant from the point p2 as shown by L3 in the figure, this straight line L3 is excluded.

Further, as shown in FIG. 9, a white line having a small deviation from the angle of the white line extracted one frame before is selected. That is, when the white line extracted one frame before is the line L4 shown in the figure, the angle θ0 of the line L4 is
A line L6 having an angle close to is extracted as a white line. That is, since the angle θ1 is closer to the angle θ0 than the angle θ2, the line L5 is excluded and the line L6 is selected.

Then, as shown in FIG. 10, when the angles of the left and right white lines (white lines extracted by the above procedure) when traveling on a flat straight road are θLO and θRO, respectively, a certain frame is obtained. When the angle of the left white line is θL and the lane width is always constant, the angle of the right white line θR can be expressed by the following equation (7).

[0067]

[Equation 4] Therefore, the white line of the combination of angles that satisfies the above-described expression (7) is set as a candidate. And the selected white line is 1
It is determined as the white line model of the stage.

(A-3) Second-step white line recognition Based on the first-step white line recognition result, the second-step white line recognition is performed. As shown in FIG. 11, first, with the middle line Lc of the left and right white lines subjected to the linear approximation in the first step as a boundary,
In the white line edge search range at the stage (area S3), the differential processing is performed in the left-right direction to extract edge points. next,
Hough transformation is performed using the extracted edge points to obtain approximate straight line candidates for each of the left and right lines.

As a criterion for selecting the optimum combination from these, (1) F by the first-stage linear approximation is used.
Misalignment between the Y coordinate of the OE point (vertical coordinate in the figure) and the Y coordinate of the FOE point due to the second-stage linear approximation, Consider the two existing in the vicinity. As for (1), assuming a flat road plane, the FOE point obtained by the straight line approximation of the white line near the own vehicle and the FOE point obtained by the straight line approximation of the white line at a certain distance from the own vehicle are present at the same Y coordinate. For that reason. As for (2), the white line continues from near the vehicle toward infinity.
The fact that the boundary between the white line at the second stage and the second stage is continuous is used. By these processes, the approximate straight line of the second step that is continuous from the first step is appropriately determined, and a curve road that is steep to some extent can be handled.

More specifically, as shown in FIG. 11, the center line Lc of the left and right white lines linearly approximated by the white line recognition process in the first step is set, and the center line Lc is used as a boundary to set two levels. Differentiation is performed in the left-right direction within the white line edge search range of the eye (area S3).

Then, similarly to the case shown in FIG. 6, Hough transformation is performed using the edge points to obtain approximate straight line candidates for each of the left and right lines as shown in FIG. In this case, L
Four straight line candidates 11 to L14 are obtained. Next, as shown in FIG. 13, Y at the first-stage FOE point p2
A coordinate having the Y coordinate of the intersection of two straight line candidates near the coordinate p2y is selected. That is, the straight line L14 is excluded. Furthermore, as shown in FIG. 14, the lower end points of the second-stage straight line candidates are the left and right white lines (LL, L) in the first-stage processing.
R) Select one that exists near. As a result, the straight line L11 is excluded, and the second-stage straight line candidates L12 and L13 are excluded.
Is required.

Further, as shown in FIG. 15, when the inter-vehicle distance between the front vehicle and the host vehicle is short and the white line in the second stage is hidden by the front vehicle, the second stage satisfying the above condition is satisfied. Since the approximate straight line of 1 is not obtained, the approximate straight line of the first step is extended to the second step and is used as the approximate straight line of the second step. This is because it is not necessary to consider the distant white line area when the preceding vehicle is present near the host vehicle, and thus it is possible to prevent erroneous recognition of the second-stage white line.

Then, the traveling lane recognition processing in the traveling lane recognition unit 3 is completed by the above processing.

<Speed Information Calculation Unit 4> Next, the processing in the speed information calculation unit 4 will be described. Generally, the white line drawn on a highway has the following characteristics.

(1) One of the left and right white lines is a broken line.

(2) The broken line has a line segment length of 8 m and other portions of 12 m.

In other words, it can be said that a certain pattern of features repeatedly appears on the road surface on an expressway. In the present invention, the speed information of the own vehicle is calculated by using this repeating pattern.

Specifically, as the first stage processing,
Recognize the broken line of the left and right white lines. The characteristics of the broken line are the white line segment (line part) and the road surface part (interline part).
It can be said that it can be divided into two.
Therefore, as shown in FIG. 16, a region (hereinafter, referred to as a white line region) S4 is created outside the approximate straight line of the left and right white lines, and the luminance distributions of the left and right white line regions S4 are compared.

The white line, which is a broken line, has a large luminance dispersion because line segments and road surface portions appear alternately. On the other hand, the solid line has a small luminance dispersion because it is all white. Therefore, it is possible to easily discriminate that the larger the variance of luminance is the broken line.

Here, the solid white line is the white line area S4.
It is conceivable that the broken line is recognized by simply comparing the average values of the brightness of the white line region S4 by utilizing that all are white and the brightness is high. However, there are shadows on the actual road surface, and when the shadow is reflected on the solid white line, it is possible that the average brightness of the broken white line area is higher than that of the solid white line area. Here, the method of comparing the variances of the luminance in the left and right white line regions is adopted.

Next, in the white line area S4 determined to be a broken line, a break between the line segment portion and the road surface portion is recognized. Since the line segment of the broken line is white, it is considered that the brightness is sufficiently higher than that of the road surface. Therefore, as shown in FIG. 17, the average value of the luminance in the Y-axis direction is calculated in the white line area determined to be the broken line, and the portion where the average value exceeds the threshold value is recognized as the line segment portion. This makes it possible to discriminate the position where the change in the average value of the brightness exceeds the threshold value from the road surface portion of the broken line.

Next, the speed of the host vehicle is calculated from the movement amount of the broken line obtained by the above procedure. The method of obtaining the distance from the camera to the target object from the image captured by the camera can be calculated from the optical arrangement of the camera.

FIG. 18 shows the optical arrangement of the cameras during image pickup. Since it is known that the white line, which is a broken line, is drawn on the road surface, it is possible to obtain the angles φ and ρ in the figure based on the position of the target object in the acquired image and the position of the FOE point. Therefore, the distance D to the target object can be calculated. If the height of the camera from the road surface is H [m], the distance D [m] can be expressed by the following equation (8).

D = H / tan (φ + ρ) (8) Here, the optical arrangement of the camera position of the nth frame and the camera position of the (n + 1) th frame is as shown in FIG. Then, if (φ + ρ) = θ, the distances Dn and Dn + 1 to the breaks of the respective broken lines are given by the following (9)
It can be shown by the equation and the equation (10).

Dn = H / tan θn ・ ・ ・ (9) Dn + 1 = H / tan θn + 1 ・ ・ ・ (10) Therefore, the moving distance of the vehicle from the nth frame to the (n + 1) th frame is represented by these differences. , If the image acquisition interval is T [sec], the speed V [Km /
h] can be expressed by the following equation (11).

V = H (1 / tan θ _n −1 / tan θ _{n + 1} ) × 3.6 / T (11) Here, if the image acquisition interval T is too large, the broken line break detected at the n-th frame becomes ( In the (n + 1) th frame,
It is possible that the image has already flowed to the front side of the screen and is not reflected in the image. In such a case, the fact that the interval between the broken lines is constant is used to detect the break in the next line segment that has flowed, and the moving distance is 20 m ( This problem can be solved by calculating the speed by considering the sum of (8 m + 12 m) as the actual moving distance.

By the above procedure, the speed information calculation unit 4 calculates the speed of the host vehicle.

<Optical Flow Calculating Unit 5> Next, the processing in the optical flow calculating unit 5 will be described. As described above, the optical flow is for displaying the movement of a certain point in the captured image by indicating the direction and the length of the arrow proportional to the magnitude of the velocity.

In this embodiment, it is possible to calculate the optical flow of the entire image by a "correlation method" in which the correspondence between the brightness values in the continuous images is directly calculated and the corresponding points are searched for, and by a relatively simple operation. By considering a new energy functional using both of the "concentration gradient method" and minimizing it, the corresponding points are searched to calculate the optical flow at points in the entire image at high speed. And
At the same time, the characteristics of the front vehicle are accurately detected, and the positional relationship between the own vehicle and the front vehicle is immediately calculated. Hereinafter, the procedure for calculating the optical flow will be described in detail.

(B-1) Derivation of energy functional In the forward vehicle image, it is conceivable that objects in the image will move globally, so the Global Optimiz of the concentration gradient method
Consider the energy functional based on the ation method.

If the constraint condition "the velocity field changes smoothly" is further added to the equation (3) shown in the above-mentioned prior art, the point (x, y) with the FOE by the first-step white line recognition as the origin is added. ), The concentration gradient in the x direction of the image in Ex, the concentration gradient in the y direction is Ey, the temporal concentration gradient is Et, the x component of the optical flow is u, the y component is v, and the u direction partial derivative of u is ux, u Is a partial differential in the y direction of y, and a partial differential in the x direction of v is v
An energy functional shown in the following equation (12) is set, where y is a partial differential of y in the y direction.

[0092]

[Equation 5] However, α in the equation (12) is a weighting value.

Then, (u, v) that minimizes the solution of the equation (12) is obtained, and this may be used as the optical flow.

However, when the continuity of the concentration gradient cannot be applied, the above-mentioned left term Ex · u + Ey · v + Et = 0 of (12) does not hold.

Then, instead of the density gradient value of the image, the correspondence of the spatial brightness value of the image between frames is preferentially considered like the correlation method and replaced with the left term of the expression (12). The brightness value of the point (x, y) in the current frame is G (x, y), and the brightness value of the corresponding point in the next frame is F (x + u, y + v). Set the energy functional.

[0096]

[Equation 6] It should be noted that α is a value for weighting, similar to the expression (12).

Then, the optical flow (u, v) is calculated by obtaining the minimum solution in the equation (13).

Here, the following equation (14) is established from the Euler-Lagrange equation.

[0099]

[Equation 7] Furthermore, the expression (15) is established by using the iterative calculation.

[0100]

[Equation 8] Where k is the number of iterations,

[Equation 9] Is the average value of the optical flow near the point (x, y).

(B-2) Consideration of temporal continuity In the energy functional shown in the above (B-1), since the correspondence of the spatial luminance value of the image is given priority, the temporal density gradient is considered. Information is missing. Therefore, in order to consider the temporal continuity of the moving image, an image obtained by performing temporal averaging on the image for which the optical flow is calculated is used.

20 (a) to 20 (c), images of consecutive (n-1), n, (n + 1) frames are averaged and shown in FIG. 20 (e). 1 averaging frame). Similarly, the images of the n, (n + 1), and (n + 2) frames shown in (b) to (d) of the same figure are averaged, and the image of the (n + 1) 'frame shown in (f) of the figure (second average). Frame). Then, in the images of the n ′ frame and the (n + 1) ′ frame, the solution that minimizes the energy functional is obtained using the above-described equation (15).

By performing this averaging, information on the temporal density gradient of the image can be contained in one averaged image. Then, for a moving object in the image, it is possible to make a smooth change in luminance in the direction of the object's movement, and it is possible to calculate a more accurate correspondence.

(B-3) Use of multi-resolution processing By using the energy functional obtained in (B-1) described above, a dense optical flow of the entire image can be calculated by a relatively simple calculation. However, since the minimum solution is obtained by using iterative calculation, there is a possibility that a local solution may be encountered before calculating the minimum solution in an image or the like having a large motion. Therefore, this problem is avoided by calculating the corresponding points while applying the multi-resolution processing to the continuous images averaged in (B-2).

First, as shown in FIG. 21A, a temporally averaged continuous image is reduced to 1/4 size. Then, for the reduced image, the optical flow at points of the entire image is calculated. Next, as shown in FIG. 21B, an image in which the continuous image is reduced to 1/2 size is created,
The optical flow is similarly calculated for this image. At this time, the optical flow obtained when the 1/4 size image is already used is used as the initial value.

After that, as shown in FIG. 21C, the optical flow is calculated from the image of the size of the original size. At this time, the optical flow obtained when the 1/2 size image is already used is used as the initial value.

By this processing, the problem of the local solution as described above can be solved, and at the same time, rough corresponding points can be calculated in advance with a small image size. The number of iterations is small, which can contribute to further reduction in processing time.

(B-4) Substitution of theoretical flow value based on velocity information When the optical flow is calculated by the method described in the processing up to (B-3) described above, theoretically, as shown in FIG. In the upper part, an optical flow (arrow in the figure) corresponding to the speed of the host vehicle is calculated, and when the relative speed to the host vehicle is not equal to the speed of the host vehicle such as a forward vehicle (moving object), or when the speed is high. When there is an object having a height, an optical flow having a size different from the optical flow corresponding to the speed of the host vehicle is calculated.

Here, it is considered possible to extract the optical flow of only the vehicle ahead by removing the influence of the optical flow corresponding to the speed of the own vehicle from the optical flow calculated by the above procedure. To be

However, when the optical flow is actually calculated using this method, an accurate optical flow cannot be calculated in a portion on the road surface where there is no texture, as shown in FIG. This is because the luminance pattern is too simple in a portion where there is no texture on the road surface, and thus it is not possible to calculate a delicate correspondence due to the influence of random noise that occurs when the image is captured by the camera. Therefore, the optical flow corresponding to the speed of the host vehicle can be calculated only in the textured portion on the road surface, and the optical flow only in the vehicle in front cannot be extracted.

Therefore, based on the speed information obtained by the speed information calculating unit 4, the optical flow (theoretical flow value) when all the points on the image are assumed to exist on the road plane is ¼. It is given as an initial value in advance to the points of the entire image reduced to. As a result, the initial value of the multi-resolution processing can be given, and an optical flow corresponding to the speed of the host vehicle can be given to a simple point of the brightness pattern having no texture on the road surface.

Thus, when the minimum solution of the above-mentioned energy functional is calculated, the optical flow corresponding to the speed information of the own vehicle can be calculated at all points on the road surface, and the texture etc. on the road surface can be calculated. It is possible to extract only the optical flow from the vehicle ahead without being affected.

Based on the above idea, it is assumed that all the points on the front image exist on the road plane, and the theoretical flow value is calculated from the speed information of the own vehicle.

Considering the optical arrangement as shown in FIG. 24, the point (x, y) on the image plane IP is calculated by using the real space coordinates (X, Y, Z) and the focal length f of the camera. Can be expressed by the following equations (16) and (17).

X = f (X / Y) (16) y = f (Y / Z) (17) Then, the value obtained by differentiating the point (x, y) with time, that is, the point ( The optical flow (u, v) of (x, y) can be expressed by the following equations (18) and (19).

U = (fX′−xZ ′) / Z (18) v = (fY′−yZ ′) / Z (19) Here, for a stationary object on the road surface, (20).

X ′ = Y ′ = 0 (20) Further, by setting Y = (camera height) = H and Z ′ = (speed of own vehicle) = − V, the point (x , Y) can be obtained by the following equations (21) and (22).

U = Vx / Z = Vx ² / fX = Vxy / fH (21) v = Vy / Z = Vy ² / fH (22) <separation processing unit 5 in front vehicle area> forward In the vehicle region separation processing unit 6, the theory based on the speed information obtained in the process (B-4) is applied to the temporal average image created in the process (B-2) in the optical flow calculation unit 5 described above. Using the flow value as an initial value, the minimum solution of the energy functional derived in (B-1) is obtained by using the multiresolution processing shown in (B-3). As a result, the optical flow of the entire image as shown in FIG. 25 is calculated.

By comparing this optical flow with the theoretical flow value calculated from the speed information, a region at a position higher than the road surface and having a speed different from the speed of the own vehicle as shown in FIG. Can be extracted. Further, in the separated flow, the traveling lane recognition unit 3
Based on the result of the two-step white line recognition recognized in the process of
By leaving only the optical flow in the lane in which the vehicle is traveling, only the optical flow of the vehicle in the vehicle as shown in FIG. 27 is separated.

At this time, the optical flow due to the texture on the road surface is equal to the optical flow corresponding to the speed of the host vehicle, and therefore it can be eliminated.

In this way, the optical flow of only the vehicle ahead can be effectively extracted.

<Danger Level Evaluation Unit 7> In the risk level evaluation unit 7,
Based on the optical flow of the separated forward vehicle,
A process of evaluating the risk of the own vehicle colliding with a vehicle ahead and warning the driver according to the risk is performed.

Now, when driving on an expressway, it is generally regarded as dangerous. (1) When the inter-vehicle distance between the own vehicle and the preceding vehicle is short, (2) the own vehicle suddenly moves to the preceding vehicle. If they are close, there are two points.

Here, the magnitude of the optical flow is
Originally, it has the property that in the real space, the closer the target object is to the camera, the larger it becomes, and the larger the relative speed, the larger it becomes.

Therefore, a method for evaluating the dangerous situation of the own vehicle can be easily conceived by simply adding the optical flows in the monitoring area. That is, when the magnitude of the y component in each optical flow is vi and the number of optical flows generated is N, a method of expressing the risk D in the forward field of view as shown in the following equation (23) is conceivable. .

[0126]

[Equation 10] However, when this method is used, the degree of risk D largely depends on the number N of generated optical flows because all the sizes of optical flows are added. That is, since the dense optical flow is calculated at the points of the entire image, the size of the vehicle in front directly affects the number of N. Therefore, FIG. 28 (a)
When a four-wheel vehicle is present in front of the host vehicle as shown in FIG. 2 and when a two-wheel vehicle is present in front of the host vehicle as shown in FIG. Even if they approach the vehicle by the same distance, the degree of danger will be different.
That is, there is a problem in that the degree of risk varies depending on the width of the vehicle ahead.

Therefore, in the present embodiment, as shown in FIG. 29, the average value of the y components of the optical flow at each y coordinate below the FOE point is calculated, and the sum is obtained. That is, consider the following equation (24).

[0128]

[Equation 11] Since the magnitudes of the optical flows are averaged in the lateral direction, the lateral width of the vehicle ahead does not affect the risk index. At this time, y at the lower end of the vehicle ahead
Assuming that the coordinate is y0, the average value of the y component of the optical flow at y0 is v0, and the optical flow of the preceding vehicle occurs on the same depth plane,
In the front vehicle area, y that is proportional to the size in the y direction
Since the optical flow having the component is generated, the average value of the y component of the optical flow in the vehicle ahead can be expressed by the following equation (25).

[0129]

[Equation 12] Then, the above equation (24) becomes the following equation (26) by the equation (25).

[0130]

[Equation 13] As a result, the risk D increases as y0 increases, that is, as the point at the lower end of the front vehicle approaches the host vehicle, and at the same time, as v0 increases, that is, the degree of approach of the front vehicle increases. It can be seen that the situation is generally dangerous when traveling on a highway.

Next, a method of displaying an actual warning screen will be described. FIG. 30 shows a warning screen. First, the acquired original image is displayed in the center of the warning screen. In this image, the white line recognition result in two stages is shown in green, and the optical flow of the vehicle ahead is shown in red. When a warning is issued to the driver, the risk level judged from the evaluation value of the risk level is displayed on the right side of the warning image by 10-level numbers and a level bar.

Also, in accordance with the numbers on the risk level bar, a warning sound is output with a high sound when the risk level is 10 to 8 and a low sound when the risk level is 7 to 5. Regarding the warning sound, the degree of danger of the five frames before the present is taken into consideration, and a warning is issued when the degree of danger continuously exceeds the threshold value.

In this way, when it is detected that the vehicle ahead is approaching, the driver can be surely notified and safety can be improved.

In this way, in the vehicle periphery monitoring device 1 according to this embodiment, the optical flow of the vehicle traveling in front of the host vehicle can be extracted with extremely high accuracy, and based on the optical flow, Since the distance between the host vehicle and the preceding vehicle and the speed at which the host vehicle approaches the preceding vehicle can be obtained, the risk of collision between the host vehicle and the preceding vehicle can be obtained immediately and with high accuracy. As a result, it is possible to significantly improve the safety when the vehicle is running.

Although the vehicle surroundings monitoring device of the present invention has been described above based on the illustrated embodiment, the present invention is not limited to this, and the configuration of each part may be any configuration having the same function. Can be replaced with one.

For example, in the above-described embodiment, an example in which the speed information calculation unit 4 is used to measure the speed of the host vehicle has been described. However, without using the speed information calculation unit 4, the traveling speed of the host vehicle is set to the vehicle speed. It is also possible to obtain it from the vehicle speed sensor mounted on the vehicle.

Further, in the above-described embodiment, when calculating the optical flow, a 1/4 image, a 1/2 image,
However, the present invention is not limited to this, and it is also possible to use images of other sizes such as ⅓ image.

Furthermore, in the present embodiment, the traveling lane recognition unit 3
In the above, an example in which the driving lane is extracted by the two-stage processing has been described, but the present invention is not limited to this, and 3
It is also possible to carry out processing in more than one stage.

Further, an example of obtaining the average of three frames (n-1, n, n + 1 frames) in order to obtain the temporal average image n'has been described, but the present invention is not limited to this. Alternatively, an average of 2 frames or 4 frames or more may be obtained.

Further, in the present embodiment, an example in which an image of the front of the own vehicle is taken and the presence of a forward vehicle traveling in the traveling lane of the own vehicle is detected has been described, but the present invention is not limited to this. Instead of the one, it is also possible to obtain the distance to the moving object existing behind the own vehicle.

[0141]

As described above, in the vehicle periphery monitoring device according to the present invention, the optical flow can be obtained easily and accurately, so that the movement of a moving object such as a vehicle ahead can be surely recognized. You can As a result, it is possible to immediately recognize the distance between the moving object and the own vehicle and the approaching speed between the moving object and the own vehicle, and it is possible to improve the safety when driving the vehicle.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a vehicle periphery monitoring device according to an embodiment of the present invention.

FIG. 2 is an explanatory diagram showing a front area of a vehicle captured by a continuous image acquisition unit.

FIG. 3A shows a front image when traveling on a straight road, and FIG. 3B shows a front image when traveling on a curved road.

FIG. 4 is an explanatory diagram showing a process for recognizing a white line on a road.

FIG. 5 is an explanatory diagram showing how the luminance changes at the edge portion of the white line.

FIG. 6 is an explanatory diagram showing edge points detected in the first-stage white line recognition processing and lines of straight line candidates based on the edge points.

FIG. 7 is an explanatory diagram showing lines of straight line candidates that pass through an FOE point one frame before.

FIG. 8 is an explanatory diagram for excluding a line candidate of a straight line that passes a position away from an FOE point one frame before.

FIG. 9 is an explanatory diagram for excluding a line candidate line that is greatly separated from the angle of the white line extracted one frame before.

FIG. 10 is an explanatory diagram showing two extracted white lines.

FIG. 11 is an explanatory diagram showing a search range in the second-stage white line recognition processing.

FIG. 12 is an explanatory diagram showing how a white line on a curved road is recognized in the second-stage white line processing.

FIG. 13 is a schematic diagram illustrating the second-stage white line processing in which the first-stage F
It is explanatory drawing at the time of excluding the line of a straight line candidate which passes the point different from the Y coordinate of OE point.

FIG. 14 is an explanatory diagram when a line candidate line that does not pass through a point close to the white line obtained in the first step white line processing is excluded in the second step white line processing.

FIG. 15 is an explanatory diagram of a process when the white line process in the second stage cannot be performed due to the presence of a vehicle ahead.

FIG. 16 is an explanatory diagram showing how a white line area is set in the extracted white line data.

FIG. 17 is an explanatory diagram showing a change in luminance of a broken line between a white line portion and a line portion.

FIG. 18 is an explanatory diagram showing a positional relationship between a camera and an object to be photographed.

FIG. 19 is an explanatory diagram showing a positional relationship between the camera and an object to be photographed when the camera moves.

FIG. 20 is an explanatory diagram showing a state in which one averaged image is created using three frame images.

FIG. 21 is an explanatory diagram showing a procedure of sequentially obtaining an optical flow from a reduced image and obtaining an optical flow of a full-scale image.

FIG. 22 is an explanatory diagram when optical flows are obtained at all points on the screen.

FIG. 23 is an explanatory diagram showing a state where an optical flow does not exist in a portion on the road surface where a texture does not exist.

FIG. 24 is an optical layout diagram for obtaining a theoretical flow.

FIG. 25 is an explanatory diagram showing a calculated optical flow.

26 is an explanatory diagram showing an optical flow obtained by subtracting the theoretical flow from the optical flow shown in FIG.

FIG. 27 is an explanatory diagram showing a state where an optical flow existing in the lane of the own vehicle is left from the optical flow shown in FIG. 26.

FIG. 28 (a) shows an optical flow when a front four-wheel vehicle exists, and (b) shows an optical flow when a two-wheel vehicle exists in the front.

FIG. 29 is an explanatory diagram for obtaining an optical flow of a vehicle ahead.

FIG. 30 is an explanatory diagram showing a state of risk display.

[Explanation of symbols]

1 Vehicle Surrounding Monitoring Device 2 Continuous Image Acquisition Unit (Image Acquisition Unit) 3 Driving Lane Recognition Unit (Driving Lane Recognition Unit) 4 Speed Information Calculation Unit 5 Optical Flow Calculation Unit (Optical Flow Calculation Means) 6 Separation Processing Unit for Front Vehicle Area 7 Risk Assessment Section (Notification Means)

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) G06T 1/00 320 G06T 1/00 320A 330 330A 3/00 500 3/00 500A 3/40 3/40 A 5/50 5/50 7/60 200 7/60 200J G08G 1/04 G08G 1/04 D 1/16 1/16 C H04N 7/18 H04N 7/18 J K (72) Inventor Masato Nakajima Tokyo Chofu 3-14-18 Iruma-cho, Ichi (72) Satoshi Kato 21-13 F term, Wakamiyadai, Yokosuka City, Kanagawa Prefecture (reference) 5B057 AA16 AA19 CA08 CA12 CA16 CB08 CB12 CB16 CD05 CE05 CF05 CH01 DA06 DB02 DC16 5C054 AA02 AA04 FC12 FC14 FE28 HA30 5H180 AA01 CC04 LL04 LL09 5L096 BA02 DA02 EA03 EA06 EA39 FA06 FA32 FA67 GA02 HA03

Claims (9)

[Claims] 1. A vehicle periphery monitoring device for sequentially capturing peripheral images of a vehicle while the vehicle is traveling, and monitoring the situation around the vehicle based on the captured images, an image acquisition unit for capturing peripheral images of the vehicle, and the image. Traveling lane recognition means for recognizing a traveling lane from an image captured by the acquisition means; traveling lane data obtained by the traveling lane recognition means;
Further, the speed data of the vehicle, and an optical flow calculation means for extracting the moving object from the image captured by the image acquisition means, and a situation of the moving object extracted by the optical flow calculation means to the occupant of the vehicle. Notifying means for notifying, the optical flow calculating means, the optical flow calculating means, the brightness value of the desired point in the image at the time of the current frame shooting, and the brightness at the point corresponding to the desired point in the image at the next frame shooting A vehicle periphery characterized by calculating an optical flow based on a value and using an energy functional created on the condition that the speed change of a moving object photographed by the image acquisition means is smooth Monitoring equipment. 2. The vehicle periphery monitoring according to claim 1, wherein the optical flow calculation means calculates the energy functional E (u, v) shown below as (u, v) that minimizes it. apparatus. [Equation 1] However, G (x, y) is the desired point (x, y) in the current frame
, F (x + u, y + v) is a brightness value at a point corresponding to a desired point (x, y) in the next frame, α is a weighting constant, ux is a partial differential of u in the x-axis direction, and uy is u y
Partial differential in the axial direction, vx is the partial differential in the x-axis direction of v, and vy is v
Partial differential along the y-axis 3. The optical flow calculation means obtains a first averaged frame from an average value of pixel data of a plurality of frames including a frame at the current shooting, and pixel data of a plurality of frames including a frame at the next shooting. 2. The second averaged frame is obtained from the average value of, and the optical flow is calculated based on the first averaged frame and the second averaged frame. 2. The vehicle periphery monitoring device according to any one of 2. 4. The optical flow calculation unit reduces the image size by regarding a plurality of pixels included in each of the first averaged frame and the second averaged frame as one pixel. Then, one reduced image is generated or a plurality of reduced images are generated at different reduction rates, and an optical flow is calculated based on the first averaged frame and the second averaged frame having the highest reduction rate. At the same time, based on the calculated data of the optical flow, the process of calculating the optical flow of each averaging frame having a reduction ratio close to the actual size is repeated to calculate the optical flow of the actual size. 3. The vehicle periphery monitoring device according to 3. 5. The traveling lane recognition means detects a plurality of edge points that are boundaries between a road surface and a partition line by obtaining a change in brightness in an image captured by the image acquisition means, and the plurality of edge points are detected. 5. A straight line approximation process is performed based on each edge point to extract a right side partition line and a left side partition line of the own vehicle's travel lane, according to any one of claims 1 to 4. Vehicle surroundings monitoring device. 6. The driving lane recognition means divides the image captured by the image acquisition means into a first area near the own vehicle and a second area farther than the first area. Then, for the first region, linear approximation is performed based on a plurality of edge points, and further, the left and right partitions are based on at least one of the following conditions (1-1) to (1-3). The line is extracted, and (1-1) the deviation from the FOE point of the image one frame before is small (1-2) the deviation from the partition line angle of the image one frame before is small (1-3) from the calculation For the second region that matches the theoretical angle required, linear approximation is performed based on a plurality of edge points, and further, at least one of the following (2-1) and (2-2) The vehicle periphery monitoring device according to claim 5, wherein the left and right partition lines are extracted based on one condition. . (2-1) The deviation between the Y coordinate of the FOE point and the Y coordinate of the FOE point of the partition line extracted in the first area is small. (2-2) The lower end of the approximated straight line is the first area. Exists near the partition line extracted by 7. A line portion and a line portion of a broken line included in the image captured by the image acquisition unit are extracted, and preset lengths of the line portion and the line portion are set. The vehicle periphery monitoring device according to any one of claims 1 to 6, further comprising a speed information calculation unit that calculates the speed data based on a length. 8. The notification means indicates the degree of danger of the own vehicle according to the distance between the moving object and the own vehicle or the speed at which the moving object approaches the own vehicle. ~ The vehicle periphery monitoring device according to claim 7. 9. The image acquisition means captures a front image of the host vehicle, and the moving object is a vehicle traveling in front of the host vehicle. The vehicle periphery monitoring device according to item 1.