WO2007002964A2 - Method for lane recognition - Google Patents

Abstract

The invention relates to a method for lane recognition by means of an image recording unit carried on a vehicle. According to the invention, in each image recorded, the edges are detected and the position of the pixels of the edges determined in the image recorded, the image is analysed to determine vanishing points, for generation of an image dividing line, the foot of the given optical centre of the image is connected to the appropriate vanishing point, for each edge pixel, a connecting line to the appropriate vanishing point is generated and the orientation thereof determined, for a given number of said connection lines, the edge pixels thereof are determined and a connection line is generated through each vanishing point, said connection lines are divided into candidates for the definition of the left or right lane marking and allocated to left (LL, RL) or right (LR, RR) lane defining candidates, based on spatial distance ranges of corresponding pairs, candidate pairs for the left (3) and right (4) lane markings are determined, a quality measure is applied to said candidate pairs, and the above procedure carried out on all determined vanishing points, and form the obtained candidate pairs, those with the best quality measures are selected and the result of the quality measure used for evaluation of the relevance of the obtained pairs of lane markings.

Description

 Lane detection method

The invention relates to a method according to the preamble of claim 1.

The aim of the invention is to recognize with the least possible effort and with the greatest possible accuracy within a short time or with as little recorded images of a vehicle from a lane or a lane marking or other along the road extending markings or attached markings.

These objects are achieved with a method of the type mentioned, is characterized in the present invention with the features specified in the characterizing part of claim 1. The images can be taken in any direction to the roadway, in particular to the front. With a relatively small number of recorded

Pictures are determined exactly the position of the markers.

A quick finding of vanishing points is achieved with the features of claim 2! A quick and accurate histogram creation is done with the characteristics of the

Claim 3 reached.

Suitable results are achieved quickly and accurately when the features of claim 4 are used.

An improved and exact evaluation of the calculation results, in particular a more exact determination of the position of the lane markings is achieved with the features of claim 5.

For the ongoing evaluation during the movement of the vehicle, the features of claim 6 and the claims 7 and 8 are advantageous. Thus obtained values for the following images can be made available as parameters or limiting values, which in particular reduces the computational effort.

The computer program product according to the invention makes it possible for the method according to the invention to be executed on a computer.

In the following the invention will be explained in more detail with reference to the drawings, for example. Fig. 1 shows a view of a traffic lane from above. Fig. 2 shows a detection of lane markings by determining four lane boundaries. Fig. 3 shows edge pixels superimposed on the recorded original image to the lane boundaries.

For example, Fig. 4 shows the process of voting for a vanishing point based on a pixel with the orientation or inclination angle α of a connecting line. FIGS. 5 and 5a show the calculation of an image division line. 6 shows a 1 D

Hough space, which was determined from the edge pixels corresponding to FIG. 3 for the vanishing point of FIG. In the following the invention will be explained in more detail with reference to the drawing, for example.

In Fig. 1 shows schematically a Bildaufnahme- or camera system 1, which moves in a lane 2, seen from above in the Euclidean space. The lane 2 is marked on the road by white bands or lane markings 3, 4 left and right. For example, the right mark is continuous, while the left is piecewise continuous. L and R are each the distance measured from the optical center to the left and right lane markers. ΔL and M describe the width of the left and right

Track mark.

For lane detection, the lane delimitations, that is, the existing of two line segments boundaries of the left and right lane markers, as shown in Figure 2 can be seen. From left to right, the four lane boundaries are labeled LL (left lane marker left boundary), RL (left lane marker, right boundary), LR (right lane marker, left boundary), and RR (right lane marker right boundary). It is assumed that in the present case forward-looking image pickup unit 1, which is aligned substantially parallel to a flat road. In the following, with (x _o , y _o ) the vanishing point, ie the intersection of the 4 lane boundaries in the recorded image; θ is the inclination angle of the camera to the road, designated, φ is the rotation angle of the camera to the right or left; / is the focal length of the camera; H is the height of the camera above the road. a _LL , a _RL , a ^ and a _RR are the respective slopes of the 4 lane boundaries in the image. a _L and a _R are the average slopes of the left and right lane markings in the image. L and R are the Euclidean distances from the camera to the left and right lane markers. Basic formulas are:

Aus (2) kann daher die Breite der Spurmarkierung ΔLand AR berechnet werden; zusätzlich kann der Fehler von L, R, ΔL and AR als Funktion des Fehlers der ermittelten Anstiege a_LL , a_RL , <7_ω und a_RR berechnet werden.

From (2), therefore, the width of the lane mark ΔLand AR can be calculated; in addition, the error of L, R, ΔL and AR can be calculated as a function of the error of the detected slopes a _LL , a _RL , <7 _ω and a _RR .

The lane recognition according to the invention comprises the following basic

Steps:

1. Edge detection with known optical image evaluation methods and gradient measurement or determination of the orientation and position of the identified edge pixels 2. Finding vanishing points that are caused by the position and orientation of the edges, preferably using a weighted 2D Hough transform in multiple resolutions and a following 2D maximum search.

3. For each vanishing point found a. Calculation of the pitch angle or an image division line between the left and right lane markers and their boundaries b. Locating lines passing through the vanishing point, advantageously using a 1-D Hough transform and a 1-dimensional maximum search c. Finding the support of each line and forming one

Histogram with the angles of inclination of the determined connecting lines, d. Improvement of the orientation or the angle of inclination of each connecting line by the least squares method (minimizing the sum of the quadratic orthogonal distances of the edge pixel from the connecting line passing through the vanishing point e) Calculating the average orthogonal gradient for each connecting line or chiaroscuro or dark-light transitions for finding lane boundary candidates LL, RL, LR and RR g locating lane marker candidates (pairs LL, RL and LR, RR) h search among those of the left lane marker and / or the left and right boundary candidates associated with the right lane marking the two lane markers for matching pairs or finding track candidates and creating a complete hypothesis of 4 lane delimitations for the respective

Vanishing Point. i. Calculation of a quality measure or parameter for each hypothesis based on advantageously 5 criteria, namely mean orthogonal

Gradient of each pair of lane boundaries and / or medium

Support size for any bounds and / or similarities within a pair of bounds and / or continuity of the boundary-supporting edge pixels and / or mean lateral errors of each pair.

4. Selection of hypotheses with the highest characteristic values

5. The determined parameters concerning track width (R + L) and / or inclination angle θ are stored and used as a-priori information for the evaluation of the next picture at the next point in time.

For edge detection and gradient measurement, a known edge detection method is used to obtain a binary threshold image. The location and orientation of each edge pixel is measured. It filters out the pixels whose edge orientation is horizontal or nearly horizontal, since horizontal lane boundaries are not interesting, as this would mean that the vehicle is traveling orthogonal to the lane. Figure 3. shows an image in which the lane markers edge pixels are superimposed.

In order to find the vanishing point F, a 2D Hough transformation is advantageously used. Other methods are known and usable. It is expedient to use a weighted, multiresolute 2D Hough transformation in order to find vanishing points F in the image. One of these vanishing points should correspond to the intersection of four lane boundaries. To find this, a rectangular Hough space centered around the optical center of the image is assumed. If there is no a-priori knowledge of the position of the vanishing point, the size of the rectangle is defined by the maximum possible values of inclination and horizontal rotation. However, if a-priori knowledge is given from previous measurements or image evaluations and / or values of tilt and / or rotation from other sensor data, the size of the Hough space can be reduced to a smaller area, thus facilitating the evaluation.

Each data point, together with its orientation, defines a segment,

bi ^~ y ₀ ) ^cos (bi) = (* / ^~ * ₀ ) ^sül ( ^a t) ( ³ )

and voted for all cells in Hough space that intersect with this segment (see Fig. 4). The resulting 2D Hough space can be smoothed by convolution with a Gaussian mask. Then the maxima are detected and the strongest maxima are kept.

For finding the inclination angles for straight lines or connecting lines, with known vanishing point for distinguishing between the boundaries of the left and right lane markings, the pitch angle is calculated, as shown in Figure 5 and 5a.

The intersection of the illustrated horizontal and vertical lines corresponds to the optical center Z (x, y). The drawn point is a determined vanishing point F for the

Track boundaries. Before the detection of the four boundary lines, a line is drawn between the vanishing point F (x ₀ , y ₀ ) and the lower end of the vertical line. These

Image division line divides the image with respect to candidates for the boundaries of the left and right lane markers according to the equation of the tilt angle

ß = atan2 ((Cy - NumRows) - y _o , -Xo (4)

where Cy represents the vertical position of the optical center in the image and NumRows represents the number of image pixels in the vertical direction.

For a given Diminishing {χ _Q, y _Q) of a one-dimensional Hough transform are straight line segments searched conveniently effected under from previous measurements or evaluations using image that pass through the vanishing point F. The detection of the lane boundaries at a given vanishing point F is more robust than the detection without a given vanishing point, since the position of the vanishing point severely restricts the interpretation of the data points as lines.

For a given vanishing point and a data point (*,.,)>. ), this vote for a possible existence of a line with inclination

Taking into account the error in the orientation (which is not caused by the measured gradient of the pixels but by the localization error of each data point (*,.,)>,.)),

wird, wie in Fig. 5b näher erläutert, für alle Neigungswinkel im Intervall [a_t - Aa_nOi₁ + Aa₁] votiert. Danach wird zweckmäßigerweise der 1 D-Hough-Raum

is, as explained in more detail in Fig. 5b, for all inclination angles in the interval [a _t - Aa _n Oi ₁ + Aa ₁ ] voted. Thereafter, the 1 D-Hough space is expediently

(Histogram) based on orientation or inclination angle of the connecting lines, for example, smoothed with a Gaussian mask. This histogram searches for connecting lines with the strongest maxima.

The resulting 1 D Hough space associated with the vanishing point (in Fig. 2) is shown in Fig. 6. The 4 maximas in the Hough space or histogram correspond to the lane boundaries of Fig. 2 and are labeled LL, LR, RL and RR.

The fact that the vanishing point is detected first and then the lane delimitations makes the method more robust. The vanishing point is a very robust indication, since it is derived from different information distributed in the image. The search problem for straight lines at a given vanishing point is reduced from a 2D to a 1D problem. When using a top-down heuristic (use of the geometric instead of the measured gradient), the robustness can be further increased. To find support for each line is for each maximum found

(corresponding to a certain line slope) of the 1 D Hough space whose support is calculated in the image. For this purpose, the number of edge pixels that support this line is determined, allowing some orthogonal deviation of the pixel from the line. To improve the determination of line slope, i. the orientation or the

Inclination angle, is given a given line inclination and given edge pixels that support the running through the respective vanishing line, the exact orientation or its course improved by the method of the robust least squares. For this, the tilt angle is calculated, which minimizes the sum of the square, orthogonal distances of the edge pixels to the connecting line.

If AI and ΔJ is the result of a convolution of the original image with the appropriate orientation mask, then the gradient image of the edge pixels is given by - All AJ. The mean orthogonal gradient of a line is then given by the equation

i »PerpGrad = - J] (- sin (a,) ^■ AI (x ^,, y _t ) + cos (a,) • ΔJ (x,, y,)) (8) n - ir = \

The lane boundary candidates LL, RL, LR and RR are then selected based on their average orthogonal gradient (PerpGrad) (transition from dark to light or vice versa) and classified according to their inclination angles smaller or larger than the pitch angle of the image splitting line: LL = {a, \ a ₍ ≤β + Aβ A PerpGrad,.> 0}

LR = {a _t \ a _t ≤β + Aβ Λ PerpGrad _t <0}

O)

RL = {a, \ a _t ≥β -Aß Λ PerpGrad _t > 0}

LR = (^ _1. \ A _t ≥ß-Aß A PerpGrad. <0}

where eg Aβ = \ ° can be used. To find pairs of lane marker candidates (pairs LL ₁ RL and

LR ₁ RR), the procedure is as follows:

To find the left lane marker candidates, all LL and RL candidates are grouped in pairs according to

a _j eLR> a _(eLL Λ Λ W≥mϊnW-dW W≤m & xW + dW (10)

where W is the width of the lane marker, and dW is the detected error of W due to slope errors. In the same way, the right lane marker candidates are calculated:

ct _j GRR> a ₍ G RL Λ W≥minW-dW Λ W≤msixW + dW (11)

wherein, for example, practically predetermined ones could be used.

To find the track candidates, i. According to a complete hypothesis of the four lane delineations LL, RL, LR, RR, pairs of left and right lane markers are grouped and it is checked whether the total width of the lane is reasonable according to usual road conditions according to

R + L≥mmLR A R + L≤mzxLR (12)

For the calculation of a characteristic value or quality measure for each hypothesis 5 criteria can be used. First, separate characteristics and quality measures are calculated for the candidates for the left and right lane marker pairs. The final characteristic value is a combination of the two individual values. The characteristic value of each lane marking is calculated by means of at least one of the following 5 values: 1. middle orthogonal gradient of each pair (gradient)

As described in (8). Calculate the mean orthogonal gradient for each of the 2 lane delineations and calculate the mean.

2. Medium support size for each pair (SupportSize) Determines the number of edge pixels that supports each of the two lane boundaries and calculates the average.

3. Line equality within a pair (similarity)

For this, it is calculated how equal the edge data of 2 lane boundaries of the same lane marker are. 4. average continuity of edge pixels supporting a pair (continuity)

5. Lateral lateral error of each pair [Lateral Erroή, for which the average orthogonal distance of the data pixels from the line and then the mean is calculated for each of the 2 lane delineations.

Below is a selection of lane markers with maximum characteristic. It is also envisaged that candidates for the left as well as the right

Lane marking is applied or calculated in each case a quality measure and the two results obtained are summarized to a final quality measure, which refers to the respective vanishing point and extending through this limitations of the left and right lane markers. Advantageously, it is provided that this procedure is carried out or applied to all vanishing vanishing points found and selected from the candidate pairs for the left and right lane marking those with the best quality measure and that the results of

Quality measure used to assess the relevance of the obtained pairs of lane markers. Finally, storage and further use of the values or parameters obtained take place.

To trace the vanishing point, the last images can be examined. If an investigation was successful, the result of the last survey step is started as the starting point for the search for the new vanishing point in a given region centered around the previous result. This limitation of the search region limits the duration and the number of detected vanishing points.

The width of the lane, R + L, is a parameter that does not change as fast over time as the vanishing point. Therefore, the average track width is usually considered over a longer period and this value is used for the following evaluations. It is advantageous if, based on the hypotheses of a number of successively recorded images, the width of the lane is calculated and that this value is used to define or narrow down the range of the lane Distance between the left and right lane marking is used and / or for the respective vanishing point of an hypothesis, the inclination angle or the pitch of the image acquisition unit is calculated and that for successive hypotheses determined values of this slope for determining or delimiting the used for finding vanishing points 2D Hough space is used to create hypotheses of a subsequent image.

claims

Claims

claims: 1. A method for lane detection and / or tracking with one, in particular a single, on a vehicle entrained image recording unit, in particular camera, with a predetermined position and orientation with respect to the vehicle and / or the road, are taken with the images of the road, characterized in that edges present in the respectively recorded image are detected and the position and the orientation of the pixels are determined by the edges detected in the image, that the image is examined with regard to vanishing points caused by the position and orientation of the edge pixels, that the foot point of the predetermined optical center of the image is connected to the respective vanishing point in order to produce an image division line for the individual detected vanishing points, that for each edge pixel a connecting line to the respective vanishing point is created and its orientation or angle of inclination is determined, that a histogram is formed for the respective vanishing point with the determined orientations or angles of inclination of all created connecting lines, that the histogram is examined for connecting lines with the most strongly represented orientations or angles of inclination, that for a given number of these connecting lines the supporting edge pixel determines and based on this edge pixel with the method of least squares robust one (n) improved (n ) Orientation or inclination angle connecting line is created by the respective vanishing point, that these improved connecting lines are divided into candidates for the limitations of the left or the right lane mark with respect to their orientation or inclination angle with respect to the image dividing line, - That these candidates are examined in a direction perpendicular to their (m) orientation or inclination angle with respect to a light-dark or dark-light transition and depending on the transition direction, a division into left (LL, RL) or right (LR, RR) lane boundary candidates for left (3) or right (4) lane marking in that, based on predetermined spatial distance ranges and / or predetermined orientation or inclination angle ranges, among the left and right boundary candidates of the two track markings assigned to the left-hand track marking and / or the right-hand track marking, mutually associated pairs (LL, RL or LR, RR), - That based on predetermined spatial distance ranges and / or Orientierungsbzw. Tilt angle ranges of matched pairs candidate pairs for the left (3) and right (4) lane markers are determined - that a quality measure is applied or calculated on these or for these candidate pairs, - That this procedure is made or applied to all vanishing vanishing points and selected from the candidate pairs for the left and right lane marking those with the best quality measure are selected and - That the results of the quality measure are used to evaluate the relevance of the obtained pairs of lane markers. 2. The method according to claim 1, characterized in that the vanishing points are searched with a weighted, multiresolution 2D Hough transformation in the recorded image, in particular taking into account the position and / or orientation of the edge pixels, where appropriate, the resulting 2D Hough space , in particular by convolution with a Gaussian mask, is smoothed and determines the maxima and in particular the strongest maxima are used as vanishing points. 3. The method according to claim 1 or 2, characterized in that the histogram using a weighted one-dimensional Hough transformation under Considering the angle error of the connecting line between the respective edge pixels and the associated vanishing point is created. 4. The method according to any one of claims 1 to 3, characterized in that the candidates for the left as well as for the right lane marking each one Quality measure is applied or calculated and summarized the two results obtained to a final quality measure, which refers to the respective vanishing point and extending through this limitations of the left and right lane markers. 5. The method according to claim 4, characterized in that the quality measure for the candidates for the left and the right boundary of the lane marker is determined based on at least one of the following criteria or quantities: 1.) Average orthogonal gradient of each pair of lane boundaries calculated from the average mean orthogonal gradients for each of the two Gauge limits and / or 2.) average assist size for each pair of lane boundaries, determined from the average of the number of edge pixels that supports and / or each of the two lane boundaries 3.) Line equality within a pair or similarity of the distribution of edge pixels, which support the two lane boundaries and / or 4.) continuity of the edge lines supporting a pair of lines and / or 5.) average lateral error of each pair, where appropriate for each of the two lane delineations determining the average orthogonal distance of the edge pixels from the line and then calculating the mean. 6. The method according to any one of claims 1 to 5, characterized in that for each recorded image at least one hypothesis is created and / or stored, the course or the orientation or inclination angle of the four lane markings, the associated vanishing point and the associated End quality measure includes. Method according to one of claims 1 to 6, characterized in that, based on the hypotheses of a number of consecutively taken pictures, the width of the lane is calculated and that this value is for defining the range of the distance between the left and right lane markings is used in a subsequently recorded image. 8. The method according to claim 6 or 7, characterized in that for the respective vanishing point of a hypothesis of the inclination angle or the pitch angle of the image pickup unit is calculated and that for successive hypotheses determined values of this tendency to define or delimiting the for finding vanishing points used 2D Hough space for the creation of hypotheses of a subsequent image is used. A computer program product having program code means stored on a computer readable medium for carrying out the method of any one of claims 1 to 8 when the program product is executed on a computer.