JP2000194831A - Marked area image extraction method and apparatus - Google Patents

Abstract

(57) [Problem] To reduce the extraction of an incorrect sign area image from a landscape image using a visible light image and an infrared image. SOLUTION: A candidate for a marker region image is extracted from a visible light image of a landscape, information on a contour of a subject is extracted from an infrared image corresponding to the visible light image, and a candidate for the marker region image and information on the contour are extracted. Is a sign region image extraction method for extracting a sign region image using, calculating the complexity of each candidate region of the sign region image and the region corresponding to the candidate in the infrared image, based on the complexity The candidates for the marker area image are narrowed down, and the ratio between the number of contour pixels present in the region near the outer edge of the narrowed down candidate in the infrared image and the contour length of the narrowed down candidate is calculated based on this ratio. This is a marker region image extraction method for determining a candidate marker region image and extracting a marker region image.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and an apparatus for extracting an image of a sign area using a visible light camera and an infrared camera, and more particularly to a technique which is effective when applied to a car driver assistance apparatus by automatically recognizing signs. It is.

[0002]

2. Description of the Related Art Conventionally, as a method of extracting a sign area image from a visible light image of a landscape by using a feature of a sign, for example, first, a hue of a target sign color is extracted from a visible light image. There is a sign region image extraction method that extracts a set of regions having similar hues and extracts, from the set, a sign region image that is close to the sign of interest as a sign region image (Nakai, Matsuo, Ueda, Umeda, "Examination of Road Sign Extraction Method from Color Road Scene Image", IEICE, Academic Advancement Report, PRU95-158 (1995-11), pp.5
~ 8).

Further, Japanese Patent Application Laid-Open No. H10-255019 discloses a technique for detecting a vehicle existing around an automobile at night using a visible light image and an infrared image corresponding to the visible light image. ing.

[0004]

SUMMARY OF THE INVENTION As a result of studying the above prior art, the present inventor has found the following problems.

In the conventional sign area image extracting method, when an image similar to the characteristic of a target sign is present in a visible light image, there is a problem that the image is erroneously extracted as a sign area image.

For example, from a visible light image showing a building in which a speed regulation sign having a red circle on the outside and a red circle similar to the sign on the wall is shown, the sign having a red color and a circular outer edge. If you try to extract speed signs by using the feature that there is, a red circle drawn on the wall of the building will be extracted as a sign in addition to the speed sign you want to extract,
There is a problem that it is necessary to judge again whether the output result is a sign.

In the technique described in Japanese Patent Application Laid-Open No. H10-255019, a high-luminance portion such as a tail lamp of an automobile is extracted from a visible light image, and the temperature information around the high-luminance portion in an infrared image is extracted. To extract the outer shape of the vehicle in the infrared image based on, when the difference in brightness between the tail lamp and the surroundings during the day is small, or when the sign itself is not a light source, the visible light image is used. Extraction is difficult.

Further, since only the outer shape is extracted from the infrared image, there is a problem that even if a marker is extracted, it is not possible to know what the marker is.

An object of the present invention is to provide a technique capable of reducing the extraction of an incorrect sign area image from a landscape image using a visible light image and an infrared image.

The above and other objects and novel features of the present invention will become apparent from the description of the present specification and the accompanying drawings.

[0011]

SUMMARY OF THE INVENTION Among the inventions disclosed in the present application, the outline of a representative one will be briefly described.
It is as follows.

(1) A candidate for a marker region image is extracted from a visible light image of a landscape, information on a contour of a subject is extracted from an infrared image corresponding to the visible light image, and the candidate for the marker region image is extracted from the contour image. A sign region image extracting method for extracting a sign region image by comparing with the information of the candidate region, calculating the complexity of each of the candidate for the sign region image and the region corresponding to the candidate in the infrared image, The candidate of the marker area image is narrowed based on the degree, and the ratio of the number of pixels representing the outline existing in the area corresponding to the vicinity of the outer edge of the narrowed candidate in the infrared image and the outline length of the narrowed candidate Is calculated, the candidate of the marker region image is determined based on the ratio, and the marker region image is extracted.

(2) In the method for extracting a marker region image according to (1), the calculation of the complexity is performed by selecting each candidate of the marker region image and each pixel of the region corresponding to the candidate in the infrared image. Are binarized by brightness, and the number of times the binarized brightness is replaced by scanning each pixel of the candidate for the marker area image and the area corresponding to the candidate in the infrared image in the vertical and horizontal directions. This is a method of calculating and calculating by dividing the number of times of replacement by the area of the candidate.

(3) In the method for extracting a marker area image according to the above (1) or (2), the pixel representing the outline is a circumscribed rectangle of the area corresponding to the narrowed candidate in the infrared image and the circumscribed rectangle. Calculate the center of the rectangle, from the center of the circumscribed rectangle, draw a straight line that reaches the circumscribed rectangle through a predetermined pixel existing in the region near the outer edge of the candidate for the marker region image, and on the pixels on the straight line, Pixels present in a region near the outer edge of the candidate for the marker region image.

(4) A visible light camera for photographing a landscape with visible light, an infrared camera having the same optical axis and angle of view as the visible light camera, and a visible light image photographed by the visible light camera,
A marker region image candidate extracting means for extracting a marker region image candidate based on a feature of the marker; and a subject for extracting, from the infrared image captured by the infrared camera, information on a contour of the subject based on temperature information in the image. A marker region image extracting apparatus comprising: a contour extracting unit; and a marker region image candidate evaluating unit that compares the marker region image candidate with the contour information and extracts a marker region image from the marker region image candidates. Wherein the marker region image candidate evaluating means calculates the complexity of the marker region image candidate and the region corresponding to the candidate in the infrared image, and the marker region image candidate based on the complexity. Means for calculating the ratio between the number of contour pixels existing in the area near the outer edge of the narrowed candidate in the outline image and the outline length of the narrowed candidate. If, based on the ratio, determined by the number of pixels indicating the contour of the subject in the infrared image present in the vicinity of the outer edge of the candidate, an apparatus having a means for extracting the label area image.

(5) In the marker region image extracting device according to (4), the means for calculating the complexity includes a candidate for the marker region image and each pixel in a region corresponding to the candidate in the infrared image. Means for binarizing the image according to the brightness, and scanning the pixels in the candidate corresponding to the marker area image and the area corresponding to the candidate in the infrared image in the vertical direction and the horizontal direction to obtain the binary brightness. An apparatus comprising: means for calculating the number of times of replacement; and means for calculating the complexity by dividing the number of times of replacement by the area of the candidate.

(6) In the sign region image extracting device according to (4) or (5), the means for extracting a sign region image from the candidates for the sign region image includes the narrowed candidate in the infrared image. A means for calculating a circumscribed rectangle of an area corresponding to the above, a center of the circumscribed rectangle, and a pixel representing a contour present in an area near an outer edge of the narrowed-down candidate from the center of the circumscribed rectangle. The apparatus includes means for drawing a straight line reaching a rectangle, and means for extracting, from pixels on the straight line, pixels existing in a region near the outer edge of the candidate for the marker region image.

Hereinafter, the present invention will be described in detail together with embodiments (examples) with reference to the drawings.

In all the drawings for explaining the embodiments, parts having identical functions are given same symbols and their repeated explanation is omitted.

[0020]

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 is a block diagram showing a schematic configuration of a sign area image extracting apparatus according to the present embodiment.

As shown in FIG. 1, a marker region image extracting apparatus according to this embodiment includes a visible light camera 1 for photographing a landscape with visible light, and an infrared camera having the same optical axis and angle of view as the visible light camera 1. 2, a marker region image candidate extracting unit 3 for extracting a marker region image candidate from the visible light image captured by the visible light camera 1 using the characteristic of the marker, and an infrared image captured by the infrared camera 2. And the subject contour extracting means 4 for extracting the contour information of the subject based on the temperature information in the infrared image, collating the candidate for the marker area image with the contour information of the infrared image, and comparing the candidate in the infrared image with the candidate. It is determined by the number of contour pixels existing in the region near the outer edge of the marker region image whether or not the image is a marker region image, and the marker region image candidate evaluation unit 5 for extracting the marker region image and the marker region image candidate evaluation unit 5 Sign area And an output device 6 such as a display for displaying an image.

A personal computer (CPU) is used as the marker region image candidate extracting unit 3, the subject outline extracting unit 4, and the marker region image candidate evaluating unit 5, for example.

The visible light camera 1 and the infrared camera 2 are arranged at predetermined positions of the vehicle, and photograph a scene in front of the vehicle.

The infrared camera 2 has the same optical axis and angle of view as the visible light camera 1, and can capture the same region as the visible light image Gv in the image showing the temperature of each part of the subject in the visible light image Gv. It is like that.

FIG. 2 is a schematic diagram for explaining the processing of the marker area image candidate extracting means 3, and FIG. 2A is a visible light image Gv taken by the visible light camera 1, and FIG. )
Is an extraction result image Gv 'obtained by extracting candidate image marker regions by the marker region image candidate extracting means 3.

First, a road sign or a building is photographed by the visible light camera 1 and the visible light image Gv (FIG. 2A) is inputted to the sign area image candidate extracting means 3, and the characteristic of the sign is used. For example, when extracting a speed regulation sign, a set of regions having a red color in the visible light image is first extracted by using the fact that the outside of the sign is a red circle, and the outer edge is circular from the set. Is selected as a candidate R for a marker area image, and an extraction result image G as shown in FIG.
v ′ (Nakai, Matsuo, Ueda, Umeda, “Examination of Road Sign Extraction Method from Color Road Scene Image”, IEICE, Advanced Technical Report, PRU95-158, pp.5
~ 8).

For example, at this time, in addition to the speed regulation sign,
Assuming that a red circle similar to a speed regulation sign is drawn on the wall surface of the building, as shown in FIG. 2B, candidate R1 and R2 of the sign area image are extracted.

FIG. 3 is a schematic diagram for explaining the processing of the object outline extracting means 4, and FIG.
3B is an infrared image Gi showing the temperature of each part of the subject in the visible light image Gv, and FIG. 3B is a contour image Gi ′ in which only the contour of each subject is extracted by the subject contour extracting means 4. is there.

As shown in FIG. 3A, the infrared image Gi represents the temperature of each part of the subject shown in the visible light image Gv. Since the temperature of the subject has a property that tends to be uniform regardless of the state of the surface and a property that varies depending on the subject, the sign and its support can be clearly distinguished from the building behind it.

The subject contour extracting means 4 extracts the contour information of the subject from the infrared image Gi, and outputs the contour image Gi '.
For example, “Takagi and Shimoda,“ Image Analysis Handbook ””, University of Tokyo Press, pp. 689-69
0 ”, the infrared image Gi is divided (region division) into partial images that can be regarded as having a locally uniform density by the simple region extension method, and then the pixels constituting the outline of each divided region are divided. To create a contour image Gi 'as shown in FIG.

At this time, in the infrared image Gi of FIG. 3A, the red circle drawn on the wall of the building as shown in FIG. 2A is not shown because it is not different from the temperature of the building. , The outline of the red circle is not extracted.

The candidate R of the marker area image obtained by the marker area image candidate extracting means 3 is used by the marker area candidate determining means 5 to obtain the contour image G obtained by the subject contour extracting means 4.
The image is compared with i ′, and a marker region image is extracted from the candidates R.

FIG. 4 is a flowchart showing the processing procedure of the marker area image candidate evaluation means 5, and FIG. 5 is a flowchart showing the procedure of calculating the complexity F in steps 501 and 502 in FIG.

First, the complexity Fv of the candidate R for the marker area image
Is calculated (step 501), and then the complexity Fi of the area corresponding to the candidate R in the contour image Gi 'is calculated in the same procedure as the complexity Fv (step 502).

In the method of calculating the complexity F in steps 501 and 502, first, as shown in FIG. 5, each pixel in the candidate R is binarized by brightness, and for example, two colors of white and black are used. Display is made (step 510). Then, the pixels in the binarized candidate R are scanned in the vertical direction to check the number of times that black changes to white, and then scanned in the horizontal direction to check the number of times that black changes to white. The total number T is calculated (step 511). Step 51
The complexity F is obtained by dividing the total number T obtained in 1 by the area of the candidate R (step 512).

Next, using the above-mentioned complexity Fv and Fi,
The candidates R for the marker area image are narrowed down (step 503).

At step 503, the extraction result image Gv '
In this example, the character drawn on the sign is also photographed, so that the sign region image is complicated. The contour image Gi 'has the feature that the density other than the contour portion is uniform because it is not complicated. The threshold value tv for the complexity Fv of the candidate R and the threshold value ti for the complexity Fi of the region in the contour image Gi 'are determined in advance, and Fv> tv and Fi <ti
Are narrowed down to only the marker region image R which becomes.

Next, the contour of the candidate R for the marker area image narrowed down in step 503 is checked (step 50).
4).

In step 504, a contour present in a region corresponding to a band-like region (hereinafter, referred to as the vicinity of the outer edge) within a small fixed distance around the outer edge of the candidate R of the marker region image in the outline image Gi 'is represented. The number of contour pixels of the candidate R is determined from the pixels (hereinafter referred to as contour pixels), and the ratio Fp between the number of contour pixels and the contour length of the candidate R is determined.

FIG. 6 is a diagram for explaining how to determine the contour pixels of the candidate R in step 504.
A set of contour pixels in an area corresponding to candidate R in i ′, where C is a circumscribed rectangle corresponding to candidate R, and O is a circumscribed rectangle C
, X is a predetermined pixel existing near the outer edge of the candidate R for the marker area image, and P is a point where an extension of a straight line connecting the center O and the predetermined pixel X is in contact with the circumscribed rectangle C.

FIG. 7 is a flowchart showing the procedure for obtaining the contour pixels in step 504. Hereinafter, a method of obtaining an outline pixel will be described with reference to FIGS.

First, a circumscribed rectangle C of an area corresponding to the candidate R in the contour image Gi 'is obtained, and a center O of the circumscribed rectangle C is calculated (step 515).

Next, a straight line OX connecting the center O and a predetermined pixel X existing in the area near the outer edge of the candidate R is drawn, and a straight line OP extending from the straight line to the circumscribed rectangle C is drawn (step 516). At this time, a value of a minute constant distance around the outer edge which is set near the outer edge is given from outside the program.

Next, with respect to the pixels existing on the straight line OP obtained in step 516, it is checked whether or not there is a contour pixel other than the pixel X (step 517).

As a result of the check in step 517, if there is no contour pixel other than the pixel X to be determined on the straight line OP, the pixel X to be determined is set as the contour pixel of the candidate R, and at least one contour pixel other than the pixel X to be determined is determined. , The pixel X to be determined is not set as the contour pixel of the candidate R.

The processing of steps 515 to 517 is performed on the contour pixels near the outer edge of the candidate R, and the total number T of contour pixels is obtained.

Next, although not shown in FIG. 7, the length (outline length) of the outer edge of the candidate R is determined, and the total number T of the outline pixels is calculated.
Is divided by the outer edge length to obtain a value Fp.

Next, using the Fp obtained in step 504, a marker area image is extracted from the narrowed candidates for the marker area image (step 505).

The extraction method at this time is determined by determining whether or not there are many contour pixels (the number of contour pixels) in a region corresponding to the vicinity of the outer edge of the candidate R in the contour image Gi '. The value tp is determined, and the candidate R satisfying Fp> tp is extracted as a marker region image.

For example, FIG. 8 is a diagram in which candidates R1 and R2 of the marker area image of FIG. 2B are superimposed on the contour image Gi 'of FIG. 3B, and the judgment is made based on FIG. In this case, the candidate R1 is determined and extracted as a marker region image because the divided value Fp becomes large because there are many contour pixels in the vicinity R1 ′ of the outer edge of the candidate R1 in the contour image Gi ′. . On the other hand, for the candidate R2, the contour image G
Since there is no contour pixel in the vicinity R2 ′ of the outer edge of the candidate R2 in i ′, the divided value Fp is small, and is not determined as a marker area image and is not extracted.

In FIG. 8, for the sake of simplicity, the neighborhoods R1 ', R1 near the outer edges of the marker region image candidates R1, R2 are shown.
2 'is drawn larger.

According to the above-described procedure, the sign area image extracted by the sign area image evaluation means 5 is output to an output device 6 such as a display.

In the present embodiment, the temperature of each subject is often constant for each subject regardless of the surface condition, and the contour of each subject can be extracted from the infrared image without being affected by the surface condition. Therefore, even if the marker region image candidate R extracted from the visible light image Gv using the features of the marker includes an erroneously extracted marker region image, the contour information obtained from the infrared image Gi is included. , It is possible to reduce the extraction of an erroneous sign area image from a landscape image.

As described above, the present invention has been specifically described based on the above-described embodiment. However, the present invention is not limited to the above-described embodiment, and may be variously modified without departing from the gist thereof. Of course.

[0055]

The effects obtained by typical aspects of the invention disclosed in the present application will be briefly described as follows.

The candidate of the marker region image extracted from the visible light image is compared with the infrared image, and the outline pixel of the candidate of the marker region image is examined to extract the wrong marker region image from the landscape image. Can be reduced.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a schematic configuration of a marker region image extracting device according to an embodiment of the present invention.

FIG. 2 is a schematic diagram for explaining processing of a marker area image extracting unit of the embodiment.

FIG. 3 is a schematic diagram for explaining a process of a subject contour extraction unit according to the embodiment;

FIG. 4 is a flowchart showing a processing procedure of a marker area image candidate evaluation unit of the embodiment.

FIG. 5 is a flowchart illustrating a procedure for calculating the complexity in FIG. 4;

FIG. 6 is a diagram for explaining how to obtain the contour pixels in FIG. 4;

FIG. 7 is a flowchart illustrating a processing procedure for obtaining an outline pixel in FIG. 4;

FIG. 8 is a schematic diagram for explaining a processing result of a marker area image candidate evaluation unit of the embodiment.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Visible light camera, 2 ... Infrared camera, 3 ... Signed area image candidate extraction means, 4 ... Subject outline extraction means, 5 ... Signed area image candidate evaluation means, 6 ... Output equipment, Gv ... Visible light image, Gi ... Infrared Image, Gv '... Extraction result image, Gi'
... Contour image, R, R1, R2 ... Candidate of marker area image, F
v: complexity of a candidate for a marker area image; Fi: complexity of a region corresponding to a candidate in a contour image; Fp: ratio of the number of contour pixels to a contour length; S: contour of a region corresponding to a candidate in a contour pixel ,
C: circumscribed rectangle of the outer edge of the candidate, O: center of the circumscribed rectangle, X:
A predetermined pixel, P: a point where the extension of the straight line OX contacts the circumscribed rectangle C, near the outer edge of R1 '... R1, near the outer edge of R2' ... R2.

Continued on the front page (51) Int.Cl. ⁷ Identification code FI Theme coat II (Reference) G06F 15/70 350D 355 (72) Inventor Motohiro Machida 3-19-2 Nishishinjuku, Shinjuku-ku, Tokyo Nippon Telegraph and Telephone Corporation (72) Inventor Osamu Nakamura 3-19-2 Nishi-Shinjuku, Shinjuku-ku, Tokyo Nippon Telegraph and Telephone Corporation (72) Inventor Toshiaki Sugimura 3-2-1-2 Nishi-Shinjuku, Shinjuku-ku, Tokyo Nippon Telegraph and Telephone Corporation F term (reference) 2D064 AA15 BA01 EA03 EA17 EB08 FA04 5B050 BA11 DA04 DA05 EA06 EA18 5B057 AA16 BA02 CA01 CA12 CA16 DA06 DB02 DB06 DC03 DC16 5H180 AA01 CC02 CC04 LL01 LL08 5L096 AA01 FA04 FA06 FA05 FA04

Claims (6)

[Claims] 1. A candidate for a marker region image is extracted from a visible light image of a landscape, information on a contour of a subject is extracted from an infrared image corresponding to the visible light image, and the candidate for the marker region image is extracted from the contour image. A sign region image extraction method for extracting a sign region image by collating with information, calculating a complexity of each of the candidate for the sign region image and a region corresponding to the candidate in the infrared image; Based on the narrowed candidates of the marker area image, calculate the ratio of the number of pixels representing the contour present in the region near the outer edge of the narrowed candidate in the infrared image and the outline length of the narrowed candidate, A method for extracting a marker region image, comprising: determining a candidate for the marker region image based on the ratio; and extracting a marker region image. 2. The marker region image extracting method according to claim 1, wherein the calculation of the complexity includes each pixel of a candidate of the marker region image and a region corresponding to the candidate in the infrared image. Each pixel of the candidate corresponding to the marker area image and the area corresponding to the candidate in the infrared image is scanned in the vertical direction and the horizontal direction, and
A method for extracting a marker region image, comprising: calculating the number of times the quantified lightness is replaced; and dividing the number of times by the area of the candidate. 3. The marker region image extracting method according to claim 1, wherein the pixel representing the outline includes a circumscribed rectangle corresponding to an area corresponding to the narrowed candidate in the infrared image. The center of the circumscribed rectangle is calculated, and a straight line reaching the circumscribed rectangle is drawn from the center of the circumscribed rectangle through a predetermined pixel existing in a region near the outer edge of the candidate for the marker region image, and a pixel on the straight line is drawn. A marker region image extracting method, wherein the pixels are present in a region near an outer edge of the candidate marker region image. 4. A feature of a sign based on a visible light camera for photographing a landscape with visible light, an infrared camera having the same optical axis and angle of view as the visible light camera, and a visible light image photographed by the visible light camera. A marker region image candidate extracting means for extracting a candidate for a marker region image based on the image information; and a subject contour extracting means for extracting information on the contour of the subject based on temperature information in the image from the infrared image captured by the infrared camera. A marker region image extraction device having a marker region image candidate evaluation unit that compares the marker region image candidate with the contour information, and extracts a marker region image from the marker region image candidates. A sign region image candidate evaluating unit that calculates a complexity of the candidate for the sign region image and a region corresponding to the candidate in the infrared image, and a sign region image candidate evaluation unit based on the complexity. Means for narrowing down candidates, means for calculating the ratio of the number of contour pixels in the area near the outer edge of the narrowed-down candidate in the outline image to the length of the narrowed-down candidate, and based on the ratio, Means for extracting a marker region image by determining the number of pixels representing the contour of the subject in the infrared image existing near the outer edge of the candidate, and extracting a marker region image. 5. The marker region image extracting apparatus according to claim 4, wherein the means for calculating the complexity includes a candidate for the marker region image and a region corresponding to the candidate in the infrared image. Means for binarizing pixels according to lightness, and the binarized lightness by scanning each pixel of the candidate for the marker area image and the area corresponding to the candidate in the infrared image in a vertical direction and a horizontal direction. A sign area image extracting apparatus, comprising: means for calculating the number of times the pieces are replaced; and means for calculating the complexity by dividing the number of times the pieces are replaced by the area of the candidate. 6. The marker region image extracting device according to claim 4, wherein the means for extracting a marker region image from the marker region image candidates includes selecting the marker candidates in the infrared image. Means for calculating the circumscribed rectangle of the corresponding area, the center of the circumscribed rectangle, and from the center of the circumscribed rectangle, passing through the pixel representing the contour existing in the area near the outer edge of the narrowed candidate, And a means for extracting a pixel existing in a region near the outer edge of the candidate for the marker region image from pixels on the straight line.