KR20110119933A - Image recognition device and method using stereo camera - Google Patents

Abstract

Disclosed are an image recognition apparatus and method using a stereo camera. The image processing apparatus of the present invention recognizes the object by extracting a moving object from the image, obtaining a central axis of the object and a length representing the object, and comparing the object with a preset value or a pattern mask. Here, the actual length representing the object may be obtained using 3D depth map data acquired using two cameras.

Description

Video Analysing Apparatus and Method Using Stereo Cameras

The present invention relates to an image recognition apparatus and method using a stereo camera that can recognize an object based on three-dimensional depth map data acquired using two cameras.

In order to obtain a depth map on a three-dimensional space from an image, that is, a distance from a subject in a three-dimensional space, a method of using a stereo camera, a method of using a laser scan, or a method of using a time of flight (TOF) The back is known.

Among these, stereo matching using a stereo camera is a hardware implementation of a process of recognizing a stereoscopic object using two eyes, and a pair of images obtained by photographing the same subject with two cameras. It is a method of extracting information about depth (or distance) in space through the interpretation process of. To this end, binocular differences on the same Epipolar Line of images obtained from two cameras are calculated. The binocular difference includes distance information, and the geometrical characteristic calculated from the binocular difference becomes the depth. When the binocular difference value is calculated in real time from the input image, three-dimensional distance information of the observation space can be measured.

Known as a stereo matching algorithm, for example, "image matching method using a plurality of image lines" of the Republic of Korea Patent No. 0517876 or "binocular difference estimation method for three-dimensional object recognition" of the Republic of Korea Patent No. 0601958.

An object of the present invention is to provide an image recognition apparatus and method using a stereo camera, which can recognize a subject based on 3D depth map data acquired using two cameras.

In order to achieve the above object, an image recognition method using a stereo camera of the present invention includes generating a pair of digital images using two cameras photographing the same region, and using the converted pair of digital images. Calculating 3D depth map data, extracting a region of a moving object by comparing one of the digital images with a reference background image, extracting a central axis of the extracted object, and identifying the depth map data Calculating a representative length which is an actual length representing the object based on distance information to the object, and when the representative length of the calculated object is within a first range, a plurality of pattern masks mapped to the first range; Recognizing the object by a method of comparing the central axis of the extracted object.

Here, the representative length of the object is preferably any one of the actual length of the central axis, the actual width of the object and the actual height of the object.

According to an embodiment, in the image processing method of the present invention, when the representative length of the calculated object is within a second range different from the first range, the plurality of pattern masks mapped to the second range and the extracted object The method may further include recognizing the object by a method of comparing the central axes.

Image recognition apparatus using a stereo camera according to another embodiment of the present invention, a stereo camera unit, a distance information calculation unit, an object extraction unit, a central axis extraction unit and an object recognition unit.

The stereo camera unit includes two cameras for capturing the same area, and generates a pair of digital images, and the distance information calculator calculates 3D depth map data using the pair of digital images generated by the stereo camera unit. do.

The object extractor extracts an area of the moving object by comparing one of the digital images generated by the stereo camera unit with a reference background image, and extracts a central axis of the extracted object.

The object recognition unit calculates a representative length which is an actual length representing the object based on the distance information to the object identified from the depth map data, and if the calculated representative length of the object falls within the first range, The object is recognized by comparing a plurality of mapped pattern masks with a central axis of the extracted object.

The image recognition device of the present invention can recognize a moving object in the photographing area in a simpler method. In this method, the recognition algorithm is relatively simple compared to the two-dimensional image processing, instead of processing the image generated using the two cameras, the recognition speed and efficiency is improved, and above all, the recognition rate is excellent.

1 is a block diagram of a 3D image recognition device according to an embodiment of the present invention;
2 is a flowchart provided to explain a three-dimensional image recognition process of the present invention;
3 is a view showing an image processing result in the step of extracting an object region from an image;
4 is a view provided for explaining a method of extracting a central axis of an object; and
5 is a diagram provided to explain a method of calculating a representative length of an object.

BEST MODE FOR CARRYING OUT THE INVENTION The present invention will be described in more detail with reference to the drawings.

Referring to FIG. 1, the image recognition device 100 of the present invention includes a stereo camera unit 110 and an image processor 130 to recognize a subject in a three-dimensional space.

The stereo camera unit 110 includes a first camera 111, a second camera 113, and an image receiver 115.

The first camera 111 and the second camera 113 are a pair of cameras spaced apart from each other to photograph the same area, and are called a stereo camera. The first camera 111 and the second camera 113 output an analog image signal photographing an area to the image receiver 115.

The image receiver 115 converts a video signal (or image) of a continuous frame input from the first camera 111 and the second camera 113 into a digital image and synchronizes the frame to the image processor 130 in synchronization with the frame. to provide.

According to an exemplary embodiment, the first camera 111 and the second camera 113 of the stereo camera unit 110 may be a camera that generates a digital video signal instead of an analog image. In this case, the image receiver 115 may be different. It provides an interface with the image processor 130 without conversion processing and serves to match frame synchronization of a pair of images.

In addition, the stereo camera unit 110 may further include a wired or wireless interface for connecting to the image processing unit 130 through an IP (Internet Protocol) network.

The image processor 130 extracts an area of an object moving on the shooting area from the pair of digital image frames output from the stereo camera unit 110 to determine whether the object is an object of interest, and continuously from the stereo camera unit 110. The above determination process may be performed in real time on all frames of the image (video) that is input to the image.

For the above process, the image processor 130 includes a distance information calculator 131, an object extractor 133, and an object recognizer 137. Hereinafter, operations of the distance information calculator 131, the object extractor 133, the central axis extractor 135, and the object recognizer 137 will be described with reference to FIG. 2.

First, when the first camera 111 and the second camera 113 generates an analog video signal, the image receiver 115 converts the analog video signal into a digital video signal and then synchronizes the frame to the image processor 130. Provided to (step S201).

Depth Map Data Calculation: Step S203

The distance information calculator 131 calculates 3D depth map data including distance information of each pixel from a pair of digital images received in real time from the image receiver 115.

Here, the distance information of each pixel is binocular difference information obtained by the stereo matching method described in the prior art, and the "three-dimensional image matching method using a plurality of image lines" of the Republic of Korea Patent No. 0517876 or the Republic of Korea Patent No. 0601958 The depth map data calculated by the distance information calculator 131 may include distance information about each pixel. .

<Extract region of moving object: step S205>

The object extractor 133 extracts a region of the moving object from one image of the pair of digital images input through the image receiver 115. Here, the moving object refers to an object existing in the photographing area of the camera and an object whose position or motion is changed or newly entered into the photographing area.

The method of extracting the area of the moving object may be variously performed. For example, the object extracting unit 133 of the present invention extracts a region of a moving object by a method of subtracting a background image previously held from an input image frame. Here, the subtraction operation is performed by subtracting pixel values of each pixel of two corresponding image frames. In addition, the reference background image is an image in which no moving object is set, and the object extractor 133 may store and use the reference background image in a storage medium (not shown).

Furthermore, since a difference image may be generated due to camera noise or a change in illumination of the photographing area, the object extractor 133 may perform a Gaussian distribution on the result of the subtraction operation. Background Modeling, which applies the Distribution process, can cope with noise or light changes.

Referring to FIG. 3, (a) is an image input from the image receiver 115, (b) is a basic background image, and (c) is a result image of a subtraction operation. Referring to FIG. 3C, it can be seen that a region of a moving object is extracted from an image input from the image receiver 115.

<Detect Edge of Moving Object: Step S207>

The object extractor 133 detects an outline of a moving object by performing outline detection on the resultant image of the subtraction operation of step S207. Edge detection is handled using different types of edges, depending on the borderline width and shape of the object.

The object extractor 133 may remove a noise by applying a morphology operation to a subtraction image and simplify an outline or a skeleton line to detect an outline. The morphology operation can basically use erosion operation to remove noise and dilation operation to fill small holes in an object.

<Extract central axis of moving object: step S209>

The central axis extractor 135 extracts a media axis of an object having a width of 1 pixel by applying a skeletonization or thinning algorithm to the object extracted by the object extractor 133. Various known methods can be applied, such as the Medial Axis Transform (MAT) algorithm using the outline or Zhang Suen algorithm.

For example, when the central axis is transformed, the central axis a of the object is a set of points having a plurality of boundary points among the respective points (or pixels) in the object R as shown in FIG. 4. Here, the boundary point refers to a point closest to the point in the object among the points on the outline B, and the points b1 and b2 on the outline become the boundary point of the point P1 in the object R. Therefore, the central axis algorithm is a process of extracting points having a plurality of boundary points and may be expressed as in Equation 1 below.

Where P _ma is a central axis represented by a set of x, x is a point present in the object R, and b _min (x) is the number of boundary points of the point x. Thus, the central axis is a set of points x whose number of boundary points is greater than one. Here, in order to calculate the boundary point, the structure of the skeleton may change somewhat according to a method of obtaining a distance from an internal point x to an arbitrary pixel on the outline (for example, 4-Distance, 8-Distance, Euclidean Distance, etc.). .

In addition, when the object is of a relatively simple form, the center line may be extracted by extracting a peak value of the Gaussian value for the object, and in this case, the edge detection step of step S207 may be omitted. .

<Calculation of representative length of moving object: step S211>

The object recognition unit 137 obtains the representative length of the object extracted in the step S207 or S209 using the depth map data obtained in the step S205.

The representative length of the object is a value calculated from an image as an actual length of an object set to represent the object, and may correspond to an actual length of a central axis, an actual width of an object, or an actual height of an object. However, the representative length of the object is affected by the position of the camera, the shooting angle, and the characteristics of the shooting area.

Further, the calculation of the actual length of the object, after calculating the actual length per pixel (hereinafter referred to as the 'unit length' of the pixel) at the distance (do) where the object extracted in step S205 is located, the pixel representing the object This is done by multiplying the number of. Here, the number of pixels representing the object may correspond to the number of pixels forming the central axis, the number of pixels to be the width or height of the object.

The width or height of the object, as the number of pixels representing the object, can be obtained through the range of the x-axis coordinate or the y-axis coordinate of the object area, and the length of the central axis is, for example, the number of pixels included in the central axis. It can be obtained by adding.

The unit length of a particular pixel varies from pixel to pixel (exactly depending on the depth of the pixel), and can be obtained as follows with reference to FIG. 5. Here, for convenience of explanation, it is assumed that the size of the image frame is 720x640 pixels.

5, the actual length Lmax corresponding to the vertical axis (or horizontal axis) of the entire frame at the maximum depth D based on the existing background image, and the vertical axis (or horizontal axis) of the entire frame at the position do of the extracted object. The corresponding actual length L (do) is indicated. First, the actual length L (do) corresponding to the vertical axis (or the horizontal axis) of the entire frame at the depth do where the object is located may be obtained as in Equation 2 below.

Here, L (do) is the actual length corresponding to the vertical axis (or the horizontal axis) of the entire frame at the depth do, and Lmax is the vertical axis (or the horizontal axis) of the entire frame at the maximum distance L based on the existing background image. The corresponding actual length, do is the depth of the object, and D is the maximum depth.

Next, the actual length L (do) corresponding to the vertical axis (or horizontal axis) of the entire frame at the distance do is located, is the number of pixels Px, Py in the vertical axis (or horizontal axis) of the entire frame. = 720, Py = 640), the unit length L _p (do) of the pixels included in the object region can be obtained as in Equation 3 below.

Here, L _p (do) is the unit length of the pixels included in the object region located at the depth do, and Py is the number of pixels along the vertical axis of the entire frame. According to Equation 2, it can be seen that L _p (do) depends on the depth to the corresponding object confirmed from the distance information of the 3D depth map data and the maximum depth on the map data.

When the unit length of the pixel is obtained, the object recognition unit 137 obtains the representative length of the object. The representative length of the object may be obtained by the following equation (4) by multiplying the unit length L _p (do) of the pixel by the number po of pixels representing the object.

Here, po is the number of pixels representing the object.

<Object Recognition by Skeleton Pattern and Size of Moving Object: Steps S213 to S217>

When the unit length of the pixel is obtained, the object recognition unit 137 recognizes the object using the representative length of the object and the skeleton of the object.

First, the object recognition unit 137 primarily determines whether the representative length of the object falls within the first range (S213).

Here, the first range is a range of a representative length of the first thing (or a group of things) to be searched. As described above, the representative length of the object is affected by the position of the camera, the shooting angle, and the characteristics of the shooting area. For example, in the case of a camera photographing a road at an angle of 45 ° downward from an intersection and a photograph of a closed space at a relatively low height, the representative length of the same object may be different. Also, depending on the position of the camera, the representative lengths of various objects existing in the same image may also vary according to their morphological characteristics. Therefore, the representative length should be set in consideration of these points.

If the representative length of the object does not belong to the first range, the object recognition unit 137 may again determine whether the representative length of the object belongs to the second range corresponding to the representative length of the second object (S215).

When the representative length of the object is within the first range, the object recognition unit 137 secondly determines whether the pattern of the central axis of the object matches one of the pattern masks of the first group, and finally recognizes the object. do. In other words, as the first range corresponds to the first object or the objects of the first group, the pattern mask of the first group may also correspond to the first object or the objects of the first group. Here, the coincidence of the pattern may mean a coincidence within an acceptable range.

Through the above process, the image recognition device of the present invention obtains 3D depth map data using a stereo camera and recognizes an object captured in the image.

Unlike what is described herein as time series preceding, the calculation of the depth map data of step S203 can be performed in parallel with the extraction process of the moving object of steps S205 and S207 as shown in FIG. And after step S207.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is clearly understood that the same is by way of illustration and example only and is not to be construed as limiting the scope of the invention as defined by the appended claims. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present invention.

Claims (5)

Generating a pair of digital images using two cameras photographing the same area;
Calculating 3D depth map data using the converted pair of digital images;
Extracting an area of a moving object by comparing one of the digital images with a reference background image;
Extracting a central axis of the extracted object;
Calculating a representative length which is an actual length representing the object based on distance information to the object identified from the depth map data; and
If the representative length of the calculated object is within a first range, recognizing the object by comparing a plurality of pattern masks mapped to a first range with a central axis of the extracted object; Image recognition method using stereo camera.
The method of claim 1,
The representative length of the object is
The image recognition method using a stereo camera, characterized in that any one of the actual length of the central axis, the actual width of the object and the actual height of the object.
The method of claim 1,
Recognizing the object by a method of comparing the central axis of the extracted object with a plurality of pattern masks mapped to the second range when the representative length of the calculated object falls within a second range different from the first range. Image recognition method using a stereo camera comprising a.
A stereo camera unit having two cameras for capturing the same area and generating a pair of digital images;
A distance information calculator for calculating 3D depth map data using a pair of digital images generated by the stereo camera unit;
An object extracting unit extracting an area of a moving object by comparing one of the digital images generated by the stereo camera unit with a reference background image;
A central axis extracting unit extracting a central axis of the extracted object; And
A representative length that is an actual length representing the object is calculated based on the distance information to the object identified from the depth map data, and a plurality of mapped to the first range when the representative length of the calculated object falls within a first range And an object recognizing unit for recognizing the object by comparing two pattern masks with a central axis of the extracted object.
The method of claim 4, wherein
The representative length of the object is
And an actual length of the central axis, an actual width of the object, and an actual height of the object.

Images