KR101404018B1 - Device for recognizing the hand gesture and method thereof - Google Patents

Abstract

The hand movement recognition apparatus according to the present invention detects a movement event or a click event according to a coordinate change of a finger end point in an image captured every frame, rather than detecting an event when a hand shape such as a specific motion stored in advance is photographed, And it is possible to accurately detect a coordinate change even in a variety of hands having different hand sizes or lengths, and does not occupy a large memory for storing specific operation information.
Further, in the present invention, since the infrared camera is used, the outline of the hand can be detected with high robustness against the skin color, and the accuracy of the recognition of the hand movement can be improved.

Description

Technical Field [0001] The present invention relates to a device for recognizing a hand gesture,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technique for recognizing a hand gesture, and more particularly, to a technique for capturing a hand gesture to detect an event.

Conventionally, a hand motion recognition method using a camera sets a recognizable hand shape or a hand pattern in advance, and detects a hand shape in an image input through a camera. The control unit searches for a preset hand shape that best matches the hand shape detected in the database stored in the memory, and generates an event according to the retrieved result. That is, the conventional hand shape recognition method defines various hand shapes corresponding to input signals, and generates an event by comparing hand shapes stored in a previously learned or stored hand shape with those in a current input image.

However, in the conventional hand shape recognition method, a region in which a predetermined color is detected as a skin color of a person is recognized as a hand shape in an area photographed by a camera, and a region having a color similar to that of the skin color is hand- There is a perceived problem. In this case, a hand shape different from a hand shape taken by a user is recognized, and an event different from a user-intended event is performed.

In addition, the conventional hand recognition method uses the motion center point of the hand in recognizing the direction of movement of the hand, which can be easily changed depending on the shape of the hand, and thus the recognition rate is high. In addition, the database needs to store various hand data and corresponding event operations, which also causes an excessive memory occupation.

It is an object of the present invention to provide a technical solution for capturing a movement of a finger coordinate to perform a position movement event or a click event.

According to an aspect of the present invention, there is provided a hand gesture recognition apparatus including a receiving unit for receiving an image including a user's hand, an extracting unit for extracting coordinate values of the end points of the fingers of the hand, And a sensing unit for sensing an event corresponding to a change in the values.

Here, the extracting unit may extract a hand region and a background region from the image using the brightness of the image, detect a curvature of the outline of the hand region, and determine a coordinate value And sequentially associates the predetermined number of coordinate values with the end point coordinate values.

The sensing unit senses a position movement event when it detects a change of a coordinate value corresponding to a stop end point among the end point coordinate values that is greater than or equal to a first value. The sensing unit detects a line segment connecting the detection end point coordinates and the thumb end point coordinates And a change in an angle formed by a line segment connecting the stop end point coordinates and the detection end point coordinates to a second value or more is detected as a click event.

The receiving unit receives an image of the back of the user while the palm of the user touches the floor.

According to another aspect of the present invention, there is provided a method of recognizing a hand movement by a hand movement recognition apparatus, comprising: receiving an image of a user's hand captured by an infrared camera; And detecting an event corresponding to a change in the extracted end point coordinate values.

The step of extracting the end point coordinate values may include separating a hand region and a background region from the image, detecting a curvature of an outline of the hand region, And a step of sequentially mapping the extracted coordinate values to the end point coordinate values.

The detecting of the event may include detecting a line segment connecting the detection end point coordinates and the thumb end point coordinates and detecting a movement event when a change of a coordinate value corresponding to the stop end point among the end point coordinate values is detected, When it detects a change of an angle formed by a segment connecting the stop end point coordinates and the detection end point coordinates to a second value or more, it is detected as a click event.

According to another aspect of the present invention, there is provided a method for recognizing a hand movement by a hand movement recognition apparatus, comprising the steps of continuously acquiring an image of a hand, extracting an image of the hand, And generating an event corresponding to a mouse click when an angle formed by a line segment connecting the end point of the detection and the end point of the thumb and a line segment connecting the end point of the stop and the end point of the detection changes by a predetermined value or more .

The hand-operated recognition apparatus using a camera according to the present invention creates an effect of accurately detecting a coordinate change even in various hands of different sizes or lengths.

1 is a block diagram of a hand-held recognition apparatus according to an embodiment of the present invention;
2 is a view showing a hand position of a camera and a user according to the present invention;
FIG. 3 is a diagram for explaining a method of generating an outline by processing a photographed image of a handgrip recognition apparatus according to the present invention. FIG.
FIG. 4 is a diagram for explaining a method of extracting coordinate values of a finger end point of a hand movement recognition apparatus according to the present invention. FIG.
5 is a view for explaining a method of detecting a movement event of a hand movement recognition apparatus according to the present invention.
6 is a diagram for explaining a method of detecting a click event in the hand-operated recognition apparatus according to the present invention.
FIG. 7 is a flowchart illustrating an operation method of a hand movement recognition apparatus according to an embodiment of the present invention; FIG.

BRIEF DESCRIPTION OF THE DRAWINGS The foregoing and further aspects of the present invention will become more apparent from the following detailed description of preferred embodiments with reference to the accompanying drawings. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of a hand-held recognition apparatus using a camera according to an embodiment of the present invention. 2 to 6 are views for explaining the operation of the hand-operated recognition apparatus according to the present invention. 1, the handwriting recognition apparatus 100 includes a receiving unit 110, an extracting unit 120, and a sensing unit 130. As shown in FIG. In addition, the handgrip recognition apparatus 100 may further include a camera 200 for photographing a user's hand.

Here, the camera 200 may be an infrared camera that detects infrared radiation energy radiated from an object by a detector, extracts the radiation temperature of the object as an electrical signal, and displays it as a two-dimensional visual image. Thus, the use of the infrared camera in the camera 200 is robust to the skin color, and the outline of the hand is clearly extracted. The mounting position of the camera 200 is located at a certain height above the user's hand so that the back of the user can be photographed with the palm of the user touching the floor as illustrated in FIGS. 2A and 2B.

First, the receiving unit 110 of the handwriting recognition apparatus 100 receives an image including a user's hand. Specifically, the receiving unit 110 receives every frame of the image of the user's hand as illustrated in Fig. 3 (a) by the camera 200. [ Since it is a well-known technique to receive the image photographed by the camera 200, a detailed description thereof will be omitted.

The extracting unit 120 extracts coordinate values of the user's fingertip using the image received through the receiving unit 110.

Specifically, the extraction unit 120 separates the hand region and the background region from the image using the brightness of the received image. That is, the extracting unit 120 may divide an area having brightness of a predetermined value or more in the received image into a hand area and the other area into a background area. As described above, the separated area is binarized so that the hand area is separated into white color and the background color is separated into black color as shown in FIG. 3 (b). The extracting unit 120 can generate an outline as shown in FIG. 3 (c) using the connecting element in the separated hand region.

)을 중심으로 하여 양쪽으로 n 간격만큼 떨어져 인접한 두 점(

)을 이용하여 곡률을 검출할 수 있다. 이후, 추출부(120)는 k+1 번째 점(

)을 중심으로 양쪽으로 n 간격만큼 떨어져 인접한 두 점(

)을 이용하여 곡률을 검출할 수 있다. 위와 같이 추출부(120)는 계속적으로 n 간격씩 이동하여 다음 점을 중심으로 하는 곡률을 검출한다. 이렇게 검출된 곡률은 도 4 (a)와 같을 수 있다.The extracting unit 120 detects the curvature of the generated outline. For example, to detect the curvature, adjacent three points among the points having an arbitrary interval (n) on the outline are used. That is, the extracting unit 120 extracts the kth point (

), And two adjacent points spaced apart by n intervals on both sides (

) Can be used to detect the curvature. Then, the extracting unit 120 extracts k + 1th point (

) Adjacent to each other and spaced by n intervals on both sides (

) Can be used to detect the curvature. As described above, the extraction unit 120 continuously moves by n intervals to detect a curvature centered on the next point. The curvature thus detected may be as shown in Fig. 4 (a).

The extraction unit 120 extracts coordinate values of five points having a large curvature among the detected curvature data. The extraction unit 120 sequentially associates the extracted coordinate values with the end point coordinate values of the respective fingers as shown in FIG. 4 (b). For example, when a user's right-handed image is received by the setting of the user, the extraction unit 120 extracts the extracted five coordinate values from the coordinate values located on the left side of the right-handed image as shown in FIG. 4 (b) Quot ;, " stop ", " index finger ", and " thumb " In another example, when the user's left-handed image is received, the extraction unit 120 extracts the extracted five coordinate values from the coordinate value located on the left-hand side of the left-hand image in order of thumb, index, And corresponds to the end point coordinate value of the corresponding finger. Here, whether or not the image including the user's right or left hand is received can be known by using information previously input or set by the user.

The sensing unit 130 senses an event corresponding to the change of the coordinate values of the end points of the finger extracted by the extracting unit 120. The sensing unit 130 may perform an operation corresponding to the sensed event.

For example, the sensing unit 130 senses the position movement event according to the change of the stop end coordinate value among the extracted end point coordinate values. Specifically, the sensing unit 130 correlates the stop end point coordinate value with the displacement of the mouse cursor, and detects the event that the stop end point of the user is shifted by the change of the stop end point coordinate value extracted for each frame by the first value or more . In response, the sensing unit 130 may move the position of the mouse cursor as illustrated in FIG. Here, the first value is a value predetermined by the user in advance, and is a value equal to or larger than a predetermined radius which can be ignored in the coordinate value change caused by hand tremor or fine movement of the user. If the user's hand illustrated in FIG. 5 is assumed to be the right hand, the mouse cursor moves to the left by the user's setting when the user's hand moves to the left, or the mouse cursor moves to the right It can also be moved. In addition, depending on the setting of the user, it can be detected as a position movement event by a change of the end point coordinate value of a finger (for example, a finger) other than the stop.

In another example, the sensing unit 130 senses a click event using a predetermined number of end point coordinate values among the extracted end point coordinate values. More specifically, as illustrated in FIG. 6 (a), the sensing unit 130 may include a line segment connecting a detection end point coordinate and a thumb end point coordinate, and a line segment connecting a stop end point coordinate and a detection end point coordinate, When a change is detected, it is detected as a click event. This is because if the user performs a click operation with the index finger as usual when the user clicks the finger, the coordinates of the end point of the finger tip change according to the click operation and monitor the sudden change of the angle as shown in FIG. 6 (b) . This angle is characterized by the fact that it does not greatly change during hand movement or rotation, but changes very sensitively to click movement.

The sensing unit 130 may perform a click operation of the mouse cursor in response to a change of the angle of the second angle or more. Here, the second value is a value predetermined by the user in advance, and the value of the angle of incidence due to the hand tremor or the fine movement of the user is a value equal to or larger than a predetermined angle of incidence. Also, depending on the setting of the user, it can be detected as a click event even by an angle change using the finger coordinates on both sides of the other finger coordinates.

As described above, according to one embodiment of the present invention, instead of sensing an event when a hand shape such as a specific motion stored in advance is sensed, a change in coordinates of a finger end point in an image captured every frame is detected, Perform a click event. Thus, it is possible to accurately detect the change even for various hands having different characteristics (e.g., size or length of the hand), and does not occupy a large memory for storing specific operation information.

FIG. 7 is a flowchart illustrating an operation method of a hand movement recognition apparatus using a camera according to an embodiment of the present invention.

First, the handwriting recognition apparatus 100 receives an image captured by the camera 200 and including a user's hand (S100). Specifically, the handgrip recognition apparatus 100 receives every frame of the image in which the user's hand is photographed by the camera 200, as illustrated in Fig. 3 (a).

The hand gesture recognition apparatus 100 separates the hand region and the background region from the image using the brightness of the received image (S200). That is, the hand-held recognition apparatus 100 separates an area having brightness of a predetermined value or more from the received image into a hand area, and divides the other area into a background area and binarizes the hand area. As shown in FIG. 3 (b), the hand region may be represented by white, and the background may be represented by black. For accuracy of area separation, the camera 200 may be an infrared camera.

)을 중심으로 하여 양쪽으로 n 간격만큼 떨어져 인접한 두 점(

)을 이용하여 곡률을 검출할 수 있다. 이후, 손동작 인식 장치(100)는 k+1 번째 점(

)을 중심으로 양쪽으로 n 간격만큼 떨어져 인접한 두 점(

)을 이용하여 곡률을 검출할 수 있다. 위와 같이 손동작 인식 장치(100)는 계속적으로 n 간격씩 이동하여 다음 점을 중심으로 하는 곡률을 검출한다. 이렇게 검출된 곡률은 도 4 (a)와 같을 수 있다.Using the connecting element in the separated hand region, the handgrip recognition apparatus 100 generates an outline as shown in FIG. 3 (c), and detects the curvature of the generated outline (S300). For example, in order to detect the curvature, the handgrip recognition apparatus 100 uses adjacent three points among the points having an arbitrary interval n on the outline. That is, the handwriting recognition apparatus 100 recognizes the kth point out of the outline points (

), And two adjacent points spaced apart by n intervals on both sides (

) Can be used to detect the curvature. Thereafter, the handgrip recognition apparatus 100 recognizes the (k + 1) th point

) Adjacent to each other and spaced by n intervals on both sides (

) Can be used to detect the curvature. As described above, the hand-held recognition apparatus 100 continuously moves by n intervals to detect the curvature centered on the next point. The curvature thus detected may be as shown in Fig. 4 (a).

The hand gesture recognition apparatus 100 extracts a predetermined number of coordinate values in order of increasing curvature among the detected curvature data (S400). Here, the predetermined number may be five, and the predetermined number of values may be changed by the user.

The hand gesture recognition apparatus 100 sequentially associates the extracted coordinate values with the end point coordinate values of each of the fingers as shown in FIG. 4B (S500). Specifically, the handgrip recognition apparatus 100 sequentially associates the extracted coordinate values with the end point coordinate values of the respective fingers as shown in FIG. 4 (b).

For example, when a user's right-handed image is received by the setting of the user, the handwriting recognition apparatus 100 extracts the extracted five coordinate values from the coordinate values located on the left side of the right-handed image as shown in FIG. 4 (b) , Stop, detection, and thumb in correspondence with the coordinate values of the end points of the corresponding finger. Alternatively, when a user's left-handed image is received by the user's setting, the handwriting recognition apparatus 100 extracts the extracted five coordinate values from the coordinate values located on the left-hand side of the left-hand image from thumb, Corresponding to the coordinate value of the end point of the corresponding finger. Here, whether or not the image including the user's right or left hand is received can be known by using information previously input or set by the user.

The hand gesture recognition apparatus 100 senses an event corresponding to a change in the end point coordinate values of the extracted fingers (S600). Then, the handgrip recognition apparatus 100 is detected and can perform an operation corresponding to the event.

For example, the hand gesture recognition apparatus 100 detects a position movement event according to a change of the stop end coordinate value among the extracted end point coordinate values. Specifically, the hand movement recognition apparatus 100 associates the stop end point coordinate value with the displacement of the mouse cursor, and when the stop end point coordinate value extracted for each frame changes by a first value or more, the stop end point of the user moves, Detection. Correspondingly, the hand gesture recognition apparatus 100 can move the position of the mouse cursor as illustrated in FIG. Here, the first value is a value predetermined by the user in advance, and is a value equal to or larger than a predetermined radius which can be ignored in the coordinate value change caused by hand tremor or fine movement of the user.

As another example, the hand gesture recognition apparatus 100 senses a click event using a predetermined number of end point coordinate values among the extracted end point coordinate values. Specifically, as illustrated in FIG. 6A, the handgrip recognition apparatus 100 recognizes a second value of an angle formed by a line segment connecting the detection end point coordinates and the thumb end point coordinates, and a line segment connecting the stop end point coordinates and the detection end point coordinates, When the change is detected, it is detected as a click event. This is because, if the user performs a click operation with the index finger as usual when the user clicks the mouse, the coordinate of the end point of the finger tip changes according to the click operation and monitors the change of the angle of abruptness as shown in FIG. 6 (b) . The handgrip recognition apparatus 100 can perform a mouse cursor click operation. Here, the second value is a value predetermined by the user in advance, and the value of the angle of incidence due to the hand tremor or the fine movement of the user is a value equal to or larger than a predetermined angle of incidence.

As described above, according to one embodiment of the present invention, instead of sensing an event when a hand shape such as a specific motion stored in advance is sensed, a coordinate change of a finger end point in an image captured every frame is detected, Perform a click event. Thus, it is possible to accurately detect the change even for various hands having different characteristics (e.g., size or length of the hand), and does not occupy a large memory for storing specific operation information. Further, in the present invention, since the infrared camera is used, the outline of the hand can be detected with high robustness against the skin color, and the accuracy of the recognition of the hand movement is improved.

The present invention has been described with reference to the preferred embodiments. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the present invention is defined by the appended claims rather than by the foregoing description, and all differences within the scope of equivalents thereof should be construed as being included in the present invention.

100: a hand movement recognition device 110:
120: Extracting unit 130:
200: camera

Claims (9)

A receiving unit for receiving an image including a user's hand;
An extracting unit for extracting coordinate values of end points of the fingers of the hand; And
A sensing unit for sensing an event corresponding to a change in the extracted endpoint coordinate values;
, ≪ / RTI &
When the sensing unit senses a change of a line segment connecting the detection end point coordinates and the thumb end point coordinates of the end point coordinate values and a change of an angle formed by a line segment connecting the stop end point coordinates and the detection end point coordinates to a second value or more, And a hand-operated recognition device. The method according to claim 1,
The extraction unit separates the hand region and the background region from the image using the brightness of the image, detects the curvature of the outline of the hand region, and extracts a predetermined number of co-ordinates having a large curvature from the curvature data And sequentially associates the predetermined number of coordinate values with the end point coordinate values. The method according to claim 1,
Wherein the sensing unit senses a position movement event when it detects a change of a coordinate value corresponding to a stop end point of the end point coordinate values that is greater than a first value. delete The method according to claim 1,
Wherein the receiving unit receives an image of the back of the user while the palm of the user is touching the floor. A hand movement recognition method using a hand movement recognition device,
Receiving an image of a user's hand photographed by an infrared camera;
Extracting end point coordinate values of the fingers of the hand; And
Sensing an event corresponding to a change in the extracted endpoint coordinate values;
, ≪ / RTI &
The step of detecting the event comprises:
And generating an event corresponding to a mouse click when an angle formed by a line segment connecting the end point of the index finger and the end point of the thumb and a line segment connecting the end point of the stop and the end point of the fingerprint is changed by more than a predetermined value Hand gesture recognition method. 7. The method of claim 6, wherein extracting the endpoint coordinate values comprises:
Separating a hand region and a background region from the image;
Detecting a curvature of an outline of the hand region;
Extracting a predetermined number of coordinate values having a large curvature among the curvature data; And
Associating the extracted coordinate values sequentially with the endpoint coordinate values;
And a handwriting recognition method. 7. The method of claim 6, wherein detecting the event comprises:
Detecting a change in a coordinate value corresponding to a stop end point of the end point coordinate values that is greater than or equal to a first value, as a position movement event. Successively acquiring a video image of a hand;
Extracting an image of the hand and estimating a fingertip point therefrom; And
Generating an event corresponding to a mouse click when an angle formed by a line segment connecting the end point of the index finger and the end point of the thumb and a line segment connecting the end point of the stop and the end point of the finger exceeds a predetermined value;
And a handwriting recognition method.

Images