CN113296675A

CN113296675A - Natural gesture finger recognition algorithm

Info

Publication number: CN113296675A
Application number: CN202110454722.4A
Authority: CN
Inventors: 刘传义; 李雅洁
Original assignee: Lanzhou University
Current assignee: Lanzhou University
Priority date: 2021-04-26
Filing date: 2021-04-26
Publication date: 2021-08-24

Abstract

本发明公开一种手指识别算法，涉及人机交互界面，尤其涉及人机交互界面中丰富输入原语和多点触控界面中操作手的判断和手指的识别，以解决现有识别算法依赖于特定设备、运算复杂度高、实时性较差等问题。该算法通过手指触点间的相对方位关系和距离大小等自然手势的信息识别手指在多点触控界面中的触点对应的手指，根据5个手指触点的几何中心点分别到大拇指和食指触点间向量的叉乘积的正负识别操作手是左手或是右手。本发明可以丰富多点触控界面中的输入原语，为人们在多点触控界面中实现盲打文本输入等“盲操作”人机交互和为盲人有效利用多点触控设备等提供便利并提高交互绩效。The invention discloses a finger recognition algorithm, which relates to a human-computer interaction interface, in particular to rich input primitives in the human-computer interaction interface and the judgment of an operator and finger recognition in a multi-touch interface, so as to solve the problem that existing recognition algorithms rely on Specific equipment, high computational complexity, and poor real-time performance. The algorithm identifies the fingers corresponding to the contacts of the fingers in the multi-touch interface through the relative orientation relationship and distance between the finger contacts and other natural gesture information. According to the geometric center points of the five finger contacts, the thumb and The positive and negative signs of the cross product of the vectors between the index finger contacts identify whether the operator is left-handed or right-handed. The invention can enrich the input primitives in the multi-touch interface, and provide convenience for people to realize "blind operation" human-computer interaction such as touch typing text input in the multi-touch interface, and for blind people to effectively use multi-touch devices and the like. and improve interaction performance.

Description

Natural gesture finger recognition algorithm

Technical Field

The invention relates to a human-computer interaction interface, in particular to rich input primitives in the human-computer interaction interface and recognition of an operating hand and fingers in a multi-point touch interface.

Background

Multipoint touch screen devices, such as smart phones, tablet computers, and touch screens as large as wall bodies, are widely used in daily work, study, and life of people. A touch screen is both an output device and an input device. Touch screen equipment does not have external input devices such as mouse, keyboard usually, and people adopt bare-handed operation above when using touch screen equipment usually: this provides on the one hand convenience of operation; but on the other hand, the input primitives are not rich enough, so that the interaction performance of people is limited. For example, when people input text on a mobile phone or a tablet computer, the existing method mainly uses a single finger to click a soft keyboard for inputting, and the input mode is generally very inefficient compared with the mode of inputting by two hands through a physical keyboard.

If the fingers touching the screen can be recognized and it is determined whether the operator's hand is left or right, then the characters on the physical keyboard can be assigned to the corresponding fingers according to the correct fingering entered by the person on a standard keyboard (QWERTY keyboard): therefore, people can realize an input mode similar to that on a physical keyboard on the multi-point touch screen, the input efficiency of the multi-point touch screen can be close to that on the physical keyboard, and touch typing input can be carried out similarly. More input primitives can be provided for the multi-touch screen through finger recognition, so that people can obtain more interactive instructions during free-hand operation. The finger recognition algorithm supports people to perform blind operation on the multi-point touch screen, and great convenience is provided for the blind to use the multi-point touch screen.

Typical identification methods currently available require specially-made equipment (e.g., documents "Masson D, Goguey A, Malaria S, et al. Whichfingers: Identifying transistors on Touch Surfaces and keys Using vibrations [ C ]// Proceedings of the 30th Annual ACM Symposium on User Interface Software and technology. Quebec City, QC, Canada: ACM,2017: 41-48."), or a camera specifically positioned relative to the operative surface (e.g., documents "Zheng J, volume D. Finger-Aware Shortcuts [ C ]// Proceedings of the CHI 2016 family J, CA, USA: 2016: 4285): such a condition is not provided in a general touch device. The finger recognition algorithm based on image processing has high operation complexity, and can realize good real-time performance of finger recognition only by equipment with high processing capacity.

Disclosure of Invention

The invention provides a finger recognition algorithm based on the contact points of fingers on a touch screen or other operation surfaces, which comprises that the contact point T which is the largest according to the sum of the distances from one contact point to the other 4 contact points is recognized as a thumb contact point T, and the thumb contact point T is shown in a figure 2 and a figure 3; calculating the distances from the rest contacts to the thumb contact, and identifying the index finger contact I according to the distances; calculating geometric center points C of the 5 contact points, and judging whether the operating hand is a left hand or a right hand according to the positive and negative of the cross product of two vectors respectively taking the point C as a starting point and the thumb contact point T and the forefinger contact point I as an end point; calculate 3 vectors starting at C and ending at the remaining 3 fingers

To

In the rotating direction of the finger, the fingers corresponding to the contact points of the 3 vectors are a middle finger, a ring finger and a little finger in sequence.

And calculating the distances from the rest contact points to the thumb contact point T, wherein the contact point with the minimum distance is the index finger contact point I, and the corresponding finger is the index finger.

Drawings

FIG. 1 is a diagram of the effect of algorithm execution, wherein the fingers and their touch points are identified by the initials of the fingers in English (i.e., thumb-T, index finger-I, middle finger-M, ring finger-R, and little finger-L, the fingers in FIGS. 2 and 3 being identified in the same manner);

FIG. 2 is a schematic diagram of the algorithm principle using a right hand as an example, where C is the geometric center point of 5 finger contacts;

fig. 3 is a schematic diagram of the principle of the algorithm with the left hand as an example (C has the same meaning as in fig. 2).

Detailed Description

When 5 contacts touch the interface, triggering the algorithm to execute, firstly obtaining the coordinates of the 5 contacts, and then circularly calculating the sum of the distances from one contact to the other 4 contacts, wherein the largest contact is the thumb contact T.

And calculating the distances from the rest 4 touch points to the point T, wherein the minimum distance is the index finger touch point I.

Calculating the geometric center point C of 5 contact points

And judging whether the manipulator is a left hand or a right hand according to the positive and negative of the cross product.

Calculate 3 vectors with C as the starting point and the remaining 3 finger touch points as the ending point

To

In the rotating direction of the touch screen, the finger contacts corresponding to the 3 vectors are a middle finger contact M, a ring finger contact R and a small finger contact L in sequence.

Claims

1. A finger recognition algorithm, including the recognition method of the thumb, the recognition method of the index finger and the recognition method of the left and right hands, and the recognition method of the middle finger, the ring finger and the little finger, it is characterized in that according to the distance from one contact to the remaining 4 contacts The largest sum is identified as the thumb contact T, and the corresponding finger is the thumb; the index finger contact I is identified according to the distance from the remaining contacts to the thumb contact, and the corresponding finger is the index finger; the geometric center point C of the 5 finger contacts is calculated , according to the two vectors starting from the center point and ending at the touch points of the thumb and forefinger respectively

and

The positive and negative of the cross product can determine whether the operator is left-handed or right-handed, and then calculate the 3 vectors starting from the center point C and ending with the remaining 3 contacts.

arrive

In the rotation direction of , the three vectors correspond to the touch points and the corresponding fingers are the middle finger, the ring finger and the little finger.

2. The index finger identification method according to claim 1, wherein the contact point closest to the thumb contact distance is identified as the index finger contact point I, and the corresponding finger is the index finger.