KR20130133223A - User identification with biokinematic input - Google Patents

Abstract

Disclosed herein are systems and methods for user identification based on biokinetic input. The system includes a multi-touch sensitive display comprising a sensor configured to receive a biokinetic input comprising data indicative of a detected location of a finger touch made by a user's fingers at each of a series of consecutive time intervals of a defined identification gesture. It may include. The system can further include a user identification module executed by the processor of the computing device. The user identification module may be configured to receive biokinetic input from the sensor and to compare the relative positions of finger touches and / or the relative rate of change of the positions of finger touches with a storage template of biokinetic data validated for the user. . If a match is determined, an indication may be displayed that the user was successfully identified.

Description

USER IDENTIFICATION WITH BIOKINEMATIC INPUT}

Biometric techniques have been developed to identify people based on characteristic physical characteristics such as fingerprints and palm prints. For example, palm scanners have been developed to allow or deny building access, and fingerprint scanners have been developed for web site and file access or login to the operating system. These systems use a dedicated scanner with a high resolution imaging function sufficient to obtain detailed scans of skin patterns on fingers or palms, thereby enabling identification of characteristic physical properties within skin patterns that distinguish users from each other. It can be done.

There are several obstacles to the adoption of such biometric techniques on tablet computing devices and touch screen mobile phones. In the past, dedicated fingerprint and palm scanners have been used, for example, on door access panels and laptop computer housings with sufficient space to mount the scanners. It does not have enough space to mount it. Also, touch screen technologies used in conventional tablet computing devices and mobile phones are not high enough resolution to obtain a usable image for the user's distinguishable fingerprint or palm skin patterns. As a result, biometric techniques related to fingerprint and palm scanning are not widely adopted for tablet computing devices and touch screen mobile phones.

Disclosed herein are systems and methods for user identification based on biokinetic input. The system includes a multi-touch sensitive display comprising a sensor configured to receive a biokinetic input comprising data indicative of a detected location of a finger touch made by a user's fingers at each of a series of consecutive time intervals of a defined identification gesture. It may include. The system can further include a user identification module executed by the processor of the computing device. The user identification module receives biokinetic input from the sensor and may be configured to compare the relative positions of the finger touches and / or the relative rate of change of the positions of the finger touches with a storage template of biokinetic data validated for the user. have. If a match is determined, an indication may be displayed that the user was successfully identified.

This summary is given to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to implementations that solve any or all disadvantages noted in any part of this disclosure.

1 is a schematic diagram of one embodiment of a computing device configured to perform user identification using biokinematic input.
FIG. 2 is a perspective view of a computing device configured to perform user identification using the biokinetic input of FIG. 1 illustrating a series of hand positions in a palm-fist switch gesture on a tablet computing device.
3 is a graphic diagram illustrating a plurality of touches corresponding to a series of hand positions shown in FIG. 2.
4 is a schematic diagram of one embodiment of a computerized user identification method for user identification using biokinetic input.

1 is a schematic diagram of one embodiment of a computing device configured to perform user identification using biokinematic input. Computing device 10 may be configured as a portable computing device such as, for example, a mobile phone, a tablet computing device, or a laptop computer. In other examples, computing device 10 may be configured as a personal computer (PC), desktop computer, tabletop computer, or kiosk computer. Computing device 10 includes a display 22, a processor 24, a memory unit 26, and a mass storage unit 28. Programs stored in mass storage unit 28 may be executed by processor 24 using memory unit 26 and display 22 to perform the various functions described herein. Computing device 10 may be configured to receive biokinetic input from a user during a defined identification gesture and to compare the biokinetic input to the stored user template 48. Thus, if a match is determined, the user can be identified successfully and access to the software, data, or functionality of the device can be granted.

The display 22 may be a multi-touch sensitive display that includes a sensor 32 for detecting touch input. Display 22 may use a variety of display technologies for generating visible images. Thus, the display may be configured as a liquid crystal display (LCD) or an organic light emitting diode (OLED) display composed of a plurality of light emitting pixels, for example. Display 22 may be a tabletop optical touch sensitive display, such as a Microsoft SURFACE® display.

Sensor 32 may be configured to receive biokinetic input 18 that includes data indicative of detection locations of each of a plurality of finger touches made of a corresponding plurality of fingers of a user. The sensor 32 may be configured to receive data at each of a series of consecutive time intervals during a defined user's identification gesture. The plurality of finger touches of the defined identification gesture may also include at least one palm touch. Thus, for example, the prescribed identification gesture may be a palm-fist switching gesture in which the user switches to an open palm after showing a clenched fist on a multi-touch display, or otherwise turns to a clenched fist after showing an open palm. Such gestures will be discussed in more detail with reference to FIGS. 2 and 3 below. It will be appreciated that this is just one example of an identification gesture that may be used by the computing device 10 to distinguish individuals. Other identifying gestures may also be used, including hands, palms, fingers and / or thumbs that move in various ways.

The sensor 32 may be any one of various touch sensors. For example, sensor 32 may be a capacitive or resistive multi-touch sensor. In such embodiments the sensor 32 may be configured to detect a touch on the upper surface of the display 22 through a change in capacitance or resistance detected caused by each of the plurality of finger and / or palm touches.

As another alternative, other touch sensing techniques may be employed. For example, the display 22 may include light sensors 32 that may be located at each pixel of the display, and the output from these light sensors detects multiple touches on the upper surface of the display. Can be processed to Such light sensors can be configured to sense visible light and infrared light as an example. For example, the optical sensor may be an APS, such as a complementary metal oxide semiconductor (CMOS) or any other active pixel sensor (APS) configured to sense visible and infrared light.

Computing device 10 may include a mass storage unit 28, such as a hard drive. The mass storage unit 28 may be in effective communication with the display 22, the processor 24, and the memory unit 26 via the data bus 30, and may utilize the portion of the memory 26 to process the processor 24. Programs executed by and other data utilized by such programs. For example, mass storage device 28 may store user identification module 34 and associated database 38. The mass storage device 28 may also be a program 36, such as an application, requesting that identification be performed by the user identification module 34, an operating system component such as a login program or a user authentication program, or another program running on a computing device. You can save the program. The user identification module 34 and program 36 may be executed by the processor 24 of the computing device 10 using a portion of the memory 26.

The user identification module 34 may operate on various various steps, such as the registration step 50 and the identification step 60, for example. Each step may be associated with various components of the user identification module 34 as described below.

User identification module 34 may be configured to receive biokinetic input 18 from sensor 32. The biokinetic input comes from a sensor 32 that senses a moving body part of the user. To distinguish one user from another user, the user identification module 34 may be configured to compare the relative positions of the plurality of finger touches and / or palm touches included in the biokinetic input. It will be appreciated that touches from more, less or other body parts may also be considered. Additionally or alternatively, user identification module 34 may be configured to compare the relative rates of change of the locations of the plurality of touches over time.

The received biokinetic input 18 is a stored user template 48 of validated biokinetic data for the user that may have been previously verified and stored by the user identification module 34 during the registration phase, as discussed below. ) Can be compared. Once a match between the biokinetic input and the stored user template 48 is determined, the display 22 may display an indication that the user was successfully identified.

As another example, the user identification module 34 may be configured to measure the relative positions of the plurality of finger touches and / or the rate of change of the positions relative to the at least one palm touch during the palm-fist switching gesture. The user identification module 34 may then compare the measured relative positions and / or rate of change with the stored user template 48 to determine whether a match exists. To do this, the user identification module 34 may generate a template of user biokinetic data and compare the template with a stored user template 48 representing the biokinetic data.

The registration step 50 and the user, in which the internal operations of the user identification module 34 are now registered using the biokinematic input used to verify the user ID, in the manner described above. Will be described with reference to the identification step 60 where they are identified and granted or denied access. As illustrated, the user identification module may include a feature extractor 42, a template generator 44, and a matching engine 46 that operate as described in detail below.

During the registration step 50, the feature extractor 42 may be configured to receive a first biokinetic input from the sensor 32 and extract user identification features from the first biokinetic input. User identification features may be unique to each user, that is, a set of specific features may be associated with one user and may be different from certain features of another user. Thus, the specific features can be used to identify one user and distinguish that user from other potential users. The first biokinetic input may include relative positions of the plurality of finger touches and at least one palm touch in some embodiments, and may include relative rates of change of the positions of these touches. As described above, the first biokinetic input may include data from a series of consecutive time intervals during a defined identification gesture, such as a palm-fist transition.

Feature extractor 42 may extract user identification features using any number of techniques. For example, feature extractor 42 may process the image through filtering, contour identification of the user's hand, and / or calculating the center of gravity of one or more fingers and / or palm touches. It will be appreciated that the examples described above may be used in combination or subcombination without further departing from the scope of the present disclosure and may further include additional or alternative means for extracting user identification characteristics. Feature extractor 42 may send the output of the extracted user identification characteristics to template generator 44 for further processing.

Continuing at registration step 50, template generator 44 may generate a template of data characteristic for the user from the first biokinetic input received by feature extractor 42. Template generator 44 may be configured to generate a user template 48 that includes user identification characteristics for a given user and a given identification gesture and store the user template 48 in a data store, such as database 38. By storing such feature data for each user, user templates 48 may be used to determine whether a match exists to identify the user by comparing with biokinetic data received during the next user identification step 60. Can be.

Database 38 may store a plurality of user templates 48 for a plurality of users that are authorized for software or data on access computing device 10. In some embodiments various classes may be used to register various users, such that more than one user template 48 may exist for each user. For example, one class may be a palm-fist switch gesture, another class may be a finger-thumb pinch gesture, and a separate user template 48 is registered to use the computing device 10. Can be stored for each authenticated user.

Turning now to the user identification step 60, this step is an arbitrary user authentication operation such as login operation, opening a protected folder, opening a protected file, accessing a protected program, accessing a protected network, or any other application for which user authentication is desired. Can be implemented. For example, user identification module 34 may be executed during a user authentication operation, such as a user login for a user session on computing device 10.

Feature extractor 42 may receive a second biokinetic input from sensor 32 during user identification step 60. The second biokinetic input may include relative positions of the plurality of finger touches and at least one palm touch, and may include relative rates of change of the positions of the touches. The second biokinetic kick input may also include data for these touches from a series of consecutive time intervals during the defined identification gesture. For example, user identification step 60 may be performed through a gesture-based user login operation on a login screen, such as a palm-fist switching, as described above.

Output of feature extractor 42 may be sent to matching engine 46 during user identification step 60. The matching engine 46 may compare the second biokinetic input with the stored user template 48 that was generated based on the characteristics extracted from the first biokinetic input to verify the user's biokinetic data. In some embodiments, a user may enter a user name into computing device 10, for example to identify the user's identity instead of or in addition to a PIN number or password, and biokinetic input is described herein. Can be matched as shown. If multiple templates for different gesture classes are available for a given user, the matching engine 46 may recognize the second biokinetic input as belonging to a particular zester class, such as a palm-fist transition, thus The second biokinetic input may be compared with the stored user templates 48 belonging to the same gesture class.

In other embodiments particularly useful where the total number of registered users of the computing device 10 is small, the user simply presents an identification gesture without entering a user name and the biokinetic input for the identification gesture presented by the user identification module. Find a user template that matches the properties extracted from.

The matching engine 46 may determine whether a match exists between the user templates and the features extracted from the second biokinetic data. If a match exists, the user is determined to have been successfully identified. Matching engine 46 may use any number of techniques to determine that a match exists. For example, the matching engine 46 may, within a certain error, correspond to one or more user templates 48 with a feature or combination of features of the second biokinetic input corresponding to the plurality of finger and / or palm touches. You can compare the area of these. The feature may be the total area, center of gravity, the length of each finger from fingertip to palm, the relative lengths of each finger from fingertip to palm, and the rate of change of any of those features over time during the gesture. Alternatively, matching engine 46 may compare such a feature or combination of features of the second biokinetic input to a predetermined normal model for all users. That is, the matching engine 46 is configured to calculate a deviation of the second biokinetic input from the given normal model and determine whether the deviation of the second biokinetic input matches the deviation of the corresponding user template within the predetermined threshold. Can be.

When a match is determined, an indication for such an event may be displayed to the user. For example, such an indication may be a message output to the graphical user interface 31 of the display 22 after a successful user login operation. In one embodiment, graphical user interface 31 may include a login screen, and the indication may be a message displayed on the login screen for a successful login. Alternatively, the indication may be an interruption of the login screen being displayed to the user. Following the discontinuance of the display and / or login screen of the message, the user may be granted access to programs 36, files, folders, applications, networks, operating systems, and the like on computing device 10.

FIG. 2 is a perspective view of a computing device configured to perform user identification using the biokinetic input of FIG. 1 illustrating a series of hand positions in a palm-fist switch gesture on a portable tablet computing device. 2 illustrates a user's hand 80 interacting with computing device 10 by contacting display 22. Figure 2 shows the user's hand 80 in different postures in AD, showing the positions of the transition from the clenched fist to the open palm and vice versa according to the order (A to D, or D to A), where The clenched fist is depicted in A, the open palm is depicted in D, and the middle figure is depicted in B and C. Such movement may be defined as a palm-fist switching gesture and may be used as an identifying gesture to gain access to the computing device 10 and / or the characteristics of the computing device 10 as already described. An example message that serves as an indication of successful identification is shown in FIG. It should be understood that the palm-fist switching gesture is merely one exemplary embodiment of any identifying gesture that may be used by the systems and methods described herein. Numerous other identifying gestures are contemplated, including relative movement between the user's plurality of fingers and / or palms, or other body parts.

3 is a graphic diagram illustrating a plurality of touches corresponding to a series of hand positions shown in FIG. 2. 3 illustrates areas of the user's hand 80 that may be in physical contact with the display 22 during the palm-fist switching gesture. Such points of contact may include a plurality of finger touches 82 and palm touch 84. For example, when the user's hand 80 makes the same fist shown in A of FIG. 2, a plurality of finger touches 82 and palm touch 84 as shown in A of FIG. Can be received by. As the user hand 80 continues the palm-fist switching gesture, the relative positions of the finger touches 82 and palm touch 84 can vary as seen from A-D and likewise D-A. Also, the relative rates of change between each of the finger touches 82 and palm touch 84 can be measured as already described.

It should be understood that biokinetic data may be received from a plurality of finger and / or palm touches without the user's hand making a physical contact with the device. That is, a sensor of the device may be configured to receive biokinetic data from a user's hand substantially in proximity to the device. As another alternative, the sensor may be configured to sense the position of the hand when the hand is in close proximity to the display 22, such as in the case of optical sensors and certain capacitive sensors, for example.

Biokinetic input is unique to the user because the user's hands may vary greatly in size, relative position, and relative rates of change between the plurality of finger touches and palm touches as the hand moves during identification zester. Thus such biokinetic input can be used as a virtual signature representing a plurality of users and thus create means for allowing user authentication to be verified.

4 is a schematic diagram of one embodiment of a computerized user identification method 400. The method 400 may be performed using software and hardware components or other suitable components of the computing device 10 described above. The method 400 includes a user biokinetic input (e.g., a multi-touch sensitive display such as the display 22 of the computing device 10 associated with an identification gesture or other identification gesture, such as a palm-fist switching gesture as shown in FIG. 18) and start at 401 receiving.

At 402, the method 400 proceeds to extract user identification characteristics. For example, as illustrated in FIG. 3, a plurality of finger touches and / or palm touches made by a plurality of fingers and / or palms of a corresponding user may be extracted. User identification characteristics may be extracted at each of a series of consecutive time intervals.

At 403 the method 400 proceeds by measuring a plurality of finger touches and / or palm touches made by the user. For example, the biokinetic input 18 corresponding to the palm-fist switching gesture may be measured according to the relative values and / or relative rates of change of each of the plurality of finger touches and / or palm touches.

During the registration step as described above, the method 400 may proceed to 404, wherein the user of the biokinetic input (referred to as the first biokinetic input in connection with FIG. 1) in response to the user palm-fist switching gesture. Generating the template 48. Generating the user template 48 may include verifying that the first biokinetic input has high integrity or repeatability through additional steps. For example, the verifying step may include the user repeating the same identification gesture one or more times before generating the user template 48. This allows the template generator to recognize user-specific patterns with greater confidence.

At 405, the method 400 includes a procedure for storing a user template 48 obtained during a registration step in a data store that can be used during other steps of the computing device 10, such as the database 38 of the computing device 10. Step on. The method loops back from 404 to 401 to receive another biokinetic input during the identification phase of the method.

During the identifying step, the method 400 may include receiving a biokinetic input corresponding to an identifying gesture, such as a palm-fist switching gesture, as a second biokinetic input during the second pass through these steps 401-403. Repeat steps 401-403, which may be possible. The method proceeds to a step 406 of comparing the second biokinetic input to a stored user template 48 of verified biokinetic data in the database along the identification path. The comparing step may include an analysis of respective positions of the plurality of finger touches and / or palm touches and / or respective rate of change among at least a portion of an identification gesture, such as a palm-fist switching gesture as described above. .

At 407 the method 400 determines whether a match exists between the second biokinetic input and the stored user template 48 within a predetermined error. If the answer to 407 is yes, the method 400 proceeds to 408 and an indication in the form of a message indicating that the user was successfully identified can be output to the display. Alternatively, the indication may be an interruption of the login screen being displayed to the user.

The method 400 now proceeds to 409 where the user is granted access to programs, files, folders, applications, operating systems, and the like on the computing device. The identification step of the method ends from 409.

If the answer to 407 is no, the method 400 proceeds to 410 where the user is denied access to a program, file, folder, application, operating system, or the like on the computing device. The identification step of the method also ends at 410. In contrast, in some embodiments, the disclosed method may additionally display some indication to the user when no match occurs and access is denied and / or the user may re-enter a biokinetic input to compare with the user template 48. You can display a GUI that allows the user to correct any input errors that might have occurred at the first time.

It will be appreciated that the method 400 may include additional or alternative steps. As one example, the method may include a mechanism for determining which step (eg, registration step 50 or identification step 60) to operate. Typically, the registration phase can be triggered when the user identification module is called by any operating system component that registers a new user account on the computing device. And an identification phase is triggered when any operating system component or application program, such as a login program, calls the user identification module. When the user identification module is part of an application programming interface (API), appropriate application programming interface function calls may be provided for each step.

The systems and methods described above can be used to embody biokinematic-based security for controlling access to software, data, or other functions of a computing device. These systems and methods provide the user convenience while having the advantage that they can be implemented without the need for a dedicated fingerprint or palm scanner, which was a feature on previous devices.

The terms “module”, “program” and “engine” are used herein to refer to software that, when executed by a computing device, performs one or more specific functions. These terms are intended to encompass, for example, executable files, data files, libraries, drivers, scripts, individual database records, or groups. While the embodiments described herein show some typical configuration of such modules, programs, and engines, it should be understood that the functions described herein may be achieved by other configured software elements.

It is to be understood that the configurations and / or approaches described herein are illustrative in nature and numerous specific embodiments or examples are not to be considered in a limiting sense because numerous variations are possible. Certain routines or methods described herein may represent one or more of any of the processing strategies. Thus, the various acts illustrated may be performed in the illustrated order, in other orders, in parallel, or in some cases omitted. Likewise, the order of the above-described processors may be reversed.

The subject matter of the present disclosure is directed to all novel and non-obvious combinations and subcombinations of the various processes, systems, and configurations disclosed herein, as well as to other features, functions, actions, and / or properties, and to Includes any and all equivalents.

Claims (10)

A sensor configured to receive a biokinematic input comprising data indicative of the respective detection location of a corresponding plurality of finger touches made by the plurality of fingers of the user at each series of consecutive time intervals during the defined identification gesture. Including a multi-touch sensitive display, and
A user identification module executed by a processor of the computing device, the user identification module configured to receive the biokinetic input from the sensor, wherein the user identification module includes a relative position of the plurality of finger touches and a relative position of the position of the plurality of touches. Compare one or more of the rate of change to a stored user template of biokinetic data validated for the user and, if a match is determined, display an indication on the display that the user was successfully identified.
Computing device.
The method of claim 1,
The sensor is a capacitive multi-touch sensor
Computing device.
The method of claim 1,
The plurality of finger touches of the prescribed identification gesture include at least one palm touch, wherein the defined identification gesture is a palm-fist switching gesture.
Computing device.
The method of claim 3,
The user identification module measures one or more of the relative position of the plurality of finger touches and the rate of change of the position with respect to the at least one palm touch during the palm-fist switch gesture and also measures to determine if a match exists. And configured to compare one or more of the associated relative positions and rate of change with the stored user template.
Computing device.
The method of claim 1,
The computing device is selected from the group consisting of mobile telephones, tablet computing devices, laptop computers, personal computers, desktop computers, tabletop computers, or kiosk computers.
Computing device. The method of claim 1,
The user identification module is executed during a user authentication operation on the computing device, the user authentication is implemented during the user's login to a user session on the computing device, and the indication is a message output to the display after a successful user login operation. Or the indication is an interruption of the login screen displayed to the user and allowed access to a program or file or operating system on the computing device.
Computing device.
Computerized user identification method,
Receiving a user biokinetic input of an identification gesture comprising a corresponding plurality of finger touches made by the plurality of fingers of the user via the multi-touch sensitive display of the computing device at each successive time interval,
Comparing at least one of the individual positions of the plurality of finger touches and the individual rate of change of the positions during at least a portion of the identification gesture of the biokinetic input to a stored user template of biokinetic data validated for the user, And
If a match is determined by the comparison, displaying an indication on the display that the user was successfully identified.
Way.
The method of claim 7, wherein
The plurality of finger touches of the identification gesture include at least one palm touch, wherein the identification gesture is a palm-fist switching gesture.
Way.
9. The method of claim 8,
Prior to said comparing step, further comprising measuring at least one of said relative position of said plurality of finger touches and said rate of change of said position relative to said at least one palm touch during said palm-fist switching gesture,
The comparing step further includes comparing one or more of the relative position of the plurality of finger touches to the at least one palm touch and the rate of change of the position during the palm-fist switching gesture.
Way.
The method of claim 7, wherein
The receiving, comparing, and displaying are performed during a user authentication operation on the computing device, the user authentication is implemented during the user's login to a user session on the computing device, and the indication is performed after a successful user login operation. Is a message output to a display, or the indication is an interruption of the login screen displayed to the user and access granted to a program or file or operating system on the computing device.
Way.

Images