WO2010044073A1

WO2010044073A1 - System and method for aiding a disabled person

Info

Publication number: WO2010044073A1
Application number: PCT/IB2009/054546
Authority: WO
Inventors: Aaron Shafir
Original assignee: Individual
Current assignee: Individual
Priority date: 2008-10-16
Filing date: 2009-10-15
Publication date: 2010-04-22
Anticipated expiration: 2011-04-16
Also published as: GB2464486A; GB0818942D0

Abstract

A system for aiding a disabled person comprising: at least one robotic arm; at least one camera for producing at least one image of the disabled person's face; an image processing module for processing the image; and a control sub-system for controlling the position of the robotic arm(s), based on repositioning of one or more facial elements by the disabled person. One or more facial elements is an artificial mark being a transparent mark detectable by the camera(s) operably connected to the robotic arm and/or a mark projected onto the person by a projection mechanism.

Description

Title of Invention: SYSTEM AND METHOD FOR AIDING A

DISABLED PERSON

[ 1 ] FIELD OF THE INVENTION

[2] The present invention relates to a system for aiding a disabled person, particularly to aid disabled person with limited use of the upper limbs.

[3] BACKGROUND OF THE INVENTION

[4] Disabled people have difficulties in carrying out simple every day operations such as eating and drinking.

[5] To help such disabled persons, patent application JP09224965 discloses a meal support robot dedicated to handling food. A person having handicap in an upper limb can independently operate the robot by providing a light projecting part for projecting a directional light, a light receiving part for receiving the light, and a holding part which is elastically deformed by contact with an operator and irradiating the light in a desired position on the light receiving part.

[6] U.S. patent 6,961,623 discloses a remote control method for use by disabled person, e.g. to actuate a muscle stimulator cuff, which involves triggering a signal by using mechanical vibrations detected to control operation of the device or process.

[7] SUMMARY OF THE INVENTION

[8] According to one aspect of the present invention there is provided a system for aiding a disabled person comprising: at least one robotic arm; at least one camera for producing at least one image of the face of the disabled person; an image processing module for processing the image; and a control sub-system for controlling the position of the at least one robotic arm, based on repositioning of one or more facial elements by the disabled person , wherein the one or more facial elements is an artificial mark being a transparent mark detectable by the at least one camera operably connected to the robotic arm and/or a mark projected onto the person by a projection mechanism.

[9] The camera is adapted to sense movement of the artificial marks including gestures

(e.g. movement of an eye, ear, mouth and/or nose; smiling, tongue movement, raising an eyebrow, and the like); and head movements (e.g. nodding yes or no or tilting, etc). In some embodiments, the mark relates to a sticker applied to the face with an 'x' written thereon or a mark (e.g. an 'x') projected onto the face by a projection mechanism or device such as a projector, laser, etc., which capability according to some embodiments is incorporated into the camera(s).

[10] According to another aspect , the present invention relates to a method for aiding a disabled person comprising: applying at least one artificial mark on the person; detecting the at least one artificial mark by at least one camera; and moving an appliance attachable to a controllable robotic arm corresponding to the detected artificial mark(s), wherein the artificial mark is a transparent mark detectable by the at least one camera operably connected to the robotic arm and/or a mark projected onto the person by the camera(s).

[11] The present system and method can be used for a variety of activities including eating/feeding, drawing, writing, teeth brushing and the like.

[12] BRIEF DESCRIPTION OF THE DRAWINGS

[13] The invention will be understood more clearly and other features and advantages shall become apparent from the following detailed description of exemplary embodiments, with reference to the following figures, wherein:

[14] Fig. 1 is an isometric view of an exemplary system in accordance with the present invention as operated by a person in a wheel chair;

[15] Fig. 2A is an isometric front view of the disable person face;

[16] Fig. 2B is an isometric profile view of the disable person face;

[17] Fig. 3 is an isometric side view of a robotic arm;

[18] Fig. 4A is a front view of the disable person face including arrows designating the person face movement; and

[19] Fig. 4B is a profile view of the disable person face including arrows designating the person face movement.

[20] DESCRIPTION OF SOME EMBODIMENTS OF THE PRESENT INVENTION

[21] Fig. 1 illustrates an embodiment of a system for aiding a disabled person 30 in accordance with the present invention, operated by the disabled person, for example, in a chair or wheel chair 32 . A digital camera 34 is attached to a chair support 36 of chair 32 . R esting upon a table 38 , a food plate 40 , a robotic arm 42 , and a storage means 44 which stores appliances that can be attached to a robotic catch 45 , disposed on an end 46 of the robotic arm. Examples of such appliances include a fork, a cup and so forth. Robotic arm 42 has freedom to move in one or more axes and preferably about those axes. Examples of such robotic arms are the VP 5-axis series and VP 6-axis series arms (DENSO Robotics, 3900 Via Oro Avenue, Long Beach, CA 90810, USA), which can perform in five and six freedoms of movement, respectively.

[22] Typically, an additional camera 48 is attached to table 38 . Camera 48 images a front view of face 58 of disabled person 30 and camera 34 is used for imaging the profile of the disabled person's face. The digital images are stored in a digital memory means, not shown, for further processing.

[23] According to some embodiments, the system further includes a voice recognition sub-system, not shown, for recognizing voice commands of disabled person 30 . An example of such voice commands are, 'change tool', which commands robot arm 42 to substitute the appliance currently attached to catch 45 with another appliance that is stored in storage means 44 . The command 'open catch' for example is another voice command that commands catch 45 of robotic arm 42 to open.

[24] An isometric front view and isometric profile view of disabled person 30 is shown in

Figs. 2A and 2B respectively, to which references are now made. In this example, on the face 58 of the disabled person 30 are placed four marks 60, 62, 64 and 66 (e.g. via stickers or projected thereon). These marks can be natural marks appearing on face 58, facial gestures or marks artificially placed on the disabled person face 58. Marks 60, 62 , 64 and 66 are detected by cameras 34 and 48 as known in the art, implemented by an image processing module, not shown. Examples of useful natural marks are wrinkles and moles. In some embodiments of the present invention, a transparent sticker which is invisible to the human eye but is detectable by cameras 34 and 48 can be applied on the disabled person's face 58. Mark 60 is positioned on the forehead of disabled person 30. The disabled person moving his forehead is accompanied by a corresponding movement of mark 60. Mark 64 is disposed on the chin of the disabled person. When the disabled person moves his chin, mark 60 moves as well. Marks 62 and 66 are positioned on the cheeks of the disabled person. In some embodiments of the present invention marks 62 and 66 are moved by moving the tongue of the disabled person towards the person's right and left cheeks, respectively.

[25] A side view of robotic arm 42 is shown in Fig . 3 to which reference is now made.

Robotic arm 42 can be described as being controllable with reference to a Cartesian grid. Double headed arrow 70 designates the movement direction of robotic arm 42 in the x-axis. Double headed arrow 72 designates the movement direction of robotic arm 42 in the z-axis. Double headed arrow 74 designates the movement direction of robotic arm 42 in the y-axis. Rotational movements around axes 70 , 72 and 74 are designated by arrows 76 , 78 and 80 respectively.

[26] A front and profile view of the face 58 of disabled person 30, including arrows describing the facial movement, is shown in Figs. 4A and 4B to which reference is now made. The movements of the disabled person's head is analyzed by determining the distance between any of marks 60, 62, 64 and 66 before and after a facial movement(s) or facial gestures respectively are performed. Nodding of the disabled person's head up and down, in the direction designated by double headed arrow 90, is accompanied by a corresponding movement of marks 60, 64. Sideways turning of the disabled person's head in the direction designated by double headed arrow 92, is accompanied by a corresponding movement of marks 60 and 64 to the left or to the right. Sideways tiliting of the disabled person head in the direction designated by double headed arrow 94, is accompanied by a corresponding movement of mark 60 to the right, and mark 64 to the left, or, mark 60 moves to the left and mark 64 moves to the right. The movements of the disabled person's chin in the direction designated by double headed arrow 96 moves mark 64 up or down in respect to mark 60. Movement of the disabled person's chin in the direction designated by arrows 98 is accompanied by a corresponding movement of mark 64 left or right with respect to mark 60. A movement of the disabled person's tongue in the direction designated by arrow 100 can cause mark 66 to move to the left with respect to marks 60 and 64. The movement of the disabled person's tongue in the direction designated by arrow 102 moves mark 62 to the right with respect to marks 60 and 64.

[27] A control sub-system, not shown, controls the movement of robotic arm 42. Robotic arm 42 is controlled/driven by analyzing the aforementioned movement(s) of the head of disabled person 30 and issuing commands to driving mechanisms, not shown, for controlling the robotic arm (Fig. 3). Head movement in the directions designated by double headed arrow 90 will actuate robotic arm 42 in the directions designated by double headed arrow 70. A movement in direction 92 will actuate robotic arm 42 in the direction designated by double headed arrow 72. A movement in direction 96 will actuate robotic arm 42 in the direction designated by double headed arrow 74. A movement in direction 98 will actuate robotic arm 42 in the direction designated by arrow 76. A movement in direction 94 will actuate robotic arm 42 in the direction designated by arrow 78. A movement in direction 100 or direction 102 will actuate robotic arm 42 in the direction designated by arrow 80.

[28] In some embodiments of the present invention marks 60, 62, 64 and 66 can be projected by one of the cameras 34 and 48 or an additional other component onto the person's face 58 ; and robotic arm 42 is controlled by the relative movement of the face and the mark.

Claims

[Claim 1] A system for aiding a disabled person comprising: at least one robotic arm; at least one camera for producing at least one image of the face of said disabled person; an image processing module for processing said image; and a control sub-system for controlling the position of said at least one robotic arm, based on repositioning of one or more facial elements by said disabled person, wherein said one or more facial elements is an artificial mark being a transparent mark detectable by the at least one camera operably connected to said robotic arm and/or a mark projected onto the person by a projection mechanism.

[Claim 2] A system as in claim 1, comprising two cameras, wherein one camera images a front view of the face of said disabled person and a second camera images a profile of the face of said disabled person.

[Claim 3] A system as in claim 1, wherein said system further comprises storage means which stores appliances that can be attached to said at least one robotic arm.

[Claim 4] A system as in claim 1, wherein said robotic arm is adapted to move in a plurality of axes. [Claim 5] A system as in claim 1, further comprising a voice recognition subsystem for recognizing voice commands of said disabled person. [Claim 6] A system as in claim 1, w herein the projection mechanism is produced by a laser. [Claim 7] A system as in claim 1, w herein the projection mechanism is produced by a projector. [Claim 8] A system as in claim 1, w herein the camera comprises the projection mechanism. [Claim 9] A method for aiding a disabled person comprising: applying at least one artificial mark on said person; detecting said at least one artificial mark by at least one camera; and moving a robotic arm corresponding to movement of the detected artificial mark(s), wherein said artificial mark is a transparent mark detectable by the at least one camera operably connected to said robotic arm and/or a mark projected onto the person by the camera(s).