WO2009093027A1

WO2009093027A1 - Wrist-mounted computer periferal

Info

Publication number: WO2009093027A1
Application number: PCT/GB2009/000172
Authority: WO
Inventors: Erik Lea
Original assignee: GUILDFORD TECHNOLOGIES Ltd
Current assignee: GUILDFORD TECHNOLOGIES Ltd
Priority date: 2008-01-24
Filing date: 2009-01-22
Publication date: 2009-07-30
Anticipated expiration: 2010-07-24
Also published as: GB0801287D0

Abstract

A new type of computer peripheral is described, which is intended to be used as a wireless device, which can interface computers, mobile phones, Personal Digital Assistants (PDAs) and other devices with wireless interface. The peripheral is strapped to a person's wrist, in a similar fashion to a watch, and it detects the person's wrist movements, which are transmitted to a computer or other device as programme instructions. In addition to detecting gestures, which are detected as pitch, roll, yaw and linear movement of the wrist, the device can pick up short transient movements in the wrist, which can be generated by snapping or tapping of the fingers.

Description

WRIST-MOUNTED COMPUTER PERIFERAL

1 FIELD OF THE INVENTION

The invention relates to the interface and control of computer programmes in electronic devices, such as computers, mobile phones and PDAs, which are today mostly controlled via keyboards, mouse devices, keypads and touch screens. More specifically, the device described here detects the movements of the user's wrist, and sends control signals to the electronic device based upon these movements. In most implementations, it is intended that the device be able to send the control signals to the electronic device wirelessly.

2 BACKGROUND OF THE INVENTION

Most computers today are interfaced by a keyboard and a mouse. Smaller electronic devices, such as mobile phones and PDAs, often have an integrated keypad or a touch-screen interface, which can be controlled by fingers or a pen. This patent describes a peripheral device which can complement or replace such devices as keyboards, mouse, keypads or touch screens. Several other methods for interfacing or controlling complex computer programs exist, as described below.

2.1 Prior Art

1. Keyboard, Button and Keypad Devices

Keyboards have been the main interface device for computers since their introduction to the market, and they are still essentially the same input device as patented by Christopher Sholes for typewriters in 1878, except that they are now electrical. Along with button and keypad devices, which are normally used on mobile phones, PDAs and teller machines, they offer a relatively quick way of entering alphanumeric data and predefined commands into a computer programme. A major drawback with these devices, especially full-sized computer keyboards, is that they take up a lot of space, and they are one of the main factors why portable computers are not becoming smaller. Several manufacturers have implemented full, but miniaturized, keyboards on mobile phones and PDAs. While this allows for smaller electronic devices, it practically takes away the speed and convenience of the keyboard once it becomes difficult for humans to place their fingers on the miniature buttons at speed. Another method for reducing the size of keyboards, which is common on mobile phones, is to put several letters on a limited number of buttons, requiring the user to press each button several times to reach the required alphanumeric character. Again, this significantly reduces the convenience and speed of the user input.

2. Mouse and Pointing Devices

The computer mouse was invented in the 1960s by Douglas Engelbart [1], and was introduced to the popular market by Logitech in the 1980s [2]. Virtually all computers today are supplied with a computer mouse. The mouse normally gives input based upon a two-dimensional movement relative to a table or a mouse pad. This normally gives a positional input to the computer, which is often used to position a point or a cursor on a graphical user interface (GUI) on a computer monitor. In addition, most mouse devices come with one or more buttons, or scrolling wheels, to be used in conjunction with the positional information. The button inpxit is generally used for selecting, grabbing or instantiating whatever the cursor is pointing at. Several alternative methods for pointing, giving positional information, selecting, grabbing, moving, or instantiating on a computer monitor or any GUI have been proposed or implemented. They include the trackball [3], touchpad [4], pointing stick [5] and lightpen [6]. All these devices have essentially the same purpose as the computer mouse. Their major drawback is that they require the user to be in a certain working position, such as by a mouse pad or by a computer monitor. Although some pointing devices have wireless connection to computers, pointing devices are generally not intended to be carried around for use on multiple computers.

3. Voice Control

Voice control has been implemented in various applications, and is commercially available for PCs and mobile units in Microsoft's Voice Command programme. The useful aspect of voice control is that it does not restrict the user's working position, and it does not necessarily take more space than a microphone. It could also easily be implemented for the same user on multiple systems, such as computers, teller machines, mobile phones and PDAs, without making the users carry hardware with them. The major drawback of voice control is that it lacks the quick and precise positional control, which is common with pointing devices.

4. Wearable Motion Detectors and Computer Gloves

Various methods for mechanical detection of hand movements have been patented [7], [8], [9], and some solutions are commercially available, for instance the Essential Reality P5 Gaming Glove. These detectors all resemble gloves in some way. Yoshihiro et al. [10] present a system, which claims to measure the left-right and up-down movement of the wrist, as well as the angle of the wrist. Since their system measures only the angle or bending of the wrist, and not the rotation of the wrist, it would not recognise any hand gesture consisting of both linear and rotational movements, as is possible with the invention described in this patent. For instance, detecting the difference between sliding one's hand along a table top and sliding it around a football would require rotational information as well as linear movement. Also, although [10] claims to measure wrist movements, it specifically describes a system where the entire sensor unit is on the hand, with only a mechanical mount on the wrist. Tatsuto [11] describes another interesting sensor, which is a smaller unit without a significant glove structure mounted on the hand. It works by having a small sensor strip following the palm of the hand, as well as a wrist mount. Again, this unit is not a rotation sensor, and it focusses on measuring the bending of the hand, thus requiring sensors mounted along the palm of the hand. What can be said for all the devices mentioned above is that they are all really focussing on the movements of the hand, rather than extracting the wrist movement only, as is done with the invention described here. Also, they are generally a bit too bulky to carry around between multiple computers.

5. Hand-Held Motion Detectors

Various products have been implemented to detect their own movement and to use this as input to a computer programme. In effect, these devices work in a similar way to a computer mouse, but without requiring close contact with a mouse pad, table or computer monitor. Notable motion detecting devices include the Revolution console system from Nintendo, presented at the Tokyo Game Show on 16 September 2005, Compaq's Rock n Scroll photo album [12], the Sharp V603SH mobile phone and the GO 2.4 Series Cordless Optical Air Mouse from Gyration.

6. Hand and Finger Detection by Image Analysis or Holography

A lot of work has been done in gesture recognition, including eye, head, facial expressions and hand and finger movements. An interesting implementation is the FogScreen technology [13] by FogScreen Inc., which allows for both display and interactive work. However, whether these systems are set up with fog screens or cameras for image analysis or holographic light rasters, they tend to be dedicated systems, which are not suited for moving between computers.

2.2 Issues Addressed by the New Invention

The main purpose of the new invention is to control computer programmes in a convenient, intuitive and efficient manner. Several specific issues are addressed in its design:

1. Its control functionality must be powerful enough to run most common computer programs as the only input peripheral. This will allow for computers to be designed without a keyboard as the main input device.

2. It is intended to provide a powerful alternative to the computer mouse, giving up to six dimensions of control, without requiring a table or mouse pad, without having to clean the mouse mechanics, and with the ability to be used in any location, such as on the street to control a PDA or a bank teller, while not restricting the position or posture of the user.

3. It is intended to provide much of the functionality of computer gloves, while being small enough to be carried around at all times, just like a wrist watch. This will enable the user to interface a wide variety of electronic devices, such as mobile phones, bank tellers and shop window demonstrations, all with the same input device. 4. It is intended to make a device which is useful for people who could not use a keyboard or hold a pointing device due to loss of fingers, rheumatism or other handicap.

5. It is intended that the device be able to work in wireless mode, and so enabling the user to control a computer from any position, where the only restriction is the choice of wireless range in the particular implementation of the device.

3 Summary

As an overall description of the invention in this patent, it is a wrist-mounted device, worn in a similar way to a watch, as shown in Figure 3. According to claim 1, it detects the linear and rotational movements of the user's wrist, and control signals are sent to a computer programme based upon these wrist movements. The computer programme can run on any type of electronic equipment, such as a computer, a mobile phone, or a PDA. Normally, the signals generated by the wrist-mounted device would be transmitted wirelessly to the computer programme, as stated in claim 8. In addition, transient movements, such as those generated by snapping of fingers, where the position of the wrist can be the same before and after the transient movement, can be detected by the wrist-mounted device, and used to generate singular programme instructions, similar to a computer mouse click, as described in claim 3.

4 Description of the Drawings

Figure 1 shows various common hand gestures: Grabbing, waving, pushing and lifting.

Figure 2 shows the six degrees of freedom of movement measured on a device, where the rotational movements are roll, pitch and yaw, and the linear movements are x, y and z. The arrow marked n indicates the normal vector relative to the top of the device.

Figure 3 shows a hand with the proposed computer peripheral strapped to the wrist.

5 Preferred Embodiments

A significant amount of hand gesture information can be extracted from the user's wrist, without knowing the details of the user's hand. Some common hand gestures are illustrated in Figure 1, showing a grabbing motion, waving left to right, pushing forward and lifting. Grabbing, pointing and moving objects are normally motions of the hand, which are related to movements of the wrist. The invention aims to detect the linear motion as well as the rotation of the wrist of the user, and use these data to generate control information for computer programmes.

Figure 2 illustrates a device, which can detect six degrees of freedom of motion of the wrist. As illustrated, it detects three rotational movements, namely roll, pitch and yaw, and three linear movements of the device, being x, y and z in a Cartesian coordinate system.

The preferred embodiment of the invention would be like illustrated in Figure 3, where a detector like in Figure 2 is strapped to the user's wrist, in the same way as a watch. While it is possible to have a bigger device, extending over more of the hand or the forearm, or a device extending further round the wrist, like a bracelet, a device similar to a wrist watch is preferable, in that it is small and portable, and because it is similar to what people are used to wearing as wrist watches.

The detection of the wrist motions can be implemented in various ways. At least three linear sensors are required to detect the six degrees of freedom of movement illustrated in Figure 2. All the degrees of freedom illustrated in Figure 2 will be the same for the device and the user's wrist, except for the roll.

Rotational movements can be detected by placing two or more linear sensors in the plane normal to the rotational axis. For a wrist detector, this can be implemented by placing a linear sensor at the left and right ends of the device, to allow the measurement of the roll to be relative to the centre of the user's wrist. This is to provide the user with a more natural and intuitive feel.

Notice, however, that neither the number of sensors, their placement, or the specific orientation used are unique, and the measurements may be implemented in various ways. Positioning the sensors at the edge of the device has however got the advantage of leaving space for a microcontroller in the centre of the device. A microcontroller, being a programmable integrated circuit, with integrated memory and analogue- to-digital converters, is readily available from companies like Texas Instruments and Microchip, and their size is typically less than 1 cm². It is intended that the sensor signals be recorded and processed within the device, before sending the signals to another electronic device to control a computer programme.

This is most practically done with a microprocessor with integrated analogue-to- digital circuitry. It should be mentioned that the motion sensors for a wrist sensor will be force or acceleration sensors, and the conversion to position and relative movement will be calculated in a processing circuit. It is intended that this takes place in a microcontroller located inside the device.

While a sensor like illustrated in Figure 3 can detect any gesture indicated in Figure 1, there is another important kind of movement which can be detected by the device. This is the transient movement, generated by snapping or tapping of the fingers, or by a quick shake of the hand. In this case, the wrist generally stays in the same position before and after the transient, while for instance a snap of the fingers will be detectable as a very rapid shake of the wrist. It is intended that this be used as singular computer instructions, similar to clicking a computer mouse.

The signals detected from the wrist are used to generate control signals for a computer programme located on an electronic device. The transmission can be performed in several ways. For the preferred embodiment of the device, it is intended that the signals be transmitted wirelessly, for instance by Bluetooth, which is a wireless protocol commonly implemented in computers, mobile phones and PDAs. This will allow the device to control computer programmes on multiple electronic devices supporting the same wireless interface. While at a dedicated home or work computer, however, the device may be plugged in by physical connection if required, to allow the wireless connection to be switched off.

6 REFERENCES

[I] Douglas C. Engelbart, X- Y Position Indicator for a Display System, Patent No.: US3541541, 1970 [2] Jean-Luc Mazzone and Marc Bidiville, Light Ball Electronic Mouse, Patent No.: US4951034, 1990 [3] Terry L. Dellinger, Hand-held Trackball Computer Pointing Device, Patent No.: US6816151, 2004

[4] William M. Louis and Canstance M. Louis, Integrated Pointing and Drawing Graphics System for Computers, Patent No.: US6674425, 2004

[5] Patrick J. Franz, Pointing Stick in a Computer Keyboard for Cursor Control, Patent No.: US5568987, 1996

[6] Nicholas P. Mati and Frederick J. Porter and Robert W. Fredrickson, Graphics Light Pen and Method for Raster Scan CRT, Patent No.: US4367465, 1983

[7] Rita Terns and Christopher S. Terns and Edward C. Meagher, Golf Glove and Method of Forming Same, Patent No.: US6708346, 2004

[8] Frank Zngf and Stan Huang, Glove Mouse with Virtual Tracking Ball, Patent No.: US 6870526, 2005 [9] Steve Harmon, Data Input Glove Having Conductive Finger Pads and Thumb Pad, and Uses Therefor, Patent No.: US6141643, 2000

[10] Shimada Yoshihiro, Shiwa Shinichi and Ishibashi Satoshi, Hand Wrist Type Control Device and Method for Using the Same, Patent No.: JP2001034407A, 2001

[II] Fujie Tatsuto, Pointing Device for Controlling Pointer by Wrist Movement, Patent No.: JP2005050159A, 2005

[12] Joel F. Bartlett, Rock n Roll is Here to Stay, IEEE Computer Graphics and Applications, May/June 2000, pp. 40-45

[13] Karri Palovuori and Ismo Rakkolainen, Method and Apparatus for Forming a Projection Screen or a Projection Volume, Patent No.: US6819487, 2004

Claims

7 CLAIMSWhat is claimed is:

1. A wrist- mounted device for controlling a computer programme πinning on any type of electronic equipment, characterized by:

• that the linear motion and the rotation of the user's wrist are detected by the device,

• and that control signals are sent to the computer programme based upon these signals.

2. A device like in claim 1, where the control signals generated by the device represent two-dimensional or three-dimensional positional coordinates.

3. A device like in claim 1, where transient movements in the wrist, generated by snapping, tapping or flicking of the fingers or quick shaking of the hand are detected and used to generate singular instructions to the computer programme, in the same way as a computer mouse button click.

4. A device like in claim 1, where the roll sensing, i.e. the sensing of the rotation around the axis going though the forearm and into the hand, is made to detect the roll close to the centre of the wrist, by the use of two or more sensors located inside the device.

5. A device like in claim 1, where the movements of the wrist are detected by acceleration or force sensors inside the device, and that the sensor signals are sampled by a microcontroller with analogue- , to-digital signal conversion, where the microcontroller produces acceleration or force data to be sent to a computer programme.

6. A device like in claim 1, where the movements of the wrist are detected by acceleration or force sensors inside the device, and that the sensor signals are sampled by a microcontroller with analogue-to-digital signal conversion, where the microcontroller converts the acceleration or force data into position and movement data to be sent to a computer programme.

7. A device like in claim 1, where the movements of the wrist are detected by acceleration or force sensors inside the device, and that the sensor signals are sampled by a microcontroller with analogue-to-digital signal conversion, where the microcontroller converts the acceleration or force data into position and movement data, and subsequently generates corresponding control code, which can be used directly for control of computer programs.

8. A device like in claim 1, where the computer control signals are transmitted to the computer programme wirelessly.