TRAINING PLATFORM. LANGUAGE. AND METHOD FOR CONTROLLING MULTI-FUNCTION MYOELECTRIC PROSTHESIS
FIELD JOF THE INVENTION
The present invention relates generally to prosthetics, and specifically to apparatus and methods for active control of a prosthetic hand and wrist.
BACKGROUND OF THE INVENTION
Below-elbow amputees have long had the unsatisfied need for a prosthesis which adequately and attractively reproduces the function and appearance of the lost forearm and hand. Low-tech solutions have ranged from a basic hook to simple mechanical graspers, which open and close in response to the application of appropriate forces to elements thereof. The advent of the microprocessor has enabled prostheses to incorporate somewhat increased functionality compared to their strictly-mechanical counterparts, but even these electro-mechanical hands are typically limited to two commands, such as "open grip" and "close grip." A greater number of motions is only attained, if at all, by using a cumbersome and time-consuming command structure and/or by assigning one of the subject's muscles to myoelectrically control one function (e.g., open/close grip), and another muscle to control another function (e.g., pronate/supinate wrist). There is no prosthetic hand which offers a practical substitute to the large variety of functions performed by a biological hand. In particular, the electro-mechanical prostheses use electrically-driven motors, which are greatly limited in their practical applications. Whereas an electro-mechanical prosthesis could theoretically give an amputee an almost unlimited amount of dexterity, in practice, each additional degree of freedom would be accompanied by an increase in weight, noise, and energy consumption such as to render the prosthesis unacceptable. US Patents 5,888,213, 5,679,004, 5,673,367, 5,443,525, 5,341,813, 5,336,269,
5,219,366, 4,808,187, 4,558,704, 4,158,196, and 4,030,141, which are incorporated herein by reference, describe various aspects of the use of myoelectric signals in the control of a prosthesis.
An article entitled, "Neural control of a virtual prosthesis," by Eriksson et al., ICANN 98, Perspectives in Neural Computing, which is incorporated herein by reference, describes the use of a neural network to interpret electromyographic signals
and to control a virtual prosthesis, in order to simulate the behavior of future prostheses which are beyond the current state-of-the-art to implement.
SUMMARY OF THE INVENTION
It is an object of some aspects of the present invention to provide apparatus and methods for preparing a subject to use a prosthesis.
It is a further object of some aspects of the present invention to provide improved apparatus and methods for controlling a prosthesis.
In preferred embodiments of the present invention, a subject is prepared to use a prosthesis, such as an electro-mechanical prosthetic hand, by training the subject to control a virtual prosthesis displayed on a screen of a training computer. Preferably, a set of one or more electromyographic (EMG) electrodes are placed on the subject's skin, in the vicinity of at least one muscle selected by the subject or by a healthcare professional. During a training session, the subject contracts and relaxes the muscle according to pre-designated and/or subject-defined patterns, in order to control the virtual prosthesis.
Typically, the training computer and/or a dedicated signal collection and processing unit external thereto receives from the electrodes a myoelectric signal generated by the subject's muscle, amplifies the signal, and performs other initial processing and filtering of the signal as appropriate to yield a processed signal. The training computer interprets aspects of the processed signal so as to determine a desired disposition of the virtual prosthesis from amongst a relatively-large list of possible dispositions. These may include, by way of illustration and not limitation:
• open hand,
• close hand - cylindrical grip less than 50 grams, • turn wrist right 90 degrees,
• adopt hook grip - 5 kilograms,
• adopt lateral grip - 2 kilograms,
• fine grip (i.e., cause tip of thumb to contact tip of index finger), and/or
• stop motion. The disposition of the virtual prosthesis displayed on the monitor is preferably updated in real-time responsive to the training computer's interpretation of the processed signal, so as to provide realistic visual feedback to the subject indicative of
the substantially identical effect that will be caused in an actual electro-mechanical prosthesis, when it is eventually fitted to the subject.
Preferably, the subject can select one disposition from the list of possible .dispositions of the virtual prosthesis using a series of contractions and relaxations of the muscle. This series of muscle activations is typically easily performed by the subject in a very short time period, e.g., less than about two seconds or, further preferably, less than one second. The inventors believe that training the subject to perform the contractions and relaxations associated with each possible disposition of the prosthesis is most efficiently (although not necessarily) achieved using a virtual prosthesis, as described herein. For example, a virtual prosthesis displayed on the screen of a training computer can be combined with displayed suggestions for improvement or refinement of characteristics of the subject's activation of the muscle, such as the timing or magnitude of a contraction or a relaxation. Additionally, the virtual prosthesis can be used to determine whether a given subject is likely to be able to successfully adapt to the actual electro-mechanical prosthesis, prior to investing the time of the subject and numerous healthcare professionals, and incurring the expenses associated with obtaining and customizing an actual prosthesis for the subject's use.
Preferably, the training computer analyzes the processed EMG signal to determine three or more consecutive components thereof, each component having a magnitude associated therewith. The magnitudes are preferably interpreted by the training computer to fall within ranges corresponding to no contraction (0), low- magnitude contraction (1), or high-magnitude contraction (2). Thus, by way of illustration and not limitation, a command telling the prosthesis to adopt a cylindrical grip may be 2-0-1, i.e., hard contraction, relaxation/pause, soft contraction. It is noted that, by analogy to the use of the <Control>, <Function>, and <Alt> keys commonly placed on computer keyboards, the digital language governing the prosthesis can also be enriched by allowing, for example, 2-2-2 to indicate that the following command issued by the subject is to be interpreted according to a different set of rules.
Alternatively or additionally, a second set of one or more electrodes coupled to another one of the subject's muscles may be used in combination with or separately from the first set of electrodes described hereinabove. Thus, for example, the subject could generate a 5-component EMG signal, wherein the first three components are detected by the first set of electrodes, and the last two components are detected by the
second set of electrodes. Alternatively, the second set of electrodes could be configured to detect signals relating to a particular aspect of the prosthesis' functioning, such as speed of motion and force of grip. It is believed that after a sufficient period of training with the virtual prosthesis and/or regular jise of the electro-mechanical prosthesis, the subject will be able to give even complex multi-component commands without investing a great amount of conscious effort, in the same way that most adults typically flawlessly perform the complex and delicate task of tying their shoes, without even the smallest amount of concentration.
Preferably, the language restricts the first component to a value of 1 or 2, thereby excluding 0 from being the first component. In this manner, the possible ambiguity of 0-1-2 and 1-2-0 (zero indicating no contraction) is avoided. It will therefore be seen that the basic set of commands available to the subject has up to 18 possibilities for controlling the prosthesis, a significantly greater number than that provided by prior art prosthetic control systems. This figure does not include the even greater number of possible commands available by using 2-2-2 or another sequence, as described hereinabove. Typically, some of the 18 commands are reserved for system commands, but in any case, most embodiments of the present invention enable the subject to select from at least 12 unique dispositions of the hand, including a range of movement dispositions (e.g., right, left, open, close, fast, slow, stop), grip dispositions (e.g., cylindrical grip, fine grip, lateral grip, hook grip), and complex sequences enabled by a single command.
Complex sequences as provided by these embodiments of the present invention may include, for example, "Open locked door," by which the subject generates a 2-2-1 EMG signal in order to cause the prosthesis to execute the following steps: (a) lateral grip, (b) pause 2 seconds, (c) right 90 degrees - force setting high, and (d) left 90 degrees - force setting low. This level of functionality thus approaches, to a certain extent, the way that a locked door is opened by a natural hand, i.e., healthy adults typically do not consciously consider every single step in a routinely-performed operation. Rather, they decide "I will open the door with the key," and the entire operation proceeds essentially automatically.
Preferably, each subject is enabled to program the virtual prosthesis with customized complex sequences, or particular dispositions that are generally unique to the subject. The pre-programmed sequences and customized dispositions are
subsequently downloaded into a control unit of the actual electro-mechanical prosthesis prior to fitting it to the subject. In some embodiments, during regular operation of the electro-mechanical prosthesis, the subject is enabled to continually add additional functionality to the prosthesis by entering in new sequences and customized dispositions or by editing existing sequences and dispositions.
It is to be understood that whereas preferred embodiments of the present invention are described herein with respect to interpreting magnitudes of the consecutive components of the EMG signal, the scope of the present invention includes, alternatively or additionally, interpreting other aspects of the EMG signal as well. In a preferred embodiment, for example, the rate of onset of each component is used in interpreting the signal. Thus, 1-0-2 may be input by some subjects in the form of a slow contraction, a pause, and a rapid contraction. Alternatively or additionally, for some subjects, a product or other function of the rate and the magnitude is used to define each component of the EMG signal. This mode is particularly advantageous for those subjects who intermittently experience difficulty controlling the degree of contraction, but who, nevertheless, tend to involuntarily couple the rate of a muscle's contraction with the desired magnitude of the muscle's contraction.
In some preferred embodiments of the present invention, one or more calibration parameters are modified, so as to optimize the relationship between the subject and the virtual prosthesis displayed on the screen of the training computer, and, more importantly, the relationship between the subject and the actual electromechanical prosthesis with which s/he will ultimately be fitted. For example, during a calibration period, the training computer may instruct the subject to repeatedly enter a calibration pattern, such as 1-2-1, 1-2-1, 1-2-1, until the computer has determined a range of timing and magnitude parameters characteristic of the EMG signal generated by the subject's muscles when the subject produces the calibration pattern. Optionally, the training computer instructs the subject to enter several calibration patterns, such as 1-2-1, 1-1-2, 2-0-2, in which the various values 0, 1, and 2 occupy different locations in the sequence. In this manner, it can be determined if, for example, the subject tends to generate a 2 which has a different timing or magnitude when it is in one of the positions compared to the corresponding characteristic when it is in another one of the positions.
It is expected that during regular use of the electro-mechanical prosthesis, the subject will not always be strictly consistent with the calibration parameters determined
during the initial calibration period — nor is there any need for such strict consistency. Late at night, for example, a subject may be reasonably expected to increase the time between each of the components of the EMG signal, or to reduce the magnitude of the muscle contractions. Preferably, ttxe control unit of the electro-mechanical prosthesis monitors such gradual changes in the manner in which the subject enters commands to the prosthesis, and automatically re-calibrates the prosthesis responsive to these gradual changes. Alternatively or additionally, if the subject detects that the prosthesis is not responding optimally to entered commands, s/he may enter a repeated series of 1-2-1's, so as to put the control unit into a recalibration mode and restore the optimal behavior of the prosthesis.
It is noted that conditions external to the subject may also be responsible for a transient decline of function of any electromyographic system. In particular, electromagnetic radiation from a number of common sources can potentially contaminate an electromyographic reading, and is a substantial obstacle to the proper functioning of even the simple electromyographic prostheses which are known in the art. Therefore, it is preferable for a subject in such a high-noise environment to intermittently recalibrate the prosthesis to ensure the best possible performance. Various techniques known in the art for improving signal-to-noise ratio may be adapted for use in improving the quality of the EMG reading, such as encouraging the subject to overcome the environmental noise by increasing the precision, strength and/or duration of both the low magnitude and the high magnitude contractions. Moreover, a non- myoelectric sensor, such as a mechanical sensor, may be used in addition to or instead of electrodes placed on the skin, in order to detect contractions of the subject's muscle. In some situations, the non-myoelectric sensor is believed to produce data which are at least as good as the data produced by myoelectric electrodes.
Optionally, a button coupled to the control unit, or other input means (including electromyographic signaling), is provided to enable the subject to indicate to the prosthesis whenever an incorrect response is made to a command given electromyographically by the subject. All responses to commands which are not so labeled by the subject are, correspondingly, assumed by the control unit to have been correctly executed. Preferably, the control unit re-analyzes the inputted electromyographic signals which were incorrectly interpreted, so as to determine characteristics thereof (e.g., component magnitudes and durations) which differ from
other signals which were correctly interpreted. If substantial differences are detected, then the control unit preferably re-calibrates the prosthesis accordingly.
There is therefore provided, in accordance with a preferred embodiment of the present invention, a method for training a subject, including: receiving a signal generated responsive to at least one contraction of one or more muscles of the subject; processing the signal; modifying a disposition of a virtual prosthesis responsive to processing the signal; and providing visual feedback to the subject indicative of the disposition of the virtual prosthesis, so as to train the subject to contract the one or more muscles in a manner which results in a desired disposition of the virtual prosthesis.
Preferably, processing the signal includes: determining two or more consecutive components of the signal; and determining for each component a value range which includes the component, the value range being selected from four or fewer designated value ranges.
Further preferably, processing the signal includes determining an indication of a desired disposition selected from eight or more dispositions of the prosthesis.
In a preferred embodiment, receiving the signal includes receiving a myoelectric signal conveyed by three or fewer electrodes coupled to at least one muscle of the subject, and processing the signal includes determining an indication of a desired disposition selected from five or more dispositions of the prosthesis.
Alternatively or additionally, the virtual prosthesis includes a virtual prosthetic hand, and processing the signal includes determining an indication of a desired grip selected from two or more grips of the virtual prosthetic hand.
Preferably, processing the signal includes: determining two or more consecutive components of the signal; and adjusting a calibration parameter of the virtual prosthesis responsive to a characteristic of at least one of the components. In a preferred embodiment, processing the signal includes determining a desired sequence of two or more dispositions of the prosthesis, and wherein modifying the
, disposition of the virtual prosthesis includes causing the virtual prosthesis to adopt each of the dispositions in the sequence, subsequent to processing the signal.
, Preferably, receiving the signal includes receiving a myoelectric signal or receiving .a non-fnyoelectric signal. In a preferred embodiment, the method includes transferring data responsive to the signal to a control unit of a physical prosthesis.
There is further provided, in accordance with a preferred embodiment of the present invention, a method for controlling a prosthesis, including: receiving a signal generated responsive to at least one contraction of one or more muscles of a subject; processing the signal to determine two or more consecutive components of the signal; determining for each component a value range which includes the component, the value range being selected from four or fewer designated value ranges; and modifying a disposition of the prosthesis responsive to the determined value ranges.
There is still further provided, in accordance with a preferred embodiment of the present invention, a method for controlling a prosthesis, including: receiving a signal generated responsive to at least one contraction of one or more muscles of a subject; processing the signal to determine an indication of a desired disposition selected from eight or more dispositions of the prosthesis; and actuating the prosthesis to adopt the desired disposition responsive to processing the signal. Preferably, processing the signal includes determining an indication of a desired disposition selected from twelve or more dispositions of the prosthesis.
There is yet further provided, in accordance with a preferred embodiment of the present invention, a method for controlling a prosthesis, including: receiving a myoelectric signal conveyed by three or fewer electrodes coupled to at least one muscle of a subject; processing the signal to determine an indication of a desired disposition selected from five or more dispositions of the prosthesis; and
actuating the prosthesis to adopt the desired disposition responsive to processing the signal.
There is also provided, in accordance with a preferred embodiment of the present invention, a method for controlling a prosthetic hand, including: receiving a signal generated responsive to at least one contraction of one or more muscles of a subject; processing the signal to determine an indication of a desired grip selected from two or more grips of the prosthetic hand; and actuating the prosthetic hand to adopt the desired grip responsive to processing the signal.
There is additionally provided, in accordance with a preferred embodiment of the present invention, a method for regulating a prosthesis, including: receiving a signal generated responsive to at least one contraction of one or more muscles of a subject during regular operation of the prosthesis; processing the signal to determine two or more consecutive components of the signal; and adjusting a calibration parameter of the prosthesis responsive to a characteristic of at least one of the components.
There is yet additionally provided, in accordance with a preferred embodiment of the present invention, a method for controlling a prosthesis, including: receiving a signal generated responsive to at least one contraction of one or more muscles of a subject; processing the signal to determine a desired sequence of two or more dispositions of the prosthesis; and subsequent to processing the signal, actuating the prosthesis to adopt each of the dispositions in the sequence.
There is still additionally provided, in accordance with a preferred embodiment of the present invention, a method for controlling a prosthesis, including: receiving during a first period a first signal generated responsive to at least one contraction of one or more muscles of a subject;
receiving from the subject during the first period an indication of a desired disposition of the prosthesis and an assignment of the desired disposition to the first signal; receiving during a second period a second signal; processing the second signal during the second period to determine whether it corresponds to the first signal; and actuating the prosthesis to adopt the desired disposition during the second period, responsive to determining that the second signal corresponds to the first signal.
There is also provided, in accordance with a preferred embodiment of the present invention, a method for controlling a prosthesis coupled to a limb of a subject, including: receiving by wireless communication a signal generated responsive to at least one contraction of one or more muscles of the subject which are not in the limb; processing the signal to determine an indication of a desired disposition of the prosthesis; and actuating the prosthesis to adopt the desired disposition responsive to processing the signal.
There is further provided, in accordance with a preferred embodiment of the present invention, a method for controlling a prosthesis, including: receiving an instruction from a subject while the subject is not wearing the prosthesis; adjusting an operational parameter of the prosthesis responsive to the instruction; receiving a signal generated responsive to at least one contraction of one or more muscles of the subject while the subject is wearing the prosthesis; and modifying a disposition of the prosthesis while the subject is wearing the prosthesis, responsive to the signal and the operational parameter.
Typically, receiving the instruction includes receiving an assignment of a myoelectric signal pattern to a disposition of the prosthesis. There is still further provided, in accordance with a preferred embodiment of the present invention, a method for controlling a prosthesis, including: determining an onset of a change of state of the prosthesis;
receiving a signal generated responsive to at least one contraction of one or more muscles of a subject wearing the prosthesis, responsive to determining the onset of the change of state; processing the signal; and adjusting a calibration parameter of the prosthesis responsive to processing the signal.
Preferably, determining the onset of the change of state includes determining a mode change of the prosthesis into an active mode, or determining a change in an ambient noise level. There is yet further provided, in accordance with a preferred embodiment of the present invention, a method for controlling a prosthesis, including: receiving a non-myoelectric signal generated responsive to contraction of one or more muscles of a subject; processing the signal to determine an indication of a desired disposition of the prosthesis; and electrically actuating the prosthesis to adopt the desired disposition responsive to processing the signal.
Typically, receiving the non-myoelectric signal includes receiving a signal generated responsive to a mechanical aspect of contraction of the one or more muscles. There is also provided, in accordance with a preferred embodiment of the present invention, a computer program product for training a subject, the product including a computer-readable medium having program instructions embodied therein, which instructions, when read by a computer, cause the computer to: receive a signal generated responsive to at least one contraction of one or more muscles of the subject; process the signal; modify a disposition of a virtual prosthesis responsive to processing the signal; and provide visual feedback to the subject indicative of the disposition of the virtual prosthesis, so as to train the subject to contract the one or more muscles in a manner which results in a desired disposition of the virtual prosthesis.
There is additionally provided, in accordance with a preferred embodiment of the present invention, apparatus for training a subject, including: a data port, adapted to receive a signal generated responsive to at least one contraction of one or more muscles of the subject; a display, adapted to display a virtual prosthesis; and a processor, adapted to process the signal, and to modify a disposition of the virtual prosthesis on the display responsive to processing the signal, so as to train the subject to contract the one or more muscles in a manner which results in a desired disposition of the virtual prosthesis. There is yet additionally provided, in accordance with a preferred embodiment of the present invention, apparatus for controlling a prosthesis, including: a data port, adapted to receive a signal generated responsive to at least one contraction of one or more muscles of a subject; and a processor, adapted to process the signal to determine two or more consecutive components of the signal, to determine, for each component, a value range which includes the component, the value range being selected from four or fewer designated value ranges, and to actuate the prosthesis to modify a disposition thereof responsive to the determined value ranges.
There is still additionally provided, in accordance with a preferred embodiment of the present invention, apparatus for controlling a prosthesis, including: a data port, adapted to receive a signal generated responsive to at least one contraction of one or more muscles of a subject; and a processor, adapted to process the signal to determine an indication of a desired disposition selected from eight or more dispositions of the prosthesis, and adapted to actuate the prosthesis to adopt the desired disposition.
There is also provided, in accordance with a preferred embodiment of the present invention, apparatus for controlling a prosthetic hand, including: a data port, adapted to receive a signal generated responsive to at least one contraction of one or more muscles of a subject; and a processor, adapted to process the signal to determine an indication of a desired grip selected from two or more grips of the prosthetic hand, and adapted to actuate- the prosthetic hand to adopt the desired grip.
There is further provided, in accordance with a preferred embodiment of the present invention, apparatus for controlling a prosthesis, including: a data port, adapted to receive a signal generated responsive to ' at least one contraction of one or more muscles; of a subject; and a processor, adapted to process the signal to determine a desired sequence of two or more dispositions of the prosthesis, and adapted to actuate the prosthesis to adopt each of the dispositions in the sequence, subsequent to the processor processing the signal.
There is yet further provided, in accordance with a preferred embodiment of the present invention, apparatus for controlling a prosthesis coupled to a limb of a subject, including: a data port, adapted to receive by wireless communication a signal generated responsive to at least one contraction of one or more muscles of the subject which are not in the limb; and a processor, adapted to process the signal to determine an indication of a desired disposition of the prosthesis, and adapted to actuate the prosthesis to adopt the desired disposition.
There is still further provided, in accordance with a preferred embodiment of the present invention, apparatus for controlling a prosthesis, including: a non-myoelectric sensor, adapted to generate a signal responsive to contraction of one or more muscles of a subject; and an electronic processor, adapted to process the signal to determine an indication of a desired disposition of the prosthesis, and adapted to electrically actuate the prosthesis to adopt the desired disposition. Preferably, the sensor includes a mechanical sensor, thermal sensor, or an ultrasound sensor.
There is additionally provided, in accordance with a preferred embodiment of the present invention, a method for controlling an electronic device, including: receiving a signal generated responsive to at least one contraction of one or more muscles of a subject; processing the signal to determine two or more consecutive components of the signal;
determining for each component a value range which includes the component, the value range being selected from four or fewer designated value ranges; and modifying a disposition of the electronic device responsive to the determined value ranges. There is still additionally provided, in accordance with a preferred embodiment of the present invention, a method for controlling an electronic device, including: receiving a signal generated responsive to at least one contraction of one or more muscles of a subject; processing the signal to determine an indication of a desired disposition selected from four or more dispositions of the electronic device; and actuating the electronic device to adopt the desired disposition responsive to processing the signal.
There is yet additionally provided, in accordance with a preferred embodiment of the present invention, apparatus for controlling an electronic device, including: a data port, adapted to receive a signal generated responsive to at least one contraction of one or more muscles of a subject; and a processor, adapted to process the signal to determine two or more consecutive components of the signal, to determine, for each component, a value range which includes the component, the value range being selected from four or fewer designated value ranges, and to actuate the electronic device to modify a disposition thereof responsive to the determined value ranges.
There is also provided, in accordance with a preferred embodiment of the present invention, apparatus for controlling an electronic device, including: a data port, adapted to receive a signal generated responsive to at least one contraction of one or more muscles of a subject; and a processor, adapted to process the signal to determine an indication of a desired disposition selected from four or more dispositions of the electronic device, and adapted to actuate the electronic device to adopt the desired disposition.
The present invention will be more fully understood from the following detailed description of the preferred embodiments thereof, taken together with the drawings, in which:
BRIEF DESCRIPTION OF THE DRAWINGS
Fig. 1 is a simplified pictorial illustration showing apparatus for preparing a subject to use a prosthesis, in accordance with a preferred embodiment of the present invention;
Fig. 2- is a simplified pictorial illustration showing an electro-mechanical prosthesis in use, in accordance with a preferred embodiment of the present invention; and
Fig. 3 is a simplified pictorial illustration showing an electro-mechanical prosthesis in use, in accordance with another preferred embodiment of the present invention. DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS
Reference is now made to Figs. 1 and 2. Fig. 1 is a simplified pictorial illustration showing training apparatus 20 for preparing a subject 30 to use a prosthesis, in accordance with a preferred embodiment of the present invention. Fig. 2 is a simplified pictorial illustration showing an electro-mechanical prosthesis 110 in use by subject 30, in accordance with a preferred embodiment of the present invention.
For clarity and simplicity, a significant portion of the description with reference to the figures is stated with respect to either a virtual prosthesis 50 (Fig. 1), whose dispositions are controlled by a training computer 40, or with respect to electromechanical prosthesis 110 (Fig. 2), whose dispositions are controlled by a control unit 120. It is nevertheless to be understood that a large number of the techniques and apparatus which are used in creating and operating one of the prostheses can be applied directly to creating and operating the other one of the prostheses. Thus, for example, a digital language is described hereinbelow with respect to training the subject to use the virtual prosthesis. This digital language is subsequently typically utilized by the subject, without any substantive changes, in order to control the electro-mechanical prosthesis. Moreover, the subject is preferably enabled to transfer to the electromechanical prosthesis, as appropriate, substantially all of the calibrations performed with the virtual prosthesis and dispositions and sequences programmed into the virtual prosthesis. In this manner (and as is apparent from the description hereinbelow), the virtual prosthesis serves as a sophisticated training and preparation platform for the subject prior to the subject being fitted with the electro-mechanical prosthesis, and which additionally provides a smooth transition from training to regular operation.
Preferably, apparatus 20 trains the subject to control virtual prosthesis 50 displayed on a screen 42 of training computer 40. For this purpose, a set of one or more electromyographic (EMG) electrodes 100 is preferably placed on the subject's skin, in the vicinity of at least one muscle selected by the subject or by a healthcare professional. Thus, although electrodes 100 are shown in Fig. 1 as being coupled to the subject's upper arm, it is to be understood that the subject could elect to have some or all of electrodes 100 placed at different sites on the subject's body, responsive to the subject's medical condition, or as convenient for the subject. Often a site remote from the subject's arm is used to advantage when muscles on the arm have atrophied from non-use and/or because the subject lacks suitable coordination of the muscles on the arm. It is noted that, because of the opportunity for frequent re-calibration of apparatus 20 and of prosthesis 110, as described herein, the subject is not limited during regular operation to only using the muscle which was used during the initial training sessions. Prior art prosthetic devices, by contrast, are limited in their ability to permit a subject to change electrode locations, because their re-calibration abilities are more restricted, and therefore their ability to function properly would be compromised responsive to varied electrode placement sites.
Preferably, during a series of one or more training sessions, a human trainer gives instructions, or instructions are displayed in a box 46 on screen 42, so as to train subject 30 to contract and relax the muscle according to pre-designated and/or subject- defined patterns, in order to control virtual prosthesis 50. Typically, training computer 40 and/or a dedicated signal collection and processing unit 22 external thereto is coupled by a cable 102 or by wireless communication means (not shown) to receive from electrodes 100 a myoelectric signal generated by the subject's muscle. The inventors have found that a signal sampled at between about 100 and 1,000 Hz is appropriate for most purposes. Successful results have been obtained, for example, at 400 Hz. Using techniques described hereinbelow, the signal is preferably amplified, filtered, and processed by unit 22, in order to yield a processed EMG signal 60, components thereof being indicative of contractions of the muscle intentionally caused by the subject for the purpose of modifying a disposition of virtual prosthesis 50. Alternatively or additionally, apparatus and methods described in one or more of the above-cited US Patents are preferably utilized, at least in part, in the sensing, amplification, filtering and/or initial processing of the signal.
Training computer 40 preferably interprets aspects of EMG signal 60 so as to distinguish the desired new disposition of virtual prosthesis 50 from amongst a relatively-large list of possible dispositions. These may include, by way of illustration and not limitation, one or more of the dispositions shown in a window 44 on screen 42 and/or in sample Table I, which is typically developed to suit the particular needs of a given subject. Aspects of the three digit EMG code shown in Table I are described further hereinbelow.
TABLE I
From the perspective of subject 30, the large range of EMG words constitutes a rich digital language with which the subject can communicate with the prosthesis. Preferably, subject 30 selects one disposition from the list of possible dispositions of the virtual prosthesis using a series of contractions and relaxations of the muscle. This series of muscle activations is typically easily performed by the subject in a very short time period, e.g., less than about two seconds or, further preferably, less than one second. In experiments performed by the inventors, for example, volunteers who underwent less than an hour of training have learned to direct virtual prosthesis 50 in about 1.8 seconds to perform the commands specified by any of the three digit EMG words shown in Table I. Advantageously, the skills which the subject developed while learning to control virtual prosthesis 50 are preferably directly transferable to the subject's subsequent regular operation of electro-mechanical prosthesis 110.
Preferably, training computer 40 displays EMG signal 60 on screen 42, and substantially simultaneously analyzes the signal to determine three or more consecutive components thereof, each component having a magnitude. The magnitudes are preferably interpreted by the training computer to fall within ranges corresponding to no contraction (0), low magnitude (1), or high magnitude (2). Thus, for example, Fig. 1 shows that subject 30 has entered a 1-2-2 command called "Get a key" (lateral grip), by producing a soft contraction followed by two hard contractions. It will be appreciated that other patterns, different from but based on those described herein, could also be used to control a prosthesis while remaining within the scope of the present invention. For example, four consecutive "contraction" or "no contraction" EMG signals may be used to define up to sixteen dispositions (e.g., 1-1-1-0), without differentiating between hard contractions and soft contractions. Similarly, two consecutive EMG signals, each having four possible values, may also be used to define up to sixteen different dispositions of the prosthesis (e.g., 2-4). Alternatively or additionally, other digital or analog techniques known in the art for communicating a selected value drawn from a range of values may be employed. The inventors have found, however, that three consecutive components each having one of three possible values (no contraction, small contraction, large contraction) tends to be easy to learn, and provides a sufficient number of dispositions of the prosthesis for most purposes.
Table I provides an option for one user-defined sequence, and options for a greater number of user-defined and/or pre-programmed sequences could be provided
using techniques described herein. Complex sequences as provided by this embodiment of. the present invention may include, for example, "Open locked door," by which the subject generates a 1-2-2 EMG word in order to cause virtual prosthesis 50 (and subsequently' electro-mechanical prosthesis 110) to execute the following steps: (a) lateral grip, (b) pause 2 seconds, (c) right 90 degrees - force setting high, and (d) left 90 degrees - force setting low.
Preferably, subject 30 is enabled to program virtual prosthesis 50 and electromechanical prosthesis 110 with customized complex sequences, or particular dispositions that are generally unique to the subject. For example, a carpenter may program the electro-mechanical prosthesis to respond to a 1-2-2 command by (a) adopting a disposition suitable for holding a wooden beam firmly against a table, and (b) setting a maximum force to be applied to the beam. Similarly, pre-programmed sequences and customized dispositions which are used during the subject's training on computer 40 are subsequently downloaded into control unit 120 of electro-mechanical prosthesis 110 prior to fitting it to the subject. For some applications, a playback- control window 48 on screen 42 displays single dispositions of virtual prosthesis 50 as well as entire sequences which are created by the subject or are pre-programmed in the prosthesis. Preferably, during regular operation of electro-mechanical prosthesis 110, subject 30 is enabled to continually add additional functionality to the prosthesis by entering in new sequences and customized dispositions.
As appropriate, the digital language provided by the use of three consecutive components each having one of three values can be supplemented using techniques known in the art. For example, by analogy to the use of the <Control>, <Function>, and <Alt> keys commonly placed on computer keyboards, the digital language described herein can be enriched by allowing, as shown in Table I, 2-2-2 to indicate that a following command issued by the subject is to be interpreted according to a different set of rules. Advantageously, whereas practical considerations generally exclude the use of a zero as the first component in an EMG word (leaving a maximum of 18 out of 27 possible words), use of 2-2-2 to introduce a subsequent command allows the subsequent command to include a zero as the first component. In this manner, a total of 18 + 27 = 35 commands can easily be expressed by subject 30. Although those of ordinary skill in the art would be able to supplement even further the digital prosthesis-control language described herein, the inventors note that for most
practical purposes, the ability to communicate even 6-10 commands (and certainly 18- 35 commands) is sufficient for most prosthesis users.
In order to correctly interpret EMG signal 60, and thereby determine the value (0, 1, or 2) of each component thereof, a statistical analysis of the signal is preferably performed so as to determine transitions between: (a) periods of relaxation, in which low level muscle electricity, sensor artifacts, and environmental noise are detected, and (b) periods of intentional contraction of the muscle. A preferred method for analyzing n points X _ in EMG signal 60, so as to determine these transitions, includes estimating the kurtosis of the signal (i.e., the extent to which outliers in signal 60 have a higher rate of occurrence than they would have in a normal distribution having the same mean (μ ) and standard deviation), using the following equation:
A higher value of Kc is indicative of a greater likelihood that subject 30 has intentionally contracted the muscle to which electrodes 100 are coupled. By contrast, a value of Kc equal to zero is understood to indicate that variations in the Xj_ are due to random fluctuations. The inventors have found that if Kc is less than about two during relaxation of the muscle, then a threshold value for Kc of approximately six is generally suitable for determining whether subject 30 has intentionally contracted the muscle. Experiments performed with a number of volunteers have shown that this threshold is high enough to eliminate signal noise from adversely affecting interpretation of signal 60 in the vast majority of cases, while not requiring the subjects to exert themselves too hard in order to generate correctly-interpreted contractions. It is to be understood that this threshold value may be modified, as appropriate, depending on environmental conditions or other factors. For example, in a high-noise environment, a threshold higher than six may be suitable.
For some applications, the skewness of EMG signal 60 is calculated, in addition to or instead of the kurtosis, so as to determine the likelihood that sensed electrical activity is generated intentionally by the subject. Intentional generation is generally
associated with higher skewness, and skewness lower than two is generally associated with random noise. Those skilled in the art, having read the disclosure of the present patent application and/or using techniques which are generally known, will be able to determine other suitable methods for analyzing EMG signal 60 for the purposes of this emDodiment of the present invention.
Having determined that detected electrical activity corresponds to intentional contraction, training computer 40 preferably subsequently determines whether the contraction is: (a) intended as part of an EMG word, so as to control virtual prosthesis 50, or (b) a contraction for purposes substantially unrelated to the virtual prosthesis. Typically, a calibration period is provided in which training computer 40 learns characteristics of the subject's contractions which are indicative of either (a) or (b). Thus, for example, subject 30 may be instructed to make the duration of each contraction in a 3-component EMG word be between about 0.04 and 1.0 second. Detected contractions having durations outside of this range would be rejected. Typically, the duration of the relaxation period between successive contractions is between about 0.08 and 1.0 second. The ability of the subject to produce contractions having generally consistent characteristics (e.g., durations between 0.50 and 1 second) is determined by training computer 40 during the calibration period, so as to enable the computer to reduce both false-positive and false-negative identifications of contractions as being a part of ~ or not a part of ~ an EMG word. Most volunteers who have practiced generating the digital language using training apparatus 20 have learned in less than an hour to produce contractions which were correctly interpreted by computer 40.
In addition to calibrating timing parameters of the contractions, training computer 40 preferably determines magnitudes of contractions which are generated by the subject when the subject intends to communicate the digits 1 and 2 in an EMG word. In a preferred embodiment, the training computer instructs subject 30 to relax the muscle while a calibration of the inherent noise detected by electrodes 100 is performed. Alternatively, the subject is instructed to input an EMG word including a 0 (corresponding to relaxation), such as 2-0-0 or 1-1-0. A minimum threshold 61 (Fig. 1) for indicating a 1 is preferably determined responsive to a statistical analysis of the noise. For example, the noise may be characterized by a mean (μ ) and a standard deviation (Srj) thereof, and threshold 61 defined by the following equation:
Threshold 61 = μ + C * Sj, where C is a coefficient preferably determined for each subject during an" initial calibration session, and optionally updated intermittently thereafter. Typically, C is set between about 10 and 20. Using these values, EMG signal 60 is therefore only interpreted as exceeding threshold 61 if it contains data points which are greater than 10 or 20 standard deviations away from the mean of the detected noise signal. Because optimum functioning of virtual prosthesis 50 and electro-mechanical prosthesis 110 is dependent to some extent on an adequate characterization of noise, this calibration procedure is preferably performed each time electrodes 100 are placed on the subject's skin, whenever it may happen that the subject notices less than optimum functioning of the prosthesis, as well as intermittently during regular operation of the prosthesis (e.g., once every two hours).
Subsequent to the noise assessment, the subject is typically instructed to generate one or more 1-2-1 EMG words. Training computer 40 preferably continues the calibration by performing each of the steps in the following list:
• The training computer waits until it detects EMG signal 60 rising higher than threshold 61 for a "Signal Actively Generated" time period tsag^ of between about 0.04 and 1.0 second, in order to register input of the first 1 in the 1-2-1 EMG calibration word. EMG signals which are below threshold 61, and those that are above threshold 61 for a tsagι duration outside of the specified range, are rejected from consideration.
• After registering input of the first 1, training computer 40 preferably waits for a "No Signal Actively Generated" period tnsag of up to about 1 second, prior to detecting the 2 of the EMG calibration word, which typically is required to last for a period tsag2 between about 0.04 and 1.0 second. If tnsagj is greater than 1 second, or if tsa_-2 is not within the designated range, then the EMG calibration word is rejected, and the subject is instructed to try again.
• In a similar fashion, training computer 40 detects a period tnsag2 of less than 1 second duration, during which EMG signal 60 is less than threshold 61, following the input of the second component in the calibration word. The period tnsag2 precedes input of the calibration word's third component, which has a magnitude of at least threshold 61, and typically has a duration of 0.04 s < tsag3 < 1.0 s.
• Finally, to complete the calibration, training computer 40 preferably verifies that:
(a) the magnitude of the 2 in the 1-2-1 calibration word is at least a determined percentage (e.g., 50%) greater than both of the l's in the EMG calibration word, and
(b) the magnitudes of both of the l's in the calibration word do not differ by more than a factor of two.
When subject 30 has successfully entered the EMG calibration word 1-2-1, training computer 40 preferably defines a threshold 62, for determination of a 2 in an EMG word as shown in Fig. 1, according to the following equation: Threshold 62 = [A(2) + MAX {A(l), A(3)}]/2, where A(i) is the amplitude of EMG signal 60 during generation of the itn component in the EMG calibration word 1-2-1, and MAX {x, y} is defined as the greater of x or y.
In addition, a representative duration tDjt for use during regular operation is defined, as follows: tbit = 0.5 * ( 0.5 * tsagl + tsag2 + 0.5 * tsag3 + tnsagl + tnsag2)-
The particular values and other details of the calibration procedure described herein are to be understood as examples only, and it is expected that a person skilled in the art, having read this disclosure, will be able to determine other calibration procedures suitable for some subjects without departing from the general scope of the present invention.
Optionally, training computer 40 instructs subject 30 to enter several calibration patterns, such as 1-2-1, 1-1-2, and 2-0-2, in which the various values 0, 1, and 2 occupy different locations in the sequence. In this manner, it can be determined if, for example, subject 30 tends to generate a 2 which has a different timing or magnitude when it is in one of the positions compared to the corresponding characteristic when it is in another one of the positions.
During regular operation of electro -mechanical prosthesis 110, control unit 120 preferably stays in a low power consumption standby mode, receiving and processing EMG signal 60 from electrodes 100 to determine whether the signal differs from the noise previously recorded by more than, for example, 4 standard deviations. If so, then an enhanced level of signal processing is preferably initiated, to determine whether signal 60 exceeds threshold level 61. If signal 60 is found to exceed level 61 or level
62 for a time period 0.04 s < t < tbjt, then the control unit registers the input of a 1 or a 2, respectively, as the first component of a new EMG word. Otherwise, the control unit returns to the standby mode.
Preferably, control unit 120 determines the duration tn "sag > 0.08 seconds of a relaxation period of the muscle, following determining that a 1 or a 2 has been entered. If tnsag is greater than tDjt, then the control unit registers a 0 input for the second component of the EMG word. If, on the other hand, tnsag is less than ttøt, then the control unit determines that a 1 or a 2 is being entered by subject 30 for the second component, and the control unit prepares to determine whether a valid input is being entered, and whether it is a 1 or 2. In like manner, the third component of the new EMG word is input into the control unit. It is to be understood that the particular protocol for interpreting EMG signal 60 based on known calibration parameters is described herein by way of illustration and not limitation.
It is expected that during regular use of electro-mechanical prosthesis 110, subject 30 will not always be strictly consistent with the calibration parameters determined during the initial calibration period ~ nor is there any need for such strict consistency. Late at night, for example, subject 30 may be reasonably expected to increase the time between each of the components of EMG signal 60, or to reduce the magnitude of the muscle contractions. Preferably, control unit 120 monitors such gradual changes in the manner in which the subject enters commands to the prosthesis, and automatically re-calibrates the prosthesis responsive to these gradual changes.
Alternatively or additionally, if subject 30 detects that the prosthesis is not responding optimally to entered commands, s/he may enter a repeated series of 1-2-1's, so as to put the control unit into a recalibration mode and restore the optimal behavior of the prosthesis, as described hereinabove.
In summary, those skilled in the art will appreciate that this embodiment of the present invention is highly resistant to incorrect interpretation of environmental noise as a valid instruction, because the calibration functions described hereinabove typically define threshold level 61 as being statistically distinct (e.g., by at least 10-20 standard deviations) from the ambient noise level, and they further define level 62 as being above level 61 by an amount particular to the subject's contractions demonstrated during calibration. Moreover, the use of defined time periods (i.e., with respect to q-jt) significantly reduces the possibility that muscle contractions, even if they are of the
appropriate magnitudes, will be mistakenly interpreted as words in the digital language if these muscle contractions are unrelated to intentional control of the prosthesis.
Typically, the digital language used by training computer 40 and electromagnetic prosthesis 110 restricts the first component of each EMG word to a value of 1 or 2, thereby excluding 0 from being the first component. In this manner, the possible ambiguity of 0-1-2 and 1-2-0 (zero indicating no contraction) is avoided. It will therefore be seen that the basic set of commands available to subject 30 has up to 18 possibilities for controlling the prosthesis, a significantly greater number than that provided by prior art prosthetic control systems. In a preferred embodiment, a watch 130 (Fig. 2) comprises one or more buttons
134 and a display 132, and is provided in one integral unit with control unit 120 or separate therefrom. Preferably, watch 130 enables the control unit to convey a large range of diagnostic or other messages to subject 30, including, for example: "Please enter 1-2-1 calibration sequence" • "Increase distinction between l's and 2's"
"Contractions are too long - try producing short bursts" "Press '>' to save new sequence" or
"Select current mode: Kitchen / Outdoors / Office / Billiards / etc." Preferably, one of buttons 134 enables subject 30 to indicate to the prosthesis whenever an incorrect response is made to an electromyographic command. All responses to commands which are not so labeled by the subject are, correspondingly, typically assumed by control unit 120 to have been correctly executed. Preferably, the control unit re-analyzes the inputted electromyographic signal which was incorrectly interpreted, so as to determine characteristics thereof (e.g., component magnitudes and durations) which differ from other signals which were correctly interpreted. If substantial differences are detected, then the control unit preferably re-calibrates the prosthesis accordingly, either automatically or by prompting the subject to enter 1-2-1.
It is to be understood that whereas preferred embodiments of the present invention are described herein with respect to interpreting magnitudes of the consecutive components of EMG signal 60, the scope of the present invention includes, alternatively or additionally, interpreting other aspects of the EMG signal as well. In a preferred embodiment, for example, the rate of onset of each component is used in interpreting signal 60. Thus, 1-0-2 may be input by some subjects in the form of a slow
contraction, a pause, and a rapid contraction. Alternatively or additionally, for some subjects, a product or other function of the rate and the magnitude is used to define each component of the EMG signal. This mode is particularly advantageous for those subjects who intermittently experience difficulty controlling the degree of contraction, but who, nevertheless, tend to involuntarily couple the rate of a muscle's contraction with the desired magnitude of the muscle's contraction. Further alternatively or additionally, other analog aspects of signal 60 are interpreted in concert with the digital aspects generally described herein, or separately therefrom.
It is to be further understood that whereas some preferred embodiments of the present invention are described herein with respect to interpreting an EMG signal generated responsive to contractions of the subject's muscle, the invention is not limited to EMG signals per se. For some applications, one or more sensors 140 convey non- myoelectric signals to training computer 40 and control unit 120, in addition to or instead of EMG signal 60 conveyed by electrodes 100. These non-myoelectric signals are preferably interpreted using substantially the same techniques as those described herein with respect to EMG signal 60 (e.g., multiple components, each having a limited number of possible values), mutatis mutandis. Thus, a large range of sensors which are known in the art may be adapted for use in order to generate an electric signal responsive to muscle contraction. For example, sensors 140 may comprise one or more of the following:
• a mechanical sensor,
• a strain gauge,
• a vibration sensor,
• a vibration transducer, which transmits a vibration into the muscle, and detects phase, frequency, or other characteristics of return vibrations, which characteristics vary responsive to changes in the state . of contraction of the muscle,
• a thermal sensor,
• an ultrasound transducer, • a motion sensor, and
• a sensor which uses tissue resonance analysis techniques.
Fig. 3 is a simplified pictorial illustration showing electro-mechanical prosthesis
110 in use, in accordance with a preferred embodiment of the present invention.
Preferably, one or more sets of electrodes 100 are coupled to one of the subject's muscles that is not on the subject's arm. This configuration may be used in combination with or separately from the set of electrodes 100 shown on the subject's arm and described hereinabove with reference to Figs. 1 and 2. Using the additional electrodes 100, subject 30 is enabled to generate, for example, a 5-component EMG signal, wherein the first three components are detected by one of the electrode sets, and the last two components are detected by another one of the electrode sets. Alternatively or additionally, one of the sets is configured to detect signals relating to a particular aspect of the prosthesis' functioning, such as speed of motion and force of grip, while another set conveys a signal corresponding to mode selection and action initiation. Further alternatively or additionally, subject 30 contracts a muscle not on the arm, to indicate the imminent initiation of a three-component EMG word, and after an appropriate delay (e.g., less than t_.[{), electromyographically enters a three-component EMG word via the electrodes which are on the arm. Advantageously, this embodiment allows the use of a 0 as the first component of the EMG word, because there is no ambiguity between, for example, X-0-1-2 and X-l-2-0 (where X represents the detected contraction of the muscle not on the subject's arm).
Preferably, but not necessarily, electrodes 100 which are remote from the subject's arm comprise one or more electrodes 150, which transmit EMG signals through respective wireless transmitters 160. Typically, transmitters 160 comprise low-power infrared or radio-frequency transmitters, and allow the subject to comfortably take advantage of the increased functionality provided by more than one set of electrodes. The inventors believe that, this increased functionality would be, for many applications, effectively unavailable without the use of wireless technology, because of the increased burden on the subject of having a number of wires running across>his body.
It will be understood by one skilled in the art that aspects of the present invention described hereinabove can be embodied in a computer running software, and that the software can be supplied and stored in tangible media, e.g., hard disks, floppy »disks or compact disks, or in intangible media, e.g., in an electronic memory, or on a network such as the Internet.
It will be appreciated by persons skilled in the art that the present invention is not limited to what has been particularly shown and described hereinabove. Rather, the
scope of the present invention includes both combinations and subcombinations of the various features described hereinabove, as well as variations and modifications thereof that are not in the prior art, which would occur to persons skilled in the art upon reading the foregoing, description. In particular, it is noted that whereas preferred embodiments of the present invention are described hereinabove with respect to controlling a prosthesis, the scope of the present invention includes applying these principles, mutatis mutandis, to controlling substantially any electronic device. Thus, a healthy or a handicapped person may exercise sophisticated myoelectric control over a television, e.g., by changing channels and controlling volume. Alternatively or additionally, the apparatus described herein may be adapted to enable a person to exercise generally simultaneous control over a range of electronic devices (e.g., room lights, radio, telephone, and oven), preferably using' techniques of wireless communications known in the art.