RU2756162C1 - Method and complex for bionic control of technical apparatuses - Google Patents

Abstract

FIELD: medical equipment.

SUBSTANCE: group of inventions relates to medicine, namely to methods and complexes for bionic control of prostheses, orthoses, exoskeletons and game consoles. The complex is comprised of a sensor unit, a signal recording unit and a control action implementation unit. The sensor unit includes an electrode system for recording electrophysiological signals of EMG and electrical impedance, force sensors for recording MMG, inertial sensors and additional sensors. The signal recording unit includes input amplifiers, the outputs whereof are connected to the EMG processing unit, the impedance and MMG processing unit, an input amplifier connected to a processor and communicator in series, a control action forming unit, and a modem. The control action implementation unit includes a modem, a control unit and an executive unit, connected in series. Electrophysiological signals of EMG and electrical impedance are recorded by means of an electrode system. Mechanical, inertial and additional signals are recorded using corresponding sensors. All signals, namely electrophysiological signals of EMG, electrical impedance and MMG, inertial signals and additional signals, are recorded, processed and interpreted simultaneously. Signals are processed in the recording unit, and generation of control commands is implemented in the control action forming unit in real time.

EFFECT: increased accuracy of recognition of actions of the user is achieved.

7 cl, 1 dwg

Description

The invention relates to medicine and medical technology, namely to control systems for various technical devices, prostheses, orthoses, exoskeletons, game consoles and other similar devices, and is intended for control and interaction with other electronic devices, rehabilitation of people with diseases of the musculoskeletal system and recognition of actions.

Currently, in order to improve the quality of life and interact with special electrical systems, more and more bio-controlled robotic systems are used. When developing such systems, the key is the choice of a method for determining the type and parameters of the performed action.

In modern biotechnical control systems, the most widespread use of biopotentials, especially electromyography, is a method for studying bioelectric potentials that arise in the skeletal muscles of humans and animals when muscle fibers are excited and their electrical activity is recorded. It is known that innervation of a motor unit - a functional unit of a muscle, which is a bundle of muscle fibers - leads to membrane depolarization, as a result of which an action potential is generated and spreads along the muscle fibers, causing muscle contraction. The electromyogram (EMG) signal is the sum of action potentials from the muscles involved in contraction recorded by needle or surface electrodes [De Luca, Carlo (2006), Electromyography. Encyclopedia of Medical Devices and Instrumentation, Second Edition, Volume 3. John Wiley Publisher, pp. 98-109]. Processing and segmentation, with the extraction of informative features, allows the use of an electromyogram signal for recognition of actions and subsequent control of devices [Alter, Ralph (1966), Bioelectric Control of Prostheses, Technical Report: Massachusetts Institute of Technology, Research Laboratory of Electronics].

The known method of electromyographic control of prostheses according to the patent WO No. 2012150500, A61F 2/60. A61F 2/68 A61F 2/70, according to which, based on the electromyographic signal from the muscle, a control signal is generated, transmitted to the control unit and actuating the actuator.

The disadvantage of this method, as well as all methods of electromyographic control, is that the selection of signals from a specific muscle is difficult, since due to the additive nature of the EMG signal, action potentials from adjacent muscles overlap each other. Another disadvantage is that EMG reflects well only the beginning and end of contraction, but this signal cannot be used to judge the type of perfect movement.

A partial solution to this limitation is set forth in the "Method for bionic control of technical devices" presented by patents for invention and WO2017160183, A61B 5/0488, A61F 2/54. The essence of this method of bionic control is to obtain a control electrophysiological signal by passing a probing alternating electric current through the muscle using surface electrodes, and registering the electrical impedance, the change signal of which due to muscle contractions is used in addition to the EMG signal recorded from the same electrodes. The control signal is generated using both channels.

The above method makes it possible to ensure a sufficiently accurate recognition of basic hand movements, such as flexion-extension, grasp-opening, and to ensure the proportionality of the reproduction of the action.

However, the disadvantage of this method and device selected as a prototype is the sensitivity of the bioimpedance signal to the distortions of the electrode system, changes in the electrode-skin contact area and the pressing force, the lack of control of these parameters can lead to the registration of changes in the bioimpedance signal that are not associated with the performance of the action, and errors in the management of a technical device. Also, using only electrical impedance and EMG signals, it is not possible to avoid false alarms of the actuator in case of performing actions with the upper part of the limb, for example, with the shoulder when the sensors are located on the forearm while walking. This problem can be solved by using inertial sensors.

Known methods for controlling prostheses based on the use of various inertial sensors, such as an accelerometer, gyroscope, Hall sensor, etc., as well as their combinations and IMU-sensors to generate control signals.

The main disadvantage of such methods is that in order to recognize finger movements and gestures, it is necessary to use a special “glove” and place inertial sensors on each finger, as is implemented in patent RU 2670649 C1 G06F 3/01. What determines the main limitations of these systems is the impossibility of their use for the rehabilitation of patients with the loss of a limb, as well as the inconvenience caused by the dimensions of the structure. Also, the use of sensors is limited to recognizing types of gestures in which twisting is present, not allowing to recognize movements in which there is no twisting along the axis of the sensor [Munusamy T. R. Hand Pattern Recognition Using Smart Band // Theses and Dissertations. 2015. Vol. 1126].

Hall sensors allow you to determine angular displacements, are used in prosthetics to implement feedback, control the position of the limb in space and provide proprioceptive perception for the patient [Goršič, M., Kamnik, R., Ambrožič, L., Vitiello, N., Lefeber, D ., Pasquini, G., & Munih, M. (2014). Online Phase Detection Using Wearable Sensors for Walking with a Robotic Prosthesis. Sensors, 14 (2), 2776-2794]. However, systems using magnetic field sensors are susceptible to external fields surrounding the wearer of the device, which can be superimposed on the desired signal [Guanglin Li, & Kuiken, T. A. (2008). Modeling of Prosthetic Limb Rotation Control by Sensing Rotation of Residual Arm Bone. IEEE Transactions on Biomedical Engineering, 55 (9), 2134-2142].

The present invention is aimed at solving the above technical problem, namely, to overcome the limitations associated with the use of known methods of registration of movements for control tasks, i.e. ensuring proportional and anthropomorphic control of the executing mechanism (such as a prosthesis, orthosis, exoskeleton, game console, etc.), control of distortions and pressing force of the recording electrode system, determining the position of the executing mechanism in space, determining linear and angular displacements and increasing the information content of the resulting data by integrating biopotential (EMG, bioimpedance), mechanical (MMG, IMU) and additional (GPS, RFID, ultrasound, optical, voice) signals.

The expected technical result of the application of the proposed invention is an increase in the efficiency of bionic control of technical devices, namely, an increase in the accuracy of recognition of actions, provision of more accurate and anthropomorphic control, proportionality of the reproduction of actions, an increase in the number of recognizable actions, a decrease in signal distortions due to the control of distortions of the electrode system, as well as an increase in the accuracy of determining the position of the limb and / or the actuator in space.

To achieve this technical result, a method is proposed for bionic control of prostheses, orthoses, exoskeletons and game consoles, which consists in measuring signals, processing them and forming a control action, in which electrophysiological signals of an electromyogram (EMG) and electrical impedance are recorded by means of a joint tetrapolar system of electrodes, and to register an electrical impedance signal, an alternating high-frequency current is passed through the measurement area, and mechanical, inertial and additional signals are recorded using appropriate sensors, and all signals: electrophysiological signals of EMG, electrical impedance, mechanomyogram (MMG), inertial signals, and additional signals are registered, processed and interpreted at the same time, and the signal processing takes place in the registration unit, and the generation of control commands is implemented in the control its impact in real time.

A method for bionic control of technical devices is proposed, in which an additional signal is a signal to determine the location of a patient, which is registered using an RFID tag and GPS.

A method for bionic control of technical devices is proposed, in which an additional signal is a feedback signal and confirmation of the fact of command execution by the actuator and determination of the change in optical density of the measurement area when performing an action for supplying control commands, and this signal is recorded using optical sensors.

A method for bionic control of technical devices is proposed, in which an additional signal is a voice signal, which is recorded using a microphone for giving control voice commands by an operator through a pronounced phrase.

The technical result is also achieved by a complex for bionic control of prostheses, orthoses, exoskeletons and game consoles, consisting of a sensor unit, a signal registration unit and a control action implementation unit, in which the sensor unit includes a joint tetrapolar electrode system consisting of two current electrodes and measuring electrodes for registration of electrophysiological signals EMG and electrical impedance, force sensors for registering MMG, inertial sensors and additional sensors, and the signal registration unit includes input amplifiers, the outputs of which are connected to the EMG processing unit, the impedance and MMG processing unit, an input amplifier serially connected to the processor and a communicator, a control action generating unit and a modem, and a control action implementation unit includes a serially connected modem, a control unit and an executive unit.

A complex of bionic control of technical devices is proposed, in which inertial sensors are a gyroscope, an accelerometer and Hall sensors.

A complex of bionic control of technical devices is proposed, in which additional sensors are RFID tags, GPS, ultrasound, optical and voice sensors.

Thus, in the implemented method, a bioimpedance signal is used as electrophysiological signals, which can be used to determine the type of action performed (gripping, flexion, extension), and an electromyographic signal, which can be used to register muscle activity, determine the moment when the action starts. and also for the implementation of proportional control.

In the implemented method, a mechanomyographic signal is used as mechanical signals, with the help of which the pressing force and distortions of the electrode system are controlled, from this signal additional information can be obtained about the action taken to improve the control accuracy.

An accelerometer is used as inertial sensors, which allows to determine the presence of motion, a gyroscope, with which it is possible to determine and transmit angular displacements to the actuator, a Hall sensor is used to determine the position of a limb and / or an actuator in space, to ensure proprioceptive perception.

In the implemented method, additional information channels can be used corresponding to additional sensors, such as RFID tags and GPS, which can be used to determine the patient's location, to narrow the range of possible control commands sent to the actuator, optical sensors to implement feedback and confirmation of the fact of the execution of the command by the actuator, voice sensors - for recognizing voice commands.

To register electrical impedance in accordance with electrical safety standards, it is preferable to use a current with a frequency of 100 kHz and an amplitude of not more than 10 mA.

In this case, it is preferable to use a tetrapolar scheme of superposition of electrodes for recording EMG signals and electrical impedance. In this case, to increase the control efficiency, a multichannel arrangement of electrode systems should be used.

It is preferable that the whole process, starting from the registration of electrophysiological, mechanical and inertial signals and ending with the generation of control commands, takes no more than 100 ms.

Preferably, robotic devices, for example, prostheses, orthosis, exoskeleton, etc., as well as multimedia devices are used as the technical device.

Fig. 1: diagram of a complex of bionic control of technical devices

The bionic control complex for technical devices includes a sensor unit (1), a signal registration unit (2) and a control action implementation unit (3). The sensor unit (1) includes, but is not limited to, a joint tetrapolar electrode system consisting of current electrodes (4) and (5), and measuring electrodes for recording electrophysiological EMG and bioimpedance signals (6) and (7), force sensors for recording MMG 8, 9 inertial sensors (gyroscope, accelerometer and Hall sensor), and RFID tags, GPS, ultrasound, optical signal, voice can be used as additional sensors. The signal registration unit (2) includes input amplifiers (11) and (15), the outputs of which are connected to the EMG processing unit (12), the impedance processing unit (13) and MMG (16), the input amplifier (17) connected in series to the processor (18) and a communicator (19), a control action generating unit (20) and a modem (21). The block for the implementation of control actions (3) includes a modem (22), a control unit (23) and an executive device (24).

A high-frequency probing current passed through the muscle at the moment of contraction using current electrodes (4) and (5) is generated in a current source (10). The voltage recorded by the measuring electrodes (6) and (7) is the sum of the common-mode noise, the amplitude-modulated bioimpedance signal at a frequency of 100 kHz and the EMG signal. Common mode noise is suppressed by the input instrumentation amplifier (11). The EMG and electrical impedance signals are separated by bandpass filters with dynamic ranges corresponding to the dynamic ranges of signals in the EMG and impedance processing units (12) and (13). The mechanomyographic signal is recorded using force sensors, to which a reference voltage is supplied from a reference voltage source (14), passes through an input amplifier (15) to suppress common-mode noise. After that, the signal from the force sensors enters the MMG signal processing unit (16). The signals from the inertial sensors are also fed to the input amplifier (17), then they are processed and interpreted in the processor (18), which converts the data from the inertial sensors into orientation signals, and the communicator (19) is designed to interface information from all the inertial sensors. All registered signals, after their processing, enter the control action generating unit (20), which includes a microcontroller with the corresponding software, where control commands are generated, transmitted through the modem (21) to the modem (22) of the control unit (23), which interprets the control commands to manipulate the executive device 24 (executive unit (2)).

The following example illustrates the invention without limiting it in essence.

Example 1. Controlling an exoskeleton

Sensor units 1, including an electrode system for recording electrophysiological signals of EMG and bioimpedance, force sensors for recording MMG 8, inertial sensors 9 (gyroscope, accelerometer and Hall sensor) are placed in the projections of antagonist muscles (for example, flexor-extensors of the hand) on a healthy hand or on a stump (if the operator is disabled). The operator tenses and relaxes the muscles in the same way as in natural flexion and extension movements. The control electrophysiological signals of EMG and impedance, carrying information about the degree of muscle contraction and the type of action performed, come from the measuring electrodes 6 and 7 to the input amplifier 11 and, after separation and filtering in the EMG and impedance processing units 12 and 13, are fed to the control action formation unit 20 , where they are complexed with the MMG signal carrying information about the perfect muscle movement and pressing force, coming from the force sensors 8 and passing through the input amplifier 15 and the MMG processing unit 16, and the signal from the inertial sensors 9, which are interpreted and converted into orientation signals in the processor 18 and are combined in communicator 19.

In the block for generating the control action 20, due to the joint analysis of the mechanomyographic signal, EMG and impedance, the type of the performed action (flexion / extension) is determined and the corresponding commands are generated for the control object (exoskeleton), feedback is formed due to signals from inertial sensors, and the presence of movement is monitored and the position of the parts of the actuator (exoskeleton) and the limbs of the operator in space is determined.

The control commands are then transmitted through the modem 21 to the modem 22 of the control unit 23, which interprets the control commands in the manipulation of the actuator 24. Robotic devices can be used as the actuator: prostheses, orthoses, exoskeletons, multimedia devices.

The control unit can include a microcontroller that analyzes all registered signals simultaneously by analyzing their amplitude-time and frequency parameters, phase and shape analysis, neural networks, etc. materials.

Addition of a third mechanomyographic (MMG) channel to the above-described complex for bionic control, based on a non-invasive assessment of neuromuscular activity by recording mechanical muscle vibrations associated with a change in muscle cross-section during contraction using strain-gauge force sensors. Adding an MMG sensor to the bioimpedance channel and the EMG channel will allow not only to control the registration parameters affecting the quality of biopotential measurements, but also add information about the action performed, since the mechanomyogram signals have characteristic patterns of change for some types of movement, which makes it possible to use them for control tasks [Yungher D, Craelius W. Discriminating 6 grasps using force myography of the forearm // Northeast American Society of Biomechanics (NEASB) Conference. 2007] [Kadkhodayan A., Jiang X., and Menon C. Continuous Prediction of Finger Movements Using Force Myography // Journal of Medical and Biological Engineering. 2016. Vol. 36 (4). P. 594-604].

Claims (7)

1. A method of bionic control of prostheses, orthoses, exoskeletons and game consoles, which consists in measuring signals, processing them and generating a control action, characterized in that electrophysiological signals of electromyogram (EMG) and electrical impedance are recorded by means of a joint tetrapolar system of electrodes, and for registration an electrical impedance signal, an alternating high-frequency current is passed through the measurement area, and the registration of mechanical, inertial and additional signals is carried out using appropriate sensors, and all signals: electrophysiological signals of EMG, electrical impedance, mechanomyogram (MMG), inertial signals, and additional signals are recorded, are processed and interpreted at the same time, and the signal processing takes place in the registration unit, and the generation of control commands is implemented in the control action generating unit in real time. 2. The bionic control method according to claim 1, characterized in that the additional signal is a signal to locate the patient, which is registered using an RFID tag and GPS. 3. The method of bionic control according to claim 1, characterized in that the additional signal is a feedback signal and confirmation of the fact of command execution by the actuator and determination of the change in optical density of the measurement area when performing an action for supplying control commands, said signal being recorded using optical sensors ... 4. The method of bionic control according to claim 1, characterized in that the additional signal is a voice signal, which is recorded using a microphone for giving control voice commands by the operator through the spoken phrase. 5. A complex for bionic control of prostheses, orthoses, exoskeletons and game consoles, consisting of a sensor unit, a signal recording unit and a control action implementation unit, characterized in that the sensor unit includes a joint tetrapolar electrode system consisting of two current electrodes and measuring electrodes for registration of electrophysiological signals EMG and electrical impedance, force sensors for registering MMG, inertial sensors and additional sensors, and the signal registration unit includes input amplifiers, the outputs of which are connected to the EMG processing unit, the impedance and MMG processing unit, an input amplifier serially connected to the processor and a communicator, a control action generating unit and a modem, and a control action implementation unit includes a serially connected modem, a control unit and an executive unit. 6. The complex for bionic control according to claim 5, characterized in that the inertial sensors are a gyroscope, an accelerometer and Hall sensors. 7. The complex for bionic control according to claim 5, characterized in that the additional sensors are RFID tags, GPS, ultrasound, optical and voice sensors.

Images