RU2843784C1 - Method for diagnosing motor function disorders in individuals with motor disorders of various geneses by means of an intelligent system with biological feedback - Google Patents

Abstract

FIELD: medical science.

SUBSTANCE: invention refers to medicine and can be used in neurology, integrated medical rehabilitation for objective diagnosis of a nature of human motor function disorder and its recovery. Disclosed is a method for the diagnosis and rehabilitation of motor function disorders of individuals with motor disorders of various geneses by means of an intelligent system with biological feedback, characterized by the fact that, with the help of a virtual reality helmet, a user is shown a room, an avatar prompt appears in front of the user, clearly demonstrating the movement to be reproduced. Further, the video stream from the camera used to monitor the process of making the target movements by the patient is transmitted to a personal computer, which determines the anatomical landmarks on each frame, simultaneously with said electromyograph, recording the activity of the target muscles of the patient, determining the amplitude and speed of muscle contraction when performing the movement, part of electromyographic sensors is placed in points corresponding to maximum amplitude of contraction with target motions of upper extremities: shoulder abduction/adduction, shoulder flexion/extension, and the other group is positioned in similar points on the lower extremities; the EMG data are pre-processed, including filtering and removing artefacts. After that, the obtained video data on the spatial position of the joints and electromyography data are synchronized for further joint analysis, the combined data are analysed using machine learning algorithms, level of activation of each muscle in different phases of movement is assessed, taking into account electromyogram parameters, information on the position of the body and joints, as well as on the time for performing each exercise obtained using a stereo camera, avatar performs a target action when the avatar moves correctly in the game virtual environment. Wherein the leading control mechanism for the actions of the avatar is EMG data, an additional feedback channel when doing the exercises is sensory stimulation by means of technical devices that have a vibration effect on the skin in the points of the projection of the target muscles involved in the movement, if the exercise involves contact with the object of the virtual environment, providing the formation of tactile sensations closest to the natural ones, wherein the stimulating devices are connected to a microcontroller connected to a computer, after processing, the data are sent to a server, where they are used to classify the type and severity of motor disorders taking into account the individual pathology features in a patient, using a pre-trained neural network model, a conclusion is made.

EFFECT: higher accuracy and quality of motor disturbance type and severity classification.

1 cl, 1 dwg, 1 tbl

Description

The invention relates to medicine and can be used in neurology, complex medical rehabilitation for objective diagnostics of the nature of human motor function impairment and its restoration.

In the modern world, acute cerebrovascular accidents (ACVA) continue to be one of the leading causes of mortality and long-term disability all over the world. In developed countries, ACVA ranks first among all causes of disability. Most patients who have had a stroke experience neurological disorders, which are mainly represented by a decrease in motor, speech and cognitive functions. Research shows that movement disorders in the upper or lower extremities are observed in more than 80% of stroke survivors. Loss of the ability to perform voluntary movements (plegia) is the most common cause of disability after ACVA. Movement disorders also occur in other diseases of the central nervous system, such as Parkinson's disease, multiple sclerosis, and traumatic brain injury. Rehabilitation of these patients is aimed at the maximum possible restoration of motor functions and the ability to self-care.

A method for rehabilitation of movement disorders (RU 2 645 604 C1) is known through therapeutic exercise with feedback. Two variants of this method are proposed: the first - with partial immersion of the patient in a virtual environment and the creation of the appearance of repetition of movements in front of him; the second - with full immersion in a virtual environment. The classes include exercises to restore the functions of the upper and lower extremities and balance, as well as coordination of movements; after the patient has reached the initial level of movement performance, the required level is automatically increased to ensure more effective rehabilitation. The level of motor activity is assessed before and during the exercises using a stereo camera, which allows recording a three-dimensional image of the user. The disadvantages of the above-described hardware and software complex (HSC) are that it uses only information about the position of body parts in space to evaluate the movements performed; the HSC does not have a function for collecting electromyographic data from muscles involved in the exercises performed, which together forms an incomplete picture of the nature of changes in the patient's motor function, leads to a less accurate diagnosis and the possibility of forming an incorrect rehabilitation course. In addition, the rehabilitation method RU 2 645 604 C1 does not include a sensory feedback channel that triggers neuroplasticity processes and increases the effectiveness of the therapeutic measures.

A method for rehabilitation of patients at various stages of disorders of the central or peripheral nervous system using virtual reality is known (RU 2 655 200 C1), intended for use in the rehabilitation of patients at various times after the onset of pathology of the musculoskeletal system. This method uses a virtual environment with elements of control and sensory interaction with a virtual object. Taking into account the information received from the recording electroencephalographic and electromyographic sensors installed on the head and the affected limb, respectively, as well as the patient's ability to move, the volume of control virtual movements is regulated in such a way that a feeling of completion of the performed movement is created when demonstrating virtual reality tasks. The effect of associating the patient with the virtual avatar is ensured by using the visual, auditory channel, as well as tactile and proprioceptive stimulation of the limb receptors upon contact with virtual objects. This method is taken as a prototype of the present development. The disadvantage of the method RU 2 655 200 C1 is the lack of a function for diagnosing the nature of the patient's existing motor disorders with the formation of a conclusion and subsequent monitoring of the dynamics of motor dysfunction and the effectiveness of the rehabilitation carried out using the integration of artificial intelligence methods. In addition, feedback in the above-described method is implemented through the use of pneumatic cuffs that act in a non-localized manner on a fairly large area of the skin, which does not create a realistic impression of contact with objects of the environment.

The purpose of the present invention is to improve the accuracy and quality of classification of the type and severity of motor disorders even in cases of severe paresis by using a combination of optical tracking and EMG recording for diagnostics, and using machine learning algorithms for data analysis. Performing exercises in a game form in virtual reality conditions, the presence of polymodal feedback will provide an opportunity to reduce the rehabilitation period for patients with motor disorders of various origins, form an individual rehabilitation trajectory for each patient and increase its effectiveness. Automatic formation of a conclusion using a trained neural network model will minimize the participation of medical personnel in the rehabilitation process and improve the system of support for medical decision-making by integrating the developed algorithm.

The diagram of the hardware and software complex is shown in Fig. 1. The proposed method for diagnosing and rehabilitating the motor function of individuals with motor disorders is implemented as follows:

Using a virtual reality helmet (7), the user is shown a room with a minimalist design, done in neutral colors, with soft lighting. An avatar-hint appears in front of him, clearly demonstrating the movement that must be reproduced. A set of motor exercises in a virtual environment imitates the most common household actions (using cutlery, putting on clothes and shoes) and typical movements from everyday life (kicking a ball, etc.). The user follows the instructions displayed on the screen, performing the specified movements. Virtual reality helps to create a controlled, reproducible environment in which it is possible to measure and track the patient's movements with high accuracy; the technology allows you to adapt scenarios and tasks in accordance with the individual capabilities and type of activity of the patient, as well as imitate any scenes from real life depending on the clinical tasks. The inclusion of interactive game elements in the tasks helps to increase the patient's motivation. All these factors increase the effectiveness of the therapeutic procedures. Patients can perform tasks in a safe virtual environment, where the risk of physical injury is minimal. This is especially important for people with limited mobility.

The video stream from the camera (1), used to monitor the patient's performance of target movements (2), is transmitted to a personal computer (3). The computer runs a specially developed program with a neural network algorithm that identifies anatomical landmarks in each frame. These landmarks include the neck (central point), nose, shoulder joints (left and right), elbow joints (left and right), wrist joints (left and right), pelvis (central point), knees (left and right), and ankle joints (left and right).

At the same time, the electromyograph (4), connected to the computer, records the activity of the patient's target muscles, determines the amplitude and speed of muscle contraction during movement. Some of the electromyographic (EMG) sensors (5, 6) are placed at points corresponding to the maximum contraction amplitude during target movements of the upper limbs (e.g., shoulder abduction/adduction, shoulder flexion/extension, etc.), and another group is located at similar points on the lower limbs. The EMG data undergo preliminary processing, including filtering and artifact removal, using a pre-trained neural network.

The obtained video data on the spatial position of the joints and electromyography data are synchronized for further joint analysis. The combined data is analyzed using machine learning algorithms, the activation level of each muscle is assessed in different phases of the movement, taking into account the electromyogram indicators, information on the position of the body and joints, as well as the time of each exercise, obtained using a stereo camera. When the movement is performed correctly (according to the instructions), the avatar in the gaming virtual environment performs the target action, and the leading control mechanism for the avatar's actions is EMG data, which is especially important for patients with severe paresis, whose amplitude of movements is too low for full analysis using an optical tracker. The combination of optical tracking technologies (allows for an objective assessment of the position of the body and limbs in space, speed, amplitude and trajectory of movements performed, including complex movements and fine motor skills) and EMG (the most objective non-invasive method for measuring muscle activity, even in the absence of visually discernible movement) allows for a comprehensive analysis of the patient's motor disorders and the most accurate conclusion about the nature of the pathology.

Sensory stimulation by means of technical devices (8) acts as an additional feedback channel during exercise performance, exerting a vibration effect on the skin at the projection points of the target muscles involved in the movement, if the exercise includes contact with an object of the virtual environment, ensuring the formation of tactile sensations that are closest to natural ones. The stimulating devices are connected to a microcontroller (9) connected to a computer.

Thus, the presented method uses two channels of biological feedback: visual and vibrotactile, which helps to launch the mechanisms of neuroplasticity and restore impaired motor functions. It has been proven that multimodal feedback, combining several sensory feedback channels, is more effective in teaching motor skills compared to unimodal feedback.

After processing, the data is sent to the server, where it is used to classify the type and severity of movement disorders, taking into account the individual characteristics of the pathology in the patient, using a pre-trained neural network model, a conclusion is formed.

Thanks to the use of artificial intelligence methods, the analysis of the entire array of multimodal data obtained during the patient's exercise (video data, EMG) can be performed synchronized and in real time, with the user being able to control an avatar in virtual reality with visual and vibrotactile feedback. The use of a neural network trained on large data sets, as well as its ability to further self-train and improve in the process of processing newly incoming information, increases the accuracy and reliability of the diagnostics.

The algorithms of this hardware and software system can be integrated into the system of support for medical decision-making, which will allow for a personalized approach to motor rehabilitation, creating individual programs of rehabilitation therapy with minimal participation of medical personnel, which reduces the likelihood of errors associated with the human factor. Thus, this method will help increase the speed and efficiency of recovery of neurological patients with pathologies accompanied by impaired motor functions (acute cerebrovascular accident, multiple sclerosis, Parkinson's disease, etc.), as well as increase the speed of rehabilitation of patients with injuries of the musculoskeletal system.

The application of the developed method is illustrated by a clinical example. Patient L., 68 years old. Clinical diagnosis: Acute cerebrovascular accident - ischemic stroke in the basin of the right middle cerebral artery. At the time of admission to the rehabilitation department, 7 days had passed since the stroke, during which the patient received drug therapy according to the federal standard of medical care for patients with acute cerebrovascular accident. In the neurological status: general condition is satisfactory, consciousness is clear, oriented in place and time. Sense of smell is preserved. Fields of vision are not limited, there is no ptosis, the range of motion of the eyeballs is full, the pupils are of equal diameter, direct and consensual reaction to light are preserved. Palpation of the exit points of the trigeminal nerve is painless. Hearing acuity is normal. Dizziness, tinnitus, nystagmus are absent. The nasolabial fold is smoothed on the left. Swallowing is not impaired, there is no dysphonia. The pharyngeal reflex is preserved. The tongue deviates to the right, there is no dysarthria. The gait is hemiparetic, auxiliary devices are required when walking (cane). The muscle strength of the upper limbs is 2 points on the left, 5 points proximally and distally on the right. The muscle strength of the lower limbs is 3 points on the left, 5 points proximally and distally on the right. Muscle tone is increased on the left. Tendon reflexes are high on the left. Babinski reflex is on the left. In the Romberg position, the patient is unstable. The finger-nose test is performed with a miss on the left. Left-sided hemihypesthesia. Symptoms of tension and meningeal symptoms are negative. The functions of the pelvic organs are not impaired. Higher cortical functions are not impaired.

Motor function test results: Barthel Index - 70 points. Total motor function score according to the Fugl-Meyer scale - 42 points, total score according to the Fugl-Meyer scale: 122 points. NIHSS scale - 8 points.

The diagnostics of the motor function disorder using the developed method was carried out in the medical rehabilitation room. During the procedure, the patient was in a standing position, in front of a camera that performed optical tracking of the movements performed. A virtual reality helmet was put on the patient's head. The rehabilitation doctor gave the patient an introductory briefing and, via a personal computer, selected 2 exercises in a virtual environment for diagnostic and rehabilitation sessions: "Pouring water into a glass", "Kicking a ball".

To record the activity of target muscles during task performance, EMG sensors were placed on the following muscles of the left arm: deltoid muscle (Musculus deltoideus), biceps brachii muscle (Musculus biceps brachii), pronator teres muscle (Musculus pronator teres), radial flexor of the wrist (Musculus flexor carpi radialis), superficial flexor of the fingers (Musculus flexor digitorum superficialis), short flexor of the pollicis brevis. On the left lower limb, EMG electrodes were placed at the projection points of the abdomen of the following muscles: gluteus maximus (Musculus Gluteus maximus), rectus femoris (Musculus rectus femoris), vastus lateralis (Musculus vastus lateralis), biceps femoris (Musculus Biceps femoris), gastrocnemius (Musculus Gastrocnemius), and anterior tibialis (Musculus Tibialis anterior).

Stimulating vibration devices were placed on the palmar surface of the fingers and hand, as well as on the anterior surface of the shin and the dorsum of the foot on the skin surface, which simulated tactile sensations at the moment of the patient's contact with the object of the virtual environment. Thus, during the exercise, multimodal biological feedback was formed, simulating tactile sensations, using vibration motors producing vibration with the following characteristics: frequency of 10-120 Hz with a change step of 5%, amplitude of 0.2-0.5 mm, stimulation duration - 1 sec.

Diagnostics of motor disorders at the first visit using the developed method showed a violation of the motor function of the left upper and lower extremities (compared to the norm). Integrative indicators of the performance of target movements, taking into account the angle of movement in the joint and the level of activity of the muscles involved in the movement, were as follows (on a 100-point scale):

1) Exercise "Pouring water into a glass"

- shoulder flexion - 28 points

- shoulder abduction - 22 points

- elbow flexion - 20 points

- forearm pronation - 25 points

- flexion in the wrist joint - 16 points

- cylindrical grip with fingers of the hand - 14 points

2) Exercise "Kicking the ball"

- hip flexion - 64 points

- flexion and extension of the knee joint - 68 points

- dorsiflexion of the ankle joint - 48 points

EMG muscle activity indicators recorded by the PAC, using the example of the “Ball kicking” exercise:

Muscle Maximum signal amplitude in the swing phase (mV) Maximum signal amplitude in the strike phase (mV) gluteus maximus muscle 0.4 0.2 Rectus femoris muscle 0.6 1,1 vastus lateralis muscle 0.3 0.8 Biceps femoris muscle 0.7 0.5 gastrocnemius muscle 0.3 0.7 Anterior tibialis muscle 0.2 0.4

The patient then underwent a course of rehabilitation exercises on the developed system, including performing the above exercises in a virtual environment. The rehabilitation course consisted of 10 daily sessions lasting 30 minutes.

At the end of the rehabilitation course, the patient's motor functions were re-evaluated using clinical scales. Test results: Barthel Index - 90 points. Overall motor function score according to the Fugl-Meyer scale - 53 points, total score according to the Fugl-Meyer scale: 140 points. NIHSS scale - 6 points.

Integrative indicators of movement performance on the hardware and software complex after completing 10 training sessions on the PAC:

1) Exercise "Pouring water into a glass"

- shoulder flexion - 36 points

- shoulder abduction - 29 points

- elbow flexion - 26 points

- forearm pronation - 33 points

- flexion in the wrist joint - 21 points

- cylindrical grip with fingers of the hand - 18 points

2) Exercise "Kicking the ball"

- hip flexion - 83 points

- flexion and extension of the knee joint - 88 points

- dusty flexion in the ankle joint - 62 points

Completion of the training complex using the developed method allowed the patient to significantly restore key motor skills necessary for returning to independent movement. Due to this, the patient's overall physical activity increased, which increased his independence in performing everyday tasks and in self-care.

Claims (7)

A method for diagnosing motor function disorders in individuals with motor disorders using an intelligent system with biofeedback, characterized by the fact that using a virtual reality helmet, the user is shown a room with a minimalist design, done in neutral tones, with soft lighting, an avatar-hint appears in front of him, clearly demonstrating the movement that needs to be reproduced, then the video stream from the camera used to monitor the patient's performance of target movements is transmitted to a personal computer running a program with a neural network algorithm that identifies anatomical landmarks in each frame, with these landmarks including the central point of the neck, nose, shoulder joints, elbow joints, wrist joints, central point of the pelvis, knees and ankle joints, simultaneously with this electromyograph (EMG) connected to a computer, the patient's muscle activity is recorded, the amplitude and speed of muscle contraction are determined when performing a movement, and some of the electromyographic sensors are placed at points corresponding to the maximum contraction amplitude during movements of the upper limbs, namely: abduction or adduction of the shoulder, flexion or extension of the shoulder; and the remaining part of the electromyographic sensors are placed at points corresponding to the maximum contraction amplitude during movements of the lower limbs, and the EMG data undergo preliminary processing, including filtering and removal of artifacts, using a pre-trained neural network, the obtained video data on the spatial position of the joints and electromyography data are synchronized for further joint analysis, the combined data is analyzed using machine learning algorithms, an assessment is made of the activation level of each muscle in different phases of the movement taking into account the electromyogram indicators, information on the position of the body and joints, as well as the time of execution of each exercise obtained using a stereo camera, when the movement is correctly performed, the avatar in the gaming virtual environment performs an action, and the leading control mechanism for the avatar's actions is the EMG data, sensory stimulation through technical devices designed to exert vibrational effects on the skin at the points of the muscles involved in the movement acts as an additional feedback channel when performing exercises, while the stimulating devices are connected to a microcontroller connected to a computer, After processing, the data is sent to the server, where it is used to classify the type and severity of movement disorders, taking into account the individual characteristics of the pathology in the patient, and a conclusion is formed using a pre-trained neural network model.