RU2844624C1 - Neurorehabilitation system based on brain-computer interface with multimodal feedback circuits for correction of cognitive and motor disorders of various nature - Google Patents

Abstract

FIELD: medical equipment.

SUBSTANCE: neurorehabilitation system based on brain-computer interface comprises brain electrical activity recording unit (1), preliminary data processing unit (2), significant features detection unit (3), states binary classification unit based on pre-trained machine learning model (4), adaptive classification unit (5) with possibility of additional training of model on individual patient data, classifier selection unit (6), rehabilitation scenario implementation unit for launching personalized rehabilitation procedures (7), classifier adaptation unit. Adaptive classification unit provides classification on the basis of the extracted significant features from the EEG data recorded during the patient’s rehabilitation procedure. Classifier adaptation unit provides additional training and optimization of the classifier model parameters on the basis of the data recorded during the rehabilitation procedure.

EFFECT: achieving the possibility of controlling the complexity of tasks by varying the level of distracting sensory stimuli that increases the effectiveness of neurorehabilitation of patients with cognitive and motor disorders, making it possible to adapt the rehabilitation process to the needs of the patient.

1 cl, 2 dwg, 1 ex

Description

The invention relates to medicine, namely to neurology, and can be used for rehabilitation and prevention of patients with cognitive and motor disorders of various natures based on multimodal biological feedback. The invention is intended for use in inpatient medical organizations of general, neurological and rehabilitation profiles for working with patients with mild (subclinical) and severe disorders of cognitive processes and motor functions of the upper limbs.

A method for rehabilitation of patients who have suffered a stroke is known [RU 2523349, IPC A61B 5/0476, A61B 5/0482, published 20.07.2014, Bulletin No. 20], including a training complex that allows the patient to perform tasks on imagining the movement of the paretic limb with subsequent control. The device records an electroencephalogram (EEG), transmits data to a computer for synchronous processing and identification of desynchronization of the sensorimotor rhythm responsible for the imaginary movement. The device determines the result of the mental task and displays the results on the screen as a mark, allowing the patient to visually control the correctness of the exercise. A disadvantage of this invention is the use of only visual feedback to control the correctness of the task. This approach limits the possibilities of rehabilitation, since patients with visual or cognitive impairments may experience difficulties in interpreting visual information.

A known method for rehabilitation of post-stroke and post-traumatic patients by conducting training of a paretic limb includes presenting the patient with a task of kinesthetic imagination of limb movement, analyzing the patient's brain activity patterns that arise during movement imagination, transmitting this data to a computer to extract signals responsible for movement imagination [RU 2622206, IPC A61B 5/0476, A61B 5/0482, published 13.06.2017, Bulletin No. 17], used after a stroke or injury using a robotic complex including a human limb exoskeleton controlled via a brain-computer interface by means of movement imagination. The invention analyzes the patient's brain activity patterns during kinesthetic imagination of limb movement. The computer extracts signals responsible for the imagined movement and transmits the results to the screen in the form of a mark. If the task is performed correctly, the exoskeleton on the paretic limb moves it in the direction of the imagined movement, and if an error occurs, in the opposite direction. This increases the effectiveness of treatment by using visual, tactile and proprioceptive feedback. A disadvantage of the invention is the limitation of its use in patients with impaired tactile and proprioceptive sensitivity, which reduces the effectiveness of the method in such cases. Interaction with the exoskeleton may be difficult or inadequately perceived by patients with severe sensory impairments, which makes it impossible to effectively use the proposed rehabilitation method. In addition, the movement of the limb in the opposite direction when the task is performed incorrectly may cause psychological discomfort and decreased motivation for further training.

A known method for stimulating morphofunctional structures of the brain to implement rehabilitation and/or prevent neurodegeneration [RU 2823580, IPC G16H 10/00, A61B 5/05, published 24.07.2024, Bulletin No. 21] includes an assessment of the morphofunctional state of the brain to identify areas with degenerative changes or to improve intellectual functions in healthy people. The user is trained to use a brain-computer interface for mental control in a virtual environment. The classifier adapts based on brain signals to accurately classify activity patterns. In the process of adaptive learning, neuroplasticity mechanisms are activated in the target areas of the brain, which helps to improve intellectual abilities or restore functions. The method focuses on visual feedback and mental control through the brain-computer interface. The lack of integration of other sensory modalities (tactile, proprioceptive, auditory) limits the ability to stimulate various sensory pathways of the brain, which may reduce the overall effectiveness of rehabilitation. Thus, the lack of a multisensory environment limits the ability of the system to create a realistic simulation for training complex cognitive and motor skills. It should also be noted that inappropriate or excessive stimulation of certain areas of the brain may lead to side effects such as increased symptoms of anxiety, depression, impaired cognitive function, or even the occurrence of epileptic activity. This is especially true for patients with multiple lesions or with disorders affecting several functional systems of the brain.

A system and method for the rehabilitation (restoration) of the sense of smell and taste, as well as the correction of emotional and cognitive disorders using a brain-computer interface are known [RU 2768172, IPC A61H 99/00, A61M 11/00, A61M 15/00, A61B 5/369, 23.03.2022, Bulletin No. 9]. The system analyzes signals from the olfactory cortex and evoked potentials, comparing them with a database. Based on the interpretation of the signals, commands are generated for the olfactory display and the display device, which stimulate the restoration of the sense of smell and taste. The system provides neural feedback between stimuli and the patient's sensations, which increases the effectiveness of rehabilitation by activating brain activity and coordinating the work of various areas of the brain. A disadvantage of the invention is that the proposed system is based on the interpretation of brain activity signals using a pre-prepared database. However, each patient may have individual differences in neural patterns associated with olfaction and taste, which may lead to a decrease in the accuracy of signal recognition and interpretation. In particular, patients may have different experience in the perception of smells and tastes, as well as different levels of damage to the olfactory and taste pathways. Using a standard database without taking into account individual patient characteristics may limit the accuracy of classification and, therefore, reduce the effectiveness of rehabilitation.

A known method for the rehabilitation of cognitive functions in patients with focal brain lesions is based on the use of a brain-computer interface [RU 2749408, IPC A61B 5/16, A61B 5/369, G06F 3/01, 09.06.2021, Bulletin No. 16], in which the patient performs tasks using the NeuroChat neurocommunication complex, which records EEG using a headset. During the calibration stage, EEG reactions to the illumination of letters are recorded. At the next stage, the patient mentally, by shifting attention from letter to letter, types a given word without touching the keyboard. Sessions are held daily for 1 hour for 10 days. The invention helps restore cognitive functions through the use of brain-computer technology. The disadvantage of this method of neurorehabilitation is the use of monotonous tasks that require a high level of concentration, which quickly tires patients and reduces their motivation and reduces the effectiveness of restoring cognitive functions.

The closest to the claimed invention is a neurorehabilitation system and a neurorehabilitation method [RU 2741215, IPC A61B 5/0476, A61F 2/72, published 01/22/2021, Bulletin No. 3], including a visual display device, a brain activity recording device, a robotic device for influencing a trained object, and a computer with a database. The computer recognizes and interprets brain activity signals, transmits commands to the robotic device or the visual display device. The system can use virtual reality, an exoskeleton, various methods of recording brain activity, such as EEG, functional near infrared spectroscopy or MRI, as well as a combination of these methods. The neurorehabilitation method includes a visual task, signal recording, processing them, transmitting commands to the robotic device and visual display of task completion. Additionally, electrical myostimulation and muscle activity recording can be used. The disadvantages of this invention include the use of standard databases and signal processing methods, which may not take into account the individual neural patterns and physiological characteristics of each patient. This reduces the accuracy of signal recognition and, accordingly, the effectiveness of rehabilitation. The use of electrical stimulation is also associated with a number of limitations and potential risks. In particular, when using it, patients may experience unpleasant or painful sensations, especially if the current parameters are not individually adjusted, which can reduce their motivation to continue therapy and cause unwanted stress reactions. Moreover, forced muscle contraction without control by the patient can cause psychological discomfort and a feeling of helplessness, which negatively affects the emotional state and motivation for rehabilitation. It is also worth noting that the complexity and multi-component nature of the system, including virtual reality, an exoskeleton and various activity recording methods, makes it difficult and labor-intensive to set up and operate. As a result, the use of the invention requires the participation of highly qualified personnel, which limits the availability of the method and increases the costs of its implementation.

To eliminate these shortcomings, a system based on a brain-computer interface with multimodal biofeedback circuits is proposed for the correction of cognitive and motor disorders of various origins.

The technical problem of the present invention is the need to develop an effective neurorehabilitation system with a wide range of applications, which is capable of taking into account the individual neural patterns and physiological characteristics of each patient, while providing multisensory feedback and adaptability to various levels of cognitive and motor impairments.

The technical result of the claimed invention is a system that, using multimodal feedback, an individually adaptable brain-computer interface and the ability to regulate the complexity of tasks by varying the level of distracting sensory stimuli, significantly increases the effectiveness of neurorehabilitation of patients with cognitive and motor disorders, allowing the rehabilitation process to be adapted to the patient's needs.

The specified result is achieved by the fact that the neurorehabilitation system based on the brain-computer interface with multimodal feedback loops for the correction of cognitive and motor disorders of various natures contains the following blocks: a visual stimulation loop block, a brain electrical activity recording block, a vibrotactile stimulation loop block, a transcranial magnetic stimulation loop block, a block implementing preliminary processing processes, a block for identifying significant features in motor imagination, a classification block based on a machine learning model trained on data from a pre-formed patient database, an adaptive classification block based on training a classifier on individual patient data obtained during the rehabilitation procedure, a block for implementing a rehabilitation scenario.

The invention is explained by drawings, where Fig. 1 shows a detailed functional block diagram of the system, and Fig. 2 shows a functional diagram of the adaptive classification block.

The positions in the drawing Fig. 1 indicate:

1 - block for recording electrical activity of the brain;

2 - data pre-processing block;

3 - block for identifying significant features;

4 - classification block based on the database;

5 - adaptive classification block;

6 - classifier selection block;

7 - block for implementing the rehabilitation scenario;

8 - visual stimulation circuit block;

9 - vibrotactile stimulation circuit block;

10 - transcranial magnetic stimulation circuit block.

The arrows in Fig. 1 indicate the connections between the corresponding blocks, defining their inputs and outputs.

The positions in the drawing Fig. 2 indicate:

1a - classification block;

2a - storage block;

3a - classifier adaptation block.

The arrows in Fig. 2 indicate the connections between the corresponding blocks, defining their inputs and outputs.

The neurorehabilitation system with multisensory feedback that takes into account individual neural patterns of the patient and ensures effective restoration of cognitive and motor functions via a brain-computer interface with the ability to adjust the complexity of tasks, contains the following units: an EEG recording unit (1) that records and pre-amplifies the signal; a data pre-processing unit (2) that pre-filters electroencephalographic signals and removes unwanted artifacts; a significant feature detection unit (3) that calculates significant characteristics from EEG time series; a database-based classification unit (4) that classifies the states of motor imagery of the target limb based on a machine learning model trained on a patient database; an adaptive classification unit (5) that classifies the states of motor imagery of the target limb based on data individually recorded for each patient; a classifier selection unit (6) that compares the accuracy of the classification algorithms used; a rehabilitation scenario implementation unit (7) that carries out a personalized rehabilitation procedure and is capable of displaying rehabilitation results; a visual stimulation circuit unit (8) that displays visual stimuli; a vibrotactile stimulation circuit unit (9) that vibrates the subject's target limb; a transcranial magnetic stimulation circuit unit (10) that pre-modulates the activity of the patient's target (damaged or trained) cerebral cortex, such as the motor cortex.

The adaptive classification block contains: a classification block (1a) that carries out classification based on the significant features selected from the EEG data recorded during the patient's rehabilitation procedure, and takes into account the results of the classification block based on the database (4); a storage block (2a) that stores previous classification results, significant features selected, and EEG data time series; a classifier adaptation block (3a) that carries out additional training and optimization of the classifier model parameters based on the data recorded during the rehabilitation procedure.

The device works as follows.

The EEG recording unit (1) records the electrical activity of the patient's brain. This unit includes preliminary amplification of signals from electrodes installed on a pre-selected area, for example, the motor cortex. The system allows the use of dry or wet electrodes connected according to a mono- or bipolar scheme and providing the recording of at least two EEG signals. The recorded signal is fed to the input of the data pre-processing unit (2), where the signal is filtered in the range of 1-40 Hz, and unwanted artifacts are removed, in particular, oculomotor ones. After pre-processing and filtering, the EEG signals are fed to the significant feature detection unit (3). Various algorithms used in brain-computer interfaces can be used to detect significant features, for example, a spatial filter(CSP - Common Spatial Pattern) or methods based on Riemann geometry such as the augmented covariance method(ACM (Augmented Covariance Matrix) with projection into the linear tangent space Tangent Space (TS). The identified set of features or spatial filters is simultaneously fed to the database-based classification block (4) and to the adaptive classification block (5). The database-based classification block is a pre-trained machine learning model on the data of conditionally healthy subjects. The adaptive classifier block (5) allows for additional training of the machine learning model used as the patient undergoes the rehabilitation procedure. Depending on the implementation, the following models can be used: linear discriminant analysis (LDA) or support vector machine (SVM). In addition to these methods, more complex algorithms can be used, such as gradient boosting or deep neural networks, etc. Figure 2 shows a functional block diagram of the adaptive classifier, where the selected set of features and the results of the classification block based on the database (4) are simultaneously fed to the classification block (1a), which is based on one of the above machine learning models. The outputs of the classification block (1a) are connected to two outputs of the adaptive classification block (5) and to the input of the storage block (2a). The other two inputs of the storage block (2a) are connected to two inputs of the adaptive classification block (5). Data is recorded and stored in the read-only memory (ROM) of the platform. The output of the storage block (2a) is connected to the input of the classifier adaptation block (3a), which is connected to one of the inputs of the classification block (1a), and implements the adaptation procedure of the classification block (1a) based on the current data, taking into account the previous classification result and the stage of the rehabilitation procedure.

The outputs of blocks 4, 5 are connected to the input of the classifier selection block (6) (Fig. 1). The block analyzes the classification accuracy and connects the output that corresponds to the higher accuracy value to the input of the rehabilitation scenario implementation block (7), one of the outputs of which is connected to the input of the adaptive classification block (5). This block allows you to launch an individually selected rehabilitation program based on both a rehabilitation scenario adapted by the doctor and one of the previously implemented scenarios. Depending on the selected rehabilitation scenario, the system launches feedback loops: a visual stimulation loop block (8), a vibrotactile stimulation loop block (9), and a transcranial magnetic stimulation loop block (10). Note that the vibrotactile and visual stimulation loop blocks can be used to create sensory interference, thereby increasing the complexity of the rehabilitation procedure task.

Let us consider an example of using the device for rehabilitation of a patient with impaired cognitive or motor functions of the upper limbs. Based on the results of an in-person examination of the patient, a medical specialist can prescribe a COURSE of one or more rehabilitation PROCEDURES for this patient. A COURSE can be selected from those previously implemented in the system. The duration of a COURSE is up to 30 days. There are from 10 to 20 PROCEDURES in a COURSE. The duration of a PROCEDURE is no more than 40 minutes. No more than one PROCEDURE can be prescribed per day. Pauses between procedures can be up to 3 days. The PROCEDURE parameters determine the type and location of at least 2 electrodes on the surface of the patient's skull according to the international 10-20 electrode placement scheme for recording an electroencephalogram. These electrodes must be installed in the required positions properly immediately before the procedure. The input EEG signal undergoes a preliminary processing procedure in block 2 (Fig. 1), where filtering in the range of 1-40 Hz and removal of unwanted artifacts are performed. The frequency range was chosen as optimal, since it includes all the main rhythms of brain activity (delta 1-4 Hz, theta 4-8 Hz, alpha 8-13 Hz, beta 13-30 Hz, lower gamma range 30-40 Hz), and at the same time cuts off network interference of 50 Hz, low-frequency drift and high-frequency noise. It should be noted that reducing the lower limit of the frequency range to less than 1 Hz can lead to the manifestation of low-frequency components of the signal, including drift and artifacts of slow potentials, which will significantly distort the spectral characteristics of the recorded bioelectrical activity of the brain and reduce the reliability of the subsequent analysis. Expanding the upper limit above 40 Hz will entail the manifestation of artifacts of a myographic nature and electromagnetic interference from the measuring equipment, which will critically worsen the signal-to-noise ratio and make it impossible to correctly isolate informative EEG features. Then the purified signals are sent to the block of significant features extraction (3), based on the CSP spatial filter method. After that, the extracted features are sent to classification blocks 4 and 5, based on the LDA linear discriminant analysis models. In the process of passing the rehabilitation scenario, the accumulation and storage of data on the electrical activity of the patient's brain occurs, on the basis of which, simultaneously with the accumulation, the adaptation of the classifier to the individual patterns of electrical activity of the patient's brain is implemented. After the accumulation of the required amount of individual patient data, the classifier selection block (6) compares the classification results (blocks 4 and 5) of the state of the mental task performance, as a result of which the best classifier is selected. This approach allows monitoring the accuracy of classification, which ensures high efficiency of rehabilitation. After selecting the classifier, the results are sent to the rehabilitation scenario implementation block (7), which monitors the implementation of the rehabilitation PROCEDURE and launches the necessary feedback loops.

The PROCEDURE may consist of two STAGES: FAMILIARIZATION and REHABILITATION. The need for the FAMILIARIZATION stage is determined by the doctor during an in-person diagnosis of the patient. The FAMILIARIZATION stage within the rehabilitation PROCEDURE consists of demonstrating to the patient on a video monitor an animated material explaining the course of the REHABILITATION stage and, if necessary, accompanied by explanatory comments from the doctor. This stage is necessary to familiarize the patient with the rehabilitation methodology. If the patient is already familiar with the procedure and fully understands all the key points of the rehabilitation course, then this stage is excluded from the PROCEDURE.

The REHABILITATION stage within the PROCEDURE is basic and is aimed at stimulating neuroplasticity and restoring neural pathways through the use of multimodal biofeedback. This stage is based on the method of imagining the movement of the upper limbs with subsequent detection of the execution of the target action and launching the positive feedback circuit. The rehabilitation program includes two main SCENARIOS for the restoration of both motor and cognitive impairments.

The cognitive impairment rehabilitation SCENARIO is aimed at improving attention, executive functions, information processing speed and cognitive control. The feedback loop is defined by three modalities: visual only, vibrotactile only, and both visual and vibrotactile. The combination of feedback types helps to minimize the effect of the nervous system “habituation” to the same type of stimulation, adding variety during the rehabilitation procedure, and improve the quality of neural pathway restoration. The patient uses both upper limbs during rehabilitation. With high task performance efficiency, it is possible to increase the complexity of the rehabilitation SCENARIO by adding sensory interference, for example, in the form of random stimulation of a non-target limb by a vibrotactile sensor. In this case, the patient needs to suppress sensory noise and perform a movement with the target limb. In this case, the noise level allows you to adjust the complexity of the rehabilitation SCENARIO and adapt to the patient's results.

The SCENARIO for rehabilitation of motor disorders is aimed at training motor activity and restoring motor functions. The feedback loop for the SCENARIO for rehabilitation of motor disorders is determined by both vibrotactile and visual stimulation. During the rehabilitation process, the patient uses only one pre-selected target limb.

The REHABILITATION stage is determined by the following steps:

1. Diagnostics and selection of the target limb.

2. Movement imagination (one or two target limbs depending on the selected SCENARIO).

3. Stimulation of the imagination of movement of the target limb through biofeedback.

4. Repeat points 2 and 3 20 to 50 times (optional).

5. Outputting results to a report file.

Before rehabilitation, the patient undergoes a comprehensive diagnosis to identify problem areas and assess the functional state of the limb(s). During this stage, the diagnostician determines the degree and nature of both cognitive and motor impairments of the patient. The criteria for selecting the target limb are determined based on the severity of the impairment. In the case of severe impairments, the doctor conducts an objective assessment of the degree of damage and determines the target limb. In the case of subclinical impairments, the patient determines the target limb based on their own subjective sensations.

After diagnostics and selection of the target limb, the patient proceeds to the REHABILITATION stage. Within this stage, the doctor prescribes a rehabilitation SCENARIO, where the patient must imagine a movement with the limb(s) selected at the diagnostic stage. The imagination process begins with a trigger event, indicating a command to begin the imagination. The trigger event is a visual stimulus "cross" shown on the video monitor. In case of detection within 10 seconds after successful imagination, the positive biofeedback circuit is launched, and the system produces stimulation using one of three possible modalities (depending on the selected SCENARIO), selected randomly. If the patient fails to successfully imagine the movement, the system automatically proceeds to the next task in the rehabilitation scenario. The stimulation modalities are selected randomly, but are balanced for the entire rehabilitation PROCEDURE, i.e. the number of stimulations of each type is equal to each other.

Based on the results of the first rehabilitation procedure, namely, if it is of low efficiency, a transcranial magnetic stimulation procedure is prescribed. One of the stimulation options is to focus a magnetic beam on the left dorsolateral prefrontal cortex to ensure the transition from the patient's internally focused mental state to a state focused on the cognitive task and to improve the processing of sensory information. Note that if the rehabilitation scenario is highly effective, this procedure may not be prescribed. Conversely, if there are pronounced cognitive or motor disorders, this procedure is performed immediately before the first rehabilitation procedure. The brain stimulation procedure is determined by the following parameters:

1. Stimulation area - left side of the dorsolateral prefrontal cortex.

2. Coil - focal, figure-eight type, with an outer diameter of 7 cm for each wing.

3. The coil is positioned tangentially above the head, the handle is oriented posterolaterally at an angle of 45 relative to the sagittal plane.

4. Duration of stimulation - 6 minutes.

5. Stimulation frequency - 5 Hz.

6. Number of pulses - 1800.

7. Stimulation intensity - 90% of the motor threshold.

As a result of the rehabilitation PROCEDURE, a report is generated in pdf format. The report may reflect such characteristics as: the number of successful imaginations, the duration of the alpha rhythm drop, the reaction time (the time from the moment of the command to start the imagination until the actual drop of the alpha rhythm), the level of the midfrontal theta rhythm, as well as the dynamics of all indicators. This allows for objective monitoring of the rehabilitation results.

Thus, the proposed invention allows for effective neurorehabilitation of patients with cognitive and motor disorders using a brain-computer interface and multimodal feedback adapted to the individual neural patterns of the patient. The system provides a high degree of flexibility and personalization of the rehabilitation process due to the possibility of additional training of the machine learning model on the data of a specific patient and the use of various types of stimulation, which increases the effectiveness of the restoration of cognitive and motor functions.

Claims (1)

A neurorehabilitation system based on a brain-computer interface, comprising a brain electrical activity recording unit including at least two sensors for recording electroencephalographic signals, a data pre-processing unit connected to the output of the brain electrical activity recording unit and configured to filter signals in the range from 1 to 40 Hz and remove unwanted artifacts in real time, a significant feature detection unit connected to the output of the data pre-processing unit and designed to isolate key characteristics of the brain electrical activity, a database-based classification unit configured to at least binary classify states based on a pre-trained machine learning model, connected to the output of the significant feature detection unit, an adaptive classification unit connected to the output of the significant feature detection unit and the output of the database-based classification unit and implemented with the capability of at least binary classification and additional training of the model on the patient's individual data, a classifier selection unit connected to the output of the adaptive classification unit and the output of the database-based classification unit, configured to compare and select a classifier, a unit implementation of a rehabilitation scenario, connected to the output of the classifier selection unit and configured to launch personalized rehabilitation procedures, wherein the rehabilitation scenario implementation unit is connected to the input of the visual stimulation circuit unit, the input of the vibrotactile stimulation circuit unit and the input of the transcranial magnetic stimulation circuit unit to provide multimodal biological feedback, wherein the adaptive classification unit comprises a classification unit configured to classify based on the selected significant features from EEG data recorded during the patient's rehabilitation procedure and to take into account the results of the classification unit based on a database, a storage unit configured to store previous classification results, selected significant features and time series of EEG data, a classifier adaptation unit configured to further train and optimize the parameters of the classifier model based on the data recorded during the rehabilitation procedure, wherein the outputs of the classification unit are connected to two outputs of the adaptive classification unit and to the input of the storage unit, the other two inputs of the storage unit are connected to two inputs of the adaptive classification block, the output of the storage block is connected to the input of the classifier adaptation block, which is connected to one of the inputs of the classification block and is designed with the possibility of implementing the adaptation procedure of the classification block based on current data, taking into account the previous classification result and the stage of the rehabilitation procedure.