WO2023113742A1

WO2023113742A1 - Movement analysis and patient follow-up system in multiple sclerosis patients

Info

Publication number: WO2023113742A1
Application number: PCT/TR2022/051457
Authority: WO
Inventors: Murat TERZI; Sema Gul TURK
Original assignee: Ondokuz Mayis Universitesi
Current assignee: Ondokuz Mayis Universitesi
Priority date: 2021-12-15
Filing date: 2022-12-08
Publication date: 2023-06-22
Anticipated expiration: 2024-06-15

Abstract

The invention relates to a detection and follow-up system that enables early diagnosis and treatment in patients with multiple sclerosis (MS). In particular, the invention relates to a system that provides movement analysis and patient follow-up, enabling the treatment process to be effective after the diagnosis of the patients and providing the appropriate individual treatment, in the autoimmune disease known as multiple sclerosis that affects the central nervous system and progresses with attacks.

Description

MOVEMENT ANALYSIS AND PATIENT FOLLOW-UP SYSTEM IN MULTIPLE

SCLEROSIS PATIENTS

Technical field of the invention

State of the Art

Multiple Sclerosis (MS) is a chronic nervous system disease that shows its effect in the central nervous system and manifests itself with attacks. Attacks can be seen at different times and frequencies in the natural course of the disease. Attacks may occur with complaints such as loss of vision, imbalance, arm and leg weakness. Attack times are different for each MS patient. In the natural course of the disease, clinical worsening with an increase in complaints that exist independently of the attacks can also be seen. In studies on the early diagnosis and treatment of MS disease, it is of great importance to follow the course of the disease. In the current system, MS (Multiple sclerosis) patients are followed up with face-to-face interviews or with technological products such as telephone. In this case, it is not possible to monitor whether the patients' examinations can be performed and whether they can use their treatments effectively.

In the state of the art, the data of the patients are followed by observation or various camera recordings. The systems in which the examinations to be made and the medical conditions to be followed are processed can be processed when the patient applies to the health personnel, and these medical conditions can only be seen by the health professional. In the invention that is the subject of the patent no "US2014153794A1” in the state of the art, a system has been developed for the use of motion analysis in the field of medicine. It can be used in many diseases such as Parkinson's, ALS, Cerebral Palsy. The selected movements of the patient are recorded with a camera and the movement data is analysed by the pattern recognition algorithm. The system records images from a single angle.

In the invention that is the subject of the patent no "US2009040041 A1” in the state of the art, the physical locations of Alzheimer's patients are tracked, and the locations of the patients are marked on the map. Missing patients can be found by following them in this way. The signals sent from a wristband worn on the patients are recorded on the designated computer, and the position of the patient can be tracked in case of loss. It is not a system where health data such as fever, pulse and heartbeat of patients can be obtained.

As a result, due to the negativities described above and the inadequacy of the existing solutions on the subject, it was necessary to make an improvement in the relevant technical field.

Brief Description and Aims of the Invention

The most important aim of the invention is to provide a system in which physical and cognitive disability can be prevented by early detection of the developments that may occur in the natural course of the disease and to increase the health service in the follow-up of MS (Multiple Sclerosis) patients by following the patients with a live system on the online platform. By this way, it is aimed to increase the quality of life of MS patients.

Another aim of the invention is to enable the patient follow-up to progress properly by activating the warning system when the patients do not have the required examinations. By this way, it is ensured that the patient can be intervened in the early period regarding the examinations.

Another aim of the invention is to develop a system through which the patients and the health personnel who uses the application and authorised by the health professional can access the patient information. Another aim of the invention is to provide a system in which the progression of MS disease and the findings of patient can be followed by performing motion analysis with the data received from the image recorder. By this way, abnormalities in the patient can be detected early with movement analysis.

Another aim of the invention is to develop a system in which the patient's data can be transferred to the phone application by using the sensor kit and these data can be shared with the healthcare professionals. By this way, specialists themselves can examine the anomalies detected by the server.

Another aim of the invention is to enable movement analysis, patient follow-up and disease findings to be carried out on a single system.

Another aim of the invention is to provide a system that can be fixed on patients. To this end, the sensor kit can be in the form of a wristband.

Another aim of the invention is to detect the acceleration and angle changes of patient movements in three axes (x, y, z). To this end, there is a gyroscope (gyro) sensor in the system.

Another aim of the invention is to develop an artificial intelligence-based model with machine learning by the virtue of all these data, to determine the correct and early diagnosis of MS patients, to determine their prognostic features, and to provide appropriate individual treatments to patients.

Description of the Figures

FIGURE-1 is the drawing showing the motion analysis and patient follow-up system developed in the system that is the subject of the invention.

Definition of Elements/Parts Composing the Invention

In order to better explain the detection and follow-up system that provides early diagnosis and effective treatment in Multiple Sclerosis (MS) patients developed with this invention, the parts and elements in the figures are numbered and the corresponding number of each number is given below: 100. Sensor kit

200. Image recorder

300. Telephone

301 . Tablet

400. Server

Detailed Description of the Invention

The detection system that is the subject of the invention includes a sensor kit (100), an image recorder (200), a hand terminal (phone (300) and tablet (301 )) and a server (400). The sensor kit (100) is attached to the wrists, ankles and torso of MS patients and enables the patient's movement information to be obtained through sensors with properties to measure acceleration, heart rate and body temperature. The sensor kit (100) may be in the form of a wristband so that it can be fixed on the patients.

The results of procedures such as radiological imaging and blood analysis of the patient are recorded in the hospitals where these were performed. The patient's exercise capacity is increased by creating movement data of MS (Multiple Sclerosis) patients with a sensor kit (100) software system that records the movement repertoire of patients when they come from home or to the hospital. At the same time, the movements of the patient are analysed and recorded with the image recorder (200). By means of the image recorder (200), the motion images of the patient are recorded. The image recorder (200) is placed at six points on the top of the room where the patient is located, and recording is provided. In this way, the patient can be recorded while performing the movements shown, and abnormality detection can be made. In the preferred embodiment of the invention, the patient's six-minute walk and number of steps are also recorded with the image recorder (200). With the data to be obtained during this period, sufficient preliminary information about the mobility and physical characteristics of the patients can be obtained. The validity and reliability test of the 6- m inute walk test in MS patients has been performed in previous studies.

Even if the patient is at home, all data of the patient continues to be transferred with the sensor kit (100). All data and examinations requested by doctors are followed through the mobile application. The mobile application is run on the sensor kit (100) and provides the image recorder (200) and sensor kit (100) data to be sent to the server. The application also enables experts to view images captured from the image recorder (200). By this way, specialists themselves can examine the anomalies detected by the server.

The application reports the abnormality information obtained from the server (400) to the patients and doctors. The application includes an interface where experts can watch videos systematically and inform patients. The interface enables all examinations and follow-ups of MS patients.

Animated motion videos were also created in order to reflect the movements of the patient on the screen and to detect motion abnormalities over a standard motion repertoire. By watching these videos, the patient is asked to make movements. The movements of the patient are recorded in three axes by means of the sensor kit (100).

Server (400) features cloud storage and cloud data processing. Cloud service is used for file storage and access, virtual machine support and machine learning tools and automation services. The collected videos are kept in the cloud provider. The cloud processing of the videos transferred to the cloud storage service on the server is carried out with the tool to be selected. These processes determine the movement of each limb and body in the image from the images obtained from the image recorder (200) and the displacement of the movement in unit time. The server (400) also detects three-axis acceleration and angle changes from the sensor kit (100) data. By means of the sensor kit (100), the movements of the received sensor data in three axes (x, y, z) can be determined numerically. Acceleration and rotation angles are recorded in these three axes. This information can be obtained by means of the gyroscope (gyro) sensor. Due to the sensor kit (100), the movement of the patient's limb can be obtained in three dimensions, so that the angle changes in the movements can be measured. It can also detect temperature change, which is one of the most important factors in MS disease. Feature vectors obtained from the image recorder (200) data taken simultaneously with the sensor kit (100) data with machine learning run in the cloud environment are given as vector input to the artificial intelligence classifier and as a result, the abnormality status of each limb and trunk and the general abnormality status are determined. By developing a diagnostic model based on movement analysis, skeletal extraction is performed, and in this way, which limb has the abnormality can be examined. Skeleton extraction processes are performed with ready-made libraries over patient videos. After the skeleton extraction is done, a data set for artificial intelligence is created with these data.

The server (400) determines the movement repertoire of multiple sclerosis disease by using the displacement, acceleration, and angle changes of the movement per unit time in the convolutional neural networks and determines in which region of the patient the abnormal movement is. These determinations can be made by motion analysis. With the data set created by the skeletal data extracted from the video images and the analysis of the videos, the abnormalities in the limbs of the patients can be noticed by teaching the artificial intelligence. Data is taken from various libraries to compare abnormal motion and normal motion data. In an embodiment of the invention, the server (400) uses extreme learning machine, support vector machine, random forest and naive bayes classifier machine learning and deep learning techniques. An artificial intelligence model for the progression of MS disease was developed using these techniques with the created data set.

The server (400) collects the results obtained by performing machine learning through the cloud storage property from the data of the image recorder (200) and the sensor kit (100). While the server (400) creates the motion repertoire of the multiple sclerosis patient with the sensor kit (100) and the image recorder (200), it also provides to transmit the patient's examinations and the doctor's follow-up system to the application via the mobile application.

By means of the sensor kit (100) and its software that can perform motion analysis in 3 axes, the patient is provided with the right exercise model at home or in the environment s/he is in. At the same time, with the developed system, it is ensured that the patient performs all examinations and doctor checks at the right time. Situations in which the patients do not come to their appointments are determined and the possibility of cancellation of the next appointment is evaluated, and if the underlying problems are created by the health professional and the health system (for example, if patients cannot find an appointment with their doctor, this will be tried to be prevented) the system will try to eliminate these situations and help the patient to make use of the health system.

In the invention, patients are followed up with a live system over the online platform. With this system, a warning system is activated when patients do not have the required tests done. By this way, early intervention regarding the examinations is provided to the patient.

Claims

CLAIMS A detection and follow-up system that enables early diagnosis and treatment in patients with multiple sclerosis (MS), comprising

• A sensor kit (100) that comprises sensors with acceleration, heart rate and body temperature measurement features which are attached to the wrists, ankles and body of multiple sclerosis (MS) patients, and enables the patient to receive information with these sensors, records the movement repertoire of the patients when they come from home or to the hospital with the sensors it contains,

• A video recording device (200), which is placed on the top of the room where the patient is located, takes a recording so that the movements of the patient can be analysed and enables abnormal motion detection with the help of the records it receives (200),

• A mobile application that enables the data taken from the sensor kit (100) and the image recorder (200) to be displayed for the examinations requested by the user or the doctor, and transmits these data or examinations to the server (400) to report the abnormalities to the patients and doctors to be followed up, and

• The server (400) which detects the movement of each limb and body in the image and the displacement of the movement in unit time from the images obtained from the image recorder (200) and the three-axis acceleration and angle changes with the data obtained from the sensor kit (100) and detects in which region of the patient the abnormal movement is. A detection and follow-up system that enables early diagnosis and treatment in patients with multiple sclerosis (MS) according to Claim 1 , comprising the sensor kit (100) produced in wristband form for easy fixation on patients. A detection and follow-up system that enables early diagnosis and treatment in patients with multiple sclerosis (MS) according to Claim 1 , comprising the image recorder (200) that allows recording the patient's walking distance and number of steps.

8

4. A detection and follow-up system that enables early diagnosis and treatment in patients with multiple sclerosis (MS) according to Claim 1 , comprising the mobile application that allows specialists to monitor the data received from the image recorder (200).

5. A detection and follow-up system that enables early diagnosis and treatment in patients with multiple sclerosis (MS) according to Claim 1 , comprising the server (400) that detects situations where the patients do not come to their appointments and evaluates the possibility of the next appointment cancellation and prevents the problems, if they are caused by the health professional and the health system.

6. A detection and follow-up system that enables early diagnosis and treatment in patients with multiple sclerosis (MS) according to Claim 1 , comprising the warning system, which is activated when patients do not perform the examinations requested by the doctor.

7. A detection and follow-up system that enables early diagnosis and treatment in patients with multiple sclerosis (MS) according to Claim 1 , comprising the sensor kit (100) that detects the change in the patient's body temperature.

8. A detection and follow-up system that enables early diagnosis and treatment in patients with multiple sclerosis (MS) according to Claim 1 and Claim 7, comprising the gyroscope sensor that allows recording of acceleration and rotation angles in three axes (x, y, z).

9