KR20130010207A - System for analyze the user's health and stress - Google Patents

Abstract

The present invention relates to a watch-type health state analysis system by acquiring an unconstrained involuntary biosignal, and analyzes and determines a health state and a stress state of a user using pulse wave (PPG) and electrocardiogram (ECG) signals collected from a user. And display the same, and transmit the same to the image display unit to play the information, the image, and the music set according to the health and stress state of the user.
An object of the present invention is a sensor unit for measuring the ECG and pulse wave signals from the ECG sensor and pulse wave sensor; An analog / digital conversion unit for converting the biometric information detected from the sensor unit into a digital signal; Analyze the stress state and pulse rate of the user using ECG data inputted through the analog / digital conversion unit, calculate the respiratory state, blood oxygen saturation (SpO2) and vascular elasticity using pulse wave data (PPG). A main processor configured to calculate blood pressure by using electrocardiogram data (ECG) and pulse wave data (PPG), and to determine and alert a dangerous state by comparing the calculated health state data with a reference value; An LCD display for displaying each data calculated by the main processor; A buzzer, a vibration motor, and an LED display unit for alerting the user as the health state of the user is determined by the main processor as a dangerous state; And a Zigbee communication unit for wirelessly transmitting the user health state information calculated by the main processor to an external device.

Description

System for Analyze the User's Health and Stress}

The present invention relates to a system for analyzing and displaying a human health state, and in particular, to collect a user's biosignal in an unconstrained insensitive state, analyze a user's current health condition from the collected biosignal, and analyze the health state. The present invention relates to a watch type health state analysis system by acquiring an unconstrained involuntary biosignal for displaying health information or an image corresponding to a health state of a user through a display unit.

U-Healthcare service, which has been converged with the development of IT technology and medical information technology, utilizes various devices around the living area, so it is not limited to one-time or medical treatment in hospitals. Patients can be monitored remotely anytime, anywhere in real life without the limitation of space.

Furthermore, the development of the convergence technology of ubiquitous healthcare and robot technology that can interact with robots can recognize emotions using information such as human voice, video, bio signals or gestures, and realize communication with robots. The development of a robot-based human emotional sympathy module is in progress.

A service apparatus and method for a robot that recognizes and responds to human emotion recognition in the prior art are disclosed in Korean Patent Publication No. 10-2010-0001928.

1 is a block diagram of a service apparatus based on emotion recognition according to the related art, and includes a data input unit 10 that receives voice and video data from a voice and video device, and recognizes a user's emotion from a received voice and video signal. An emotion recognition unit 20 to determine the service to be provided to the user based on the recognized user's emotion, and an output unit 40 to provide the service to the user based on the determined service. It consists of.

Here, the data input unit 10 receives audio and video data from the audio and video device. The data input unit 110 is composed of a voice and video input device such as a microphone or a camera.

The emotion recognition unit 20 recognizes a user's emotion from the received voice and image data. To this end, the emotion recognizer 20 includes a voice recognizer 22, an expression recognizer 24, and an emotion determiner 26.

The speech recognition unit 22 extracts features such as pitch statistic, loudness, number of sections, crossing rate, increasing rate, etc. from the speech signal. The speech recognizer 22 classifies emotion-specific patterns using Multi Layer Perception (MLP), which learns these extracted features by Backpropagation (BP) of artificial neural networks.

The facial expression recognition unit 24 performs image processing such as a skin tone accumulation algorithm on the input face image to extract a feature. In this process, the face region of the user moving from the background image should be detected. The facial expression recognition unit 24 measures the background at the time of capturing and acquires the detected background image according to the image difference from the input image. By subtracting the detected background image from the background image, only the newly input user's image is obtained. In this case, it should be noted that the input image may be changed by external light at the time of obtaining the image. This problem can be solved by re-shooting the image within a few seconds when the image is taken and receiving a continuous image to make the background image and the newly acquired image almost the same.

The emotion determination unit 26 determines the final user's emotion based on the recognition result obtained by the voice recognition unit 22 and / or the facial expression recognition unit 24. In determining emotion, a decision fusion method or a feature fusion method is used to determine a user's final emotional state.

The service determiner 30 determines a service to be provided to the user based on the recognized emotion of the user. First, the service determiner 30 first checks a service providing partner for confirming whether a person corresponds to a specific person in a database based on voice and video data.

If the service providing partner corresponds to a specific person in the database, the service determination unit 30 determines the type of service to be provided. In this case, the service determination unit 30 determines a service to be provided to the user according to each emotion classified by the emotion recognition unit 20. In contrast, if the service providing partner is not a specific person in the database, the service determination unit 30 determines not to provide the service.

The output unit 40 provides a service to a user based on the determined service type. For example, when the user's feelings are sad, the user speaks comforting words or actions, and when the user's feelings are joy, the user acts as a word of congratulations to make the user happy. That is, the output unit 40 provides various services to the user according to the emotional state of the user.

According to such a service apparatus based on emotion recognition according to the related art, it is possible to recognize a user's emotion through voice and video or complex data, and thereby provide a service corresponding to the user's emotion by the service robot and the service device. It becomes.

However, the service apparatus and method based on emotion recognition according to the prior art extracts only the voice and facial expression information of the user and recognizes and judges the emotion, and extracts and learns the voice feature and facial feature of the user from a database. The emotion is determined through the extracted user's face image and voice feature vectors. Since the face image and voice extract information through microphones and cameras, it is sensitive to noise depending on the surrounding environment, and thus cannot extract accurate data. There was this.

Therefore, the present invention analyzes and determines the health state and stress state of the user using pulse wave (PPG) and electrocardiogram (ECG) signals collected from the user in order to improve the problems of the prior art, and transmits them to the image display unit. It is an object of the present invention to provide a watch type health state analysis system by acquiring an unconstrained involuntary biosignal to reproduce information, video, and music set according to a user's health and stress state.

In order to achieve the object of the present invention, the watch type health state analysis system through the acquisition of an unconstrained involuntary biosignal is obtained by extracting biometric information of a user from a watch type biometric information acquisition device equipped with sensors for acquiring a plurality of biosignals. A watch type health state analysis system through the acquisition of an unconstrained involuntary biosignal to analyze a health state and a stress state of a user, comprising: a sensor unit measuring an electrocardiogram and a pulse wave signal from the ECG sensor and the pulse wave sensor; An analog / digital conversion unit for converting the biometric information detected from the sensor unit into a digital signal; Analyze the stress state and pulse rate of the user using ECG data inputted through the analog / digital conversion unit, and use the pulse wave data (PPG) to measure the respiratory state, blood oxygen saturation (SpO2) and vascular elasticity. A main processor configured to calculate a blood pressure using the ECG data and the pulse wave data PPG, and to compare the calculated health state data with a reference value to determine and alarm a dangerous state; An LCD display for displaying each data calculated by the main processor; A buzzer, a vibration motor, and an LED display unit for alerting the user as the health state of the user is determined by the main processor as a dangerous state; And a Zigbee communication unit for wirelessly transmitting the user health state information calculated by the main processor to an external device.

According to the present invention, the watch type health state analysis system through the acquisition of an unconstrained involuntary biosignal is a user's stress state, pulse rate, blood pressure, respiration, vascular elasticity with only the biosignal obtained from an electrocardiogram sensor and a pulse wave sensor provided as a watch type. There is an effect that can analyze and extract health status information, such as.

In addition, the watch type dry state analysis apparatus according to the present invention is configured as a watch type, when the user wears on the wrist, such as a watch has an effect that can analyze the health state and stress state of the user in the unconscious state have.

In addition, the present invention compares the calculated health state data with a reference value to determine whether there is a risk, and if it is determined that the risk state has an effect of alerting the user with a buzzer, vibration, LED, and the like.

In addition, the present invention wirelessly receives the health status information from the location type health status analysis device, and displays the health information suitable for the user's health status, or play the appropriate picture, video or music according to the stress state, The provided user has the effect of appropriately acquiring necessary information according to his or her health condition or relieving stress.

1 is a block diagram of a service apparatus based on emotion recognition according to the prior art,
2 is a block diagram of a watch type health state analysis system through the acquisition of an unconstrained insensitive biosignal according to an embodiment of the present invention;
3 is a flowchart illustrating an operation of the watch type health state analysis system by acquiring an unconstrained insensitive biosignal according to an embodiment of the present invention.

With reference to the accompanying drawings, a detailed configuration and operation of the watch-type health state analysis system by acquiring unrestrained bio-signal biosignal according to an embodiment of the present invention will be described in detail as follows.

2 is a block diagram of a watch type health state analysis system through the acquisition of an unconstrained involuntary biosignal signal according to an embodiment of the present invention. Sensor including an electrocardiogram sensor 111 and pulse wave sensor 112 for measuring, an accelerometer 113 for measuring the amount of movement of the user to correct the electrocardiogram and pulse wave modified by the user's movement And an analog / digital converter 120 for converting the user's biometric information obtained from the sensor unit into a digital signal, and electrocardiogram data (ECG) input through the analog / digital converter 120. Analyze the stress state and pulse rate of the user, calculate the respiratory state, blood oxygen saturation (SpO2) and vascular elasticity by using the pulse wave data (PPG), the electrocardiogram data (ECG) and Blood pressure is calculated using pulse wave data (PPG), and the main processor 130 for determining and alarming a dangerous state by comparing the calculated health state data with a reference value, and the respective data calculated by the main processor LCD display 142 to display, the buzzer 151, the vibration motor 152 and the LED display 143 to alert the user as the health state of the user is determined by the main processor 130, and Switching unit 144 for user selection and operation control, a memory 141 for storing the measured biometric data and calculation data, risk determination reference data, and LCD display for displaying the user selection, calculation data and operation process ( 142, a Zigbee communication unit 153 for wirelessly transmitting the user health state information calculated by the main processor 130 to an external device, and a power supply unit 101 for supplying driving power to each unit. .

On the other hand, the health state response device 200 for wirelessly receiving the health state information transmitted through the ZigBee communication unit 153 to play the necessary health information, images and music according to the health state of the health state analysis device 100 The ZigBee communication unit 201 receiving the health state information transmitted from the ZigBee communication unit 153 and the necessary health information according to the health state information received through the ZigBee communication unit 201 through an image, and according to the stress state Health state response unit 210 for reproducing the image and music set to relieve stress of the user, the image reproducing unit 221 and the music reproducing unit for reproducing the image and music under the control of the health stimulus response unit 210 223, a health information storage unit 211 storing health information to be provided to the user through the health state reaction unit 210, and a picture to be provided to the user through the health state reaction unit 210, and a video. Or it consists of the image storage section 212 for storing music.

Referring to the operation of the present invention configured as described above in detail with reference to Figures 2 and 3 as follows.

Figure 2 is a block diagram of a watch type health status analysis system through the acquisition of unrestrained involuntary bio-signal according to an embodiment of the present invention.

First, the health state analysis device 100 is configured to be worn on the user's body part, in particular in the present invention is configured as a watch type to be worn on the user's wrist, ECG signal, pulse wave signal (PPG) from the wrist ) And the amount of movement change, and by receiving and analyzing the obtained sensor values, and analyzes and determines the current health state and stress state of the user and wirelessly transmits them to the health state reaction device 200.

The electrocardiogram sensor 111 extracts an electrocardiogram signal from a contact contacting the skin of a wrist of a user and outputs the electrocardiogram signal to the main processor 130 through the A / D converter 120.

The pulse wave sensor 112 irradiates infrared (IR) and red light (RED) to the radial artery located on the user's wrist, and detects the pulse wave signal (PPG) which is a change amount of BVP (Blood Volume Plus) by the reflected infrared ray A Through the / D conversion unit 120 is output to the main processor (130).

The accelerometer 113 is composed of a MEMS-type three-axis acceleration sensor, converts the degree of movement of the user into an electrical signal and outputs it to the main processor 130 through the A / D conversion unit 130.

The main processor 130 analyzes the health state of the user through the process as shown in FIG.

3 is a flowchart illustrating a process of analyzing a user's health state according to an exemplary embodiment of the present invention, wherein the ECG, pulse wave signal (PPG), and user motion signal inputted from the respective sensors in the main processor 130 are parameterized. The process of analyzing and determining the health state and stress state of the user in detail as follows.

Receives the user bio-signal and the motion signal obtained from the sensor unit 111, 112, 113 (S101), modified according to the movement of the user included in the input ECG signal and pulse wave signal (PPG) Each of the noises is removed. (S111) (S121)

That is, the main processor 130 removes the noise included in the ECG signal and the pulse wave signal PPG through the noise removing process S111 and S121, and the low pass through the preprocessing process S112 and S122. High- and low-frequency components are removed through a low-pass filter and a high-pass filter, and an envelope waveform is generated by a square root function.

In the noise removing process S111, when a user moves during the measurement of the ECG signal and the pulse wave signal PPG, noise according to the movement is generated. At this time, it is impossible to obtain an accurate envelope due to the generated noise, and the noise detected in the ECG signal and the pulse wave signal PPG including the noise should be removed according to the amount of change in the user's movement.

In this case, the noise includes a DC component in each ECG and pulse wave signal according to a user's movement. The DC value is compensated for the measured ECG and pulse wave signal to obtain an accurate ECG and pulse wave signal.

The user's motion determines whether the user is stationary or in motion according to the signal obtained through the accelerometer 114, and detects more accurate envelopes by removing noise of ECG and pulse wave signals according to the amount of change in the motion. do.

The envelope in which the ECG signal ECG is detected in the preprocessing step S112 is used to calculate an RRI from the envelope using an envelope detection algorithm. The time interval becomes a data point between R-peak and R-peak.

In addition, the ratio of the sympathetic activity (LF: Low Frequency), the sympathetic activity (HF: High Frequency) through the frequency sequence analysis process (S113), the ratio of the sympathetic activity (LF) and para-sympathetic activity (HF) is Autonomic nerve balance (LF / HF), sympathetic activity (LF) is the area of the frequency domain between 0.05Hz and 0.15Hz, parasympathetic activity (HF) is the area of the frequency domain between 0.15Hz and 0.4Hz it means.

Time series analysis (S114) calculates the standard deviation (SDNN) and heart rate variability (HRV-index) parameters of the peak time interval (RRI).

The stress state of the user is analyzed using the calculated sympathetic activity (LF), parasympathetic activity (HF), standard deviation (SDNN), and heart rate variability (HRV-index) parameters.

The pulse rate is calculated from the peak time interval RRI calculated from the ECG signal ECG passing through the preprocessing step S112. (S116) (S117).

The pulse wave signal PPG extracted from the pulse wave sensor 112 removes the noise through the noise removing process S121 and then calculates the pulse wave signal PPG peak value from the envelope calculated through the preprocessing process S122. The pulse transit time (PTT) is calculated through the R peak and the PPG peak of the ECG signal, and the user's current blood pressure is calculated from the PTT (S131) (S132).

In addition, due to the relationship between the respiration and blood flow changes, the respiration signal is extracted through the pulse wave signal (PPG), which is basically distinguished by the difference in the frequency characteristics of the respiration signal and the time signal. That is, the respiratory signal is calculated by analyzing the frequency component of the pulse wave signal PPG through the frequency sequence analysis S123.

Among the components extracted by the pulse wave sensor 112, the oxygen saturation level SpO 2 is calculated from each signal calculated by red light RED and infrared light IR.

Analyze the change in blood flow in the PPG waveform calculated through the preprocessing step (S122), extract the existing frequency of the pulse, and calculate the vascular elasticity through the second derivative process (S126) (S127).

The main processor 130 calculates the stress state and the dry state parameters (pulse rate, blood pressure, respiration, vascular elasticity) calculated through each process and stores the health state information of the user in the memory 141 by On the LCD display unit 142.

In addition, the main processor 130 determines whether the calculated health state parameters (pulse rate, blood pressure, respiration, vascular elasticity) is a dangerous state of the user by comparing the reference parameters stored in the memory 141.

If it is determined that the dangerous state, the alarm through the buzzer 151, the vibration motor 152, the LED display unit 143, and the like.

On the other hand, the main processor 130 transmits the user's current health state and stress state through the ZigBee communication unit 153 to the health state reaction device 200.

The health state response device 200 performs an operation set according to the transmitted health state and the stress state of the user, and the current state of the user in the health information storage unit 211 is controlled by the health state reaction unit 210. The necessary health information corresponding to the health state is displayed on the LCD screen through the image reproducing unit 221, or.

If the stress state of the user is high, the user accesses the photo, video, and music stored in the image storage unit 212 to play the image and playback through the image playback unit 221 or music playback unit 223 screen and speaker (SPK) It is output through, which helps the user to relieve stress while watching video and music.

For example, the expression change according to the calculated stress state of the user is displayed on the LCD screen, but with a numerical value. Or the face image of the stressed expression is displayed on the screen, the blood saturation degree is displayed on the screen and the face image indicating the color change, and the vascular elasticity is displayed on the face image of the child, young man, and the old man who meet the value. Print out the screen and induce interest.

Emotional reaction system and control method according to the emotional state analysis of the user according to the present invention described above is not limited to the above embodiment, it can be variously changed according to the configuration of the emotional reaction device, active user emotion, It can be variously modified according to the emotional state other than comfort, relaxing, anxiety state, any person having ordinary knowledge in the field of the present invention without departing from the gist of the present invention claimed in the claims below. The technical spirit is to the extent that various modifications can be made.

100: emotion analysis device 101: power supply
111: pulse wave sensor 112: skin electrical resistance sensor
113: skin temperature sensor 114: accelerometer
120: A / D conversion unit 130: main processor
141: memory 142: LCD display
143: LED display unit 144: switching unit
151: buzzer 152: vibration motor
153: Zigbee transmitter
200: Emotional response device 201: Zigbee receiving unit
210: emotional response control unit 221: operation driver
222: motion database 223: music player
224: sound source database

Claims (6)

Watch type through the acquisition of the unbounded bio-signal that analyzes the user's health state and stress state by analyzing the extracted user's bio-information obtained from the watch-type bio-information acquisition device equipped with sensors for acquiring a plurality of bio-signals In the health state analysis system,
A sensor unit for measuring electrocardiogram and pulse wave signals from the electrocardiogram sensor and pulse wave sensor;
An analog / digital conversion unit for converting the biometric information detected from the sensor unit into a digital signal; And
Analyze the stress state and pulse rate of the user using ECG data inputted through the analog / digital conversion unit, and use the pulse wave data (PPG) to measure the respiratory state, blood oxygen saturation (SpO2) and vascular elasticity. A main processor configured to calculate a blood pressure using the ECG data and the pulse wave data PPG, and to compare the calculated health state data with a reference value to determine and alarm a dangerous state;
An LCD display for displaying each data calculated by the main processor;
A buzzer, a vibration motor, and an LED display unit for alerting the user as the health state of the user is determined by the main processor as a dangerous state; And
And a ZigBee communication unit for wirelessly transmitting the user health state information calculated by the main processor to an external device. The method of claim 1,
The stress analyzed by the main processor calculates autonomic neural balance (LF / HF) parameters through the frequency sequence analysis of the ECG signal, and the standard deviation (SDNN) of the peak time interval (RRI) through the time series analysis process. Watch type health status analysis system through the acquisition of unrestrained involuntary bio-signal, characterized in calculating the heart rate variability (HRV-index) parameter.
The method of claim 1,
In addition to the sensor unit to correct the electrocardiogram and pulse wave deformed by the user's movement, further comprising an accelerometer (Accelerometer) for measuring the change in the user's movement characterized in that the acquisition of the non-involunted insensitive bio-signal Watch type health status analysis system through.
The method of claim 1,
Unrestrained indetermination, characterized in that it further comprises a health state response device for wirelessly receiving health state information transmitted through the Zigbee communication unit of the health state analysis device to play the necessary health information, images and music according to the state of health. Watch type health status analysis system through biometric signal acquisition.
The method of claim 3, wherein
The health state reaction apparatus includes a Zigbee communication unit for receiving health state information transmitted from the Zigbee communication unit of the health state analysis device;
A health state response unit providing necessary health information according to the health state information received through the ZigBee communication unit through an image, and playing the image and music set for the user's stress relief according to the stress state;
An image reproducing unit and a music reproducing unit for reproducing images and music under the control of the health intensive reaction unit;
A health information storage unit for storing health information to be provided to the user through the health state reaction unit; And
And a video storage unit for storing a photo, video or music to be provided to the user through the health state response unit.
The method of claim 3, wherein
The dry state reaction apparatus displays the expression change according to the stress state of the user on the LCD screen, and outputs the face image of the stressed expression along with the numerical value, and the blood oxygen saturation is displayed on the face image indicating the change in color with the numerical value. The watch type health status analysis system by acquiring an unconstrained involuntary biosignal, characterized by outputting, and outputting an image of a face of a child, a young man, an old man with a blood vessel elasticity corresponding to the numerical value.

Images