[go: up one dir, main page]

WO2021101073A1 - Appareil d'acquisition de données biométriques et procédé associé - Google Patents

Appareil d'acquisition de données biométriques et procédé associé Download PDF

Info

Publication number
WO2021101073A1
WO2021101073A1 PCT/KR2020/014098 KR2020014098W WO2021101073A1 WO 2021101073 A1 WO2021101073 A1 WO 2021101073A1 KR 2020014098 W KR2020014098 W KR 2020014098W WO 2021101073 A1 WO2021101073 A1 WO 2021101073A1
Authority
WO
WIPO (PCT)
Prior art keywords
heart rate
electronic device
data
value
processor
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/KR2020/014098
Other languages
English (en)
Korean (ko)
Inventor
김민철
이종하
김찬일
한경완
허동혁
황석민
임연욱
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Samsung Electronics Co Ltd
Industry Academic Cooperation Foundation of Keimyung University
Original Assignee
Samsung Electronics Co Ltd
Industry Academic Cooperation Foundation of Keimyung University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Samsung Electronics Co Ltd, Industry Academic Cooperation Foundation of Keimyung University filed Critical Samsung Electronics Co Ltd
Publication of WO2021101073A1 publication Critical patent/WO2021101073A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/02—Detecting, measuring or recording for evaluating the cardiovascular system, e.g. pulse, heart rate, blood pressure or blood flow
    • A61B5/024—Measuring pulse rate or heart rate
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Measuring devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor or mobility of a limb
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Measuring devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor or mobility of a limb
    • A61B5/1126—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor or mobility of a limb using a particular sensing technique
    • A61B5/1128—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor or mobility of a limb using a particular sensing technique using image analysis
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7271—Specific aspects of physiological measurement analysis
    • A61B5/7278—Artificial waveform generation or derivation, e.g. synthesizing signals from measured signals
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/74—Details of notification to user or communication with user or patient; User input means
    • A61B5/742—Details of notification to user or communication with user or patient; User input means using visual displays
    • A61B5/7445—Display arrangements, e.g. multiple display units
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/74—Details of notification to user or communication with user or patient; User input means
    • A61B5/746—Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/74—Details of notification to user or communication with user or patient; User input means
    • A61B5/7465—Arrangements for interactive communication between patient and care services, e.g. by using a telephone network
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B24/00—Electric or electronic controls for exercising apparatus of preceding groups; Controlling or monitoring of exercises, sportive games, training or athletic performances
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B24/00—Electric or electronic controls for exercising apparatus of preceding groups; Controlling or monitoring of exercises, sportive games, training or athletic performances
    • A63B24/0075—Means for generating exercise programs or schemes, e.g. computerized virtual trainer, e.g. using expert databases
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B71/00—Games or sports accessories not covered in groups A63B1/00 - A63B69/00
    • A63B71/06—Indicating or scoring devices for games or players, or for other sports activities
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B71/00—Games or sports accessories not covered in groups A63B1/00 - A63B69/00
    • A63B71/06—Indicating or scoring devices for games or players, or for other sports activities
    • A63B71/0619—Displays, user interfaces and indicating devices, specially adapted for sport equipment, e.g. display mounted on treadmills
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B2230/00—Measuring physiological parameters of the user
    • A63B2230/04—Measuring physiological parameters of the user heartbeat characteristics, e.g. ECG, blood pressure modulations

Definitions

  • Various embodiments relate to a method of measuring a biosignal and an apparatus for obtaining biometric data using the same.
  • vital signs refer to biological signals that show elements for maintaining life in the form of signals.
  • Several international medical standards define heart rate (heart rate), respiratory rate, blood pressure, and body temperature as essential vital signs. Technologies that utilize medical big data including information related to the human body such as vital signs have been proposed.
  • Heart rate refers to the heart rate per unit time.
  • the heart rate value can be expressed as the number of pulses per minute.
  • the heart rate changes according to the body's need for oxygen absorption or carbon dioxide excretion caused by physical activity.
  • the heart rate may be measured at a position in the body where the pulse rate is sensed. That is, the heart rate may be measured at a location where an artery through which blood flows is located by the heart rate.
  • the heart rate may be measured in the neck where the carotid artery is located, the groin where the high artery is located, or the wrist.
  • ECG electrocardiogram
  • Continuous heart rate monitoring can be used to diagnose and treat disease.
  • photoplethysmograph may be used to measure heart rate.
  • Biometric information such as blood flow rate, blood oxygen saturation, or heart rate may be obtained by using the optical volume pulse wave signal.
  • the photovoltaic pulse wave may be detected using a photoelectric conversion device that converts light into electric charge.
  • An optical sensor including a photodiode may be used as an optical sensor for detecting a volumetric pulse wave. When light reaches the photoelectric conversion device through the lens, the light is converted into electric charges. Light in a designated band is irradiated to the body, and a light volume pulse wave can be obtained based on a signal that detects light reflected from the body or transmitted through the body.
  • a Doppler radar radiates a designated frequency signal and measures a reflected signal according to the movement of an organ.
  • the digital signal processor can calculate the user's heart rate and respiration rate from the high-frequency signal using digital signal processing that performs frequency domain analysis.
  • Cardiovascular diseases such as tachycardia among arrhythmia, have a high frequency of occurrence of a specific heartbeat during exercise.
  • the user attaches contact equipment to the body during exercise load test and performs a test, or the user attaches the equipment to the body for a long period of time (e.g., 3 days) in daily life. You need to measure your heart rate. In this case, it causes inconvenience to the user or makes daily life uncomfortable.
  • the electronic device can measure the user's heart rate from the change in pixel values for the same part of the face. have.
  • a feature point for recognizing a face a method of identifying two eyes in an image may be used.
  • the electronic device may recognize a region in which a face is photographed in an image based on the identified two eyes.
  • the electronic device can recognize the face and measure the heart rate normally only when the user is staring at the camera.
  • the electronic device may not be able to recognize the face because a correlation between the feature points cannot be obtained.
  • the user measures the heart rate while exercising, the user's head moves, so the facial recognition rate is lowered, and the normal heart rate is not measured, so that the normal heart rate is not provided to the user.
  • a method of measuring biometric information using a Doppler radar is not suitable as a method of measuring a heart rate for a moving body because it uses reflected waves according to movement of an organ.
  • An electronic device may include a memory and a processor connected to the memory.
  • the processor acquires image data captured using a camera, analyzes the image data to calculate a heart rate value, determines whether the heart rate value is normally acquired, and the heart rate value is not normally acquired.
  • it may be configured to determine a correction value for a section in which the heart rate value is not normally obtained, and to store biometric information data including the correction value in a memory.
  • the method of operating the electronic device includes an operation of acquiring image data captured using a camera, an operation of calculating a heart rate value by analyzing the image data, an operation of determining whether the heart rate value is normally obtained, and an operation of obtaining a heart rate value normally. If not, an operation of determining a correction value for a section in which the heart rate value is not normally obtained and an operation of generating biometric information data including the correction value may be included.
  • a computer-readable recording medium recording a program includes an operation of acquiring image data captured using a camera when executed, an operation of calculating a heart rate value by analyzing image data, and the heart rate value To perform an operation of determining whether or not the heart rate value is normally acquired, an operation of determining a correction value for a section in which the heart rate value is not normally obtained, and an operation of generating biometric information data including the correction value when the heart rate value is not normally obtained It may be a record of a program that does.
  • An electronic device may include a memory for storing heart rate data of an arrhythmia patient and a processor connected to the memory.
  • the processor acquires biometric information data including a heart rate value, and if the biometric information data does not contain a heart rate value or there is a section other than the normally calculated value, the value of the section is corrected to a correction value, and the corrected If the biometric information data and the heart rate data of the arrhythmia patient are compared, and the number of times the corrected biometric information data includes the abnormal pattern included in the heart rate data of the arrhythmia patient is greater than or equal to the threshold value, information on the abnormal heart rate pattern is It can be configured to output.
  • a method and apparatus capable of increasing a facial recognition rate for measuring a heart rate from an image may be provided.
  • a method and apparatus capable of providing heart rate information to a user based on information acquired from an image may be provided even when a user's face is not normally recognized from an image. Accordingly, even when the user moves, a method and apparatus capable of providing normal heart rate information to the user without attaching or wearing the device to the body can be provided.
  • a method and an apparatus capable of inducing a user to easily achieve an exercise target value by presenting an appropriate exercise intensity based on a user's bio-signal may be provided.
  • FIG. 1 is a block diagram of an electronic device in a network environment, according to various embodiments.
  • FIG. 2 is a block diagram illustrating a program according to various embodiments.
  • FIG. 3 is a diagram illustrating a configuration of an electronic device according to an exemplary embodiment.
  • FIG. 4 is a flowchart illustrating a process of generating biometric information data by an electronic device according to an exemplary embodiment.
  • FIG 5 illustrates an example of an electronic device according to an embodiment.
  • FIG. 6 illustrates an example of a system for acquiring biometric information data, according to an embodiment.
  • FIG. 7 illustrates another example of an electronic device for obtaining biometric information data, according to an exemplary embodiment.
  • FIG. 8 is a flowchart illustrating a process of calculating a heart rate value by an electronic device according to an exemplary embodiment.
  • FIG. 9 is a diagram conceptually illustrating a method of recognizing a face area for calculating a heart rate value by an electronic device, according to an exemplary embodiment.
  • FIG. 10 is a flowchart illustrating a process of determining whether a heart rate value is normally calculated by an electronic device, according to an exemplary embodiment.
  • FIG. 11 is a flowchart illustrating a process of determining a correction value when a heart rate value is not normally calculated by an electronic device, according to an exemplary embodiment.
  • FIG. 12 conceptually illustrates an operation of correcting a heart rate value by using candidate data by an electronic device according to an exemplary embodiment.
  • FIG. 13 is a flowchart illustrating a process of additionally correcting the corrected biometric information data when a normal heart rate value is calculated by the electronic device, according to an exemplary embodiment.
  • 15 is a flowchart illustrating a process in which an electronic device generates candidate data and biometric information data based on a feature point, according to an exemplary embodiment.
  • 16 illustrates an example of a screen displayed by an electronic device according to an embodiment.
  • 17 is a flowchart illustrating a process for an electronic device to provide biometric information data together with guide image content, according to an exemplary embodiment.
  • FIG. 18 is a flowchart illustrating a process of providing recommended exercise intensity information based on biometric information data by an electronic device according to an exemplary embodiment.
  • 19 is a flowchart illustrating a process of providing recommended exercise information based on biometric information data by an electronic device according to an exemplary embodiment.
  • 20 is a diagram illustrating an electronic device in a module unit according to an exemplary embodiment.
  • FIG. 1 is a block diagram of an electronic device 101 in a network environment 100 according to various embodiments.
  • the electronic device 101 communicates with the electronic device 102 through a first network 198 (for example, a short-range wireless communication network), or a second network 199 It is possible to communicate with the electronic device 104 or the server 108 through (eg, a long-distance wireless communication network).
  • the electronic device 101 may communicate with the electronic device 104 through the server 108.
  • the electronic device 101 includes a processor 120, a memory 130, an input device 150, an audio output device 155, a display device 160, an audio module 170, and a sensor module ( 176, interface 177, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196, or antenna module 197 ) Can be included.
  • a sensor module 176, interface 177, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196, or antenna module 197
  • at least one of these components may be omitted or one or more other components may be added to the electronic device 101.
  • some of these components may be implemented as one integrated circuit.
  • the sensor module 176 eg, a fingerprint sensor, an iris sensor, or an illuminance sensor
  • the display device 160 eg, a display.
  • the processor 120 for example, executes software (eg, a program 140) to implement at least one other component (eg, a hardware or software component) of the electronic device 101 connected to the processor 120. It can be controlled and can perform various data processing or operations. According to an embodiment, as at least a part of data processing or operation, the processor 120 may transfer commands or data received from other components (eg, the sensor module 176 or the communication module 190) to the volatile memory 132. It is loaded into, processes commands or data stored in the volatile memory 132, and the result data may be stored in the nonvolatile memory 134.
  • software eg, a program 140
  • the processor 120 may transfer commands or data received from other components (eg, the sensor module 176 or the communication module 190) to the volatile memory 132. It is loaded into, processes commands or data stored in the volatile memory 132, and the result data may be stored in the nonvolatile memory 134.
  • the processor 120 includes a main processor 121 (eg, a central processing unit or an application processor), and a secondary processor 123 (eg, a graphic processing unit, an image signal processor) that can be operated independently or together with the main processor 121 (eg, a central processing unit or an application processor). , A sensor hub processor, or a communication processor). Additionally or alternatively, the coprocessor 123 may be set to use lower power than the main processor 121 or to be specialized for a designated function. The secondary processor 123 may be implemented separately from the main processor 121 or as a part thereof.
  • main processor 121 eg, a central processing unit or an application processor
  • a secondary processor 123 eg, a graphic processing unit, an image signal processor
  • the coprocessor 123 may be set to use lower power than the main processor 121 or to be specialized for a designated function.
  • the secondary processor 123 may be implemented separately from the main processor 121 or as a part thereof.
  • the co-processor 123 is, for example, in place of the main processor 121 while the main processor 121 is in an inactive (eg, sleep) state, or the main processor 121 is active (eg, executing an application). ) While in the state, together with the main processor 121, at least one of the components of the electronic device 101 (for example, the display device 160, the sensor module 176, or the communication module 190) It is possible to control at least some of the functions or states associated with it.
  • the coprocessor 123 eg, an image signal processor or a communication processor
  • may be implemented as a part of other functionally related components eg, the camera module 180 or the communication module 190). have.
  • the memory 130 may store various data used by at least one component of the electronic device 101 (eg, the processor 120 or the sensor module 176 ).
  • the data may include, for example, software (eg, the program 140) and input data or output data for commands related thereto.
  • the memory 130 may include a volatile memory 132 or a nonvolatile memory 134.
  • the program 140 may be stored as software in the memory 130, and may include, for example, an operating system 142, middleware 144, or an application 146.
  • the input device 150 may receive a command or data to be used for a component of the electronic device 101 (eg, the processor 120) from outside (eg, a user) of the electronic device 101.
  • the input device 150 may include, for example, a microphone, a mouse, a keyboard, or a digital pen (eg, a stylus pen).
  • the sound output device 155 may output an sound signal to the outside of the electronic device 101.
  • the sound output device 155 may include, for example, a speaker or a receiver.
  • the speaker can be used for general purposes such as multimedia playback or recording playback, and the receiver can be used to receive incoming calls.
  • the receiver may be implemented separately from the speaker or as part of the speaker.
  • the display device 160 may visually provide information to the outside of the electronic device 101 (eg, a user).
  • the display device 160 may include, for example, a display, a hologram device, or a projector and a control circuit for controlling the device.
  • the display device 160 may include a touch circuitry set to sense a touch, or a sensor circuit (eg, a pressure sensor) set to measure the strength of a force generated by the touch. have.
  • the audio module 170 may convert sound into an electrical signal, or conversely, may convert an electrical signal into sound. According to an embodiment, the audio module 170 acquires sound through the input device 150, the sound output device 155, or an external electronic device (eg: Sound can be output through the electronic device 102) (for example, a speaker or headphones).
  • an external electronic device eg: Sound can be output through the electronic device 102
  • Sound can be output through the electronic device 102
  • the sensor module 176 detects an operating state (eg, power or temperature) of the electronic device 101, or an external environmental state (eg, a user state), and generates an electrical signal or data value corresponding to the detected state. can do.
  • the sensor module 176 is, for example, a gesture sensor, a gyro sensor, an atmospheric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a biometric sensor, It may include a temperature sensor, a humidity sensor, or an illuminance sensor.
  • the interface 177 may support one or more specified protocols that may be used for the electronic device 101 to connect directly or wirelessly with an external electronic device (eg, the electronic device 102 ).
  • the interface 177 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.
  • connection terminal 178 may include a connector through which the electronic device 101 can be physically connected to an external electronic device (eg, the electronic device 102).
  • the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).
  • the haptic module 179 may convert an electrical signal into a mechanical stimulus (eg, vibration or movement) or an electrical stimulus that a user can perceive through tactile or motor sensations.
  • the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.
  • the camera module 180 may capture a still image and a video.
  • the camera module 180 may include one or more lenses, image sensors, image signal processors, or flashes.
  • the power management module 188 may manage power supplied to the electronic device 101.
  • the power management module 188 may be implemented as at least a part of, for example, a power management integrated circuit (PMIC).
  • PMIC power management integrated circuit
  • the battery 189 may supply power to at least one component of the electronic device 101.
  • the battery 189 may include, for example, a non-rechargeable primary cell, a rechargeable secondary cell, or a fuel cell.
  • the communication module 190 includes a direct (eg, wired) communication channel or a wireless communication channel between the electronic device 101 and an external electronic device (eg, the electronic device 102, the electronic device 104, or the server 108). It is possible to support establishment and communication through the established communication channel.
  • the communication module 190 operates independently of the processor 120 (eg, an application processor) and may include one or more communication processors supporting direct (eg, wired) communication or wireless communication.
  • the communication module 190 is a wireless communication module 192 (eg, a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (eg : A local area network (LAN) communication module, or a power line communication module) may be included.
  • a corresponding communication module is a first network 198 (for example, a short-range communication network such as Bluetooth, WiFi direct or IrDA (infrared data association)) or a second network 199 (for example, a cellular network, the Internet, or It can communicate with external electronic devices through a computer network (for example, a telecommunication network such as a LAN or WAN).
  • the wireless communication module 192 uses subscriber information stored in the subscriber identification module 196 (eg, International Mobile Subscriber Identifier (IMSI)) in a communication network such as the first network 198 or the second network 199.
  • IMSI International Mobile Subscriber Identifier
  • the electronic device 101 can be checked and authenticated.
  • the antenna module 197 may transmit a signal or power to the outside (eg, an external electronic device) or receive from the outside.
  • the antenna module may include one antenna including a conductor formed on a substrate (eg, a PCB) or a radiator formed of a conductive pattern.
  • the antenna module 197 may include a plurality of antennas. In this case, at least one antenna suitable for a communication method used in a communication network such as the first network 198 or the second network 199 is, for example, provided by the communication module 190 from the plurality of antennas. Can be chosen.
  • the signal or power may be transmitted or received between the communication module 190 and an external electronic device through the at least one selected antenna.
  • other components eg, RFIC
  • other than the radiator may be additionally formed as part of the antenna module 197.
  • At least some of the components are connected to each other through a communication method (e.g., a bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI))) between peripheral devices and a signal ( E.g. commands or data) can be exchanged with each other.
  • a communication method e.g., a bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)
  • GPIO general purpose input and output
  • SPI serial peripheral interface
  • MIPI mobile industry processor interface
  • commands or data may be transmitted or received between the electronic device 101 and the external electronic device 104 through the server 108 connected to the second network 199.
  • Each of the electronic devices 102 and 104 may be a device of the same or different type as the electronic device 101.
  • all or part of the operations executed by the electronic device 101 may be executed by one or more of the external electronic devices 102, 104, or 108.
  • the electronic device 101 needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device 101
  • One or more external electronic devices receiving the request may execute at least a part of the requested function or service, or an additional function or service related to the request, and transmit a result of the execution to the electronic device 101.
  • the electronic device 101 may process the result as it is or additionally and provide it as at least a part of a response to the request.
  • cloud computing distributed computing, or client-server computing technology may be used.
  • An electronic device may be a device of various types.
  • the electronic device may include, for example, a portable communication device (eg, a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance.
  • a portable communication device eg, a smart phone
  • a computer device e.g., a smart phone
  • a portable multimedia device e.g., a portable medical device
  • a camera e.g., a camera
  • a wearable device e.g., a smart bracelet
  • the electronic device according to the embodiment of the present document is not limited to the above-described devices.
  • a or B “at least one of A and B”, “at least one of A or B,” “A, B or C,” “at least one of A, B and C,” and “A”
  • Each of the phrases such as “at least one of, B, or C” may include any one of the items listed together in the corresponding one of the phrases, or all possible combinations thereof.
  • Terms such as “first”, “second”, or “first” or “second” may be used simply to distinguish the component from other Order) is not limited.
  • Some (eg, a first) component is referred to as “coupled” or “connected” to another (eg, a second) component, with or without the terms “functionally” or “communicatively”. When mentioned, it means that any of the above components may be connected to the other components directly (eg by wire), wirelessly, or via a third component.
  • module used in this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic blocks, parts, or circuits.
  • the module may be an integrally configured component or a minimum unit of the component or a part thereof that performs one or more functions.
  • the module may be implemented in the form of an application-specific integrated circuit (ASIC).
  • ASIC application-specific integrated circuit
  • Various embodiments of the present document include one or more instructions stored in a storage medium (eg, internal memory 136 or external memory 138) that can be read by a machine (eg, electronic device 101). It may be implemented as software (for example, the program 140) including them.
  • the processor eg, the processor 120 of the device (eg, the electronic device 101) may call and execute at least one command among one or more commands stored from a storage medium. This enables the device to be operated to perform at least one function according to the at least one command invoked.
  • the one or more instructions may include code generated by a compiler or code executable by an interpreter.
  • a storage medium that can be read by a device may be provided in the form of a non-transitory storage medium.
  • a method according to various embodiments disclosed in the present document may be provided by being included in a computer program product.
  • Computer program products can be traded between sellers and buyers as commodities.
  • the computer program product is distributed in the form of a device-readable storage medium (e.g. compact disc read only memory (CD-ROM)), or through an application store (e.g. Play Store TM ) or two user devices (e.g., compact disc read only memory (CD-ROM)) It can be distributed (e.g., downloaded or uploaded) directly between, e.g. smartphones).
  • a device-readable storage medium e.g. compact disc read only memory (CD-ROM)
  • an application store e.g. Play Store TM
  • two user devices e.g., compact disc read only memory (CD-ROM)
  • It can be distributed (e.g., downloaded or uploaded) directly between, e.g. smartphones).
  • At least a part of the computer program product may be temporarily stored or temporarily generated in a storage medium that can be read by a device such as a server of a manufacturer, a server of an application store, or a memory of a relay server.
  • each component (eg, module or program) of the above-described components may include a singular number or a plurality of entities.
  • one or more components or operations among the above-described corresponding components may be omitted, or one or more other components or operations may be added.
  • a plurality of components eg, a module or program
  • the integrated component may perform one or more functions of each component of the plurality of components in the same or similar to that performed by the corresponding component among the plurality of components prior to the integration. .
  • operations performed by a module, program, or other component may be sequentially, parallel, repeatedly, or heuristically executed, or one or more of the operations may be executed in a different order or omitted. Or one or more other actions may be added.
  • the program 140 includes an operating system 142 for controlling one or more resources of the electronic device 101, middleware 144, or an application 146 executable in the operating system 142.
  • the operating system 142 may include, for example, Android TM , iOS TM , Windows TM , Symbian TM , Tizen TM , or Bada TM .
  • At least some of the programs 140 are, for example, preloaded on the electronic device 101 at the time of manufacture, or when used by a user, an external electronic device (eg, electronic device 102 or 104), or a server ( 108)).
  • the operating system 142 may control management (eg, allocation or retrieval) of one or more system resources (eg, process, memory, or power) of the electronic device 101.
  • Operating system 142 additionally or alternatively, other hardware devices of the electronic device 101, for example, the input device 150, the sound output device 155, the display device 160, the audio module 170 , Sensor module 176, interface 177, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196, or One or more driver programs for driving the antenna module 197 may be included.
  • the middleware 144 may provide various functions to the application 146 so that a function or information provided from one or more resources of the electronic device 101 can be used by the application 146.
  • the middleware 144 is, for example, an application manager 201, a window manager 203, a multimedia manager 205, a resource manager 207, a power manager 209, a database manager 211, and a package manager 213. ), a connectivity manager 215, a notification manager 217, a location manager 219, a graphic manager 221, a security manager 223, a call manager 225, or a voice recognition manager 227.
  • I can.
  • the application manager 201 may manage the life cycle of the application 146, for example.
  • the window manager 203 may manage one or more GUI resources used on a screen, for example.
  • the multimedia manager 205 for example, identifies one or more formats required for playback of media files, and performs encoding or decoding of a corresponding media file among the media files by using a codec suitable for the selected corresponding format. You can do it.
  • the resource manager 207 may manage the source code of the application 146 or a memory space of the memory 130, for example.
  • the power manager 209 manages the capacity, temperature, or power of the battery 189, for example, and may determine or provide related information necessary for the operation of the electronic device 101 by using the corresponding information. . According to an embodiment, the power manager 209 may interwork with a basic input/output system (BIOS) (not shown) of the electronic device 101.
  • BIOS basic input/output system
  • the database manager 211 may create, search, or change a database to be used by the application 146, for example.
  • the package manager 213 may manage installation or update of an application distributed in the form of, for example, a package file.
  • the connectivity manager 215 may manage, for example, a wireless connection or a direct connection between the electronic device 101 and an external electronic device.
  • the notification manager 217 may provide a function for notifying the user of the occurrence of a designated event (eg, incoming call, message, or alarm), for example.
  • the location manager 219 may manage location information of the electronic device 101, for example.
  • the graphic manager 221 may manage, for example, one or more graphic effects to be provided to a user or a user interface related thereto.
  • the security manager 223 may provide, for example, system security or user authentication.
  • the telephony manager 225 may manage, for example, a voice call function or a video call function provided by the electronic device 101.
  • the voice recognition manager 227 transmits, for example, a user's voice data to the server 108, and a command corresponding to a function to be performed in the electronic device 101 based at least in part on the voice data, Alternatively, text data converted based at least in part on the voice data may be received from the server 108.
  • the middleware 244 may dynamically delete some of the existing components or add new components.
  • at least a part of the middleware 144 may be included as a part of the operating system 142 or implemented as separate software different from the operating system 142.
  • the application 146 is, for example, a home 251, a dialer 253, an SMS/MMS 255, an instant message (IM) 257, a browser 259, a camera 261, and an alarm 263. , Contacts (265), voice recognition (267), email (269), calendar (271), media player (273), album (275), watch (277), health (279) (e.g. Biometric information measurement), or environmental information 281 (eg, air pressure, humidity, or temperature information measurement) application may be included. According to an embodiment, the application 146 may further include an information exchange application (not shown) capable of supporting information exchange between the electronic device 101 and an external electronic device.
  • an information exchange application (not shown) capable of supporting information exchange between the electronic device 101 and an external electronic device.
  • the information exchange application may include, for example, a notification relay application configured to deliver specified information (eg, a call, a message, or an alarm) to an external electronic device, or a device management application configured to manage an external electronic device.
  • the notification relay application for example, transmits notification information corresponding to a specified event (eg, mail reception) generated by another application (eg, email application 269) of the electronic device 101 to an external electronic device. I can. Additionally or alternatively, the notification relay application may receive notification information from an external electronic device and provide it to the user of the electronic device 101.
  • the device management application includes, for example, an external electronic device that communicates with the electronic device 101 or some components thereof (for example, the display device 160 or the camera module 180). -Off) or a function (eg, brightness, resolution, or focus of the display device 160 or the camera module 180) may be controlled.
  • the device management application may additionally or alternatively support installation, deletion, or update of an application operating in an external electronic device.
  • FIG. 3 illustrates a configuration of an electronic device 300 (eg, the electronic device 101 of FIG. 1) according to an exemplary embodiment.
  • the electronic device 300 may include a processor 310 (eg, the processor 120 of FIG. 1) and a memory 320 (eg, the memory 130 of FIG. 1 ).
  • the processor 310 may be configured to be operatively connected to the memory 320.
  • the processor 310 may control components of the electronic device 300 or process data by executing instructions stored in the memory 320.
  • the electronic device 300 includes a camera 330 (eg, the camera module 180 of FIG. 1 ), a display 340 (eg, the display device 160 of FIG. 1 ), and a communication module 350. It may further include at least one of (eg, the communication module of FIG. 1).
  • the processor 310 may acquire image data including an image photographed of the user 1 through the camera 330.
  • the processor 310 may obtain image data including an RGB value and a depth value.
  • the camera 330 may detect a distance value to a subject based on a Time of Flight (TOF) method.
  • the processor 310 may recognize the motion of the subject photographed in the image by using the depth value.
  • the processor 310 may track the user's face photographed in the image by using the recognized motion information.
  • the processor 310 may acquire image data captured using the camera 330 included in the electronic device 300.
  • the processor 310 may acquire image data captured through a camera 330 included in another device.
  • the processor 310 is located near the electronic device 300 and may receive image data from a device equipped with a camera through a communication connection (eg, short-range wireless communication or wired communication) using the communication module 350. I can. As another example, the processor 310 may obtain image data downloaded from an external server (eg, the server 108 of FIG. 1) by the electronic device 300. The method of obtaining the image data by the processor 310 may be variously modified.
  • a communication connection eg, short-range wireless communication or wired communication
  • the processor 310 may obtain image data downloaded from an external server (eg, the server 108 of FIG. 1) by the electronic device 300. The method of obtaining the image data by the processor 310 may be variously modified.
  • the processor 310 that has obtained the image data may analyze the image data.
  • the processor 310 may recognize a user's face or skin in an image and track a location of the recognized face or skin in the image.
  • a user's biometric signal eg, a pulse wave
  • a pixel value included in an area in which the user's skin is photographed may be required among information included in the image data.
  • the processor 310 may identify a location of an area in which the user's face is photographed in the image.
  • the processor 310 convolves the image before and after the image sequence included in the image data to identify the location of the region where the user's face is photographed, and determines the region where the user's face is photographed based on the result. I can.
  • the processor 310 may divide the image channels included in the area where the face of the image data is captured into color channels (eg, RGB channels), and select a channel necessary for measuring a biosignal from among the image channels.
  • the processor 310 may remove color values that are not suitable for extracting a bio-signal from among color values of an image channel. For example, the processor 310 may exclude color values of a region in which the reflected light is photographed or a region other than the skin (eg, eyebrows) photographed from the analysis target.
  • the processor 310 recognizes a region in which the user's face has been photographed in the image, and the pixel value of pixels within a range in which it can be determined that the skin has been photographed among regions recognized as photographing the user's face A biosignal may be obtained based on.
  • the processor 310 may amplify the values of the selected color channels and remove noise to extract the biosignal. For example, the processor 310 may adjust a signal to noise ratio (SNR) by applying a nonlinear scale factor to each RGB channel. The processor 310 may calculate a biosignal based on a signal to which a scale factor is applied.
  • SNR signal to noise ratio
  • the processor 310 may image-process the preprocessed image signal and then calculate the heart rate based on the time domain. Alternatively, the processor 310 may convert the image signal into the frequency domain and then calculate the heart rate based on the frequency domain. The processor 310 may control the display 340 to display a screen including information on the calculated heart rate. Alternatively, the processor 310 may store biometric information data including the calculated heart rate in the memory 320 or other device (eg, the electronic device 104 or server 108 of FIG. 1) through the communication module 350. Can be transferred to. In the present specification, the biometric information data may refer to information generated by collecting biometric information on the user 1. The biometric information data may be used to diagnose the health status of the user 1 or to determine the characteristics of the user 1's body.
  • a heart rate value may not be calculated from a bio-signal extracted from image data or a heart rate value that is out of a normal range may be calculated. For example, if the face of the user 1 moves outside the area photographed by the camera 330 due to the movement of the user 1, the heart rate value cannot be calculated from the image data.
  • the processor 310 may determine a correction value for a section in which the normal heart rate value is not calculated. The determined correction value may be output through the display 340 in place of the calculated heart rate value, or may be included in the biometric information data.
  • the processor 310 may determine a correction value to replace the measured biosignal (biosignal extracted from image data) using the candidate data.
  • candidate data may mean data created by predicting how a biosignal of an unmeasured section will be formed.
  • the candidate data is a feature point for predicting how the biosignal of the correction target section will be formed from the biosignal last measured before the section in which the biosignal is not normally measured (hereinafter, "correction target section"). May include information about.
  • the candidate data may include an artificial biosignal (eg, artificial heartbeat model data (AHD)) created based on a feature point extracted from a previously measured biosignal.
  • AHD artificial heartbeat model data
  • the processor 310 may determine whether the calculated value is a normal value by comparing a value calculated from the image data (eg, a heart rate value) with candidate data. For example, the processor 310 may compare the R-R peak interval of data (measured biosignal) associated with the heart rate value and the R-R peak interval of the candidate data. As a result of the comparison, when the R-R peak intervals are similar to each other, the processor 310 may determine that the calculated heart rate value is a normal value.
  • a value calculated from the image data eg, a heart rate value
  • the processor 310 may determine a correction value in consideration of information on an exercise being performed by the user 1. For example, when a guide image content (eg, 1620 in FIG. 16) for guiding a user's exercise is being played through the display 340, the user performs an exercise according to the guide image content (eg, 1620 in FIG. 16). It can be seen as being performed. In this case, the processor 310 may acquire the amount of change in heart rate associated with the guide image content being played (eg, 1620 in FIG. 16 ). The amount of change in heart rate may be included in the guide image content (eg, 1620 in FIG. 16) or may be searched using an indicator indicating an exercise provided by the guide image content (eg, 1620 in FIG. 16 ). The processor 310 may determine a correction value such that the biosignal within the correction target section becomes a signal indicating the acquired heart rate change amount.
  • a guide image content eg, 1620 in FIG. 16
  • the processor 310 may determine a correction value such that the biosignal within
  • the processor 310 may determine a correction value according to the length of the region to be corrected. For example, if the correction target section is short (e.g. 3 seconds), the correction value is determined based on the candidate data, and if the correction target section is longer than a threshold value (e.g. 5 seconds), a pattern database (not shown) The correction value may be determined based on the data pattern stored in (not).
  • the data pattern may include user information (eg, gender, age, weight, or height) of the user 1 and data defining an amount of change in heart rate according to the type of exercise that the user 1 is performing.
  • the processor 310 may search for a heart rate change amount from the pattern database based on user information and the type of exercise, and determine a correction value for the correction target section based on the searched heart rate change amount and a previous heart rate value.
  • User information may refer to information indicating a user's attributes (eg, gender, age, weight, height).
  • the processor 310 may recalibrate the correction value of the correction target section among the biometric information data based on the measured biosignal again. . Since the correction value input to the correction target section is not actually a measured value, non-linear data may be generated at the point when the correction target section ends. The processor 310 may correct the value of the correction target section again based on the biosignal normally measured after the time point to which the correction target section ends.
  • the processor 310 may control the display 340 to display at least a part of the corrected biometric information data. Alternatively, the processor 310 may control the display 340 to determine recommendation information to be provided to the user based on the biometric information data and display the determined recommendation information.
  • the display 340 may be a device that is hardware coupled to the electronic device 300, but may be a device to which communication is connected through the communication module 350. Alternatively, the display 340 may be a device controlled through an external server in the IoT system.
  • the recommended information may include information related to exercise of the user 1.
  • the processor 310 may collect biometric information data while outputting guide image content (eg, 1620 in FIG. 16) for guiding a user's exercise through the display 340. Since the guide image content (e.g., 1620 in FIG. 16) is an image that guides the user to perform a specified exercise, the processor 310 may use biometric information data based on information included in the guide image content (e.g., 1620 in FIG. 16). It is possible to determine which exercise the user is performing at the time when is collected. However, the method of determining which exercise the user is performing by the electronic device is not limited thereto.
  • the processor 310 may detect the user's movement and determine the user's exercise progress using an exercise matching algorithm corresponding to the detected movement.
  • the processor 310 may track a user's movement using the image and depth information acquired through the 3D camera, and determine which exercise the user is performing based on exercise information matched with the user's movement.
  • the processor 310 may compare the exercise performed by the user at the time the biometric information data is collected and reference data including average biometric information of other users while performing the same type of exercise.
  • the reference data may be biometric information data measured when the user 1 previously performed the same exercise.
  • the reference data may be stored in the memory 320 or may be downloaded through the communication module 350.
  • the processor 310 may transmit the biometric information data to an external server, and may receive a comparison result of the biometric information data and the reference data from the external server. For example, the processor 310 may compare the change amount of the heart rate value included in the biometric information data with the change amount of the reference heart rate. The processor 310 may output recommended exercise intensity information for increasing exercise intensity when the increase in the heart rate value is smaller than the value of the reference heart rate change amount. Conversely, when the increase in the heart rate value is greater than the value of the reference heart rate change amount, the processor 310 may output recommended exercise intensity information for lowering the exercise intensity.
  • the processor 310 may output recommended exercise information recommending an exercise to be performed by the user based on a comparison result of biometric information of users having the same or similar user information as the user 1 in the same exercise program. have.
  • the processor 310 may analyze the biometric information data of the user 1 for each exercise program and determine the ability value for each body part.
  • the processor 310 may provide the capability value for each body part to the user 1 or may output recommended exercise information according to the capability value for each body part. For a more specific example, when the measured heart rate increases significantly and the number of failures is high when the exercise program for training the thigh muscles is provided, the processor 310 may output the exercise program for training the thigh muscles as recommended exercise information. have.
  • the electronic device 300 may easily track a change in heart rate even during exercise of the user 1, and thus provide data that facilitates early detection of arrhythmia findings according to a change in heart rate during exercise.
  • FIG. 4 is a flowchart 400 illustrating a process of generating biometric information data by an electronic device (eg, the electronic device 101 of FIG. 1 and the electronic device 300 of FIG. 3) according to an embodiment.
  • the processor of the electronic device eg, the processor 120 of FIG. 1, the processor 310 of FIG. 3
  • the memory eg, the memory 130 of FIG. 1, FIG. It can be understood that it is executed by executing instructions stored in the memory 320 of 3.
  • the electronic device may acquire image data photographing a user.
  • the electronic device may acquire image data including an image captured using a camera (eg, the camera module 180 of FIG. 1 and the camera 330 of FIG. 3) provided in the electronic device. have.
  • the electronic device may receive image data acquired through an external device.
  • the electronic device may analyze the acquired image data. According to an embodiment, the electronic device may recognize a user from an image included in image data and classify an image channel to be used to extract a biosignal. In operation S430, the electronic device may calculate a heart rate value from the biosignal extracted from the classified image channel.
  • the electronic device may determine whether the heart rate value is normally calculated. If the user's face is not recognized in operation 420 or the biosignal that is the basis of the heart rate value is out of the normal range in operation S430, the electronic device may determine that the heart rate value has not been normally calculated. According to an embodiment, the electronic device may not capture a shape of the user by the camera while the user performs other actions (eg, drinking water, talking on the phone, or resting) during exercise. If the user is not photographed, the electronic device may repeatedly perform operation S410 until the user is photographed. Alternatively, when the user is not recognized in the image (eg, when the user is not photographed for a specified time or longer), the electronic device may terminate a program for generating biometric information data.
  • the electronic device may terminate a program for generating biometric information data.
  • biometric information data including the heart rate calculated in operation S450 may be generated.
  • the generated biometric information data may be stored in a memory (eg, the memory 130 of FIG. 1 or the memory 320 of FIG. 3 ), or may be transmitted to an external server.
  • the electronic device may determine a correction value in operation S445.
  • the electronic device may determine a correction value based on at least one of a last measured value of a normal biosignal, a feature point extracted from existing data, information on an exercise being performed by a user, and a data pattern.
  • the electronic device may generate biometric information data in which a value of the biosignal associated with the heart rate value is replaced with the determined correction value.
  • FIG 5 illustrates an example of an electronic device 500 according to an embodiment.
  • the electronic device 500 (for example, the electronic device 101 of FIG. 1 and the electronic device 300 of FIG. 3) that measures a biosignal may be configured as a stand-alone device. .
  • the electronic device 500 may photograph a user 1 using a camera 530 provided on one surface, and obtain biometric information data from the captured image.
  • the electronic device 500 includes at least a portion of the biometric information data or biometric data through an output device (for example, the display device 160 of FIG. 1, the sound output device 155, and the display 340 of FIG. 3) provided in the electronic device. Information related to data can be output.
  • FIG. 6 shows an example of a system 600 for acquiring biometric information data, according to an embodiment.
  • an apparatus for acquiring and processing a bio-signal may be configured as a system 600 including a plurality of hardware devices connected through a network 699.
  • the plurality of devices 601, 630, and 640 may be devices connected to an Internet of Things (IoT) system controlled by the server 610.
  • IoT Internet of Things
  • the camera device 630 may transmit image data including an image of the user 1 to the server 610.
  • the server 610 may analyze image data to obtain biometric information data.
  • the server 610 may display at least part of the biometric information data or information related to the biometric information data through the display device 640.
  • the server 610 displays information on the exercise state of the user 1 required to correct the biometric information data or determine information related to the biometric information data. It can be obtained based on information related to the content being played through the device 640. For example, when an image guiding the lower body exercise is being played through the display device 640, or the exercise device 601 in motion is a device for lower body exercise, the server 610 provides the user 1 to exercise the lower body. It can be judged as being on the job. According to another embodiment, the server 610 may analyze an image acquired from the camera device 630 to obtain information on a user's exercise state.
  • heart rate information can be obtained while continuously monitoring the user's heart rate.
  • the system 600 may be configured to automatically transmit a message to a designated contact (eg, an emergency agency or a guardian's communication device) when a singularity occurs in the heart rate information acquired during the user's daily life.
  • a designated contact eg, an emergency agency or a guardian's communication device
  • the server 610 may receive image data obtained from the camera device 630.
  • the server 610 may generate biometric information data including a heart rate value calculated from the received image data.
  • the server 610 may obtain biometric information data including a heart rate value calculated based on image data obtained from the camera device 630.
  • the server 610 may compare the heart rate data of the arrhythmia patient and the biometric information data stored in the memory of the server 610.
  • the biometric information data may be corrected data.
  • the arrhythmia patient heart rate data may include information on a heart rate value measured while the arrhythmia patients perform exercise.
  • the server 610 may determine the number of times that a characteristic (eg, an abnormal pattern) included in the heart rate data of the arrhythmia patient occurs in the biometric information data.
  • a characteristic eg, an abnormal pattern
  • the server 610 may extract a feature point (data representing the feature) from the arrhythmia patient heart rate data and the biometric information data through a designated process, and compare the extracted feature points.
  • the server 610 may output information on the abnormal heartbeat pattern to the display device 640 to display information.
  • FIG. 6 is for explaining the structure of an embodiment, and the shape of the device may be differently modified according to the embodiment.
  • the system 600 of FIG. 6 may be implemented in the form of a healthcare device having functions of a display 640, a camera 630, and a server 610 without a separate server 610 being built. have.
  • FIG. 7 illustrates another example of an electronic device 700 (eg, the electronic device 101 of FIG. 1) that acquires biometric information data, according to an exemplary embodiment.
  • the electronic device 700 receives image data from the camera device 730 and processes the image data to transmit at least part of the biometric information data or information related to the biometric information data through the display device 740. Can be printed.
  • the electronic device 700 may be connected to the camera device 730 or the display device 740 through short-range wireless communication (eg, BluetoothTM, Wi-FiTM).
  • FIGS. 5 to 7 are for explaining several embodiments, and the hardware configuration is not limited thereto, and the hardware configuration may be variously changed according to the embodiments.
  • FIG. 8 is a flowchart 800 illustrating a process of calculating a heart rate value by an electronic device (eg, the electronic device 101 of FIG. 1 and the electronic device 300 of FIG. 3) according to an embodiment.
  • the processor of the electronic device eg, the processor 120 of FIG. 1, the processor 310 of FIG. 3
  • the memory eg, the memory 130 of FIG. 1, FIG. It can be understood that it is executed by executing instructions stored in the memory 320 of 3.
  • the electronic device may convolve the before and after images in the image sequence included in the image data. Based on the convolution result, the electronic device may compare the previous image frame and the current image frame in operation S820 to determine a first area within the current image frame in which the user's face is photographed.
  • the user's face includes skin, eyes, lips, or eyebrows, but since only information on the user's skin color is required for heart rate detection, the remaining elements can be removed.
  • the electronic device may divide an image channel in the first region.
  • the electronic device may remove an unnecessary color value from among values in the first area. For example, the electronic device may select color values excluding channels having values within a range similar to that of a skin value from among the divided image channels.
  • the electronic device may amplify the selected image channel values. That is, the electronic device can increase the size of the signal by amplifying the intensity of each pixel. For example, the electronic device may assign a scale factor to each color channel. By amplifying the value of the video channel, a signal having a large value becomes relatively larger, thereby improving a signal-to-noise ratio.
  • the electronic device may calculate a heart rate value based on the amplified color channel value.
  • FIG. 9 is a conceptual diagram illustrating a method of recognizing a face region for calculating a heart rate value by an electronic device (eg, the electronic device 101 of FIG. 1 and the electronic device 300 of FIG. 3 ), according to an exemplary embodiment It is a drawing for.
  • an electronic device eg, the electronic device 101 of FIG. 1 and the electronic device 300 of FIG. 3
  • the electronic device may determine the first area 910 in the image 900 in which the user's face is photographed. Also, the electronic device may find a skin color in the first area 910 and determine a center point 920 thereof. The electronic device may determine the second area 931 in which pixels determined to be skin color values are located while amplifying the pixel value radially from the center point 920. The area 941 in which the eyebrows other than the color value of the skin are photographed may not be included in the second area 931.
  • the electronic device determines a plurality of second regions 931, 932, 933, 934, and values of pixels included in an area combining the plurality of second regions 931, 932, 933, 934
  • the bio-signal can be extracted based on areas having a color value of the skin excluding the eyebrow area 941, the eye area 942, and the area 943 in which the reflected light is photographed. have.
  • FIG. 10 is a flowchart 1000 illustrating a process of determining whether a heart rate value is normally calculated by an electronic device (eg, the electronic device 101 of FIG. 1 and the electronic device 300 of FIG. 3 ), according to an exemplary embodiment.
  • the processor of the electronic device eg, the processor 120 of FIG. 1, the processor 310 of FIG. 3
  • the memory eg, the memory 130 of FIG. 1, FIG. It can be understood that it is executed by executing instructions stored in the memory 320 of 3.
  • the electronic device may detect an object in the image data. According to an embodiment, the electronic device may determine whether an object corresponding to a user exists among the detected objects. If an object corresponding to the user does not exist, the electronic device may output a message inducing the user to be photographed in the image. Alternatively, the electronic device may terminate the process.
  • the electronic device may determine whether the user's face is detectable within the image data. When the user's face is not detected in the image data, the electronic device may determine that the heart rate is not normally calculated.
  • the electronic device may calculate a heart rate value based on information in the detected face area. For example, the electronic device may extract a pulse wave as a bio-signal from a skin color value in the facial area, and determine a heart rate generated per unit time from the pulse wave.
  • the electronic device may determine whether the heart rate value is normally calculated. For example, when the R-R peak interval value of the biosignal is included in the normal range, the electronic device may determine that the heart rate value is normally calculated. Conversely, when the R-R peak interval value of the bio-signal is out of the normal range, the electronic device may determine that the heart rate value is not normally calculated.
  • FIG. 11 is a diagram illustrating a process of determining a correction value when a heart rate value is not normally calculated by an electronic device (eg, the electronic device 101 of FIG. 1 or the electronic device 300 of FIG. 3 ), according to an embodiment
  • One is a flow chart 1100.
  • the processor of the electronic device eg, the processor 120 of FIG. 1, the processor 310 of FIG. 3
  • the memory eg, the memory 130 of FIG. 1, FIG. It can be understood that it is executed by executing instructions stored in the memory 320 of 3.
  • the electronic device may determine whether a heart rate value that is not normally calculated can be corrected using candidate data. According to an embodiment, when the size of a section in which the heart rate value is not normally calculated is less than or equal to the threshold value, the electronic device may determine that the heart rate value can be corrected based on candidate data. Conversely, when the size of the section in which the heart rate value is not normally calculated is greater than the threshold value, the electronic device may determine that the heart rate value cannot be corrected as candidate data. That is, data may be corrected based on candidate data for a biosignal lost for a short period of time, and data may be corrected based on a different method when the period in which the biosignal is lost becomes longer.
  • the candidate data may include data modeled to have characteristics similar to those of the measured heart rate data by comparing with the previously measured heart rate data.
  • the candidate data may be referred to as a heart rate data prediction model.
  • the electronic device may update candidate data based on a result of learning measured data using machine learning using a genetic algorithm.
  • the electronic device may determine a correction value for correcting a heart rate value that is not normally calculated based on the candidate data. If it is determined that data cannot be corrected based on the candidate data, in operation S1125, the electronic device may determine a correction value based on the pattern similarity. In operation S1125, in order to determine the correction value based on the pattern similarity, the electronic device may determine the correction value based on user information and a normally measured last heart rate value.
  • the user information may include information indicating a user's attributes (eg, gender, age, weight, height) and information related to an exercise that the user is performing (eg, type of exercise, intensity of exercise).
  • the electronic device may search for a data pattern corresponding to user information from the pattern database.
  • the data pattern may include information on an amount of change in heart rate corresponding to user information.
  • the electronic device may determine a correction value for correcting the heart rate value based on the searched heart rate change amount and the last heart rate value.
  • the electronic device may search for a data pattern from a pattern database stored in the electronic device.
  • the electronic device may provide user information to an external server, and the external server may retrieve a data pattern from a pattern database and transmit it to the electronic device.
  • the electronic device may generate biometric information data including a correction value.
  • the biometric information data including the correction value may include a heart rate value calculated based on the correction value.
  • the electronic device may generate the biometric information data including an indicator indicating that the value has been corrected.
  • the flowchart 1100 illustrated in FIG. 11 is for explaining a method of correcting data that is not normally measured based on an embodiment, and a process of correcting data may be applied differently from the flowchart 1100.
  • FIG. 12 conceptually illustrates an operation of correcting a heart rate value by using candidate data by an electronic device (eg, the electronic device 101 of FIG. 1 and the electronic device 300 of FIG. 3) according to an embodiment.
  • an electronic device eg, the electronic device 101 of FIG. 1 and the electronic device 300 of FIG. 3
  • the electronic device may calculate a normal heart rate value from the pulse wave extracted in the normal section 1210.
  • the heart rate value cannot be normally extracted. Therefore, it is necessary to correct the data of the abnormal section 1220.
  • the electronic device since the data of the abnormal section 1220 does not have a normal RR peak interval, the electronic device according to an exemplary embodiment uses the normal section 1210 based on the RR peak interval of the data. And the abnormal section 1220 can be identified.
  • the electronic device may replace the data of the abnormal section 1220 with candidate data 1200.
  • the candidate data 1200 may be data created by predicting data to occur in the abnormal section 1220 by learning feature points extracted from data in the normal section 1210.
  • FIG. 13 illustrates an electronic device (eg, the electronic device 101 of FIG. 1, the electronic device 300 of FIG. 3) additionally correcting the corrected biometric information data when a normal heart rate value is calculated, according to an exemplary embodiment.
  • the processor of the electronic device eg, the processor 120 of FIG. 1, the processor 310 of FIG. 3
  • the processor of the electronic device is a memory (eg, the memory 130 of FIG. 1, FIG. It can be understood that it is executed by executing instructions stored in the memory 320 of 3.
  • the electronic device may determine whether data corresponding to the heart rate value before the calculated heart rate value in the biometric information data is a corrected value.
  • the previous data may be a heart rate value prior to the calculated heart rate value, or may mean another biosignal value for calculating a previous heart rate value.
  • an error may occur with the actual heart rate value.
  • a non-linear section may occur between the previous data and the calculated heart rate value.
  • Data may be discontinuously formed because the corrected value of the previous data does not match the actual value.
  • the electronic device evaluates the correction value of the previous data based on the heart rate value calculated in operation S1320, and if the correction value is not valid, the correction value of the previous data Can be recalibrated. For example, if the correction value of previous data and the calculated heart rate value do not have a linear relationship, the electronic device may determine that the correction value is not valid. The electronic device may recalibrate the previous data so that the corrected value of the previous data has a linear value with the calculated heart rate value.
  • the electronic device may generate biometric information data including the calculated heart rate value.
  • FIG. 14 conceptually illustrates an operation of recalibrating a corrected value by an electronic device (eg, the electronic device 101 of FIG. 1 and the electronic device 300 of FIG. 3) according to an exemplary embodiment.
  • an electronic device eg, the electronic device 101 of FIG. 1 and the electronic device 300 of FIG. 3
  • a point 1430 at which is formed non-linearly may occur.
  • the electronic device converts data of at least a portion of the section 1425 including the second point of view 1420 of the corrected section 821 into the data of the first point of view 1410. It can be corrected to have a linear relationship with.
  • FIG. 14 is merely a description of an exemplary embodiment of recalibrating a bio-signal, but is not limited thereto.
  • the electronic device corrects the value of the corrected section based on the heart rate values normally calculated before and after the corrected data to a value that linearly connects the heart rate value normally calculated before and after the corrected data. You may.
  • the heart rate value was calculated as 80 at the end point of the first section, the heart rate value was corrected to 90 in the second section after the first section, and the heart rate value at the point in the third section after the second section
  • the electronic device may recalibrate the heart rate value of the second section to a heart rate value that linearly increases so that the heart rate value of the second section starts at 80 at the time point of the second section and becomes 105 at the end point.
  • FIG. 15 is a diagram illustrating a process in which an electronic device (eg, the electronic device 101 of FIG. 1, the electronic device 300 of FIG. 3) generates candidate data based on a feature point and generates biometric information data according to an embodiment.
  • an electronic device eg, the electronic device 101 of FIG. 1, the electronic device 300 of FIG. 3
  • One is a flow chart 1500.
  • the processor of the electronic device eg, the processor 120 of FIG. 1, the processor 310 of FIG. 3
  • the memory eg, the memory 130 of FIG. 1, FIG. It can be understood that it is executed by executing instructions stored in the memory 320 of 3.
  • the electronic device may generate candidate data.
  • candidate data may be configured by estimating a target suitability.
  • Target fit may mean an individual's average normal heart rate. That is, the electronic device may configure initial candidate data to indicate the user's normal heart rate.
  • the electronic device may generate candidate data based on the feature point data.
  • the candidate data may be artificially generated heartbeat data.
  • the electronic device may determine whether input data exists.
  • the input data may mean a calculated heart rate value or a biosignal for calculating a heart rate value.
  • the electronic device may determine that the input signal does not exist. If the input data does not exist, since the heart rate value cannot be calculated, in operation S1525, the electronic device may generate biometric information data based on the candidate data.
  • the electronic device may determine whether the amount of feature point data is sufficient. For example, in an electronic device, if the feature point data contains more than 500 frames and candidate data is created based on the feature point data, there is no change in the result at each run, and the candidate data is stably created. It can be judged that the amount is sufficient.
  • the feature point data may mean data including information on feature points extracted from a bio-signal so that the device can learn by using a machine learning model. If the amount of feature point data is not sufficient, since candidate data generated in operation S1510 is not reliable, the electronic device may not use the candidate data to generate a correction value. In this case, the electronic device may extract the feature point from the input data in operation S1550, update the feature point data, and generate biometric information data based on a heart rate value included in the input data or calculated from the input data.
  • the electronic device may compare a heart rate value based on the input data and candidate data in operation S1540.
  • the electronic device may compare the R-R peak interval for the heart rate value and the R-R peak interval for the candidate data. Since the candidate data is a biosignal that is expected to be measured when the biosignal is normally measured, the biosignal normally measured has an R-R peak interval similar to the R-R peak interval of the candidate data. Accordingly, it may be determined whether the measured bio-signal or heart rate value is a value requiring correction through comparison of the R-R peak interval.
  • the R-R peak interval with respect to a heart rate value may mean an R-R peak interval of a pulse wave curve for calculating a heart rate value. If the difference between the RR peak interval for the calculated heart rate value and the RR peak interval for the candidate data is within the specified range, the electronic device extracts and updates feature points from information related to heart rate calculation in operation S1550, and updates the calculated heart rate value. Biometric information data can be generated on the basis of it. When the difference between the RR peak interval for the calculated heart rate value and the RR peak interval for the candidate data is outside the specified range (or greater than the value in the specified range), the electronic device determines the heart rate value based on the candidate data in operation S1555. It is possible to generate biometric information data replaced (corrected) with the corrected value.
  • the electronic device may repeatedly perform the operation of operation S1510.
  • the electronic device may allow only candidate data having a similar R-R peak interval value to remain among the generated candidate data.
  • the electronic device may store the R-R peak interval value of the surviving candidate data and apply the stored R-R peak interval value to the next generation of candidate data.
  • FIG. 16 illustrates an example of a screen 1600 displayed by an electronic device (eg, the electronic device 101 of FIG. 1 and the electronic device 300 of FIG. 3) according to an embodiment.
  • an electronic device eg, the electronic device 101 of FIG. 1 and the electronic device 300 of FIG. 3
  • the screen 1600 displayed on the display (for example, the display 340 of FIG. 3) by the electronic device includes an image 1610 captured by a camera (eg, the camera 330 of FIG. 3) and the guide image content ( 1620).
  • the guide image content 1620 may include, for example, a video stored in an electronic device or streamed from an external server.
  • the screen 1600 may include a heart rate display area 1630 indicating a heart rate value measured based on image data.
  • a heart rate display area 1630 indicating a heart rate value measured based on image data.
  • the screen 1600 displays a reference heart rate display area (not shown) representing a range of normal heart rate values for users having the same attributes as user information (eg, gender, age, height, or weight).
  • the reference heart rate display area may be expressed in various ways according to exemplary embodiments. For example, the reference heart rate display area may be displayed on the heart rate display area 1630 to be compared with the measured heart rate value.
  • the screen 1600 may include a calorie display area 1640 that displays a calorie consumption amount calculated based on a heart rate value.
  • the electronic device may calculate a more accurate amount of heat consumption by using the heart rate value measured during exercise.
  • the electronic device may calculate the amount of heat consumed based on a type of exercise and a heart rate value corresponding to the guide image content 1620. For example, the electronic device may calculate the amount of heat consumed based on Equation 1 below.
  • M count is the oxygen consumption coefficient by exercise
  • weight is the user's weight
  • HR is the heart rate
  • HR REST is the resting heart rate
  • k AGE is the age correlation coefficient.
  • FIG. 17 illustrates an electronic device (eg, the electronic device 101 of FIG. 1, the electronic device 300 of FIG. 3) according to an embodiment together with a guide image content (eg, the guide image content 1620 of FIG. 16 ).
  • a guide image content eg, the guide image content 1620 of FIG. 16 .
  • the processor of the electronic device eg, the processor 120 of FIG. 1, the processor 310 of FIG. 3
  • the memory eg, the memory 130 of FIG. 1, FIG. It can be understood that it is executed by executing instructions stored in the memory 320 of 3.
  • the electronic device displays image data (eg, the image (eg, the image of FIG. 16)) including an image captured through a camera (eg, the camera 330 of FIG. 3) on the display (eg, the display 340 of FIG. 1610)) and guide video content can be output. Also, the electronic device may calculate a heart rate value based on image data including the captured image.
  • image data eg, the image (eg, the image of FIG. 16)
  • the electronic device may calculate a heart rate value based on image data including the captured image.
  • the electronic device may determine whether the heart rate value is normally calculated. When the heart rate value is normally calculated, the electronic device may generate biometric information data in operation S1750 based on the calculated heart rate value.
  • the electronic device may acquire the normal heart rate change amount from the heart rate change amount data of the exercise program.
  • the heart rate change amount data may be big-data in which measured data is accumulated.
  • the electronic device may acquire information about an exercise guided by the guide image content (eg, exercise intensity, exercise portion, and type of exercise such as strength exercise or aerobic exercise).
  • the electronic device may analyze an image acquired through a 3D camera to obtain information on exercise (eg, type of exercise, number of repetitions of exercise). The electronic device may search for information on the amount of change in heart rate measured when performing the same exercise from the data on the amount of change in heart rate, based on the information on the exercise.
  • the electronic device may determine a correction value based on the heart rate change amount and the previous heart rate value. For example, in operation S1720, the electronic device may determine a correction value as a changed value based on a heart rate change amount from a heart rate value normally acquired before a heart rate value that is a target of determination in operation S1720. When the correction value is determined, in operation S1750, the electronic device may generate the corrected biometric information data based on the correction value.
  • the process illustrated by flow chart 1700 may be executed repeatedly while measuring heart rate values.
  • FIG. 18 is a flowchart illustrating a process of providing recommended exercise intensity information based on biometric information data by an electronic device (eg, the electronic device 101 of FIG. 1 and the electronic device 300 of FIG. 3) according to an embodiment (1800).
  • the processor of the electronic device eg, the processor 120 of FIG. 1, the processor 310 of FIG. 3
  • the memory eg, the memory 130 of FIG. 1, FIG. It can be understood that it is executed by executing instructions stored in the memory 320 of 3.
  • the electronic device may acquire exercise type information of the user.
  • the exercise type information may mean information indicating what kind of exercise the user is performing (eg, exercise intensity, exercise part, and type of exercise such as strength exercise or aerobic exercise).
  • the electronic device may acquire a reference heart rate change amount corresponding to the exercise type information. For example, the electronic device may search for a reference heart rate change amount related to exercise type information from the database.
  • the reference heart rate change amount may mean information on the amount of change in heart rate measured when a user or another person performs an exercise corresponding to the exercise type information.
  • the electronic device may compare the obtained reference heart rate change amount with the change amount of the heart rate value included in the generated biometric information data.
  • the electronic device may determine recommended exercise intensity information based on the comparison result. For example, when the amount of change in the heart rate value is smaller than the reference amount of change in heart rate by a threshold value or more, the electronic device may determine information for increasing the exercise intensity as the recommended exercise intensity information. Conversely, when the amount of change in the heart rate value is larger than the reference amount of change in heart rate by a threshold value or more, the electronic device may determine information to reduce the exercise intensity or recommend rest as the recommended exercise intensity information. In operation S1840, the electronic device may output a notification for delivering recommended exercise intensity information to the user in a designated form (eg, video or sound).
  • a designated form eg, video or sound
  • the electronic device may determine recommended exercise intensity information by further considering information on the user's exercise state according to the user's movement in the amount of change in the heart rate value. For example, if the user's movement is not matched with the reference data specified in the electronic device's program (that is, the user's exercise movement is not appropriate), and the change in the heart rate value is less than the threshold value, the electronic device determines the exercise intensity. You can determine the recommended exercise intensity information that you do not want to change. When the user's movement matching the specified reference data in the program of the electronic device is detected above the specified threshold, and the increase in the user's heart rate value is less than the specified threshold, the electronic device is recommended to increase the user's exercise intensity. Strength information can be determined.
  • FIG. 19 is a flowchart illustrating a process of providing recommended exercise information based on biometric information data by an electronic device (eg, the electronic device 101 of FIG. 1 and the electronic device 300 of FIG. 3) according to an embodiment ( 1900).
  • the processor of the electronic device eg, the processor 120 of FIG. 1, the processor 310 of FIG. 3
  • the memory eg, the memory 130 of FIG. 1, FIG. It can be understood that it is executed by executing instructions stored in the memory 320 of 3.
  • the electronic device may acquire user information on the user.
  • the user information may mean information indicating an attribute of a user.
  • the user information may include information on an exercise performed by the user, information on at least one of sex, age, weight, and height.
  • the electronic device may determine recommended exercise information based on user information and biometric information data.
  • the electronic device may compare an average body signal of a user having the same or similar condition as the user information with biometric information data. For example, if the amount of change in heart rate of the biometric data data is smaller than the amount of change in average heart rate information corresponding to user information, the electronic device may determine that the user's exercise ability is high.
  • the electronic device may output information on the determined exercise capability of the user and determine an exercise program according to the determined exercise capability value.
  • the electronic device determines that the lower body exercise ability is low and recommends an exercise program that can improve the lower body exercise ability Can be determined as exercise information.
  • the electronic device may output the determined recommended exercise information.
  • FIG. 20 is a diagram illustrating an electronic device (eg, the electronic device 300 of FIG. 3) according to an exemplary embodiment in units of modules.
  • An electronic device includes a camera input unit 2010, an image preprocessor 2020, a biosignal processing unit 2030, a correction module 2040, a recalibration module 2050, and an output unit 2060. Can be.
  • the camera input unit 2010 may acquire an image by photographing a user's movement.
  • the image preprocessor 2020 may recognize whether a user is captured in an image input through the camera input unit 2010. When an area in which the user is photographed is recognized in the image, the image preprocessor 2020 may separate an area necessary for measuring a biosignal from the image.
  • the biosignal processing unit 2030 may include an image segmentation unit 2031, a signal generation unit 2032, a signal amplification unit 2033, and a biosignal extraction unit 2034.
  • the biosignal processing unit 2030 may extract a biosignal from an area separated by the image preprocessor 2020.
  • the image segmentation unit 2031 may separate and classify an image channel required for extracting a bio-signal, and select a color channel having a high effective value.
  • the signal preprocessor 2032 may exclude a color value unnecessary for extracting the bio-signal (eg, a color value in which the reflected light is photographed) and select a color channel necessary for extracting the bio-signal.
  • the signal amplifying unit 2033 may amplify a value of the selected color channel.
  • the biosignal extractor may remove noise from the amplified color channel and extract a heart rate value.
  • the correction module 2040 corrects the biosignal. can do.
  • the correction module 2040 may correct the biosignal based on the data model correction unit 2041 or the pattern information correction unit 2042.
  • the data model correction unit 2041 may correct the biosignal based on the data model modeled based on the feature points.
  • the pattern information correction unit 2042 may correct the biosignal based on the data pattern stored in the pattern database 2043.
  • the correction module 2040 may perform correction using the data model correction unit 2041 for a section in which the biosignal is not normally extracted for a short period (eg, 2 seconds).
  • the correction module 2040 may perform correction using the pattern information correction unit 2042 for a section in which the biosignal is not normally extracted for a long period (eg, 2 minutes).
  • the recalibration module 2050 performs the correction of the biosignal corrected by the correction module 2040. At least some can be recalibrated.
  • the output unit 2060 may output information based on the measured or corrected biosignal.
  • an electronic device including a memory (eg, the memory 130 of FIG. 1, the memory 320 of FIG. 3) and a processor connected to the memory (eg, the processor 120 of FIG. 1)
  • the processor is a camera (eg, the camera module 180 of FIG. 1, the camera 330 of FIG. 3, the camera 530 of FIG. 5, the camera device 630 of FIG. 6, the camera device 730 of FIG.
  • the processor may extract a feature point based on heart rate data calculated before the section, generate modeled candidate data based on the feature point, and determine a correction value based on the candidate data.
  • the processor may compare the calculated heart rate value with candidate data, and determine the correction value based on the comparison result.
  • the processor uses the calculated heart rate value when the difference between the RR peak interval for the calculated heart rate value and the RR peak interval for the candidate data is within a specified range as the normally acquired heart rate value, and the image data is It can be determined by the biometric information data at the time of acquisition.
  • the processor may include the candidate data as the correction value in the biometric information data when the difference between the R-R peak interval for the calculated heart rate value and the R-R peak interval for the candidate data is greater than a specified range.
  • the electronic device may further include a pattern database storing a plurality of heart rate data patterns.
  • the processor determines a correction value based on the candidate data, and when the size of the section is greater than the threshold value, the processor is based on the similarity between the heart rate data and the plurality of heart rate data patterns.
  • the correction value may be determined based on the selected heart rate data pattern.
  • the biometric information data at the second time point is the heart rate value.
  • the biometric information data can be corrected to have a linear relationship with.
  • the processor may determine the first region captured by the user in the image data by convolving images included in the image sequence of the image data.
  • the processor determines a second region including pixels within a specified range, among pixels included in the first region, and a heart rate value based on the pixel value included in the second region. Can be calculated.
  • the electronic device may further include a display (eg, the display device 160 of FIG. 1 and the display 340 of FIG. 3 ).
  • the processor may be configured to control the display to display an image included in the image data and guide image content, and to determine the correction value based on a heart rate change amount associated with the guide image content.
  • the processor may control the display to output recommended exercise intensity information determined based on a comparison result of a change amount of a heart rate value included in the biometric information data and a reference heart rate change amount.
  • the processor may control the display to obtain user information related to image data and to output biometric information data and recommended exercise information determined based on the user information.
  • the processor may control the display to calculate the calorie consumption amount based on the biometric information data and output the calorie consumption amount.
  • An electronic device for example, the electronic device 101 of FIG. 1, the electronic device 300 of FIG. 3, the electronic device 500 of FIG. 5, the server 610 of FIG. 6, and the electronic device of FIG. 7)
  • the method of operating (700)) is a camera (e.g., the camera module 180 of FIG. 1, the camera 330 of FIG. 3, the camera 530 of FIG. 5, the camera device 630 of FIG. 6, and 7
  • an operation of determining a correction value for a section in which the heart rate value is not normally obtained and an operation of generating biometric information data including the correction value may be included.
  • an operation of extracting a feature point based on heart rate data calculated before the section and an operation of generating modeled candidate data based on the feature point are further included, and the operation of determining the correction value is performed on the candidate data. It may include an operation of determining a correction value based on it.
  • the determining of the correction value based on the candidate data may include comparing the calculated heart rate value with the candidate data, and determining the correction value based on the comparison result.
  • the comparing operation may include comparing an R-R peak interval for the calculated heart rate value and an R-R peak interval for the candidate data.
  • the operation of determining the correction value is a comparison result, when the difference between the two values is within a specified range, the calculated heart rate value is determined as the biometric information data at the time the image data is acquired as a normally acquired heart rate value, and the difference between the two values is specified. If it is out of the range, an operation of including the candidate data as the correction value in the biometric information data may be included.
  • the operation of calculating a heart rate value by analyzing image data includes an operation of determining a first region photographed by a user in the image data by convolving images included in an image sequence of the image data. An operation of determining a second region including pixels whose pixel value is within a specified range among pixels included in the first region, and an operation of calculating the heart rate value based on the pixel value included in the second region. I can.
  • the method of operating the electronic device further includes an operation of displaying an image and guide image content included in the image data, and the operation of determining a correction value includes acquiring a heart rate change amount associated with the guide image content. It may include an operation and an operation of determining the correction value based on the amount of change in heart rate.
  • a computer-readable recording medium eg, nonvolatile memory 134, memory 320 in which one or more programs including a plurality of instructions executed by at least one processor according to an embodiment are recorded, is executed.
  • a camera e.g., the camera module 180 of FIG. 1, the camera 330 of FIG. 3, the camera 530 of FIG. 5, the camera device 630 of FIG. 6, the camera device 730 of FIG. 7) Acquiring the image data captured by using, calculating the heart rate value by analyzing the image data, determining whether or not the heart rate value is normally acquired, and when the heart rate value is not acquired normally, the heart rate value is normally acquired.
  • a program for determining a correction value for an unchanged section and an operation for generating biometric information data including the correction value may be recorded.
  • the electronic device may include a memory for storing heart rate data of an arrhythmia patient and a processor connected to the memory.
  • the processor acquires biometric information data including a heart rate value, and if the biometric information data does not contain a heart rate value or there is a section other than the normally calculated value, the value of the section is corrected to a correction value, and the corrected
  • biometric information data and the heart rate data of the arrhythmia patient are compared, and the number of times the corrected biometric information data includes the abnormal pattern included in the heart rate data of the arrhythmia patient is greater than or equal to the threshold value, information on the abnormal heart rate pattern Can be configured to output.
  • a computer-readable storage medium storing one or more programs (software modules) may be provided.
  • One or more programs stored in a computer-readable storage medium are configured to be executable by one or more processors in an electronic device (device).
  • the one or more programs include instructions that cause the electronic device to execute methods according to embodiments described in the claims or specification of the present disclosure.
  • These programs include random access memory, non-volatile memory including flash memory, read only memory (ROM), and electrically erasable programmable ROM.
  • EEPROM electrically erasable programmable read only memory
  • magnetic disc storage device compact disc-ROM (CD-ROM), digital versatile discs (DVDs), or other types of It may be stored in an optical storage device or a magnetic cassette. Alternatively, it may be stored in a memory composed of a combination of some or all of them. In addition, a plurality of configuration memories may be included.
  • the program is through a communication network composed of a communication network such as the Internet, an intranet, a local area network (LAN), a wide LAN (WLAN), or a storage area network (SAN), or a combination thereof. It may be stored in an accessible storage device. Such a storage device may access a device performing an embodiment of the present disclosure through an external port. In addition, a separate storage device on the communication network may access a device performing an embodiment of the present disclosure.
  • a communication network such as the Internet, an intranet, a local area network (LAN), a wide LAN (WLAN), or a storage area network (SAN), or a combination thereof. It may be stored in an accessible storage device. Such a storage device may access a device performing an embodiment of the present disclosure through an external port.
  • a separate storage device on the communication network may access a device performing an embodiment of the present disclosure.

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Medical Informatics (AREA)
  • Veterinary Medicine (AREA)
  • Physics & Mathematics (AREA)
  • Public Health (AREA)
  • Biophysics (AREA)
  • Pathology (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Surgery (AREA)
  • Physiology (AREA)
  • Cardiology (AREA)
  • Physical Education & Sports Medicine (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Dentistry (AREA)
  • Artificial Intelligence (AREA)
  • Psychiatry (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • Signal Processing (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nursing (AREA)
  • Human Computer Interaction (AREA)
  • Measuring And Recording Apparatus For Diagnosis (AREA)

Abstract

Selon divers modes de réalisation, l'invention concerne un dispositif électronique pouvant comprendre : une mémoire pour stocker des instructions ; et un processeur connecté à la mémoire. Le processeur peut être conçu pour : acquérir des données d'image capturées à l'aide d'une caméra ; calculer une valeur de fréquence cardiaque par analyse des données d'image ; déterminer si la valeur de fréquence cardiaque a été normalement acquise ou non ; lorsque la valeur de fréquence cardiaque n'a pas été normalement acquise, déterminer une valeur de correction pour une section dans laquelle la valeur de fréquence cardiaque n'a pas été normalement acquise ; et stocker, dans la mémoire, des données biométriques comprenant la valeur de correction.
PCT/KR2020/014098 2019-11-18 2020-10-15 Appareil d'acquisition de données biométriques et procédé associé Ceased WO2021101073A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR1020190148077A KR20210060246A (ko) 2019-11-18 2019-11-18 생체 데이터를 획득하는 장치 및 그 방법
KR10-2019-0148077 2019-11-18

Publications (1)

Publication Number Publication Date
WO2021101073A1 true WO2021101073A1 (fr) 2021-05-27

Family

ID=75981322

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2020/014098 Ceased WO2021101073A1 (fr) 2019-11-18 2020-10-15 Appareil d'acquisition de données biométriques et procédé associé

Country Status (2)

Country Link
KR (1) KR20210060246A (fr)
WO (1) WO2021101073A1 (fr)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102593722B1 (ko) * 2021-08-30 2023-10-24 연세대학교 산학협력단 딥러닝 기반 재실자 맞춤형 대사량 예측시스템 및 예측방법
KR102545641B1 (ko) * 2021-10-20 2023-06-20 고려대학교 산학협력단 프라이버시를 보호하는 심전도 기반의 개인 인증 기법
KR102588481B1 (ko) * 2023-03-23 2023-10-12 상명대학교산학협력단 산소포화도 추정 방법 및 장치

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20130044856A (ko) * 2011-10-25 2013-05-03 주식회사 라이프사이언스테크놀로지 보행보조기 사용자의 생체신호 보정 획득방법
KR20150093036A (ko) * 2014-02-06 2015-08-17 한국전자통신연구원 생체 정보 측정장치 및 측정방법
KR20160054325A (ko) * 2014-11-06 2016-05-16 장재윤 맞춤형 개인 트레이닝 관리 시스템 및 방법
JP6135255B2 (ja) * 2013-04-02 2017-05-31 富士通株式会社 心拍測定プログラム、心拍測定方法及び心拍測定装置
WO2019003549A1 (fr) * 2017-06-28 2019-01-03 ソニー株式会社 Dispositif de traitement d'informations, procédé de traitement d'informations, et programme

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20130044856A (ko) * 2011-10-25 2013-05-03 주식회사 라이프사이언스테크놀로지 보행보조기 사용자의 생체신호 보정 획득방법
JP6135255B2 (ja) * 2013-04-02 2017-05-31 富士通株式会社 心拍測定プログラム、心拍測定方法及び心拍測定装置
KR20150093036A (ko) * 2014-02-06 2015-08-17 한국전자통신연구원 생체 정보 측정장치 및 측정방법
KR20160054325A (ko) * 2014-11-06 2016-05-16 장재윤 맞춤형 개인 트레이닝 관리 시스템 및 방법
WO2019003549A1 (fr) * 2017-06-28 2019-01-03 ソニー株式会社 Dispositif de traitement d'informations, procédé de traitement d'informations, et programme

Also Published As

Publication number Publication date
KR20210060246A (ko) 2021-05-26

Similar Documents

Publication Publication Date Title
WO2020105841A1 (fr) Liquide de traitement à haute température avec de la nanocellulose
WO2019088769A1 (fr) Procédé et système de fourniture d'informations médicales basé sur une api ouverte
WO2017209528A1 (fr) Appareil électronique et son procédé de fonctionnement
WO2019039716A1 (fr) Dispositif électronique et son procédé de mesure de contrainte
WO2022131657A1 (fr) Dispositif électronique pour fournirun guide d'exercice et son procédé de fonctionnement
WO2019078507A1 (fr) Dispositif électronique et procédé de fourniture d'un indice de stress correspondant à l'activité d'un utilisateur
WO2020189930A1 (fr) Dispositif électronique et procédé de notification d'informations biométriques par un dispositif électronique
WO2016098950A1 (fr) Procédé pour mettre à jour un modèle de caractéristique biologique, et dispositif électronique associé
WO2021101073A1 (fr) Appareil d'acquisition de données biométriques et procédé associé
US11832925B2 (en) Electronic device for measuring biometric information and method for operating the same
WO2018164411A1 (fr) Dispositif électronique comprenant un module de caméra, et procédé de commande d'un dispositif électronique
WO2019194651A1 (fr) Procédé et dispositif de mesure d'informations biométriques dans un dispositif électronique
WO2023033286A1 (fr) Appareil électronique pour soins de santé mentale et son procédé de commande
WO2019093637A1 (fr) Dispositif électronique et système de module de charge le comprenant
WO2021045375A1 (fr) Dispositif électronique et procédé pour obtenir un signe vital
WO2020209618A1 (fr) Dispositif électronique et procédé de fourniture d'informations pour soulager le stress par ce dernier
WO2018021723A1 (fr) Procédé et appareil d'affichage continu d'images sur la base de la similarité d'images
WO2022124647A1 (fr) Procédé de détection de l'apnée du sommeil et dispositif électronique le prenant en charge
WO2019164126A1 (fr) Dispositif électronique et procédé de fourniture d'informations sur l'état cardiovasculaire d'un utilisateur
WO2019221438A1 (fr) Dispositif électronique de mesure de la tension artérielle et son procédé de fonctionnement
WO2020242087A1 (fr) Dispositif électronique et procédé de correction de données biométriques sur la base de la distance entre le dispositif électronique et l'utilisateur, mesurée à l'aide d'au moins un capteur
WO2021080292A1 (fr) Dispositif électronique pour intégrer et présenter des données physiologiques entre de multiples dispositifs et son procédé
WO2018021726A1 (fr) Dispositif électronique et procédé de commande d'activation d'un module de caméra
WO2024210322A1 (fr) Dispositif électronique, serveur et système pour fournir des biosignaux hautement précis sur la base d'informations acquises sans contact, et procédé de fonctionnement associé
WO2021080369A1 (fr) Dispositif de vérification d'utilisateur en utilisant des données d'électrocardiogramme et procédé associé

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20889793

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20889793

Country of ref document: EP

Kind code of ref document: A1