KR20160054397A - The method and apparatus for early warning the danger - Google Patents

Abstract

A method for acquiring image information of a target object through a first wearable device worn by a user, detecting physical characteristics of the target object based on the obtained image information, and determining a degree of threatening the safety of the user using the detected physical characteristics Disclosed is a method and apparatus for early warning of a risk, which determines a risk, and warns the user based on the risk of the determined event.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and apparatus for early warning of danger,

And to methods and apparatus for early warning of danger.

There are some wearable devices, such as a head-mounted display (HMD), capable of detecting dangerous objects such as vehicles and capable of transmitting warning information. In such a device, an image signal is used to detect a danger from the vehicle. In other wearable devices, the audio signal is used to detect the danger and provide warning information.

And to methods and apparatus for early warning of danger. It also provides a computer program stored on a medium in combination with hardware for processing the method.

A method for early warning a wearable device according to one aspect includes obtaining image information of a target object, detecting physical characteristics of the target object based on the acquired image information, Determining a degree of risk that is a degree that threatens the safety of the user, and providing an alert to the user based on the risk of the determined event.

The obtaining step may acquire surrounding audio information, and the detecting step may detect the audio characteristic based on the obtained audio information.

The step of determining the risk also includes the steps of determining the time taken for the target object to reach the wearable device based on the image information and determining a time period for the target object to reach the wearable device, And determining a risk for the target object according to at least one of the characteristics.

The physical properties of the target object may also include a person's face, trunk, limb, animal's teeth, tail, or eyes.

The audio properties of the target object also include at least one of volume, pitch, Mel Frequency Cepstrum Coefficient (MFCC), human conversation and Power Normalized Cepstral Coefficient (PNCC) .

The acquiring step may also include acquiring weather information or odor information, and the step of determining the degree of risk may determine the degree of risk based on the weather information or the odor information.

The step of providing an alert may also provide an alert in a different manner depending on the risk.

And receiving image information or audio information obtained through the second wearable device, wherein the step of determining the risk can determine the risk based on the information received from the second wearable device.

The step of detecting includes detecting whether a target object is a person using image information and detecting a face of a person if the target object is a person, The risk can be determined according to whether or not it is in the set face database.

A wearable device for early warning a danger according to another aspect includes a sensing unit for acquiring image information of a target object, a characteristic detector for detecting a physical characteristic of the target object based on the acquired image information, A risk determination unit that determines a degree of risk that is a degree that threatens the safety of the user, and a warning unit that warns the user based on the risk of the determined event.

Further, the sensing unit acquires the surrounding audio information, and the characteristic detecting unit can detect the audio characteristic based on the obtained audio information.

The risk determining unit determines the time taken for the target object to reach the wearable device based on the image information, and determines at least one of the physical characteristics of the target object, the time taken by the target object to reach the wearable device, The risk for the target object can be determined.

The physical properties of the target object may also include a person's face, trunk, limb, animal's teeth, tail, or eyes.

The audio properties of the target object also include at least one of volume, pitch, Mel Frequency Cepstrum Coefficient (MFCC), human conversation and Power Normalized Cepstral Coefficient (PNCC) .

Further, the sensing unit may acquire weather information or odor information, and the risk determining unit may determine the risk based on weather information or odor information.

The alerting unit can also provide alerts in different ways depending on the risk.

And a receiving unit that receives image information or audio information acquired through the second wearable device, and the risk determining unit may determine the risk based on the information received from the second wearable device.

The detection unit detects whether the target object is a person using image information, detects a face of a person if the target object is a person, and the risk determination unit determines a risk level according to whether the detected face is in a predetermined face database You can decide.

1 is a block diagram illustrating an apparatus for early warning of a danger according to one embodiment.
2 is a block diagram illustrating an apparatus for early warning of a risk according to another embodiment.
3 is a block diagram illustrating an apparatus for alerting a risk early in accordance with another embodiment.
4 is a flow chart illustrating a method for early warning a risk according to an embodiment.
5 is a flow chart for explaining a method for early warning a risk according to another embodiment.
6 is a flow chart illustrating an example of a method for early warning a risk according to an embodiment.

FIG. 7 is a flow chart for explaining another example of a method for early warning a danger.
Fig. 8 is a flow chart for explaining another example of a method for early warning a danger.
Fig. 9 is a flowchart for explaining an example of implementing a method for early warning a risk.

Brief Description of the Drawings The advantages and features of the present invention, and how to accomplish them, will become apparent with reference to the embodiments described hereinafter with reference to the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. To fully disclose the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims.

The terms used in this specification will be briefly described and the present invention will be described in detail.

While the present invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not limited to the disclosed embodiments. Also, in certain cases, there may be a term selected arbitrarily by the applicant, in which case the meaning thereof will be described in detail in the description of the corresponding invention. Therefore, the term used in the present invention should be defined based on the meaning of the term, not on the name of a simple term, but on the entire contents of the present invention.

When an element is referred to as "including" an element throughout the specification, it is to be understood that the element may include other elements as well, without departing from the spirit or scope of the present invention. Also, as used herein, the term "part " refers to a hardware component such as software, FPGA or ASIC, and" part " However, "part" is not meant to be limited to software or hardware. "Part" may be configured to reside on an addressable storage medium and may be configured to play back one or more processors. Thus, by way of example, and not limitation, "part (s) " refers to components such as software components, object oriented software components, class components and task components, and processes, Subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables. The functions provided in the components and "parts " may be combined into a smaller number of components and" parts " or further separated into additional components and "parts ".

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. In order to clearly explain the present invention in the drawings, parts not related to the description will be omitted.

1 is a block diagram illustrating an apparatus for early warning of a danger according to one embodiment.

Referring to FIG. 1, a wearable device 100 for early warning a danger includes a sensing unit 110, a characteristic detection unit 120, a risk determination unit 130, and a warning unit 140. The wearable device may include a head mount display, a smart glass or a smart watch.

The sensing unit 110 may acquire image information of a target object. Also, the sensing unit 110 can acquire surrounding audio information. And the sensing unit 110 can acquire weather information or odor information. In addition, the sensing unit 110 can acquire biometric information of the user through the first wearable device.

The characteristic detecting unit 120 may detect a physical property of the target object based on the obtained image information. In addition, the characteristic detecting unit 120 can detect the audio characteristic based on the obtained audio information. The audio properties of the target object may include at least one of volume, pitch, Mel Frequency Cepstrum Coefficient (MFCC), human conversation and Power Normalized Cepstral Coefficient (PNCC) have. The characteristic detecting unit 120 can detect whether the target object is a person using image information. The characteristic detecting unit 120 can detect a face of a person if the target object is a person.

The risk determining unit 130 can determine a risk level that is a degree that threatens the safety of the user by using the detected physical characteristics. Also, the risk determination unit 130 may determine, based on the image information, the time taken for the target object to reach the first wearable device. The risk determining unit 130 may determine the risk to the target object according to at least one of the physical characteristics of the target object, the time taken for the target object to reach the first wearable device, and the audio characteristics of the target object. Also, the risk determination unit 130 can determine the risk based on weather information or odor information. For example, on a rainy day, you have to be careful about slipping, which can increase your risk. Or high concentrations of toxic gases in the air can increase the risk. The risk determination unit 130 can determine the risk based on the biometric information. Also, the risk determining unit 130 can determine the risk according to whether the detected face is in the predetermined face database.

The warning unit 140 may provide a warning to the user based on the risk of the determined event. The warning unit 140 can also provide a warning based on the biometric information. For example, a warning can be provided when the user's blood pressure drops sharply. In addition, the warning unit 140 may change the warning according to the risk. For example, if you are at low risk, you can be alerted with a weak vibration, and if you are at high risk, you can give a warning with a strong vibration. Or if the risk is low, the lamp blinks slowly, and if the risk is high, the lamp blinks quickly. Another example is to repeat the warning sound slowly if the risk is low, and to quickly repeat the warning if the risk is high. In addition, it is possible to provide an alarm in a different manner depending on various risks.

2 is a block diagram illustrating an apparatus for early warning of a risk according to another embodiment.

Referring to FIG. 2, an apparatus 200 for early warning of a danger includes a sensing unit 110, a characteristic detecting unit 120, a risk determining unit 130, a warning unit 140, and a receiving unit 210.

The description of the sensing unit 110, the characteristic detection unit 120, the risk determination unit 130, and the warning unit 140 of FIG. 2 is the same as that of FIG. 1. The receiving unit 210 can receive the information acquired through the second wearable device. At this time, the information may include at least one of biometric information of the second user and image information and audio information obtained by the second wearable device. The risk determination unit 130 may also determine the risk based on the information received from the second wearable device.

In addition, the possibility of fraud can be determined by analyzing conversation contents, facial features, or motion transitions of the target object based on the fraud reference model. The risk determination unit 130 can determine the risk based on the probability of the determined fraud.

3 is a block diagram illustrating an apparatus for alerting a risk early in accordance with another embodiment.

3, the wearable device 300 may include a sensing unit 310, a processing unit 320, and an alarm unit 330. At this time, the wearable device 300 may correspond to the wearable device 100 that warns of an early danger shown in FIG.

The sensing unit 310 may include various sensors for sensing different types of information. Generally, the sensing unit 310 includes an image sensor 312 audio sensor 314, a humidity sensor 316, and an odor sensor 318.

An image sensor, also known as a photosensitive element, is an apparatus for converting an optical image into an electronic signal. In this embodiment, the image sensor 312 can be used to obtain real-time image information of the surrounding environment. The wearable device 300 may also include a camera, and the image sensor 312 may be included in the camera. The real-time image information may include any surrounding information that may threaten the safety of the user. For example, real-time image information is not limited to road conditions, vehicle operating conditions, and information from nearby creatures, including humans and animals.

The audio sensor 314 may comprise an array of pick-up devices or pick-ups. In this embodiment, the audio sensor 314 may be used to obtain real-time audio information of the surrounding environment. The real time audio information may include any surrounding information that may jeopardize the safety of the user. For example, the real-time audio information may include, but is not limited to, the horn of the vehicle, the thunder, the human voice, the animal scream, and the like.

In addition, the sensing unit 310 may include another type of sensor. For example, the sensing unit 310 may comprise a humidity sensor 316, an odor sensor 318, a speed sensor (not shown), an atmospheric pressure sensor (not shown), and the like. The humidity sensor 316 may be used to sense humidity to determine the weather conditions of the surrounding environment. The odor sensor 318 may be used, for example, to detect the odor of the ambient environment or some type of gas. For example, this gas can be a gas that contains dangerous chemicals such as carbon monoxide fumes that jeopardize the safety of the user. The speed sensor may be used, for example, to sense the speed of the user wearing the wearable device 300. [ The atmospheric pressure sensor can be used, for example, to sense atmospheric pressure to determine the weather conditions of the surrounding environment. Those skilled in the art will appreciate that the sensing portion 310 may include any type of sensors that have already been made or any type of sensors for sensing the environmental conditions to be developed in the future. On the other hand, the wearable device 300 may further include a network communication unit 350 that can be connected to a network such as the Internet 20 in order to acquire information. The wearable device 300 can receive the weather forecast information in order to acquire weather information of the surrounding environment in the network by the network communication unit 350.

In addition, the wearable device 300 may further include a biometric information sensor 360 in addition to the sensing unit 310. The bio-information sensor 360 is a device used to detect a biological material and convert the concentration of a substance into an electrical signal.

In one embodiment, the bio-information sensor 360 may be used to sense biometric information of the user 10 (i.e., user) wearing the wearable device 300. [ The biometric information of the user may include various information indicating a physiological state of the user. For example, the biometric information may include, but is not limited to, heart rate, blood pressure, body temperature, respiration rate.

In the wearable device 300, the processing unit 320 is used to determine the risk level based on various pieces of information acquired through the wearable device. In response to the risk, the processing unit 320 may provide warning information. For example, the processing unit 320 may analyze image information and audio information, and may determine a risk based on the analysis.

In one embodiment, the processing unit 320 may analyze the image information to acquire information that may jeopardize the safety of the user. The processing unit 320 may include the characteristic detecting unit 120 of FIGS. 1 and 2. The processing unit 320 may detect the physical characteristics of the target object based on the environmental image information transmitted by the sensing unit 310. For example, the target object may be a vehicle, an animal, or a person. The vehicle may also include, but is not limited to, trucks, cars, motorcycles. When the target object is an animal (e.g., a stray dog, a fierce dog, etc.), its physical characteristics may include, but are not limited to, tail and eye. When the target object is a person, the physical characteristics of the person include, but are not limited to, a person's face, limbs and torso. If the target object is moving, the processing unit 320 can determine the time taken for the target object to reach the wearable device, i.e., the time the target object took to reach the user.

In another embodiment, the processing unit 320 may analyze the audio information to acquire information that may jeopardize the safety of the user. For example, the processing unit 320 may extract the audio characteristics of the target object based on the audio information transmitted from the sensing unit 310. For example, the target object may include, but is not limited to, a vehicle, an animal, and a person. When the target object is a vehicle, the audio characteristics of the vehicle may include, but are not limited to, the horn sound of the vehicle. When the target object is an animal, the audio characteristics of the animal may include, but are not limited to, audio size, pitch, and Mel Frequency Cepstrum Coefficient (MFCC). When the target object is a person, the audio characteristics of a person may include, but are not limited to, speech normalized coefficients (PNCC, Power Normialized Cepstral Coefficient).

The processing unit 320 may also be configured based on image and audio analysis to determine the risk to the target object. For example, the processing unit 320 can determine a risk for a target object based on the following three items.

The three items are the physical characteristics of the target object, the time taken for the target object to reach the corresponding wearable device, and the audio characteristics of the target object.

In addition, the processing unit 320 can be formed based on various factors that adjust the risk level to be evaluated. In one embodiment, the processing unit 320 may determine a risk based on weather information or odor information sensed by the sensing unit 310.

For example, on rainy or snowy days, on slippery roads, the vehicle may slip and create unexpected hazards. Therefore, the risk can be increased. As another example, when poison gas is detected by the sensing unit 310, the risk may be increased. In another embodiment, the processing unit 320 may be based on physiological condition information of the user sensed by the biometric information sensor 360 For example, when the user's unfavorable physiological condition such as rapid heart rate, hypertension, shortness of breath, etc. is detected by the biometric information sensor 360, the risk may be increased.

The processing unit 320 is configured to provide warning information in response to a risk. In another embodiment, the processing unit 320 is configured to obtain feedback information of a user wearing the wearable device, and to provide warning information based on the feedback information. For example, when a user inputs information about a high-risk element, warning information can be provided according to a high risk. For example, you can enter the current weather, or enter your health status. In another embodiment, the early warning system may include a stored user preference database (e.g., user preference data such as reaction data or feedback data on various risks).

Through the user preference database search, the processing unit 320 can provide warning information in a manner suitable for a user or a similar user.

The warning unit 330 may issue a warning based on the warning information provided by the processing unit 320. [ Alerting section 330 may include various devices (e.g., speakers, displays, vibrators, etc.) that can interact with a user to provide another way of alerting.

The manner of warning can be set according to user preference. User preferences include, but are not limited to, visual, auditory or tactile modalities such as images, text, sounds, or vibrations. In some embodiments, wearable device 300 may include third party support functions. For example, the wearable device 300 may include a transmitter 340. The transmitting unit 340 can receive information from other wearable devices. (E.g., a wearable device worn by the user 40).

The above information includes, but is not limited to, biometric information of users wearing other wearable devices, information generated by other wearable devices (e.g., the risk generated by the wearable device's processing unit).

In addition, the processing unit 320 can determine the risk based on information from other wearable devices. Through information sharing, the user can be provided with an initial warning from a third party. Therefore, the safety of the user can be increased.

The early warning system may include a shared database 30 that allows the wearable device to upload its information to the shared database 30 to share information with other devices. The receiving unit 210 of FIG. 2 may be included in the transmitting unit 340.

In this embodiment, each wearable device can receive information from another wearable device via the shared database 30. [

In another embodiment, the connection may be established directly to transmit information between the two wearable devices. For example, a wearable device worn by an adult and a wearable device worn by a child can be used together by setting up a direct connection. Thus, the two wearable devices can share information with each other.

The wearable device 300 includes, but is not limited to, a head mount device, a wrist mount device, a smart bracelet, a smart wrist watch, and the like. A person skilled in the art can design a device in the wearable device 300 suitable for carrying out the detection via various sensors.

4 is a flow chart illustrating a method for early warning a risk according to an embodiment.

In step S400, the image information of the target object can be obtained. And can receive information acquired through the second wearable device worn by the second user. At this time, the information may include at least one of biometric information of the second user, image information, audio information, and information generated by the second wearable device. The weather information can be obtained by sensing the weather condition using the sensor or receiving the weather forecast through the network. The weather information or the smell information can be obtained. In addition, the biometric information of the first user acquired through the first wearable device can be received.

In step S410, the physical property of the target object can be detected based on the obtained image information.

In step S430, the risk may be determined based on the information received from the second wearable device. In step S430, based on the risk of the determined event, Thereby providing a warning to the user. The warning can be made different based on the biometric information.

5 is a flow chart for explaining a method for early warning a risk according to another embodiment.

In step S500, surrounding information can be acquired through the wearable device.

In step S510, information acquired through the second wearable device worn by the second user can be received. At this time, the information may include at least one of biometric information of the second user, information, and information generated by the second wearable device. The weather information can be obtained by sensing the weather condition using the sensor or receiving the weather forecast through the network. The information may include weather information or odor information. In addition, biometric information of the first user acquired through the first wearable device can be received.

In step S520, the risk level can be determined based on the acquired information. Also, the risk can be determined based on the information acquired through the second wearable device. The risk can be determined based on weather information or odor information. In addition, the physical characteristics of the target object can be detected, and the time taken for the target object to reach the corresponding wearable device based on the image information can be determined. And audio characteristics of the target object can be extracted based on the audio information. The risk for the target object can be determined according to at least one of the physical characteristics of the target object, the time taken for the target object to reach the corresponding wearable device, and the audio characteristics of the target object. The audio properties of the target object may include at least one of volume, pitch, Mel Frequency Cepstrum Coefficient (MFCC), human conversation and Power Normalized Cepstral Coefficient (PNCC) have.

In step S530, a warning can be provided according to the determined risk. Further, the means for providing the warning information based on the biometric information can be adjusted.

6 is a flow chart illustrating an example of a method for early warning a risk according to an embodiment.

In the following description, "first wearable device" refers to the wearable device mentioned. The "second wearable device" refers to another wearable device having a similar function in addition to the wearable device mentioned above. This is a way to distinguish the wearable device itself without limiting it.

In step S600, the information is acquired through the first wearable device. The information includes image information and audio information. For example, when a user is walking on the street or staying indoors, there may be some potential risk. For example, potential hazards may be due to vehicle hazards, hazards from nearby animals, toxic gas in the air, or potential fraud from strangers while driving.

In this embodiment, the wearable device worn by the user may obtain ambient information to detect possible potential risks. Generally, the information includes image information and audio information.

For example, when the user walks the street, the image information may include information of the vehicle, information of a person or an animal, information of the operation state of a person or an animal during driving the vehicle. The information of the plurality of images may include a direction of movement, a speed, and a target object. The image sensor 312 shown in FIG. 3 may be used to sense image information. The audio information may include the horn frequency of the vehicle, the human voice around the user, and the scream of the animal while driving the vehicle. The audio sensor 314 shown in FIG. 3 may be used to sense audio information.

The risk is determined based on the information obtained in step S610. For example, it is determined in detail whether the driving direction of the vehicle is directed toward the user or away from the user, and whether or not the distance between the vehicle and the user is too close to the safety of the user . The method also includes determining whether the nearby animal is in an anger state and whether it is possible to beat the user through the audio information.

In step S620, a warning is issued in accordance with the determined risk. For example, when the risk is high, it includes warning the user to leave the place or change the walking direction. In some embodiments, the alert method may be set according to the field or user preferences. In addition, the wearable device may store the user preference database and transmit the feedback information of the user for later use. (E. G., Response data or feedback data for various risks)

The wearable device may obtain user feedback information from a user preference database that stores user preferences and may provide a corresponding alert based on the feedback information. For example, the manner of the warning may include, but is not limited to, visual, auditory, and tactile. Thus, the content of the warning may include images, text, sounds, or any combination of the above-mentioned items. In a further embodiment, the warning may be different depending on the determined risk. For example, if the risk is low, the sound of the warning can be set to low volume and low frequency. If the risk is high, the sound of the warning can be set to high volume and high frequency. Also, if the risk is high, various ways of warning can be used at the same time, for example, vibration can be generated to warn the user, rather than a beep. In another embodiment, it is possible to adjust the warning method based on the biometric information of the user wearing the wearable device. Since the user's biometric information can reflect the psychological or mental state of the user, other biological states or mental states can lead to different responses and remedies for the oncoming risks.

Thus, based on other reactions and treatments, the wearable device may provide various ways of alerting the user. For example, if the user's mental state is not good, the user is likely to overlook the risk if the current user does not respond to the risk. In order to prevent possible damage, the manner of the warning operated by the wearable device may become more sensitive. For example, a wearable device can use a high warning volume, a fast frequency, and so on.

A method of warning danger early uses a wearable device to obtain information. The acquired information may include image information, audio information, and the like. The wearable device determines the risk based on the obtained information. The wearable device warns the user according to the determined risk. In summary, the method provides a user's comprehensive and rapid early warning of danger through various sensors. Thereby enhancing the safety of the user. In addition, the detection of danger includes the detection of animals, such as animals and people, as well as the detection of vehicles.

FIG. 7 is a flow chart for explaining another example of a method for early warning a danger.

In step S700, based on the image information, determines the time it takes for the target object to reach the corresponding wearable device. Image information is analyzed to obtain information that may jeopardize the safety of the user. The image information may not only be captured by the wearable device itself, but may also include images captured by other wearable devices. For example, when two devices are paired, these two devices can transmit data to each other. The data may include, but are not limited to, photographed image information, audio information, and information (e.g., determined risk) each generated by the wearable device.

In this embodiment, analyzing the image information may include detecting physical characteristics of the object of interest. In one embodiment, the sharing property may be used to detect various types of object objects. For example, the target object includes three types of automobiles, animals, and people. If the target object is an animal (e.g., a stray dog, a wild dog, etc.), the physical characteristics include, but are not limited to, an animal's teeth, tail, and eyes. When the target object is a person, the physical characteristics of the person include, but are not limited to, the face, torso, and limbs of a person. A person's physical feature information can be used to determine whether there is a possibility of fraud. The step of analyzing the image information may also include determining the time it takes for the target object to arrive at the corresponding wearable device based on the plurality of images.

For example, multiple images may be used to determine the distance between the target object and the wearable device, the direction of motion, and the velocity of the target object. Thus, the time taken for the target object to reach the corresponding wearable device can be calculated. Several methods can be used to calculate distance, direction of motion and speed. For example, the distance can be estimated by comparing the actual size of the target object with the image size of the target object in the image. For example, the direction of motion can be estimated according to the offset of the target object in the image. One skilled in the art can understand that if the direction of motion of the target object is not towards the user, then the target object is not at risk.

In one embodiment, analyzing the image information may analyze an image from another wearable device. For example, the first wearable device and the second wearable device are paired and share image information with each other. In this embodiment, the first wearable device can analyze the image information from the second wearable device and calculate the time it takes for the target object to reach the second wearable device. The first wearable device can then determine the time it takes for the target object to reach the first wearable device based on the mutual positional relationship of the first wearable device and the second wearable device. In another embodiment, the second wearable device may share the time it takes for the target object to reach the first wearable device and the second wearable device. Therefore, the amount of transmitted data can be reduced, and the processing efficiency can be improved.

In step S710, the audio property of the target object can be detected based on the audio information. The audio information is analyzed to obtain information that may jeopardize the safety of the user. Likewise, the audio information not only includes audio information captured by the wearable device itself, but also includes audio captured by other wearable devices. Specifically, analyzing the audio information includes detecting audio properties of the target object from the audio information.

Depending on the baseband decomposition and reconstruction technique of the speech signal, it is possible to separate the sound of interest (for example, the sounds of vehicles, animals or people) from the background sound. For example, if the target object is a vehicle, the audio characteristics include, but are not limited to, the horn frequency of the vehicle. If the target object is an animal, the audio properties include, but are not limited to, volume, pitch, and Mel-frequency spectral coefficients (MFCC). When the target object is a person, the audio characteristics of the person may include, but are not limited to, human conversation, power normalization coefficient coefficients (PNCC).

In step S720, the risk can be determined according to the physical characteristics of the target object, the time it takes for the target object to reach the corresponding wearable device, and the audio characteristics of the target object. The risk level for the target object is determined by analyzing the image and audio information . ≪ / RTI > In particular, the risk to the target object can be determined according to the physical characteristics of the target object and the time required for reaching the wearable device to which the target object corresponds.

In one embodiment, when the target object is a vehicle, it can be weighted and discretized by the time it takes for the vehicle to reach the user and the horn frequency of the vehicle. Thus, the risk level, i.e., the level that threatens the user, can be determined. In another embodiment, when the target object is an animal, the degree of threat of the animal (e.g., degree of anger and degree of ferocity) can be evaluated according to physical characteristics, audio characteristics and acoustic characteristics of the animal. On the other hand, the attack probability from the animal can be evaluated according to the time it takes for the animal to reach the user. Finally, threats and probabilities can be weighted and discretized to determine animal threats, or risk levels. In another embodiment, if the target object is a person, evaluating the risk level for the target object may include determining whether the target object is deceiving the user.

In step S730, the risk can be determined based on additional factors (weather information, odor information, biological information). The risk can be further adjusted based on a number of factors to provide an accurate early warning.

In one embodiment, the risk can be determined based on meteorological information or odor information. As mentioned above, the weather condition can be detected by the humidity sensor in the sensing portion of the wearable device. Alternatively, the weather forecast can be obtained via the network by the network communication unit of the wearable device. As an alternative, the weather conditions can be sensed by other types of sensors, such as temperature sensors used to sense the temperature. Acoustic sensors can be used to detect sounds such as thunder, rain, wind, and so on. The image sensor can be used to detect images of rainy, snowy, clear, cloudy, and lightning blades. In addition, the odor sensor can sense odor information.

For example, the risk level caused by a vehicle on a rainy day should be much higher than on a clear day. Because on rainy days the road is slippery and difficult to see.

Therefore, due to these factors, there is a possibility that the driver or the user may make more mistakes and accidents occur. In another embodiment, the risk level can be adjusted based on the biometric information of the user wearing the wearable device (i.e., the user). As described above, the biometric information of the user may reflect the physiological state or the mental state of the user. When the physiological state or mental state is weak, the user is more vulnerable and easy to deceive. Thus, the risk level can be raised accordingly.

Fig. 8 is a flow chart for explaining another example of a method for early warning a danger. And a risk level for the target object when the target object is a person. In one embodiment, the risk level for the target object is primarily determined by the likelihood of fraud by the target object.

 In step S800, it is determined whether or not the target object is a person.

In step S810, if the target object is a person, the face of the target person is detected.

In step S820, the risk level is determined according to whether or not the detected face is present in the predetermined face database. At this time, faces that are not in the face database can be determined to be strangers. It can be understood that the danger is usually caused by a stranger. It focuses primarily on the detection of fraud committed by strangers. The stranger is recognized according to the image information or the audio information obtained by the wearable device. In one embodiment, whether the target object is a stranger can be determined according to image information. For example, the processing unit of the wearable device can detect a face according to image information. A variety of facial recognition techniques can be used to detect faces. In an embodiment, Adaboost machine learning algorithms and Haar-like features can be used to detect faces. Whether or not the detected face is a face of a stranger can be determined by searching an acquaintance database. The acquaintance database is a database storing the acquaintances' face image data of the previous user.

In another embodiment, whether the target object is a stranger may be determined according to the audio information. For example, the processing unit of the wearable device can perform speech recognition on the audio information. Various speech recognition techniques can be used to recognize speech. In an embodiment, the target object may be recognized by a grammar recognition technique. Further, whether the detected voice is the voice of the acquaintance or a person who can be trusted can be determined by searching the acquaintance database. The acquaintance database has previously stored voice data or audio feature data of the acquaintances of the user.

It can be seen that this can arbitrarily combine the two embodiments described above. For example, in one implementation, the target object may be considered an acquaintance only if it is recognized as being acquaintance by both embodiments. In other embodiments, it can be considered an acquaintance if it is recognized as being acquaintance by one of both embodiments.

Also, whether or not the detected person is interacting with the user, i.e., whether the detected person is talking to the user, can be determined according to the image information of. In other words, whether the target object talks to the user is determined by the physical characteristics. In one embodiment, the determining step first determines the distance between the target object and the user according to the face size of the target object. For example, it is determined whether or not the distance is within a preset range. Second, it determines whether the target object is facing the user according to the face direction of the target object. Finally, it determines whether the lips of the target object are moving. If all three of these conditions are satisfied at the same time, the target object can be regarded as a person who communicates with the user. Generally, a stranger made a story through a story. Thus, in one embodiment, strangers who do not talk to the user can be excluded from the investigation. The excluding step may be performed before and after the step of detecting the face and the voice, and is not limited to this point.

The likelihood of fraud can be determined based on one or more embodiments and is determined including, but not limited to, aspects such as conversation, representation or operation of the object of interest. In one embodiment, the speech content of the target object is analyzed based on a fraud criterion model to determine the likelihood of fraud. A fraud reference model can be created by studying examples of fraud that existed. For example, a fraud criteria model may be generated by the following steps. We collect various examples of fraud from the network and use the Bayesian classifier to act on the fraud language model.

In one embodiment, the audio information is analyzed by the processing unit of the wearable device. For example, the analysis step first extracts the power normalization cepstrum coefficient (PNCC) feature of the target object and acquires speech recognition. Second, we segment words and extract sensitive words based on the Markov random field. Finally, we use a fraud reference model to predict the likelihood of fraud.

For example, when the target object says, "Send $ 100,000 to this bank card," the extracted sensitive words are "bank card" and "dollar". According to Bayes' theorem, fraud possibility P (fraud | "bank card", "money") = P ("bank card" "," Money "). At this time, P ("bank card", "money" | fraud) represents the probability of simultaneous occurrence of keywords "bank card" and "money" in all fraud cases. P (fraud) is a prior probability. P ("bank card," money ") is the prior probability that the keywords" bank card "and" money "appear simultaneously. These three possibilities can be determined by learning a moral language model.

In another embodiment, the likelihood of fraud can be estimated based on a change in facial characteristics (i.e. facial expression) of the target object. In one embodiment, the processing portion of the wearable device may be used to analyze changes in facial features of a target object based on image information. According to an analysis of psychology or practice, some changes in facial features can be used to infer corresponding psychological activities (such as lying). Thus, the likelihood of fraud can be assigned according to a change or combination of special facial features. For example, the possibility of fraud is assigned according to psychology or practice. The likelihood of fraud is adjusted based on statistical analysis of fraud events. For example, a change in a facial feature may include, but is not limited to, relative displacement of the eye, relative change in eyebrow shape, facial color, and the like. Temporal feature changes and fragile likelihood templates can be generated. In an embodiment, the template may be a look-up table. Each part of the data in the lookup table is configured by the search index and the likelihood of fraud.

 For example, the search index can be defined as follows. If the eye moves to the right and the eyebrows rise upward and the face turns pink, the probability of corresponding fraud is 0.7. In another embodiment, the likelihood of fraud can be estimated based on an action change of the target object. The processing unit of the wearable device can be used to analyze a behavior change of a target object based on image information. According to the analysis of psychology or practice, certain special behavioral changes can be deduced as corresponding psychological activities (eg, psychological activities such as lying)

The possibility of fraud can be assigned according to these particular behavioral changes and combinations thereof. For example, the possibility of fraud can be assigned on the basis of psychology or pragmatics. The likelihood of fraud is adjusted based on statistical analysis of fraud events. Special behavioral behavior changes include, but are not limited to, neck tilting, shoulder shrug. Templates of behavior change and fraud possibilities can be generated. Detection of behavioral behavior changes may be implemented in various ways. For example, in the detecting step, the hand is first detected in the skin color, and the shoulder is detected through skeletal analysis. Second, tracking the hand and shoulder using a tracking-learning detection (TLD) tracking algorithm to obtain the order of the hand and shoulder trajectories. Finally, we estimate the possibility of fraud based on the order of orbits.

For example, if an act of touching the neck with the hand is detected, the possibility of fraud on the hand is P (fraud | hand touching the neck). If the shrugging of the shoulder is detected, the possibility of fraud on the shoulder is P (fraud | motion shrugging), the possibility of comprehensive fraud is P (fraud | hand tie, shrug ) = P (fraud | action to tie the neck with the hand) + P (fraud | action to shrug the shoulder) - P (fraud |

It can be seen that the three embodiments described above can be arbitrarily combined. For example, in one embodiment, the likelihood of fraud determined by each can be calculated separately, from which the possibilities of three frauds can be weighted to determine the likelihood of an overall fraud.

In general, the likelihood of estimated fraud can be expressed as a percentage or other value. The probability of fraud can be mapped to a corresponding risk to later provide corresponding warning information according to the corresponding risk. This method extends the range of risk detection and improves user safety.

The specific sequence of steps in this figure is described, but does not mean that it should be performed in a particular order. And you do not have to perform all the steps to achieve the desired result. Instead, the order of execution of the steps shown in the flowchart may be changed.

Fig. 9 is a flowchart for explaining an example of implementing a method for early warning a risk.

Referring to FIG. 9, the user's companion loses on the street (such as a young girl). When the user is very uneasy, the wearable device of the present invention worn by the user senses the user's anxiety by the bio-information sensor and transmits the anxiety information to the wearable device worn by another nearby user. Then we open the image sharing function to detect this anxiety and expand the user's view. This makes it easier for other users to find a lost girl.

In step S900, the user discovers that the companion (girl) is lost.

In step S910, the wearable device (first wearable device) worn by the user senses the psychological state of the user by the bio-information sensor. When the first wearable device senses the user's anxiety, it can request information sharing at one or more nearby wearable devices. For example, the server of the early warning system may send an information sharing request to one or more second wearable devices in the vicinity. As another example, the first wearable device can directly broadcast a request for information sharing to inform a nearby second wearable device.

In step S930, the second wearable device (i.e., the wearable device) worn by another user receives the information sharing request.

In step S940, these second wearable devices enable the information sharing function to help.

In step S950, the second wearable device may upload the view-of-view image to the cloud database optionally with audio information to share with the first wearable device.

As shown in step S960, these second wearable devices can provide positional information (e.g., positional information obtained through a GPS module). At the same time, in step S940, the first wearable device can acquire the surrounding image for searching.

In step S970, the object is searched according to information acquired by the first wearable device itself and information shared by the one or more second wearable devices. For example, based on image information, a young girl is searched for according to her characteristic or physical characteristics.

In step S980, by way of information sharing, the user's field of view can be expanded, so that the girl can be found more quickly.

In one embodiment, a search function based on image processing may be performed at the server of the early warning system. Therefore, the load of the first wearable device can be reduced. In another embodiment, the search functionality based on image processing may be distributed among other wearable devices that can accelerate processing speed.

In addition, each part (for example, a part) of the device described in the embodiments of the present application may be implemented in the form of software as well as in the form of hardware. Each portion of the device may be provided in a processor (e.g., processing portion 320).

An apparatus according to the present embodiments may include a processor, a memory for storing and executing program data, a permanent storage such as a disk drive, a communication port for communicating with an external device, a user such as a touch panel, a key, Interface devices, and the like. Methods implemented with software modules or algorithms may be stored on a computer readable recording medium as computer readable codes or program instructions executable on the processor. Here, the computer-readable recording medium may be a magnetic storage medium such as a read-only memory (ROM), a random-access memory (RAM), a floppy disk, a hard disk, ), And a DVD (Digital Versatile Disc). The computer-readable recording medium may be distributed over networked computer systems so that computer readable code can be stored and executed in a distributed manner. The medium is readable by a computer, stored in a memory, and executable on a processor.

This embodiment may be represented by functional block configurations and various processing steps. These functional blocks may be implemented in a wide variety of hardware and / or software configurations that perform particular functions. For example, embodiments may include integrated circuit components such as memory, processing, logic, look-up tables, etc., that may perform various functions by control of one or more microprocessors or other control devices Can be employed. Similar to how components may be implemented with software programming or software components, the present embodiments may be implemented in a variety of ways, including C, C ++, Java (" Java), an assembler, and the like. Functional aspects may be implemented with algorithms running on one or more processors. In addition, the present embodiment can employ conventional techniques for electronic environment setting, signal processing, and / or data processing. Terms such as "mechanism", "element", "means", "configuration" may be used broadly and are not limited to mechanical and physical configurations. The term may include the meaning of a series of routines of software in conjunction with a processor or the like.

The specific implementations described in this embodiment are illustrative and do not in any way limit the scope of the invention. For brevity of description, descriptions of conventional electronic configurations, control systems, software, and other functional aspects of such systems may be omitted. Also, the connections or connecting members of the lines between the components shown in the figures are illustrative of functional connections and / or physical or circuit connections, which may be replaced or additionally provided by a variety of functional connections, physical Connection, or circuit connections.

In this specification (particularly in the claims), the use of the terms " above " and similar indication words may refer to both singular and plural. In addition, when a range is described, it includes the individual values belonging to the above range (unless there is a description to the contrary), and the individual values constituting the above range are described in the detailed description. Finally, if there is no explicit description or contradiction to the steps constituting the method, the steps may be performed in an appropriate order. It is not necessarily limited to the description order of the above steps. The use of all examples or exemplary terms (e. G., Etc.) is merely intended to be illustrative of technical ideas and is not to be limited in scope by the examples or the illustrative terminology unless the context clearly dictates otherwise. It will also be appreciated by those skilled in the art that various modifications, combinations, and alterations may be made depending on design criteria and factors within the scope of the appended claims or equivalents thereof.

Claims (19)

Acquiring image information of a target object;
Detecting physical characteristics of the target object based on the obtained image information;
Determining a risk level that is a degree that threatens the safety of a user by using the detected physical characteristics; And
And providing a warning to the user based on the risk of the determined event.
The method according to claim 1,
Wherein the acquiring comprises:
Acquires surrounding audio information,
Wherein the detecting comprises:
And detecting audio characteristics based on the obtained audio information.

3. The method of claim 2,
Wherein the step of determining the risk comprises:
Determining, based on the image information, the time it took for the target object to reach the wearable device; And
Determining a risk to the target object according to at least one of a physical characteristic of the target object, a time taken for the target object to reach the wearable device, and an audio characteristic of the target object. How to warn of danger.
The method according to claim 1,
Wherein the physical characteristics of the target object include a human face, torso, limb, animal's teeth, tail or eye.
3. The method of claim 2,
Wherein the audio properties of the target object include at least one of volume, pitch, Mel Frequency Cepstrum Coefficient (MFCC), human conversation and Power Normalized Cepstral Coefficient (PNCC) , A way to warn a wearable device of an early warning of danger.

The method according to claim 1,
Wherein the acquiring comprises:
Obtaining weather information or odor information,
Wherein the step of determining the risk comprises:
Wherein the risk is determined based on the weather information or odor information.

The method according to claim 1,
The providing of the alert may comprise:
A method for alerting a wearable device early to provide a warning in a different manner depending on the risk. The method according to claim 1,
Further comprising receiving image information or audio information acquired through a second wearable device,
Wherein the step of determining the risk comprises:
Wherein the risk is determined based on information received from the second wearable device.
The method according to claim 1,
Wherein the detecting comprises:
Detecting whether the target object is a person using the image information;
And detecting the face of the person if the target object is a person,
Wherein the step of determining the risk comprises:
And determines the risk according to whether the detected face is in a predetermined face database.
A computer-readable recording medium on which a computer program for executing the method according to any one of claims 1 to 9 is recorded.
A sensing unit for acquiring image information of a target object;
A characteristic detector for detecting physical characteristics of the target object based on the obtained image information;
A risk determining unit that determines a risk level that is a degree that threatens the safety of a user by using the detected physical characteristics; And
And a warning section for warning the user based on the risk of the determined event.
12. The method of claim 11,
The sensing unit includes:
Acquiring the surrounding audio information,
Wherein the characteristic detecting unit comprises:
And detects an audio characteristic on the basis of the obtained audio information.
13. The method of claim 12,
The risk-
Determining a time taken for the target object to reach the wearable device based on the image information,
Wherein the risk of the target object is determined based on at least one of a physical characteristic of the target object, a time taken for the target object to reach the wearable device, and an audio characteristic of the target object.
12. The method of claim 11,
Wherein the physical characteristics of the target object include a human face, torso, limb, animal's teeth, tail or eye.
13. The method of claim 12,
Wherein the audio properties of the target object include at least one of volume, pitch, Mel Frequency Cepstrum Coefficient (MFCC), human conversation and Power Normalized Cepstral Coefficient (PNCC) , A wearable device that warns of danger early.
12. The method of claim 11,
The sensing unit includes:
Obtain weather information or odor information,
The risk-
And the risk is determined based on the meteorological information or odor information.
12. The method of claim 11,
The warning unit,
A wearable device that warns of an early warning that provides an alert in a different way depending on the above risk.
12. The method of claim 11,
Further comprising a receiver for receiving image information or audio information acquired through a second wearable device,
The risk-
And the risk is determined based on information received from the second wearable device. 12. The method of claim 11,
Wherein the characteristic detecting unit comprises:
Detecting whether the target object is a person using the image information, detecting the face of the person if the target object is a person,
The risk-
And determines the risk according to whether or not the detected face is in a predetermined face database.

Images