TW201918828A - Contactless gesture determining system for wearable device and determining method thereof - Google Patents

Abstract

A contactless gesture determining system for a wearable device is provided, including a storage module, a sensing module, a processing module and a displaying module. The storage module is stored with gesture information corresponding to a plurality of gestures. The sensing module senses the variations of muscles of a wrist of a user and generates a sensing signal. The processing module is connected to the storage module and the sensing module, receives the sensing signal, and compares the sensing signal with the gesture information stored in the storage module, to obtain the gesture of the user. The displaying module displays the information corresponding to the gesture of the user.

Description

 Non-contact gesture judgment system for wearing device and judgment method thereof  

The present invention relates to a sensing technology for a wearable device, and more particularly to a non-contact gesture determination system for a wearable device and a method for determining the same.

With the evolution of technology, electronic devices have become more and more widely used. In recent years, popular wearable devices, such as Google Glass or Apple Watch, can be easily accessed through human-machine interface input in addition to portable devices. And the use of sensing technology to provide more functionality and applications, this will become the direction of many manufacturers to research and development.

At present, the development of the wear-through device is in the initial state of the wear-through device. In short, the function of the traditional physiological measurement device is miniaturized to the device that can be worn by the user to provide health monitoring and control. Mobile phone operation and other functions, but such wear-through devices still have some limitations on use and improvement of space. As mentioned above, due to the combination of traditional physiological measurement equipment, there are many noises based on physiological signals, so it is likely to cause high interference. The measurement results, in addition, due to the use of body surface contact measurement, if the position of the physiological measurement equipment is offset, it may lead to measurement error, and the contact measurement requirements will cause the wear-through device to be close to the wearer. Skin, long-term wear is a problem of comfort, in addition, the wearable device can only passively communicate with the mobile phone, can not actively communicate with the mobile phone. The above-mentioned improvement is such that the application of the wear-through device is limited or inconvenient, and the existing physiological measurement device is limited, which may also cause discomfort in the use of the wear-through device.

It can be seen from the above that in the comfort and accuracy of the wear-through device, it is necessary to find a sensing technology suitable for the wearable device, in particular, the state sensing can be performed in a non-contact state, and the physiological sensing possibility can be reduced. The noise interference caused to provide a comfortable and practical wear-through device is a technical problem that is urgently solved by those skilled in the art.

The object of the present invention is to provide a non-contact sensing technology for a wearable device. The non-contact type considers the wearing comfort, but at the same time, it still needs to have a certain sensing precision, so the wearer is determined by the non-contact gesture judging mechanism. The act to be performed, thereby providing a wearable device that is non-contact and can simultaneously achieve human-computer interaction.

The present invention provides a non-contact gesture determination system for a wearable device, including a storage module, a sensing module, a processing module, and a display module, wherein the storage module prestores gesture information corresponding to a plurality of different gestures. The sensing module is configured to sense a muscle change of a wrist portion of the wearer to generate a sensing signal, the processing module is connected to the storage module and the sensing module, and the processing module receives the sensing signal to The sensing signal is compared with the gesture information in the storage module, and the gesture performed by the wearer is obtained, and the display module is configured to display information corresponding to the gesture of the wearer.

In one embodiment, the sensing module includes a single sensor or a plurality of sensors that sense muscle changes in the wearer's wrist portion by electric, magnetic or electromagnetic waves.

In another embodiment, the sensing module includes one or more sets of corresponding transmitters and receivers, and the transmitter and the receiver sense muscle changes of the wearer's wrist portion through electromagnetic wave penetration characteristics. .

In still another embodiment, the processing module further includes a signal processing unit and a gesture processing unit, wherein the signal processing unit performs processing for removing noise and interference of the sensing signal, and the noise and interference are removed. The sensing signal is converted into a signal value, and the gesture processing unit obtains the corresponding gesture information from the storage module according to the signal value.

In addition, the gesture processing unit further includes determining the signal feature from the signal value to compare with the gesture information in the storage module.

In another embodiment, the contactless gesture determination system for the wearable device further includes a communication module, and the communication module is configured to establish a network connection with the external device to perform data transmission and command reception.

In still another embodiment, the non-contact gesture determination system for the wearable device further includes a power module, the power module provides the sensing module, the processing module, and operating power of the display module, and The power information of the power module is transmitted to the display module.

The present invention further provides a non-contact gesture determination method for a wearable device, comprising: sensing a muscle change of a wrist portion of a wearer to generate a sensing signal; converting the sensing signal into a signal value; performing the signal value and The pre-stored gesture information is compared to obtain a gesture performed by the wearer corresponding to the muscle change; and information corresponding to the gesture of the wearer is displayed.

In the above-described non-contact gesture determination method for a wearable device, sensing a muscle change of a wrist portion of a wearer means sensing a muscle change of a wrist portion of the wearer by an electric field, a magnetic field, or an electromagnetic wave.

In the above method for determining a non-contact gesture for a wearable device, the step of converting the sensing signal into the signal value includes performing a process of removing noise and interference of the sensing signal, and removing the noise and the noise. The sensing signal of the interference is converted into the signal value.

Compared with the prior art, the non-contact gesture judging system for the wearable device and the judging method thereof can obtain the behavior that the wearer wants to perform through the gesture judging mechanism, for example, by using an electric field, a magnetic field or an electromagnetic wave. Judging the change of the wearer's gesture, the wearable device does not need to be close to the wearer's skin, and the sensing effect can also reduce the noise problem that may be caused by the traditional physiological signal. In addition, the wearable device can transmit instructions based on the gesture judgment or The message to the corresponding electronic device can be an active human-machine interface. Therefore, in conjunction with the non-contact gesture determination mechanism of the present invention, the wearable device can be a wearable device having characteristics such as low sensing interference, high comfort, high accuracy, and good human interface.

1‧‧‧ Non-contact gesture judgment system

11‧‧‧ Storage Module

12‧‧‧Sensor module

121‧‧‧first sensor

122‧‧‧Second sensor

13‧‧‧Processing module

131‧‧‧Signal Processing Unit

132‧‧‧ gesture processing unit

14‧‧‧Display module

15‧‧‧Communication module

16‧‧‧Power Module

200‧‧‧Wrist part

201‧‧‧ skin surface

300‧‧‧Wearing device

400‧‧‧External equipment

S51~S54‧‧‧Steps

1 is a system architecture diagram of a contactless gesture judging system for a wearable device according to the present invention; and FIG. 2 is a system architecture diagram of another embodiment of a contactless gesture judging system for a wearable device according to the present invention; 3A and 3B are schematic views of a gesture sensing technology according to the present invention; 4A and 4B are schematic views of a wearable device to which the present invention is applied; and FIG. 5 is a schematic diagram of a non-contact gesture determination method for a wearable device according to the present invention; FIG. 6 is a flow chart of the wearable device with a non-contact sensing mechanism in the learning phase and the operation phase of the present invention; and FIG. 7A-7C is a sensing result of different gestures for different signal sources according to the present invention. Waveform diagram.

The technical contents of the present invention are described below by way of specific embodiments, and those skilled in the art can easily understand the advantages and effects of the present invention from the contents disclosed in the present specification. The invention may be embodied or applied by other different embodiments.

The gesture judgment technology is an emerging human-machine interface tool in recent years. The more common one is image recognition technology, but the disadvantage is that there is a photographic lens, which is not suitable for the wearer that emphasizes miniaturization. Furthermore, image recognition needs Considering ambient light sources, wearable devices are often used in different outdoor environments, which may have a great impact on the accuracy of image recognition.

Other gesture judgment techniques, such as myoelectric technology, that is, the skeletal muscle is contracted by nerve stimulation to produce contraction. The principle is that the ion channel is opened and closed. When the nerve impulse is transmitted to the muscle fiber, the cell membrane is depolarized, so that the ion is in the cell. The change of the internal and external concentration causes the change of the potential, which is called the action potential, and the potential change caused when the action potential is transmitted to both sides. It can be seen that the electromyography technology is contact sensing, and the dry electrode or the wet electrode is required to contact the skin, and the wearing part is close to the arm, which is not only uncomfortable in use, but may even cause limitation on the movement of the wearer.

The above gesture judgment mechanism has its defects, and is not necessarily applicable to the wearable device. Therefore, the present invention proposes a non-contact sensing mechanism that can be applied to the wearable device to determine the wearer's desire to perform through the gesture sensing result. In addition to the behavior, the wearable device can be connected to an external electronic device such as a mobile phone in addition to being able to operate alone.

Please refer to FIG. 1 , which is a system architecture diagram of a contactless gesture judging system for a wearable device of the present invention. The non-contact gesture judgment system 1 for a wearable device of the present invention is mainly disposed in a wearable device, such as a wristband, but not limited thereto, and in particular, the present invention emphasizes that the wearable device provides a non-contact feeling. The measurement mechanism helps to provide a high degree of comfort and a good sensing mechanism. As shown in the figure, the non-contact gesture determination system 1 for a wearable device includes a storage module 11, a sensing module 12, a processing module 13, and a display module 14.

The storage module 11 is configured to pre-store gesture information corresponding to a plurality of different gestures. In short, in order to allow the non-contact gesture determination system 1 for the wearable device to instantly determine the gesture performed by the wearer, the storage module 11 prestores gesture information corresponding to a plurality of different gestures, such as muscle changes. What is the gesture of the wearer?

The sensing module 12 is configured to sense a muscle change of the wearer's wrist portion 200 to obtain a sensing signal corresponding to the muscle change. The sensing module 12 can send a sensing wave to the wearer's skin by the wearing device, the sensing wave will reflect or transmit at the wearer's sensing, and finally the sensing module 12 receives the sensed signal, that is, the foregoing Sensing signal, this sensing signal is the muscle change corresponding to the wearer. The design of the sensing module 12 in the wearable device may be one or more sensors, that is, the sensing wave and the receiving sensing signal are the same, or one or more groups of corresponding transmitters and In the receiver, the transmitter emits a sensing wave, and the receiver receives the sensing signal.

The processing module 13 is connected to the storage module 11 and the sensing module 12, and the processing module 13 receives the sensing signal obtained from the sensing module 12 and stores the sensing signal and the gesture information pre-stored in the storage module 11. The comparison is made to thereby obtain the gesture performed by the wearer. As described above, the storage module 11 prestores the gesture information corresponding to the plurality of different gestures. When the operation module is in operation, the sensing module 12 senses the muscle change of the wrist portion 200 of the wearer to obtain the sensing signal, and the gesture. After the information is compared, the corresponding person is found, that is, the corresponding person is the gesture information corresponding to the muscle change at this time.

The display module 14 is configured to display information corresponding to the gesture of the wearer. After the processing module 13 obtains the corresponding gesture information by the muscle change, at this time, the display module 14 can display the meaning of the muscle change, for example, pre-defining when my five fingers are straight and presenting the guessing punch "scissors stone When the shape of the "cloth" in the cloth indicates that the time is to be displayed, when the processing module 13 confirms that the muscle change corresponds to the "cloth", the current time can be displayed by the display module 14. As for the information corresponding to the gesture, it can be preset according to the user's needs.

The contactless gesture judging system 1 for a wearable device of the present invention can obtain the behavior to be performed by the wearer by the gesture judging mechanism, and the sensing of the muscle change can be performed by, for example, an electric field, a magnetic field or an electromagnetic wave. The mechanism also makes it unnecessary for the wearable device to be in close contact with the wearer's skin, and also reduces the noise problems that may be present when the traditional physiological signal is taken. Therefore, through the non-contact gesture determination mechanism of the present invention, the wearable device can have characteristics such as low sensing interference, high comfort, and high accuracy.

Please refer to FIG. 2, which is a system architecture diagram of another embodiment of the contactless gesture judging system for a wearable device of the present invention. As shown in the figure, the storage module 11, the sensing module 12, the processing module 13, and the display module 14 of the non-contact gesture determining system 1 for the wearable device of the present embodiment are the same as those described in FIG. In the embodiment, the processing module 13 further includes a signal processing unit 131 and a gesture processing unit 132. In addition, the contactless gesture determination system 1 for the wearable device further includes a communication module 15 and a power module 16.

The signal processing unit 131 is configured to perform the process of removing noise and interference of the sensing signal sensed by the sensing module 12, and further converting the sensing signal from which noise and interference have been removed into signal values. The sensing signal sensed by the sensing module 12 may have other noise or interference. The signal processing unit 131 may perform noise and interference removal on the sensing signal, and convert the physical phenomenon into a measurable electronic signal, that is, Convert to signal value.

Then, the gesture processing unit 132 is configured to obtain the corresponding gesture information from the storage module 11 according to the signal value. More specifically, the gesture processing unit 132 can find the signal feature from the signal value to be in the storage module 11 The gesture information is compared. The signal feature described here may be a signal waveform change, such as the amplitude of the waveform, that is, the signal value may generate a change in the signal waveform corresponding to the current muscle change, and the gesture information in the storage module 11 is various pre-stored. The signal waveform changes, and when the alignment matches, the gesture performed by the wearer can be pushed.

In order to achieve information exchange between the wearable device and the external device, the contactless gesture determination system 1 for the wearable device further includes a communication module 15, and the communication module 15 can be used for establishing wired communication with an external device, for example, the external device can be The computer or the mobile phone can perform data transmission and command reception after the connection, for example, the information of the non-contact gesture judgment system 1 of the wearable device is transmitted back to the external device, and even the corresponding control command is found according to the current gesture to be transmitted back to the outside. The device, for example, activates the mobile phone camera when the fist is fisted, so when the sensing module 12 senses the gesture, the processing module 13 determines the gesture, and the gesture is pre-defined to activate the mobile phone camera, the communication module 15 Pass this message to your phone.

The non-contact gesture judging system 1 for the wearable device further includes a power module 16 that can provide operating power of the modules such as the sensing module 12, the processing module 13, the display module 14, or the communication module 15. In addition, the power information about the power module can also be transmitted to the display module 14 for display.

Please refer to FIGS. 3A and 3B, which are schematic diagrams of the gesture sensing technology of the present invention. Figures 3A and 3B show different implementations of the sensing module, and first explain the principle of measuring muscle changes. The scaling (or flexion and extension) of the forearm muscles will cause changes in the skin of the wrist. The muscles at different positions will exhibit different muscle changes during flexion and extension. This change will affect the skin surface of the skin. Different undulating states, such as stretching and thinning due to muscle contraction, will cause the corresponding skin surface to collapse. In summary, different gestures will cause different skin fluctuations.

As shown in FIG. 3A, the sensing module can be a single sensor or multiple sensors, such as the first sensor 121 and the second sensor 122, the first sensor 121 and the second sensing. The device 122 is a sensor that can transmit a sensing wave and receive a sensing signal, and the sensor can sense the muscle change of the wearer's wrist portion through an electric field, a magnetic field or an electromagnetic wave.

Specifically, both the first sensor 121 and the second sensor 122 can each emit a sensing wave, and when reaching the skin surface 201 of the wearer, there is different interference due to the fluctuation of the skin surface, when the sensing wave After the reflection, it can be received by the first sensor 121 or the second sensor 122 that originally transmitted the sensing wave, so that the sensing signal can be obtained.

As shown in FIG. 3B, the sensing module can be one or more sets of corresponding transmitters and receivers. The first sensor 121 in the figure is a transmitter, and the second sensor 122 is a receiver. The transmitter and receiver are permeable to electromagnetic wave penetration characteristics to sense muscle changes in the wrist portion of the wearer. The first sensor 121 and the second sensor 122 are distributed on both sides of the wrist, and the technology of ultra-wideband (UWB) can be used to learn the skin and muscle tissue by the penetration ability of electromagnetic waves. Change situation.

Specifically, the first sensor 121 transmits a sensing wave that penetrates the skin surface 201 of the wearer, but different muscles have different changes (such as muscles 1 to 3) because different gestures change when the sensing wave changes due to different muscles. The effect is different, and finally the receiver receives a sensing signal through each muscle at the other end (relative to the transmitter) to become a sensing signal.

Please refer to Figures 4A and 4B, which are schematic views of a wearable device to which the present invention is applied. As shown in FIG. 4A, the wearable device 300 can be worn on the wearer wrist portion 200. As shown in FIG. 4B, the wearable device 300 can communicate with the external device 400. The wearable device 300 includes at least one sensor. Each sensor may operate separately or in groups, and the sensed signal is sensed. Processing to obtain the gesture performed by the wearer, the related information can be displayed on the display module 14, such as a display.

Please refer to FIG. 5, which is a step diagram of a non-contact gesture determination method for a wearable device of the present invention. In step S51, the muscle change of the wrist portion of the wearer is sensed to generate a sensing signal. Among them, sensing the muscle change of the wearer's wrist portion means sensing the muscle change of the wrist portion of the wearer by an electric field, a magnetic field or an electromagnetic wave.

In step S52, the sensing signal is converted into a signal value. In this step, the processing of removing noise and interference of the sensing signal is performed, and the sensing signal with the removed noise and interference is converted into the signal value, that is, the physical phenomenon is converted into the measurable value. The electronic signal is converted to a signal value.

In step S53, the comparison between the signal value and the pre-stored gesture information is performed to obtain a gesture performed by the wearer corresponding to the muscle change. This step indicates that one of the pre-stored gesture information can be found to correspond to the signal value. As described above, different gestures will have their corresponding signal waveform changes, so that the waveform formed by the signal value can be found. The corresponding waveform can be used to get the gesture.

In step S54, the information corresponding to the gesture of the wearer is displayed. A gesture can be pre-defined to perform an operation, such as a fist gesture indicating that the camera is activated, so that when a fist gesture is obtained, the handset can be notified to prepare to activate the camera, thus making such a wearable device a good human interface.

Please refer to FIG. 6 , which is a flow chart of the wearable device with the non-contact sensing mechanism in the learning phase and the operational phase of the present invention. In short, the wearable device defines the gesture before use, which is the gesture learning phase, and then the wearer wears the general operational state.

The gesture learning phase is an initial phase. Before entering the general operational phase, it must go through this phase, or if there are new gestures to join, you need to go through this phase. In this phase, the user makes an expected gesture, and the sensing result of the gesture will be recorded as a reference for the gesture determination. However, this stage is not absolutely necessary, but it can be used to adjust the individual differences, which helps the gesture processing unit to make more accurate judgment results.

Specifically, the gesture learning phase mainly senses the gesture and performs signal processing. If the gesture learning is unsuccessful, the gesture returns to the front end once again. If the gesture learning is successful, the gesture information of the gesture is recorded.

The general operation phase is the normal use process of the general user (wearer). In this stage, the system of the present invention continuously senses the gesture action of the user, and after the appropriate signal processing, calculates the gesture judgment result.

Specifically, the general operation stage first determines whether it is the normal operation stage state, and if not, returns to the gesture learning stage, and also determines whether a new gesture is to be provided at present, and if so, returns to the gesture learning stage. Then, in the normal operation phase, the gesture is also sensed and signal processed, and finally the gesture information corresponding to the gesture is obtained and transmitted to the processing module to perform related information display or corresponding operation control.

Please refer to FIG. 7A-7C, which is a waveform diagram of sensing results for different gestures under different signal sources of the present invention. As shown in FIG. 7A, the magnetic force is used as the sensing result of the sensing signal source, and the sensing method is to detect the skin surface, wherein the waveform with a low signal value is a “stone” gesture, and the waveform with a high signal value is The "cloth" gesture, as can be seen from the figure, can clearly determine the continuous cloth and stone gestures.

As shown in FIG. 7B, the magnetic force is used as the sensing result of the sensing signal source, and the sensing method is also to detect the skin surface, wherein the waveform with the low signal value is a “stone” gesture, and the signal with a high signal value. For the "scissors" gesture, as can be seen from the figure, the continuous scissors and stone gestures can be clearly judged.

As shown in Fig. 7C, it is a sensing result of magnetic force as a source of sensing signals, and the sensing method is also to detect the surface of the skin, wherein three gestures of making stones and scissors are sequentially performed, as shown in the figure. It is possible to clearly determine the three gestures.

In summary, the non-contact gesture judging system for the wearable device of the present invention and the judging method thereof can obtain the behavior to be performed by the wearer through the gesture judging mechanism, and the mechanism can be sensed by using an electric field, a magnetic field or an electromagnetic wave. Therefore, the wearing device does not need to be in close contact with the wearer's skin. In addition, the wearing device can transmit an instruction or a message to the corresponding electronic device based on the gesture judgment, so that the wearing device becomes an active human-machine interface. In combination with the non-contact gesture judgment mechanism of the present invention, the wearable device can have the advantages of low sensing interference, high comfort, high precision, and good human-machine interface, so that the wearable device can be widely applied to the state, environment or demand. under.

The above embodiments are merely illustrative of the principles and effects of the invention and are not intended to limit the invention. Modifications and variations of the above-described embodiments can be made by those skilled in the art without departing from the spirit and scope of the invention. Therefore, the scope of protection of the present invention should be as set forth in the scope of the claims described below.

Claims (10)

 A non-contact gesture judging system for a wearable device includes: a storage module for pre-storing gesture information corresponding to a plurality of different gestures; and a sensing module for sensing muscle changes of a wearer's wrist portion, Obtaining a sensing signal corresponding to the muscle change; the processing module is connected to the storage module and the sensing module, and configured to receive the sensing signal to the sensing signal and the gesture in the storage module The information is compared to obtain a gesture performed by the wearer; and the display module is configured to display information corresponding to the gesture of the wearer.    The non-contact gesture judging system for a wearable device according to claim 1, wherein the sensing module comprises a single sensor or a plurality of sensors, and the sensor transmits an electric field, a magnetic field or an electromagnetic wave. The muscle changes in the wrist portion of the wearer are measured.    The non-contact gesture judging system for a wearable device according to claim 1, wherein the sensing module comprises one or more sets of corresponding transmitters and receivers, the transmitter and the receiving The device senses the muscle change of the wearer's wrist portion through the electromagnetic wave penetration characteristic.    The non-contact gesture judging system for a wearable device according to claim 1, wherein the processing module further comprises: a signal processing unit that performs processing for removing noise and interference of the sensing signal, And the sensing signal that has been removed from the noise and the interference is converted into a signal value; and the gesture processing unit obtains the corresponding gesture information from the storage module according to the signal value.    The non-contact gesture judging system for the wearable device of claim 1, wherein the gesture processing unit further comprises: identifying the signal feature by the signal value to perform the gesture information in the storage module. Comparison.    The non-contact gesture judging system for the wearable device described in claim 1 further includes a communication module for establishing a network connection with an external device to perform data transmission and command reception.    The non-contact gesture judging system for a wearable device, as described in claim 1, further comprising a power module, the sensing module, the processing module, operating power of the display module, and The power information of the power module is transmitted to the display module.    A non-contact gesture judging method for a wearable device, comprising: sensing a muscle change of a wrist portion of a wearer to generate a sensing signal; converting the sensing signal into a signal value; performing the signal value and a pre-storing gesture Information comparison to obtain a gesture performed by the wearer corresponding to the muscle change; and information indicating the wearer's gesture.    The non-contact gesture judging method for a wearable device according to claim 8, wherein sensing the muscle change of the wearer's wrist portion means sensing the wearer's wrist by an electric field, a magnetic field or an electromagnetic wave. Part of the muscle changes.    The non-contact gesture judging method for a wearable device according to claim 8, wherein the step of converting the sensing signal into the signal value comprises performing noise removal and interference of the sensing signal. Processing, and converting the sensing signal from which noise and interference have been removed into the signal value.