WO2020195172A1 - Wearable device, information processing unit, and information processing method - Google Patents

Abstract

A wearable device (1) is provided with: a position sensor (3) for detecting the position of a hand (H) of a user; a myoelectric sensor (4) for detecting myoelectric potential information of the hand (H); and an estimating unit which, when the position of the hand (H) detected by the position sensor (3) is a grasping position (P2), estimates a grasping state of the hand (H) of the user on the basis of the myoelectric potential information detected by the myoelectric sensor (4).

Description

Wearable device, information processing unit and information processing method

This disclosure relates to a wearable device, an information processing unit, and an information processing method.

Patent Document 1 discloses a motion capture device or the like that predicts muscle motion by calculating muscle tonus prediction data based on frequency analysis data of myoelectric potential information.

Japanese Unexamined Patent Publication No. 2001-054507

In the above-mentioned conventional technology, when the electrode of the myoelectric sensor is used in contact with the skin surface of the user, the output of the electrode may be affected by noise or the like depending on the contact condition of the electrode. Therefore, in the prior art, it is desired to improve the accuracy of estimating the muscular strength of the hand by using the myoelectric sensor that comes into contact with the skin surface.

Therefore, in this disclosure, we propose a wearable device, an information processing unit, and an information processing method that can improve the estimation accuracy of the muscle strength of the hand using the myoelectric sensor.

In order to solve the above problems, the wearable device of one form according to the present disclosure uses a posture sensor for detecting the posture of the user's hand, an electromyographic sensor for detecting the myoelectric potential information of the hand, and the posture sensor. When the detected posture is the grasping posture, the estimation unit includes an estimation unit that estimates the grasping state of the hand of the user based on the myoelectric potential information detected by the myoelectric sensor.

Further, the information processing unit of one form according to the present disclosure includes a posture sensor that detects the posture of the user's hand, an acquisition unit that acquires the detection results of the myoelectric potential sensor that detects the myoelectric potential information of the hand, and the posture sensor. When the posture detected by the user is the grasping posture, the estimation unit includes an estimation unit that estimates the grasping state of the hand of the user based on the myoelectric potential information detected by the myoelectric sensor.

In addition, one form of information processing method according to the present disclosure includes a step in which a computer acquires detection results of a posture sensor for detecting the posture of a user's hand and an electromyographic sensor for detecting the myoelectric potential information of the hand. When the posture detected by the posture sensor is the grasping posture, the step includes estimating the grasping state of the hand of the user based on the myoelectric potential information detected by the myoelectric sensor.

It is a figure for demonstrating an example of the wearable apparatus which concerns on 1st Embodiment. It is a figure which shows the output example of the myoelectric sensor of the wearable device which concerns on 1st Embodiment. It is a figure which shows the output example of the posture sensor of the wearable device which concerns on 1st Embodiment. It is a figure which shows the configuration example of the wearable device which concerns on 1st Embodiment. It is a flowchart which shows an example of the processing procedure executed by the wearable apparatus which concerns on 1st Embodiment. It is a figure for demonstrating an example of the wearable apparatus which concerns on the modification (1) of 1st Embodiment. It is a figure which shows an example of the grasping posture of the wearable device which concerns on the modification (2) of 1st Embodiment. It is a figure for demonstrating an example of the system which concerns on 2nd Embodiment. It is a figure for demonstrating an example of the wearable apparatus in the system which concerns on 2nd Embodiment.

The embodiments of the present disclosure will be described in detail below with reference to the drawings. In each of the following embodiments, the same parts are designated by the same reference numerals, so that duplicate description will be omitted.

(First Embodiment)
[Configuration of Wearable Device According to First Embodiment]
FIG. 1 is a diagram for explaining an example of a wearable device according to the first embodiment. As shown in FIG. 1, the wearable device 1 is attached to the user's hand H. In the present embodiment, the user's hand H is described as a portion of the human body from the wrist to the fingertip, but for example, only the finger or the back of the hand may be used. The wearable device 1 will be described, for example, in the case of a glove interface object, but the wearable device 1 is not limited thereto. For example, the wearable device 1 may be a glove, a racing glove, a wearing belt, or the like worn on the user's hand H. In the present embodiment, the case where the wearable device 1 is attached to one of the user's hands H will be described, but the wearable device 1 may be attached to both of the user's hands H.

In the example shown in FIG. 1, the wearable device 1 covers the entire finger, instep, etc. of the user's hand H. The wearable device 1 can be used as an interface for detecting a user's grip strength in, for example, AR (Augmented Reality), VR (Virtual Reality), and the like. For example, how to hold sports equipment differs for each sport such as baseball, golf, tennis, table tennis, boxing, and racing, and it is difficult to individually make a controller for a game machine corresponding to each sport. Grip strength is also a very important muscle strength in some sports. For example, in baseball, it is important to firmly fix the bat during batting and use grip strength to transmit the force of the entire body to the bat. Therefore, by using the wearable device 1 as a glove interface object, it is possible to estimate the grip strength and the like in a plurality of types of sports.

In addition, it is difficult to realize freehand or UI operation with tools by the conventional grip strength measurement method in which the piezoelectric element is provided in the controller of the user, and the grip strength is estimated unless the position of the piezoelectric element is accurately grasped. It was difficult to do. Therefore, the wearable device 1 aims to improve the estimation accuracy of the muscle strength of the hand H and the convenience of the user.

The wearable device 1 includes a mounting portion 2, a posture sensor 3, a myoelectric sensor 4, and an information processing unit 10. In the first embodiment, the case where the wearable device 1 has five posture sensors 3 and three myoelectric sensors 4 corresponding to each of the fingers of the hand H will be described, but the number of sensors is limited. It's not a thing.

The mounting portion 2 is a glove on which the wearable device 1 is mounted on the user's hand H. The mounting portion 2 is formed of, for example, cloth, synthetic fibers, leather, or the like. The mounting portion 2 covers the user's hand H and deforms according to the operation of the hand H. The mounting unit 2 is provided with a posture sensor 3, a myoelectric sensor 4, and an information processing unit 10. When the attachment portion 2 is attached to the user's hand H, for example, the posture sensor 3 and the myoelectric sensor 4 are brought into direct or indirect contact with the skin of the hand H. The mounting unit 2 has a non-grasping posture P1 indicating a posture in which the user does not hold the hand H and a grasping posture P2 indicating a posture in which the user holds the hand H when the user is mounted on the hand H. The non-grasp posture P1 includes, for example, a posture in which the hand H is open. The non-grasping posture P1 includes a posture when the object to be detected is not grasped by the force of the hand H, and a part of the fingers of the hand H may be bent. The grasping posture P2 includes, for example, a posture when the hand H holds an object to be detected, a posture when the force of the hand H is applied, and the like. The posture of the hand H in the present embodiment includes, for example, the posture, shape, and grasping state of the hand H.

The posture sensor 3 detects the posture of the user's hand H. The posture sensor 3 is configured to detect the bending of a part of the user's hand H, such as the user's finger. As the attitude sensor 3, for example, a bending sensor, a camera, an inertial measurement unit (IMU: Inertial Measurement Unit), or the like can be used. The posture sensor 3 may be used in combination with, for example, a bending sensor, a camera, and an inertial measurement unit.

In the present embodiment, the case where the posture sensor 3 is a bending sensor will be described. For example, the posture sensor 3 is arranged along the upper surface of each of the fingers of the hand H when the mounting portion 2 of the wearable device 1 is mounted on the hand H. The upper surface of the finger is, for example, a surface that faces upward when the back of the hand H is turned upward. In the present embodiment, the case where the posture sensor 3 is arranged on the back side of the hand H so as to straddle the proximal phalanx and the metacarpal bone of each finger of the hand H will be described, but the present invention is limited to this. Not done. For example, the posture sensor 3 may be arranged on the entire finger, the inner surface of the finger, or the like so that the bending of the finger of the hand H can be detected.

The posture sensor 3 is a sensor whose resistance value increases when it is bent. The posture sensor 3 detects at least a part of the bending of the user's hand. In this embodiment, the posture sensor 3 includes, for example, a resistor that is sensitive to bending. The resistance value of the attitude sensor 3 changes based on the amount of bending of the resistor. In other words, the posture sensor 3 detects the posture of the hand H as a resistance value. For example, the posture sensor 3 arranged on the index finger of the hand H detects the bending of the index finger, and as the bending increases, the resistance value also increases. One of the plurality of posture sensors 3 is connected to the power supply of the wearable device 1, and the other is grounded. In the present embodiment, the posture sensor 3 is electrically connected to the information processing unit 10 and outputs the detection result to the information processing unit 10.

Further, for example, in the case of a camera, the posture sensor 3 is provided so that the mounting portion 2 can be imaged. In other words, the posture sensor 3 is provided outside the mounting portion 2, and estimates the user's hand H from the image captured by the mounting portion 2. The posture sensor 3 detects the hand H and its posture from an image by using well-known techniques such as background subtraction method and pattern matching.

Further, for example, in the case of an inertial measurement unit, the posture sensor 3 is provided on the mounting portion 2 so as to be arranged on the finger, fingertip, or the like of the hand H. The posture sensor 3 detects the three-dimensional angular velocity and acceleration according to the movement of the finger, the fingertip, and the like, and detects the posture of the hand H based on the detection result.

The myoelectric sensor 4 detects the myoelectric potential information of the user's hand H. The myoelectric potential information is, for example, information necessary for estimating the grasped state of the hand H. The myoelectric sensor 4 has, for example, a plurality of electrodes 41 that come into contact with the skin of the user's hand H. The myoelectric sensor 4 detects the myoelectric potential information generated when the muscle of the hand H is operated by the electrode 41. One of each of the plurality of electrodes 41 is connected to the power supply of the wearable device 1, and the other is grounded. The myoelectric sensor 4 is electrically connected to the information processing unit 10, and outputs the myoelectric potential information detected by the plurality of electrodes 41 to the information processing unit 10. The muscle electronic information includes, for example, information indicating the potential of each of the plurality of electrodes 41 required for estimating the grasping power of the hand H, the shape of the fingers, etc., and information indicating the part of the hand H in which the electrodes 41 are arranged. Including. In this embodiment, the case where the electrode 41 is a dry type electrode will be described, but a wet type electrode may be used. The advantages of the dry type electrode 41 are that it can be easily attached to the user's skin and that the running cost can be suppressed as compared with the case of using the wet type disposable electrode.

The information processing unit 10 is, for example, a dedicated or general-purpose computer. In this embodiment, the information processing unit 10 is formed in a chip shape. The information processing unit 10 is a computer for processing data from another device. The information processing unit 10 is an electronic device that uses the posture sensor 3 and the myoelectric sensor 4 in combination and processes the information detected by the posture sensor 3 and the myoelectric sensor 4. The information processing unit 10 has, for example, a function of determining the posture of the user's hand H. The information processing unit 10 has, for example, a function of estimating the grasping state of the user's hand H. The grasping state includes, for example, a state such as a posture of fingers and a grip strength. The information processing unit 10 obtains the detection results of the posture sensors 3 of the non-grasping posture P1 and the gripping posture P2 by calibration at the start of use of the user, and sets the detection result of the non-grasping posture P1 as the initial value. It may be.

FIG. 2 is a diagram showing an output example of the myoelectric sensor 4 of the wearable device 1 according to the first embodiment. In each of the graphs shown in FIG. 2, the vertical axis represents the amplitude and the horizontal axis represents the time. The graphs G11 and G12 shown in FIG. 2 show an output example of the myoelectric sensor 4 in a state where the user's hand H is in the non-grasping posture P1, that is, the hand H is open. In the non-grasping posture P1, for example, a change in the posture of the hand H causes a recess in the back of the hand H, or the electrode 41 partially or completely does not come into contact with the skin. In such a case, the amplitude of the signal of the myoelectric sensor 4 becomes unstable due to the change of the contact impedance. For example, when the electrode 41 is completely separated from the hand H, the output of the myoelectric sensor 4 does not change at the baseline as shown in the graph G11, and the boundary of the A / D converter (ADC) as shown in the graph G12. It becomes stuck to the value.

The graphs G21 and G22 shown in FIG. 2 show an output example of the myoelectric sensor 4 in a state where the user's hand H holds the grasping posture P2, that is, the hand H. The grasping posture P2 has a first state P21 and a second state P22. The first state P21 indicates a state in which the user lightly grasps the hand H without exerting any force. The second state P22 indicates a state in which the user puts a lot of effort into holding the hand H firmly. In the first state P21, the posture of the hand H is fixed, and the contact area between the electrode 41 and the hand H is constant. In this case, the myoelectric sensor 4 outputs myoelectric potential information indicating a myoelectric signal having a small amplitude. Further, the second state P22 outputs myoelectric potential information indicating a myoelectric signal in which the average amplitude increases in proportion to the grip strength of the hand H.

As shown in FIG. 2, in the myoelectric sensor 4, the characteristics of the myoelectric signal change depending on whether the user's hand H is in the non-grasping posture P1 and the grasping posture P2. That is, it can be seen that the myoelectric sensor 4 outputs a myoelectric signal effective for estimating the grasping state of the hand H when the hand H changes from the non-grasping posture P1 to the grasping posture P2. Further, in the myoelectric sensor 4, the myoelectric signal in the first state P21 of the grasping posture P2 is the myoelectric potential information at the beginning of grasping the hand H, and the myoelectric potential information can be used as the initial value.

FIG. 3 is a diagram showing an output example of the posture sensor 3 of the wearable device 1 according to the first embodiment. In the graph shown in FIG. 3, the vertical axis shows the resistance value of the posture sensor 3, and the horizontal axis shows the amount of change from the non-grasping posture P1 of the hand H to the gripping posture P2. The graph shows that the posture sensor 3 outputs the resistance value Rs when the hand H is opened in the non-grasp posture P1. Then, in the graph, when the user gradually changes the state in which the hand H is grasped from the non-grasping posture P1 to the first state P21 of the grasping posture P2, the resistance value is around Rm according to the increase in the deformation amount of the posture sensor 3. It shows that it increases to. Then, the graph shows that the resistance value Rm output by the posture sensor 3 does not change in the first state P21 and the second state P22 of the grasping posture P2.

As shown in FIG. 3, the posture sensor 3 changes the resistance value from the non-grasping posture P1 to the gripping posture P2 of the user's hand H, but it can be seen that the change in the gripping force of the hand H cannot be captured. Further, the wearable device 1 sets the reference resistance value Rth based on the first state P21 of the grasping posture P2 for the posture sensor 3, thereby changing the hand H from the non-grasping posture P1 to the grasping posture P2. The change of the hand H from the grasping posture P2 to the non-grasping posture P1 can be discriminated based on the detection result of the posture sensor 3.

Returning to FIG. 1, when the wearable device 1 is attached to the user's hand H, the wearable device 1 determines the posture of the hand H based on the detection result of the posture sensor 3. For example, it is assumed that the user's hand H is the non-grasp posture P1 with the hand H open. In this case, since the resistance value detected by the posture sensor 3 does not exceed the reference resistance value Rth, the wearable device 1 determines that the posture of the user's hand H is the non-grasp posture P1 and does not detect the myoelectric sensor 4. .. After that, it is assumed that the user's hand H changes from the non-grasping posture P1 to the grasping posture P2 by, for example, holding a tool. In this case, since the resistance value detected by the posture sensor 3 exceeds the reference resistance value Rth, the wearable device 1 determines that the posture of the user's hand H is the grasping posture P2. Then, the wearable device 1 starts the detection of the myoelectric sensor 4, and estimates the grasping state of the hand H based on the detection result of the myoelectric sensor 4. For example, the wearable device 1 estimates the state of fingers, grip strength, and the like based on the frequency component of the myoelectric potential information for each electrode 41 of the myoelectric sensor 4.

For example, the wearable device 1 estimates the grasping state of the movement of the hand H, the grip strength, etc. by utilizing the characteristic that the myoelectric signals of the myoelectric potential information of the myoelectric sensor 4 show different frequencies depending on the movement. For example, the wearable device 1 can extract an integrated value average potential, a frequency spectrum, and the like as parameters of a myoelectric signal. The integrated value average potential is a rectified and averaged myoelectric signal, and it is known that the greater the muscle strength, the more active the muscle activity and the larger the value. Therefore, the wearable device 1 detects the grip strength of the hand H by, for example, a detection method using the integrated value average potential, which is a well-known technique. Further, the wearable device 1 may estimate the fine operation of the hand H by using, for example, a gesture detection method using a frequency spectrum, which is a well-known technique.

After that, it is assumed that the user's hand H transitions from the grasping posture P2 to the non-grasping posture P1. In this case, since the resistance value detected by the posture sensor 3 does not exceed the reference resistance value Rth, the wearable device 1 determines that the posture of the user's hand H is the non-grasp posture P1 and ends the detection of the myoelectric sensor 4. ..

As described above, in the wearable device 1 according to the first embodiment, when the posture of the hand H detected by the posture sensor 3 is the grasping posture P2, the hand H is based on the myoelectric potential information detected by the myoelectric sensor 4. Estimate the grasping state of. For example, when the user changes the hand H to the grasping posture P2, the user tends to maintain the grasping posture P2. As a result, the wearable device 1 can estimate the grasping state of the hand H using the myoelectric sensor 4 in a state where there is a high possibility that the grasping posture P2 of the hand H does not change. As a result, since the wearable device 1 can suppress the influence of noise and the like of the myoelectric sensor 4, it is possible to improve the estimation accuracy of the muscle strength of the hand H using the myoelectric sensor 4.

Further, the wearable device 1 has a dry type electrode 41 in which the myoelectric sensor 4 detects myoelectric potential information. For example, in the case of the dry electrode 41 of the myoelectric sensor 4, since the identification of the myoelectric signal changes depending on the contact condition, the value may vary from measurement to measurement even with the same muscle strength. Since the wearable device 1 can estimate the grasping state of the hand H using the myoelectric sensor 4 in the state where the grasping posture P2 of the hand H does not change, the measurement result of the dry electrode 41 can be stabilized. it can. As a result, the wearable device 1 can easily attach the myoelectric sensor 4 to the hand H, and can improve convenience.

Further, the wearable device 1 uses a bending sensor for detecting the movement of the finger of the hand H as the posture sensor 3. As a result, the wearable device 1 can determine the posture of the hand H based on the degree of bending of the fingers of the hand H. As a result, the wearable device 1 can improve the accuracy of discriminating the grasping posture P2 of the hand H, so that the estimation accuracy of the muscle strength of the hand H using the myoelectric sensor 4 can be further improved. Further, although the wearable device 1 can determine the posture from the image obtained by capturing the hand H, the user needs to position the hand H inside the imaging range, which may reduce convenience. On the other hand, the wearable device 1 does not have to limit the operating range of the hand H by using the bending sensor for the posture sensor 3, so that the convenience can be improved.

Further, in the wearable device 1, when the mounting portion 2 is mounted, the posture sensor 3 is positioned at the position of the hand H where the mounting portion 2 can detect the grasping posture P2. As a result, in the wearable device 1, the posture sensor 3 can be arranged at the measurement position of the hand H only by mounting the mounting portion 2 on the hand H. As a result, the wearable device 1 can improve the accuracy of detecting the posture of the hand H by the posture sensor 3.

Further, in the wearable device 1, the mounting portion 2 is a glove, and the posture sensor 3 is positioned on the back of the hand H by the mounting portion 2. As a result, the wearable device 1 can stabilize the contact between the posture sensor 3 and the hand H by detecting the posture on the back of the hand H by the posture sensor 3. As a result, the wearable device 1 can further improve the accuracy of detecting the posture of the hand H by the posture sensor 3.

Further, when the posture of the hand H detected by the posture sensor 3 is the grasping posture, the information processing unit 10 according to the first embodiment grasps the hand H based on the myoelectric potential information detected by the myoelectric sensor 4. To estimate. As a result, the information processing unit 10 can estimate the grasping state of the hand H using the myoelectric sensor 4 while the grasping posture P2 of the hand H does not change. As a result, the information processing unit 10 can suppress the influence of noise and the like of the myoelectric sensor 4, so that the accuracy of estimating the muscle strength of the hand H using the myoelectric sensor 4 can be improved.

[Configuration Example of Wearable Device According to First Embodiment]
FIG. 4 is a diagram showing a configuration example of the wearable device 1 according to the first embodiment. As shown in FIG. 4, the wearable device 1 includes a posture sensor 3, a myoelectric sensor 4, and an information processing unit 10. The posture sensor 3, the myoelectric sensor 4, and the information processing unit 10 are operated by electric power from the power source 1B of the wearable device 1.

The information processing unit 10 includes an acquisition unit 11, a storage unit 12, an estimation unit 13, and a communication unit 14. In the present embodiment, in each processing unit of the acquisition unit 11 and the estimation unit 13, for example, a program stored in the information processing unit 10 by a CPU (Central Processing Unit), an MCU (Micro Control Unit), or the like is stored in a RAM ( It is realized by executing Random Access Memory) or the like as a work area. In addition, each processing unit may be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array).

The acquisition unit 11 acquires the detection results output by the posture sensor 3 and the myoelectric sensor 4. The acquisition unit 11 stores the acquired information in the storage unit 12. The acquisition unit 11 can acquire the detection result of the posture sensor 3 and the detection result of the myoelectric sensor 4 at different timings. The acquisition unit 11 acquires the information requested by the estimation unit 13 and outputs the acquired information to the estimation unit 13. In the present embodiment, the acquisition unit 11 is electrically connected to the plurality of electrodes 41 of the myoelectric sensor 4, obtains the myoelectric signal from the electrodes 41, and obtains the active part, the magnitude of the amplitude, and the amount of change. The acquisition unit 11 outputs the obtained information as myoelectric potential information to the estimation unit 13. In other words, the acquisition unit 11 outputs the myoelectric potential information indicating the myoelectric signal of the posture sensor 3 to the estimation unit 13.

The storage unit 12 stores various data. For example, the storage unit 12 can store data indicating the detection results of the posture sensor 3 and the myoelectric sensor 4. The storage unit 12 stores, for example, posture data 12A and the like. The posture data 12A includes data indicating the posture of the hand H to be detected. For example, when the wearable device 1 is used in a game machine, the posture of the hand H to be detected may differ depending on the game. Therefore, the posture data 12A can detect the posture of the hand H according to the game by setting the posture of the hand H to be detected. The posture data 12A may include data indicating the posture of the hand H according to activities of daily living, which is indispensable for conducting daily life. The posture data 12A may include data actually measured when the set user's hand H is the grasping posture P2. In other words, the posture data 12A may include data used for calibration. In the posture data 12A, the measurement results for each of the plurality of electrodes 41 are linked to the grip strength and the state of the hand H. The posture data 12A may include data indicating a measurement result corresponding to a general hand H size. The posture data 12A may include data such as a calculation program for obtaining grip strength based on the position of the electrode 41 and the amount of change in myoelectric potential information, a conversion table, and the like.

When the posture of the hand H detected by the posture sensor 3 is the grasping posture P2, the estimation unit 13 estimates the grasping state of the user's hand H based on the myoelectric potential information detected by the myoelectric sensor 4. The estimation unit 13 estimates the grasping state of the hand H when the grasping posture P2 continues for the determination time. For the determination time, for example, a time for avoiding an erroneous determination can be set. The estimation unit 13 sets the myoelectric potential information of the myoelectric sensor 4 when the start of the grasping posture P2 is detected as an initial value, and estimates the grasping state based on the initial value and the myoelectric potential information detected after the start of the grasping posture P2. To do. For example, the estimation unit 13 estimates the grasped state based on the amount of change between the initial value myoelectric potential information and the detected myoelectric potential information. When the estimation unit 13 detects the end of the grasping posture P2 when the grasping state of the hand H is being estimated, the estimation unit 13 ends the estimation of the grasping state. The end of the grasping posture P2 includes, for example, a change from the grasping posture P2 to the non-grasping posture P1. The estimation unit 13 is based on the myoelectric potential information detected by the myoelectric sensor 4 when the posture of the hand H detected by the posture sensor 3 is the grasping posture indicated by the posture data 12A stored in the storage unit 12. , Estimate the grasping state of the user's hand H. The estimation unit 13 outputs the estimated estimation result to the communication unit 14.

The communication unit 14 communicates wirelessly. The communication unit 14 supports a short-range wireless communication system. The communication unit 14 has a function of wirelessly communicating information with an external communication device or the like. The communication unit 14 transmits the information from the estimation unit 13 to an external electronic device or the like. External electronic devices include, for example, game consoles, televisions, smartphones, smart speakers, computers and the like. The communication unit 14 outputs information received from an external electronic device or the like to the estimation unit 13.

The functional configuration example of the wearable device 1 according to the present embodiment has been described above. The above configuration described with reference to FIG. 4 is merely an example, and the functional configuration of the wearable device 1 according to the present embodiment is not limited to such an example. The functional configuration of the wearable device 1 according to the present embodiment can be flexibly modified according to specifications and operations. For example, the wearable device 1 may be configured such that the information processing unit 10 is provided outside the mounting unit 2 and the communication unit for transmitting the detection result of the sensor to the external information processing unit 10 is provided in the mounting unit 2.

[Processing procedure of the wearable device 1 according to the first embodiment]
Next, an example of the information processing method executed by the information processing unit 10 of the wearable device 1 according to the first embodiment will be described. FIG. 5 is a flowchart showing an example of a processing procedure executed by the wearable device 1 according to the first embodiment. The processing procedure shown in FIG. 5 is realized by the information processing unit 10 of the wearable device 1 executing a program. The processing procedure shown in FIG. 5 is repeatedly executed by the information processing unit 10.

As shown in FIG. 5, the information processing unit 10 acquires the detection result of the posture sensor 3 (step S101). The information processing unit 10 functions as the acquisition unit 11 by executing step S101. When the information processing unit 10 stores the acquired detection result in the storage unit 12, the processing proceeds to step S102.

The information processing unit 10 determines the posture of the user's hand H based on the detection result of the posture sensor 3 (step S102). For example, when the posture of the hand H based on the detection result of the posture sensor 3 and the grasping posture indicated by the posture data 12A match or are similar, the information processing unit 10 determines the posture of the hand H as the grasping posture P2. The determination result is stored in the storage unit 12. When the information processing unit 10 determines that the posture of the hand H is not the grasping posture P2, the information processing unit 10 stores the determination result in the storage unit 12. Then, when the determination of the posture of the hand H is completed, the information processing unit 10 proceeds to the process in step S103.

The information processing unit 10 determines whether or not the posture of the hand H is the grasping posture P2 based on the determination result in step S102 (step S103). When the information processing unit 10 determines that the posture of the hand H is the grasping posture P2 (Yes in step S103), the information processing unit 10 proceeds to the process in step S104. The information processing unit 10 determines whether or not the detection by the myoelectric sensor 4 has started (step S104). When the information processing unit 10 determines that the detection by the myoelectric sensor 4 has started (Yes in step S104), the information processing unit 10 proceeds to step S107, which will be described later. If the information processing unit 10 determines that the detection by the myoelectric sensor 4 has not started (No in step S104), the information processing unit 10 proceeds to step S105.

The information processing unit 10 determines whether or not the grasping posture P2 has continued for the determination time (step S105). For example, when the elapsed time from the time when the grasping posture P2 is detected reaches the determination time, the information processing unit 10 determines that the grasping posture P2 has continued for the determination time. Then, when the information processing unit 10 determines that the grasping posture P2 has continued for the determination time (Yes in step S105), the information processing unit 10 proceeds to the process in step S106.

The information processing unit 10 starts detection by the myoelectric sensor 4 (step S106). For example, the information processing unit 10 supplies electric power from the power source 1B to the myoelectric sensor 4. As a result, the myoelectric sensor 4 starts detecting myoelectric potential information by the plurality of electrodes 41. When the information processing unit 10 starts the detection by the myoelectric sensor 4, the process proceeds to step S107.

The information processing unit 10 acquires the detection result of the myoelectric sensor 4 (step S107). The information processing unit 10 functions as the acquisition unit 11 by executing step S107. Then, the information processing unit 10 estimates the grasping state of the hand H based on the myoelectric potential information detected by the myoelectric sensor 4 (step S108). For example, the information processing unit 10 uses the myoelectric potential information of the myoelectric sensor 4 when detecting the start of the grasping posture P2 as an initial value, and the amount of change and the posture data between the initial value myoelectric potential information and the detected myoelectric potential information. Based on 12A, the grasping state of the hand H is estimated. When the information processing unit 10 stores the estimation result in the storage unit 12, the information processing unit 10 proceeds to the process in step S109.

The information processing unit 10 executes the output processing of the estimation result (step S109). For example, the information processing unit 10 transmits the estimation result to an external electronic device or the like via the communication unit 14 by executing the output process. Then, when the output process is completed, the information processing unit 10 proceeds to step S111, which will be described later.

Further, when the information processing unit 10 determines that the grasping posture P2 does not continue for the determination time (No in step S105), the information processing unit 10 proceeds to the process in step S110. The information processing unit 10 executes an output process of an estimation result indicating that the grasping state of the hand H has not been estimated (step S110). For example, the information processing unit 10 transmits the estimation result to an external electronic device or the like via the communication unit 14 by executing the output process. Then, when the output process is completed, the information processing unit 10 proceeds to step S111, which will be described later.

The information processing unit 10 determines whether or not it has ended (step S111). For example, the information processing unit 10 determines that the attachment unit 2 has been removed from the hand H based on the detection result of the myoelectric sensor 4. For example, when the information processing unit 10 receives the end request via the communication unit 14, it determines that the end is complete. Then, when the information processing unit 10 determines that the process is not completed (No in step S111), the information processing unit 10 returns the process to step S101 already described, and repeats a series of processes after step S101. When the information processing unit 10 determines that the process is complete (Yes in step S111), the information processing unit 10 terminates the processing procedure shown in FIG.

If the information processing unit 10 determines that the posture of the hand H is not the grasping posture P2 (No in step S103), the information processing unit 10 proceeds to step S112. The information processing unit 10 determines whether or not the detection by the myoelectric sensor 4 has started (step S112). When the information processing unit 10 determines that the detection by the myoelectric sensor 4 has not started (No in step S112), the information processing unit 10 proceeds to step S110 already described. If the information processing unit 10 determines that the detection by the myoelectric sensor 4 has started (Yes in step S112), the information processing unit 10 proceeds to step S113.

The information processing unit 10 ends the detection by the myoelectric sensor 4 (step S113). For example, the information processing unit 10 stops the power supply from the power supply 1B to the myoelectric sensor 4. As a result, the myoelectric sensor 4 ends the detection of myoelectric potential information by the plurality of electrodes 41. When the information processing unit 10 finishes the detection by the myoelectric sensor 4, the information processing unit 10 proceeds to step S111 already described.

In the processing procedure shown in FIG. 5 above, the information processing unit 10 functions as the estimation unit 13 by executing the processes of steps S102 to S106 and steps S108 to S111.

Further, in the processing procedure shown in FIG. 5 above, the processing in step S110 may be deleted. That is, if the information processing unit 10 has not estimated the grasped state of the hand H, the processing procedure may be such that the estimation result is not output. Further, in the processing procedure, the determination process (step S105) of whether or not the grasping posture P2 has continued for the determination time may be deleted.

As described above, in the wearable device 1, when the posture of the hand H changes from the non-grasping posture P1 to the grasping posture P2 by the posture sensor 3, the myoelectric sensor 4 starts detecting the myoelectric potential information. As a result, the wearable device 1 does not need to drive the myoelectric sensor 4 when the posture of the hand H is the non-grasp posture P1. As a result, the wearable device 1 can reduce the power consumption by setting the posture P1 in which the posture of the hand H is not grasped into the standby state of the myoelectric sensor 4.

Further, in the wearable device 1, if the posture of the hand H detected by the posture sensor 3 is not the grasping posture P2, the estimation unit 13 does not estimate the grasping state of the hand H. As a result, the wearable device 1 can avoid guessing the grasping state of the hand H in the case of the non-grasping posture P1 in which the myoelectric potential information of the myoelectric sensor 4 may include noise or the like. As a result, the wearable device 1 can improve the estimation accuracy of the grasped state of the hand H based on the detection result of the myoelectric sensor 4.

Further, the wearable device 1 estimates the grasping state of the hand H by the estimation unit 13 when the grasping posture P2 of the hand H continues over the determination threshold value. As a result, the wearable device 1 does not estimate the grasping state when the hand H temporarily changes to the grasping posture P2. As a result, the wearable device 1 can improve the detection accuracy that the user has changed the hand H to the grasping posture P2, so that the grasping state of the hand H is estimated even though the user's hand H is not the grasping posture P2. Can be suppressed.

Further, the wearable device 1 uses the myoelectric potential information of the myoelectric sensor 4 when detecting the start of the grasping posture P2 of the hand H as an initial value, and sets the initial value and the myoelectric potential information detected after the start of the grasping posture P2. Based on this, the grasping state of the hand H is estimated. As a result, the wearable device 1 can estimate the grasping state based on the amount of change from the reference based on the myoelectric potential information at the start of the grasping posture P2 of the hand H. As a result, the wearable device 1 can estimate the grasping state in consideration of the contact state between the myoelectric sensor 4 and the hand H, so that the estimation accuracy of the grasping state can be improved.

Further, when the wearable device 1 detects the end of the grasping posture P2 of the hand H when the grasping state of the hand H is estimated, the estimation of the grasping state ends. As a result, the wearable device 1 can avoid estimating the grasping state of the hand H using the myoelectric sensor 4 when the grasping posture P2 of the hand H changes to another posture. As a result, the wearable device 1 can suppress the influence of the displacement between the myoelectric sensor 4 and the hand H, and can improve the estimation accuracy of the muscle strength of the hand H using the myoelectric sensor 4.

Further, the wearable device 1 uses the myoelectric potential information detected by the myoelectric sensor 4 when the posture of the hand H detected by the posture sensor 3 is the grasping posture indicated by the posture data 12A stored in the storage unit 12. Based on this, the grasping state of the user's hand H is estimated. As a result, the wearable device 1 can estimate the grasping state of the hand H when the grasping posture P2 of the hand H indicated by the posture data 12A. As a result, the wearable device 1 can estimate the grasping state of the hand H according to the set grasping posture P2 by setting various different grasping postures P2.

The above-mentioned first embodiment shows an example, and various modifications and applications are possible.

The wearable device 1 according to the first embodiment described above has a grasped state of the hand H based on the myoelectric potential information detected by the myoelectric sensor 4 when the posture of the hand H detected by the posture sensor 3 is the grasped posture. Is estimated, but is not limited to this. For example, the wearable device 1 may add the detection result of the posture sensor 3 to the estimation of the grasping state of the hand H.

[Modified example of the first embodiment (1)]
For example, in the wearable device 1 according to the first embodiment, the arrangement of the posture sensor 3 can be changed.

FIG. 6 is a diagram for explaining an example of the wearable device 1 according to the modified example (1) of the first embodiment. As shown in FIG. 6, the wearable device 1 is an open finger glove, a thimble glove, or the like that exposes the fingers, fingertips, etc. of the user's hand H when worn on the user's hand H. The wearable device 1 may or may not expose the palm of the hand H, for example.

The wearable device 1 according to the modified example (1) of the first embodiment includes a mounting portion 2A, a posture sensor 3, a myoelectric sensor 4, and an information processing unit 10. In the modification (1) of the first embodiment, the wearable device 1 has one posture sensor 3 straddling the metacarpal bones of five fingers of the hand H and three myoelectric sensors 4. However, the number of sensors is not limited.

The mounting portion 2A is a glove that exposes the fingers, fingertips, etc. of the user's hand H on the wearable device 1. The mounting portion 2A is formed of, for example, cloth, synthetic fibers, leather, or the like. The mounting portion 2A covers the back of the user's hand H and deforms according to the movement of the hand H. The mounting portion 2A is provided with a posture sensor 3, a myoelectric sensor 4, and an information processing unit 10. When the attachment portion 2A is attached to the user's hand H, for example, the posture sensor 3 and the myoelectric sensor 4 are brought into direct or indirect contact with the skin of the hand H. The mounting portion 2A has a non-grasping posture P1 indicating a posture in which the user does not hold the hand H and a grasping posture P2 indicating a posture in which the user holds the hand H when mounted on the hand H. For example, when the mounting portion 2A is in the grasping posture P2, the user's hand H is in a state where all or part of the fingers are bent inward. In this case, the back of the hand H changes to a rounded state in the direction in which the fingers are lined up. Therefore, in the mounting portion 2A, one posture sensor 3 is arranged on the back of the hand H straddling the metacarpal bones of a plurality of fingers, and myoelectric sensors 4 are arranged on both sides thereof.

The posture sensor 3 is configured to detect the bending of the back of the user's hand H. The posture sensor 3 detects the bending of the back of the hand H, and as the bending increases, the resistance value also increases. One of the posture sensors 3 is connected to the power supply 1B of the wearable device 1, and the other is grounded. In the present embodiment, the posture sensor 3 is electrically connected to the information processing unit 10 and outputs the detection result to the information processing unit 10.

In the present embodiment, the wearable device 1 describes a case where one posture sensor 3 is arranged on the back of the hand H with a length straddling the metacarpal bones of five fingers, but the present invention is not limited to this. For example, in the wearable device 1, each of the plurality of posture sensors 3 may be arranged side by side so as to straddle each of the metacarpal bones of five fingers. Further, the wearable device 1 may have a configuration in which the posture sensor 3 is arranged on a part of the back of the hand H that bends when the grasping posture P2.

The information processing unit 10 is a device that processes information detected by one posture sensor 3 and three myoelectric sensors 4. The information processing unit 10 has, for example, a function of determining the posture of the user's hand H. The information processing unit 10 has, for example, a function of estimating the grasping state of the user's hand H.

As shown in FIG. 6, when the wearable device 1 is attached to the user's hand H, the wearable device 1 determines the posture of the hand H based on the detection result of the posture sensor 3. For example, it is assumed that the user's hand H is the non-grasp posture P1 with the hand H open. In this case, since the resistance value detected by the posture sensor 3 does not exceed the reference resistance value Rth, the wearable device 1 determines that the posture of the user's hand H is the non-grasp posture P1 and does not detect the myoelectric sensor 4. .. After that, it is assumed that the user's hand H changes from the non-grasping posture P1 to the grasping posture P2 by, for example, holding a tool. In this case, since the resistance value detected by the posture sensor 3 exceeds the reference resistance value Rth, the wearable device 1 determines that the posture of the user's hand H is the grasping posture P2. Then, the wearable device 1 starts the detection of the myoelectric sensor 4, and estimates the grasping state of the hand H based on the detection result of the myoelectric sensor 4. For example, the wearable device 1 estimates the state of fingers, grip strength, and the like based on the frequency component of the myoelectric potential information for each electrode 41 of the myoelectric sensor 4.

For example, the wearable device 1 estimates the movement, grip strength, etc. of the hand H by utilizing the characteristic that the myoelectric signals indicated by the myoelectric potential information of the myoelectric sensor 4 indicate different frequency identification depending on the movement of the hand H. For example, the wearable device 1 can extract an integrated value average potential, a frequency spectrum, and the like as parameters of a myoelectric signal. The integrated value average potential is a rectified and averaged myoelectric signal, and it is known that the greater the muscle strength, the more active the muscle activity and the larger the value. Therefore, the wearable device 1 detects the force of the hand H by the detection method using the integrated value average potential. Further, the wearable device 1 may estimate the fine operation of the hand H by using the gesture detection method using the frequency spectrum.

After that, it is assumed that the user's hand H transitions from the grasping posture P2 to the non-grasping posture P1. In this case, since the resistance value detected by the posture sensor 3 does not exceed the reference resistance value Rth, the wearable device 1 determines that the posture of the user's hand H is the grasping posture P2, and ends the detection of the myoelectric sensor 4.

As described above, in the wearable device 1 according to the modified example (1) of the first embodiment, the mounting portion 2A is an open finger glove, and the posture sensor 3 is positioned on the back of the hand H by the mounting portion 2A. In the wearable device 1, when the mounting portion 2A is mounted on the hand H, the fingertips of the hand H are exposed, so that the user can freely use the fingertips. As a result, the wearable device 1 can improve the accuracy of detecting the posture of the hand H by the posture sensor 3 and improve the operability of the user.

[Modified Example (2) of the First Embodiment]
For example, the wearable device 1 according to the first embodiment can correspond to various different postures of the hand H.

For example, activities of daily living are indicators of basic behaviors that are indispensable for conducting daily living. In many cases, complicated movements are combined with daily movements such as eating, excreting, and bathing, and various problems occur when the grip strength is weakened. Problems include, for example, not being able to hold heavy objects for a long time, being unable to open the lids of bottles and PET bottles, having difficulty opening twist-type doors, and having difficulty eating. Therefore, the wearable device 1 according to the modified example (2) of the first embodiment can grasp and support the user's movement by combining the detection of the posture of the hand H and the estimation of the grip strength.

The wearable device 1 according to the modified example (2) of the first embodiment includes a mounting portion 2, a posture sensor 3, a myoelectric sensor 4, and an information processing unit 10. In the wearable device 1 according to the modification (2) of the first embodiment, the mounting portion 2 may be replaced with the mounting portion 2A. The information processing unit 10 includes an acquisition unit 11, a storage unit 12, an estimation unit 13, and a communication unit 14.

FIG. 7 is a diagram showing an example of the grasping posture P2 of the wearable device 1 according to the modified example (2) of the first embodiment. As shown in FIG. 7, the wearable device 1 is associated with the user's scene and the grasping posture P2 by the posture data 12A. Scenes include, for example, holding luggage, opening lids, opening doors, eating meals, and the like. Information such as the bending amount and shape of the finger corresponding to the posture of the hand H is associated with the grasping posture P2, for example.

In the example shown in FIG. 7, the posture data 12A associates information on the posture of hanging the bag with the hand H with the scene of holding the luggage. The posture data 12A associates information on the posture of holding the lid with a finger with the scene of opening the lid. The posture data 12A associates information on the posture of grasping the doorknob with the scene of opening the door. The posture data 12A associates information on the posture of grasping the tableware with the scene of eating a meal.

The estimation unit 13 of the information processing unit 10 specifies the posture corresponding to the scene to be estimated based on the posture data 12A. When the posture of the hand H determined based on the detection result of the posture sensor 3 and the posture corresponding to the target scene match or are similar to each other, the estimation unit 13 uses the myoelectric sensor 4 to grip the hand H. Etc. is estimated. The estimation unit 13 can provide a function of outputting the estimated grip strength as an operation result in the target scene of the user.

For example, assume that the target scene is a scene with luggage. In this case, when the wearable device 1 is attached to the user's hand H, the posture sensor 3 detects the posture of the hand H. The wearable device 1 specifies the grasping posture P2 of the target scene based on the posture data 12A. When the posture detected by the posture sensor 3 is the grasping posture P2, the wearable device 1 estimates the grip strength (grasping state) of the user's hand H based on the myoelectric potential information detected by the myoelectric sensor 4. The wearable device 1 associates the estimated grip strength with the identification information indicating the target scene, stores it in the storage unit 12, and transmits it to an external electronic device. As a result, the wearable device 1 can indicate whether or not the user's grip strength is weakened by showing the user's grip strength in the target scene.

As described above, in the wearable device 1 according to the modified example (2) of the first embodiment, the posture data 12A includes the grasping posture P2 according to the scene, and the posture of the hand H detected by the posture sensor 3 is in the scene. In the case of the corresponding grasping posture, the grasping state of the hand H is estimated. As a result, the wearable device 1 can be worn by the user in daily life to estimate the grasping state of the hand H according to the scene of daily life. For example, when the user cannot hold a heavy object for a long time, by setting "hold luggage" as the scene of the posture data 12A, the wearable device 1 can grasp the posture P2 when holding the luggage. Estimate the grasped state. As a result, the wearable device 1 can contribute to the support of daily life based on the grasped state of the user by outputting the grasped state of the hand H estimated in the scene.

Note that the modified example (2) of the first embodiment may be applied to the wearable device 1 of another embodiment or modified example.

(Second Embodiment)
[Outline of the system using the wearable device 1 according to the second embodiment]
Next, the second embodiment will be described. FIG. 8 is a diagram for explaining an example of the system according to the second embodiment. As shown in FIG. 8, system 100 is, for example, a system for interactive gameplay of video games. The system 100 includes a wearable device 1 according to the first embodiment, and a head-mounted display (HMD) 110. The HMD 110 is worn, for example, in a manner similar to eyeglasses, goggles, or helmets and is configured to display a video game or other content to User U. The HMD 110 provides an immersive experience for the user U by providing a display mechanism that is close to the eyes of the user U. The HMD 110 can provide a display area for each of the user U's eyes, which occupies most or the entire field of view of the user U.

The HMD 110 can be connected to the computer 120 by wire or wirelessly. Computer 120 includes, but is not limited to, for example, game consoles, personal computers, laptops, tablet computers, mobile devices, mobile phones, tablets, thin clients, set-top boxes, media streaming devices. In this embodiment, the computer 120 is configured to run a video game and output video and audio from the video game for rendering by the HMD 110.

The system 100 can use the wearable device 1 as a means for the user U to input a video game. In the example shown in FIG. 8, the system 100 has the wearable device 1 attached to the left and right hands H of the user U. Then, the wearable device 1 outputs the estimated grasping state of the hand H, the grip strength, and the like to the cooperation device such as the HMD 110 and the computer 120. For example, the computer 120 can execute a process of reflecting the grasping state of the hand H, the grip strength, and the like in AR, VR, a video game, and the like. For example, in the case of a boxing game, the computer 120 changes the damage given to the opponent according to the grip strength of the user U input from the wearable device 1, and outputs a video and audio showing the result to the HMD 110. The HMD 110 outputs video and audio from the computer 120 to the user U.

The camera 130 is connected to the computer 120. The camera 130 can be configured to capture an image of the interactive environment in which the user U is located. These captured images can be analyzed to determine the position and movement of the user U, HMD110, and wearable device 1. The HMD 110 may include one or more lights that can be tracked to determine the position and orientation of the HMD 110. The camera 130 may include one or more microphones for capturing sound from an interactive environment. The camera 130 may be configured to include a plurality of image capture devices (eg, stereoscopic pairs of cameras), an IR camera, a depth of field camera, and combinations thereof. In the present embodiment, the case where the camera 130 includes an IR camera will be described.

The computer 120 functions as a thin client that communicates with the cloud gaming provider 150 on the network 200. The cloud gaming provider 150 maintains and runs the video game being played by user U. The computer 120 transmits the inputs from the wearable device 1, the HMD 110, and the camera 130 to the cloud gaming provider. The cloud gaming provider 150 processes inputs that affect the game state of the video game being run. The output from the running video game, such as video data, audio data, and tactile feedback data, is transmitted to the computer 120.

FIG. 9 is a diagram for explaining an example of the wearable device 1 in the system 100 according to the second embodiment. As shown in FIG. 9, the wearable device 1 is attached to the hand H of the user U. Then, the user U holds the pen 160 with the hand H equipped with the wearable device 1. The pen 160 is provided with a reflection marker at the pen tip. When the posture of the hand H detected by the posture sensor 3 is the grasping posture, the wearable device 1 estimates the grasping state of the hand H based on the myoelectric potential information detected by the myoelectric sensor 4. In this case, the wearable device 1 estimates the grasping state and the grip strength in which the object is grasped by the thumb and the index finger.

The computer 120 detects the position of the hand H and the pen tip of the pen 160 in space by analyzing the image captured by the camera 130 of the user U. For example, the computer 120 irradiates infrared rays from the HMD 110, the camera 130, and the like, and detects the position of the pen tip of the pen 160 in space based on the reflected light. Then, the computer 120 obtains the grip strength when the user U is holding the pen 160 based on the grasping state and the grip strength estimated by the wearable device 1, and determines the thickness of the line according to the grip strength. For example, the computer 120 increases the drawing line as the force of the user U holding the pen 160 increases. The computer 120 causes the HMD 110 to display an image drawn based on the position of the pen tip of the pen 160 that has moved in space and the grip strength of the hand H. As a result, the system 100 can display the image drawn by the user U in the space freehand on the HMD 110 based on the grasped state of the hand H estimated by the wearable device 1.

In this embodiment, the wearable device 1, the HMD 110, and the camera 130 may themselves be networked devices that connect to the network 200 to communicate with the cloud gaming provider 150. For example, computer 120 may be a local network device, such as a router, that does not perform video game processing separately, but facilitates the passage of network traffic. The connection to the network by the wearable device 1, the HMD 110, and the camera 130 may be wired or wireless.

As described above, when the posture of the hand H detected by the posture sensor 3 of the wearable device 1 is the grasping posture, the system 100 estimates the grasping state of the hand H based on the myoelectric potential information detected by the myoelectric sensor 4. To do. The system 100 outputs the estimation result estimated by the wearable device 1 to the computer 120, and outputs the processing result of the computer 120 to the HMD 110. As a result, the system 100 improves the accuracy of estimating the muscle strength of the hand H using the myoelectric sensor 4 of the wearable device 1, so that the reality of the output of the HMD 110 can be improved. Further, in the system 100, when the posture of the hand H is the grasping posture, the hand H may be imaged by the camera 130, so that the power consumption can be reduced.

Note that the second embodiment may be combined with the technical ideas of other embodiments and modifications.

Although the preferred embodiments of the present disclosure have been described in detail with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person with ordinary knowledge in the technical field of the present disclosure can come up with various modifications or modifications within the scope of the technical ideas described in the claims. Of course, it is understood that it belongs to the technical scope of the present disclosure.

Further, the effects described in the present specification are merely explanatory or exemplary and are not limited. That is, the technique according to the present disclosure may exhibit other effects apparent to those skilled in the art from the description herein, in addition to or in place of the above effects.

In addition, it is possible to create a program for causing the hardware such as the CPU, ROM, and RAM built in the computer to exhibit the same function as the configuration of the information processing unit 10, and read the program recorded by the computer. Possible recording media may also be provided.

Further, each step related to the processing of the wearable device 1 in the present specification does not necessarily have to be processed in chronological order in the order described in the flowchart. For example, each step related to the processing of the information processing unit may be processed in an order different from the order described in the flowchart, or may be processed in parallel.

The following configurations also belong to the technical scope of the present disclosure.
(1)
A posture sensor that detects the posture of the user's hand and
An electromyographic sensor that detects the myoelectric potential information of the hand and
When the posture detected by the posture sensor is the grasping posture, an estimation unit that estimates the grasping state of the hand of the user based on the myoelectric potential information detected by the myoelectric sensor.
Wearable device with.
(2)
The wearable device according to (1) above, wherein the myoelectric sensor has a dry electrode that comes into contact with the hand and detects the myoelectric potential information indicating the action potential generated in the muscle of the hand.
(3)
The wearable device according to (1) or (2) above, wherein the posture sensor includes a bending sensor that detects the movement of the fingers of the hand.
(4)
The wearable device according to (1), wherein the myoelectric sensor starts detecting myoelectric potential information when the posture detected by the posture sensor becomes the grasping posture.
(5)
The wearable device according to (1), wherein the estimation unit does not estimate the grasping state of the hand of the user when the posture detected by the posture sensor is not the grasping posture.
(6)
The wearable device according to any one of (1) to (5) above, wherein the estimation unit estimates the grasping state when the grasping posture continues for a determination time.
(7)
The estimation unit uses the myoelectric potential information of the myoelectric sensor when the start of the grasping posture is detected as an initial value, and the estimation unit is based on the initial value and the myoelectric potential information detected after the start of the grasping posture. The wearable device according to any one of (1) to (6) above, which estimates the grasping state.
(8)
The wearable device according to any one of (1) to (7) above, wherein when the estimation unit detects the end of the grasping posture when estimating the grasping state, the estimation of the grasping state ends.
(9)
Further provided with a mounting portion to be mounted on the hand of the user.
The wearable device according to any one of (1) to (8), wherein the mounting portion positions the posture sensor at a position of the hand that can detect the grasping posture when the mounting portion is mounted on the hand.
(10)
The wearing portion is a glove that exposes at least the fingertips of the user.
The wearable device according to (9), wherein the posture sensor is positioned on the back of the hand by the mounting portion.
(11)
Further, a storage unit for storing posture data indicating the grasping posture of the user's hand is provided.
The estimation unit is based on the myoelectric potential information detected by the myoelectric sensor when the posture detected by the posture sensor is the grasping posture indicated by the posture data stored in the storage unit. The wearable device according to any one of (1) to (10), which estimates the grasping state of the hand of the user.
(12)
The wearable device according to (11), wherein the posture data includes data indicating the grasping posture according to a scene in which the user operates.
(13)
An acquisition unit that acquires the detection results of a posture sensor that detects the posture of the user's hand and an electromyographic sensor that detects the myoelectric potential information of the hand.
When the posture detected by the posture sensor is the grasping posture, an estimation unit that estimates the grasping state of the hand of the user based on the myoelectric potential information detected by the myoelectric sensor.
Information processing unit equipped with.
(14)
The computer
A step of acquiring the detection results of the posture sensor that detects the posture of the user's hand and the myoelectric sensor that detects the myoelectric potential information of the hand, and
When the posture detected by the posture sensor is the grasping posture, the step of estimating the grasping state of the hand of the user based on the myoelectric potential information detected by the myoelectric sensor, and
Information processing methods including.
(15)
Wearable devices worn in the user's hands
A linkage device that cooperates with the wearable device and
With
The wearable device is
A posture sensor that detects the posture of the user's hand and
An electromyographic sensor that detects the myoelectric potential information of the hand and
When the posture detected by the posture sensor is the grasping posture, an estimation unit that estimates the grasping state of the hand of the user based on the myoelectric potential information detected by the myoelectric sensor.
With
The cooperation device is a system that executes processing based on the grasping state estimated by the wearable device.

1 Wearable device 2 Mounting unit 2A Mounting unit 3 Attitude sensor 4 Myoelectric sensor 10 Information processing unit 11 Acquisition unit 12 Storage unit 12A Attitude data 13 Estimating unit 14 Communication unit 100 System 110 Head-mounted up display (HMD)
120 Computer 130 Camera H Hand P1 Non-grasp posture P2 Grasp posture U User

Claims (14)

A posture sensor that detects the posture of the user's hand and
An electromyographic sensor that detects the myoelectric potential information of the hand and
When the posture detected by the posture sensor is the grasping posture, an estimation unit that estimates the grasping state of the hand of the user based on the myoelectric potential information detected by the myoelectric sensor.
Wearable device with. The wearable device according to claim 1, wherein the myoelectric sensor has a dry electrode that comes into contact with the hand and detects the myoelectric potential information indicating the action potential generated in the muscle of the hand. The wearable device according to claim 2, wherein the posture sensor includes a bending sensor that detects the movement of the fingers of the hand. The wearable device according to claim 1, wherein the myoelectric sensor starts detecting the myoelectric potential information when the posture detected by the posture sensor becomes the grasping posture. The wearable device according to claim 1, wherein the estimation unit does not estimate the grasping state of the hand of the user when the posture detected by the posture sensor is not the grasping posture. The wearable device according to claim 1, wherein the estimation unit estimates the grasping state when the grasping posture continues for a determination time. The estimation unit uses the myoelectric potential information of the myoelectric sensor when the start of the grasping posture is detected as an initial value, and the estimation unit is based on the initial value and the myoelectric potential information detected after the start of the grasping posture. The wearable device according to claim 1, wherein the grasping state is estimated. The wearable device according to claim 1, wherein when the estimation unit detects the end of the grasping posture when the grasping state is estimated, the estimation of the grasping state is completed. Further provided with a mounting portion to be mounted on the hand of the user.
The wearable device according to claim 1, wherein the mounting portion positions the posture sensor at a position of the hand that can detect the grasping posture when the mounting portion is mounted on the hand. The wearing portion is a glove that exposes at least the fingertips of the user.
The wearable device according to claim 9, wherein the posture sensor is positioned on the back of the hand by the mounting portion. Further, a storage unit for storing posture data indicating the grasping posture of the user's hand is provided.
The estimation unit is based on the myoelectric potential information detected by the myoelectric sensor when the posture detected by the posture sensor is the grasping posture indicated by the posture data stored in the storage unit. The wearable device according to claim 1, wherein the state of grasping the hand of the user is estimated. The wearable device according to claim 11, wherein the posture data includes data indicating the grasped posture according to a scene in which the user operates. An acquisition unit that acquires the detection results of a posture sensor that detects the posture of the user's hand and an electromyographic sensor that detects the myoelectric potential information of the hand.
When the posture detected by the posture sensor is the grasping posture, an estimation unit that estimates the grasping state of the hand of the user based on the myoelectric potential information detected by the myoelectric sensor.
Information processing unit equipped with. The computer
A step of acquiring the detection results of the posture sensor that detects the posture of the user's hand and the myoelectric sensor that detects the myoelectric potential information of the hand, and
When the posture detected by the posture sensor is the grasping posture, the step of estimating the grasping state of the hand of the user based on the myoelectric potential information detected by the myoelectric sensor, and
Information processing methods including.

Images