WO2024171759A1 - Information processing device and information processing method - Google Patents

Abstract

This information processing device comprises: a measurement unit that measures a biological signal of a user; and an operation condition calculation unit that calculates, on the basis of the measured biological signal of the user, an operation condition for execution of an operation of the user.

Description

Information processing device and information processing method

This disclosure relates to an information processing device and an information processing method.

Several systems have been proposed that can measure muscle strength and estimate biological conditions. For example, previously, means have been disclosed for appropriately measuring myoelectric potential without being affected by conditions or characteristics such as the attachment of the myoelectric potential measuring device or the shape of the skin surface.

JP 2016-27848 A

However, when operating a system using muscle activity generated by a user's movements or application of force as input, it is necessary to take into account the effects of individual differences in muscle strength and muscle sensation, fatigue due to operation, and changes in muscle strength due to growth and aging. These factors can lead to failure to meet the operating conditions for the user's operation input, resulting in a loss of consistency in operation and an increase in operational errors and an increase in the operating burden. Patent Document 1 discloses technology that can properly measure myoelectric potential, but does not disclose optimizing operations that use muscle activity as input. Therefore, even if muscle strength can be accurately measured, Patent Document 1 has the problem that it is not possible to achieve consistent operability or accurate, low-burden operation that is not affected by the characteristics and condition of the user's muscles.

The present disclosure has been made in consideration of the above problems, and provides an information processing device and information processing method that can achieve consistent operability, accurate, and low-stress operation without being affected by the characteristics or condition of the user's muscles.

The information processing device of the present disclosure includes: a measurement unit that measures a biosignal of a user;
The device further includes an operation condition calculation unit that calculates an operation condition for performing an operation by the user based on the measured biological signal of the user.

1 is a diagram illustrating a configuration of an information processing device according to a first embodiment. 2 is a functional block diagram of a calculation unit in the information processing device according to the first embodiment. FIG. 1 is a functional block diagram of an information processing device according to a first embodiment, in which an input mechanism is provided for a calculation unit. FIG. 1 is a diagram illustrating a hardware configuration of an information processing device according to a first embodiment. 4 is a diagram for explaining the operation of an application program in the information processing device according to the first embodiment; FIG. 4 is a flowchart illustrating an operation of the information processing device according to the first embodiment. 6 is an example of a screen showing the contents of an operation for the configuration of the information processing device according to the first embodiment. FIG. 4 is a diagram showing operation contents of the information processing device according to the first embodiment. 4A and 4B are diagrams illustrating a measurement state of a user of the information processing device according to the first embodiment; 11A to 11C are diagrams illustrating an example of instructions given until a user of the information processing device according to the first embodiment assumes an appropriate posture. 1A and 1B are diagrams illustrating an example of a cue to start measurement, such as a countdown in the information processing device according to the first embodiment; 11A and 11B are diagrams illustrating an example of a termination signal given after a certain period of time has elapsed by the information processing device according to the first embodiment; 5A and 5B are diagrams illustrating an example of a display indicating the lapse of a remaining time in the information processing device according to the first embodiment; 5A and 5B are diagrams illustrating measurement results of myoelectric potentials by the information processing device according to the first embodiment. 5 is a diagram showing the RMS of a myoelectric potential measured by the information processing device according to the first embodiment; FIG. FIG. 11 is a functional block diagram of an information processing device according to a second embodiment. FIG. 13 is a functional block diagram of an information processing device according to a third embodiment. FIG. 1 is a hardware configuration diagram showing an example of a computer that realizes an arithmetic unit of an information processing device according to the first to fourth embodiments.

Below, a preferred embodiment of the present disclosure will be described in detail with reference to the attached drawings. Note that in this specification and drawings, components having substantially the same functional configuration will be denoted by the same reference numerals to avoid duplicated explanations. Note that the explanation will be given in the following order.

0. Background of the embodiment 1. First embodiment 1-1. Configuration of the information processing device 100 1-2. Outline of the flow of the calibration process 1-3. Functional block diagram of the arithmetic device 102 1-4. Description of the operation of the information processing device 100 1-5. Adjustment of threshold value 2. Second embodiment 3. Third embodiment 4. Fourth embodiment 5. Hardware configuration

<0. Background of the embodiment>
When using muscle activity to input motion and force, there are individual differences in muscle strength and force sensation. In addition, muscle strength changes with fatigue, growth, and aging. These effects can lead to operational errors and failures, leading to reduced operability and increased strain on the user.

Several systems have been proposed that can measure muscle strength and estimate biological states, but in order to optimize the conditions for operations that use movements resulting from muscle activity as input, it is necessary to properly estimate the characteristics and state of the muscles active during the movement. To do this, it is necessary to consider appropriate measurement and estimation methods, such as having the user perform specific movements depending on the content of the operation.

In an embodiment, the system instructs the user to perform an appropriate action, or implicitly detects when the user has performed a pre-defined action. By measuring biometric information in association with the muscle activity caused by this action and estimating the individual muscle characteristics and physical condition, it is possible to calculate the optimal execution conditions for a specific operation.

According to this embodiment, it is possible to realize consistent operability and accurate operation with less strain, without being affected by the individual characteristics or state of the user's muscles.
1. First embodiment
<1-1. Configuration of information processing device 100>
1 is a diagram showing a configuration of an information processing device 100 according to a first embodiment. As shown in FIG. 1, the information processing device 100 includes a sensor 101, a computing device 102, and an output mechanism 103.

The sensor 101 is attached to the user's body and measures biosignals indicating the activity of the user's muscles. The sensor 101 can also measure muscle tension and muscle strength. The biosignals include biosignals indicating the user's skin surface myoelectric potential. The sensor 101 may also measure physical changes in the user. Measurements of physical changes may use ultrasound, magnetism, infrared rays, vibration (inertial sensor), blood flow, deformation of the skin surface (camera, distortion sensor), etc. The sensor 101 may also be of a type that measures the force applied by the user. In this case, a pressure sensor, force sensor, etc. is used.

The sensor 101 may also measure factors that affect muscle activity as an auxiliary measure. The sensor 101 may measure joint flexibility and range of motion. For example, an inertial sensor, a camera, etc. are used to measure joint flexibility and range of motion. The sensor 101 may also measure mental states such as excitement level. For example, blood flow, heart rate, body temperature, etc. are measured to measure mental states such as excitement level.

The computing device 102 calculates operation conditions for a user operation based on the measurement results from the sensor 101. The operation conditions are conditions for executing an operation in response to a user operation. The computing device 102 also has an application program that executes an operation when the operation conditions are satisfied.

The computing device 102 may be a PC (personal computer), a smartphone, AR (Augmented Reality)/XR (Extended Reality) goggles, etc.

The output mechanism 103 is a device that gives instructions to the user. The output mechanism 103 is, for example, a display, a speaker, a vibration device, etc.

Operations that can be measured include maintaining or changing the user's posture, applying or releasing force, straining or relaxing, and complex movements. Examples of complex movements include walking, running, grasping, moving, transforming, and releasing objects.

As an example of the configuration of the information processing device 100, the sensor 101, the computing device 102, and the output mechanism 103 may be included in one device. For example, the information processing device 100 is a wearable device such as a smart watch. The sensor (and in some cases some output mechanisms such as vibration) may be independent as a measurement device. For example, an armband-type wearable device, AR, VR (Virtual Reality) goggles, a PC, a smartphone, etc. All of the sensors 101, the computing device 102, and the output mechanism 103 may be independent. For example, the information processing device 100 is an armband-type wearable device, a PC, a display, etc.

<1-2. Overview of calibration process flow>
Next, an overview of the flow of calibration of the processing execution process for muscle activity input will be described. The overview of the flow of the calibration process of the information processing device 100 is as follows:

1. Starting up the device (information processing device 100) 2. Starting up an application program 3. Calling up calibration processing 3-1. Instructing the user to perform a predetermined operation through the output mechanism 103 3-2. Measuring a biological signal through the sensor 101 3-3. Executing processing of the operation 3-4. Reflecting the measurement result in the operating conditions of the operation 4. Ending the application program 5. Ending the device

<1-3. Functional block diagram of the arithmetic device 102>
FIG. 2 is a functional block diagram of the arithmetic unit 102 in the information processing device 100 according to the first embodiment.

As shown in FIG. 2, the computing device 102 has a muscle activity measuring unit 201, a muscle activity estimating unit 202, an operation condition calculating unit 203, an application control unit 204, and an instruction generating unit 205.

The muscle activity measuring unit 201 acquires signals related to the user, including biosignals measured by the sensor 101.

The muscle activity estimation unit 202 outputs an index of the signal related to the user acquired by the muscle activity measurement unit 201. The "index" is an index that indicates the state of muscle strength. For example, from voltage fluctuations in mV measured at periods of several to several tens of ms through an electromyography sensor, indexes such as RMS (Root Mean Square), maximum amplitude, average amplitude, and integral value are calculated and used as indexes of muscle strength. Here, RMS is the square root of the arithmetic mean of the value obtained by squaring data for a certain period of time.

The operation condition calculation unit 203 calculates the operation conditions for the user's operation based on the indicators output from the muscle activity estimation unit 202. If necessary, the operation condition calculation unit 203 updates the operation conditions for the user's operation in the application control unit 204. In other words, the operation condition calculation unit 203 applies the calculated operation conditions to the application program that judges the operation.

The application control unit 204 executes a specific operation within the application program when the user's muscle activity satisfies the operation conditions.

The instruction generation unit 205 issues an instruction to the output mechanism 103 to cause the user to perform a calibration operation. The instruction generation unit 205 issues the instruction to the output mechanism 103 before the operation condition calculation unit 203 calculates the operation conditions.

FIG. 3 is a functional block diagram of the information processing device 100 according to the first embodiment, in which an input mechanism 104 is provided for the arithmetic device 102. As shown in FIG. 3, the arithmetic device 102 according to the modified example has an input mechanism 104 in addition to the functional block diagram shown in FIG. 2.

The input mechanism 104 is used when the user specifies the timing of calibration or performs auxiliary operations during calibration. Inputs to the input mechanism 104 include, for example, touch input from a smartwatch, smartphone, etc., mouse or keyboard input from a PC, controller input from AR/VR goggles, and hand tracking. The operation condition calculation unit 203 calculates the operation conditions when the timing of calibration is specified from the input mechanism 104.

<1-4. Operation of the information processing device 100>
The operation of the information processing device 100 will be described. As shown in Fig. 4, the hardware configuration of the information processing device 100 is such that a user wears a wristband 105 equipped with a sensor 101 on the arm and wears VR goggles 106. Then, the force from the fingertip to the arm is measured through the sensor 101. For example, as shown in Fig. 5, an application program of the information processing device 100 displays a menu on the VR goggles 106 when a specific position 108 on a desk is pressed with a finger 109 with a certain amount of force or more (when the input force exceeds a threshold).

The calibration also adjusts the force applied to the operation to match the user's muscle strength and sense. The reason for this type of calibration is that if the force required for an operation is too great, more force than necessary is applied, and the operation must be repeated. Also, if the force required is too small, there will be frequent errors.

FIG. 6 is a flowchart for explaining the operation of the information processing device 100 according to the first embodiment. In FIG. 6, a case is explained in which an operation threshold is set based on the user's muscle strength when an application program is initialized.

The timing for reading out the calibration process can be 1. when the application program is initialized, or 2. while the application program is running. "When the application program is initialized" is immediately after the application program starts and before the main processing of the application program starts. "While the application program is running" is when the main processing of the application program is running.

Calibration processing is performed while the main processing of an application program is running, for example, when a calibration start signal is input by the user through the input mechanism 104, or when a specific condition for the operation content (such as the occurrence of an erroneous operation) is satisfied. In this case, the calibration processing goes into a standby state after the application program has finished starting up (step S0).

In FIG. 6, the device (information processing device 100) starts an application program (step S0). Next, it is determined whether a calibration signal or the like has been input (step S1), and if a calibration start signal or the like has been input (Yes in step S1), the calibration process starts (step S2). On the other hand, if a calibration start signal or the like has not been input (No in step S1), the device goes into a standby state until a calibration start signal or the like is input.

Examples of parameters to be calibrated include parameters that determine how much force is required to execute a process, and parameters that determine how long force must be applied before a process is executed.

Factors that affect these parameters include differences in muscle strength, changes due to fatigue, changes due to practice, muscle development, weakness, range of motion of joints, flexibility, etc.

Next, the user is notified of the specific operation to be performed (step S3). This notification notifies the user of the content of the previously set calibration operation via the output mechanism 103.

For example, as shown in FIG. 7, a text and diagram explanation of the action of "pressing a specific position on the screen with maximum force with only the index finger extended" is displayed on the screen 110. The action at this time is selected and set so that the characteristics of the muscles used in the operation can be measured. For example, elements such as posture, operation content (pressing, grasping, stretching, pinching, etc.), position, and operation target (when operating on a specific object) are specified. FIG. 8 is a diagram showing the operation content. In FIG. 8, examples of a pressing operation 111_1, a grasping operation 111_2, a pinching operation 111_3, and a stretching operation 111_4 are shown.

Also, it is possible to calibrate a wide range of muscles by including multiple operations in one movement. For example, to calibrate operations that use the arm all at once, the muscle strength of the entire arm can be measured by performing a gripping movement with maximum strength with the hand lowered, as shown in Figure 9.

In addition to maximum muscle strength, a relaxed state or an appropriate amount of force may also be specified. Explanations of the movements may be given in the form of text, audio, video, etc. Before starting measurement, the user may be asked to practice the posture that has been notified to them so that they can correctly perform the posture.

Then, the information processing device 100 waits until the user is ready (step S4). Specifically, the information processing device 100 waits until the user is in an appropriate posture to start the operation. As shown in FIG. 10, the information processing device 100 performs the following operations until the user is in an appropriate posture.

1. Display the target position and posture and instruct the robot to maintain the same posture.
For example, a region and a target posture are displayed at a specific position on the screen, and the user is instructed to assume the same posture (FIG. 10(1)).

2. The difference to the target position and posture is fed back via the output mechanism 103 to prompt the user to adjust the posture.
For example, sound can be played from the target position, the direction to the target position can be displayed by an arrow, and the distance and angle deviation can be displayed in text (Figure 10 (2)).

3. When the robot is close enough to the target position and attitude, a signal is sent through the output mechanism.
For example, when the user assumes a suitable posture, the word "OK" is displayed on the screen, and a sound or vibration is generated to indicate completion (FIG. 10(3)).

Feedback and completion signals can be expressed visually, such as letters, symbols, or shapes, or audio, or tactilely, such as vibration.

Next, the start of the action is notified (step S5), and the user is prompted to perform the action. The biosignals during the action are measured by the sensor 101 (step S6). After that, the user is notified of the timing for the action to end (step S7).

In the first embodiment, the user is instructed to continue the instructed action from the start signal to the end signal, and the biosignal during that time is measured by the sensor 101. For example, the action of "pressing a specific position on the screen with maximum force with only the index finger extended" is continued for three seconds from start to end.

In this case, for example, as shown in FIG. 11, after a countdown ((1) in FIG. 11), a signal to start measurement is given (displaying START in (2) in FIG. 11). Then, as shown in FIG. 12, after a certain period of time has elapsed, a signal to end is given (displaying END in FIG. 12). The signal may be presented visually, audibly, tactilely, etc. During measurement, a display 114 may be given to show the remaining time, etc., as shown in FIG. 13.

The criteria for termination can be other than the passage of time. For example, the process can end automatically when the voltage value of the input myoelectric potential falls below a certain level.

It is also possible for the system to notify the user that it has entered a standby state, and allow the user to start and end the action at any time. For example, the system could notify the user that it has entered a standby state, and have the user perform the action of "pressing a specific position on the screen with maximum force with only the index finger extended, then releasing it." In this case, the system could automatically set the measurement period based on changes in input, or the user could input start and end signals through the input mechanism.

When measuring biosignals, auxiliary sensors may be used as necessary in addition to measuring muscle tension using an electromyography sensor or the like. For example, finger range of motion information using a camera or excitement level information using a heart rate sensor. To show whether the sensor is being operated correctly during measurement, the sensor may be equipped with its own output mechanism and information measured through visual, auditory or tactile stimulation may be presented. A separate device may be prepared that can measure the force applied by the user, and the force value measured by the prepared device may be combined with the obtained biosignal.

Next, it is checked whether there is a problem with the measurement of the biosignals (noise, incorrect signals, etc.) (step S8). If it is determined in step S8 that there is a problem (No in step S8), an instruction is given to repeat the operation (step S9) and the process returns to step S4. If it is determined in step S8 that there is no problem (Yes in step S8), muscle activity is estimated from the measurement results of the biosignals (step S10).

For example, if the occurrence time, frequency, or proportion of abnormal signals such as noise or value jumps during EMG measurement exceeds the tolerance, an instruction will be displayed on the screen to repeat the measurement. If auxiliary sensors are used, these measurement problems will also be included.

　It may also be possible for the user to notify the system if a physical problem occurs, such as a misalignment of the measurement device. If a problem is detected during measurement, the measurement may be interrupted and immediately restarted.

In step S10, muscle activity is estimated from the measurement results of the biosignals, for example, by calculating indices such as RMS, maximum amplitude, average amplitude, and integral value from voltage fluctuations in mV measured at periods of several to several tens of ms through an electromyography sensor, and using these as indices of muscle strength. Here, RMS is the square root of the arithmetic mean of the value obtained by squaring data over a certain period of time. These indices may or may not correspond linearly to muscle strength, and appropriate correction processing may be performed if there is strong nonlinearity due to the measurement conditions. Other signal processing methods such as frequency analysis, pattern matching, and machine learning may also be used.

FIG. 14 is a diagram showing the measurement results of myoelectric potentials by the information processing device 100 according to the first embodiment. FIG. 15 is a diagram showing the RMS of the myoelectric potentials measured by the information processing device 100 according to the first embodiment. As shown in FIG. 15, it is shown that the RMS of the myoelectric potentials is correlated with the myoelectric potentials.

If other sensors are used, a processing method appropriate for the biosignal is applied. Multiple sensors or multiple types of sensors may be used in combination. For example, eight myoelectric potential sensors may be worn wrapped around the arm and their average value may be used. The estimation method may be adjusted based on user attributes such as age and gender. The user may be able to input values such as the current level of fatigue to adjust the estimated state.

Next, a determination is made as to whether there is a problem with the estimated muscle activity (step S11). A problem would be, for example, if the user performs a movement different from the one instructed, or if it is determined that maximum muscle strength is not being obtained due to a difference in the timing of the start of measurement, etc. In step S11, if a problem is determined (No in step S11), the process returns to step S9 and an instruction is given to repeat the movement.

The judgment in step S11 is made by waveform pattern matching, peak detection, machine learning, and other methods of estimating movement. Personal information such as the user's age and gender may be confirmed in advance, and if an inappropriate result is obtained by comparing this information, the judgment may be redone.

The measurement result may be notified to the user, and the measurement may be redone if a signal requesting redo is received from the user via the input mechanism 104. Information obtained from an auxiliary sensor may also be utilized. If measurement does not go well with a certain operation due to the range of motion of the joints, etc., another movement may be performed as a backup.

If it is determined in step S11 that there is a problem (Yes in step S11), a determination is made as to whether there is any other action that should be performed (step S12). If it is determined in step S12 that there is any other action that should be performed (Yes in step S12), a specific notification is given to the user in step S3. For example, this may be the case when multiple actions have been set in advance and there are actions that have not yet been measured. It may be possible to automatically determine whether there is any other action that should be performed based on the measurement results up to this point.

If it is determined in step S12 that there are no other movements to be performed (No in step S12), all movements are completed, and optimal execution conditions are calculated from the estimated muscle activity (step S13).

The calculation of the execution conditions is done by, for example, setting a certain percentage as the threshold value for the muscle strength at maximum muscle exertion or an index equivalent thereto estimated from the measurement. For example, if the muscle strength index when pushing a desk with maximum force is 100 as a result of the measurement and estimation of muscle activity, then multiplying it by a coefficient K = 0.3 registered in the system beforehand, and calculating 30 as the threshold value.

The threshold value may be a value directly input by the user with an appropriate strength, or may be calculated using a nonlinear function (such as a model generated by deep learning). For example, the user may input a strength of their choice, and the peak value of muscle strength change at that time may be used as the threshold. It may also be calculated using a model of muscle strength and threshold value previously obtained by deep learning. The threshold value is a value based on the change in strength of the force input by the user. The threshold value may be the peak value of muscle strength change of the force input by the user, the average value of muscle strength change, a derivative value, etc.

It is also possible to increase the number of movements to be measured and to utilize the measurement results and the estimated muscle activity results in a composite manner. For example, muscle strength during relaxation can also be measured, and a threshold can be determined at a certain internal division point between the relaxation and maximum muscle strength.

Other parameters may be optimized depending on the conditions for executing the operation. For example, the duration (if the execution condition is a certain period of continuous input), the similarity threshold (for pattern matching, etc.), learning (for machine learning, etc.), estimation parameters, etc.

Information obtained from auxiliary sensors can also be used. For example, if a measurement is taken during a state of high excitement, the threshold can be set lower according to that level. If the range of motion is narrow, more muscle strength is required to maintain posture, so the threshold can be set higher.

After the execution conditions are calculated in step S13, the calculated execution conditions are applied to the control process of the application program (step S14). For example, the threshold value calculated in step S13 is reflected as a judgment condition for an application control program that judges whether a specific process should be executed based on the strength of the pressing on the desk. When measuring multiple actions, the calculation process of the execution conditions and application to the control process may be performed after each action has been completed, rather than after all actions have been completed.

Then, the calibration process ends (step S15), and the application program and device are terminated (step S16).

<1-5. Threshold adjustment>
The calibration process for adjusting the threshold may be started when a signal to start the calibration process is input by the user through the input mechanism 104 during the course of an application program. The calibration process may be started, for example, by selecting "Start Calibration" from the application menu, pressing a calibration button on the controller, etc.

The system may detect problems during operation and automatically start a calibration process. For example, the system may detect frequent occurrences of operational errors and start a calibration process.

By setting the timing of the calibration process to any timing, the threshold can be optimized to match the individual's sense of force, which cannot be absorbed by the initial calibration process. The threshold can also be adjusted to the user's preferred level of force.

2. Second embodiment
The information processing device 200 of the second embodiment automatically detects an operation failure and adjusts the threshold value without explicitly performing a specific operation. In the second embodiment, the muscle activity detection unit 306 detects a specific operation from muscle activity without explicitly instructing the user, and calculates the operation conditions according to the contents of the detected operation.

FIG. 16 is a functional block diagram of an information processing device 200 according to a second embodiment. In FIG. 16, the same parts as those in FIG. 2 are given the same reference numerals, and their description will be omitted. As shown in FIG. 16, the calculation device 212' of the information processing device 200 has a muscle activity measurement unit 201, a muscle activity estimation unit 202, an operation condition calculation unit 303, an application control unit 204, and a muscle activity detection unit 306.

The muscle activity detection unit 306 detects a specific motion registered in advance from the muscle activity estimation result. That is, the muscle activity detection unit 306 detects a specific motion of the user from the indicators output from the muscle activity estimation unit 202. The operation condition calculation unit 303 calculates optimal execution conditions from the muscle activity of the specific motion part. That is, the operation condition calculation unit 303 calculates operation conditions according to the specific motion detected by the muscle activity detection unit 306.

The process of the information processing device 200 according to the second embodiment is outlined below.
1. Start of application program 2. Start of calibration process 3. Measure biological signals while operating the application program with a sensor 4. Estimate muscle activity from the measurement results 5. Detect specific movement parts registered in advance from the estimated muscle activity results 6. Calculate optimal execution conditions from muscle activity of specific movement parts 7. Apply the calculated execution conditions to the control process of the app 8. End of calibration process 9. End of application program

"5. Detecting specific movement parts registered in advance from the estimated muscle activity results" may result in, for example, when pushing a desk, the threshold being too high and not enough force being used, resulting in repeated pushing of the desk, or the threshold being too low and frequent erroneous operations occurring.

"6. Calculating optimal execution conditions from muscle activity of a specific motion part" includes, for example, the following processes 6-1 to 6-4.
6-1. Identify a certain period before and after the moment of operation failure. 6-2. Calculate muscle strength within the identified period. 6-3. Detect the peak of muscle strength change within the period. 6-4. Adjust the threshold from the peak value (if the peak is higher than the threshold, raise the threshold to near the peak. If the peak is lower than the threshold, lower the threshold to near the peak.)

According to the information processing device 200 of the second embodiment, the calibration process can be performed without performing any calibration operations, reducing the burden on the user.

3. Third embodiment
Fig. 17 is a functional block diagram of an information processing device 300 according to the third embodiment. In Fig. 17, the same parts as those in Fig. 2 are given the same reference numerals, and their description will be omitted. As shown in Fig. 17, a calculation device 312'' of the information processing device 300 has a muscle activity measurement unit 201, a muscle activity estimation unit 202, an operation condition calculation unit 303', an application control unit 204, and a muscle activity detection unit 406. Furthermore, the information processing device 300 has a storage device 107 in addition to the sensor 101.

The muscle activity detection unit 406 detects the movement to be recorded from the muscle activity estimation result, and records the muscle activity during the detected movement in the storage device 107. That is, the muscle activity detection unit 406 detects the indicators of the movement to be recorded from the indicators output from the muscle activity estimation unit 202, and stores the detected indicators in the storage device 107. The operation condition calculation unit 403 calculates optimal execution conditions from past muscle activity recorded at appropriate times and current muscle activity. That is, the operation condition calculation unit 403 calculates the operation conditions based on the indicators stored in the storage device 107.

The outline of the process of the information processing device 300 according to the third embodiment is as follows.
1. Start of application 2. Start of calibration process 3. Measure biological signals during application program operation with a sensor 4. Estimate muscle activity from the measurement results 5. Detect movements to be recorded from the estimated muscle activity results 6. Record muscle activity during the detected movements in storage device 107 7. Calculate optimal execution conditions from past muscle activity and current muscle activity recorded at appropriate times 8. Apply the calculated execution conditions to the control process of the app 9. End of calibration process 10. End of application program

"5. Detecting the movements to be recorded from the estimated muscle activity results" involves, for example, calculating the peak value of muscle strength changes over a certain period of time before and after the user pushes a desk. The objects of detection are movements and timing at which muscle strength can be measured under certain conditions.

"6. Recording the muscle activity during the detected movement in the storage device 107" means, for example, storing the muscle strength values during the detected movement as time-series data in the storage memory in the VR goggles. The data may be stored at regular intervals while the app is running, or when the application is closed. Data that has been stored for a certain period of time may be deleted. The storage device 107 does not have to be physically built into the device, such as a cloud storage device connected via the Internet.

"7. Calculating optimal execution conditions from past and current muscle activity recorded at appropriate times" involves, for example, comparing the average muscle strength value over a certain period of time in the past with the current muscle strength value to estimate increases or decreases in muscle strength and adjust the threshold to a certain level. This may be based on other indicators such as a moving average of muscle activity or a derivative value of muscle strength change. Values from auxiliary sensors may also be recorded and used for judgment. If necessary, indicators related to the execution of an operation, such as the success rate of the operation, may also be recorded and used for judgment.

According to the information processing device 300 of the third embodiment, the calibration process can be performed without performing any calibration operations, reducing the burden on the user. In addition, the effects of muscle strength increases or decreases, and changes in strength due to long-term practice can be reduced for users with large changes in muscle strength such as children and elderly people.

4. Fourth embodiment
In the fourth embodiment, the user may perform an arbitrary movement at a specific part of the user's body, and the biosignal at that time may be integrated with the movement information measured by another sensor to perform calibration. For example, the user may perform a movement of appropriately stretching or bending a finger, and the myoelectric potential at that time and the movement may be measured by a camera or the like, and calibration may be performed based on the correspondence between the force exerted when a certain finger is bent.

5. Hardware Configuration
FIG. 18 is a hardware configuration diagram showing an example of a computer that realizes the arithmetic unit 102 of the information processing device 100 according to the first to fourth embodiments.
A computer that realizes the arithmetic device 102 of the information processing device 100 according to each of the above-described embodiments is realized by a computer 1000 having a configuration as shown in Fig. 18, for example. Fig. 18 is a hardware configuration diagram showing an example of the computer 1000 that realizes the functions of the arithmetic device 102. The computer 1000 has a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, a HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input/output interface 1600. Each unit of the computer 1000 is connected by a bus 1050.

The CPU 1100 operates based on the programs stored in the ROM 1300 or the HDD 1400 and controls each part. For example, the CPU 1100 loads the programs stored in the ROM 1300 or the HDD 1400 into the RAM 1200 and executes processes corresponding to the various programs.

The ROM 1300 stores boot programs such as the BIOS (Basic Input Output System) that is executed by the CPU 1100 when the computer 1000 starts up, as well as programs that depend on the hardware of the computer 1000.

HDD 1400 is a computer-readable recording medium that non-temporarily records programs executed by CPU 1100 and data used by such programs. Specifically, HDD 1400 is a recording medium that records application programs related to the present disclosure, which are an example of program data 1450.

The communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (e.g., the Internet). For example, the CPU 1100 receives data from other devices and transmits data generated by the CPU 1100 to other devices via the communication interface 1500.

The input/output interface 1600 is an interface for connecting the input/output device 1650 and the computer 1000. For example, the CPU 1100 receives data from an input device such as a keyboard or a mouse via the input/output interface 1600. The CPU 1100 also transmits data to an output device such as a display, a speaker, or a printer via the input/output interface 1600. The input/output interface 1600 may also function as a media interface that reads programs and the like recorded on a specific recording medium. Examples of media include optical recording media such as DVDs (Digital Versatile Discs) and PDs (Phase change rewritable Discs), magneto-optical recording media such as MOs (Magneto-Optical Disks), tape media, magnetic recording media, and semiconductor memories.

For example, when the computer 1000 functions as the arithmetic device 102 according to the first embodiment, the CPU 1100 of the computer 1000 executes an image processing program loaded onto the RAM 1200 to realize functions such as the reissue request unit 43. The HDD 1400 also stores the SE management processing program according to the present disclosure and data in the content storage unit 121. The CPU 1100 reads and executes the program data 1450 from the HDD 1400, but as another example, it may also obtain these programs from other devices via the external network 1550.

The above describes in detail preferred embodiments of the present disclosure with reference to the attached drawings, but the technical scope of the present disclosure is not limited to such examples. It is clear that a person with ordinary knowledge in the technical field of the present disclosure can conceive of various modified or revised examples within the scope of the technical ideas described in the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure.

Furthermore, the effects described in this specification are merely descriptive or exemplary and are not limiting. In other words, the technology disclosed herein may achieve other effects that are apparent to a person skilled in the art from the description in this specification, in addition to or in place of the above effects.

This technology can also be configured as follows:

(1) a measurement unit that measures a biosignal of a user;
and an operation condition calculation unit that calculates an operation condition for performing an operation of the user based on the measured biological signal of the user.
(2) the biosignal of the user is an index based on an estimation result of muscle activity based on the biosignal of the user;
The information processing device according to (1), wherein the operation condition is a threshold value of the index.
(3) The information processing device according to (2), wherein the threshold value is calculated by multiplying the index by a coefficient.
(4) The information processing device according to (2), wherein the threshold value is based on a strength of force input by the user.
(5) The information processing device according to (2), wherein the threshold is calculated by using a nonlinear function on the index, and the threshold is a strength of force input by the user.
(6) The information processing device according to (2), wherein the threshold value is a value based on a change in strength of the force input by the user.
(7) The information processing device according to any one of (1) to (6), wherein the operation condition calculation unit applies the calculated operation condition to an application program that determines an operation.
(8) an instruction generating unit that outputs an instruction for causing the user to perform an operation for calibrating the operating condition;
The information processing device according to any one of (1) to (7), further comprising: an output unit that outputs the instruction output by the instruction generating unit to the user.
(9) The information processing device according to (7), wherein the operation condition calculation unit calculates the operation condition when the application program is initialized.
(10) The information processing device according to (7), wherein the operation condition calculation unit calculates the operation condition while the application program is being executed.
(11) The information processing device according to (8), wherein the instruction generation unit issues the instruction to the output unit before the operation condition calculation unit calculates the operation condition.
(12) The biosignal is a voltage signal of the user,
The information processing device according to any one of (2) to (11), wherein the index includes at least one of an RMS, a maximum amplitude, an average amplitude, and an integral value of the voltage signal.
(13) A detection unit that detects a specific action of the user from the indicator,
The information processing device according to any one of (2) to (12), wherein the operation condition calculation unit calculates the operation condition in response to the detected specific action.
(14) A detection unit that detects an indicator of an action to be recorded from the indicators and stores the detected indicator in a storage unit,
The information processing device according to any one of (2) to (13), wherein the operation condition calculation unit calculates the operation condition based on the index stored in the storage unit.
(15) measuring a biosignal of a user;
An information processing method for calculating, based on the measured biological signal of the user, an operation condition for performing an operation of the user.

REFERENCE SIGNS LIST 100, 200, 300 Information processing device 101 Sensor 102, 212, 312 Computing device 103 Output mechanism 104 Input mechanism 105 Wristband 106 VR goggles 107 Storage device 108 Position 109 Finger 110 Screen 111_1 Pressing operation 111_2 Grasping operation 111_3 Pinch operation 111_4 Stretching operation 114 Display showing remaining time and other progress 201 Muscle activity measuring unit 202 Muscle activity estimating unit 203, 303 Operation condition calculating unit 204 Application control unit 205 Instruction generating unit 206, 306, 406 Muscle activity detecting unit

Claims (15)

A measurement unit that measures a biosignal of a user;
and an operation condition calculation unit that calculates an operation condition for performing an operation of the user based on the measured biological signal of the user. the biosignal of the user is an index based on an estimation result of muscle activity based on the biosignal of the user;
The information processing apparatus according to claim 1 , wherein the operation condition is a threshold value of the index. The information processing apparatus according to claim 2 , wherein the threshold value is calculated by multiplying the index by a coefficient. The information processing apparatus according to claim 2 , wherein the threshold value is based on the strength of a force input by the user. The threshold is calculated using a nonlinear function of the index;
The information processing apparatus according to claim 2 , wherein the threshold value is a strength of a force input by the user. The information processing apparatus according to claim 2 , wherein the threshold value is a value based on a change in strength of the force input by the user. The information processing apparatus according to claim 1 , wherein the operation condition calculation unit applies the calculated operation condition to an application program that performs operation determination. an instruction generating unit that outputs an instruction for causing the user to perform an operation for calibrating the operating condition;
The information processing apparatus according to claim 1 , further comprising an output unit that outputs the instruction output by the instruction generating unit to the user. The information processing apparatus according to claim 7 , wherein the operation condition calculation section calculates the operation condition when the application program is initialized. The information processing apparatus according to claim 7 , wherein the operation condition calculation section calculates the operation condition during execution of the application program. The information processing apparatus according to claim 8 , wherein the instruction generation unit issues the instruction to the output unit before the operation condition calculation unit calculates the operation condition. the biosignal is a voltage signal of the user;
The information processing device according to claim 2 , wherein the index includes at least one of an RMS, a maximum amplitude, an average amplitude, and an integral value of the voltage signal. a detection unit that detects a specific action of the user from the indicator;
The information processing apparatus according to claim 2 , wherein the operation condition calculation unit calculates the operation condition in response to the detected specific action. a detection unit that detects an indicator of an action to be recorded from the indicators and stores the detected indicator in a storage unit;
The information processing apparatus according to claim 2 , wherein the operation condition calculation unit calculates the operation condition based on the index stored in the storage unit. Measure the user's biosignals,
An information processing method for calculating, based on the measured biological signal of the user, an operation condition for performing an operation by the user.

Images