WO2025004362A1 - Remote operation assistance system and method - Google Patents

Abstract

According to one aspect of this invention, when executed by a system that connects a user terminal and an operator terminal over a network and transfers an operation of the operator to a user in a state in which a display screen is shared, first, operation information in which the movement of a finger of the operator is reflected is detected in the operator terminal, and the detected operation information is transmitted from the operator terminal to the user terminal over the network. With regard to this, the user terminal receives the operation information transmitted from the operator terminal, generates a stimulus control signal for generating the same movement as the finger of the operator in the finger of the user on the basis of the received operation information, and presents a stimulus to a body part related to the movement of the finger of the user in response to the generated stimulus control signal.

Description

Remote operation support system and method

One aspect of the present invention relates to a remote operation support system and a remote operation support method used to remotely support, for example, the operation of a smartphone.

For example, one service that supports users who are unfamiliar with operating smartphones is known, which connects the user's smartphone to an operator terminal at a service center via a network, and the operator provides instructions on how to operate the smartphone while the screen is shared between the two terminals (see, for example, non-patent document 1). By using this service, the user can receive assistance with how to operate the smartphone from the comfort of their own home, as if the operator were face-to-face.

"Anshin Remote Support", NTT Docomo, Internet <URL: https://www.docomo.ne.jp/service/remote_support/>

However, in this type of conventional service, the user confirms the operation method by visually viewing the screen as the operator operates. As a result, when the user later tries to perform the operation that they have seen the operator perform, they may find that they are unable to reproduce the operation because their hand and finger movements differ from the operator's. In particular, for users who have difficulty moving their hands and fingers at will due to a disability or other reason, it is difficult to reproduce the operation of the operator that they have seen with their own hands and fingers.

This invention was made with the above in mind, and aims to provide technology that enables users who are unfamiliar with operation or who have difficulty moving their fingers at will to reliably operate an information terminal using their fingers.

In order to solve the above problems, one aspect of the remote operation support system or method according to the present invention is a system that connects a user terminal and an operator terminal via a network and transmits the operator's operations to a user while sharing a display screen. When the system is executed, the operator terminal first detects motion information reflecting the movement of the operator's fingers, and transmits the detected motion information from the operator terminal to the user terminal via the network. In response to this, the user terminal receives the motion information transmitted from the operator terminal, generates a stimulation control signal based on the received motion information to cause the user's fingers to move in the same way as the movement of the operator's fingers, and presents a stimulation to a body part related to the movement of the user's fingers in response to the generated stimulation control signal.

According to one aspect of the invention, motion information reflecting the movement of the operator's fingers, such as an electromyographic signal, is detected and transmitted to a user terminal, and the user terminal applies stimuli such as electrical stimulation to the areas related to the movement of the user's fingers based on the motion signal, so as to cause the user's fingers to move in the same way as the operator's fingers. As a result, the movement of the operator's fingers is transmitted as is to the user's fingers, and the user can directly recognize the movement of the operator's fingers at a remote location as the movement of his or her own fingers.

In other words, one aspect of the present invention provides technology that enables even users who are unfamiliar with operation or who have difficulty moving their fingers at will to reliably operate an information terminal using their fingers.

FIG. 1 is a diagram showing an example of the configuration of a remote operation support system according to an embodiment of the present invention. FIG. 2 is a block diagram showing an example of the configuration of an operator terminal in the remote operation support system shown in FIG. FIG. 3 is a block diagram showing an example of the configuration of a user terminal in the remote operation support system shown in FIG. FIG. 4 is a flowchart showing an example of a processing procedure and processing contents of the operator terminal and the user terminal shown in FIG. 2 and FIG. FIG. 5A is a diagram showing an example of an operation to be supported. FIG. 5B is a diagram showing another example of an operation to be supported.

Below, an embodiment of the present invention will be described with reference to the drawings.

[One embodiment]
(Configuration example)
(1) System FIG. 1 is a diagram showing an example of the configuration of a remote operation support system according to an embodiment of the present invention.

In one embodiment of the present invention, the remote operation support system connects a user terminal UT used by the user receiving support with an operator terminal OT used by the operator providing the support via a network NW, and in this state a display screen is shared between the user terminal UT and the operator terminal OT, and while looking at this display screen, the operator supports the user in operating the terminal using their fingers.

Operation gloves HE1 and HE2 are worn on the hands of the operator and the user, respectively. A sensor unit and a stimulus presentation unit are installed or connected to gloves HE1 and HE2, respectively.

The sensor unit is composed of electromyogram sensors ES1 and ES2 and their detection electrodes. The detection electrodes include, for example, multiple sets of two electrodes that form a pair, and these sets are placed in locations inside the glove that correspond to each finger. The electromyogram sensors ES1 and ES2 measure the surface electromyogram (EMG) of the skin, and measure the skin surface potential difference that appears between the two electrodes for each set of electrodes as an electromyogram signal.

More specifically, the electromyogram sensors ES1 and ES2 detect amplitude changes corresponding to muscle activity associated with the finger movement accompanying the swipe operation. The electromyogram sensors ES1 and ES2 then transmit the detected electromyogram signals together with identification information of the electrode set that detected the signals to the operator terminal OT and the user terminal UT via, for example, a signal cable or a wireless line that employs a low-power wireless data transmission standard such as Bluetooth (registered trademark).

The stimulation presentation unit is composed of electrical stimulation generators ED1 and ED2 and stimulation presentation electrodes. The stimulation presentation electrodes include, for example, multiple sets of two electrodes that form a pair. Each electrode set is placed at a position in the glove that corresponds to the muscle that moves each finger.

The electrical stimulation generators ED1 and ED2 individually apply electrical stimulation to the muscles that move each finger, generating a low-frequency stimulation signal suitable for muscular contraction by, for example, superimposing a stimulation frequency of 15 to 200 Hz on a reference frequency of several kHz to 20 kHz. The reference frequency, stimulation frequency and signal strength of the electrical stimulation signal can be arbitrarily controlled by the electrical stimulation control signal output from the operator terminal OT and the user terminal UT, and can be arbitrarily set according to the type and condition of the muscle to be stimulated, etc.

The network NW comprises, for example, a wide area network with the Internet at its core, and an access network for accessing this wide area network. Examples of the access network that can be used include a public communication network using wired or wireless connections, a Local Area Network (LAN) using wired or wireless connections, and a Cable Television (CATV) network. Note that when the business support system is operated within a company or business establishment, for example, the network NW is configured as an in-house network such as a LAN or wireless LAN.

(2) Operator terminal (OT)
FIG. 2 is a block diagram showing an example of the hardware and software configuration of the operator terminal OT.

The operator terminal OT is, for example, a smartphone, and has a control unit 11 that uses a hardware processor such as a central processing unit (CPU). A storage unit having a program storage unit 12 and a data storage unit 13, an input/output interface (hereinafter, the interface will be abbreviated as I/F) unit 14, and a communication I/F unit 5 are connected to this control unit 11 via a bus (not shown).

In addition, in addition to a smartphone, a PDA terminal with a touch panel or a notebook personal computer may also be used as the operator terminal OT.

A touch panel type input/output device is connected to the input/output I/F unit 14. The touch panel type input/output device is configured by arranging an input device 162, which is, for example, a piezoelectric or electrostatic input sheet, on the display surface of a display device 161, which uses, for example, liquid crystal or organic EL.

The input/output I/F unit 14 is also connected to the electromyogram sensor ES1 and the electrical stimulation generator ED1 described above. Note that E11 and E12 are a detection electrode and a stimulation presentation electrode connected to the electromyogram sensor ES1 and the electrical stimulation generator ED1, respectively.

The communication I/F unit 15 transmits and receives information data to and from the user terminal UT via the network NW in accordance with the communication protocol defined by the network NW.

The program storage unit 12 is configured by combining, for example, a non-volatile memory such as a solid state drive (SSD) as a storage medium that can be written to and read from at any time, and a non-volatile memory such as a read only memory (ROM), and stores application programs necessary to execute various control processes according to one embodiment, in addition to middleware such as an operating system (OS). Hereinafter, the OS and each application program will be collectively referred to as the program.

The data storage unit 13 is, for example, a combination of a non-volatile memory such as an SSD, which can be written to and read from at any time, and a volatile memory such as a RAM (Random Access Memory), and is used to temporarily store transmitted and received data, as well as data generated in the process of providing operational assistance to the user using electrical stimulation.

The control unit 11 includes a screen sharing control processing unit 111, an electromyographic signal receiving processing unit 112, an electrical stimulation control processing unit 113, and an electromyographic signal transmitting processing unit 114 as processing functions necessary to implement one embodiment. All of these processing units 111 to 114 are realized by causing the hardware processor of the control unit 11 to execute application programs stored in the program storage unit 12.

The screen sharing control processing unit 111 shares display screen data used for operational support with the user terminal UT when a communication link is established with the user terminal UT.

The myoelectric signal receiving and processing unit 112 receives the user's myoelectric signal transmitted from the user terminal UT via the communication I/F unit 15, and passes the received user's myoelectric signal to the electrical stimulation control processing unit 113.

The electrical stimulation control processing unit 113 generates an electrical stimulation control signal in response to the user's myoelectric signal to cause the operator's finger to move in the same way as the user's finger as represented by the myoelectric signal, and outputs the generated electrical stimulation control signal from the input/output I/F unit 14 to the electrical stimulation generator ED1.

The myoelectric signal transmission processing unit 114 receives the operator's myoelectric signal detected by the myoelectric sensor ES1 from the input/output I/F unit 14, and transmits the received operator's myoelectric signal from the communication I/F unit 15 to the user terminal UT.

(3) User Terminal UT
FIG. 3 is a block diagram showing an example of the hardware and software configuration of the user terminal UT.

The user terminal UT is a smartphone, just like the operator terminal OT described above. Note that the user terminal UT can also be an information terminal using a touch panel, such as a PDA.

The user terminal UT comprises a control unit 21 using a hardware processor such as a central processing unit (CPU), a storage unit having a program storage unit 22 and a data storage unit 23, an input/output I/F unit 24, and a communication I/F unit 25, all connected via a bus.

The input/output I/F unit 24 is connected to a touch panel type input/output device in which a sheet-like input device 162 is arranged on the display surface of a display device 161.

The input/output I/F unit 14 is also connected to an electromyogram sensor ES2 and an electrical stimulation generator ED2. A detection electrode E21 is connected to the electromyogram sensor ES2, and a stimulation presentation electrode E22 is connected to the electrical stimulation generator ED2.

The communication I/F unit 25 transmits and receives information data to and from the operator terminal OT via the network NW in accordance with the communication protocol defined by the network NW.

The program storage unit 22 is configured by combining, for example, a non-volatile memory such as an SSD as a storage medium that can be written to and read from at any time, and a non-volatile memory such as a ROM, and stores application programs necessary for executing various control processes according to one embodiment, in addition to middleware such as an OS. Hereinafter, the OS and each application program will be collectively referred to as a program.

The data storage unit 23 is, for example, a combination of a non-volatile memory such as an SSD that can be written to and read from at any time as a storage medium, and a volatile memory such as a RAM, and is used to temporarily store transmitted and received data, as well as data generated in the process of receiving operational assistance using electrical stimulation.

The control unit 21 includes, as processing functions necessary to implement one embodiment, a screen sharing control processing unit 211, an EMG signal transmission processing unit 212, an EMG signal reception processing unit 213, and an electrical stimulation control processing unit 214. All of these processing units 211 to 214 are realized by causing the hardware processor of the control unit 21 to execute application programs stored in the program storage unit 22.

The screen sharing control processing unit 211 shares display screen data used to receive operational assistance with the operator terminal OT when a communication link is established with the operator terminal OT.

The myoelectric signal transmission processing unit 212 receives the user's myoelectric signal detected by the myoelectric sensor ES2 from the input/output I/F unit 24, and transmits the received user's myoelectric signal from the communication I/F unit 25 to the operator terminal OT.

The myoelectric signal receiving and processing unit 213 receives the operator's myoelectric signal transmitted from the operator terminal OT via the communication I/F unit 25, and passes the received operator's myoelectric signal to the electrical stimulation control processing unit 214.

The electrical stimulation control processing unit 214 generates an electrical stimulation control signal in response to the operator's myoelectric signal to cause the user's finger to move in the same way as the operator's finger represented by the myoelectric signal, and outputs the generated electrical stimulation control signal from the input/output I/F unit 24 to the electrical stimulation generator ED2.

(Example of operation)
Next, an example of the operation of the remote operation support system configured as above will be described.

FIG. 4 is a flowchart showing an example of the processing procedure and processing content of operation support control executed by each of the control units 11, 21 of the operator terminal OT and the user terminal UT of the remote operation support system.

(1) Advance preparation for operation assistance When a user wishes to receive operation assistance from a user terminal UT, the user first puts on a glove HE2 on the hand that will perform the operation. Similarly, the operator puts on a glove HE1 on the hand that will perform the operation. As shown in Fig. 1, the gloves HE1 and HE2 are integrally provided with electromyographic sensors ES1 and ES2, and are connected to electrical stimulation generators ED1 and ED2 via signal cables.

The electromyogram sensors ES1, ES2 and the electrical stimulation generators ED1, ED2 may both be integrally installed in the gloves HE1, HE2, or may be connected by a signal cable.

Furthermore, the electromyogram sensors ES1, ES2 and the electrical stimulation generators ED1, ED2 are paired in advance with the operator terminal OT and the user terminal UT to enable communication, for example, via Bluetooth.

(2) Application Activation When a user wishes to receive guidance from an operator on how to operate a smartphone, the user inputs an operation assistance request in the user terminal UT, for example, by tapping an icon. When the control unit 11 of the user terminal UT detects the operation assistance request in step S10, the control unit 11 activates an application and executes a process for establishing a communication link with the operator terminal OT, and then transmits the operation assistance request to the operator terminal OT.

In response to this, the control unit 21 of the operator terminal OT establishes a communication link with the user terminal UT in response to access from the user terminal UT, and then, upon receiving the above-mentioned operation assistance request in step S11, launches an application and notifies the operator of this fact.

(3) Trial Operation by User In response to the trial operation by the user, in step S12, the control unit 21 of the user terminal UT reads out display screen data to be operated from the data storage unit 13 or downloads it from a website, and displays it on the display device 261. At the same time, under the control of the screen sharing control processing unit 211, the display screen data is transmitted from the communication I/F unit 25 to the operator terminal OT.

In response to this, the control unit 11 of the operator terminal OT, under the control of the screen sharing control processing unit 111, receives the display screen data transmitted from the user terminal UT via the communication I/F unit 15 in step S13. Then, the received display screen data is displayed on the display device 161.

Note that while map data is used as the display screen data, it may also be image data such as photographs or text data.

Now, in this state, assume that the user performs a swipe operation on the display screen data by moving the fingers wearing the glove HE2 on the input device 262 of the user terminal UT. Figures 5A and 5B show, as an example of such an operation, a pinch-in operation and a pinch-out operation using two fingers simultaneously. Note that other examples of the operation may include an operation using three fingers simultaneously, a drag operation using one finger, a flick operation, a double tap operation, etc.

When the control unit 21 of the user terminal UT detects the user's trial operation in step S14, under the control of the myoelectric signal transmission processing unit 212, it acquires, via the input/output I/F unit 24 in step S15, a myoelectric signal (including identification information of the electrode set from which the signal was detected) corresponding to the finger movement associated with the trial operation detected by the myoelectric sensor ES2. The myoelectric signal transmission processing unit 212 then transmits the acquired myoelectric signal of the user from the communication I/F unit 25 to the operator terminal OT in step S16. At the same time, under the control of the screen sharing control processing unit 211, the control unit 21 of the user terminal UT transmits the operated display screen data from the communication I/F unit 25 to the operator terminal OT.

In response to this, when the control unit 11 of the operator terminal OT receives the display screen data transmitted from the user terminal UT via the communication I/F unit 15 in step S17 under the control of the screen sharing control processing unit 111, in step S18, the control unit 11 updates the display screen data displayed on the display device 161 to the received display screen data.

At the same time, the control unit 11 of the operator terminal OT, under the control of the myoelectric signal receiving and processing unit 112, receives the user's myoelectric signal transmitted from the user terminal UT via the communication I/F unit 15 in step S17.

When the control unit 11 of the operator terminal OT receives the user's myoelectric signal, in step S19, under the control of the electrical stimulation control processing unit 113, generates an electrical stimulation control signal for causing the operator's finger to move in the same way as the user's finger represented by the user's myoelectric signal. This electrical stimulation control signal includes values specifying the strength (amplitude) and application time of the electrical stimulation, and an identification signal specifying the electrode set corresponding to the finger to which the electrical stimulation is to be applied. The electrical stimulation control processing unit 113 then outputs the generated electrical stimulation control signal from the input/output I/F unit 14 to the electrical stimulation generator ED1.

As a result, an electrical stimulation signal of the strength specified by the electrical stimulation control signal is generated from the electrical stimulation generator ED1 for the specified length of time, and the generated electrical stimulation signal is applied to the set of stimulation presentation electrodes corresponding to the finger to be moved. As a result, electrical stimulation is applied to the muscles that move the operator's finger that is the same as the finger used by the user in the trial operation, thereby moving the operator's finger. In other words, the movement of the user's finger is transferred to the movement of the operator's finger. Therefore, the operator can recognize the finger movement associated with the user's trial operation from the movement of his or her own finger, and can directly confirm any points that need to be corrected in the user's trial operation.

(4) Support for user operation The operator refers to the confirmation result of the user's finger movement and performs a model operation, such as a pinch-in operation or a pinch-out operation, on the display screen of the operator terminal OT. When the control unit 11 of the operator terminal OT detects the model operation in step S20, the control unit 11 acquires, under the control of the myoelectric signal transmission processing unit 114, a myoelectric signal (including identification information of the electrode set that detected the signal) corresponding to the finger movement associated with the model operation detected by the myoelectric sensor ES1 via the input/output I/F unit 24 in step S21. Then, the myoelectric signal transmission processing unit 114 transmits the acquired myoelectric signal of the user from the communication I/F unit 15 to the user terminal UT in step S22.

In response to this, the control unit 21 of the user terminal UT, under the control of the myoelectric signal receiving and processing unit 213, receives the operator's myoelectric signal transmitted from the operator terminal OT via the communication I/F unit 25 in step S23.

Upon receiving the operator's myoelectric signal, the control unit 21 of the user terminal UT, under the control of the electrical stimulation control processing unit 214, generates an electrical stimulation control signal in step S24 for causing the user's finger to move in the same way as the movement of the operator's finger represented by the operator's myoelectric signal. This electrical stimulation control signal includes values specifying the strength (amplitude) and application time of the electrical stimulation, and an identification signal specifying the electrode set corresponding to the finger to which the electrical stimulation is to be applied. The electrical stimulation control processing unit 214 then outputs the generated electrical stimulation control signal from the input/output I/F unit 24 to the electrical stimulation generator ED2.

As a result, an electrical stimulation signal of the strength specified by the electrical stimulation control signal is generated from the electrical stimulation generator ED2 for the specified length of time, and the generated electrical stimulation signal is applied to the set of stimulation presentation electrodes corresponding to the finger to be moved. As a result, electrical stimulation is applied to the muscles that move the user's finger that is the same as the finger used by the operator for the operation, thereby moving the user's finger. In other words, the operator's model finger movement is transferred to the user's finger. Therefore, the user can directly recognize the finger movement associated with the operator's model operation as the movement of his or her own finger.

When the display screen is operated in response to the user's finger movements, the display screen data after this operation is sent from the communication I/F unit 25 to the operator terminal OT in step S25 under the control of the screen sharing control processing unit 211.

In response to this, the control unit 11 of the operator terminal OT, under the control of the screen sharing control processing unit 111, receives the display image data transmitted from the user terminal UT via the communication I/F unit 15 in step S26, and in step S27 outputs the received display image data from the input/output I/F unit 14 to the display device 161 for display.

Therefore, the operator can check the results of the user's operations by looking at the display screen data.

(5) Determination of End of Assistance Control In step S28, the control unit 21 of the user terminal UT monitors whether or not the user performs an operation to end the operational assistance. If the user does not perform an operation to end the operational assistance, the control unit 21 returns to step S14 and repeatedly executes a series of processes for the operational assistance in steps S14 to S28. In response to this, when a request to end the operational assistance is detected, the control unit 21 transmits the request to end the operational assistance from the communication I/F unit 25 to the operator terminal OT, and then closes the application to end the above-mentioned operational assistance process.

In response to this, in step S29, the control unit 11 of the operator terminal OT monitors for receipt of the operation assistance end request sent from the user terminal UT. If the operation assistance end request is not received, the process returns to step S17 and continues the operation assistance control in steps S17 to S29. On the other hand, if the operation assistance end request is received, the control unit 11 of the operator terminal OPT notifies the operator of this fact with a message and closes the application to end the operation assistance.

(Action and Effects)
As described above, in one embodiment, for example, the finger movement accompanying a swipe operation that is an example of the operator is detected by the electromyographic sensor ES1, and the detected electromyographic signal is transmitted from the operator terminal OT to the user terminal UT. In response to this, when the user terminal UT receives the electromyographic signal, it generates an electrical stimulation control signal for causing the user's finger to make the same movement as the example finger movement of the operator represented by the electromyographic signal, and supplies this to the electrical stimulation generator ED2, and the electrical stimulation generator ED2 applies an electrical stimulation to the user's finger to move the finger.

Therefore, for users who are unfamiliar with operating smartphones or who have difficulty moving their fingers freely due to disabilities, the operator can directly demonstrate operations in the form of finger movements. This allows users to reliably perform relatively difficult operations such as swiping.

In one embodiment, prior to providing operational assistance to the user, the user first performs an operation and a myoelectric signal representing the finger movement at that time is transmitted from the user terminal UT to the operator terminal OT. The operator terminal OT then generates an electrical stimulation control signal based on the myoelectric signal and applies an electrical stimulation to the operator's finger, thereby directly communicating the user's finger movement to the operator.

As a result, the operator can understand what needs to be corrected in the user's finger movements and then perform model operations. This makes it possible to provide more accurate and efficient support for smartphone operation based on the characteristics of each user's finger movements.

[Other embodiments]
(1) In one embodiment, an electromyogram is detected as motion information reflecting the finger movement accompanying the model operation of the operator, but instead, for example, the position of the fingertip and its acceleration in multiple axial directions may be detected, and the user's finger movement may be controlled based on the detected fingertip position and acceleration. In this case, an actuator or the like may be used to control the user's finger movement.

(2) In one embodiment, electromyograms are detected as motion information reflecting the finger movements accompanying the user's operation. In addition, for example, skin electrodermal activity may be detected. In this case, the degree of difficulty of the user in the operation is estimated based on the detected skin electrodermal activity, and the result is presented to the operator, allowing the operator to provide more detailed support by adjusting the speed of the model operation, etc.

(3) In addition, the types and functional configurations of the operator terminal and user terminal, their processing procedures and processing contents, the types of operations to be supported, etc. may be modified and implemented in various ways without departing from the spirit of this invention.

Although the embodiments of the present invention have been described in detail above, the above description is merely an example of the present invention in every respect. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention. In other words, when implementing the present invention, specific configurations according to the embodiments may be appropriately adopted.

In short, this invention is not limited to the above-described embodiment as it is, and in the implementation stage, the components can be modified and embodied without departing from the gist of the invention. Furthermore, various inventions can be formed by appropriately combining multiple components disclosed in the above-described embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, components from different embodiments may be appropriately combined.

UT...user terminal OT...operator terminal NW...network ES1, ES2...electromyogram sensor E11, E21...detection electrode ED1, ED2...electrical stimulation generator E12, E22...stimulation presentation electrode 11, 21...control unit 12, 22...program storage unit 13, 23...data storage unit 14, 24...input/output I/F unit 15, 25...communication I/F unit 161, 261...display device 162, 262...input device 111, 211...screen sharing control processing unit 112, 213...electromyogram signal reception processing unit 113, 214...electrical stimulation control processing unit 114, 212...electromyogram signal transmission processing unit

Claims (7)

A remote operation support system that connects a user terminal and an operator terminal via a network and transmits an operator's operation to a user while sharing a display screen,
a first sensor unit that detects operator operation information reflecting a movement of the operator's fingers;
a transmission processing unit provided in the operator terminal and configured to transmit the detected operator operation information to the user terminal via the network;
a reception processing unit provided in the user terminal and configured to receive the operator operation information transmitted from the operator terminal;
a stimulus control unit provided in the user terminal and generating a user stimulus control signal for causing the user's fingers to make the same movement as the movement of the operator's fingers based on the received operator operation information;
a first stimulus presentation unit that presents a stimulus to a body part related to the movement of the user's fingers in response to the generated user stimulus control signal. a second sensor unit configured to detect user motion information reflecting a motion of the user's fingers;
a second transmission processing unit provided in the user terminal and configured to transmit the detected user operation information to the operator terminal via the network;
a second receiving processor provided in the operator terminal and configured to receive the user operation information transmitted from the user terminal;
a second stimulus control unit provided in the operator terminal and configured to generate an operator stimulus control signal for causing the operator's fingers to make the same movement as the user's fingers, based on the received user operation information;
The remote operation support system according to claim 1 , further comprising: a second stimulus presentation unit that presents a stimulus to a body part related to the movement of the operator's fingers in response to the generated operator stimulus control signal. the first sensor unit detects, as the operator operation information, a myoelectric signal reflecting a movement of the operator's fingers;
The first stimulus presentation unit applies electrical stimulation to a muscle site that moves a finger of the user.
The remote operation support system according to claim 1 . The second sensor unit detects, as the user operation information, a myoelectric signal reflecting a movement of the user's fingers;
The second stimulus presentation unit applies electrical stimulation to a muscle site that moves the operator's finger.
The remote operation support system according to claim 2 . The remote operation support system according to claim 1, wherein the first sensor unit detects the operator operation information that reflects the movement of the operator's fingers related to a swipe operation. The remote operation support system according to claim 2, wherein the second sensor unit detects the user operation information reflecting the movement of the user's fingers associated with a swipe operation. A remote operation support method executed by a system that connects a user terminal and an operator terminal via a network and transmits an operator's operation to a user while sharing a display screen, comprising:
detecting motion information reflecting the motion of the operator's fingers;
transmitting the detected motion information from the operator terminal to the user terminal via the network;
receiving the operation information transmitted from the operator terminal at the user terminal;
generating a stimulation control signal for causing the user's finger to move in the same manner as the operator's finger based on the received motion information;
and a step of presenting a stimulus to a body part related to the movement of the user's finger in response to the generated stimulus control signal.

Images