[go: up one dir, main page]

WO2000055714A1 - Remote assist system - Google Patents

Remote assist system Download PDF

Info

Publication number
WO2000055714A1
WO2000055714A1 PCT/US2000/005942 US0005942W WO0055714A1 WO 2000055714 A1 WO2000055714 A1 WO 2000055714A1 US 0005942 W US0005942 W US 0005942W WO 0055714 A1 WO0055714 A1 WO 0055714A1
Authority
WO
WIPO (PCT)
Prior art keywords
remote
video
site
information
remote communication
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/US2000/005942
Other languages
French (fr)
Inventor
James E. Messer
Walter Sullivan
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Varian Semiconductor Equipment Associates Inc
Original Assignee
Varian Semiconductor Equipment Associates Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Varian Semiconductor Equipment Associates Inc filed Critical Varian Semiconductor Equipment Associates Inc
Publication of WO2000055714A1 publication Critical patent/WO2000055714A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/14Systems for two-way working
    • H04N7/15Conference systems

Definitions

  • the present invention relates to a remote communication system.
  • system downtime can be costly.
  • the manufacturer faces a long wait whenever critical parts require expert maintenance or repair. If a technician or repair expert needs to be dispatched to the manufacturing site, it may take hours, or even days, for the repair expert to schedule and arrive at the site, especially if the site is overseas.
  • Other situations such as on construction sites, construction workers often need to communicate in real-time with site foremen, engineers or architects, particularly when problems or critical issues arise.
  • Still other situations, such as a medical setting, doctors and other medical personnel often desire to communicate to other staff or colleagues in a different location.
  • remote repair as is the semiconductor manufacturing environment described above, can be provided by telephone or by providing "patch" software over the Internet.
  • Cellular phone systems, two-way video conferencing, remote cameras also address basic telecommunication needs.
  • the communication system would be flexible and adaptable to a variety of situations and users. It would be further advantageous to provide a communication system that allows for mobile use by a user. It would be further advantageous to provide a communication system that can allow users to substantially have their hands free and not unnecessarily burdened with having to hold a communication or control device.
  • Disclosed herein is an invention for remote communication, wherein a user in a remote location has real-time access to information captured at an on-site location and visa versa.
  • on-site workers can communicate and collaborate directly with support centers and factories.
  • the invention disclosed herein allows support personnel to remotely assist on-site workers by easily directing complex procedures since remotely-located support personnel will be able to see and hear exactly what the on-site workers are experiencing. Additionally, the disclosed invention allows for emergency and preventive maintenance to be conducted in real-time at far less cost than conventional approaches. Further details of aspects, objects, and advantages of the invention are described in the detailed description, drawings and claims.
  • Figure 1 depicts a general block diagram of a remote assist communication system according to an embodiment of the invention.
  • Figure 2 depicts a representation of a remote assist communication system according to an embodiment of the invention.
  • Figure 3 depicts an embodiment of a video helmet configuration.
  • Figure 4 depicts a schematic of an embodiment of a remote assist apparatus.
  • Figure 5 depicts a schematic of an embodiment of a remote assist apparatus.
  • Figure 6 depicts a general block diagram of a remote assist communication system according to an embodiment of the invention.
  • Figure 7 depicts a general block diagram of a remote assist communication system according to an embodiment of the invention.
  • the invention combines video conferencing capabilities and a unique video helmet to allow on-site workers to communicate and collaborate directly with remote support centers and factories.
  • remote assist allows experts and support personnel at a remote location to conveniently and effectively communicate to workers at an on-site location.
  • Experts and support personnel guide direct or instruct workers so as to correct, or respond to, problems in real-time or provide on-the-spot preventive maintenance.
  • the remote assist invention offers a multi-sensory interface with distributed resources for users in a mobile environment.
  • a fundamental idea behind the present invention is that an issue to be addressed occurs at one location while additional expertise requested to help respond to the issue is located at another location.
  • an engineer or field service technician wearing a sensory device initiates a remote assist session with a location where the additional expertise is located.
  • Substantial benefits can be realized where experts at a remote location can offer additional skills to immediately solve problems at essentially any virtual location.
  • Fast and cost-effective emergency support can be available as well as support for maintenance operations.
  • Real-time process analysis and around-the-clock support from hardware and software consultants can be made available without requiring the development of large and expensive support personnel.
  • remote assist can offer cost-effective and hands-on training as well as efficient and simplified communication of fab-to-fab best practices.
  • FIG. 1 depicts a general configuration for remote assist communication system according to an embodiment of the invention.
  • Remote assist 1 comprises an on-site location unit 2 connected to a remote location unit 4 via a computer or network 3.
  • the on-site location 2 represents an on-site location from where information or direction is requested.
  • On-site location 2 can be located in a campus, building, floor, laboratory bench, operating room, subsidiary, factory, oil rig, construction site and so forth. It is at an on-site location unit 2 where information is being gathered or where an issue/problem is directly being addressed.
  • Remote location 4 represents a location where the additional expertise or help is located.
  • Remote location 4 can be a design or control center, or any site where additional expertise or information is located or distributable.
  • Remote location 4 can be a customer support center, an engineering and design center, building, campus, floor, laboratory, construction site command center, oil rig command center, operating room and so forth, where the additional expertise desired is located.
  • the remote location can preferably include access to a database for use by expert personnel in providing solutions or instruction.
  • On-site location unit 2 and remote location unit 4 interface via a network 3.
  • Network 3, or computerized conferencing center can consist of PCs, client-server networks, LANs, WANs, computer clusters, the Internet or other possible network configurations.
  • a terminal at an on-site location 2 or at a remote location 4 can include access to a database to provide real-time retrieval of information to personnel at either the on-site location 2 or remote location 4.
  • Software that controls the transfer of information from any location to another includes videoconferencing software.
  • the video conferencing software along with hardware allows users to display images from a remote site or local site on a display screen, or to display other images such as documents, video, or the like.
  • Devices at the on-site location 2 collect and/or receive data from the remote location 4 via network 3.
  • a remote communication device is provided in Fig. 2.
  • a network 3 links a remote location 503 to an on-site location 502.
  • the on-site location 502 consists of on-site sensory devices 5 providing and receiving sensory inputs and outputs to an on-site terminal 6, coupled to the network 3.
  • the remote location 503 consists of a remote sensory devices 8 providing and receiving sensory inputs to a remote terminal 7, coupled to the network 3.
  • the on-site sensory devices 5 provide two-way data information transfer.
  • a communication system will include devices for data gathering and data reception.
  • such sensory devices include, but are not limited to, cameras, microphones, and other sensory measurement and gathering devices.
  • a virtual presence is established by personnel at an on-site location for communication of information with a user at a remote location.
  • the on-site sensory devices 5 provide information (e.g., video, audio and data signals) regarding certain aspects of an on-site location 502.
  • the signals are transmitted to an on-site computer terminal 6 and/or to personnel at the on-site location by, for example, a communication tether 9.
  • On-site terminal 6 can store, process and communicate the signals to the network 3.
  • Terminal 6 can also receive signals from the network and provide them to personnel at the on-site location.
  • remote sensory devices 8 are located at the remote location 503.
  • the remote sensory devices 8 provide two-way data information transfer.
  • the communication system will include devices for data gathering and data reception at the remote location.
  • sensory devices include, but are not limited to, cameras, microphones, and other sensory measurement and gathering devices.
  • a virtual presence can be established by personnel at a remote location for communication of information with a user at an on-site location.
  • remote sensory devices 8 are transmitted to a remote computer terminal 7 and/or to personnel at the remote location.
  • Remote terminal 7 can store, process and communicate the signals to the network 3.
  • Remote terminal 6 can also receive signals from the network and provide them to personnel at the remote location.
  • an on-site user 500 is shown wearing a sensory device 5 having various sensory inputs and outputs, such as microphone 510.
  • the on- site sensory device can provide video, audio and data signals and information regarding certain aspects of the on-site location 502.
  • the signals are transmitted to an on-site computer terminal 6. Signals can also be provided to personnel at the on-site location.
  • Terminal 6 can store, process and communicate the signals to the network 3.
  • On-site terminal 6 can also receive signals from the network and provide them to personnel at the on-site location.
  • the user 500 in Fig. 2 can wear a video helmet having input and output sensory devices in various configurations.
  • An on-site user 500 communicates information received from the various sensory devices to remote personnel 521 at the remote location 503 via on-site terminal 6.
  • the user 500 is shown holding a camera 22 in his hand.
  • the camera 22 can be pointed at an object, such as object 515.
  • the image of the object 515 can be displayed effectively in real-time on terminals 516 and 517.
  • measurement devices such as sensor 520, can provide additional data to a remote user 521 or directly to the on-site terminal 6.
  • Remote personnel 521 can communicate back to the user 500 or to the on-site location in general, via audio input devices or video devices, such as sensory device 8, coupled to the remote terminal 7. Based on the information provided by the on-site user 500, the remote personnel 521 can respond via video and/or audio communication. It will be understood that various input and output devices can be used, and in various configurations, to support the real-time communication between the on-site user 500 and remote personnel 521.
  • FIG. 3 Shown in Fig. 3 is an embodiment of an input/output sensory device configuration 20 incorporates a video helmet 43 having a digital color camera 22, a microphone 24, earphones 26, and display screens 28 and 30 is depicted.
  • the video helmet 43 is preferably fully adjustable to adapt to the head and comfort level of the wearer.
  • control knob 32 can adjust the tension of bracket 44 so as to be comfortable to the user.
  • Volume control 34 can adjust the sound of the volume of the sounds entering earpiece 26 via audio cable 42.
  • Image control button 32 allows for the focusing of display screen 28 and 30. Additional adjustment controls for brightness, eye spacing, depth, and so forth are also available. Further adjustments are possible by camera bracket 38 that is preferably slidable over bracket 44.
  • the display screens 28 and 30 can be independently adjustable. Each display screen 28 and 30 can be adjusted by flipping or rotating upward. In a preferred embodiment, the display screens are sufficiently spaced in front of a users to allow for easy viewing of images on the display screens yet still allow a user to directly see and work upon physical objects before him.
  • the camera 22, attachable to video cable 40 is preferably removable from the video helmet 43 and can thereby be positioned or held, placed, or otherwise mounted at other locations to view an object. Additionally, the camera 22 can include a magnification feature where either a helmet wearer can magnify what is seen by the camera 22 and display the results on a display screen, or the camera is controlled by a user at another location.
  • the video helmet configuration 20 preferably is also clean-room safe. Clean-room fabric 36 is loosely or fixedly connected to the video helmet to create a clean-room compatible head mount.
  • a general video helmet is available from Kaiser Electro-Optics, Inc.
  • model Proview 30 is available having a display screen, microphone, and headset adjustment controls. Clean-room fabric 36, digital camera 22, camera bracket 38 and microphone 24 are designed, as shown in Fig. 3, to attach to the helmet model.
  • the digital camera 22 is preferably a wide-angle camera requiring minimum adjustment and focusing.
  • An available camera that is attachable to Proview 30 is the ELMO headset camera by Elmo Manufacturing Corporation. While the camera 22 is shown attached to a video helmet in Fig. 3, a camera or other image sensory device can be designed to be attachable to other parts of the body. If the camera is hand-held, the user can hold the camera to access difficult to reach areas or to be able to conveniently point at a desired location.
  • the display screens 28 and 30 not only are capable of showing images at the on-site location but can also independently display images being transmitted to and from the on-site location. Not only is video and audio communication possible but personnel can transmit and/or access electronic documentation such as schematics, assembly drawings, photos, videotaped demonstrations, technical specification and any documentation to be displayed on a display screen or monitor at any location.
  • the video conferencing software allows the transfer of any image to and from an on-site location to a remote location.
  • the remote assist communication system can include a controller that allows for the control of information that appears on any of the display screens 28 or 30.
  • a controller comprises a control unit that manipulates a cursor that appears on display screens 28 and 30.
  • the information from the controller can also appear on screens at remote locations, which may be displaying the images appearing on a display screen.
  • the controller can be used as a tool by an on-site user to communicate to remote personnel, such as areas of interest or concern.
  • An example of such a controller is a wired or wireless mouse that attaches to the wrist or other body part of an on-site person. Use of a portable mouse (e.g., a wrist mouse) allows the user to keep his hands free yet maintain convenient access to a control device.
  • the remote assist invention can also support the transfer of digital data and measurements provided by various measuring instruments such as heat sensors, digital calipers, digital micrometers and the like. These devices can supply digital information to the communication system.
  • display screen 28 or 30 displays an image transmitted from the on-site location while the other display screen displays an image at the remote location.
  • at least one display screen depicts information from a database. That is, at least one display screen can show information such as schematics, technical specifications, written instructions, or any other data that can be compiled or collected in a database for easy reference by the wearer of a video helmet. While the disclosed embodiment of the invention is shown as a helmet having sensory devices, it is noted that the invention can be easily adapted to operate without a helmet. Thus, sensory devices can be utilized that attach to various parts of the body other than to a user's head or headset. In addition, other sensory devices (e.g., temperature probes or other input devices) can be utilized that do not attach at all to the user.
  • communication can be done via a tethered cable or by a wireless local loop, telephone, fiber optics, ISDN, wireless or other transmission configuration that allows for mobility of the video helmet user.
  • the transmission of data and information from one on-site location to a remote location is effectively bi-directional; in other words, audio and video transmission, as well as access to database information, can be initiated by a user at any location.
  • Configuration 100 depicts an IBM Aptiva SC3 computer terminal 102, or computerized conferencing center, with remote assist software that links the on-site terminal 102 to a network.
  • An on-site location can include a desktop video camera 104.
  • the desktop video camera can record audio and video and transmit the recorded information to a desktop camera port 103 of computer terminal 102.
  • the desktop camera 104 can be connected to the computer terminal 102 by a variety of methods. For example, as noted earlier, a low-loss tether audio cable interlaced with a video cable can be used to conduct signals recorded by the desktop camera 104 to the computer terminal 102 via a camera control unit 145. Also, the transmission of signals can be done via wireless local loop or other wireless networks. For example, an antenna can be attached to the desktop camera 104, capable of two-way communication of digital signals with a camera control unit 145 or directly to a computer network 102.
  • the on-site location can supply an audio signal to the microphone port 105 of the terminal 102.
  • the audio signal is preferably detected by a sensory input device, such as a microphone 24 standing alone, attachable to the person or attached to a video helmet as shown in Fig. 3.
  • the signals can be transmitted over an audio cable 42 or other transmission configuration, such as wireless local loop.
  • the headset camera 22 in Fig. 3 can be represented by headset camera 116 of Fig. 4.
  • the signals from the headset camera 116 can be routed to a camera control unit 145 via camera port 147.
  • the signals received by the camera control unit 145 at camera port 146 can be transmitted by s- video port 148 and video port 149 to VCR video in 151 and video top 153 of the network computer 102.
  • Camera control unit 145 can also send the signal detected by the headset camera 116 to a display screen such as display screen 28 or 30 shown in Fig. 3. In this way, if desired, an on-site user wearing a video helmet can see what is detected by a camera.
  • One way to provide the signal from the headset camera 116 is via the external sync port 146 of the camera control unit 145.
  • the audio signals received by the terminal 102 from an on-site location can be preferably directed or routed via a VCR Audio port 129.
  • the audio signals from port 129 are routed to line-in port 109 to provide audio signals to the wearer.
  • the sound output port 117 can then preferably route the signals to a speaker port such as port 166 of a Kaiser Electro-Optics control box 160.
  • the control box can bi-directionally route signals to a headset 180 or headphones 190.
  • the video signal can be preferably routed via the VGA port 143 of the network 102 to the control box 160 and eventually for display on monitor 170.
  • the on-site computer network terminal 102 also has standard ports for input/output devices. These standard ports include printer port 141, COM port 139, mouse 139, phone connection 115, modem line 113, speakers 131 and keyboard 135.
  • the control box 160 can bi-directionally route audio and video signals detected by a sensory input/output device to output units at another location. In this way audio and video signals can be easily transmitted between an on-site location and a remote location via the on-site computer or network terminal 102. Controls can be found at both locations to allow a user to control the what another user sees or hears on a display screen, display monitor, earphone or other sensory device. Cameras or images located at the remote site can provide images to the on-site user on their display screen 28, 30. Similarly, cameras and images located at the on-site location can be displayed on the display screen 28, 30 of a video helmet and can also be displayed in real-time on a display at the remote location.
  • a terminal at a location can integrate a database or have access to a database.
  • the database can provide access to stored information, such as schematics, drawings, and specifications.
  • This information can be displayed on a display or display screen 28, 30 so that personnel at remote and on-site locations can refer to common reference material accessible at various locations.
  • the bi-directional configuration in Fig. 4 allows the video helmet incorporating the digital camera 22, microphone 24, earphone 26 and two display screens 28 and 30 to enable a wearer of the video helmet to see images being transmitted from an on-site location via a PC based video conferencing system.
  • Personnel at a remote location can transmit images to a display screen 28, 30 or can control what images are being seen on a display screen.
  • expert personnel could transmit information from a database or provide the images detected from the digital camera located at the on-site location.
  • a storage device stores information being captured by the digital camera 22, microphone 24 or any other input device.
  • FIG. 5 Shown in Fig. 5 is an embodiment of remote location network configuration 200.
  • the network configuration 200 can route the signals from the remote location by way of ISDN lines 210, 211 and 212.
  • the transmission capacity from at least three ISDN lines are used.
  • Three ISDN lines can transmit at approximately 384 kbs.
  • Full motion display capabilities typically require about 390 kbs.
  • the equivalent of three ISDN lines provides nearly full motion.
  • Other networking mechanisms can also be employed within the scope of the invention.
  • Remote location configuration 200 depicts an IBM Aptiva SC3 computer network 203, or computerized conferencing center, with remote assist software that ultimately links the remote network 203 to an on-site location.
  • the embodiment of Fig. 5 also includes a video camera 201. Support personnel at the remote location can transmit video images taken at the remote location to on-site locations.
  • the video camera 201 is shown connected to a desktop camera port 204.
  • Microphone speaker 202 is also connected to computer terminal 203 to ultimately provide audio signal for communication with an on-site location.
  • the audio signals from microphone 202 are split to microphone port 205 and speakers 225.
  • the audio signals can be routed to VCR audio out jack 224 for routing to a sound board at the line in port 206.
  • the audio signals are routed to the speakers of a monitor 232.
  • the video signals displayed on the monitor 232 arrive from a VGA port 231.
  • the computer terminal 203 can include other standard ports such as microphone port 208, modem line port 209, modem phone jack 220, disk drive port 226, keyboard port 227, mouse port 228, COM port 229 and printer 230.
  • a cleanroom-safe device frees the wearer's hands while he receives and transmits real-time images, sound and data related to any problem at hand.
  • the wearer of the video helmet 43 can interact with experts at a remote location.
  • the wearer, or on-site user can hold up components and lay them on a table to show them to repair persons at a different location.
  • the repair or support persons could provide real-time direction, for example by circling certain parts to indicate that the parts need to replaced, repairs, cleaned, etc.
  • the on-site user can see this information on one of the display screens.
  • the on-site user can also see any factory engineers he is talking to as well as schematic drawing and other relevant material. If the user chooses, he can view any of these images on a large monitor.
  • multiple remote users and multiple remote links can easily be established, supported, and maintained. Such multiple links are shown in Figs. 6 and 7.
  • the network 52 and related videoconferencing software can simultaneously support multiple users at one or more on-site locations, such as on-site locations 54 and 56.
  • the network 62 and related videoconferencing software can simultaneously support multiple support personnel at one or more remote locations, such as remote locations 66 and 68.
  • all the skills of a manufacturing support team can be available immediately to address any problem.
  • remote users at one location and repairpersons at another location can view images simultaneously, coordinate efforts, and thereby more easily solve any problems.
  • the disclosed configuration is a convenient way to reduce maintenance and repair costs.
  • repair experts can guide the on-site user by providing visual information to the on-site user over one of the-display screens 28, 30. In this way, the on-site user can "see” on a display screen in real-time what he should actually be seeing before him.
  • the on-site user can have access to a database on a network, such as network 102, that could contain easy access to such information as technical specifications, written test procedures, or, any other information.
  • doctors at remote locations can effectively establish two-way communication.
  • the off-site or remote doctor can provide real-time input or response by communicating to the doctor in several ways.
  • the remote doctor can provide visual guidance to the on-site operating doctor via one of the display screens 28, 30 or over an ear piece 26.
  • the remote doctor or medical personnel can also provide real-time information to indicate the status of body parts or patients vitals on a display screen 28, 30 or over an ear piece 26.
  • Remote assist can be used as a form of remote repair, diagnostics, and maintenance (RRDM).
  • RRDM enables a factory repairperson to respond to service problems effectively from afar without having to make a costly and time-consuming visit.
  • the computerization of many kinds of business equipment and processes makes RRDM possible. While it is impossible to physically send a spare part over a communication link, when a machine that stalls has electronic controls, factory engineers can connect on-line and in real-time to troubleshoot the system and make adjustments that can get the equipment back on-line. In other environments, such as on construction sites, construction workers can communicate in real-time with site foremen, engineers or architects, particularly when problems or critical issues arise. Engineers, site foremen, architects and the like, do not necessarily have to be present to respond to all situations when only feedback, approval, or simple instruction and the like are needed. Other applications of the remote assist communication system are also possible and are considered within the spirit and scope of the invention.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Telephonic Communication Services (AREA)

Abstract

Disclosed herein is an invention for remote communication, wherein a user (500) in one location (502) has real-time access to information captured at a remote location (503). By combining video conferencing capabilities and a unique video helmet on-site workers (500) can communicate and collaborate directly with support centers and factories. In an embodiment, the invention disclosed herein allows support personnel (521) to remotely assist on-site workers (500) by easily directing complex procedures since remotely-located support personnel (521) will be able to see and hear exactly what the on-site workers (500) are experiencing. Additionally, the disclosed invention allows for emergency and preventive maintenance to be conducted in real-time at far less cost than conventional approaches.

Description

REMOTE ASSIST SYSTEM
Field of the Invention
The present invention relates to a remote communication system.
Background of the Invention
The need for information delivery solutions in commercial and industrial environments rapidly increasing. As systems become more complex and worksites become more distributed throughout the world, the need increases for real-time access to data, information and communication with personnel located at remote areas.
For example, in certain industrial environments, such as semiconductor chip manufacturing environments, system downtime can be costly. Typically, the manufacturer faces a long wait whenever critical parts require expert maintenance or repair. If a technician or repair expert needs to be dispatched to the manufacturing site, it may take hours, or even days, for the repair expert to schedule and arrive at the site, especially if the site is overseas. Other situations, such as on construction sites, construction workers often need to communicate in real-time with site foremen, engineers or architects, particularly when problems or critical issues arise. Still other situations, such as a medical setting, doctors and other medical personnel often desire to communicate to other staff or colleagues in a different location.
Present communication systems do provide some features. For example, remote repair, as is the semiconductor manufacturing environment described above, can be provided by telephone or by providing "patch" software over the Internet. Cellular phone systems, two-way video conferencing, remote cameras also address basic telecommunication needs.
However, these present communication systems are limited in some way, especially in remote assist situations. For instance, while two-way video and sound communication exists, such as in video conferencing, present systems do not offer convenient access to databases as well as a solution that offers substantial mobility. Consequently, it would be advantageous to provide a communication system that provides simple and direct two-way video and audio communication. Further, it would be advantageous for the communication system to provide convenient access to reference and database information. Further, it would be advantageous to provide a communication system that is secure. It would be further advantageous to provide a communication system that not only allows a remote user to communicate to a local user, but also equally allows a local user to directly communicate with another user while both users can reference a common database. It would be advantageous for the communication system to be flexible and adaptable to a variety of situations and users. It would be further advantageous to provide a communication system that allows for mobile use by a user. It would be further advantageous to provide a communication system that can allow users to substantially have their hands free and not unnecessarily burdened with having to hold a communication or control device.
Summary of the Invention
Disclosed herein is an invention for remote communication, wherein a user in a remote location has real-time access to information captured at an on-site location and visa versa. In one embodiment, by combining video conferencing capabilities and a unique video helmet, on-site workers can communicate and collaborate directly with support centers and factories. The invention disclosed herein allows support personnel to remotely assist on-site workers by easily directing complex procedures since remotely-located support personnel will be able to see and hear exactly what the on-site workers are experiencing. Additionally, the disclosed invention allows for emergency and preventive maintenance to be conducted in real-time at far less cost than conventional approaches. Further details of aspects, objects, and advantages of the invention are described in the detailed description, drawings and claims.
Brief Description of the Drawings
Figure 1 depicts a general block diagram of a remote assist communication system according to an embodiment of the invention. Figure 2 depicts a representation of a remote assist communication system according to an embodiment of the invention.
Figure 3 depicts an embodiment of a video helmet configuration.
Figure 4 depicts a schematic of an embodiment of a remote assist apparatus.
Figure 5 depicts a schematic of an embodiment of a remote assist apparatus.
Figure 6 depicts a general block diagram of a remote assist communication system according to an embodiment of the invention.
Figure 7 depicts a general block diagram of a remote assist communication system according to an embodiment of the invention.
Detailed Description
In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be apparent, however, to one skilled in the art, that the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagrams in order to avoid unnecessarily obscuring the present invention. As noted, the need for effective remote communication increases, especially as worksites become more distributed. Disclosed herein is an invention for remote communication that allows at least two individuals to establish two-way communication wherein a user at a remote location has real-time access to information captured at an on-site location. Such an approach effectively permits a user at one location to have access and control to information obtained at another location. A user in one location has, in effect, a virtual presence in another location.
In one embodiment, the invention combines video conferencing capabilities and a unique video helmet to allow on-site workers to communicate and collaborate directly with remote support centers and factories.
In general, remote assist allows experts and support personnel at a remote location to conveniently and effectively communicate to workers at an on-site location. Experts and support personnel guide, direct or instruct workers so as to correct, or respond to, problems in real-time or provide on-the-spot preventive maintenance. By establishing bi-directional video, audio, and data retrieval interfaces, the remote assist invention offers a multi-sensory interface with distributed resources for users in a mobile environment.
A fundamental idea behind the present invention is that an issue to be addressed occurs at one location while additional expertise requested to help respond to the issue is located at another location. To illustrate in a manufacturing environment, an engineer or field service technician wearing a sensory device initiates a remote assist session with a location where the additional expertise is located. Substantial benefits can be realized where experts at a remote location can offer additional skills to immediately solve problems at essentially any virtual location. Fast and cost-effective emergency support can be available as well as support for maintenance operations. Real-time process analysis and around-the-clock support from hardware and software consultants can be made available without requiring the development of large and expensive support personnel. Additionally, remote assist can offer cost-effective and hands-on training as well as efficient and simplified communication of fab-to-fab best practices.
Moreover, remote assist can help manufacturers identify problems early on and thereby allow for easier escalation management. When used as part of an escalation management process, remote assist becomes an effective tool for quickly attending to intermittent problems, which can often prove difficult to reproduce during on-site visits. Figure 1 depicts a general configuration for remote assist communication system according to an embodiment of the invention. Remote assist 1 comprises an on-site location unit 2 connected to a remote location unit 4 via a computer or network 3. The on-site location 2 represents an on-site location from where information or direction is requested. On-site location 2 can be located in a campus, building, floor, laboratory bench, operating room, subsidiary, factory, oil rig, construction site and so forth. It is at an on-site location unit 2 where information is being gathered or where an issue/problem is directly being addressed.
Remote location 4 represents a location where the additional expertise or help is located. Remote location 4 can be a design or control center, or any site where additional expertise or information is located or distributable. Remote location 4 can be a customer support center, an engineering and design center, building, campus, floor, laboratory, construction site command center, oil rig command center, operating room and so forth, where the additional expertise desired is located. The remote location can preferably include access to a database for use by expert personnel in providing solutions or instruction.
On-site location unit 2 and remote location unit 4 interface via a network 3. Network 3, or computerized conferencing center, can consist of PCs, client-server networks, LANs, WANs, computer clusters, the Internet or other possible network configurations. In an embodiment, a terminal at an on-site location 2 or at a remote location 4 can include access to a database to provide real-time retrieval of information to personnel at either the on-site location 2 or remote location 4. Software that controls the transfer of information from any location to another includes videoconferencing software. The video conferencing software along with hardware allows users to display images from a remote site or local site on a display screen, or to display other images such as documents, video, or the like. Devices at the on-site location 2 collect and/or receive data from the remote location 4 via network 3.
One embodiment of a remote communication device is provided in Fig. 2. In Fig. 2, a network 3 links a remote location 503 to an on-site location 502. The on-site location 502 consists of on-site sensory devices 5 providing and receiving sensory inputs and outputs to an on-site terminal 6, coupled to the network 3. Similarly, the remote location 503 consists of a remote sensory devices 8 providing and receiving sensory inputs to a remote terminal 7, coupled to the network 3.
The on-site sensory devices 5 provide two-way data information transfer. A communication system will include devices for data gathering and data reception. For data gathering purposes, such sensory devices include, but are not limited to, cameras, microphones, and other sensory measurement and gathering devices. In effect, a virtual presence is established by personnel at an on-site location for communication of information with a user at a remote location. The on-site sensory devices 5 provide information (e.g., video, audio and data signals) regarding certain aspects of an on-site location 502. The signals are transmitted to an on-site computer terminal 6 and/or to personnel at the on-site location by, for example, a communication tether 9. On-site terminal 6 can store, process and communicate the signals to the network 3. Terminal 6 can also receive signals from the network and provide them to personnel at the on-site location. Similarly, remote sensory devices 8 are located at the remote location 503. The remote sensory devices 8 provide two-way data information transfer. The communication system will include devices for data gathering and data reception at the remote location. For data gathering purposes, such sensory devices include, but are not limited to, cameras, microphones, and other sensory measurement and gathering devices. In effect, a virtual presence can be established by personnel at a remote location for communication of information with a user at an on-site location.
The signals and information captured by remote sensory devices 8 are transmitted to a remote computer terminal 7 and/or to personnel at the remote location. Remote terminal 7 can store, process and communicate the signals to the network 3. Remote terminal 6 can also receive signals from the network and provide them to personnel at the remote location.
In an embodiment of the invention in Fig. 2, an on-site user 500 is shown wearing a sensory device 5 having various sensory inputs and outputs, such as microphone 510. The on- site sensory device can provide video, audio and data signals and information regarding certain aspects of the on-site location 502. The signals are transmitted to an on-site computer terminal 6. Signals can also be provided to personnel at the on-site location. Terminal 6 can store, process and communicate the signals to the network 3. On-site terminal 6 can also receive signals from the network and provide them to personnel at the on-site location.
As will be described with respect to Fig. 3, the user 500 in Fig. 2 can wear a video helmet having input and output sensory devices in various configurations. An on-site user 500 communicates information received from the various sensory devices to remote personnel 521 at the remote location 503 via on-site terminal 6. In Fig. 2, the user 500 is shown holding a camera 22 in his hand. The camera 22 can be pointed at an object, such as object 515. The image of the object 515 can be displayed effectively in real-time on terminals 516 and 517. If desired, measurement devices, such as sensor 520, can provide additional data to a remote user 521 or directly to the on-site terminal 6. Remote personnel 521 can communicate back to the user 500 or to the on-site location in general, via audio input devices or video devices, such as sensory device 8, coupled to the remote terminal 7. Based on the information provided by the on-site user 500, the remote personnel 521 can respond via video and/or audio communication. It will be understood that various input and output devices can be used, and in various configurations, to support the real-time communication between the on-site user 500 and remote personnel 521.
Shown in Fig. 3 is an embodiment of an input/output sensory device configuration 20 incorporates a video helmet 43 having a digital color camera 22, a microphone 24, earphones 26, and display screens 28 and 30 is depicted.
The video helmet 43 is preferably fully adjustable to adapt to the head and comfort level of the wearer. For instance, control knob 32 can adjust the tension of bracket 44 so as to be comfortable to the user. Volume control 34 can adjust the sound of the volume of the sounds entering earpiece 26 via audio cable 42. Image control button 32 allows for the focusing of display screen 28 and 30. Additional adjustment controls for brightness, eye spacing, depth, and so forth are also available. Further adjustments are possible by camera bracket 38 that is preferably slidable over bracket 44.
Additionally, the display screens 28 and 30 can be independently adjustable. Each display screen 28 and 30 can be adjusted by flipping or rotating upward. In a preferred embodiment, the display screens are sufficiently spaced in front of a users to allow for easy viewing of images on the display screens yet still allow a user to directly see and work upon physical objects before him.
The camera 22, attachable to video cable 40, is preferably removable from the video helmet 43 and can thereby be positioned or held, placed, or otherwise mounted at other locations to view an object. Additionally, the camera 22 can include a magnification feature where either a helmet wearer can magnify what is seen by the camera 22 and display the results on a display screen, or the camera is controlled by a user at another location.
The video helmet configuration 20 preferably is also clean-room safe. Clean-room fabric 36 is loosely or fixedly connected to the video helmet to create a clean-room compatible head mount.
A general video helmet is available from Kaiser Electro-Optics, Inc. For example, model Proview 30 is available having a display screen, microphone, and headset adjustment controls. Clean-room fabric 36, digital camera 22, camera bracket 38 and microphone 24 are designed, as shown in Fig. 3, to attach to the helmet model. The digital camera 22 is preferably a wide-angle camera requiring minimum adjustment and focusing. An available camera that is attachable to Proview 30 is the ELMO headset camera by Elmo Manufacturing Corporation. While the camera 22 is shown attached to a video helmet in Fig. 3, a camera or other image sensory device can be designed to be attachable to other parts of the body. If the camera is hand-held, the user can hold the camera to access difficult to reach areas or to be able to conveniently point at a desired location. In an embodiment, the display screens 28 and 30 not only are capable of showing images at the on-site location but can also independently display images being transmitted to and from the on-site location. Not only is video and audio communication possible but personnel can transmit and/or access electronic documentation such as schematics, assembly drawings, photos, videotaped demonstrations, technical specification and any documentation to be displayed on a display screen or monitor at any location. The video conferencing software allows the transfer of any image to and from an on-site location to a remote location.
Additionally, the remote assist communication system can include a controller that allows for the control of information that appears on any of the display screens 28 or 30. According to an embodiment, one such controller comprises a control unit that manipulates a cursor that appears on display screens 28 and 30. The information from the controller can also appear on screens at remote locations, which may be displaying the images appearing on a display screen. In this way, the controller can be used as a tool by an on-site user to communicate to remote personnel, such as areas of interest or concern. An example of such a controller is a wired or wireless mouse that attaches to the wrist or other body part of an on-site person. Use of a portable mouse (e.g., a wrist mouse) allows the user to keep his hands free yet maintain convenient access to a control device.
The remote assist invention can also support the transfer of digital data and measurements provided by various measuring instruments such as heat sensors, digital calipers, digital micrometers and the like. These devices can supply digital information to the communication system.
In one configuration, display screen 28 or 30 displays an image transmitted from the on-site location while the other display screen displays an image at the remote location. In an alternate embodiment, at least one display screen depicts information from a database. That is, at least one display screen can show information such as schematics, technical specifications, written instructions, or any other data that can be compiled or collected in a database for easy reference by the wearer of a video helmet. While the disclosed embodiment of the invention is shown as a helmet having sensory devices, it is noted that the invention can be easily adapted to operate without a helmet. Thus, sensory devices can be utilized that attach to various parts of the body other than to a user's head or headset. In addition, other sensory devices (e.g., temperature probes or other input devices) can be utilized that do not attach at all to the user.
In the discussion that follows, and as considered throughout in the specification, communication can be done via a tethered cable or by a wireless local loop, telephone, fiber optics, ISDN, wireless or other transmission configuration that allows for mobility of the video helmet user. Further, the transmission of data and information from one on-site location to a remote location is effectively bi-directional; in other words, audio and video transmission, as well as access to database information, can be initiated by a user at any location.
Further details of an on-site configuration embodiment are provided in Fig. 4. Configuration 100 depicts an IBM Aptiva SC3 computer terminal 102, or computerized conferencing center, with remote assist software that links the on-site terminal 102 to a network.
An on-site location, similar to configuration 100 in Fig. 3, can include a desktop video camera 104. The desktop video camera can record audio and video and transmit the recorded information to a desktop camera port 103 of computer terminal 102. The desktop camera 104 can be connected to the computer terminal 102 by a variety of methods. For example, as noted earlier, a low-loss tether audio cable interlaced with a video cable can be used to conduct signals recorded by the desktop camera 104 to the computer terminal 102 via a camera control unit 145. Also, the transmission of signals can be done via wireless local loop or other wireless networks. For example, an antenna can be attached to the desktop camera 104, capable of two-way communication of digital signals with a camera control unit 145 or directly to a computer network 102.
Additionally, the on-site location can supply an audio signal to the microphone port 105 of the terminal 102. The audio signal is preferably detected by a sensory input device, such as a microphone 24 standing alone, attachable to the person or attached to a video helmet as shown in Fig. 3. The signals can be transmitted over an audio cable 42 or other transmission configuration, such as wireless local loop. The headset camera 22 in Fig. 3 can be represented by headset camera 116 of Fig. 4. In an embodiment, the signals from the headset camera 116 can be routed to a camera control unit 145 via camera port 147. In an embodiment, the signals received by the camera control unit 145 at camera port 146 can be transmitted by s- video port 148 and video port 149 to VCR video in 151 and video top 153 of the network computer 102.
Camera control unit 145 can also send the signal detected by the headset camera 116 to a display screen such as display screen 28 or 30 shown in Fig. 3. In this way, if desired, an on-site user wearing a video helmet can see what is detected by a camera. One way to provide the signal from the headset camera 116 is via the external sync port 146 of the camera control unit 145.
The audio signals received by the terminal 102 from an on-site location can be preferably directed or routed via a VCR Audio port 129. In the disclosed embodiment, the audio signals from port 129 are routed to line-in port 109 to provide audio signals to the wearer. The sound output port 117 can then preferably route the signals to a speaker port such as port 166 of a Kaiser Electro-Optics control box 160. The control box can bi-directionally route signals to a headset 180 or headphones 190. The video signal can be preferably routed via the VGA port 143 of the network 102 to the control box 160 and eventually for display on monitor 170.
The on-site computer network terminal 102 also has standard ports for input/output devices. These standard ports include printer port 141, COM port 139, mouse 139, phone connection 115, modem line 113, speakers 131 and keyboard 135.
The control box 160 can bi-directionally route audio and video signals detected by a sensory input/output device to output units at another location. In this way audio and video signals can be easily transmitted between an on-site location and a remote location via the on-site computer or network terminal 102. Controls can be found at both locations to allow a user to control the what another user sees or hears on a display screen, display monitor, earphone or other sensory device. Cameras or images located at the remote site can provide images to the on-site user on their display screen 28, 30. Similarly, cameras and images located at the on-site location can be displayed on the display screen 28, 30 of a video helmet and can also be displayed in real-time on a display at the remote location. A terminal at a location, such as terminal 102, can integrate a database or have access to a database. The database can provide access to stored information, such as schematics, drawings, and specifications. This information can be displayed on a display or display screen 28, 30 so that personnel at remote and on-site locations can refer to common reference material accessible at various locations. The bi-directional configuration in Fig. 4 allows the video helmet incorporating the digital camera 22, microphone 24, earphone 26 and two display screens 28 and 30 to enable a wearer of the video helmet to see images being transmitted from an on-site location via a PC based video conferencing system. Personnel at a remote location can transmit images to a display screen 28, 30 or can control what images are being seen on a display screen. For instance, expert personnel could transmit information from a database or provide the images detected from the digital camera located at the on-site location. Additionally, in an embodiment, a storage device stores information being captured by the digital camera 22, microphone 24 or any other input device. Use of various sensory devices allow an on-site user to "see" the expert personnel at remote locations in real-time and/or receive images, instruction or guidance as to what a user should be "seeing" or doing.
Shown in Fig. 5 is an embodiment of remote location network configuration 200. The network configuration 200 can route the signals from the remote location by way of ISDN lines 210, 211 and 212. As shown in the embodiment of Fig. 4, the transmission capacity from at least three ISDN lines are used. Three ISDN lines can transmit at approximately 384 kbs. Full motion display capabilities typically require about 390 kbs. Thus, the equivalent of three ISDN lines provides nearly full motion. Other networking mechanisms can also be employed within the scope of the invention.
Remote location configuration 200 depicts an IBM Aptiva SC3 computer network 203, or computerized conferencing center, with remote assist software that ultimately links the remote network 203 to an on-site location.
The embodiment of Fig. 5 also includes a video camera 201. Support personnel at the remote location can transmit video images taken at the remote location to on-site locations. The video camera 201 is shown connected to a desktop camera port 204. Microphone speaker 202 is also connected to computer terminal 203 to ultimately provide audio signal for communication with an on-site location. In an embodiment, the audio signals from microphone 202 are split to microphone port 205 and speakers 225. The audio signals can be routed to VCR audio out jack 224 for routing to a sound board at the line in port 206. The audio signals are routed to the speakers of a monitor 232. The video signals displayed on the monitor 232 arrive from a VGA port 231.
The computer terminal 203 can include other standard ports such as microphone port 208, modem line port 209, modem phone jack 220, disk drive port 226, keyboard port 227, mouse port 228, COM port 229 and printer 230.
As related to the sensory input/output device depicted in Fig. 3, a cleanroom-safe device frees the wearer's hands while he receives and transmits real-time images, sound and data related to any problem at hand. The wearer of the video helmet 43 can interact with experts at a remote location. The wearer, or on-site user, can hold up components and lay them on a table to show them to repair persons at a different location. The repair or support persons could provide real-time direction, for example by circling certain parts to indicate that the parts need to replaced, repairs, cleaned, etc. The on-site user can see this information on one of the display screens. The on-site user can also see any factory engineers he is talking to as well as schematic drawing and other relevant material. If the user chooses, he can view any of these images on a large monitor.
It will be further understood that multiple remote users and multiple remote links can easily be established, supported, and maintained. Such multiple links are shown in Figs. 6 and 7. In the remote assist embodiment 50 shown in Fig. 6, for example, the network 52 and related videoconferencing software can simultaneously support multiple users at one or more on-site locations, such as on-site locations 54 and 56. Similarly, in the remote assist embodiment 60 shown in Fig. 7, the network 62 and related videoconferencing software can simultaneously support multiple support personnel at one or more remote locations, such as remote locations 66 and 68. In any of the remote assist embodiments disclosed herein, all the skills of a manufacturing support team can be available immediately to address any problem. If so desired, remote users at one location and repairpersons at another location can view images simultaneously, coordinate efforts, and thereby more easily solve any problems. For manufacturing environments, the disclosed configuration is a convenient way to reduce maintenance and repair costs. For example, repair experts can guide the on-site user by providing visual information to the on-site user over one of the-display screens 28, 30. In this way, the on-site user can "see" on a display screen in real-time what he should actually be seeing before him. Additionally, the on-site user can have access to a database on a network, such as network 102, that could contain easy access to such information as technical specifications, written test procedures, or, any other information. In other environments, such as surgical environments, doctors at remote locations can effectively establish two-way communication. While an on-site operating doctor can transmit real-time images over the digital camera 22, the off-site or remote doctor can provide real-time input or response by communicating to the doctor in several ways. For example, the remote doctor can provide visual guidance to the on-site operating doctor via one of the display screens 28, 30 or over an ear piece 26. The remote doctor or medical personnel can also provide real-time information to indicate the status of body parts or patients vitals on a display screen 28, 30 or over an ear piece 26.
Remote assist can be used as a form of remote repair, diagnostics, and maintenance (RRDM). RRDM enables a factory repairperson to respond to service problems effectively from afar without having to make a costly and time-consuming visit. The computerization of many kinds of business equipment and processes makes RRDM possible. While it is impossible to physically send a spare part over a communication link, when a machine that stalls has electronic controls, factory engineers can connect on-line and in real-time to troubleshoot the system and make adjustments that can get the equipment back on-line. In other environments, such as on construction sites, construction workers can communicate in real-time with site foremen, engineers or architects, particularly when problems or critical issues arise. Engineers, site foremen, architects and the like, do not necessarily have to be present to respond to all situations when only feedback, approval, or simple instruction and the like are needed. Other applications of the remote assist communication system are also possible and are considered within the spirit and scope of the invention.

Claims

CLAIMSWhat is claimed is:
1. A system for remote communication comprising: a video helmet comprising a video camera, a first display screen, an earphone, and a microphone, the video helmet capable of sending and receiving audio, video and data signals; a network for sending and receiving the audio, video and data signals from the video helmet to at least one remote location unit, and the at least one remote location unit capable of sending and receiving the audio, video and data signals with the video helmet via the network.
2. The system for remote communication as in claim 1 further comprising a data storage device coupled to the network, the storage device containing stored data accessible by controls on the video helmet, the stored data displayable on the first display screen.
3. The system for remote communication as in claim 1 wherein audio, video and data signals are controlled by a control device attachable to a wearer of the video helmet.
4. The system for remote communication as in claim 1 wherein the video helmet further comprises a second display screen.
5. The system for remote communication as in claim 4 wherein the first display screen simultaneously displays information received by the video camera and the second display screen is capable of displaying data received from the at least one remote location via the network.
6. The system for remote communication as in claim 4 wherein the first display screen displays information received from a database and the second display screen is capable of displaying data received from the at least one remote location via the network.
7. The system for remote communication as in claim 1 wherein images on the first display screen are controllable by signals sent by said at least one remote location via the network.
8. The system for remote communication as in claim 1 wherein the remote video helmet is lined with clean room compatible fabric.
9. The system for remote communication as in claim 1 wherein said at least one remote location is configured to communicate with a plurality of video helmets.
10. A system for remote communication comprising: a first mobile sensory apparatus at an on-site location, the first mobile sensory apparatus comprising a first set of information reception devices and a first set of information gathering devices said first set of information gathering devices generating a first set of information signals; at least one remote location unit comprising a second set of information reception devices and a second set of information gathering devices in connection with the first mobile sensory apparatus via the network, the at least one remote location having a remote display terminal, and a network configured to transmit the first set of information signals from the first mobile sensory apparatus to the at least one remote location unit and to transmit the second set of information signals from the at least one remote location unit to the first mobile sensory apparatus.
11. The system for remote communication as in claim 10 further comprising: a second mobile sensory apparatus at the on-site location, the second mobile sensory apparatus comprising a third set of information reception devices and a third set of information gathering devices said third set of information gathering devices generating a third set of information signals, and the network configured to transmit the third set of information signals from the second mobile sensory apparatus to the at least one remote location unit and to transmit the second set of information signals from the at least one remote location unit to the second mobile sensory apparatus.
12. The system for remote communication as in claim 10 further comprising a data storage device coupled to the network, the storage device containing stored data accessible by controls on the second set of information gathering devices, the stored data displayable on a display screen at the at least one remote location.
13. The system for remote communication as in claim 10 further comprising a data storage device coupled to the network, the storage device containing stored data accessible by controls on the first mobile sensory apparatus and the stored data displayable on a display screen of the first mobile sensory apparatus.
14. The system for remote communication as in claim 11 further comprising a second mobile sensory apparatus at the on-site location, the second mobile sensory apparatus comprises a fourth set of information reception devices and a fourth set of information gathering devices said fourth set of information gathering devices generating a fourth set of information signals, and the network configured to transmit the fourth set of information signals from the second mobile sensory apparatus to the first mobile sensory apparatus.
15. The system for remote communication as in claim 10 having a first display screen attached to the first is configured to display information received from the first set of information signals and further having a second display is configured to display information received from the second set of information signals.
16. The system for remote communication as in claim 10 wherein the first mobile sensory apparatus is lined with clean-room compatible fabric.
17. A system for remote communication comprising: a video helmet comprising a display screen, a detachable on-site camera, an on-site microphone, and audio speaker; a communication tether connecting the video helmet to an on-site terminal for communicating video and audio signals; the on-site terminal connected to a network, said on-site terminal having a processor and display monitor, and a remote terminal connected to the network, the remote terminal having a processor and video display to display information received from the video helmet, and the remote terminal generating information for display at the on-site location.
18. The system for remote communication as in claim 17 wherein images from the remote terminal are displayed on the display screen.
19. The system for remote communication as in claim 17 wherein images captured by the on-site camera are displayed on the display screen.
20. The system for remote communication as in claim 17 wherein a controller provides input to the display screen.
21. The system for remote communication as in claim 20 wherein the controller is a mouse attachable to the wrist of a wearer of the video helmet.
22. The system for remote communication as in claim 17 wherein the on-site terminal can access a database.
23. The system for remote communication as in claim 17 wherein the remote terminal can access a database.
PCT/US2000/005942 1999-03-15 2000-03-07 Remote assist system Ceased WO2000055714A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US27030099A 1999-03-15 1999-03-15
US09/270,300 1999-03-15

Publications (1)

Publication Number Publication Date
WO2000055714A1 true WO2000055714A1 (en) 2000-09-21

Family

ID=23030755

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2000/005942 Ceased WO2000055714A1 (en) 1999-03-15 2000-03-07 Remote assist system

Country Status (2)

Country Link
TW (1) TW480425B (en)
WO (1) WO2000055714A1 (en)

Cited By (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1152311A1 (en) * 2000-05-04 2001-11-07 Focke & Co. (GmbH & Co.) Mobile computer-assisted maintenance unit
WO2002067067A3 (en) * 2001-02-20 2003-06-19 Siemens Ag Combined eye tracking information in an augmented reality system
EP1237373A3 (en) * 2001-02-15 2004-01-07 Xybernaut Corporation An electronic news gathering system, a method of electronic news gathering, a method of fabricating an apparatus for facilitating electronic news gathering and an electronic news audio and video coverage system
DE102004019989B3 (en) * 2004-04-23 2005-12-15 Siemens Ag Arrangement and method for carrying out videoconferencing
FR2875989A1 (en) * 2004-09-24 2006-03-31 Renault Sas VIDEO DEVICE FOR INCREASING REALITY INCREASED AND METHOD FOR COMPARING TWO ENVIRONMENTS
AT505204B1 (en) * 2000-12-29 2009-03-15 Sticht Fertigungstech Stiwa OPERATIONAL MANAGEMENT FOR A MANUFACTURING AND / OR ASSEMBLING DEVICE AND METHOD FOR THE OPERATION THEREOF
US7664815B2 (en) 2002-02-22 2010-02-16 Lars Gundersen Communication method and system
ITTR20100009A1 (en) * 2010-11-11 2012-05-12 Advanced Technology Srl ADVANCED TECHNOLOGY GEOREFERECING OF FIND - GEOREFERENTATOR FOR OPERATORS AND FINDINGS ON THE SCENE OF CRIME OR EVENTS IN GENERAL
WO2012164155A1 (en) * 2011-05-27 2012-12-06 Nokia Corporation Method and apparatus for collaborative augmented reality displays
EP2584758A1 (en) * 2011-10-19 2013-04-24 Santarius s.r.o. Method of interactive communication
WO2015023443A1 (en) 2013-08-16 2015-02-19 Siemens Healthcare Diagnostics Inc. User interface tool kit for mobile devices
CN107422686A (en) * 2016-04-06 2017-12-01 劳斯莱斯电力工程有限公司 Equipment for allowing the remote control to one or more devices
WO2018145133A1 (en) * 2017-02-02 2018-08-09 Raubenheimer Pieter Jacobus Adriaan A supervisory, diagnostic and technical assistance device and a system for promoting remote supervision, diagnostic and technical assistance
US10257463B2 (en) 2012-02-16 2019-04-09 Covidien Lp Multifunctional conferencing systems and methods
CN111464792A (en) * 2020-05-12 2020-07-28 滨州学院 Network security emergency disposal device capable of remotely assisting operation
IT201900019776A1 (en) * 2019-10-25 2021-04-25 Amc S R L TROUBLESHOOTING AND MACHINERY MAINTENANCE SYSTEM IN A COMPANY'S PRODUCTION PLANT
CN116671720A (en) * 2023-06-14 2023-09-01 青岛飞奕科技有限公司 Backpack guidance system and method applied to remote maintenance of air conditioner
JP2025082208A (en) * 2023-11-16 2025-05-28 エヌ・ティ・ティ・コミュニケーションズ株式会社 Communication system and communication method

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI409730B (en) * 2009-07-29 2013-09-21 Young Optics Kunshan Co Ltd Head mounted display system

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5619183A (en) * 1994-09-12 1997-04-08 Richard C. Ziegra Video audio data remote system
EP0821257A2 (en) * 1993-10-22 1998-01-28 Kopin Corporation Portable communication display system

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0821257A2 (en) * 1993-10-22 1998-01-28 Kopin Corporation Portable communication display system
US5619183A (en) * 1994-09-12 1997-04-08 Richard C. Ziegra Video audio data remote system

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
MIAH T ET AL: "WEARABLE COMPUTERS - AN APPLICATION OF BT'S MOBILE VIDEO SYSTEM FORTHE CONSTRUCTION INDUSTRY", BT TECHNOLOGY JOURNAL,GB,BT LABORATORIES, vol. 16, no. 1, 1 January 1998 (1998-01-01), pages 191 - 199, XP000736939, ISSN: 1358-3948 *
REKIMOTO J ET AL: "THE WORLD THROUGH THE COMPUTER: COMPUTER AUGMENTED INTERACTION WITHREAL WORLD ENVIRONMENTS", ACM SYMPOSIUM ON USER INTERFACE SOFTWARE AND TECHNOLOGY,US,NEW YORK, NY: ACM, 1995, pages 29 - 36, XP000634412, ISBN: 0-89791-709-X *
STANSFELD S ET AL: "AN APPLICATION OF SHARED VIRTUAL REALITY TO SITUATIONAL TRAINING", PROCEEDINGS OF THE VIRTUAL REALITY ANNUAL INTERNATIONAL SYMPOSIUM,US,LOS ALAMITOS, IEEE COMP. SOC. PRESS, 1995, pages 156 - 161, XP000529981, ISBN: 0-7803-2543-5 *

Cited By (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1152311A1 (en) * 2000-05-04 2001-11-07 Focke & Co. (GmbH & Co.) Mobile computer-assisted maintenance unit
AT505204B1 (en) * 2000-12-29 2009-03-15 Sticht Fertigungstech Stiwa OPERATIONAL MANAGEMENT FOR A MANUFACTURING AND / OR ASSEMBLING DEVICE AND METHOD FOR THE OPERATION THEREOF
EP1237373A3 (en) * 2001-02-15 2004-01-07 Xybernaut Corporation An electronic news gathering system, a method of electronic news gathering, a method of fabricating an apparatus for facilitating electronic news gathering and an electronic news audio and video coverage system
WO2002067067A3 (en) * 2001-02-20 2003-06-19 Siemens Ag Combined eye tracking information in an augmented reality system
US7664815B2 (en) 2002-02-22 2010-02-16 Lars Gundersen Communication method and system
DE102004019989B3 (en) * 2004-04-23 2005-12-15 Siemens Ag Arrangement and method for carrying out videoconferencing
FR2875989A1 (en) * 2004-09-24 2006-03-31 Renault Sas VIDEO DEVICE FOR INCREASING REALITY INCREASED AND METHOD FOR COMPARING TWO ENVIRONMENTS
EP1641287A3 (en) * 2004-09-24 2009-11-04 Renault s.a.s. Video device for enhancing augmented reality and method for comparing of two surroundings
ITTR20100009A1 (en) * 2010-11-11 2012-05-12 Advanced Technology Srl ADVANCED TECHNOLOGY GEOREFERECING OF FIND - GEOREFERENTATOR FOR OPERATORS AND FINDINGS ON THE SCENE OF CRIME OR EVENTS IN GENERAL
WO2012062816A1 (en) * 2010-11-11 2012-05-18 Advanced Technology S.R.L. System for georeferencing forensic evidence
WO2012164155A1 (en) * 2011-05-27 2012-12-06 Nokia Corporation Method and apparatus for collaborative augmented reality displays
EP2584758A1 (en) * 2011-10-19 2013-04-24 Santarius s.r.o. Method of interactive communication
US10257463B2 (en) 2012-02-16 2019-04-09 Covidien Lp Multifunctional conferencing systems and methods
EP2826241B1 (en) * 2012-02-16 2020-02-12 Covidien LP Use of a conferencing system for providing remote assistance
US10728501B2 (en) 2012-02-16 2020-07-28 Covidien Lp Multifunctional conferencing systems and methods
WO2015023443A1 (en) 2013-08-16 2015-02-19 Siemens Healthcare Diagnostics Inc. User interface tool kit for mobile devices
EP3033594A4 (en) * 2013-08-16 2018-03-28 Siemens Healthcare Diagnostics Inc. User interface tool kit for mobile devices
CN107422686A (en) * 2016-04-06 2017-12-01 劳斯莱斯电力工程有限公司 Equipment for allowing the remote control to one or more devices
WO2018145133A1 (en) * 2017-02-02 2018-08-09 Raubenheimer Pieter Jacobus Adriaan A supervisory, diagnostic and technical assistance device and a system for promoting remote supervision, diagnostic and technical assistance
IT201900019776A1 (en) * 2019-10-25 2021-04-25 Amc S R L TROUBLESHOOTING AND MACHINERY MAINTENANCE SYSTEM IN A COMPANY'S PRODUCTION PLANT
CN111464792A (en) * 2020-05-12 2020-07-28 滨州学院 Network security emergency disposal device capable of remotely assisting operation
CN116671720A (en) * 2023-06-14 2023-09-01 青岛飞奕科技有限公司 Backpack guidance system and method applied to remote maintenance of air conditioner
JP2025082208A (en) * 2023-11-16 2025-05-28 エヌ・ティ・ティ・コミュニケーションズ株式会社 Communication system and communication method

Also Published As

Publication number Publication date
TW480425B (en) 2002-03-21

Similar Documents

Publication Publication Date Title
WO2000055714A1 (en) Remote assist system
CA2251080C (en) Computerized optometer and medical office management system
CN102473081B (en) Portable telepresence apparatus
US12485547B2 (en) Head mounted display for remote operation of machinery
JP2024036696A (en) Remote support method and remote support system for surgical support robot
CN106650200A (en) Remote presence system including a cart that supports a robot face and an overhead camera
KR20100139037A (en) Remote presence system on operating room hardware
JPH09247637A (en) Video / audio communication system, video communication system, transmitting terminal and receiving terminal
JP2005107758A (en) Maintenance system and information sharing system
US20180336500A1 (en) System, equipment and method for performing and documenting in real-time a remotely assisted professional procedure
JP2021010101A (en) Remote work support system
US20030097042A1 (en) Endoscopic system
US7574371B2 (en) Medical diagnosis management system
US20040205256A1 (en) System and method for communicating between two or more locations
US11354863B2 (en) Systems and methods for immersive and collaborative video surveillance
JP2003308117A (en) Maintenance system
US20100066802A1 (en) Remote communications device and method for facilitating manual operations
JP2017056069A (en) Nurse call system
WO2022130395A1 (en) A remote medical proctoring system and method thereof
JP4277756B2 (en) Camera fixing jig, camera fixing jig parts, and camera device
JP7727934B2 (en) Information display system, information processing device, and information display method
JP2001060109A (en) Maintenance system
JP7402586B2 (en) Business monitoring system
JPH11345021A (en) Machine tool maintenance support system
JP2001009680A (en) Machine tool maintenance support system

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): JP KR

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE

121 Ep: the epo has been informed by wipo that ep was designated in this application
DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
122 Ep: pct application non-entry in european phase