[go: up one dir, main page]

WO2025224986A1 - Terminal portable et procédé de traitement d'informations - Google Patents

Terminal portable et procédé de traitement d'informations

Info

Publication number
WO2025224986A1
WO2025224986A1 PCT/JP2024/016486 JP2024016486W WO2025224986A1 WO 2025224986 A1 WO2025224986 A1 WO 2025224986A1 JP 2024016486 W JP2024016486 W JP 2024016486W WO 2025224986 A1 WO2025224986 A1 WO 2025224986A1
Authority
WO
WIPO (PCT)
Prior art keywords
wired
distance
wearable
processor
wired connection
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/JP2024/016486
Other languages
English (en)
Japanese (ja)
Inventor
宏司 中森
義憲 岡田
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Maxell Ltd
Original Assignee
Maxell Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Maxell Ltd filed Critical Maxell Ltd
Priority to PCT/JP2024/016486 priority Critical patent/WO2025224986A1/fr
Publication of WO2025224986A1 publication Critical patent/WO2025224986A1/fr
Pending legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/64—Constructional details of receivers, e.g. cabinets or dust covers

Definitions

  • the present invention relates to a wearable device and an information processing method.
  • HMD head-mounted display
  • Patent Document 1 discloses technology related to security when transmitting power wirelessly, and discloses technology in which, if a smartwatch determines that it is a predetermined distance away from a charging device, it determines that a third party has taken the charging device and issues a warning.
  • a variety of devices may be connected to a wearable device via a wired cable.
  • the wired cable may be pulled unintentionally by the user, which may interfere with the use of the wearable device.
  • the following wearable terminal comprises a display, a wired connection interface, and a processor.
  • the display displays a virtual space that may include virtual objects, an image of real space and virtual objects, or only an image of real space on a display screen.
  • a wired cable is connected to the wired connection interface, which is used for a wired connection with a device.
  • the processor then identifies the establishment of a wired connection with the device and, when it identifies that the processor is more than a predetermined distance away from the device's location, issues a notification to the user advising them to be careful about using the wearable terminal.
  • This information processing method uses a wearable device that includes a display that displays on a display screen a virtual space that may include virtual objects, an image of real space and virtual objects, or only an image of real space; a wired connection interface that is connected to a wired cable and is used for a wired connection with a device; and a processor.
  • This method includes the steps of: determining, by the processor, the establishment of a wired connection with the device; acquiring, by the processor, the distance from the device; and, if the processor identifies the establishment of a wired connection with the device and identifies that the wearable device is located a predetermined distance or more away from the device, issuing a notification to the user of the wearable device to warn them about use.
  • the present invention provides technology that enables effective use of a wearable device when a device is wired to the wearable device and the wearable device is used by a user. Note that other issues, configurations, and advantages will become clear from the description of the embodiments of the invention below.
  • FIG. 10 is a diagram illustrating an example of a display screen in use assistance control for a wired connection.
  • FIG. 10 is a diagram illustrating an example of a display screen in use assistance control for a wired connection.
  • FIG. 10 is a diagram illustrating an example of a display screen in use assistance control for a wired connection.
  • FIG. 10 is a diagram illustrating an example of a display screen in use assistance control for a wired connection.
  • FIG. 10 is a diagram illustrating an example of a display screen in use assistance control for a wired connection.
  • FIG. 10 is a diagram illustrating an example of a display screen in use assistance control for a wired connection.
  • FIG. 10 is a diagram illustrating an example of a display screen in use assistance control for a wired connection.
  • FIG. 10 is a diagram illustrating an example of a display screen in use assistance control for a wired connection.
  • FIG. 10 is a diagram illustrating an example of a display screen in use assistance control for a wired connection.
  • FIG. 2 is a diagram illustrating an example of a hardware configuration of a wearable terminal.
  • 10A and 10B are diagrams illustrating an example of information data and an example of the functional configuration of a wearable terminal.
  • FIG. 10 is a diagram illustrating an example of a display screen in use assistance control for a wired connection.
  • FIG. 10 is a diagram illustrating an example of a display screen in use assistance control for a wired connection.
  • FIG. 10 is a diagram illustrating an example of a display screen in use assistance control for a wired connection.
  • FIG. 10 is a diagram illustrating an example of a display screen in use assistance control for a wired connection.
  • FIG. 10 is a diagram illustrating an example of a display screen in use assistance control for a wired connection.
  • FIG. 10 is a diagram illustrating an example of a display screen in use assistance control for a wired connection.
  • FIG. 10 is a diagram illustrating an example of a display screen in use assistance control for a wired connection.
  • FIG. 10 is a diagram illustrating an example of a display screen when a user is located inside a third distance.
  • 10 is a flowchart illustrating an example of a processing flow of a wearable terminal or an information system.
  • FIG. 10 is a flowchart illustrating an example of a processing flow of a wearable terminal or an information system.
  • FIG. 10 is a diagram illustrating an example of a condition for starting use assistance control for a wired connection.
  • FIG. 10 is a diagram illustrating an example of auxiliary control for use of a wired connection.
  • FIG. 10 is a diagram illustrating an example of a condition for ending the use assistance control of a wired connection.
  • a computer executes a program using a processor (e.g., a CPU or GPU), and performs the processes defined in the program using storage resources (e.g., memory) and interface devices (e.g., communication ports). Therefore, the entity that performs the processes by executing a program may be the processor. Similarly, the entity that performs the processes by executing a program may be a controller, device, system, computer, or node that has a processor.
  • the entity that performs the processes by executing a program may be any computing unit, and may include a dedicated circuit that performs specific processing.
  • a dedicated circuit is, for example, an FPGA (Field Programmable Gate Array), ASIC (Application Specific Integrated Circuit), CPLD (Complex Programmable Logic Device), etc.
  • a program may be installed on a computer from a program source.
  • the program source may be, for example, a program distribution server or a computer-readable storage medium. If the program source is a program distribution server, the program distribution server may include a processor and storage resources for storing the program to be distributed, and the processor of the program distribution server may distribute the program to be distributed to other computers.
  • two or more programs may be realized as one program, or one program may be realized as two or more programs.
  • the wearable device 100 is an immersive HMD that is worn on the head of the user 10 so as to cover the eyes of the user 10.
  • the wearable device 100 includes, for example, gaze sensors (101, 102), a camera 103, a distance sensor 104, a positioning sensor 105, a microphone 106, a speaker 107, and a vibrator 108. Each component will be described in detail later.
  • the wearable device 100 can perform display operations based on, for example, a VR display mode in which a virtual space that may include virtual objects is displayed on the display screen, an AR display mode in which both an image of real space captured by the camera 103 and virtual objects are displayed on the display screen, or a captured image display mode in which only an image captured by the camera 103 is displayed on the display screen.
  • a VR display mode in which a virtual space that may include virtual objects is displayed on the display screen
  • an AR display mode in which both an image of real space captured by the camera 103 and virtual objects are displayed on the display screen
  • a captured image display mode in which only an image captured by the camera 103 is displayed on the display screen.
  • a character 151 which is an example of a virtual object, is displayed on the display screen 150a.
  • the wearable terminal 100 can also display various types of information.
  • status information regarding the wired connection of the wearable terminal 100 is displayed by the wired connection usage assistance control.
  • a message 152 indicating that the wearable terminal 100 is currently connected to the external device 120 via a wired connection
  • a graphic 153 an icon in this example
  • the type of information to be displayed is not limited to this example.
  • the display format is not particularly limited.
  • the wearable terminal 100 may display information based on symbols and colors in addition to the message 152 and the graphic 153.
  • the wearable terminal 100 may display information based on a combination of multiple of the message 152, graphic 153, symbols, colors, etc.
  • an information system including a wearable terminal 100 and a device 120 that is wired to the wearable terminal 100.
  • the wearable terminal 100 includes a connection unit 109 to which a wired cable 130 is connected.
  • this connection unit 109 can be configured to include a wired connection input/output interface, which will be described later.
  • the device 120 also includes a connection unit 121 to which the wired cable 130 is connected, forming an input/output interface.
  • the wearable terminal 100 and the device 120 are connected via the wired cable 130 (wired connection medium).
  • device 120 is a monitor that outputs video processed by wearable terminal 100.
  • any device that is wired to wearable terminal 100 may be used, and other types of devices may also be used in the information system.
  • Device 120 may also be a charging device for wearable terminal 100, an audio input device connected to wearable terminal 100 that inputs external audio, an audio output device that outputs audio processed by wearable terminal 100, a camera separate from camera 103 of wearable terminal 100, a personal computer, etc.
  • the wired device 120 is placed on the table 122.
  • the device 120 may be placed in a location other than on the table 122 in real space.
  • the distance 140 shown in the figure is the distance between the device 120 and the user 10 in real space.
  • Figure 2A shows an example of the relative positions of a wearable device, a device connected to the wearable device via a wired connection, and a wired cable.
  • a user 10 wearing an HMD 100 uses the HMD 100, for example, at a predetermined distance (i.e., the distance corresponding to the distance 140 described above) from a device 120 connected via a wired cable 130.
  • a predetermined distance i.e., the distance corresponding to the distance 140 described above
  • a first distance 201, a second distance 203, and a third distance 205 are input by the user 10 to the HMD 100, and the HMD 100 sets these distances (201, 203, 205).
  • These distances (201, 203, 205) indicate the predetermined distance from the device 120, based on the position of the device 120.
  • the first distance 201 is, for example, a distance set by the user 10 based on the judgment that a distance greater than this could cause inconvenience, such as being pulled by the connected wired cable 130.
  • the second distance 203 is, for example, a distance longer than the first distance 201 but shorter than the maximum length to which the wired cable 130 can be extended.
  • the third distance 205 is, for example, a distance shorter than the first distance 201 and closer to the device 120.
  • the area 202 between the first distance 201 and the third distance 205 is an area where the user 10 is unlikely to feel annoyed by being pulled by the wired cable 130 when moving.
  • the area 204 between the first distance 201 and the second distance 203 is an area where the user 10 is likely to feel annoyed by being pulled by the wired cable 130 when moving.
  • the area inside the third distance 205 is the area near the device 120 (in this example, the monitor).
  • the HMD 100 may perform use assistance control for a wired connection.
  • the HMD 100 in use assistance control for a wired connection, can display a wired connection area 211 in the field of view 206 of the camera of the HMD 100 when facing the device 120.
  • the wired connection area 211 relates, by way of example, to the area 204 between the first distance 201 and the second distance 203. Also, by way of example, it relates to the area within the first distance 201. Also, by way of example, it relates to the area within the second distance 203.
  • FIG. 2B shows an example of a display screen when displaying a wired connection area.
  • a character 151 is displayed on the display screen 210b.
  • the HMD 100 displays, as an example, a wired connection area 211 within the field of view 206 on the display screen 210b.
  • the HMD 100 may also display an image of real space corresponding to the wired connection area 211 acquired using the camera 103.
  • the HMD 100 may also generate and display a virtual object indicating the wired connection area 211.
  • the HMD 100 when the user 10 faces away from the device 120, the HMD 100, in the use assistance control of the wired connection, can display a wired connection area 221 in the field of view 207 of the camera of the HMD 100 when the user faces away from the device 120.
  • the wired connection area 221 relates to the area 204 between the first distance 201 and the second distance 203, for example. Also, for example, it relates to the area within the second distance 203.
  • Figure 2C shows an example of a display screen when displaying a wired connection area.
  • a character 151 is displayed on the display screen 210c.
  • the HMD 100 displays, as an example, a wired connection area 221 within the field of view 207 on the display screen 210c.
  • the HMD 100 may also display an image of real space corresponding to the wired connection area 221 acquired using the camera 103.
  • the HMD 100 may also generate and display a virtual object indicating the wired connection area 221.
  • the user 10 by making the user 10 aware of the wired connection area (211, 221) that takes into account the length of the connected wired cable 130, it is possible to help prevent annoyances that the user 10 may experience, such as the user 10 being pulled by the wired cable 130 (wired connection medium).
  • the HMD 100 when it identifies that it is a predetermined distance or more from the device 120, it may provide assistance by notification as a control to assist in the use of a wired connection.
  • the HMD 100 can display, for example, information on the display screen regarding the direction toward the connection unit 121 of the device 120, the position of the connection unit 121 of the device 120, the distance between the HMD 100 or the user 10 and the device 120, and the length of the wired cable 130.
  • Figure 2D shows an example of a display screen including notification information.
  • a character 151 is displayed on the display screen 210d.
  • the HMD 100 displays, as auxiliary controls for use of the wired connection, a direction mark 231 indicating the direction toward the monitor's connection point, a position mark 232 indicating the position of the monitor's connection point, a message 233 indicating the distance between the HMD 100 or user 10 and the monitor, and a message 234 indicating the length of the wired cable 130, "wired cable length 200 cm.”
  • real-space objects such as the monitor, table, and wired cable, shown with thin dashed lines, do not need to actually be displayed on the display screen 210d.
  • the real-space objects are illustrated to clarify their positions on the display screen.
  • the user 10 By allowing the user 10 to recognize notification information such as the direction of the wired connected device 120, the position of the device 120, the distance between the device 120 and the HMD 100 or the user 10, and the length of the wired cable 130, it is possible to help prevent events that the user 10 finds annoying, such as the user 10 being pulled by the wired cable 130.
  • the HMD 100 may provide assistance in notifying the user of the real-space situation as a control to assist in using a wired connection.
  • the HMD 100 can, for example, display on the display screen an object in the real space around the HMD 100 that has been photographed by the camera 103, or a virtual object of this object.
  • Figure 2E shows an example of a display screen.
  • a character 151 is displayed on the display screen 210e.
  • the HMD 100 may also display an object in real space captured by the camera 103, or a virtual object of this object.
  • the HMD 100 displays a table 122, a monitor which is an example of a device 120, and a wired cable 130.
  • the HMD 100 may provide assistance in notifying the user of the status of the wired cable 130 as a means of assisting in the use of a wired connection.
  • the HMD 100 may, for example, display the wired cable 130 on a display screen.
  • Figure 2F shows an example of a display screen.
  • a character 151 is displayed on the display screen 210f.
  • the HMD 100 may display an image of the wired cable 130 captured by the camera 103, or a virtual object 241 of this wired cable 130.
  • real-space objects such as the monitor and table shown with thin dashed lines do not need to be actually displayed on the display screen 210f.
  • the monitor and table are illustrated to clarify their positions on the display screen.
  • the HMD 100 may display the wired cable 130 or the virtual object 241 of the wired cable 130 on the display screen 210f so that it corresponds to its position in real space. However, the HMD 100 may also display the wired cable 130 or the virtual object 241 of the wired cable 130 on the display screen 210f so that it does not correspond to its position in real space. The HMD 100 may, for example, display the wired cable 130 or the virtual object 241 of the wired cable 130 on the edge of the display screen 210f so as not to interfere with the user 10's viewing.
  • an area on the display screen 210f for displaying the wired cable 130 or the virtual object 241 of the wired cable 130 may be set in advance, and the HMD 100 may adjust the size of the displayed wired cable 130 or the virtual object 241 of the wired cable 130 so that it fits within this area.
  • the user 10 by allowing the user 10 to recognize the state of the wired cable 130 (for example, the shape of the wired cable, whether the wired cable is bent, whether the wired cable is tangled, etc.), it is possible to help prevent events that the user 10 finds annoying, such as being pulled by the wired cable 130. Alternatively, it is possible to help the user 10 immediately recognize and deal with events that become annoying, such as being pulled by the wired cable 130.
  • the state of the wired cable 130 for example, the shape of the wired cable, whether the wired cable is bent, whether the wired cable is tangled, etc.
  • the HMD 100 may display, for example, as shown in Figures 2G-2H.
  • Figure 2G shows an example of a display screen.
  • a character 151 is displayed on the display screen 210g.
  • the HMD 100 may also display information indicating that the wired cable 130 is taut and information indicating that pressure is being applied to the connection terminal in the use assistance control of the wired connection.
  • the HMD 100 displays, on the display screen 210g, a figure 261 (icon) and a message 262 indicating that the wired cable 130 is taut, and a figure 263 (icon) and a message 264 indicating that pressure is being applied to the connection terminal.
  • FIG. 2H shows an example of a display screen.
  • a character 151 is displayed on the display screen 210h.
  • the HMD 100 may also display information indicating that the wired cable 130 has been detached during use assistance control for a wired connection.
  • a figure 271 (icon) and a message 272 indicating that the wired cable 130 has been detached are displayed on the display screen 210h.
  • the wearable device 100 (e.g., HMD) includes gaze sensors (101, 102), a camera 103, a distance sensor 104, a positioning sensor 105, a microphone 106, a speaker 107, and a vibrator 108.
  • the gaze sensors (101, 102) include, for example, a left eye gaze sensor 101 and a right eye gaze sensor 102 for the right eye.
  • the left eye gaze sensor 101 detects the left eye of the user 10 and can capture the viewpoint of the left eye at the gaze point by detecting the movement of the left eye, movement in the gaze direction, etc.
  • the right eye gaze sensor 102 detects the right eye of the user 10 and can capture the viewpoint of the right eye at the gaze point by detecting the movement of the right eye, movement in the gaze direction, etc.
  • the wearable terminal 100 may perform predetermined processing using gaze information acquired from the gaze sensors (101, 102).
  • the wearable terminal 100 may, for example, display an input operation screen such as an operation menu on the display 308.
  • the wearable terminal 100 may then acquire the position of an icon, button, etc. on the input operation screen at which the user 10 is looking as input operation information, and perform predetermined processing.
  • the process for detecting eye movement can utilize well-known technology commonly used for eye tracking.
  • infrared light is directed at the face from an infrared LED (Light Emitting Diode), and an image is taken with an infrared camera, an example of an eye gaze sensor.
  • eye movement using the light reflection point on the cornea as a reference point, eye movement (eye direction) can be estimated from changes in eye position and other geometric features.
  • the camera 103 captures a captured image by converting light incident from the lens into an electrical signal using an imaging element such as a CCD image sensor or CMOS image sensor.
  • the camera 103 can be positioned so that it can capture images of the area around the wearable device 100.
  • the wearable device 100 can acquire information such as the position of the wired connected device 120, the position of the connection part 121 of the wired cable 130 in the wired connected device 120, and the position and status of the wired cable 130. Note that there are no particular restrictions on the placement, orientation, number, viewing angle, etc. of the cameras 103 as long as appropriate processing can be performed.
  • the ranging sensor 104 is a sensor that can measure the distance from the wearable device 100 to an object, the object's position, and capture the three-dimensional shape of the object.
  • ranging sensors that can be used include LiDAR (Light Detection and Ranging), which irradiates an object with laser light such as infrared light and measures the scattered light that bounces back; TOF (Time of Flight) sensors, which measure distance by measuring the reflection time of pulsed light irradiated on the subject for each pixel; and millimeter-wave radar, which emits millimeter-wave radio waves and captures the reflected waves.
  • the ranging sensor 104 may also be a sensor that performs measurements based on the angle at which the light reflected by the object is received.
  • the ranging sensor 104 may be a triangulation sensor.
  • the ranging sensor 104 may also be a stereo camera that performs measurements based on parallax images.
  • the wearable device 100 processes the measurement results from the distance measurement sensor 104 and can obtain information such as the position of the wired connection destination device 120, the position of the connection part 121 of the wired cable 130 on the wired connection destination device 120, and the position and status of the wired cable 130.
  • the positioning sensor 105 is, for example, a GPS (Global Positioning System) receiver. In this case, by receiving satellite radio waves from GPS satellites and performing positioning calculations, it is possible to determine, for example, the coordinates on the Earth at which the wearable device 100 is located. The positioning calculations may be performed by the GPS receiver, or by other components (such as the processor 320 of the wearable device 100 or an external device).
  • the wearable device 100 may also acquire location information using other methods. For example, the wearable device may determine its current location by analyzing images captured by the camera 103 and/or three-dimensional point cloud data measured by the ranging sensor 104; techniques such as SLAM (Simultaneous Localization and Mapping) are known. If location information is acquired without using the positioning sensor 105, the positioning sensor 105 may be omitted.
  • GPS Global Positioning System
  • VPS Vehicle Positioning System
  • the wearable device 100 may acquire location information by VPS using images captured by the camera 103.
  • the wearable device 100 may also transmit the images captured by the camera 103 to an external device.
  • the external device may then use the captured images to acquire location information based on the VPS and distribute the acquired location information to the wearable device 100. Note that if location information is acquired without using the positioning sensor 105, the positioning sensor 105 may be omitted.
  • the wearable device 100 may also acquire location information by Pedestrian Dead Reckoning (PDR), for example, using a sensor described below. If location information is acquired without using the positioning sensor 105, the positioning sensor 105 may be omitted.
  • PDR Pedestrian Dead Reckoning
  • the microphone 106 collects the user's 10 own speech, external sounds, and other sounds, and converts them into audio data.
  • the user 10 may speak to give instructions such as input operations, and the wearable device 100 may acquire the audio data collected by the microphone 106 as instruction information for input operations, etc., and execute predetermined processing.
  • the speaker 107 outputs audio based on the audio data, and can, for example, notify the user 10 of notification information by audio.
  • the wearable device 100 can, for example, issue a warning message such as "Connected via wire!” by audio from the speaker 107 to notify the user 10. By providing audio notifications, usability can be improved.
  • the vibrator 108 is a device that generates vibrations under the control of the processor 320, and converts notification information sent by the wearable device 100 to the user 10 into vibrations.
  • the vibrator 108 transmits vibrations to the user 10 wearing the wearable device 100, allowing the user 10 to be notified of notification information such as a warning message such as "Wired connection in progress!” or the display operation of the wired connection area, thereby improving usability.
  • the wearable terminal 100 also includes an acceleration sensor 304, a gyro sensor 305, and a geomagnetic sensor 306.
  • the acceleration sensor 304 can detect, for example, the tilt and direction of the wearable terminal 100.
  • the gyro sensor 305 is a sensor that detects angular velocity or angular acceleration in the rotational direction, and can capture the posture of an object. Therefore, the acceleration sensor 304 and gyro sensor 305 can be used to detect the posture, such as the tilt and direction, of the wearable terminal 100.
  • the geomagnetic sensor 306 is a sensor that detects the Earth's magnetic force, and detects the direction in which the wearable terminal 100 is facing.
  • the wearable device 100 can use these sensors to acquire information about the posture and movements of the user 10. The wearable device 100 can then use the acquired information to understand the state of the user 10, and as a result, can analyze with greater precision the position and state of the wired connected device 120, the position and state of the wired cable 130, and so on.
  • the wearable terminal 100 also includes an operation input interface 307 and a display 308.
  • the operation input interface 307 is an interface that realizes input means, such as gaze, hands, a pointer, or even operating devices separate from the wearable terminal 100, such as a keyboard, key buttons, or touch keys, and the user 10 inputs the information they wish to input via the operation input interface 307.
  • the wearable terminal 100 may display an input operation screen such as an operation menu on the display screen of the display 308, take in the position on the input operation screen at which the user 10 is looking as input operation information, and perform an operation according to the input operation information.
  • the wearable terminal 100 may also display a cursor such as a hand or pointer on the input operation screen, take in operations performed by the user 10 using the cursor as input operation information, and perform an operation according to the input operation information.
  • the wearable terminal 100 may also take in gestures made by the user 10 as input operation information, and perform an operation according to the input operation information.
  • an input operation device such as a keyboard, key buttons, or touch keys
  • the input operation device may be connected to the wearable terminal 100 main body using a wired cable 130, or may be connected wirelessly.
  • Examples of user 10 input operations include setting the first, second, and third distances (201, 203, 205) related to assistive control of wired connections, initial settings for warning messages (152, 234, 262, 264, 272), and settings to disable control when connected via wire.
  • Other examples of user 10 input operations include application operations, volume adjustments, and various other setting operations.
  • the display 308 can be configured using a display such as an LCD display, a touch panel display, or an organic EL display, and displays virtual objects, camera-captured images, etc.
  • a display such as an LCD display, a touch panel display, or an organic EL display
  • the display 308 can be configured using a LCD panel, a display that displays holographic images, etc., and displays virtual objects, camera-captured images, etc.
  • the wearable terminal 100 also includes a processor 320, memory 330, a wired connection input/output interface 345, a short-range wireless communication interface 346, and a network communication interface 347.
  • the processor 320 is the main body that executes predetermined processes and is configured using a CPU (Central Processing Unit) and the like. As shown in the figure, each component of the wearable terminal 100 is connected to the processor 320 via a bus 350.
  • the processor 320 executes an operating system (OS) 333 and application programs 334 for operation control and the like stored in the memory 330, thereby controlling each component of the wearable terminal 100 and realizing the functions of the OS, middleware, applications, etc.
  • OS operating system
  • Memory 330 is composed of a main memory device, a non-volatile memory device, etc., and stores various programs 331 and information data 332 handled by processor 320, etc. An example of information data 332 will be described later.
  • Memory 330 stores, as programs 331, for example, an operating system 333, and various application programs 334 for controlling operations used by processor 320, etc. Memory 330 may also store various settings, settings made by user 10, etc.
  • the wired connection input/output interface 345 (wired connection interface) is configured for wired connection and is connected to the wired connection destination device 120 via the wired cable 130.
  • the wired connection destination device 120 is, as an example, an information processing device 360 (wired connection destination device), and the wired connection input/output interface 345 is an input/output interface that performs wired communication of information with the wired connection destination information processing device 360 via the wired cable 130.
  • a charging device or the like may be connected as the wired connection destination device 120, and the wired connection input/output interface 345 may be an input/output interface that inputs or outputs power via the wired cable 130.
  • the wired connection input/output interface 345 may also be configured to be connected via Power of Ethernet (PoE; Ethernet is a registered trademark).
  • PoE Power of Ethernet
  • Devices 120 that can be connected via a wired connection using the wired connection input/output interface 345 include display devices such as monitors, information processing devices such as personal computers, and charging devices, as well as audio devices such as earphones or loudspeakers, and input operation devices such as keyboards, key buttons, and touch keys, as shown in FIG. 1.
  • the wired cable 130 is a connection cable to the device 120 that is the wired connection destination. More specifically, the wired cable 130 is a connection cable such as a charging cable used to connect to a charging device, a twisted pair cable used with POE, a cable for wired earphones, or an audio cable. Furthermore, if the information processing device 360 is a personal computer, the wired cable 130 is, for example, a LAN cable.
  • the short-range wireless communication interface 346 is a communication interface that performs short-range wireless communication with a device (in this example, the information processing device 360) within the range where short-range wireless communication is possible.
  • Short-range wireless communication is performed using, for example, an electronic tag, but is not limited to this. If the wearable terminal 100 is located near a device that is at least capable of wireless communication, short-range wireless communication may be performed using Bluetooth (registered trademark), IrDA (Infrared Data Association, registered trademark), Zigbee (registered trademark), HomeRF (Home Radio Frequency, registered trademark), or wireless LAN (IEEE802.11a, IEEE802.11b, IEEE802.11g).
  • the network communication interface 347 is a communication interface that performs wireless communication with an external information server 380 or the like via an external network 381 using base station communication or the like.
  • the network communication interface 347 can receive various information stored in the information server 380.
  • long-distance wireless communication such as W-CDMA (Wideband Code Division Multiple Access) or GSM (Registered Trademark) (Global System for Mobile communications) can be used for base station communication.
  • W-CDMA Wideband Code Division Multiple Access
  • GSM Registered Trademark
  • Global System for Mobile communications Global System for Mobile communications
  • the memory 330 can store, as information data 332, image information 335, ranging information 336, location information 337, wired connection area information 338, virtual space information 339, wired connection medium status information 340, wired connection status information 341, and video information 342, for example.
  • Image information 335 is information relating to images acquired by the wearable device 100, and includes, for example, information relating to images of real space acquired by the camera 103 and information relating to images acquired by the distance measurement sensor 104. Note that image information 335 may also include information relating to images used to acquire position information of the wearable device 100.
  • the ranging information 336 is information relating to the distance to an object around the wearable device 100, and includes, for example, information relating to the distance to the object measured by the ranging sensor 104.
  • the ranging information 336 may also include information relating to the position of the object obtained using the ranging sensor 104.
  • the ranging information 336 may also include information relating to the shape of the object obtained using the ranging sensor 104.
  • Location information 337 is information about the location of the wired connected device 120 and information about the location of the wearable device 100 (i.e., the location of the user 10). The location information 337 is acquired by the wired connection destination device/wearable terminal location information processing unit 321, which will be described later.
  • Wired connection area information 338 is information about the wired connection area, which is an area where a connected wired cable 130 may be present. As an example, as described in the example display screen above, wired connection area information 338 may include information about the area between first distance 201 and second distance 203, which is an area where the wired cable 130 is likely to be an annoyance. Also, as an example, wired connection area information may include information about the area within first distance 201. Also, as an example, it may include information about the area within second distance 203. Also, as an example, it may include information about the area between first distance 201 and third distance 205. Wired connection area information 338 is acquired by wired connection area setting processing unit 323, which will be described later.
  • the virtual space information 339 is information about the virtual space and virtual objects displayed by the wearable terminal 100.
  • the virtual space information 339 may be generated by the processor 320 of the wearable terminal 100 executing a program, application software, or the like.
  • the virtual space information 339 may also be information generated by an external device (for example, an information server 380) and acquired by the wearable terminal 100 via communication.
  • Wired connection medium status information 340 is information regarding the status of the wired cable 130. Wired connection medium status information 340 is acquired by the wired connection medium status identification processing unit 326, which will be described later.
  • Wired connection status information 341 is information indicating whether the wired cable 130 is connected to the device 120. In other words, wired connection status information 341 indicates whether the wired cable 130 is disconnected. Wired connection status information 341 is acquired by the wired connection status identification processing unit 327, which will be described later.
  • Video information 342 is information relating to video captured using camera 103.
  • Video information 342 includes video captured by camera 103 of connection part 121 of wired device 120. This video is video showing the disconnection of wired cable 130, and includes, for example, a scene in which wired cable 130 is disconnected, a scene when connected wired cable 130 is disconnected, etc.
  • the timing for starting to acquire video information may be, for example, when the wired cable 130 is detached. That is, the wearable device 100 may start capturing this video (playback video) when the wired cable 130 is unplugged.
  • the timing for acquiring video information may also be, for example, when the wired cable 130 reaches a predetermined state. That is, the wearable device 100 may start capturing this video (playback video) when, for example, it acquires information that a predetermined amount of pressure or more has been applied to the wired cable 130, the connection terminal of the wiredly connected device 120, and/or the connection terminal of the wearable device 100. Note that the wearable device 100 may stop recording a predetermined time after the wired cable 130 is detached from the connection terminal.
  • the wearable terminal 100 includes, as an example, a wired connection destination device/wearable terminal position information processing unit 321, an icon/message notification processing unit 322, a wired connection area setting processing unit 323, a captured image display processing unit 324, a virtual space generation and display processing unit 325, a wired connection medium status identification processing unit 326, a wired connection status identification processing unit 327, and a video generation and display processing unit 328.
  • These components are realized by the processor 320 storing data such as programs in the main storage device of the memory 330 and processing the data. Note that other components (sensors, interfaces, etc.) may also be used to realize these components.
  • the wired connection destination device/wearable terminal location information processing unit 321 acquires the position of the wired connection destination device 120 and the position of the wearable terminal 100.
  • the wired connection destination device/wearable terminal location information processing unit 321 acquires the position of the wired connection destination device 120, for example, using image information 335 and/or distance measurement information 336.
  • the wired connection destination device/wearable terminal location information processing unit 321 may also acquire the position of the wired connection destination device 120, for example, by communicating with the wired connection destination device 120.
  • the wired connection destination device/wearable terminal location information processing unit 321 may, for example, obtain the position of the wearable terminal 100 using the positioning sensor 105.
  • the wired connection destination device/wearable terminal location information processing unit 321 may also obtain the position of the wearable terminal 100 using, for example, technology related to self-position estimation, VPS technology, etc.
  • the wired connection destination device/wearable terminal location information processing unit 321 can store information regarding the location of the wired connection destination device 120 and the location of the wearable terminal 100 in the memory 330 as location information 337.
  • the icon/message notification processing unit 322 generates an icon and/or message for notification information and outputs it to the display 308 to notify the user 10.
  • the icon/message notification processing unit 322 may also notify the user 10 of the content based on the notification information by generating audio data related to the message and outputting the audio data.
  • the icon/message notification processing unit 322 may also notify the user 10 of the content based on the notification information by generating vibration data corresponding to the content of the message and vibrating the vibrator 108 based on the generated vibration data.
  • the wired connection area setting processing unit 323 sets a wired connection area based on the information about the set distances (201, 203, 205) and stores the set details in memory 330 as wired connection area information 338.
  • the wired connection area setting processing unit 323 sets the area within the second distance 203 as wired connection area information 338 and stores the wired connection area information 338 in memory 330.
  • the captured image display processing unit 324 uses image information 335 to generate an image captured by camera 103 as a background and displays it on display 308. Furthermore, when performing a captured image display format in which only images captured by camera 103 are displayed on the display screen, the captured image display processing unit 324 uses image information 335 to display the image captured by camera 103 on display 308.
  • the captured image display processing unit 324 uses image information 335 to display images of surrounding objects (such as the wired cable 130) on the display 308 during auxiliary control of wired connection use.
  • the virtual space generation and display processing unit 325 uses virtual space information 339 to generate the virtual objects to be displayed and displays them on the display 308.
  • the virtual space generation and display processing unit 325 uses virtual space information 339 to generate the virtual spaces, virtual objects, etc. to be displayed and displays them on the display 308.
  • the virtual space generation and display processing unit 325 uses image information 335 and the like to generate virtual objects of surrounding objects (such as the wired cable 130) and displays the virtual objects on the display 308. Furthermore, in assistive control of wired connections, the virtual space generation and display processing unit 325 generates virtual objects such as direction marks and position marks and displays the virtual objects on the display 308.
  • the wired connection medium state identification processing unit 326 acquires wired connection medium state information 340 regarding the state of the wired cable 130 and stores it in memory 330.
  • the wired connection medium state identification processing unit 326 uses, for example, image information 335, distance measurement information 336, etc. to identify the shape of the wired cable 130, and detects whether the wired cable 130 is taut, whether the wired cable 130 is slack, whether the wired cable 130 is tangled, etc., to acquire the wired connection medium state information 340.
  • the wired connection medium state identification processing unit 326 may also compare the acquired information regarding the length of the wired cable 130 with information regarding the distance between the wired connection destination device 120 and the wearable terminal 100 or user 10 to detect whether the wired cable 130 is taut, whether the wired cable 130 is slack, etc., and acquire the wired connection medium state information 340.
  • the wired connection medium status identification processing unit 326 may also acquire information regarding whether pressure is being applied to the wired cable 130, the connection terminal of the wired connected device 120, and/or the connection terminal of the wearable terminal 100, and store this information in the memory 330.
  • the pressure at each connection terminal is detected by a piezoelectric element incorporated in each connection terminal.
  • the wearable terminal 100 acquires the pressure at the connection terminal on the device 120 side, for example, via the wired cable 130.
  • the wearable terminal 100 also acquires the pressure at the connection terminal on the device 120 side, for example, by communicating with the device 120 (wired communication or wireless communication).
  • the wearable terminal 100 acquires the pressure on the wearable terminal 100 side from the connection terminal of the wearable terminal 100.
  • the pressure on the wired cable 130 can be, for example, the same as the pressure applied to the connection terminal.
  • a piezoelectric element may be provided in the connection terminal of the wired cable 130, and the wearable terminal 100 may acquire the pressure applied to the connection terminal of the wired cable 130 from this piezoelectric element.
  • the wired connection status identification processing unit 327 acquires wired connection status information 341 related to the attachment and detachment of the wired cable 130 and stores it in the memory 330.
  • the wired connection status identification processing unit 327 may, for example, identify the presence or absence of input/output of data, etc. via the wired connection input/output interface 345, and acquire the wired connection status information 341.
  • the wired connection status identification processing unit 327 may, for example, acquire wired connection status information 341 indicating that the wired connection has been disconnected when input/output of data, signals, power, etc. is not possible between the wearable terminal 100 and the device 120 via the wired connection input/output interface 345.
  • the wired connection status identification processing unit 327 may also use image information 335, distance measurement information 336 indicating the shape of objects, and the like, around the connection unit 121 to identify whether the wired cable 130 is connected and acquire wired connection status information 341.
  • the wired connection status identification processing unit 327 may also acquire information regarding whether a wired connection has been established from the device 120 via communication.
  • the video generation and display processing unit 328 performs processing to output a video based on the video information 342. For example, in assistive control of wired connection use, the video generation and display processing unit 328 uses the video information 342 to generate a video including, for example, a scene in which the wired cable 130 is disconnected and/or a scene when the connected wired cable 130 is disconnected, and outputs the video to the display 308.
  • Figures 4 to 7A show examples of display screens for use assistance control for wired connections.
  • the portion of the wired cable 130 enclosed by the encircling line 401 is in an area outside the first distance 201, a virtual object of the character 151 is displayed on the display screen 150b, and the user 10 is using the HMD 100 in the area 202 between the first distance 201 and the third distance 205. Even when the user 10 is located in the area 202 between the first distance 201 and the third distance 205, a portion of the wired cable 130 is located outside the first distance 201. Therefore, in the use assistance control for the wired connection, the HMD 100 displays a message and/or an icon indicating that a wired connection is in progress on the display screen 150b, for example, by executing the icon/message notification processing unit 322.
  • the display screen 150b displays or re-displays an icon 153 indicating that a wired connection is in progress.
  • the icon 153 may be highlighted based on factors such as color or size.
  • a warning message 152 saying "Wired connection in progress!” is displayed on the display screen 150b.
  • the HMD 100 may display the wired cable 130 or a virtual object of the wired cable 130 on the display screen so as to correspond to a position in real space; however, as shown in Figure 5, when the user 10 faces away from the device 120, the wired cable 130 is not within the field of view 501 of the user 10, and it may not be possible to display the wired cable 130 or a virtual object of the wired cable 130 on the display screen so as to correspond to a position in real space.
  • the HMD 100 may not display the wired cable 130 or a virtual object of the wired cable 130 to correspond to a position in real space during use assistance control for a wired connection.
  • the HMD 100 may generate a virtual object 502 of the wired cable 130 by executing the virtual space generation and display processing unit 325, and display the virtual object 502 of the wired cable 130 at an appropriate position on the display screen 210i.
  • the display position may be, for example, on the edge of the display screen 210i so as not to interfere with the user 10's viewing of content, etc.
  • the HMD 100 may generate and display a virtual object of the entire wired cable 130, or may generate a virtual object 502 of a portion of the wired cable 130 (for example, a portion nearby that is outside the field of view) and display it on the display screen 210i.
  • the user 10 can recognize the presence of the wired cable 130 as a virtual object. This allows the user 10 to sense any annoying events, such as being pulled by the wired cable 130, and can prevent such events from occurring.
  • the user 10 is using the HMD 100 with the distance between the device 120 (in this example, a monitor) and the HMD 100 large and the wired cable 130 taut.
  • the HMD 100 can display the wired cable 130 or a virtual object of the wired cable 130 on the display screen so as to correspond to a position in real space.
  • the HMD 100 generates a virtual object 603 of the wired cable 130 by executing the virtual space generation and display processing unit 325 and displays it on the display screen 210j.
  • the HMD 100 executes the virtual space generation and display processing unit 325 to generate and display virtual objects: a direction mark 601 indicating the direction toward the connection unit 121 of the device 120, and a position mark 602 indicating the position of the connection unit 121 of the device 120. Furthermore, the HMD 100 executes the icon/message notification processing unit 322 to display on the display screen 210j a message 262 and/or an icon 261 indicating that the wired cable 130 is taut. For example, the display screen 210j displays or re-displays the icon 261 indicating that the wired cable 130 is taut. Note that the icon 261 may be highlighted based on factors such as color and size. For example, the display screen 210j displays a warning message 262 stating "A wired cable is taut!
  • the user 10 is using the HMD 100 with the wired cable 130 caught on the object 610 and pressure acting on the connection terminal.
  • the HMD 100 can display the wired cable 130 or a virtual object of the wired cable 130 on the display screen so as to correspond to a position in real space.
  • the HMD 100 executes the virtual space generation and display processing unit 325 to generate a virtual object 604 of the wired cable 130 caught on the object 610 and displays it on the display screen 210k.
  • the HMD 100 executes the virtual space generation and display processing unit 325 to display a direction mark 601 pointing toward the device 120 and a position mark 602 indicating the connection part 121 of the device 120, and executes the icon/message notification processing unit 322 to display an icon and/or message.
  • an icon 263 indicating that pressure is being applied to the connection terminal is displayed or re-displayed on the display screen 210k.
  • the icon 263 may be highlighted based on color, size, etc.
  • a warning message 264 stating "Pressure is being applied to the connection terminal!” is displayed on the display screen 210k.
  • the table, equipment (in this example, a monitor), and objects on the display screen shown in the same figure are illustrated to clarify their positions on the display screen, and do not actually need to be displayed.
  • the user 10 is using the HMD 100 with a tangled portion 605 in the wired cable 130.
  • the HMD 100 can display the wired cable 130 or a virtual object of the wired cable 130 on the display screen so as to correspond to a position in real space.
  • the HMD 100 generates a virtual object 607 of the wired cable 130 by executing the virtual space generation and display processing unit 325, and displays it on the display screen 210l.
  • this virtual object 607 includes a tangled portion 606 that corresponds to the tangled portion 605 in real space.
  • the HMD 100 may also display an icon and/or a message indicating that the wired cable 130 is tangled by executing, for example, the icon/message notification processing unit 322.
  • the wearable device 100 may identify the tangled portion 605 of the wired cable 130, and the wired connection medium state identification processing unit 327 may identify the state of the wired cable 130 excluding that portion. For example, if the wired cable 130 is tangled and taut, the HMD 100 may identify the tangled portion 605 of the wired cable 130, and the wired connection medium state identification processing unit 327 may identify the state of the wired cable 130 as being straight.
  • FIG. 7A shows an example of a situation in which the wired cable 130 connected to the device 120 becomes disconnected when the user 10 is using the HMD 100 at a position greater than the third distance 205.
  • the HMD 100 can display the wired cable 130 or a virtual object of the wired cable 130 on the display screen so as to correspond to a position in real space.
  • the HMD 100 generates a virtual object 703 of the wired cable 130 by executing the virtual space generation and display processing unit 325 and displays it on the display screen 210m.
  • the HMD 100 executes the virtual space generation and display processing unit 325 to display a direction mark 701 pointing toward the device 120 and a position mark 702 indicating the connection part 121 of the device 120, and executes the icon/message notification processing unit 322 to display an icon and/or message.
  • an icon 271 indicating that the wired cable 130 has been disconnected is displayed or re-displayed on the display screen 210m.
  • the icon 271 may be highlighted based on color, size, etc.
  • a warning message 272 stating "The wired cable has been disconnected! is displayed on the display screen 210m.
  • the HMD 100 may display a video (playback video) showing the detachment of the wired cable 130 on the display screen.
  • the HMD 100 can execute the video generation and display processing unit 328 to display a video 704 including the situation in which the wired cable 130 has been detached on the display screen 210n. By viewing the video 704, the user 10 can recognize the situation in which the wired cable 130 has been detached.
  • the user 10 may set the video playback time, which is the time from the start to the end of the video, and the video playback display processing unit 328 may display the video 704 showing the situation in which the wired cable 130 has been disconnected, for the video playback time set by the user 10.
  • the video playback time may be determined automatically, and the video playback display processing unit 328 may display the video 704 showing this situation for the automatically determined video playback time.
  • the video generation and display processing unit 328 generates a 15-second video 704 including the situation in which the wired cable 130 has been disconnected, based on the video playback time.
  • the video 704 displays objects around the connection unit 121 of the device 120, and in the example of the display screen 210n in FIG. 7A, a table, the device 120 (in this example, a monitor), and the wired cable 130 are displayed.
  • the display method is not particularly limited, but for example, the user 10 may select a display related to display screen 210m or a display related to display screen 210n, input the selection, and the wearable terminal 100 (in this example, an HMD) may display display screen 210m or display screen 210n based on the user's selection. On the other hand, the wearable terminal may automatically transition between display screen 210m and display screen 210n. Note that the wearable terminal 100 may simultaneously display display screen 210m and display screen 210n on the display 308.
  • Figure 7B shows an example of a display screen when the user 10 is located inside the third distance 205. If the wired cable 130 becomes unplugged while the user 10 is located within the third distance 205, it is assumed that the user 10 intentionally unplugged the wired cable 130 from the device 120. Therefore, the wearable terminal 100 (in this example, the HMD) does not display information indicating that the wired cable 130 has become unplugged on the display screen 210o.
  • the third distance 205 is a criterion for determining that the user 10 intentionally unplugged the wired cable 130.
  • the processor 320 of the wearable terminal 100 confirms the establishment of a wired connection between the wearable terminal 100 and the device 120 (S801). Then, while the user 10 is using the wearable terminal 100, the processor 320 determines whether the conditions for executing usage assistance control for a wired connection (usage assistance control during a wired connection) are met (S802). Then, if the processor 320 determines that the conditions for executing usage assistance control for a wired connection are met, it executes usage assistance control for a wired connection (S803).
  • the processor 320 performs wired connection usage assistance control on the condition that, for example, the distance between the wearable terminal 100 and the device 120 (apparatus) to which it is wired connected, or the connection unit 121 (wired connection point) of the device 120 (i.e., the distance 140 between the device 120 and the user 10) is equal to or greater than the first distance 201 (condition 1).
  • the processor 320 performs auxiliary control of the wired connection, for example, on the condition that at least a portion of the wired cable 130 (wired connection medium) is located at a position equal to or greater than the first distance 201 from the device 120 to which the wired connection is connected or the connection section 121 of the device 120 (Condition 2). For example, if a portion of the wired cable 130 is located outside the first distance 201, the processor 320 performs auxiliary control of the wired connection.
  • the processor 320 performs auxiliary control of the use of the wired connection, for example, on the condition that the second distance 205 is greater than the length of the wired cable 130 (condition 3).
  • the processor 320 performs auxiliary control of the wired connection, for example, on the condition that the pressure applied to the connection terminal of the wired cable 130 is equal to or greater than a first pressure amount (condition 4).
  • the first pressure amount is an amount of pressure that is set appropriately.
  • the processor 320 performs auxiliary control of the wired connection, for example, if the pressure applied to the connection terminal of the wired device 120 is equal to or greater than a second pressure amount (condition 5).
  • the second pressure amount is an amount of pressure that is set as appropriate.
  • the processor 320 performs auxiliary control of wired connection use, for example, on the condition that the pressure applied to the connection terminal of the wearable device 100 is equal to or greater than a third pressure amount (condition 6).
  • the third pressure amount is an amount of pressure that is set as appropriate.
  • the processor 320 performs auxiliary control of the wired connection, for example, on the condition that the wired cable 130 is in a straight line (condition 7).
  • the processor 320 can determine the state of the wired cable 130, for example, using the wired connection medium information 340.
  • the processor 320 performs assistance control for use of the wired connection, for example, when the distance between the wearable terminal 100 and the device 120 to which it is wired connected, or the connection unit 121 of the device 120, is equal to or greater than the third distance 205, and the wired connection has been disconnected (condition 8).
  • the processor 320 performs assistance control for use of the wired connection, when the distance 140 between the device 120 and the user 10 is equal to or greater than the third distance 205, and the wired connection has been disconnected.
  • the processor 320 can determine whether the wired connection has been disconnected, for example, using the wired connection status information 341.
  • control content related to S803 will be described. That is, an example of the control content for controlling assistance in using a wired connection will be described.
  • the processor 320 may execute a combination of multiple control contents.
  • the processor 320 issues a notification indicating that a wired connection is in progress (control 1).
  • the processor 320 confirms that a wired connection is in progress, for example, based on the wired connection status information 341.
  • the manner of notification is not particularly limited, and the processor 320 may issue a notification to that effect by displaying an icon, a message, a virtual object, outputting sound, outputting vibration, etc.
  • the processor 320 displays, for example, the device 120 connected via a wired connection and/or the area within the second distance 203 from the device 120 (control 2).
  • the processor 320 generates and displays a virtual object indicating the wired connection area, for example, based on the wired connection area information 338.
  • the processor 320 may also display, for example, an image of real space corresponding to the wired connection area acquired using the camera 103.
  • the processor 320 may, for example, issue a warning that the set second distance 203 is longer than the length of the wired cable 130 (control 3). Note that the length of the wired cable 130 used to connect to the device 120 may, for example, be input in advance by the user 10. The processor 320 may then store the length of the wired cable 130 set by the user 10 as information data 332 in the memory 330 and use it for processing.
  • the processor 320 displays, for example, a direction mark indicating the direction from the wearable device 100 to the device 120 to which it is wired connected, or the connection unit 121 of that device 120 (control 4).
  • the processor generates and displays a virtual object of the direction mark using the virtual space generation and display processing unit 325.
  • the processor 320 displays, for example, the wired device 120 or the connection part 121 of the device 120 (control 5).
  • the processor 320 displays, for example, the wired device 120 or the connection part 121 of the device 120 captured by the camera 103.
  • the processor 320 generates and displays, for example, a virtual object of a mark indicating the wired device 120 or the connection part 121 of the device 120, using the virtual space generation and display processing unit 325.
  • the processor 320 displays, for example, the distance from the wearable terminal 100 to the device 120 to which it is wired connected, or to the connection unit 121 of the device 120 (control 6).
  • the processor 320 displays the distance based on, for example, the display of an icon, a message, or a virtual object.
  • the processor 320 may also, for example, provide an audio notification regarding the distance.
  • the processor 320 displays the length of the connected wired cable 130 (control 7).
  • the processor 320 may store the length of the wired cable 130 set by the user 10 as information data 332 in the memory 330 and use it for processing.
  • the processor 320 issues a notification, for example, indicating that the wired cable 130 is straight (control 8). That is, the processor 320 issues a notification, for example, indicating that the wired cable 130 is taut.
  • the processor 320 checks the shape of the wired cable 130, for example, based on the wired connection medium status information 340.
  • the manner of notification is not particularly limited, and the processor 320 issues a notification by displaying an icon, displaying a message, displaying a virtual object, outputting sound, outputting vibration, etc.
  • the processor 320 issues a notification indicating that pressure is being applied to, for example, the wired cable, the wearable terminal, or at least one connection terminal of the wired connected device (control 9).
  • the processor 320 acquires information indicating that pressure is being applied, for example, based on the wired connection medium status information 340.
  • the manner of notification is not particularly limited, and the processor 320 issues the notification by displaying an icon, displaying a message, displaying a virtual object, outputting audio, outputting vibration, etc.
  • the processor 320 displays an image of real space (control 10).
  • the processor 320 may, for example, confirm that the wired cable 130 is within the field of view of the camera 103, and display an image of real space including the wired cable 130.
  • the processor 320 may also, for example, confirm that the wired device 120 is within the field of view of the camera 103, and display an image of real space including the wired device 120.
  • the processor 320 for example, generates and displays a virtual object corresponding to the connected wired cable 130 (control 11). Note that, for example, if the wired cable 130 is tangled, a virtual object of the wired cable 130 including the tangle may be displayed.
  • the processor 320 may also display an image of the wired cable 130 acquired by the camera 103.
  • the processor 320 displays the position of the disconnected wired cable 130 (control 12).
  • the processor 320 for example, generates and displays a virtual object indicating the position of the connection part 121 of the device 120 to which the wired cable 130 was connected.
  • the processor 320 may also generate and display a virtual object indicating the position of the end of the disconnected wired cable 130.
  • the processor 320 may also issue a notification indicating that the cable has been disconnected.
  • the manner of notification is not particularly limited, and the processor 320 may issue a notification indicating that the cable has been disconnected by displaying an icon, a message, a virtual object, outputting sound, outputting vibration, or the like.
  • the processor 320 displays a playback video (control 13). That is, the processor 320 displays a video of a situation in which the wired cable 130 has become disconnected, for example.
  • the processor 320 of the wearable terminal 100 determines whether the conditions for canceling use assistance control for wired connections are met (S804). If the conditions for canceling use assistance control for wired connections are met, the processor 320 cancels use assistance control for wired connections (S805).
  • conditions 1 to 8 relate to conditions 1 to 8 described above using Figure 9A.
  • the processor 320 releases the wired connection usage assistance control (conditions 1 to 8).
  • the processor 320 releases the use assistance control of the wired connection, for example, if the wired connection is released when the distance between the wearable terminal 100 and the wired device 120 or the connection unit 121 of the device 120 is less than the third distance 205 (Condition 9). In other words, since it is considered that the wired connection has been intentionally released by the user 10, the use assistance control of the wired connection is terminated.
  • processor 320 acquires the input from user 10 and cancels the use assistance control of the wired connection (condition 10).
  • the user 10 disconnects the wired connection within the third distance 205 (S806). Then, when the user 10 ends use of the wearable terminal 100, the processing ends. On the other hand, when the user 10 continues use of the wearable terminal 100 and makes a new wired connection, the same processing as above is performed (S807).
  • a wearable device 100 such as an HMD
  • the use assistance control for wired connections issues notifications urging caution regarding use of the wearable terminal 100, allowing the user 10 to be aware of the situation, thereby enabling better use of the wearable terminal 100. For example, even when a wired connection is made to another peripheral device, the inconvenience caused by the wired connection medium is reduced, allowing for comfortable and easy operation.
  • an immersive HMD that is worn on the user's head and displays virtual space, real space, etc. on the display unit has been used as a specific example of the wearable device 100.
  • an optically transparent HMD that displays virtual objects on the display unit while allowing the user to see physical objects in front of them, it can notify the user 10 by displaying an icon or message indicating the wired connection status, thereby providing the effect of reducing the inconvenience caused by the wired cable 130.
  • the present invention is not limited to the above-described embodiments, and includes various modifications.
  • the above-described embodiments have been described in detail to clearly explain the present invention, and are not necessarily limited to those including all of the described configurations.
  • it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment and it is also possible to add the configuration of another embodiment to the configuration of one embodiment.
  • the above-mentioned configurations, functions, processing units, acquisition sensors, etc. may be realized in hardware, for example by designing them as integrated circuits.
  • the above-mentioned configurations, functions, etc. may be realized in software by a processor interpreting and executing a program that realizes each function.
  • the processor includes transistors and other circuits, and may be considered circuitry or processing circuitry.
  • Information such as programs, tables, and files that realize each function may be stored in memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD, or may be stored in a device on a communications network.
  • control and information lines shown are those considered necessary for the explanation, and do not necessarily represent all control and information lines on the product. In reality, it is safe to assume that almost all components are interconnected.
  • the HMD 100 may also be configured by connecting a device such as a smartphone or tablet.
  • a device such as a smartphone or tablet.
  • the HMD 100 may be configured to use the display of the attached smartphone or tablet as the display 308.
  • the first distance 201, the second distance 203, and the third distance 205 may be determined, input, and set by the user 10.
  • the first distance 201, the second distance 203, and the third distance 205 may be automatically determined by the user 10 inputting the length of the wired cable 130.
  • the length of the wired cable 130 may be input in a format in which the user selects from predetermined lengths, such as 3 m, 5 m, and 7 m, for example.
  • the wearable terminal 100 may then store data (such as a table) in which the first distance 201, the second distance 203, and the third distance 205 are associated with the length of the wired cable 130, and set these distances by reading out the corresponding first distance 201, second distance 203, and third distance 205 from the input length of the wired cable 130. Note that the first distance 201, second distance 203, and third distance 205 are pre-registered in this data as appropriate distances.
  • Wearable terminal 101 Left eye gaze sensor 102 Right eye gaze sensor 103 Camera 104 Distance measurement sensor 105 Positioning sensor 106 Microphone 107 Speaker 108 Vibrator 130 Wired cable 304 Acceleration sensor 305 Gyro sensor 306 Geomagnetic sensor 307 Operation input interface 308 Display 320 Processor 330 Memory 345 Wired connection input/output interface 346 Near field wireless communication interface 347 Network communication interface

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

L'invention concerne un terminal portable porté par un utilisateur, ledit terminal portable comprenant : un dispositif d'affichage qui affiche, sur un écran d'affichage, un espace virtuel qui peut comprendre un objet virtuel, une image d'un espace réel et un objet virtuel, ou seulement une image d'un espace réel; une interface de connexion filaire à laquelle un câble filaire est connecté et qui est utilisée pour une connexion filaire avec un dispositif; et un processeur. Lorsque le processeur identifie que la connexion filaire avec le dispositif a été établie et identifie que la position du dispositif est au moins à une distance prescrite, le processeur fournit à l'utilisateur une notification favorisant l'attention de l'utilisateur à l'utilisation du terminal portable.
PCT/JP2024/016486 2024-04-26 2024-04-26 Terminal portable et procédé de traitement d'informations Pending WO2025224986A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/JP2024/016486 WO2025224986A1 (fr) 2024-04-26 2024-04-26 Terminal portable et procédé de traitement d'informations

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2024/016486 WO2025224986A1 (fr) 2024-04-26 2024-04-26 Terminal portable et procédé de traitement d'informations

Publications (1)

Publication Number Publication Date
WO2025224986A1 true WO2025224986A1 (fr) 2025-10-30

Family

ID=97489485

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2024/016486 Pending WO2025224986A1 (fr) 2024-04-26 2024-04-26 Terminal portable et procédé de traitement d'informations

Country Status (1)

Country Link
WO (1) WO2025224986A1 (fr)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016541035A (ja) * 2013-09-30 2016-12-28 株式会社ソニー・インタラクティブエンタテインメント ヘッドマウントディスプレイのユーザー向けカメラベース安全機構
JP2017184172A (ja) * 2016-03-31 2017-10-05 株式会社バンダイナムコエンターテインメント シミュレーションシステム
US20180260022A1 (en) * 2017-03-07 2018-09-13 Htc Corporation Method suitable for a head mounted device and virtual reality system

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016541035A (ja) * 2013-09-30 2016-12-28 株式会社ソニー・インタラクティブエンタテインメント ヘッドマウントディスプレイのユーザー向けカメラベース安全機構
JP2017184172A (ja) * 2016-03-31 2017-10-05 株式会社バンダイナムコエンターテインメント シミュレーションシステム
US20180260022A1 (en) * 2017-03-07 2018-09-13 Htc Corporation Method suitable for a head mounted device and virtual reality system

Similar Documents

Publication Publication Date Title
US12476881B2 (en) Electronic device with intuitive control interface
US10412379B2 (en) Image display apparatus having live view mode and virtual reality mode and operating method thereof
US11361497B2 (en) Information processing device and information processing method
JP2015095802A (ja) 表示制御装置、表示制御方法、およびプログラム
US20180254038A1 (en) Information processing device, information processing method, and program
WO2014016986A1 (fr) Système de partage d'un environnement tridimensionnel et procédé de partage d'un environnement tridimensionnel
US20160379591A1 (en) Information processing apparatus, control method, and storage medium
JP2015118442A (ja) 情報処理装置、情報処理方法およびプログラム
JP2018063540A (ja) ヘッドマウントデバイスを用いてコンテンツを提供するアプリケーションへの入力を支援するための方法、当該方法をコンピュータに実行させるためのプログラム、およびコンテンツ表示装置
WO2021236274A1 (fr) Localisation et visualisation de son
US10771707B2 (en) Information processing device and information processing method
JPWO2020012955A1 (ja) 情報処理装置、情報処理方法、およびプログラム
WO2017007643A1 (fr) Systèmes et procédés fournissant des indications non intrusives sur des obstacles
CN109714585B (zh) 图像传输方法及装置、显示方法及装置、存储介质
US11422622B2 (en) Electronic device and operating method thereof
JP2018067157A (ja) 通信装置およびその制御方法
WO2025224986A1 (fr) Terminal portable et procédé de traitement d'informations
US11523246B2 (en) Information processing apparatus and information processing method
CN117676070A (zh) 多媒体会议的开展方法和装置
WO2025120854A1 (fr) Dispositif de traitement d'informations, procédé de traitement d'informations et système de traitement d'informations
US20250036349A1 (en) Information processing device, information processing system, display device, and information processing method
CN115698967A (zh) 信息处理系统、信息处理方法和信息处理终端
US20250321632A1 (en) Information processing apparatus capable of properly controlling virtual object, method of controlling information processing apparatus, and storage medium
EP4535132A1 (fr) Dispositif de traitement d'informations, procédé de commande de dispositif de traitement d'informations, programme et support lisible par ordinateur
US20230091669A1 (en) Information processing apparatus, information processing system, and non-transitory computer readable medium storing program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24936902

Country of ref document: EP

Kind code of ref document: A1