[go: up one dir, main page]

WO2025225189A1 - Système de traitement d'informations et procédé de traitement d'informations - Google Patents

Système de traitement d'informations et procédé de traitement d'informations

Info

Publication number
WO2025225189A1
WO2025225189A1 PCT/JP2025/009269 JP2025009269W WO2025225189A1 WO 2025225189 A1 WO2025225189 A1 WO 2025225189A1 JP 2025009269 W JP2025009269 W JP 2025009269W WO 2025225189 A1 WO2025225189 A1 WO 2025225189A1
Authority
WO
WIPO (PCT)
Prior art keywords
pose
image
information processing
processing system
estimated
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/JP2025/009269
Other languages
English (en)
Japanese (ja)
Inventor
遵 五味田
洋佑 今野
翔 加賀美
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Group Corp
Original Assignee
Sony Group Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Group Corp filed Critical Sony Group Corp
Publication of WO2025225189A1 publication Critical patent/WO2025225189A1/fr
Pending legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer

Definitions

  • the present disclosure relates to an information processing system and an information processing method.
  • Patent Document 1 discloses a technology that estimates in real time the changes in the position and orientation (hereinafter referred to as "pose") of a controller held in a user's hand based on images captured by a camera equipped in a head-mounted display, and performs control based on the estimated pose of the controller.
  • an information processing system comprising: an estimation unit that estimates a relative pose between a first device and a second device based on a first image captured by a first device worn on a user's head and a second image captured by a second device worn on a part of the user different from the head; and a presentation control unit that controls the presentation of information indicating the positional relationship between the first device and the second device based on the estimated relative pose.
  • an information processing method including: a processor estimating a relative pose between a first device and a second device based on a first image captured by a first device worn on a user's head and a second image captured by a second device worn on a part of the user different from the head; and controlling the presentation of information indicating the positional relationship between the first device and the second device based on the estimated relative pose.
  • FIG. 1 is a diagram illustrating the Outside-In method.
  • FIG. 1 is a diagram illustrating an inside-out method.
  • FIG. 10 is a diagram illustrating an error in a relative pose estimated by the Inside-Out method.
  • 10A and 10B are diagrams illustrating an example of a situation in which a relative pose cannot be estimated in the Inside-Out method.
  • FIG. 2 is a block diagram illustrating an example of a functional configuration of a head-mounted display 10 according to an embodiment of the present disclosure.
  • FIG. 2 is a block diagram showing an example of a functional configuration of a controller 20 according to the embodiment.
  • FIG. 2 is a diagram for explaining an overview of processing performed by the information processing system according to the embodiment.
  • FIGS. 10A and 10B are diagrams illustrating examples of information presentation control for each case according to the embodiment.
  • 10 is a diagram showing an example of a superimposed display of a first image I1 and a second image I2 according to the embodiment.
  • FIG. 10 is a flowchart showing an example of a processing flow of the information processing system according to the embodiment.
  • 10 is a flowchart showing an example of a processing flow of the information processing system according to the embodiment.
  • FIG. 2 is a block diagram showing an example of a hardware configuration of an information processing device 90 according to the embodiment.
  • Embodiment ⁇ 1.1. Background In systems using AR, VR, or MR, for example, a virtual space is constructed by estimating in real time the changes in the pose of a device such as a head-mounted display worn on the user's head, and changing the image presented to the user based on the estimated head movement.
  • a virtual space is constructed by estimating in real time the changes in the pose of a device such as a head-mounted display worn on the user's head, and changing the image presented to the user based on the estimated head movement.
  • One method for estimating the pose of a controller (or the user's hand holding the controller) is the so-called outside-in method, which estimates the pose of the controller based on an image of the controller captured by a camera equipped in a head-mounted display, as disclosed in Patent Document 1, for example.
  • Figure 1 is a diagram explaining the Outside-In method.
  • a camera attached to a head-mounted display 10 worn on the head of a user 50 captures an image of a controller 20 held in the hand of the user 50, and the relative pose of the controller 20 with respect to the head-mounted display 10 is directly estimated based on the captured image.
  • the controller 20 must be within the angle of view AV1 of the camera provided on the head-mounted display 10, which limits the range of movement of the hand holding the controller 20.
  • the pose of the controller 20 can also be estimated based on images captured by a camera equipped on the controller 20 (the so-called inside-out method).
  • Figure 2 is a diagram explaining the Inside-Out method.
  • the pose of the head-mounted display 10 is estimated based on an image captured by a camera provided in the head-mounted display 10
  • the pose of the controller 20 is estimated based on an image captured by a camera provided in the controller 20.
  • the Inside-Out method has the advantage that there are no restrictions on the range of movement of the hand holding the controller 20.
  • the relative pose of the controller 20 with respect to the head-mounted display 10 is estimated based on the pose of the head-mounted display 10 and the pose of the controller 20. Therefore, the error in the relative pose estimated with the Inside-Out method may appear as a sum of the error in the pose of the head-mounted display 10 and the error in the pose of the controller 20.
  • Figure 3 is a diagram explaining the error in the relative pose estimated using the Inside-Out method.
  • the error in the relative pose estimated based on the pose of the head-mounted display 10 and the pose of the controller 20 is ⁇ 1 + ⁇ 2.
  • the Inside-Out method has an issue not present in the Outside-In method: if it is not possible to estimate either the pose of the head-mounted display 10 or the pose of the controller 20 relative to the common environmental map 60, it is also not possible to estimate the relative pose.
  • Figure 4 is a diagram illustrating an example of a situation in which the relative pose cannot be estimated using the Inside-Out method.
  • Figure 4 shows an example of a case where the pose of the controller 20 can be estimated based on an image taken by a camera provided on the controller 20 and the environmental map 60, but where, for some reason, the pose of the head-mounted display 10 cannot be estimated based on an image taken by a camera provided on the head-mounted display 10 and the environmental map 60.
  • the relative pose also cannot be estimated. Note that, contrary to the example shown in Figure 4, if the pose of the head-mounted display 10 can be estimated but the pose of the controller 20 cannot be estimated, the relative pose also cannot be estimated.
  • An information processing system includes at least a head-mounted display 10 and a controller 20.
  • FIG. 5 is a block diagram showing an example of the functional configuration of the head-mounted display 10 according to this embodiment.
  • the head-mounted display 10 is an example of a first device.
  • the head-mounted display 10 includes a control unit 110, a camera 120, a display unit 130, an operation reception unit 140, a memory unit 150, and a communication unit 160.
  • Control unit 110 The control unit 110 according to this embodiment controls the operation of each component included in the head mounted display 10 .
  • control unit 110 The functions of the control unit 110 according to this embodiment are realized through cooperation between various processors, such as a Central Processing Unit (CPU) and a Graphics Processing Unit (GPU), and memory.
  • processors such as a Central Processing Unit (CPU) and a Graphics Processing Unit (GPU), and memory.
  • the control unit 110 may operate, for example, as an estimation unit, a presentation control unit, a first pose estimation unit, and a generation unit.
  • the camera 120 captures an image of the environment around the head-mounted display 10 under the control of the control unit 110 .
  • Display section 130 The display unit 130 according to this embodiment displays various types of visual information under the control of the control unit 110 .
  • control unit 110 creates a virtual space by changing the image displayed on the display unit 130 based on the pose of the head-mounted display 10 (which can also be said to be the pose of the user's head) estimated in real time.
  • control unit 110 changes the image displayed on the display unit 130 based on the relative pose of the controller 20 with respect to the head-mounted display 10 (which can also be considered the relative pose of the user's hand with respect to their head), thereby realizing an intuitive interface for actions such as grasping objects in the virtual space and various operations.
  • the operation reception unit 140 receives various operations by the user.
  • the operation reception unit 140 may include, for example, a button, a switch, a lever, and the like.
  • the storage unit 150 stores various types of information used by the head mounted display 10 .
  • the storage unit 150 stores an environmental map 60.
  • the communication unit 160 communicates information with other devices including at least the controller 20. Any communication standard may be adopted for the information communication.
  • FIG. 6 is a block diagram showing an example of the functional configuration of the controller 20 according to this embodiment.
  • the controller 20 according to this embodiment is an example of a second device.
  • the controller 20 is worn on a part of the head other than the head of the user wearing the head-mounted display 10.
  • the controller 20 according to this embodiment may be held in the user's hand.
  • the controller 20 according to this embodiment may be attached to the user's fingers, wrist, toes, ankle, etc.
  • the controller 20 When the controller 20 is attached to the user's toes, ankle, etc., it becomes possible to detect actions such as foot operations and kicks.
  • the controller 20 includes a control unit 210, a camera 220, an operation reception unit 240, a memory unit 250, and a communication unit 260.
  • Control unit 210 The control unit 210 according to this embodiment controls the operation of each component included in the controller 20 .
  • control unit 210 The functions of the control unit 210 in this embodiment are realized through cooperation between various processors, such as a CPU and GPU, and memory.
  • the control unit 210 may, for example, operate as a second pose estimation unit.
  • the camera 220 captures an image of the environment around the controller 20 under the control of the control unit 210 .
  • the operation reception unit 240 receives various operations by the user.
  • the operation reception unit 240 may include, for example, a button, a switch, a lever, and the like.
  • the storage unit 250 stores various types of information used by the controller 20 .
  • the storage unit 250 stores an environmental map 60.
  • the communication unit 260 performs information communication with other devices including at least the head mounted display 10. Any communication standard may be adopted for the information communication.
  • the functional configuration of the head-mounted display 10 described using FIG. 5 and the functional configuration of the controller 20 described using FIG. 6 are merely examples, and the functional configurations of the head-mounted display 10 and controller 20 according to this embodiment are not limited to these examples.
  • the head-mounted display 10 and the controller 20 may further include an audio output unit, a vibration output unit, etc.
  • the functional configuration of the head-mounted display 10 and controller 20 can be flexibly modified depending on specifications, operation, etc.
  • the information processing system generates an environmental map 60 based on an image (hereinafter also referred to as the first image) captured by the camera 120 provided in the head-mounted display 10 (S101).
  • the generation of the environmental map 60 in step S101 may be performed, for example, by the control unit 110 of the head-mounted display 10.
  • the information processing system estimates the pose of the head-mounted display 10 (hereinafter also referred to as the first pose) based on the first image and the environmental map 60 (S102).
  • the first pose estimation in step S102 may be performed, for example, by the control unit 110 of the head-mounted display 10.
  • the information processing system estimates the pose of the controller 20 (hereinafter also referred to as the second pose) based on an image captured by the camera 220 provided in the controller 20 (hereinafter also referred to as the second image) and the environmental map 60 (S103).
  • the second pose estimation in step S103 may be performed, for example, by the control unit 210 of the controller 20.
  • the information processing system estimates the relative pose between the head-mounted display 10 and the controller 20 based on the first pose estimated in step S102 and the second pose estimated in step S103 (S104).
  • the relative pose estimation based on the two poses in step S104 may be performed by, for example, the control unit 110 of the head-mounted display 10.
  • the information processing system estimates the relative pose between the head-mounted display 10 and the controller 20 based on the first image and the second image (S105).
  • step S105 The relative pose estimation based on both images in step S105 may be performed by, for example, the control unit 110 of the head-mounted display 10.
  • the information processing system determines the relative pose to be used for controlling the presentation of information indicating the positional relationship between the head-mounted display 10 and the controller 20 based on at least one of the relative pose estimated based on both poses in step S104 and the relative pose estimated based on both images in step S105 (S106).
  • step S106 the information processing system according to this embodiment is capable of detecting, for example, an error in the relative pose estimated based on both poses, based on both images.
  • the premise is that if the error in the relative pose estimated based on both poses is sufficiently small, then when the first image is transformed using that relative pose, it should be consistent with the second image.
  • the distance to the subject In order to achieve a perfect match, the distance to the subject must be known; if the distance is unknown or contains an error, the match will not be a perfect match, but rather will be determined by finding corresponding points on the epipolar line. However, in any case, if the error in the relative pose estimated based on both poses is sufficiently small, then transforming the first image using that relative pose will result in a match with the second image.
  • the information processing system can determine the magnitude of the error in the relative pose estimated based on both poses by checking for the presence or absence of such discrepancies.
  • the information processing system may perform processing such as controlling the presentation to display a less noticeable error, controlling the presentation to guide the user's movements to reduce the error, or prompting calibration.
  • the information processing system may correct an error in the relative pose estimated based on both poses, based on the relative pose estimated based on both images. Furthermore, the information processing system according to this embodiment may use the corrected relative pose to present information indicating the positional relationship between the head-mounted display 10 and the controller 20.
  • the relative pose estimated in step S105 does not have a unique scale (actual physical length), but rather is a position vector (orientation) with an indefinite orientation and scale, i.e., five of the six degrees of freedom for position and orientation are determined.
  • the information processing system is able to correct errors in the relative pose estimated based on both poses for the above five degrees of freedom, based on the relative pose estimated based on both images.
  • step S106 may be performed, for example, by the control unit 110 of the head-mounted display 10.
  • the information processing system controls the presentation of information indicating the positional relationship between the head-mounted display 10 and the controller 20 based on the relative pose determined in step S106 (S107).
  • control of the presentation of information indicating the positional relationship in step S107 may be performed, for example, by the control unit 110 of the head-mounted display 10.
  • the entity that performs each of the processes in steps S101 to S107 may be the control unit 110 of the head-mounted display 10 or the control unit 210 of the controller 20.
  • the information processing system includes a computer different from the head-mounted display 10 and the controller 20, the above-mentioned subject may be that computer.
  • the environmental map 60 may also be generated based on a second image.
  • the information processing system according to this embodiment is capable of estimating a relative pose based on both images. Therefore, even if it is not possible to estimate either or both of the two poses, the information processing system according to this embodiment can present information indicating the positional relationship between the head-mounted display 10 and the controller 20.
  • the information that can be presented may vary depending on the estimation conditions of the first and second poses and the contents of the first and second images.
  • Figure 8 is a diagram illustrating examples of information presentation control for each case according to this embodiment.
  • CASE A-1 is a case where both poses cannot be estimated and one of the cameras does not capture the other device (the controller 20 is not captured in the first image, and the head-mounted display 10 is not captured in the second image).
  • control may be performed to present only the orientation of the controller 20.
  • CASE: A-2 is a case where both poses cannot be estimated and one of the cameras captures the other device (the controller 20 is captured in the first image, or the head-mounted display 10 is captured in the second image).
  • control may be performed to present the position of the controller 20 using an approximate scale.
  • CASE: A-1 information may be displayed to guide the other device to be captured by one camera.
  • Figure 9 shows an example of guidance in CASE: A-1.
  • control unit 110 may cause display unit 130 to display an arrow corresponding to vector b-a, as shown in FIG. 9.
  • control unit 110 may cause the display unit 130 to display a message guiding the user to face the controller 20, as illustrated in FIG. 9.
  • control unit 110 may cause the display unit 130 to display virtual objects using Computer Graphics (CG) that match the worldview of the application (e.g., game, navigation, etc.), and may provide natural guidance to the user, such as by having the user follow the virtual object.
  • CG Computer Graphics
  • CASE B is the case where only one of the two poses can be estimated.
  • CASE: B it is possible to estimate the relative pose including the scale based on the one pose that can be estimated, the first image, and the second image.
  • the position of the controller 20 relative to the head-mounted display 10 may be presented based on the relative pose including the scale estimated as described above.
  • CASE C is the case where both poses can be estimated.
  • the position of the controller 20 relative to the head-mounted display 10 may be presented based on the relative pose including the scale estimated and corrected as described above.
  • both images In order to estimate the relative pose based on both images, both images must have a common field of view (the same area must be visible in both images).
  • the information processing system may control the presentation of information that guides the two images to have a common field of view if the two images do not have a common field of view.
  • the information processing system may display the first image I1 and the second image I2 superimposed on each other on the display unit 130, as shown in FIG. 10.
  • FIG. 10 shows an example of superimposed display of a first image I1 and a second image I2 according to this embodiment.
  • FIGS. 11A and 11B are flowcharts showing an example of the processing flow of the information processing system according to this embodiment.
  • Figures 11A and 11B are connected to each other by the circled letters.
  • the control unit 110 estimates the pose (first pose) of the head-mounted display 10 based on the image (first image) captured by the camera 120 (S201).
  • control unit 210 estimates the pose of the controller 20 (second pose) based on the image (second image) captured by the camera 220 (S202).
  • control unit 110 determines whether both poses have been estimated (S203).
  • control unit 110 determines that both poses have been estimated (S203: YES), it estimates a relative pose based on both poses (S204).
  • control unit 110 determines whether both poses have been estimated (S203: YES), it determines whether the two images have a common field of view (S205).
  • control unit 110 determines that both images have a common field of view (S205: YES), it estimates the relative pose based on both images (S206).
  • control unit 110 corrects the relative pose estimated based on both poses in step S204 with the relative pose estimated based on both images in step S206, and determines the corrected relative pose as the relative pose to be used for presenting information indicating the positional relationship between the head-mounted display 10 and the controller 20 (S207).
  • control unit 110 may determine the relative pose estimated based on both poses in step S204 as the relative pose to be used for presenting the information indicating the positional relationship, and skip step S207.
  • the control unit 110 controls the presentation of the position of the controller 20 based on the determined relative pose (S208).
  • the control unit 110 may also detect various operations within the virtual space based on the determined relative pose.
  • control unit 110 determines whether both poses have not been estimated (S203: NO), it determines whether one of the two poses has been estimated (S209).
  • control unit 110 determines whether one of the poses has been estimated (S209: YES), it determines whether the two images have a common field of view (S210).
  • control unit 110 determines that both images have a common field of view (S210: YES), it estimates a relative pose based on one of the estimated poses and both images (S211), and determines the estimated relative pose as the relative pose to be used for presenting information indicating the positional relationship.
  • control unit 110 controls the presentation of the position of the controller 20 based on the determined relative pose (S208).
  • control unit 110 determines whether the two images have a common field of view (S212).
  • control unit 110 determines that both images have a common field of view (S212: YES), it estimates a relative pose based on both images (S213) and determines the estimated relative pose as the relative pose to be used for presenting information indicating the positional relationship.
  • control unit 110 determines whether the scale can be roughly calculated (S214).
  • the control unit 110 determines that the scale can be roughly estimated (S214: YES), and roughly estimates the scale of the relative pose estimated in step S213 based on the size of the device shown in the image (S215).
  • control unit 110 controls the presentation of the position of the controller 20 based on the determined relative pose (S208).
  • control unit 110 determines that the scale cannot be roughly estimated (S214: NO)
  • it controls the presentation of the orientation of the controller 20 based on the determined relative pose (S216).
  • control unit 110 may control the presentation of information indicating the orientation of the controller 20 relative to the head-mounted display 10.
  • control unit 110 controls the presentation of information indicating the position of the controller 20 relative to the head-mounted display 10, as described above.
  • control unit 110 performs presentation control to guide the other device to appear in the image (so that the controller 20 appears in the first image, or so that the head-mounted display 10 appears in the second image) (S217).
  • control unit 110 determines that the two images do not have a common field of view (S205: NO, S210: NO, or S212: NO), it performs presentation control to guide the two images so that they have a common field of view (S218).
  • the information indicating the positional relationship between the head-mounted display 10 and the controller 20 is presented primarily using visual information, but the information indicating the positional relationship may also be presented using sound, vibration, etc.
  • the head-mounted display 10 and the controller 20 may also be associated in a one-to-N relationship.
  • the device combination according to this embodiment is not limited to the combination of the head-mounted display 10 and the controller 20.
  • multiple users will each wear a head-mounted display 10 that performs pose estimation based on the same environmental map 60, and share a virtual space (for example, an application in which a virtual object placed in a certain location from one user's perspective appears to be in the same location from another user's perspective).
  • a virtual space for example, an application in which a virtual object placed in a certain location from one user's perspective appears to be in the same location from another user's perspective.
  • FIG. 12 is a block diagram showing an example hardware configuration of an information processing device 90 according to an embodiment of the present disclosure.
  • the information processing device 90 may be a device having a hardware configuration equivalent to that of each of the devices described above.
  • the information processing device 90 includes, for example, a processor 871, a ROM 872, a RAM 873, a host bus 874, a bridge 875, an external bus 876, an interface 877, an input device 878, an output device 879, storage 880, a drive 881, a connection port 882, and a communication device 883.
  • a processor 871 for example, a processor 871, a ROM 872, a RAM 873, a host bus 874, a bridge 875, an external bus 876, an interface 877, an input device 878, an output device 879, storage 880, a drive 881, a connection port 882, and a communication device 883.
  • the hardware configuration shown here is an example, and some of the components may be omitted.
  • the information processing device 90 may include further components in addition to the components shown here.
  • the processor 871 functions, for example, as an arithmetic processing device or control device, and controls the overall operation or part of the operation of each component based on various programs recorded in the ROM 872, the RAM 873, the storage 880, or the removable storage medium 901.
  • the ROM 872 is a means for storing programs to be read into the processor 871, data to be used for calculations, etc.
  • the RAM 873 temporarily or permanently stores, for example, the programs to be read into the processor 871 and various parameters that change as appropriate when the programs are executed.
  • the processor 871, ROM 872, and RAM 873 are connected to one another via, for example, a host bus 874 that is capable of high-speed data transmission.
  • the host bus 874 is connected to an external bus 876 that has a relatively low data transmission speed via, for example, a bridge 875.
  • the external bus 876 is connected to various components via an interface 877.
  • the input device 878 may be, for example, a mouse, keyboard, touch panel, button, switch, lever, etc. Furthermore, a remote controller (hereinafter referred to as a remote control) capable of transmitting control signals using infrared rays or other radio waves may also be used as the input device 878.
  • the input device 878 may also include an audio input device such as a microphone.
  • the output device 879 is a device capable of visually or audibly notifying the user of acquired information, such as a display device such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), or an organic EL display, an audio output device such as a speaker or a headphone, a printer, a mobile phone, a facsimile, etc.
  • the output device 879 according to the present disclosure also includes various vibration devices capable of outputting tactile stimuli.
  • the storage 880 is a device for storing various types of data.
  • a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, or a magneto-optical storage device may be used as the storage 880.
  • the drive 881 is a device that reads information recorded on a removable storage medium 901 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, or writes information to the removable storage medium 901 .
  • a removable storage medium 901 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory
  • the removable storage medium 901 is, for example, a DVD medium, a Blu-ray (registered trademark) medium, an HD DVD medium, various semiconductor storage media, etc.
  • the removable storage medium 901 may also be, for example, an IC card equipped with a contactless IC chip, an electronic device, etc.
  • connection port 882 is a port for connecting an external device 902, such as a Universal Serial Bus (USB) port, an IEEE 1394 port, a Small Computer System Interface (SCSI), an RS-232C port, or an optical audio terminal.
  • USB Universal Serial Bus
  • SCSI Small Computer System Interface
  • RS-232C Small Computer System Interface
  • the externally connected device 902 is, for example, a printer, a portable music player, a digital camera, a digital video camera, or an IC recorder.
  • the communication device 883 is a communication device for connecting to a network, such as a communication card for wired or wireless LAN, Bluetooth (registered trademark), Wi-Fi (registered trademark) or Wireless USB (WUSB), a router for optical communication, a router for Asymmetric Digital Subscriber Line (ADSL), or a modem for various types of communication.
  • a network such as a communication card for wired or wireless LAN, Bluetooth (registered trademark), Wi-Fi (registered trademark) or Wireless USB (WUSB), a router for optical communication, a router for Asymmetric Digital Subscriber Line (ADSL), or a modem for various types of communication.
  • an information processing system includes an estimation unit that estimates a relative pose between a first device and a second device based on a first image taken by a first device worn on a user's head and a second image taken by a second device worn on a part of the user different from the head, and a presentation control unit that controls the presentation of information indicating the positional relationship between the first device and the second device based on the estimated relative pose.
  • the above configuration makes it possible to accurately estimate the relative pose between devices without restricting the user's movements.
  • steps involved in the processing described in this disclosure do not necessarily have to be processed chronologically in the order depicted in the flowcharts or sequence diagrams.
  • steps involved in the processing of each device may be processed in an order different from that depicted, or may be processed in parallel.
  • each device described in this disclosure may be realized by a program stored on a non-transitory computer-readable storage medium.
  • Each program is, for example, loaded into RAM when executed by a computer, and executed by a processor such as a CPU.
  • the storage medium may be, for example, a magnetic disk, optical disk, magneto-optical disk, flash memory, etc.
  • the program may be distributed, for example, via a network, without using a storage medium.
  • an estimation unit that estimates a relative pose between a first device and a second device based on a first image captured by a first device worn on a user's head and a second image captured by a second device worn on a part of the user different from the head; a presentation control unit that controls presentation of information indicating a positional relationship between the first device and the second device based on the estimated relative pose; Equipped with Information processing system.
  • the estimation unit determines the relative pose to be used for presenting the information indicating the positional relationship based on at least one of the relative pose estimated based on the first image and the second image, or the relative pose estimated based on a pose of the first device and a pose of the second device; The information processing system according to (2) above.
  • the estimation unit determines the relative pose to be used for presenting the information indicating the positional relationship by correcting the relative pose estimated based on the pose of the first device and the pose of the second device with the relative pose estimated based on the first image and the second image.
  • the estimation unit when either the pose of the first device or the pose of the second device has been estimated and the first image and the second image have a common field of view, determines the relative pose estimated based on either the pose of the first device or the pose of the second device that has been estimated, the first image, and the second image as the relative pose to be used for presenting information indicating a positional relationship between the first device and the second device;
  • the information processing system according to (3) above.
  • the estimation unit determines the relative pose estimated based on the first image and the second image as the relative pose to be used for presenting information indicating a positional relationship between the first device and the second device.
  • the estimation unit roughly estimates a scale related to the relative pose when the second device is captured in the first image or when the first device is captured in the second image;
  • the presentation control unit controls presentation of information indicating a position of the second device relative to the first device when the relative pose determined by the estimation unit has a scale.
  • the information processing system according to any one of (3) to (7).
  • the presentation control unit controls presentation of information indicating an orientation of the second device relative to the first device when the relative pose determined by the estimation unit does not have a scale.
  • the information processing system according to any one of (3) to (7).
  • the presentation control unit controls, when the first image and the second image do not have a common field of view, presentation of information that guides the first image and the second image to have a common field of view.
  • the information processing system according to any one of (4) to (6).
  • the presentation control unit displays the first image and the second image in a superimposed manner; The information processing system according to (10) above.
  • the presentation control unit controls the presentation of at least one of information guiding the second device to appear in the first image or information guiding the first device to appear in the second image when both the pose of the first device and the pose of the second device have not been estimated, the first image and the second image have a common field of view, the second device is not shown in the first image, and the first device is not shown in the second image.
  • the first device includes a head-mounted display; The information processing system according to any one of (1) to (13).
  • the presentation control unit causes a display unit included in the first device to display information indicating a positional relationship between the first device and the second device;
  • the second device includes a controller worn on the user's hand or foot; The information processing system according to any one of (1) to (15).
  • the processor Estimating a relative pose between a first device and a second device based on a first image taken by a first device worn on a user's head and a second image taken by a second device worn on a part of the user different from the head; controlling presentation of information indicating a positional relationship between the first device and the second device based on the estimated relative pose; Including, Information processing methods.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

Le problème décrit par la présente invention est d'estimer avec une grande précision une pose relative entre des dispositifs sans limiter le mouvement d'un utilisateur. La solution selon l'invention porte sur un système de traitement d'informations qui comprend : une unité d'estimation qui estime une pose relative entre un premier dispositif et un second dispositif, sur la base d'une première image capturée par le premier dispositif fixé à la tête d'un utilisateur, et d'une seconde image capturée par le second dispositif fixé à une partie du corps différente de la tête de l'utilisateur; et une unité de commande de présentation qui commande la présentation d'informations indiquant une relation de position entre le premier dispositif et le second dispositif sur la base de la pose relative qui est estimée.
PCT/JP2025/009269 2024-04-23 2025-03-12 Système de traitement d'informations et procédé de traitement d'informations Pending WO2025225189A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2024-070034 2024-04-23
JP2024070034 2024-04-23

Publications (1)

Publication Number Publication Date
WO2025225189A1 true WO2025225189A1 (fr) 2025-10-30

Family

ID=97489745

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2025/009269 Pending WO2025225189A1 (fr) 2024-04-23 2025-03-12 Système de traitement d'informations et procédé de traitement d'informations

Country Status (1)

Country Link
WO (1) WO2025225189A1 (fr)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170352184A1 (en) * 2016-06-06 2017-12-07 Adam G. Poulos Optically augmenting electromagnetic tracking in mixed reality
JP2018036720A (ja) * 2016-08-29 2018-03-08 株式会社タカラトミー 仮想空間観察システム、方法及びプログラム
WO2019155840A1 (fr) * 2018-02-07 2019-08-15 ソニー株式会社 Dispositif de traitement d'informations, procédé de traitement d'informations et programme

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170352184A1 (en) * 2016-06-06 2017-12-07 Adam G. Poulos Optically augmenting electromagnetic tracking in mixed reality
JP2018036720A (ja) * 2016-08-29 2018-03-08 株式会社タカラトミー 仮想空間観察システム、方法及びプログラム
WO2019155840A1 (fr) * 2018-02-07 2019-08-15 ソニー株式会社 Dispositif de traitement d'informations, procédé de traitement d'informations et programme

Similar Documents

Publication Publication Date Title
US20230315202A1 (en) Object Engagement Based on Finger Manipulation Data and Untethered Inputs
US9355452B2 (en) Camera and sensor augmented reality techniques
CN110780742B (zh) 眼球追踪处理方法及相关装置
US10185399B2 (en) Image processing apparatus, non-transitory computer-readable recording medium, and image processing method
JPWO2013132886A1 (ja) 情報処理装置、情報処理方法及びプログラム
US12455641B2 (en) Method and device for dynamically selecting an operation modality for an object
US12393279B2 (en) Information processing device and information processing method
WO2025225189A1 (fr) Système de traitement d'informations et procédé de traitement d'informations
EP3702008A1 (fr) Affichage d'une fenêtre d'un espace virtuel
US20240175717A1 (en) Information processing method, information processing apparatus, and program
US12401883B2 (en) System, imaging apparatus, information processing apparatus, information processing method, and information processing program
WO2019054037A1 (fr) Dispositif de traitement d'informations, procédé de traitement d'informations et programme
KR101549031B1 (ko) 포인팅 기능을 제공하는 장치, 방법 및 기록매체
WO2013132885A1 (fr) Dispositif de traitement d'informations, procédé de traitement d'informations et programme
US12412353B1 (en) Anchoring to multiple positions of an auxiliary device
US20240184376A1 (en) Control apparatus, control method, and control system for force-sense device
WO2024190223A1 (fr) Suivi de caméra cinématographique à l'aide d'une caméra de suivi ir et étalonnage pour une production virtuelle
US20240103634A1 (en) Motion Mapping for Continuous Gestures
US20250139914A1 (en) Control apparatus and information presentation method
JP7790203B2 (ja) 指示者側装置、方法およびプログラム
US12158988B2 (en) Mapping a computer-generated trackpad to a content manipulation region
EP4597263A1 (fr) Dispositif de traitement d'informations, procédé de traitement d'informations et programme
US20230245379A1 (en) Information processing apparatus for acquiring actual viewpoint position and orientation and virtual viewpoint position and orientation of user, information processing method, and storage medium
WO2020042589A1 (fr) Procédé et appareil d'estimation de distance d'utilisateur, dispositif, et support de stockage
US20230067564A1 (en) Virtual image display system and calibration method for pointing direction of controller thereof

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 25794210

Country of ref document: EP

Kind code of ref document: A1