WO2024195562A1 - Information processing device, information processing method, and program - Google Patents

Abstract

This information processing device is rendered to comprise a display control unit that: performs a carving-out process in which from a first image captured with an imaging device worn by an image-capturing user, an angle-of-view range narrower than the first image is carved out to be a second image for an observing user to observe; and in accordance with status information, varies displacement speed, produced by alterations in the carved-out range of the second image when there has been head-on directed displacement of the first image, in a displayed image that is based on the second image.

Description

Information processing device, information processing method, and program

This technology relates to information processing devices, information processing methods, and programs, and in particular to the technical field of image display.

In order to transmit a person's experience to others as it is, a system has been proposed that transmits first-person perspective images using a wearable device such as a head-mounted camera, allowing others to experience the first-person perspective images.
The following Patent Document 1 mentions that in such a system, when an image is cut out from a 360° video and displayed, the speed of movement of the displayed image is limited to prevent visual sickness.

WO2015/122108

When a person wearing a camera moves the camera quickly during real-time image distribution, such as live streaming, the image seen by the user watching the image will move faster across the entire screen, which can cause motion sickness. For this reason, the camera user must avoid moving the camera quickly while recording, for example by being careful about the movement of the head where the camera is worn. However, it is difficult to completely prevent motion sickness from occurring. Also, being too careful will prevent free recording.

On the other hand, as in Patent Document 1, slowing down the movement of the image displayed on the viewing side can prevent motion sickness, but by limiting the speed, a time delay occurs before the image in front of the camera is displayed, compromising real-time performance.

This disclosure therefore proposes technology that prevents visually induced motion sickness while minimizing the loss of real-time viewing for the viewer.

The information processing device related to the present technology performs a cut-out process to cut out a range of an angle of view narrower than that of a first image from a first image captured by an imaging device worn by an imaging user, and to create a second image for observation by an observing user, and is equipped with a display control unit that varies the displacement speed of a displayed image based on the second image, which is caused by a change in the cut-out range of the second image when the first image is displaced in a forward direction, in accordance with situational information.
In a system in which a second image is cut out from a first image and displayed, the cut-out position can be made to follow the change in the front direction of the first image, for example, the front direction of an imaging device that captures the first image. In this case, the speed of change in the displayed image is variably set according to the situation in order to reduce discomfort felt by the viewer of the second image.

FIG. 1 is an explanatory diagram of a system configuration according to an embodiment of the present technology; FIG. 2 is an explanatory diagram of an example of a display device used in the system of the embodiment. 5A and 5B are explanatory diagrams of displacement of a display image in the system according to the embodiment. FIG. 1 is a block diagram of a configuration example of a system according to an embodiment. FIG. 2 is a block diagram of an information processing device used in the system of the embodiment. 11 is a flowchart of a process relating to an image motion speed according to an embodiment. 11 is an explanatory diagram of a speed limit on the viewer side for the movement of a captured image according to an embodiment. FIG. 11 is an explanatory diagram of a speed limit on the viewer side for the movement of a captured image according to an embodiment. FIG. 11 is an explanatory diagram of a speed limit on the viewer side for the movement of a captured image according to an embodiment. FIG. 11 is an explanatory diagram of a speed limit on the viewer side for the movement of a captured image according to an embodiment. FIG. 11 is an explanatory diagram of a speed limit when a viewer instructs a movement in the same direction as the movement of a captured image in the embodiment. FIG. 11 is an explanatory diagram of a speed limit when a viewer instructs a movement in the same direction as the movement of a captured image in the embodiment. FIG. 11 is an explanatory diagram of a speed limit when a viewer instructs a reverse movement to the movement of a captured image in the embodiment. FIG. 11 is an explanatory diagram of a speed limit when a viewer instructs a reverse movement to the movement of a captured image in the embodiment. FIG. 11 is a flowchart of an example of processing according to status information according to an embodiment. 11 is a flowchart of an example of processing according to status information according to an embodiment. 11 is a flowchart of an example of processing according to status information according to an embodiment. 11 is a flowchart of an example of processing according to status information according to an embodiment. 11 is a flowchart of an example of processing according to status information according to an embodiment. 13 is a flowchart of a modified example of the process relating to the image motion speed according to the embodiment.

The embodiments will be described below in the following order.
1. Configuration of information processing system
2. Configuration of information processing device
<3. Processing example>
4. Summary and Modifications

In this disclosure, the term "image" includes both moving images and still images. Furthermore, the term "image" is used to mean not only the state displayed on a display device, but also image data in a signal processing process, a transmission process, or a state recorded on a recording medium.

1. System configuration
An example of a configuration of an information processing system according to an embodiment of the present disclosure will be described with reference to FIG.

FIG. 1 shows a schematic diagram of an information processing system 1. The information processing system 1 includes an imaging device 40, an information processing device 10, an information processing device 20, and an output device 30. The information processing system 1 may also include a content server 90.

The information processing device 10, the information processing device 20, and the content server 90 are connected to each other via a network N11 so as to be able to transmit and receive information to and from each other.
The type of the network N11 is not particularly limited. As a specific example, the network N11 may be configured as a so-called wireless network, such as a network based on the Wi-Fi (registered trademark) standard. As another example, the network N11 may be configured as the Internet, a dedicated line, a local area network (LAN), a wide area network (WAN), or the like. The network N11 may include a plurality of networks, and a part of the network may be configured as a wired network.

The imaging device 40 includes an imaging section 41, and the imaging section 41 captures images (e.g., moving images or still images) of the environment surrounding the imaging user Ua.
For example, the imaging device 40 is configured as a head-mounted type, and holds the imaging unit 41 at a predetermined position on the head of a user Ua (hereinafter referred to as "imaging user Ua") who wears the imaging device 40 and captures images. The imaging unit 41 includes, for example, an imaging element and an optical system (for example, a lens, etc.) for forming a subject image on the imaging element. Examples of the imaging element include a charge coupled device (CCD) and a complementary metal oxide semiconductor (CMOS).

The imaging device 40 may also include multiple imaging units 41 (e.g., imaging units 41a and 41b) that are arranged to capture images in different directions, for example, based on the imaging user Ua. By synthesizing the images captured by each of the multiple imaging units 41 through image processing or the like according to the angle of view of each of the multiple imaging units 41, it is possible to obtain an image that captures a wider range than the angle of view of each imaging unit 41, such as a full-sky image that is a 360° panoramic image.

Furthermore, it is preferable that one or more of the imaging units 41 are equipped with a wider-angle lens, and images in each direction based on the imaging user Ua are captured by one or more of the imaging units 41. The imaging unit 41 may be one that captures wide-angle images using a fisheye lens.

In the following explanation, we will use as an example a case where a panoramic image is synthesized based on images captured by each of the multiple imaging units 41a and 41b, but this is not necessarily limited to this configuration.

For example, images captured by one or more imaging units 41 and obtained through synthesis processing and the like in the information processing device 10 (output images) can be a variety of images, such as a full-sphere image captured by two fisheye lenses, a hemispherical image captured by one fisheye lens, and a panoramic image (a full-sphere image and a panoramic image with a range of less than 360°). For the sake of explanation, such images transmitted from the information processing device 10 will be referred to as "captured images." As this captured image, it is sufficient to output an image with a wider angle of view than the image displayed on the output device 30 (called a "display image" to distinguish it from the captured image). On the output device side, a partial area of such a captured image is cut out and used as the display image.

The imaging device 40 is not limited to a head-worn wearable device, but may take a variety of forms, such as a neckband type worn around the neck of the imaging user Ua, a pendant type, a glasses type, a shoulder-worn type, or a general camera held by the imaging user Ua. In this embodiment, a type in which the imaging direction changes depending on the behavior of the imaging user Ua is assumed. The imaging direction refers to the front direction of the imaging angle of view. If the captured image is not a 360° image, the imaging direction may refer to the optical axis direction of the incident light being captured. Alternatively, in the case of a head-worn imaging device 40, the imaging direction may be the front direction of the head of the imaging user Ua.

The imaging device 40 is configured to be capable of transmitting and receiving information to and from the information processing device 10 held by the imaging user Ua, for example, via a wireless or wired communication path. Based on this configuration, the imaging device 40 transmits images captured by each of the imaging units 41a and 41b to the information processing device 10.

The information processing device 10 acquires images captured by the imaging units 41a and 41b from the imaging device 40. The information processing device 10 generates an image such as a panoramic image by synthesizing the images captured by the imaging units 41a and 41b.

The information processing device 10 also generates information on the behavior of the imaging user Ua, such as head rotation, i.e., in this case, information indicating the position and direction (front direction) of the viewpoint of the captured image, and associates it with the captured image. The behavior of the imaging user Ua, such as head rotation, is one aspect of the imaging user Ua's operation that affects the displayed image. In particular, the head behavior is one aspect of a position-unspecified operation that does not specify the position of image displacement.
The information processing device 10 may recognize changes in the position and direction of the viewpoint associated with the rotation of the head of the imaging user Ua, and generate a panoramic image, etc. so that rotation of the image associated with the change in the position and direction of the viewpoint is suppressed.

Changes in the position and orientation of the viewpoint can be recognized as changes in the position and orientation of the imaging device 40 based on the detection results of an acceleration sensor or an angular velocity sensor provided in the imaging device 40, for example. As another example, changes in the position and orientation of the viewpoint can be recognized by performing image analysis processing on the images captured by the imaging units 41a and 41b, respectively.

Then, the information processing device 10 transmits the generated captured image, for example a panoramic image, to the content server 90 via the network N11.

In addition, at least a part of the configuration of the information processing device 10 may be provided in the imaging device 40. For example, a configuration for generating a panoramic image may be provided in the imaging device 40. In addition, the information processing device 10 and the imaging device 40 may be configured integrally.

The content server 90 acquires, via the network N11, captured images (e.g., panoramic images) based on the results of imaging by the imaging device 40 from the information processing device 10 held by the imaging user Ua, and distributes the acquired images to the information processing device 20 held by another user (observing user Ub). Note that the observing user Ub refers to a user who views the images captured by the imaging user Ua. There may be one or more observing users Ub.

The content server 90 may also temporarily or permanently store the captured images acquired from the information processing device 10 in, for example, the storage unit 95, and distribute the captured images stored in the storage unit 95 to the information processing device 20. With this configuration, it becomes possible to transmit captured images based on the imaging results of the imaging device 40 synchronously or asynchronously from the information processing device 10 held by the imaging user Ua to the information processing device 20 held by the observing user Ub.

The output device 30 is configured as a so-called head mounted display (HMD) equipped with a display unit 31 such as a display, and presents an image via the display unit 31 to the observation user Ub wearing the output device 30.

For example, FIG. 2A shows an example of an output device 30 applied in the information processing system 1. The output device 30 is configured to be worn on the user's head and to hold a display unit 31 (e.g., a display panel) for displaying an image in front of the user's eyes. Note that head-mounted display devices (HMDs) that can be used as the output device 30 include immersive HMDs, see-through HMDs, video see-through HMDs, and retinal projection HMDs.

When an immersive HMD is worn on the user's head or face, it is worn so as to cover the user's eyes, and a display unit such as a display is held in front of the user's eyes. Therefore, it is difficult for a user wearing an immersive HMD to directly view the outside scenery (i.e., the scenery of the real world), and only the image displayed on the display unit comes into view. With this configuration, the immersive HMD can give a sense of immersion to the user viewing the image. Note that the output device 30 shown in Figure 2A corresponds to an immersive HMD.

A see-through HMD holds a virtual image optical system consisting of a transparent light guide section in front of the user's eyes using, for example, a half mirror or a transparent light guide plate, and displays an image inside the virtual image optical system. Therefore, a user wearing a see-through HMD can view the outside scenery even while viewing an image displayed inside the virtual image optical system. One specific example of a see-through HMD is a so-called glasses-type wearable device in which the part equivalent to the lenses of glasses is configured as a virtual image optical system.

Like immersive HMDs, video see-through HMDs are worn over the user's eyes, with a display unit such as a display held in front of the user's eyes. On the other hand, video see-through HMDs have an imaging unit for capturing images of the surrounding scenery, and the image of the scenery captured by the imaging unit in the user's line of sight is displayed on the display unit. With this configuration, a user wearing a video see-through HMD can check the outside scenery from the image displayed on the display unit, although it is difficult for the user to directly view the outside scenery.

A retinal projection HMD has a projection unit held in front of the user's eyes, and the image is projected from the projection unit towards the user's eyes so that it is superimposed on the external scenery. More specifically, in a retinal projection HMD, an image is projected directly from the projection unit onto the retina of the user's eye, and the image is formed on the retina. This configuration allows users with myopia or hyperopia to view clearer images. Furthermore, a user wearing a retinal projection HMD can view the external scenery even while viewing an image projected from the projection unit.

So far, the HMD type has been described, but the output device 30 applied to the information processing system 1 may be a monitor display device as shown in FIG. 2B, or a display screen of a portable information processing device such as a smartphone as shown in FIG. 2C. Furthermore, the output device 30 may also be a display device composed of a projection device and a screen (not shown), or a large spherical display formed in a dome-shaped space that the observing user Ub views inside the dome.

The information processing device 20 on the observing user Ub's side acquires an image (e.g., a panoramic image) based on the imaging results by the imaging device 40 from the content server 90 via the network N11. The information processing device 20 then displays the acquired image on the display unit 31 of the output device 30. With this configuration, the observing user Ub wearing the output device 30 can view an image of the environment surrounding the imaging user Ua wearing the imaging device 40, for example, via the display unit 31 of the output device 30.

The information processing device 20 may also receive notification of the detection results of the head movement (head posture) of the observing user Ub from the output device 30. With this configuration, the information processing device 20 may recognize changes in the position and direction of the viewpoint of the observing user Ub based on the head movement of the observing user Ub, and present an image corresponding to the recognized change in viewpoint to the observing user Ub via the output device 30 (display unit 31).

For example, FIG. 3 shows an example of an operation for presenting an image according to a change in the viewpoint of the observing user Ub. The example shown in FIG. 3 shows an example of simulating a situation in which the observing user Ub views an image v0 that is expanded wider than the observing user Ub's field of view, such as a panoramic image, via an output device 30 (not shown) while moving the viewpoint as if looking around. Note that image v0 may be a moving image or a still image. Image v0 here corresponds to a captured image based on the imaging results by the imaging device 40.

The information processing device 20 recognizes changes in the viewpoint of the observing user Ub by detecting the movement of the head of the observing user Ub. The information processing device 20 then cuts out an image v11 of a portion of the image v0, which is expanded wider than the field of view of the observing user Ub, that corresponds to the position and direction of the recognized viewpoint, based on a pre-set field of view angle, and presents the cut-out image v11 to the observing user Ub via the output device 30. That is, the image v11 shown in FIG. 3 is a schematic representation of the image presented on the display unit 31 of the output device 30, and is the display image that is actually viewed by the observing user Ub.

The viewing angle for extracting image v11 may be fixed, or may be changeable based on user operation, etc. For example, when a portion of image v11 is to be further enlarged and presented to observing user Ub, information processing device 20 may control image v11 to be relatively smaller in size with respect to image v0 by setting the viewing angle narrower.

For such processing, the output device 30 may be provided with a detection unit (detection unit 33 in FIG. 4) such as an acceleration sensor or an angular velocity sensor (gyro sensor) and configured to detect the behavior of the observing user Ub, such as head movement (head posture). For example, if the output device 30 is an HMD as shown in FIG. 2A, the output device 30 may detect the yaw, pitch, and roll components of the user's head movement. The output device 30 notifies the information processing device 20 of the detection result of the head movement of the observing user Ub. Note that detection of head movement here means detection of a change in the line of sight of the observing user Ub.

It should be noted that the display image is not necessarily displaced in conjunction with the movement of the head of the observing user Ub.
For example, it is possible to displace the display image in conjunction with the direction of the arm of the observing user Ub depending on the type and model of the output device 30. For example, an acceleration sensor or an angular velocity sensor may be mounted on a device (such as the information processing device 20) held or worn by the observing user Ub to detect the behavior of the observing user Ub, such as the movement of the arm.
Furthermore, an acceleration sensor or an angular velocity sensor may be prepared as a sensing device separate from the output device 30 and the information processing device 20 and attached to the observing user Ub to detect the behavior of the observing user Ub.

For example, by being able to sense the behavior of the observing user Ub in this manner, it is possible to displace the forward direction of the displayed image in accordance with the behavior of the observing user Ub. However, it is not essential for the information processing system 1 to be provided with a configuration for detecting the behavior of the observing user Ub. For example, the output device 30 or the information processing device 20 may be provided with an operator or GUI (Graphical User Interface) that allows the observing user Ub to operate the viewing direction, and the displayed image may be displaced in accordance with that operation.

Furthermore, the method of detecting the behavior of the observing user Ub is not limited to a method based on the detection results of various sensors provided in the output device 30 or the information processing device 20. For example, a configuration for detecting the head and body movements of the observing user Ub may be provided outside the output device 30. As a specific example, an image of the observing user Ub may be captured by an imaging device, and the captured image may be analyzed to detect the head and body movements of the observing user Ub. Furthermore, various sensors such as so-called optical sensors may be provided outside the output device 30, and the head movements of the observing user Ub may be detected by sensing the observing user Ub with the sensors.

With the information processing system 1 configured as described above, the observing user Ub can see the view from the viewpoint of the imaging user Ua, or can see the view around the imaging user Ua as if he is looking around while moving his own viewpoint, allowing him to experience a very realistic image as if he were in the same place as the imaging user Ua. Therefore, for example, by presenting an image (panoramic image) based on the imaging results by the imaging device 40 as image v0, the observing user Ub can have the experience of sharing a space with the imaging user Ua who is wearing the imaging device 40.

In addition, at least a part of the configuration of the information processing device 20 may be provided in the output device 30. For example, the output device 30 may be provided with a configuration for extracting a partial image from the panoramic image according to the position and direction of the viewpoint of the observing user Ub and presenting the extracted partial image on the display unit 31. In addition, the information processing device 20 and the output device 30 may be configured integrally.

As mentioned above, the cut-out range of the display image within the captured image is displaced according to the behavior of the observing user Ub, but the captured image itself displaces in the forward direction according to the behavior of the imaging user Ua, for example, head movement. This is because the imaging direction of the imaging unit 41 changes. Therefore, even if the observing user Ub does not change his/her line of sight, i.e., the cut-out range does not change, the entire scene viewed by the observing user Ub as the display image will be displaced in the up/down/left/right directions due to the behavior of the imaging user Ua, for example, shaking his/her head up/down/left/right.

In the above example, a captured image (such as a panoramic image) based on the imaging results by the imaging device 40 is transmitted from the information processing device 10 to the information processing device 20 via the content server 90, but this is not necessarily limited to this configuration. For example, the image based on the imaging results by the imaging device 40 may be transmitted directly from the information processing device 10 to the information processing device 20 without going through the content server 90.

Next, the functional configuration of the information processing system 1 will be described. Figure 4 is a block diagram showing an example of the functional configuration of the information processing system 1.

The imaging device 40 includes an imaging section 41 (in the figure, multiple imaging sections 41a and 41b) and a detection section 43. Note that the example in which the imaging device 40 includes two imaging sections 41a and 41b is just an example, and the imaging device 40 may include one imaging section 41 or three or more imaging sections 41.

Each of the imaging units 41a and 41b captures an image of the surroundings of the imaging device 40 (i.e., an image of the surroundings of the imaging user Ua wearing the imaging device 40) and outputs the captured image to the information processing device 10.

The detection unit 43 is composed of, for example, an acceleration sensor and an angular velocity sensor (gyro sensor), detects changes in the position and attitude of the imaging device 40, and outputs the detection results to the information processing device 10.

The information processing device 10 includes a communication unit 101, a recognition unit 103, and an image synthesis unit 105. The communication unit 101 is a communication interface that enables each component in the information processing device 10 to communicate with an external device (e.g., the content server 90 or the information processing device 20) via the network N11.

The configuration of the communication unit 101 can be changed as appropriate depending on the type of network N11. For example, if the network N11 is a wireless network, the communication unit 101 may include a baseband processor, an RF circuit, and the like. In the following explanation, when each component in the information processing device 10 transmits and receives information to and from an external device via the network N11, it is assumed that the information is transmitted and received via the communication unit 101, even if no special explanation is given.

The recognition unit 103 acquires the detection results of changes in the position and posture of the imaging device 40 from the detection unit 43, and recognizes the head direction (in other words, the line of sight of the imaging user Ua) determined from the head position and posture of the imaging user Ua wearing the imaging device 40 based on the acquired detection results. The recognition unit 103 then outputs information indicating the recognition results of the head position and posture of the imaging user Ua to the image synthesis unit 105.

The image synthesis unit 105 acquires from each of the multiple imaging units 41 an image captured by that imaging unit 41. The image synthesis unit 105 synthesizes the images captured by each of the multiple imaging units 41 according to the imaging direction of each imaging unit 41 and the angle of view of that imaging unit 41 to generate an image (e.g., a panoramic image) that captures a wider range than the angle of view of each imaging unit 41. As the imaging unit 41 is attached to the head of the imaging user Ua, the front direction of the panoramic image generated by the image synthesis unit 105 changes according to changes in the posture of the head of the imaging user Ua.

Although the image synthesis unit 105 is described as generating a captured image as a panoramic image, the captured image generated by the image synthesis unit 105 is not necessarily limited to a panoramic image. The captured image is an image having a wider angle of view than the display image on the observing user Ub side, and may be, for example, an image of 180 degrees horizontally, or a hemispherical image or panoramic image in which the subject scene is displayed over 360 degrees in the front, back, left, and right directions.
Although the configuration includes the image synthesis unit 105, image synthesis processing is not necessary when capturing images using, for example, a single image capturing unit 41. In that sense, there are also examples in which the image synthesis unit 105 is not provided.

The image synthesis unit 105 also calculates, based on the information indicating the recognition result of the position and posture of the head of the imaging user Ua, information indicating the direction and magnitude of the acceleration applied to the head as acceleration information. Note that, at this time, the image synthesis unit 105 may calculate the acceleration information by excluding the translational motion component of the head movement of the imaging user Ua and taking into account only the rotational motion.
The acceleration information may be one-axis (for example, yaw direction) or two-axis (for example, yaw and pitch directions) information, or may be three-axis (yaw, pitch, and roll) information.

Then, the image synthesis unit 105 associates the captured image, such as the generated panoramic image, with the calculated acceleration information and transmits them to the content server 90 connected via the network N11.

There is no particular limitation on the timing at which the above-described imaging device 40 and information processing device 10 perform the process of acquiring an image (e.g., a panoramic image) of the surroundings of the imaging user Ua and the process of transmitting the image. As a specific example, the imaging device 40 and the information processing device 10 may capture an image of the surroundings of the imaging user Ua in real time, generate a panoramic image based on the captured image, and transmit the panoramic image to the content server 90.
As another example, the imaging device 40 and the information processing device 10 may record the captured image in a recording unit (not shown), and at a desired timing (for example, at the timing of receiving an operation from the imaging user Ua), read out the recorded captured image from the recording unit and transmit it from the communication unit 101. In this case, the information generating device 10 records the captured image in the storage unit in association with acceleration information indicating the recognition result of the position and posture of the head of the imaging user Ua, and transmits the acceleration information together with the captured image when transmitting.
In addition, the information processing device 10 may transmit the panoramic image to the content server 90 in synchronization with the generation of the panoramic image, or may associate the acceleration information with the generation of the panoramic image and record it in a recording unit not shown, and transmit the captured image and the acceleration information to the content server 90 at a later point in time.

The content server 90 includes a communication unit 901 and a content control unit 903. The content server 90 may also be configured to be able to access the storage unit 95.

The communication unit 901 is a communication interface that enables each component in the content server 90 to communicate with an external device (e.g., information processing devices 10 and 20) via the network N11. The communication unit 901 may have a configuration similar to that of the communication unit 101 described above. In the following explanation, when each component in the content server 90 transmits or receives information to or from an external device via the network N11, it is assumed that the information is transmitted or received via the communication unit 901, even if no special explanation is given.

The content control unit 903 acquires captured images, such as panoramic images, generated by the information processing device 10 connected via the network N11 and associated with acceleration information indicating the direction and magnitude of acceleration applied to the head of the imaging user Ua, that is, images of the surroundings of the imaging user Ua.

Furthermore, the content control unit 903 distributes the captured image associated with the acceleration information acquired from the information processing device 10 to the other information processing device 20 .
The content control unit 903 may also temporarily or permanently store the captured image acquired from the information processing device 10 in the storage unit 95 and distribute the captured image stored in the storage unit 95 to the information processing device 20.

The memory unit 95 is configured from a storage device and stores various data such as the captured images and acceleration information described above. The memory unit 95 may be included in the content server 90. With this configuration, the content control unit 903 can transmit captured images acquired from the information processing device 10 to another information processing device 20 synchronously or asynchronously.

The output device 30 includes a display unit 31, a detection unit 33, and a sensation induction unit 35. The display unit 31 corresponds to the display unit 31 described with reference to Fig. 1 and Fig. 2. The sensation induction unit 35 is configured to present a force sense to the observing user Ub wearing the output device 30.
The detection unit 33 is composed of, for example, an acceleration sensor or an angular velocity sensor (gyro sensor), detects changes in the position and attitude of the output device 30 , and outputs the detection result to the information processing device 20 .

The information processing device 20 includes a communication unit 201, an analysis unit 203, a display control unit 205, an input unit 206, and a force feedback control unit 207.

The communication unit 201 is a communication interface that enables each component in the information processing device 20 to communicate with an external device (e.g., the content server 90 or the information processing device 10) via the network N11. The communication unit 201 may have a configuration similar to the communication unit 101 or the communication unit 901 described above. In the following explanation, when each component in the information processing device 20 transmits or receives information to or from an external device via the network N11, it is assumed that the information is transmitted or received via the communication unit 201, even if no special explanation is given.

The analysis unit 203 obtains the captured image to be presented to the observing user Ub from the content server 90. The captured image is associated with acceleration information indicating the direction and magnitude of acceleration applied to the head of the imaging user Ua holding the information processing device 10 by the information processing device 10 that generated the captured image.

The analysis unit 203 also acquires detection results of changes in the position and posture of the output device 30 from the detection unit 33, and recognizes the position and posture of the head of the observing user Ub wearing the output device 30 (in other words, the position and direction of the viewpoint) based on the acquired detection results.
Then, the analysis unit 203 outputs the acquired captured image and information indicating the recognition results of the position and posture of the head of the observing user Ub to the display control unit 205.

The analysis unit 203 also recognizes the direction and magnitude of the acceleration applied to the head of the imaging user Ua based on the acceleration information associated with the captured image. By using the recognition result, the analysis unit 203 recognizes a relative change in the head direction (line of sight) of the observing user Ub (i.e., the user wearing the output device 30) with respect to the head direction (line of sight) of the imaging user Ua (i.e., the user wearing the imaging device 40). That is, the analysis unit 203 is able to calculate acceleration information indicating the direction and magnitude of the acceleration applied to the head of the observing user Ub when simulating the acceleration applied to the head of the imaging user Ua to the head of the observing user Ub. Then, the analysis unit 203 outputs the acceleration information indicating the direction and magnitude of the acceleration applied to the head of the observing user Ub to the force sense control unit 207.
In addition, the imaging user Ua wears the imaging device 40 on his/her head as described above, and the imaging unit 41 moves together with the head of the imaging user Ua. Therefore, the display control unit 205 regards the direction of the head of the imaging user Ua as the direction of the line of sight.
In the case where the imaging user Ua wears the imaging unit 41 on, for example, the shoulder instead of on the head, the direction of the imaging user Ua's body may be regarded as the direction of the line of sight.

The display control unit 205 acquires the captured image acquired from the analysis unit 203 and information indicating the recognition result of the position and posture of the head of the observing user Ub. Based on the information indicating the recognition result, the display control unit 205 cuts out and extracts an image corresponding to the head direction (line of sight) of the observing user Ub from the acquired captured image based on a preset viewing angle. Then, the display control unit 205 displays the cut-out image as a display image on the display unit 31. With this configuration, the information processing device 20 can present to the observing user Ub an image in which the surroundings of the imaging user Ua are captured, for example, a panoramic image, in a direction corresponding to the position and posture of the head of the observing user Ub.
Note that the display control unit 205 can regard the direction of the head of the observing user Ub as the direction of the line of sight, for example, but may also detect the line of sight of the observing user Ub itself.

The input unit 206 is an input device that allows various inputs by the observing user Ub. Various forms are assumed, such as an operator such as a key or switch, a touch panel, a trackball, a mouse, a joystick, or a pointer. Here, the input unit 206 is an input device that allows the observing user Ub to instruct the displacement of the displayed image. For example, it allows a position non-designated operation that moves the displayed image left and right without designating a position, and a position designation operation that designates a certain point in the panoramic image and displaces the displayed image toward that point.
Note that the above-mentioned head behavior or some other behavior of the observing user Ub is also one aspect of the position designation operation or the position non-designation operation. For example, the observing user moving his/her head can be considered as the position non-designation operation, and the observing user Ub's behavior of pointing in a certain direction can be considered as the position designation operation.

The display control unit 205 can also change the cut-out range from the captured image based on the operation information from the input unit 206 and display the cut-out image on the display unit 31. With this configuration, the information processing device 20 can provide the observing user Ub with an image in a direction according to the operation of the observing user Ub.

The force sense control unit 207 acquires acceleration information indicating the direction and magnitude of acceleration applied to the head of the observing user Ub from the analysis unit 203. The force sense control unit 207 then presents a force sense to the observing user Ub by driving the body sensation induction unit 35 according to the direction and magnitude of acceleration indicated by the acceleration information. With this configuration, the information processing device 20 can present a force sense to the observing user Ub in conjunction with the image presented to the observing user Ub via the display unit 31.

Note that there is no particular limitation on the timing at which the above-mentioned output device 30 and information processing device 20 execute the process of acquiring the captured image from the content server 90 and the process of presenting the captured image and presenting the force feedback. As a specific example, the output device 30 and information processing device 20 may execute the process of acquiring the captured image from the content server 90 and the process of presenting the captured image and presenting the force feedback synchronously or asynchronously.

Furthermore, as long as the information processing device 20 can recognize the direction and magnitude of the force sensation presented to the observing user Ub wearing the output device 30, in other words, the direction and magnitude of the acceleration applied to the head of the observing user Ub, the content of the information associated with the captured image transmitted from the information processing device 10 is not particularly limited.

As a specific example, the information processing device 10 may associate information indicating the recognition result of the position and posture of the head of the imaging user Ua with the imaging image to be transmitted. In this case, for example, the information processing device 20 may calculate the direction and magnitude of the acceleration applied to the head of the imaging user Ua based on the information acquired from the information processing device 10.
As another example, the information processing device 20 may indirectly recognize a change in the position or posture of the imaging device 40 that is the image capture source of the omnidirectional image (and thus a change in the position or posture of the head of the imaging user Ua) by performing image analysis on the captured image. In this case, only the captured image may be transmitted to the information processing device 20.

It is also assumed that the way the presented force sensation is felt may differ depending on the user. Therefore, the information processing device 20 may recognize the observing user Ub wearing the output device 30, and change the magnitude of the presented force sensation, i.e., the magnitude of acceleration, depending on the recognized observing user Ub.

An example of the functional configuration of the information processing system 1 has been described above with reference to FIG.
The functional configuration of the information processing system 1 is not necessarily limited to the example shown in Fig. 4. For example, a part of the configuration of the information processing device 10 may be provided on the imaging device 40 side, or the imaging device 40 and the information processing device 10 may be configured integrally. Similarly, a part of the configuration of the information processing device 20 may be provided on the output device 30, or the output device 30 and the information processing device 20 may be configured integrally. In addition, the information processing device 10 may transmit the generated panoramic image directly to the information processing device 20 without going through the content server 90.

Furthermore, the display control unit 205 that cuts out the display image is provided on the information processing device 20 side, but a configuration example in which the display control unit 205 is provided on the content server 90 or the information processing device 10 is also possible.

2. Configuration of information processing device
Next, a configuration example of an information processing device 70 that can be used as the information processing device 20, the information processing device 10, the content server 90, etc. will be described with reference to FIG.
The information processing device 70 is a device capable of information processing, particularly image processing, such as a computer device. Specific examples of the information processing device 70 include personal computers, workstations, mobile terminal devices such as smartphones and tablets, video editing devices, etc. The information processing device 70 may also be a computer device configured as a server device or a computing device in cloud computing.

The CPU (Central Processing Unit) 71 of the information processing device 70 shown in FIG. 5 executes various processes according to programs stored in a ROM (Read Only Memory) 72 or a non-volatile memory unit 74 such as an EEP-ROM (Electrically Erasable Programmable Read-Only Memory), or programs loaded from a storage unit 79 to a RAM (Random Access Memory) 73. The RAM 73 also stores data necessary for the CPU 71 to execute various processes as appropriate.

The image processing unit 85 is configured as a processor that performs various types of image processing, such as image synthesis, spherical image generation, image clipping, image analysis, image signal processing including color/brightness adjustment and color conversion, and image editing, or a combination of these processes.
For example, when this information processing device 70 is used as the information processing device 20 on the observing user Ub side, the image processing unit 85 is a processor that executes the processing of the display control unit 205 and the analysis unit 203 .
Furthermore, for example, when the information processing device 70 is used as the information processing device 10 on the imaging user Ua side, the image processing unit 85 is a processor that executes the processing of the image synthesis unit 105 .

This image processing unit 85 can be realized, for example, by a CPU separate from the CPU 71, a graphics processing unit (GPU), a general-purpose computing on graphics processing units (GPGPU), an artificial intelligence (AI) processor, or the like.
The image processing unit 85 may be provided as a function within the CPU 71 .

The CPU 71, ROM 72, RAM 73, non-volatile memory unit 74, and image processing unit 85 are interconnected via a bus 83. The input/output interface 75 is also connected to this bus 83.

An input unit 76 consisting of operators and operation devices is connected to the input/output interface 75. For example, the input unit 76 may be various operators and operation devices such as a keyboard, a mouse, a key, a trackball, a dial, a touch panel, a touch pad, a remote controller, or the like.
An operation by the user is detected by the input unit 76 , and a signal corresponding to the input operation is interpreted by the CPU 71 .
A microphone may also be used as the input unit 76. Voice uttered by the user may also be input as operation information.
For example, when this information processing device 70 is used as the information processing device 20 on the observing user Ub side, the input unit 76 corresponds to the input unit 206 in FIG.

Further, the input/output interface 75 is connected, either integrally or separately, to a display unit 77 formed of an LCD (Liquid Crystal Display) or an organic EL (electro-luminescence) panel, or the like, and an audio output unit 78 formed of a speaker, or the like.
The display unit 77 is a display unit that performs various displays, and is configured, for example, by a display device provided in the housing of the information processing device 70, or a separate display device connected to the information processing device 70, or the like.
The display unit 77 displays various images, operation menus, icons, messages, etc., that is, GUI, on the display screen based on instructions from the CPU 71 .
The display unit 77 may be used as the output device 30 in FIGS.

The input/output interface 75 may also be connected to a memory unit 79 and a communication unit 80, which may be configured using a hard disk drive (HDD) or solid state drive (SSD).

The storage unit 79 can store various data and programs. A database can also be configured in the storage unit 79.

The communication unit 80 performs communication processing via a transmission path such as the Internet, and communication with various devices such as an external DB, an editing device, and an information processing device via wired/wireless communication, bus communication, etc. This communication unit 80 can function as the communication unit 201, communication unit 101, or communication unit 901 described in FIG. 4.

A drive 81 is also connected to the input/output interface 75 as required, and a removable recording medium 82 such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory is appropriately mounted thereon.
The drive 81 allows video data, various computer programs, and the like to be read from the removable recording medium 82. The read data is stored in the storage unit 79, and the video and audio contained in the data are output on the display unit 77 and the audio output unit 78. In addition, the computer programs, etc. read from the removable recording medium 82 are installed in the storage unit 79 as necessary.

In this information processing device 70, for example, software for the processing of this embodiment can be installed via network communication by the communication unit 80 or via a removable recording medium 82. Alternatively, the software may be stored in advance in the ROM 72, the storage unit 79, etc.

<3. Processing example>
Hereinafter, a processing example of the information processing device 20 according to the embodiment, in particular, a processing example of the display control unit 205 will be described.
When the display control unit 205 is provided in the content server 90, the following processing example is executed in the content server 90. When the display control unit 205 is provided in the information processing device 10, the following processing example is executed in the information processing device 10.

FIG. 6 shows an example of processing by the display control unit 205 that controls the displacement of the display image viewed by the observing user Ub, that is, the image cut out from a captured image such as a panoramic image. In particular, this processing example is an example of processing that slows down the displacement speed of the displayed image so that the observing user Ub does not experience visual sickness due to displacement of the cut-out range. However, rather than simply slowing down the displacement speed of the displayed image, the displacement speed is varied according to situation information so as not to impair real-time performance as much as possible. For example, in a situation where visual sickness is likely to occur, the displacement of the displayed image is executed slowly, but in a situation where visual sickness is unlikely to occur, the delay in the displayed image is suppressed.

In step S101, the display control unit 205 sets the speed of displacement in the received captured image as a speed v. For example, when a displacement in the front direction of the captured image occurs due to a movement of the head of the imaging user Ua wearing the imaging unit 41, the speed of the displacement is set as the speed v. When there is no movement of the head of the imaging user Ua, the speed v=0.
Note that velocity v also includes a component in the direction of displacement. When considering displacement only in the horizontal left-right direction, the value of velocity v is set with one side of the left and one side of the left as a + value and the other as a - value. When considering the left-right direction (x-axis) and the up-down direction (y-axis), velocity v for each axis can be set by setting one side of the left and one side of the left as a + value and the other as a - value in the x-axis direction, and one side of the up-down direction as a + value and the other as a - value in the y-axis direction.
The display control unit 205 may obtain the velocity v, for example, from acceleration information associated with the captured image, or may obtain the velocity v by calculating the amount of displacement of the same subject by matching between frames of the captured image.

In step S102, the display control unit 205 branches the process depending on whether or not there has been an input of a displacement operation by the observing user Ub. In this case, the input refers to an input instructing the displacement of the cropping range of the displayed image, for example, detection of the movement of the head of the observing user Ub. Alternatively, the input can also refer to an operation by the input unit 206 instructing the displacement of the displayed image.

If the observing user Ub does not particularly move his/her head to change the direction of the line of sight or perform a position non-designation operation or a position designation operation by dragging an operating device such as a mouse, the display control unit 205 proceeds to step S103 and sets the upper speed limit vmax to a speed vmax1. Note that while the above-mentioned velocity v is a vector, the upper speed limit vmax here is a scalar and takes a positive value.
The speed vmax1 assigned as the upper speed limit value vmax is a value set as the upper limit of the speed at which image sickness is unlikely to occur even if the displayed image is displaced regardless of the will of the observing user Ub, and is a value that is slower than the speed vmax2 described later.

In step S104, the display control unit 205 calculates the cut-out range of each of the subsequent frames according to the speed v, with the speed upper limit vmax set to the speed vmax1. That is, the display control unit 205 calculates the cut-out range of each of the subsequent frames, taking into consideration the amount of displacement of the image per frame period due to the speed v and the direction of the displacement.
Here, when looking at the amount of displacement per frame period, if the speed v is equal to or less than the speed vmax1, the cut-out range of each frame is set so that a displacement amount equivalent to the speed v occurs per frame. Therefore, if the imaging user Ua slowly turns his head to the left at a speed less than the speed vmax1, the front of the display image seen by the observing user Ub is displaced to the left in the same manner as the imaging user Ua changes his line of sight.
In addition, if the captured image is an image that directly reflects the displacement in the imaging direction due to the behavior of the imaging user Ua, if the cut-out range is fixed in each frame, a displacement equivalent to the velocity v will also occur in the displayed image viewed by the observing user Ub.
On the other hand, when the information processing device 10 generates an image so as to suppress rotation of the image due to a change in the line of sight caused by the rotation of the imaging user Ua's head, etc., the cut-out range is set to shift for each frame depending on information on the speed and direction of the movement of the imaging user Ua, so that a displacement equivalent to a speed v occurs in the displayed image viewed by the observing user Ub.

When it is determined in step S104 that the speed v exceeds the speed vmax1, that is, when the imaging user Ua turns his head to the left at a speed exceeding the speed vmax1, the front of the display image seen by the observing user Ub is displaced to the left at a speed vmax1 slower than the line of sight change speed (speed v) of the imaging user Ua, that is, in a delayed state. That is, in step S104, the display control unit 205 causes the frames to progress at a speed vmax1 or less during the frame period until the display image catches up with the amount of displacement (change in scenery) at the speed v due to the behavior of the imaging user Ua.
Therefore, the cut-out range according to the speed v is set in a state where the speed component is limited at the speed vmax1.

This state will be explained typically with reference to Figures 7, 8 and 9. In each figure, an arrow 51 indicates the front direction of the imaging user Ua, that is, the front direction of an imaged image such as a panoramic image.
It is assumed that the imaging user Ua is capturing images at a live music venue, and that in front of the imaging user Ua are lined up, from the left, as subjects 50: a piano player, a guitar player, a drummer, and a saxophone player.

At time t=t0 in Figure 7, the guitar player is directly in front of the line of sight of the imaging user Ua. In this case, the position of the guitar player on the front side on the projection surface 52 of the panoramic image, which is the captured image, becomes the center of cropping for the displayed image. In the figure, normal cropped image 54 shows a state in which delay processing to prevent motion sickness is not performed. In normal cropped image 54, the guitar player is shown in the center of the displayed image.

On the other hand, the projection plane 56 and speed-variable cut-out image 58 show the case where delay processing is performed. Since there is no displacement of the captured image at time t=t1, the position of the guitar player on the projection plane 56 becomes the cut-out center for the displayed image. Also, the speed-variable cut-out image 58 is the same as the normal cut-out image 54.

At time t=t1 in FIG. 8, it is assumed that the head of the image capturing user Ua moves quickly and the line of sight is directed toward the saxophone player.
If the cut-out center 53 is found at the speed v of that movement, the position of the saxophonist in the foreground on the projection plane 52 becomes the cut-out center 53, and the saxophonist is in the center of the image in the normal cut-out image 54. In other words, the scene of the displayed image seen by the observing user Ub appears to flow quickly sideways, which can easily cause motion sickness.

In such a case, the speed of displacement of the display image is limited to a speed vmax1. For example, even if a saxophone player is positioned in front of the projection surface 56, the cut-out center 57 is set to be somewhere between the guitar player and the drummer. As a result, when the speed-variable cut-out image 58 is used as the display image, the center will be located between the guitar player and the drummer. In other words, the movement of the display image from time t0 to time t1 is slower than the movement in the line of sight of the imaging user Ua.

At time t=t2 in FIG. 9, it is assumed that the imaging user Ua is still looking at the saxophone player. Because the displacement speed of the displayed image is limited to speed vmax1, even if the saxophone player is positioned in front, as in the projection surface 56, the cut-out center 57 is set to be around the drummer. As a result, when the speed-variable cut-out image 58 is used as the displayed image, the drummer will be positioned in the center.

Then, at time points t3 and t4, the displacement speed of the displayed image is similarly limited to speed vmax1, so that the displacement of the displayed image becomes slower. For example, as shown in FIG. 10, at time point t4, the saxophone player is in front in the variable speed cut-out image 58 (=display image).

In this way, the captured image has its front direction shifted from the guitar player to the saxophone player at times t1 (FIG. 7) and t2 (FIG. 8), whereas the displayed image has its front direction shifted from the guitar player to the saxophone player over the course of from time t1 to time t4 (FIG. 9).
That is, even if the imaging user Ua moves his/her head quickly, the entire scene of the display image visually recognized by the observing user Ub changes slowly. In particular, when proceeding to steps S103 and S104, the observing user Ub has not performed an input (position designation operation or position non-designation operation) to instruct the change, and the change in the display image is an unexpected change. For this reason, fast movement of the display image is likely to induce motion sickness. Therefore, the change speed of the display image is limited to a low value.

If there is an input from the observing user Ub in step S102 of FIG. 6, the display control unit 205 proceeds to step S110 and first sets the upper speed limit vmax = 0. This is processing that assumes that even if there is movement in the captured image and the speed v is set in step S101 according to that movement, the displacement speed of the displayed image according to the displacement of the captured image is "0". In other words, this processing is for assuming that there is no movement in the captured image.

Then, in step S111, the display control unit 205 calculates the cutout reference position using the velocity v set in step S101. This means that a cutout range is set so that the display image seen by the observing user Ub does not displace even if the front direction is displaced by the imaging user Ua, and this is set as the reference position for the displacement due to the input of the observing user Ub. The position of the displacement equivalent to the velocity (-v) for the displacement of the velocity v (i.e., the position before the displacement due to the movement of the imaging user Ua) becomes the cutout reference position.
For example, even if the head (gaze direction) of the imaging user Ua faces left, if the cut-out range is moved to the right by the same amount of displacement, the display image seen by the observing user Ub will not be displaced. The center of the cut-out range that prevents such a display image from being displaced due to the influence of the imaging user Ua becomes the cut-out reference position.

In step S112, the display control unit 205 branches the process depending on whether the user input was a position designation operation. A position designation operation here refers to an operation in which the observing user Ub designates a certain cut-out range (the center position of the cut-out range). For example, an operation in which a specific position or subject is designated on the projection surface, or an operation in which an amount of displacement is designated such as "rotate left 90 degrees", is a position designation operation referred to here. In other words, it is an operation in which the destination of displacement is designated. On the other hand, an operation in which continuous displacement is designated, such as a drag operation or a scroll operation, is not a position designation operation as referred to here, but a non-position designation operation.

If the operation is a position designation operation, the display control unit 205 sets a speed v of the displacement of the display image based on the difference between the designated position and the current position in step S113. That is, the display control unit 205 changes the speed v set in step S101 to a speed based on the difference between the designated position and the current position. The speed v is set as a vector from the current position to the designated position as a target.
Furthermore, in step S114, the display control unit 205 sets the upper speed limit vmax to the speed vmax2. As described above, vmax2>vmax1.
Then, in step S117, the display control unit 205 calculates the cropping range for each subsequent frame such that the display image is displaced toward the specified position according to the speed v, with the upper speed limit vmax set to speed vmax2.

If the speed v is equal to or less than speed vmax2, the display control unit 205 sets the cutout range of each frame so that a displacement amount equivalent to the speed v occurs for each frame. For example, if the distance between the position specified by the position specification operation and the current position is short and the transition can be made at a speed equal to or less than vmax2, the cutout range of each frame is set so that the display image transitions at that speed v.
On the other hand, when the speed v exceeds the speed vmax2, the display control unit 205 sets the cutout range of each frame so that the display image transitions at the speed vmax2.
Therefore, the cut-out range according to the speed v is set in a state where the speed component is limited at the speed vmax2.

Here, vmax2>vmax1 means that the displayed image is displaced at that speed, even if it is faster than when there is no user input. This is because the displacement of the displayed image occurs due to the observation user Ub's own operation, so there is little risk of visual sickness occurring.

This will be explained with reference to Figures 7, 11, and 12.
Suppose that the line of sight of the imaging user Ua changes from the direction of the guitar player to the direction of the saxophone player at times t1 and t2 in Figures 11 and 12 from the state at time t=t0 in Figure 7 described above. However, if the observing user Ub is performing a position designation operation, this change in line of sight is not reflected in the displayed image.
However, suppose that the observing user Ub specifies the position of the saxophone player. In response to this operation, the speed-variable cut-out image 58 is displaced at a speed v that does not exceed the speed vmax2. For example, at time t1, the drummer is displaced to the center of the displayed image as shown in FIG. 11, and at time t2, the saxophone player is displaced to the center of the displayed image as shown in FIG. 12.

In the examples of Figures 7, 8, 9, and 10, the displayed image catches up to a state where the saxophone player is facing forward at time t4, but if a position specification operation is performed, the saxophone player will face forward at an earlier time, time t2.

In the above, we have shown a case where the displacement of the imaging user Ua's line of sight and the displacement of the observing user Ub's line of sight toward the designated position are in the same direction, but there are also cases where the designated position of the observing user Ub moves with a displacement in the opposite direction to the displacement of the imaging user Ua's line of sight. Even in such cases, the displacement is always toward the designated position.

This will be explained with reference to Figures 7, 13 and 14.
7, the forward direction of the line of sight of the imaging user Ua is changed from the direction of the guitar player to the direction of the saxophone player at times t1 and t2 in Figures 13 and 14. However, suppose that the observing user Ub performs a position designation operation and designates the position of the piano player.
Then, based on the operation, the speed-variable cut-out image 58 is displaced at a speed v that does not exceed the speed vmax2. For example, at time t1, the image is displaced until the midpoint between the guitar player and the piano player is at the center of the displayed image, as shown in Figure 13, and at time t2, the image is displaced until the piano player is at the center of the displayed image, as shown in Figure 14.

That is, in steps S114 and S117 in FIG. 6, the upper speed limit vmax is set to a relatively high speed vmax2, but the displayed image is displaced in a direction and by an amount of displacement that strictly conforms to the operation of the observing user Ub.
This is because, since the observing user Ub is aware of the displacement, there is little possibility that visual sickness will occur even if the delay amount of the displacement speed is reduced, and also because the operation of the observing user Ub can be responded to more responsively.

If the input by the observing user Ub is not a position designation operation, for example, if it is a non-position designation operation that is a displacement operation with an indefinite target position, such as a drag operation or a scroll operation, the display control unit 205 proceeds from step S112 to step S115, and replaces the velocity v set in step S101 with a velocity v according to the operation amount of the drag operation, etc. The replaced velocity v is set as a vector including components of the operation direction and operation speed of the drag operation.
Furthermore, the display control unit 205 sets the upper speed limit vmax=|v| in step S116. This means that the velocity component, which is the absolute value of the velocity v, has the upper limit set to the displacement velocity due to the position non-designation operation, which means that there is essentially no upper limit, and the displacement velocity due to an operation such as a drag operation is used as is to set the cut-out range.

Then, in step S117, the display control unit 205 calculates the cropping range for each subsequent frame so that the display image is displaced in the direction specified by the operation with the upper speed limit vmax = |v|, that is, at a displacement speed according to the operation.

In this case, too, the forward displacement of the captured image due to the behavior of the capturing user Ua (non-position designation operation on the capturing user Ua's side) is not reflected in the displayed image, and the displayed image is displaced only in the direction of the operation amount, such as a drag operation. And since the observing user Ub specifies the speed and direction himself, and the possibility of image-induced sickness is extremely low, the image is displaced according to the amount of operation. This makes it possible to realize a display with good responsiveness to operations.

The above processing in Figure 6 has described an example of processing when the behavior of the imaging user Ua causes a displacement of the captured image in the forward direction. In that case, the displacement speed of the displayed image will differ depending on situational information such as whether or not there is an input from the observing user Ub and whether or not the input content is a position designation operation.
This means that in situations where motion sickness is likely to occur, the displacement speed is slowed down, and in situations where motion sickness is unlikely to occur, the amount of reduction in the displacement speed is suppressed, thereby achieving a display that is as real-time and responsive to operations as possible.

In FIG. 6, the speed of the displacement of the displayed image is variable depending on the situation, whether the displacement is due to the behavior of the imaging user Ua or due to input by the observing user Ub, but it may also be changed depending on other situation information.

For example, assuming that the processing in Figure 6 is performed, the settings of speed vmax2 and speed vmax1 can be changed in the processing in Figures 15 to 19 below.

FIG. 15 shows the setting changes according to the output device 30.
In step S201, the display control unit 205 checks whether or not information about the output device 30 has been acquired.
If the speed vmax2 and/or the speed vmax1 are acquired, the process proceeds to step S202, where the settings of the speed vmax2 and/or the speed vmax1 are changed.

2, the output device 30 may be an HMD type, a large monitor display, a relatively small display provided on a smartphone, etc. There are cases where information on the type of these devices can be acquired.
Also, there are cases where it is possible to obtain information on the current display mode, such as whether the displayed image is being displayed as a window occupying part of the screen or being displayed on the entire screen.

Change the speed settings with information about these output devices.
For example, when a small-sized monitor device or a smartphone display is used as the output device 30, or when an image is displayed smaller than a predetermined size in a window within the screen, motion sickness is less likely to occur, so the values of the speed vmax2 and the speed vmax1 are increased. In other words, the values are updated to reduce the delay.

On the other hand, when a large monitor device or smartphone display is used as the output device 30, or when a window larger than a certain size is displayed, motion sickness is likely to occur, so the values of speed vmax2 and speed vmax1 are lowered. In other words, the tendency for a delay in displacement to prevent motion sickness to occur is increased.

FIG. 16 shows the change of settings according to user information.
In step S210, the display control unit 205 checks whether or not user information of the observing user Ub has been input.
If the predetermined user information has been input, the process proceeds to step S211, where the settings of one or both of the speeds vmax2 and vmax1 are changed.

For example, the observer user Ub inputs whether or not he or she is prone to visual motion sickness by operating the input unit 206. The display control unit 205 changes the speed setting based on this input information.
For example, if the observing user Ub is not prone to motion sickness, the values of the speeds vmax2 and vmax1 are increased. In other words, the values are updated to reduce the delay.
On the other hand, if the observing user Ub is prone to motion sickness, the values of the speeds vmax2 and vmax1 are reduced. In other words, emphasis is placed on preventing motion sickness even if the real-time performance is suppressed.

FIG. 17 shows how settings are changed depending on image contents.
The display control unit 205 analyzes the captured image in step S221, and determines whether or not to change the settings based on the analysis result in step S222. If it is determined that the settings are to be changed, the display control unit 205 proceeds to step S223, where it changes one or both of the settings of the speed vmax2 and the speed vmax1.

For example, by analyzing the captured image, it is possible to determine whether the image is of a dark scene, a distant view, has a small dynamic range, has few high frequency components, etc. These are image contents that are unlikely to cause motion sickness. Therefore, if the captured image falls into any of these categories, the values of speed vmax2 and speed vmax1 are increased.

On the other hand, images of bright scenes, images of close-up subjects, images containing meaningful text information (text that the observing user Ub wants to read), images with a large dynamic range, images with a lot of high-frequency components, etc. are likely to cause motion sickness. Therefore, if the captured image falls into any of these categories, the values of speed vmax2 and speed vmax1 are reduced.

FIG. 18 shows how settings are changed depending on the viewing environment.
In step S231, the display control unit 205 determines whether or not information on the viewing environment has been acquired. If information on the viewing environment has been acquired, in step S232, the display control unit 205 compares the acquired information with the previously acquired information on the viewing environment to check whether or not there has been a change in the environment.
If it is determined that there is an environmental change, the display control unit 205 proceeds to step S223, and changes the settings of one or both of the speed vmax2 and the speed vmax1.

For example, when viewing a monitor display in a bright room, motion sickness is unlikely to occur, but even in a dark room, viewing a bright image on a monitor display can easily cause motion sickness. Therefore, if the viewing environment is bright, the values of speed vmax2 and speed vmax1 should be increased. Also, if the viewing environment is dark, the values of speed vmax2 and speed vmax1 should be decreased.

In addition, if the viewing environment is not simply dark, but is also dark and the image content of the displayed image is relatively bright, the values of speed vmax2 and speed vmax1 may be reduced.

FIG. 19 shows a setting change according to an output other than visual.
In step S241, the display control unit 205 determines whether there is any output other than visual output, such as audio, vibration, or force output to the observing user Ub.

If there is any output other than the visual sense, the display control unit 205 judges the information related to the sound, vibration, force sense, etc. in step S242. For example, it judges the volume, vibration level, force sense level, etc. Then, depending on the judgment result, the display control unit 205 changes the setting of one or both of the speed vmax2 and the speed vmax1 in step S243.

For example, if the volume is high, or if the observing user Ub is given a sense of force or vibration at the same time as the displacement, then motion sickness may be less likely to occur. Therefore, it is determined whether output other than vision has an effect on suppressing motion sickness, and the values of speed vmax2 and speed vmax1 are increased based on the result of the determination.

As in the above examples, by taking into account various situational information and variably controlling the displacement speed of the displayed image, it is possible to increase the delay as necessary, while prioritizing real-time display in situations where visual sickness is unlikely to occur.

A modified example of the above-mentioned FIG. 6 is explained in FIG. 20. The same processes as those in FIG. 6 are given the same step numbers to avoid duplicate explanations. FIG. 20 shows a more specific example of the calculation of the cutout range.

In step S101, the display control unit 205 sets the speed of displacement in the captured image to speed v, and in step S151 calculates the cut-out reference position P0. As with step S111 in FIG. 6, this sets a cut-out range that prevents displacement of the displayed image seen by the observing user Ub even if the forward displacement caused by the capturing user Ua occurs, and sets this as the reference position for displacement. Specifically, the cut-out reference position P0 is set to a position equivalent to -v. In other words, this is set to the position where the forward displacement caused by the capturing user Ua is cancelled.

In step S102, the display control unit 205 branches the process depending on whether or not a displacement operation has been input by the observing user Ub. If no displacement operation has been input, a speed v' is set in step S152. In this case, the speed v' is set to a speed equivalent to the difference between the gaze position of the imaging user Ua and the current position. The gaze position of the imaging user Ua is the position of the center point in the front direction of the captured image, and the current position is the gaze position before the imaging user Ua moves.
The display control unit 205 then proceeds to step S103, sets the upper speed limit vmax to the speed vmax1, and proceeds to step S155.

If it is determined in step S102 that there is an input from the observing user Ub, the display control unit 205 branches the process in step S112 depending on whether the input is a position designation operation or a position non-designation operation.

In the case of a position specification operation, in step S113A, the display control unit 205 sets the speed v' of the displacement of the displayed image based on the difference between the specified position and the current position. Then, in step S114, the display control unit 205 sets the upper speed limit vmax to speed vmax2, and proceeds to step S155.

In the case of a non-position-specified operation, in step S115A, the display control unit 205 sets a speed v' according to the amount of operation such as a drag operation. Then, in step S117, the display control unit 205 sets the upper speed limit vmax = |v| and proceeds to step S155.

In step S155, the display control unit 205 calculates the cut-out range using the cut-out reference position P0, the speed v' set in either step S152, step S113A, or step S115A, and the upper speed limit vmax set in either step S103, step S114, or step S116.

Specifically, it is set as P0+min(max(v', -vmax), vmax), where vmax is a positive value as the upper speed limit, and -vmax is a negative value as the upper speed limit.
That is, the larger of the speeds v' and -vmax is selected, and the displacement amount equivalent to the smaller of the selected speed and the upper speed limit vmax is added to the cut-out reference position P0. Then, the cut-out range is determined with the position displaced by the calculated speed as the center.

As a result, when steps S152 and S103 are performed, an extraction range is obtained that has a displacement amount due to the smaller of the velocity v' due to the movement of the image capturing user Ua and the upper velocity limit vmax (=vmax1).
Also, if steps S113A and S114 have been performed, an extraction range is obtained in which the displacement amount is the smaller of the speed v' corresponding to the displacement amount from the current position to the specified position of the observing user Ub and the upper speed limit vmax (=vmax2).
Furthermore, if steps S115A and S116 have been carried out, an extraction range is obtained that is a displacement amount due to the velocity v' corresponding to the operation amount of the non-designated position of the observing user Ub.

4. Summary and Modifications
According to the above embodiment, the following effects can be obtained.

The information processing device 20 of the embodiment performs a cut-out process to cut out a range with a narrower angle of view than the angle of view of the captured image from an captured image (first image) captured by an imaging device worn by the imaging user Ua, and to create a display image (second image) for observation by the observing user Ub, and is equipped with a display control unit 205 that varies the displacement speed of the display image due to a change in the cut-out range when the captured image is displaced in the forward direction during the cut-out process, in accordance with situation information.
As a result, when the captured image is displaced in the front direction due to the movement (non-position designation operation) of the image capturing user Ua, the movement of the display image (speed-variable cut-out image 58) viewed by the observing user Ub can be changed in displacement speed depending on the situation. Therefore, while reducing the displacement speed of the display image so as not to cause image-induced sickness, in a situation where image-induced sickness is unlikely to occur, it is possible to suppress the decrease in the displacement speed and prevent a large time delay. In other words, while preventing image-induced sickness, it is possible to reduce the delay of the display image relative to the displacement of the captured image as much as possible in a situation where image-induced sickness is unlikely to occur, and to prevent a loss of real-time performance.

In the embodiment, an example has been given in which the captured image is an image captured by the imaging device 40 worn by the imaging user Ua, and the displacement of the captured image in the forward direction occurs due to a position non-designation operation by the imaging user Ua.
This system cuts out a display image from an image captured by an imaging device 40 worn by an imaging user Ua, for example, on the head. In this case, the displacement speed of the display image is reduced so as not to cause motion sickness, and the displacement speed is varied according to the situation, so that the imaging user Ua does not need to be conscious of preventing motion sickness in the observing user Ub. This allows the imaging user Ua to capture images freely, and the system can provide the observing user Ub with images that are unconsciously captured to better reflect the local situation.

In the embodiment, the display control unit 205 variably sets the displacement speed of the displayed image within a range equal to or less than the displacement speed of the captured image, regardless of whether or not the captured image is displaced in the forward direction (see Figures 6 and 20).
Making the displacement of the displayed image slower than the displacement of the actual captured image is effective in preventing image-induced motion sickness, but depending on the situation, the delay in the displacement of the displayed image relative to the captured image may be reduced or eliminated. By varying the displacement speed within a range equal to or less than the displacement speed of the captured image, it is preferable to prevent image-induced motion sickness while maintaining real-time performance as much as possible.
The processes in FIGS. 6 and 20 can also be applied when there is a displacement in the front direction of the captured image.

In the embodiment, the situation information includes the presence or absence of a displacement operation on the display image of the observing user Ub. Then, an example has been given in which the display control unit 205 makes the displacement speed of the display image when the displacement operation of the observing user Ub has been performed faster than the displacement speed of the display image when the displacement operation has not been performed.
When an operation is performed by the observing user Ub, the observing user Ub spontaneously displaces the displayed image, so that the observing user is unlikely to get motion sickness even if there is movement in the display. Therefore, when an operation is performed by the observing user Ub, the degree of delay in displacing the displayed image is made smaller than when there is no operation, thereby preventing unnecessary delays from occurring.

In the embodiment, when the observing user Ub performs a displacement operation, the display control unit 205 assumes that there is no displacement in the forward direction of the captured image and sets the cut-out range in accordance with the displacement operation of the observing user Ub (see steps S110 to S117 in FIG. 6).
When an operation is performed by the observing user Ub, even if there is a displacement of the captured image (displacement in the front direction) due to the behavior of the imaging user Ua, the displacement is canceled and the cropping range is set according to the operation of the observing user Ub. This allows the observing user Ub to view the display image that displaces according to his/her own operation, regardless of the behavior of the imaging user Ua.

In the embodiment, the display control unit 205 sets the displacement speed of the display image in the absence of a displacement operation by the observing user Ub to a first upper speed limit value (speed vmax1) or less. Also, in the case where a user viewing the display image performs an operation to specify a cutout range as a displacement operation, an example has been given in which the displacement speed according to the displacement distance of the display image is set to a second upper speed limit value (vmax2) or less, which is a speed faster than the first upper speed limit value (see steps S113, S114, and S117 in FIG. 6).
When the observing user Ub performs an operation to directly or indirectly specify the cut-out range, the speed is determined according to the difference in distance from the position of the current displayed image to the position of the cut-out region, and the speed is set to be equal to or less than speed vmax2. Speed vmax2 is a speed value faster than speed vmax1. In other words, since the displacement of the displayed image is caused by a voluntary operation of the observing user Ub, the degree to which the displacement of the displayed image is delayed is reduced, and the delay is reduced.
The operation of specifying the cut-out range may be, for example, an operation in which the observing user Ub specifies a point on an image or a specific subject, etc., and the display control unit 205 may set a predetermined range around the point, etc. as the cut-out range.

In the embodiment, an example was given in which, when the observing user Ub performs a non-position designation operation as a displacement operation, the display control unit 205 sets the displacement speed of the displayed image to a speed corresponding to the non-position designation operation (see steps S115, S116, and S117 in Figure 6).
When the observing user Ub performs a position-unspecified operation to move the display image by an operation such as dragging, swiping, scrolling, etc., the cropping range is set so that the display image is displaced at a speed and in a direction according to the position-unspecified operation, even if there is a displacement in the front direction of the captured image. This allows the observing user Ub to displace the display image as desired by a dragging operation, etc., regardless of the behavior of the capturing user Ua.

In the embodiment, an example has been given in which, when a displacement operation is performed to instruct a displacement in a direction different from the displacement direction of the captured image, the display control unit 205 sets the cropping range so that the image is displaced in a direction corresponding to the displacement operation. In other words, when the direction in which the observing user Ub performs a displacement operation on the displayed image is different from the displacement direction of the captured image due to the behavior of the imaging user Ua, the displayed image is displaced in a direction corresponding to the displacement operation of the observing user Ub. This allows the observing user Ub to displace the displayed image as desired by operation, regardless of the behavior of the imaging user Ua.

In the embodiment, an example is given in which, when there is a displacement of the captured image in the forward direction, the display control unit 205 sets a cut-out reference position equivalent to the cut-out range of the displayed image in the case where there is no displacement of the captured image in the forward direction, and variably sets the displacement speed of the displayed image based on the cut-out reference position and the displacement amount according to the displacement operation.
This makes it possible to displace the display image while canceling the movement of the entire screen caused by the movement of the imaging user Ua.

In the embodiment, the situation information includes information about the output device 30 that displays the display image.
The displacement speed of the display image is set according to information such as the type, size, and display mode of the output device 30 having the display unit 31 viewed by the observing user Ub. For example, the upper speed limit vmax is changed to a value of a speed vmax2 or a speed vmax1. This makes it possible to control the displacement speed of the display image according to a display device that is likely to cause visual sickness and a display device that is unlikely to cause visual sickness.

In the embodiment, the situation information includes the input information of the observing user Ub.
For example, the observing user Ub inputs whether he or she is prone to motion sickness. In response to this, the display control unit 205 changes the value of the speed vmax2 or the speed vmax1 applied as the upper speed limit vmax. This makes it possible to control the displacement speed of the displayed image according to the individual user's susceptibility to motion sickness.

In the embodiment, the situation information includes information about the image contents of the display image.
For example, whether or not a person is susceptible to motion sickness varies depending on the brightness, dynamic range, close view/distant view, high frequency components, etc. of the displayed image. Therefore, the display control unit 205 changes the values of the speed vmax2 and the speed vmax1, for example, according to information about the image content. This makes it possible to control the displacement speed according to the susceptibility of the displayed image to motion sickness.

In the embodiment, the situation information includes information about the viewing environment of the displayed image.
For example, the value of the speed vmax2 or the speed vmax1 is changed depending on the brightness of the room the display device is placed in. This makes it possible to control the displacement speed depending on whether the environment is prone to visually-induced motion sickness.

In the embodiment, the situation information includes information regarding extra-visual information that is output together with the display image.
For example, when audio, vibration, haptic output, etc. are to be provided together with the displayed image, the values of the speed vmax2 and the speed vmax1 are changed according to the level of these outputs. This makes it possible to control the displacement speed according to whether or not the situation is prone to visual sickness.

The processing by the display control unit 205, i.e., the processing of varying the displacement speed of the display image according to the situation information, may be executed by the information processing device 10 on the imaging user Ua side, and the display image after the speed setting processing may be transmitted to the information processing device 20 on the observing user Ub side via the content server 90.
Alternatively, the content server 90 may perform processing by the display control unit 205 and transmit the display image after the speed setting processing to the information processing device 20 .

The program of the embodiment is a program that causes a processor such as a CPU or a DSP, or a device including these, to execute the process of FIG. 6 described above or the processes of FIGS.
In other words, the program of the embodiment is a program that causes the information processing device 20 (or the information processing device 10, or the content server 90) to perform a cut-out process to cut out a range with a narrower angle of view than the first image from a first image captured by an imaging device worn by the imaging user Ua, and to create a second image for observation by the observing user Ub, and also to execute display control to vary the displacement speed of the displayed image based on the second image, which is caused by changing the cut-out range of the second image when there is a displacement of the first image in the forward direction, in accordance with situational information.
Such a program can realize the information processing device 20 according to the embodiment.

The programs of the above-described embodiments can be pre-recorded in a HDD as a recording medium built into a device such as a computer device, or in a ROM in a microcomputer having a CPU. Such programs can also be temporarily or permanently stored (recorded) in removable recording media such as flexible disks, CD-ROMs (Compact Disc Read Only Memory), MO (Magneto Optical) disks, DVDs (Digital Versatile Discs), Blu-ray Discs (registered trademark), magnetic disks, semiconductor memories, and memory cards. Such removable recording media can be provided as so-called package software.
Such a program can be installed in a personal computer or the like from a removable recording medium, or can be downloaded from a download site via a network such as a LAN (Local Area Network) or the Internet.

Furthermore, such a program is suitable for widespread provision of the information processing device 10 of the embodiment. For example, by downloading the program to personal computers, communication devices, mobile terminal devices such as smartphones and tablets, mobile phones, game devices, video devices, PDAs (Personal Digital Assistants), etc., these devices can function as the information processing device 10 of the present disclosure.

Note that the effects described in this specification are merely examples and are not limiting, and other effects may also be present.

The present technology can also be configured as follows.
(1)
An information processing device comprising: a display control unit that performs a cut-out process for cutting out a range of an angle of view narrower than that of a first image from a first image captured by an imaging device worn by an imaging user to create a second image for observation by an observing user; and that varies a displacement speed of a displayed image based on the second image, which is caused by changing the cut-out range of the second image, in accordance with situation information.
(2)
The information processing device according to (1), wherein the displacement in the front direction of the first image occurs due to a position non-designation operation of the image capturing user.
(3)
The display control unit is
The information processing device according to (2) above, wherein a displacement speed of the display image caused by changing the cut-out position of the second image is variably set within a range equal to or less than a displacement speed of the first image.
(4)
the situation information includes whether or not an observing user who observes a display image based on the second image performs a displacement operation on the display image,
The display control unit is
The information processing device according to (2) or (3), wherein a displacement speed of the display image when the displacement operation is performed is made faster than a displacement speed of the display image when the displacement operation is not performed.
(5)
the situation information includes whether or not an observing user who observes a display image based on the second image performs a displacement operation on the display image,
The display control unit is
setting a displacement speed of the display image in the absence of the displacement operation to a first upper speed limit value or less;
The information processing device according to any one of (2) to (4) above, wherein a displacement speed of a display image when the displacement operation is performed is set to a second upper speed limit value or less that is faster than the first upper speed limit value.
(6)
the situation information includes whether or not an observing user who observes a display image based on the second image performs a displacement operation on the display image,
The display control unit is
The information processing device according to any one of (2) to (5) above, wherein when the displacement operation is performed, the cropping range of the second image is set according to the displacement operation, assuming that there is no displacement of the first image in the forward direction.
(7)
the situation information includes whether or not an observing user who observes a display image based on the second image performs a displacement operation on the display image,
The display control unit is
setting a displacement speed of the display image in the absence of the displacement operation to a first upper speed limit value or less;
An information processing device as described in any of (2) to (6) above, wherein when the observing user performs an operation to specify a cut-out range of the second image as the displacement operation, a displacement speed corresponding to the displacement distance of the displayed image is set to be equal to or lower than a second speed upper limit value that is faster than the first speed upper limit value.
(8)
the situation information includes whether or not an observing user who observes a display image based on the second image performs a displacement operation on the display image,
The display control unit is
The information processing device according to any one of (2) to (7) above, wherein, when the observing user performs a position non-designation operation as the displacement operation, a displacement speed of the displayed image is set to a speed corresponding to the position non-designation operation.
(9)
the situation information includes whether or not an observing user who observes a display image based on the second image performs a displacement operation on the display image,
The display control unit is
The information processing device described in any of (2) to (8) above, wherein when the displacement operation includes an operation instructing a displacement in a direction different from the displacement direction of the first image, the cut-out range of the second image is set so that it is displaced in a direction corresponding to the displacement operation.
(10)
the situation information includes whether or not an observing user who observes a display image based on the second image performs a displacement operation on the display image,
The display control unit is
An information processing device as described in any of (2) to (9) above, wherein, when there is a displacement of the first image in the forward direction, a cut-out reference position is set that corresponds to the cut-out range of the second image in the case where there is no displacement of the first image in the forward direction, and a displacement speed of the displayed image is variably set based on the cut-out reference position and a displacement amount corresponding to the displacement operation.
(11)
The information processing device according to any one of (2) to (10), wherein the status information includes information of a display device that displays a display image based on the second image.
(12)
The information processing device according to any one of (2) to (11) above, wherein the situation information includes input information of an observing user observing a display image based on the second image.
(13)
The information processing device according to any one of (2) to (12) above, wherein the status information includes information regarding the image content of the display image based on the second image.
(14)
The information processing device according to any one of (2) to (13) above, wherein the situation information includes information regarding a viewing environment of an image displayed by the second image.
(15)
The information processing device according to any one of (2) to (14) above, wherein the status information includes information regarding a non-visual output that is output together with an image displayed by the second image.
(16)
An information processing device,
An information processing method comprising: performing a cut-out process for cutting out a range of a narrower angle of view than a first image from a first image captured by an imaging device worn by an imaging user to create a second image for observation by an observing user; and performing display control for varying the displacement speed of a displayed image based on the second image, which is caused by a change in the cut-out range of the second image when the first image is displaced in a forward direction, in accordance with situational information.
(17)
A program that causes an information processing device to perform a cut-out process for cutting out a range of a narrower angle of view than a first image from a first image captured by an imaging device worn by an imaging user to create a second image for observation by an observing user, and to execute display control for varying the displacement speed of a displayed image based on the second image, which is caused by a change in the cut-out range of the second image when the first image is displaced in a forward direction, in accordance with situational information.

REFERENCE SIGNS LIST 1 Information processing system 10, 20 Information processing device 30 Output device 31 Display unit 33 Detection unit 40 Imaging device 41, 41a, 41b Imaging unit 90 Content server 95 Storage unit 101 Communication unit 103 Image synthesis unit 105 Recognition unit 201 Communication unit 203 Analysis unit 205 Display control unit 206 Input unit 207 Force/sense control unit 901 Communication unit 903 Content control unit Ua Imaging user Ub Observation user

Claims (17)

An information processing device comprising: a display control unit that performs a cut-out process for cutting out a range of an angle of view narrower than that of a first image from a first image captured by an imaging device worn by an imaging user to create a second image for observation by an observing user; and that varies a displacement speed of a displayed image based on the second image, which is caused by changing the cut-out range of the second image, in accordance with situation information. The information processing device according to claim 1 , wherein the displacement of the first image in the forward direction occurs due to a position non-designation operation by the image capturing user. The display control unit is
The information processing apparatus according to claim 2 , wherein a speed at which the display image is displaced by changing the cut-out position of the second image is variably set within a range equal to or lower than a speed at which the first image is displaced. the situation information includes whether or not an observing user who observes a display image based on the second image performs a displacement operation on the display image,
The display control unit is
The information processing device according to claim 2 , wherein a movement speed of the display image when the movement operation is performed is set to be faster than a movement speed of the display image when the movement operation is not performed. the situation information includes whether or not an observing user who observes a display image based on the second image performs a displacement operation on the display image,
The display control unit is
setting a displacement speed of the display image in the absence of the displacement operation to a first upper speed limit value or less;
The information processing device according to claim 2 , wherein a displacement speed of the display image when the displacement operation is performed is set to a second upper speed limit value or less that is faster than the first upper speed limit value. the situation information includes whether or not an observing user who observes a display image based on the second image performs a displacement operation on the display image,
The display control unit is
The information processing device according to claim 2 , wherein, when the displacement operation is performed, the cutout range of the second image is set in accordance with the displacement operation, assuming that there is no displacement of the first image in the front direction. the situation information includes whether or not an observing user who observes a display image based on the second image performs a displacement operation on the display image,
The display control unit is
setting a displacement speed of the display image in the absence of the displacement operation to a first upper speed limit value or less;
3. The information processing device according to claim 2, wherein when the observing user performs an operation to specify a cut-out range of the second image as the displacement operation, a displacement speed corresponding to a displacement distance of the displayed image is set to a second upper speed limit value that is faster than the first upper speed limit value. the situation information includes whether or not an observing user who observes a display image based on the second image performs a displacement operation on the display image,
The display control unit is
The information processing device according to claim 2 , wherein, when the observer performs a position non-designation operation as the displacement operation, a displacement speed of the displayed image is set to a speed corresponding to the position non-designation operation. the situation information includes whether or not an observing user who observes a display image based on the second image performs a displacement operation on the display image,
The display control unit is
The information processing device according to claim 2 , wherein when the displacement operation includes an operation instructing displacement in a direction different from the displacement direction of the first image, the cropping range of the second image is set so as to be displaced in a direction corresponding to the displacement operation. the situation information includes whether or not an observing user who observes a display image based on the second image performs a displacement operation on the display image,
The display control unit is
3. The information processing device according to claim 2, wherein, when there is a displacement of the first image in the forward direction, a cut-out reference position is set that corresponds to the cut-out range of the second image in a case where there is no displacement of the first image in the forward direction, and a displacement speed of the displayed image is variably set based on the cut-out reference position and a displacement amount corresponding to the displacement operation. The information processing device according to claim 2 , wherein the status information includes information about a display device that displays an image based on the second image. The information processing apparatus according to claim 2 , wherein the situation information includes input information of an observing user who observes a display image based on the second image. The information processing device according to claim 2 , wherein the status information includes information related to an image content of the display image based on the second image. The information processing device according to claim 2 , wherein the situation information includes information related to a viewing environment of the display image based on the second image. The information processing device according to claim 2 , wherein the status information includes information regarding a non-visual output that is output together with the display image based on the second image. An information processing device,
An information processing method comprising: performing a cut-out process for cutting out a range of a narrower angle of view than a first image from a first image captured by an imaging device worn by an imaging user to create a second image for observation by an observing user; and performing display control for varying the displacement speed of a displayed image based on the second image, which is caused by a change in the cut-out range of the second image when the first image is displaced in a forward direction, in accordance with situational information. A program that causes an information processing device to perform a cut-out process for cutting out a range of a narrower angle of view than a first image from a first image captured by an imaging device worn by an imaging user to create a second image for observation by an observing user, and to execute display control for varying the displacement speed of a displayed image based on the second image, which is caused by a change in the cut-out range of the second image when the first image is displaced in a forward direction, in accordance with situational information.

Images