WO2020115824A1 - Virtual space display system - Google Patents

Abstract

[Problem] To provide a virtual space display system capable of increasing the appeal to a user. [Solution] A virtual space display system which displays, as images on a display device mounted on a head part of a user floating in a water area, a virtual space and an object arranged in the virtual space. The virtual space display system is provided with: a head part motion detection unit which detects motion of the head part of the user; a resistance detection unit that is mounted on an arbitrary body part of the user and detects resistance that the user receives from the water area at the body part; and a virtual space processing program which transforms the virtual space and the object on the basis of the motion of the head part of the user detected by the head part motion detection unit and the resistance that the user receives from the water area and detected by the resistance detection unit.

Description

Virtual space display system

The present invention relates to a virtual space display system, and more particularly to a virtual space display system for displaying a virtual space and an object arranged in the virtual space on a display device mounted on a user's head in a water area as an image.

A technology for displaying a virtual space created by VR (Vertical Reality) technology on a head mounted display (Head Mount Display: HMD) mounted on the user's head is popular.

Patent Document 1 discloses a virtual space display system that displays a virtual space on the HMD of a user who wears an HMD and floats in a water area.

According to the virtual space display system of Patent Document 1, the motion of the user's head in a state of being suspended in the water in the prone position is detected, and an image of the virtual space is generated according to the detected motion of the head. The user can enjoy the image in the virtual space while feeling floating in the water.

Japanese Patent Laid-Open No. 2018-41180

By the way, when the user visually recognizes the image of the virtual space in the water area, the interaction between the image of the virtual space and the user can be realized if it is possible to reflect not only the movement of the head but also the sensation that the user receives from the water area. The interest of the user is improved.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a virtual space display system capable of improving the interest of the user.

A virtual space display system according to the present invention for achieving the above object is a virtual space display system for displaying a virtual space and an object arranged in the virtual space on a display device mounted on a user's head in a water area as an image. In, a head motion detection unit that detects a motion of the user's head, a resistance detection unit that is attached to an arbitrary body part of the user and detects resistance that the user receives from the water area in the body part, and a head motion detection unit And a virtual space processing program that transforms a virtual space and an object based on the resistance of the user's head detected by 1. and the resistance detected by the resistance detection unit from the water area.

This virtual space display system is characterized by including a gyro sensor and an acceleration sensor that are attached to an arbitrary body part of the user and detect movements of the body part in the water area.

According to this virtual space display system, the display device mounted on the user's head based on the motion of the user's head detected by the head motion detection unit and the resistance received by the user from the water area detected by the resistance detection unit. The virtual space and the objects displayed as images are transformed.

Therefore, since the virtual space and the object are transformed based on the user's action, the interaction between the image of the virtual space and the image of the object and the user is realized, and the interest of the user is improved.

In particular, since the resistance detection unit that detects the resistance received from the water area is provided, the user in the water area can reflect a specific action force received from the water area, such as resistance or buoyancy, in the virtual space and the object, and the user's interest can be enhanced. Is expected to improve further.

Furthermore, the virtual space processing program of this virtual space display system is configured to detect the motion of the head of one user detected by the head motion detection unit that detects the motion of the head of one user and the resistance that the user receives from the water area. The resistance detected by the resistance detection unit that detects the movement of the head of another user detected by the head movement detection unit that detects the movement of the head of another user, and the resistance received by the other user from the water area. It is characterized in that the virtual space and the object are transformed based on the resistance detected by the resistance detecting section.

As described above, based on the movements of the heads of a plurality of users detected by the head movement detection unit and the resistance received by the users from the water area detected by the resistance detection unit, the display device attached to the heads of the respective users is displayed. The virtual space and the object displayed as an image are transformed.

Therefore, the virtual space and the object are transformed based on the action of each user, so that the interaction between the image of the virtual space and the image of the object and each user is realized, and the interaction between each user is realized. Therefore, the interest of each user is further improved among a plurality of users.

According to the present invention, since the virtual space and the object are transformed based on the user's action, the interaction between the image of the virtual space and the image of the object and the user is realized, and the interest of the user is improved.

It is a figure explaining an outline of composition of a virtual space display system concerning an embodiment of the invention. Similarly, it is a block diagram illustrating an outline of a configuration of a server of the virtual space display system according to the present exemplary embodiment. Similarly, it is a block diagram explaining an outline of a configuration of a smartphone stored in a head mounted display of the virtual space display system according to the present embodiment. Similarly, it is a block diagram illustrating an outline of a configuration of an operation unit of the virtual space display system according to the present embodiment. Similarly, it is a block diagram illustrating an outline of a configuration of a virtual space processing program according to the present embodiment. Similarly, it is a diagram for explaining an outline of operation when the virtual space display system according to the present embodiment is used by a plurality of users.

Next, a virtual space display system according to an embodiment of the present invention will be described based on FIGS. 1 to 6.

FIG. 1 is a diagram for explaining the outline of the configuration of the virtual space display system according to this embodiment. As shown in the figure, the virtual space display system 10 is a server 20 that creates a virtual space 11 and the like, and is a display device that is attached to a user 1 located in a water area W and displays the virtual space 11 created by the server 20. A head mounted display (Head Mount Display: HMD) 30 and an operation unit 40 connected to the server 20 and attached to the hand of the user 1 are provided. The user 1 in the present embodiment may be floating in the water body W, or may be used with the legs attached to the bottom of the water body (or a member such as a table placed on the bottom). ..

In this embodiment, in the virtual space 11, there are a virtual sphere 12 which is a spherical object expressed in three dimensions and a virtual hand 13 which is an object in which a hand which is a body part of the user 1 is displayed in three dimensions. Will be placed.

The virtual hand 13 is operated by the operation unit 40 attached to the hand of the user 1 in the present embodiment.

FIG. 2 is a block diagram for explaining the outline of the configuration of the server 20 according to this embodiment. As illustrated, the server 20 includes a processor 21, a memory 22, a storage 23, a transmission/reception unit 24, and an input/output unit 25 as main components, which are electrically connected to each other via a bus 26.

The processor 21 is an arithmetic device that controls the operation of the server 20, controls the transmission and reception of data between each element, and performs the processing necessary for executing an application program.

The processor 21 is, for example, a CPU (Central Processing Unit) in the present embodiment, and executes each application by executing an application program stored in a storage 23 described later and expanded in a memory 22.

The memory 22 includes a main storage device configured by a volatile storage device such as a DRAM (Dynamic Random Access Memory) and an auxiliary storage device configured by a non-volatile storage device such as a flash memory or an HDD (Hard Disc Drive). ..

The memory 22 is used as a work area of the processor 21, and also stores a BIOS (Basic Input/Output System) executed when the server 20 is started, and various setting information.

The storage 23 stores application programs and data used for various processes. In this embodiment, the virtual space processing program 23A is stored. The outline of the configuration of the virtual space processing program 23A will be described later.

The transmission/reception unit 24 connects the server 20 to the network, and in the present embodiment, the HMD 30 and the operation unit 40 are connected to the transmission/reception unit 24 via the network.

The transmission/reception unit 24 may have a short-range communication interface such as Bluetooth (registered trademark) or BLE (Bluetooth Low Energy).

The input/output unit 25 may be connected with an information input device such as a keyboard and a mouse or an output device such as a display, if necessary.

The bus 26 transmits, for example, an address signal, a data signal, and various control signals among the connected processor 21, memory 22, storage 23, transmission/reception unit 24, and input/output unit 25.

As shown in FIG. 1, in the present embodiment, the HMD 30 is formed in a helmet type to be worn on the head of the user 1, and includes a smartphone storage unit 31 and a respirator 32.

In the present embodiment, the smartphone storage unit 31 stores a smartphone 33 which is formed at a portion corresponding to the position of the eyes of the user 1 and which is connected to the server 20 via the network.

In the present embodiment, the respirator 32 is formed in a tubular shape or a pipe shape that extends upward from an area corresponding to the position of the mouth of the user 1 and opens, and inspires and exhausts according to the breathing of the user 1. Is done.

FIG. 3 is a block diagram illustrating an outline of the configuration of the smartphone 33 according to the present embodiment. As illustrated, the smartphone 33 includes a display 33A and a head movement detection unit 33B.

In the present embodiment, the virtual space 11, the virtual sphere 12, and the virtual hand 13 generated by the server 20 are displayed on the display 33A as images. This display 33A is mounted as a display of the HMD 30 in the present embodiment.

In the present embodiment, the head motion detection unit 33B includes a geomagnetic sensor 33Ba, a gyro sensor 33Bb, and an acceleration sensor 33Bc, detects the motion of the head of the user 1 wearing the HMD 30 in which the smartphone 33 is stored, Generate a head motion signal.

In particular, the gyro sensor 33Bb detects the angular velocities of the HMD 30 around the three axes with time according to the movement of the head of the user 1 and the corresponding movement of the HMD 30, and determines the time change of the angle (tilt) around each axis. can do.

When the smartphone 33 mounts the HMD 30 stored in the smartphone storage unit 31, in the present embodiment, the user 1 observes only the display 33A of the smartphone 33.

With this, the user 1 loses all the external field of view, and thus can completely immerse himself in the virtual space 11 displayed on the display 33A of the smartphone 33.

In the present embodiment, the user 1 who is immersed in the virtual space 11 can execute the interaction with the virtual sphere 12 by using the virtual hand 13 that operates in conjunction with the operation of his or her hand.

As shown in FIG. 1, the operation unit 40 has a glove shape that can be worn on the hand of the user 1, and is attached to both hands of the user 1 in the present embodiment.

FIG. 4 is a block diagram illustrating an outline of the configuration of the operation unit 40 according to the present embodiment. As illustrated, the operation unit 40 includes a first motion detection unit 41 that detects a motion of the hand of the user 1, a second motion detection unit 42 that detects a motion of the finger of the user 1, a first motion detection unit 41, and a first motion detection unit 41. The motor 43 is driven based on the finger motion detected by the two-motion detector 42, and the controller 44 controls various operations of the operation unit 40.

In the present embodiment, the first motion detection unit 41 includes a resistance sensor 41a, which is a resistance detection unit, a gyro sensor 41b, and an acceleration sensor 41c. Here, as the characteristics of water, there are four elements of buoyancy, resistance, water pressure, and water temperature. The present invention identifies the movement or displacement of the user 1 by detecting/detecting his/her own resistance among them. Not only the resistance but also the water pressure due to the difference in water depth may be detected if necessary. Further, the “detection unit” may be considered as a single unit, or may be built in a controller (operation unit) or the like.

In the present embodiment, the resistance sensor 41a is a sensor that detects the resistance that the user 1 receives from the water area W, and detects the movement of the hand of the user 1 in cooperation with the gyro sensor 41b and the acceleration sensor 41c. 1 Generate an operation signal. The sensor according to the present embodiment is not limited to this, for example, a strain sensor, a torque sensor, a semiconductor resistance sensor, etc., the movement of the object, the displacement, such as speed, it is possible to obtain the information of the change, and underwater Any material may be used as long as it can be used (or processed so that it can be used in water).

The second motion detection unit 42 is composed of a motion capture mechanism including a sensor that detects the motion of the finger of the user 1, and detects the motion of the finger of the user 1 to generate a second motion signal.

FIG. 5 is a block diagram for explaining the outline of the configuration of the virtual space processing program 23A according to this embodiment. As illustrated, the virtual space processing program 23A includes a video generation unit 23Aa, a video transformation signal generation unit 23Ab, and a feedback signal generation unit 23Ac.

In the present embodiment, the image generation unit 23Aa generates the virtual space 11, the virtual sphere 12 and the virtual hand 13 based on the preset virtual space generation data D, and also virtualizes the virtual sphere 12 and the virtual hand 13. It is placed in the space 11.

Furthermore, the image generation unit 23Aa changes the virtual space 11, the virtual sphere 12, and the virtual hand 13 based on the image conversion signal generated by the image generation signal generation unit 23Ab described below.

In the present embodiment, the image transformation signal generation unit 23Ab includes the head movement signal S0 generated by the head movement detection unit 33B of the smartphone 33 and the first movement generated by the first movement detection unit 41 of the operation unit 40. The image transformation signal S3 is generated based on the signal S1 and the second motion signal S2 generated by the second motion detector 42.

In the present embodiment, the feedback signal generation unit 23Ac includes the head motion signal S0 generated by the head motion detection unit 33B of the smartphone 33 and the first motion signal generated by the first motion detection unit 41 of the operation unit 40. The feedback signal S4 is generated based on S1 and the second motion signal S2 generated by the second motion detector 42.

Next, an outline of the operation of the virtual space display system 10 will be described. In the following description, the state in which the user 1 floats in the water area W will be mainly described, but the present embodiment is not limited to this, and a part of the body is the bottom of the water area (or a structure such as a table in the water area). May be in contact with.

In a state in which the user 1 wearing the HMD 30 and the operation unit 40 floats in the water W, the virtual space 11, the virtual sphere based on the virtual space generation data D preset by the video generation unit 23Aa of the virtual space processing program 23A. 12 and the virtual hand 13 are generated, the virtual sphere 12 and the virtual hand 13 are arranged in the virtual space 11, and these are displayed as an image on the HMD 30 (display 33A of the smartphone 33).

In this state, when the user 1 floating in the water W moves his/her head, the head motion detection unit 33B detects the motion of the head of the user 1 and generates the head motion signal S0 according to the motion. To do.

When the user 1 floating in the water area W moves his or her hand, the resistance sensor 41a of the first motion detection unit 41 detects the resistance that the user 1 receives from the water area W in his hand in response to the motion, and the gyro sensor 41b. Also, in cooperation with the acceleration sensor 41c, the motion of the hand of the user 1 is detected to generate the first motion signal S1.

Similarly, when the user 1 floating in the water W moves his/her finger, the second motion detection unit 42 detects the motion of the finger of the user 1 according to the motion, and generates the second motion signal S2.

When the head movement signal S0, the first movement signal S1, and the second movement signal S2 generated in this way are input to the video transformation signal generation unit 23Ab, the video transformation signal S3 is generated by the video transformation signal generation unit 23Ab. Is generated.

When the image transformation signal S3 is generated, the virtual space 11, the virtual sphere 12, and the virtual hand 13 generated by the image generation unit 23Aa are transformed by the image generation unit 23Aa, and the transformed virtual space 11, virtual sphere 12, and The image of the virtual hand 13 is displayed on the HMD 30.

On the other hand, when the head movement signal S0, the first movement signal S1, and the second movement signal S2 are input to the feedback signal generation unit 23Ac, the feedback signal generation unit 23Ac generates the feedback signal S4.

When the feedback signal S4 is generated, the motor 43 of the operation unit 40 is driven, and the torque of the motor 43 limits the amount of movement of the hand and fingers of the user 1. When touching the virtual sphere 12, a pseudo tactile sensation when touching the virtual sphere 12 is fed back to the user 1 via the operation unit 40.

As described above, the virtual space 11, the virtual sphere 12, and the virtual hand 13 generated by the image generation unit 23Aa are the head motion signal S0 generated by the head motion detection unit 33B, and the resistance sensor of the first motion detection unit 41. It is transformed based on the first motion signal S1 generated by 41a and the like and the second motion signal S2 generated by the second motion detector 42.

Therefore, since the virtual space 11, the virtual sphere 12, and the virtual hand 13 are transformed based on the motion of the user 1, the interaction between the image of the virtual space 11, the image of the virtual sphere 12, and the image of the virtual hand 13 and the user 1. Is realized, and the interest of the user 1 is improved.

In particular, in the present embodiment, the resistance sensor 41a that detects the resistance received from the water area W cooperates with the gyro sensor 41b and the acceleration sensor 41c to detect the motion of the hand of the user 1 and generate the first motion signal S1. Therefore, the specific acting force (buoyancy, resistance, etc.) that the user 1 floating in the water area W receives from the water area W can be reflected in the virtual space 11, the virtual sphere 12, and the virtual hand 13.

Therefore, it is expected that the interest of users will be further improved.

FIG. 6 is a diagram illustrating an operation outline when the virtual space display system 10 is used by a plurality of users. As illustrated, one user 1 and another user 2 floating in different water areas W1 and W2 respectively mount an HMD 30 and an operation unit 40 that can communicate with the server 20, and each HMD 30 has a virtual space 11 , The virtual sphere 12, the virtual hand 13, and the virtual hand 14 of the user 2 are displayed.

In this state, the head motion signal S0, the first motion signal S1 and the second motion signal S2 are respectively supplied to the users 1 and 2 depending on the motions of the heads, hands and fingers of the users 1 and 2 floating in the water areas W1 and W2. 1(2) is generated respectively, and the video alteration signal generation unit 23Ab generates the video alteration signal S3 for each user 1(2).

When the image transformation signal S3 for each user 1(2) is generated, the virtual sphere 12 and the virtual hand 13(14) are transformed in the virtual space 11, and the transformed virtual space 11, virtual sphere 12 and virtual are created. The image of the hand 13 (14) is displayed on the HMD 30 of each user 1 (2).

On the other hand, when the head motion signal S0, the first motion signal S1, and the second motion signal S2 are generated according to the motion of the head, hand, and finger of each user 1(2) floating in the water areas W1 and W2. In the feedback signal generation unit 23Ac, the feedback signal S4 for each user 1(2) is generated.

When the feedback signal S4 for each user 1(2) is generated, the motor 43 of the operation unit 40 is driven, and the torque of the motor 43 limits the amount of movement of the hand and finger of the user 1(2). The pseudo tactile sense of the virtual hand 13 (14) in the space 11 is fed back to the user 1 (2) via the operation unit 40.

As described above, the virtual space 11, the virtual sphere 12, and the virtual hand 13 (14) generated by the video generation unit 23Aa are generated for each user 1 (2) based on the motions of the plurality of users 1 (2). The transformation is performed based on the head movement signal S0, the first movement signal S1, and the second movement signal S2.

Therefore, the virtual space 11, the virtual sphere 12, and the virtual hand 13 are transformed based on the respective movements of the user 1 (2), so that the image of the virtual space 11, the image of the virtual sphere 12, and the virtual hand 13 (14). Since the interaction between the video and the user 1(2) is realized and the interaction between the user 1 and the user 2 is realized, the interest of the user 1(2) is further improved.

The present invention is not limited to the above-mentioned embodiments, and various modifications can be made without departing from the spirit of the invention. In the above embodiment, the case where the operation unit 40 is attached to the hand of the user 1(2) has been described, but the operation unit 40 may be attached to another body part such as a leg.

In the above embodiment, the case where the virtual space display system 10 is used by the user 1 or a plurality of users 1(2) has been described, but the number of users is not particularly limited.

In the above embodiment, the case where the display of the HMD 30 is implemented by the display 33A of the smartphone 33 stored in the HMD 30 has been described, but it may be implemented by incorporating the display in the HMD 30.

In the above embodiment, the case where the water areas W1 and W2 are different water areas has been described, but they may be the same water area.

Further, the water areas W, W1 and W2 may be pools and various kinds of water storage tanks, seas, lakes and the like, or may be pools and bathtubs at home or other facilities where the body below the neck can be immersed. Any body of water may be used. Further, the above-mentioned HMD 830 was mainly a device related to VR (Vertical Reality) technology, but in the field of AR (Augmented Reality) technology and MR (Mixed Reality) technology, for example. The present invention can also be applied. In this case, for example, the position of the user (including the displacement of a part of the body) may be grasped by the position tracking technique, and the process according to the position may be performed.

1, 2 User 10 Virtual space display system 11 Virtual space 12 Virtual sphere (object)
13, 14 Virtual hand (object)
20 server 23A virtual space processing program 23Aa video generation unit 23Ab video transformation signal generation unit 23Ac feedback signal generation unit 30 HMD (display device)
33 Smartphone 33A Display 33B Head motion detection unit 40 Operation unit 41 First motion detection unit 41a Resistance sensor (resistance detection unit)
41b Gyro sensor 41c Acceleration sensor 42 Second motion detector S0 Head motion signal S1 First motion signal S2 Second motion signal S3 Video transformation signal S4 Feedback signals W, W1, W2 Water area

Claims (3)

In a virtual space display system for displaying a virtual space and an object arranged in the virtual space on a display device mounted on a user's head in a water area as an image,
A head motion detection unit that detects a motion of the head of the user,
A resistance detection unit that is attached to any body part of the user and detects resistance received by the user from the body of water in the body part,
A virtual space processing program that transforms the virtual space and the object based on the motion of the head of the user detected by the head motion detection unit and the resistance received by the user from the water area detected by the resistance detection unit. When,
A virtual space display system comprising: The virtual space display system according to claim 1, further comprising a sensor that is attached to an arbitrary body part of the user and detects a motion of the body part in the water area. The virtual space processing program,
The resistance detected by the resistance detection unit that detects the movement of the head of the one user detected by the head movement detection unit that detects the movement of the head of the one user and the resistance that the one user receives from the water area. And a resistance detection unit that detects the movement of the head of another user detected by the head movement detection unit that detects the movement of the head of another user and the resistance that the other user receives from the water area. The virtual space display system according to claim 1, wherein the virtual space and the object are transformed based on the resistance.

Images