KR20160099981A - Virtual reality device based on two-way recognition including a three-dimensional marker with a patten for multiple users - Google Patents

Abstract

In order to solve the above-described problems, the present invention provides information on mutual position and direction of users through a wearable virtual reality apparatus, a virtual reality image processing apparatus, and a stereoscopic marker, And to provide a customized virtual reality image.
To this end, the multi-user cooperation method according to the present invention recognizes the user wearing the wearable virtual reality realization device equipped with the camera, the infrared camera or the various short-range and long-distance measurable sensors and the marker of the stereoscopic markers, , Estimating a direction; Estimating an absolute position and a direction of the wearable virtual reality apparatus by mutually recognizing the plurality of users; Recognizing the operation of each user, verifying the virtual reality image of the virtual reality realization device, and displaying the corresponding image to each user.

Description

TECHNICAL FIELD [0001] The present invention relates to a virtual reality apparatus capable of bi-directional recognition including a plurality of users using stereoscopic markers having a pattern,

The present invention relates to a wearable virtual reality realization device and a related system and a method thereof capable of bidirectional recognition including a plurality of users based on a stereoscopic marker having a pattern, The object of the present invention is to provide a virtual reality image and a space to a single or plural users based on a mutual recognition based position and direction detection method of a sensor and a stereoscopic marker.

In general, a screen image displayed through a display is a real image delivery method that directly transmits real images, and a virtual reality image delivery method that delivers virtual images created through a computer or virtual images that do not exist in the real world.

With the recent development of image processing and information communication technology, it is possible to provide a virtual reality environment capable of detecting the position and direction of a single or a plurality of users in real time and simultaneously providing images corresponding to respective user's viewpoints.

Generally, in the virtual reality environment, a method of detecting a user's position or direction includes a method of recognizing a single or a plurality of markers provided by a fixed sensor provided in a fixed environment, a method of recognizing a plurality of sensors provided by a user . There is a problem that the fixed sensor and the singular or plural markers are recognized in the fixed environment and the virtual reality or augmented reality is implemented on the marker in a limited space. Also, there is a problem that the markerless method of recognizing a single or a plurality of sensors provided by a user can not be used in an open environment due to instability of mutual recognition in addition to the case of using a real image photographed in advance of a fixed environment.

In the conventional method, a method of estimating only a direction of a pedestrian through pattern recognition of a user's action and a walking gesture has been proposed. However, in the conventional method, it is impossible to follow the position of a user and the mutual position of a plurality of users Do. In addition, although a method of estimating the position of a pedestrian based on motion recognition has been proposed (Patent Document 0002), it is impossible to follow the mutual position of a plurality of users even in the above invention.

In the conventional invention, a directional trekking system for augmented reality apparatuses using a plurality of sensors has been proposed in order to follow the mutual position of a single user or a plurality of users. It is impossible to follow the mutual position among a plurality of users in an open environment or an arbitrary environment in which environmental information is uncertain, although the direction and position of a single or plural users are tracked through a device and a plurality of sensors.

The virtual reality of the marker or markerless method can be used in various fields. However, in order to provide a service that realizes a virtual reality according to a user's eyes such as a collaboration among a plurality of users, real-time rendering, And equipment operation cost.

KR 10-1472960 B1 KR 10-1473653 B1 KR 10-1285913 B1

The present invention has been made to solve the above problems, and it is an object of the present invention to provide a virtual reality image processing apparatus and a wearable image realizing apparatus having a single sensor or a plurality of sensors and a stereoscopic marker for detecting a position of a user and a position between users, It is an object of the present invention to provide a virtual reality environment image and space capable of recognizing the position and direction of a single user or a plurality of users in an open environment and enabling collaboration among users.

A virtual reality realization platform constituted by a stereoscopic marker and a virtual reality image processing device in which a marker is recorded and a wearable virtual reality realization device having a single or a plurality of sensors is constituted, and a marker of the stereoscopic markers A system capable of estimating a position and a direction of a user through a wearable virtual reality realizing device recognizing a wearable virtual reality realizing device and a system capable of estimating respective user relative positions by operating the plurality of wearable virtual reality realizing devices, The present invention provides a system for transmitting a virtual reality image to a wearable virtual reality apparatus and visualizing the virtual reality image, and an apparatus and method for implementing the system.

According to the present invention, there is provided a limited virtual reality or augmented reality which is conventionally transmitted to a user in a fixed environment or an environment in which prior information exists. However, It is possible to implement a system that can cooperate with each other even in an open environment or an environment lacking advance information through marker recognition of markers, and it is possible to provide a virtual reality implementation environment optimization through frame optimization through mutual recognition.

BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a conceptual diagram of a plurality of wearable virtual reality devices and stereoscopic marker period position and orientation detection according to a preferred embodiment of the present invention; Fig.
FIG. 2 is a block diagram of a position, direction detection and corresponding virtual reality image implementation method between a plurality of users wearing a wearable virtual reality apparatus according to a preferred embodiment of the present invention
3 is a system block diagram of a wearable virtual reality apparatus and a virtual reality image processing apparatus according to a preferred embodiment of the present invention
4 is a frame optimization block diagram of a virtual reality image processing apparatus through acceleration sensing of a user.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals are used to designate the same or similar components throughout the drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear. In addition, the preferred embodiments of the present invention will be described below, but it is needless to say that the technical idea of the present invention is not limited thereto and can be variously modified by those skilled in the art.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a conceptual diagram of a plurality of wearable virtual reality devices and stereoscopic marker period position and direction detection. FIG.

The present invention can provide a system capable of extracting relative coordinates and absolute coordinates between users by using one or more wearable virtual reality realization devices and stereoscopic markers in an environment where open environment and space information are insufficient.

1 is a block diagram of a wearable virtual reality apparatus 120 of a first user and a wearable virtual reality apparatus 120 of a first user according to an embodiment of the present invention. So that it is possible to confirm the relative position of the user.

The wearable virtual reality apparatuses 110 and 120 may include a space 110a recognizable by a wide angle of view 112 by having a camera 111, a plurality of cameras, an infrared camera, a sensor, The surround images can be provided to the user by combining the images obtained through the camera.

When the wearable virtual reality apparatus 110 of the first user and the wearable virtual reality apparatus 120 of the second user view the stereoscopic marker 130, the stereoscopic marker 130 is recorded The recognized marker can be recognized (130a).

The stereoscopic markers 130 may have a square three-dimensional shape, and may have a single or a plurality of markers on each side thereof, and may include a single or a plurality of auxiliary markers for extracting positions and directions of the respective markers .

A marker recognized by the wearable virtual reality apparatuses 110 and 120 is measured by comparing the extended line of the left marker 131 and the right marker 132 to measure the horizontal inclination of the left marker 131 and the upper marker 133 The vertical gradient is measured by comparing the extensibility. Also, by re-comparing the right marker 132 and the upper marker 133, the direction can be estimated by correcting the horizontal and vertical inclination. In addition, by comparing the pixel units of the marker plane image 130a recognized by the wearable virtual reality apparatuses 110 and 120 with the resolutions of the wearable virtual reality apparatuses 110 and 120, It is possible to estimate the position with respect to the depth.

The location and direction information of the wearable virtual reality apparatuses 110 and 120 can be transmitted in real time via the wired / wireless communication network and the virtual reality images can be transmitted through the wearable virtual reality apparatuses 110 and 120 of each user, .

FIG. 2 is a block diagram of location and direction detection of a plurality of users wearing a wearable virtual reality apparatus according to a preferred embodiment of the present invention, and a corresponding virtual reality image realization method.

In the present invention, a plurality of users are recognized through the interworking 210 between the first user and the second user through the wearable image implementing device through the Bluetooth, the camera and various sensors.

(220) of the stereoscopic markers of the stereoscopic marking apparatus of the wearable image implementing apparatus of the first user. After the obtained landmark image is transmitted 230 to the virtual reality image processing apparatus, the direction and position of the first user are detected 240.

Recognizes the first user's photographing and operation recognition (260) through the photographing and operation recognition (250) of the second user and the wearable image realizing device of the second user through the wearable image implementing apparatus of the first user.

The images of the first user and the second user obtained from the respective users are transmitted to the virtual reality image processing apparatus (270), and mutual positions and directions of the users are detected (280).

And transmits and implements 290 a virtual reality image to the wearable virtual reality image realizing device through the virtual reality image processing device to the first user and the second user.

The plurality of users participating in the virtual reality realization method proposed by the present invention are not limited to the first and second users, and after the mutual position and direction are detected during the process 240, the process is repeatedly performed in real time And high-quality images are displayed to a plurality of users.

3 is a system block diagram of a wearable virtual reality apparatus and a virtual reality image processing apparatus according to a preferred embodiment of the present invention.

The virtual reality image processing apparatus 320, the virtual reality image storage unit 330, the communication unit 340, the stereoscopic marker 350, the stereoscopic image display unit 350, And a central control unit 360.

In the present invention, the wearable image implementing apparatus 310 that can be worn by the user can follow the mutual positions and directions between the stereoscopic markers 350 and the virtual reality image processing apparatuses 320 located at arbitrary positions, Direction, and image information transmitted from the real-time image processing apparatus 320. [

The wearable image implementing apparatus 310 includes a marker image input unit 311 for obtaining a marker image from the stereoscopic markers, a marker image transmission number 312 for transmitting a marker image to the virtual reality image processing apparatus 320, A virtual reality image receiving unit 313 for receiving an image from the processing apparatus and outputting the image to a user, and a virtual reality image output unit 314. [ The apparatus may include a single sensor or a plurality of sensors, a camera, and the like.

In the present invention, the virtual reality image processing apparatus 320 includes a marker image receiving unit 321 for receiving and detecting a marker image acquired from the wearable image implementing apparatus, and a marker image recognizing unit 322 for detecting a marker image . A position calculating unit 323 for calculating the position and direction of the user from the obtained marker image, a position for transmitting the information, and a direction transmitting unit 324. A virtual reality computing unit 325 for computing a virtual reality image to be transmitted to the virtual reality image processing apparatus, and a virtual reality transmitting unit 326 for transmitting the information.

The virtual reality image storing unit 330 storing the virtual reality image includes a virtual reality DB 331 storing virtual reality image information and a virtual reality transmitting unit 332 transmitting the image to the virtual reality image processing apparatus Consists of. The apparatus is constituted by a nonvolatile memory, and not only virtual reality image information but also related application programs can be provided. And may be provided as a storage device so as to store images obtained from a single sensor or a plurality of sensors to a wearable image implementing apparatus.

The communication unit 340 includes a communication module that uses a wired / wireless communication network for communication between the wearable image implementing apparatus 310 and the virtual reality image processing apparatus 320, The network is capable of providing service to a single user or a plurality of users in another space.

And a square stereoscopic marker 350 may be provided so that the stereoscopic marker 350 can be recognized through the wearable image implementing apparatus 310. The stereoscopic markers 350 have a square stereoscopic shape and independent markers are recorded on each surface, so that the shape of the markers can be seen differently in any direction.

The centralized control unit 360 for performing the integrated power management and environmental management of each system below can include an integrated system management program for system on / off, system initialization, environment saving, and the like.

4 is a frame optimization block diagram of a virtual reality image processing apparatus through acceleration sensing of a user.

Real-time image processing of the mutual recognition and virtual reality image processing apparatus 320 through a single sensor or a plurality of sensors of the wearable image implementing apparatus 310 requires much physical signal processing. Therefore, the user can optimize the frame of the acquisition information and the virtual reality image processing through optimization of the mutual recognition rate and the motion recognition rate.

Through the wearable image implementing apparatus 310, motion and motion of a plurality of users are detected 410 and a plurality of user motion patterns are determined 420. An acceleration calculation 430 according to motion of a plurality of users is performed through an acceleration sensor and various sensors provided in the wearable image implementing apparatus 310 and a frame operation and an operation check according to an operation recognition rate to be obtained are performed 440 .

Through the above process, the user operation recognition rate can be set to the operation speed 450 and the photographing speed according to the frame 450. For example, when a plurality of users are in a static state or a static state corresponding to the plurality of users, it is detected that the change in operation is relatively small, and then the operation recognition rate to be obtained is calculated. By optimizing the minimum acquisition frame, Can be minimized.

It will be apparent to those skilled in the art that various modifications, substitutions and substitutions are possible, without departing from the scope and spirit of the invention as disclosed in the accompanying claims. will be. Therefore, the embodiments disclosed in the present invention and the accompanying drawings are intended to illustrate and not to limit the technical spirit of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments and accompanying drawings. The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

100: Multiple wearable virtual reality devices and three-dimensional marker system
200: Location and direction detection between multiple users wearing a wearable virtual reality device and a block diagram of a corresponding virtual reality image implementation method
300: Wearable Virtual Reality Implementation System and System of Virtual Reality Image Processing System
400: Block optimization block diagram of virtual reality image processing device through acceleration sensing

Claims (9)

A virtual reality realization platform composed of a wearable virtual reality realization device having a single sensor or a plurality of sensors, a stereoscopic markers recorded with markers, and a virtual reality image processing device;
A system capable of estimating a position and a direction of a user through a wearable virtual reality realization device capable of recognizing a marker of a stereoscopic marker on which the marker is recorded;
A system capable of estimating respective user-relative positions by operating a plurality of wearable virtual reality devices;
A platform for transmitting a virtual reality image to the one or more wearable virtual reality devices and visualizing the virtual reality image and an implementation method thereof
The method according to claim 1,
A position and direction estimation system of a user using a wearable virtual reality realization device characterized by recognizing a polygonal stereoscopic marker and markers in which a single or a plurality of markers are recorded on each face
The method according to claim 1,
Recognizes mutual recognition and operation through a plurality of user-to-user sensors wearing a wearable virtual reality apparatus equipped with a single or a plurality of sensors.
The method of claim 3,
Wherein the absolute position of each user can be obtained through mutual recognition between the plural users and recognition of the stereoscopic markers in the above step
5. The method of claim 4,
And transmits the video corresponding to the operation through the wearable virtual reality realization device of each user after collating the operation information of each user with the virtual reality database
The method of claim 3,
Wherein a wide angle-of-view screen can be displayed to a user after combining screens of a plurality of sensors of a wearable virtual reality apparatus
The method of claim 3,
And provides mutual cooperation virtual reality images and spaces using the mutual recognition and operation contrast by providing respective user position and direction information.
6. The method of claim 5,
Characterized in that, through mutual recognition and motion recognition, the virtual virtual reality realizing device performs a virtual overlay on a part which is not recognized by the wearable virtual reality realizing device
(a) estimating a position and a direction of a user by recognizing a marker of a user wearing a wearable virtual reality apparatus equipped with the camera, an infrared camera, or a sensor capable of measuring various near and long distances, and a marker of the stereoscopic markers;
(b) estimating an absolute position and a direction of the wearable virtual reality apparatus by mutually recognizing the plurality of users;
(c) recognizing the operation of each user and checking the virtual reality images of the virtual reality realizing device;
(d) transferring a corresponding image to each wearable virtual reality realizing device in operation of each user;
(e) displaying the delivered image on each wearing virtual reality realizing device through virtual overwriting, and implementing the same

Images