WO2016036065A1 - Panoramic stereo photographic device and method - Google Patents

Abstract

This panoramic stereo photographic device may comprise: multiple camera modules configured to obtain images; and a control unit configured to generate a left-eye panoramic image and a right-eye panoramic image by using the images obtained by the multiple camera modules and to generate a stereo image by using the left-eye panoramic image and right-eye panoramic image. The panoramic stereo photographic device is configured such that the centers of the left-eye panoramic image and right-eye panoramic image are always aligned at the same reference point even if the direction of view changes. Accordingly, the occurrence of flaws in the boundaries of images due to the limitation of a rig structure in conventional panoramic stereo photographic devices can be prevented, and a panoramic stereo image can be obtained using a relatively small number of cameras.

Description

Panoramic stereo photography device and method

Embodiments are directed to an apparatus and method for panoramic stereo imaging, and more particularly, to an apparatus and method for acquiring a stereoscopic stereo image by combining a plurality of panoramic cameras.

The stereoscopic image refers to a stereoscopic image in which an image corresponding to the left eye and an image corresponding to the right eye are separately configured. For example, when the polarization of the image corresponding to both eyes different from each other, and wearing a special glasses to which the polarizer is applied to watch a stereo image, it is possible to feel a three-dimensional feeling from the image displayed on the plane.

The panoramic image refers to a wide viewing angle image obtained by capturing an image using a plurality of cameras arranged along a trajectory of an image to be acquired and then concatenating the obtained images. When the stereo image is to be obtained in the form of a panorama, since the stereo image is divided into the left eye image and the right eye image, several pairs of a left eye camera and a right eye camera must be used to obtain a panoramic stereo image.

1 is a schematic configuration diagram of a conventional panoramic stereo imaging apparatus.

Referring to FIG. 1, in the conventional panoramic stereo photographing apparatus, a pair of left eye cameras 101, 102, 103, and 104 that captures a left eye image and a right eye camera 111, 112, 113, and 114 that capture a right eye image are paired one by one. And a plurality of left eye camera-right camera pairs are disposed around the reference point 100. The reference point 100 corresponds to a pivot point of the trajectory of the panoramic image. For example, when the panoramic stereo imaging apparatus is applied to a head-mounted display device (HMD), the head position of the user wearing the HMD may correspond to the reference point 100.

When the user wearing the HMD looks at a specific gaze direction, the image may be acquired by a left eye camera-right camera pair positioned in the corresponding direction. For example, as shown by arrows in FIG. 1, when the line of sight is at the top of the drawing, a stereo image may be obtained using the left eye camera 101 and the right eye camera 111 arranged to photograph the direction. Since the left-eye camera-right camera pairs are arranged in various directions, even if the line of sight changes, a stereo image of the corresponding direction can be obtained, and as a result, a panoramic stereo image can be obtained.

However, the conventional panoramic stereo imaging apparatus has a limitation in matching the image boundary due to the limitation of the rig structure connecting the left eye camera and the right eye camera. That is, since the positions of the cameras are slightly separated from each other, the center of the left eye camera 101-right eye camera 111 pair and the left eye camera 102-right eye camera 112 pair do not coincide with each other in FIG. 1. As a result, a defective area is generated between the photographing area by the left eye camera 101-right eye camera 111 pair and the photographing area by the left eye camera 102-right eye camera 112 pair. This causes the panoramic image to be disconnected without naturally connecting when the user rotates the gaze direction from the front (ie, top in FIG. 1) to the right (ie, right in FIG. 1).

According to an aspect of the present invention, it is possible to provide a panoramic stereo photographing apparatus and method which can always improve the defects of the image by always matching the center point of the camera combination for taking a stereo image.

According to an exemplary embodiment, a panoramic stereo photographing apparatus includes: a plurality of camera modules configured to acquire an image; And a control unit configured to generate a left eye panoramic image and a right eye panoramic image using the images acquired by the plurality of camera modules, and generate a stereo image using the left eye panoramic image and the right eye panoramic image.

Each of the plurality of camera modules may include a plurality of cameras arranged in different directions.

The controller selects a left eye camera and a right eye camera from each of the plurality of camera modules, generates the left eye panorama image by combining images of the selected left eye cameras, and generates the right eye panorama image by combining images of the selected right eye cameras. It can be configured to.

The controller may be further configured to determine a left eye camera and a right eye camera among cameras included in each camera module based on a relative position between the cameras included in each camera module and a reference point.

The plurality of camera modules may be symmetrically arranged with respect to a reference point, and each camera module may have the same camera arrangement as other camera modules disposed at symmetric positions with respect to each camera module with the reference point therebetween. .

According to an embodiment, a panorama stereo photographing method includes: capturing an image by a plurality of camera modules; Generating a left eye panoramic image and a right eye panoramic image by combining images of the plurality of camera modules by the panoramic stereo photographing apparatus; And generating a stereo image using the left eye panoramic image and the right eye panoramic image.

Each of the plurality of camera modules may include a plurality of cameras arranged in different directions. The generating of the left eye panoramic image and the right eye panoramic image may include selecting a left eye camera and a right eye camera from each of the plurality of camera modules; Combining the images of selected left eye cameras to generate the left eye panoramic image; And combining the images of the selected right eye cameras to generate the right eye panoramic image.

The selecting of the left eye camera and the right eye camera may include determining a left eye camera and a right eye camera among cameras included in each camera module based on a relative position between the cameras included in each camera module and a reference point. It may include.

The photographing of the image may include photographing the image using a plurality of camera modules symmetrically arranged with respect to a reference point. In this case, each camera module may have the same camera arrangement as another camera module disposed in a symmetrical position with respect to each camera module with the reference point therebetween.

According to an aspect of the present invention, the center of the left eye panoramic image and the right eye panoramic image are always aligned with the same reference point even if the line of sight corresponding to the stereo image changes. Therefore, in the conventional panoramic stereo photographing apparatus, defects may be prevented from occurring due to the limitation of the rig structure, and a panoramic stereo image may be obtained by using a relatively small number of cameras.

1 is a schematic configuration diagram of a conventional panoramic stereo imaging apparatus.

2 is a schematic block diagram of a panoramic stereo photographing apparatus according to an embodiment.

3 is a conceptual diagram illustrating a principle of generating a stereo image.

4 is a conceptual diagram illustrating the configuration of a panoramic image in the panoramic stereo photographing apparatus according to an embodiment.

5A to 5C are schematic views illustrating arrangement of camera modules in the panoramic stereo photographing apparatus according to another embodiment.

6 to 8 are schematic views showing the arrangement of camera modules in the panoramic stereo imaging apparatus according to another embodiment.

9 is a schematic flowchart of a panoramic stereo photographing method according to an exemplary embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

2 is a schematic block diagram of a panoramic stereo photographing apparatus according to an embodiment.

Referring to FIG. 2, the panoramic stereo photographing apparatus includes a plurality of camera modules 21-24 and a controller 20 for controlling the same. Each camera module 21-24 may include a plurality of cameras. In addition, the plurality of camera modules 21-24 may be symmetrically arranged with respect to the predetermined reference point 100. For example, when the panoramic stereo imaging apparatus according to the present embodiment is applied to a head-mounted display device (HMD), FIG. 2 corresponds to a plan view of the user wearing the HMD and the reference point 100. May correspond to the head position of the user.

In the drawings of the present specification, each of the cameras included in the camera modules 21-24 is shown in a fan shape, which is intended to conceptually indicate the shooting direction and angle of view of each camera, and the actual shape or size of the camera, or It should be understood that it does not represent the actual shooting area by the camera. Also, in the present specification, an image is intended to include both a single image such as a picture or a series of consecutive images such as an image. For example, a video including both an image and an audio may be included in the image described herein.

The cameras constituting each camera module 21-24 are arranged to face in different directions from each other, and are configured to capture images of different directions. That is, each camera module 21-24 may be one panoramic camera. Each camera module 21-24 may be a panoramic camera capable of 360-degree omnidirectional photographing, or may be a panoramic camera capable of capturing only some angles. For example, referring to the camera module 21, the camera module 21 may include a camera 211 disposed toward the lower left of the drawing and another camera 212 disposed toward the upper right of the drawing opposite thereto. have. However, this is only an example, and the number of cameras constituting each camera module 21 and its arrangement may be different from those shown in the drawings.

The plurality of camera modules 21-24 may be symmetrically arranged with respect to the reference point 100. In addition, each camera module 21-24 may have the same camera arrangement form as other camera modules 21-24 disposed symmetrically with respect to the reference point 100. For example, the camera module 21 of FIG. 2 has an arrangement in which two cameras 211 and 212 face left and right upper ends, respectively. In this case, the camera module 24 disposed symmetrically with the camera module 21 with the reference point 100 also has an arrangement in which two cameras 241 and 242 face the lower left and upper right sides of the drawing, respectively. The same applies to the cameras 221 and 222 of the camera module 22 and the cameras 231 and 232 of the camera module 24.

The controller 20 is electrically connected to the plurality of camera modules 21-24, and is configured to generate a panoramic stereo image using the images obtained from the plurality of camera modules 21-24. The control unit 20 may be directly connected to the plurality of camera modules 21-24 through a wired connection, or may be located at a remote location with the plurality of camera modules 21-24, and the plurality of camera modules 21 through wireless communication. -24 may be configured to receive and process the captured image. In one embodiment, the control unit 20 may be further configured to control operations by the plurality of camera modules 21-24 by transmitting control signals to the plurality of camera modules 21-24.

The controller 20 is configured to obtain a panoramic stereo image by using the images acquired by the plurality of camera modules 21-24. The controller 20 may generate a left eye panoramic image and a right eye panoramic image, respectively, by using the images acquired by the plurality of camera modules 21-24, and generate a stereo image by combining the generated left eye and right eye panoramic images. have. The stereo image generated as described above may include an omnidirectional stereo image, and a panorama stereo image may be obtained by presenting a specific portion of the stereo image according to the direction of the user's gaze. A detailed operation of the controller 20 will be described later with reference to FIG. 4.

3 is a conceptual diagram illustrating a principle of generating a stereo image.

Referring to FIG. 3, it is assumed that a line of sight corresponding to a stereo image is the upper right of the drawing. In this case, among the plurality of cameras 211, 212, 221, 222, 231, 232, 241, and 242 constituting the plurality of camera modules 21-24, two cameras arranged in a line of sight direction may be used. Stereo images can be generated with the right eye camera. If there is no camera arranged to match the gaze direction, the camera arranged in the direction closest to the gaze direction may be used. As described above, the camera modules 21-24 are symmetrically disposed with respect to the reference point 100 and the camera modules 21-24 disposed at the symmetrical positions have the same camera arrangement form. Thus, two cameras 212 and 241 facing in the eye direction belong to two camera modules 21 and 24 positioned at symmetrical positions with respect to the reference point 100, respectively.

Next, the left eye camera and the right eye camera are determined among the two cameras 212 and 241 and the reference point 100 in the eye direction. In FIG. 3A, the camera 212 is a left eye camera because the camera 212 is located on the left side when looking at the gaze direction from the reference point 100, and the camera 241 is located on the right side when looking at the gaze direction at the reference point 100. Becomes In this case, a stereo image may be generated by using the image photographed by the camera 212 as a left eye image and the image photographed by the camera 241 as a right eye image.

In the embodiments of the present invention, a stereo image is generated using the above principles, and includes a plurality of cameras 211, 212, 221, 222, 231, 232, 241, which constitute the plurality of camera modules 21-24. Select the left eye cameras and the right eye cameras from 242 and generate the left eye and right eye panorama images using the selected left eye and right eye cameras. It will be described in detail with reference to Figure 4 below.

4 is a conceptual diagram illustrating the configuration of a panoramic image in the panoramic stereo photographing apparatus according to an embodiment.

Referring to FIG. 4A, in the present exemplary embodiment, the controller may select one left eye camera from each camera module. For example, among the cameras constituting each camera module, the camera arranged to photograph the left direction when the camera module is viewed from the reference point 100 may be selected as the left eye camera. As a result, in FIG. 4A, the camera 211, the camera 221, the camera 232, and the camera 242 were selected as the left eye camera.

4B conceptually illustrates an image photographed by each of the cameras 211, 221, 232, and 242 selected as the left eye camera. In this case, the controller may configure the left eye panoramic image by combining the images photographed by the respective left eye cameras as illustrated in FIGS. 4C and 4D. The panomamara image generated as described above includes an omnidirectional left eye image. The above process may be similarly applied to cameras 212, 222, 231, and 241 not previously selected among the cameras constituting the camera module, thereby generating a right eye panoramic image.

The controller may generate a stereo image using the left eye panoramic image and the right eye panoramic image generated as described above. Since each of the left and right eye images constituting the stereo image is itself composed of a panoramic image, the generated stereo image includes the omnidirectional stereo image. By presenting a specific portion of the stereo image according to the direction of the user's gaze, a panoramic stereo image can be obtained.

In the embodiments described above with reference to FIGS. 1 to 4, four camera modules 21-24 are symmetrically positioned in the up, down, left, and right directions of the reference point 100 on a plane. However, in another embodiment, the number of camera modules is not limited thereto. For example, the panoramic stereo photographing apparatus may include more or fewer camera modules arranged with reference points interposed therebetween.

FIG. 5A is a schematic diagram showing the arrangement of the camera modules 51-53 in the panoramic stereo photographing apparatus according to an embodiment, and FIG. 5B is a schematic diagram of the right eye images obtained by the camera modules 51-53, and FIG. 5C. Is a schematic diagram of the left eye images obtained by the camera modules 51-53. In the present exemplary embodiment, the panoramic stereo photographing apparatus includes camera modules 51-53 and a controller (not shown), and the camera modules 51-53 are arranged in a triangle shape. For example, the camera modules 51-53 may be disposed at each vertex position of the equilateral triangle, and the center of the equilateral triangle may correspond to the reference point. Like the above-described embodiments, each of the camera modules 51-53 may function as one panoramic camera.

A right eye panoramic image is generated using the right eye images shown in FIG. 5B in the same manner as the embodiment of FIG. 4, and a left eye panoramic image is generated in the same manner as the embodiment of FIG. 4 using the left eye images shown in FIG. 5C. can do. In addition, a panoramic stereo image may be generated using the generated right eye panoramic image and the left eye panoramic image.

In another embodiment, the panoramic stereo photographing apparatus may include a plurality of camera modules and a controller (not shown) disposed three-dimensionally in space as illustrated in FIGS. 6 to 8. In the embodiment of FIG. 6, eight camera modules 61-68 are disposed at each vertex position of the cube, in the embodiment of FIG. 7, six camera modules 71-76 are disposed at each vertex position of the octahedron, In the embodiment of FIG. 8, four camera modules 81-84 are disposed at each vertex position of the tetrahedron.

Like the above-described embodiments, each of the camera modules 61-68, 71-76 and 81-84 functions as one panoramic camera. Thus, using the camera modules 61-68, 71-76 and 81-84, the left eye and right eye panorama images are respectively generated in the same manner as described above with reference to FIG. 4, and the left eye and right eye panorama images are used. To create a panoramic stereo image.

6 to 8 merely illustrate an arrangement of three-dimensional camera modules, and the arrangement of camera modules in the panoramic stereo imaging apparatus according to the embodiments is not limited thereto, and is not illustrated herein. Other different forms.

9 is a schematic flowchart of a panoramic stereo photographing method according to an exemplary embodiment. The panorama stereo imaging method according to the present embodiment may be performed by the above-described panorama stereo imaging apparatus.

9, an image may be acquired using a plurality of camera modules (S101). Next, one left eye camera and one right eye camera may be determined in each camera module (S102). Specifically, when the camera module is viewed from the reference point among the cameras constituting each camera module, the cameras disposed on the left side may be selected as the left eye camera, and the cameras disposed toward the right side may be selected as the right eye camera. . Next, a left eye panoramic image may be generated using the selected left eye cameras (S103), and a right eye panoramic image may be generated using the selected right eye cameras (S104). Finally, a panoramic stereo image may be generated using the left eye panoramic image and the right eye panoramic image (S105).

According to embodiments of the present invention, the left eye panoramic image and the right eye panoramic image of the whole orientation may be generated using the left eye camera and the right eye camera selected from each camera module, respectively. In addition, a panoramic stereo image may be obtained by combining the left eye and right eye panorama images thus generated. Therefore, a panoramic stereo image in the corresponding gaze direction may be obtained regardless of the direction of the user's gaze.

Although the present invention described above has been described with reference to the embodiments shown in the drawings, this is merely exemplary, and it will be understood by those skilled in the art that various modifications and variations may be made therefrom. However, such modifications should be considered to be within the technical protection scope of the present invention. Therefore, the true technical protection scope of the present invention will be defined by the technical spirit of the appended claims.

Embodiments are directed to an apparatus and method for panoramic stereo imaging, and more particularly, to an apparatus and method for acquiring a stereoscopic stereo image by combining a plurality of panoramic cameras.

Claims (9)

A plurality of camera modules configured to acquire an image; And And a control unit configured to generate a left eye panoramic image and a right eye panoramic image by using the images acquired by the plurality of camera modules, and generate a stereo image using the left eye panoramic image and the right eye panoramic image. . The method of claim 1, Each of the plurality of camera modules includes a plurality of cameras arranged toward different directions from each other. The method of claim 2, The controller selects a left eye camera and a right eye camera from each of the plurality of camera modules, generates the left eye panorama image by combining images of the selected left eye cameras, and generates the right eye panorama image by combining images of the selected right eye cameras. Panoramic stereo shooting device. The method of claim 3, wherein The controller is further configured to determine a left eye camera and a right eye camera among cameras included in each camera module based on a relative position between the cameras included in each camera module and a reference point. The method of claim 1, The plurality of camera modules are arranged symmetrically with respect to the reference point, each camera module has the same camera arrangement as the other camera module disposed in a symmetrical position with respect to each camera module with the reference point in between . Photographing an image by a plurality of camera modules; Generating a left eye panoramic image and a right eye panoramic image by combining images of the plurality of camera modules by the panoramic stereo photographing apparatus; And And generating a stereo image using the left eye panoramic image and the right eye panoramic image. The method of claim 6, Each of the plurality of camera modules includes a plurality of cameras arranged in different directions from each other, Generating the left eye panoramic image and the right eye panoramic image, Selecting a left eye camera and a right eye camera from each of the plurality of camera modules; Combining the images of selected left eye cameras to generate the left eye panoramic image; And And combining the images of the selected right eye cameras to generate the right eye panoramic image. The method of claim 7, wherein The selecting of the left eye camera and the right eye camera may include determining a left eye camera and a right eye camera among cameras included in each camera module based on a relative position between the cameras included in each camera module and a reference point. Panoramic stereo shooting method that includes. The method of claim 6, The photographing of the image may include photographing an image using a plurality of camera modules arranged symmetrically with respect to a reference point. And each camera module has the same camera arrangement as other camera modules disposed in symmetrical positions with respect to each camera module with the reference point therebetween.

Images