WO2024150720A1 - Three-dimensional information acquisition device - Google Patents

Abstract

This three-dimensional information acquisition device comprises: a first distance image acquisition unit that acquires a first distance image which includes a plurality of distance values obtained in accordance with the time from when light was emitted until light reflected off of an object was received; a second distance image acquisition unit that acquires a second distance image which is obtained by emitting light onto the same object as in the first distance image and by emitting light at a different emission angle than in the first distance image; and a generation unit that determines the impact of a multipath accompanying the reflection of light on the basis of the first distance image and the second distance image and that generates three-dimensional information of the object, the three-dimensional information being composited from the first distance image and the second distance image on the basis of the result of said determination.

Description

Three-dimensional information acquisition device

The present invention relates to a three-dimensional information acquisition device.
This application claims priority to Japanese Patent Application No. 2023-002476 filed in Japan on January 11, 2023, Japanese Patent Application No. 2023-002483 filed in Japan on January 11, 2023, and Japanese Patent Application No. 2023-002486 filed in Japan on January 11, 2023. The contents of which are incorporated herein by reference.

Conventionally, a method for measuring three-dimensional shapes is known in which near-infrared light is irradiated onto an object, and the distance to the object is measured based on the time of flight of the light reflected from the object. A ToF (Time of Flight) camera is known as a device using this measurement method. When using this ToF camera, the light received by the sensor is a mixture of direct light reflected directly from the object and indirect light reflected from an object other than the object (for example, a wall located behind the object). When an object with high near-infrared light reflectance is present near the object, there is a problem in that an error occurs in the distance to the object due to the effect of disturbance reflection (so-called multipath), and distortion occurs when trying to reproduce the three-dimensional shape of the object based on the measurement results. In order to reduce the effect of such multipath, a measurement technology using a specular reflector has been proposed (for example, see Patent Document 1).

JP 2019-15706 A

However, when using the above-mentioned technology, even if you want to measure both the object and the background, it is necessary to place a specular reflector around the object. Because placing a specular reflector hides the background, it is not possible to use the above-mentioned technology when measuring the three-dimensional shapes of both the object and the background. Furthermore, when combining a ToF camera and an RGB camera to obtain point cloud data that also includes color information of the object, the inability to obtain an image of the background is a problem.

The present invention was made in consideration of these circumstances, and aims to provide a distance measurement technology that can reduce the effects of multipath.

[1] One aspect of this embodiment is a three-dimensional information acquisition device that includes a first distance image acquisition unit that acquires a first distance image, which is a distance image including multiple distance values obtained according to the time between irradiating light and receiving the light reflected from an object; a second distance image acquisition unit that acquires a second distance image, which is a distance image obtained by irradiating light on the same object as the first distance image, but by irradiating light at a different irradiation angle from the first distance image; and a generation unit that determines the effect of multipath caused by light reflection based on the first distance image and the second distance image, and generates three-dimensional information of the object by combining the first distance image and the second distance image based on the determination result.

[2] Also, in one aspect of this embodiment, in the three-dimensional information acquisition device described in [1] above, the distance image has distance values corresponding to each coordinate in a two-dimensional coordinate system, and the generation unit compares the distance values contained in the first distance image and the distance values contained in the second distance image at the same coordinate, and determines the effect of multipath due to light reflection based on the comparison result.

[3] In one aspect of this embodiment, the three-dimensional information acquisition device described in [1] or [2] above further includes an irradiation unit that irradiates light to acquire the distance image, and the irradiation unit is capable of changing the direction in which the light is irradiated depending on the position of the target object.

[4] In one aspect of this embodiment, the three-dimensional information acquisition device described in [3] above further includes a visible light image acquisition unit that acquires a visible light image of the object, and the irradiation unit is capable of varying the direction of light irradiation according to the position of the object obtained by analyzing the visible light image.

[5] In addition, in one aspect of this embodiment, the three-dimensional information acquisition device described in any one of [1] to [4] above further includes an irradiation unit that irradiates light to acquire the distance image, and the irradiation unit further includes a light-emitting unit that emits light and a diffusion unit that diffuses the light emitted by the light-emitting unit, and irradiates light at different irradiation angles by varying the distance between the light-emitting unit and the diffusion unit.

The reference examples are as follows:

[A1] One reference aspect of this embodiment is a three-dimensional information acquisition device that includes a first distance image acquisition unit that acquires a first distance image, which is a distance image including multiple distance values obtained according to the time from when light is irradiated to when the light reflected off the object and the background is received; a second distance image acquisition unit that acquires a second distance image, which is a distance image obtained by irradiating light on the same object as the first distance image, and which is obtained using a background that has a lower near-infrared reflectance than the background in the first distance image; and a generation unit that determines the effect of multipath caused by light reflection based on the first distance image and the second distance image, and generates three-dimensional information of the object by combining the first distance image and the second distance image based on the determination result.

[A2] In one aspect, in the three-dimensional information acquisition device described in [A1] above, the distance image has distance values corresponding to each coordinate in a two-dimensional coordinate system, and the generation unit compares the distance values contained in the first distance image and the distance values contained in the second distance image at the same coordinate, and determines the effect of multipath caused by light reflection based on the comparison result.

[A3] In one aspect, the three-dimensional information acquisition device described in [A1] or [A2] above further includes a first visible light image acquisition unit that acquires a first visible light image, which is a visible light image capturing an image of the object and background corresponding to the first distance image, and a second visible light image acquisition unit that acquires a second visible light image, which is a visible light image capturing an image of the object and background corresponding to the second distance image, and the first visible light image and the second visible light image have different imaging conditions for determining the brightness of the visible light images.

[A4] In one aspect, the three-dimensional information acquisition device described in [A1] or [A2] above further includes a first visible light image acquisition unit that acquires a first visible light image, which is a visible light image of the object and background corresponding to the first distance image; a second visible light image acquisition unit that acquires a second visible light image, which is a visible light image of the object and background corresponding to the second distance image; and a visible light image generation unit that generates pixel information to be assigned to the three-dimensional information of the object based on the acquired first visible light image and the acquired second visible light image, and the visible light image generation unit generates the pixel information according to a result of comparing pixel values of the acquired first visible light image and pixel values of the acquired second visible light image.

[A5] Also, one aspect is a three-dimensional information acquisition system comprising a three-dimensional information acquisition device described in any one of [A1] to [A5] above, and an infrared absorbing sheet that is used when acquiring the second distance image and that is a background having low reflectance of near-infrared rays.

[B1] One reference aspect of this embodiment is a three-dimensional information acquisition device that includes a first distance image acquisition unit that acquires a first distance image, which is a distance image including a plurality of distance values obtained according to the time between irradiating light and receiving the light reflected by an object; a second distance image acquisition unit that acquires a second distance image, which is a distance image obtained by irradiating light onto a part of the object that is the same as the first distance image, with light at an irradiation angle different from that of the first distance image, and which is obtained by enlarging a part of the object that is the same as the first distance image with an optical member; and a generation unit that determines the effect of multipath caused by light reflection based on the first distance image and the second distance image, and generates three-dimensional information of the object by combining the first distance image and the second distance image based on the determination result.

[B2] In one aspect, in the three-dimensional information acquisition device described in [B1] above, the first distance image and the second distance image are acquired while keeping the optical axis fixed.

[B3] In one aspect, the three-dimensional information acquisition device described in [B1] or [B2] above further includes a visible light image acquisition unit that acquires a visible light image of the object, and the magnification achieved by the optical member is based on the visible light image acquired by the visible light image acquisition unit.

[B4] In one aspect, in the three-dimensional information acquisition device described in [B3] above, the visible light image acquisition unit acquires a first visible light image, which is a visible light image having substantially the same angle of view as the first distance image, and a second visible light image, which is a visible light image having substantially the same angle of view as the second distance image, and the irradiation angle of the light irradiated to acquire the distance image is determined according to the angle of view of the visible light image.

[B5] In one aspect, the three-dimensional information acquisition device described in [B4] above further includes a visible light image generation unit that generates pixel information to be added to the three-dimensional information of the object based on the acquired first visible light image and the acquired second visible light image, and the visible light image generation unit complements pixel values at coordinates in the first visible light image that have a lower resolution than the second visible light image using a predetermined method.

According to this embodiment, the effects of multipath can be reduced and the distance to the target object can be measured.

1 is a diagram for explaining an overview of a three-dimensional information acquisition device according to a first embodiment. FIG. 2 is a functional configuration diagram showing an example of a functional configuration of a three-dimensional information acquisition device according to the first embodiment. FIG. 4A to 4C are diagrams for explaining a first irradiation by the three-dimensional information acquisition device according to the first embodiment. 5A to 5C are diagrams for explaining a second irradiation by the three-dimensional information acquisition device according to the first embodiment. FIG. 2 is a second diagram for explaining the first irradiation by the three-dimensional information acquisition device according to the first embodiment. FIG. 2 is a second diagram for explaining the second irradiation by the three-dimensional information acquisition device according to the first embodiment. 5A to 5C are diagrams for explaining a method of changing an irradiation angle in the three-dimensional information acquisition device according to the first embodiment. FIG. 11 is a functional configuration diagram showing an example of a functional configuration of a three-dimensional information acquisition device according to a second embodiment. 11A and 11B are diagrams for explaining first and second irradiation by a three-dimensional information acquisition device according to a second embodiment. 13A to 13C are diagrams illustrating an example of a visible light image obtained by a three-dimensional information acquisition device according to a second embodiment when a first background is used and a visible light image obtained by a three-dimensional information acquisition device according to a second embodiment when a second background is used. 13A to 13C are diagrams showing an example of point cloud data when a first distance image is used and a second distance image is used by a three-dimensional information acquisition device according to a second embodiment. FIG. 11 is a schematic diagram showing an example of a cross section of a three-dimensional information acquisition device according to a third embodiment. FIG. 11 is a functional configuration diagram showing an example of a functional configuration of a three-dimensional information acquisition device according to a third embodiment. 13A and 13B are diagrams showing an example of point cloud data obtained by combining a distance image and a visible light image at the wide-angle end captured by the three-dimensional information acquisition device of the third embodiment. 13A and 13B are diagrams showing an example of point cloud data obtained by combining a distance image and a visible light image at the telephoto end by the three-dimensional information acquisition apparatus according to the third embodiment. 13A and 13B are diagrams showing an example of point cloud data obtained by combining a distance image and a visible light image at the wide-angle end and a distance image and a visible light image at the telephoto end using a three-dimensional information acquisition device according to a third embodiment. 13A to 13C are diagrams for explaining pixel value complementation by a three-dimensional information acquisition device according to a third embodiment.

Below, an embodiment of the present invention will be described with reference to the drawings. The embodiment described below is merely an example, and the embodiment to which the present invention is applied is not limited to the following embodiment. Furthermore, "based on XX" in this application means "based on at least XX," and includes cases where it is based on other elements in addition to XX. Furthermore, "based on XX" is not limited to cases where XX is used directly, but also includes cases where it is based on XX that has been subjected to calculations or processing. "XX" is any element (for example, any information).

[Embodiment 1]
First, a first embodiment will be described with reference to FIGS.

FIG. 1 is a diagram for explaining an outline of a three-dimensional information acquisition device according to embodiment 1. With reference to the same figure, an outline of a three-dimensional information acquisition device 10 will be explained. The three-dimensional information acquisition device 10 acquires three-dimensional information of an object T existing in a three-dimensional space. The three-dimensional information acquired by the three-dimensional information acquisition device 10 includes at least information on the three-dimensional shape of the object T. The three-dimensional information acquisition device 10 measures the distance L1 from the three-dimensional information acquisition device 10 to the object T. The three-dimensional information acquisition device 10 acquires the three-dimensional shape of the object T by measuring the distance L1 to the object T at each coordinate of a two-dimensional coordinate system. The object T targeted by the three-dimensional information acquisition device 10 includes anything that is the subject of acquisition of three-dimensional information, such as animals and objects. Behind the object T, there is a background BG. The background BG may be, for example, a screen such as a green screen used for shooting, or may be a simple wall, floor, building, natural object, or other background that is reflected during normal shooting. The three-dimensional information acquisition device 10 may be used, for example, by a professional photographer in an indoor photography studio, or by a general user in a situation similar to normal photography, indoors or outdoors.

The three-dimensional information acquisition device 10 includes a light receiving unit 110, an irradiation unit 120, and a control unit 130. The light receiving unit 110 may include an optical lens such as an objective lens. The irradiation unit 120 irradiates the object T with irradiation light. The irradiation unit 120 may be a light source such as a laser diode. More specifically, the irradiation unit 120 may be a VCSEL (Vertical Cavity Surface Emitting Laser) capable of emitting a laser beam in the vertical direction. The light irradiated by the irradiation unit 120 (first irradiation light BM1) is reflected by the object T, and the light reflected by the object T (second irradiation light BM2) is incident on the light receiving unit 110. The control unit 130 measures the distance L1 from the three-dimensional information acquisition device 10 to the object T according to the time (flight time) between emitting and receiving light. The light receiving unit 110 has a plurality of light receiving elements (not shown) arranged in a two-dimensional array. The control unit 130 measures the three-dimensional shape of the object T according to the distance measurement information of each of the plurality of light receiving elements. In the illustrated example, a pair of light receiving units 110 and irradiating units 120 is shown, but a configuration in which a plurality of irradiating units 120 are provided for one light receiving unit 110 may also be used. In this case, it is preferable that the plurality of irradiating units 120 are provided in the vicinity of the light receiving unit 110, with the light receiving unit 110 at the center.

The three-dimensional information acquisition device 10 may be equipped with an image sensor (not shown). The image sensor has a plurality of pixels arranged in a two-dimensional array, and the plurality of pixels of the image sensor receive visible light imaged by an optical lens provided in the light receiving unit 110, and generate a visible light image (RGB image) based on the received information. The visible light image format is not limited to the RGB format, and may be a YCbCr format or a grayscale (monochrome) format. The light (e.g., near-infrared light) irradiated by the irradiation unit 120 and reflected by the object T and the visible light for generating the visible light image may pass along the same optical axis and enter the light receiving unit 110. In this case, the three-dimensional information acquisition device 10 may be equipped with a visible light reflecting dichroic film (not shown) or the like, and may be split into two parts: near-infrared light that passes through the visible light reflecting dichroic film and visible light that is reflected by the visible light reflecting dichroic film. In this case, the light receiving element for distance measurement is placed on the optical axis of the light passing through the visible light reflecting dichroic film, and the image sensor for generating a visible light image is placed on the optical axis of the light reflected by the visible light reflecting dichroic film.

FIG. 2 is a functional configuration diagram showing an example of the functional configuration of the three-dimensional information acquisition device according to the first embodiment. With reference to the diagram, an example of the functional configuration of the three-dimensional information acquisition device 10 will be described. The three-dimensional information acquisition device 10 includes a light receiving unit 110, an irradiation unit 120, and a control unit 130.

The irradiation unit 120 includes a first irradiation unit 121 and a second irradiation unit 122. The first irradiation unit 121 and the second irradiation unit 122 irradiate irradiation light having different irradiation angles. The light receiving unit 110 receives the reflected light of the irradiation light irradiated by either the first irradiation unit 121 or the second irradiation unit 122 and reflected by the object T or the background BG. Here, the irradiation angle of the first irradiation unit 121 is different from the irradiation angle of the second irradiation unit 122. Specifically, the irradiation angle of the first irradiation unit 121 is wider than the irradiation angle of the second irradiation unit 122. The irradiation angle of the first irradiation unit 121 and the irradiation angle of the second irradiation unit 122 will be described with reference to FIG. 3 and FIG. 4. In the following description, the term "irradiation angle" may be used synonymously with "field of view (FoV)".

FIG. 3 is a diagram for explaining the first irradiation by the three-dimensional information acquisition device according to embodiment 1. The first irradiation performed by the first irradiation unit 121 will be explained with reference to the figure. The figure is a diagram of the arrangement in FIG. 1 as viewed from directly above. In the example shown in the figure, the three-dimensional information acquisition device 10 measures the three-dimensional shape of an object T that is in front of a background BG. The three-dimensional information acquisition device 10 includes a light receiving unit 110, a first irradiation unit 121, and a second irradiation unit 122. The three-dimensional information acquisition device 10 includes a first irradiation unit 121-1 and a first irradiation unit 121-2 as the first irradiation unit 121. The irradiation angle by the first irradiation unit 121 is shown as angle α. For example, the light irradiated by the first irradiation unit 121 and reflected by the object T (the path of the light is described as irradiation light BM1-1) is not affected by multipath. However, depending on the light irradiated by the first irradiating unit 121 and reflected off the background BG (the path of this light is referred to as irradiated light BM1-2), the reflected light from the background BG may be further reflected by the object T, etc., and such light is affected by multipath. Due to the effect of multipath, the shape of the object T is actually as shown by the dotted line, but is measured as the shape shown by the solid line. This is because when affected by multipath, the Depth value (a value indicating the distance from the three-dimensional information acquisition device 10 to the object T) is measured as being larger (farther) than the actual value.

In this embodiment, for convenience, the first irradiation unit 121 is described as being divided into first irradiation unit 121-1 and first irradiation unit 121-2, but one first irradiation unit 121 may irradiate light having an irradiation angle of angle α. When two first irradiation units 121 are provided as shown in the figure, it is preferable that the two first irradiation units 121 are provided at positions symmetrical to each other with the light receiving unit 110 at the center.

4 is a diagram for explaining the second irradiation by the three-dimensional information acquisition device according to the first embodiment. The second irradiation performed by the second irradiation unit 122 will be explained with reference to the same figure. The same figure is a diagram of the arrangement in FIG. 1 as seen from directly above. In the example shown in the same figure, the three-dimensional information acquisition device 10 measures the three-dimensional shape of the object T that is in front of the background BG. The three-dimensional information acquisition device 10 includes a light receiving unit 110, a first irradiation unit 121, and a second irradiation unit 122. The irradiation angle by the second irradiation unit 122 is shown as angle β. The angle β is narrower than the angle α, and is an angle at which the object T fits just within the angle of view. The path of the light irradiated by the second irradiation unit 122 and reflected by the object T is described as irradiation light BM1-3. The angle β is narrower than the angle α, and is an angle at which the object T fits just within the angle of view, so less light reaches the background BG and less light reaches the light receiving unit 110 through multiple paths. Therefore, the second irradiation can reduce the influence of multipath. In the shape measurement of the object T based on the light irradiated by the second irradiation unit 122, the influence of multipath is reduced, so that the shape of the object T is originally as shown by the dotted line, but the measurement result is also as shown by the solid line. In this way, the second irradiation can accurately measure the shape of the object T. In the illustrated example, since it is a view from directly above, only one second irradiation unit 122 is visible, but a second irradiation unit 122 (not shown) may be provided at a position facing the second irradiation unit 122 across the light receiving unit 110 (which can also be said to be below the light receiving unit 110). In other words, it can be said that the first irradiation unit 121 is arranged in the left-right direction across the light receiving unit 110, and the second irradiation unit 122 is arranged in the up-down direction across the light receiving unit 110.

In this embodiment, for convenience, the irradiation units that irradiate irradiation light at different irradiation angles are described as being divided into the first irradiation unit 121 and the second irradiation unit 122, but irradiation light at different irradiation angles may be irradiated by a single irradiation unit. In other words, the first irradiation unit 121 and the second irradiation unit 122 may be the same unit.

Returning to FIG. 2, the functional configuration of the control unit 130 will be described. The control unit 130 controls the light irradiation by the irradiation unit 120 and the exposure of the light receiving unit 110, and acquires information on the light received by the light receiving unit 110. Based on the information on the light irradiation and light reception, the control unit 130 measures the distance to the object T at each coordinate in the two-dimensional coordinate system, and acquires the three-dimensional shape of the object T.

The control unit 130 includes a ToF sensor control unit 131, a first distance image acquisition unit 132, a second distance image acquisition unit 133, and a three-dimensional information generation unit 134. Each of these functional units is realized, for example, using electronic circuits. Furthermore, each functional unit may include internal storage means such as a semiconductor memory or a magnetic hard disk device as necessary. Furthermore, each function may be realized by a computer and software.

The ToF sensor control unit 131 controls the irradiation of light by the irradiation unit 120. Specifically, the ToF sensor control unit 131 controls the timing of the irradiation of light by the first irradiation unit 121 by outputting the first control information CI1, and controls the timing of the irradiation of light by the second irradiation unit 122 by outputting the second control information CI2. The ToF sensor control unit 131 also controls the exposure when the light receiving unit 110 receives the reflected light when the irradiation unit 120 irradiates by outputting the third control information CI3. As a result, the ToF sensor control unit 131 acquires information about the reflected light by acquiring the received light information RI from the light receiving unit 110. The received light information RI may include information about the distance to the object T from which the reflected light is acquired according to each coordinate of the two-dimensional coordinates. The received light information RI may include each piece of information acquired by the light receiving unit 110 through exposure, or may include infrared light information and distance information for each coordinate acquired by signal processing based on the reflected light acquired by the light receiving unit 110 through multiple exposures. The infrared light information and distance information may be obtained by the ToF sensor control unit 131 performing signal processing based on information about the reflected light obtained from the light receiving unit 110.

The ToF sensor control unit 131 generates a distance image based on the acquired light reception information RI. The distance image includes multiple distance values obtained according to the time from when light is emitted to when the light reflected by the object is received. The multiple distance values correspond to a two-dimensional coordinate system. The distance image can also be called a depth image obtained by the ToF sensor. Specifically, the ToF sensor control unit 131 generates a first distance image DI1 based on the reflected light obtained as a result of irradiation by the first irradiation unit 121 by outputting the first control information CI1. In addition, the ToF sensor control unit 131 generates a second distance image DI2 based on the reflected light obtained as a result of irradiation by the second irradiation unit 122 by outputting the second control information CI2. Here, the irradiation by the first irradiation unit 121 and the irradiation by the second irradiation unit 122 are performed on the same object T. In addition, the irradiation angle of the irradiation by the first irradiation unit 121 is different from the irradiation angle of the irradiation by the second irradiation unit 122. In other words, the second distance image DI2 is a distance image obtained by irradiating the same object T as the first distance image DI1 with light, and can also be said to be a distance image obtained by irradiating light at a different illumination angle than the first distance image DI1.

The first distance image acquisition unit 132 acquires the first distance image DI1 generated by the ToF sensor control unit 131. The first distance image acquisition unit 132 outputs the acquired first distance image DI1 to the three-dimensional information generation unit 134.

The second distance image acquisition unit 133 acquires the second distance image DI2 generated by the ToF sensor control unit 131. The second distance image acquisition unit 133 outputs the acquired second distance image DI2 to the three-dimensional information generation unit 134.

The three-dimensional information generating unit 134 acquires the first distance image DI1 from the first distance image acquiring unit 132, and acquires the second distance image DI2 from the second distance image acquiring unit 133. The three-dimensional information generating unit 134 generates three-dimensional information of the object T based on the acquired first distance image DI1 and second distance image DI2. Here, the irradiation angle of the light irradiated to obtain the first distance image DI1 is different from the irradiation angle of the light irradiated to obtain the second distance image DI2, specifically, the irradiation angle of the light irradiated to obtain the first distance image DI1 is wider. Since the irradiation angle of the light irradiated to obtain the first distance image DI1 is wide, it can be said that the first distance image DI1 is affected by multipath. On the other hand, since the irradiation angle of the light irradiated to obtain the second distance image DI2 is narrower than the irradiation angle of the first distance image DI1 (ideally because of an angle that is irradiated only to the size or range of the object T), the second distance image DI2 has reduced effects of multipath. However, according to the second distance image DI2, the area outside the object T does not have a distance value (depth value) (value is 0). Therefore, there are areas where the second distance image DI2 does not provide a distance value. The three-dimensional information generating unit 134 generates three-dimensional information of the object T using the distance value of the first distance image DI1 for areas where the second distance image DI2 does not provide a distance value. The three-dimensional information of the object T generated by the three-dimensional information generating unit 134 has reduced effects of multipath caused by light reflection compared to the first distance image DI1. Furthermore, the three-dimensional information of the object T generated by the three-dimensional information generating unit 134 also includes information about the periphery of the object T (e.g., background BG) compared to the second distance image DI2. In the following description, the three-dimensional information generating unit 134 may be simply referred to as a generating unit.

When the three-dimensional information generating unit 134 can obtain distance values from both the first distance image DI1 and the second distance image DI2, the three-dimensional information generating unit 134 may determine which distance value to use depending on the result of comparing the distance value of the first distance image DI1 with the distance value of the second distance image DI2. In other words, the influence of multipath on the three-dimensional information of the object T is determined based on the result of comparing the distance value of the first distance image DI1 with the distance value of the second distance image DI2, and which distance value to use is determined depending on the determination result. Here, the first distance image DI1 and the second distance image DI2 each have a distance value corresponding to each coordinate of the two-dimensional coordinate system. Therefore, the three-dimensional information generating unit 134 may compare the distance value included in the first distance image DI1 with the distance value included in the second distance image DI2 at the same coordinate, and generate three-dimensional information of the object T depending on the comparison result.

"Depending on the result of the comparison" may mean determining that there is a multipath effect when the difference in distance values exceeds a certain amount of variation due to the sensor's noise characteristics, depth accuracy, individual variation, temperature characteristics, etc. For example, when the depth accuracy is ±2%, the three-dimensional information generating unit 134 may determine that there is a multipath effect if the comparison result is ±5% or ±10%. For coordinates determined to be affected by multipath, the distance value of the second distance image DI2 may be used.

The three-dimensional information generating unit 134 may determine that multipath has occurred and perform correction if the distortion of the object due to multipath is visibly unnatural. The determination of whether the distortion of the object due to multipath is visibly unnatural may be based on the size of the object T. For example, if the size of the object T is a circle with a diameter of 100 mm (millimeters), then a distortion of 5 mm is tolerated if the threshold is 5%, and 10 mm is tolerated if the threshold is 10%. In this way, the threshold may be set based on the allowable distortion.

As described above, in the case of a configuration in which a visible light image can be acquired using a visible light reflecting dichroic film or the like, a visible light image acquisition unit (not shown) may be further provided to acquire a visible light image of the object T, and the three-dimensional information generation unit 134 may determine which distance value of the first distance image DI1 or the second distance image DI2 to use based on the acquired visible light image. By using the visible light image, it is possible to detect the edge (contour) of the object T by performing image analysis such as differentiation of pixel values and frequency analysis. Furthermore, according to the distance image, a position where there is a certain amount of difference in distance value can be detected as the edge of the object T. Therefore, when the edge based on the visible light image and the edge based on the distance image are different, it can be determined that there is an effect of multipath. In such a case, the three-dimensional information generation unit 134 can use the distance value of the first distance image DI1 for the outside of the edge of the object T based on the edge of the object T in the visible light image, and the distance value of the second distance image DI2 for the inside of the edge of the object T.

In the above example, it is assumed that the object T is located at the center of the angle of view. However, this embodiment can also be applied to cases where the object T is not located at the center of the angle of view. Next, with reference to Figures 5 and 6, an example of a case where the object T is not located at the center of the angle of view will be described.

5 is a second diagram for explaining the first irradiation by the three-dimensional information acquisition device according to the first embodiment. With reference to the diagram, the first irradiation by the three-dimensional information acquisition device 10 when the object T is not in the center of the angle of view will be explained. The diagram shows a state in which the position of the object T has moved from the center of the angle of view to the left, as compared with the arrangement in FIG. 3. Even in this state, the object T is present within the range of the angle α. In the illustrated example, as in the situation in FIG. 3, the light irradiated by the first irradiating unit 121 and reflected by the object T (the path of this light is described as irradiated light BM1-1) is not affected by multipath. However, depending on the light irradiated by the first irradiating unit 121 and reflected by the background BG (the path of this light is described as irradiated light BM1-2), the reflected light from the background BG may be further reflected by the object T, etc., and such light is affected by multipath.

6 is a second diagram for explaining the second irradiation by the three-dimensional information acquisition device according to the first embodiment. With reference to this diagram, the second irradiation by the three-dimensional information acquisition device 10 when the object T is not in the center of the angle of view will be explained. This diagram shows a state in which the position of the object T has moved from the center of the angle of view to the left, as compared to the arrangement in FIG. 4. In this state, if light with a narrow irradiation angle is irradiated to the center of the angle of view as in FIG. 4, the object T will not fit within the range of angle β. According to this embodiment, the angle at which the irradiation unit 120 irradiates light is changed to match the object T, thereby fitting the object T within the narrow angle of view. Specifically, by making the direction in which the second irradiation unit 122 faces variable, the irradiation unit 120 can change the direction in which light is irradiated depending on the position of the object T.

As described above, in the case of a configuration in which a visible light image can be further acquired using a visible light reflecting dichroic film or the like, the irradiation unit 120 may change the direction in which the second irradiation unit 122 faces based on the visible light image. In this case, the three-dimensional information acquisition device 10 further includes a visible light image acquisition unit that acquires a visible light image obtained by capturing an image of the object T, and the irradiation unit 120 can change the direction in which light is irradiated according to the position of the object T obtained by analyzing the acquired visible light image. The direction in which light is irradiated by the irradiation unit 120 may be, for example, by performing object detection processing on the acquired visible light image, and controlling the irradiation direction angle and irradiation range angle based on the center point of the range in which it is determined that an object exists. In addition, when the object T is a person, for example, face recognition may be performed by image processing the visible light image, and the irradiation direction angle and irradiation range angle may be controlled based on the center point of the range recognized as a face. This makes it possible to generate three-dimensional information on the face of a person with reduced effects of multipath.

In the above example, the irradiation unit 120 is provided with a first irradiation unit 121 with a wide irradiation angle and a second irradiation unit 122 with a narrow irradiation angle, and the first irradiation and the second irradiation are switched by irradiating light from either one of them. With a VCSEL that is normally available on the market, the position of the diffuser is fixed from the viewpoint of eye safety, and it is difficult to change the irradiation angle. Therefore, in order to change the irradiation angle, it is necessary to prepare a VCSEL with a different irradiation angle in advance, and it is considered to adopt the embodiment described above. However, if it becomes possible to easily change the irradiation angle by making the distance between the diffuser and the VCSEL variable, it is possible to realize a configuration similar to the above embodiment with one VCSEL. Below, an example of a case where a configuration similar to the above embodiment is realized with one VCSEL will be described with reference to FIG. 7.

FIG. 7 is a diagram for explaining a method of changing the irradiation angle in the three-dimensional information acquisition device according to the first embodiment. The method of changing the irradiation angle will be explained with reference to the same figure. The illustrated irradiation unit 140 is a modified example of the irradiation unit 120. The same figure shows an example of the internal configuration of the irradiation unit 140. The irradiation unit 140 is a member for emitting light to acquire a distance image. The irradiation unit 140 is configured to include a light emitting unit 141 and a diffusion unit 142. The light emitting unit 141 is a member that emits light and may be, for example, a VCSEL. The diffusion unit 142 is a member that diffuses the light emitted by the light emitting unit 141 and may be, for example, a diffuser. The irradiation unit 140 irradiates light at different irradiation angles by varying the distance between the light emitting unit 141 and the diffusion unit 142.

7(A) to 7(C) show different distances between the light emitting unit 141 and the diffusion unit 142. According to FIG. 7(A) to FIG. 7(C), the irradiation unit 140 emits light at different irradiation angles. For example, the diffusion unit 142 may be configured to shift in the front-rear direction (up-down direction indicated by the double-headed arrow in the figure) relative to the fixed light emitting unit 141, thereby making the distance between the light emitting unit 141 and the diffusion unit 142 variable. In this case, the ToF sensor control unit 131 may use the shift amount of the diffusion unit 142 to calculate the distance value, and may perform correction. Note that this correction may be ignored if it is within a range that can be ignored at the speed of light. From the viewpoint of eye safety, it is preferable that the front-rear position shift mechanism of the diffusion unit 142 is configured so as not to come off easily. In the example shown in FIG. 7(A), the distance between the light emitting unit 141 and the diffusion unit 142 is distance L41, which is the shortest among the distances shown in the figure. In this case, the irradiation angle is angle γ1, and light can be irradiated at the widest angle among the angles shown. In the example shown in FIG. 7(C), the distance between the light-emitting section 141 and the diffusion section 142 is distance L43, which is the longest among the angles shown. In this case, the irradiation angle is angle γ3, and light can be irradiated at the narrowest angle among the angles shown. In the example shown in FIG. 7(B), the distance between the light-emitting section 141 and the diffusion section 142 is distance L42, which is the distance between the example shown in FIG. 7(A) and the example shown in FIG. 7(C). In this case, the irradiation angle is angle γ2, and light can be irradiated at an irradiation angle between the example shown in FIG. 7(A) and the example shown in FIG. 7(C).

[Summary of the First Embodiment]
According to the embodiment described above, the three-dimensional information acquisition device 10 acquires the first distance image DI1 by including the first distance image acquisition unit 132, and acquires the second distance image DI2 by including the second distance image acquisition unit 133. The first distance image DI1 and the second distance image DI2 are both distance images (depth images) including a plurality of distance values obtained according to the time from when light is irradiated to when the light reflected by the object is received. The first distance image DI1 and the second distance image DI2 are distance images obtained by irradiating light to the same object T, and are distance images obtained by irradiating light with different irradiation angles. The first distance image DI1 has distance information over a wide range because the distance to the object T is measured using light with a wide irradiation angle. The second distance image DI2 is less affected by multipath because the distance to the object T is measured using light with a narrow irradiation angle. The three-dimensional information acquisition device 10 includes a three-dimensional information generating unit 134, and generates three-dimensional information of the object T with reduced effects of multipath caused by light reflection based on the first distance image DI1 and the second distance image DI2. Therefore, according to this embodiment, a distance measurement technique capable of reducing the effects of multipath can be provided. Conventionally, the effects of multipath could not be reduced unless an environment susceptible to the effects of multipath (e.g., surrounding objects such as a background with high near-infrared light reflectance) was excluded. However, according to this embodiment, a distance measurement technique capable of reducing the effects of multipath without excluding an environment susceptible to the effects of multipath can be provided.

Furthermore, according to the embodiment described above, the first distance image DI1 and the second distance image DI2 both have distance values corresponding to each coordinate in the two-dimensional coordinate system. Furthermore, the three-dimensional information generating unit 134 compares the distance values contained in the first distance image DI1 and the second distance image DI2 at the same coordinate, and generates three-dimensional information of the object according to the comparison result. That is, the three-dimensional information acquisition device 10 determines which distance value to use according to the difference in the distance values. Coordinates with a large difference in distance values are likely to be affected by multipath in the first distance image DI1, which has a wide irradiation angle. For such coordinates, by using the distance values of the second distance image DI2, three-dimensional information can be generated in which the effects of multipath are reduced. Therefore, according to this embodiment, a distance measurement technology that can easily reduce the effects of multipath can be provided.

Furthermore, according to the embodiment described above, the illumination unit 120 is provided to illuminate light for acquiring a distance image. The illumination unit 120 can change the direction in which light is illuminated depending on the position of the object T. In other words, the three-dimensional information acquisition device 10 can change the illumination angle depending on the position of the subject. Therefore, according to this embodiment, even if the object T is not in the center of the angle of view, a second illumination can be performed in which light is illuminated only on the object T. Therefore, according to this embodiment, it is possible to provide a distance measurement technique that can reduce the effects of multipath even if the object T is not in the center of the angle of view.

Furthermore, according to the embodiment described above, the three-dimensional information acquisition device 10 further includes a visible light image acquisition unit, and thereby acquires a visible light image of the object T. Furthermore, the irradiation unit 120 can change the direction of light irradiation and the irradiation angle according to the position and size of the object T obtained by analyzing the visible light image. That is, the irradiation unit 120 can identify the position and size of the object T based on the visible light image, and change the irradiation direction and irradiation angle. Therefore, according to this embodiment, even if the object T is not at the center of the angle of view, the direction for performing the second irradiation can be automatically determined, and the second irradiation can be performed. Therefore, according to this embodiment, it is possible to provide a distance measurement technique that can reduce the effects of multipath even if the object T is not at the center of the angle of view.

Furthermore, according to the embodiment described above, the three-dimensional information acquisition device 10 is provided with an irradiation unit 140, which irradiates light for acquiring a distance image, and the irradiation unit 140 is provided with a light-emitting unit 141 that emits light, and a diffusion unit 142 that diffuses the light emitted by the light-emitting unit 141. The irradiation unit 140 irradiates light at different irradiation angles by varying the distance between the light-emitting unit 141 and the diffusion unit 142. In other words, according to this embodiment, it is not necessary to previously provide an irradiation unit 120 having a plurality of different irradiation angles, and the irradiation angle can be changed according to the position and size of the object T. Therefore, according to this embodiment, it is possible to irradiate light at an appropriate irradiation angle according to the position and size of the object T, and it is possible to provide a distance measurement technology that can accurately reduce the effects of multipath.

In the above example, it is assumed that only one object T exists within the angle of view. However, this embodiment is not limited to this example, and can also be applied to a case where multiple objects T exist within the angle of view. When multiple objects T exist within the angle of view, for example, the second irradiation may be performed on a main object T among the multiple objects T, and the second irradiation may not be performed on the other objects T among the multiple objects T. In another embodiment, the second irradiation corresponding to the first irradiation may be performed as many times as the number of objects T to obtain multiple second distance images DI2 that are less affected by multipath. In this case, the three-dimensional information generating unit 134 may generate three-dimensional information of multiple objects T based on one first distance image DI1 and multiple second distance images DI2.

In the above example, the irradiation angle is narrowed in the second irradiation according to the position and size of the object T, but if there is a limit to the adjustment of the irradiation angle (i.e., even when the irradiation angle is set to the narrowest, light is irradiated to the outside of the object T), it is possible to reduce the effect of multipath (although it may not be possible to completely eliminate the effect of multipath) by positioning the object T closer to the optical axis. The effect of multipath may also be reduced by adjusting the direction in which the second irradiation unit 122 faces so that the optical axis of the light irradiated by the second irradiation unit 122 passes through the center of the object T.

The magnitude of the multipath effect is determined by the shape of the object T and the background BG. Depending on the shape of the object T and the background BG, the multipath effect may be more in the horizontal direction or more in the vertical direction. Therefore, the three-dimensional information acquisition device 10 may perform different control depending on whether the multipath effect is more in the horizontal direction or more in the vertical direction. Whether the multipath effect is more in the horizontal direction or more in the vertical direction may be determined, for example, by a visible light image. When the multipath effect is more in the horizontal direction, the second irradiation unit 122 may be arranged vertically above and below the light receiving unit 110 (i.e., similar to the arrangement shown in FIG. 3). When the multipath effect is more in the vertical direction, the second irradiation unit 122 may be arranged horizontally above and below the light receiving unit 110 (i.e., an arrangement rotated 90 degrees from the arrangement shown in FIG. 3).

[Embodiment 2]
Next, a second embodiment will be described with reference to Fig. 8 to Fig. 11. In the first embodiment, an example of acquiring a second distance image DI2 by irradiating light with a different irradiation angle (specifically, light with a narrow irradiation angle) has been described. The second embodiment differs from the first embodiment in that a second distance image DI2 in which multipath is reduced is acquired by changing the background while keeping the irradiation angle fixed.

8 is a functional configuration diagram showing an example of the functional configuration of a three-dimensional information acquisition device according to the second embodiment. With reference to the same figure, an example of the functional configuration of a three-dimensional information acquisition device 10A according to the second embodiment will be described. The irradiation unit 120A of the three-dimensional information acquisition device 10A differs from the three-dimensional information acquisition device 10 in that it does not include a plurality of irradiation units that irradiate light at different irradiation angles. Specifically, the three-dimensional information acquisition device 10A differs from the three-dimensional information acquisition device 10 in that it includes an irradiation unit 120A instead of an irradiation unit 120. The three-dimensional information acquisition device 10A also differs from the three-dimensional information acquisition device 10 in that it further includes a background information acquisition unit 150. The three-dimensional information acquisition device 10A also differs from the three-dimensional information acquisition device 10 in that it includes a control unit 130A instead of a control unit 130. In the description of the three-dimensional information acquisition device 10A, the same components as those of the three-dimensional information acquisition device 10 may be denoted by the same reference numerals and description thereof may be omitted.

The irradiation unit 120A emits light to obtain a distance image based on control information CI from the ToF sensor control unit 131A.

The background information acquisition unit 150 acquires information about the background of the object T. The background information acquisition unit 150 outputs the acquired information to the ToF sensor control unit 131A as background information BGI. The background information acquisition unit 150 may acquire information about the background BG of the object T, for example, by an operation from a user. As another example, the background information acquisition unit 150 may acquire information about the background BG of the object T by capturing a visible light image and performing image analysis on the captured visible light image. The background information BGI may include information about whether the background has a high near-infrared reflectance or a low near-infrared reflectance. When the background information acquisition unit 150 acquires information about the background BG of the object T by performing image analysis on a visible light image, for example, the edge of the object T may be detected, and if the outside of the object T is close to white, the background may be determined to have a high near-infrared reflectance, and if the outside of the object T is close to black, the background may be determined to have a low near-infrared reflectance.

Control unit 130A includes a ToF sensor control unit 131A instead of ToF sensor control unit 131, a first distance image acquisition unit 132A instead of first distance image acquisition unit 132, a second distance image acquisition unit 133A instead of second distance image acquisition unit 133, and a three-dimensional information generation unit 134A instead of three-dimensional information generation unit 134. Control unit 130A is a modified example of control unit 130.

The ToF sensor control unit 131A controls the light irradiation by the irradiation unit 120A. Specifically, the ToF sensor control unit 131A controls the timing of light irradiation by the irradiation unit 120A by outputting control information CI. The ToF sensor control unit 131A also acquires information about the reflected light received as a result of irradiation by the irradiation unit 120A. Specifically, the ToF sensor control unit 131A acquires information about the reflected light by acquiring light reception information RI from the light receiving unit 110. The light reception information RI may include information about the timing of acquiring the reflected light according to each coordinate of the two-dimensional coordinate system. The ToF sensor control unit 131A generates a distance image based on the acquired light reception information RI. The ToF sensor control unit 131A generates a first distance image DI1A based on the reflected light obtained when light is irradiated when the background of the object T is a first background BG1, which is a background with high reflectance of near-infrared rays (hereinafter referred to as first irradiation). In addition, when the background of the object T is a second background BG2 that has a lower reflectance of near-infrared rays than the first background BG1, the ToF sensor control unit 131A irradiates light (hereinafter referred to as the second irradiation) and generates a second distance image DI2A based on the resulting reflected light. Here, the first irradiation and the second irradiation are performed on the same object T.

The first distance image acquisition unit 132A acquires the first distance image DI1A generated by the ToF sensor control unit 131A. The first distance image acquisition unit 132A outputs the acquired first distance image DI1A to the three-dimensional information generation unit 134A.

The second distance image acquisition unit 133A acquires the second distance image DI2A generated by the ToF sensor control unit 131A. The second distance image acquisition unit 133A outputs the acquired second distance image DI2A to the three-dimensional information generation unit 134A.

The three-dimensional information generating unit 134A acquires the first distance image DI1A from the first distance image acquiring unit 132A, and acquires the second distance image DI2A from the second distance image acquiring unit 133A. The three-dimensional information generating unit 134A generates three-dimensional information of the object T based on the acquired first distance image DI1A and second distance image DI2A. Here, when the second background BG2 is used, the reflectance of near-infrared rays is lower than that of the first background BG1, so it can be said that it is less susceptible to the influence of multipath caused by light reflection compared to the case where the first background BG1 is used. Therefore, it can be said that the second distance image DI2A is less affected by multipath caused by light reflection compared to the first distance image DI1A. Therefore, the three-dimensional information generating unit 134A can generate three-dimensional information of the object T with reduced influence of multipath caused by light reflection based on the first distance image DI1A and the second distance image DI2A. In the following description, the three-dimensional information generation unit 134A may be referred to simply as the generation unit.

Note that, when distance values can be obtained from both the first distance image DI1A and the second distance image DI2A, the three-dimensional information generation unit 134A may determine which distance value to use depending on the result of comparing the distance value in the first distance image DI1A with the distance value in the second distance image DI2A. Here, the first distance image DI1A and the second distance image DI2A each have distance values corresponding to each coordinate in a two-dimensional coordinate system. Thus, the three-dimensional information generation unit 134A may compare the distance value contained in the first distance image DI1A with the distance value contained in the second distance image DI2A at the same coordinates, and generate three-dimensional information of the object T depending on the comparison result.

9 is a diagram for explaining the first irradiation and the second irradiation by the three-dimensional information acquisition device according to the second embodiment. With reference to the figure, the first irradiation using the first background BG1 and the second irradiation using the second background BG2, which are irradiation by the three-dimensional information acquisition device 10A according to the second embodiment, will be explained. In the example shown in the figure, the three-dimensional information acquisition device 10A measures the three-dimensional shape of the object T that is in front of the background. The three-dimensional information acquisition device 10A includes a light receiving unit 110 and an irradiation unit 120A. The three-dimensional information acquisition device 10A includes irradiation units 120A-1 and 120A-2 as the irradiation unit 120A. FIG. 9(A) shows an example of the first irradiation, and FIG. 9(B) shows an example of the second irradiation. In the example shown in the figure, a cylindrical gum case is used as the object T. In addition, a white wall with high near-infrared reflectance is used as the first background BG1, and a black wall with low near-infrared reflectance is used as the second background BG2.

As shown in FIG. 9(A), the light irradiated by the irradiation unit 120A and reflected by the object T (the path of this light is referred to as irradiation light BM2-1) is not affected by multipath. However, depending on the light irradiated by the irradiation unit 120A and reflected by the first background BG1 (the path of this light is referred to as irradiation light BM2-2), the reflected light from the first background BG1 may be further reflected by the object T, etc., and such light is affected by multipath. Due to the effect of multipath, the shape of the object T is actually as shown by the dotted line, but is measured as shown by the solid line. This is because when affected by multipath, the Depth value (a value indicating the distance from the three-dimensional information acquisition device 10A to the object T) is measured as being larger (farther) than the actual value.

As shown in FIG. 9B, the light irradiated by the irradiation unit 120A and reflected by the object T (the path of the light is described as irradiation light BM2-1) is not affected by multipath. In addition, depending on the light irradiated by the irradiation unit 120A and reflected by the second background BG2 (the path of the light is described as irradiation light BM2-3), the reflected light from the second background BG2 may be further reflected by the object T, etc. However, since the second background BG2 has a low reflectance of near-infrared rays, the reflected light is attenuated and such light is not affected by multipath (even if it is affected, the effect is minor). Therefore, when the second background BG2 is used, since it is not affected by multipath (or the effect is minor), the shape of the object T is originally as shown by the dotted line, but the measurement result is also as shown by the solid line. In this way, the second irradiation using the second background BG2 allows the shape of the object T to be accurately measured. In the following description, a configuration including the three-dimensional information acquisition device 10A and the second background BG2 may be referred to as a three-dimensional information acquisition system.

The second background BG2 may be the first background BG1, such as a white wall, covered with a black cloth (a cloth with low near-infrared reflectance). Preferably, the second background BG2 may be an infrared absorbing sheet or the like, which is a background with low near-infrared reflectance.

The three-dimensional information generating unit 134A generates three-dimensional information of the object T, which reduces the effects of multipath caused by light reflection, based on the first distance image DI1A and the second distance image DI2A, in a manner similar to that described in the first embodiment.

As described above, in the case of a configuration in which a visible light image can be further acquired using a visible light reflecting dichroic film or the like, the luminance of the visible light image of the object T may deviate from an appropriate level depending on the luminance of the background. Therefore, in such a case, the imaging conditions (exposure, electronic amplification, shutter speed, etc.) when the first background BG1 is used may be different from the imaging conditions when the second background BG2 is used, and level correction may be performed. Specifically, the three-dimensional information acquisition device 10A may realize such a function by including a first visible light image acquisition unit (not shown) and a second visible light image acquisition unit (not shown). The first visible light image acquisition unit acquires a first visible light image that is a visible light image obtained by capturing the object T and the first background BG1 corresponding to the first distance image DI1A. The second visible light image acquisition unit acquires a second visible light image that is a visible light image obtained by capturing the object T and the second background BG2 corresponding to the second distance image DI2A. The first visible light image and the second visible light image were captured under different imaging conditions for determining the brightness of the visible light image.

Furthermore, the level correction as described above may be performed after imaging. In this case, the three-dimensional information acquisition device 10A may further include a visible light image generation unit (not shown) to perform level correction after imaging. The visible light image generation unit generates pixel information to be added to the three-dimensional information of the object T based on the acquired first visible light image and second visible light image. The visible light image generation unit also generates pixel information according to a result of comparing the pixel values of the acquired first visible light image with the pixel values of the second visible light image. When the visible light image generation unit detects that the difference between the visible light image level of the second background BG2, which has a low near-infrared reflectance, and the visible light image level of the first background BG1, which has a high near-infrared reflectance, is equal to or greater than a certain level, the visible light image generation unit may determine that the level is outside the appropriate level and perform level correction by changing the exposure, electronic amplification, shutter speed, etc.

Next, with reference to Figures 10 and 11, examples of visible light images and distance images will be described for the cases where the first background BG1 is used and where the second background BG2 is used.

10 is a diagram showing an example of a visible light image when the first background is used and when the second background is used by the three-dimensional information acquisition device according to the second embodiment. With reference to the figure, an example of a visible light image will be described for each of the cases where the first background BG1 and the second background BG2 are used. FIG. 10(A) shows an image of a gum bottle and blocks around it, etc., with a general wall as the background. In this state, the general wall can be said to be the first background BG1, which has a high reflectance of near-infrared rays. Therefore, if an attempt is made to acquire a distance image in this state, it is likely to be affected by multipath. FIG. 10(B) shows an image of the same subject as FIG. 10(A), but a black cloth or the like is inserted between the wall and the object T. The black cloth or the like is preferably an infrared absorbing sheet. The black cloth or the like can be said to be the second background BG2, which has a low reflectance of near-infrared rays. Therefore, if an attempt is made to acquire a distance image in this state, it is unlikely to be affected by multipath.

11 is a diagram showing an example of point cloud data when a first distance image is used and when a second distance image is used by a three-dimensional information acquisition device according to embodiment 2. With reference to the figure, an example of a distance image will be described for each of the cases where a first background BG1 is used and where a second background BG2 is used. In the illustrated example, the image is rotated diagonally so that the distortion can be easily seen. FIG. 11(A) is an example of a first distance image DI1A, which is a distance image acquired under the conditions in which FIG. 10(A) was captured. As can be seen from the figure, the cylindrical gum bottle is distorted elongated in the depth direction. This is because the effect of multipath caused the distance value to be larger in the depth direction than it should be. FIG. 11(B) is an example of a second distance image DI2A, which is a distance image acquired under the conditions in which FIG. 10(B) was captured. As can be seen from the figure, the distortion in the depth direction is less than that in FIG. 11(A). This is because the effect of multipath was reduced by using a background with low reflectance of near-infrared rays.

[Summary of the second embodiment]
According to the embodiment described above, the three-dimensional information acquisition device 10A acquires the first distance image DI1A by including the first distance image acquisition unit 132A, and acquires the second distance image DI2A by including the second distance image acquisition unit 133A. The first distance image DI1A and the second distance image DI2A are both distance images (depth images) including a plurality of distance values obtained according to the time from irradiating light to receiving the light reflected by the object. The first distance image DI1A and the second distance image DI2A are distance images obtained by irradiating light to the same object T, and are distance images obtained using different backgrounds BG. Specifically, the first distance image DI1A is a distance image obtained using the first background BG1, and the second distance image DI2A is a distance image obtained using the second background BG2. The second background BG2 has a lower reflectance of near-infrared rays than the first background BG1. In addition, the three-dimensional information acquisition device 10A includes a three-dimensional information generating unit 134A, and generates three-dimensional information of the object T with reduced effects of multipath caused by light reflection based on the first distance image DI1A and the second distance image DI2A. Therefore, according to this embodiment, a distance measurement technique capable of reducing the effects of multipath can be provided. Conventionally, the effects of multipath could not be reduced unless an environment susceptible to the effects of multipath (for example, surrounding objects such as a background with high near-infrared light reflectance) was excluded. However, according to this embodiment, a distance measurement technique capable of reducing the effects of multipath without excluding an environment susceptible to the effects of multipath can be provided (i.e., three-dimensional information including information about the background of the object T can be generated).

Furthermore, according to the embodiment described above, the first distance image DI1A and the second distance image DI2A both have distance values corresponding to each coordinate in the two-dimensional coordinate system. Furthermore, the three-dimensional information generating unit 134A compares the distance values contained in the first distance image DI1A and the second distance image DI2A at the same coordinate, and generates three-dimensional information of the object according to the comparison result. That is, the three-dimensional information acquisition device 10A determines which distance value to use according to the difference in the distance values. Coordinates with a large difference in distance values are likely to be affected by multipath in the first distance image DI1A using the first background BG1, which has a high reflectance of near-infrared rays. For such coordinates, the effect of multipath can be reduced by using the distance values of the second distance image DI2A. Therefore, according to this embodiment, a distance measurement technique that can easily reduce the effect of multipath can be provided.

Furthermore, according to the embodiment described above, the three-dimensional information acquisition device 10A further includes a first visible light image acquisition unit and a second visible light image acquisition unit. The first visible light image acquisition unit acquires a first visible light image, which is a visible light image obtained by capturing an object T and a first background BG1 corresponding to the first distance image DI1A. The second visible light image acquisition unit acquires a second visible light image, which is a visible light image obtained by capturing an object T and a second background BG2 corresponding to the second distance image DI2A. The first visible light image and the second visible light image have different imaging conditions for determining the brightness of the visible light image. That is, according to the three-dimensional information acquisition device 10A, the visible light image using the first background BG1 and the visible light image using the second background BG2 are captured under different imaging conditions (exposure, electronic amplification, shutter speed, etc.). Therefore, according to this embodiment, it is possible to capture a visible light image under imaging conditions that provide an appropriate luminance level for the image.

According to the embodiment described above, the three-dimensional information acquisition device 10A further includes a first visible light image acquisition unit, a second visible light image acquisition unit, and a visible light image generation unit. The first visible light image acquisition unit acquires a first visible light image, which is a visible light image obtained by capturing an image of the object T and the first background BG1 corresponding to the first distance image DI1A. The second visible light image acquisition unit acquires a second visible light image, which is a visible light image obtained by capturing an image of the object T and the second background BG2 corresponding to the second distance image DI2A. The visible light image generation unit generates pixel information to be added to the three-dimensional information of the object T based on the acquired first and second visible light images. The visible light image generation unit also generates pixel information to be added to the three-dimensional information of the object T according to a result of comparing the pixel values of the acquired first and second visible light images. That is, the three-dimensional information acquisition device 10A may perform correction after capturing the visible light image. Therefore, according to this embodiment, it is possible to correct a visible light image so that the brightness level of the image is optimal.

Furthermore, according to the embodiment described above, the three-dimensional information acquisition system includes the three-dimensional information acquisition device 10A and the second background BG2. The second background BG2 is an infrared absorbing sheet that is used when acquiring the second distance image DI2A and has a low reflectance of near-infrared rays. Therefore, according to this embodiment, a normal wall, floor, building, natural object, or other background that is reflected during normal imaging is used as the first background BG1, and an infrared absorbing sheet is used as the second background BG2, so that the first distance image DI1A and the second distance image DI2A can be acquired. The second distance image DI2A is a distance image in which the influence of multipath is reduced. Therefore, according to this embodiment, a distance measurement technique that can reduce the influence of multipath can be provided. Note that the first background BG1 is not limited to a normal wall, floor, or the like, and a background with a high reflectance such as a white chart may be used as the first background BG1.

[Embodiment 3]
Next, a third embodiment will be described with reference to Fig. 12 to Fig. 17. The third embodiment differs from the first and second embodiments in that a distance image and a visible light image having different angles of view are captured using a zoom lens.

FIG. 12 is a schematic diagram showing an example of a cross section of a three-dimensional information acquisition device according to embodiment 3. An example of a cross section of three-dimensional information acquisition device 10B will be described with reference to the same figure. Three-dimensional information acquisition device 10B includes lens 111, visible light reflective dichroic film 112, image sensor 113, and sensor 114. Three-dimensional information acquisition device 10B differs from three-dimensional information acquisition device 10 and three-dimensional information acquisition device 10A in that it further includes lens 111 that functions as a zoom lens. Note that the configurations of three-dimensional information acquisition device 10 and three-dimensional information acquisition device 10A, except for lens 111, may be similar to that of three-dimensional information acquisition device 10B.

The three-dimensional information acquisition device 10B has a plurality of irradiation units 120 at positions symmetrical to each other with the light receiving unit 110 in between. In the illustrated example, the three-dimensional information acquisition device 10B has irradiation units 120B-1 and 120B-2 as the irradiation units 120 near the light receiving unit 110. The irradiation units 120B-1 and 120B-2 are arranged at positions symmetrical to each other with the light receiving unit 110 in between. The irradiation units 120B-1 and 120B-2 irradiate the object T with light. The light reflected by the object T is incident on the light receiving unit 110. In the figure, the optical axis of the incident light is depicted as optical axis OA. The light incident on the light receiving unit 110 passes through the lens 111 and is incident on the visible light reflecting dichroic film 112.

Lens 111 is specifically a zoom lens. A zoom lens can change the magnification (zoom magnification) by changing the spacing between multiple lens groups. Three-dimensional information acquisition device 10B may be equipped with multiple lenses (not shown). Lens 111 changes the magnification by sliding in the optical axis direction (left and right in the figure). The magnification is controlled by a control unit (not shown).

The visible light reflecting dichroic film 112 is provided on the optical path between the light receiving unit 110 and the sensor 114. The visible light reflecting dichroic film 112 transmits a portion of the incident light (specifically, near-infrared light) and reflects the other light (specifically, visible light). The visible light reflecting dichroic film 112 transmits a portion of the light irradiated by the irradiation unit 120 and reflected by the object T, thereby guiding the light to the sensor 114. The light transmitted by the visible light reflecting dichroic film 112 is described as infrared light IL. The light reflected by the visible light reflecting dichroic film 112 is described as visible light VL. The infrared light IL and the visible light VL before passing through the visible light reflecting dichroic film 112 pass through approximately the same optical axis between the light receiving unit 110 and the visible light reflecting dichroic film 112. The approximately same range may be, for example, a range in which an optical path is formed by a common lens.

The light split into two optical paths by the visible light reflecting dichroic film 112 is received by sensors arranged in each optical path. Specifically, the infrared light IL that passes through the visible light reflecting dichroic film 112 is received by the sensor 114, and the light reflected by the visible light reflecting dichroic film 112 is received by the image sensor 113.

The image sensor 113 has a plurality of pixels arranged in a two-dimensional array. The image sensor 113 may have pixels of each of the colors RGB arranged in a Bayer array.

The sensor 114 has multiple pixels arranged in a two-dimensional array. Each of the multiple pixels receives infrared light IL multiple times to obtain the information required for distance conversion.

FIG. 13 is a functional configuration diagram showing an example of the functional configuration of a three-dimensional information acquisition device according to embodiment 3. With reference to the same figure, an example of the functional configuration of a three-dimensional information acquisition device 10B according to embodiment 3 will be described. Three-dimensional information acquisition device 10B differs from three-dimensional information acquisition device 10 or three-dimensional information acquisition device 10A in that it includes an irradiation unit 120B instead of irradiation unit 120, a control unit 130B instead of control unit 130A, and a magnification information acquisition unit 160 instead of background information acquisition unit 150. In the description of three-dimensional information acquisition device 10B, components similar to those of three-dimensional information acquisition device 10 or three-dimensional information acquisition device 10A are denoted by similar reference numerals and description thereof may be omitted.

The magnification information acquisition unit 160 acquires information related to the magnification. The information related to the magnification may be, for example, information related to a change in the magnification caused by the lens 111 moving forward or backward. The magnification information acquisition unit 160 outputs the acquired information related to the magnification to the control unit 130B as magnification information ZI.

The irradiation unit 120B irradiates light with an irradiation angle corresponding to the angle of view corresponding to the magnification. Specifically, the irradiation unit 120B acquires information related to the magnification from the magnification information acquisition unit 160, and irradiates light with an irradiation angle substantially the same as the angle of view corresponding to the magnification, based on the acquired information related to the magnification.

Control unit 130B includes a ToF sensor control unit 131B instead of ToF sensor control unit 131, a first distance image acquisition unit 132B instead of first distance image acquisition unit 132, a second distance image acquisition unit 133B instead of second distance image acquisition unit 133, and a three-dimensional information generation unit 134B instead of three-dimensional information generation unit 134. Control unit 130B also includes a visible light image acquisition unit 135. Control unit 130B is a modified example of control unit 130.

The ToF sensor control unit 131B controls the irradiation of light by the irradiation unit 120B. Specifically, the ToF sensor control unit 131 controls the timing of the irradiation of light by the first irradiation unit 121B by outputting first control information CI1, and controls the timing of the irradiation of light by the second irradiation unit 122B by outputting second control information CI2. The ToF sensor control unit 131B also acquires information about the reflected light received as a result of the irradiation by the irradiation unit 120B. Specifically, the ToF sensor control unit 131B acquires information about the reflected light by acquiring light reception information RI from the light receiving unit 110. The light reception information RI may include information about the timing at which the reflected light was acquired according to each coordinate of the two-dimensional coordinate system. The ToF sensor control unit 131B generates a distance image based on the acquired light reception information RI. Specifically, the ToF sensor control unit 131B generates a first distance image DI1B based on reflected light obtained as a result of performing irradiation by the first irradiating unit 121B (hereinafter referred to as the first irradiation) by outputting the first control information CI1. Also, the ToF sensor control unit 131B generates a second distance image DI2B based on reflected light obtained as a result of performing irradiation by the second irradiating unit 122B (hereinafter referred to as the first irradiation) by outputting the second control information CI2. Here, the first irradiation and the second irradiation are performed on the same object T. Also, the first irradiation and the second irradiation are obtained by setting the magnification of the lens 111 to different values. The second irradiation is magnified compared to the first irradiation. In other words, the second distance image DI2B is a distance image obtained by magnifying a part of the same object T as the first distance image DI1B by an optical member.

The first distance image acquisition unit 132B acquires the first distance image DI1B generated by the ToF sensor control unit 131B. The first distance image acquisition unit 132B outputs the acquired first distance image DI1B to the three-dimensional information generation unit 134B.

The second distance image acquisition unit 133B acquires the second distance image DI2B generated by the ToF sensor control unit 131B. The second distance image acquisition unit 133B outputs the acquired second distance image DI2B to the three-dimensional information generation unit 134B.

The three-dimensional information generating unit 134B acquires a first distance image DI1B from the first distance image acquiring unit 132B, and acquires a second distance image DI2B from the second distance image acquiring unit 133B. The three-dimensional information generating unit 134B generates three-dimensional information of the object T based on the acquired first distance image DI1B and second distance image DI2B. Here, the second distance image DI2B, which has a high magnification ratio, has less illumination of the background compared to the first distance image DI1B, and therefore is less susceptible to the effects of multipath caused by light reflection. Therefore, the three-dimensional information generating unit 134B can generate three-dimensional information of the object T with reduced effects of multipath caused by light reflection based on the first distance image DI1B and the second distance image DI2B. In the following description, the three-dimensional information generating unit 134B may be simply referred to as a generating unit.

Here, when attempting to generate three-dimensional information with less influence of multipath based on the first distance image DI1B and the second distance image DI2B, synthesis is difficult (although not impossible) if the optical axis is misaligned. Therefore, it is preferable that the first distance image DI1B and the second distance image DI2B are acquired with the optical axis fixed. Note that this embodiment is not limited to this example, and the first distance image DI1B and the second distance image DI2B may be acquired with the optical axis misaligned from each other. In this case, a certain calculation is required when attempting to generate three-dimensional information with less influence of multipath.

The visible light image acquisition unit 135 acquires a visible light image of the object T from the image sensor 113. The visible light image acquisition unit 135 acquires a visible light image with the same magnification as the magnification when the distance image is acquired. Specifically, the visible light image acquisition unit 135 acquires a first visible light image, which is a visible light image having approximately the same angle of view as the first distance image DI1B, and a second visible light image, which is a visible light image having approximately the same angle of view as the second distance image DI2B.

The magnification ratio at which the second visible light image and the second distance image DI2B are acquired may be determined based on the results of analyzing the first visible light image. For example, the first visible light image may be analyzed to detect the edge (contour) of the object T, and the second visible light image and the second distance image DI2B may be acquired at a magnification ratio that just contains the edge. The analysis of the first visible light image may involve differentiation of pixel values or frequency analysis, or a machine learning algorithm such as face recognition may be used. In this case, the magnification ratio determined by the optical member may be based on the visible light image acquired by the visible light image acquisition unit 135.

In this embodiment, either the distance image or the visible light image may be acquired first. For example, if the visible light image is acquired first and then the distance image is acquired, the irradiation unit 120B may determine the irradiation angle according to the magnification acquired by the magnification information acquisition unit 160. In other words, the irradiation angle of the light irradiated to acquire the distance image may be determined according to the angle of view of the visible light image.

Next, an example of point cloud data generated by the three-dimensional information acquisition device 10B will be described with reference to Figures 14 to 16.

FIG. 14 is a diagram showing an example of point cloud data in which a distance image and a visible light image at the wide end are combined by a three-dimensional information acquisition device according to embodiment 3. With reference to the same figure, an example of point cloud data generated based on the first distance image DI1B and the first visible light image will be described. In the example shown, the first distance image DI1B and the first visible light image are acquired when the focal length is, for example, 4 mm (millimeters). At the wide end, wide-angle information is obtained, but the density of the point cloud is low. In addition, at the wide end, the image is also affected by multipath due to light reflected from the background. It can also be said that the pixel density of a visible light image in a certain area at the wide end is lower than the pixel density of the corresponding area at the telephoto end.

FIG. 15 is a diagram showing an example of point cloud data in which a distance image and a visible light image at the telephoto end are combined by the three-dimensional information acquisition device according to the third embodiment. With reference to the same figure, an example of point cloud data generated based on the second distance image DI2B and the second visible light image will be described. In the example shown, the second distance image DI2B and the second visible light image are acquired when the focal length is, for example, 13 mm. At the telephoto end, only narrow-angle information is obtained, but the density of the point cloud is dense. Also, at the telephoto end, light is only irradiated onto the object T, so light is not irradiated onto the background, and the effect of multipath caused by reflected light is reduced. It can also be said that the pixel density of a visible light image in a certain area at the telephoto end is higher than the pixel density of the corresponding area at the wide-angle end.

16 is a diagram showing an example of point cloud data obtained by combining a distance image and a visible light image at the wide end and a distance image and a visible light image at the telephoto end by a three-dimensional information acquisition device according to the third embodiment. With reference to the figure, an example of three-dimensional information obtained by combining a distance image and a visible light image at the wide end and a distance image and a visible light image at the telephoto end will be described. The example shown in the figure is obtained by replacing part of the point cloud data at the wide end shown in FIG. 14 with the point cloud data at the telephoto end shown in FIG. 15. The part is an area enlarged by the lens 111. In this example, the density of the point cloud data and the resolution of the visible light image are different between the area where the point cloud data at the telephoto end exists and the other areas. As shown in the figure, the density of the point cloud data and the resolution of the visible light image are increased in the area where the point cloud data at the telephoto end exists. However, this embodiment is not limited to this example, and for the area where the point cloud data at the telephoto end does not exist, the data may be supplemented by supplementing the density of the point cloud data and the resolution of the visible light image.

FIG. 17 is a diagram for explaining the complementation of pixel values by the three-dimensional information acquisition device according to the third embodiment. An example of complementation of the density of point cloud data and the resolution of a visible light image will be described with reference to the diagram. FIG. 17(A) shows an image of a case where three-dimensional information at the wide-angle end is simply replaced with three-dimensional information at the telephoto end. Each rectangle shown in the diagram represents information corresponding to one pixel. As shown in the diagram, the pixel density is four times higher in the central portion of FIG. 17(A). FIG. 17(B) is a diagram showing an image of complementation. As shown in the diagram, the pixel density is the same in FIG. 17(B). In other words, the density of the peripheral portion is complemented to match the density of the central portion. Existing technology can be used as the complementation method, and linear interpolation, for example, may be used.

Specifically, pixel complementation may be performed by the visible light image generation unit. The visible light image generation unit generates pixel information to be added to the three-dimensional information of the object T based on the first visible light image and the second visible light image acquired by the visible light image acquisition unit 135. The visible light image generation unit also complements pixel values at coordinates in the first visible light image that have a lower resolution than the second visible light image (e.g., the peripheral part in FIG. 17(A)) using a predetermined method. The predetermined method may be linear interpolation, etc.

[Summary of the Third Embodiment]
According to the embodiment described above, the three-dimensional information acquisition device 10B acquires the first distance image DI1B by including the first distance image acquisition unit 132B, and acquires the second distance image DI2B by including the second distance image acquisition unit 133B. The first distance image DI1B and the second distance image DI2B are both distance images (depth images) including a plurality of distance values obtained according to the time from when light is irradiated to when the light reflected by the object is received. The first distance image DI1B and the second distance image DI2B are distance images obtained by irradiating light to the same object T, and are distance images obtained by irradiating light at different irradiation angles. In addition, the second distance image DI2B is a distance image obtained by enlarging a part of the same object T as the first distance image DI1B by an optical member. Since the second distance image DI2B is an enlarged part of the first distance image DI1B, it can be said that the light reflected on the background is small and the influence of multipath is reduced. Furthermore, the three-dimensional information acquisition device 10B includes a three-dimensional information generating unit 134B, and generates three-dimensional information of the object T with reduced effects of multipath caused by light reflection based on the first distance image DI1B and the second distance image DI2B. Therefore, according to this embodiment, a distance measurement technique capable of reducing the effects of multipath can be provided. Furthermore, in the past, the effects of multipath could not be reduced unless an environment susceptible to the effects of multipath (for example, surrounding objects such as a background with high near-infrared light reflectance) was excluded. However, according to this embodiment, a distance measurement technique capable of reducing the effects of multipath without excluding an environment susceptible to the effects of multipath can be provided.

Furthermore, according to the embodiment described above, the first distance image DI1B and the second distance image DI2B are acquired while the optical axis remains fixed. Therefore, the three-dimensional information acquisition device 10B can easily synthesize the first distance image DI1B by replacing a portion of the first distance image DI1B with the second distance image DI2B. Therefore, according to this embodiment, three-dimensional information in which the effects of multipath are reduced can be easily generated.

Furthermore, according to the embodiment described above, the three-dimensional information acquisition device 10B further includes a visible light image acquisition unit 135, thereby acquiring a visible light image of the object T. Furthermore, the magnification by the optical member is based on the visible light image acquired by the visible light image acquisition unit 135. For example, the three-dimensional information acquisition device 10B detects the position and size of the object T based on the visible light image, and enlarges the image to an extent that the object T fits in just right. Therefore, according to this embodiment, since light is not irradiated outside the object T, the effects of multipath can be reduced, and by enlarging the object T to obtain a distance image, high-resolution three-dimensional information can be generated.

Furthermore, according to the embodiment described above, the visible light image acquisition unit 135 acquires a first visible light image, which is a visible light image having substantially the same angle of view as the first distance image DI1B, and a second visible light image, which is a visible light image having substantially the same angle of view as the second distance image DI2B. That is, the angle of view of the distance image and the angle of view of the visible light image are substantially the same. Therefore, according to this embodiment, the three-dimensional information acquisition device 10B can easily combine the distance image and the visible light image. Furthermore, according to the embodiment described above, the irradiation angle of the light irradiated to acquire the first distance image DI1B and the second distance image DI2B is determined according to the angle of view of the visible light image. That is, according to this embodiment, light with an irradiation angle corresponding to the angle of view at which the visible light image was captured is irradiated. Therefore, according to this embodiment, the effect of multipath can be reduced without irradiating light outside the angle of view.

Furthermore, according to the embodiment described above, the three-dimensional information acquisition device 10B further includes a visible light image generation unit, which generates pixel information to be added to the three-dimensional information of the object T based on the acquired first and second visible light images. Furthermore, the visible light image generation unit complements pixel values at coordinates in the first visible light image that have a lower resolution than the second visible light image using a predetermined method. The predetermined method may be, for example, linear interpolation. Therefore, according to this embodiment, high-resolution three-dimensional information can be easily generated.

Although the embodiments of the present invention have been described above, the present invention is not limited to the above-mentioned embodiments, and various modifications can be made without departing from the spirit of the present invention. Furthermore, the above-mentioned embodiments may be combined as appropriate.

The present invention reduces the effects of multipath and makes it possible to measure the distance to an object.

10...3D information acquisition device, 10A...3D information acquisition device, 110...light receiving unit, 111...lens, 112...visible light reflecting dichroic film, 113...image sensor, 114...sensor, 120...irradiation unit, 121...first irradiation unit, 122...second irradiation unit, 130...control unit, 131...ToF sensor control unit, 132...first distance image acquisition unit, 133...second distance image acquisition unit, 134...3D information generation unit, 140...irradiation unit, 141...light emitting unit, 142...diffusion unit, 150...background information acquisition unit, 160...magnification information acquisition unit, T...object, BG...background, BM...irradiation light, DI1...first distance image, DI2...second distance image, BGI...waveform information, ZI...magnification information, VL...visible light, IL...infrared light, OA...optical axis

Claims (17)

a first distance image acquisition unit that acquires a first distance image that is a distance image including a plurality of distance values obtained according to the time from when light is emitted to when the light reflected by the object is received;
a second distance image acquisition unit that acquires a second distance image, the second distance image being the distance image obtained by irradiating the same object as the first distance image with light, the second distance image being the distance image obtained by irradiating the object with light at an illumination angle different from that of the first distance image;
a generation unit that determines the effect of multipath caused by light reflection based on the first distance image and the second distance image, and generates three-dimensional information of the object by combining the first distance image and the second distance image based on the determination result. the distance image has distance values corresponding to each coordinate in a two-dimensional coordinate system;
The three-dimensional information acquisition device according to claim 1 , wherein the generation unit compares the distance values contained in the first distance image with the distance values contained in the second distance image at the same coordinates, and determines the effect of multipath caused by light reflection based on the comparison result. an illumination angle of light irradiated to obtain the first distance image is wider than an illumination angle of light irradiated to obtain the second distance image;
the generation unit determines that the first distance image is affected by multipath when a comparison result between a distance value included in the first distance image and a distance value included in the second distance image at the same coordinate shows a difference equal to or greater than a predetermined value, and adopts the distance value of the second distance image as the distance value of the coordinate.
The three-dimensional information acquisition device according to claim 2 . An illumination unit that illuminates light for acquiring the distance image,
The three-dimensional information acquisition device according to claim 1 or 2, wherein the irradiating section is capable of changing a direction in which the light is irradiated depending on a position of the object. A visible light image acquisition unit that acquires a visible light image of the object is further provided,
The three-dimensional information acquisition device according to claim 4 , wherein the irradiating section is capable of changing a direction in which the light is irradiated depending on a position of the object obtained by analyzing the visible light image. a light receiving unit that receives, on the same optical axis, the light irradiated by the irradiating unit and reflected by the object and visible light for acquiring the visible light image,
The three-dimensional information acquisition device according to claim 5 . An illumination unit that illuminates light for acquiring the distance image,
The irradiation unit includes:
A light emitting unit that emits light;
A diffusion unit that diffuses the light emitted by the light emitting unit,
The three-dimensional information acquisition device according to claim 1 or 2, wherein the distance between the light emitting section and the diffusion section is variable, so that light is irradiated at different irradiation angles. a first distance image acquisition unit that acquires a first distance image, which is a distance image including a plurality of distance values obtained according to the time from when light is emitted to when the light reflected by the object and the background is received;
a second distance image acquisition unit that acquires a second distance image, the second distance image being the distance image obtained by irradiating light onto the same object as the first distance image, the second distance image being the distance image obtained using a background having a lower reflectance of near-infrared rays than the background in the first distance image;
a generation unit that determines the effect of multipath caused by light reflection based on the first distance image and the second distance image, and generates three-dimensional information of the object by combining the first distance image and the second distance image based on the determination result. the distance image has distance values corresponding to each coordinate in a two-dimensional coordinate system;
The three-dimensional information acquisition device according to claim 8 , wherein the generation unit compares the distance values contained in the first distance image with the distance values contained in the second distance image at the same coordinates, and determines the effect of multipath caused by light reflection based on the comparison result. a first visible light image acquisition unit that acquires a first visible light image that is a visible light image obtained by capturing an image of the object and a background corresponding to the first distance image;
a second visible light image acquisition unit that acquires a second visible light image that is a visible light image obtained by capturing an image of the object and a background corresponding to the second distance image,
The three-dimensional information acquisition device according to claim 8 , wherein the first visible light image and the second visible light image are imaged under different imaging conditions for determining brightness of the visible light image. a first visible light image acquisition unit that acquires a first visible light image that is a visible light image obtained by capturing an image of the object and a background corresponding to the first distance image;
a second visible light image acquisition unit that acquires a second visible light image that is a visible light image obtained by capturing an image of the object and a background corresponding to the second distance image;
a visible light image generating unit that generates pixel information to be assigned to the three-dimensional information of the object based on the acquired first visible light image and the acquired second visible light image,
The three-dimensional information acquisition device according to claim 8 , wherein the visible light image generation section generates the pixel information according to a result of comparing pixel values of the acquired first visible light image with pixel values of the acquired second visible light image. A three-dimensional information acquisition device according to any one of claims 8 to 11,
an infrared absorbing sheet that is used when acquiring the second distance image and that serves as a background with low reflectance of near-infrared rays. a first distance image acquisition unit that acquires a first distance image that is a distance image including a plurality of distance values obtained according to the time from when light is emitted to when the light reflected by the object is received;
a second distance image acquisition unit that acquires a second distance image, the second distance image being obtained by irradiating a part of the object identical to the first distance image with light at a different illumination angle from the first distance image, and the second distance image being obtained by magnifying the part of the object identical to the first distance image with an optical member;
a generation unit that determines the effect of multipath caused by light reflection based on the first distance image and the second distance image, and generates three-dimensional information of the object by combining the first distance image and the second distance image based on the determination result. The three-dimensional information acquisition device according to claim 13 , wherein the first distance image and the second distance image are acquired while keeping an optical axis fixed. A visible light image acquisition unit that acquires a visible light image of the object is further provided,
The three-dimensional information acquisition device according to claim 13 or 14, wherein a magnification ratio achieved by the optical member is based on a visible light image acquired by the visible light image acquisition unit. the visible light image acquisition unit acquires a first visible light image which is a visible light image having substantially the same angle of view as the first distance image, and a second visible light image which is a visible light image having substantially the same angle of view as the second distance image,
The three-dimensional information acquisition device according to claim 15 , wherein an illumination angle of light emitted to acquire each of the distance images is determined in accordance with an angle of view of the corresponding visible light image. a visible light image generating unit configured to generate pixel information to be assigned to three-dimensional information of the object based on the acquired first visible light image and the acquired second visible light image,
The three-dimensional information acquisition device according to claim 16 , wherein the visible light image generation section complements pixel values at coordinates in the first visible light image that have a lower resolution than the second visible light image, using a predetermined method.

Images