CN111721239A - Depth data measurement equipment and structured light projection device - Google Patents

Abstract

A depth data measuring apparatus and a structured light projecting device included therein are disclosed. The apparatus comprises: a projection device for projecting the structured light to a photographic subject; an imaging device for photographing the photographic subject to obtain a two-dimensional image frame under the structured light irradiation, wherein the structured light projecting device includes: a laser generator for generating laser light; a Liquid Crystal On Silicon (LCOS) device to acquire the laser light and generate structured light for projection. The invention uses LCOS to carry out fine projection of structured light, and improves the imaging precision of depth data. LCOS can also transform various projection codes including speckle or fringe to meet various imaging scenarios. Further, a VCSEL structure can be employed in combination to achieve low power consumption and miniaturization of the projection apparatus.

Description

Depth data measurement equipment and structured light projection device

technical field

The invention relates to the field of three-dimensional imaging, in particular to a depth data measurement device and a structured light projection device.

Background technique

A depth camera is a collection device that collects the depth information of a target object. This type of camera is widely used in 3D scanning, 3D modeling and other fields. For example, more and more smartphones are now equipped with depth sensors for face recognition. camera device.

Although 3D imaging has been a research hotspot in the field for many years, the existing depth cameras still have many problems, such as high power consumption, large size, poor anti-interference ability, and inability to achieve precise real-time imaging.

To this end, there is a need for an improved depth data measurement device.

SUMMARY OF THE INVENTION

A technical problem to be solved by the present disclosure is to provide an improved depth data measurement device, which uses LCOS to perform fine projection of structured light, thereby improving the imaging accuracy of depth data. LCOS can also transform various projection codes including speckle or fringe to meet various imaging scenarios. Further, the VCSEL structure can be adopted to achieve low power consumption and miniaturization of the projection device.

According to a first aspect of the present disclosure, there is provided a depth data measurement device, comprising: a projection device for projecting structured light to a photographic object; an imaging device for photographing the photographed object to obtain the structure A two-dimensional image frame under light irradiation, wherein the structured light projection device comprises: a laser generator for generating laser light; a liquid crystal on silicon (LCOS) device for acquiring the laser light and generating a structure for projection Light. Thus, pixel-level precision projection pattern control is performed using LCOS. Further, the switching of each pixel of the LCOS device can be controlled by, for example, a processing device to generate different projected structured light patterns. This expands the application field of the device.

Optionally, the laser generator includes a vertical cavity surface emitting laser (VCSEL) for generating the laser light. Thus, the vertical emission performance of the VCSEL can be utilized to further reduce the volume, power consumption and heat generation.

Optionally, the projection device may include: a diffusion sheet arranged on the propagation light path of the laser light to convert the laser light generated by the VCSEL into a surface light source. The projection device may further include: a shaping optical component for providing the surface light source generated by the diffusing sheet to the LCOS device.

Optionally, polarized light can be generated by utilizing the characteristics of the VCSEL, and the LCOS device can control the reflection of the light by adjusting the phase difference of the liquid crystal corresponding to each pixel.

Optionally, the VCSEL may be a light-emitting array including a plurality of light-emitting units, and the VCSEL may turn off a specific row, column or light-emitting unit according to the projected structured light pattern. In other words, the VCSEL itself can emit various light-emitting patterns

Optionally, the projection device may further include: a lens group for projecting the structured light generated by the LCOS device.

Optionally, the device may be a monocular imaging device, so the imaging device further includes: an image sensor with a fixed relative distance from the projection device, wherein the image sensor captures the two-dimensional image of the structured light obtained by shooting Image frames are used for comparison with reference structured light image frames to obtain depth data for the subject.

As an alternative, the device may also be a binocular imaging device, so the imaging device may further include: first and second image sensors with a fixed relative distance from the projection device, used for photographing the photographed object to obtain first and second two-dimensional image frames under the illumination of the structured light, wherein based on the first and second two-dimensional image frames and predetermined relative positions between the first and second image sensors The relationship obtains the depth data of the photographed object.

Optionally, the structured light projected by the projection device is infrared structured light, and the imaging device further includes: a visible light sensor for photographing the photographed object to obtain a two-dimensional image frame under visible light irradiation. This provides color information of the subject.

Optionally, the apparatus may further include: a processing device connected to the projection device and the imaging device, for controlling the projection of the projection device and the imaging of the imaging device. Further, the processing device is configured to: obtain the depth data of the photographed object by using the two-dimensional image frame photographed by the imaging device.

In different implementations, the LCOS device may be used for: projecting encoded discrete light spots distributed in a two-dimensional plane, and the imaging device is used for synchronously photographing the projected structured light distributed in a two-dimensional plane to The two-dimensional image frame is acquired. The LCOS device can also be used to project a group of structured lights with different fringe codes respectively, and the imaging device is used for synchronously photographing each projected structured light to obtain a group of two-dimensional image frames, the group of two The three-dimensional image frames are jointly used to obtain the depth data of the photographed object once. Specifically, the LCOS device may be used for: scanning and projecting the fringe code, and the imaging device includes: a rolling shutter sensor that synchronously turns on pixel columns in the fringe direction corresponding to the current scanning position to perform imaging.

The camera head can be set independently, and for this purpose, the apparatus may further include: a casing for accommodating the projection device and the imaging device, and for fixing the relative positions of the projection device and the imaging device. Further, the apparatus may further comprise: a signal transmission device connected to the projection device and the imaging device through the casing, for inwardly transmitting control for the projection device and the imaging device signal, and transmit the two-dimensional image frame outward.

According to a second aspect of the present disclosure, there is provided a structured light projection device comprising: a vertical cavity surface emitting laser (VCSEL) for generating the laser light. A liquid crystal on silicon (LCOS) device for capturing the laser and generating structured light for projection. Further, the device may further include: a diffuser, arranged on the propagation light path of the laser light, to convert the laser light generated by the VCSEL into a surface light source; a shaping optical component for converting the surface light source generated by the diffuser provided to the LCOS device; and a lens group for projecting the structured light generated by the LCOS device outward. Therefore, it can be used for structured light projection of various depth data computing devices.

Therefore, the depth data measurement device of the present invention uses LCOS to perform fine projection of structured light, thereby improving the imaging accuracy of depth data, and is especially suitable for depth data measurement of tiny objects or details. LCOS can also transform various projection codes including speckle or fringe to meet various imaging scenarios. Further, a VCSEL structure can be used to achieve low power consumption and miniaturization of the projection device, the VCSEL can have an array structure, and can partially emit light to further reduce power consumption and device heat generation.

Description of drawings

The above and other objects, features and advantages of the present disclosure will become more apparent from the more detailed description of the exemplary embodiments of the present disclosure taken in conjunction with the accompanying drawings, wherein the same reference numerals generally refer to the exemplary embodiments of the present disclosure. same parts.

FIG. 1 shows a schematic diagram of an example of an object to be measured.

Figure 2 shows a schematic diagram of discrete spots projected by the laser beam onto the surface of the object to be measured.

FIG. 3 shows a schematic structural diagram of a depth data measurement device according to an embodiment of the present invention.

Figure 4 illustrates the principle of depth imaging with fringe-encoded structured light.

FIG. 5 shows a schematic diagram of the composition of a depth data measurement device according to an embodiment of the present invention.

FIG. 6 shows a schematic composition diagram of a depth data measurement device according to an embodiment of the present invention.

FIG. 7 shows a light emitting path of the projection apparatus shown in FIG. 3 .

Detailed ways

Preferred embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

The three-dimensional measurement method based on structured light detection adopted in the present invention can perform three-dimensional measurement on the surface of an object in real time.

The three-dimensional measurement method based on structured light detection is a method that can perform real-time three-dimensional detection on the surface of moving objects. Simply put, the measurement method first projects a two-dimensional laser texture pattern with encoded information on the surface of a natural body, such as a discretized speckle pattern, and the laser texture is continuously collected by another relatively fixed image acquisition device. Compare the collected laser texture pattern with the reference surface texture pattern with known depth distance stored in the memory in advance, and calculate the projection on the surface of the natural body according to the difference between the collected texture pattern and the known reference texture pattern. The depth distance of each laser texture sequence segment is obtained, and the three-dimensional data of the surface of the object to be measured is obtained by further measurement. The three-dimensional measurement method based on structured light detection adopts the method of parallel image processing, so it can detect moving objects in real time, and has the advantages of fast and accurate three-dimensional measurement, which is especially suitable for use environments with high real-time measurement requirements.

FIG. 1 shows a schematic diagram of an example of an object to be measured. The figure schematically shows the human hand as the object to be tested. Figure 2 shows a schematic diagram of discrete spots projected by the laser beam onto the surface of the object to be measured. In the scene of monocular imaging, the discrete spot image shown in Figure 2 can be compared with the reference standard image to calculate the depth data of each discrete spot, and thus integrate the overall image of the object to be measured. depth data. As can be seen from Figure 2, since there is a certain distance between the discrete laser spots, more spot information cannot be emitted for the narrower position of the projection surface, so it is easy to lose part of the real depth information.

In the prior art, there is no structured light projection device capable of fine projection, so it is impossible to measure the depth data of fine objects with high precision.

To this end, the present invention provides an improved depth data measurement device, which uses LCOS to perform fine projection of structured light, thereby improving the imaging accuracy of depth data. LCOS can also transform various projection codes including speckle or fringe to meet various imaging scenarios. Further, the VCSEL structure can be adopted to achieve low power consumption and miniaturization of the projection device.

FIG. 3 shows a schematic structural diagram of a depth data measurement device according to an embodiment of the present invention. As shown, the depth data measurement apparatus 300 may include a projection device 310 and an imaging device 320 .

The projection device 310 is used for projecting structured light to the photographed object. The imaging device 320 is used for photographing the photographed object to obtain a two-dimensional image frame under the illumination of the structured light.

In order to show the internal structure of the projection device 310, FIG. 3 does not show the housing and/or the fixing parts of the projection device 310, and the above-mentioned housings and/or fixing parts can be used to fix the relative positions of the various components shown in the figure, and It can have the effect of protecting the device from external contamination and external shock damage.

As shown in the figure, the projection device for projecting structured light mainly includes two devices: a laser generator 311 and a liquid crystal on silicon (LCOS) device 312 .

Here, the laser generator 311 is used to generate laser light. A liquid crystal-on-silicon (LCOS) device can be used as a generator of projected patterns for capturing the laser and generating structured light for projection. As a result, extremely high-precision projection pattern control is performed using LCOS. Further, the switching of each pixel of the LCOS device can be controlled by, for example, a processing device inside or outside the device, so as to generate different projected structured light patterns. This expands the application scenarios of the device.

Here, LCOS (Liquid Crystal on Silicon), that is, liquid crystal on silicon, also called liquid crystal on silicon, is a very small matrix liquid crystal display device based on reflection mode. This matrix is fabricated on a silicon chip using CMOS technology.

Specifically, LCOS can use a CMOS integrated circuit chip coated with liquid crystal silicon as a substrate of a reflective LCD. It is polished with advanced technology and then coated with aluminum as a mirror to form a CMOS substrate. Then, the CMOS substrate is attached to the glass substrate containing the transparent electrode, and then injected into the liquid crystal package. LCOS places the control circuit behind the display device, which can improve light transmittance, resulting in greater light output and higher resolution.

LCOS can be regarded as a kind of LCD. The traditional LCD is made on a glass substrate, and the LCOS is made on a silicon wafer. Due to the reflective projection, the light utilization efficiency can reach more than 40%. The structure of LCOS panel is similar to that of TFT LCD. After the partitions are distributed between the upper and lower substrates for isolation, the liquid crystal is filled between the substrates to form a light valve. The switch of the circuit drives the rotation of the liquid crystal molecules to determine the projected brightness. with dark. The upper substrate of the LCOS panel may be ITO conductive glass, and the lower substrate may be a CMOS substrate coated with liquid crystal silicon. Since the material of the lower substrate is single-crystal silicon, it has good electron mobility, and single-crystal silicon can form thin lines, so high resolution can be achieved. The pixel pitch (ie, the horizontal distance between two pixels of the same color) of existing LCOS devices can be extremely small, eg, 8 to 20 micrometers (10-6).

In the present invention, since the laser generator preferably projects infrared light, for example, infrared light of 940 nm, the LCOS device used in the present invention is different from the LCOS panel commonly used for displaying RGB three colors in the prior art to generate a projection of the pattern at one wavelength (ie, only "monochromatic"). Therefore, the LCOS device of the present invention can have a smaller pixel pitch, thereby realizing the projection of extremely fine structured light patterns.

In one embodiment, the laser generator 311 may comprise a vertical cavity surface emitting laser (VCSEL) or be implemented in particular. A VCSEL can be used to generate the laser. Thus, the vertical emission performance of the VCSEL can be utilized to further reduce the volume, power consumption and heat generation.

Further, as shown in FIG. 3 , the projection device 310 may further include: a diffuser 313 arranged on the propagation light path of the laser light to convert the laser light generated by the VCSEL into a surface light source. Thus, the LCOS device 312 is provided with the background light it needs. Further, the projection device may further include: a shaping optical component 314 for shaping the surface light source generated by the diffusing sheet (for example, shaping into a shape conforming to the LCOS device) to provide the LCOS device.

In addition, the projection device 310 may further include: a lens group for projecting the structured light generated by the LCOS device.

It should be understood that although the diffuser 313, shaping optics 314, and lens group 315 for projection are shown, in other embodiments, one or more of the above components may be omitted (eg, such that the VCSEL 311 The outgoing shape of the LCOS directly conforms to the cross-sectional shape required by the LCOS, so as to omit the shaping optical component 314), or replace or add other components. All of these conventional optical modifications are within the scope of the present invention.

Further, based on the principle of LCOS reflecting polarized light, the VCSEL 311 can directly generate polarized light, and the LCOS device controls the light reflection by adjusting the phase difference of the liquid crystal corresponding to each pixel. Since the polarized light projected by the LCOS 312 through the lens group 315 can reduce the adverse effect of specular reflection on the imaging quality of the imaging device 320, thereby improving the imaging quality. Further, the device can also be used for high precision flaw inspection of reflective surfaces (eg glass surfaces).

Additionally, while the LCOS 312 itself is a pixel matrix consisting of multiple pixels, the projected pattern can be precisely controlled by controlling the "switch" of each pixel (eg, controlling the angle of the liquid crystal in the pixel to incident polarized light). However, on the other hand, the VCSEL 311 may also include a matrix structure, eg, a light-emitting array composed of a plurality of light-emitting units. To this end, in some embodiments, the VCSEL 311 can also turn off a specific row, column or light-emitting unit according to the projected structured light pattern when emitting laser light. In other words, although the VCESL 311 is used as the surface light source of the LCOS 312 , the emission pattern of the VCESL 311 still has a certain correlation with the pattern of the surface light source received by the LCOS 312 , and can be fine-tuned by the LCOS 312 precisely.

For example, in some cases, the projection device 310 may project a fringe pattern as structured light and finely image it. According to the principle of structured light measurement, whether the scanning angle α can be accurately determined is the key to the entire fringe pattern measurement system. In the present invention, the determined scanning angle can be realized by LCOS. The scanning angle of the surface structured light system. Figure 4 illustrates the principle of depth imaging with fringe-encoded structured light. For the convenience of understanding, the coding principle of stripe structured light is briefly described in the figure with two-gray-level three-bit binary time coding. The projection device can sequentially project three patterns as shown in the figure to the measured object in the shooting area, and the projection space is divided into 8 areas by light and dark grayscale respectively in the three patterns. Each area corresponds to its own projection angle, wherein it can be assumed that the bright area corresponds to the code "1", and the dark area corresponds to the code "0". Combine the encoding values of a point on the scene in the projection space in the three encoding patterns in the projection order to obtain the area encoding value of the point, thereby determining the area where the point is located and then decoding to obtain the scan angle of the point.

When projecting the leftmost pattern of Figure 4, in one embodiment, VCESL 311 may be fully lit and projected by LCOS 312 by turning off the left column of pixels corresponding to 0-3. In another embodiment, the VCESL 311 may be partially lit, eg, the lighting corresponds to the right side portion (typically need not be an exact 4-7, but could be a wider 3-7 portion), thereby ensuring that the LCOS 312 The pixel columns corresponding to 4-7 receive sufficient backlight and are projected by LCOS 312 by turning off the left pixel column corresponding to 0-3.

Thus, by turning off some of the light-emitting units of the VCSEL during projection, the power consumption of the VCSEL can be further reduced, thereby reducing the heat generated by the device, and obtaining more rest time for each light-emitting unit of the VCSEL. Therefore, it is especially suitable for use in heat-sensitive scenarios, and can prolong the life of the VCSEL.

As shown in FIG. 3 , the depth data measurement device of the present invention may be a monocular device, that is, only one camera is included to capture structured light. To this end, the imaging device 320 may be implemented as an image sensor with a fixed relative distance from the projection device, wherein the two-dimensional image frame of the structured light captured by the image sensor is used for comparison with the reference structured light image frame , to obtain the depth data of the photographed object.

As an alternative, the depth data measurement device of the present invention may be a binocular device, that is, it includes two cameras to capture structured light synchronously, and uses the parallax in the two images to obtain depth data. To this end, the imaging device further includes: first and second image sensors with a fixed relative distance from the projection device, used for photographing the photographed object to obtain the first and second image sensors under the illumination of the structured light. and a two-dimensional image frame, wherein the depth data of the photographed object is obtained based on the predetermined relative positional relationship between the first and second two-dimensional image frames and the first and second image sensors.

In a binocular imaging system, the above-described decoding process such as the fringe encoding shown in FIG. 4 can be simplified by directly matching the encoded values of the respective points in the first and second image sensors. In order to improve the matching accuracy, the number of projected patterns in the time encoding can be increased, such as a five-bit binary time encoding with two gray levels. In the application scenario of binocular imaging, this means that, for example, each pixel in each image frame of the left and right contains 5 or 0 or 1 area coding values, which can be realized with higher precision (eg, pixel level) Left and right image matching. Under the condition that the projection rate of the projection device is unchanged, compared with the three encoding patterns in FIG. 4 , the five encoding patterns are equivalent to achieving higher-precision image matching at a higher time-domain cost. This is still quite desirable when the original projection rate of the projection device is extremely high (eg, fast switching of the LCOS projection pattern).

As mentioned above, the structured light projected by the projection device is preferably infrared structured light, so as to avoid the interference of visible light. At this time, the depth data measurement device of the present invention may further include: a visible light sensor, used for photographing the photographed object to obtain a two-dimensional image frame under the illumination of visible light. For example, an RGB sensor may be included to obtain color 2D information of the photographed object for combining with the obtained depth information, for example, to obtain 3D information, or as a supplement or correction for deep learning.

At this time, the LCOS device can be used to: respectively project a group of structured lights with different fringe codes (for example, three groups as shown in FIG. 4, or more groups of fringe patterns), and the imaging device is used for synchronous shooting Each type of projected structured light is used to obtain a group of two-dimensional image frames, and the group of two-dimensional image frames is used to obtain the depth data of the photographed object once.

In some cases, LCOS devices can project a complete pattern at a time. In other cases, the LCOS device can be used to scan and project the fringe code, and the imaging device includes: a rolling shutter sensor that simultaneously turns on pixel columns in the fringe direction corresponding to the current scanning position for imaging. For example, a VCSEL can turn on some of its own arrays in turn, and cooperate with the LCOS's turn-by-turn reflection (ie, the LCOS projects a structured light pattern of several turns lit up), and synchronizes with the pixel arrays of the rolling shutter sensor. Thereby, the heat dissipation of the VCSEL is further reduced, and the interference of ambient light on structured light imaging is avoided.

FIG. 5 shows a schematic diagram of the composition of a depth data measurement device according to an embodiment of the present invention. As shown in FIG. 5 , the depth data measurement head 500 includes a projection device 510 and two image sensors 520_1 and 520_2.

The projection device 510 is used for scanning and projecting structured light with fringe coding to the shooting area. For example, in successive 3 image frame projection periods, the projection device 510 can successively project the three patterns shown in FIG. 4 , and the imaging results of the three patterns can be used to generate depth data. The first and second image sensors 520_1 and 520_2 respectively have a predetermined relative positional relationship, and are used for photographing the photographing area to obtain first and second two-dimensional image frames under the illumination of structured light, respectively. For example, in the case where the projection device 510 projects three patterns as shown in FIG. 1 , the first and second image sensors 520_1 and 520_2 may respectively perform the projection of the three patterns projected with the three patterns in three synchronized image frame imaging periods. The photographing area (for example, the imaging plane in FIG. 5 and the area within a certain range before and after it) is imaged.

As shown in FIG. 5 , the projection device 510 may project linear light extending in the x direction in the z direction (ie, toward the shooting area). In different embodiments, the projection of the above-mentioned line-shaped light may be already shaped (ie, the outgoing light itself is the line-shaped light), or it may be a light spot moving in the x-direction (ie, the scanned line-shaped light) Light). Correspondingly, the LCOS in the projection device 510 may reflect one or more pixel columns (line light), or a pixel block (light spot) composed of one or more pixels. The projected linear light can move continuously in the y-direction to cover the entire imaging area. The lower part of Figure 5 gives a more understandable illustration of the scanning of the line-shaped light for the perspective view of the shot area.

In the embodiment of the present invention, the direction in which the light exits the measuring head is defined as the z direction, the vertical direction of the photographing plane is the x direction, and the horizontal direction is the y direction. Therefore, the striped structured light projected by the projection device may be the result of the linear light extending in the x direction moving in the y direction. Although in other embodiments, it is also possible to perform synchronization and imaging processing on the stripe structured light obtained by moving the linear light extending in the horizontal y direction in the x direction, but in the embodiment of the present invention, it is still preferable to use vertical stripe light Be explained.

Further, the measuring head 500 also includes a synchronization device 550, which can be implemented by, for example, the following processing device. The synchronization device 550 is connected to the projection device 510 (including both VCSEL and LCOS) and the first and second image sensors 520_1 and 520_2, respectively, to achieve precise synchronization among the three. Specifically, the synchronization device 550 may, based on the scanning position of the projection device 510, simultaneously turn on the pixel columns in the stripe direction corresponding to the current scanning position in the first and second image sensors 520_1 and 520_2 to perform imaging. As shown in Figure 5, the current stripe is being scanned to the center of the shot area. To this end, in the image sensors 520_1 and 520_2, pixel columns (eg, 3 adjacent pixel columns) located in the central region are turned on for imaging. As the fringes move in the y direction (as shown by the arrows in the lower perspective view of FIG. 5 ), the pixel columns in image sensors 520_1 and 520_2 that are turned on for imaging also move synchronously (as shown above the matrix in the upper left block diagram of FIG. 5 ) arrows). In this way, the one-dimensional characteristic of the fringe image can be used to control the range of the pixel column for imaging at each moment, thereby reducing the adverse effect of ambient light on the measurement result. In order to further reduce the influence of ambient light, the projection device is particularly suitable for projecting light that is not easily confused with ambient light, such as infrared light. In addition, since the corresponding relationship between the pixel row and the scanning light is affected by many factors such as the width, power, speed of the projected light, and the photosensitive efficiency of the image sensor, the range (and the corresponding number) of pixel rows that are turned on synchronously each time can be based on calibration, for example. operation to confirm.

In other embodiments, the LCOS device can also be used to project the coded discrete light spots distributed in a two-dimensional plane, and the imaging device is used for synchronously photographing the projected structured light distributed in the two-dimensional plane to obtain all the light spots. the two-dimensional image frame. For example, an LCOS device can project a discrete spot of light like the one shown in Figure 2 (but with much greater precision and typically a much smaller subject).

As mentioned above, the structured light projected by the projection device is preferably infrared structured light, so as to avoid the interference of visible light. At this time, the depth data measurement device of the present invention may further include: a visible light sensor, used for photographing the photographed object to obtain a two-dimensional image frame under the illumination of visible light. For example, an RGB sensor may be included to obtain color 2D information of the photographed object for combining with the obtained depth information, for example, to obtain 3D information, or as a supplement or correction for deep learning.

In different embodiments, the apparatus may be implemented as a measuring head only for capturing functions, or may contain processing and computing means. In addition, in the case of including processing and computing equipment, according to different applications, the measuring head and the processing and computing device can be packaged in the same housing, or connected separately via a signal transmission device.

Although not shown in FIG. 3 , the depth data measuring apparatus of the present invention may further include: a processing device (control function) connected to the projection device and the imaging device for controlling the projection of the projection device and the imaging Imaging of the device. For example, the processing device may be used to: control the switching of pixels of the LCOS device to generate different projected structured light patterns.

In addition, the processing device may further include a computing function and be configured to: obtain depth data of the photographed object by using the two-dimensional image frame photographed by the imaging device.

Further, the depth data measuring apparatus of the present invention may further comprise: a casing for accommodating the projection device and the imaging device, and for fixing the relative positions of the projection device and the imaging device. The fixture 330 shown in Figure 3 may be part of the housing.

In some embodiments, processing means for control and/or computation may be included within the housing. But in some cases, it is necessary to separate the camera and handle this setup. At this time, the apparatus may include: a signal transmission device connected to the projection device and the imaging device through the casing, for inwardly transmitting a control signal for the projection device and the imaging device, and transmitting the two-dimensional image frame outward. When the depth data measuring apparatus of the present invention includes a processing device, the above-mentioned signal transmission device may be a signal connection line with the processing device, such as an optical fiber or a coaxial cable. When the device itself does not include a processing function, the above-mentioned signal transmission device may be a connection interface with the processing device.

FIG. 6 shows a schematic composition diagram of a depth data measurement device according to an embodiment of the present invention.

As shown, the depth data measurement apparatus includes a separate measurement head 600 , a signal transmission device 640 and a processor 650 . The figure schematically shows a perspective view of the measuring head 600 , as well as a schematic diagram of a cable of a signal transmission device (transmission cable) 640 and a schematic diagram of a symbol of the processor 650 . It should be understood that, in different implementations, the processor 650 may be surrounded by a separate processor housing, or inserted into other devices, such as the computing motherboard of the acquisition device described below, or fixed in other ways. There is no restriction on this disclosure.

The measuring head completes the active projection of structured light and the binocular measurement for structured light. The measurement head 600 may include a structured light projection device 610 , first and second image sensors 620_1 and 620_2 having a predetermined relative positional relationship, and a housing 630 .

The structured light projection device 610 can be used to project structured light to the photographed object, and includes the structure of the VCSEL combined with the LCOS as described above. The first and second image sensors 620_1 and 620_2 are used to photograph the photographed object to obtain first and second two-dimensional image frames under the illumination of the structured light, respectively. The casing 630 is used to accommodate the structured light projection device and the first and second image sensors, and to fix the relative positions of the structured light projection device and the first and second image sensors.

The signal transmission device 640 can be connected to the structured light projection device 111 and the first and second image sensors through the casing 630, and is used to transmit (inside the casing) a target for the structured light projection device. 610 and the control signals of the first and second image sensors, and transmit the first and second two-dimensional image frames captured by the image sensor to the outside (outside the casing).

The processor 650 is connected to the signal transmission device 640 and is located outside the housing 630, and is used for sending the control signal through the signal transmission device, and based on the continuously acquired first and second two-dimensional image frames and The predetermined relative positional relationship between the first and second image sensors calculates motion data of the photographed object.

Therefore, the depth data measurement apparatus of the present invention can set the measurement head to be miniaturized, light weight and low heat dissipation by separating the measurement head from the processor (eg, processing circuit), thereby facilitating imaging in, for example, medical imaging equipment installation in the space.

Here, the signal transmission device 640 may include a coaxial cable, whereby control signals and image data are directly transmitted through electrical signals. Further, in a high magnetic field environment such as MRI acquisition, in order to avoid using iron-nickel materials, an optical fiber can be used as the signal transmission device 640 . At this time, the structured light projection device, the image sensor and the processor may each include a photoelectric converter for converting an optical signal transmitted by the optical fiber into an electrical signal, or converting a signal to be transmitted into an optical signal.

In another embodiment, the present invention can also be implemented as a structured light projection device. The apparatus may include: a vertical cavity surface emitting laser (VCSEL) for generating the laser light; and a liquid crystal on silicon (LCOS) device for capturing the laser light and generating structured light for projection. Further, the device may further include: a diffusion sheet, arranged on the propagation light path of the laser light, to convert the laser light generated by the VCSEL into a surface light source; a shaping optical component for converting the surface light generated by the diffusion sheet a light source is provided to the LCOS device; and a lens group for projecting the structured light generated by the LCOS device outward. The above structured light projection device can cooperate with various imaging devices to realize depth data measurement and calculation for various scenes.

It should be understood that since the present invention adopts the LCOS for projection using the reflection principle, the laser generator and the projection lens group can be arranged on the folded optical path, thereby contributing to the compactness and miniaturization of the device. FIG. 7 shows a light emitting path of the projection apparatus shown in FIG. 3 . As shown in the figure, the laser emitted by the laser generator 711, such as VCSEL, is sent to the LCOS 712 via the diffuser 713 and the shaping component 714, and is projected by the lens group 715 after being reflected by the relevant liquid crystal inside the LCOS 712.

The depth data measuring apparatus according to the present invention and the structured light projection device constituting the apparatus have been described in detail above with reference to the accompanying drawings. The present invention uses LCOS to perform fine projection of structured light, thereby improving the imaging accuracy of depth data, and is especially suitable for measuring the depth data of tiny objects or details. LCOS can also transform various projection codes including speckle or fringe to meet various imaging scenarios. Further, a VCSEL structure can be used to achieve low power consumption and miniaturization of the projection device, the VCSEL can have an array structure, and can partially emit light to further reduce power consumption and device heat generation.

Various embodiments of the present invention have been described above, and the foregoing descriptions are exemplary, not exhaustive, and not limiting of the disclosed embodiments. Numerous modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the various embodiments, the practical application or improvement over the technology in the marketplace, or to enable others of ordinary skill in the art to understand the various embodiments disclosed herein.

Claims (20)

1. A depth data measuring apparatus comprising:

a projection device for projecting the structured light to a photographic subject;

an imaging device for photographing the photographic subject to obtain a two-dimensional image frame under the structured light irradiation,

wherein the projection device comprises:

a laser generator for generating laser light;

a Liquid Crystal On Silicon (LCOS) device to acquire the laser light and generate structured light for projection.

2. The apparatus of claim 1, wherein the laser generator comprises:

a Vertical Cavity Surface Emitting Laser (VCSEL) to generate the laser light.

3. The apparatus of claim 2, wherein the projection device comprises:

a diffusion sheet disposed on a propagation path of the laser light to convert the laser light generated by the VCSEL into a surface light source.

4. The apparatus of claim 3, wherein the projection device further comprises:

and the shaping optical assembly is used for providing the surface light source generated by the diffusion sheet to the LCOS device.

5. The apparatus of claim 2, wherein the VCSEL generates polarized light and the LCOS device controls the reflection of the light by adjusting a phase difference of a corresponding liquid crystal for each pixel.

6. The apparatus of claim 2, wherein the VCSEL includes a light emitting array comprised of a plurality of light emitting cells, and the VCSEL turns off a specific row, column or light emitting cell according to a projected structured light pattern when emitting laser light.

7. The apparatus of claim 1, wherein the projection device further comprises:

a lens group to project structured light generated by the LCOS device.

8. The apparatus of claim 1, wherein the imaging device further comprises:

and the image sensor is fixed in relative distance with the projection device, wherein the two-dimensional image frame of the structured light obtained by the image sensor is used for being compared with the reference structured light image frame to obtain the depth data of the shooting object.

9. The apparatus of claim 1, wherein the imaging device further comprises:

and first and second image sensors fixed in relative distance from the projection device, for photographing the photographic subject to obtain first and second two-dimensional image frames under the structured light irradiation, wherein the depth data of the photographic subject is found based on the first and second two-dimensional image frames and a predetermined relative positional relationship between the first and second image sensors.

10. The apparatus of claim 1, wherein the structured light projected by the projection device is infrared structured light, and the depth data measuring apparatus further comprises:

and the visible light sensor is used for shooting the shooting object to obtain a two-dimensional image frame under the irradiation of visible light.

11. The apparatus of claim 1, further comprising:

and the processing device is connected with the projection device and the imaging device and is used for controlling the projection of the projection device and the imaging of the imaging device.

12. The apparatus of claim 10, wherein the processing device is to:

and obtaining the depth data of the shooting object by utilizing the two-dimensional image frame shot by the imaging device.

13. The apparatus of claim 10, wherein the processing device is to:

controlling the LCOS device pixels to open and close to generate different projected structured light patterns.

14. The apparatus of claim 1, wherein the LCOS device is to:

projecting the encoded discrete spots in a two-dimensional planar distribution,

and the imaging device is used for synchronously shooting the projected structured light distributed in a two-dimensional plane to acquire the two-dimensional image frame.

15. The apparatus of claim 1, wherein the LCOS device is to:

a set of structured light with different fringe codes is projected separately,

and the imaging device is used for synchronously shooting each kind of projected structured light to acquire a group of two-dimensional image frames, and the group of two-dimensional image frames are commonly used for solving the depth data of the shooting object once.

16. The apparatus of claim 15, wherein the LCOS device is to:

scan projecting the stripe code, an

The image forming apparatus includes:

and synchronously opening the rolling curtain sensor for imaging the pixel columns in the stripe direction corresponding to the current scanning position.

17. The apparatus of claim 1, further comprising:

and the shell is used for accommodating the projection device and the imaging device and fixing the relative positions of the projection device and the imaging device.

18. The apparatus of claim 17, further comprising:

and the signal transmission device penetrates through the shell and is connected with the projection device and the imaging device, and is used for transmitting control signals for the projection device and the imaging device inwards and transmitting the two-dimensional image frames outwards.

19. A structured light projection device, comprising:

a Vertical Cavity Surface Emitting Laser (VCSEL) to generate the laser light.

A Liquid Crystal On Silicon (LCOS) device to acquire the laser light and generate structured light for projection.

20. The apparatus of claim 19, further comprising:

a diffusion sheet disposed on a propagation light path of the laser light to convert the laser light generated by the VCSEL into a surface light source;

a shaping optical assembly for providing the surface light source produced by the diffuser to the LCOS device; and

a lens group for projecting structured light generated by the LCOS device outward.

Images