CN1203292C

CN1203292C - Method and system for measruing object two-dimensiond surface outline

Info

Publication number: CN1203292C
Application number: CNB031535046A
Authority: CN
Inventors: 钟约先; 李仁举; 张吴明; 马扬飚; 袁朝龙; 叶成蔚
Original assignee: Beijing Tenyoun 3d Technology Co Ltd; Tsinghua University
Current assignee: Beijing Tenyoun 3d Technology Co Ltd; Tsinghua University
Priority date: 2003-08-15
Filing date: 2003-08-15
Publication date: 2005-05-25
Anticipated expiration: 2023-08-15
Also published as: CN1483999A

Abstract

The present invention relates to a method and a system for measuring the outline of the three-dimensional surface of an object, which belongs to the technical field of the three-dimensional measurement of the object. The present invention is characterized in that the present invention uses the combination of a phase and stereo vision technique for projecting a grating on the object surface, grating images occurring aberrance are shot by using two cameras, and each point of phase on the images shot by a left camera and a right camera is obtained by using encoding light and a phase-shifting method. The phase and an outer polar line are used for realizing point matching on two images so that a calibrated camera system can be used for calculating the coordinate of the point in three-dimensional space to measure the three-dimensional outline of the object surface. The present invention has the advantages of non-contact, high speed, large data quantity, high precision, simple operation, easy realization, etc. Only 2 seconds are needed to measure the single surface of the object, high-density data can be obtained and reaches 400, 000 points, and measured precision is more than 0.05mm.

Description

Method for Measuring 3D Surface Profile of Objects

技术领域technical field

测量物体表面三维轮廓的方法属于物体三维测量方法技术领域。A method for measuring a three-dimensional profile of an object surface belongs to the technical field of object three-dimensional measurement methods.

背景技术Background technique

物体的三维测量技术在产品设计与制造、质量检测与控制、机器人视觉等领域中应用很普遍。近些年来其应用也拓展到产品仿制、快速制造系统、产品反求设计、在线检测、服装制作、影视特技、虚拟现实、艺术雕塑等领域。The three-dimensional measurement technology of objects is widely used in product design and manufacturing, quality inspection and control, robot vision and other fields. In recent years, its application has also expanded to product imitation, rapid manufacturing system, product reverse design, online inspection, clothing production, film and television special effects, virtual reality, art sculpture and other fields.

三维轮廓测量主要有两类方法即接触式和非接触式。接触式测量的主要方法为三坐标测量仪，其测量精度高，可达0.5μm。但这种方法不适合柔软实物的测量，测量速度慢，对工作环境要求很高，必须防震、防灰、恒温等，因此应用范围受到较大限制。从整体来看，机械式三坐标测量仪难以满足当今快速、高效率测量的需求。There are two main types of methods for 3D profile measurement, contact and non-contact. The main method of contact measurement is a three-coordinate measuring instrument, which has a high measurement accuracy of up to 0.5 μm. But this method is not suitable for the measurement of soft objects, the measurement speed is slow, and it has high requirements on the working environment. It must be shockproof, dustproof, constant temperature, etc., so the application range is relatively limited. On the whole, the mechanical three-coordinate measuring instrument is difficult to meet the needs of today's fast and high-efficiency measurement.

非接触的三维测量方法有：光学传感器法、激光扫描法、立体视觉法、投影栅相位法等。Non-contact three-dimensional measurement methods include: optical sensor method, laser scanning method, stereo vision method, projection grid phase method, etc.

1.光学传感器法1. Optical sensor method

该方法的工作原理类似于机械式三坐标测量仪，只不过采用专门的光学探头来检测物体表面形状。其光学探头能够直接得到被测点与探头之间的距离，由其工作位置得到其它两个方向上的坐标。日本Toshiba公司研制了一台用于大型高精度光学表面测量的非接触式光学探测仪，面形测量精度达到0.1μm，粗糙度到1nmRa，安装在超精CNC车床的工作台上使用。其关键技术在于光学探头的制造，属精密仪器设备，因此造价昂贵。The method works similar to a mechanical CMM, except that a specialized optical probe is used to detect the surface shape of the object. Its optical probe can directly obtain the distance between the measured point and the probe, and obtain the coordinates in the other two directions from its working position. Toshiba Corporation of Japan has developed a non-contact optical detector for large-scale high-precision optical surface measurement. The surface measurement accuracy reaches 0.1μm and the roughness reaches 1nmRa. It is installed on the workbench of an ultra-precision CNC lathe. Its key technology lies in the manufacture of the optical probe, which is a precision instrument and equipment, so it is expensive.

2.激光扫描法2. Laser scanning method

该方法利用激光扫描物体表面，通过出射点、投影点和成像点三者之间的几何成像关系确定物体各点的三维坐标。根据工作激光光源的特点和性质可以分为点式激光扫描、线状激光扫描等。激光扫描的速度较块，但是扫描精度却受到工件的材质及表面特性等因素的影响。另外，激光扫描系统的价格十分昂贵，非一般用户所能承受。The method uses laser to scan the surface of the object, and determines the three-dimensional coordinates of each point of the object through the geometric imaging relationship among the exit point, projection point and imaging point. According to the characteristics and properties of the working laser light source, it can be divided into point laser scanning, linear laser scanning and so on. The speed of laser scanning is relatively fast, but the scanning accuracy is affected by factors such as the material and surface characteristics of the workpiece. In addition, the price of the laser scanning system is very expensive, which cannot be afforded by ordinary users.

3.立体视觉法3. Stereo vision method

立体视觉法是根据人双眼视觉系统的仿生学原理建立起来的，该方法已经能够达到一定的测量精度。它根据三角测量原理，利用对应点的视差可以计算视野范围内的立体信息，用于双目和多目视觉。The stereo vision method is established according to the bionics principle of the human binocular vision system, and this method has been able to achieve a certain measurement accuracy. Based on the principle of triangulation, it uses the parallax of corresponding points to calculate the stereoscopic information within the field of view, which is used for binocular and multi-eye vision.

这种方法对应用场合要求较宽松，一次能获得一块区域的三维信息，特别是具有不受物体表面反射特性影响的优点。但其中的对应点的匹配问题较难解决，算法复杂，耗时较长。在物体表面特征点较稀疏时，也很难获得精确的形状。This method has relatively loose requirements on the application occasion, and can obtain three-dimensional information of one area at a time, especially has the advantage of not being affected by the reflection characteristics of the object surface. However, the matching problem of corresponding points is difficult to solve, and the algorithm is complex and time-consuming. It is also difficult to obtain an accurate shape when the feature points on the surface of the object are sparse.

4.投影栅相位法4. Projection grating phase method

投影栅采用在物体表面投射栅线，利用调制栅线的相位畸变信息得到物体的三维信息，它采用数学的方法解调相位，并利用相位值计算每个相对于参考面的高度值。投影栅相位法存在的很大问题是其系统操作性不好，难以实现实用化。The projection grid uses the projection grid line on the surface of the object, and uses the phase distortion information of the modulation grid line to obtain the three-dimensional information of the object. It uses a mathematical method to demodulate the phase, and uses the phase value to calculate each height value relative to the reference plane. The big problem with the projected grating phase method is that its system operability is not good, and it is difficult to realize practical application.

发明内容Contents of the invention

本方法的目的在于在于提供一种测量精确、便于操作且易于实用化的测量物体三维表面轮廓的方法，为了达到这个目的本发明开创性的提出将利用相位和立体视觉技术的结合，在物体表面投射光栅，采用双摄像机拍摄图象，利用相位和极线实现匹配，从而达到对物体三维坐标的反求与重构，开发出了可操作性好、测量精度较高、速度快、适合于不同大小物体的新型的三维无接触测量系统。The purpose of this method is to provide a method for measuring the three-dimensional surface profile of an object that is accurate, easy to operate, and easy to use. Projection grating, using dual cameras to shoot images, using phase and epipolar line to achieve matching, so as to achieve the reverse and reconstruction of the three-dimensional coordinates of the object, developed a good operability, high measurement accuracy, fast speed, suitable for different New 3D non-contact measuring system for large and small objects.

具有某种特性的光(称为结构光)投射到物体上，物体上不同高度的点对光栅进行调制使光栅发生畸变。用两架摄像机拍摄发生畸变的光栅图像，利用编码光和相移方法获得左右摄像机拍摄图象上每一点的相位。利用相位和外极线几何实现两幅图像上的点的匹配。对定标了的摄像机系统，便可计算其在三维空间的坐标。Light with certain characteristics (called structured light) is projected onto an object, and points at different heights on the object modulate the grating to distort the grating. The distorted grating image is captured by two cameras, and the phase of each point on the image captured by the left and right cameras is obtained by using coded light and phase shift method. Matching of points on the two images is achieved using phase and epipolar geometry. For the calibrated camera system, its coordinates in three-dimensional space can be calculated.

本发明提出的测量物体三维表面轮廓的方法，其特征在于它是一种利用相位和立体视觉技术的结合，在物体表面投射光栅，再采用双摄像机拍摄发生畸变的光栅图像，利用编码光和相移方法获得左右摄像机拍摄图像上每一点的相位。利用相位和外极线实现两幅图像上的点的匹配，从而达到对物体表面点三维坐标的反求的方法；它包括如下步骤：The method for measuring the three-dimensional surface profile of an object proposed by the present invention is characterized in that it uses a combination of phase and stereo vision technology to project a grating on the surface of the object, and then uses dual cameras to shoot the distorted grating image, and uses coded light and phase The shift method obtains the phase of each point on the image captured by the left and right cameras. Using the phase and the epipolar line to realize the matching of the points on the two images, so as to achieve the method of inversely seeking the three-dimensional coordinates of the surface points of the object; it includes the following steps:

(1)利用计算机生成虚拟光栅，其中包括编码光栅和相移光栅，利用投影仪将生成的光栅投射在物体上；(1) Use a computer to generate a virtual grating, including a coded grating and a phase shift grating, and use a projector to project the generated grating on the object;

相移光栅的光强如下式来表达：The light intensity of the phase shift grating is expressed as follows:

I_i(u，v)＝a(u，v)+b(u，v)cos(φ(u，v)+φ_i)I _i (u, v)=a(u, v)+b(u, v) cos(φ(u, v)+φ _i )

其中：in:

(u，v)为某点的坐标；(u, v) is the coordinates of a certain point;

I_i(u，v)为第i幅图像中(u，v)点的光强；I _i (u, v) is the light intensity of point (u, v) in the i-th image;

a(u，v)为背景光强函数；a(u, v) is the background light intensity function;

b(u，v)为条纹对比度；b(u, v) is the fringe contrast;

φ(u，v)表示每个点的相位，周期为T；φ_i为相移角；φ(u, v) represents the phase of each point, the period is T; φ _i is the phase shift angle;

编码光栅共有N幅，构造方法为第一幅为半黑半白，后面各幅逐渐细分，细分方法为上一幅的黑色部分，分为半黑半白，上一幅的白色分为半白半黑。对每一个点，根据在各幅图像中每一幅图像上为黑还是为白进行编码，为黑则编码为1，为白则编码为0，从而得到此点的编码序列；N幅编码光栅可有2^N个编码序列，整个图象被分成2^N个长条，每个长条的宽度为相移光栅的周期T；对第n个长条(n＝1，2，...2^N，又称周期数)，按照这样的编码光构造方法对应一个唯一的编码序列，其十进制编码数为nc，建立周期数n和编码数nc之间的映射关系，可实现两者的互相转换；There are N pieces of coded gratings. The construction method is that the first piece is half black and half white, and the subsequent pieces are gradually subdivided. The subdivision method is that the black part of the previous picture is divided into half black and half white, and the white part of the previous picture is divided into Half white and half black. For each point, encode according to whether it is black or white on each image in each image, if it is black, it is encoded as 1, and if it is white, it is encoded as 0, so as to obtain the encoding sequence of this point; N encoding gratings There can be 2 ^N coding sequences, and the whole image is divided into 2 ^N strips, and the width of each strip is the period T of the phase shift grating; for the nth strip (n=1, 2, ... 2 ^N , also known as cycle number), according to such coded light construction method corresponds to a unique code sequence, its decimal code number is nc, and the mapping relationship between cycle number n and code number nc can be established to realize mutual conversion between the two ;

(2)利用两台CCD摄像机采集投射的光栅图像，并保存在程序中分配的数组中；(2) Utilize two CCD cameras to collect the projected raster image, and save it in the array allocated in the program;

(3)对每一个相机拍摄的各幅图像分别进行处理，得到每个点的相位值，相位值由相主值加上周期数乘以2π，即2nπ，由于两个摄像机拍摄的每个点的相位值应该是相等的，用两个摄像机拍摄得到的每个点相位值作为匹配的依据；(3) Each image taken by each camera is processed separately to obtain the phase value of each point. The phase value is multiplied by the phase master value plus the cycle number by 2π, that is, 2nπ. Since each point captured by two cameras The phase values of should be equal, and the phase values of each point captured by the two cameras are used as the basis for matching;

(4)对两个摄像机进行定标，得到摄像机的内参数及相对于世界坐标系的外参数：f^(j)，R ^(j)，T^(j)，j＝1，2(4) Calibrate the two cameras to obtain the internal parameters of the cameras and the external parameters relative to the world coordinate system: f ^(j) , R ^(j) , T ^(j) , j=1, 2

f^(j)：镜头焦距长度，j为摄像机编号；f ^(j) : the focal length of the lens, j is the camera number;

R^(j)：旋转矩阵， $R = [\begin{matrix} r_{1} & r_{2} & r_{3} \\ r_{4} & r_{5} & r_{6} \\ r_{7} & r_{8} & r_{9} \end{matrix}];$ R ^(j) : rotation matrix, $R = [\begin{matrix} r_{1} & r_{2} & r_{3} \\ r_{4} & r_{5} & r_{6} \\ r_{7} & r_{8} & r_{9} \end{matrix}];$

T^(j)：平移向量，T＝[TxTyTz]′；T ^(j) : translation vector, T=[TxTyTz]';

先计算R，Tx，Ty；First calculate R, Tx, Ty;

(4.1)计算每个点的图像坐标(4.1) Calculate the image coordinates of each point

u＝u₀+x/dxu=u ₀ +x/dx

v＝v₀+y/dyv=v ₀ +y/dy

对空间一个点(Xw，Yw，Zw)，经摄像机拍摄后，图像坐标(mm)为(x，y)，象素坐标为(u，v)，(u₀，v₀)为图像坐标系的原点在像素坐标系中的像素坐标，(dx，dy)为CCD相邻象素之间的x和y方向的距离，由CCD厂家提供。For a point (Xw, Yw, Zw) in space, after being captured by the camera, the image coordinates (mm) are (x, y), the pixel coordinates are (u, v), and (u ₀ , v ₀ ) is the image coordinate system The pixel coordinates of the origin of is in the pixel coordinate system, (dx, dy) is the distance in the x and y directions between adjacent pixels of the CCD, provided by the CCD manufacturer.

(4.2)计算五个未知量Ty^-1r₁，Ty^-1r₂，Ty^-1Tx，Ty^-1r₄，Ty^-1r₅ (4.2) Calculate five unknown quantities Ty ^-1 r ₁ , Ty ^-1 r ₂ , Ty ^-1 Tx, Ty ^-1 r ₄ , Ty ^-1 r ₅

对于每一个三维对象点(Xw_k，Yw_k，Zw_k)(因为点共面，故将Z坐标取作0)以及相应的图像坐标(x_k，y_k)，由共线方程：For each three-dimensional object point (Xw _k , Yw _k , Zw _k ) (because the points are coplanar, the Z coordinate is taken as 0) and the corresponding image coordinates (x _k , y _k ), the collinear equation is:

${x x}_{k k} = = f f \frac{{r r}_{11} X x {w w}_{k k} + + {r r}_{22} Y Y {w w}_{k k} + + Tx Tx}{{r r}_{77} X x {w w}_{k k} + + {r r}_{88} Y Y {w w}_{k k} + + Tz Tz}$

${y the y}_{k k} = = f f \frac{{r r}_{44} X x {w w}_{k k} + + {r r}_{55} Y Y {w w}_{k k} + + Ty Ty}{{r r}_{77} X x {w w}_{k k} + + {r r}_{88} Y Y {w w}_{k k} + + Tz Tz}$

将上面二式相除得到：利用最小二乘法求得上述5个未知量；(4.3)计算r₁，...，r₉，Tx，TyDivide the above two formulas to obtain: use the least square method to obtain the above five unknown quantities; (4.3) Calculate r ₁ ,..., r ₉ , Tx, Ty

(4.31)计算|Ty|(4.31) Calculate |Ty|

定义矩阵：Define the matrix:

$C C = = [\begin{matrix} {r r}_{11}^{' '} & {r r}_{22}^{' '} \\ {r r}_{44}^{' '} & {r r}_{55}^{' '} \end{matrix}] = = \begin{matrix} [\begin{matrix} {r r}_{11} / / Ty Ty & {r r}_{22} / / Ty Ty \\ {r r}_{44} / / Ty Ty & {r r}_{55} / / Ty Ty \end{matrix}] \end{matrix}$

则有：Then there are:

${y the y}^{22} = = \frac{S S r r - - {[[S S {r r}_{22} - - 44 {(({r r}_{11}^{' '} {r r}_{55}^{' '} - - {r r}_{44}^{' '} {r r}_{22}^{' '}))}^{22}]]}^{11 / / 22}}{22 {(({r r}_{11}^{' '} {r r}_{55}^{' '} - - {r r}_{44}^{' '} {r r}_{22}^{' '}))}^{22}}$

其中：in:

Sr＝r₁′²+r₂′²+r₅′²+r₄′²，Sr=r ₁ ′ ² +r ₂ ′ ² +r ₅ ′ ² +r ₄ ′ ² ,

如果矩阵C的某行或某列元素为全为0，按下式计算：If a certain row or column of matrix C is all 0, it is calculated as follows:

Ty²＝(r_i′²+r_j′²)^-1，其中r_i′，r_j′，是矩阵C剩余的另两个元素。Ty ² =(r _i ′ ² +r _j ′ ² ) ⁻¹ , where ri _′ , r _j ′ are the remaining two elements of matrix C.

(4.32)决定Ty的符号(4.32) Determine the sign of Ty

①假设Ty为正号；①Assume that Ty is a positive sign;

②选择所拍摄图像中，离图像中心较远的一点。设其图像坐标为(x_k，y_k)，在三维世界中的坐标为(Xw_k，Yw_k，Zw_k)。② Select a point farther from the center of the image in the captured image. Suppose its image coordinates are (x _k , y _k ), and its coordinates in the three-dimensional world are (Xw _k , Yw _k , Zw _k ).

③由上面的结果计算下面各式的值：③Calculate the following values from the above results:

r₁＝(Ty^-1r₁)*Ty，r₂＝(Ty^-1r₂)*Ty，r₄＝(Ty^-1r₄)*Tyr ₁ =(Ty ^-1 r ₁ )*Ty, r ₂ =(Ty ^-1 r ₂ )*Ty, r ₄ =(Ty ^-1 r ₄ )*Ty

r₅＝(Ty^-1r₅)*Ty，Tx＝(Ty^-1Tx)*Ty，r ₅ =(Ty ⁻¹ r ₅ )*Ty, Tx=(Ty ⁻¹ Tx)*Ty,

x_k′＝r₁Xw_k+r₂Yw_k+Txx _k ′＝r ₁ Xw _k +r ₂ Yw _k +Tx

y_k′＝r₄Xw_k+r₅Yw_k+Tyy _k ′＝r ₄ Xw _k +r ₅ Yw _k +Ty

如果x_k和x_k′、y_k和y_k′都同号，那么sgn(Ty)＝+1，否则sgn(Ty)＝-1；If x _k and x _k ', y _k and y _k ' all have the same sign, then sgn(Ty)=+1, otherwise sgn(Ty)=-1;

(4.33)计算旋转矩阵R(4.33) Calculate the rotation matrix R

根据Ty的值，重新计算r₁，r₂，r₄，r₅，Tx。According to the value of Ty, r ₁ , r ₂ , r ₄ , r ₅ , Tx are recalculated.

$R R = = [\begin{matrix} {r r}_{11} & {r r}_{22} & {((11 - - {r r}_{11}^{22} - - {r r}_{22}^{22}))}^{11 / / 22} \\ {r r}_{44} & {r r}_{55} & s the s {((11 - - {r r}_{44}^{22} - - {r r}_{55}^{22}))}^{11 / / 22} \\ {r r}_{77} & {r r}_{88} & {r r}_{99} \end{matrix}]$

其中s＝sgn(r₁r₄+r₂r₅)where s=sgn(r ₁ r ₄ +r ₂ r ₅ )

r₇，r₈，r₉是由前两行的外积得到。r ₇ , r ₈ , r ₉ are obtained from the outer product of the first two lines.

如果按照这样的R计算下面的焦距f为负值，则：If the following focal length f is calculated according to such R as a negative value, then:

$R R = = [\begin{matrix} {r r}_{11} & {r r}_{22} & - - {((11 - - {r r}_{11}^{22} - - {r r}_{22}^{22}))}^{11 / / 22} \\ {r r}_{44} & {r r}_{55} & - - s the s {((11 - - {r r}_{44}^{22} - - {r r}_{55}^{22}))}^{11 / / 22} \\ - - {r r}_{77} & - - {r r}_{88} & {r r}_{99} \end{matrix}]$

接着计算焦距长度f和T_z的值Then calculate the value of focal length f and T _z

对每一个标定点，建立包含f和T_z作为未知参数的线性方程：For each calibration point, set up a linear equation including f and T _z as unknown parameters:

$[[{Y Y}_{i i} - - {y the y}_{i i}]] [\begin{matrix} f f \\ Tz Tz \end{matrix}] = = {w w}_{i i} {y the y}_{i i}$

其中：in:

Y_i＝r₄x_wi+r₅y_wi+r₆*0+TyY _i ＝r ₄ x _wi +r ₅ y _wi +r ₆ *0+Ty

w_i＝r₇x_wi+r₈y_wi+r₉*0w _i =r ₇ x _wi +r ₈ y _wi +r ₉ *0

通过解方程，可求f和T_z；By solving the equation, f and T _z can be obtained;

(5)根据相位和外极线几何对每点进行三维重构，得到物体表面点的三维坐标：(5) Perform three-dimensional reconstruction of each point according to the phase and epipolar geometry, and obtain the three-dimensional coordinates of the surface point of the object:

(5.1)计算基础矩阵F(5.1) Calculate the fundamental matrix F

F＝A₂ ^-TEA₁ ^-1 F＝A ₂ ^-T EA ₁ ^-1

其中in

${A A}_{22} = = [\begin{matrix} {f f}_{22} / / d d {x x}_{22} & 00 & {u u}_{0202} \\ 00 & {f f}_{22} / / d d {y the y}_{22} & {v v}_{0202} \\ 00 & 00 & 11 \end{matrix}],,$ ${A A}_{11} = = [\begin{matrix} {f f}_{11} / / d d {x x}_{11} & 00 & {u u}_{0101} \\ 00 & {f f}_{11} / / d d {y the y}_{11} & {v v}_{0101} \\ 00 & 00 & 11 \end{matrix}]$

其中f₁，(u₀₁，v₀₁)，dx₁，dy₁为第一个摄像机的内参数，Where f ₁ , (u ₀₁ , v ₀₁ ), dx ₁ , dy ₁ are internal parameters of the first camera,

f₂，(u₀₂，v₀₂)，dx₂，dy₂为第二个摄像机的内参数；f ₂ , (u ₀₂ , v ₀₂ ), dx ₂ , dy ₂ are internal parameters of the second camera;

$E = {[T]}_{x} R^{(2)} R^{{(1)}^{- 1}}$ (称为本质矩阵，Essential Matrix) $E. = {[T]}_{x} R^{(2)} R^{{(1)}^{- 1}}$ (called the essential matrix, Essential Matrix)

$T T = = {T T}^{((22))} - - {R R}^{((22))} {R R}^{{((11))}^{- - 11}} {T T}^{((11))}$

把T表示为[Tx，Ty，Tz]，则反对称矩阵为[T]_x Express T as [Tx, Ty, Tz], then the antisymmetric matrix is [T] _x

${[[T T]]}_{x x} = = [\begin{matrix} 00 & - - Tz Tz & Ty Ty \\ Tz Tz & 00 & - - Tx Tx \\ - - Ty Ty & Tx Tx & 00 \end{matrix}]$

(5.2)计算右摄像机拍摄图像上一点P，其坐标为

它在左摄像机拍摄图像上的极线方程参数。并在上述直线上寻找其匹配点Q点，方法是点P和点Q的相位值相等。(5.2) Calculate a point P on the image captured by the right camera, and its coordinates are

Its epipolar equation parameters on the image captured by the left camera. And look for its matching point Q on the above straight line, the method is that the phase values of point P and point Q are equal.

已知

根据下述直线方程，找右摄像机拍摄图像上点P在左摄像机拍摄图像上的点Q，其坐标为 A known

According to the following straight line equation, find the point Q of the point P on the image captured by the right camera on the image captured by the left camera, and its coordinates are

${\overset{~ ~}{m m}}_{22}^{T T} F f {\overset{~ ~}{m m}}_{11} = = 00,,$

其中：

是P点的像素坐标(u₂，v₂，1)^T；in:

is the pixel coordinate (u ₂ , v ₂ , 1) ^T of point P;

是Q点像素坐标(u₁，v₁，1)^T；

is the pixel coordinate of point Q (u ₁ , v ₁ , 1) ^T ;

根据点P和点Q相位相等的原则，在由上述直线方程决定的一条直线上寻找Q点。According to the principle that the phases of point P and point Q are equal, find point Q on a straight line determined by the above straight line equation.

(5.3)找到Q点，则存储此匹配点，按照下述的三维空间点S(Xw，Yw，Zw)和两个摄像机拍摄图像的对应图像坐标的关系为：(5.3) Find the Q point, then store this matching point, according to the relationship between the following three-dimensional space point S (Xw, Yw, Zw) and the corresponding image coordinates of the images captured by the two cameras:

${x x}^{((j j))} = = {f f}^{((j j))} \frac{{R R}^{((j j))}_{1111} Xw wxya + + {R R}^{((j j))}_{1212} Yw w + + {R R}^{((j j))}_{1313} Zw Zw + + {T T}^{((j j))} x x}{{R R}^{((j j))}_{3131} Xw wxya + + {R R}^{((j j))}_{3232} Yw w + + {R R}^{((j j))}_{3333} Zw Zw + + {T T}^{((j j))} z z},,$

${y the y}^{((j j))} = = {f f}^{((j j))} \frac{{R R}^{((j j))}_{21 twenty one} Xw wxya + + {R R}^{((j j))}_{22 twenty two} Yw w + + {R R}^{((j j))}_{23 twenty three} Zw Zw + + {T T}^{((j j))} y the y}{{R R}^{((j j))}_{3131} Xw wxya + + {R R}^{((j j))}_{3232} Yw w + + {R R}^{((j j))}_{3333} Zw Zw + + {T T}^{((j j))} z z},,$

j＝1，2，为摄像机的编号。j=1, 2, is the number of the camera.

所述相移光栅的相移角φ_i＝i*90，i＝1，...4，即相移光栅图像有4幅，其相主值计算公式为：The phase shift angle φ _i =i*90 of the phase shift grating, i=1, ... 4, that is, there are 4 phase shift grating images, and the calculation formula of the phase master value is:

$φ (u, v) = a \tan (\frac{I_{2} - I_{4}}{I_{3} - I_{1}}),$ 其中I₁、I₂、I₃、I₄，是点(u，v)在四幅相移光栅中的相位。 $φ (u, v) = a the tan (\frac{I_{2} - I_{4}}{I_{3} - I_{1}}),$ Among them, I ₁ , I ₂ , I ₃ , and I ₄ are the phases of the point (u, v) in the four phase shift gratings.

所述编码光栅共有7幅，每一点的二进制编码序列即可转化成该点在图像中的位置，即周期数。There are 7 coded gratings in total, and the binary code sequence of each point can be converted into the position of the point in the image, that is, the number of cycles.

本发明提出的测量物体三维表面轮廓的系统，其特征在于它含有计算机、投影仪、两台CCD摄像机。The system for measuring the three-dimensional surface profile of an object proposed by the invention is characterized in that it contains a computer, a projector, and two CCD cameras.

实验证明本发明具有非接触、速度快、数据量大、精度高、操作简单、易于实现等优点。Experiments prove that the present invention has the advantages of non-contact, high speed, large amount of data, high precision, simple operation and easy realization.

附图说明Description of drawings

图1：本发明所述系统的示意图。Figure 1: Schematic representation of the system of the present invention.

图2：摄像机定标用的标定块。Figure 2: Calibration block for camera calibration.

图3：两阶段法摄像机定标程序流程图。Figure 3: Flowchart of the two-stage camera calibration procedure.

图4：三维重构算法流程图。Figure 4: Flowchart of the 3D reconstruction algorithm.

图5：人手测量结果示意图。Figure 5: Schematic diagram of human measurement results.

具体实施方式Detailed ways

本发明提出的一种三维测量的方法的实施例结合说明如下：The embodiment of the method for a kind of three-dimensional measurement that the present invention proposes is described as follows in conjunction with:

本实施例的测量系统如图1所示。本系统由CCD摄像机1，3，投影仪2，计算机4等组成。The measurement system of this embodiment is shown in FIG. 1 . The system is composed of CCD cameras 1, 3, projector 2, computer 4 and so on.

计算机为PIII 1G带有1394图象卡，显卡支持双显示器输出。The computer is PIII 1G with 1394 graphics card, and the graphics card supports dual monitor output.

系统采用的ASKC20+高清晰数字投影仪，其亮度为1500ANSI流明，分辨率为800×600。The ASKC20+ high-definition digital projector used in the system has a brightness of 1500 ANSI lumens and a resolution of 800×600.

CCD采用德国Basler公司的A302f数字摄像机，其分辨率达到了780×582，符合IEEE1394工业标准，配合Computar M1214-MP定焦镜头使用。The CCD adopts the A302f digital camera of Basler Company in Germany, its resolution reaches 780×582, conforms to the IEEE1394 industrial standard, and is used with the Computar M1214-MP fixed-focus lens.

方法采取的步骤如下，其中软件的实现采用Visual C++6.0平台进行开发：The steps taken by the method are as follows, wherein the implementation of the software is developed using the Visual C++6.0 platform:

1)利用计算机生成虚拟光栅，其中包括编码光栅和相移光栅，利用投影仪将生成的光栅投射在物体上；1) Use a computer to generate a virtual grating, including a coded grating and a phase shift grating, and use a projector to project the generated grating on the object;

I_i(u，v)＝a(u，v)+b(u，v)cos(φ(u，v)+φ_i) (1)I _i (u, v)=a(u, v)+b(u, v) cos(φ(u, v)+φ _i ) (1)

其中： in:

(u，v)为某点的坐标；(u, v) is the coordinates of a point;

b(u，v)为条纹对比度；b(u, v) is the stripe contrast;

φ(u，v)表示每个点的相位，周期为T；φ(u, v) represents the phase of each point, and the period is T;

对于每间隔90度的相移，φ_i＝i*90，i＝1，...4，For every phase shift of 90 degrees, φ _i =i*90, i=1,...4,

在构造相移光栅时，光栅图象大小为1024×768，光栅的周期大小为8即点的坐标x取值范围在(1，1024)之间的整数，y取值在(1，768)之间的整数。a(x，y)取0，b(x，y)取255。每个点相位值φ(x，y)设置为x除以8得到的余数除以8，再乘以2π。When constructing a phase-shifted grating, the size of the grating image is 1024×768, and the period size of the grating is 8, that is, the coordinate x of the point is an integer in the range of (1, 1024), and the value of y is in the range of (1, 768). Integer between. a(x, y) takes 0, and b(x, y) takes 255. The phase value φ(x, y) of each point is set as the remainder obtained by dividing x by 8, divided by 8, and then multiplied by 2π.

编码光栅共有7幅，构造方法为第一幅为半黑半白，后面各幅逐渐细分，细分方法为上一幅的黑色部分，分为半黑半白，上一幅的白色分为半白半黑。对每一个点，根据在各幅图像中每一幅图像上为黑还是为白进行编码，为黑则编码为1，为白则编码为0，从而得到此点的编码序列；7幅编码光栅可有2⁷个编码序列，整个图象被分成2⁷个长条，每个长条的宽度为相移光栅的周期8；对第n个长条(n＝1，2，...2⁷)，按照这样的编码光构造方法对应一个唯一的编码序列，其十进制编码数为nc，按照这样的方法建立每一个周期数n和编码数nc之间的映射关系，可实现两者的互相转换；There are 7 coded gratings. The construction method is that the first frame is half black and half white, and the subsequent frames are gradually subdivided. The subdivision method is that the black part of the previous frame is divided into half black and half white, and the white part of the previous frame is divided into Half white and half black. For each point, encode according to whether it is black or white on each image in each image, if it is black, it is encoded as 1, and if it is white, it is encoded as 0, so as to obtain the encoding sequence of this point; 7 encoding gratings There may be 2 ⁷ coding sequences, and the whole image is divided into 2 ⁷ strips, and the width of each strip is the period 8 of the phase shift grating; for the nth strip (n=1, 2, ... 2 ⁷ ), according to such code light construction method corresponds to a unique code sequence, its decimal code number is nc, according to this method to establish the mapping relationship between each cycle number n and code number nc, can realize the mutual between the two conversion;

2)利用两台CCD摄像机采集投射的光栅图象，并保存在程序中分配的数组中。2) Use two CCD cameras to collect the projected raster images and save them in the array allocated in the program.

3)对每一个相机拍摄的11幅图象分别进行处理，得到每个点的相位值，相位值由相主值加上周期数乘以2π。3) Process the 11 images taken by each camera separately to obtain the phase value of each point, and the phase value is multiplied by the phase master value plus the number of cycles multiplied by 2π.

相移法中相主值计算公式为：The formula for calculating the phase principal value in the phase shift method is:

$φ φ ((u u,, v v)) = = a a tan the tan ((\frac{{I I}_{22} - - {I I}_{44}}{{I I}_{33} - - {I I}_{11}})) - - - - - - ((22))$

其中I₁、I₂、I₃、I₄，是点(u，v)在四幅相移光栅中的光强。Among them, I ₁ , I ₂ , I ₃ , and I ₄ are the light intensities of the point (u, v) in the four phase shift gratings.

对七幅编码光栅，通过图象处理技术，对图象进行二值化处理，可以对每个求得每个点在每幅图象中是黑点(编码为1)还是白点(编码为0)，综合7幅图象，可得到编码序列，然后按照编码数与周期数之间的映射关系，将编码序列转化为周期数n。每个点的相位由相主值加上2nπ即可得到。For the seven coded gratings, through image processing technology, the image is binarized, and each point can be obtained for each image whether it is a black point (coded as 1) or a white point (coded as 1) or a white point (coded as 0), the 7 images are synthesized to obtain the coding sequence, and then the coding sequence is converted into the cycle number n according to the mapping relationship between the coding number and the cycle number. The phase of each point can be obtained by adding 2nπ to the phase principal value.

对同一个点来说，按照上面计算的相位是与摄像机拍摄的位置无关的，也就是说在对两个摄像机拍摄的每个点的应该是相等的。因而我们用两个摄像机拍摄的每个点的相位值作为匹配的依据。For the same point, the phase calculated above has nothing to do with the position of the camera, that is to say, it should be equal to each point of the two cameras. Therefore, we use the phase value of each point captured by the two cameras as the basis for matching.

4)对两个摄像机进行定标，定标中使用的标定块如图2所示。摄像机定标是求得摄像机的内参数及相对于世界坐标系的外参数的过程。对空间的一个点(Xw，Yw，Zw)，经摄像机拍摄后，图象坐标(mm)为(x，y)，象素坐标(象素)为(u，v)：4) Calibrate the two cameras, and the calibration blocks used in the calibration are shown in Figure 2. Camera calibration is the process of obtaining the internal parameters of the camera and the external parameters relative to the world coordinate system. For a point (Xw, Yw, Zw) in space, after being captured by the camera, the image coordinates (mm) are (x, y), and the pixel coordinates (pixels) are (u, v):

$x x = = f f \frac{{r r}_{11} Xw wxya + + {r r}_{22} Yw w + + {r r}_{33} Zw Zw + + Tx Tx}{{r r}_{77} Xw wxya + + {r r}_{88} Yw w + + {r r}_{99} Zw Zw + + Tz Tz}$

$y the y = = f f \frac{{r r}_{44} Xw wxya + + {r r}_{55} Yw w + + {r r}_{66} Zw Zw + + Ty Ty}{{r r}_{77} Xw wxya + + {r r}_{88} Yw w + + {r r}_{99} Zw Zw + + Tz Tz} - - - - - - ((33))$

u＝u₀+x/dxu=u ₀ +x/dx

v＝v₀+y/dy (4)v=v ₀ +y/dy (4)

其中：in:

$R = [\begin{matrix} r_{1} & r_{2} & r_{3} \\ r_{4} & r_{5} & r_{6} \\ r_{7} & r_{8} & r_{9} \end{matrix}],$ T＝[TxTyTz]′为旋转矩阵和平移向量，f为镜头焦距，为要标定的参数。 $R = [\begin{matrix} r_{1} & r_{2} & r_{3} \\ r_{4} & r_{5} & r_{6} \\ r_{7} & r_{8} & r_{9} \end{matrix}],$ T=[TxTyTz]' is the rotation matrix and translation vector, f is the focal length of the lens, and is the parameter to be calibrated.

(u₀，v₀)为图像坐标系的原点在像素坐标系中的像素坐标，对我们的摄像机的分辨率是780*582，(u₀，v₀)取值为(390，291)。(dx，dy)为CCD相邻象素之间的x和y方向的距离，可由CCD厂家提供。(u ₀ , v ₀ ) is the pixel coordinate of the origin of the image coordinate system in the pixel coordinate system. The resolution of our camera is 780*582, and the value of (u ₀ , v ₀ ) is (390, 291). (dx, dy) is the distance in the x and y directions between adjacent pixels of the CCD, which can be provided by the CCD manufacturer.

定标中的标定块是精密加工，每个点的三维坐标是精确知道，利用图象处理基数可以得到每个点的图象坐标。在本系统中采取Tsai的两阶段方法得到摄像机的旋转矩阵R，平移矩阵T，焦距f。具体流程如图3所示，其过程如下：The calibration block in the calibration is precisely processed, and the three-dimensional coordinates of each point are precisely known, and the image coordinates of each point can be obtained by using the image processing base. In this system, Tsai's two-stage method is adopted to obtain the camera's rotation matrix R, translation matrix T, and focal length f. The specific process is shown in Figure 3, and the process is as follows:

阶段1：计算旋转矩阵R，Tx，TyPhase 1: Calculation of rotation matrices R, Tx, Ty

(4.1)计算图像坐标(4.1) Calculate image coordinates

按照(4)式计算每个标志点的图象坐标(x，y)。Calculate the image coordinates (x, y) of each marker point according to formula (4).

(4.2)计算五个未知量Ty^-1r₁，Ty^-1r₂，Ty^-1T_x，Ty^-1r₄，Ty^-1r₅对于每一个三维对象点(Xw_k，Yw_k，Zw_k)(因为点共面，故将Z坐标取作0)以及相应的图像坐标(x_k，y_k)，由共线方程：(4.2) Calculate five unknown quantities Ty ^-1 r ₁ , Ty ^-1 r ₂ , Ty ^-1 T _x , Ty ^-1 r ₄ , Ty ^-1 r ₅ for each three-dimensional object point (Xw _k , Yw _k , Zw _k ) (because the points are coplanar, the Z coordinate is taken as 0) and the corresponding image coordinates (x _k , y _k ), by the collinear equation:

${y the y}_{k k} = = f f \frac{{r r}_{44} X x {w w}_{k k} + + {r r}_{55} Y Y {w w}_{k k} + + Ty Ty}{{r r}_{77} X x {w w}_{k k} + + {r r}_{88} Y Y {w w}_{k k} + + Tz Tz} - - - - - - ((55))$

将上面二式相除得到：Divide the above two equations to get:

[y_kXw_k y_kYw_k y_k -x_kXw_k -x_kYw_k]L＝x_k (6)[y _k Xw _k y _k Yw _k y _k -x _k Xw _k -x _k Yw _k ]L＝x _k (6)

其中：in:

L＝[Ty^-1r₁ Ty^-1r₂ Ty^-1Tx Ty^-1r₄ Ty^-1r₅]^T (7)L＝[Ty ^-1 r ₁ Ty ^-1 r ₂ Ty ^-1 Tx Ty ^-1 r ₄ Ty ^-1 r ₅ ] ^T (7)

在上式中，有5个未知数，我们的点的个数一般很多，利用最小二乘法求得方程组的解。In the above formula, there are 5 unknowns, and the number of our points is generally many, and the solution of the system of equations is obtained by the method of least squares.

(4.3)计算r₁，...，r₉，Tx，Ty(4.3) Calculate r ₁ , ..., r ₉ , Tx, Ty

(4.31)计算|Ty|(4.31) Calculate |Ty|

定义矩阵：Define the matrix:

$C C = = [\begin{matrix} {r r}_{11}^{' '} & {r r}_{22}^{' '} \\ {r r}_{44}^{' '} & {r r}_{55}^{' '} \end{matrix}] = = [\begin{matrix} {r r}_{11} / / Ty Ty & {r r}_{22} / / Ty Ty \\ {r r}_{44} / / Ty Ty & {r r}_{55} / / Ty Ty \end{matrix}] - - - - - - ((88))$

则有：Then there are:

$T T {y the y}^{22} = = \frac{{Sr Sr - - [[S S {r r}_{22} - - 44 {(({r r}_{11}^{' '} {r r}_{55}^{' '} - - {r r}_{44}^{' '} {r r}_{22}^{' '}))}^{22}]]}^{11 / / 22}}{22 {(({r r}_{11}^{' '} {r r}_{55}^{' '} - - {r r}_{44}^{' '} {r r}_{22}^{' '}))}^{22}} - - - - - - ((99))$

其中：in:

Sr＝r₁′²+r₂′²+r₅′²+r₄′²， (10)Sr=r ₁ ′ ² +r ₂ ′ ² +r ₅ ′ ² +r ₄ ′ ² , (10)

(4.32)决定Ty的符号(4.32) Determine the sign of Ty

①假设Ty为正号；①Assume that Ty is a positive sign;

②选择所拍摄图像中，离图像中心较远的一点。设其图像坐标为(x_k，y_k)，在三维世界中的坐标为(Zw_k，Yw_k，Zw_k)。② Select a point farther from the center of the image in the captured image. Let its image coordinates be (x _k , y _k ), and its coordinates in the three-dimensional world be (Zw _k , Yw _k , Zw _k ).

r_i＝(Ty^-1r₂)*Ty，r₂＝(Ty^-1r₂)*Ty，r₄＝(Ty^-1r₄)*Tyr _i =(Ty ^-1 r ₂ )*Ty, r ₂ =(Ty ^-1 r ₂ )*Ty, r ₄ =(Ty ^-1 r ₄ )*Ty

r₅＝(Ty^-1r₅)*Ty，Tx＝(Ty^-1Tx)*Ty， (11)r ₅ =(Ty ⁻¹ r ₅ )*Ty, Tx=(Ty ⁻¹ Tx)*Ty, (11)

x_k′＝r₁Xw_k+r₂Yw_k+Txx _k ′＝r ₁ Xw _k +r ₂ Yw _k +Tx

y_k′＝r₄Xw_k+r₅Yw_k+Tyy _k ′＝r ₄ Xw _k +r ₅ Yw _k +Ty

(4.33)计算旋转矩阵R(4.33) Calculate the rotation matrix R

$R R = = [\begin{matrix} {r r}_{11} & {r r}_{22} & {((11 - - {r r}_{11}^{22} - - {r r}_{22}^{22}))}^{11 / / 22} \\ {r r}_{44} & {r r}_{55} & s the s {((11 - - {r r}_{44}^{22} - - {r r}_{55}^{22}))}^{11 / / 22} \\ {r r}_{77} & {r r}_{88} & {r r}_{99} \end{matrix}] - - - - - - ((1212))$

其中s＝sgn(r₁r₄+r₂r₅)where s=sgn(r ₁ r ₄ +r ₂ r ₅ )

$R R = = [\begin{matrix} {r r}_{11} & {r r}_{22} & - - {((11 - - {r r}_{11}^{22} - - {r r}_{22}^{22}))}^{11 / / 22} \\ {r r}_{44} & {r r}_{55} & - - s the s {((11 - - {r r}_{44}^{22} - - {r r}_{55}^{22}))}^{11 / / 22} \\ - - {r r}_{77} & - - {r r}_{88} & {r r}_{99} \end{matrix}] - - - - - - ((1313))$

$[[{Y Y}_{k k} - - {y the y}_{k k}]] [\begin{matrix} f f \\ Tz Tz \end{matrix}] = = {w w}_{k k} {y the y}_{k k} - - - - - - ((1414))$

其中：in:

Y_k＝r₄x_wk+r₅y_wk+r₆*0+Ty (15)Y _k ＝r ₄ x _wk +r ₅ y _wk +r ₆ *0+Ty (15)

w_k＝r₇x_wk+r₈y_wk+r₉*0w _k ＝r ₇ x _wk +r ₈ y _wk +r ₉ *0

5)根据相位和外极线几何对每点进行三维重构，得到物体表面点的三维坐标。对两个摄像机拍摄的图象，三维空间点S(Xw，Yw，Zw)和对应图像坐标的关系为：5) Perform three-dimensional reconstruction for each point according to the phase and epipolar geometry, and obtain the three-dimensional coordinates of the surface point of the object. For the images captured by two cameras, the relationship between the three-dimensional space point S(Xw, Yw, Zw) and the corresponding image coordinates is:

${y the y}^{((j j))} = = {f f}^{((j j))} \frac{{R R}^{((j j))}_{21 twenty one} Xw wxya + + {R R}^{((j j))}_{22 twenty two} Yw w + + {R R}^{((j j))}_{23 twenty three} Zw Zw + + {T T}^{((j j))} y the y}{{R R}^{((j j))}_{3131} Xw wxya + + {R R}^{((j j))}_{3232} Yw w + + {R R}^{((j j))}_{3333} Zw Zw + + {T T}^{((j j))} z z} - - - - - - ((1616))$

其中j＝1，2表示两个摄像机拍摄的图像。Where j=1, 2 represent images taken by two cameras.

摄像机定标后，f^(j)，R^(j)，T^(j)是已知的，对个摄像机拍摄的图像，共有四个方程，对标定好的摄像机，利用最小二乘法可以得到三维点的坐标。After the camera is calibrated, f ^(j ), R ^(j) and T ^(j) are known. There are four equations for the image taken by a camera. For the calibrated camera, the three-dimensional point can be obtained by using the least square method coordinate of.

这是立体视觉三维重建的基本原理，其中最大的问题是如何找到相匹配的两个点，也就是说如何实现三维空间中一个点在两幅图象上的对应关系。由计算机视觉中的外极线理论知道，对第一幅图像中的一个点，在第二幅图像中的对应点是在一条直线上。其关系可有如下方程表示：This is the basic principle of stereoscopic 3D reconstruction, and the biggest problem is how to find two matching points, that is to say, how to realize the correspondence between a point in three-dimensional space and two images. According to the epipolar line theory in computer vision, for a point in the first image, the corresponding point in the second image is on a straight line. Its relationship can be expressed by the following equation:

${\overset{~ ~}{m m}}_{22}^{T T} F f {\overset{~ ~}{m m}}_{11} = = 00 - - - - - - ((1717))$

其中，

是第一幅图像中某点的像素坐标(u₁，v₁，1)^T，

是第二幅图像中某点的像素坐标(u₂，v₂，1)^T，F为基础矩阵(Fundamental Matrix)，其元素是摄像机内外参数。in,

is the pixel coordinate (u ₁ , v ₁ , 1) ^T of a point in the first image,

is the pixel coordinate (u ₂ , v ₂ , 1) ^T of a certain point in the second image, and F is the fundamental matrix (Fundamental Matrix), whose elements are internal and external parameters of the camera.

F＝A₂ ^-TEA₁ ^-1 (18)F＝A ₂ ^-T EA ₁ ^-1 (18)

其中in

其中f₁，(u₀₁，v₀₁)，dx₁，dy₁为第一个摄像机的内参数，f₂，(u₀₂，v₀₂)，dx₂，dy₂为第二个摄像机的内参数，参数的意义同前面所述。Where f ₁ , (u ₀₁ , v ₀₁ ), dx ₁ , dy ₁ are internal parameters of the first camera, f ₂ , (u ₀₂ , v ₀₂ ), dx ₂ , dy ₂ are internal parameters of the second camera , the meanings of the parameters are the same as those mentioned above.

$T T = = {T T}^{((22))} - - {R R}^{((22))} {R R}^{{((11))}^{- - 11}} {T T}^{((11))} - - - - - - ((1919))$

将T表示为[Tx，Ty，Tz]，[T]_x为反对称矩阵，定义为：Denote T as [Tx, Ty, Tz], [T] _x is an anti-symmetric matrix, defined as:

${[[T T]]}_{x x} = = [\begin{matrix} 00 & - - Tz Tz & Ty Ty \\ Tz Tz & 00 & - - Tx Tx \\ - - Ty Ty & Tx Tx & 00 \end{matrix}] - - - - - - ((2020))$

可以看出基础矩阵是由摄像机内外参数所决定的，通过摄像机标定，我们就能得到极线方程。It can be seen that the fundamental matrix is determined by the internal and external parameters of the camera. Through camera calibration, we can get the epipolar equation.

从极线方程可以看出，对右摄像机拍摄图象上一点P，其坐标

其在左摄像机拍摄图象上如果有对应点Q，应该是在由和基础矩阵F决定的一条直线上。It can be seen from the epipolar equation that for a point P on the image captured by the right camera, its coordinates

If there is a corresponding point Q on the image captured by the left camera, it should be and on a straight line determined by the fundamental matrix F.

关键是在这条直线上的哪一个点呢?The key is which point on this straight line?

在前面相位获取时，曾提到，同一个点在不同摄像机拍摄时相位值应该是相同的。而对每一个点的相位值，我们有相位获取是得到了的，我们正是利用这一点实现两幅图象上点的精确匹配。In the previous phase acquisition, it was mentioned that the phase value of the same point should be the same when it is shot by different cameras. For the phase value of each point, we have obtained the phase acquisition, and we use this point to realize the precise matching of the points on the two images.

算法流程图见附图4。The flowchart of the algorithm is shown in Figure 4.

找到匹配点后，利用式(16)可以计算点的三维坐标。After finding the matching point, the three-dimensional coordinates of the point can be calculated by using formula (16).

按照上述过程对人手进行三维测量，测量得到的点如图5所示。According to the above process, the three-dimensional measurement of the human hand is carried out, and the measured points are shown in Figure 5.

本发明所构成的三维测量系统具有非接触、速度快、数据量大、精度高、操作简单等特点。对物体的单面测量只要2秒钟能得到极高密度的数据(40万个点)，测量的精度在0.05mm以上。The three-dimensional measuring system constituted by the present invention has the characteristics of non-contact, fast speed, large amount of data, high precision, simple operation and the like. It only takes 2 seconds to measure one side of an object to obtain extremely high-density data (400,000 points), and the measurement accuracy is above 0.05mm.

Claims

1. The method for measuring the three-dimensional profile of the surface of an object is characterized in that: it is a combination of phase and stereo vision technology, projecting a grating on the surface of the object, and then using dual cameras to shoot the distorted grating image, using coded light and phase shift The method obtains the phase of each point on the image captured by the left and right cameras; uses the phase and the epipolar line to realize the matching of the points on the two images, so as to achieve the method of inversely seeking the three-dimensional coordinates of the surface point of the object; it includes the following steps:

(1) Use a computer to generate a virtual grating, including a coded grating and a phase shift grating, and use a projector to project the generated grating on the object;

The light intensity of the phase shift grating is expressed as follows:

I _i (u, v)=a(u, v)+b(u, v) cos(φ(u, v)+φ _i )

in:

(u, v) is the coordinates of a certain point;

I _i (u, v) is the light intensity of point (u, v) in the i-th image;

a(u, v) is the background light intensity function;

b(u, v) is the fringe contrast;

φ(u, v) represents the phase of each point, and the period is T;

φ _i is the phase shift angle;

There are N pieces of coded gratings. The construction method is that the first piece is half black and half white, and the subsequent pieces are gradually subdivided. The subdivision method is that the black part of the previous picture is divided into half black and half white, and the white part of the previous picture is divided into Half white and half black; for each point, encode according to whether it is black or white on each image in each image, if it is black, it is encoded as 1, and if it is white, it is encoded as 0, so as to obtain the encoding sequence of this point ; N coding gratings can have 2 ^N coding sequences, and the whole image is divided into 2 ^N strips, and the width of each strip is the period T of the phase shift grating; for the nth strip, n=1,2 , ... 2 ^N , also known as the number of cycles, according to this coded light construction method corresponds to a unique code sequence, its decimal code number is nc, and the mapping relationship between the cycle number n and the code number nc can be established to realize two the mutual conversion of those;

(2) Utilize two CCD cameras to collect the projected raster image, and save it in the array allocated in the program;

(3) Each image taken by each camera is processed separately to obtain the phase value of each point. The phase value is multiplied by the phase master value plus the cycle number by 2π, that is, 2nπ. Since each point captured by two cameras The phase values of should be equal, and the phase values of each point captured by the two cameras are used as the basis for matching;

(4) Calibrate the two cameras to obtain the internal parameters of the cameras and the external parameters relative to the world coordinate system: f ^(j) , R ^(j) , T ^(j) , j=1, 2

f ^(j) : the focal length of the lens, j is the camera number;

R ^(j) : rotation matrix,

R = [\begin{matrix} r_{1} & r_{2} & r_{3} \\ r_{4} & r_{5} & r_{6} \\ r_{7} & r_{8} & r_{9} \end{matrix}];

T ^(j) : translation vector, T=[Tx Ty Tz]';

First calculate R, Tx, Ty;

(4.1) Calculate the image coordinates of each point

u=u ₀ +x/dx

v=v ₀ +y/dy

For a point (Xw, Yw, Zw) in space, after being captured by the camera, the image coordinates (mm) are (x, y), the pixel coordinates are (u, v), and (u ₀ , v ₀ ) is the image coordinate system The pixel coordinates of the origin in the pixel coordinate system, (dx, dy) is the distance in the x and y directions between adjacent pixels of the CCD, provided by the CCD manufacturer;

(4.2) Calculate five unknown quantities Ty ^-1 r ₁ , Ty ^-1 r ₂ , Ty ^-1 Tx, Ty ^-1 r ₄ , Ty ^-1 r ₅

For each 3D object point (Xw _k , Yw _k , Zw _k ), since the points are coplanar, the Z coordinate is taken as 0, and the corresponding image coordinates (x _k , y _k ), are defined by the collinear equation:

{x x}_{k k} = = f f \frac{{r r}_{11} X x {w w}_{k k} + + {r r}_{22} Y Y {w w}_{k k} + + Tx Tx}{{r r}_{77} X x {w w}_{k k} + + {r r}_{88} Y Y {w w}_{k k} + + Tz Tz}

{y the y}_{k k} = = f f \frac{{r r}_{44} {Xw wxya}_{k k} + + {r r}_{55} Y Y {w w}_{k k} + + Ty Ty}{{r r}_{77} X x {w w}_{k k} + + {r r}_{88} Y Y {w w}_{k k} + + Tz Tz}

Divide the above two equations to obtain: use the least square method to obtain the above five unknown quantities;

(4.3) Calculate r ₁ , ..., r ₉ , Tx, T _y

(4.31) Calculate |Ty|

Define the matrix:

C C = = [\begin{matrix} {r r}_{11}^{' '} & {r r}_{22}^{' '} \\ {r r}_{44}^{' '} & {r r}_{55}^{' '} \end{matrix}] = = [\begin{matrix} {r r}_{11} / / Ty Ty & {r r}_{22} / / Ty Ty \\ {r r}_{44} / / Ty Ty & {r r}_{55} / / Ty Ty \end{matrix}]

Then there are:

{Ty Ty}^{22} = = \frac{Sr Sr - - {[[{Sr Sr}_{22} - - 44 {(({r r}_{11}^{' '} {r r}_{55}^{' '} - - {r r}_{44}^{' '} {r r}_{22}^{' '}))}^{22}]]}^{11 / / 22}}{22 {(({r r}_{11}^{' '} {r r}_{55}^{' '} - - {r r}_{44}^{' '} {r r}_{22}^{' '}))}^{22}}

in:

Sr=r ₁ ′ ² +r ₂ ′ ² +r ₅ ′ ² +r ₄ ′ ² ,

If a certain row or column of matrix C is all 0, it is calculated as follows:

Ty ² =(r _i ′ ² +r _j ′ ² ) ^-1 , where ri _′ , r _j ′ are the remaining two elements of matrix C;

(4.32) Determine the sign of Ty

①Assume that Ty is a positive sign;

② Select a point in the captured image that is far from the center of the image; set its image coordinates as (x _k , y _k ), and its coordinates in the three-dimensional world as (Xw _k , Yw _k , Zw _k );

③Calculate the following values from the above results:

r ₁ =(Ty ^-1 r ₁ )*Ty, r ₂ =(Ty ^-1 r ₂ )*Ty, r ₄ =(Ty ^-1 r ₄ )*Ty

r ₅ =(Ty ⁻¹ r ₅ )*Ty, Tx=(Ty ⁻¹ Tx)*Ty,

x _k ′＝r ₁ Xw _k +r ₂ Yw _k +Tx

y _k ′＝r ₄ Xw _k +r ₅ Yw _k +Ty

If x _k and x _k ', y _k and y _k ' all have the same sign, then sgn(Ty)=+1, otherwise sgn(Ty)=-1;

(4.33) Calculate the rotation matrix R

According to the value of Ty, recalculate r ₁ , r ₂ , r ₄ , r ₅ , Tx;

R R = = [\begin{matrix} {r r}_{11} & {r r}_{22} & {((11 - - {r r}_{11}^{22} - - {r r}_{22}^{22}))}^{11 / / 22} \\ {r r}_{44} & {r r}_{55} & s the s {((11 - - {r r}_{44}^{22} - - {r r}_{55}^{22}))}^{11 / / 22} \\ {r r}_{77} & {r r}_{88} & {r r}_{99} \end{matrix}]

where s=sgn(r ₁ r ₄ +r ₂ r ₅ )

r ₇ , r ₈ , r ₉ are obtained from the outer product of the first two rows;

If the following focal length f is calculated according to such R as a negative value, then:

R R = = [\begin{matrix} {r r}_{11} & {r r}_{22} & {- - ((11 - - {r r}_{11}^{22} - - {r r}_{22}^{22}))}^{11 / / 22} \\ {r r}_{44} & {r r}_{55} & - - s the s {((11 - - {r r}_{44}^{22} - - {r r}_{55}^{22}))}^{11 / / 22} \\ - - {r r}_{77} & {- - r r}_{88} & {r r}_{99} \end{matrix}]

Then calculate the value of focal length f and T _z

For each calibration point, set up a linear equation including f and T _z as unknown parameters:

[\begin{matrix} {Y Y}_{k k} & - - {y the y}_{k k} \end{matrix}] [\begin{matrix} f f \\ Tz Tz \end{matrix}] = = {w w}_{k k} {y the y}_{k k}

in:

Y _k ＝r ₄ x _wk +r ₅ y _wk +r ₆ *0+Ty

w _k ＝r ₇ x _wk +r ₈ y _wk +r ₉ *0

By solving the equation, f and T _z can be obtained;

(5) Perform three-dimensional reconstruction of each point according to the phase and epipolar geometry, and obtain the three-dimensional coordinates of the surface point of the object:

(5.1) Calculate the fundamental matrix F

F＝A ₂ ^-T EA ₁ ^-1

in

{A A}_{22} = = [\begin{matrix} {f f}_{22} / / {dx dx}_{22} & 00 & {u u}_{0202} \\ 00 & {f f}_{22} / / {dy dy}_{22} & {v v}_{0202} \\ 00 & 00 & 11 \end{matrix}],, {A A}_{11} = = [\begin{matrix} {f f}_{11} / / {dx dx}_{11} & 00 & {u u}_{0101} \\ 00 & {f f}_{11} / / {dy dy}_{11} & {v v}_{0101} \\ 00 & 00 & 11 \end{matrix}]

Where f ₁ , (u ₀₁ , v ₀₁ ), dx ₁ , dy ₁ are internal parameters of the first camera,

f ₂ , (u ₀₂ , v ₀₂ ), dx ₂ , dy ₂ are internal parameters of the second camera;

E＝[T] _x R ⁽²⁾ R ^(1)-1 , called the essential matrix;

T＝T ⁽²⁾ -R ⁽²⁾ R ^(1)-1T ⁽¹⁾

Express T as [Tx, Ty, Tz], then the antisymmetric matrix is [T] _x

{[[T T]]}_{x x} = = [\begin{matrix} 00 & - - Tz Tz & Ty Ty \\ Tz Tz & 00 & - - Tx Tx \\ - - Ty Ty & Tx Tx & 00 \end{matrix}]

(5.2) Calculate a point P on the image captured by the right camera, and its coordinates are Find its epipolar equation parameters on the image captured by the left camera; and find its matching point Q on the above straight line, the method is that the phase values of point P and point Q are equal;

A known According to the following straight line equation, find the point Q of the point P on the image captured by the right camera on the image captured by the left camera, and its coordinates are

{\overset{~ ~}{m m}}_{22}^{T T} F f {\overset{~ ~}{m m}}_{11} = = 00,,

in: is the pixel coordinate (u ₂ , v ₂ , 1) ^T of point P:

is the pixel coordinate of point Q (u ₁ , v ₁ , 1) ^T ;

According to the principle that the phases of point P and point Q are equal, find point Q on a straight line determined by the above straight line equation;

(5.3) Find the Q point, then store this matching point, according to the relationship between the following three-dimensional space point S (Xw, Yw, Zw) and the corresponding image coordinates of the images captured by the two cameras:

{x x}^{((j j))} = = {f f}^{((j j))} \frac{{R R}^{((j j))}_{1111} Xw wxya + + {R R}^{((j j))}_{1212} Yw w + + {R R}^{((j j))}_{1313} Zw Zw + + {T T}^{((j j))} x x}{{R R}^{((j j))}_{3131} Xw wxya + + {R R}^{((j j))}_{3232} Yw w + + {R R}^{((j j))}_{3333} Zw Zw + + {T T}^{((j j))} z z},,

{y the y}^{((j j))} = = {f f}^{((j j))} \frac{{R R}^{((j j))}_{21 twenty one} Xw wxya + + {R R}^{((j j))}_{22 twenty two} Yw w + + {R R}^{((j j))}_{23 twenty three} Zw Zw + + {T T}^{((j j))} y the y}{{R R}^{((j j))}_{3131} Xw wxya + + {R R}^{((j j))}_{3232} Yw w + + {R R}^{((j j))}_{3333} Zw Zw + + {T T}^{((j j))} z z},,

j=1, 2, which is the number of the camera.

2. The method for measuring the three-dimensional profile of a surface according to claim 1, characterized in that the phase shift angle φ _i =i*90 of the phase shift grating, i=1, ... 4, that is, the phase shift grating image has 4 frames, the calculation formula of the phase master value is:

φ (x, the y) = a the tan (\frac{I_{2} - I_{4}}{I_{3} - I_{1}}),

Among them, I ₁ , I ₂ , I ₃ , and I ₄ are the light intensities of the point (u, v) in the four phase shift gratings.

3. The method for measuring the three-dimensional surface profile of an object according to claim 1, wherein the coded grating has 7 pieces, and the binary code sequence of each point can be converted into the position of the point in the image, i.e. the number of cycles.