CN103942829A - Single-image human body three-dimensional posture reconstruction method - Google Patents

Abstract

The invention discloses a single-image human body three-dimensional posture reconstruction method which comprises the following steps that first, a human body standard three-dimensional skeleton model is built; second, the positions of human body articulation points and limb end points are generated in a human body image; third, proportion parameters of a weak perspective projection are estimated; fourth, all joints in the standard three-dimensional skeleton model are primarily flush with corresponding marks in the image; fifth, an optimization algorithm is used for optimization. The method improves three-dimensional posture reconstruction efficiency.

Description

Human 3D Pose Reconstruction from a Single Image

technical field

The invention relates to a method for reconstructing a three-dimensional posture, in particular to a method for reconstructing a three-dimensional posture of a human body from a single image.

Background technique

The three-dimensional reconstruction of human body posture is a frontier direction that has attracted much attention in the field of computer vision in recent years. It understands and describes human body detection, motion analysis, three-dimensional human body tracking and behavior in human body image sequences, and belongs to the category of image analysis and understanding. . From a technical point of view, the research content of human motion analysis is quite rich, mainly involving pattern recognition, image processing, computer vision, artificial intelligence and other subject knowledge; Self-occlusion also brings great challenges to the 3D reconstruction of human poses.

Due to the wide application prospects and potential economic value of human body 3D posture reconstruction and human body 3D motion analysis in advanced human-computer interaction, security monitoring, video conferencing, medical diagnosis, and content-based image storage and retrieval, it has stimulated the domestic The strong interests of scientific research workers and related businesses at home and abroad, especially in the United States, the United Kingdom and other countries have carried out a large number of related research projects. Three-dimensional reconstruction of human body posture is currently mainly used in intelligent video surveillance, intelligent human-machine interface, virtual reality and game production, sports analysis and rehabilitation evaluation.

Considering the method, the 3D reconstruction of human pose can be divided into multi-camera method and single-camera method. Multi-camera methods include motion capture, space sculpting, stereo vision, and structured light methods; Kinects developed by Microsoft uses a structured light method; although a camera is used for 3D image acquisition, it is essentially a multi-camera method. Relatively speaking, since the 2D image loses the 3D information of the target object, it is more difficult and challenging to reconstruct the 3D pose of the human body from an image of a single camera. In single-camera human 3D pose reconstruction, image features usually used include 2D human contour features, human image edge features, texture features, etc., but the reconstruction efficiency is low. This method adopts the two-dimensional node mark feature of the human body image, and cooperates with the three-dimensional skeleton model of the human body, which is a new and effective method.

Contents of the invention

The technical problem to be solved by the present invention is to provide a method for reconstructing a three-dimensional posture of a human body from a single image, which improves the efficiency of three-dimensional posture reconstruction.

The present invention solves the above-mentioned technical problems through the following technical solutions: a method for reconstructing a three-dimensional posture of a human body from a single image, wherein the method for reconstructing a three-dimensional posture of a human body from a single image comprises the following steps:

Step 1, establishing a standard three-dimensional skeleton model of the human body;

Step 2, generating the positions of human body joints and limb endpoints in the human body image;

Step 3. Estimate the scale parameter of the weak perspective projection. The scale parameter is the maximum value of the distance between the adjacent joint points of the human body in the image and the corresponding limb length ratio in the standard 3D skeleton model. According to the estimated scale parameter of the weak perspective projection Scale up the standard 3D skeleton;

Step 4: keep the length of the limbs of the skeleton unchanged, start from the root node, and adjust the positions of the human body nodes in the standard 3D skeleton in turn, so that all the nodes in the standard 3D skeleton are basically aligned with the corresponding marker points in the image;

Step 5, using an optimization algorithm to optimize the human body posture, so that the sum of the differences between the weak perspective projections of all nodes of the standard 3D skeleton on the human body image and the positions of the corresponding image marker points is minimized.

Preferably, the method for reconstructing the three-dimensional pose of the human body from a single image uses the motion constraints of the structure of the human skeleton on the skeleton nodes.

Preferably, said image is generated using a weak perspective projection model.

Preferably, the human body pose optimization in Step 5 realizes the best alignment between the weak perspective projection of each node of the three-dimensional human skeleton on the image plane and the human body node marks in the image.

Preferably, after the step five is optimized, the three-dimensional coordinate positions of the human skeleton nodes and the weak perspective projection scale parameters will be determined, thereby reconstructing the three-dimensional posture of the human body.

The positive and progressive effect of the present invention is that the method for reconstructing the three-dimensional posture of a human body from a single image of the present invention improves the efficiency of three-dimensional posture reconstruction. The invention has the advantages of few reconstruction parameters and high efficiency, and can be applied to intelligent video monitoring, intelligent man-machine interface, virtual reality, game production, sports analysis, health evaluation and the like. The invention specifically relates to the realization of three-dimensional posture reconstruction of a human skeleton by using two-dimensional markers of an image human body and node position registration of a three-dimensional skeleton of a human body in the case of weak perspective projection, and belongs to the field of computer vision. The present invention reduces seven global variables required in the process of three-dimensional posture reconstruction to one. Since the number of variables to be determined in the three-dimensional reconstruction is reduced, the nonlinear degree of the calculation process is reduced, the convergence range of the calculation is increased, and the tolerance to the initial three-dimensional posture error of the human body is improved in the reconstruction. At the same time, the calculation is simplified. The process improves the calculation speed and greatly improves the efficiency of 3D pose reconstruction.

Description of drawings

FIG. 1 is a schematic diagram of a standard three-dimensional skeleton model of a human body used in an embodiment of the present invention.

FIG. 2 is a schematic diagram of a single human body image used in the present invention.

Fig. 3 is a schematic diagram of a human body image and two-dimensional markers used in the present invention.

Fig. 4 is a schematic diagram of aligning two-dimensional image markers with three-dimensional skeleton nodes of a human body according to the present invention.

Detailed ways

The preferred embodiments of the present invention are given below in conjunction with the accompanying drawings to describe the technical solution of the present invention in detail.

The invention relates to a method for reconstructing a three-dimensional posture of a human body from a single two-dimensional image, in particular to realizing three-dimensional posture reconstruction of a human body skeleton by using two-dimensional human body markers in the image and node position registration of a three-dimensional skeleton of the human body under the condition of weak perspective projection. The present invention is based on the following three provisions: 1) weak perspective projection model is used for image generation; 2) human body joint points or limb endpoints in the image are used as image marker points; 3) the three-dimensional posture of human body is represented by a standard three-dimensional skeleton, as shown in Figure 1 , where the dots represent skeleton nodes. The invention adopts a three-dimensional skeleton model of a human body to realize three-dimensional reconstruction of a human body posture, and restricts positions of joint points of a human body in a three-dimensional space by means of limb constraints of the skeleton model of a human body. The 3D pose of a standard 3D skeleton is achieved by optimally aligning the nodes of the 3D skeleton with the corresponding body node markers in the image. The present invention uses the three-dimensional posture of the human body obtained from the previous frame image as the estimated posture of the three-dimensional posture of the human body corresponding to the current frame image, and applies the three-dimensional posture reconstruction method of the human body in a single image to the three-dimensional posture reconstruction and three-dimensional tracking of the human body in the video image. In the case of weak perspective projection, there is a proportional relationship between the image coordinates of human body marker points and the corresponding three-dimensional node space coordinates, that is, X=sx, Y=sy, where (x, y, z) are nodes on the human body three-dimensional skeleton model Coordinates; (X, Y) are the corresponding two-dimensional marker coordinates on the image.

The human body three-dimensional pose reconstruction method of a single image of the present invention comprises the following steps:

Step 1, according to Figure 1, establish a standard 3D skeleton model of the human body; define the human body nodes in the standard 3D skeleton in turn, for example, define the waist joint point as the root node (that is, node zero 0); sequentially define the neck joint point, the top of the head point, left shoulder point, left elbow point, left hand point, right shoulder point, right elbow point, right hand point, left hip point, left knee point, left foot point, right hip point, right knee point, right Foot nodes, etc., respectively defined as node 1, node 2, node 3, node 4, node 5, node 6, node 7, node 7, node 8, node 9 9. Node 10 on the 10th, Node 11 on the 11th, Node 12 on the 12th, Node 13 on the thirteenth, Node 14 on the 14th, etc.; according to application requirements, the model can also be modified during implementation, such as at the root node A "chest node" is added between the neck node and the neck node.

Step 2: Generate (automatically generated by software or manually) the positions of human body joints and limb endpoints in the human body image; the position coordinates of these human body nodes can be obtained by human-computer interaction, or automatically by software (such as Technique disclosed in: M. Andriluka, S. Roth, and B. Schiele, "Pictorial Structures Revisited: People Detection and Articulated Pose Estimation," CVPR, 2009);

Step 3, estimate the scale parameter of the weak perspective projection, the scale parameter s is the maximum value of the distance between the adjacent joint points of the human body in the image and the corresponding limb length ratio in the standard 3D skeleton model, according to the estimated weak perspective projection scale The parameter enlarges the standard 3D skeleton;

Step 4: keep the length of the limbs of the skeleton unchanged, start from the root node, and adjust the positions of the human body nodes in the standard 3D skeleton in turn, so that all the nodes in the standard 3D skeleton are basically aligned with the corresponding marker points in the image. As follows: adjust the position of the first node in the standard three-dimensional skeleton in order to align with the neck node mark in the image; adjust the second node to align with the head node mark in the image; adjust the third node to align with the Align the left shoulder node mark in the image; adjust node 4 to align with the left elbow node mark in the image; adjust node 5 to align with the left-hand node mark in the image, and operate in sequence until node 14 is adjusted So far, all the nodes in the standard three-dimensional skeleton are basically initially aligned with the corresponding marker points in the image;

Step 5, using an optimization algorithm to optimize the human body posture, so that the sum of the differences between the weak perspective projections of all nodes of the standard 3D skeleton on the human body image and the positions of the corresponding image marker points is minimized. After optimization, the three-dimensional coordinate positions of the human skeleton nodes and the weak perspective projection ratio parameters will be determined, so that the reconstruction of the three-dimensional pose of the human body is completed, so that the obtained three-dimensional coordinate positions of the skeleton joint points and the three-dimensional pose of the human body can also be converted into skeleton limbs Marked by rotating around the upper node of the limb (that is, the node closer to the root node);

The method proposed by the invention can be extended to the three-dimensional pose reconstruction and three-dimensional tracking of the human body of the video image. The specific method is to use the three-dimensional posture of the human body obtained from the previous frame image as the result of step five, and repeat the process of step five and step six until the end of the video. The invention adopts the motion constraints of the skeleton nodes by the structure of the human skeleton. The images were generated using a weak perspective projection model. The human body posture optimization in step five realizes the best alignment of the weak perspective projection of each node of the human body three-dimensional skeleton on the image plane and the human body node marks in the image. After step five is optimized, the three-dimensional coordinate positions of the human skeleton nodes and the weak perspective projection scale parameters will be determined, thereby reconstructing the three-dimensional posture of the human body.

The technical solution of the present invention will be further described in detail below in conjunction with FIG. 2 to FIG. 4 . In this example, the human body image shown in Figure 2 is selected, and it is hoped that the method provided by the present invention can be used to reconstruct the posture of the three-dimensional skeleton corresponding to the human body in the figure. The specific implementation steps are as follows:

First, establish a standard three-dimensional skeleton model of the human body. According to the statistics of human anatomy, the length of the torso is 550 pixel units, the length between the head and neck is 32 units, the neck node to the shoulder node is 220 units, the upper arm is 280 units, the forearm is 250 units, the root node to the cross node is 90 units, the thigh is 450 units, Lower leg 420 units. Define the waist joint point in the skeleton as the root node (that is, node zero); in turn, define the neck joint point, head node, left shoulder joint point, left elbow joint point, left hand joint point, etc. as the first node and the second node , Node No. 3, Node No. 4, Node No. 5, etc.

Second, in the human body image, use the human-computer interaction method to mark the joint point positions of the human body image with the mouse, and the result is shown in Figure 3. Obtain the image coordinate value (Xi, Yi) of the corresponding node, where the subscript i corresponds to the node number of the human skeleton.

Third, estimate the weak perspective projection scale parameter. According to the Pythagorean theorem, the length of human body limbs between image markers is calculated. The maximum value of the proportional coefficient between the length of the two-dimensional human body limbs in the image and the corresponding limb length in the three-dimensional skeleton is used as the scale parameter of the weak perspective projection, which is the following formula (1):

s=Max{L _ij /l _ij }………………………………………(1)

Among them, L _ij is the length of the human limb in the image; l _ij is the corresponding length of the human limb in the three-dimensional skeleton.

Fourth, align the root node in the enlarged three-dimensional skeleton with the waist node mark in the image, and the z coordinate of the root node of the skeleton can be any value. In this embodiment, in order to ensure that the z coordinate of any node in the three-dimensional skeleton of the human body does not have a negative value in any posture, z=3000 is taken here. Adjust the root node position of the human three-dimensional skeleton so that it basically satisfies the following formula (2):

X ₀ ≈sx ₀ ; Y ₀ ≈sy ₀ ; z ₀ =3000………………(2)

Under the constraints of the three-dimensional skeleton of the human body, that is, the limbs in the three-dimensional skeleton can only perform three-dimensional rotation around the node on the limb that is closer to the root node. Starting from the first node, the positions of each node in the three-dimensional skeleton of the human body are adjusted sequentially , so that it is basically aligned with the corresponding joint points in the image. That is, the following formula (3):

X _i ≈s x _i ; Y _i ≈sy _i

Among them, i=1, 2, 3, ...

Fifth, using the annealing particle filter method (in fact, any numerical optimization method can be used), under the constraints of the three-dimensional human skeleton, the weak perspective projection of each node of the three-dimensional human skeleton on the human body image plane and the human body node marks in the image are optimal Alignment, that is, the following formula (4):

After optimization, the node coordinates (x _i , y _i , z _i ), i=1, 2, 3, ... on the three-dimensional skeleton of the human body are finally determined, thus completing the reconstruction of the three-dimensional posture of the human body. Figure 4 shows the results of 3D reconstruction.

The present invention adopts the following five models or modules: (1) A standard three-dimensional skeleton model of the human body, wherein the length of each limb is as described above. In the standard 3D skeleton model, the transformation relationship between the rotation angle of the skeleton limb around the node and the spatial position change of other limb nodes due to the limb rotation is established. (2) The data read-in module has two ways of inputting the position of the image mark. If the human body node marks in the image are obtained by an automatic method, read in the position of the human body node marks in the image in sequence according to the node numbers in the human three-dimensional skeleton; otherwise, use the mouse to manually determine the image in the human-computer interaction interface of the read-in module The positions of human body node markers in . (3) Human body posture initialization module, including two functions: First, calculate and draw the projection of the human body three-dimensional skeleton on the image plane (x-y plane). 2. Provide a human-computer interaction interface, use human-computer interaction to change the orientation angle of the limbs in the three-dimensional skeleton of the human body, thereby adjusting the position of the limb nodes, so that on the image plane, the weak perspective projection of the three-dimensional skeleton nodes of the human body and the human body image The node markers are located relatively close together. (4) Human body posture optimization module: realize the optimal alignment of the weak perspective projection of each node of the human body's 3D skeleton on the image plane and the human body node marks in the image. (5) Result output module: according to the node data of the optimized 3D human skeleton, draw the 3D view of the human skeleton; or, according to the 3D posture of the human skeleton, draw the triangular surface representation of the 3D human body shell, and finally perform the result output.

Various modifications and changes can be made to the present invention by those skilled in the art. Thus, the present invention covers the modifications and changes that come within the scope of the appended claims and their equivalents.

Claims (5)

1. a human body three-dimensional attitude reconstruction method for single image, is characterized in that, the human body three-dimensional attitude reconstruction method of described single image comprises the following steps:

Step 1, sets up the standard three-dimensional framework model of human body;

Step 2 generates the position of human joint points and limbs end points in human body image;

Step 3, estimate the scale parameter of weak perspective projection, scale parameter is distance between the human body adjacent segment point in image and the maximal value of limbs length ratio corresponding in standard three-dimensional framework model, according to the estimated weak perspective projection scale parameter going out, standard three-dimensional framework is amplified;

Step 4, keeps the limbs length of skeleton constant, from root node, the position of the people's body node in adjustment criteria three-dimensional framework successively, realize all nodes in standard three-dimensional framework all with corresponding basic tentatively the aliging of gauge point in image;

Step 5, adopts optimized algorithm to carry out human body attitude optimization, and the weak perspective projection of all nodes that make standard three-dimensional framework on human body image is minimum with the sum total of the difference of corresponding image tagged point position.

2. the human body three-dimensional attitude reconstruction method of single image as claimed in claim 1, is characterized in that, the kinematic constraint of the structure that the human body three-dimensional attitude reconstruction method of described single image has adopted human skeleton to skeleton node.

3. the human body three-dimensional attitude reconstruction method of single image as claimed in claim 1, is characterized in that, the generation of described image adopts weak perspective projection model.

4. the human body three-dimensional attitude reconstruction method of single image as claimed in claim 1, it is characterized in that, the human body attitude optimization of described step 5 realizes the weak perspective projection of each node of human body three-dimensional skeleton on the plane of delineation and the best alignment of the human body vertex ticks in image.

5. the human body three-dimensional attitude reconstruction method of single image as claimed in claim 1, it is characterized in that, after described step 5 is optimized, the three-dimensional coordinate position of human skeleton node and weak perspective projection scale parameter will be determined, thus reconstruct the 3 d pose of human body.