CN113706512B - Live pig weight measurement method based on deep learning and depth camera - Google Patents

Abstract

The invention discloses a pig weight measurement method based on deep learning and a depth camera, which obtains a depth image of a pig; converts the depth image into three-dimensional point cloud data; and preprocesses the three-dimensional point cloud data to remove noise; Put the noise-removed 3D point cloud data into the point cloud convolutional neural network PointNet++ model for deep learning, remove the background 3D point cloud data; project the 3D point cloud data after removing the background into the 3D coordinate system, and find the corresponding extreme Value point position, that is, to obtain the corresponding feature point coordinates, so as to obtain the corresponding body size data; take the body size data of the live pig as the independent variable, and take the weight of the live pig as the dependent variable to obtain the weight of the live pig. The present invention no longer needs a lot of manpower and material resources to weigh live pigs one by one, but accurately estimates the weight by collecting images of live pigs, and performing image analysis and calculation, so as to achieve the purpose of batch weight measurement.

Description

A pig weight measurement method based on deep learning and deep camera

technical field

The invention relates to the field of agriculture, in particular to a pig weight measurement method based on deep learning and a deep camera.

Background technique

my country is the largest country in the world for the production of live pigs, ranking first in the world in both the scale of live pig breeding and the consumption of raw pork. The weight of live pigs is an important index to evaluate its growth and development status, and it is also the characteristic basis for the selection and evaluation of gilt pigs. Body weight index is an important basis for evaluating the reproductive ability and feeding ability of sow pigs. A sow with a suitable body weight has a high litter size and a high healthy piglet rate. At the same time, the feeding amount of the pigs is also regulated by the data of the pigs' weight. In the process of raising and managing pigs, the feeding work of pigs can be adjusted appropriately according to their weight data. Keep abreast of the nutritional status of live pigs to avoid a decline in the health of sow pigs.

However, the weight measurement of pigs has always been a troublesome and troublesome matter, and even many small pig farms and family farms are only visually inspected or directly ignored. Visually measure the weight of live pigs. Most of the current methods for evaluating the weight of live pigs in China are based on the experience of breeders to visually measure the weight of live pigs and score them. According to statistics, the scores often fluctuate greatly, indicating that there are large errors in the existing visual scoring methods.

There are two more accurate methods: body size estimation and direct measurement. Body measurement and estimation mainly include: tape measurement and estimation, backup body size and PIC body weight speed measuring ruler. However, manual measurement errors are large, and there are problems such as consuming a lot of manpower and extremely low efficiency. At the same time, because most of the pigs are large in size, it is difficult to ensure the stability of the pigs, which will pose a threat to the safety of the surveyors. The most widely used method of direct measurement is: platform scale (weighbridge) method. Compared with the former method, this method is more accurate in measurement, but the "platform scale (weighbridge)" is costly, consuming a lot of materials and money. In the face of modern large-scale farming with a huge base, the traditional measurement method is obviously powerless and inefficient. Weighing one by one The heavy method will undoubtedly increase the input of labor costs, and it is time-consuming and labor-intensive, and the focus cycle is longer.

Contents of the invention

In order to solve the above problems, the present invention provides a pig weight measurement method based on deep learning and a deep camera.

In order to achieve the above object, the technical scheme adopted in the present invention is as follows:

A method for measuring pig body weight based on deep learning and deep cameras, comprising the following steps:

Obtain the depth image of the pig;

Converting the depth image into three-dimensional point cloud data;

Preprocessing the 3D point cloud data to remove noise;

Use the PyTorch deep learning framework to run the point cloud convolutional neural network PointNet++ model, put the noise-removed 3D point cloud data into the point cloud convolutional neural network PointNet++ model for deep learning, and remove the background 3D point cloud data;

Project the 3D point cloud data after removing the background to the 3D coordinate system, find the minimum point of z value on each slice point cloud, merge each point with the minimum z value into a new point column, and perform discrete points on the point column Remove, get the corresponding two-dimensional fitting line, and find the corresponding extreme point position, that is, get the corresponding feature point coordinates, so as to get the corresponding body size data; Y=a ₀ +a ₁ x ₁ +a ₂ x ₂ +…+a _n x _n +ε

Taking the body size data of the live pig as the independent variable and the weight of the live pig as the dependent variable, the weight of the live pig is obtained according to the following formula:

Y＝α ₀ +α ₁ X ₁ +α ₂ X ₂ +…+α _n X _n +ε

Among them, Y is the weight of the pig, a ₀ , a ₁ , a ₂ ...a _n and ε are coefficients corresponding to different types of pigs, and x ₁ , x ₂ ... x _n are the body size data of the pig.

Optionally, obtaining the depth image of the live pig includes: installing the Kinect depth camera on the side of the feeding machine to capture the depth image of the still pig while eating, with a distance range of

Among them, a is the angle of view of the Kinect depth camera, L is the length of the pig eating area, and D is the distance between the Kinect depth camera and the pig.

Optionally, converting the depth image into 3D point cloud data includes: using its built-in point cloud acquisition function in the Kinectfor WindowsSDK2.0 software development environment to convert the acquired depth image into 3D point cloud data.

Optionally, preprocessing the 3D point cloud data, and removing noise includes: performing filtering and noise reduction processing on the 3D point cloud data set using a bilateral filter method, the expression of which is:

J ⁰ ＝IJ ^t+1 ＝f(J ^t )

Among them, J ⁰ is the initial image, J ^t is the result after t iterations, and f(Jt) is the filter.

Optionally, the feature points include the ground plane, the top of the head of the pig, the first tail vertebra of the pig, the chinchilla bone of the pig and the phalanx bone of the pig, wherein the equation of the ground plane is

ax+by+cz+d=0

Corresponding body size data, including body length X ₁ , body height X ₂ , body oblique length X ₃ and body width X ₄ ;

Among them, the body length X ₁ is the linear distance from N(x ₁ , y ₁ , z ₁ ) to M(x ₂ , y ₂ , z ₂ ) of the pig’s head, and its expression is: X ₁ =|x ₁ -x ₂ |;

Body height X ₂ is the straight-line distance from the pig's first tailbone M (x ₂ , y ₂ , z ₂ ) to the ground plane ax+by+cz+d=0, and its expression is:

Body oblique length X ₃ is the straight-line distance from pig’s bun bone A (x ₃ , y ₃ , z ₃ ) to pig’s phalanx B (x ₄ , y ₄ , z ₄ ), and its expression is:

The body width X ₄ is measured by the Kinect depth camera, and its expression is: X ₄ = |z _u -z _d |.

Compared with the prior art, the technical progress achieved by the present invention lies in:

In the present invention, the live pigs only need to pass through in an orderly manner, and after setting up the shooting area of the depth camera in advance, the weight of the live pigs can be measured through various extraction parameters. Emphasis on efficiency, conform to the current trend of large-scale farming

Compared with the traditional method of measuring the weight of live pigs, the method of using deep learning to measure the weight of live pigs is more convenient and safer. It can realize the non-contact measurement of pig weight, reduce human intervention, guarantee the natural growth requirements of pigs to a great extent, and achieve the purpose of welfare breeding.

The present invention focuses on researching a non-contact pig body weight estimation system. Collecting whole-body images of live pigs and estimating the weight of live pigs through deep learning algorithms is of great significance for promoting precise, modern and intelligent farming. The present invention aims to design an accurate and efficient pig weight non-contact detection system, solve the weight measurement problem through deep learning algorithm, reduce unnecessary loss of manpower and material resources; realize pig breeding informatization and standardized management, thereby Help farms to expand the scale of farming and create higher benefits.

Description of drawings

The accompanying drawings are used to provide a further understanding of the present invention, and constitute a part of the description, and are used together with the embodiments of the present invention to explain the present invention, and do not constitute a limitation to the present invention.

In the attached picture:

Fig. 1 is a schematic flow chart of the method of the present invention.

FIG. 2 is a schematic structural diagram of the installation of the depth camera of the present invention.

Detailed ways

The following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present invention will be described below with reference to the accompanying drawings.

As shown in Figure 1, the present invention discloses a pig weight measurement method based on deep learning and a deep camera, including: S01: Obtain a pig's depth image;

As shown in Figure 2, obtaining the depth image of the live pig includes: setting up the Kinect depth camera on the side of the feeding machine, and capturing the depth image of the still pig while eating, with a distance range of

Among them, a is the angle of view of the Kinect depth camera, L is the length of the pig’s eating area, and D is the distance between the Kinect depth camera and the pig. The distance between the Kinect depth camera and the pig can be adjusted from 2.5m to 2.7m through the above formula.

The Kinect depth camera uses TOF (Time Of Light) ranging technology to obtain depth information, that is, the infrared emitter actively sends out continuous infrared light pulses, and the depth sensor receives the infrared light pulses returned from the object, and the object distance is obtained according to the round-trip flight time of the infrared light pulses. The depth value of the sensor, so that the feature recognition of the measured object is not limited by lighting, which reduces the requirements for external lighting, and through higher depth fidelity and greatly improved noise floor, it can collect smaller object data information.

Kinect depth camera calibration is a necessary process to accurately measure the target object. Through Kinect depth camera calibration, the distortion of the lens can be corrected and the coordinates of the target object in metric units in the world coordinate system can be obtained. During the image acquisition process of live pigs, the Kinect depth camera can be controlled by infrared trigger, and installed next to the pig feeder, the system will be turned on whenever the pigs eat collectively, and when the pigs block the infrared device, The system starts the Kinect depth camera work to take pictures for the live pig at this moment. The infrared trigger device adopts E18-8MNK photoelectric sensor, the detection distance is 0-8 meters, the rated current is 100mA, and the rated voltage is 5v.

The focus and difficulty of the pig weight detection system are reflected in the acquisition and processing of pig images. Therefore, the acquisition process of pig morphological information is particularly important, especially the structure and layout of the Kinect depth camera. In order to optimize the amount of calculation and increase the speed of weight measurement, we Considering that the biological scale data is roughly symmetrical and the accuracy loss is minimal, the dual Kinect depth camera solution is discarded, and only a single Kinect depth camera is installed on the side.

Whether the front-end can successfully collect and preprocess the images of live pigs. Accurate and clear images are extremely characteristic in uncertain and relatively chaotic farm environments. In this embodiment, a Kinect depth camera is set up on the side of the feeding machine. Capture the image of the still pig while eating to prevent the head twisting of the pig from affecting the image acquisition, and convert the data from the depth image to the 3D point cloud data.

In the application of deep learning, the early image acquisition tasks are extremely characteristic, and the number and fineness of the produced data sets directly affect the results of later model training, which in turn has a great impact on the accuracy of pig body measurement data . We obtain a large number of depth images of pigs, including different body shapes and the same orientation (side-view images), and at the same time, after selecting the obtained depth image data, they are all converted into 3D point clouds to save and create point cloud data sets of pigs .

Image quality is the primary condition to ensure the accuracy of calculated body weight data. Due to the complex environment of image extraction, it is difficult to ensure the absolute stillness of live pigs during image extraction, in order to ensure the reliability and stability of image quality. At the same time, in the process of image transmission and conversion into a 3D point cloud dataset, unnecessary noise is reduced in the image, which affects the quality of depth image acquisition. Therefore, we also need to preprocess the collected data to ensure the accuracy of point cloud data and improve the quality of point cloud data.

S02: Convert the depth image into three-dimensional point cloud data;

Point cloud is divided into two types in terms of composition characteristics, one is ordered point cloud, and the other is disordered point cloud. The point cloud data restored from the depth image can be arranged according to the three-dimensional data, and it is easy to find the information of its adjacent points, exclude the invalid point cloud sequence, and retain the valid sequence. The processing of point cloud data is easier and faster than that of depth images, so it is necessary to convert depth images into point cloud data and make corresponding point cloud data sets. In the Kinectfor Windows SDK2.0 software development environment, use its own corresponding point cloud acquisition function to convert the acquired depth image into point cloud data.

S03: Preprocessing the 3D point cloud data to remove noise;

Due to the unavoidable accuracy of the scanning equipment, and the slight shaking of the equipment during the test, or the complex changes in the environment, these may cause the extracted point cloud data to contain a large number of hash points and isolated points. The production of the data set These noises need to be filtered out. The present invention intends to mainly use the bilateral filtering method for filtering and noise reduction processing. Compared with other filtering methods, the bilateral filtering method has a stronger ability to maintain the edge data of the point cloud, and the bilateral filtering is combined with the spatial proximity and similarity of the point cloud data. A kind of compromise processing, considering spatial information and similarity at the same time, to achieve the purpose of edge preservation and denoising. When continuously applying a bilateral filter or other average-based boundary extraction filters, bilateral filtering is continuously performed, and the general expression is: J ⁰ =IJ ^t+1 =f(J ^t ), where J ⁰ is the initial image, J ^t is the result after t iterations, f(J ^t ) is the filter, continuous execution of the bilateral filter will not remove small structures, on the contrary, it can better retain the details of the data, but will remove the pig's body Noise such as rough edges.

S04: Use the PyTorch deep learning framework to run the point cloud convolutional neural network PointNet++ model, put the noise-removed 3D point cloud data into the point cloud convolutional neural network PointNet++ model for deep learning, and remove the background 3D point cloud data;

Although the noise reduction process has been carried out, it is difficult to obtain the point cloud data of individual pigs in the environment of the farm and ignore the complex background point cloud data set in the collection of 3D point cloud data. Unnecessary background and other point cloud data are removed from the data volume, reducing the amount of point cloud data that needs to be processed. Therefore, we need to segment the point cloud data to obtain the point cloud data of live pigs separately.

The present invention utilizes the PyTorch deep learning framework to run the point cloud convolutional neural network PointNet++ model, puts the noise-removed 3D point cloud data into the point cloud convolutional neural network PointNet++ model for deep learning, and converts the 3D data into a depth image for use by image processing The method of applying 3D convolutional neural network after traditional convolutional neural network or voxelization usually leads to a part of data loss or excessive calculation cost. The method of directly processing point cloud can maximize the use of 3D point cloud data features to get more feature information. The PointNet++ model can extend the traditional two-dimensional convolutional neural network to three-dimensional use. The PointNet++ model can directly process point cloud data, and extend the traditional image input format to the input of point cloud data. Using the PointNet++ model to directly segment the 3D point cloud data, and then process the point cloud data, the obtained result is more direct, the number is faster, and the required data set is smaller than the above method, and the accuracy is higher. At the same time, compared with the time of depth image preprocessing, this method consumes less energy, saving cost and manpower.

The PyTorch deep learning framework is a combination of Caffe2 and Torch. It refactors and unifies the code libraries of the two frameworks, deletes duplicate components and shares upper-level abstractions, and obtains a unified framework that supports efficient graph mode execution, mobile deployment and Extensive vendor integrations and more. Relatively speaking, development is more flexible, and code writing is easier. The PointNet++ model has higher precision, can directly process point cloud data, the processing steps are simpler, and it is more effective for the collection and detection of body weight, making the measurement of subsequent data more accurate.

SO5: The calculation of body size data can be converted into the distance length of the corresponding feature points. Project the processed point cloud data into the three-dimensional coordinate system, first find the minimum point of z value on each slice point cloud, merge these points with minimum z value into a new point column, and then perform discrete point removal on the point column , that is, the corresponding two-dimensional fitting line is obtained. When extracting feature points, it is only necessary to program and identify the x and z values of the point column, and then project in each direction accordingly, and fit the two-dimensional curve under the corresponding two-dimensional coordinate system, and obtain them respectively through mathematical functions Corresponding extreme point positions, the corresponding feature point coordinates can be obtained, and the corresponding body size data can be obtained. Since what is collected is the depth image of the side body of the pig, the data length should be multiplied by 2 when calculating the body width of the pig.

The body measurement point is extracted from the curve to obtain the characteristic points of the top of the pig's head, the first tail vertebra of the pig, the chinchilla bone of the pig and the characteristic parts of the phalanx bone of the pig. Using the feature points and the ground plane equation ax+by+cz+d=0, calculate the pixel length of the body-scale data:

Corresponding body size data, including body length X ₁ , body height X ₂ , body oblique length X ₃ and body width X ₄ ;

Among them, the body length X ₁ is the linear distance from N(x ₁ , y ₁ , z ₁ ) to M(x ₂ , y ₂ , z ₂ ) of the pig’s head, and its expression is: X ₁ =|x ₁ -x ₂ |;

Body height X ₂ is the straight-line distance from the pig's first tailbone M (x ₂ , y ₂ , z ₂ ) to the ground plane ax+by+cz+d=0, and its expression is:

Body oblique length X ₃ is the straight-line distance from pig’s bun bone A (x ₃ , y ₃ , z ₃ ) to pig’s phalanx B (x ₄ , y ₄ , z ₄ ), and its expression is:

The body width X ₄ is measured by the Kinect depth camera, and its expression is: X ₄ = |z _u -z _d |.

S06: Carry out correlation analysis between the weight of live pigs and their body size data, and use the least square method of multi-factor linear regression to conduct multiple linear regression analysis, and calculate the body length X ₁ , body height X ₂ , body oblique length X ₃ and body width X ₄ The body size data was used as the multivariate independent variable of the least square method, and the weight of the live pig was analyzed as the independent variable at the same time. The following model can be derived:

Y＝α ₀ +α ₁ X ₁ +α ₂ X ₂ +…+α _n X _n +ε

Among them, Y is the weight of the pig, a ₀ , a ₁ , a ₂ ...a _n and ε are coefficients corresponding to different types of pigs, and x ₁ , x ₂ ... x _n are the body size data of the pig.

Finally, it should be noted that: the above is only a preferred embodiment of the present invention, and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, for those skilled in the art, it still The technical solutions recorded in the foregoing embodiments may be modified, or some technical features thereof may be equivalently replaced. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the claims of the present invention.

Claims (4)

1. A live pig body weight measurement method based on depth learning and depth camera, is characterized in that, comprises the following steps: Obtain the depth image of the pig; Converting the depth image into three-dimensional point cloud data; Preprocessing the 3D point cloud data to remove noise; Use the PyTorch deep learning framework to run the point cloud convolutional neural network PointNet++ model, put the noise-removed 3D point cloud data into the point cloud convolutional neural network PointNet++ model for deep learning, and remove the background 3D point cloud data; Project the 3D point cloud data after removing the background to the 3D coordinate system, find the minimum point of z value on each slice point cloud, merge each point with minimum z value into a new point column, and perform discrete points on the point column Remove, get the corresponding two-dimensional fitting line, and find the corresponding extreme point position, that is, get the corresponding feature point coordinates, so as to get the corresponding body size data; Taking the body size data of the live pig as the independent variable and the weight of the live pig as the dependent variable, the weight of the live pig is obtained according to the following formula: Y＝α ₀ +α ₁ X ₁ +α ₂ X ₂ +…+α _n X _n +ε Among them, Y is the weight of live pigs, a ₀ , a ₁ , a ₂ ... a _n and ε are coefficients corresponding to different types of live pigs, x ₁ , x ₂ ... x _n are the body size data of live pigs; The feature points include the ground plane, the top of the head of the pig, the first tail vertebra of the pig, the chinchilla bone of the pig and the phalanx of the pig, wherein the equation of the ground plane is ax+by+cz+d=0 Corresponding body size data, including body length X ₁ , body height X ₂ , body oblique length X ₃ and body width X ₄ ; Among them, the body length X ₁ is the linear distance from N(x ₁ , y ₁ , z ₁ ) to M(x ₂ , y ₂ , z ₂ ) of the pig’s head, and its expression is: X ₁ =|x ₁ -x ₂ |; Body height X ₂ is the straight-line distance from the pig's first tailbone M (x ₂ , y ₂ , z ₂ ) to the ground plane ax+by+cz+d=0, and its expression is: Body oblique length X ₃ is the straight-line distance from pig’s bun bone A (x ₃ , y ₃ , z ₃ ) to pig’s phalanx B (x ₄ , y ₄ , z ₄ ), and its expression is: The body width X ₄ is measured by the Kinect depth camera, and its expression is: X ₄ = |z _u -z _d |. 2. The pig body weight measurement method based on deep learning and depth camera according to claim 1, characterized in that: obtaining the depth image of the pig comprises: setting up the Kinect depth camera on the side of the feeding machine, for the still pigs when eating Capture the depth image, the distance range is Among them, a is the angle of view of the Kinect depth camera, L is the length of the pig eating area, and D is the distance between the Kinect depth camera and the pig. 3. the pig body weight measurement method based on depth learning and depth camera according to claim 2, is characterized in that: described depth image is converted into three-dimensional point cloud data and comprises: under Kinectfor Windows SDK2.0 software development environment, use its The built-in point cloud acquisition function converts the acquired depth image into 3D point cloud data. 4. The live pig body weight measurement method based on deep learning and depth camera according to claim 3, characterized in that: the three-dimensional point cloud data is preprocessed, and removing noise comprises: using a bilateral filter method to process the three-dimensional point cloud data set Perform filtering and noise reduction processing, the expression is: J ⁰ ＝IJ ^t+1 ＝f(J ^t ) Among them, J ⁰ is the initial image, J ^t is the result after t iterations, and f(J ^t ) is the filter.