CN111402320A - A deep learning-based fiber cross-section diameter detection method - Google Patents

Abstract

The invention provides a method for detecting the diameter of fiber cross-sections based on deep learning, which includes: creating a convolutional neural network model by using pre-training parameters; wherein, the convolutional neural network model is a Mask R-CNN model, and its convolution kernel is a different length. Equal rectangular convolution kernel; obtain the training set and verification set of the fiber image, train the convolutional neural network model through the training set and verification set of the fiber image, and obtain the trained convolutional neural network model; determine the fiber image to be detected , identify the fiber pictures to be detected according to the trained convolutional neural network model, and obtain the shape masks of each fiber cross-section in the fiber pictures; among them, the training, validation set and the fiber pictures to be identified are all fiber pictures with a size of 1024×1024; Based on the shape mask of the fiber section, parameters of each section mask profile are calculated; the parameters include the area, perimeter, and diameter of the section mask profile. The invention can automatically determine the fiber characteristics and accurately calculate the diameter of the fiber section.

Description

A deep learning-based fiber cross-section diameter detection method

technical field

The invention belongs to the technical field of fiber detection, and in particular relates to a method for detecting the diameter of a fiber cross-section based on deep learning.

Background technique

At present, the method for measuring the diameter of the fiber cross-section in the prior art is to cut the fiber bundle to be measured into small segments of millimeter level, and then use a dispersing device to disperse the fiber segments on a glass slide, and place them under a microscope to image them through a camera. Using the classic digital image method, the diameters of a large number of fiber segments in the image are measured separately. This process is commonly called longitudinal measurement in the textile field. A key problem here is that a large number of fiber segments are scattered on the glass slide, and there are a lot of crosses, bends and partial overlaps. Because the crossed fibers are divided into multiple segments, and these multiple segments are not necessarily attributed to the same fiber) and combined measurements (two combined together are counted as one), these are actually fake data. The data tested by such a system has a certain degree of randomness, especially for samples with large dispersion of fiber diameters, the error will be large. On the other hand, the average number of measurable fibers in a field of view is usually about 10 in the measurement of dispersion fiber acquisition. To achieve a large-capacity test of thousands of fibers, it needs to correspond to the mobile acquisition of hundreds of fields of view, and the efficiency is low. In addition, for fibers whose cross-sectional shape is not close to a circle, the measured diameter also has a certain deviation.

Therefore, how to accurately and efficiently detect the diameters of a large number of fibers is a problem that needs to be solved by those skilled in the art.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a method for detecting the diameter of a fiber cross-section based on deep learning in view of the deficiencies of the prior art, using the Mask-RCN model to automatically determine the fiber characteristics and accurately calculate the diameter (area) of the fiber cross-section.

An embodiment of the present invention provides a method for detecting the diameter of a fiber cross-section based on deep learning, comprising the following steps:

Step 1, using pre-training parameters to create a convolutional neural network model; wherein, the convolutional neural network model is a Mask R-CNN model, and its convolution kernel is a rectangular convolution kernel of unequal length;

Step 2, obtaining the training set and the verification set of the fiber picture, and training the convolutional neural network model through the training set and the verification set of the fiber picture to obtain the trained convolutional neural network model;

Step 3: Determine the fiber picture to be detected, identify the fiber picture to be detected according to the trained convolutional neural network model, and obtain the shape mask of each fiber cross section in the fiber picture; wherein, the training and verification sets and the fiber pictures to be identified are all fiber pictures in the size of 1024×1024;

Step 4, based on the shape mask of the fiber cross-section, calculate the parameters of each cross-sectional mask contour; the parameters include the area, perimeter, and diameter of the cross-sectional mask contour.

Compared with the prior art, the beneficial effects of the present invention are:

1. No matter what the shape of the fiber cross-section is, the cross-sectional area can be accurately obtained, so as to calculate the accurate equivalent diameter.

2. There is no cross-over problem of fibers in the field of view in cross-section mode, so there is no problem of repeated measurement.

3. The number and density of fibers in the field of view are high. Usually, there are hundreds of fibers in a field of view for measurement, and the efficiency is greatly improved compared with the longitudinal method.

4. Based on the AI neural network algorithm, the problem that the adjacent sections are difficult to be accurately divided by the traditional digital image algorithm is well solved, which lays a foundation for the section calculation.

Description of drawings

Fig. 1 is the flow chart of the fiber section diameter detection method based on deep learning of the present invention;

FIG. 2 is a simplified diagram of the Mask R-CNN structure of the present invention.

Detailed ways

The present invention will be described in detail below with reference to the various embodiments shown in the accompanying drawings, but it should be noted that these embodiments do not limit the present invention. Equivalent transformations or substitutions all fall within the protection scope of the present invention.

Referring to FIG. 1 , this embodiment provides a method for detecting the diameter of a fiber cross-section based on deep learning, including:

Step S1, using the pre-training parameters on ImageNet to create a convolutional neural network model; wherein, the convolutional neural network model is a Mask R-CNN model, and its convolution kernel is a rectangular convolution kernel of unequal length;

Step S2, obtaining the training set and the verification set of the fiber picture, and training the convolutional neural network model through the training set and the verification set of the fiber picture, to obtain the trained convolutional neural network model;

Step S3: Determine the fiber picture to be detected, identify the fiber picture to be detected according to the trained convolutional neural network model, and obtain the shape mask of each fiber cross section in the fiber picture; wherein, the training and verification sets and the fiber pictures to be identified are all fiber pictures in the size of 1024×1024;

Step S4, based on the shape mask of the fiber cross-section, calculate the parameters of each cross-sectional mask contour; the parameters include the area, perimeter, and diameter of the cross-sectional mask contour.

The Mask R-CNN model consists of 5 parts, namely the feature extraction network, the feature combination network, the region submission network (RPN), the region feature aggregation network (ROIAlign) and the functional network, as shown in Figure 2.

The feature extraction network is the backbone network of the deep neural network and is the most computationally intensive part of the entire model. According to different application requirements, different feature extraction networks can be selected. Taking ResNet50 as an example, take the 4 feature maps output by its 4 ResidualBlocks, denoted as C ₂ , C _3, C _{4, and} C _5, which represent the features of different depths of the image respectively. The function of the feature combination network is to recombine the features of different depths, and the newly generated feature map contains the feature information of different depths at the same time. Mask R-CNN uses FPN to combine feature maps C ₂ , C _{3 ,} C _{4 ,} C ₅ into new feature maps P ₂ , P ₃ , P ₄ , P ₅ , P ₆ . For i=5, 4, 3 , 2, U ₆ =0, the feature combination processing process is shown in formula (1): P' _i

Among them: conv represents the convolution operation, sum represents the element-by-element bitwise sum operation, upsample represents the upsampling operation that doubles the feature length and width, and pooling represents the maximum pooling operation with a stride of 2.

The role of the region submission network is to use the feature map to calculate the candidate frame that can represent the position of the object in the image, and use the anchor technology to complete the region submission function. RPN in Mask R-CNN is based on the five feature maps P ₂ , P ₃ , P ₄ , P ₅ , and P ₆ , and regresses each feature vector in each feature map to obtain a 5n-dimensional vector to describe Correction values of n anchors, each of which includes △x, △y, △h, △w, and p. where p is the foreground and background confidence. Anchor is a pre-set Box. For each point in P ₂ , P ₃ , P ₄ , P ₅ , and P ₆ , multiple Anchors will be preset with different widths and heights centered on their coordinates. Then, the center, width and height of each Anchor are corrected according to the correction value obtained by the regression of the RPN network, so as to obtain a new Box. Equation (2) gives the calculation process of Anchor correction.

Among them: x and y represent the coordinates of the center of the Anchor, and w and h represent the width and height of the Anchor, respectively.

When the Anchor correction is completed, a large number of Boxes will be generated. At this time, according to the p value of each Box, a more accurate candidate box can be transitioned by using Non-Maximum Suppression (NMS).

After obtaining the candidate frame, the traditional method will cut out the corresponding area from the original image according to the position of each candidate frame, and then classify and segment the area. However, considering that the input required by these functional networks comes from the feature maps P ₂ , P ₃ , P ₄ , P ₅ , P ₆ , the ROIAlign algorithm is used to directly crop the features of the corresponding positions of the candidate frames from the feature maps, And add bilinear interpolation and pooling to transform the features into a unified ruler. The ROIAlign algorithm can be regarded as a pooling process that introduces bilinear interpolation, which turns the original discrete pooling into continuous.

After obtaining the features of the same size in the corresponding area of each candidate frame, they are used as the input of some functional networks called heads to participate in the subsequent calculation. For the classification head, a fixed combination of the fully connected layer and the Softmax layer is used. For the two-stage correction of the candidate frame, MaskRCNN regresses each category to obtain a correction value of a 5-dimensional vector, and the correction process is consistent with formula (2).

The Mask-RCNN network in this embodiment has two main parts.

The first is the Region Proposal Network, which generates around 2000 region proposals per image. During training, each of these proposals (ROIs) goes through the second part, the object detection and mask prediction network. Since the mask prediction branch runs in parallel with the label and box prediction branches, for each given ROI, the network predicts masks belonging to all classes.

The second is that during inference, region proposals undergo non-maximal suppression, and the mask prediction branch only processes the top 1000 detection boxes. Therefore, with 1000 ROIs and 2 object categories, the mask prediction part of the network will output a 4D tensor of size 1000 × 2 × 28 × 28, where each mask is of size 28 × 28.

It should be noted that since the fibers to be detected in this solution are mainly wool wool as an example, there may be certain differences in other fibers using the present invention.

In this scheme, the size of the fiber picture is set to 1024×1024. Therefore, the size of the fiber picture in the training set, the fiber picture in the verification set, and the fiber picture to be identified in this scheme are all 1024×1024, so as to ensure that the length of the picture can be Divisible by 32. Since the image is large, and there may be cashmere sections in every part of the fiber image, the Anchor is set to [64, 128, 256, 512, 1024], and the maximum number that can be detected per fiber image is set to 1000. Moreover, during training, since the woolen foreground on the fiber image exceeds 90%, the positive and negative samples are seriously unbalanced, and the loss function needs to be adjusted on the model. Therefore, the Focal Loss function is used to reduce the imbalance caused by the positive and negative samples. Loss. In addition, the model uses the Adam algorithm for gradient calculation to ensure that the gradient optimal solution is found.

The specific process is: the fiber image is collected by a special device, Mask-RCNN splits the network through FCN, takes the 14×14 ROIAlign output feature map as input, and passes through four 3×3 convolutional layers, keeping the size of 14×14 unchanged. Then, the output size is upsampled to 28×28 through a 2×2 deconvolution layer, and finally a 28×28 output is obtained through a 1×1 convolution layer and a sigmoid activation layer. The points represent the foreground and background confidence of the shape of a certain category of the candidate box. Finally, an object shape mask is obtained with a confidence threshold of 0.5. For the displayed image, the network needs to detect multiple objects. For each object, it outputs an array containing the predicted class score (indicating the probability that the object belongs to the predicted class), the left, top, right and bottom positions of the detected object's bounding box in the box. The class ids from this array are used to extract the corresponding mask from the output of the mask prediction branch.

For the displayed image, the network needs to detect multiple objects. For each object, it outputs an array containing the predicted class score (indicating the probability that the object belongs to the predicted class), the left, top, right and bottom positions of the detected object's bounding box in the box. The class ids from this array are used to extract the corresponding mask from the output of the mask prediction branch.

In this scheme MaskRCNN, based on the ResNet50 model structure, there are more than 50 operation blocks composed of conv and BatchNorm. After the network model training is completed, the network model that is only used for forward operation has some redundant calculation steps, which can be completed in advance by means of parameter merging. In addition, the derivative of the model, the number of parameters and the value of the parameters are adjusted, and a more suitable model for the detection of fiber cross-section diameter (area) is proposed.

The input image of the model selected in this scheme is a 1024×1024 3-channel color image, and the output includes categories, regressions and their corresponding mask values, and then calculates the cross-sectional diameter (area) of wool in the fiber image based on these values. . Finally, MaskRCNN uses FPN technology, and each input image will generate multiple feature maps with different depths and scales after the feature combination of FPN. MaskRCNN will select one of the feature maps according to the size of the candidate frame for ROIAlign operation. The selected The principle is to select a feature map with a larger depth for a candidate frame with a larger area.

The input of the system constructed in this example is the cross-sectional image of the fluff fiber under the optical microscope, and the output is the shape mask of each fiber cross-section in the image. Then, according to the classical image algorithm, the area, perimeter, and diameter of the outline of each cross-sectional mask are calculated. and other parameters. The image uses Mask R-CNN for forward calculation to obtain the specific information of the main image in the image, and then uses the fiber cross-section mask result output by the model to calculate the diameter of the image fiber, without the need for personnel to participate in each link. Compared with the traditional section calculation method, the accuracy and speed are greatly improved. This solution applies deep learning to the field of fiber cross-section diameter (area) testing and detection, which makes the fiber cross-sectional diameter (area) detection simple. It no longer needs to manually set and extract a large number of features, but can learn fiber features independently, which can achieve high efficiency. And analysis operation, improve efficiency, but also greatly reduce the operation cost.

It will be apparent to those skilled in the art that the present invention is not limited to the details of the above-described exemplary embodiments, but that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments are to be regarded in all respects as illustrative and not restrictive, and the scope of the invention is to be defined by the appended claims rather than the foregoing description, which are therefore intended to fall within the scope of the claims. All changes within the meaning and scope of the equivalents of , are included in the present invention.

Claims (1)

1. A fiber section diameter detection method based on deep learning is characterized by comprising the following steps:

step 1, creating a convolutional neural network model by using pre-training parameters; the convolutional neural network model is a MaskR-CNN model, and the convolutional kernel of the model is a rectangular convolutional kernel with different lengths;

step 2, acquiring a training set and a verification set of the fiber picture, and training the convolutional neural network model through the training set and the verification set of the fiber picture to obtain a trained convolutional neural network model;

step 3, determining a fiber picture to be detected, and identifying the fiber picture to be detected according to the trained convolutional neural network model to obtain a shape mask of each fiber section in the fiber picture, wherein the training and verifying set and the fiber picture to be identified are 1024 × 1024 fiber pictures;

step 4, calculating the parameters of the mask contour of each cross section based on the shape mask of the fiber cross section; the parameters include area, perimeter, diameter of the cross-section mask outline.

Images