CN117036427A - Industrial printed matter image registration method and device based on lightweight network - Google Patents

Abstract

The invention discloses an industrial printed matter image registration method based on a lightweight network, which comprises the following steps: s1, image acquisition and data set construction, namely acquiring the front side and the back side of a printed matter image through a CCD industrial camera, and constructing a training data set by combining a multispectral image; s2, constructing a DHSiam network, wherein two parallel feature extraction networks are covered; s3, providing an attention isomerism mechanism, covering a position regression unit and a light weight unit, enhancing data expression and reducing model parameters; s4, constructing a cross entropy loss function with additional margin, and enhancing network measurement performance; according to the invention, 8 GPU parallel hardware platforms are built, the AMC-Loss resolution image similarity is only utilized, the attention heterogeneous mechanism is utilized to extract important information of different scale feature images, meanwhile, model parameters are adjusted, the performance of an algorithm at a mobile end is enhanced, and the registration capability of the model on different visual angles and different deformation printed matter images is comprehensively improved.

Description

A lightweight network-based image registration method and device for industrial printed matter

Technical field

The present invention relates to the technical field of target image registration, specifically a lightweight network-based industrial printed matter image registration method and device.

Background technique

In many computer vision tasks, how to find matching images from a large number of data sets plays a decisive role, such as CT image detection, multi-view reconstruction, image retrieval, and image-based localization. As a popular technology, image registration obtains specified image information in a specific way, determines the similarity of image pairs under different environments (illumination, shooting angle, shooting position, etc.), and achieves geometric alignment. Currently, image registration can be divided into traditional registration methods and deep learning-based image registration methods. Traditional registration methods include traditional rigid registration methods and traditional non-rigid registration algorithms. Traditional rigid registration algorithms, such as SIFT features and AKAZE features, use affine transformation to achieve registration based on the mapping relationship between features between source images. The features between image sources are relatively stable, and this type of registration algorithm has certain scale and rotation invariance. However, due to the limited number of low-level image features and manually designed features, it is difficult to accurately represent the true deformation between images, and cannot be used when complex deformations occur in the source image; traditional non-rigid registration algorithms, such as DEMONS registration based on optical flow field A quasi-algorithm, through iterative operations, optimizes the displacement field between source images until the algorithm converges. This algorithm can obtain good registration accuracy, but the algorithm converges slowly and registration takes a long time, which cannot meet the requirements of industrial real-time detection.

With the rapid development of visual imaging technology, data-driven deep learning has been introduced into real life and has been successfully applied in various visual fields, such as pixel-based high-resolution classification, image segmentation, high-level semantic feature extraction, change detection, etc. Deep learning is also widely used in the field of feature region registration. For example, the MatchNet network fuses image region features through a fully connected layer to achieve similarity measurement; the DeepCompare method implements similarity measurement for grayscale images; the L2-Net network structure implements registration.

In recent years, image registration algorithm models based on deep learning have become larger and more complex, severely restricting their application in the industrial field. In the consumer electronics industry, printed matter such as instructions and packaging boxes must undergo strict screening before leaving the factory to ensure that there are no defects in appearance. The general process of detecting defects in paper printed matter is to compare the image to be tested with the template image to find the differences and then determine whether the product has defects. Difference processing is a method to quickly extract the difference between two images. The only information retained in the difference map is the difference artifacts and defects. However, paper printed matter is a non-rigid material and is easily deformed by external forces. Even if it is fixed by a clamp, it is difficult to obtain the exact same shape as the template image. Traditional registration methods are difficult to effectively solve the problem of non-rigid registration of paper printed images, and cannot fundamentally reduce misjudgments caused by defect detection. And most of its image registration algorithms use Euclidean distance loss for feature learning. This type of loss introduces more noise information of the output features and the actual registration information accounts for a low proportion, seriously restricting the registration efficiency, causing a high cost of registration, and the overall model convergence. The speed is slow and the accuracy is low; for the existing image registration system, it is necessary to determine whether a given patch pair is registered. The registration accuracy for image sets with larger size and higher resolution is obviously insufficient, and the computational burden is heavy in practical applications. Larger and less transplantable.

Contents of the invention

The purpose of the present invention is to provide a lightweight network-based industrial printed matter image registration method and device. The present invention builds an 8-GPU parallel hardware platform and optimizes the cuDNN library function. Inspired by the successful application of deep convolutional neural networks in large-scale image classification and feature extraction tasks, we integrate traditional registration methods with current convolutional neural network architectures and design a registration method using GPU technology, combined with large-scale The Margin Cross Entropy Loss function (Large MarginCosine Loss, LMCL) proposes an additional angular margin cross entropy loss (Additive Margin CrossEntropy Loss, AMC-Loss), which only uses the modulus of the feature vector to determine the registration attributes, using To overcome or alleviate the above-mentioned defects in the prior art.

2. In order to solve the above technical problems, the present invention provides the following technical solution: a lightweight network-based industrial print image registration method, which is characterized by including the following specific steps:

S1: Image collection and data set construction. Use CCD industrial cameras to collect images of the front and back sides of printed matter on the production line, and combine multi-spectral images to build a training data set;

S2: Network construction and optimization, relying on Pseudo-Siamese to build a DHSiam network, covering two parallel feature extraction networks;

S3: Attention heterogeneous mechanism, covering two modules, namely position regression unit and lightweight unit;

S4: AMC-Loss loss construction, constructing a cross-entropy loss function with additional margin, covering cross-entropy loss design and Margin edge value optimization.

According to the above technical solution, the size of the edge value in the loss function is quantitatively adjusted through parameter values, and a specific m quantitative adjustment loss method is further designed.

The attention heterogeneous mechanism is embedded in the residual module architecture of each DHSiam twin network to quantitatively adjust the position information of the feature vector.

According to the above technical solution, a DHSiam twin network model is proposed in step S2;

The DHSiam twin network model is based on a parallel weight-sharing network structure and consists of two different branches. Each branch is connected to the corresponding deep neural network. The original image branch is connected to the ResNet50 structure, and the branch to be registered is connected to the ResNet101 structure;

Pass the original image Ix through the ResNet50 branch, and the image Iy to be registered through the ResNet101 branch, and output the corresponding 7×7 feature maps F(Ix) and F(Iy);

Each branch structure includes a convolutional layer, a nonlinear activation unit and a normalization layer. During training, the image pairs Ix and Iy are input to different branches. The best feature representation of the image pair is input and output in the form of a feature vector. ;

Global depth convolution is used as the measurement layer to output the corresponding 1-dimensional feature vectors G(Ix) and G(Iy).

According to the above technical solution, in step S2, the convolution kernel of the traditional residual network input Conv1 layer is replaced with multiple 3×3 serial convolution combinations.

According to the above technical solution, in step S4, a cross-entropy loss function with additional margin is constructed to realize image pair attribute determination;

The loss function retains the advantage of expanding the differences between classes, and reduces the sensitivity to different signals, paying more attention to the difference in vector module values;

The loss function of modular value design is as follows:

During the loss calculation process, it is necessary to ensure that only the modulus of the feature vector is obtained, so the L2 algorithm is used to normalize the weight of a single feature. L is a binary label, which is used to determine whether the input Ix, Iy image pair is a positive sample (l=1) or a negative sample. sample(l=0);

According to the above technical solution, Margin information is introduced in step S4 to enhance the distinguishability between feature vectors of the DHSiam network;

An additional edge penalty m is added between the original image and the output feature vectors C _x and C _y of the image to be registered, which simultaneously enhances the intra-class tightness and inter-class differences;

The cross-entropy loss function with additional margin is defined as follows:

In the formula, N is the training sample number, p _j is the modulus calibration s=1 of the j-th feature vector output by the twin network; the loss uses additive margin addition to weaken the marginal penalty.

According to the above technical solution, an attention heterogeneous mechanism is added in step S2;

In the attention heterogeneous mechanism, the feature rotation problem is first solved through the position regression unit. Based on this algorithm, a heterogeneous attention mechanism is added to model the importance of the channel, reduce the amount of model parameters, and output refined feature maps.

According to the above technical solution, in step S3, the position regression unit is first embedded in the ResNet residual block, merged along the channel axis to generate an effective feature descriptor of R ^h×w×2 , and then connected to cascade convolution, using the convolution layer Generate spatial mapping area of a single image

The position regression unit uses two channel pooling operations to aggregate the channel information of a feature channel and generate two two-dimensional mappings and/> Represents the maximum fusion feature and average fusion feature at the channel level respectively, and is then connected to a standard convolution layer to generate/> The formula is as follows:

In the formula, Represents the feature value of the i-th image input in a single batch M/> and the eigenvalues of the i-1th picture The included angle is the rotation angle of the i-position feature matrix. The angle-corrected feature information is output through the rotation layer, and padding is added for each rotated area to avoid feature loss.

According to the above technical solution, the lightweight unit in step S3 consists of a feature segmentation module and a feature mapping module. A new hyperparameter K is introduced, which specifies the number of splits of feature channels. The obtained hyperparameter is called a base array. (K=32 in this article). On this basis, a new hyperparameter r is introduced in lightweighting. This parameter limits the number of splits of channel groups in Unit. Therefore, the total number of channels is c=Kr, and the input will be The input block c is divided into F _i = {F ₁ , F ₂ ,...F _c } along the channel dimension, where i∈{1,2,...c}, followed by a global maximum pooling operation to aggregate feature channels Information, generate feature descriptor U ^r (F)∈R ^H×W×c/N/r , where r∈1,2,...N, where the number of feature vector channels in each group after N grouping, H and W are the length and width of the channel vector. At this time, H=W after global pooling. Through global average pooling across channel dimensions, global context information s ^r ∈R ^c/K/ with embedded channel statistics can be collected. ^r , the formula is as follows

Each feature map channel is generated by a weighted combination of segmentations. The calculation formula for the c-th channel is:

where a _j (c) represents the soft channel allocation weight:

which maps is the weight ratio of the s ^r channel position represented by context information;

Dynamic convolution kernels are connected in parallel, which contain different convolution kernel sizes. Randomly reducing the convolution kernel size will cause insufficient extraction of some important features or loss of features; introduce soft channel allocation weight a through r-Softmax, according to The weight factor of each channel adjusts the convolution and size of each channel. The specific formula is as follows:

Among them, s is the threshold parameter given according to the actual detection requirements, that is, s-1 1×1 convolution kernel is added to every other K×K convolution kernel. According to the weight of each channel obtained by the formula, the weight proportion is smaller. The size of the convolution kernel on the channel is reduced from 3×3 to 1×1, by maintaining the 3×3 convolution kernel for important channels.

According to the above technical solution, the position regression unit and the lightweight unit are arranged in series in step S3, and the positions and quantities of the two mechanisms in the entire model are adjusted according to the sample set.

Compared with the existing technology, the beneficial effects achieved by the present invention are: the present invention uses cross entropy loss to convert the Euclidean space into a modulus space, improves the registration accuracy, and the matching cost is directly determined by the modulus of the two vectors in the embedded space. value calculation. The size of the edge value can be quantitatively adjusted through the parameter m. The present invention further derives a specific m to quantitatively adjust the loss method. In addition, an attention heterogeneous mechanism is proposed in the DHSiam architecture to adjust the position information of feature vectors, reduce mismatching caused by image rotation, and reduce the amount of model parameters. Cross-entropy loss and attention heterogeneous mechanisms have shown obvious advantages in a large number of image matching analyses, and experimental results have verified the portability, robustness, and effectiveness of the method.

Description of the drawings

The drawings are used to provide a further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. In the attached picture:

Figure 1 is a flow chart of the lightweight network-based industrial print image registration method of the present invention;

Figure 2 is a schematic structural diagram of the image acquisition component in the embodiment of the present invention;

Figure 3 is a schematic structural diagram of a separate collection device for an industrial camera in an embodiment of the present invention;

Figure 4 is a schematic diagram of the deep learning sub-network in the present invention;

Figure 5 is a schematic diagram of the position regression unit network of the attention heterogeneous mechanism in the present invention;

Figure 6 is a schematic diagram of the lightweight unit network of the attention heterogeneous mechanism in the present invention;

Figure 7 is a schematic diagram of a specific form of the lightweight unit in the present invention;

Figure 8 is a schematic diagram of an example of a printed image sample in the present invention;

In the picture: 1. Industrial camera; 2. Fill light; 3. Position sensor; 4. Grooved conveyor belt; 5. Electrical connection interface; 6. Vertical slide rail; 7. Horizontal slide rail; 8. Print card board.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

Please refer to Figures 1-8. The present invention provides a technical solution: a deep learning-based industrial print image registration method, including:

S1: Image collection and data set construction. As shown in Figure 2, a CCD industrial camera is used to collect images of the front and back sides of printed matter on the production line, and a training data set is constructed using multispectral images.

Network construction and optimization, as shown in Figure 4, relies on Pseudo-Siamese (non-weight sharing twin network) to build a DHSiam (Dynamic and Heterogeneous Siamese) network, covering two parallel feature extraction networks (ResNet residual network);

S3: Attention heterogeneous mechanism, as shown in Figure 5 and Figure 6, covers position regression units and lightweight units, enhances data expression, and reduces model parameters;

S4: AMC-Loss loss construction, constructing an additional margin cross-entropy loss function (AMC-Loss), covering cross-entropy loss design and margin edge value optimization, and enhancing network measurement performance.

The embodiments of the present invention can effectively improve the robustness and accuracy of the system during the registration process, and have strong adaptability to mobile devices in the industrial field; the present invention uses cross-entropy loss to transform the Euclidean space Modular space is formed to improve the registration accuracy, and the matching cost is directly calculated by the modulus of the two vectors in the embedding space;

The size of the edge value in the loss function in the embodiment of the present invention can be quantitatively adjusted through the parameter m. The present invention further designs a specific m to quantitatively adjust the loss method.

The attention heterogeneous mechanism of the embodiment of the present invention is embedded in each residual module structure of the DHSiam network to quantitatively adjust the position information of the feature vector, reduce mismatching caused by image rotation, and solve the problem of insufficient algorithm transplantation due to an oversized training model.

In the embodiment of the present invention, the registration system has verified the portability, robustness and effectiveness of the method through a large number of experiments.

In the embodiment of the present invention, the application environment of the registration system is not limited to industrial printed images, but is also applicable to medical CT images, remote sensing satellite images, three-dimensional modeling, industrial defect detection and other fields.

In the embodiment of the present invention, the print image acquisition device is shown in Figure 2. The device includes a groove conveyor belt component and image acquisition components located on both sides of the groove. The image acquisition component includes an industrial camera 1, a fill light 2, and a position sensor 3. , vertical slide rail 6 and horizontal slide rail 7, the position of the industrial camera 1 is adjustable; the groove conveyor belt component includes the groove conveyor belt 4 and the electrical connection port 5; the distance between the image acquisition device and the groove conveyor belt 4 is adjustable, and the fill light 2 is placed Outside the lens of the industrial camera 1, the industrial camera 1 is connected to the mobile computing platform;

Specifically, the fill light 2 is placed outside the lens of the industrial camera 1 to ensure that the collected image of the LCD screen to be tested is clear and not affected by external light;

Specifically, the image acquisition component includes a vertical slide rail 6 and a horizontal slide rail 7. The industrial camera 1 is installed on the slider of the vertical slide rail 6; the vertical slide rail 6 is connected to the horizontal slide rail 7; the horizontal slide rail 7 is connected to the industrial control electromechanical device. connect;

In the embodiment of the present invention, a position sensor 3 is installed on the chute of the vertical slide rail 6. The position sensor 3 is connected to the industrial control electromechanical device. The distance between the industrial camera 1 and the groove conveyor belt 4 is adjusted according to the detection information of the position sensor 3. According to the position sensor 3. Detection information adjusts the height of industrial camera 1;

In the embodiment of the present invention, an industrial camera separate collection device is shown in Figure 3, which includes: an industrial camera 1, a fill light 2, and a printed matter card board 8. Specifically, a simulated background can be added to the printed matter card board 8, and industrial printed matter can be pasted on the printed matter card board 8 to achieve front and back collection.

In the embodiment of the present invention, the printed matter card board 8 in Figure 3 can be placed vertically in the grooved conveyor belt 4 in Figure 2 to achieve simultaneous collection of front and back viewing angles. The specific printed matter card board 8 can simulate various collection environments and add collection backgrounds.

DHSiam's pair-based non-weight sharing network structure consists of two different branches, each branch is connected to the corresponding deep neural network, the original image branch is connected to the ResNet50 structure, and the branch to be registered is connected to the ResNet101 structure to ensure full extraction of image information;

The network passes the original image Ix through the ResNet50 branch, and the image Iy to be registered passes through the ResNet101 branch, and outputs the corresponding feature maps F(Ix) and F(Iy);

In the embodiment of the present invention, each branch structure of the network includes a convolution layer, a nonlinear activation unit and a normalization layer. During training, image pairs Ix and Iy are input into different branches, and the best feature representation of the input image pair is output in the form of a feature vector.

Furthermore, the registration network includes a measurement layer to achieve feature dimensionality reduction. The network uses Global Deep Learning Convolutional (GDC) as the measurement layer to output the corresponding 1×1 feature vectors G(Ix) and G(Iy);

In the embodiment of the present invention, the convolution kernel of the traditional residual network input Conv1 layer is replaced by multiple 3×3 serial convolution combinations to avoid larger convolution kernels bringing more parameters and reduce the calculation of cuDNN library functions. quantity.

In step S4, in order to ensure the robustness and stability of the system, an additional margin cross entropy loss function (Additive Margin Cross Entropy Loss, AMC-Loss) is constructed to implement image pair attribute determination;

The loss function retains the advantage of expanding the differences between classes and pays more attention to the difference in vector module values.

The present invention designs the loss function based on the modulus value as follows:

During the loss calculation process, it is necessary to ensure that only the modulus value of the feature vector is obtained, so the present invention uses the L2 algorithm to normalize the weight of a single feature. l is a binary label, used to determine whether the input Ix, Iy image pair is a positive or negative sample;

The present invention designs the loss function based on cosine similarity as follows:

This invention adds an additional edge penalty m between the original image and the output feature vectors C _x and C _y of the image to be registered, while enhancing the intra-class tightness and inter-class differences;

The additional margin cross entropy loss function (Additive Margin Cross Entropy Loss, AMC-Loss) is defined as follows:

In the formula, N is the training sample number, p _j is the module value calibration s=1 of the j-th feature vector output by the twin network. The loss uses additive margin addition to weaken the marginal penalty and avoid the problem of increased difficulty in model training caused by the multiplicative marginal penalty and the complex double-angle formula.

During the design process of the registration system, the rotation of the image to be registered brings great uncertainty to the registration. How to effectively avoid the impact of image rotation on feature extraction determines the algorithm performance.

The embodiment of the present invention will add an attention heterogeneous mechanism based on the DHSiam network. This mechanism includes two sub-units: a position regression unit and a lightweight unit. First, the feature rotation problem is solved through the position regression unit. On this basis, heterogeneous lightweight units are added to reduce the number of model parameters and enhance its deployment on the mobile terminal.

In the embodiment of the present invention, given a set of pictures, the registration picture and the picture to be registered are different in proportion, angle and field of view. Since convolution uses local receptive fields, the corresponding features may have some angular differences. These differences will introduce intra-class inconsistencies and affect the registration accuracy. In order to solve this problem, the present invention proposes a position regression unit to explore and learn angle information between images, thereby avoiding misregistration caused by image rotation. This method can adaptively aggregate feature context information and improve the feature expression ability of image registration.

To avoid registration differences caused by image rotation, obtaining the angular information of features is key. As shown in Figure 5, the position regression unit is designed to map the global context information to the features generated by the extended residual network, and rotate the corresponding angle to obtain better feature expression. In order to obtain angle information, the present invention first adds a pooling operation along the channel axis based on the ResNet residual block, merges them to generate an effective feature descriptor of R ^h×w×2 , and performs pooling along the channel axis. The operation can effectively highlight location information. Then connect the cascade convolution, and use the convolution layer to generate the spatial mapping area of a single picture. Finally, the rotation angle is obtained through normalization operation and inner product operation.

The embodiment of the present invention uses two channel pooling operations to aggregate the channel information of a feature channel and generate two two-dimensional mappings. and/> represent the maximum fusion feature and the average fusion feature at the channel level respectively. Then concatenated with a standard convolutional layer to generate/> This layer uses a 5×5 convolution kernel to obtain more spatial information with a larger receptive field. Then perform normalization and inner product operations, the formula is as follows:

In the formula, Represents the feature value of the i-th image input in a single batch M/> and the eigenvalues of the i-1th picture The included angle is the rotation angle of the i-position feature matrix, and then the angle-corrected feature information is output through the rotation layer, and padding is added for each rotated area to avoid feature loss caused by rotation.

The lightweight unit in the embodiment of the present invention is a feature calculation unit, which is composed of a feature segmentation module and a feature mapping module. Figure 6 and Figure 7 describe the specific form of the lightweight unit.

In Figure 6, the lightweight unit in step S3 consists of a feature segmentation module and a feature mapping module. The present invention introduces a new hyperparameter K, which specifies the number of splits of feature channels. In this invention, the obtained hyperparameters are called base arrays (K=32 in this invention).

In further quantification, the present invention introduces a new hyperparameter r, which limits the number of splits of channel groups in Unit, so the total number of channels is c=Kr. The input block c is divided into F _i = {F ₁ , F ₂ ,...F _c } along the channel dimension, where i∈{1,2,...c}, followed by a global maximum pooling operation. To aggregate the feature channel information, generate the feature descriptor U ^r (F)∈R ^H×W×c/N/r where r∈1,2,...N where the number of feature vector channels in each group after N grouping, H and W are the length and width of the channel vector. At this time, H=W after global pooling. Through global average pooling across channel dimensions, global context information with embedded channel statistics s ^r ∈ R ^c/ can be collected. ^K/r , the formula is as follows:

Each feature map channel is generated by a weighted combination of segmentations. The calculation formula for the c-th channel is:

where a _j (c) represents the soft channel allocation weight:

which maps is the weight ratio of the s ^r channel position represented by context information.

In Figure 7, a dynamic convolution kernel is connected, which contains different convolution kernel sizes. However, randomly reducing the convolution kernel size will cause insufficient extraction of some important features or loss of features. In order to solve this type of problem, this paper The invention introduces soft channel allocation weight a through r-Softmax, and adjusts the convolution sum size of each channel according to the weight factor of each channel. The specific formula is as follows:

Among them, s is the threshold parameter given according to the actual detection requirements (that is, adding s-1 1×1 convolution kernels to every other K×K convolution kernel). According to the obtained weight of each channel, the size of the convolution kernel on the channel with a smaller weight is reduced from 3×3 to 1×1. By maintaining the 3×3 convolution kernel for the important channels, the present invention ensures the convolution The feature information extracted by the kernel is sufficient.

Compared with traditional isomorphic convolution, the lightweight module of the present invention replaces some of the larger convolution kernels, reducing the amount of model parameters in this way. Moreover, the model of the present invention uses the attention mechanism to dynamically adjust, using larger convolution kernels. The convolution kernel extracts important information, and a smaller convolution kernel is used to extract non-important information. Let the entire model ensure that the extraction accuracy remains unchanged on the basis of reducing the number of parameters.

In the embodiment of the present invention, in order to make full use of context information, the position regression unit and the lightweight unit are aggregated. Specifically, the present invention transforms feature information through pooling layers, and accumulates different weights of pixels or channels through different forms of pooling layers to model the importance of features, and finally outputs the final prediction through different residual blocks. picture. Through experiments, it can be found that the serial arrangement of the two mechanisms is better than the parallel arrangement. In the specific model, the position and quantity of the two mechanisms in the entire model will be adjusted to improve the computing efficiency of the GPU acceleration library cuDNN.

In order to ensure that the output of the library function corresponds to the input, it is necessary to ensure that the data structure and parameter amounts of adjacent library functions are completely corresponding and equal. This is similar to the transmission of the upper and lower layers of a convolutional neural network. The embodiment of the present invention uses the C++ object-oriented method to write the library functions as abstract classes to facilitate developers to call them.

In the embodiment of the present invention, the visual application using deep learning algorithms is a method supported by image sets. A complete and efficient industrial printed matter data set plays a decisive role in the deep learning task, and the existing public data sets There are obvious flaws in scale, diversity, and applicability. The training program of the present invention requires a supervised network composed of image pairs and geometric relationships, and the process usually requires a large number of richly representative data sets. To compile a robust data set, a wide variety of imaging equipment is required. Therefore, two parts of the data set are designed in the experiment of the present invention:

The first part uses a set of 3,000 pairs of printed images of different landmarks (including the Great Wall, the Forbidden City, the Summer Palace, the Potala Palace, Buckingham Palace, the Taj Mahal, etc.) and inlays the printed matter into the collection device shown in Figure 2;

In order to ensure the richness of the samples, 100,000 pairs of printed image samples were collected from the Internet, consisting of some printed book collections and public printed book collections.

In the stage where image preprocessing is necessary in the embodiment of the present invention, the training image is expanded using data enhancement technology. The outline of the method is shown in Figure 8, where the left end is the original image and the right end is the image after Unity pose transformation. The aligned images are cropped to remove useless areas, and the image data is standardized using methods such as histogram equalization, Z-score normalization, and downsampling. During the training phase, random rotation and horizontal flipping are used to increase the data set. Use the expanded training data to train the CNN to ensure the best performance of the CNN model.

The negative sample pairs selected in the embodiment of the present invention are about three to four times that of the positive sample pairs. Using this sample generation method can ensure the rich representativeness of the negative samples and try to cover all situations that occur in the actual registration process of the system. Afterwards, different degrees of data enhancement are performed on the data set. The rich training data makes the model training results more efficient and the obtained model more robust.

It should be understood that parts not elaborated in this specification belong to the prior art.

It should be understood that the application scope of the present invention is applicable to, but not limited to, the field of industrial print image processing. The above description of the preferred embodiments is relatively detailed and should not be considered as limiting the scope of patent protection of the present invention.

It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that these entities or operations are mutually exclusive. any such actual relationship or sequence exists between them. Furthermore, the terms "comprises," "comprises," or any other variations thereof are intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus that includes a list of elements includes not only those elements, but also those not expressly listed other elements, or elements inherent to the process, method, article or equipment.

Finally, it should be noted that the above are only preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still The technical solutions described in the foregoing embodiments may be modified, or some of the technical features may be equivalently replaced. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

Claims (10)

1. A lightweight network-based industrial print image registration method, which is characterized by including the following specific steps: S1: Image collection and data set construction. Use CCD industrial cameras to collect images of the front and back sides of printed matter on the production line, and combine multi-spectral images to build a training data set; S2: Network construction and optimization, relying on Pseudo-Siamese to build a DHSiam network, covering two parallel feature extraction networks; S3: Attention heterogeneous mechanism, covering two modules, namely position regression unit and lightweight unit; S4: AMC-Loss loss construction, constructing a cross-entropy loss function with additional margin, covering cross-entropy loss design and Margin edge value optimization. 2. A lightweight network-based industrial print image registration method according to claim 1, characterized in that the size of the edge value in the loss function is quantitatively adjusted through parameter values, and a specific m quantitative adjustment loss is further designed. method. The attention heterogeneous mechanism is embedded in the residual module architecture of each DHSiam twin network to quantitatively adjust the position information of the feature vector. 3. A lightweight network-based industrial print image registration method according to claim 1, characterized in that a DHSiam twin network model is proposed in step S2; The DHSiam twin network model is based on a parallel weight-sharing network structure and consists of two different branches. Each branch is connected to the corresponding deep neural network. The original image branch is connected to the ResNet50 structure, and the branch to be registered is connected to the ResNet101 structure; Pass the original image Ix through the ResNet50 branch, and the image Iy to be registered through the ResNet101 branch, and output the corresponding 7×7 feature maps F(Ix) and F(Iy); Each branch structure includes a convolutional layer, a nonlinear activation unit and a normalization layer. During training, the image pairs Ix and Iy are input to different branches. The best feature representation of the image pair is input and output in the form of a feature vector. ; Global depth convolution is used as the measurement layer to output the corresponding 1-dimensional feature vectors G(Ix) and G(Iy). 4. A lightweight network-based industrial print image registration method according to claim 3, characterized in that in step S2, the convolution kernel of the traditional residual network input Conv1 layer is replaced with multiple 3×3 serial convolution combination. 5. A lightweight network-based industrial print image registration method according to claim 1, characterized in that in step S4, a cross-entropy loss function with additional margin is constructed to realize image pair attribute determination; The loss function retains the advantage of expanding the differences between classes, and reduces the sensitivity to different signals, paying more attention to the difference in vector module values; The loss function of modular value design is as follows: During the loss calculation process, it is necessary to ensure that only the modulus value of the feature vector is obtained, so the L2 algorithm is used to normalize the weight of a single feature. L is a binary label, which is used to determine whether the input Ix, Iy image pair is a positive sample (l=1) or a negative sample. Sample(l=0). 6. A lightweight network-based industrial print image registration method according to claim 5, characterized in that Margin information is introduced in step S4 to enhance the distinguishability between DHSiam network feature vectors; An additional edge penalty m is added between the original image and the output feature vectors C _x and C _y of the image to be registered, which simultaneously enhances the intra-class tightness and inter-class differences; The cross-entropy loss function with additional margin is defined as follows: In the formula, N is the training sample number, p _j is the modulus calibration s=1 of the j-th feature vector output by the twin network; the loss uses additive margin addition to weaken the marginal penalty. 7. A lightweight network-based industrial print image registration method according to claim 1, characterized in that an attention heterogeneous mechanism is added in step S2; In the attention heterogeneous mechanism, the feature rotation problem is first solved through the position regression unit. Based on this algorithm, a heterogeneous attention mechanism is added to model the importance of the channel, reduce the amount of model parameters, and output refined feature maps. 8. A lightweight network-based industrial print image registration method according to claim 7, characterized in that in step S3, the position regression unit is first embedded in the ResNet residual block and merged along the channel axis to generate R Effective feature descriptor of ^h×w×2 , followed by cascade convolution, using the convolution layer to generate the spatial mapping area of a single image The position regression unit uses two channel pooling operations to aggregate the channel information of a feature channel and generate two two-dimensional mappings and/> Represents the maximum fusion feature and average fusion feature at the channel level respectively, and is then connected to a standard convolution layer to generate/> The formula is as follows: In the formula, Represents the feature value of the i-th image input in a single batch M/> and the eigenvalues of the i-1th picture/> The included angle is the rotation angle of the i-position feature matrix. The angle-corrected feature information is output through the rotation layer, and padding is added for each rotated area to avoid feature loss. 9. A lightweight network-based industrial print image registration method according to claim 7, characterized in that in step S3, the lightweight unit is composed of a feature segmentation module and a feature mapping module, and a new hyperparameter is introduced K, which stipulates the number of splits of feature channels, and calls the obtained hyperparameters the base array (K=32 in this article). On this basis, a new hyperparameter r is introduced in lightweighting. This parameter The number of splits of channel groups in Unit is limited, so the total number of channels is c=Kr. The input will be input block c, which is divided into F _i = {F ₁ , F ₂ ,...F _c } along the channel dimension, where i ∈{1,2,...c}, followed by a global maximum pooling operation to aggregate feature channel information and generate feature descriptor U ^r (F)∈R ^H×W×c/N/r , where r∈1 ,2,...N, where the number of channels of each group of feature vectors after N grouping, H and W are the length and width of the channel vector. At this time, after global pooling, H=W, through the global average across channel dimensions Pooling, global context information s ^r ∈R ^c/K/r with embedded channel statistics can be collected, the formula is as follows Each feature map channel is generated by a weighted combination of segmentations. The calculation formula for the c-th channel is: where a _j (c) represents the soft channel allocation weight: which maps is the weight ratio of the s ^r channel position represented by context information; Dynamic convolution kernels are connected in parallel, which contain different convolution kernel sizes. Randomly reducing the convolution kernel size will cause insufficient extraction of some important features or loss of features; introduce soft channel allocation weight a through r-Softmax, according to The weight factor of each channel adjusts the convolution and size of each channel. The specific formula is as follows: Among them, s is the threshold parameter given according to the actual detection requirements, that is, s-1 1×1 convolution kernel is added to every other K×K convolution kernel. According to the weight of each channel obtained by the formula, the weight proportion is smaller. The size of the convolution kernel on the channel is reduced from 3×3 to 1×1, by maintaining the 3×3 convolution kernel for important channels. 10. A lightweight network-based industrial print image registration method according to claim 7, characterized in that in step S3, the position regression unit and the lightweight unit are arranged in series, and the entire model is adjusted according to the sample set. The location and quantity of the two mechanisms in .