CN116129219B - SAR target class increment recognition method based on knowledge robust-rebalancing network - Google Patents
SAR target class increment recognition method based on knowledge robust-rebalancing networkInfo
- Publication number
- CN116129219B CN116129219B CN202310066776.2A CN202310066776A CN116129219B CN 116129219 B CN116129219 B CN 116129219B CN 202310066776 A CN202310066776 A CN 202310066776A CN 116129219 B CN116129219 B CN 116129219B
- Authority
- CN
- China
- Prior art keywords
- class
- target
- network
- learning
- sar
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Engineering & Computer Science (AREA)
- Molecular Biology (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to a SAR target class increment recognition method based on a knowledge robust-rebalancing network, which comprises the steps of obtaining a SAR target image training set, constructing a class increment learning model based on the knowledge robust-rebalancing network, carrying out iterative training on the class increment learning model by utilizing the SAR target image training set to obtain a class increment learning model after training, and realizing target recognition of an SAR image to be detected by utilizing the class increment learning model after training. The class increment learning model comprises an old target teacher sub-network, a new target increment learning sub-network and a mixed knowledge distillation module. The SAR target class increment recognition method can fully consider and maintain old class feature separability information and enhance the feature separability of new and old classes, and can realize recognition model increment learning to recognize new classes and ensure that the new classes are still robust to old target recognition.
Description
Technical Field
The invention belongs to the technical field of target recognition, and particularly relates to a SAR target class increment recognition method based on a knowledge robust-rebalancing network.
Background
Conventional target recognition methods are generally limited to static environment settings, i.e., it is considered that the tag training data of all target classes to be recognized can be acquired at one time. However, most of the actual recognition environments do not meet the assumption of static conditions, and the marking data of the new target class may be gradually acquired with continuous acquisition of sensor data, so that the actual recognition environments have dynamics.
At present, radar Automatic Target Recognition (RATR) is widely developed and applied in the military and civil fields. However, RATR is a typical dynamic environment object recognition task, which is directed to a dynamic observation environment in which object classes exhibit time-varying characteristics, and new class objects continuously appear, so that it is difficult to acquire data of all object classes to be recognized simultaneously in a training stage to build a complete object recognition library. In the dynamic environment, once a new target class is observed, the conventional target recognition method needs to retrain the recognition model by combining all new and old target class training data so as to ensure the recognition capability of the recognition model on all observed target classes, and the time and space resources are obviously consumed, so that the application of RATR technology in the real dynamic environment is greatly limited.
The Class Increment Learning (CIL) method is a learning algorithm which can be continuously updated and evolved along with the acquired new class data in a dynamic environment, can update the existing recognition model based on the new target class data, and endows the updated recognition model with the capability of simultaneously recognizing the new and old target classes. The Class Increment Learning (CIL) method can be used for efficiently updating the SAR target recognition model based on the new target class data acquired by the SAR sensor, and meanwhile, the recognition of new and old SAR target classes is realized, so that the recognition model is prevented from being repeatedly trained in a dynamic environment in an inefficient manner.
However, in the mainstream class increment learning method, in the process of updating the recognition model by combining new target class data, a serious catastrophic forgetting problem is faced, that is, the recognition performance of the updated recognition model on the old target class is obviously reduced. In order to solve the above problems, researchers have improved both in terms of how to fully migrate old knowledge using old object recognition models and in terms of how to fully correct unbalanced learning of new and old object class data in model updates.
However, the old knowledge migration and class unbalance correction strategies of the existing class incremental learning method only focus on the classification space, neglect key information of the old target class characteristics and influence of class unbalance on the characteristic diversity learning of the new and old target classes, and cause obvious degradation of the recognition performance of the updated model on the old target classes.
Disclosure of Invention
In order to solve the problems in the prior art, the invention provides a SAR target class increment identification method based on a knowledge robust-rebalancing network. The technical problems to be solved by the invention are realized by the following technical scheme:
the invention provides a SAR target class increment identification method based on a knowledge robust-rebalancing network, which comprises the following steps:
Acquiring an SAR target image training set;
constructing a class increment learning model based on a knowledge robust-rebalancing network, wherein the class increment learning model comprises an old target teacher sub-network, a new target increment learning sub-network and a mixed knowledge distillation module,
The old target teacher sub-network is used for providing old target knowledge for the new target incremental learning sub-network and comprises a first trunk feature extraction module and a classification module which are connected in cascade;
The new target incremental learning sub-network is used for incrementally learning new target classes to realize the identification of the new target classes and old target classes at the same time, and comprises a cascaded second trunk feature extraction module and a multi-prototype rebalancing module, wherein the multi-prototype rebalancing module comprises an unbiased classifier learning branch and a separable feature learning branch which are arranged in parallel;
The mixed knowledge distillation module is used for transferring knowledge for correctly identifying an old target from the old target teacher sub-network to the new target incremental learning sub-network;
The loss function of the class increment learning model is as follows:
Ltotal＝λ1·[Lmp+Lfrd]+(1-λ1)·[(1-λ2)·Lce+λ2·Lrd];
Wherein L mp represents a loss function of a separable characteristic learning branch, L ce represents a loss function of an unbiased classifier learning branch, L frd represents a characteristic topological relation distillation loss function, L rd represents a classification layer response distillation loss function, lambda 1 represents a first balance coefficient, lambda 2 represents a second balance coefficient;
performing iterative training on the class incremental learning model by using the SAR target image training set to obtain a class incremental learning model after training;
And (3) utilizing the training-completed class increment learning model to realize target identification of the SAR image to be detected.
In one embodiment of the invention, acquiring a SAR target image training set comprises:
acquiring a plurality of SAR target images, cutting each SAR target image into images with 64 multiplied by 64 pixels, and giving category labels to each SAR target image;
Dividing the plurality of SAR target images into a plurality of groups according to categories, wherein each group comprises 2 category SAR target images;
One group of images is used as an initial training set for training an old target recognition model, the other groups of images are used as a new target class training set for carrying out incremental updating on the class incremental learning model, wherein in each incremental updating, SAR target images of not more than 300 old target classes and the new target class training set are selected to form an incremental learning training set together, and the class incremental learning model is trained and updated.
In one embodiment of the invention, the first trunk feature extraction module and the second trunk feature extraction module are identical in structure and comprise a first convolution layer, a plurality of residual convolution layers and a pooling layer which are cascaded, wherein,
The convolution kernel size of the first convolution layer is 3×3, and the number of convolution kernels is set to 64;
the residual convolution layer comprises a second convolution layer, a first Batch Norm layer, a first ReLU nonlinear activation function layer, a third convolution layer, a second Batch Norm layer and a second ReLU nonlinear activation function layer, wherein,
The input feature map of the residual convolution layer sequentially passes through the second convolution layer, the first Batch Norm layer and the first ReLU nonlinear activation function layer to obtain a first feature map, the first feature map sequentially passes through the third convolution layer and the second Batch Norm layer to be mapped to obtain a second feature map, and the second feature map is added with the input feature map of the residual convolution layer after downsampling to obtain an output feature map of the residual convolution layer through the second ReLU nonlinear activation function layer;
the pooling layer adopts global average pooling.
In one embodiment of the present invention, the classification module includes a first full-connection layer, where the number of input nodes of the first full-connection layer is 512, and the number of output nodes is N, where N represents the total number of classes of the old target class.
In one embodiment of the present invention, the unbiased classifier learning branch includes a second full connection layer, where the number of input nodes in the second full connection layer is 512, the number of output nodes is n+m, and M represents the total number of classes of the newly added target class;
The loss function of the unbiased classifier learning branch is as follows:
Wherein B represents the number of training samples of the learning branch of the unbiased classifier input in the current batch training process, y b represents the real class label of the B-th training sample x b, Representing the predicted classification score obtained by the training sample x b through the second stem feature extraction module and the unbiased classifier learning branch, softmax () represents the normalized exponential function.
In one embodiment of the present invention, the separable feature learning branch includes a third full-connection layer and a fourth full-connection layer in cascade, where the number of input nodes of the third full-connection layer is 512, the number of output nodes of the third full-connection layer is 512, the number of input nodes of the fourth full-connection layer is 512, and the number of output nodes of the fourth full-connection layer is 128;
the loss function of the separable characteristic learning branch is as follows:
wherein B represents the number of training samples for inputting the separable characteristic learning branches in the current batch training process, Representing the embedded features of the d-th training sample x d obtained through the second trunk feature extraction module and the separable feature learning branch, y d representing the true class label of the training sample x d,Representing the s-th prototype parameters in the class prototype set of class j,Representing the S-th prototype parameter in the prototype set of the class corresponding to the class y d, and S represents the total number of prototype parameters in the prototype set of the class.
In one embodiment of the invention, the characteristic topological relation distillation loss function is:
wherein Γ o represents a feature similarity matrix of sample features extracted by the old target teacher sub-network, Γ t represents a feature similarity matrix of sample features extracted by the new target incremental learning sub-network, and W mask is a mask matrix;
The classifying layer response distillation loss function is:
In the formula, Representing the softened output probability of the new target incremental learning sub-network output for sample x i in the j-th class, q j(xi) represents the softened output probability of the old target teacher sub-network output for sample x i in the j-th class.
In one embodiment of the present invention, when the class incremental learning model is iteratively trained by using the SAR target image training set, a class incremental learning model after training is obtained, the method further includes:
And training an old target recognition model by using the initial training set, wherein the trained old target recognition model is used as an old target teacher sub-network of the class incremental learning model.
In one embodiment of the present invention, the training set of SAR target images is used to iteratively train the class incremental learning model to obtain a trained class incremental learning model, which includes the following steps:
step 3a, initializing parameters of the new target incremental learning sub-network;
step 3B, selecting 2B SAR target images from the incremental learning training set, wherein the B SAR target images are obtained by randomly sampling the incremental learning training set, and the B SAR target images are obtained by uniformly sampling the incremental learning training set;
Step 3c, inputting 2B SAR target images into a second trunk feature extraction module of the new target increment learning sub-network to obtain corresponding features, inputting features corresponding to random sampling samples into the separable feature learning branches, calculating loss functions of the corresponding separable feature learning branches, inputting features corresponding to class-balanced sampling samples into the unbiased classifier learning branches, and calculating loss functions of the corresponding unbiased classifier learning branches;
Step 3d, inputting 2B SAR target images into a first trunk feature extraction module of the old target teacher sub-network to obtain corresponding features, calculating corresponding feature topological relation distillation loss functions based on features corresponding to random sampling samples, inputting features corresponding to class-balanced sampling samples into a classification module of the old target teacher sub-network, and calculating corresponding classification layer response distillation loss functions;
Step 3e, according to the calculated loss function of the separable characteristic learning branch, the loss function of the unbiased classifier learning branch, the characteristic topological relation distillation loss function and the classification layer response distillation loss function, updating the parameters of the new target increment learning sub-network by using a gradient descent method;
And 3f, repeating the steps 3b-3e for a plurality of times of iterative training until a preset training stopping condition is reached, and finishing one-time incremental updating to obtain a training-finished class incremental learning model.
In one embodiment of the present invention, the target recognition of the SAR image to be detected is realized by using the trained class increment learning model, which comprises the following steps:
Inputting the SAR image to be detected into a new target increment learning sub-network of a class increment learning model with training completed, after extracting the characteristics of the SAR image to be detected, inputting the extracted characteristics into a learning branch of the unbiased classifier to obtain the prediction classification probability of the SAR image to be detected, and obtaining the prediction class of the SAR image to be detected according to the maximum probability.
Compared with the prior art, the invention has the beneficial effects that:
The SAR target class increment recognition method based on the knowledge robust-rebalancing network utilizes a trained class increment learning model based on the knowledge robust-rebalancing network to realize target recognition of SAR images to be detected, the class increment learning model enhances the feature separability between new and old classes of samples based on multi-prototype metric learning at a feature level and optimizes the class weight balanced classifier based on class balance sampling at a classification level, the problem that the old target class test sample is easily misjudged as a new target class due to the new and old class imbalance learning of the network is fully corrected, the old target knowledge robust maintenance strategy combining designed feature topological relation maintenance and classification prediction maintenance is utilized, the old class feature separability learned by the old target recognition model is maintained in incremental updating based on topological relation distillation at the feature level, and the old class correct prediction capability learned by the old target recognition model is maintained based on response knowledge distillation at the classification level. The SAR target class increment recognition method can fully consider and maintain old class feature separability information and enhance the feature separability of new and old classes, and can realize recognition model increment learning to recognize new classes and ensure that the new classes are still robust to old target recognition.
The foregoing description is only an overview of the present invention, and is intended to be implemented in accordance with the teachings of the present invention, as well as the preferred embodiments thereof, together with the following detailed description of the invention, given by way of illustration only, together with the accompanying drawings.
Drawings
FIG. 1 is a flowchart of a SAR target class increment recognition method based on a knowledge robust-rebalancing network provided by an embodiment of the present invention;
FIG. 2 is a schematic diagram of a framework of a knowledge-based robust-rebalancing network-based class incremental learning model according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a residual convolution layer according to an embodiment of the present disclosure;
Fig. 4 is a graph of the recognition accuracy of the method of the present invention and the existing class increment method based on the MSTAR dataset increment experiment according to the embodiment of the present invention.
Detailed Description
In order to further explain the technical means and effects adopted by the invention to achieve the preset aim, the following describes in detail a SAR target class increment recognition method based on a knowledge robust-rebalance network according to the invention with reference to the attached drawings and the detailed description.
The foregoing and other features, aspects, and advantages of the present invention will become more apparent from the following detailed description of the preferred embodiments when taken in conjunction with the accompanying drawings. The technical means and effects adopted by the present invention to achieve the intended purpose can be more deeply and specifically understood through the description of the specific embodiments, however, the attached drawings are provided for reference and description only, and are not intended to limit the technical scheme of the present invention.
Example 1
Referring to fig. 1, fig. 1 is a flowchart of a method for identifying SAR target class increments based on a knowledge robust-rebalancing network according to an embodiment of the present invention, as shown in the fig. 1, the method for identifying SAR target class increments based on the knowledge robust-rebalancing network according to the embodiment includes:
Step 1, acquiring a SAR target image training set;
in an alternative embodiment, step 1 comprises:
Step 1a, acquiring a plurality of SAR target images, cutting each SAR target image into images with 64 multiplied by 64 pixels, and giving category labels to each SAR target image;
in this embodiment, all 5172 images in the SAR vehicle target dataset MSTAR (moving and stationary target acquisition) are used as a sample set, each of 2746 images with the radar working pitch angle of 17 ° in the sample set is cut into images with the pixels of 64×64, and each image is given a category label of 1-10, that is, 2746 images are divided into 10 categories.
Step 1b, dividing a plurality of SAR target images into a plurality of groups according to categories, wherein each group comprises 2 category SAR target images;
in this embodiment, 10 classes of SAR target images are randomized into 5 groups, each group containing 2 classes of SAR target images.
And 1c, taking one group of images as an initial training set for training an old target recognition model, taking the other groups of images as a new target class training set for carrying out incremental updating on a class incremental learning model, wherein in each incremental updating, the SAR target images of not more than 300 old target classes and the new target class training set are selected to form an incremental learning training set together, and training and updating are carried out on the class incremental learning model.
In this embodiment, the first group of images is used as an initial training set, the other 4 groups are used as new target class training sets, and in each incremental update, 300 SAR target images of the old target class and the new target class training sets are selected to form an incremental learning training set together, and it is to be noted that the proportion of each class in the SAR target images of the 300 old target classes is equal.
Step 2, constructing a knowledge-based robust-rebalancing network-based class incremental learning model;
Please refer to the schematic frame diagram of a knowledge-based robust-rebalance network-based class incremental learning model provided in the embodiment of the present invention shown in fig. 2, wherein the knowledge-based robust-rebalance network-based class incremental learning model of the present embodiment includes an old target teacher sub-network M o, a new target incremental learning sub-network M t and a mixed knowledge distillation module HKD.
Wherein the old target teacher sub-network M o is used to provide old target knowledge for the new target incremental learning sub-network M t, in this embodiment the old target teacher sub-network M o is a given old target identification network.
Optionally, the old target teacher sub-network M o includes a first trunk feature extraction module F and a classification module C in cascade.
In an alternative embodiment, the first trunk feature extraction module F comprises a first convolution layer, a plurality of residual convolution layers and a pooling layer in cascade, and the classification module C comprises a first full-connection layer.
In the embodiment, the first trunk feature extraction module F comprises nine residual convolution layers, and the specific structure of the first trunk feature extraction module F is that a first layer of convolution layers, a first layer of residual convolution layers, a second layer of residual convolution layers, a third layer of residual convolution layers, a fourth layer of residual convolution layers, a fifth layer of residual convolution layers, a sixth layer of residual convolution layers, a seventh layer of residual convolution layers, an eighth layer of residual convolution layers, a ninth layer of residual convolution layers and a pooling layer. Wherein the convolution kernel size of the first layer of convolution layers is 3×3, and the number of convolution kernels is set to 64.
Referring to fig. 3, a schematic structural diagram of a residual convolution layer according to an embodiment of the present invention is shown, where the residual convolution layer in this embodiment includes a second convolution layer, a first Batch Norm layer, a first ReLU nonlinear active function layer, a third convolution layer, a second Batch Norm layer, and a second ReLU nonlinear active function layer. The input feature map of the residual convolution layer sequentially passes through a second convolution layer, a first Batch Norm layer and a first ReLU nonlinear activation function layer to obtain a first feature map, the first feature map sequentially passes through a third convolution layer and a second Batch Norm layer to be mapped to obtain a second feature map, and the second feature map is added with the input feature map of the residual convolution layer subjected to downsampling and then passes through the second ReLU nonlinear activation function layer to obtain an output feature map of the residual convolution layer. In this embodiment, the convolution kernel sizes of the first to ninth residual convolution layers are all 3×3, and the number of convolution kernels is sequentially set to 64,64,128,128,256,256,256,512,512;
In this embodiment, the pooling layer is set to global average pooling, and the feature map of 512×h×w output by the ninth layer residual convolution layer is mapped to a feature vector of 512×1. The number of input nodes of the first full connection layer is 512, the number of output nodes is N, and N represents the total category number of the old target category.
The new target incremental learning sub-network M t is a network obtained by expanding the old target teacher sub-network M o, and is used for incremental learning of a new target class to realize identification of the new target class and the old target class at the same time.
In an alternative embodiment, the new target incremental learning sub-network M t includes a cascaded second stem feature extraction module F' and a multi-prototype re-balancing module MPR. In this embodiment, the second trunk feature extraction module F' and the first trunk feature extraction module F have the same structure, and are not described herein. The multi-prototype re-balance module MPR comprises an unbiased classifier learning branch and a separable feature learning branch arranged in parallel.
Optionally, the unbiased classifier learning branch includes a second full connection layer, where the number of input nodes in the second full connection layer is 512, the number of output nodes is n+m, and M represents the total class number of the newly added target class. The loss function of the unbiased classifier learning branch is as follows:
Wherein B represents the number of training samples of the learning branch of the unbiased classifier input in the current batch training process, y b represents the real class label of the B-th training sample x b, Representing the predicted classification score obtained by the training sample x b through the second stem feature extraction module and the unbiased classifier learning branch, softmax () represents the normalized exponential function.
In this embodiment, the loss function L ce of the unbiased classifier learning branch represents the cross entropy loss of the predicted label and the real label obtained by the unbiased classifier learning branch for the B samples of the current lot. It should be noted that, the above B samples come from class-balanced sampling of the incremental learning training set, so as to ensure that the classifier learns unbiased class weight parameters.
Optionally, the separable feature learning branch includes a third full-connection layer and a fourth full-connection layer, where the number of input nodes of the third full-connection layer is 512, the number of output nodes is 512, the number of input nodes of the fourth full-connection layer is 512, and the number of output nodes is 128. The loss function of the separable characteristic learning branch is as follows:
wherein B represents the number of training samples for inputting the separable characteristic learning branches in the current batch training process, Representing the embedded features of the d-th training sample x d obtained through the second trunk feature extraction module and the separable feature learning branch, y d representing the true class label of the training sample x d,Representing the s-th prototype parameters in the class prototype set of class j,Representing the S-th prototype parameter in the prototype set of the class corresponding to the class y d, and S represents the total number of prototype parameters in the prototype set of the class.
In the present embodiment, the loss function L mp of the separable characteristic learning branch represents the embedded characteristic obtained by the separable characteristic learning branch of the current lot B samplesAnd various prototype setsLoss of multi-prototype comparison between. Optimizing the loss can restrict the embedded characteristics of various samplesApproach to a prototype set of this class in feature spaceAway from other prototype setsAnd further, the method ensures that all new target class features and old target class features extracted by the second trunk feature extraction module F' have good intra-class compactness and inter-class separability. It should be noted that, the above B samples are derived from random sampling of the incremental learning training set, so as to ensure that the second trunk feature extraction module F' has a good separable feature extraction capability under the original data distribution.
Wherein the hybrid knowledge distillation module HKD is intended to migrate knowledge that correctly identifies an old target from the old target teacher sub-network M o to the new target incremental learning sub-network M t by constraining the feature responses and the classification decision responses of the old target teacher sub-network M o and the new target incremental learning sub-network M t to agree. In this embodiment, the hybrid knowledge distillation module HKD includes a characteristic topological relationship distillation loss function L frd and a classification layer response distillation loss function L rd, i.e., L HKD＝Lfrd+Lrd.
Wherein, the characteristic topological relation distillation loss function is:
Where Γ o represents the feature similarity matrix of the sample features extracted by the old target teacher sub-network, Γ t represents the feature similarity matrix of the sample features extracted by the new target incremental learning sub-network, and W mask is the mask matrix.
In particular, the method comprises the steps of,Wherein z i andRepresenting the features obtained by the first trunk feature extraction module F of the old target teacher sub-network M o and the second trunk feature extraction module F' of the new target incremental learning sub-network M t, respectively, for sample x i, representing cosine similarity calculation, W mask is used for extracting feature similarity terms related to the old target class samples in Γ o and Γ t.
In this embodiment, the feature topology distillation loss function L frd constrains similarity agreement between features of the current lot B sample inputs to the old target teacher sub-network M o and the new target delta learning sub-network M t. This loss transfers the discriminative knowledge of the old target teacher sub-network M o that learns that the old target class features are highly similar and that the inter-class similarity is low to the new target incremental learning sub-network M t so that the new target incremental learning sub-network M t retains the ability to extract the old target separable features.
Wherein, the response distillation loss function of the classifying layer is:
In the formula, Representing the softened output probability of the new target incremental learning sub-network output for sample x i in the j-th class, q j(xi) represents the softened output probability of the old target teacher sub-network output for sample x i in the j-th class.
In particular, the method comprises the steps of,Representing the classification score of sample x i in the new target delta learning sub-network M t, τ is the softening temperature coefficient,
Representing the classification score of sample x i on the old target teacher sub-network M o.
In this embodiment, the classification layer responds to the distillation loss function L rd to constrain the classification responses of the current lot B samples input to the old target teacher sub-network M o and the new target delta learning sub-network M t to be consistent so that the new target delta learning sub-network M t still retains the correct prediction ability for the old target class.
Step 3, performing iterative training on the class increment learning model by utilizing the SAR target image training set to obtain a class increment learning model after training;
In the embodiment, the method comprises the steps of performing iterative training on the class increment learning model by utilizing the SAR target image training set to obtain a trained class increment learning model, and training an old target recognition model by utilizing the initial training set, wherein the trained old target recognition model is used as an old target teacher sub-network of the class increment learning model.
In this embodiment, the specific training process for training the old target recognition model is consistent with the conventional target recognition training method under the static environment setting, and will not be described herein.
In an alternative embodiment, the iterative training of the class incremental learning model using the SAR target image training set includes the steps of:
step 3a, initializing parameters of a new target increment learning sub-network;
Optionally, the parameters of the new target incremental learning sub-network are randomly initialized based on normal distribution, and the set iterative training parameters of the class incremental learning model include the iterative times Q and the maximum iterative times Q, wherein Q is greater than or equal to 100 and q=0, and in the embodiment, q=200.
Step 3B, selecting 2B SAR target images from the incremental learning training set, wherein the B SAR target images are obtained by randomly sampling the incremental learning training set, and the B SAR target images are obtained by uniformly sampling the incremental learning training set;
Optionally, the B SAR target images obtained by random sampling are denoted as D rad, and the B SAR target images obtained by class-balanced sampling are denoted as D bal.
Wherein the sampling probability of selecting one sample from the j-th training data is thatWhere q e [0,1], n j represents the number of SAR target images contained in the j-th class of training data, and when q=0 corresponds to random sampling, q=1 corresponds to class equalization sampling, in this embodiment b=128.
Step 3c, inputting 2B SAR target images into a second trunk feature extraction module of the new target increment learning sub-network to obtain corresponding features, inputting features corresponding to random sampling samples into separable feature learning branches, calculating loss functions of the corresponding separable feature learning branches, inputting features corresponding to class-balanced sampling samples into unbiased classifier learning branches, and calculating loss functions of the corresponding unbiased classifier learning branches;
In this embodiment, D rad and D bal are input into the second trunk feature extraction module F' of the new target incremental learning subnetwork M t and undergo global averaging pooling to obtain corresponding K-dimensional features AndFeatures corresponding to samples to be randomly sampledInputting the separable feature learning branches of the multi-prototype re-balance module MPR to obtain the mapped embedded featuresAnd is based onThe loss function L mp of the separable feature learning branch is calculated. Corresponding characteristics of class-balanced sampling sampleUnbiased classifier learning branches of input multi-prototype re-balance module MPR to obtain mapped classification scoresAnd is based onAnd calculating a loss function L ce of the unbiased classifier learning branch.
Step 3d, inputting 2B SAR target images into a first trunk feature extraction module of an old target teacher sub-network to obtain corresponding features, calculating corresponding feature topological relation distillation loss functions based on features corresponding to random sampling samples, inputting features corresponding to class equilibrium sampling samples into a classification module of the old target teacher sub-network, and calculating corresponding classification layer response distillation loss functions according to prediction classification scores of the old target teacher sub-network and prediction classification scores of new target increment learning sub-networks;
in this embodiment, D rad and D bal are input into the first trunk feature extraction module F of the old target teacher sub-network M o and undergo global averaging pooling to obtain corresponding K-dimensional features AndRespectively calculating extracted features of old target teacher sub-network M o Feature similarity matrix of (a)And extracted features of new target incremental learning sub-network M t Feature similarity matrix of (a)And calculates a characteristic topological relation distillation loss function L frd based on Γ o and Γ t. Corresponding characteristics of class-balanced sampling sampleInputting the obtained classification scores into a classification module C to obtain mapped classification scoresAnd based on the classification score O bal predicted by the old target teacher sub-network M o for the class-balanced sampling sample and the classification score predicted by the new target incremental learning sub-network M t for the class-balanced sampling sampleThe loss classification layer response distillation loss function L rd is calculated.
Step 3e, according to the calculated loss function of the separable characteristic learning branch, the unbiased classifier learns the loss function of the branch, the characteristic topological relation distillation loss function and the classification layer response distillation loss function, and the gradient descent method is utilized to update the parameters of the new target increment learning sub-network;
In this embodiment, the gradient descent method is used to update the parameters of each layer of the new target incremental learning sub-network M t according to the loss function of the incremental learning model by using the loss functions L mp and L ce calculated in step 3d and the loss functions L frd and L rd calculated in step 3e, so as to obtain an updated incremental learning model.
The loss function of the class increment learning model is as follows:
Ltotal＝λ1·[Lmp+Lfrd]+(1-λ1)·[(1-λ2)·Lce+λ2·Lrd];
wherein λ 1 represents a first balance coefficient, and λ 2 represents a second balance coefficient;
Optionally, the training process uses a momentum SGD optimizer, the weight regularization term is 1×10 -4, the momentum factor is 0.9, the initial learning rate is set to 0.5, and the initial learning rate is attenuated to 1/10 of the original learning rate in 120 iterations and 160 iterations, respectively.
And 3f, repeating the steps bc-3e for a plurality of times for iterative training until a preset training stopping condition is reached, and finishing one-time increment updating to obtain a training-completed class increment learning model.
In this embodiment, the preset training stop condition is that the iteration number Q reaches the maximum iteration number Q, or that the loss function of the class-increment learning model reaches a preset threshold.
After the incremental update is completed, if a new incremental update is to be performed, the class incremental learning model training process is performed according to the class incremental learning model training process in the steps 3b-3f, and then the class incremental learning model training is performed by using the new incremental learning training set.
It is worth to say that, the class increment learning model based on the knowledge robust-rebalancing network constructed in the embodiment carries out updating training of the new object increment recognition model based on a small amount of stored old object class samples and acquired new object class samples, key feature knowledge and key classification knowledge which correctly recognize the old object in the old object recognition model are transferred to the new object increment recognition model in the updating training process, the robustness of the learned old object knowledge is guaranteed, the balancing learning of the new object increment recognition model on the new object class samples and the old object class samples is realized based on the rebalancing module, the problem that the feature separability of the old object class extracted by the new object increment recognition model is poor and the preference exists in classification prediction is solved, and the problem that the old object class recognition performance cannot be effectively maintained while the new object class is learned in an increment mode in the prior art is solved.
And 4, utilizing the trained class increment learning model to realize target identification of the SAR image to be detected.
In an alternative embodiment, the SAR image to be detected is input into a new target increment learning sub-network of the trained class increment learning model, the second trunk feature extraction module extracts features of the SAR image to be detected, the extracted features are input into an unbiased classifier learning branch to obtain the prediction classification probability of the SAR image to be detected, and the prediction class of the SAR image to be detected is obtained according to the maximum probability.
In this embodiment, each image in 2426 images with radar working pitch angle of 15 ° in a sample set formed by target data of SAR vehicles is cut into images with pixels of 64×64, class labels of each image belong to 1-10, and all cut images are divided into test sets corresponding to 5 sets of training sets according to class labels, so as to be used as SAR images to be tested, and the training-completed class incremental learning model is tested.
The SAR target class increment recognition method based on the knowledge robust-rebalancing network can fully consider the feature separability information of the old class and enhance the feature separability of the new class, and can realize recognition model increment learning to recognize the new class and ensure that the new class is still robust to old target recognition.
Referring to the graph of the recognition accuracy based on the MSTAR data set increment experiment provided by the embodiment of the invention and the conventional class increment method, as shown in FIG. 4, the recognition accuracy of the method of the invention is better than that of the conventional class increment method.
It is to be understood that the above description is only one specific example of the present invention and is not to be construed as limiting the invention in any way, and that various modifications and changes in form and detail may be made to the invention without departing from the principles and construction thereof, which, however, are intended to be within the scope of the appended claims. For example, a possible alternative is to optimize the hidden feature extraction of new and old object class samples based on example contrast learning at the feature level, describe the learned feature topology relationship of the old object recognition model based on any metric mode, but still aim to ensure the robustness of the old object feature relationship and the separability of the new and old object features, and the improvement of the basic optimization strategy is still within the scope of the claims of the invention without departing from the concept of the invention.
It should be noted that in this document relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that an article or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed. Without further limitation, an element defined by the phrase "comprising one does not exclude the presence of other like elements in an article or apparatus that comprises the element. The terms "connected" or "connected," and the like, are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.
The foregoing is a further detailed description of the invention in connection with the preferred embodiments, and it is not intended that the invention be limited to the specific embodiments described. It will be apparent to those skilled in the art that several simple deductions or substitutions may be made without departing from the spirit of the invention, and these should be considered to be within the scope of the invention.
Claims (10)
1. A knowledge-based robust-rebalancing network-based SAR target class increment identification method, comprising:
Acquiring an SAR target image training set;
constructing a class increment learning model based on a knowledge robust-rebalancing network, wherein the class increment learning model comprises an old target teacher sub-network, a new target increment learning sub-network and a mixed knowledge distillation module,
The old target teacher sub-network is used for providing old target knowledge for the new target incremental learning sub-network and comprises a first trunk feature extraction module and a classification module which are connected in cascade;
The new target incremental learning sub-network is used for incrementally learning new target classes to realize the identification of the new target classes and old target classes at the same time, and comprises a cascaded second trunk feature extraction module and a multi-prototype rebalancing module, wherein the multi-prototype rebalancing module comprises an unbiased classifier learning branch and a separable feature learning branch which are arranged in parallel;
The mixed knowledge distillation module is used for transferring knowledge for correctly identifying an old target from the old target teacher sub-network to the new target incremental learning sub-network;
The loss function of the class increment learning model is as follows:
Ltotal＝λ1·[Lmp+Lfrd]+(1-λ1)·[(1-λ2)·Lce+λ2·Lrd];
Wherein L mp represents a loss function of a separable characteristic learning branch, L ce represents a loss function of an unbiased classifier learning branch, L frd represents a characteristic topological relation distillation loss function, L rd represents a classification layer response distillation loss function, lambda 1 represents a first balance coefficient, lambda 2 represents a second balance coefficient;
performing iterative training on the class incremental learning model by using the SAR target image training set to obtain a class incremental learning model after training;
And (3) utilizing the training-completed class increment learning model to realize target identification of the SAR image to be detected.
2. The knowledge-based robust-rebalancing network based SAR target class delta identification method according to claim 1, wherein obtaining the SAR target image training set comprises:
acquiring a plurality of SAR target images, cutting each SAR target image into images with 64 multiplied by 64 pixels, and giving category labels to each SAR target image;
Dividing the plurality of SAR target images into a plurality of groups according to categories, wherein each group comprises 2 category SAR target images;
One group of images is used as an initial training set for training an old target recognition model, the other groups of images are used as a new target class training set for carrying out incremental updating on the class incremental learning model, wherein in each incremental updating, SAR target images of not more than 300 old target classes and the new target class training set are selected to form an incremental learning training set together, and the class incremental learning model is trained and updated.
3. The knowledge-based robust-rebalancing network SAR target class increment recognition method of claim 1, wherein the first and second backbone feature extraction modules are structurally identical and each comprise a first convolution layer, a plurality of residual convolution layers and a pooling layer in cascade connection,
The convolution kernel size of the first convolution layer is 3×3, and the number of convolution kernels is set to 64;
the residual convolution layer comprises a second convolution layer, a first Batch Norm layer, a first ReLU nonlinear activation function layer, a third convolution layer, a second Batch Norm layer and a second ReLU nonlinear activation function layer, wherein,
The input feature map of the residual convolution layer sequentially passes through the second convolution layer, the first Batch Norm layer and the first ReLU nonlinear activation function layer to obtain a first feature map, the first feature map sequentially passes through the third convolution layer and the second Batch Norm layer to be mapped to obtain a second feature map, and the second feature map is added with the input feature map of the residual convolution layer after downsampling to obtain an output feature map of the residual convolution layer through the second ReLU nonlinear activation function layer;
the pooling layer adopts global average pooling.
4. The method for incremental identification of SAR target classes based on a knowledge robust-rebalance network according to claim 1, wherein said classification module comprises a first fully connected layer having an input node number of 512 and an output node number of N, wherein N represents the total class number of the old target class.
5. The knowledge-based robust-rebalancing network-based SAR target class increment recognition method according to claim 4, wherein the unbiased classifier learning branch includes a second full-connection layer, the number of input nodes of the second full-connection layer is 512, the number of output nodes is N+M, and M represents the total class number of the newly added target class;
The loss function of the unbiased classifier learning branch is as follows:
Wherein B represents the number of training samples of the learning branch of the unbiased classifier input in the current batch training process, y b represents the real class label of the B-th training sample x b, Representing the predicted classification score obtained by the training sample x b through the second stem feature extraction module and the unbiased classifier learning branch, softmax () represents the normalized exponential function.
6. The knowledge-based robust-rebalancing network-based SAR target class increment recognition method according to claim 5, wherein the separable feature learning branch comprises a third fully connected layer and a fourth fully connected layer in cascade, the number of input nodes of the third fully connected layer is 512, the number of output nodes of the third fully connected layer is 512, the number of input nodes of the fourth fully connected layer is 512, and the number of output nodes of the fourth fully connected layer is 128;
the loss function of the separable characteristic learning branch is as follows:
wherein B represents the number of training samples for inputting the separable characteristic learning branches in the current batch training process, Representing the embedded features of the d-th training sample x d obtained through the second trunk feature extraction module and the separable feature learning branch, y d representing the true class label of the training sample x d,Representing the s-th prototype parameters in the class prototype set of class j,Representing the S-th prototype parameter in the prototype set of the class corresponding to the class y d, and S represents the total number of prototype parameters in the prototype set of the class.
7. The knowledge-based robust-rebalancing network based SAR target class delta identification method of claim 6, wherein said characteristic topological relation distillation loss function is:
wherein Γ o represents a feature similarity matrix of sample features extracted by the old target teacher sub-network, Γ t represents a feature similarity matrix of sample features extracted by the new target incremental learning sub-network, and W mask is a mask matrix;
The classifying layer response distillation loss function is:
In the formula, Representing the softened output probability of the new target incremental learning sub-network output for sample x i in the j-th class, q j(xi) represents the softened output probability of the old target teacher sub-network output for sample x i in the j-th class.
8. The knowledge-based robust-rebalancing network-based SAR target class increment recognition method according to claim 2, wherein, before performing iterative training on the class increment learning model by using the SAR target image training set to obtain a trained class increment learning model, further comprising:
And training an old target recognition model by using the initial training set, wherein the trained old target recognition model is used as an old target teacher sub-network of the class incremental learning model.
9. The knowledge-based robust-rebalancing network-based SAR target class increment recognition method according to claim 8, wherein the iterative training of the class increment learning model is performed by using the SAR target image training set to obtain a trained class increment learning model, comprising the steps of:
step 3a, initializing parameters of the new target incremental learning sub-network;
step 3B, selecting 2B SAR target images from the incremental learning training set, wherein the B SAR target images are obtained by randomly sampling the incremental learning training set, and the B SAR target images are obtained by uniformly sampling the incremental learning training set;
Step 3c, inputting 2B SAR target images into a second trunk feature extraction module of the new target increment learning sub-network to obtain corresponding features, inputting features corresponding to random sampling samples into the separable feature learning branches, calculating loss functions of the corresponding separable feature learning branches, inputting features corresponding to class-balanced sampling samples into the unbiased classifier learning branches, and calculating loss functions of the corresponding unbiased classifier learning branches;
Step 3d, inputting 2B SAR target images into a first trunk feature extraction module of the old target teacher sub-network to obtain corresponding features, calculating corresponding feature topological relation distillation loss functions based on features corresponding to random sampling samples, inputting features corresponding to class-balanced sampling samples into a classification module of the old target teacher sub-network, and calculating corresponding classification layer response distillation loss functions;
Step 3e, according to the calculated loss function of the separable characteristic learning branch, the loss function of the unbiased classifier learning branch, the characteristic topological relation distillation loss function and the classification layer response distillation loss function, updating the parameters of the new target increment learning sub-network by using a gradient descent method;
And 3f, repeating the steps 3b-3e for a plurality of times of iterative training until a preset training stopping condition is reached, and finishing one-time incremental updating to obtain a training-finished class incremental learning model.
10. The knowledge-based robust-rebalancing network-based SAR target class increment recognition method according to claim 1, wherein the training-completed class increment learning model is utilized to realize target recognition of the SAR image to be detected, comprising:
Inputting the SAR image to be detected into a new target increment learning sub-network of a class increment learning model with training completed, after extracting the characteristics of the SAR image to be detected, inputting the extracted characteristics into a learning branch of the unbiased classifier to obtain the prediction classification probability of the SAR image to be detected, and obtaining the prediction class of the SAR image to be detected according to the maximum probability.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310066776.2A CN116129219B (en) | 2023-01-16 | 2023-01-16 | SAR target class increment recognition method based on knowledge robust-rebalancing network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310066776.2A CN116129219B (en) | 2023-01-16 | 2023-01-16 | SAR target class increment recognition method based on knowledge robust-rebalancing network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116129219A CN116129219A (en) | 2023-05-16 |
| CN116129219B true CN116129219B (en) | 2025-07-29 |
Family
ID=86300617
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310066776.2A Active CN116129219B (en) | 2023-01-16 | 2023-01-16 | SAR target class increment recognition method based on knowledge robust-rebalancing network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116129219B (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116843620B (en) * | 2023-05-29 | 2025-08-26 | 合肥综合性国家科学中心人工智能研究院(安徽省人工智能实验室) | A printed circuit board defect detection method based on non-example-based incremental learning |
| CN116797928B (en) * | 2023-06-21 | 2025-09-05 | 西安电子科技大学 | Incremental classification method for SAR targets based on stability and plasticity of equilibrium model |
| CN116977870B (en) * | 2023-06-28 | 2025-07-29 | 西安电子科技大学 | SAR target increment identification method based on dynamic structure and multi-level distillation |
| CN116824262A (en) * | 2023-07-06 | 2023-09-29 | 长安大学 | Remote sensing image scene classification method |
| CN116977635B (en) * | 2023-07-19 | 2024-04-16 | 中国科学院自动化研究所 | Category-incremental semantic segmentation learning method and semantic segmentation method |
| CN117235618B (en) * | 2023-08-07 | 2025-07-29 | 西安电子科技大学 | Multifunctional radar working mode increment identification method based on depth feature expansion |
| CN116778264B (en) * | 2023-08-24 | 2023-12-12 | 鹏城实验室 | Object classification method, image classification method and related equipment based on incremental learning |
| CN117274668B (en) * | 2023-08-30 | 2025-10-17 | 浙江中烟工业有限责任公司 | Defect increment learning method and device based on small sample |
| CN118395239B (en) * | 2024-05-15 | 2025-06-17 | 西安电子科技大学 | Incremental recognition method of radar working mode based on attention distillation |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114943889A (en) * | 2022-03-24 | 2022-08-26 | 西安电子科技大学 | SAR image target identification method based on small sample incremental learning |
| CN115170872A (en) * | 2022-06-23 | 2022-10-11 | 江苏科技大学 | Class increment learning method based on knowledge distillation |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2612866B (en) * | 2021-11-09 | 2024-10-23 | Samsung Electronics Co Ltd | Method and apparatus for class incremental learning |
| CN115294385A (en) * | 2022-07-07 | 2022-11-04 | 西北工业大学 | Sample labeling method and device, electronic equipment and computer-readable storage medium |
-
2023
- 2023-01-16 CN CN202310066776.2A patent/CN116129219B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114943889A (en) * | 2022-03-24 | 2022-08-26 | 西安电子科技大学 | SAR image target identification method based on small sample incremental learning |
| CN115170872A (en) * | 2022-06-23 | 2022-10-11 | 江苏科技大学 | Class increment learning method based on knowledge distillation |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116129219A (en) | 2023-05-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116129219B (en) | SAR target class increment recognition method based on knowledge robust-rebalancing network | |
| CN114241282B (en) | Knowledge distillation-based edge equipment scene recognition method and device | |
| CN113378632B (en) | Pseudo-label optimization-based unsupervised domain adaptive pedestrian re-identification method | |
| CN111191732B (en) | Target detection method based on full-automatic learning | |
| CN112347970B (en) | Remote sensing image ground object identification method based on graph convolution neural network | |
| CN113222011B (en) | Small sample remote sensing image classification method based on prototype correction | |
| CN113361476B (en) | A method for identifying abnormal signals before the Zhangheng No. 1 earthquake based on artificial intelligence technology | |
| CN118734196B (en) | Method for predicting failure of gyroscope group based on MBKA-GBDT | |
| CN116910571B (en) | Open-domain adaptation method and system based on prototype comparison learning | |
| CN105701502A (en) | Image automatic marking method based on Monte Carlo data balance | |
| CN112364974B (en) | YOLOv3 algorithm based on activation function improvement | |
| CN109508746A (en) | Pulsar candidate's body recognition methods based on convolutional neural networks | |
| CN113344243A (en) | Wind speed prediction method and system for optimizing ELM based on improved Harris eagle algorithm | |
| CN114782752A (en) | Small sample image ensemble classification method and device based on self-training | |
| CN112651317A (en) | Hyperspectral image classification method and system for sample relation learning | |
| CN118172674B (en) | Remote sensing estimation method and system for reset value of residential building | |
| CN108985378B (en) | A Domain Adaptation Method Based on Hybrid Cross Deep Network | |
| CN115457596A (en) | Unsupervised pedestrian re-identification method based on camera perception map learning | |
| CN111695531A (en) | Cross-domain pedestrian re-identification method based on heterogeneous convolutional network | |
| CN111160526A (en) | Online testing method and device for deep learning system based on MAPE-D annular structure | |
| CN115496948B (en) | A network-supervised fine-grained image recognition method and system based on deep learning | |
| CN115294381B (en) | Small sample image classification method and device based on feature transfer and orthogonal prior | |
| CN113887636B (en) | Selectable data enhancement method and system based on genetic algorithm | |
| CN116977870B (en) | SAR target increment identification method based on dynamic structure and multi-level distillation | |
| JP6993250B2 (en) | Content feature extractor, method, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |