CN111401203A - Target identification method based on multi-dimensional image fusion - Google Patents

Abstract

The invention belongs to the technical field of artificial intelligence and computer vision, and particularly relates to a target identification method based on multi-dimensional image fusion. According to the fusion algorithm of the Laplacian pyramid decomposition structure based on multi-resolution analysis, correlation between imaging characteristics and image features of different sensors is researched, and the fusion algorithm of the Laplacian pyramid decomposition structure based on multi-resolution analysis is realized. By using pyramidal decomposition of the image, objects of different sizes in the image can be analyzed. Meanwhile, information obtained by analyzing the lower layer with high resolution can be used for guiding the analysis of the upper layer with low resolution, so that the analysis and calculation can be greatly simplified. Because the image fusion method of the invention accords with the natural reality condition in the complex environment of the battlefield, compared with other existing image fusion methods, the invention has the characteristics of good fusion effect, rich details and the like.

Description

Target recognition method based on multi-dimensional image fusion

technical field

The invention belongs to the technical field of artificial intelligence and computer vision, and in particular relates to a target recognition method based on multi-dimensional image fusion.

Background technique

Image target recognition is to realize image recognition by comparing the stored target information with the current image information. The description of the image is the premise of target recognition. By using numbers or symbols to represent the relevant features of each target in the image or scene, and even the relationship between the targets, the abstract expression of the target features and the relationship between them is finally obtained. Image recognition technology can use template matching model when extracting individual features in images. In some specific applications, image recognition not only needs to give what object the recognized object is, but also needs to give the position of the object. At present, image recognition technology has been widely used in many fields, such as biomedicine, satellite remote sensing, robot vision, cargo detection, target tracking, autonomous vehicle navigation, public security, banking, transportation, military, e-commerce and multimedia network communication. With the rapid development of artificial intelligence and computer vision technology, target recognition based on machine vision and target recognition based on deep learning have appeared, which greatly improves the accuracy and efficiency of image recognition.

However, the image information obtained by a single-band sensor has shortcomings. For example, visible light images are rich in details, but cannot be imaged at night or when the light is weak; infrared images can be imaged 24 hours a day, but the temperature distribution of objects can be obtained, and details cannot be observed. Using image fusion methods, the multi-band information of a single sensor or the information provided by different types of sensors can be integrated to eliminate the possible redundancy and contradiction between the multi-sensor information, so as to enhance the transparency of the information in the image and improve the accuracy of interpretation , reliability, and usage to form a clear, complete, and accurate description of the target. The efficient image fusion method can comprehensively process the information of multi-source channels according to the needs, so as to effectively improve the utilization rate of image information, the reliability of the system's target recognition and the degree of automation of the system.

In systems such as unmanned reconnaissance aircraft, vehicle-mounted panoramic situational awareness, and shipborne photoelectric search and tracking, the target recognition technology based on multi-dimensional image fusion can meet many needs of military photoelectric systems, and solve the ability to automate and intelligently perceive external scenes. At the same time, the target recognition technology based on multi-dimensional image fusion is also widely used in aerial survey and industrial survey in the civil field, so it can bring huge social and economic benefits to my country's military and reconnaissance fields.

In recent years, many scholars have carried out research on target recognition methods, but at present, the method of single-band sensor image is mainly used for target recognition. The Chinese journal "Command and Control and Simulation" 2019, Vol.28, No.1, pp.1-5 published a paper entitled "Recognition of Image Targets in Naval Battlefields Based on Deep Learning". In this paper, the advantages and disadvantages of the R-CNN series models based on regional proposal and the YOLO model based on regression are analyzed, and the application status of deep learning technology in image target recognition in naval battlefields is sorted out. It can be seen that the traditional target recognition algorithm cannot identify the effective target in the image under the condition of low signal-to-noise ratio. Therefore, to achieve effective image recognition, it is necessary to research and find more effective, accurate and real-time technical approaches.

SUMMARY OF THE INVENTION

(1) Technical problems to be solved

The technical problem to be solved by the present invention is: how to provide a target recognition method based on multi-dimensional image fusion for unmanned systems in order to meet the target recognition requirements in complex environments.

(2) Technical solutions

In order to solve the above technical problems, the present invention provides a target recognition method based on multi-dimensional image fusion, the method comprising:

Step 1: Preprocess the image; including:

Step 11: Calculate the relative parameter transformation matrix of the image;

After receiving the identification command issued by the unmanned reconnaissance system device, the visible light image is collected as the reference image g _B through the corresponding sensor, and the infrared image is used as the candidate image g _c ;

Assuming that the number of matching points is N, the number of N is at least 3, and assuming that N=3, select 3 feature points in the reference image, namely B ₁ , B ₂ and B ₃ ; select the corresponding feature points in the candidate image g _c 3 matching points, namely C ₁ , C ₂ and C ₃ ;

For the two multi-source images of the reference image g _B and the candidate image g _C , a total of N pairs of matching points are found; then the relative parameter transformation matrix P _C←B between the reference image g _B and the candidate image g _C adopts the following least squares method Formula calculation:

P _C←B =C*B ^T *(B*B ^T ) ^-1

Among them, C represents the odd coordinate of the matching point in the candidate image coordinate system, C is a 3×N matrix, B represents the odd coordinate of the matching point in the reference image coordinate system, B is a 3×N matrix, B ^T is B The transpose matrix of , the relative parameter transformation matrix P _C←B is a 3×3 matrix;

Step 12: Transform the candidate image into the reference image coordinate system;

Using inverse mapping, starting from the reference image g _B , the corresponding position of each pixel position on the reference image g _B on the candidate image g _C is obtained through the transformation function; according to the odd coordinate of each point B ⁰ in the reference image g _B , the odd-order coordinates of the corresponding point C ⁰ in the candidate image g _C can be calculated according to the following formula:

其取值范围为(1,2,…,W)；垂直坐标为

其取值范围为(1,2,…,H)；C⁰的水平坐标为

其取值范围为(1,2,…,W)；垂直坐标为

其取值范围为(1,2,…,H)；Among them, the horizontal coordinate of B ⁰ is

Its value range is (1,2,…,W); the vertical coordinate is

Its value range is (1,2,…,H); the horizontal coordinate of C ⁰ is

Its value range is (1,2,…,W); the vertical coordinate is

Its value range is (1,2,…,H);

After assigning the pixel gray value of point C ⁰ in the candidate image g _C to the corresponding pixel point B ⁰ of the reference image g _B , the transformed candidate image g _B←C is obtained, and the image g _B←C is used as the output and the reference image. g _B for fusion;

Step 2: Image fusion based on multi-source features; including:

The reference image g _B is fused with the transformed candidate image g _B←C , and the image fusion algorithm based on the pyramid decomposition structure of multi-resolution analysis is adopted. The steps are as follows:

Step 21: Perform Gauss tower decomposition of the image;

For the reference image g _B as the source image, with G ₀ as the zero layer of the Gaussian pyramid, the image G _l of the l-th layer of the Gaussian pyramid is:

In the formula: N1 is the top layer number of the Gauss pyramid; C _l is the number of columns of the image of the first layer of the Gauss pyramid; R _l is the number of rows of the image of the first layer of the Gauss pyramid; w(m,n) represents the 5×5 window function, It has low-pass characteristics, as follows:

Step 22: Build the Laplace pyramid of the image;

in:

是由G_l内插放大得到的图像，其尺寸与G_l-1的尺寸相同，但

与G_l-1并不相等，在原有像素间内插的新像素的灰度值是通过对原有像素灰度值的加权平均确定的；由于G_l是对G_l-1低通滤波得到的，即G_l是模糊化、降采样的G_l-1，因此

的细节信息比G_l-1的少；In the formula,

is the image enlarged by the interpolation of G _l , and its size is the same as that of G _l-1 , but

It is not equal to G _l-1 , the gray value of the new pixel interpolated between the original pixels is determined by the weighted average of the original pixel gray value; since G _l is obtained by low-pass filtering of G _l-1 , that is, G _l is the blurred, down-sampled G _l-1 , so

The detailed information is less than that of G _l-1 ;

From this, the decomposed image LP _l of each layer of the Laplace pyramid is obtained:

In the formula: N2 is the layer number of the top layer of the Laplace pyramid; LP _l represents the l-th layer image decomposed by the Laplace pyramid;

Step 23: Reconstruct the source image by the Laplace pyramid;

From the above transformation, we can get

Step 24: Image fusion based on Laplace pyramid decomposition;

Let A and B be two source images, and F be the fused image. The fusion process is as follows:

Step 241: Perform Laplace pyramid decomposition on each source image to establish its own Laplace pyramid;

Step 242: Perform fusion processing on each decomposition layer of the image pyramid to obtain the Laplace pyramid of the fused image;

Step 243: Perform image reconstruction on the fused Laplace pyramid to obtain a final fused image;

Step 3: carry out target identification; this step includes:

Step 31: Perform big data annotation;

For the M collected big data images, use the labeling tool to select a rectangular area, define the label of the background area as 0, and define the label of the target area as 1; the classification constitutes a training set of a certain scale for training deep learning models. and the validation set to realize the recognition of class 1 targets; the number of M is at least 12000;

Step 32: perform data training;

Use the data set marked in the previous step to train the target classification model;

Step 33: perform real-time image fusion;

Collect visible light images and infrared images in real time, and fuse them to obtain fused images;

Step 34: Carry out target identification and positioning;

Use the trained classification model to detect ship targets on the fusion image obtained in the previous step, record the size and position of all ship targets identified by the classification model, and use a rectangular identification frame to mark this rectangular area on the image. identified.

(3) Beneficial effects

Compared with the prior art, the present invention has the following beneficial effects:

(1) The present invention proposes a method for fusion based on multi-dimensional image information. By studying the imaging characteristics of different sensors and the correlation between image features, the fusion algorithm of Laplacian pyramid decomposition structure based on multi-resolution analysis is realized. Using the pyramid decomposition of the image, it is possible to analyze objects of different sizes in the image, for example, a high-resolution layer (lower layer) can be used to analyze details, and a low-resolution layer (top layer) can be used to analyze larger objects. At the same time, the information obtained by analyzing the high-resolution lower layer may also be used to guide the analysis of the low-resolution upper layer, which can greatly simplify the analysis and calculation. The tower decomposition of images provides a convenient and flexible method for multi-resolution analysis of images. The Laplacian decomposition of images can decompose the importance of images (such as edges) into different tower decomposition layers according to different scales. superior. Since the image fusion method of the present invention conforms to the natural reality conditions in the complex environment of the battlefield, the present invention has the characteristics of good fusion effect and rich details compared with other existing image fusion methods.

(2) In the present invention, the convolutional neural network based on deep learning is used for target recognition on the fused image, which has the characteristics of accurate recognition, strong ability to resist scale and illumination changes, and the like.

(3) In the present invention, feature points are used for heterologous image registration. And the least square method is used to calculate the image transformation parameters, which has the advantages of high calculation accuracy, fast speed, and good fusion recognition effect.

Description of drawings

Figure 1(a)-Figure 1(d) are the results of multi-dimensional image fusion experiments. Wherein, Figure 1(a) is the startup interface of the multi-dimensional image fusion software; Figure 1(b) is the interface after the multi-dimensional image fusion software loads the image and video; Figure 1(c) is the visible light and infrared image registration selection interface; Figure 1(d) is the result of multi-dimensional image fusion.

FIG. 2 is an operation flow chart of the target recognition method based on multi-dimensional image fusion of the present invention.

FIG. 3 is a schematic diagram of the image fusion based on pyramid decomposition of the present invention.

FIG. 4(a) and FIG. 4(b) are diagrams of experimental results of target recognition on ship target images and videos according to the preferred embodiment of the present invention.

Detailed ways

In order to make the purpose, content, and advantages of the present invention clearer, the specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

In order to meet the target recognition requirements in complex environments, the present invention provides a target recognition method based on multi-dimensional image fusion for unmanned systems. ability.

The main task of the present invention is to use the image fusion method to fuse the multi-band video image sequence, and finally to intelligently identify the target by using the deep learning method. It can be seen that the video image sequence is the object to be processed by the present invention. The target recognition technology based on multi-dimensional image fusion is to label, train and learn the multi-band fusion image big data to automatically identify multiple targets in the image.

The image acquisition device of the present invention adopts the visible light camera of Quanrui Video, and the infrared adopts the EXA-IR2 uncooled thermal imager. The computer hardware in this preferred embodiment adopts the I7-7700 processor, the main frequency is 2.80G, and the size of the hard disk is 1T. It only takes about 0.1 second to use the computer to calculate the target recognition algorithm for one frame of image. Collect 1 frame of visible light image and 1 frame of infrared image for image fusion operation. It is assumed that the visible light image is the reference image and the infrared image is the candidate image. The basic process of image fusion technology is to first register the reference image and the candidate image to establish a mathematical transformation model of the two images; then according to the established mathematical transformation model, a unified coordinate transformation is performed, that is, all candidate image sequences are transformed to the benchmark. In the coordinate system of the image, the fused image is formed. For two images, due to the existence of scale, rotation and translation transformation, the transformation relationship needs at least 3 pairs of matching points to solve.

The target recognition method based on multi-dimensional image fusion provided by the preferred embodiment of the present invention completes the real-time recognition of images according to the workflow shown in FIG. 2 , and the recognition process includes the following four major parts.

Specifically, the target recognition method based on multi-dimensional image fusion provided by the present invention includes:

Step 1: Preprocess the image; including:

Step 11: Calculate the relative parameter transformation matrix of the image;

After receiving the identification command issued by the unmanned reconnaissance system device, the visible light image is collected as the reference image g _B through the corresponding sensor, and the infrared image is used as the candidate image g _c ;

Assuming that the number of matching points is N, the number of N is at least 3, and assuming that N=3, select 3 feature points in the reference image, namely B ₁ , B ₂ and B ₃ ; select the corresponding feature points in the candidate image g _c 3 matching points, namely C ₁ , C ₂ and C ₃ ;

For the two multi-source images of the reference image g _B and the candidate image g _C , a total of N pairs of matching points are found; if N≥3, the relative parameter transformation matrix P _C←B between the reference image g _B and the candidate image g _C Calculated using the following least squares formula:

P _C←B =C*B ^T *(B*B ^T ) ^-1

Among them, C represents the odd coordinate of the matching point in the candidate image coordinate system, C is a 3×N matrix, B represents the odd coordinate of the matching point in the reference image coordinate system, B is a 3×N matrix, B ^T is B The transpose matrix of , the relative parameter transformation matrix P _C←B is a 3×3 matrix;

Step 12: Transform the candidate image into the reference image coordinate system;

Since there is a certain transformation relationship between the reference image g _B and the candidate image g _C , it is necessary to transform them into the same coordinate system for image fusion. In the present invention, the candidate image g _C needs to be transformed into the reference image g _B coordinate system.

Using inverse mapping, starting from the reference image g _B , the corresponding position of each pixel position on the reference image g _B on the candidate image g _C is obtained through the transformation function; according to the odd coordinate of each point B ⁰ in the reference image g _B , the odd-order coordinates of the corresponding point C ⁰ in the candidate image g _C can be calculated according to the following formula:

其取值范围为(1,2,…,W)；垂直坐标为

其取值范围为(1,2,…,H)；C⁰的水平坐标为

其取值范围为(1,2,…,W)；垂直坐标为

其取值范围为(1,2,…,H)；Among them, the horizontal coordinate of B ⁰ is

Its value range is (1,2,…,W); the vertical coordinate is

Its value range is (1,2,…,H); the horizontal coordinate of C ⁰ is

Its value range is (1,2,…,W); the vertical coordinate is

Its value range is (1,2,…,H);

After assigning the pixel gray value g _c (i,j) of the point C ⁰ in the candidate image g _C to the corresponding pixel point B ⁰ of the reference image g _B , the transformed candidate image g _B←C is obtained, and the image g _B is _←C is used as the output to fuse with the reference image g _B ;

Usually image transformation can use two mapping methods: forward mapping and reverse mapping; forward mapping is to transform the candidate image to the coordinate space where the reference image is located according to the calculated image transformation parameters; that is, scan every pixel of the candidate image. , through the transformation function, calculate the position of each pixel corresponding to the reference image in turn. When two adjacent pixels of the candidate image are mapped to two non-adjacent pixels of the reference image, discrete marseille and void voids will appear. Therefore, it is necessary to change the way of thinking, and reverse thinking can be used to find the coordinates of the corresponding candidate image for each point of the reference image. The inverse mapping starts from the reference image g _B , and uses the transformation function to solve the corresponding position of each pixel position on the reference image g _B on the candidate image g _C. First scan each pixel position of the reference image g _B , and then calculate the corresponding sampling pixel point on the candidate image g _C according to the transformation function, and assign the gray value of this point to the corresponding pixel point of the reference image g _B.

The effect of inverse mapping is better than that of forward mapping, because each pixel of the reference image can be scanned to obtain the appropriate gray value, thus avoiding the appearance of virtual points in the forward mapping that some points of the output image may not be assigned. The case of voids and mosaics.

Step 2: Image fusion based on multi-source features; including:

The reference image g _B is fused with the transformed candidate image g _B←C , and the image fusion algorithm based on the pyramid decomposition structure of multi-resolution analysis is adopted. The steps are as follows:

Step 21: Perform Gauss tower decomposition of the image;

For the reference image g _B as the source image, with G ₀ as the zero layer (bottom layer) of the Gaussian pyramid, the image G _l of the l-th layer of the Gaussian pyramid is:

In the formula: N1 is the top layer number of the Gauss pyramid; C _l is the number of columns of the image of the first layer of the Gauss pyramid; R _l is the number of rows of the image of the first layer of the Gauss pyramid; w(m,n) represents the 5×5 window function ( generation kernel), with low-pass characteristics, as follows:

Step 22: Build the Laplace pyramid of the image;

in:

是由G_l内插放大得到的图像，其尺寸与G_l-1的尺寸相同，但

与G_l-1并不相等，在原有像素间内插的新像素的灰度值是通过对原有像素灰度值的加权平均确定的；由于G_l是对G_l-1低通滤波得到的，即G_l是模糊化、降采样的G_l-1，因此

的细节信息比G_l-1的少；In the formula,

is the image enlarged by the interpolation of G _l , and its size is the same as that of G _l-1 , but

It is not equal to G _l-1 , the gray value of the new pixel interpolated between the original pixels is determined by the weighted average of the original pixel gray value; since G _l is obtained by low-pass filtering of G _l-1 , that is, G _l is the blurred, down-sampled G _l-1 , so

The detailed information is less than that of G _l-1 ;

From this, the decomposed image LP _l of each layer of the Laplace pyramid is obtained:

In the formula: N2 is the layer number of the top layer of the Laplace pyramid; LP _l represents the l-th layer image decomposed by the Laplace pyramid;

Step 23: Reconstruct the source image by the Laplace pyramid;

From the above transformation, we can get

Step 24: Image fusion based on Laplace pyramid decomposition;

The image fusion method based on Laplace pyramid decomposition is shown in Figure 3; let A and B be two source images, F is the fused image, and the fusion process is as follows:

Step 241: Perform Laplace pyramid decomposition on each source image to establish its own Laplace pyramid;

Step 242: Perform fusion processing on each decomposition layer of the image pyramid to obtain the Laplace pyramid of the fused image;

Step 243: Perform image reconstruction on the fused Laplace pyramid to obtain a final fused image;

Step 3: carry out target identification; this step includes:

Step 31: Perform big data annotation;

For the M collected big data images, the labeling tool is used to select the rectangular area, the label of the background area is defined as 0, and the label of the area defining the ship target is 1; the classification constitutes a certain scale for training the deep learning model. Training set and validation set to realize the recognition of Class 1 ship targets; the number of M is at least 12,000;

Step 32: perform data training;

Use the data set marked in the previous step to train the ship target classification model;

Step 33: perform real-time image fusion;

Collect visible light images and infrared images in real time, and fuse them to obtain fused images;

Step 34: Carry out target identification and positioning;

Use the trained classification model to detect ship targets on the fusion image obtained in the previous step, record the size and position of all ship targets identified by the classification model, and use a rectangular identification frame to mark this rectangular area on the image. identified.

Figures 4(a) and 4(b) show the experimental results of target recognition based on multi-dimensional image fusion using this preferred embodiment. There are two ship targets in Figure 4(a) and one ship target in Figure 4(b). It can be seen that since the present invention adopts the target recognition method based on multi-dimensional image fusion, it has a better target recognition effect.

The above are only the preferred embodiments of the present invention. It should be pointed out that for those skilled in the art, without departing from the technical principle of the present invention, several improvements and modifications can also be made. These improvements and modifications It should also be regarded as the protection scope of the present invention.

Claims (1)

1. A target identification method based on multi-dimensional image fusion is characterized by comprising the following steps:

step 1: preprocessing the image; the method comprises the following steps:

step 11: calculating an image relative parameter transformation matrix;

when receiving an identification command sent by the unmanned reconnaissance system device, acquiring a visible light image as a reference image g through a corresponding sensor_BInfrared image as candidate image g_c；

Assuming that the number of matching points is N, the number of N is at least 3, and assuming that N is 3, 3 feature points, each B, are selected from the reference image₁，B₂And B₃(ii) a In the candidate image g_cIn the method, 3 corresponding matching points are selected, and are respectively C₁，C₂And C₃；

For reference image g_BAnd candidate image g_CN pairs of matching points are found in the two multi-source images; reference image g_BAnd candidate image g_CRelative parameter transformation matrix P between_C←BThe following least square method formula is adopted for calculation:

P_C←B＝C*B^T*(B*B^T)^-1

where C denotes odd coordinates of a matching point in the candidate image coordinate system, C is a 3 × N matrix, B denotes odd coordinates of a matching point in the reference image coordinate system, B is a 3 × N matrix, and B is^TFor the transpose matrix of B, the relative parameter transform matrix P_C←BIs 3 × 3 matrix;

step 12: transforming the candidate image into a reference image coordinate system;

from the reference image g, using inverse mapping_BStarting from this, the reference image g is solved by a transformation function_BOn each pixel point in the candidate image g_CA corresponding position on; from the reference image g_BIn each point B⁰The odd coordinates of (g) can be calculated according to the following formula_CPoint C corresponding thereto⁰Odd-order coordinates of (c):

wherein, B⁰Has a horizontal coordinate of

The value range is (1,2, …, W); vertical coordinate is

The value range is (1,2, …, H); c⁰Has a horizontal coordinate of

The value range is (1,2, …, W); vertical coordinate is

The value range is (1,2, …, H);

candidate image g_CMidpoint C⁰Giving the reference image g a pixel gray value of_BCorresponding pixel point B⁰Then, a transformed candidate image g is obtained_B←CAnd combine the image g_B←CAs output and reference image g_BCarrying out fusion;

step 2: performing image fusion on the image based on the multi-source characteristics; the method comprises the following steps:

a reference image g_BAnd the transformed candidate image g_B←CThe fusion is carried out, and an image fusion algorithm of a pyramid decomposition structure based on multi-resolution analysis is adopted, and the method comprises the following steps:

step 21: carrying out Gauss tower type decomposition on the image;

for a reference image g as a source image_BIn the order of G₀As the zero layer of the Gaussian pyramid, the image G of the first layer of the Gaussian pyramid_lComprises the following steps:

0≤l≤N1，0≤i<C_l，0≤j<R_l；

in the formula: n1 is the top layer number of Gauss pyramid; c_lThe number of columns of the image of the l layer of the Gauss pyramid is obtained; r_lW (m, n) represents a 5 × 5 window function, and has a low-pass characteristic, which is as follows:

step 22, establishing L aplace pyramid of the image;

0≤l≤N2，0≤i<C_l，0≤j<R_l；

wherein:

in the formula (I), the compound is shown in the specification,

is composed of G_lInterpolated and enlarged image, size and G_l-1Are the same size, but

And G_l-1The values of the gray values of the new pixels interpolated between the original pixels are determined by a weighted average of the gray values of the original pixels; due to G_lIs to G_l-1Obtained by low-pass filtering, i.e. G_lIs fuzzification, down-sampling G_l-1Thus, therefore, it is

Ratio of detail information of G_l-1Less;

thus, a decomposed image L P of L aplace pyramid layers was obtained_l：

Wherein N2 is layer number of L aplace pyramid top layer, L P_lA layer i image representing L aplace pyramid decomposition;

step 23, reconstructing a source image by L aplace pyramid;

by transformation of the above formula

Step 24, fusing images based on L aplace pyramid decomposition;

and setting A and B as two source images and F as a fused image, wherein the fusion process is as follows:

241, performing L aplace pyramid decomposition on each source image to establish respective L aplace pyramids;

242, respectively fusing all decomposition layers of the image pyramid to obtain an L aplace pyramid of the fused image;

step 243, carrying out image reconstruction on the fused L aplace pyramid to obtain a final fused image;

and step 3: carrying out target identification; the method comprises the following steps:

step 31: carrying out big data annotation;

selecting a rectangular area by using a marking tool for M collected big data images, defining the label of a background area as 0, and defining the label of a target area as 1; classifying to form a training set and a verification set which have a certain scale and are used for training the deep learning model, and realizing the identification of the class 1 target; the number of M is at least 12000;

step 32: carrying out data training;

training a target classification model by using the data set marked in the previous step;

step 33: performing real-time image fusion;

collecting a visible light image and an infrared image in real time, and fusing to obtain a fused image;

step 34: carrying out target identification and positioning;

and carrying out ship target detection on the fusion image obtained in the last step by using the trained classification model, recording the size and the position of all ship targets identified by the classification model, and identifying the rectangular area on the image by using a rectangular identification frame.

Images