CN108681691A - A kind of marine ships and light boats rapid detection method based on unmanned water surface ship - Google Patents

Abstract

The invention provides a rapid detection method for marine ships based on unmanned surface ships. Ship detection at sea is one of the most important tasks of the vision system of unmanned surface vehicle (USV). The invention firstly extracts the edge information of the image, and establishes a "target" scoring function to obtain target candidate frames. Then, the sea-antenna in the sea-sky environment image is detected, and the target candidate frame is screened again based on the sea-antenna prediction. Then, the ship target is modeled with Histograms of Oriented Gradient (HOG) features, and the classifier is iteratively trained using the "bootstrap method" using the support vector machine. Finally, the feature descriptor of the target candidate box is input into the classifier for ship detection. Compared with the traditional detection method, the detection method provided by the present invention can detect the marine ship target more quickly and accurately, has a higher detection rate, and has stronger robustness to changes in scale and illumination conditions.

Description

A rapid detection method for marine vessels based on unmanned surface vehicles

technical field

The invention relates to an unmanned surface vehicle target detection technology, in particular to a rapid detection method for a marine vessel based on an unmanned surface vehicle.

Background technique

Unmanned surface vehicle (USV) is a new type of maritime intelligence, which can be used to perform military tasks such as reconnaissance, anti-submarine, and patrol, as well as civilian tasks such as search and rescue, navigation, and hydrographic survey. Among them, the function of the vision system of the unmanned surface vehicle is to replace the human eye to detect, track and measure the targets and obstacles at sea, and to understand the scene and behavior. Vision-based detection of marine vessels is one of the most important tasks of the vision system of unmanned surface vehicles, and it is the basis for realizing the recognition and tracking of marine vessels by unmanned surface vehicles. Therefore, it is of great significance to study the feature model and target detection method of marine vessels for the development of unmanned surface vehicles.

Traditional target detection algorithms have one thing in common, they all use a "sliding window" search strategy. This strategy is to detect the position of the object in the image by sliding the classifier through each window position of the image. The number of sliding windows is linearly related to the detection scale of the classifier. At a single scale, about 10 ⁴ -10 ⁵ windows need to be tested for each image classifier, and at multiple scales, the number of test windows will increase by several orders of magnitude. In addition, today's detectors also require to predict the aspect ratio of the object, so the number of test windows will reach 10 ⁶ -10 ⁷ . Obviously, this "exhaustive" detection method will generate many redundant windows, resulting in a large amount of calculation and very time-consuming. Therefore, many target detection systems using this method will choose some simpler classifiers. These simple classifiers often use weaker feature models to obtain faster calculation speeds to make up for the disadvantages of sliding window search strategies. However, although the use of weak feature models improves the calculation speed of the classifier, it loses the detection rate and detection accuracy.

Through the above analysis, if the detection algorithms of these sliding window search strategies are directly transplanted into the unmanned surface vehicle visual detection system for ship detection, and a simple classifier is used to make up for the computational efficiency, then the detection rate and detection rate of the ship will Accuracy will be greatly reduced, while more false positives will be generated. If a more complex classifier is used, although it can improve the detection rate and reduce false positives, the "exhaustive" feature of the sliding window search strategy will make the entire detection process very time-consuming. In this way, the position information of the ship detected by the unmanned surface vehicle will lose its timeliness, which will have a certain impact on the execution of subsequent tasks.

Contents of the invention

The purpose of the invention is to aim at the defects in the prior art and the characteristics of the water surface, to provide a rapid detection method for marine vessels based on unmanned surface vehicles, which can detect targets of marine vessels more quickly and accurately, and has a higher detection rate, It is also robust to changes in scale and lighting conditions.

To achieve the above object, the present invention adopts the following technical solutions:

A method for rapid detection of ships at sea based on unmanned surface vehicles, which is operated by using the vision system of the unmanned surface vessel, the vision system includes a camera, an image acquisition card, and an industrial computer; wherein the camera is installed directly above the unmanned surface vessel , the industrial computer is installed and fixed in the cabin of the unmanned surface boat, the camera is connected to the industrial computer through the IEEE 1394 interface, and the image acquisition card is connected to the industrial computer through the PCI card slot. The operation steps of this method are as follows:

(1) Downsampling is performed on the image collected by the camera to obtain a downsampled image;

(2) Utilize the edge detector to obtain the edge response of each pixel in the original image, and combine these edge responses together to obtain the edge map of the original image;

(3) Establish a "target" scoring function to screen target candidate boxes in the edge map;

(4) Detect the sea antenna, and re-screen the target candidate frame based on the sea antenna prediction;

(5) Carry out direction gradient histogram and HOG feature modeling to ship target, obtain a complex feature vector with 5796 dimensions;

(6) Utilize the support vector machine and use the "bootstrap method" to iteratively train the classifier;

(7) Input the feature descriptors of the screened target candidate boxes into the classifier for ship detection, and output the position information of the ship in the image if there is a ship.

Compared with prior art, the beneficial effect of the present invention is:

The detection speed of the method provided by the invention can be increased by several orders of magnitude, so the marine ship target can be detected more quickly and accurately, and at the same time, it has a higher detection rate. In addition, it is also robust to changes in scale and lighting conditions.

Description of drawings

Other characteristics, objects and advantages of the present invention will become more apparent by reading the detailed description of non-limiting examples with reference to the following drawings:

Fig. 1 is a schematic diagram of the system structure of the present invention;

Fig. 2 is a block diagram of the present invention;

Fig. 3 is a schematic diagram of obtaining target candidate frames in the present invention;

Fig. 4 is a schematic diagram of the present invention based on sea antenna screening target candidate frames;

Fig. 5 is a flow chart of classifier training in the present invention.

Detailed ways

The technical solutions of the present invention will be described in detail below, but the protection scope of the present invention is not limited to the embodiments.

As shown in Figure 1, the present invention uses the vision system of the unmanned surface vehicle to operate, and the vision system includes equipment such as a camera, an image acquisition card, and an industrial computer. Among them, the camera is installed directly above the unmanned surface vehicle at a position about 1.5 meters away from the bow of the vessel, and the industrial computer is installed and fixed in the cabin of the unmanned surface vessel. The camera is connected to the industrial computer through the IEEE 1394 interface, and the image acquisition card is connected to the industrial computer through the PCI card slot.

As shown in Fig. 2, the present invention provides a method for rapid detection of marine vessels based on unmanned surface vehicles. First, the edge information of the image is extracted, and the "targetness" scoring function is established to obtain the target candidate box. Then, the sea-antenna in the sea-sky environment image is detected, and the target candidate frame is screened again based on the sea-antenna prediction. Then, the ship target is modeled with Histograms of Oriented Gradient (HOG) features, and the support vector machine (Support Vector Machine, SVM) is used to iteratively train the classifier with the "bootstrap method". Finally, the feature descriptor of the target candidate box is input into the classifier for ship detection. Compared with the traditional detection method, the detection method provided by the present invention can detect the marine ship target more quickly and accurately, has a higher detection rate, and has stronger robustness to changes in scale and illumination conditions.

The above-mentioned rapid detection method for ships at sea based on unmanned surface vehicles specifically comprises the following steps:

(1) down-sampling the image collected by the camera, and down-sampling the collected image of 1440×1080 pixels to 640×480 pixels for processing;

(2) As shown in Figure 3, the edge detector is used to obtain the edge response of each pixel in the original image, and these edge responses are combined to obtain the edge map of the original image, so that the directly obtained edge map is relatively close, By performing a Non-Maximal Suppression (NMS) to obtain the local maximum of the edge response, a relatively sparse edge map is obtained, as shown in Figure 3(b).

(3) Establish a "targetness" scoring function to screen target candidate boxes in the edge map. The specific steps are:

Given a set s _i ∈ S of edge groups, compute the similarity between each pair of adjacent edge groups. For edge groups s _i and s _j , the calculation formula of their similarity a(s _i , s _j ) is as follows:

a(s _i ,s _j )＝|cos(θ _i -θ _ij )cos(θ _j -θ _ij )| ^γ

where _{θi and θj are the average orientations of two edge groups, and θij is the} angle between their average positions _xi and _xj . The γ value is used to control the sensitivity of the direction to the similarity, and γ=2 is taken in this method.

Given a set S of edge groups and calculating the similarity between them, the candidate bounding box b is scored by establishing a scoring function. Calculate the sum of the edge values of all pixels p in the edge group _si , denoted as m _i . Select the position of any pixel in the edge group s _i , denoted as

For each edge group s _i , a continuous value w _b (s _i )∈[0,1] is calculated to measure whether s _i is completely contained in the bounding box b. The calculation formula of w _b (s _i ) is as follows:

In the formula, T is an ordered path starting from t ₁ ∈ S _b and ending at t _|T| = _si , and a(t _j ,t _j+1 ) is the similarity between edge groups. If no such path exists, let w _b (s _i )=1.

Using the calculated w _b (s _i ), the scoring formula for the bounding box b is as follows:

In the formula, b _w and b _h are the width and height of the bounding box respectively, w _b (s _i )∈[0,1] measures whether s _i is completely contained in the bounding box b, m _i is the edge group s _i The sum of the edge values of all pixels p. Since larger bounding boxes will contain more edges, our method takes κ = 1.5 to offset this bias.

Since edges inside the bounding box are less important than those near the bounding box. The scoring formula is improved, and the edge value inside the bounding box is subtracted from the scoring h _b . The improved scoring formula is as follows:

In the formula, b _w and b _h are the width and height of the bounding box respectively, the width and height of bin ⁱⁿ are b _w /2 and b _h /2 respectively, m _p is the edge value of each pixel p in the edge map, Similarly, take κ=1.5. Finally, 1000 bounding boxes with larger scores are selected as object candidates.

(4) As shown in Figure 4, the sea antenna divides the image into three regions: the sky region, the sea surface region and the sea antenna region. When a ship sails on the sea, it will only be in the sea surface area and the sea line area, and will not be in the sky area. Based on such a characteristic, the present invention further improves the target candidate frame generation method, screens again 1000 target candidate frames with higher scores, and eliminates the target candidate frames that are completely above the sea antenna area, as shown in Figure 4(a) Shown in red box. The target candidate boxes under and intersecting the sea antenna are reserved, as shown by the green box in Fig. 4(a). By performing a simple Hough change detection sea antenna on the edge map, the target candidate box is screened again based on the above sea antenna prediction.

(5) The present invention carries out directional gradient histogram (Histograms of OrientedGradient, HOG) feature modeling to the boat target, according to the shape characteristics of the boat, the aspect ratio of the boat feature model is designed to be 3:1, and the feature window The size is designed to be 192×64 pixels. The cell size of the HOG feature is designed to be 8×8 pixels. The number of histogram channels per cell was set to 9. In this way, the calculation formula of the feature dimension V of the HOG feature descriptor of the ship is as follows:

(6) As shown in FIG. 5 , the present invention utilizes a support vector machine (Support Vector Machine, SVM) with a linear kernel, and adopts a "bootstrap method" to iteratively train a classifier. The specific training steps are as follows: First, the initial positive samples are composed of all the ground truth boxes (Ground Truth) of the boat, and the total number is 2000. Then, object candidate boxes with 20%-50% overlapping area with the ground-truth boxes are selected as the initial negative samples. In order to avoid selecting similar duplicate negative samples, two negative samples with an overlapping area of more than 70% are selected and discarded. Finally, 10,000 negative samples are randomly selected from all negative samples as negative samples for SVM training. After obtaining the initial classifier, a retraining process is required. The initial classifier is used to detect the ship target on the original image of the negative sample (not including the ship target), so that all the detected rectangular boxes belong to false positives (FalsePositives). These false positive rectangles are a hard example for the classifier (Hard Example). Save these difficult examples as images, add them to the initial negative sample set, and retrain the classifier. In this way, the classifier obtained through retraining has better classification ability, that is, the ability to detect ship targets. The retraining process can be carried out iteratively until the performance of the classifier is not significantly improved.

(7) Input the feature descriptors of the filtered target candidate boxes into the classifier for ship detection. If there is a boat, then output the location information of the boat in the image.

Claims (5)

1. a kind of marine ships and light boats rapid detection method based on unmanned water surface ship, is grasped using unmanned water surface ship vision system Make, which includes camera, image pick-up card, industrial personal computer；Wherein, camera be mounted on unmanned water surface ship just on Side, industrial personal computer are fixed in the ship cabin of unmanned water surface ship, and camera is connected to industrial personal computer by 1394 interfaces of IEEE On, image pick-up card is connected to by pci card slot on industrial personal computer, which is characterized in that this method operating procedure is as follows：

(1) down-sampled to the image progress of camera acquisition, obtain down-sampled image；

(2) edge detector is utilized, the skirt response of each pixel in original image is obtained, these skirt responses combination is existed The edge graph of original image is obtained together；

(3) " Objective " score function is established, target candidate frame is screened in edge graph；

(4) sea horizon is detected, target candidate frame is screened based on sea horizon anticipation again；

(5) histograms of oriented gradients is carried out to ships and light boats target, HOG feature modelings obtain a complicated spy with 5796 dimensions Sign vector；

(6) support vector machines is utilized, using " boot strap " repetitive exercise grader；

(7) Feature Descriptor of the target candidate frame after screening is input in grader, ships and light boats detection is carried out, if there is ship Ship then exports the location information of ships and light boats in the picture.

2. a kind of marine ships and light boats rapid detection method based on unmanned water surface ship according to claim 1, it is characterised in that： " Objective " score function is established in the step (3), target candidate frame is screened in edge graph is specially：

The set s of given edge group_i∈ S calculate the similarity between each pair of neighboring edge group, for edge group s_iWith s_j, the similarity a (s between them_i,s_j) calculation formula is as follows：

a(s_i,s_j)=| cos (θ_i-θ_ij)cos(θ_j-θ_ij)|^γ

In formula, θ_iAnd θ_jIt is the mean direction of the edges Liang Ge group, θ_ijIt is their mean place x_iAnd x_jBetween angle, γ Value is the sensibility to similarity for control direction, and γ=2 are taken in this method；

The set S of given edge group, and they are calculated between any two after similarity, by establishing a scoring letter It is several to score boundary candidate frame b, calculate edge group s_iThe marginal value summation of middle all pixels p, is denoted as m_i；Choose edge group s_iIn any one pixel position, be denoted as

For each edge group s_i, calculate a successive value w_b(s_i) ∈ [0,1], for weighing s_iWhether side is completely contained in In boundary frame b；w_b(s_i) calculation formula it is as follows：

In formula, T is to start from t₁∈S_b, end at t_|T|=s_iOrdered path, a (t_j,t_j+1) similar between edge group Degree；If there is no such path, w is enabled_b(s_i)=1；

Utilize the w for calculating gained_b(s_i), it is as follows to the scoring formula of bounding box b：

In formula, b_w、b_hIt is the width and height of bounding box, w respectively_b(s_i) ∈ [0,1] measurements s_iWhether bounding box b is completely contained in In, m_iIt is edge group s_iThe marginal value summation of middle all pixels p；Since the bounding box of bigger can include more edges, we Method takes κ=1.5 to offset this deviation；

Since the edge inside bounding box is in the edge near bounding box compared to those, importance will be come low；To commenting Point formula is improved, by the marginal value inside bounding box from scoring h_bIn cut, improved scoring formula is as follows：

In formula, b_w、b_hIt is the width and height of bounding box, b respectivelyⁱⁿWidth and height be respectively b_w/ 2 and b_h/ 2, m_pFor edge The marginal value size of each pixel p, similarly takes κ=1.5 in figure；Finally, target candidate frame is screened in edge graph, chooses 1000 The larger bounding box of a scoring is as target candidate frame.

3. a kind of marine ships and light boats rapid detection method based on unmanned water surface ship according to claim 1, it is characterised in that： Sea horizon is detected in the step (4), screening target candidate frame is specially again based on sea horizon anticipation：

The detection of sea horizon is by executing a Hough variation on edge graph, to obtain the position of sea horizon in the picture Confidence ceases；Ships and light boats ride the sea, and can only be in water area and sea horizon region, be not at sky areas；Based in this way One characteristic screens 1000 larger target candidate frames of scoring, weeds out and be completely on sea horizon region again The target candidate frame of side retains the target candidate frame intersected below sea horizon and with sea horizon.

4. a kind of marine ships and light boats rapid detection method based on unmanned water surface ship according to claim 1, it is characterised in that： Carrying out histograms of oriented gradients feature modeling to ships and light boats target in the step (5) is specially：

According to the sShape features of ships and light boats, the ratio of width to height of ships and light boats characteristic model is designed as 3:1, characteristic window is sized so as to The cell factory lattice of 192 × 64 pixels, HOG features are designed and sized to 8 × 8 pixels, the histogram channel of each cell factory lattice Number is set as 9, in this way, the intrinsic dimensionality V calculation formula of the HOG Feature Descriptors of ships and light boats are as follows：

5. a kind of marine ships and light boats rapid detection method based on unmanned water surface ship according to claim 1, it is characterised in that： It is specially using support vector machines repetitive exercise grader in the step (6)：

Using the support vector machines of linear kernel, using " boot strap " repetitive exercise grader, specific training step is as follows：First, Initial positive sample is made of all true value frames of ships and light boats, and sum is 2000；Then, selection is accounted for true value frame overlapping area The target candidate frame of 20%-50% is as initial negative sample, in order to avoid choosing the negative sample approximately repeated, by faying surface Product is more than 70% two negative samples, selects one of abandon；Finally, 10000 works are randomly selected from all negative samples For the negative sample of SVM training, after obtaining preliminary classification device, the process of a retraining is carried out；By preliminary classification device negative Sample artwork, that is, do not include the detection of the enterprising ship target of navigating of ships and light boats target, and all rectangle frames detected in this way belong to wrong report； The rectangle frame of these wrong reports is a difficult example for grader, these hardly possible examples are saved as image, are added to initial negative sample In set, the training of grader is re-started, in this way, just there is better classification capacity by the grader that retraining obtains, Namely detect the ability of ships and light boats target, the process of retraining can iteration carry out, until the performance of grader is not bright Until aobvious promotion.