CN119810801A - Unmanned aerial vehicle runway obstacle recognition and avoidance method based on improved YOLO v5 - Google Patents

Abstract

The present invention provides an unmanned aerial vehicle runway obstacle recognition method and an obstacle avoidance method based on an improved YOLO v5, and belongs to the technical field of obstacle recognition. The unmanned aerial vehicle runway obstacle recognition method comprises acquiring a data image; applying a sky segmentation method to segment the data image to obtain a segmented data image. An improved YOLO v5 visual model is configured, and the target recognition submodel in the backbone network Backbone included in the improved YOLO v5 visual algorithm model is CenterNet, and the MSN component is deleted in the detection head Head. The segmented data image is input into the improved YOLO v5 visual algorithm model, and the obstacle recognition result is output. In view of the actual application scenario in which the types and numbers of obstacles on the runway in the unmanned aerial vehicle runway obstacle recognition are relatively small, a unmanned aerial vehicle runway obstacle recognition method based on the improved YOLOv5 is proposed to solve the problems faced by YOLOv5, such as complex computing power, low efficiency, and unsuitability for unmanned aerial vehicle runways.

Description

Unmanned plane runway obstacle recognition method and obstacle avoidance method based on improved YOLO v5

Technical Field

The invention belongs to the technical field of obstacle recognition, and particularly relates to an unmanned plane runway obstacle recognition method and an obstacle avoidance method based on improved YOLO v 5.

Background

With the rapid development of vision sensors, the application scenes of the vision sensors are more and more, and obstacle recognition is one of the core applications in the field of machine vision. Meanwhile, with the development of AI technology, more and more intelligent algorithms are constructed to be applied to a machine vision system as a core algorithm for obstacle recognition. YOLO v5 was constructed in this context. YOLO v5 is improved based on the precursor YOLO v4, so that the method has the advantages of high speed, high precision, light weight and the like, but also has many problems such as poor effect of small targets and dense targets, overlarge training data set and the like, so that the algorithm of YOLO v5 needs to be improved, and the problems faced by the algorithm are eliminated as far as possible under the condition of keeping the advantages.

Disclosure of Invention

The invention aims to provide an unmanned aerial vehicle runway obstacle recognition method and an obstacle avoidance method based on improved YOLO v5, and provides an unmanned aerial vehicle runway obstacle recognition method based on improved YOLOv to solve the problems of complex calculation, low efficiency, inapplicability to unmanned aerial vehicle runways and the like faced by YOLOv5, aiming at practical application scenes of fewer types and numbers of obstacles on runways in unmanned aerial vehicle runway obstacle recognition, and solving the problems of difficulty in recognition of small targets on the opposite end side, particularly poor dense target recognition effect of a relatively small data set based on YOLOv algorithm.

In order to achieve the above object, the present invention provides the following technical solutions:

in a first aspect, the invention provides an unmanned aerial vehicle runway obstacle recognition method based on improved YOLO v5, which is characterized by comprising the following steps:

S10, acquiring a data image;

S11, dividing the data image by using a sky division method to obtain a divided data image;

S12, configuring an improved YOLO v5 visual model, wherein a target recognition sub-model in a Backbone network backup included in the improved YOLO v5 visual algorithm model is CENTERNET, and deleting an MSN component in a detection Head;

S13, inputting the segmented data image into an improved YOLO v5 visual algorithm model, and outputting an obstacle recognition result.

As a possible implementation manner, the Backbone network Backbone is sequentially provided with three layers of components including CBL and CSP-x after CENTERNET, where x represents the number of repetitions.

As a possible implementation manner, the Neck network in the improved YOLO v5 visual model includes a Backbone network Backbone, neck and a Prediction, where the target identification submodel in the Backbone network backup is CENTERNET, and the MSN component is deleted in the detection header.

In a second aspect, the invention also provides an obstacle avoidance method, which is used for identifying an obstacle by using the unmanned aerial vehicle runway obstacle identification method based on the improved YOLO v5 in the first aspect, and comprises the following steps:

S20, determining threat level of the obstacle;

S21, determining an obstacle avoidance path according to the threat level.

As one possible implementation, the threat level specifically includes having an obstacle but not affecting the flight of the aircraft, having an obstacle and possibly affecting the flight of the aircraft, having an obstacle and certainly affecting the flight of the aircraft.

As a possible implementation manner, determining the obstacle avoidance path according to the threat level specifically includes:

When the threat level is judged to be an obstacle but the flight of the aircraft is not affected, the path planning module does not carry out path planning again, namely the original path is provided for the aircraft, and the flight task is continued to be executed;

When the threat level is judged to be an obstacle and the flight of the aircraft is possibly influenced, the obstacle recognition module sends the recognized judgment information to the path planning module, and the path planning module carries out path planning again and provides a new route for the aircraft to continue the execution of the flight task;

When the threat level is judged to be an obstacle and the flight of the aircraft is positively influenced, and when no new path is provided by the path planning, the emergency stop information is sent to the control system of the unmanned aerial vehicle, so that the aircraft is braked.

Compared with the prior art, the invention has the following effects:

1. Aiming at the special application scene of an airport, the method adopts an optimized YOLOv visual algorithm to realize the recognition and safety pre-warning of the runway obstacle of the unmanned aerial vehicle, and effectively solves the problems that the recognition of small targets at the opposite end side of a relatively small data set is difficult, and particularly the recognition effect of dense targets is poor.

2. By modifying the target recognition algorithm of the Backbone network Backbone of the YOLO v5, namely adopting CENTERNET to replace FCOS, the structure can omit the MSN component in the original YOLOv Head, effectively reduce the complexity of the algorithm and meet the requirements of limited space, computational power and instantaneity;

3. The invention applies the sky segmentation and Reduced YOLO v5 algorithm to fuse the runway obstacle recognition algorithm, and after fusing the sky segmentation, the invention can rapidly process the visual image and remove the influence of noise caused by the sky part on the result of the subsequent obstacle recognition.

4. The intelligent path planning method constructs the intelligent path planning of the three-level safe obstacle avoidance rule supporting unmanned aerial vehicle platform.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the invention and do not constitute a limitation on the invention.

FIG. 1 is a general flow chart for identifying obstacle in an unmanned aircraft runway based on the improved YOLO v5 algorithm provided by an embodiment of the invention;

FIG. 2 is a modified YOLO v5 algorithm framework;

FIG. 3 illustrates three different obstacle avoidance modes.

Detailed Description

In order to clearly describe the technical solution of the embodiments of the present invention, in the embodiments of the present invention, the words "first", "second", etc. are used to distinguish the same item or similar items having substantially the same function and effect. For example, the first threshold and the second threshold are merely for distinguishing between different thresholds, and are not limited in order. It will be appreciated by those of skill in the art that the words "first," "second," and the like do not limit the amount and order of execution, and that the words "first," "second," and the like do not necessarily differ.

In the present invention, the words "exemplary" or "such as" are used to mean serving as an example, instance, or illustration. Any embodiment or design described herein as "exemplary" or "for example" should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete fashion.

In the present invention, "at least one" means one or more, and "a plurality" means two or more. "and/or" describes an association of associated objects, meaning that there may be three relationships, e.g., A and/or B, and that there may be A alone, while A and B are present, and B alone, where A, B may be singular or plural. The character "/" generally indicates that the context-dependent object is an "or" relationship. "at least one of" or the like means any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one (a, b or c) of a, b, c, a and b combination, a and c combination, b and c combination, or a, b and c combination, wherein a, b and c can be single or multiple.

Referring to fig. 1 and 2, in a first aspect, an embodiment of the present invention provides an unmanned runway obstacle recognition method based on improved YOLO v5, including the steps of:

S10, acquiring a data image;

S11, dividing the data image by using a sky division method to obtain a divided data image. Sky information occupying large area information in the image is removed so as to adapt to an improved YOLO v5 visual algorithm model and accurately realize the identification of runway obstacles. The method and the device for identifying the obstacle by using the sky segmentation and the Reduced YOLO v5 fusion runway obstacle identification algorithm can be used for rapidly processing the visual image after the sky segmentation is fused, and eliminating the influence of noise caused by the sky part on the result of the subsequent obstacle identification.

S12, configuring an improved YOLO v5 visual model, wherein a target identification sub-model in a Backbone network Backbone included in the improved YOLO v5 visual algorithm model is CENTERNET, and deleting an MSN component in a detection Head. Improving the efficiency of the YOLO v5 algorithm is achieved by replacing FCOS with CENTERNET. By using CENTERNET instead of FCOS as the framework for obstacle recognition, recognition of obstacles on the runway is realized rapidly by means of fusing length and width at one central point of CENTERNET, and the algorithm is modified mainly because the obstacles on the runway are simpler, less in number and less than common obstacles, so that a complex framework for target monitoring is not needed, and the application scene in the embodiment of the invention is more suitable for adopting CENTERNET as the method for target monitoring. The unmanned plane runway obstacle recognition method frame can cancel the NMS frame in the Head part after CENTERNET is used, so that the whole YOLO v5 frame can be further reduced, the complexity of an algorithm is effectively reduced, and the requirements of limited space, calculation power and instantaneity are met.

In addition, aiming at the special application scene of the airport, the invention adopts the optimized YOLOv visual algorithm to realize the recognition and the safety pre-warning of the runway obstacle of the unmanned aerial vehicle, and effectively solves the problems of difficult recognition of small targets, particularly poor dense target recognition effect at the opposite end side of a relatively small data set. Specifically, CENTERNET is based on a key point detection method, in contrast to conventional FCOS, CENTERNET converts the target detection problem into a regression task that predicts the object center point and size. It uses thermodynamic diagrams (heatmap) to represent the center position of an object and performs regression of the bounding box in combination with width and height. By center point positioning, the model can effectively avoid complex designs (such as anchors and multi-scale feature pyramids) in traditional target detection. The object detection method has the advantages that the object can be directly detected from the thermodynamic diagram, so that the position of the object can be better captured, and the object detection method has certain advantages in particular in small object detection. In addition, it is more suitable for detection of irregular objects, non-rectangular areas, because it detects by regression of the object center and size, rather than by regression of four bounding box coordinates. Furthermore, the model is relatively easy to train because the complex anchor configuration is not involved, and the sensitivity of the model to the hyper-parameters is low.

S13, inputting the segmented data image into an improved YOLO v5 visual algorithm model, and outputting an obstacle recognition result.

As one possible implementation, the Backbone network Backbone is followed by CenterNe sequentially three layers of components comprising CBL and CSP-x, where x represents the number of repetitions, as an example x=3.

As a possible implementation manner, the Neck network in the improved YOLO v5 visual model includes a Backbone network Backbone, neck and a Prediction, where the target identification submodel in the Backbone network backup is CENTERNET, and the MSN component is deleted in the detection header.

In a second aspect, the embodiment of the invention further provides an obstacle avoidance method, which is used for identifying an obstacle by using the unmanned aerial vehicle runway obstacle identification method based on the improved YOLO v5 in the first aspect, and comprises the following steps:

And S20, determining threat levels of the obstacles, wherein the threat levels specifically comprise the obstacle which does not affect the flight of the aircraft, the obstacle which possibly affects the flight of the aircraft, and the obstacle which certainly affects the flight of the aircraft.

S21, determining an obstacle avoidance path according to the threat level. When the threat level is judged to be the obstacle but the flight of the aircraft is not affected, the path planning module does not carry out path planning again, namely the original path is provided for the aircraft, and the flight task execution is continued. When the threat level is judged to be an obstacle and the flight of the aircraft is possibly influenced, the obstacle recognition module sends the recognized judgment information to the path planning module, and the path planning module carries out path planning again and provides a new route for the aircraft to continue the execution of the flight mission. When the threat level is judged to be an obstacle and the flight of the aircraft is positively influenced, and when no new path is provided by the path planning, the emergency stop information is sent to the control system of the unmanned aerial vehicle, so that the aircraft is braked.

In the implementation, referring to fig. 3 (a), there is an obstacle but the flight of the aircraft is not affected, the information of the obstacle is identified by the configured obstacle identifying module, and the normal operation of the aircraft is not affected at this time according to the proportion of the obstacle to the runway, for example, only one tenth, i.e. at this time, because the speed of the aircraft is very low, even personnel will not affect the normal operation of the aircraft.

Referring to fig. 3 (b), there are obstacles, but the obstacle may affect the flight safety, at this time, the obstacle recognition module may send the recognized judgment information to the path planning module, and the path planning module may re-plan the path and provide a new route to the aircraft to continue the flight mission.

Referring to fig. 3 (c), there are obstacles, and the position of the obstacle affects the safe operation of the aircraft, and when no new path is provided in the path planning, emergency stop information is sent to the control system of the unmanned aerial vehicle, so as to realize the braking of the aircraft.

The intelligent path planning method constructs the intelligent path planning of the three-level safe obstacle avoidance rule supporting unmanned aerial vehicle platform.

Although the invention has been described herein in connection with various embodiments, other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended drawings. In the description, the word "comprising" does not exclude other elements or steps, and the "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the specification. Some measures are described in mutually different embodiments, but this does not mean that these measures cannot be combined to produce a good effect.

Although the invention has been described in connection with specific features and embodiments thereof, it will be apparent that various modifications and combinations can be made without departing from the spirit and scope of the invention. Accordingly, the specification and figures are merely exemplary of the invention and are to be regarded as covering any and all modifications, variations, combinations, or equivalents that are within the scope of the invention. It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the present invention and the equivalent techniques thereof, the present invention is also intended to include such modifications and variations.

Claims (6)

1. An unmanned aerial vehicle runway obstacle recognition method based on improved YOLO v5 is characterized by comprising the following steps:

S10, acquiring a data image;

S11, dividing the data image by using a sky division method to obtain a divided data image;

S12, configuring an improved YOLO v5 visual model, wherein a target recognition sub-model in a Backbone network backup included in the improved YOLO v5 visual algorithm model is CENTERNET, and deleting an MSN component in a detection Head;

S13, inputting the segmented data image into an improved YOLO v5 visual algorithm model, and outputting an obstacle recognition result.

2. The improved YOLO v5 based unmanned runway obstacle recognition method of claim 1, wherein the Backbone network Backbone is followed by CENTERNET in sequence with three layers of components comprising CBL and CSP-x, where x represents the number of repetitions.

3. The unmanned aerial vehicle runway obstacle recognition method based on improved YOLO v5 of claim 1, wherein the Neck networks in the improved YOLO v5 visual model comprise a Backbone network Backbone, neck and a Prediction, wherein the target recognition submodel in the Backbone network Backbone is CENTERNET, and the MSN component is deleted in the Head.

4. An obstacle avoidance method characterized in that an unmanned runway obstacle recognition method based on the modified YOLO v5 according to any one of claims 1 to 3 is applied to recognize an obstacle, and the obstacle avoidance method comprises the following steps:

S20, determining threat level of the obstacle;

S21, determining an obstacle avoidance path according to the threat level.

5. The obstacle avoidance method of claim 4 wherein the threat level comprises, in particular, having an obstacle but not affecting the flight of the aircraft, having an obstacle and potentially affecting the flight of the aircraft, having an obstacle and certainly affecting the flight of the aircraft.

6. The obstacle avoidance method of claim 5 wherein determining the obstacle avoidance path based on the threat level comprises:

When the threat level is judged to be an obstacle but the flight of the aircraft is not affected, the path planning module does not carry out path planning again, namely the original path is provided for the aircraft, and the flight task is continued to be executed;

When the threat level is judged to be an obstacle and the flight of the aircraft is possibly influenced, the obstacle recognition module sends the recognized judgment information to the path planning module, and the path planning module carries out path planning again and provides a new route for the aircraft to continue the execution of the flight task;

When the threat level is judged to be an obstacle and the flight of the aircraft is positively influenced, and when no new path is provided by the path planning, the emergency stop information is sent to the control system of the unmanned aerial vehicle, so that the aircraft is braked.