[go: up one dir, main page]

CN119200625A - Dynamic planning method and system for UAV inspection paths in three-dimensional scenes - Google Patents

Dynamic planning method and system for UAV inspection paths in three-dimensional scenes Download PDF

Info

Publication number
CN119200625A
CN119200625A CN202410876598.4A CN202410876598A CN119200625A CN 119200625 A CN119200625 A CN 119200625A CN 202410876598 A CN202410876598 A CN 202410876598A CN 119200625 A CN119200625 A CN 119200625A
Authority
CN
China
Prior art keywords
flight path
flight
drone
path
unmanned aerial
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202410876598.4A
Other languages
Chinese (zh)
Inventor
袁超
袁杰祺
李林
张精平
吴凤敏
向强
程宇翔
黄震
张智棚
李煜东
蒲冠宇
陈立川
朱豪
郭洲余
周伟民
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chongqing Geological Disaster Prevention And Control Center
Chongqing Geology Environment Monitoring Master Station
Chongqing Geographic Information And Remote Sensing Application Center (chongqing Surveying And Mapping Product Quality Inspection And Testing Center)
Chongqing Institute of Geology and Mineral Resources
Original Assignee
Chongqing Geological Disaster Prevention And Control Center
Chongqing Geology Environment Monitoring Master Station
Chongqing Geographic Information And Remote Sensing Application Center (chongqing Surveying And Mapping Product Quality Inspection And Testing Center)
Chongqing Institute of Geology and Mineral Resources
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chongqing Geological Disaster Prevention And Control Center, Chongqing Geology Environment Monitoring Master Station, Chongqing Geographic Information And Remote Sensing Application Center (chongqing Surveying And Mapping Product Quality Inspection And Testing Center), Chongqing Institute of Geology and Mineral Resources filed Critical Chongqing Geological Disaster Prevention And Control Center
Priority to CN202410876598.4A priority Critical patent/CN119200625A/en
Publication of CN119200625A publication Critical patent/CN119200625A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/40—Control within particular dimensions
    • G05D1/46—Control of position or course in three dimensions
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)

Abstract

The invention discloses a dynamic planning method and a system for an unmanned aerial vehicle inspection path under a three-dimensional scene, wherein the method comprises the steps of S1, obtaining an initial flight path according to a topographic map, S2, carrying out flight according to the initial flight path, scanning depth information of a flight area in real time at each flight path point in the initial flight path to obtain a ranging value in front of an unmanned aerial vehicle and a depth actual measurement value below the unmanned aerial vehicle, S3, comparing the depth actual measurement value with a depth reference value in the initial flight path, if absolute values of difference values of a plurality of continuous depth actual measurement values and the depth reference value H are larger than a preset first threshold value, then changing the topography, entering a slow-going mode, and reducing the flight speed, S4, judging whether collision risks exist according to the ranging value in front of the unmanned aerial vehicle and the depth actual measurement value below the unmanned aerial vehicle, carrying out dynamic planning of an automatic obstacle avoidance and a route, and if collision risks exist, collecting images in front of the unmanned aerial vehicle, and carrying out obstacle avoidance decision by image processing. The invention realizes the automatic execution of obstacle avoidance and inspection.

Description

Dynamic planning method and system for unmanned aerial vehicle inspection path in three-dimensional scene
Technical Field
The invention relates to the technical field of path planning, in particular to a dynamic planning method for an unmanned aerial vehicle inspection path in a three-dimensional scene.
Background
With the rapid development of aviation science and technology, unmanned aerial vehicle technology is rapidly improved, and unmanned aerial vehicles are used for replacing manual operation in recent years to become hot spots. Particularly, the unmanned aerial vehicle is utilized for inspection, so that the running condition of the line can be accurately and comprehensively inspected, and the inspection efficiency and quality are greatly improved.
At present, when natural disasters occur, unmanned aerial vehicles enter destinations to patrol, and at present, the unmanned aerial vehicles are generally controlled by remote manual operation of people, but the manual control difficulty is high, and collision accidents are easy to occur. Or the original map is adopted for path planning and planning, but after the natural disaster occurs, the topography and the topography of the disaster site can be changed, so that the planned path is easy to have obstacles, and the unmanned plane is difficult to smoothly reach the target site to execute the inspection task.
Disclosure of Invention
The invention aims to provide a dynamic planning method and a system for an unmanned aerial vehicle inspection path in a three-dimensional scene, which effectively aim at geological changes occurring in disaster places, automatically execute obstacle avoidance and inspection, and ensure safe and error-free execution of inspection tasks.
In order to achieve one of the above purposes, the present invention adopts the following technical scheme:
A dynamic planning method for unmanned aerial vehicle inspection paths in a three-dimensional scene comprises the following steps:
s1, acquiring an initial flight path according to an existing topographic map;
s2, carrying out flight according to the initial flight path in the step S1, and scanning the depth information of the flight area in real time at each flight path point W i in the initial flight path to obtain a ranging value S (W i) in front of the unmanned aerial vehicle and a depth actual measurement value h (W i) below the unmanned aerial vehicle;
S3, comparing the depth measured value H (W i) with a depth reference value H (W i) in the initial flight path, if more than two continuous depth measured values H (W i) and depth reference value H (W i) are larger than a preset first threshold value, changing the terrain, and enabling the unmanned aerial vehicle to enter a creep mode to reduce the flight speed;
S4, judging whether collision risks exist or not according to a ranging value S (W i) in front of the unmanned aerial vehicle and a depth actual measurement value h (W i) below the unmanned aerial vehicle, and carrying out automatic obstacle avoidance and dynamic planning of a route, if the collision risks exist, acquiring images in front of the unmanned aerial vehicle, and carrying out obstacle avoidance decision by utilizing image processing, wherein the method specifically comprises the following steps:
Firstly, acquiring an image in front of an unmanned aerial vehicle, and calibrating a ranging point O in the image so that the ranging point O is positioned at the right center of the image;
Then, the distance between the position of the unmanned aerial vehicle and the point O of the ranging point is a ranging value S (W i), if the fixed visual angle of the camera is alpha, namely the sum of the included angles formed by the ranging line and the visual angle edge lines on the two horizontal sides is alpha, namely the sum of the included angles formed by the ranging line and the visual angle edge lines on the two vertical sides is alpha';
In the image, the sum of the distances from the ranging point O to the extreme edges of the left view and the right view is L1, and the sum of the distances from the ranging point O to the extreme edges of the upper view and the lower view is L1';
Extracting the boundary of the obstacle, and determining that the distance between the O point of the ranging point and the left, right, upper and lower boundaries is L2, L3, L2', L3', and obtaining a left included angle beta, a right included angle gamma, an upper included angle beta ', and a lower included angle gamma' of the ranging line and the boundary line at the moment;
Calculating the actual distances corresponding to L2, L3, L2', L3' by using trigonometric functions S(L2)、S(L3)、S(L2')、S(L3'),S(L2)＝S(Wi)×tanβ,S(L3)＝S(Wi)×tanγ,S(L2')＝S(Wi)×tanβ',S(L3')＝S(Wi)×tanγ';
And the distance between the current position of the unmanned aerial vehicle and the upper, lower, left and right boundaries of the front obstacle is selected, the shortest boundary distance is selected to determine the obstacle avoidance direction, and meanwhile, the obstacle avoidance flight distance is determined according to the actual distance corresponding to the shortest boundary distance and the obstacle avoidance safety distance threshold S.
Further, acquiring the front image of the unmanned aerial vehicle comprises fusing the distance information acquired by the ranging sensor of the unmanned aerial vehicle with data acquired by the camera of the unmanned aerial vehicle to acquire the front image of the unmanned aerial vehicle.
Further, the judging whether collision risk exists or not, and performing automatic obstacle avoidance and dynamic planning of the route specifically includes:
a. If the ranging value S (W i) in front of the unmanned aerial vehicle is larger than a preset first collision threshold value and the depth measured value h (W i) below the unmanned aerial vehicle is larger than a preset second collision threshold value, judging that the unmanned aerial vehicle has no collision risk, and the unmanned aerial vehicle flies along an original flight path to a target point;
b. If the ranging value S (W i) in front of the unmanned aerial vehicle is larger than a preset first collision threshold value, and the depth measured value h (W i) below the unmanned aerial vehicle is smaller than or equal to a preset second collision threshold value, judging that the unmanned aerial vehicle has a bottom collision risk, at the moment, the unmanned aerial vehicle pauses to advance, and when the depth measured value h (W i) below the unmanned aerial vehicle is larger than the preset second collision threshold value, continuing the flight of the front target point;
c. If the ranging value S (W i) in front of the unmanned aerial vehicle is smaller than or equal to a preset first collision threshold value, and the depth measured value h (W i) below the unmanned aerial vehicle is larger than a preset second collision threshold value, judging that the unmanned aerial vehicle has a front collision risk, at the moment, acquiring an image in front of the unmanned aerial vehicle, and performing obstacle avoidance decision by utilizing image processing;
d. If the ranging value S (W i) in front of the unmanned aerial vehicle is smaller than or equal to a preset first collision threshold, and the depth measured value h (W i) below the unmanned aerial vehicle is smaller than or equal to a preset second collision threshold, the unmanned aerial vehicle is judged to have collision risks of the bottom and the front, at the moment, the unmanned aerial vehicle pauses to advance, and when the depth measured value h (W i) below the unmanned aerial vehicle is larger than the preset second collision threshold, the front image of the unmanned aerial vehicle is acquired, and the obstacle avoidance decision is carried out by image processing.
Further, the step S1 of acquiring an initial flight path according to the existing topographic map specifically comprises the following steps:
S101, carrying out three-dimensional environment modeling according to the existing topographic map to obtain a three-dimensional map;
S102, determining a plurality of flight path points W i from the patrol area of the three-dimensional map, wherein i represents the number of the path points, and obtaining an initial flight path;
and S103, mapping the initial flight path into the three-dimensional map, and obtaining a depth reference value H (W i) of each flight path point W i.
Further, the step S102 of determining a flight path point W i from the patrol area of the three-dimensional map to obtain an initial flight path comprises the following steps:
Firstly, creating a preliminary flight path from a three-dimensional map based on a patrol area;
Secondly, setting constraint conditions and constructing an objective function, wherein the constraint conditions at least comprise a flight path cost constraint, an obstacle avoidance cost constraint and a flight height cost constraint, and the objective function comprises a shortest flight path and a minimum flight threat;
And finally, determining the positions of a plurality of unmanned aerial vehicle path points W i of the preliminary flight path according to the constraint condition and the objective function to obtain the initial flight path.
Further, the constraint conditions at least comprise a flight path cost constraint, an obstacle avoidance cost constraint and a flight height cost constraint, and the objective function comprises a shortest flight path and a minimum flight threat, and specifically comprises the following steps:
The flight path cost constraint:
The flight path comprises a plurality of flight path points W i, the coordinates of the flight path point W i are W i(xi,yi,zi, and the distance between two adjacent flight path points is D (W i,Wi+1);
The total flight path cost is
The obstacle avoidance cost constraint:
A contour topographic map of the patrol area is obtained,
When the unmanned aerial vehicle is set to a flight altitude depth actual measurement value h (W i), determining the shortest distance s min between the unmanned aerial vehicle and a contour line, determining a corresponding contour line from a contour line topographic map, and setting the contour line outer width d as a region threat zone;
For any obstacle, the risk cost R j of the unmanned aerial vehicle at the flying height h (W i) is expressed as:
wherein θ is a set first cost coefficient, and j is the number of obstacle avoidance points in flight;
the total risk cost is
The flying height cost constraint:
Setting a threat zone, namely, a height range higher than the lowest height h min and lower than the lowest height h min plus one unit height h 0 (h min,hmin+h0),h0 is determined based on the local vegetation height, and the lowest height h min is determined by the terrain height in the three-dimensional map;
The flying height cost h i of the flying path point W i is:
The highest height h max is the flying highest height of the unmanned aerial vehicle;
The total height cost is
The shortest flight path f 1 (a):
The flight threat minimum f 2 (a):
Further, determining positions of a plurality of unmanned aerial vehicle path points W i of the preliminary flight path according to the constraint condition and the objective function to obtain an initial flight path, which specifically includes:
Substituting the constraint conditions and the objective function into a matlab mathematical tool, solving by using a fmincon function, and calculating the positions of a plurality of unmanned aerial vehicle path points W i of the preliminary flight path to obtain the preliminary flight path.
Further, the reduced flying speed is reduced to 20% -50%.
In order to achieve the second purpose, the invention adopts the following technical scheme:
The unmanned aerial vehicle inspection path dynamic planning system in the three-dimensional scene comprises an unmanned aerial vehicle and flight control equipment, wherein the flight control equipment adopts any one of the unmanned aerial vehicle inspection path dynamic planning methods in the three-dimensional scene.
In order to achieve the third object, the present invention adopts the following technical scheme:
a flight control device comprises a processor and a memory for storing instructions executable by the processor, wherein the processor executes the method for dynamically planning the unmanned aerial vehicle inspection path in the three-dimensional scene.
The invention has the beneficial effects that:
The method comprises the steps of firstly constructing constraint conditions and objective functions by using a three-dimensional map, a contour map and the like, so as to obtain an initial flight path, then executing a flight task according to the initial flight path, judging whether the terrain has change according to depth reference information obtained by the initial flight path in the flight process of the unmanned aerial vehicle, and slowing down the flight speed if the terrain has change, so as to avoid collision phenomenon caused by the fact that the unmanned aerial vehicle flies too fast. Meanwhile, the unmanned aerial vehicle executes automatic obstacle avoidance in the flight process, and the optimal obstacle avoidance direction is determined by combining image recognition. And after the obstacle avoidance is successful, returning to the original flight path to continue to execute the flight inspection task. Therefore, the invention can effectively and automatically execute obstacle avoidance and inspection aiming at natural disasters, ensures that inspection tasks are executed safely and without errors, and has higher practical application value.
Drawings
FIG. 1 is a flow chart of an embodiment 1 of the present invention;
FIG. 2 is a contour map of an inspection area according to embodiment 1 of the present invention;
FIG. 3 is a schematic diagram of the calibration of the ranging point O in the image of embodiment 1 of the present invention;
fig. 4 is a schematic diagram illustrating the horizontal positions of the ranging point O and the unmanned aerial vehicle in fig. 3;
fig. 5 is a schematic view of the vertical positions of the ranging point O and the unmanned aerial vehicle in fig. 3.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments.
Specific example 1:
referring to fig. 1 to 5, a dynamic planning method for an unmanned aerial vehicle inspection path in a three-dimensional scene is characterized by comprising the following steps:
s1, acquiring an initial flight path according to an existing topographic map;
Comprising the following steps:
S101, carrying out three-dimensional environment modeling according to an existing topographic map to obtain a three-dimensional map, wherein the existing topographic map can be a direct three-dimensional map, namely, the three-dimensional map has height information;
S102, determining a plurality of flight path points W i from the patrol area of the three-dimensional map, wherein i represents the number of the path points, and obtaining an initial flight path;
the method specifically comprises the following steps:
Firstly, a preliminary flight path is established from a three-dimensional map based on a patrol area, the preliminary flight path is composed of a plurality of unmanned aerial vehicle path points W i, and the construction process of the preliminary flight path is as follows:
The flying spot S, the patrol point IP i, and the return point R are marked from the three-dimensional map, and the flying spot S and the return point R may be the same.
A plurality of unmanned aerial vehicle path points exist between the unmanned aerial vehicle taking off from the flying point S and the inspection point IP i, and the scheme obtains the positions of the unmanned aerial vehicle path points by configuring a plurality of constraint conditions and objective functions;
Secondly, setting constraint conditions and constructing an objective function, wherein the constraint conditions at least comprise a flight path cost constraint, an obstacle avoidance cost constraint and a flight height cost constraint, and the objective function comprises a shortest flight path and a minimum flight threat;
The constraint conditions at least comprise a flight path cost constraint, an obstacle avoidance cost constraint and a flight height cost constraint, and the objective function comprises a shortest flight path and a minimum flight threat, and specifically comprises the following steps:
The flight path cost constraint:
Let the flight path point a include a plurality of flight path point coordinates W i(xi,yi,zi), the distance between two adjacent flight path points D (W i,Wi+1);
The total flight path cost is
The obstacle avoidance cost constraint:
Because unmanned aerial vehicle can bring extra flight energy consumption, flight distance and flight time when avoiding the barrier. Therefore, the total obstacle avoidance cost in the flight path is calculated through the obstacle avoidance cost constraint, and the method is mainly applied to regional inspection, so that the main obstacles are mountains, hills, buildings and the like. The mountain and the hills are characterized in that the obstacle boundary contracts as the height rises, so that the area of the obstacle is reduced. The building is characterized by an obstacle boundary that does not change as the height rises.
A contour topographic map of the patrol area is obtained,
When the unmanned aerial vehicle is set at the flying height h (W i), a corresponding contour line is determined from a contour line topographic map, then the shortest distance s min between the unmanned aerial vehicle and the contour line is determined, and the outer width d of the contour line is set as a region threat zone, wherein in the specific embodiment, d is 5-20m;
For any obstacle, the risk cost R j of the unmanned aerial vehicle at the flying height h (W i) is expressed as:
Wherein, theta is a first set cost coefficient, theta is 0.5, j is the number of obstacle avoidance points in flight, max represents that the safety problem exists;
the total risk cost is
The flying height cost constraint:
The unmanned aerial vehicle cannot exceed the set highest height h max or cannot be lower than the set lowest height h min in the flight process, a threat zone is set, the threat zone is higher than the lowest height h min and the lowest height h min +a height range of a unit height h 0 (h min,hmin+h0),h0 is determined based on the local vegetation height, the lowest height h min is determined by the terrain height in the three-dimensional map, and the highest height h max is the flight highest height of the unmanned aerial vehicle;
The flying height cost h i of the flying path point W i is:
wherein, the value of m is the second cost coefficient, and the value of x is 0.5;
The total height cost is
The shortest flight path f 1 (a):
The flight threat minimum f 2 (a):
n and m are natural numbers.
And finally, determining the positions of a plurality of unmanned aerial vehicle path points W i of the preliminary flight path according to the constraint condition and the objective function to obtain the initial flight path.
Substituting the constraint conditions and the objective function into a matlab mathematical tool, solving by using a fmincon function, and calculating the positions of a plurality of unmanned aerial vehicle path points W i of the preliminary flight path to obtain the preliminary flight path.
And S103, mapping the initial flight path into the three-dimensional map, and obtaining a depth reference value H (W i) of each flight path point W i.
S2, carrying out flight according to the initial flight path in the step S1, scanning by using a ranging sensor, and scanning the depth information of the flight area in real time at each flight path point W i in the initial flight path to obtain a ranging value S (W i) in front of the unmanned aerial vehicle and a depth actual measurement value h (W i) below the unmanned aerial vehicle;
S3, comparing the depth measured value H (W i) with the depth reference value H (W i) in the initial flight path, if the absolute value of the difference values of the depth measured values H (W i) and the depth reference value H (W i) is larger than a preset first threshold value, the preset first threshold value is 5-20m, the terrain is changed, the unmanned aerial vehicle enters a creep mode, the flight speed is reduced, and the reduced flight speed is reduced to 20% -50% of the existing flight speed.
And S4, judging whether collision risks exist or not according to a ranging value S (W i) in front of the unmanned aerial vehicle and a depth actual measurement value h (W i) below the unmanned aerial vehicle, carrying out automatic obstacle avoidance and dynamic planning of a route, and if the collision risks exist, acquiring images in front of the unmanned aerial vehicle, and carrying out obstacle avoidance decision by utilizing image processing. The method specifically comprises the following steps:
a. if the ranging value S (W i) in front of the unmanned aerial vehicle is larger than a preset first collision threshold value and the depth measured value h (W i) below the unmanned aerial vehicle is larger than a preset second collision threshold value, judging that the unmanned aerial vehicle has no collision risk, enabling the unmanned aerial vehicle to fly along an original flight path to a target point, enabling the second collision threshold value and the second collision threshold value to be between 5 and 20m,
B. If the ranging value S (W i) in front of the unmanned aerial vehicle is larger than a preset first collision threshold value, and the depth measured value h (W i) below the unmanned aerial vehicle is smaller than or equal to a preset second collision threshold value, judging that the unmanned aerial vehicle has a bottom collision risk, at the moment, the unmanned aerial vehicle pauses to advance, and when the depth measured value h (W i) below the unmanned aerial vehicle is larger than the preset second collision threshold value, continuing the flight of the front target point;
c. If the ranging value S (W i) in front of the unmanned aerial vehicle is smaller than or equal to a preset first collision threshold value, and the depth measured value h (W i) below the unmanned aerial vehicle is larger than a preset second collision threshold value, judging that the unmanned aerial vehicle has a front collision risk, at the moment, acquiring an image in front of the unmanned aerial vehicle, and performing obstacle avoidance decision by utilizing image processing;
the method specifically comprises the following steps:
Firstly, acquiring an unmanned aerial vehicle front image, fusing distance information acquired by a distance measuring sensor of the unmanned aerial vehicle with data acquired by a camera of the unmanned aerial vehicle to acquire the unmanned aerial vehicle front image, calibrating a distance measuring point O in the fused image to enable the distance measuring point O to be located at the right center of the image, wherein the distance measuring point O is a point located in front of the unmanned aerial vehicle and is shot by the camera, and if a plurality of objects exist in the image, the same calibration method is adopted for each object.
Then, the distance between the position of the unmanned aerial vehicle and the point O of the ranging point is a ranging value S (W i), as shown in fig. 3, 4 and 5, if the fixed view angle of the camera is alpha, if the obstacle is regarded as a plane, namely the sum of the angles formed by the ranging line and the view angle edge lines on the two horizontal sides is alpha, namely the sum of the angles formed by the ranging line and the view angle edge lines on the two vertical sides is alpha';
in the image, the sum of the distances from the ranging point O to the extreme edges of the left view angle and the right view angle is L1, the L1 is determined by counting the pixel points, and the sum of the distances from the ranging point O to the extreme edges of the upper view angle and the lower view angle is L1';
Extracting the boundary of the obstacle, and determining that the distance between the O point of the ranging point and the left, right, upper and lower boundaries is L2, L3, L2', L3', and obtaining a left included angle beta, a right included angle gamma, an upper included angle beta ', and a lower included angle gamma' of the ranging line and the boundary line at the moment;
Alpha ' calculates the actual distance corresponding to L2, L3, L2', L3' by trigonometric function S(L2)、S(L3)、S(L2')、S(L3'),S(L2)＝S(Wi)×tanβ,S(L3)＝S(Wi)×tanγ,S(L2')＝S(Wi)×tanβ',S(L3')＝S(Wi)×tanγ';
The distance between the current position of the unmanned aerial vehicle and the upper, lower, left and right boundaries of the front obstacle is calculated through the formula, the shortest boundary distance is selected to determine the obstacle avoidance direction, and meanwhile, the obstacle avoidance flight distance is determined according to the fact distance and the obstacle avoidance safety distance threshold S, wherein the obstacle avoidance safety distance threshold S is a first collision threshold or a second collision threshold.
For example, the upper side is selected as the obstacle avoidance direction, the actual distance between the unmanned aerial vehicle and the upper boundary is S (L2 '), and the obstacle avoidance flight distance is S (L2')+s, where S is the obstacle avoidance distance threshold. At this time, S (L2') +S is raised, flown forward, passed over the obstacle, and flown toward the target point.
D. If the ranging value S (W i) in front of the unmanned aerial vehicle is smaller than or equal to the preset first collision threshold, and the depth measured value h (W i) below the unmanned aerial vehicle is smaller than or equal to the preset second collision threshold, it is determined that the unmanned aerial vehicle has a bottom and a front collision risk, at this time, the unmanned aerial vehicle pauses to advance, and when the depth measured value h (W i) below the unmanned aerial vehicle is larger than the preset second collision threshold, the front image of the unmanned aerial vehicle is acquired again, an obstacle avoidance decision is made by image processing, and the obstacle avoidance decision is the same as the above.
In the embodiment, the unmanned aerial vehicle flies according to an initial flight path and scans regional depth information in real time so as to carry out obstacle avoidance and dynamic planning of a route, taking a point A to a point IP i as an example;
Flying from the point A, flying along an initial path, and simultaneously scanning at each flying path point W i by using a range finder to obtain a ranging value S (W i) in front of the unmanned aerial vehicle and a depth actual measurement value h (W i) below the unmanned aerial vehicle;
Comparing the depth actual measurement value H (W i) with the depth reference value H (W i);
When the absolute value of the difference value between the depth measured value H (W i) and the depth reference value H (W i) which are more than two continuous values is larger than a preset first threshold value, the change of the terrain is indicated, the unmanned aerial vehicle enters a creep model, and the flying speed is reduced by 20% -50%. Meanwhile, a ranging value S (W i) in front of the unmanned aerial vehicle and a depth actual measurement value h (W i) below the unmanned aerial vehicle are continuously detected so as to automatically shoot and avoid the obstacle.
After the obstacle avoidance of the obstacle point is completed, the unmanned aerial vehicle flies to the target point according to a straight line, if the obstacle is encountered again, the obstacle avoidance is performed again according to the steps S2 to S4, and meanwhile, the unmanned aerial vehicle continues to fly until returning to an initial flight path, so that the inspection task is completed.
Specific example 2:
The embodiment provides a dynamic planning system for an unmanned aerial vehicle inspection path in a three-dimensional scene, which comprises an unmanned aerial vehicle and flight control equipment, wherein the flight control equipment adopts the dynamic planning method for the unmanned aerial vehicle inspection path in the three-dimensional scene in embodiment 1, and the dynamic planning method in embodiment 1 is not repeated.
Specific example 3:
a flight control device comprises a processor and a memory for storing executable instructions of the processor, wherein the processor executes the dynamic planning method for the unmanned aerial vehicle inspection path in the three-dimensional scene in the embodiment 1, and the dynamic planning method in the embodiment 1 is not repeated.
The technical scheme provided by the invention is described in detail. The principles and embodiments of the present invention have been described herein with reference to specific examples, the description of which is intended only to facilitate an understanding of the method of the present invention and its core ideas. It should be noted that it will be apparent to those skilled in the art that various modifications and adaptations of the invention can be made without departing from the principles of the invention and these modifications and adaptations are intended to be within the scope of the invention as defined in the following claims.

Claims (10)

1.一种三维场景下无人机巡查路径的动态规划方法，其特征在于，包括如下步骤：1. A method for dynamically planning a UAV inspection path in a three-dimensional scene, characterized in that it includes the following steps: S1:根据地形图获取初始飞行路径；S1: Obtain the initial flight path based on the terrain map; S2:根据所述步骤S1中的初始飞行路径进行飞行，在所述初始飞行路径中的各个飞行路径点Wi实时扫描飞行地区深度信息，得到无人机前方的测距值S(Wi)以及无人机下方的深度实测值h(Wi)；S2: Fly according to the initial flight path in step S1, scan the depth information of the flight area in real time at each flight path point Wi in the initial flight path, and obtain the distance measurement value S( Wi ) in front of the UAV and the depth measurement value h( Wi ) below the UAV; S3:将所述深度实测值h(Wi)与所述初始飞行路径中的深度参照值H(Wi)进行对比，若存在连续两个以上深度实测值h(Wi)与深度参照值H(Wi)差值的绝对值大于预设第一阈值时，则地形存在变化，所述无人机进入缓行模式，降低飞行速度；S3: Compare the measured depth value h(W i ) with the depth reference value H(W i ) in the initial flight path. If the absolute value of the difference between two or more consecutive measured depth values h(W i ) and the depth reference value H(W i ) is greater than a preset first threshold, the terrain changes, and the UAV enters a slow-motion mode to reduce the flight speed. S4:根据无人机前方的测距值S(Wi)以及无人机下方的深度实测值h(Wi)，判断是否存在碰撞风险，进行自动避障和航线的动态规划；若有碰撞风险，则采集无人机前方图像，利用所述图像处理来进行避障决策，具体包括：S4: According to the range measurement value S(W i ) in front of the drone and the measured depth value h(W i ) below the drone, determine whether there is a collision risk, and perform automatic obstacle avoidance and dynamic planning of the route; if there is a collision risk, collect the image in front of the drone, and use the image processing to make obstacle avoidance decisions, specifically including: 首先、获取无人机前方图像，对所述图像中的测距点O进行标定，使得测距点O位于图像的正中心；First, obtain the image in front of the drone, and calibrate the ranging point O in the image so that the ranging point O is located at the exact center of the image; 然后、测距线，无人机位置与测距点O点的距离为测距值S(Wi)，若摄像头的固定视角为α，即测距线与水平两侧的视角边缘线构成的夹角之和为α，即测距线与竖向两侧的视角边缘线构成的夹角之和为α’；Then, the distance between the drone position and the distance measurement point O is the distance measurement value S(W i ). If the fixed viewing angle of the camera is α, the sum of the angles formed by the distance measurement line and the viewing angle edge lines on both sides horizontally is α, and the sum of the angles formed by the distance measurement line and the viewing angle edge lines on both sides vertically is α'; 所述图像中，从测距点O至左右两个视角最边缘的距离之和为L1，从测距点O点至上下两个视角最边缘的距离之和为L1’；In the image, the sum of the distances from the distance measuring point O to the edges of the left and right viewing angles is L1, and the sum of the distances from the distance measuring point O to the edges of the upper and lower viewing angles is L1'; 提取障碍物边界，并确定测距点O点与左、右、上、下边界的距离分别为L2、L3、L2’、L3’，此时，得到测距线与边界线的左夹角β、右夹角γ、上夹角β’、下夹角γ’；Extract the obstacle boundary and determine the distances between the ranging point O and the left, right, upper and lower boundaries as L2, L3, L2’, L3’ respectively. At this time, the left angle β, right angle γ, upper angle β’ and lower angle γ’ between the ranging line and the boundary line are obtained; 通过三角函数计算出L2、L3、L2’、L3’对应的实际距离S(L2)、S(L3)、S(L2')、S(L3')，S(L2)＝S(Wi)×tanβ，S(L3)＝S(Wi)×tanγ，S(L2')＝S(Wi)×tanβ'，S(L3')＝S(Wi)×tanγ'；The actual distances S(L2), S(L3), S(L2'), S(L3') corresponding to L2, L3, L2', and L3' are calculated by trigonometric functions: S(L2) = S(W i ) × tanβ, S(L3) = S(W i ) × tanγ, S(L2') = S(W i ) × tanβ', S(L3') = S(W i ) × tanγ'; 通过无人机当前位置与前方障碍物上下左右边界的距离，选择其中最短的边界距离确定避障方向，同时，根据最短的边界距离对应的实际距离加上避障安全距离阈值S确定避障飞行距离。The obstacle avoidance direction is determined by selecting the shortest boundary distance between the current position of the drone and the upper, lower, left, and right boundaries of the obstacle in front. At the same time, the obstacle avoidance flight distance is determined based on the actual distance corresponding to the shortest boundary distance plus the obstacle avoidance safety distance threshold S. 2.根据权利要求1所述一种三维场景下无人机巡查路径的动态规划方法，其特征在于：所述获取无人机前方图像包括：根据无人机的测距传感器获取的距离信息与无人机的摄像头采集的数据进行融合，获得无人机前方图像。2. According to the dynamic planning method of the drone inspection path in a three-dimensional scene described in claim 1, it is characterized in that: the acquisition of the image in front of the drone includes: fusing the distance information obtained by the drone's ranging sensor with the data collected by the drone's camera to obtain the image in front of the drone. 3.根据权利要求1所述一种三维场景下无人机巡查路径的动态规划方法，其特征在于：所述判断是否存在碰撞风险，进行自动避障和航线的动态规划，具体包括：3. According to the method for dynamic planning of a drone inspection path in a three-dimensional scene in claim 1, it is characterized in that: the determination of whether there is a collision risk, automatic obstacle avoidance and dynamic planning of the route specifically include: a、若无人机前方的测距值S(Wi)大于预设第一碰撞阈值，且无人机下方的深度实测值h(Wi)大于预设第二碰撞阈值，则判断无人机不存在碰撞风险，无人机沿原飞行路径向目标点飞行；a. If the distance measurement value S(W i ) in front of the drone is greater than the preset first collision threshold, and the depth measurement value h(W i ) below the drone is greater than the preset second collision threshold, it is determined that there is no collision risk for the drone, and the drone flies along the original flight path to the target point; b、若在无人机前方的测距值S(Wi)大于预设第一碰撞阈值，且无人机下方的深度实测值h(Wi)小于或者等于预设第二碰撞阈值时，判定无人机存在底部碰撞风险，此时，无人机暂停前进，升高至是无人机下方的深度实测值h(Wi)大于预设第二碰撞阈值时，继续前目标点飞行；b. If the distance measurement value S(W i ) in front of the drone is greater than the preset first collision threshold, and the actual depth measurement value h(W i ) below the drone is less than or equal to the preset second collision threshold, it is determined that the drone has a bottom collision risk. At this time, the drone stops moving forward and rises until the actual depth measurement value h(W i ) below the drone is greater than the preset second collision threshold, and then continues to fly to the target point; c、若在无人机前方的测距值S(Wi)小于或者等于预设第一碰撞阈值，无人机下方的深度实测值h(Wi)大于预设第二碰撞阈值时，判定无人机存在前方碰撞风险，此时，采集无人机前方图像，利用图像处理来进行避障决策；c. If the range measurement value S(W i ) in front of the drone is less than or equal to the preset first collision threshold, and the depth measurement value h(W i ) below the drone is greater than the preset second collision threshold, it is determined that the drone has a front collision risk. At this time, the image in front of the drone is collected and image processing is used to make obstacle avoidance decisions; d、若在无人机前方的测距值S(Wi)小于或者等于预设第一碰撞阈值，且无人机下方的深度实测值h(Wi)小于或者等于预设第二碰撞阈值时，判定无人机存在底部和前方的碰撞风险，此时，无人机暂停前进，升高至是无人机下方的深度实测值h(Wi)大于预设第二碰撞阈值时，再进行采集无人机前方图像，利用图像处理来进行避障决策。d. If the distance measurement value S(W i ) in front of the UAV is less than or equal to the preset first collision threshold, and the actual depth measurement value h(W i ) below the UAV is less than or equal to the preset second collision threshold, it is determined that the UAV is at risk of collision with the bottom and the front. At this time, the UAV stops moving forward and rises until the actual depth measurement value h(W i ) below the UAV is greater than the preset second collision threshold. Then, the image in front of the UAV is collected and the image processing is used to make obstacle avoidance decisions. 4.根据权利要求1所述一种三维场景下无人机巡查路径的动态规划方法，其特征在于：所述S1:根据现有地形图获取初始飞行路径；具体包括：4. According to the method for dynamically planning the inspection path of an unmanned aerial vehicle in a three-dimensional scene in claim 1, it is characterized in that: the step S1: obtaining an initial flight path according to an existing topographic map; specifically comprising: S101:根据现有地形图进行三维环境建模，得到三维地图；S101: Perform three-dimensional environment modeling according to the existing topographic map to obtain a three-dimensional map; S102:从所述三维地图的巡查地区中确定多个飞行路径点Wi，i表示路径点个数，得到初始飞行路径；S102: determining a plurality of flight path points W i from the patrol area of the three-dimensional map, where i represents the number of path points, and obtaining an initial flight path; S103:将所述初始飞行路径映射至所述三维地图中，并得到各个飞行路径点Wi的深度参照值H(Wi)。S103: Mapping the initial flight path to the three-dimensional map, and obtaining a depth reference value H(W i ) of each flight path point W i . 5.根据权利要求4所述一种三维场景下无人机巡查路径的动态规划方法，其特征在于：所述S102:从所述三维地图的巡查地区中确定飞行路径点Wi，得到初始飞行路径；包括：5. The method for dynamically planning a drone inspection path in a three-dimensional scene according to claim 4, characterized in that: S102: determining a flight path point W i from the inspection area of the three-dimensional map to obtain an initial flight path; comprising: 首先、基于巡查地区从三维地图中创建一个初步飞行路径；First, create a preliminary flight path from a 3D map based on the patrol area; 其次、设置约束条件、构建目标函数，所述约束条件至少包括：飞行路径成本约束、避障成本约束和飞行高度成本约束，所述目标函数包括：飞行路径最短和飞行威胁最小；Secondly, setting constraints and constructing an objective function, wherein the constraints include at least: a flight path cost constraint, an obstacle avoidance cost constraint and a flight altitude cost constraint, and the objective function includes: the shortest flight path and the smallest flight threat; 最后、根据所述约束条件和所述目标函数确定所述初步飞行路径的多个无人机路径点Wi位置，得到初始飞行路径。Finally, the positions of multiple UAV path points Wi of the preliminary flight path are determined according to the constraint conditions and the objective function to obtain the initial flight path. 6.根据权利要求5所述一种三维场景下无人机巡查路径的动态规划方法，其特征在于：所述约束条件至少包括：飞行路径成本约束、避障成本约束和飞行高度成本约束，所述目标函数包括：飞行路径最短和飞行威胁最小，具体包括：6. According to the dynamic planning method of the inspection path of the unmanned aerial vehicle in a three-dimensional scene in claim 5, it is characterized in that: the constraint conditions at least include: flight path cost constraint, obstacle avoidance cost constraint and flight altitude cost constraint, the objective function includes: shortest flight path and minimum flight threat, specifically including: 所述飞行路径成本约束:The flight path cost constraint: 飞行路径包括多个飞行路径点Wi，飞行路径点Wi的坐标为Wi(xi，yi，zi)，两个相邻飞行路径点的距离为D(Wi，Wi+1)；The flight path includes a plurality of flight path points W i , the coordinates of the flight path point W i are W i (x i , y i , z i ), and the distance between two adjacent flight path points is D (W i , W i+1 ); 总飞行路径成本为 The total flight path cost is 所述避障成本约束:The obstacle avoidance cost constraint: 获取巡查区域等高线地形图，Obtain the contour map of the inspection area. 设无人机在飞行高度深度实测值h(Wi)时，确定所述无人机与等高线之间的最短距离smin，对应的等高线从等高线地形图中确定，设定等高线外宽度d为区域威胁区；When the UAV is at the measured value h(W i ) of the flight altitude depth, the shortest distance s min between the UAV and the contour line is determined, the corresponding contour line is determined from the contour line topographic map, and the width d outside the contour line is set as the regional threat area; 对于任何一个障碍物，无人机在飞行高度为h(Wi)时，其风险成本Rj表示为：For any obstacle, when the UAV is flying at a height of h(W i ), its risk cost R j is expressed as: 其中，θ为设定的第一成本系数，j为飞行时避障点位个数；Among them, θ is the set first cost coefficient, j is the number of obstacle avoidance points during flight; 总风险成本为 The total risk cost is 所述飞行高度成本约束：The flight altitude cost constraint: 设定威胁区：高于最低高度hmin,低于最低高度hmin+一个单位高度h0的高度范围(hmin，hmin+h0)，h0基于当地植被高度确定，最低高度hmin由三维地图中的地形高度确定；Set the threat zone: the height range from above the minimum height h min to below the minimum height h min + one unit height h 0 (h min , h min +h 0 ), where h 0 is determined based on the local vegetation height and the minimum height h min is determined by the terrain height in the three-dimensional map; 飞行路径点Wi的飞行高度成本hi为：The flight altitude cost h i of flight path point W i is: 其中，ɡ为设定的第二成本系数；最高高度hmax为无人机的飞行最高高度；Among them, ɡ is the set second cost coefficient; the maximum altitude h max is the maximum flight altitude of the UAV; 总高度成本为 The total height cost is 所述飞行路径最短f1(A)： The flight path is the shortest f 1 (A): 所述飞行威胁最小f2(A)： The minimum flight threat f 2 (A): 7.根据权利要求5所述一种三维场景下无人机巡查路径的动态规划方法，其特征在于：7. The method for dynamically planning a UAV inspection path in a three-dimensional scene according to claim 5, characterized in that: 所述根据所述约束条件和所述目标函数确定所述初步飞行路径的多个无人机路径点Wi位置，得到初始飞行路径，具体包括：Determining the positions of multiple drone path points W i of the preliminary flight path according to the constraint conditions and the objective function to obtain the initial flight path specifically includes: 将所述约束条件和目标函数代入至matlab数学工具中，并使用fmincon函数进行求解，计算出所述初步飞行路径的多个无人机路径点Wi位置，可得到初步的飞行路径。Substitute the constraints and objective function into the matlab mathematical tool, and use the fmincon function to solve it, calculate the positions of multiple drone path points W i of the preliminary flight path, and obtain the preliminary flight path. 8.根据权利要求1所述一种三维场景下无人机巡查路径的动态规划方法，其特征在于：所述降低飞行速度降低到20％-50％。8. According to the dynamic planning method of the UAV inspection path in a three-dimensional scene in claim 1, it is characterized in that: the flight speed is reduced to 20%-50%. 9.一种三维场景下无人机巡查路径的动态规划系统，包括无人机、飞行控制设备，其特征在于：所述飞行控制设备采用权利要求1至8任一所述一种三维场景下无人机巡查路径的动态规划方法。9. A dynamic planning system for an unmanned aerial vehicle inspection path in a three-dimensional scene, comprising an unmanned aerial vehicle and a flight control device, wherein the flight control device adopts a dynamic planning method for an unmanned aerial vehicle inspection path in a three-dimensional scene as described in any one of claims 1 to 8. 10.一种飞行控制设备，包括：处理器、存储所述处理器可执行指令的存储器，其特征在于，所述处理器执行权利要求1至8任一项所述一种三维场景下无人机巡查路径的动态规划方法。10. A flight control device, comprising: a processor and a memory storing executable instructions of the processor, characterized in that the processor executes the dynamic planning method of a drone inspection path in a three-dimensional scene as described in any one of claims 1 to 8.
CN202410876598.4A 2024-07-02 2024-07-02 Dynamic planning method and system for UAV inspection paths in three-dimensional scenes Pending CN119200625A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410876598.4A CN119200625A (en) 2024-07-02 2024-07-02 Dynamic planning method and system for UAV inspection paths in three-dimensional scenes

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410876598.4A CN119200625A (en) 2024-07-02 2024-07-02 Dynamic planning method and system for UAV inspection paths in three-dimensional scenes

Publications (1)

Publication Number Publication Date
CN119200625A true CN119200625A (en) 2024-12-27

Family

ID=94059235

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410876598.4A Pending CN119200625A (en) 2024-07-02 2024-07-02 Dynamic planning method and system for UAV inspection paths in three-dimensional scenes

Country Status (1)

Country Link
CN (1) CN119200625A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119759090A (en) * 2025-03-06 2025-04-04 旭日蓝天(武汉)科技有限公司 Unmanned aerial vehicle scheduling control method for multi-machine collaborative operation

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018157309A1 (en) * 2017-02-28 2018-09-07 深圳市大疆创新科技有限公司 Air route correction method and device and unmanned aerial vehicle
KR20210037998A (en) * 2019-09-30 2021-04-07 (주)대우건설 Method of providing drone route
WO2021088133A1 (en) * 2019-11-05 2021-05-14 上海为彪汽配制造有限公司 Method and system for constructing flight trajectory of multi-rotor unmanned aerial vehicle
US20220147066A1 (en) * 2020-03-02 2022-05-12 Clrobur Co., Ltd. Drone control system and intelligent drone flight planning method thereof
CN117029827A (en) * 2023-08-09 2023-11-10 河北科技大学 Unmanned aerial vehicle track planning method and device under wind and rain conditions

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018157309A1 (en) * 2017-02-28 2018-09-07 深圳市大疆创新科技有限公司 Air route correction method and device and unmanned aerial vehicle
KR20210037998A (en) * 2019-09-30 2021-04-07 (주)대우건설 Method of providing drone route
WO2021088133A1 (en) * 2019-11-05 2021-05-14 上海为彪汽配制造有限公司 Method and system for constructing flight trajectory of multi-rotor unmanned aerial vehicle
US20220147066A1 (en) * 2020-03-02 2022-05-12 Clrobur Co., Ltd. Drone control system and intelligent drone flight planning method thereof
CN117029827A (en) * 2023-08-09 2023-11-10 河北科技大学 Unmanned aerial vehicle track planning method and device under wind and rain conditions

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119759090A (en) * 2025-03-06 2025-04-04 旭日蓝天(武汉)科技有限公司 Unmanned aerial vehicle scheduling control method for multi-machine collaborative operation

Similar Documents

Publication Publication Date Title
CN109060281B (en) Integrated bridge detection system based on unmanned aerial vehicle
CN108496129B (en) An aircraft-based facility detection method and control device
CN107514993B (en) The collecting method and system towards single building modeling based on unmanned plane
US20210158009A1 (en) UAV Real-Time Path Planning Method for Urban Scene Reconstruction
US9734397B1 (en) Systems and methods for autonomous imaging and structural analysis
CN108132675B (en) A factory inspection UAV autonomous path cruise and intelligent obstacle avoidance method
CN108508916B (en) A control method, device, equipment and storage medium for unmanned aerial vehicle formation
CN110631581B (en) Method for establishing indoor 3D map and unmanned aerial vehicle
US10521664B2 (en) Systems and methods for autonomous perpendicular imaging of test squares
CN111257892A (en) An obstacle detection method for vehicle autonomous driving
CN116012336A (en) Tunnel Intelligent Geological Sketch and Surrounding Rock Grade Identification Device and Method
CN114463489A (en) Oblique photography modeling system and method for optimizing unmanned aerial vehicle air route
CN111721279A (en) A terminal path navigation method suitable for power transmission inspection work
CN117873158A (en) Unmanned aerial vehicle routing inspection complex route optimization method based on live-action three-dimensional model
CN111522360A (en) Banded oblique photography automatic route planning method based on electric power iron tower
CN119200625A (en) Dynamic planning method and system for UAV inspection paths in three-dimensional scenes
CN119440090A (en) Autonomous obstacle avoidance method, system, device and storage medium for inspection drone
TW201534512A (en) Control method about obstacle avoidance and navigation by binocular images
CN119846644A (en) Position correction method and device applied to unmanned aerial vehicle laser SLAM system
CN118115585A (en) A method for detecting the best landing area for UAVs
CN114708422B (en) A method and device for calculating cabin door coordinates based on binocular images
CN113674343B (en) Unmanned aerial vehicle inspection method, unmanned aerial vehicle inspection device, unmanned aerial vehicle inspection equipment and storage medium
CN107609468B (en) Class optimization aggregation analysis method for active safety detection of unmanned aerial vehicle landing area and application
CN115601517A (en) Rock mass structural plane information acquisition method and device, electronic equipment and storage medium
CN120279443A (en) Unmanned aerial vehicle intelligent inspection method based on lightweight model and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination