WO2024069789A1 - Aerial imaging system, aerial imaging method, and aerial imaging program - Google Patents

Abstract

[Problem] To ensure safety in aerial imaging. [Solution] An aerial imaging system 1 comprises a mobile object 100 that flies over a target area F, a camera 141 mounted on the mobile object to capture an image of the target area, an event detection unit 320 that detects an event on the basis of the image captured by the camera or an input from an external system, and an imaging condition determination unit 325 that determines an imaging condition, including at least one of the target imaging position and target imaging direction of the mobile object, according to the detected event.

Description

Aerial photography system, aerial photography method, and aerial photography program

The present invention relates to an aerial photography system, an aerial photography method, and an aerial photography program.

Patent Document 1 discloses a camera viewpoint display system that detects the aircraft's position and nose direction, as well as the pan and tilt angles of a camera device mounted on the aircraft, calculates the camera viewpoint from each of these pieces of data, and displays the viewpoint on a map on a monitor screen. With this system, an operator controls the aircraft's position and attitude, as well as the camera's shooting direction, while grasping the aircraft's position and heading from a ground station.

Patent document 2 discloses a technology that automatically controls the position and shooting direction so that a specific object is tracked and photographed by multiple UAVs.

JP 2006-281830 A JP 2020-115642 A

In order to effectively capture sports and other events, it is necessary to capture images that are suited to the details of the subject, such as the game situation. More specifically, since the range to be captured changes depending on the game situation, it is desirable to be able to appropriately control the shooting position and direction according to the situation.

In this regard, the system described in Patent Document 1 was complicated because it required simultaneous control of the camera and the flying object. Furthermore, the system described in Patent Document 2 only tracked the shooting position and direction of the object, which was inconvenient.

The present invention was made in consideration of the above problems, and aims to provide an aerial photography system that reduces the labor required for photography and enables appropriate photography according to the conditions of the subject.

In order to achieve the above object, an aerial photography system according to one aspect of the present invention comprises a moving body that flies over a target area, a camera mounted on the moving body that photographs the target area, an event detection unit that detects an event based on an image captured by the camera or an input from an external system, and a photography condition determination unit that determines photography conditions including at least one of a target photography position and a target photography direction of the moving body according to the detected event.

The shooting condition determination unit may determine the shooting conditions according to the type of the detected event.

The target shooting direction may be achieved by controlling at least one of the nose direction of the moving body and the angle of the camera relative to the moving body.

The shooting conditions may include a target zoom amount for the camera.

When the shooting condition determination unit receives input of the shooting conditions from a controller of the moving object or the external system, the shooting condition determination unit may determine the shooting conditions based on the operation received via the controller even if the event detection unit detects the event.

The camera may further include a controller that accepts input of the shooting conditions by a user, and the shooting condition determination unit may determine the shooting conditions based on the input via the controller if the event detection unit has not detected the event, and may determine the shooting conditions based on the event if the event detection unit detects the event.

The aerial photography system may include a plurality of the moving bodies, and photograph a single target area by flying the plurality of moving bodies simultaneously over the target area, and the photography condition determination unit may determine different photography conditions for each of the plurality of moving bodies.

The shooting condition determination unit may set, for multiple moving bodies flying simultaneously, shooting conditions for shooting the same shooting range from different target shooting positions, or shooting conditions for shooting an area including the same shooting range with different zoom amounts.

The shooting condition determination unit may determine the shooting conditions according to the predicted result of the ball's trajectory predicted by the event detection unit as the detection result of the event.

The shooting condition determination unit may determine the shooting conditions so that, when an event indicating a foul has occurred in a competition held in the target area is detected, the shooting condition has a shooting range of the ball used in the competition or the vicinity of the position of the referee of the competition.

The aerial photography system may include a plurality of moving bodies, and photograph a single target area by flying the plurality of moving bodies simultaneously over the target area, and the photography condition determination unit may determine the photography conditions, when an event indicating that a foul has occurred in the competition is detected, by using the plurality of moving bodies to photograph the ball or the vicinity of the point of the competition referee at different target photography positions, target photography directions, or zoom amounts.

The system may include a flight path generation unit that generates a flight path for the moving body, and the flight path generation unit may automatically generate the flight path to the target shooting position that is determined based on the event detected from the captured image.

The flight path generation unit generates the flight path within the court that is configured within the target area, and the flight path generation unit may generate the flight path to the target shooting position by connecting a plurality of shooting positions that have been set in advance, and may change the shooting positions to be connected depending on the detection status of the event.

In order to achieve the above object, an aerial photography method according to another aspect of the present invention includes an event detection step for detecting an event based on an image captured by a camera photographing a target area or an input from an external system, and a photography condition determination step for determining photography conditions including at least one of a target photography position and a target photography direction of a moving body equipped with the camera according to the detected event.

In order to achieve the above-mentioned object, an aerial photography program according to yet another aspect of the present invention causes a computer to execute an event detection command for detecting an event based on an image acquired by a camera photographing a target area or an input from an external system, and a photography condition determination command for determining photography conditions including at least one of a target photography position and a target photography direction of a moving body equipped with the camera in accordance with the detected event.
The computer program may be provided by being stored on various data-readable recording media, or may be provided so as to be downloadable via a network such as the Internet.

The present invention reduces the labor required for photography and enables appropriate photography according to the subject's circumstances.

1 is an overall configuration diagram of an aerial photography system according to an embodiment of the present invention; FIG. 2 is a simplified external perspective view of the drone according to the embodiment. FIG. 2 is a functional configuration diagram of the drone according to the embodiment. (a) is a simplified front view of the exterior of the control device of the embodiment; (b) is a schematic diagram showing the direction in which the drone moves or turns in response to input from the control device. FIG. 2 is a functional configuration diagram of the control device according to the embodiment. FIG. 2 is a functional configuration diagram of a server according to the embodiment. 1 is a schematic diagram showing an example of a shooting position of a drone that is set in advance in a shooting target field where the drone flies. FIG. 2 is a schematic state transition diagram showing the transition of flight modes of the drone. This is a schematic state transition diagram showing the state transition of the drone depending on the aircraft state of the drone. A schematic state transition diagram showing the state transition of the drone according to the aircraft behavior state of the drone. FIG. 11 is a schematic state transition diagram showing state transitions of a game in a stadium as an example of a field to be photographed. FIG. 2 is a schematic state transition diagram showing the state transition of the offensive and defensive states in the stadium. 1 is a table showing an example of the correspondence between the game state in the stadium and the shooting range, camera position, shooting direction, and zoom amount captured by the drone. 1 is a schematic diagram showing possible photographing positions and flight paths to which the photographing position can be transitioned; 1 is a flowchart of a control executed during flight of the drone. 16 is a flowchart showing control of flight restrictions in the drone (details of S1002 in FIG. 15). 16 is a flowchart showing flight mode switching control of the drone (details of S1010 in FIG. 15 ). FIG. 4 is a diagram showing a first example of a screen displayed on a terminal of the aerial photography system. FIG. 13 is a diagram showing a second example of a screen displayed on the terminal of the aerial photography system. FIG. 13 is a diagram showing a third example of a screen displayed on the terminal of the aerial photography system. FIG. 13 is a diagram showing a fourth example of a screen displayed on the terminal of the aerial photography system.

Below, a preferred embodiment of the present invention will be described in detail with reference to the attached drawings. In this specification and drawings, components having substantially the same functional configuration will be denoted with the same reference numerals to avoid repetitive explanation. Furthermore, the embodiments shown below are merely examples, and other known elements or alternative means may be adopted depending on the application, purpose, scale, etc.

A. One embodiment
[A-1. composition]
(A-1-1. Overall configuration)
FIG. 1 is an overall configuration diagram of an aerial photography system 1 (hereinafter also referred to as "system 1") according to one embodiment of the present invention. System 1 uses a drone 100 (an example of a moving body) to take aerial photographs of a competition taking place at a stadium F (FIG. 7) (an example of a target area) or an event taking place at an event venue. Stadium F is an example of a target area. A single system 1 may include multiple drones 100. In this case, system 1 can photograph a single stadium F by flying multiple drones simultaneously over the stadium F.

In the following explanation, we will use as an example the system 1 that films soccer as needed, but this system 1 can also be applied to sports and events other than soccer.

As shown in FIG. 1, in addition to the drone 100, the system 1 mainly includes a controller 200 that allows the pilot to operate the drone 100, a server 300 that manages the flight and photography of the drone 100, an external input device 600, and an external system 700.

The drone 100 and the controller 200 are connected to each other via wireless communication (which may include communication via a base station 800). The controller 200 and the server 300 are connected to each other via a communication network 400 such as an internet line. The drone 100 acquires satellite signals from an artificial satellite 500 to determine its own position, etc.

The external input device 660 is a device capable of transmitting and receiving information to and from the system 1, separate from the controller 200, and is composed of a mobile terminal such as a smartphone or tablet terminal. The external input device 660 can be operated, for example, by the manager, coach, bench player, referee, or court equipment personnel of the competition taking place at the stadium F. The external input device 660 has, for example, a function for receiving an emergency command to suspend filming, and the drone 100 performs emergency evacuation based on the command. The external input device 660 may also receive an input to switch the flight mode of the drone 100. Furthermore, the external input device 660 may be equipped with a display device, and may display information similar to that displayed on the display unit 201 of the controller 200. In particular, the external input device 660 may acquire event information that occurs during the competition. The event information is referred to when the user of the external input device 660 makes an input to switch the flight mode of the drone 100.

The external system 700 may be any system configured separately from the system 1. For example, systems such as a court facility system, a match management system, and a referee support system may be applied as systems deployed in relation to the competition held at the stadium F, and systems such as a weather observation system or an earthquake observation system deployed independently of the competition may also be applied. Multiple external systems 700 may be connected to the system 1. The system 1 may receive an emergency command to stop filming or a command to switch the flight mode of the drone 100 from the various external systems 700. In addition, the various external systems 700 may acquire event information that occurs during the competition.

The court facilities system, which is an example of the external system 700, may obtain the brightness of the captured image from the system 1, for example, and control the illuminance adjustment or blinking of the lighting in the stadium F. The court facilities system may also receive a request for lighting illuminance from the system 1 and control the illuminance adjustment or blinking.

The configuration of system 1 is not limited to that shown in FIG. 1, and the drone 100, the controller 200, the server 300, and the base station 800 may each be connected to each other so that they can communicate with each other via a communication network 400 such as an Internet line. In this case, the drone 100 may perform wireless communication directly with the communication network 400 using a communication method such as LTE without going through the controller 200. Therefore, the drone 100, the controller 200, and the base station 800 do not need to perform direct wireless communication, and it is sufficient if they can each be connected to the communication network 400 in a remote location. Therefore, this system configuration is suitable for cases where the drone 100 and the controller 200 are in a remote location (for example, when a pilot operates them remotely).

In addition, in the system 1, the drone 100, the controller 200, the base station 800, and the server 300 are each connected to each other so that they can communicate with each other via a communication network 400 such as an Internet line, and the drone 100 and the base station 800 may be communicatively connected to the communication network 400 by satellite communication via an artificial satellite 500.

Furthermore, in the system 1, multiple servers 300 may be connected to one drone 100 via multiple communication networks 400, i.e., the system may be made redundant. In this case, even if an abnormality occurs in the server 300 or communication network 400, the operation of the system 1, and therefore shooting by the drone 100, can be continued by the other redundant servers 300 and communication networks 400, thereby improving the reliability of the system 1. Note that in both of the above forms, the drone 100 and the controller 200 can be controlled even when they are remotely located, making them suitable for remote operation, but this is not limited to this, and they can also be applied to visual flight in which the pilot manually controls the drone 100 while watching it.

The device described in the above embodiment may be realized as a single device, or may be realized by multiple devices (e.g., drone 100, controller 200, cloud server 300) that are partially or completely connected by a communication network 400. For example, each functional unit and memory unit of server 300 may be realized by being implemented in different servers 300, drones 100, and controllers 200 that are connected to each other by the communication network 400.

(A-1-2. Drone 100)
(A-1-2-1. Overview of the drone 100)
Fig. 2 is a simplified external perspective view of the drone 100 of this embodiment. Fig. 3 is a functional configuration diagram of the drone 100 of this embodiment. As described above, the drone 100 takes aerial photographs of competitions held in the stadium F (Fig. 7) and events held in the event venue.

In this specification, "drone" refers to any flying object that has the ability to autonomously control its attitude, regardless of the power source (electricity, prime mover, etc.), control method (wireless or wired, and fully autonomous or partially manual, etc.), and whether manned or unmanned. Drones are also sometimes referred to as Unmanned Aerial Vehicles (UAVs), flying objects, multicopters, RPAS (Remote Piloted Aircraft Systems), or UAS (Unmanned Aircraft Systems), etc.

As shown in FIG. 2, the exterior of the drone 100 is mainly composed of a housing 101 and multiple propellers 122. The housing 101 is, for example, a roughly rectangular parallelepiped, but may have any shape. Rod-shaped connecting parts 102 extending laterally are connected to the left and right sides of the housing 101. The other ends of the connecting parts 102 are respectively connected to propellers 122 and motors 121 that rotate the propellers 122. The motors 121 are, for example, electric motors. Note that in this embodiment, there are four connecting parts 102, propellers 122, and motors 121, but the number is not limited to this. The propellers 122 may be composed of a single propeller, or may be composed of multiple propellers arranged coaxially. The number and shape of the blades of each propeller are not particularly limited.

In addition, a propeller guard (not shown) may be provided on the outside of the propeller 122 to prevent the propeller from interfering with obstacles.

In the housing 101, for example, a photographing camera 141 is held by a camera holder 142 below the housing 101. In addition, an obstacle detection camera 131 is disposed on the front surface of the housing 101. In this embodiment, the obstacle detection camera 131 is a so-called dual camera consisting of two cameras that form a pair. The obstacle detection camera 131 is disposed so as to capture an image in front of the drone 100. Note that the obstacle detection camera 131 may be disposed not only on the front surface but also on all surfaces of the housing 101, for example, on six surfaces in the case of a housing 101 that is a substantially rectangular parallelepiped.

The drone 100 is equipped with an alarm device 250 that alerts people around the drone 100 to the presence of the drone 100. The alarm device 250 has, for example, a warning light 251 and a speaker 252. The warning light 251 is provided for each propeller 122 or motor 121, and is disposed, for example, on each side of multiple motors 121. The warning light 251 may be disposed along the cylindrical side of the motor 121 so that it can be seen from all directions in addition to the front. The speaker 252 outputs an alarm sound and is provided in the housing 101 of the drone 100. The speaker 252 is provided, for example, on the underside of the housing 101, and transmits the alarm sound downwards of the drone 100.

(A-1-2-2. Functional blocks of drone 100)
As shown in FIG. 3, the drone 100 is equipped with an arithmetic device such as a CPU (Central Processing Unit) for executing information processing, and storage devices such as a RAM (Random Access Memory) and a ROM (Read Only Memory), and thereby has the following functional blocks: a measurement unit 110, a flight function unit 120, an obstacle detection unit 130, an imaging unit 140, and a communication unit 150.

The measurement unit 110 is a functional unit that measures information related to the drone 100 or its surroundings. The measurement unit 110 has, for example, a position measurement unit 111, a direction measurement unit 112, an altitude measurement unit 113, and a speed measurement unit 114. In addition to these, the measurement unit 110 may also include various sensors that acquire information such as temperature, air pressure, wind speed, and acceleration.

The position measurement unit 111 receives signals from the artificial satellites 500 and measures the position (absolute position) of the aircraft based on the signals. The position measurement unit 111 measures its current position using, for example, GNSS (Global Navigation Satellite System), GPS (Global Positioning System), etc., but is not limited to this. As a method for measuring the position, for example, RTK-GNSS (Real Time Kinematic - Global Navigation Satellite System) can also be used. The position information includes at least two-dimensional coordinate information in a planar view (e.g., latitude, longitude), and preferably includes three-dimensional coordinate information including altitude information.

In addition, the base station 800, which provides information on the reference points of fixed stations used for relative positioning such as RTK, is connected to the drone 100 and the controller 200 so as to be able to communicate wirelessly with them, making it possible to measure the position of the drone 100 with greater accuracy. Here, when performing RTK measurement using a virtual reference point method using a VRS (Virtual Reference Station), the base station 800 can be omitted, or the accuracy of the position coordinate estimation of the base station 800 or the drone 100 can be further improved.

The orientation measurement unit 112 measures the orientation of the aircraft (nose direction, heading direction). The orientation measurement unit 112 is composed of a geomagnetic sensor that measures the nose direction (heading direction) of the drone 100 aircraft by measuring geomagnetism, a compass, etc.

The altitude measurement unit 113 measures the altitude above the ground (hereinafter also referred to as "flight altitude") as the distance from the ground below the drone 100 (vertically downward).

The speed measurement unit 114 detects the flight speed of the drone 100. The speed measurement unit 114 may measure the speed using a known sensor such as a gyro sensor.

(A-1-2-3. Flight function unit 120)
The flight function unit 120 is a mechanism and function unit that causes the drone 100 to fly, and generates thrust in the airframe for lifting the drone 100 and moving it in a desired direction. As shown in Figures 2 and 3, the flight function unit 120 has a plurality of motors 121, a plurality of propellers 122, and a flight control unit 123.

The flight control unit 123 independently controls the multiple motors 121 to rotate each propeller 122, causing the drone 100 to perform various operations such as taking off, moving forward, turning, and landing, and controls the attitude angle control and flight operations of the drone 100 from takeoff, during flight, and until landing.

The flight control unit 123 has a processing unit, also called a flight controller. The processing unit can have one or more processors, such as a programmable processor (e.g., a central processing unit (CPU), MPU, or DSP). The processing unit has access to a memory (storage unit). The memory stores logic, code, and/or program instructions that the processing unit can execute to perform one or more steps. The memory may include, for example, a separable medium such as an SD card or RAM, or an external storage device. Various data acquired by the measurement unit 110, or video or still image data captured by the imaging camera 141, may be directly transmitted to and stored in the memory. Each data may also be recorded in an external memory.

The processing unit includes a control module configured to control the state of the drone 100. For example, the control module controls the flight function section 120 (thrust generating section) of the drone 100 to adjust the spatial arrangement, attitude angle, angular velocity, angular acceleration, angular jerk rate, and/or acceleration of the drone 100 having six degrees of freedom (translational motion x, y, and z, and rotational motion θx, θy, and θz).

The flight control unit 123 can control the flight of the drone 100 based on control signals from the pilot 200 or based on a preset autonomous flight program. The flight control unit 123 can also control the flight of the drone 100 by controlling the motor 121 based on various information such as the field to be photographed, flight permitted/prohibited areas, information on the corresponding flight geofences, map information including two-dimensional or three-dimensional map data, the current position information of the drone 100, attitude information (heading information), speed information, and acceleration information, and any combination of these.

Examples of Shooting Target Field and Shooting Positions In this specification, the term "shooting target field" or "target area" refers to a two-dimensional location (for example, the stadium F) that is the subject of shooting.

FIG. 7 is a schematic diagram showing an example of a playing field F, which is an example of a field to be photographed by a drone, viewed from above. The playing field F is composed of a court F100 that is roughly rectangular and is defined by, for example, a straight outer edge, and an outer court area F200 that is a predetermined area that covers the outer edge of the court F100. The outer edge of the court F100 is composed of mutually opposing goal lines F110a, F110b and mutually opposing touch lines F111a, F111b that are connected at roughly right angles. The connection points of the goal lines F110a, F110b and the touch lines F111a, F111b are the corners F112a, F113a, F112b, F113b.

Goals F120a, F120b are provided approximately in the center of the pair of goal lines F110a, F110b. Penalty areas F130a, F130b are defined in specific areas inside the court F100 adjacent to the goals F120a, F120b, and penalty lines F140a, F140b are drawn on the outer edges of the penalty areas.

A halfway line F150 is drawn in the center of the court F100, connecting the midpoints of a pair of touchlines F111a, F111b and dividing the court F100 into approximately equal parts. The halfway line F150 is approximately parallel to the goal lines F110a, F110b.

Note that the goal lines F110a, F110b, touchlines F111a, F111b, penalty lines F140a, F140b, and halfway line F150 are required by the rules for players to play the game, and therefore all of these lines are generally drawn in a manner that allows them to be seen, but the technical scope of the present invention is not limited to this. Also, in this explanation, a soccer stadium is used as an example, but the sports that are photographed by the system of the present invention are not limited to soccer, and include any type of sports, such as tennis. Furthermore, the subject of the photography is not limited to sports, and the system can also be applied to other events (concerts, ceremonies, etc.).

In the stadium F, multiple predetermined shooting positions L101-L105, L206-L215 are defined. The shooting positions L101-L105, L206-L215 may be two-dimensional coordinates on a plane, or may be three-dimensional coordinate information that also defines the height at the corresponding positions. The flight height of the drone 100 may be manually controllable based on input from the controller 200.

Photographing positions L101 to L105 are defined, for example, on the touchline F111b, at approximately equal intervals along the touchline F111b. For example, photographing position L101 is a point located in a range including the intersection of the halfway line F150 and the touchline F111b and slightly outside the court F100. Photographing positions L103 and L105 are points near the corners F112a or F112b on both sides of the touchline F111b. Photographing positions L102 and L104 are points between photographing positions L103 and L105 and photographing position L101. Note that the above positions are merely examples, and are not limited to these and may be any appropriate positions.

Photographing positions L206 to L215 are points defined within the court F100. For example, photographing positions L206 and L211 are points near the center of the penalty lines F140a and F140b on a line parallel to the goal lines F110a and F110b, and are so-called goal-front photographing positions. Photographing positions L207 and L212 are photographing positions closer to the touchline or F111a or F111b and closer to the halfway line F150 than photographing positions L206 and L215. More specifically, for example, photographing positions L207 and L212 are points on an imaginary line segment connecting photographing position L101 and goals F120a and F120b, and are, for example, points approximately in the center of the imaginary line segment. Photographing positions L209 and L215 are points that are linearly symmetrical to photographing positions L207 and L212. Shooting position L208 is a point between shooting position L207 and the halfway line F150, shooting position L210 is a point between shooting position L209 and the halfway line F150, shooting position L213 is a point between shooting position L212 and the halfway line F150, and shooting position L214 is a point between shooting position L215 and the halfway line F150.

In the outside court area F200, an evacuation point H200 is set to which the drone 100 is to be evacuated if an abnormality or malfunction of the drone 100 or the system 1 is detected. The abnormality referred to here is an abnormality related to the stability of the aerial movement of the drone 100. The abnormality includes, for example, a case where the calculation load associated with the operation control (behavior control, shooting control, etc.) of the drone 100 exceeds a load threshold. Alternatively, the abnormality may include a transient abnormality related to the environment, such as a case where the measured value of the behavior control value (e.g. speed) of the drone 100 exceeds an allowable value due to the influence of a strong wind or the like.

The evacuation point H200 is set at a point different from the shooting positions L101 to L105 and L206 to L215, and in this embodiment, it is set outside the touchline F111a and along the touchline F111a. There may be multiple evacuation points H200, and in this embodiment, there are three. The evacuation point H220 is set near the extension of the halfway line F150. The evacuation points H210 and H230 are set closer to the goals F120a and F120b than the shooting positions L206 and L211. The evacuation points H210 and H230 are set at the ends of an area partitioned by a geofence G200, which will be described later, for example. At the evacuation point H200, for example, the drone 100 is replaced or the battery installed in the drone 100 is replaced.

In the stadium F, multiple geofences G100, G200 are set, encompassing at least each of the shooting positions L101-L105, L206-L215. A geofence indicates a virtual boundary line that divides an area, and in particular, the geofence in this embodiment indicates a fence that is the boundary line between a flight-permitted area, where the drone 100 is permitted to fly or move, and a no-fly area. A geofence is a boundary line that divides an area that extends three-dimensionally, including in plane and height. If a moving object such as the drone 100 comes into contact with a geofence, flight or movement is restricted to prevent the aircraft from flying outside the flight-permitted area.

The boundary line of the geofence in the height direction may include an upper limit and a lower limit. In this embodiment, the geofences G100, G200 that are applied to whether or not flight is permitted are switched according to the control of the system 1 while the drone 100 is flying. The number of geofences G100, G200 depicted in the figure is two, but the number is arbitrary, and specifically may be three or more.

The geofence G100 is an area that includes the shooting positions L101 to L105, and defines an area that includes the touchline F111b and the area nearby. In other words, the geofence G100 is defined near the outer edge of the court F100, and a portion of it extends into the outer court area F200. The geofence G200 is a geofence that is primarily applied in the outer edge flight mode M102, which will be described later. The geofence G200 is an area that includes the shooting positions L206 to L215, and is set at least inside the court F100. This geofence G200 is a geofence that is primarily applied in the on-court flight mode M105, which will be described later.

The areas defined by the multiple geofences G100, G200 at least partially contact or overlap each other. The areas defined by the multiple geofences G100, G200 also overlap in the height direction. The heights of the multiple geofences G100, G200 may differ from each other. Specifically, the lower limit of the altitude of the geofence G200 set inside the stadium F is set higher than the lower limit of the altitude of the geofence G100 set on the outer edge of the stadium F. Since there is a high probability that subjects to be photographed, such as players, are present inside the stadium F, by setting the lower limit of the altitude of the geofence high and flying the drone 100 at a sufficient height, it is possible to prevent interference with the movements of the subjects to be photographed and collisions with the subjects to be photographed or the ball.

(A-1-2-4. Obstacle detection unit 130)
Returning to the description of Fig. 3, the obstacle detection unit 130 is a functional unit that detects obstacles around the drone 100. The obstacles may include, for example, people, for example, players, objects, animals such as birds, fixed equipment, and a ball. The obstacle detection unit 130 measures the position, speed vector, and the like of an obstacle located, for example, below the drone 100 based on the acquired image.

The obstacle detection unit 130 includes, for example, an obstacle detection camera 131, a ToF (Time of Flight) sensor 132, and a laser sensor 133. The ToF sensor 132 measures the time it takes for a laser pulse emitted from the sensor to return to the light receiving element in the sensor, and measures the distance to an object by converting this time into distance. The laser sensor 133 uses, for example, the LiDAR (Light Detection And Ranging) method to shine light such as near-infrared light, visible light, or ultraviolet light on the target object and measure the distance by capturing the reflected light with an optical sensor.

In this embodiment, FIG. 2 shows that the obstacle detection camera 131 is positioned facing forward, but the type, position and number of the camera 131, ToF sensor 132 and laser sensor 133 are arbitrary, and the ToF sensor 132 or laser sensor 133 may be positioned instead of the camera 131, or the ToF sensor 132 or laser sensor 133 may be provided on all six surfaces of the housing 101, i.e., the front, back, top, bottom and both sides.

(A-1-2-5. Photographing unit 140)
The photographing unit 140 is a functional unit that photographs images of a competition in the stadium F (FIG. 7) or an event at an event venue, and includes a photographing camera 141, a camera holding unit 142, and a photographing control unit 143. As shown in FIG. 2, the photographing camera 141 (imaging device) is disposed at the bottom of the main body of the drone 100, and outputs image data related to a peripheral image photographed around the drone 100. The photographing camera 141 is a video camera (color camera) that photographs moving images. The moving images may include audio data acquired by a microphone (not shown). In addition to or instead of this, the photographing camera 141 may also be configured to photograph still images.

The orientation of the photographic camera 141 (the attitude of the photographic camera 141 relative to the housing 101 of the drone 100) can be adjusted by a camera actuator (not shown) built into the camera holding unit 142. The photographic camera 141 may have an automatic control function for parameters such as exposure, contrast, or ISO. The camera holding unit 142 may have a so-called gimbal control mechanism that suppresses the transmission of shaking or vibration of the aircraft to the photographic camera 141. The photographic control unit 143 controls the photographic camera 141 and the camera holding unit 142 to adjust the orientation of the photographic camera 141, the photographic magnification (zoom amount), the camera's photographic conditions, etc. Image data acquired by the photographic camera 141 can be transmitted to the memory unit of the drone 100 itself, the pilot 200, the server 300, etc.

(A-1-2-6. Communication unit 150)
The communication unit 150 is capable of radio wave communication via the communication network 400 and includes, for example, a radio wave communication module. The communication unit 150 is capable of communication with the controller 200 and the like via the communication network 400 (including the wireless base station 800).

(A-1-3. Controller 200)
(A-1-3-1. Overview of Controller 200)
FIG. 4 is a front view of the appearance of the controller 200 of this embodiment in a simplified manner. FIG. 5 is a functional configuration diagram of the controller 200 of this embodiment. The controller 200 is a mobile information terminal that controls the drone 100 by the operation of the pilot and displays information received from the drone 100 (e.g., position, altitude, remaining battery level, camera image, etc.). In this embodiment, the flight state (altitude, attitude, etc.) of the drone 100 may be remotely controlled by the pilot 200, or may be autonomously controlled by the drone 100. For example, when a flight command is transmitted from the pilot to the drone 100 via the pilot 200, the drone 100 performs autonomous flight. In addition, manual operation may be possible during basic operations such as takeoff and return, and in an emergency.

As shown in FIG. 4, the controller 200 includes a display unit 201 and an input unit 202 as a hardware configuration. The display unit 201 and the input unit 202 are connected to each other so that they can communicate with each other wired or wirelessly. The display unit 201 may be configured as a touch panel or liquid crystal monitor that is integrated into the controller 200, or may be configured as a display device such as a liquid crystal monitor, tablet terminal, or smartphone that is connected to the controller 200 wired or wirelessly. The display unit 201 as a hardware configuration may be configured as a touch panel display by integrally incorporating an element that accepts input such as touch.

The input unit 202 is a mechanism through which the pilot inputs operational commands such as flight direction and takeoff/landing when piloting the drone 100.
As shown in FIG. 4A, the input unit 202 has a left slider 326L, a right slider 326R, a left input stick 327L, a right input stick 327R, a power button 328, and a return button 329. The left slider 326L and the right slider 326R are operators that accept, for example, an input of 0/1, or an input of one-dimensional stepless or stepwise information, and the operator slides the left and right index fingers to input, for example, while holding the controller 200 in his/her hand. The left input stick 327L and the right input stick 327R are operators that accept an input of multi-dimensional stepless or stepwise information, and are, for example, so-called joysticks. The left input stick 327L and the right input stick 327R may also accept an input of 0/1 by pressing them. The power button 328 and the return button 329 are operators that accept pressing them, and are constituted by mechanical switches or the like.

The left input stick 327L and the right input stick 327R accept input operations that instruct the three-dimensional flight operations of the drone 100, including, for example, takeoff, landing, ascent, descent, right turn, left turn, forward movement, backward movement, left movement, and right movement. Figure 4(b) is a schematic diagram showing the movement direction or rotation direction of the drone 100 corresponding to each input of the left input stick 327L and right input stick 327R shown in Figure 4(a). Note that this correspondence is an example.

As shown in FIG. 5, the controller 200 includes a processor such as a CPU for executing information processing, and storage devices such as RAM and ROM, which constitute the software configuration of the main functional blocks of the display control unit 210, input control unit 220, and communication unit 240.

(A-1-3-2. Display control unit 210)
The display control unit 210 displays to the pilot the drone 100 or the status information of the drone 100 acquired from the server 300. The display control unit 210 can display images relating to various information such as the shooting target field, flight permitted/prohibited areas, flight geofence, map information, current position information of the drone 100, attitude information (directional information), speed information, acceleration information, and remaining battery power. The "current position information" referred to here is sufficient to include information on the horizontal position of the current position of the drone 100 (i.e., latitude and longitude), and does not need to include altitude information (absolute altitude or relative altitude).

The display control unit 210 has a mode display unit 211 and a shooting status display unit 212.

The mode display unit 211 is a functional unit that displays at least the state, i.e., the mode, to which the drone 100 belongs on the display unit 201. The mode to which the drone 100 belongs is, for example, the flight mode shown in FIG. 8, but instead of or in addition to this, the aircraft state shown in FIG. 9, the aircraft action state shown in FIG. 10, the game state shown in FIG. 11, or the offensive and defensive states shown in FIG. 12 may be displayed on the display unit 201.

As shown in FIG. 18, the screen G1 displayed on the display unit 201 displays, for example, a display field G21 for the flight mode to which the drone 100 belongs, as well as a status display field G22 showing the aircraft status, aircraft behavior status, match status, and offensive/defensive status.

5 is a functional unit that displays, on the display unit 201, an image captured by the imaging camera 141 mounted on the drone 100. As shown in FIG 18, the screen G1 displayed on the display unit 201 displays, for example, an image field G40 in which an image being captured by the drone 100 is displayed.
The screen G1 and each state will be described in detail later.

(A-1-3-3. Input control unit 220)
The input control unit 220 shown in FIG. 5 receives various inputs from a user such as a pilot.
The input control unit 220 of this embodiment mainly has the following functional units: an aircraft position operation unit 221, an aircraft attitude operation unit 222, a camera attitude operation unit 223, a camera zoom operation unit 224, a flight mode switching unit 225, a target position receiving unit 226, a power supply input unit 227, and a return input unit 228.

The aircraft position operation unit 221 includes an up/down movement input unit 221a and a left/right movement input unit 221b. The aircraft attitude operation unit 222 includes a forward/backward movement input unit 222a and a yaw rotation input unit 222b.

The up-down movement input unit 221a is an input unit for allowing the pilot to move the drone 100 up and down, and acquires input to the right input stick 327R. That is, when the right input stick 327R is moved upward (toward the back when held in the hand), the drone 100 rises, and when the right input stick 327R is moved downward (toward the front when held in the hand), the drone 100 descends. The left-right movement input unit 221b is an input unit for allowing the pilot to move the drone 100 left and right, and acquires input to the right input stick 327R. That is, when the right input stick 327R is moved to the right, the drone 100 moves to the right, and when the right input stick 327R is moved to the left, the drone 100 moves to the left.

The forward/backward movement input unit 222a is an input unit for allowing the pilot to move the drone 100 forward/backward, and acquires input to the left input stick 327L. That is, when the left input stick 327L is moved upward (toward the rear when held in the hand), the drone 100 moves forward, and when the left input stick 327L is moved downward (toward the front when held in the hand), the drone 100 moves backward. The yaw rotation input unit 222b is an input unit for allowing the pilot to yaw rotate the drone 100, and acquires input to the left input stick 327L. That is, when the left input stick 327L is moved to the right, the drone 100 turns right, and when the left input stick 327L is moved to the left, the drone 100 turns left.

The camera attitude operation unit 223 is an input unit for operating the camera holding unit 142 via the shooting control unit 143 and for controlling the orientation of the shooting camera 141 relative to the housing 101 of the drone 100. The camera attitude operation unit 223 obtains input to the right slider 326R. The camera attitude operation unit 223 accepts operation of either or both of the pitch angle and yaw angle of the shooting camera 141 relative to the housing 101.

The camera zoom operation unit 224 is an input unit for operating the shooting magnification of the shooting camera 141, i.e., the zoom amount, and obtains input to the left slider 326L.

The flight mode switching unit 225 is an input unit for switching flight modes. Flight modes selectable by the flight mode switching unit 225 include at least, for example, the outer edge flight mode M102 (see FIG. 8), the inside court flight mode M105 (see FIG. 8), and the fixed position flight mode M103 or M107 (see FIG. 8). The flight mode switching unit 225 accepts switching of flight modes via, for example, a touch panel display integrated with the display unit 201.

The target position receiving unit 226 is a functional unit that receives input of a target shooting position to which the drone 100 should head. The target position receiving unit 226 receives input of a point on the stadium F. For example, when at least a portion of an image or schematic diagram of the stadium F is displayed on the display unit 201, the target position receiving unit 226 may receive input of the target shooting position via a touch panel display that is configured integrally with the display unit 201. The target position receiving unit 226 may receive a selection input of a target shooting position when a point that can be selected as the target shooting position, i.e., a shooting position, is specified in advance.

Flight Modes Here, we will explain the types of flight modes that can be set for the drone 100 and examples of their state transitions.
As shown in FIG. 8, the flight modes of the drone 100 mainly include a pre-preparation mode M100, an off-court takeoff and landing mode M101, an outer edge flight mode M102, an off-court fixed position flight mode M103, an on-court entry mode M104, an on-court flight mode M105, an off-court exit mode M106, an on-court fixed position flight mode M107, and an on-court takeoff and landing mode M108.

The advance preparation mode M100 is a mode in which advance settings such as geofence settings are made. The advance preparation mode M100 transitions to an off-court takeoff and landing mode M101. In this off-court takeoff and landing mode M101, the drone 100 takes off from point L101g (see FIG. 14). Note that in the off-court takeoff and landing mode M101, the drone 100 may take off from a point outside the court F100 other than point L101g.

The off-court takeoff and landing mode M101 is the mode to which the drone 100 belongs when control starts or ends. The drone 100 transitions from the off-court takeoff and landing mode M101 to the perimeter flight mode M102.

The outer edge flight mode M102 is a mode in which the drone flies above the outer edge along part or all of the outer edge of the court F100 to photograph the playing field F, and more specifically, flies at one of the photographing positions L101 to L105 (Fig. 14) to photograph. In the main embodiment in this description, the outer edge flight mode M102 is a mode in which the drone flies above the touchline F111b. However, the "outer edge" on which the outer edge flight mode M102 flies is a concept that includes not only directly above the touchline F111b but also slightly outside the court F100.

In the outer edge flight mode M102, the drone 100 receives user instructions via the target position receiving unit 226 of the pilot 200 and flies at one of the specified shooting positions L101 to L105. The shooting direction may be manually controlled according to the user's instructions, or may be fixed at a specified angle. In the outer edge flight mode M102, the drone 100 may change its shooting position while keeping the shooting direction fixed, a so-called dolly shooting technique, in which the drone 100 follows and shoots a specific player.

The outer edge flight mode M102 can transition to an outside court takeoff and landing mode M101, an outside court fixed position flight mode M103, or an inside court approach mode M104.

The outside-court fixed position flight mode M103 is a mode in which the drone 100 flies in a fixed position outside the area of the court F100. The outside-court fixed position flight mode M103 can transition to the outer edge flight mode M102. The on-court entry mode M104 is a mode in which the drone 100 performs a series of processes required for entering the area of the court F100. The drone 100 transitions to the on-court flight mode M105 via the on-court entry mode M104.

Intra-court flight mode M105 is a mode in which the drone flies above the court F100 to photograph the stadium F, and more specifically, flies at one of the photographing positions L206 to L215 (Figure 7) to photograph. In intra-court flight mode M105, as in outer edge flight mode M102, the drone accepts a user command to select a photographing position via the target position receiving unit 226 of the controller 200, and flies at one of the specified photographing positions L206 to L215. The photographing direction may be manually controlled according to the user's instructions, or may be fixed at a predetermined angle.

The on-court flight mode M105 can transition to an off-court exit mode M106, an on-court fixed position flight mode M107, or an on-court takeoff and landing mode M108.

The court exit mode M106 is a mode in which the drone 100 performs a series of processes required for the drone 100 to exit the area of the court F100. The drone 100 transitions from the court exit mode M106 to the outer edge flight mode M102. Note that the court exit mode M106 and the court entry mode M104 can transition back and forth.

The on-court fixed position flight mode M107 is a mode in which the drone flies in a fixed position within the area of the court F100. The on-court fixed position flight mode M107 can transition to the on-court flight mode M105. The on-court takeoff and landing mode M108 is a mode in which the drone takes off and lands within the area of the court F100, and is a mode to which the drone transitions mainly when a command to land on the spot is issued by manual intervention. A drone that takes off in the on-court takeoff and landing mode M108 transitions to the on-court flight mode M105.

Returning to FIG. 5, the power input unit 227 is a functional unit that accepts the power on/off command for the controller 200 via the power button 328.

The return input unit 228 is a functional unit that accepts a command to return the drone 100 located in the stadium F (Figure 7) to the target landing point L101g (see Figure 14) via the return button 329.

In addition to or instead of the above configuration, the input control unit 220 may be capable of receiving touch input to the display unit 201 and transmitting control commands to the drone 100 in response to the input. More specifically, for example, when the user selects appropriate information such as a map or schematic diagram displayed on the display unit 201, a route to the selected point may be automatically generated, causing the drone 100 to fly autonomously.

(A-1-3-4. Communication unit 240)
The communication unit 240 is a functional unit that transmits and receives signals between the controller 200 and an appropriate configuration included in the system 1. The controller 200 has a communication function that performs wireless communication with the drone 100 by wireless communication using Wi-Fi, 2.4 GHz, and 5.6 to 5.8 GHz frequency bands. The controller 200 also has a wireless communication function that can communicate with the server 300 via the communication network 400 using a communication standard such as LTE (Long Term Evolution). The communication unit 240 transmits various input signals by a user such as a pilot to the drone 100 or the server 300. The communication unit 240 also receives signals from the drone 100 or the server 300.

(A-1-4. Server 300)
(A-1-4-1. Overview of Server 300)
6 is a functional configuration diagram of the server 300 according to the present embodiment. The server 300 manages or controls the flight and photography of the drone 100. The server 300 includes an input/output unit (not shown) for inputting or outputting various types of information (image output, audio output).

The server 300 may be a general-purpose computer such as a workstation or personal computer, or may be logically realized by cloud computing.

The server 300 is equipped with a calculation device such as a CPU for executing information processing, and storage devices such as RAM and ROM, and as a software configuration, it mainly configures the following functional blocks: a presetting unit 310, an event detection unit 320, a photography condition determination unit 325, a flight mode switching unit 330, an outer edge flight control unit 340, an in-court flight control unit 350, a fixed position flight control unit 360, a communication unit 370, and a memory unit 380.

(A-1-4-2. Presetting unit 310)
The pre-setting unit 310 is a functional unit that performs the settings necessary for the flight of the drone 100 before the drone 100 flies over the field to be photographed.
The presetting unit 310 mainly includes a geofence setting unit 311 .

The geofence setting unit 311 is a functional unit that sets the geofence of the drone 100. The geofence includes information on the planar direction and the height direction.

The geofence setting unit 311 sets a geofence according to the flight mode. That is, for example, the geofence setting unit 311 activates the geofence G100 (see FIG. 7) in the outer edge flight mode M102 (see FIG. 8). The geofence setting unit 311 also activates the geofence G200 (see FIG. 7) in the on-court flight mode M105 (see FIG. 8). The geofence setting unit 311 also sets a geofence different from the geofence G100 or geofence G200, a so-called third geofence, in the intermediate modes that are intermediate in the transition between the outer edge flight mode M102 and the on-court flight mode M105, that is, the off-court exit mode M106 and the on-court entry mode M104.

If the geofence in the first flight mode and the geofence in the second flight mode, which is the flight mode at the destination, overlap at least in part, the third geofence is a geofence that covers a combined area that combines the first area defined by the geofence in the first flight mode and the second area defined by the geofence in the second flight mode. In this embodiment, the geofence G100 of the outer edge flight mode M102 and the geofence G200 of the inside court flight mode M105 overlap. Therefore, the third geofence is a geofence that divides the area that combines the geofences G100 and G200.

In addition, when the geofence in the first flight mode and the geofence in the second flight mode do not overlap, the third geofence may be a geofence that covers an area that combines a first area defined by the first geofence corresponding to the first flight mode, a second area defined by the second geofence corresponding to the second flight mode, and a gap between the first area and the second area. With this configuration, even when transitioning between flight modes in which the geofences do not overlap, the vehicle does not deviate from the geofence during the mode transition, ensuring safety.
The geofence setting unit 311 stores information about the set geofence in the memory unit 380.

(A-1-4-3. Event detection unit 320)
The event detection unit 320 is a functional unit that detects the state of the subject to be photographed or the drone 100. The event detection unit 320 detects an event based on the camera image of the photographing camera 141 or an input from the external system 700. The detection criteria for each event are stored in, for example, the storage unit 380, and the event detection unit 320 detects an event by referring to the storage unit 380. The event detection unit 320 may also detect an event by analysis using a neural network. The detection process by the event detection unit 320 can be performed using any known appropriate image analysis technology.

The event detection unit 320 detects events that trigger a change in the flight mode or shooting conditions of the drone 100.
The event detection unit 320 mainly has an aircraft state acquisition unit 321 , an aircraft action state acquisition unit 322 , a game state acquisition unit 323 , and an offensive/defensive state acquisition unit 324 .

The aircraft status acquisition unit 321 is a functional unit that acquires the aircraft status of the drone 100.
9 is a diagram showing the state transition of the aircraft state of the drone 100. The aircraft state is broadly divided into, for example, a normal operation flight mode M200, a detection and judgment mode M210, and an action mode M220. When the drone 100 starts flying, the drone 100 transitions to the normal operation flight mode M200.

If any detection or reception occurs in the normal operation flight mode M200, the aircraft state transitions to the detection and judgment mode M210. The detection and judgment mode M210 includes an abnormality detection mode M211, a failure detection mode M212, a manual intervention mode M213, and a low battery mode M214.

Specifically, if an abnormality is detected in normal operation flight mode M200, the mode transitions to abnormality detection mode M211. This abnormality is a transient, in other words, reversible disturbance such as a drop in radio wave strength or strong winds. If the abnormality is resolved in abnormality detection mode M211, the mode transitions to normal operation flight mode M200.

If an aircraft or system failure is detected in normal operation flight mode M200, the drone 100 transitions to failure detection mode M212. If a manual control command is received, the drone transitions to manual intervention mode M213, and if it is detected that the remaining battery charge is less than a predetermined value, the drone transitions to low battery mode M214. In addition, if a manual control command is received in abnormality detection mode M211, failure detection mode M212, or low battery mode M214, the drone transitions to manual intervention mode M213. The drone 100 transitions to an action mode M220 that corresponds to the detection judgment mode M210.

The action mode M220 is a state in which the drone 100 performs a series of actions that are preset for each state. The action mode M220 includes a landing mode M221 at an evacuation point, an emergency stop mode M222, a landing on the spot mode M223, a return mode M224, and a fixed position flight mode M225.

The landing at evacuation point mode M221 is set to fly the drone 100 to the evacuation point H200 and land it. The landing at evacuation point mode M221 is entered when the abnormality is not resolved in the abnormality detection mode M211.

The emergency stop mode M222 is set to stop the propellers 122 on the spot. In the emergency stop mode M222, the drone 100 falls freely. The emergency stop mode M222 can be selected in the manual intervention mode M213 when the propellers 122 are about to come into contact with a person or object.

The on-site landing mode M223 is set to perform a soft landing on the spot. The return mode M224 is set to return to the takeoff and landing point.

The fixed position flight mode M225 is a state in which the drone flies at a fixed position, and can transition to the normal operation flight mode M200 based on a user operation. The user operation is input, for example, by selecting a button displayed on the display unit 201. In addition, in the fixed position flight mode M225, if an event that can transition from the normal operation flight mode M200 to the detection and judgment mode M210, i.e., an abnormality, a malfunction, manual intervention, or a low battery, is detected, the drone 100 transitions from the fixed position flight mode M225 to each state of the detection and judgment mode M210 via the normal operation flight mode M200. In addition, the drone 100 in the fixed position flight mode M225 can transition to the return mode M224 based on a user operation.

The drone 100 in the abnormality detection mode M211 and the failure detection mode M212 transitions to a landing mode M221 at an evacuation point. The drone 100 in the manual intervention mode M213 transitions to one of the following states depending on the input command: landing mode M221 at an evacuation point, emergency stop mode M222, landing on the spot mode M223, return mode M224, and fixed position flight mode M225. The drone 100 in the low battery mode M214 transitions to the return mode M224.

The drone 100 in the normal operation flight mode M200 can also transition to the return mode M224 based on a user operation. The user operation is input, for example, by selecting a button displayed on the display unit 201.

The aircraft behavior state acquisition unit 322 is a functional unit that acquires the aircraft behavior state of the drone 100. Each mode of the aircraft behavior state is a sub-mode of the aircraft state that is performed to realize a transition of the aircraft state.
10 is a diagram showing state transitions of the aircraft's behavioral states. The aircraft's behavioral states are broadly divided into a takeoff mode M300, an evacuation mode M310, a normal mode M320, and a landing mode M340, for example.

Takeoff mode M300 is a mode in which drone 100 takes off. The state transition of the aircraft's behavior state starts from takeoff mode M300. When drone 100 takes off, the aircraft's behavior state transitions from takeoff mode M300 to evacuation mode M310 or normal mode M320. Evacuation mode M310 mainly includes evacuation point arrival stationary mode M311 and evacuation moving mode M312. Normal mode M320 also includes point arrival stationary mode M321 and moving mode M322. Evacuation mode M310 and normal mode M320 can transition to each other via temporary suspension mode M330. This is just one example.

When transitioning from takeoff mode M300 to evacuation mode M310, the drone 100 in takeoff mode M300 transitions to evacuation point arrival stationary mode M311. The evacuation point arrival stationary mode M311 is a mode in which the drone moves to the evacuation point H200 and remains stationary there, i.e., hovers. When moving from the evacuation point H200 to another point for the purpose of evacuation, the drone 100 in the evacuation point arrival stationary mode M311 transitions to evacuation in-motion mode M312. When the drone 100 reaches the specified destination or there is no movement command by manual intervention, the aircraft behavior state transitions from the evacuation point arrival stationary mode M311 or the evacuation in-motion mode M312 to the temporary suspension mode M330.

In addition, when transitioning from takeoff mode M300 to normal mode M320, the drone 100 in takeoff mode M300 transitions to point arrival stationary mode M321. Point arrival stationary mode M321 is a mode in which the drone moves to a specified destination and remains stationary on the spot, i.e., hovers. When moving to another location in normal use conditions, the drone 100 in point arrival stationary mode M321 transitions to moving mode M322. When the drone 100 reaches the specified destination or there is no movement command by manual intervention, the aircraft behavior state transitions from point arrival stationary mode M321 or moving mode M322 to temporary suspension mode M330.

The drone 100 in the evacuation point arrival stationary mode M311, the point arrival stationary mode M321, the moving mode M322, and the pause mode M330 can transition to the landing mode M340. The aircraft's operating state ends processing in the landing mode M340.

The game status acquisition unit 323 shown in FIG. 6 is a functional unit that acquires the game status of the competition held at the stadium F. The game status acquisition unit 323 detects the game status by performing image processing on the captured image. The game status acquisition unit 323 may also acquire the game status based on decision-related information input by the umpire to the external input device 600 or the umpire support system, which is an example of the external system 700. Furthermore, the game status acquisition unit 323 may acquire the game status based on information input from the external input device 600 held by a team member, for example, the manager or coach.

FIG. 11 is a diagram showing an example of state transitions in a match state. In this figure, an example of a match state in soccer is shown. The match state includes a pre-match state M400, a normal play state M410, and an end-of-match state M460. The state transition starts from the pre-match state M400, and transitions from the pre-match state M400 to the normal play state M410. The normal play state M410 is a state in which the game is progressing. When the match ends, transitions from the normal play state M410 to the end-of-match state M460. Note that a transition from the normal play state M410 to the end-of-match state M460 may occur not only at the end of the match, but also during a break during the match, such as halftime.

The game state also includes a play suspended without foul play state M420 and a play suspended with foul play state M440. When play is suspended without a foul play in the normal play state M410, a transition to the play suspended without foul play state M420 occurs. The play suspended without foul play state M420 occurs, for example, when the ball crosses the goal line F110a, F110b or the touch line F111a, F111b and goes out of the court. In the play suspended without foul play state M420, a transition to a throw-in state M421, a goal kick state M422, or a corner kick state M423 occurs in accordance with events that occur according to the rules of the game, such as the type of line the ball crossed or the affiliation of the player who kicked the ball out of the court. When the event in each state ends, the throw-in state M421, the goal kick state M422, or the corner kick state M423 transitions to the normal play state M410.

In the normal play state M410, if a foul occurs, or more precisely, if the referee recognizes the occurrence of a foul, a transition to the foul state M431 occurs. In addition, if an offside occurs or is recognized by the referee, a transition to the offside state M432 occurs. Then, a transition from the foul state M431 or the offside state M432 occurs to the foul play interruption state M440. In the foul play interruption state M440, a transition to the free kick state M441 or the penalty kick state M442 occurs depending on the location where the foul occurred and the event that occurred. In the free kick state M441, a so-called indirect free kick may be performed instead of a free kick. In addition, the free kick state M441 may be subdivided into a free kick state for the attacking side and a free kick state for the defending side. In the free kick state M441 and the penalty kick state M442, when the event in each state ends, the match is resumed and the match state transitions to the normal play state M410.

When the match ends, the normal play state M410 transitions to the post-match state M460, and the state transition for the match state ends.

The normal play state M410 may also transition to a penalty shootout state M443. Although not shown in the figure, the penalty shootout state M443 may transition to an end-of-match state M460, thereby terminating the state transition.

Some of the game states shown in FIG. 11 may trigger a change in flight mode, while other game states may not. For example, the flight mode may be changed based on a transition to the shaded states in the figure, i.e., the pre-game state M400, goal kick state M422, corner kick state M423, free kick state M441, penalty kick state M442, player substitution state M450, and end-of-game state M460. Note that when a transition occurs from another game state to the normal play state M410, the flight mode may be changed to one that corresponds to the offensive or defensive state.

The offensive and defensive state acquisition unit 324 is a functional unit that acquires the offensive and defensive states of the teams in the match held at the stadium F. The offensive and defensive state acquisition unit 324 detects the offensive and defensive states by performing image processing on the captured images. The offensive and defensive state acquisition unit 324 may also acquire the offensive and defensive states based on judgment-related information input by the umpire to the external input device 600 or the umpire support system, which is an example of the external system 700. Furthermore, the offensive and defensive state acquisition unit 324 may acquire the offensive and defensive states based on information input from the external input device 600 held by a team member, for example, the manager or coach.

FIG. 12 is a diagram showing an example of a state transition between offensive and defensive states. The figure shows an example of an offensive and defensive state in soccer. When a match starts, the offensive and defensive state transitions to an offensive state M510 or a defensive state M520. In addition, the offensive state M510 and the defensive state M520 transition to each other via an offensive/defensive change state M530 or an offensive/defensive uncertainty state M540.

The offensive state M510 is a state in which one of the teams (hereinafter also referred to as "Team A") designated in advance is on the offensive. An offensive state is, for example, a state in which Team A is in possession of the ball and is advancing toward the other team (hereinafter also referred to as "Team B"), but is not limited to this and may be a predetermined state determined by any determination criterion stored in advance in the memory unit 380.

The attack state M510 includes an A team offensive (own side) state M511, an A team offensive (enemy side) state M512, and an A team quick attack state M513. A transition is possible between the A team offensive (own side) state M511 and the A team offensive (enemy side) state M512, and between the A team offensive (own side) state M511 and the A team quick attack state M513. A transition is also possible from the A team quick attack state M513 to the A team offensive (enemy side) state M512.

The defensive state M520 includes a team A defensive (opponent's half) state M521, a team A defensive (own's half) state M522, and a team B fast attack state M523. The team A defensive (own's half) state M521 and the team A defensive (own's half) state M522, and the team A defensive (opponent's half) state M521 and the team B fast attack state M523 can transition to each other. Also, a transition can be made from the team B fast attack state M523 to the team A defensive (own's half) state M522.

The offense/defense switching state M530 and the offense/defense uncertain state M540 can be transitioned to from any of the following: Team A offensive (own side) state M511, Team A offensive (opponent's side) state M512, Team A quick attack state M513, Team A defensive (opponent's side) state M521, Team A defensive (own side) state M522, and Team B quick attack state M523.

The offensive and defensive state acquisition unit 324 detects a transition to a fast attack state M513, M523 in an offensive/defensive change state M530 or an offensive/defensive uncertain state M540. The offensive and defensive state acquisition unit 324 analyzes, for example, the acceleration of the ball or the players, the fluctuation in the ball's movement direction or the player's orientation, the number of players in a specified area, the movement direction of the players, the number of players moving in a certain direction, etc., from images captured by the image capture camera 141. The offensive and defensive state acquisition unit 324 detects a fast attack state M513, M523 based on the results of this analysis. The offensive and defensive state acquisition unit 324 also determines whether it is an A team fast attack state M513 or a B team fast attack state M523, depending on the movement direction of the players or the ball.

The offensive and defensive states are not limited to those described above, and any state that triggers a change in the shooting conditions may be specified. For example, a state that transitions upon detection of a long pass may be specified. In addition, the state may be specified appropriately depending on the content of the sport or event to be shot.

In addition, the event detection unit 320 may determine an event based on input information from the external system 700, instead of or in addition to the above-mentioned acquisition units 321 to 324. For example, the event detection unit 320 may determine an external disturbance such as a strong wind as an event based on input information from a weather information system, which is an example of the external system 700. The event detection unit 320 may also determine an event based on input information from a court facility system, which is another example of the external system 700, or facility information entered by a person involved with the court facility.

(A-1-4-4. Shooting condition determination unit 325)
The shooting condition determination unit 325 is a functional unit that determines the shooting conditions set in the shooting camera 141 of the drone 100. The shooting condition determination unit 325 determines the shooting conditions according to the event detected by the event detection unit 320. The shooting conditions include at least one of the target shooting position and the target shooting direction of the drone 100. The target shooting direction includes, for example, information on either or both of the pitch angle with respect to the horizontal and the yaw angle with respect to a predetermined reference direction. Furthermore, the target shooting direction may include information on the target zoom amount of the shooting camera 141. In the following description, the shooting direction will be described as including the pitch angle, the yaw angle, and the zoom amount.

The shooting conditions may also include information on the shooting range of the shooting camera 141. Note that the technical scope of the present invention is not limited to a configuration in which both are set, but rather that at least one of the target shooting position and the target shooting direction is automatically set depending on the event.

The target shooting direction is achieved by controlling at least one of the nose direction of the drone 100 and the shooting direction of the shooting camera 141. The nose direction of the drone 100 is controlled by the flight control unit 123 of the drone 100. The shooting direction of the shooting camera 141 is controlled, for example, by the shooting control unit 143 driving the camera holding unit 142. Note that the above-mentioned "control of the nose direction" and "control of the shooting direction" are concepts that include control not only in the left-right direction (the so-called "pan direction") but also in the up-down direction (the so-called "tilt direction").

The photographing condition determination unit 325 determines the photographing conditions according to the type of event detected. In the normal play state M410, the photographing condition determination unit 325 allows manual control via the controller 200. When an event is detected by the event detection unit 320, the photographing condition determination unit 325 refers to the event-photographing condition table T1 (see FIG. 13) stored in the memory unit 380 and determines the photographing conditions according to the event.

As shown in FIG. 13, the event-photography condition table T1 is a table in which events detected as game states and the photography conditions selected for the events are stored in association with each other. More specifically, in the event-photography condition table T1, events are associated with the photography range, the photography position where the drone 100 is located, the photography direction of the photography camera 141, and the zoom amount of the photography camera 141. For example, in the PK state M442 or the PK shootout state M443, the photography range is the penalty area F130a or F130b where the ball is located, and the photography position is the photography position L206 or the photography position L211. The photography direction is the direction of the goal F120a or F120b where the ball is located. The zoom amount is, for example, predetermined in stages, such as IN, Middle, and OUT, in descending order of zoom amount, and is IN in the PK state M442 or the PK shootout state M443.

In the goal kick state M422 and the defending free kick state M441, the shooting position is one of the shooting positions L101, L102, and L104 in the outer edge flight mode M102. This configuration reduces the risk of the ball colliding with the drone 100. In the attacking free kick state M441, the penalty kick state M442, and the penalty shootout state M443, the goal scene can be shot from a position where it is easier to see by shooting from one of the shooting positions L206 to L211 in the court F100. In the foul state M431, the entire court F100 can be shot by shooting from the shooting positions L101, L102, or L104 along the outer edge. In the foul state M431, the shooting direction is set so that the shooting range is around the ball or the referee's position. With this configuration, it is possible to reliably shoot the state of the game immediately after the foul, such as the referee's decision and the movements of the players. In the corner kick state M423, the area in front of the goal can be photographed up close by taking a photo at the shooting positions L207, L209, L212, or L215.

In sports, the ideal shooting direction required for events occurring in a match, such as free kicks, fast breaks, and corner kicks, varies even for the same shooting position. When shooting sports where the situation changes rapidly, it is difficult to quickly and accurately achieve shooting from the ideal shooting direction by manual operation, and there is a risk that important scenes will be missed as a result of operational errors or delays. Furthermore, if manual operation was attempted, multiple camera operators would have to be deployed. On the other hand, since the shooting direction required for each event is determined to a certain extent, the above-mentioned configuration, which automatically controls the drone 100 to a shooting position and shooting direction in accordance with the event, makes it possible to take appropriate photos according to the match situation. It also reduces the number of camera operators, contributing to labor savings.

In addition, in the case of an event in which multiple shooting positions are stored, the shooting condition determination unit 325 may select the shooting position that is closest to the ball among the stored shooting positions.

The event-photography condition table T1 may store different photography conditions set for the multiple drones 100. For example, in the foul state M431, the first drone 100 may photograph the ball location with a large zoom amount, and the second drone may photograph the ball location with a small zoom amount. The first drone 100 and the second drone 100 may also photograph while yaw rotating in opposite directions. Furthermore, the first drone 100 may photograph from the side, and the second drone 100 may photograph from directly above. While it is even more difficult to manually control multiple drones 100 to the appropriate photography conditions, the above-mentioned automatically controlled configuration makes it possible to quickly photograph from multiple angles using multiple drones 100.

Shooting with a zoom amount of "OUT" is so-called overhead shooting. In the event-shooting condition table T1, events for which overhead shooting is performed may be associated with opposite shooting directions. That is, for example, one event is associated with a shooting direction from team A's court to team B's court, and a shooting direction from team B's court to team A's court. The offensive/defensive state acquisition unit 324 detects which team is in possession of the ball, and the shooting condition determination unit 325 determines the shooting direction depending on the team in possession of the ball. Specifically, the shooting condition determination unit 325 determines that the shooting direction should be the offensive direction of the team in possession of the ball. With this configuration, it is possible to continuously take overhead shots of the moving ball.

Furthermore, the event-photography condition table T1 may be configured to accept correction input by the user, and the changes may be stored in the storage unit 380. With this configuration, the user's photography knowledge is reflected in the event-photography condition table T1, making it possible to realize more optimal automatic photography. Furthermore, multiple types of event-photography condition tables T1 may be stored in the storage unit 380, and the user may select the table to be applied. For example, this is because the content to be photographed differs depending on whether the purpose of photography is to watch a sports game or to instruct athletes, etc. With this configuration, automatic photography suited to the purpose can be easily realized. Note that the event-photography condition table T1 shown in FIG. 13 is an example of a table set for coaching purposes, for example, but it is merely one example, and the specific photography conditions stored in the table are arbitrary.

In the above embodiment, both the shooting positions L101 to L215 and the shooting direction are automatically set according to the detected event, but instead, only the shooting positions L101 to L215 may be automatically set. In this case, the shooting direction is determined by input via the controller 200. Also, when the user selects a shooting position L101 to L215 via the controller 200, the shooting condition determination unit 325 may be configured to determine the shooting direction according to the detected event and the selected shooting position L101 to L215.

The shooting condition determination unit 325 may determine the shooting conditions based on the input to the controller 200 when the event detection unit 320 has not detected an event, and may determine the shooting conditions based on the event when the event detection unit 320 has detected an event. With this configuration, even if there is user input, shooting can be reliably performed under the specified shooting conditions when an event is detected, so that shooting conditions for important shooting scenes can be maintained appropriately. Furthermore, when no event is detected, the user's freedom of shooting can be ensured, and the convenience of automatic control can be combined with the flexibility of manual operation.

In addition, when the shooting condition determination unit 325 receives an operation from the controller 200 or the external system 700, even if an event is detected by the event detection unit 320, the shooting condition determination unit 325 may determine the shooting conditions based on the operation received via the controller 200 or the external system 700. That is, the shooting conditions input from the controller 200 are applied in priority over the shooting conditions associated with the event. With this configuration, the user only needs to focus on operations that cannot be handled by automatic control, so the operational burden is reduced compared to a configuration in which everything is manually controlled, and operational errors can be reduced. In other words, with this configuration, it is possible to achieve both convenience and freedom, that is, to ensure the freedom of shooting according to the user's appropriate requests while maintaining the convenience of automatic shooting.

If the event detection unit 320 detects an event, the photographing condition determination unit 325 may determine the photographing conditions based on the event, and if the event detection unit 320 does not detect an event, the photographing condition determination unit 325 may automatically track and photograph the ball.

When the system 1 includes multiple drones 100, the shooting condition determination unit 325 determines different shooting conditions for each of the multiple drones 100. The multiple drones 100 may take pictures from different shooting positions and different shooting directions. For example, one drone may take a picture of a player taking a shot, while the other may take a picture of the goalkeeper of the opposing team.

The shooting condition determination unit 325 may also set shooting conditions for multiple drones 100 flying simultaneously such that they each shoot the same shooting range from different target shooting positions. With this configuration, important scenes can be shot from multiple angles. The shooting condition determination unit 325 may also set shooting conditions for multiple drones 100 flying simultaneously such that they each shoot an area including the same shooting range with different zoom amounts. With this configuration, it becomes possible to shoot areas that are particularly noteworthy in the stadium F under multiple shooting conditions, making it possible to more reliably shoot important scenes.

The shooting condition determination unit 325 may analyze the captured image, predict the shooting range to be captured based on the analysis results, and determine the shooting conditions. For example, the shooting condition determination unit 325 may predict the movement distance of the ball after a predetermined time by analyzing the movement direction and speed or acceleration of the ball from the captured image, and determine the shooting conditions such that the position of the ball after the predetermined time will be in the shooting range. Note that the speed of the ball may refer to the speed at the start of the prediction, i.e., the initial speed.

The photographing condition determination unit 325 may determine the photographing conditions according to the trajectory of the ball predicted by the event detection unit 320 as a result of the detection of an event. The photographing condition determination unit 325 may predict the trajectory of the ball, for example, when a fast attack state M513, M523, a long pass, or the like is detected, and determine the photographing conditions according to the predicted trajectory. The photographing condition determination unit 325 may change the photographing direction to the traveling direction of the ball and may determine to zoom out the photographing magnification. By zooming out, the traveling ball can be photographed more reliably.
Note that the ball trajectory prediction is not limited to a mode in which a predetermined trajectory is predicted and then detected as an event by the event detection unit 320. That is, for example, the shooting condition determination unit 325 may perform ball trajectory prediction by the shooting condition determination unit 325 or another functional unit separately from the event detection unit 320, and determine the shooting conditions based only on the result of the trajectory prediction.

In particular, when changing the shooting direction in response to a ball trajectory prediction, it is preferable to control the shooting direction of the shooting camera 141 by control of the shooting control unit 143 instead of controlling the nose direction. This is because changing the shooting direction based on a ball trajectory prediction requires changing the shooting direction with high response speed. That is, for example, when changing the shooting direction based on a ball trajectory prediction, the shooting direction of the shooting camera 141 is changed by the shooting control unit 143, and when changing the shooting direction not based on a trajectory prediction, the shooting direction may be changed by controlling the nose direction of the drone 100.

In addition, when a change of offense and defense or a fast break is detected, the shooting condition determination unit 325 may determine to change the shooting direction to the direction of the ball's movement and to zoom out the shooting magnification.

(A-1-4-5. Flight mode switching unit 330)
The flight mode switching unit 330 is a functional unit that switches the flight mode depending on the detection result by the event detection unit 320.
The flight mode switching unit 330 mainly has a mode switching input acquisition unit 331, a flight permitted area switching unit 332, a geofence switching unit 333, and a flight path generation unit 334.

The mode switching input acquisition unit 331 is a functional unit that acquires input information regarding switching of flight modes. The flight modes are, for example, the outer edge flight mode M102, the fixed position flight mode M103 or M107, and the on-court flight mode M105 (all see FIG. 8). Whether the off-court fixed position flight mode M103 or the on-court fixed position flight mode M107 is used for flight in a fixed position is determined according to the position of the drone 100 at the time the flight mode selection input is received. In other words, when the drone 100 is in the area outside the court F200, the off-court fixed position flight mode M103 is used, and when the drone 100 is inside the court F100, the on-court fixed position flight mode M107 is used.

The permitted flight area switching unit 332 is a functional unit that switches the permitted flight area in response to switching of the flight mode.

The geofence switching unit 333 is a functional unit that switches geofences in response to switching of flight modes. For example, when the flight mode is the outer edge flight mode M102, the geofence switching unit 333 sets a geofence G100. When the flight mode is the inner court flight mode M105, the geofence switching unit 333 sets a geofence G200. In addition, in the intermediate modes that are intermediate in the transition between the outer edge flight mode M102 and the inner court flight mode M105, i.e., the inner court entry mode M104 and the outer court exit mode M106, a third geofence different from the geofences G100 and G200 in the outer edge flight mode M102 and the inner court flight mode M105 is set.

The flight path generating unit 334 is a functional unit that generates a flight path of the drone 100 during movement involving switching of flight modes. The flight path generating unit 334 determines, for example, the shooting position at which the mode transitions to the inside-court entry mode M104 or the outside-court exit mode M106. The flight path generating unit 334 also determines the shooting position at which the mode transitions from the inside-court entry mode M104 to the inside-court flight mode M105, or the shooting position at which the mode transitions from the outside-court exit mode M106 to the outer edge flight mode M102. Furthermore, the flight path generating unit 334 generates a specific flight path in the inside-court entry mode M104 or the outside-court exit mode M106. This flight path is generated, in principle, within the area of the third geofence.

(A-1-4-6. Outer flight control unit 340)
The outer edge flight control unit 340 is a functional unit that controls the flight of the drone 100 in the outer edge flight mode. The outer edge flight control unit 340 has an imaging condition command unit 341 and a flight path generation unit 342.

The shooting condition command unit 341 is a functional unit that transmits commands regarding shooting conditions to the drone 100. The shooting conditions are, for example, a target shooting position or shooting direction. This shooting condition command unit 341 acquires a target shooting position located within the range of the flight area in the outer edge flight mode M102 from the shooting condition determination unit 325. The shooting condition command unit 341 may also acquire a target shooting position input by the user, for example, received via the target position receiving unit 226 of the controller 200.

The flight path generating unit 342 is a functional unit that generates a flight path along which the drone 100 moves in the outer perimeter flight mode M102. In other words, the flight path generating unit 342 generates a flight path in a flyable area in the outer perimeter flight mode M102. When the drone 100 belongs to a flyable area in the outer perimeter flight mode M102 and the acquired target position also belongs to a flyable area in the outer perimeter flight mode M102, the flight path generating unit 342 generates a flight path from the current position to the target position. In addition, when the drone 100 moves across the outer perimeter flight mode M102 and the on-court flight mode M105, the flight path generating unit 342 may generate a flight path in the flight range in the outer perimeter flight mode M102.

When the ball crosses the touchline F111b on which the drone 100 is flying and enters the outer court area F200, the flight path generation unit 342 moves the drone 100 to the outer court area F200 outside the touchline F111b. The flight path generation unit 342 may also perform shooting directly below the drone 100 or from that point toward the court F100. With this configuration, it is possible to follow and shoot the ball even if the ball rolls into the outer court area F200. The geofence G100 of the outer edge flight mode M102 may be set in advance to extend beyond the touchline F111b to the outside of the court F100. With this configuration, the drone 100 can be reliably maintained within the geofence G100 even when the drone 100 follows the ball and flies slightly outside the touchline F111b as described above.

When the flight path generating unit 342 detects an obstacle on the flight path or near the drone 100, it regenerates a flight path that bypasses the obstacle on the inside of the court F100. When the flight path generating unit 342 detects an obstacle, it may hover for a predetermined time and then move along the originally generated flight path. This is because while the safety of the drone 100 is not ensured in the area outside the court F200 in the stadium F, the safety of the drone 100 inside the court F100 is highly likely to be ensured. The obstacle may be detected, for example, by the obstacle detecting unit 130 of the drone 100, or by information from an external system 700 or the like.
Furthermore, when an obstacle is detected, the outer edge flight control unit 340 may switch to manual control after causing the flight path generation unit 342 to hover for a predetermined time. Furthermore, when an obstacle is detected, the outer edge flight control unit 340 may cause the flight path generation unit 342 to hover for a predetermined time, and then display a message prompting the user to re-input the target position via the display control unit 210.

(A-1-4-7. In-court flight control unit 350)
The in-court flight control unit 350 is a functional unit that controls the flight of the drone 100 in the in-court flight mode M105. The in-court flight control unit 350 has an image capture condition command unit 351 and a flight path generation unit 352. The image capture condition command unit 351 is a functional unit that transmits commands to the drone 100 regarding image capture conditions within the range of the flight area in the in-court flight mode M105. The image capture conditions are, for example, a target image capture position or image capture direction.

The flight path generating unit 352 generates a flight path along which the drone 100 moves in the on-court flight mode M105. That is, the flight path generating unit 352 generates a flight path in a flyable area in the on-court flight mode M105. More specifically, the flight path generating unit 352 generates a flight path by connecting multiple preset shooting positions in the on-court flight mode M105. Like the flight path generating unit 342 of the outer edge flight control unit 340, the flight path generating unit 352 may generate a flight path in a flight range in the on-court flight mode M105 when the current location and the target shooting position belong to flyable areas of different flight modes.

Furthermore, the flight path generating unit 352 changes the connected shooting positions depending on the event detection status. That is, when an event is detected, the flight path generating unit 352 changes the connection relationship of the shooting positions on the flight path and generates a flight path to the target shooting position.

If the flight path generating unit 352 detects an obstacle on the flight path or near the drone 100, it regenerates a flight path that bypasses the obstacle. The obstacle is detected by, for example, the obstacle detection unit 130. The flight path generating unit 352 may regenerate the flight path by changing the connection between multiple shooting positions that have been set in advance, or may change the flight path to a higher altitude while maintaining the flight path on a plane. In addition, the flight path generating unit 352 may hover for a predetermined time when it detects an obstacle, and then move along the flight path that was initially generated. Furthermore, the in-court flight control unit 350 may switch to manual control after hovering for a predetermined time by the flight path generating unit 352 when it detects an obstacle. Furthermore, the in-court flight control unit 350 may display a message prompting the user to re-input the target position via the display control unit 210 after hovering for a predetermined time by the flight path generating unit 352 when it detects an obstacle. The obstacle may be, for example, a bird, a fixed facility, or a player. The obstacles also include balls.

In this embodiment, flight control in the outer edge flight mode M102 is performed by the outer edge flight control unit 340, and flight control in the inner court flight mode M105 is performed by the inner court flight control unit 350. Then, the shooting positions defined in each of the outer edge flight mode M102 and the inner court flight mode M105 are presented as options, and a flight path to the selected target position is generated. However, the technical scope of the present invention is not limited to this, and the pilot 200 may control the shooting position and orientation of the drone 100 to fly freely at any position in the area within the geofences G100 and G200 set corresponding to each flight mode.

Furthermore, the functional configuration of the flight path generation units 334, 342, and 352 is an example, and for example, a single flight path generation unit may generate the flight path without subdivision.

FIG. 14 is a schematic diagram showing the routes that the drone 100 can follow, defined by the shooting positions L101-L105, L206-L215, and the evacuation point H200. Point L101g on the ground at the shooting position L101 is the takeoff and landing point for the drone 100. The position transition of the drone 100 begins with the step of taking off from point L101g and arriving at the shooting position L101. The drone 100 also descends at the shooting position L101, and lands at point L101g to end shooting.

At each of the shooting positions L101 to L105 and L206 to L215, the drone 100 may only be able to transition to adjacent shooting positions. For example, the point to which the drone 100 at shooting position L105 can transition while maintaining the outer edge flight mode is shooting position L104. Furthermore, the points to which the drone 100 at shooting position L105 can transition after switching to the inside court flight mode M105 are shooting positions L106 and L107. The flight path generation unit 334, 342, or 352 (see FIG. 6) generates a flight path for the drone 100 by referring to the possible transition paths.

When the target position receiving unit 226 receives the selection of the shooting position, the drone 100 transitions to the selected shooting position via the available shooting positions. For example, when the drone 100 is at shooting position L105 and shooting position L215 is selected, the flight path generating unit 334, 342, or 352 (see FIG. 6) generates a flight path that transitions through shooting positions L105, L207, L208, L213, L212, and L215 in that order, and the drone 100 flies along this flight path.

In addition, instead of a flight path generated by connecting the shooting positions L101 to L215, the drone 100 may fly on a flight path that connects the current position to the target shooting position in a straight line. Furthermore, the drone 100 may transition to an adjacent shooting position in the case of a transition of the shooting position accompanied by a flight mode switch, and may be able to transition directly to a non-adjacent shooting position in the case of a transition without a flight mode switch. That is, for example, when the drone moves from shooting position L105 to shooting position L215, it may transition from shooting position L105 to shooting position L207 accompanied by a mode switch, and then move linearly from shooting position L107 to shooting position L215 in the area within the geofence G200.

The outer edge flight control unit 340 and the inner court flight control unit 350 fly the drone 100 autonomously in each flight area according to the flight mode. For example, the outer edge flight control unit 340 and the inner court flight control unit 350 may perform dolly shooting within each flight area, that is, the drone 100 may automatically follow and shoot a specific object such as a ball or a specified player. The outer edge flight control unit 340 and the inner court flight control unit 350 may automatically control the flight height of the drone 100. The autonomous flight mode may differ depending on the flight mode. For example, dolly shooting may be performed when controlled by the outer edge flight control unit 340, while automatic follow-up shooting of only the shooting direction with a fixed shooting position, or automatic follow-up shooting of the position and shooting direction may be performed when controlled by the inner court flight control unit 350. The outer edge flight control unit 340 and the inner court flight control unit 350 may also generate a flight path within the court (within the stadium F) for moving the drone 100 to a target position specified by the user in each flight area.

(A-1-4-8. Fixed position flight control unit 360)
The fixed position flight control unit 360 is a functional unit that controls the flight of the drone 100 in the off-court fixed position flight mode M103 and the on-court fixed position flight mode M107. In the fixed position flight mode, the fixed position flight control unit 360 hovers at a predetermined position and controls the nose direction or the direction of the shooting camera 141 to follow a specific player or the ball and perform automatic shooting. Note that the above-mentioned "control of direction" is a concept that includes control not only in the left-right direction (so-called "pan direction") but also in the up-down direction (so-called "tilt direction").

The fixed position flight control unit 360 includes an image capture condition command unit 361. The image capture condition command unit 361 is a functional unit that transmits commands for the target position and the target image capture direction in the fixed position flight mode M103 or M107. The image capture direction may be information determined by the image capture condition determination unit 325, or may be information input by the user via the controller 200.

(A-1-4-9. Communication unit 370)
The communication unit 370 has a modem or the like (not shown) and is capable of communicating with the drone 100, the controller 200, and the like via the communication network 400. The communication unit 370 may, for example, monitor the state of the drone 100 and its surroundings and notify the controller 200.

(A-1-4-10. Storage unit 380)
The storage unit 380 is a functional unit that stores information related to flight control of the drone 100, and is, for example, a database. The storage unit 380 stores, for example, the coordinates of multiple shooting positions L101 to L105, L210 to L215 in the stadium F. These coordinates may be two-dimensional coordinates on a plane or three-dimensional coordinates including information in the height direction. The storage unit 380 also stores an event-shooting condition table T1 shown in FIG. 13. As described above, the event-shooting condition table T1 is recorded in a rewritable manner. Furthermore, multiple event-shooting condition tables T1 may be stored.

● Flowcharts Fig. 15 is a flowchart showing the overall flow of aerial photography control in this embodiment. Fig. 16 is a subroutine of the flight restriction process S1002 in Fig. 15. Fig. 17 is a subroutine of the photography condition switching process S1010 in Fig. 15.

The control described in the flowchart shown in FIG. 15 is executed in a regular loop. As shown in FIG. 15, if it is detected that the drone 100 is approaching the vicinity of geofences G100, G200 while flying (YES in step S1001), the process proceeds to flight restriction processing in step S1002. The subroutine of the flight restriction processing S1002 is explained in FIG. 15.

If it is not detected in step S1001 that the drone is approaching the vicinity of geofences G100, G200 (NO in step S1001), the presence or absence of an obstacle in the path or near the drone 100 is detected (step S1003). If an obstacle is detected in step S1003 (YES in step S1003), the drone 100 is caused to hover or a detour route is generated, and the flight route of the drone 100 is changed to the detour route (step S1004).

If no obstacle is detected in step S1003 (NO in step S1003), the presence or absence of an action determination of the aircraft state is detected (step S1005). If an action determination of the aircraft state is detected in step S1005 (YES in step S1005), the process proceeds to step S1006, where an event type determination process is executed (step S1006).

If no action determination is detected in step S1005 (NO in step S1005), it is determined whether or not there is an input from the controller 200 by the user (step S1007). If an input from the controller 200 is detected in step S1007 (YES in step S1007), a command based on the input is executed (step S1008).

If no input from the controller 200 is detected in step S1007 (NO in step S1007), the presence or absence of an event is determined (step S1009). If an event is detected (YES in step S1009), the process proceeds to the shooting condition switching process in step S1010 (step S1010). If no event is detected in step S1009 (NO in step S1009), the process returns to step S1001 and steps S1001 to S1009 are repeated.

As shown in FIG. 15, the overall processing of aerial photography control is performed in the following order: geofence restriction, obstacle detection, control based on aircraft status, control based on user input, and control based on in-game events such as the game status or offensive and defensive status. In other words, each control is executed before control based on in-game events. This order is based on the priority of performing safe control processing. With this configuration, the safety of flying the drone 100 can be more reliably guaranteed.

As shown in FIG. 16, first, the flight control unit 123 of the drone 100 issues an operation command to restrict the drone 100 from advancing outside the geofence (step S1101). In step S1101, a restriction is set on the flight target position so that the drone 100 does not advance outside the geofence even when the drone 100 is manually operated.

Next, if the drone 100 does not advance outside the geofence (NO in step S1102), the process ends.

If the drone 100 advances outside the geofence (YES in step S1102), proceed to step S1103. Possible causes of this situation include, for example, the wind being too strong and causing the aircraft to be swept away, or a malfunction in the drone 100 preventing it from flying in the intended direction. In step S1103, the flight control unit 123 issues an operation command to return to the geofence. More specifically, in step S1103, a flight target position command to return to the geofence, that is, an operation command to set the flight target position to a specified point inside the geofence, is given to the drone 100.

Next, by referring to information measured by the measurement unit 110 of the drone 100, such as information on position, direction, altitude, or speed, it is determined whether the drone 100 is approaching the inside of the geofence (step S1104). In this case, step S1104 is executed a predetermined time after step S1103. Note that in step S1104, it is sufficient if the drone 100 is closer to the inside of the geofence than at the time of step S1103, and it is not necessary to determine whether the drone 100 is located inside the geofence.

If it is determined in step S1104 that the drone 100 is not approaching the geofence (NO in step S1104), it is determined that the above operation command is ineffective, and the flight control unit 123 forces the drone 100 to land (step S1105).

If the drone 100 is approaching the geofence in step S1104 (YES in step S1104), proceed to step S1106. In step S1106, information measured by the measurement unit 110 of the drone 100, such as the position and altitude, is referenced to determine whether the drone 100 is located within the geofence. If the drone 100 is located within the geofence (YES in S1106), end the process. If the drone 100 is not located within the geofence (NO in S1106), return to step 1104 and continue operation based on the operation command to return to the geofence until the drone 100 returns within the geofence.

FIG. 17 is a diagram showing an example of a switching process flow for switching shooting conditions. First, when a specific event is detected by the event detection unit 320, the shooting condition determination unit 325 refers to the event-shooting condition table T1 to determine the target values for the shooting position and shooting direction, and transmits a control command to the drone 100 (step S1301). Next, it is determined whether an operation command due to manual intervention by the user has been received from the controller 200 (step S1302). If an operation command has not been received from the controller 200 (NO in step S1302), flight by automatic control continues until the target shooting position and target shooting direction are reached (step S1303). If an operation command has been received from the controller 200, an operation based on the operation command due to manual intervention is executed (step S1304).

When the target shooting position and the target shooting direction are reached in step S1303, the process proceeds to step S1305. In step S1305, the mode transitions to manual control mode, and a message is displayed on the operation screen G3 (see FIG. 20) indicating that manual operation is permitted.

Display Examples of the Display Unit 201 FIGS. 18 and 19 are examples of screens G1 and G2 displayed on the display unit 201 of the controller 200. FIG.

　The screen G1 shown in FIG. 18 displays a field map G10 that shows a schematic bird's-eye view of the stadium F and the shooting positions L101 to L215, an icon G11 that shows the position information of the drone 100, the shooting range G12 captured by the shooting camera 141, a display field G21 of the flight mode to which the drone 100 belongs, a status display field G22 that shows the status detected by the event detection unit 320, such as the aircraft status, aircraft behavior status, match status, and offensive and defensive status, a landing button G30 for landing the drone 100, and a video field G40 in which images captured by the drone 100 are displayed. In the example of FIG. 18, the display field G21 displays the main control modes, either automatic control mode or manual control mode. The drone is flying in the outer edge flight mode at the shooting position L101.

The position and shooting direction of the drone 100 may be controlled manually, or automatic tracking control of the ball or a specific player may be performed. When automatic tracking control is performed, information about the ball or specific player being tracked may be displayed on the screen G1.

The icon G11 representing the drone 100 displays an arrow indicating the direction of travel of the drone 100. Note that the direction of the nose of the drone 100 is not limited to the direction of travel of the drone 100, and may be pointing in any direction. The direction of the nose of the drone 100 does not have to be constant while moving, and for example, the drone 100 may move while photographing a player or the ball by rotating in a yaw motion.

FIG. 19 shows an example of the display on screen G2 when multiple drones 100 are photographing one stadium F. FIG. 19 particularly shows an example of the display when a free kick state M441 is detected. On screen G2, icons 11a and 11b of two drones are displayed on a field map G10. In addition, the shooting ranges G13a and G13b photographed by each of the multiple drones 100, and video columns G40a and G40b showing the captured images are displayed in association with the icons 11a and 11b of the corresponding drones 100.

In the example of FIG. 19, the first drone 100 corresponding to icon 11a is taking localized shots near the goal 120a, while the second drone 100 corresponding to icon 11b is taking bird's-eye shots from a shooting position L101 on the outer edge. The shooting angles of the first drone and the second drone are different from each other. It is preferable that the shooting conditions of the first drone and the second drone are such that they complement each other's positions where they cannot shoot. In this way, with a configuration in which multiple drones 100 shoot under different shooting conditions, the stadium F can be photographed from multiple angles.

FIG. 20 is an example of screen G3 displayed on the display unit 201 when an event has not been detected by the event detection unit 320. Because an event has not been detected, automatic control of the drone 100 is not performed, and the display field G21 indicates that the mode is "manual control mode."

FIG. 21 is an example of screen G4 when the event detection unit 320 detects an event in the state of screen G3. In this case, the drone 100 switches to automatic control in response to the detection of the event, and the display field G21 indicates that it is in "automatic control mode." In addition, the status display field G22 indicates that a team A quick attack state M513 has been detected as the offensive and defensive state.

When a fast attack state M513 or M523 is detected, in FIG. 21, an arrow G15 indicating the predicted trajectory of the ball and the ball G16 after movement based on the trajectory are displayed on the field map G10. Also, an arrow G17 indicating the change in the shooting direction of the shooting camera 141 is displayed. With this configuration, the reason for the operation of the drone 100 is clear to the user even in automatic control mode, providing the user with a sense of security.

[A-2. Effects of this embodiment]
According to this embodiment, it is possible to reduce the labor required for photographing and to photograph an object appropriately according to the situation of the object.

The present invention is not limited to the above embodiment, and various configurations can be adopted based on the contents of this specification.

The series of processes described in relation to the above embodiment may be implemented using software, hardware, or a combination of software and hardware. A computer program for implementing each function of the server 300 according to this embodiment may be created and implemented in a PC or the like. A computer-readable recording medium on which such a computer program is stored may also be provided. Examples of the recording medium include a magnetic disk, an optical disk, a magneto-optical disk, and a flash memory. The above computer program may also be distributed, for example, via the communication network 400, without using a recording medium.

The flowcharts used in the above embodiments do not necessarily have to be executed in the order shown. Some processing steps may be executed in parallel. In addition, additional processing steps may be employed, and some processing steps may be omitted.

1 Aerial photography system 100 Drone (mobile body)
141 Shooting camera 200 Pilot 220 Input control unit 300 Server 320 Event detection unit 330 Flight mode switching unit 331 Mode switching determination unit 380 Memory unit F Stadium F100 Court F200 Area outside the court M102 Outer edge flight mode M105 Inner court flight mode

Claims (15)

A moving object flying in a target area;
A camera mounted on the moving object for photographing the target area;
an event detection unit that detects an event based on a captured image acquired by the camera or an input from an external system;
an image capturing condition determination unit that determines image capturing conditions including at least one of a target image capturing position and a target image capturing direction of the moving object in response to the detected event;
Equipped with
Aerial photography system.
the photographing condition determination unit determines the photographing condition according to a type of the detected event.
2. The aerial photography system according to claim 1.
The target shooting direction is achieved by controlling at least one of a nose direction of the moving body and an angle of the camera relative to the moving body.
2. The aerial photography system according to claim 1.
The photographing conditions include a target zoom amount of the camera.
2. The aerial photography system according to claim 1.
When the photographing condition determination unit receives an input of the photographing conditions from a controller of the moving body or the external system, even if the event detection unit detects the event, the photographing condition determination unit determines the photographing conditions based on an operation received via the controller or the external system.
2. The aerial photography system according to claim 1.
Further comprising a controller for receiving input of the photographing conditions by a user,
The photographing condition determination unit determines the photographing condition based on an input via the controller when the event detection unit does not detect the event, and determines the photographing condition based on the event when the event detection unit detects the event.
2. The aerial photography system according to claim 1.
An aerial photography system including a plurality of the moving bodies, the aerial photography system photographing one target area by flying the plurality of the moving bodies simultaneously in the target area,
The photographing condition determination unit determines different photographing conditions for each of the plurality of moving objects.
2. The aerial photography system according to claim 1.
The photographing condition determination unit sets, for the plurality of moving bodies flying simultaneously, the photographing condition for photographing the same photographing range from the different target photographing positions, or the photographing condition for photographing an area including the same photographing range with different zoom amounts,
8. The aerial photography system according to claim 7.
the photographing condition determination unit determines the photographing condition in accordance with a prediction result of a ball trajectory predicted by the event detection unit as a detection result of the event.
2. The aerial photography system according to claim 1.
the photographing condition determination unit, when an event indicating that a foul has occurred in a game held in the target area is detected, determines the photographing condition to set a photographing range around a ball used in the game or a position of a referee of the game;
2. The aerial photography system according to claim 1.
An aerial photography system including a plurality of the moving bodies, the aerial photography system photographing one target area by flying the plurality of the moving bodies simultaneously in the target area,
the photographing condition determination unit, when an event indicating that the foul has occurred in the game is detected, determines the photographing conditions in which the photographing range is the periphery of the ball or a point of a referee of the game at the target photographing positions, the target photographing directions, or the zoom amounts different from one another by the multiple moving objects;
The aerial imaging system according to claim 10.
A flight path generating unit that generates a flight path of the moving object,
The flight path generation unit automatically generates the flight path to the target shooting position that is determined based on the event detected from the captured image.
2. The aerial photography system according to claim 1.
The flight path generation unit generates the flight path within a court configured within the target area,
the flight path generation unit generates the flight path to the target shooting position by connecting a plurality of shooting positions set in advance, and changes the shooting positions to be connected depending on the detection status of the event.
13. The aerial imaging system according to claim 12.
An event detection step of detecting an event based on an image captured by a camera capturing an image of a target area or an input from an external system;
a photographing condition determination step of determining photographing conditions including at least one of a target photographing position and a target photographing direction of a moving body equipped with the camera in response to the detected event;
Equipped with
Aerial photography method.
An event detection command for detecting an event based on an image captured by a camera capturing an image of a target area or an input from an external system;
a photographing condition determination command for determining photographing conditions including at least one of a target photographing position and a target photographing direction of a moving body equipped with the camera in response to the detected event;
Executing the above by a computer,
Aerial photography program.

Images