WO2025169696A1 - Flying body detecting device, flying body detecting system, flying body detecting method, and program storage medium - Google Patents

Abstract

In order to prevent the detection accuracy of a flying body in a detection region from decreasing due to the distance of said flying body from an imaging device that captures an image of the detection region, this flying body detecting device comprises the following functions. The flying body detecting device acquires a telephoto image, which is a captured image captured by, from among a plurality of types of imaging devices having different angles of view installed at a common installation location for capturing images of a detection region, a telephoto-type imaging device for capturing an image of a distant portion distant from the installation location in the detection region. The flying body detecting device acquires a wide-angle image, which is a captured image captured by a wide-angle-type imaging device from among the plurality of types of imaging devices. The flying body detecting device detects, from the telephoto image, a detection target flying body in a distant portion in the detection region, and detects, from the wide-angle image, a detection target flying body in a detection region portion other than the distant portion. The flying body detecting device generates a detection result reflecting image by superimposing, on the wide-angle image, information indicating the position of the detected flying body in the image.

Description

Flying object detection device, flying object detection system, flying object detection method, and program storage medium

This disclosure relates to an airborne object detection device, an airborne object detection system, an airborne object detection method, and a program storage medium for detecting airborne objects.

No-fly zones (for example, the airspace above airports and important facilities and the surrounding areas) have been designated for unmanned aerial vehicles (small unmanned aircraft), also known as drones and UAVs (Unmanned Aerial Vehicles), and a flight permit is required for unmanned aircraft to fly in these no-fly zones. However, with the increasing use of unmanned aircraft, there are concerns that an increasing number of unmanned aircraft will enter these no-fly zones without permission.

Patent Document 1 (JP 2007-116666 A) discloses technology for efficiently monitoring surveillance areas spanning several kilometers or more, and accurately capturing and photographing moving objects within the surveillance area. That is, with the technology shown in Patent Document 1, the entire surveillance area is photographed with a wide-angle camera. When a moving object is detected in an image captured by the wide-angle camera, the attitude of the telephoto camera is controlled to point the optical axis of the telephoto camera in the direction of the moving object. In other words, the telephoto camera photographs the moving object while tracking it.

Japanese Patent Application Laid-Open No. 2007-116666

In the technology disclosed in Patent Document 1, the presence or absence of moving objects in the entire monitoring area is first detected from images captured by a wide-angle camera. As a result, even if a moving object moves into a part of the monitoring area far from the wide-angle camera, it may not be possible to properly detect the moving object because the distant moving object appears small in the image captured by the wide-angle camera. In other words, the accuracy of detecting moving objects decreases in parts of the monitoring area far from the camera.

This disclosure was conceived to solve such problems. In other words, the main purpose of this disclosure is to provide technology that prevents the detection accuracy of an aerial vehicle in a detection area from decreasing due to the distance from the imaging device that captures the detection area.

In order to achieve the above-mentioned object, the flying object detection device of the present disclosure, as one aspect thereof,
an acquisition unit that acquires a telephoto image, which is an image captured by a telephoto type imaging device that captures a distant part of the detection area that is far from the installation location, among multiple types of imaging devices with different angles of view that are installed at a common installation location to capture images of the detection area, and a wide-angle image, which is an image captured by a wide-angle type imaging device among the multiple types of imaging devices;
a detection unit that detects a target flying object in a distant part of the detection area from a telephoto image and detects a target flying object in a part of the detection area other than the distant part from a wide-angle image;
a generation unit that generates an image reflecting the detection result by superimposing information indicating the position of the detected flying object in the image on the wide-angle image;
and an output unit that outputs an image reflecting the detection result.

In addition, one aspect of the flying object detection system of the present disclosure is
a telephoto type imaging device that captures an image of a distant part of the detection area that is far from the installation location, among a plurality of types of imaging devices with different angles of view that are installed at a common installation location for capturing an image of the detection area;
a wide-angle type imaging device among the plurality of types of imaging devices that captures images of a detection area including a portion that is not captured by the telephoto type imaging device; and
and the above-mentioned flying object detection device that uses images captured by the telephoto type imaging device and the wide-angle type imaging device.

Furthermore, in one aspect, the flying object detection method of the present disclosure includes:
By computer,
Among a plurality of types of imaging devices with different angles of view installed at a common installation location for imaging the detection area, a telephoto image is acquired which is an image captured by a telephoto type imaging device which captures a distant portion of the detection area that is far from the installation location, and a wide-angle image is acquired which is an image captured by a wide-angle type imaging device among the plurality of types of imaging devices,
Detecting a target flying object in a distant part of the detection area from a telephoto image, and detecting a target flying object in a part of the detection area other than the distant part from a wide-angle image;
generating an image reflecting the detection results by superimposing information representing the position of the detected flying object in the image on the wide-angle image;
An image reflecting the detection results is output.

Furthermore, in one aspect, the program storage medium of the present disclosure includes:
a process of acquiring a telephoto image, which is an image captured by a telephoto type imaging device that captures a distant portion of the detection area that is far from the installation location, among multiple types of imaging devices with different angles of view that are installed at a common installation location to capture the detection area, and a wide-angle image, which is an image captured by a wide-angle type imaging device among the multiple types of imaging devices;
A process of detecting a target flying object in a distant part of the detection area from a telephoto image, and detecting a target flying object in a part of the detection area other than the distant part from a wide-angle image;
A process of generating an image reflecting the detection result by superimposing information representing the position of the detected flying object in the image on the wide-angle image;
The computer program for causing a computer to execute a process of outputting an image reflecting the detection result is stored.

According to the present disclosure, it is possible to prevent the detection accuracy of an aerial vehicle in a detection area from decreasing due to the distance from the imaging device that captures the detection area.

1 is a diagram illustrating the configuration of an embodiment of an air vehicle detection device according to the present disclosure. FIG. 2 is a diagram illustrating an imaging device that constitutes the flying object detection system according to the present disclosure. 2 is a diagram illustrating the imaging device. FIG. FIG. 10 is a diagram illustrating an example of a detection result reflection image. FIG. 10 is a diagram illustrating another example of the detection result reflection image. FIG. 10 is a diagram illustrating yet another example of the detection result reflection image. 10 is a flowchart illustrating an example of an operation related to detecting a flying object in the flying object detection device. 10A and 10B are diagrams illustrating modified examples of the generation unit in the flying object detection device. 10A and 10B are diagrams illustrating other embodiments. FIG. 10 is a diagram illustrating another embodiment. 10A and 10B are diagrams illustrating still another embodiment of the flying object detection device. 10 is a flowchart illustrating an example of an operation of the flying object detection device for detecting another flying object.

Embodiments of this disclosure are described below with reference to the drawings.

First Embodiment
The flying object detection system 1 of the first embodiment according to the present disclosure is a system that detects a target flying object in a detection area using images captured by an imaging device, and includes multiple types of imaging devices 2 and 3 and an flying object detection device (hereinafter also simply referred to as a detection device) 5, as shown in Fig. 1. The detection device 5 is a computer device that detects flying objects in the detection area by image analysis of the images captured by the imaging devices 2 and 3.

In the first embodiment, the flying object detection system 1 is applied to a surveillance system. A surveillance system to which the flying object detection system 1 is applied is, for example, a system that monitors the airspace above and surrounding areas of important facilities such as airports and nuclear power plants, and the surveillance area includes no-fly zones where the flight of unmanned aerial vehicles is prohibited by law. Because the flying object detection system 1 is applied to such a surveillance system, the detection area of the flying object detection system 1 is the surveillance area of the surveillance system to which it is applied.

The target air vehicles to be detected by the air vehicle detection system 1 are determined in advance by a system designer, taking into consideration, for example, the type of facility being monitored by the applicable monitoring system and its surrounding environment. Examples of air vehicles set as detection targets include unmanned aerial vehicles (unmanned aerial vehicles) such as unmanned airplanes, unmanned rotorcraft, and unmanned airships, which can be flown by remote control or automatic piloting, as well as balloons, hang gliders, and paragliders, which are flown by humans using specific aviation equipment. In this way, various air vehicles can be set as detection target air vehicles, but in the first embodiment, so-called drones (unmanned aerial vehicles) are set as detection target air vehicles.

Several types of imaging devices with different angles of view are used as the imaging devices that make up the flying object detection system 1. One type of imaging device used in the flying object detection system 1 is a telephoto type imaging device equipped with a telephoto lens. Another type of imaging device used in the flying object detection system 1 is a wide-angle type imaging device equipped with a wide-angle lens. The lenses provided in the imaging devices can be classified into three types depending on the angle of view: "standard lens," "wide-angle lens," and "telephoto lens." A "standard lens" is a lens with an angle of view of approximately 45 to 50 degrees, which is said to be close to the human field of view. A "wide-angle lens" is a lens with a wider angle of view than a "standard lens," for example, 60 degrees or more. A "telephoto lens" is a lens with a narrower angle of view than a "standard lens," for example, 30 degrees or less.

In the following description, a wide-angle type imaging device will also be referred to as a wide-angle camera 2, and a telephoto type imaging device will also be referred to as a telephoto camera 3.

In the flying object detection system 1, the wide-angle camera 2 and telephoto camera 3 are installed in a common installation location where they can capture images of the detection area, with the image capture direction fixed. For example, the wide-angle camera 2 and telephoto camera 3 are installed at a height H(y) of approximately 10 meters, allowing them to view the detection area as shown in Figure 2. Figure 2 shows a schematic diagram of an example of the capture range of the detection area of the wide-angle camera 2 and telephoto camera 3 installed in this manner, viewed from a direction along the ground surface. In Figure 2, the portion of the field of view of the wide-angle camera 2 that captures the detection area is represented by the hatched area Zw. Furthermore, the portion of the detection area that is captured by the telephoto camera 3 is represented by the lightly shaded area Zt. Note that unmanned aerial vehicles (such as drones) are subject to altitude restrictions, such as prohibiting flight in airspace above 150 meters above the ground. The example in Figure 2 shows a detection area that takes into account altitude restrictions for such flying objects (unmanned aerial vehicles).

Telephoto camera 3 captures images of the distant part of the detection area that is far from the installation location of telephoto camera 3. For example, the camera settings (magnification, etc.) of telephoto camera 3 are configured to capture images of the distant part of the detection area, such as a distance D(z) of approximately 800 meters from the installation location of telephoto camera 3. Here, the image captured by telephoto camera 3 is also referred to as a telephoto image. The telephoto image is used by detection device 5 to detect the target flying object in the distant part of the detection area.

By the way, telephoto camera 3 can take magnified images of distant objects. In other words, the field of view of telephoto camera 3 is narrow. For this reason, the range of distant areas that telephoto camera 3 can capture is limited. Figure 3 shows a schematic diagram of an example of the capture range in each detection area of wide-angle camera 2 and telephoto camera 3, viewed from the zenith. In Figure 3, as in Figure 2, the portion of the field of view of wide-angle camera 2 that captures the detection area is represented by the hatched portion Zw. Furthermore, the portion of the detection area that is captured by telephoto camera 3 is represented by the lightly shaded portion Zt.

As shown in Figure 3, there are cases where a single telephoto camera 3 is unable to capture the entire distant portion of the detection area. In such cases, in the flying object detection system 1, multiple telephoto cameras 3 are installed in the same installation location, and the shooting directions of the multiple telephoto cameras 3 are set so that the shooting ranges of these telephoto cameras 3 are shifted and the entire distant portion is captured by the multiple telephoto cameras 3. In other words, the distant portion of the detection area is captured by multiple telephoto-type shooting devices with mutually shifted fields of view. Note that in the example of Figure 3, the shooting directions of the multiple telephoto cameras 3 are set so that the shooting ranges are shifted in a direction along the ground surface, but depending on the height H(d) of the detection area from the ground surface and the width of the field of view of the telephoto cameras 3, the shooting directions of the multiple telephoto cameras 3 may be set so that the shooting ranges are shifted in the vertical direction. Furthermore, the number of telephoto cameras 3 installed depends on the width of the distant portion of the detection area and the width of the field of view of the telephoto cameras 3, and is not limited.

Wide-angle camera 2 has a field of view that compensates for the portions of the detection area not captured by telephoto camera 3. Here, the image captured by wide-angle camera 2 is also referred to as a wide-angle image. The wide-angle image is used to detect flying objects in portions of the detection area that are closer to the installation location than the distant portions captured by telephoto camera 3 (detection area portions other than the distant portions (hereinafter also referred to as the portion covered by the wide-angle camera)). For this reason, the camera settings of wide-angle camera 2 are configured so that flying objects in the portion of the detection area covered by the wide-angle camera are clearly visible. Depending on the size of the detection area, multiple wide-angle cameras 2 may be installed in the same installation location, and the shooting directions of these wide-angle cameras 2 may be set so that the shooting ranges of these wide-angle cameras 2 are shifted and the entire detection area is captured by multiple wide-angle cameras 2.

Here, suppose that wide-angle camera 2 and telephoto camera 3 capture images of flying objects of the same size as the target of detection, flying in the wide-angle camera's portion and the distant portion of the detection area. In this flying object detection system 1, in such a case, the cameras are set so that the difference between the size of the target of detection object captured in the telephoto image and the size of the target of detection object captured in the wide-angle image is within a predetermined range (tolerance). Preferably, the cameras are set so that the size of the target of detection object captured in the telephoto image is the same as the size of the target of detection object captured in the wide-angle image. Such camera settings are related to reducing the load on the detection device 5 in the detection process for the target flying object. Taking into account the reduction in the load of the detection process, the tolerance range related to the difference in size of the target of detection object captured in the telephoto image and the wide-angle image described above is set.

The detection device 5 is a computer device that detects target flying objects by analyzing wide-angle images captured by the wide-angle camera 2 and telephoto images captured by the telephoto camera 3. That is, as shown in FIG. 1, the detection device 5 is directly or indirectly connected to each of the wide-angle camera 2 and the telephoto camera 3, and includes an arithmetic unit 50 and a storage device 40. The storage device 40 includes a storage medium for storing data and a computer program (hereinafter also referred to as a program) 41. There are multiple types of storage devices, such as magnetic disk drives and semiconductor memory devices. Furthermore, there are many types of semiconductor memory devices, such as RAM (Random Access Memory) and ROM (Read Only Memory). A computer device may be equipped with multiple types of storage devices depending on their intended use, but these storage devices will be collectively referred to as the storage device 40. Furthermore, the types and number of storage devices 40 included in the detection device 5 are not limited, and a description thereof will be omitted. The detection device 5 may also be connected to a database 6, which is a storage device. In this case, the detection device 5 may write information to or read information from the database 6, but to avoid complicating the explanation, we will omit the explanation here even if such a case occurs.

The arithmetic device 50 is composed of processors such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The arithmetic device 50 can have functions based on a program 41 by reading and executing the program 41 stored in the storage device 40. Here, the arithmetic device 50 has an acquisition unit 51, a detection unit 52, a generation unit 53, and an output unit 54 as functional units related to detecting flying objects.

The acquisition unit 51 acquires wide-angle images captured by the wide-angle camera 2 and telephoto images captured by the telephoto camera 3. The method by which the acquisition unit 51 acquires wide-angle images and telephoto images is not limited here; for example, the acquisition unit 51 may acquire captured images (wide-angle images and telephoto images) from the wide-angle camera 2 and telephoto camera 3, respectively, or may acquire wide-angle images and telephoto images by reading them from a database (not shown) that has temporarily stored the wide-angle images and telephoto images from the wide-angle camera 2 and telephoto camera 3, respectively.

The detection unit 52 detects target flying objects in the distant part of the detection area from the telephoto image, and detects target flying objects in the part of the detection area covered by the wide-angle camera (the part of the detection area other than the distant part) from the wide-angle image. Various methods have been proposed for detecting target flying objects from captured images (wide-angle images or telephoto images), and one example, although not limited to these, is a detection method that uses AI (Artificial Intelligence) technology. In this case, a detection model generated using AI technology is used. This detection model is generated by learning from images of the target flying object as training data. The input information to the detection model is the captured image (wide-angle image or telephoto image), and the output information from the detection model includes information indicating the presence or absence of the target flying object in the input captured image, and, if the target flying object is detected, information indicating the position of the detected flying object in the captured image (hereinafter also referred to as detection position information).

In this flying object detection system 1, as described above, the wide-angle camera 2 and telephoto camera 3 are set so that when a flying object of the same size to be detected is photographed in the wide-angle camera's portion and the distant portion of the detection area, the size of the image of the flying object captured in each of the wide-angle and telephoto images will be within a predetermined range (acceptable range). In other words, the cameras are set so that the image analysis of the wide-angle image and the telephoto image can be performed using the same detection model. As a result, in the detection process by the detection unit 52, the same detection model is used for the image analysis of the wide-angle image and the telephoto image. Note that if multiple types of flying objects are set as flying objects to be detected, a detection model that can distinguish and detect each of these multiple types of flying objects is generated by learning images of these multiple types of flying objects. The detection unit 52 may use this detection model. In this case, the detection unit 52 can also output information indicating the type of flying object detected.

The detection unit 52 uses this detection model to detect the target flying object from both the wide-angle image and the telephoto image. Furthermore, if the detection unit 52 detects a flying object from the telephoto image, the flying object should also appear in the wide-angle image, and so the detection unit 52 associates the flying object in the wide-angle image with the telephoto image. This process uses positional relationship data between the telephoto image and the wide-angle image, which associates image portions that show the same real space. This data is generated in advance and stored in the storage device 40.

Information representing the detection results by the detection unit 52 is stored in the storage device 40 in association with, for example, information identifying the imaging device (wide-angle camera 2 or telephoto camera) that captured the captured image subjected to the detection process, information representing the frame number of the captured image subjected to the detection process, and information about the time of capture.

As mentioned above, in the flying object detection system 1, the detection process for the target flying object is performed using the same detection model for both wide-angle images and telephoto images. This reduces the processing load on the detection device 5 compared to when wide-angle images and telephoto images are processed using separate detection models, and also shortens the time required for the detection process for the flying object. This contributes to real-time detection of flying objects using images captured by the wide-angle camera 2 and telephoto camera 3.

The generation unit 53 generates an image reflecting the detection results. The image reflecting the detection results is an image in which information representing the position of the detected flying object in the image is superimposed on the wide-angle image. Here, the information representing the position of the detected flying object in the image is represented, for example, by a graphic. The graphic representing the position of the detected flying object in the image (hereinafter also referred to as the flying object detection graphic) is not limited here and may be, for example, a circle, triangle, or square, or a symbol or a mark resembling an flying object, and is set appropriately by a system designer, etc. The information used in the process of generating the detection result reflection image is information about the position of the flying object in the wide-angle image detected from the wide-angle image (i.e., detection position information), and information about the position of the flying object in the wide-angle image detected from the telephoto image by matching the telephoto image with the wide-angle image. In addition, the wide-angle image and the telephoto image are synchronized so that their shooting times match. As a result, the flying object detection graphic representing the position of the flying object in the image detected from the telephoto image is superimposed on the wide-angle image captured at the same shooting time as the telephoto image in which the flying object was detected.

FIGS. 4 and 5 each show examples of detection result reflection images. In the example of FIG. 4, the detection result reflection image is an image in which a flying object detection graphic 8, which is a double circle, is superimposed on a wide-angle image. The example of FIG. 5 is an example of a detection result reflection image when multiple types of flying objects are detected. The detection result reflection image is an image in which a star-shaped flying object detection graphic 8 corresponding to the type of flying object detected, and a double circle flying object detection graphic 8 corresponding to another type of flying object detected, are superimposed on a wide-angle image. Note that in FIGS. 4 and 5, the detected flying object is indicated by the symbol "9." Furthermore, when multiple types of flying objects are detected, the same flying object detection graphic 8 may be superimposed on the wide-angle image regardless of the type of flying object detected.

)。 English: Furthermore, the alert level may differ depending on the flight location and type of the detected flying object. In such cases, the type of flying object detection graphic 8 corresponding to the alert level may be predetermined, and the generation unit 53 may generate an image reflecting the detection results by superimposing the flying object detection graphic 8 of the type corresponding to the alert level on the wide-angle image. Note that here, the alert level is calculated, for example, by the detection unit 52. In other words, if different alert levels are set depending on the distance from the installation location of the wide-angle camera 2 and the telephoto camera 3 (camera installation location), data relating to the distance from the camera installation location to the flying object and the alert level is stored in the storage device 40 as alert level judgment information used to determine the alert level. Furthermore, if the alert level is set depending on the type of flying object to be detected, data relating to the type of flying object and the alert level is stored in the storage device 40 as alert level judgment information. Furthermore, if the alert level is set based on the combination of the distance from the camera installation location to the flying object and the type of flying object, data relating to the alert level is stored in the storage device 40 as alert level judgment information. When a target flying object is detected, the detection unit 52 calculates the alert level of the detected flying object using one or both of the distance from the camera installation location to the detected flying object and the type of flying object, as well as the alert level determination information stored in the storage device 40. The generation unit 53 then generates an image reflecting the detection results by superimposing an flying object detection graphic 8 of a type corresponding to the calculated alert level onto the wide-angle image as described above. Note that the method for calculating the distance between the detected flying object and the camera installation location from the wide-angle image or telephoto image is not limited here, and therefore its description is omitted.

Furthermore, the image reflecting the detection results generated by the generation unit 53 may further include text indicating information such as the type of aircraft detected and the alert level.

The output unit 54 outputs the generated image reflecting the detection results. An example of an output destination is the display device 7 shown in Figure 1. The display device 7 is a device that notifies (provides) information by displaying the information on a screen using text and images. Here, the display device 7 receives the image reflecting the detection results from the output unit 54 and provides the image reflecting the detection results to, for example, a user of the flying object detection system (monitoring system). Note that the display device 7 may be, for example, a display device of a terminal device carried by a user of the flying object detection system (monitoring system).

In this case, the captured images (wide-angle image, telephoto image) output by the wide-angle camera 2 and the telephoto camera 3 are moving images, and the detection process for the flying object is performed using a frame image selected from the multiple frame images that make up the captured image. The detection result reflection image generated by reflecting the results of this detection process is, for example, included in the moving image, which is a wide-angle image, by replacing the frame image of the original wide-angle image, and is output to the output destination by the output unit 54. The display device 7 displays the moving image (wide-angle image) output by the output unit 54 in this way. In other words, the output unit 54 can be said to control the display operation of the display device 7.

For example, when a user inputs a request (enlarged display request) to the detection device 5 to enlarge an image of a detected flying object on the display device 7, the output unit 54 may perform display control to enlarge the flying object 9 in response to the request, as shown in FIG. 6. The method by which the user inputs the enlarged display request is not limited here, but an example is a method in which the enlarged display request is input to the detection device 5 by clicking the flying object detection graphic 8 with the cursor 71 as shown in FIG. 6. Furthermore, the manner in which the flying object is enlarged is not limited here and may be an appropriately set display manner. For example, as shown in FIG. 6, a window image in which the flying object is enlarged may be superimposed on the wide-angle image, or the wide-angle image and the enlarged image of the flying object may be displayed side by side.

The detection device 5 of the first embodiment is configured as described above. Next, an example of the operation of the detection device 5 to detect an airborne object will be described with reference to Figure 7. Figure 7 is a flowchart explaining an example of the operation of the detection device 5 to detect an airborne object. Figure 7 can also be considered a diagram explaining the method for detecting an airborne object in the detection device 5.

For example, let us say that the storage device 40 in the detection device 5 stores various information (data) used for processing by the arithmetic unit (processor) 50 as described above. The acquisition unit 51 of the detection device 5 acquires wide-angle images from the wide-angle camera 2 and telephoto images from the telephoto camera 3 (step 101 in Figure 7). In the following explanation, it is assumed that the wide-angle images and telephoto images are moving images.

The detection unit 52 executes a detection process to detect the target flying object for each of the acquired wide-angle and telephoto images (step 102). Here, the detection process is executed on frame images selected for each predetermined number of frames from the frame images that make up each of the wide-angle and telephoto images.

Then, the detection unit 52 determines whether the target flying object has been detected in at least one of the wide-angle image and the telephoto image through the detection process (step 103). If the target flying object is detected, the detection unit 52 calculates the position in the wide-angle image or telephoto image in which the flying object is detected as detected photographing information. Furthermore, if the photographed image in which the flying object is detected is a telephoto image, the detection unit 52 calculates the position in the wide-angle image of the flying object detected from the telephoto image through processing to associate the telephoto image with the wide-angle image.

Then, the generation unit 53 uses information from the detection process by the detection unit 52 to superimpose an airborne object detection graphic 8 representing the position of the detected airborne object in the image onto the wide-angle image, thereby generating an image reflecting the detection result (step 104). The output unit 54 then outputs the image reflecting the detection result (step 105). Here, since the wide-angle image is a moving image, the output unit 54 outputs a wide-angle image including the detection result reflection image (frame image), for example, to the display device 7. In other words, here, the airborne object detection process described above is performed sequentially on each of multiple frame images selected from the multiple frame images in each of the wide-angle image and the telephoto image. When the detection unit 52 detects an airborne object, the generation unit 53 generates an image reflecting the detection result. When the detection result reflection image (frame image) is generated, the output unit 54 outputs a wide-angle image including the detection result reflection image, for example, to the display device 7.

On the other hand, if the target flying object is not detected in the selected frame image, the flying object detection process for that frame image is terminated, and the system prepares for the flying object detection process for the next frame image to be selected.

As described above, the detection device 5 of the first embodiment and the flying object detection system 1 equipped with the detection device 5 use telephoto images to detect target flying objects in the distant parts of the detection area. In telephoto images, the target flying object can be captured at a detectable size even if it is far from the camera installation location. Therefore, by using telephoto images, the flying object detection system 1 can prevent situations where the target flying object captured in the captured image is too small to be detected due to its distance from the camera installation location, resulting in the flying object being missed.

Furthermore, the flying object detection system 1 uses wide-angle images to detect target flying objects in parts of the detection area other than the distant part. A wide-angle type imaging device is able to capture parts of the detection area that are not clearly visible in a telephoto image (in other words, parts of the detection area other than the distant part). By using wide-angle images from such a wide-angle type imaging device, the flying object detection system 1 can detect target flying objects in parts of the detection area other than the distant part while preventing missed detections.

Therefore, by using wide-angle images and telephoto images, the detection device 5 and the flying object detection system 1 equipped with the detection device 5 can prevent the detection accuracy of flying objects in the detection area from decreasing due to the distance from the imaging device that captures the detection area.

<Other embodiments>
The present disclosure is not limited to the first embodiment and may be embodied in various ways. For example, in the first embodiment, the output unit 54 outputs a detection result reflection image based on a wide-angle image. In addition, for example, when a user of the flying object detection system 1 requests the provision of a telephoto image, the output unit 54 may output the telephoto image to the requestor in response to the request. In this case, for example, the output unit 54 outputs a telephoto image synchronized with the detection result reflection image to be output. The telephoto image output in this manner may be displayed on the display device 7 in place of the detection result reflection image, or the detection result reflection image and the telephoto image may be displayed side by side on the display device 7.

Furthermore, when a telephoto image is output, the generation unit 53 may generate a telephoto image with information such as the following. In other words, information on the detection results detected from the wide-angle image cannot be known just by looking at the telephoto image. A telephoto image with information is a telephoto image on which the detection results detected from such a wide-angle image are superimposed. The form of the information representing the detection results from the wide-angle image superimposed on the telephoto image may be, for example, text as shown in FIG. 8, or an image of an aircraft detected from the wide-angle image. Such a telephoto image with information can also provide information on the detection results from the wide-angle image that is not shown in the telephoto image.

Furthermore, in the example of Figure 3, two telephoto cameras 3 are used to capture images of the distant parts of the detection area, but depending on the size of the distant parts of the detection area and the field of view of the telephoto cameras 3, three or more telephoto cameras 3 may be used to capture images of the distant parts, as shown in Figure 9, for example. Note that, like Figure 3, Figure 9 is a schematic diagram showing an example of the capture ranges in the detection areas of the wide-angle camera 2 and the telephoto camera 3, viewed from the zenith.

Furthermore, while the first embodiment shows an example in which the flying object detection system is applied to a surveillance system, the application of the flying object detection system in the present disclosure is not limited to surveillance systems. For example, in an unmanned aircraft traffic management system in the area surrounding a logistics hub using unmanned aircraft such as drones, the flying object detection system in the present disclosure may be applied to detect unmanned aircraft that are subject to management as flying objects to be detected.

Furthermore, in the first embodiment, an air vehicle that is a man-made object was used as an example of an air vehicle to be detected, but, for example, a bird may also be set as an air vehicle to be detected. In other words, if a bird collides with an aircraft, there is a risk of a serious accident, such as the aircraft crashing. If a bird flies near an aircraft, there is a risk of this happening, and there is concern that it may interfere with the operation of the unmanned aircraft. For this reason, for example, when the air vehicle detection system of the present disclosure is applied to an unmanned aircraft traffic management system, it will also detect birds as an air vehicle to be detected. Then, when a bird (air vehicle to be detected) is detected around a managed aircraft that is an air vehicle to be detected, information indicating that a bird is flying nearby and that there is a risk posed by the bird may be superimposed on the wide-angle image, thereby generating and outputting an image reflecting the detection results, such as that shown in FIG. 10.

Furthermore, in the first embodiment, the output unit 54 outputs an image reflecting the detection result to the display device 7 using the generation unit 53. In addition to this, when an airborne object is detected by the detection unit 52, the output unit 54 may output information indicating that the airborne object has been detected (airborne object detection information) to a predetermined notification destination other than the display device. One example of the notification destination is a computer device of a monitoring system to which the airborne object detection system is applied. Examples of the airborne object detection information include text information of a message indicating that an airborne object has been detected, and control information for an alarm sound to notify the detection of an airborne object by sound.

Furthermore, the flying object detection device may also have a configuration such as that shown in FIG. 11. That is, the flying object detection device 10 is, for example, a computer device, and includes an acquisition unit 11, a detection unit 12, a generation unit 13, and an output unit 14 as functional units realized by executing a computer program. The acquisition unit 11 acquires telephoto images, which are images captured by a telephoto type imaging device that captures distant parts of the detection area that are far from the installation location, out of multiple types of imaging devices with different angles of view that are installed in a common installation location to capture images of the detection area. The acquisition unit 11 also acquires wide-angle images, which are images captured by a wide-angle type imaging device out of the multiple types of imaging devices.

The detection unit 12 detects target flying objects in the distant part of the detection area from the telephoto image. The detection unit 12 also detects target flying objects in the part of the detection area other than the distant part from the wide-angle image.

The generation unit 13 generates an image reflecting the detection results by superimposing information indicating the position of the detected flying object in the image onto the wide-angle image. The output unit 14 outputs the image reflecting the detection results. Note that the acquisition unit 51, detection unit 52, generation unit 53, and output unit 54 of the detection device 5 in the first embodiment described above are examples of the acquisition unit 11, detection unit 12, generation unit 13, and output unit 14.

The flying object detection device 10 has the configuration described above. Together with the imaging devices 20 and 30, as shown by the dotted lines in Figure 11, the flying object detection device 10 forms an flying object detection system.

Next, an example of the operation of the flying object detection device 10 will be described with reference to Figure 12. Figure 12 is a flowchart explaining an example of the operation of the flying object detection device 10. Figure 12 can also be considered a diagram explaining an example of a flying object detection method used by the flying object detection device 10.

For example, when the acquisition unit 11 acquires a wide-angle image and a telephoto image (step 201), the detection unit 12 executes a detection process to detect the target flying object from each of the wide-angle image and the telephoto image (step 202). Then, if the target flying object is detected, the generation unit 13 generates an image reflecting the detection result by superimposing information representing the position of the detected flying object in the image on the wide-angle image (step 203). The output unit 14 outputs the generated image reflecting the detection result (step 204).

As described above, the flying object detection device 10 performs detection processing of the target flying object using telephoto images (i.e., images captured by a telephoto-type camera) for the distant parts of the detection area. Therefore, even if the target flying object is flying in the distant parts and located far from the installation location of the camera, the flying object detection device 10 can prevent the target flying object from being missed due to appearing small in the captured image, because the target flying object is magnified and captured in the telephoto image.

Furthermore, for detection area portions other than the distant portion (in other words, detection area portions closer to the installation location of the camera device than the distant portion), the flying object detection device 10 performs detection processing of the target flying object using wide-angle images, which are images captured by a wide-angle type camera device. In other words, because a telephoto type camera device focuses on the distant portion, detection area portions closer to the installation location of the camera device than the distant portion may be out of focus or have large blind spots, making it impossible to capture the flying object properly in the telephoto image. In contrast, a wide-angle type camera device has a wide field of view that can compensate for the blind spots of a telephoto type camera device, and by focusing on detection area portions closer to the installation location of the camera device than the distant portion, it is possible to clearly capture the target flying object in that detection area portion. By using wide-angle images captured by such a wide-angle type imaging device, the flying object detection device 10 can compensate for the shortcomings of using telephoto images, thereby preventing the detection accuracy of flying objects in the detection area from decreasing due to the distance from the imaging device capturing the detection area.

A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes.
[Appendix 1]
an acquisition unit that acquires a telephoto image, which is an image captured by a telephoto type imaging device that captures a distant part of the detection area that is far from the installation location, among multiple types of imaging devices with different angles of view that are installed at a common installation location to capture images of the detection area, and a wide-angle image, which is an image captured by a wide-angle type imaging device among the multiple types of imaging devices;
a detection unit that detects a target flying object in a distant part of the detection area from a telephoto image and detects a target flying object in a part of the detection area other than the distant part from a wide-angle image;
a generation unit that generates an image reflecting the detection result by superimposing information indicating the position of the detected flying object in the image on the wide-angle image;
An aircraft detection device having an output unit that outputs an image reflecting the detection result.
[Appendix 2]
The distant part of the detection area is photographed by a plurality of telephoto type photographing devices whose fields of view are shifted from each other,
the acquisition unit acquires telephoto images captured by each of the plurality of telephoto type imaging devices,
The flying object detection device described in Appendix 1, wherein the detection unit detects the target flying object using telephoto images taken by each of the multiple telephoto type imaging devices in the distant part of the detection area.
[Appendix 3]
The detection unit is an aircraft detection device described in Appendix 1 that detects multiple types of aircraft to be detected separately.
[Appendix 4]
The generating unit further generates an information-attached telephoto image by superimposing information representing the detection result detected from the wide-angle image by the detecting unit on the telephoto image.
[Appendix 5]
The output unit is an aerial vehicle detection device described in Appendix 1, which performs display control to enlarge and display an image of the detected aerial vehicle in response to a request to enlarge and display the image of the aerial vehicle on a display device.
[Appendix 6]
The output unit outputs information indicating that an air vehicle has been detected to a predetermined notification destination other than a display device.
[Appendix 7]
The telephoto and wide-angle images are acquired by the acquisition unit using the telephoto type imaging device and the wide-angle type imaging device, whose cameras are set so that when a flying object of the same size is captured in a telephoto image and a wide-angle image in a distant part of the detection area and in a part of the detection area other than the distant part, the difference in size of the flying object of the detection object captured in the telephoto image and the wide-angle image falls within a predetermined range;
The detection unit performs the detection process for the flying object using the same detection model generated by learning captured images of the flying object to be detected for each of the telephoto and wide-angle images.
[Appendix 8]
a telephoto type imaging device that captures an image of a distant part of the detection area that is far from the installation location, among a plurality of types of imaging devices with different angles of view that are installed at a common installation location for capturing an image of the detection area;
a wide-angle type imaging device among the plurality of types of imaging devices that captures images of a detection area including a portion that is not captured by the telephoto type imaging device; and
An aircraft detection system comprising the aircraft detection device described in Appendix 1, which uses images captured by the telephoto type imaging device and the wide-angle type imaging device, respectively.
[Appendix 9]
By computer,
Among a plurality of types of imaging devices with different angles of view installed at a common installation location for imaging the detection area, a telephoto image is acquired which is an image captured by a telephoto type imaging device which captures a distant portion of the detection area that is far from the installation location, and a wide-angle image is acquired which is an image captured by a wide-angle type imaging device among the plurality of types of imaging devices,
Detecting a target flying object in a distant part of the detection area from a telephoto image, and detecting a target flying object in a part of the detection area other than the distant part from a wide-angle image;
generating an image reflecting the detection results by superimposing information representing the position of the detected flying object in the image on the wide-angle image;
A flying object detection method that outputs an image reflecting the detection results.
[Supplementary Note 10]
a process of acquiring a telephoto image, which is an image captured by a telephoto type imaging device that captures a distant portion of the detection area that is far from the installation location, among multiple types of imaging devices with different angles of view that are installed at a common installation location to capture the detection area, and a wide-angle image, which is an image captured by a wide-angle type imaging device among the multiple types of imaging devices;
A process of detecting a target flying object in a distant part of the detection area from a telephoto image and detecting a target flying object in a part of the detection area other than the distant part from a wide-angle image;
A process of generating an image reflecting the detection result by superimposing information representing the position of the detected flying object in the image on the wide-angle image;
and a program storage medium for storing a computer program that causes a computer to execute a process of outputting an image that reflects the detection result.

In addition, some or all of the configurations described in Supplementary Notes 2 to 7, which are dependent on Supplementary Note 1, may also be dependent on Supplementary Notes 8 to 10 in the same dependent relationship as Supplementary Notes 2 to 7. Furthermore, not limited to Supplementary Notes 1 and Supplementary Notes 8 to 10, some or all of the configurations described as supplementary notes may also be dependent on various hardware, software, various recording means for recording software, or systems, as long as they do not deviate from the respective embodiments described above.

The present disclosure has been described above with reference to the embodiments, but the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

This application claims priority based on Japanese Patent Application No. 2024-015839, filed February 5, 2024, the disclosure of which is incorporated herein in its entirety.

REFERENCE SIGNS LIST 1 Flying object detection system 2 Wide-angle camera 3 Telephoto camera 5 Detection device 10 Flying object detection device 11, 51 Acquisition unit 12, 52 Detection unit 13, 53 Generation unit 14, 54 Output unit

Claims (20)

an acquisition means for acquiring a telephoto image, which is an image captured by a telephoto type imaging device that captures a distant portion of the detection area that is far from the installation location, among multiple types of imaging devices with different angles of view that are installed at a common installation location for capturing images of the detection area, and a wide-angle image, which is an image captured by a wide-angle type imaging device among the multiple types of imaging devices;
a detection means for detecting a target flying object in a distant part of the detection area from a telephoto image and detecting a target flying object in a part of the detection area other than the distant part from a wide-angle image;
a generating means for generating an image reflecting the detection result by superimposing information representing the position of the detected flying object in the image on the wide-angle image;
An air vehicle detection device having an output means for outputting an image reflecting the detection result. The distant part of the detection area is photographed by a plurality of telephoto type photographing devices whose fields of view are shifted from each other,
the acquisition means acquires telephoto images taken by each of the plurality of telephoto type imaging devices,
2. The flying object detection device according to claim 1, wherein the detection means detects the target flying object in the distant part of the detection area using telephoto images taken by each of the plurality of telephoto type imaging devices. 3. The flying object detection device according to claim 1, wherein the detection means detects a plurality of types of flying objects as detection targets by distinguishing between each of the types. The generating means further generates an information-added telephoto image by superimposing information representing the detection result detected from the wide-angle image by the detection means onto the telephoto image. An aircraft detection device as described in any one of claims 1 to 3. The flying object detection device according to any one of claims 1 to 4, wherein the output means performs display control to enlarge and display an image of the detected flying object on a display device in response to a request to enlarge and display the image of the flying object. The flying object detection device according to any one of claims 1 to 5, wherein the output means outputs information indicating that the flying object has been detected to a predetermined notification destination other than the display device. The telephoto and wide-angle images are acquired by the acquisition means using the telephoto type imaging device and the wide-angle type imaging device, whose cameras are set so that when a flying object of the same size is captured in a telephoto image and a wide-angle image in a distant part of the detection area and in a part of the detection area other than the distant part, the difference in size of the flying object of the detection object captured in the telephoto image and the wide-angle image falls within a predetermined range;
The detection means performs the detection process for the flying object using the same detection model generated by learning photographed images of the flying object to be detected for each of the telephoto image and the wide-angle image. An flying object detection device as described in any one of claims 1 to 6. a telephoto type imaging device that captures an image of a distant part of the detection area that is far from the installation location, among a plurality of types of imaging devices with different angles of view that are installed at a common installation location for capturing an image of the detection area;
a wide-angle type imaging device among the plurality of types of imaging devices that captures images of the detection area including a portion that is not captured by the telephoto type imaging device; and
8. A flying object detection system comprising the flying object detection device according to claim 1, which uses images captured by the telephoto type imaging device and the wide-angle type imaging device, respectively. By computer,
Among a plurality of types of imaging devices with different angles of view installed at a common installation location for imaging the detection area, a telephoto image is acquired which is an image captured by a telephoto type imaging device which captures a distant portion of the detection area that is far from the installation location, and a wide-angle image is acquired which is an image captured by a wide-angle type imaging device among the plurality of types of imaging devices,
Detecting a target flying object in a distant part of the detection area from a telephoto image, and detecting a target flying object in a part of the detection area other than the distant part from a wide-angle image;
generating an image reflecting the detection results by superimposing information representing the position of the detected flying object in the image on the wide-angle image;
A flying object detection method that outputs an image reflecting the detection results. a process of acquiring a telephoto image, which is an image captured by a telephoto type imaging device that captures a distant portion of the detection area that is far from the installation location, among multiple types of imaging devices with different angles of view that are installed at a common installation location to capture the detection area, and a wide-angle image, which is an image captured by a wide-angle type imaging device among the multiple types of imaging devices;
A process of detecting a target flying object in a distant part of the detection area from a telephoto image, and detecting a target flying object in a part of the detection area other than the distant part from a wide-angle image;
A process of generating an image reflecting the detection result by superimposing information representing the position of the detected flying object in the image on the wide-angle image;
and a program storage medium for storing a computer program that causes a computer to execute a process of outputting an image that reflects the detection result. The distant part of the detection area is photographed by a plurality of telephoto type photographing devices whose fields of view are shifted from each other,
further acquiring telephoto images by each of the plurality of telephoto type imaging devices by the computer;
The flying object detection method according to claim 9, wherein the computer detects the target flying object using telephoto images captured by each of the plurality of telephoto type imaging devices in the distant part of the detection area. 12. The flying object detection method according to claim 9 or 11, wherein when detecting a target flying object, a plurality of types of target flying objects are detected by distinguishing between each of them. 13. The flying object detection method according to claim 9, claim 11 or claim 12, wherein the computer further generates an information-added telephoto image by superimposing information representing the detection results detected from the wide-angle image onto the telephoto image. The computer performs display control to enlarge and display an image of the detected flying object on a display device in response to a request to enlarge and display an image of the flying object. The flying object detection method according to claim 9 or any one of claims 11 to 14, wherein the computer further outputs information indicating that the flying object has been detected to a predetermined notification destination other than a display device. The distant part of the detection area is photographed by a plurality of telephoto type photographing devices whose fields of view are shifted from each other,
A process of acquiring telephoto images by each of the plurality of telephoto type imaging devices;
The program storage medium of claim 10 further stores a computer program that causes a computer to execute a process of detecting the target flying object using telephoto images from each of the plurality of telephoto type imaging devices in the distant portion of the detection area. 17. The program storage medium according to claim 10 or claim 16, wherein the process of detecting the target flying object comprises distinguishing and detecting each of a plurality of types of target flying objects. 18. The program storage medium according to claim 10, claim 16, or claim 17, further storing a computer program that causes a computer to execute a process of generating an information-added telephoto image by superimposing information representing the detection result detected from the wide-angle image onto the telephoto image in the process of generating the detection result reflection image. A program storage medium as described in claim 10 or any one of claims 16 to 18, further storing a computer program that causes a computer to execute a process of performing display control to enlarge and display an image of a detected flying object in response to a request to enlarge and display the image of the flying object on a display device. A program storage medium as described in claim 10 or any one of claims 16 to 19, further storing a computer program that causes a computer to execute a process of outputting information indicating that an aircraft has been detected to a predetermined notification destination other than a display device.