WO2020165050A1

WO2020165050A1 - Method, device and system for detecting hazardous objects

Info

Publication number: WO2020165050A1
Application number: PCT/EP2020/053175
Authority: WO
Inventors: Jaanus Tamm; Kalev SARAPU; Vladimir NEPOCHATOV
Original assignee: Defsecintel Solutions Oue
Current assignee: Defsecintel Solutions Oue
Priority date: 2019-02-15
Filing date: 2020-02-07
Publication date: 2020-08-20
Anticipated expiration: 2021-08-15
Also published as: EP3924870A1

Abstract

The present invention relates to a method, device and system for detecting hazardous objects in a sector. It can comprise capturing data corresponding to a sector, analyzing captured data via an object detection algorithm, detecting at least one predetermined object at a first location based on the analyzed data, assigning an importance parameter to the detected object and in response to the importance parameter exceeding a set or self-adjusted or self-learned threshold, classifying the detected object as a hazardous object and deploying an unmanned aerial vehicle to the first location. The capturing data can be done by a sensor component, the analyzing, the detecting, and/or the classifying by a processing component. It can further comprise at least one of visual images, infrared (IR) images, and thermal images. The sensor component can thus be sensitive to one or more electromagnetic wave lengths. The predetermined object can comprise at least one of a person; and a land borne, seaborne and/or airborne vehicle, and a land borne, seaborne and/or airborne mobile robot; and a chemical component or composition.

Description

Method, device and system for detecting hazardous objects

Field

The invention relates to security. More specifically, the invention relates to detecting and surveilling hazardous objects in a sector.

Introduction

Surveillance of particular areas or sectors can be important in many industries. Security, border defense, commercial and private ventures may benefit from autonomous or semi- autonomous surveillance systems. Particularly in remote or sparsely-populated areas it may be beneficial to have systems in place that can monitor a certain region or sector for intruders and notify the appropriate operators in case those are detected.

Some such surveillance systems include components such as ground-based stations and unmanned aerial vehicles. For example, United States patent application 2018/0150087 A1 discloses a tagging drone (TD) that responds to a received query of availability initiated from a command and control drone (CCD) or command and control station (CCS), over a network, for investigating or tagging a designated target. The TD receives one or more dispatching instructions and heads to the designated target. The TD performs tagging of the designated target and determines whether the designated target was successfully tagged as a tagged target. The tagged target is tracked and tracking data is provided for recovery resource capture of the tagged target.

Summary

It is the object of the present invention to provide an improved, reliable and/or efficient way of providing surveillance to a sector. Furthermore, it is the object to provide a device, system and method for detecting hazardous objects. It is also the object to provide surveillance devices with processing capabilities and capable of reacting to detected hazardous objects timely and accurately.

The present invention relates to a method for detecting hazardous objects in a sector. The method can comprise the steps of capturing data corresponding to a sector, analyzing captured data via an object detection algorithm, detecting at least one predetermined object at a first location based on the analyzed data, assigning an importance parameter to the detected object and in response to the importance parameter exceeding a set or self-adjusted or self-learned threshold, classifying the detected object as a hazardous object and deploying an unmanned aerial vehicle to the first location.

The data captured can be any kind of data that can be used to detect an object, such as a human being, animal, any manmade structure. The data can comprise at least one of electromagnetic sensor data, particle sensor data, acoustic sensor data etc., video data, photographic data, infrared data, ultrasonic data, radar data, any kind of derived data etc. The detection algorithm can comprise any detection of an object on the basis of the before- mentioned data.

The capturing data can be done by a sensor component, the analyzing, the detecting, and/or the classifying by a processing component. It can further comprise at least one of visual images, infrared (IR) images, and thermal images. The sensor component can thus be sensitive to one or more electromagnetic wave lengths.

The predetermined object can particularly comprise at least one of a person; and a land borne, seaborne and/or airborne vehicle, and a land borne, seaborne and/or airborne mobile robot; and a chemical component or composition.

The importance parameter can be assigned based on a type of detected object. The importance parameter can be assigned based at least partially on the first location. Additionally or alternatively, it can be assigned based at least partially on a detection of an individual or aggregated amount, size, speed, volume and/or a predetermined combination of detected objects.

The importance parameter can be assigned based at least partially on behavior of the detected object. The importance parameter can be trained by a machine learning algorithm on the basis of positive and/or negative training data.

The present invention can also comprise the step of triggering an alert in response to the importance parameter exceeding a set threshold . It can further comprise the step of notifying a remote operator in response to the importance parameter exceeding a threshold, preferably a set or machined learned threshold.

The present invention can further comprise the step of, prior to deploying an unmanned aerial vehicle, assessing at least one environmental condition. This environment condition comprises at least one of precipitation, chemical parameters, physical parameters, wind speed, and ambient light. The chemical parameters could be any toxic or harmful compositions of unallowed warfare and could particularly positively trigger the deployment of the unmanned vehicles.

Prior to deploying the unmanned aerial vehicle, a charting a course to the hazardous object can be done. A charting of the unmanned aerial vehicle's course can be based on the assessed environmental condition.

While the unmanned aerial vehicle is en route to the first location, an updating of the charted course in response to the hazardous object moving can take place. An instructing of the unmanned aerial vehicle to navigate to a second location can also be realized while en route to the first location, the second location corresponding to a hazardous object's new location.

The present invention can prior to deploying the unmanned aerial vehicle further comprise the steps of verifying battery status of at least two unmanned aerial vehicles by a verifying component, and selecting the unmanned aerial vehicle to deploy based on the battery status by a selecting component.

The unmanned aerial vehicle can be configured for travelling with a speed of at least 150 km/h to the first location. The speed can even be around 250 km/h. Further the unmanned aerial vehicle can be configured for arriving to the first location within 5 minutes, preferably within 3 minutes of the designation of the hazardous object. This provides a reliable and fast enough monitoring of the section at issue.

In case of drones as unmanned aerial vehicles, these are particularly configured to take a top speed of at least 150 km/h, even to around 250 km/h as well.

The unmanned aerial vehicle can be configured for initiating data collection upon arrival to the first location wherein further data collection comprises capturing a plurality of images corresponding to the hazardous object. The method can further comprise the unmanned aerial vehicle to broadcast at least one message upon arrival to the first location. In another embodiment is can transmit the at least one message upon detection of the hazardous object of during the commuting back.

The present invention can further comprise the step of recalling to unmanned aerial vehicle while en route to the first location in response to at least one of the detected object no longer classified as a hazardous object and the hazardous object moving outside of a predetermined geographical sector.

Based on analyzing a plurality of past classified hazardous objects, predicting likely future locations associated with future hazardous objects can be also comprised.

The present invention also comprises a surveillance device for detecting hazardous objects, the device comprising at least one sensor configured to capture data corresponding to a sector at least one processing component configured to analyze captured data via an object detection algorithm, detect at least one predetermined object at a first location in the analyzed data, assign an importance parameter to the detected object, in response to the importance parameter exceeding a set threshold, classifying the detected object as a hazardous object and deploying an unmanned aerial vehicle to the first location.

The device can further comprise a ground station, a body, and/or a dock configured for an unmanned aerial vehicle.

The sensor can comprise at least one of a visual camera, an IR camera and a thermal camera.

It can further comprise a communication component wherein the communication component can be configured to communicate with a remote server.

The processing component can be further configured to transmit an alert via the communication component wherein the processing component can be further configured to transmit data associated with the hazardous object. The device can further comprise an enclosed compartment configured to store at least one unmanned aerial vehicle wherein the compartment can be temperature controlled and can comprise a lid configured to open and close automatically, and/or a battery station.

The battery station can comprise a battery swapping component configured to exchange unmanned aerial vehicle's battery. The battery station can additionally or alternatively comprise a battery charger configured to charge batteries of the unmanned aerial vehicle.

The device can further comprise at least one ambient sensor wherein the ambient sensor can be configured to detect ambient conditions in its surroundings.

The device can further comprise a surveillance module wherein the surveillance module can comprises a shaft, preferably a telescoping shaft, configured to extend to a height of at least 10m, preferably at least 15m, more preferably at least 20 m.

The surveillance module can comprise at least one idle position and at least one active position wherein in the idle position, the surveillance module can be arranged significantly flush with the body.

In the active position the surveillance module can protrude substantially vertically from the body wherein in the active position the surveillance module extends from the body by at least 10m, preferably at least 15m, such as at least 20m.

The sensor can be arranged on the surveillance module.

Additionally or alternatively, a motion component can be configured to propel the surveillance device in a direction.

The device can further comprise a trailer on which all components are arranged so that the trailer is configured to dislocate.

The present invention is also related to a system for surveying a sector. This system can be particularly adapted to carry out any of the method steps described before and below and claimed. The system can comprise at least one surveillance device comprising a body, at least one sensor configured to capture data corresponding to a sector, a dock for an unmanned aerial vehicle, and at least one unmanned aerial vehicle configured to dock at the surveillance device and to be deployed from the surveillance device.

The surveillance device can further can further comprise a processing component configured to analyze sensor data via an object detection algorithm, detect at least one predetermined object at a first location in the analyzed data, assign an importance parameter to the detected object, in response to the importance parameter exceeding a set threshold, classifying the detected object as a hazardous object and deploying the unmanned aerial vehicle to the first location.

The unmanned aerial vehicle can be configured to reach any point within the sector in a time shorter than a predetermined time after classification of the detected object as the hazardous object. The unmanned aerial vehicle can be a drone that is capable of fulfilling the requirements as set out before and below.

The predetermined time interval can comprise 10 minutes, preferably 5 minutes, more preferably 3 minutes wherein the unmanned aerial vehicle an be further configured to travel with a top speed of at least 100 km/h, preferably at least 150 km/h, more preferably at least 200 km/h, such as 250 km/h.

The body can comprise an enclosed compartment configured to store the unmanned aerial vehicle wherein the enclosed compartment can comprise a lid and the surveillance device can be configured to open the lid so as to release and the unmanned aerial vehicle and close the lid after its departure.

The dock can be arranged within the enclosed compartment.

The surveillance device can further comprise a battery station configured for at least one of exchanging a battery of an unmanned aerial vehicle and/or charging a battery of an unmanned aerial vehicle.

The system according to the present invention can also comprise a second unmanned aerial vehicle configured to dock at the surveillance device and wherein the surveillance device can be configured to monitor a battery status of the two unmanned aerial vehicles and arrange at least one of battery exchange and battery charging so that at least one of the two unmanned aerial vehicles has a charged battery at any time.

Thee surveillance device can further comprise at least one ambient sensor wherein the ambient sensor can be configured to detect ambient conditions around the surveillance device.

The surveillance device can be configured to analyze the detected ambient conditions and abort deploying the unmanned aerial vehicle if the ambient conditions are unsuitable for it.

The surveillance device can be configured to track the hazardous object. The surveillance device can be further configured to send updated information regarding the hazardous object to the unmanned aerial vehicle while it is en route to the first location.

The surveillance device can be configured to reroute the unmanned aerial vehicle to a second location while it is en route to the first location in response to detecting that the hazardous object has moved.

The unmanned aerial vehicle can further comprise at least one UAV sensor and can be configured to start transmitting data from the UAV sensor to the surveillance device upon approach to the first location.

At least one remote sensing device can be configured to at least detect predetermined objects and placed within the sector at a location different from the surveillance device and wherein the remote sensing device is configured to alert the surveillance device upon detection of a predetermined object.

The surveillance device can be configured to deploy the unmanned aerial vehicle in response to an alert from the remote sensing device.

Further the system can comprise a remote server that is configured to store and/or process information submitted by the surveillance device.

The remote server can be configured to communicate with the surveillance device and wherein the communication comprises at least one of sending instructions to the surveillance device and/or receiving data from the surveillance device. The remote server can be configured to predict likely future hazardous objects based on analyzing previously classified hazardous objects.

The server can be further configured to predict likely future locations of hazardous objects.

A system for detecting hazardous objects can further comprise a plurality of surveillance devices as described before and below, each surveillance device assigned to a different sector, a plurality of unmanned vehicles, each unmanned vehicle assigned to at least one surveillance device wherein the surveillance devices are configured to communicate with each other.

The surveillance devices can be configured to modify assignment of unmanned vehicles.

The surveillance devices can be further configured to exchange data related to at least one of ambient conditions, ongoing unmanned aerial vehicles' deployment, detected hazardous objects in a sector, and/or movement of hazardous objects from one sector to another.

The device or system as described before or below can be further configured to automatically modify assignment of the aerial vehicles according to demand and to control the relocation of the aerial vehicles. As the devices can be positioned at a distance to each other, the aerial vehicles can then be controlled to leave one surveillance device it is assigned to and located in or at to another surveillance device. This can be also done en route of one aerial vehicle so that it returns to a different surveillance device than the one it has started from.

The device or the system can also comprise an automatic guiding component for independently guiding the aerial vehicles to the surveillance device. This would enable a guiding even in case other means of navigation, such as the GPS signal, are not available.

Embodiments

Below is a list of method embodiments. Those will be indicated with a letter "M". Whenever such embodiments are referred to, this will be done by referring to "M" embodiments.

M l . A method for detecting hazardous objects in a sector, the method comprising

capturing data corresponding to a sector; analyzing captured data via an object detection algorithm;

detecting at least one predetermined object at a first location based on the analyzed data;

assigning an importance parameter to the detected object;

in response to the importance parameter exceeding a set threshold, classifying the detected object as a hazardous object and deploying an unmanned aerial vehicle to the first location.

M2. The method according to the preceding embodiment wherein the capturing data is done by a sensor component, the analyzing, the detecting, and/or the classifying by a processing component.

M3. The method according to the preceding embodiment wherein captured data comprises at least one of

visual images;

infrared (IR) images; and

thermal images.

M4. The method according to any of the preceding embodiments wherein the predetermined object comprises at least one of

a person; and

a land borne, seaborne or airborne vehicle; and

a land borne, seaborne or airborne mobile robot; and

a chemical component or composition

M5. The method according to any of the preceding embodiments wherein the importance parameter is assigned based on a type of detected object.

M6. The method according to any of the preceding embodiments wherein the importance parameter is assigned based at least partially on the first location.

M7. The method according to any of the preceding embodiments wherein the importance parameter is assigned based at least partially on a detection of an individual or aggregated amount, size, speed, volume and/or a predetermined combination of detected objects. M8. The method according to any of the preceding embodiments wherein the importance parameter is assigned based at least partially on behavior of the detected object.

M9. The method according to any of the preceding embodiments wherein the importance parameter is trained by a machine learning algorithm on the basis of positive and/or negative training data.

M 10. The method according to any of the preceding embodiments further comprising triggering an alert in response to the importance parameter exceeding a set threshold.

M i l . The method according to any of the preceding embodiments further comprising notifying a remote operator in response to the importance parameter exceeding a threshold, preferably a set or machined learned threshold.

M 12. The method according to any of the preceding embodiments further comprising, prior to deploying an unmanned aerial vehicle, assessing at least one environmental condition.

M 13. The method according to the preceding embodiment wherein the environment condition comprises at least one of

precipitation;

chemical parameters;

physical parameters;

wind speed; and

ambient light.

M 14. The method according to any of the preceding embodiments further comprising, prior to deploying the unmanned aerial vehicle, charting a course to the hazardous object.

M 15. The method according to the preceding embodiment and with the features of embodiment M 12 further comprising charting the unmanned aerial vehicle's course based on the assessed environmental condition.

M 16. The method according to any of the two preceding embodiments further comprising, while the unmanned aerial vehicle is en route to the first location, updating the charted course in response to the hazardous object moving. M17. The method according to the preceding embodiment further comprising instructing the unmanned aerial vehicle to navigate to a second location while en route to the first location, the second location corresponding to a hazardous object's new location.

M18. The method according to any of the preceding embodiments further comprising, prior to deploying the unmanned aerial vehicle,

verifying battery status of at least two unmanned aerial vehicles by a verifying component; and

selecting the unmanned aerial vehicle to deploy based on the battery status by a selecting component.

M19. The method according to any of the preceding embodiments further comprising the unmanned aerial vehicle travelling with a speed of at least 150 km/h to the first location.

M20. The method according to any of the preceding embodiments further comprising the unmanned aerial vehicle arriving to the first location within 5 minutes, preferably within 3 minutes of the designation of the hazardous object.

M21. The method according to any of the preceding embodiments further comprising the unmanned aerial vehicle initiating data collection upon arrival to the first location.

M22. The method according to the preceding embodiment wherein data collection comprises capturing a plurality of images corresponding to the hazardous object.

M23. The method according to any of the two preceding embodiments further comprising the unmanned aerial vehicle broadcasting at least one message upon arrival to the first location.

M24. The method according to any of the preceding embodiments further comprising recalling to unmanned aerial vehicle while en route to the first location in response to at least one of the detected object no longer classified as a hazardous object; and

the hazardous object moving outside of a predetermined geographical sector. M25. The method according to any of the preceding embodiments further comprising, based on analyzing a plurality of past classified hazardous objects, predicting likely future locations associated with future hazardous objects.

Below is a list of device embodiments. Those will be indicated with a letter "D". Whenever such embodiments are referred to, this will be done by referring to "D" embodiments.

Dl. A surveillance device for detecting hazardous objects, the device comprising

at least one sensor configured to capture data corresponding to a sector

at least one processing component configured to

analyze captured data via an object detection algorithm;

detect at least one predetermined object at a first location in the analyzed data; assign an importance parameter to the detected object;

D2. The device according to the preceding embodiment comprising a ground station.

D3. The device according to any of the preceding device embodiments further comprising a body.

D4. The device according to any of the preceding device embodiments further comprising a dock configured for an unmanned aerial vehicle.

D5. The device according to any of the preceding device embodiments wherein the sensor comprises at least one of a visual camera, an IR camera and a thermal camera.

D6. The device according to any of the preceding device embodiments further comprising a communication component.

D7. The device according to the preceding embodiment wherein the communication component is configured to communicate with a remote server. D8. The device according to any of the two preceding embodiments wherein the processing component is further configured to transmit an alert via the communication component.

D9. The device according to any of the three preceding embodiments wherein the processing component is further configured to transmit data associated with the hazardous object.

D10. The device according to any of the preceding device embodiments further comprising an enclosed compartment configured to store at least one unmanned aerial vehicle.

Dl l. The device according to the preceding embodiment wherein the compartment is temperature controlled.

D12. The device according to any of the two preceding embodiments wherein the compartment comprises a lid configured to open and close automatically.

D13. The device according to any of the preceding device embodiments further comprising a battery station.

D14. The device according to the preceding embodiment wherein the battery station comprises a battery swapping component configured to exchange unmanned aerial vehicle's battery.

D15. The device according to any of the two preceding embodiments wherein the battery station further comprises a battery charger configured to charge batteries of the unmanned aerial vehicle.

D16. The device according to any of the preceding device embodiments further comprising at least one ambient sensor.

D17. The device according to the preceding embodiment wherein the ambient sensor is configured to detect ambient conditions in its surroundings.

D18. The device according to any of the preceding device embodiments further comprising a surveillance module. D19. The device according to the preceding embodiment wherein the surveillance module comprises a shaft, preferably a telescoping shaft, configured to extend to a height of at least 10m, preferably at least 15m, more preferably at least 20 m.

D20. The device according to any of the two preceding embodiments and with the features of embodiment D3 wherein the surveillance module comprises at least one idle position and at least one active position.

D21. The device according to the preceding embodiment and with the features of embodiment D3 wherein in the idle position, the surveillance module is arranged significantly flush with the body.

D22. The device according to any of the two preceding embodiments and with the features of embodiment D3 wherein in the active position the surveillance module protrudes substantially vertically from the body.

D23. The device according to the preceding embodiment wherein in the active position the surveillance module extends from the body by at least 10m, preferably at least 15m, such as at least 20m.

D24. The device according to any of the six preceding embodiments wherein the sensor is arranged on the surveillance module.

D25. The device according to any of the preceding device embodiments further comprising a motion component configured to propel the surveillance device in a direction.

D26. The device according to any of the preceding device embodiments further comprising a trailer on which all components are arranged so that the trailer is configured to dislocate.

D27. The device according to any of the preceding device embodiments further comprising an automatic guiding component for independently guiding the aerial vehicles to the surveillance device.

M26. The method according to any of the preceding method embodiments performed by a device according to any of the preceding device embodiments. Below is a list of system embodiments. Those will be indicated with a letter "S". Whenever such embodiments are referred to, this will be done by referring to "S" embodiments.

51. A system for surveying a sector, the system comprising

at least one surveillance device comprising

a body;

at least one sensor configured to capture data corresponding to a sector; a dock for an unmanned aerial vehicle; and

at least one unmanned aerial vehicle configured to dock at the surveillance device and to be deployed from the surveillance device.

52. The system according to the preceding embodiment wherein the surveillance device comprises a processing component configured to

analyze sensor data via an object detection algorithm;

in response to the importance parameter exceeding a set threshold, classifying the detected object as a hazardous object and deploying the unmanned aerial vehicle to the first location.

53. The system according to the preceding embodiment wherein the unmanned aerial vehicle is configured to reach any point within the sector in a time shorter than a predetermined time after classification of the detected object as the hazardous object.

54. The system according to the preceding embodiment wherein the predetermined time interval comprises 10 minutes, preferably 5 minutes, more preferably 3 minutes.

55. The system according to any of the preceding system embodiments, wherein the unmanned aerial vehicle is configured to travel with a top speed of at least 100 km/h, preferably at least 150 km/h, more preferably at least 200 km/h, such as 250 km/h.

56. The system according to any of the preceding system embodiments wherein the body comprises an enclosed compartment configured to store the unmanned aerial vehicle. 57. The system according to the preceding embodiment wherein the enclosed compartment comprises a lid and the surveillance device is configured to open the lid so as to release and the unmanned aerial vehicle and close the lid after its departure.

58. The system according to any of the two preceding embodiments wherein the dock is arranged within the enclosed compartment.

59. The system according to any of the preceding system embodiments wherein the surveillance device further comprises a battery station configured for at least one of

exchanging a battery of an unmanned aerial vehicle; and

charging a battery of an unmanned aerial vehicle.

S10. The system according to the preceding embodiment further comprising a second unmanned aerial vehicle configured to dock at the surveillance device and wherein the surveillance device is configured to monitor battery status of the two unmanned aerial vehicles and arrange at least one of battery exchange and battery charging so that at least one of the two unmanned aerial vehicles has a charged battery at any time.

510. The system according to any of the preceding system embodiments wherein the surveillance device further comprises at least one ambient sensor and wherein the ambient sensor is configured to detect ambient conditions around the surveillance device.

511. The system according to the preceding embodiment and with features of embodiment S2 wherein the surveillance device is configured to analyze the detected ambient conditions and abort deploying the unmanned aerial vehicle if the ambient conditions are unsuitable for it.

512. The system according to any of the preceding system embodiments and with the features of embodiment S2 wherein the surveillance device is configured to track the hazardous object.

513. The system according to the preceding embodiment wherein the surveillance device is configured to send updated information regarding the hazardous object to the unmanned aerial vehicle while it is en route to the first location. 514. The system according to any of the two preceding embodiments wherein the surveillance device is configured to reroute the unmanned aerial vehicle to a second location while it is en route to the first location in response to detecting that the hazardous object has moved.

515. The system according to any of the preceding system embodiments and with the features of embodiment S2 wherein the unmanned aerial vehicle further comprises at least one UAV sensor and is configured to start transmitting data from the UAV sensor to the surveillance device upon approach to the first location.

516. The system according to any of the preceding system embodiments further comprising at least one remote sensing device configured to at least detect predetermined objects and placed within the sector at a location different from the surveillance device and wherein the remote sensing device is configured to alert the surveillance device upon detection of a predetermined object.

517. The system according to the preceding embodiment wherein the surveillance device is configured to deploy the unmanned aerial vehicle in response to an alert from the remote sensing device.

518. The system according to any of the preceding system embodiments further comprising a remote server.

519. The system according to the preceding embodiment wherein the remote server is configured to communicate with the surveillance device and wherein the communication comprises at least one of

Sending instructions to the surveillance device; and

Receiving data from the surveillance device.

520. The system according to any of the two preceding embodiments and with features of embodiment S2, wherein the remote server is configured to predict likely future hazardous objects based on analyzing previously classified hazardous objects.

521. The system according to the preceding embodiment wherein the server is further configured to predict likely future locations of hazardous objects. 522. The system according to any of the preceding system embodiments wherein the surveillance device is according to any of the preceding device embodiments.

523. A system for detecting hazardous objects, the system comprising

A plurality of surveillance devices according to any of the device claims, each surveillance device assigned to a different sector;

A plurality of unmanned vehicles, each unmanned vehicle assigned to at least one surveillance device;

wherein the surveillance devices are configured to communicate with each other.

524. The system according to the preceding embodiment wherein the surveillance devices are configured to modify assignment of unmanned vehicles.

525. The system according to any of the preceding system embodiments wherein the system is configured to automatically modify assignment of the aerial vehicles according to demand and to control the relocation of the aerial vehicles.

526. The system according to any of the preceding system embodiment wherein the system is configured to automatically modify assignment of at least one aerial vehicle when it is en route.

527. The system according to any of the two preceding embodiments wherein the surveillance devices are configured to exchange data related to at least one of

Ambient conditions;

Ongoing unmanned aerial vehicles' deployment;

Detected hazardous objects in a sector;

Movement of hazardous objects from one sector to another.

528. The system according to any of the preceding system embodiments further comprising an automatic guiding component for independently guiding the aerial vehicles to the surveillance device.

529. The system according to any of the preceding system embodiments wherein the surveillance devices are configured according to any of the preceding device embodiments. The present technology will now be discussed with reference to the accompanying drawings.

Brief description of the drawings

Figure 1 depicts a schematic embodiment of a surveillance device according to an embodiment of the invention;

Figure 2 schematically depicts a system for detecting hazardous objects in a sector;

Figure 3 shows a flowchart embodying a method for detecting hazardous objects in a sector;

Figure 4 illustrates one possible embodiment corresponding to a method for detecting hazardous objects in a sector;

Figure 5 shows another schematic embodiment of a surveillance device according to an embodiment of the invention;

Figure 6 depicts another schematic embodiment of a surveillance device according to an embodiment of the invention.

Description of embodiments

Figure 1 schematically depicts an embodiment of a surveillance device 1 according to an aspect of the present invention.

The surveillance device 1 can be used to monitor and survey a certain region or sector and to detect hazardous objects, which may correspond to intruders in the sector. The surveillance device 1 is depicted as a ground station 1. The surveillance device 1 comprises a body 2, which may have any convenient shape and may be manufactured out of plastic, metal, or another material.

The surveillance device comprises a sensor 4, which can collect data relating to the surveillance device's 1 surroundings. The sensor 4 may comprise a camera, such as a high- resolution camera and/or a plurality of cameras. For example, the sensor 4 may comprise a high-resolution camera for daytime surveillance and an infrared camera for nighttime surveillance. The camera(s) can monitor a section or can alone or in concert with each other monitor 360°. The camera(s) can also configured to capture light, infrared and/or thermal signals. The sensor 4 may be fitted on a surveillance module 12. The surveillance module 12 can comprise a shaft 14. The surveillance module 12 can, in a concrete example, comprise a telescoping or extending part (which, in the fully extended position comprises the shaft 14), and one or more sensors located on top of the extending part, in such a way that the sensors may survey the surroundings from a height. The shaft 14 may comprise a telescoping or extendable rod, antenna, or the like.

The surveillance module 12 may be configured so that it has at least one active and idle positions. Figure 1 schematically shows the surveillance module 12 in an active position, with the shaft 14 fully extended and the sensor 4 placed at the top part. In this way, the sensor 4, preferably comprising at least one camera, may survey a larger area, as it is placed at a height. Preferably, the shaft 14 may extend to a height of at least 10 m, preferably at least 15 m, such as at least 20 m above the ground.

The surveillance device 1 further comprises a processing component 6 and a communication component 8, depicted schematically in the drawing. The processing component 6 preferably comprises a computing device, such as a central processing unit, system on a chip, or the like. The processing component 6 can generally control and coordinate the operation of the surveillance device 1. The communication component 8 may comprise one or more communication modules. For example, a short-range communication module such as a Bluetooth, WiFi or a similar protocol-enabled module may be used for communication in the range below 10km, and a long-range communication module such as a SIM-Card, Modem or the like may be used for communication in the range above 10km.

The surveillance device 1 further comprises an optional ambient sensor 10. The ambient sensor 10 may sense ambient conditions around the surveillance device 1. For example, the ambient sensor 10 may detect weather conditions such as wind speed, precipitation, dust or sand storms, humidity, temperature and the like.

The surveillance device 1 as shown in figure 1 further comprises a motion component 16, depicted as wheels. The motion component 16 can also comprise tracks or the like. The motion component 16 can be used to propel the surveillance device 1 from one location to another. It may be used to bring the surveillance device 1 to its location of operation. The transfer of the surveillance device 1 can also be realized as an autonomous or semi-autonomous or remote controlled drive. During operation, the surveillance device 1 is preferably stationary, but it may move between different operational locations. The surveillance device 1 as depicted also comprises an energy component 18. It is shown as a solar cell. The energy component 18 may also comprise one or more batteries, one or more wind turbines, or other components.

Further depicted is a compartment 20 in the body 2. The compartment 20 can preferably be an enclosed compartment with a lid 22. The lid 22 may be controlled by the processing component 6 to open or close.

The compartment 20 may house one or more unmanned aerial vehicles 100. The aerial vehicles 100 may dock to a dock 24 when not in use. There may be further a battery station within the compartment 20. The battery station may have a battery swapping component 32 and a battery charger 34. The battery swapping component 32 may be used to swap or change a battery 102 of the unmanned aerial vehicle 100. The battery charger 34 may be used to charge the battery 102 of the unmanned aerial vehicle 100, either while the battery 102 is placed within the unmanned aerial vehicle 100, or when it is removed as shown in figure 1. Additionally or alternatively, the compartment can comprise a rapid charger for rapidly charging the battery/batteries of the aerial vehicle(s) 100.

The unmanned aerial vehicles 100 can be released from the surveillance device 1 (via the lid 22 of the compartment 20) upon detection of a hazardous object within the surveillance device's 1 sector. Such a hazardous object may then be investigated by the unmanned aerial vehicle 100 from close-by, whereupon its status or classification as a hazardous object may be maintained or revoked. In other words, the unmanned aerial vehicle 100 may approach a hazardous object detected by the surveillance device 1 for further investigation. Preferably, the unmanned aerial vehicle 100 is a drone configured to navigate with a speed of at least 100 km/h, preferably 150 km/h. This advantageously allows for very quick investigation of a potential intruder or threat by the unmanned aerial vehicle 100.

Figure 2 schematically depicts an embodiment of a surveillance device 1 in operation. Two neighbouring sectors 70, 70' are shown. Each sector 70, 70' has a surveillance device 1, 1' monitoring it. The sectors may correspond to a geographical area with a radius of 5-20 km, preferably 5-15 km. The sectors may be partially intersecting, and may have any shape (such as circular, elliptical and the like). The sectors may also be of different sizes and shapes, depending on local topography, vegetation, buildings and the like.

The surveillance device 1, monitoring sector 70 has detected several predetermined objects 42. The predetermined objects 42 may correspond to objects such as persons, vehicles, mobile robots, and the like. The predetermined objects 42 may also correspond to an arbitrary moving object with certain predetermined characteristics (such as size, speed of movement, general shape and the like). The detection of the predetermined objects can be done by analysing data from the sensor 4, which preferably comprises visual images. Such images can be analysed by an object-detection algorithm on the processing component 6, and predetermined objects 42 can be identified. The object-detection algorithm can be preferably a neural network-based algorithm.

The surveillance device 1 can also classify some of the detected predetermined objects 42 as hazardous objects 62. This can be done by a computer and a respective classification software. Signals, such as camera signals, can be streamed, analysed and, if decided manually or automatically by the software, send out. Meta data can be added or can be sent out without the camera signals instead. This would make the message shorter and easier and faster to be submitted. Such classification may be based on several parameters assigned a certain weight. For instance, location of the detected objects, a certain combination of the detected objects, movement patterns of the detected objects can all be used to classify a detected predetermined object 42 as a hazardous object 62. In the figure, the hazardous object 62 is shown as a first location 72.

The surveillance device 1 may dispatch an unmanned aerial vehicle 100 to the first location 72 to investigate the hazardous object 62.

The surveillance device 1', shown in the sector 70' has only detected predetermined objects 42, and not classified any of them as hazardous objects 62 (for example, the predetermined objects 42 may correspond to known and authorized persons or vehicles, animals, known and authorized mobile robots and the like). The unmanned aerial vehicle 100' corresponding to the surveillance device 1' is shown docked to the surveillance device 1', ready to be deployed if a hazardous object 62 is identified.

Figure 3 depicts a schematic embodiment of a method for detecting hazardous objects, shown as a schematic algorithm. First, data from a sensor is captured and analysed. If a predetermined object is detected, it is assigned an importance parameter. Such a parameter may be based on several factors such as location of detection, movement patterns, time of day, a certain combination of predetermined objects and the like. Different weights may be assigned to different factors. Such weights may be constantly updated. If no predetermined object is detected, previous task of capturing and analysing data is resumed.

Once an importance parameter is assigned to a detected predetermined object, it is evaluated whether the parameter exceeds a certain predetermined threshold. If it does not, the object is classified as predetermined object and the previous task is resumed. The object is then still monitored, and its importance parameter continuously updated to reflect any ongoing changes (such as movement).

If the importance parameter of the detected predetermined object exceeds the set threshold, the predetermined object is classified as a hazardous object. This then leads to UAV deployment for further investigation of the hazardous object. Additionally, an alert may be raised, and a remote operator may be contacted.

The steps of the method as shown in figure 3 may be performed by the surveillance device 1, particularly by its processing component 6 (with the data captured by the sensor 4).

Figure 4 schematically depicts the implementation of the method for detecting hazardous objects. Sketches a, b, c, d show a progression in a time of an event.

Sketch a shows a surveillance device 1 that has detected a predetermined object 42 at a first location 72. The predetermined object 42 may have just entered sector 70 (denoted by the borders of the sketch). The surveillance device 1 can also comprise software with machine learning software that optimizes the transmission decision.

Sketch b depicts the surveillance device 1 classifying the detected predetermined object 42 as a hazardous object 62. This can be based, for example, on the first location 72 where the predetermined object 42 has been detected, on the nature of the predetermined object 42 (such as a group of people for example), on the movement pattern of the predetermined object 42, on the time of day, on a combination of these factors, or the like.

In sketch c, the surveillance device 1 deploy an unmanned aerial vehicle 100 to the first location 72 to further investigate the hazardous object 62. While the unmanned aerial vehicle 100 is en route to the first location 72, the surveillance device 1 detects that the hazardous object 62 has moved to a second location 74.

In sketch d, the surveillance device 1 instructs the unmanned aerial vehicle 100 to navigate to the second location 74 where the hazardous object 62 has moved. Upon approaching the second location 74, the unmanned aerial vehicle 1 may start transmitting sensor data such as a video stream or images back to the surveillance device 1. The surveillance device 1 may then analyse this data and/or send it to a remote server or operator for further analysis.

Figure 5 shows another schematic embodiment of a surveillance device 1 according to an aspect of the invention. The surveillance module 12 comprising a shaft 14 are shown. A sensor 4 is placed at the top of the shaft 14 on the surveillance module 12. The sensor 4 may comprise a visual camera, an infrared camera, or a plurality of cameras. Additionally, a radar sensor may be used together with the visual camera.

Similarly to figure 1, the surveillance module 12 is shown in an active position, where the shaft 14 is fully expanded and monitoring of the surroundings can be performed. During transport or storage, the shaft 14 may be collapsed or telescoped so as to be generally flush with the body 2 of the surveillance device 1.

Compartment 20 is shown, along with the lid 22. The compartment may comprise the unmanned aerial vehicle, its dock, a battery station and the like (not shown). The compartment may also be temperature controlled (in all embodiments of the surveillance device 1).

An energy component 18 is shown, depicted as a solar cell. The energy component 18 may supply power to the surveillance device 1. Also shown is a motion component 16, depicted as wheels. The surveillance device 1 can be advantageously moved from one position to another by using the motion component 16.

Figure 6 depicts another schematic embodiment of the surveillance device 1. The surveillance device 1 of figure 6 is shown with a surveillance module 12 extending above the surveillance device 1 via a shaft 14. The shaft can telescope to make it more compact for transport of the device 1. Attached to the shaft 14 is a fixing component 142 which comprises a plurality of cables or the like with one end attached to the shaft 14 and another configured to be fixed to the ground, soil, or the like. In this way, the surveillance module 12 may remain stable even when significantly exceeding the height of the surveillance device's body.

List of reference numerals

1 - Surveillance device

2 - Surveillance device body

4 - Sensor

6 - Processing component

8 - Communication component

10 - Ambient sensor

12 - Surveillance module 14 - Shaft of the surveillance module

142 - Fixing component 16 - Motion component

18 - Energy component

20 - Compartment

22 - Compartment lid

24 - Dock for unmanned aerial vehicle

30 - Battery station

32 - Battery swapping component

34 - Battery charger42 - Predetermined object

62 - Hazardous object

70 - Sector

72 - First location

74 - Second location

100 - Unmanned aerial vehicle

102 - Unmanned aerial vehicle battery

200 - Remote server

Reference numbers and letters appearing between parentheses in the claims, identifying features described in the embodiments and illustrated in the accompanying drawings, are provided as an aid to the reader as an exemplification of the matter claimed. The inclusion of such reference numbers and letters is not to be interpreted as placing any limitations on the scope of the claims.

The term "at least one of a first option and a second option" is intended to mean the first option or the second option or the first option and the second option. Whenever a relative term, such as "about", "substantially" or "approximately" is used in this specification, such a term should also be construed to also include the exact term. That is, e.g., "substantially straight" should be construed to also include "(exactly) straight".

Whenever steps were recited in the above or also in the appended claims, it should be noted that the order in which the steps are recited in this text may be the preferred order, but it may not be mandatory to carry out the steps in the recited order. That is, unless otherwise specified or unless clear to the skilled person, the order in which steps are recited may not be mandatory. That is, when the present document states, e.g., that a method comprises steps (A) and (B), this does not necessarily mean that step (A) precedes step (B), but it is also possible that step (A) is performed (at least partly) simultaneously with step (B) or that step (B) precedes step (A). Furthermore, when a step (X) is said to precede another step (Z), this does not imply that there is no step between steps (X) and (Z). That is, step (X) preceding step (Z) encompasses the situation that step (X) is performed directly before step (Z), but also the situation that (X) is performed before one or more steps (Yl), ..., followed by step (Z) . Corresponding considerations apply when terms like "after" or "before" are used.

Claims

1. A method for detecting hazardous objects in a sector, the method comprising

capturing data corresponding to a sector;

analyzing captured data via an object detection algorithm;

assigning an importance parameter to the detected object;

2. The method according to the preceding embodiment wherein the capturing data is done by a sensor component, the analyzing, the detecting, and/or the classifying by a processing component.

3. The method according to the preceding embodiment wherein captured data comprises at least one of

visual images;

infrared (IR) images; and

thermal images.

4. The method according to any of the preceding embodiments wherein the importance parameter is assigned based on at least one of a type of detected object and on behavior of the detected object.

5. The method according to any of the preceding embodiments wherein the importance parameter is assigned based at least partially on a detection of an individual or aggregated amount, size, speed, volume and/or a predetermined combination of detected objects.

6. The method according to any of the preceding embodiments further comprising triggering an alert in response to the importance parameter exceeding a set threshold.

7. The method according to any of the preceding embodiments further comprising notifying a remote operator in response to the importance parameter exceeding a threshold, preferably a set or machined learned threshold.

8. A surveillance device for detecting hazardous objects, the device comprising at least one sensor configured to capture data corresponding to a sector

at least one processing component configured to

analyze captured data via an object detection algorithm;

9. The device according to the preceding embodiment comprising at least one of a ground station., a body and a dock configured for an unmanned aerial vehicle.

10. The device according to any of the preceding device embodiments wherein the sensor comprises at least one of a visual camera, an IR camera and a thermal camera.

11. The device according to any of the preceding device embodiments further comprising a communication component wherein the communication component can be configured to communicate with a remote server.

12. The device according to any of the two preceding embodiments wherein the compartment comprises at least one of a lid configured to open and close automatically and a battery station.

13. The device according to the preceding embodiment wherein the surveillance module comprises a shaft, preferably a telescoping shaft, configured to extend to a height of at least 10m, preferably at least 15m, more preferably at least 20 m.

14. A system for surveying a sector, the system comprising

at least one surveillance device comprising

a body;

at least one sensor configured to capture data corresponding to a sector; a dock for an unmanned aerial vehicle; and at least one unmanned aerial vehicle configured to dock at the surveillance device and to be deployed from the surveillance device.

15. The system according to the preceding embodiment wherein the surveillance device comprises a processing component configured to

analyze sensor data via an object detection algorithm;

16. The system according to the preceding embodiment wherein the predetermined time interval comprises 10 minutes, preferably 5 minutes, more preferably 3 minutes.

17. The system according to any of the preceding system embodiments, wherein the unmanned aerial vehicle is configured to travel with a top speed of at least 100 km/h, preferably at least 150 km/h, more preferably at least 200 km/h, such as 250 km/h.

18. The system according to any of the preceding system embodiments further comprising a remote server and the remote server is configured to communicate with the surveillance device and wherein the communication comprises at least one of

sending instructions to the surveillance device; and

receiving data from the surveillance device.

19. A system for detecting hazardous objects, the system comprising

wherein the surveillance devices are configured to communicate with each other.

20. The system according to any of the preceding system embodiments wherein the system is configured to automatically modify assignment of the aerial vehicles according to demand and to control the relocation of the aerial vehicles.