WO2018152849A1 - Control method, remote monitoring device, base station, server and steaming media server - Google Patents

Abstract

Disclosed in the present invention is a control method for a remote monitoring device (100), the remote monitoring device (100) communicating with a server (300). The control method comprises: (S11) a remote monitoring device (100) detects an operation of a user; (S12) the remote monitoring device (100) determines a control instruction used for controlling an unmanned aerial vehicle (500) according to the operation of the user; and (S13) the remote monitoring device (100) sends the control instruction for the unmanned aerial vehicle (500) to the server (300) so that the unmanned aerial vehicle (500) may acquire the control instruction. Also disclosed in the present invention are a control method for a base station (200), a control method for the server (300), a control method for a streaming media server (400), the remote monitoring device (100), the base station (200), the server (300) and the streaming media server (400).

Description

Control method, remote monitoring device, base station, server, and streaming server Technical field

The present invention relates to the field of remote monitoring technologies, and in particular, to a control method, a remote monitoring device, a base station, a server, and a streaming media server.

Background technique

For industry users (such as security monitoring, traffic inspection), the main demand for the purchase of drones is not simple aerial play, but more is to use drones to complete their work tasks. Therefore, they hope to have a complete back-end platform, which can remotely watch the real-time image transmission information, status information, etc. taken by the out-of-office personnel to control the drone, and even hope to control some functions of the drone remotely (such as Control the head of the gimbal, zoom factor of the camera, etc.), in order to achieve the highest viewing effect, complete its monitoring, inspection and other work tasks. However, at present, the control scene of the drone is mostly that the user uses the remote controller for manual control, and the control range and the range of the image transmission are also limited by the communication link of the remote controller. In addition, when the flying hand is operating the drone, only the person can view the drone picture and status information. Therefore, the real-time image transmission and status information of the drone cannot be transmitted to the remote, and it is impossible to integrate into the unified monitoring platform. Therefore, the drone cannot be used to complete a complete solution and realize the effect of remote monitoring.

Summary of the invention

Embodiments of the present invention provide a control method, a remote monitoring device, a base station, a server, and a streaming media server to enable a user to remotely control a drone and/or remotely view data such as acquired images.

The present invention provides a method for controlling a remote monitoring device, where the remote monitoring device communicates with a server, and the control method includes:

The remote monitoring device detects a user's operation;

The remote monitoring device determines a control instruction for controlling the drone according to an operation of the user; and

The remote monitoring device transmits a control command of the drone to the server to enable the drone to acquire the control command.

The present invention provides a method for controlling a base station, where the base station communicates with a drone and a server respectively, and the control method includes:

Receiving, by the base station, a control instruction sent by the server for controlling the drone; and

The base station controls the drone according to the control instruction.

The present invention provides a server control method, where the server communicates with a base station of a drone and a remote monitoring device, respectively. The control method includes:

Receiving, by the server, a control instruction sent by the remote monitoring device for controlling the drone; and

The server sends the control command to the base station.

The present invention provides a method for controlling a streaming media server, wherein the streaming media server communicates with a base station and a server of a drone, respectively. The control method includes:

Receiving, by the streaming media server, image data captured by the drone sent by the base station; and

The streaming server transmits an image acquisition address corresponding to the image data to the server.

The present invention provides a remote monitoring device, the remote monitoring device is in communication with a server, the remote monitoring device includes a processor, and the processor is configured to:

Detecting user operations;

Determining a control instruction for controlling the drone according to an operation of the user; and

Sending a control command of the drone to the server to enable the drone to acquire the control command.

The present invention provides a base station, wherein the base station communicates with a drone and a server, respectively, the base station includes a processor, and the processor is configured to:

Receiving a control instruction sent by the server for controlling the drone; and

The drone is controlled according to the control command.

The present invention provides a server that communicates with a base station and a remote monitoring device of a drone, respectively, the server including a processor, the processor is configured to:

Receiving a control instruction sent by the remote monitoring device for controlling the drone; and

Sending the control command to the base station.

The present invention provides a streaming media server that communicates with a base station and a server of a drone, respectively, wherein the streaming media server includes a processor, and the processor is configured to:

Receiving image data captured by the drone transmitted by the base station; and

An image acquisition address corresponding to the image data is transmitted to the server.

The control method, the remote monitoring device, the base station, the server, and the streaming media server of the embodiment of the present invention can transmit the image information and the state information captured by the drone to the remote monitoring device in real time, so that the remote monitoring device can see the unmanned person on the one hand. The image information and status information of the machine, on the other hand, can realize the remote control of the drone, and is suitable for various application scenarios such as security monitoring and out-of-office duty, which is convenient for the use of industrial users.

The additional aspects and advantages of the embodiments of the present invention will be set forth in part in the description which follows.

DRAWINGS

The above and/or additional aspects and advantages of the present invention will become apparent from the following description of the embodiments Obvious and easy to understand, where:

1 is a schematic flow chart of a control method according to some embodiments of the present invention;

2 is a schematic diagram of functional modules of a remote monitoring device according to some embodiments of the present invention;

3 is a schematic diagram of the principle of a control method according to some embodiments of the present invention;

4 is a schematic flow chart of a control method according to some embodiments of the present invention;

5 is a schematic flow chart of a control method according to some embodiments of the present invention;

6 is a schematic flow chart of a control method according to some embodiments of the present invention;

7 is a schematic flow chart of a control method according to some embodiments of the present invention;

8 is a schematic flow chart of a control method according to some embodiments of the present invention;

9 is a schematic flow chart of a control method according to some embodiments of the present invention;

10 is a schematic flow chart of a control method according to some embodiments of the present invention;

11 is a schematic diagram of functional blocks of a base station according to some embodiments of the present invention;

12 is a schematic flow chart of a control method according to some embodiments of the present invention;

13 is a schematic diagram of functional blocks of a base station according to some embodiments of the present invention;

14 is a schematic diagram of the principle of a control method according to some embodiments of the present invention;

15 is a schematic flow chart of a control method according to some embodiments of the present invention;

16 is a schematic diagram of functional blocks of a base station according to some embodiments of the present invention;

17 is a schematic diagram showing the principle of a control method according to some embodiments of the present invention;

18 is a schematic flow chart of a control method according to some embodiments of the present invention;

19 is a schematic flow chart of a control method according to some embodiments of the present invention;

20 is a schematic flow chart of a control method according to some embodiments of the present invention;

21 is a schematic flow chart of a control method according to some embodiments of the present invention;

22 is a schematic flow chart of a control method according to some embodiments of the present invention;

23 is a schematic flow chart of a control method according to some embodiments of the present invention;

24 is a schematic flow chart of a control method according to some embodiments of the present invention;

25 is a schematic diagram of the principle of a control method according to some embodiments of the present invention;

26 is a schematic diagram of functional modules of a server according to some embodiments of the present invention;

27 is a schematic flow chart of a control method according to some embodiments of the present invention;

28 is a schematic diagram of functional modules of a server according to some embodiments of the present invention;

29 is a schematic flow chart of a control method according to some embodiments of the present invention;

30 is a schematic diagram of functional modules of a server according to some embodiments of the present invention;

31 is a schematic diagram of the principle of a control method according to some embodiments of the present invention;

32 is a schematic flow chart of a control method according to some embodiments of the present invention;

33 is a schematic flow chart of a control method according to some embodiments of the present invention;

FIG. 34 is a schematic flow chart of a control method according to some embodiments of the present invention; FIG.

35 is a schematic flow chart of a control method according to some embodiments of the present invention;

36 is a schematic flow chart of a control method according to some embodiments of the present invention;

37 is a schematic flow chart of a control method according to some embodiments of the present invention;

38 is a schematic diagram of functional modules of a streaming media server according to some embodiments of the present invention;

39 is a schematic flow chart of a control method according to some embodiments of the present invention;

40 is a schematic flow chart of a control method according to some embodiments of the present invention;

41 is a schematic diagram of the principle of a control method in accordance with some embodiments of the present invention.

detailed description

The embodiments of the present invention are described in detail below, and the examples of the embodiments are illustrated in the drawings, wherein the same or similar reference numerals indicate the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the drawings are intended to be illustrative of the invention and are not to be construed as limiting.

Referring to FIGS. 1 to 3, the control method of the embodiment of the present invention is used for the remote monitoring device 100, and the remote monitoring device 100 communicates with the server 300. Control methods include:

S11: The remote monitoring device 100 detects the operation of the user;

S12: The remote monitoring device 100 determines a control instruction for controlling the drone 500 according to a user's operation; and

S13: The remote monitoring device 100 transmits a control command of the drone 500 to the server 300 to enable the drone 500 to acquire a control command.

The control method of the embodiment of the present invention can be implemented by the remote monitoring device 100 of the embodiment of the present invention. The remote monitoring device 100 of the embodiment of the present invention includes a first processor 10. The first processor 10 can be used to perform step S11, step S12, and step S13.

That is to say, the first processor 10 is used to:

Detecting user operations;

Determining a control instruction for controlling the drone 500 according to a user's operation; and

The control command of the drone 500 is sent to the server 300 to enable the drone 500 to acquire the control command.

As such, the remote monitoring device 100 can implement remote control of the drone 500 according to the user's operation, and the real-time image and status information of the drone 500 can be transmitted to the remote monitoring device 100 for integration into a unified monitoring platform. The user does not need to worry about the remote distance from the drone 500, which makes it impossible to realize the manipulation of the remote drone 500 and the inability to acquire various types of data of the drone 500. The control method of the embodiment of the invention is applicable In various industries such as security monitoring and traffic inspection, the user can control the drone 500 on the remote platform and view the image data collected by the drone 500, which is convenient for the user and enhances the user experience.

Referring to FIG. 3 and FIG. 4 together, in some embodiments, the control method of the embodiment of the present invention further includes:

S14: The remote monitoring device 100 acquires the working data of the drone 500 according to the control instruction.

Step S14 can be implemented by the first processor 10. That is to say, the first processor 10 is further configured to acquire the work data of the drone 500 according to the control command.

Thus, the user inputs a control command to acquire the work data of the remote drone 500, and the first processor 10 of the remote monitoring device 100 receives the work data of the drone 500 according to the user's input command. The user can view the work data of the drone 500 on the remote monitoring device 100.

Referring to FIG. 3 and FIG. 5 together, in some embodiments, the work data includes image data captured by the drone 500, the control command includes first control instructions for acquiring image data, and the server 300 and the cached image data The streaming server 400 communicates, step S14, the remote monitoring device 100 acquires the working data of the drone 500 according to the control instruction, including:

S141: The remote monitoring device 100 acquires image data from the streaming server 400 according to the first control instruction.

Step S141 can be implemented by the first processor 10. That is, the first processor 10 is further configured to acquire image data from the streaming server 400 according to the first control instruction.

Specifically, the remote monitoring device 100 determines that the user inputs a first control instruction for acquiring image data captured by the drone 500 according to an operation of the user, and the remote monitoring device 100 transmits the first control instruction to the server 300 to notify the drone 500. The captured image data is transmitted to the streaming server 400. The streaming server 400 buffers the image data. As such, the remote monitoring device 100 can acquire the image data from the streaming server 400, and the user can view the image data transmitted by the drone 500 on the remote monitoring device 100. Among them, the image data includes an image or video taken by the drone 500.

Referring to FIG. 3 and FIG. 6 together, in some embodiments, the drone 500 communicates with the base station 200. Step S141: The remote monitoring device 100 acquires image data from the streaming server 400 according to the first control instruction, including:

S1411: The remote monitoring device 100 receives an image acquisition address transmitted by the server 300, and the image acquisition address is transmitted to the server 300 when the streaming server 400 buffers image data transmitted by the base station 200 communicating with the drone 500;

S1412: The remote monitoring device 100 acquires image data from the streaming server 400 according to the image acquisition address.

In some embodiments, step S1411 and step S1412 can be implemented by the first processor 10. That is to say, the first processor 10 is further used to:

Receiving an image acquisition address transmitted by the server 300, the image acquisition address being transmitted to the server 300 when the streaming server 400 buffers image data transmitted by the base station 200 in communication with the drone 500;

Image data is acquired from the streaming server 400 in accordance with the image acquisition address.

Specifically, after the remote monitoring device 100 issues the first control command, the base station 200 transmits the image data sent by the drone 500 to the streaming server 400 for caching. At this time, the streaming server 400 transmits an image acquisition address corresponding to the image data to the server 300, and the server 300 transmits the image acquisition address to the remote monitoring device 100. The remote monitoring device 100 can acquire image data from the streaming server 400 according to the image acquisition address.

In some embodiments, the image acquisition address includes one or more formats. Step S1412: The remote monitoring device 100 acquires image data from the streaming server 400 according to the image acquisition address, including:

The remote monitoring device 100 acquires image data from the streaming server 400 according to one of an image acquisition address of one format or an image acquisition address of a plurality of formats.

It can be understood that the remote monitoring device 100 may include various devices, such as devices of the IOS system, devices of the Android system, and the like, and thus the format of the image acquisition address between the remote monitoring device 100 and the streaming media server 400 is also various. In this way, the streaming media server 400 transmits the image acquisition addresses in multiple formats, so that the selection freedom of the remote monitoring device 100 is greater, that is, the various types of remote monitoring devices 100 can obtain the addresses according to the images of the corresponding formats. Get image data.

In a specific embodiment of the present invention, the format of the image acquisition address includes three formats: HLS, RTMP, and M3U8. Among them, the image acquisition address of the HLS format is applicable to devices of the IOS system, such as an Apple mobile phone, an Apple computer, and the like. Image acquisition addresses in RTMP and M3U8 formats are available for non-IOS devices such as Android phones, Windows computers, and more.

Referring to FIG. 3 and FIG. 7, in some embodiments, the working data includes status data of the drone 500, the control command includes a second control instruction for acquiring status data, and the remote monitoring device 100 according to the control is performed in step S14. The instruction to acquire the work data of the drone 500 includes:

S142: The remote monitoring device 100 acquires status data from the server 300 according to the second control instruction.

In some embodiments, step S142 can be implemented by the first processor 10. That is, the first processor 10 is further configured to acquire status data from the server 300 according to the second control instruction.

Specifically, after the remote monitoring device 100 sends the second control command input by the user for acquiring the status data of the drone 500 to the server 300, the server 300 transmits a second control command to the base station 200, and the base station 200 transmits the second control unit 500. The status data is sent to the server 300, which then sends the status data to the remote monitoring device 100. As such, the user can observe the real-time flight status of the drone 500 through the remote monitoring device 100.

In other embodiments, the operational data may also include status data for the base station 200. The status data of the base station 200 includes location information of the base station 200 and the like. When the work data includes only the state data of the base station 200, the control method of the present embodiment can obtain the state data of the base station 200 by transmitting a control command for acquiring the state data of the base station 200 by the remote monitoring device 100. When the work data includes both the status data of the drone 500 and the status of the base station 200 For the data, the control method of the present embodiment can obtain the state data of the drone 500 and the state data of the base station 200 by executing the step S142 and transmitting a control command for acquiring the state data of the base station 200 by the remote monitoring device 100.

In some embodiments, the status data of the drone 500 includes at least one of flight parameters, attitude information, location information, pan/tilt attitude information, and power information of the drone 500.

In this way, the user can grasp the flight state of the drone 500 in real time according to the state data of the drone 500.

Referring to FIG. 8, in some embodiments, step S142: The remote monitoring device 100 acquires status data from the server 300 according to the second control instruction, including:

S1421: The remote monitoring device 100 receives the status data sent from the server 300 through a preset network protocol according to the second control instruction.

In some embodiments, step S1421 can be implemented by the first processor 10. That is to say, the first processor 10 is further configured to receive the status data transmitted from the server 300 through the preset network protocol according to the second control instruction.

It can be understood that the remote monitoring device 100 and the server 300 can communicate through various network protocols. The preset network protocol is one or more of a variety of network protocols. The remote monitoring device 100 communicates with the server 300 according to a preset network protocol to ensure the accuracy and stability of the state data transmission.

In some embodiments, the preset network protocol includes a Websocket network protocol.

As such, duplex communication between the remote monitoring device 100 and the server 300 is implemented using the Websocket network protocol. The Websocket network protocol ensures a persistent connection between the remote monitoring device 100 and the server 300, improving the stability and accuracy of state data transmission.

In some embodiments, step S142: The remote monitoring device 100 acquires status data from the server 300 according to the second control instruction. The remote monitoring device 100 receives one or more of all status data or status data according to the second control instruction.

It can be understood that the status data of the drone 500 includes flight parameters, posture information, position information, pan/tilt posture information, power amount information, and the like of the drone 500. However, the user needs to obtain all the state data of the drone 500 in some cases, and in some cases only needs to obtain one or several of the state data. As such, the remote monitoring device 100 can acquire only the state information that the user needs to know according to the indication of the second control instruction.

In some embodiments, the control instructions include third control commands for controlling the flight of the plurality of drones 500, and the third control commands are used to control the plurality of drones 500 to operate simultaneously and/or to poll for work.

It will be appreciated that in some cases a user may require multiple drones 500 to perform tasks simultaneously or in turn. Therefore, the user can issue a third control command to the remote monitoring device 100 to control the plurality of drones 500 to perform tasks according to user requirements.

In some embodiments, the control instructions include a fourth control command for controlling the flight of the drone 500, and the fourth control command is for controlling the drone 500 to execute the preset course.

In this way, the user can control the drone 500 to pass the preset navigation by issuing a fourth control command to the remote monitoring device 100. Line flight.

In some embodiments, the preset route includes a route stored in the server 300 and/or a route entered through the remote monitoring device 100.

It can be understood that the preset route can be stored in the server 300, wherein the user can operate the remote monitoring device 100, and store multiple routes or routes that need to be executed frequently through the remote monitoring device 100 in the server 300. When the aircraft performs the route, the user can select the route stored in the memory on the remote monitoring device 100, and control the drone to execute the route; in addition, the user can also edit or set the route in the remote monitoring device 100, and remotely monitor The device 100 transmits the edited or set route to the server 300, and the server 300 transmits the route to the base station 200 connected to the drone 500, and the drone 500 can fly according to the route. In this way, the drone 100 has a large degree of freedom of flight, and can further satisfy the user's use requirements.

Referring to FIG. 9, in some embodiments, the control method of the embodiment of the present invention further includes:

S15: The remote monitoring device 100 determines an image processing instruction of the image data according to a user's operation; and

S16: The remote monitoring device 100 transmits an image processing instruction to the server 300, and the image processing instruction is used to process the image data captured by the drone 500 buffered in the streaming server 400.

In some embodiments, step S15 and step S16 can be implemented by the first processor 10. That is to say, the first processor 10 can also be used to:

Determining an image processing instruction of the image data according to a user's operation; and

The image processing instructions are sent to the server 300 for processing the image data captured by the drone 500 buffered in the streaming server 400.

It can be understood that after the remote monitoring device 100 acquires the image data, the user may wish to process the image data. For example, in some cases users would like to be able to store some of the more important data in the image data. In this way, the user issues an image processing instruction to the remote monitoring device 100, and the remote monitoring device 100 transmits an image processing instruction to the server 300, and then the server 300 transmits the image processing instruction to the streaming server 400 that caches the image data corresponding to the image processing instruction. In the above, the streaming server 400 performs corresponding processing on the image data according to the image processing instruction.

In some embodiments, the image processing instructions include processing, storing, sorting, watermarking, and extracting one or more of the image objects for processing.

In this way, image data can be processed in various ways to meet the user's needs.

Referring to FIG. 3 again, in a specific embodiment of the present invention, when the user issues an image processing instruction for storing image data through the remote monitoring device 100, the server 300 receives the image processing instruction and transmits it to the streaming server 400. The streaming server 300 executes the image processing instruction to store the corresponding image data in the object storage system 600 included or external to the streaming server 400. In an embodiment of the present invention, the object storage system 600 is external. In addition, in some embodiments, the object storage system 600 and the streaming server 400 may also be the same.

Referring to FIG. 3 and FIG. 10 together, the control method of the embodiment of the present invention is used for the base station 200, and the base station 200 communicates with the drone 500 and the server 300, respectively. Control methods include:

S21: The base station 200 receives a control command sent by the server 300 for controlling the drone 500; and

S22: The base station 200 controls the drone 500 according to the control command.

Referring to FIG. 11, a control method according to an embodiment of the present invention may be implemented by a base station 200 according to an embodiment of the present invention. The base station 200 of the embodiment of the present invention includes a second processor 20. Step S21 and step S22 may be implemented by the second processor 20. That is to say, the second processor 20 is used to:

Receiving a control command sent by the server 300 for controlling the drone 500; and

The drone 500 is controlled in accordance with a control command.

In this way, after receiving the control command sent by the server 300, the base station 200 can control the flight or data transmission of the drone 500 according to the control command. Since the control command is transmitted from the remote monitoring device 100 to the server 300 and then forwarded to the base station 200 by the server 300, the effect of the user remotely controlling the drone 500 is achieved.

Referring to Figures 12-13 together, in some embodiments, base station 200 includes a communication client module 22 and a first service module 24. Step S21: The base station 200 receives the control command sent by the server 300 for controlling the drone 500, including:

S211: The communication client module 22 of the base station 200 receives the control instruction sent by the server 300; and

S212: The communication client module 22 of the base station 200 sends a control command to the first service module 24.

Step S22: The base station 200 controls the drone 500 according to the control instruction, including:

S221: The first service module 24 of the base station 200 controls the drone 500 according to the control command.

In some embodiments, step S211 and step S212 can be implemented by the second processor 20. That is to say, the second processor 20 is also used to:

The communication client module 22 of the control base station 200 receives the control command sent by the server 300;

Controlling the communication client module 22 of the base station 200 to send a control command to the first service module 24; and

The first service module 24 of the control base station 200 controls the drone 500 in accordance with a control command.

Referring to FIG. 14, in particular, the communication client module 22 receives the control command sent by the server 300 and transmits the control command to the first service module 24. The first service module 24 performs control of the drone 500 according to the received control command. action. The communication between the communication client module 22 and the first service module 24 includes communication using the Websocket network protocol.

Referring to FIG. 15 and FIG. 16, in some embodiments, the base communication client module 22 includes a first client module 221 and a second client module 222, and the communication client module 22 of the base station 200 receives the server in step S211. The control commands sent by 300 include:

S2111: The first client module 221 of the base station 200 receives the control command sent by the server 300; and

S2112: The first client module 221 of the base station 200 transparently transmits a control command to the second client module 222.

Step S212: The communication client module 22 of the base station 200 sends a control command to the first service module 24, including:

S2121: The second client module 222 of the base station 200 sends a control command to the first service module 24.

In some embodiments, step S2111 and step S2112 can be implemented by the second processor 20. That is to say, the second processor 20 is further used to:

The first client module 221 of the control base station 200 receives the control command sent by the server 300;

Controlling, by the first client module 221 of the base station 200, the control command to the second client module 222; and

The second client module 222 controlling the base station 200 sends a control command to the first service module 24.

Referring to FIG. 17 , specifically, the first client module 221 receives the control command sent by the server 300 and transmits the control command to the second client module 222 , and then sends the control command to the first service module 24 by the second client module 222 . The first service module 24 performs an action of controlling the drone 500 in accordance with the control command. As such, dividing the communication client module 22 into the first client module 221 and the second client module 222 can simplify the programming process of implementing the control method of the embodiment of the present invention, and can further modularize communication between the server and the base station. .

Referring to FIG. 18, in some embodiments, step S22 base station 200 controls drone 500 according to a control instruction, including:

S222: The base station 200 controls the drone 500 to send the working data according to the control instruction.

The control method of the embodiment of the present invention further includes:

S23: The base station 200 receives the work data; and

S24: The base station 200 transmits the working data.

In some embodiments, step S222, step S23, and step S24 can be implemented by the second processor 20. That is to say, the second processor 20 is further used to:

Controlling the drone 500 to transmit work data according to the control command;

Receiving work data; and

Send work data.

As such, the base station 200 receives the work data from the drone 500 and transmits the work data to the server 300 in accordance with the control command. Since the server 300 communicates with the remote monitoring device 100, that is, the server 300 transmits the work data to the remote monitoring device 100, the user can view the work data of the drone 500 on the remote monitoring device 100.

Referring to FIG. 3 and FIG. 19 together, in some embodiments, the work data includes image data captured by the drone 500, the control command includes first control instructions for acquiring image data, and the base station 200 and the streaming server 400 Communication, step S222, the base station 200 controls the drone 500 to transmit the working data according to the control instruction, including:

S2221: The base station 200 controls the drone 500 to transmit image data according to the first control instruction.

Step S23: The base station 200 receives the working data, including:

S231: The base station 200 receives image data.

Step S24: The base station 200 sends the working data to include:

S241: The base station 200 transmits image data to the streaming server 400.

In some embodiments, step S2221, step S231, and step S241 can be implemented by the second processor 20. That is to say, the second processor 20 is further used to:

Controlling the drone 500 to transmit image data according to the first control instruction;

Receiving image data; and

The image data is sent to the streaming server 400.

In this manner, the base station 200 receives the image data transmitted by the drone 200 in accordance with the control command, and transmits the image data to the streaming server 400. The remote monitoring device 100 and the streaming server 400 communicate to achieve acquisition of image data. Among them, the image data includes an image or video taken by the drone 500.

Referring to FIG. 20, in some embodiments, the sending, by the base station 200, the image data to the streaming server 400 in step S241 includes:

S2411: The base station 200 sends the image data to the streaming media server 400 by using a preset network protocol according to the first control instruction.

In some embodiments, step S2411 can be implemented by the second processor 20. That is, the second processor 20 is further configured to send image data to the streaming server 400 through a preset network protocol according to the first control instruction.

It can be understood that communication between the base station 200 and the streaming server 400 can be through various network protocols. The preset network protocol can be one or more of a variety of network protocols. The base station 200 communicates with the streaming server 400 according to a preset network protocol to ensure the accuracy of image data transmission.

In some embodiments, the preset network protocol includes an RTMP network protocol.

It can be understood that the RTMP network protocol is a real-time message transmission protocol. The transmission of image data using the RTMP network protocol can transmit images or video to the streaming server 400 in real time, and the remote monitoring device 100 can also acquire image data in real time.

Referring to FIG. 21, in some embodiments, the work data includes status data of the drone 500, the control command includes a second control command for acquiring status data, and the base station 200 controls the drone 500 to transmit according to the control command. Work data includes:

S2222: The base station 200 controls the drone 500 to transmit status data according to the second control instruction.

Step S23: The base station 200 receives the working data, including:

S232: The base station 200 receives the status data.

Step S24: The base station 200 sends the working data to include:

S242: The base station 200 transmits status data to the server 300.

In some embodiments, the second processor 20 is further configured to:

Controlling the drone 500 to transmit status data according to the second control instruction;

Receiving status data; and

Status data is sent to the server 300.

In this manner, the base station 200 receives the status data transmitted by the drone 500 according to the control command, and transmits the status data of the drone 500 to the server 300. The remote monitoring device 100 communicates with the server 300 to effect acquisition of status data.

In other embodiments, the operational data may also include status data for the base station 200. The status data of the base station 200 includes location information of the base station 200 and the like. When the work data includes only the state data of the base station 200, the control method of the present embodiment only needs to perform step S242. When the work data includes both the state data of the drone 500 and the state data of the base station 200, step S2222, step S232, and step S242 are performed.

Referring to FIG. 22, in some embodiments, the sending, by the base station 200, the status data to the server 300 in step S242 includes:

S2421: The first service module 24 of the base station 200 sends status data to the second client module 222;

S2422: The second client module 222 of the base station 200 transparently transmits the status data to the first client module 221; and

S2423: The first client module 221 of the base station 200 sends status data to the server 300.

In some embodiments, step S2421, step S2422, and step S2423 can each be implemented by the second processor 20. That is to say, the second processor 20 is further used to:

Controlling the first service module 24 of the base station 200 to send status data to the second client module 222;

Controlling the second client module 222 of the base station 200 to transparently transmit state data to the first client module 221; and

The first client module 221 of the control base station 200 transmits status data to the server 300.

Specifically, referring to FIG. 17, after the status data is received by the first service module 24 of the base station 200, it is sent by the first service module 24 to the second client module 222 and transparently transmitted by the second client module 222 to the first client. The end module 221 is finally sent by the second client module 221 to the server 300. In this way, the transmission of the state data is realized, and the remote monitoring device 100 and the server 300 can realize the acquisition of the state data according to the foregoing manner, wherein the first client module 221 is responsible for processing the communication with the server 300, and the second client module 222 is configured. Responsible for processing the communication with the internal modules of the base station 200, and then the data transmission between the first client module 221 and the second client module 222 enables the communication link between the server 300 and the base station 200 to be clear, facilitating the post-communication procedure. Migration and modification.

In some embodiments, the base station 200 further includes a third client module, wherein the third client module can provide the user with an interaction interface of the base station 200, and the user operates the interaction interface of the base station 200 and inputs corresponding control control commands. The base station 200 can control the drone 500 according to the control command. The client module in the present invention may be an application program, a hardware module, or a combination of an application module and a hardware module.

Referring to FIG. 23, in some embodiments, the sending, by the base station 200, the status data to the server 300 in step S242 includes:

S2424: The base station 200 sends the status data to the server 300 by using a preset network protocol according to the second control instruction.

In some embodiments, step S2424 can be implemented by the second processor 20. That is to say, the second processor 20 is configured to send the status data to the server 300 through the preset network protocol according to the second control instruction.

It can be understood that the transmission of status data can be implemented between the base station 200 and the server 300 using a variety of network protocols. The preset network protocol is one or more of a variety of network protocols. The base station 200 communicates with the server 300 according to a preset network protocol to ensure the accuracy of the status data transmission.

In some embodiments, the preset network protocol includes a Websocket network protocol.

As such, duplex communication between the base station 200 and the server 300 is implemented using the Websocket network protocol. The Websocket network protocol ensures a persistent connection between the base station 200 and the server 300, improving the stability and accuracy of state data transmission.

In some embodiments, the status data of the drone 500 includes at least one of flight parameters, attitude information, location information, pan/tilt attitude information, and power information of the drone 500.

It can be understood that the status data of the drone 500 that the user often pays attention to includes flight parameters, posture information, position information, pan/tilt posture information, power amount information, and the like. The base station 200 receives and transmits these status information to facilitate the user's grasp of the flight status of the drone 500.

In some embodiments, the base station 200 receives status data that is periodically executed; the base station 200 transmits status data to the server 300 that is periodically performed.

It can be understood that the status data of the drone 500 when it is working is changed in real time. Therefore, the base station 200 needs to periodically receive the status data of the drone 500 and periodically transmit it to the server 300 to facilitate the user to grasp the flight status of the drone in real time. The period in which each state data is received and the period in which the state data is transmitted are short, so that the purpose of updating the state data in real time can be achieved.

Referring to FIG. 24, in some embodiments, the receiving, by the base station 200, the control instruction sent by the server 300 for controlling the drone 500 includes:

S213: The base station 200 receives a control instruction sent by the server 300 for controlling the drone 500 according to a preset network protocol.

In some embodiments, step S213 can be implemented by the second processor 20. That is to say, the second processor 20 is further configured to receive a control command sent by the server 300 for controlling the drone 500 according to a preset network protocol.

It can be understood that the transmission of control commands can be implemented between the base station 200 and the server 300 using a variety of network protocols. The preset network protocol is one or more of a variety of network protocols. The base station 200 communicates with the server 300 according to a preset network protocol to ensure the accuracy of control command transmission.

In some embodiments, the preset network protocol includes a Websocket network protocol.

As such, the transmission of control commands between the base station 200 and the server 300 is implemented using the Websocket network protocol. The Websocket network protocol can ensure a persistent connection between the base station 200 and the server 300, and improve the stability of control command transmission.

In some embodiments, the control instructions include third control commands for controlling the flight of the plurality of drones 500, and the third control commands are used to control the plurality of drones 500 to operate simultaneously and/or to poll for work.

It will be appreciated that in some cases a user may require multiple drones 500 to perform tasks simultaneously or in turn. Therefore, the base station 200 can control the plurality of drones 500 to perform tasks according to user requirements according to the third control command.

In some embodiments, the control instructions include a fourth control command for controlling the flight of the drone 500, and the fourth control command is for controlling the drone 500 to execute the preset course.

As such, the base station 200 can control the drone 500 to fly according to a preset route according to the fourth control command.

In some embodiments, the preset route includes a route stored in the server 300 and/or a route entered through the remote monitoring device 100.

It can be understood that the preset route may be stored in the server 300 or may be a preset route input by the user through the remote monitoring device 100. In this way, the drone 100 has a large degree of freedom of flight, and can further satisfy the user's use requirements.

In some embodiments, base station 200 can include a dedicated remote control, smartphone, computer, watch, wristband, ground control station that can control drone 500, can automatically replace battery or charge for drone 500 Equipment, etc. and combinations thereof.

Referring to FIGS. 25-26 together, the control method of the embodiment of the present invention is used for the server 300. The server 300 communicates with the base station 200 of the drone 500 and the remote monitoring device 100, respectively. Control methods include:

S31: The server 300 receives a control instruction sent by the remote monitoring device 100 for controlling the drone 500; and

S32: The server 300 transmits a control command to the base station 100.

Referring to FIG. 26 again, the control method of the embodiment of the present invention may be implemented by the server 300 of the embodiment of the present invention. The server 300 of the embodiment of the present invention includes a third processor 30. Step S31 and step S32 may be implemented by the third processor 30. That is to say, the third processor 30 is used to:

Receiving a control command sent by the remote monitoring device 100 for controlling the drone 500; and

A control command is sent to the base station 100.

As such, the server 300 acts as a relay station for communication between the remote monitoring device 100 and the base station 200, and forwards the control command sent by the remote monitoring device 100 to the base station 200 to implement remote monitoring of the drone 500 by the remote monitoring device 100.

Referring to FIG. 27-28, in some embodiments, the server 300 includes a communication service module 32. The step S31 receives the control command sent by the remote monitoring device 100 for controlling the drone 500, including:

S311: The communication service module 32 of the server 300 receives the control instruction.

Step S32: The server 300 sends a control instruction to the base station 100, including:

S321: The communication service module 32 of the server 300 sends a control command to the base station 200.

In some embodiments, step S311 and step S321 can be implemented by the third processor 30. That is to say, the third processor 30 is used to:

The communication service module 32 of the control server 300 receives the control instruction; and

The communication service module 32 of the control server 300 transmits a control command to the base station 200.

Specifically, the remote monitoring device 100 receives the control command and is received by the communication service module 32 of the server 300, and the communication service module 32 transmits the control command to the base station 200. As such, the server 300 performs the relaying and transmitting of the control commands to enable remote monitoring of the drone 500 by the remote monitoring device 100.

Referring to FIG. 29 to FIG. 30 together, in some embodiments, the communication service module 32 includes a first communication service module 321 and a second communication service module 322. Step S311: The communication service module 32 of the server 300 receives the control instruction, including:

S3111: The first communication service module 321 of the server 300 receives the control instruction;

S3112: The first communication service module 321 of the server 300 transparently transmits the control command to the second communication service module 322.

Step S321: The communication service module 32 of the server 300 sends a control command to the base station 200, including:

S3211: The second communication service module 322 of the server 300 sends a control command to the base station 200.

In some embodiments, step S1111, step S3112, and step S3211 can each be implemented by the third processor 30. That is to say, the third processor 30 is further used to:

The first communication service module 321 of the control server 300 receives the control instruction;

The first communication service module 321 of the control server 300 transparently transmits the control command to the second communication service module 322;

The second communication service module 322 of the control server 300 transmits a control command to the base station 200.

Referring to FIG. 31, specifically, the first communication service module 321 receives the control command sent by the remote monitoring device 100 and transmits the control command to the second communication service module 322, and the second communication service module 322 transmits the control command to the base station 200. As such, dividing the communication service module 32 into the first communication service module 321 and the second communication service module 322 can simplify the programming process of the server 300 of the embodiment of the present invention, wherein the first communication service module 321 is responsible for communicating with the remote monitoring device 100. The second communication service module 322 is responsible for communicating with the base station 200, and the data between the first communication service module 321 and the second communication service module 322 are mutually transparent, so that the server 300 and the base station 200 and the remote monitoring device 100 can be The communication link is more modular, facilitating modification and porting of the communication program of the later server 300.

Referring to FIG. 32, in some embodiments, the control method of the embodiment of the present invention further includes:

S33: The server 300 receives the work data of the drone 500 according to the control command.

In some embodiments, step S33 can be implemented by the third processor 30. That is to say, the third processor 30 is further configured to receive the work data of the drone 500 according to the control command.

As such, the server 300 receives the work data of the drone 500, and the remote monitoring device 100 communicates with the server 300. Therefore, the remote monitoring 100 can acquire the working data of the drone 500 through the server 300, and realize remote monitoring of the drone 500.

Referring to FIG. 3 and FIG. 33 together, in some embodiments, the work data includes an image acquisition address of image data captured by the drone 500, and the control instruction includes a first control instruction for acquiring an image acquisition address of the image data. The server 300 communicates with the streaming media server 400 that caches image data. Step S33: The server 300 receives the working data of the drone 500 according to the control command, including:

S331: The server 300 receives an image acquisition address corresponding to the image data sent by the streaming media server 400 according to the first control instruction.

The control method of the embodiment of the present invention further includes:

S34: The server 300 sends an image acquisition address to the remote monitoring device 100.

In some embodiments, step S331 and step S34 can be implemented by the third processor 30. That is to say, the third processor 30 is also used to:

The server 300 receives an image acquisition address corresponding to the image data sent by the streaming server 400 according to the first control instruction; and

The server 300 transmits an image acquisition address to the remote monitoring device 100.

Specifically, after the user issues the first control command to the remote monitoring device 100, the remote monitoring device 100 transmits the first control command to the server 300 and is forwarded by the server 300 to the base station 200. Subsequently, the base station 200 transmits image data to the streaming server 400. When receiving the image data transmitted by the base station 200, the streaming server 400 buffers the image data, and transmits the image acquisition address corresponding to the buffered image data to the server 300. The server 300 forwards the image acquisition address to the remote monitoring device 100. The remote monitoring device 100 can obtain image data of the drone based on the image acquisition address.

In some embodiments, the step S331 receives the image acquisition address corresponding to the image data sent by the streaming server 400 according to the first control instruction, and the first communication service module 321 of the control server 300 receives the streaming media according to the first control instruction. The image acquisition address corresponding to the image data transmitted by the server 400. The streaming media server 400 sends an image acquisition address to the server 300. The first communication service module 321 in the server 300 receives the image acquisition address, and then the first communication service module 321 sends the image acquisition address to the remote monitoring device 100.

In some embodiments, the step S331 receives the image acquisition address corresponding to the image data sent by the streaming server 400 according to the first control instruction, and the server 300 receives the image data corresponding to the image data sent by the streaming server 400 according to the first control instruction. An image acquisition address of one or more formats; step S34: The server 300 transmits an image acquisition address to the remote monitoring device 100. The server 300 transmits an image acquisition address of one or more formats to the remote monitoring device 100.

It can be understood that the remote monitoring device 100 may include multiple devices, such as devices of the IOS system, Android systems. The device or the like, and thus the format of the image acquisition address between the remote monitoring device 100 and the streaming server 400 is also various. In this way, the streaming media server 400 transmits the image acquisition addresses in multiple formats, so that the selection freedom of the remote monitoring device 100 is greater, that is, the various types of remote monitoring devices 100 can obtain the addresses according to the images of the corresponding formats. Get image data.

In a specific embodiment of the present invention, the format of the image acquisition address includes three formats: HLS, RTMP, and M3U8. Among them, the image acquisition address of the HLS format is applicable to devices of the IOS system, such as an Apple mobile phone, an Apple computer, and the like. Image acquisition addresses in RTMP and M3U8 formats are available for non-IOS devices such as Android phones, Windows computers, and more.

Referring to FIG. 34, in some embodiments, the work data includes status data of the drone 500, the control command includes a second control command for acquiring status data, and the server 300 receives the drone 500 according to the control command. Work data includes:

S332: The server 300 receives the status data according to the second control instruction.

The control method of the embodiment of the present invention further includes:

S35: The server 300 sends the status data to the remote monitoring device 100.

In some embodiments, step S331 and step S36 can be implemented by the third processor 30. That is to say, the third processor 30 is also used to:

Receiving status data according to the second control instruction;

Status data is sent to the remote monitoring device 100.

In this manner, the server 300 forwards the state data of the drone 500 received from the base station 200 to the remote monitoring device 100, and the user can grasp the working flight state of the drone 500 in real time through the remote monitoring device 100.

In other embodiments, the operational data may also include status data for the base station 200. The status data of the base station 200 includes location information of the base station 200 and the like. When the working data includes only the status data of the base station 200, the control method of the present embodiment receives the status data of the base station 200 according to the control command for acquiring the status data of the base station 200 and performs the step S35 to send the status data of the base station 200 to the remote monitoring device. 100. When the working data includes the status data of the UAV 500 and the status data of the base station 200, the control method of the present embodiment receives the status data of the base station 200 according to the control command for acquiring the status data of the base station 200, and performs the steps. S332 receives the state data of the drone 500 according to the second control command, and then performs step S35 to transmit the state data of the base station 200 and the state data of the drone 500 to the remote monitoring device 100.

Referring to FIG. 35, in some embodiments, the step S332 receives the status data by the server 300 according to the second control instruction, including:

S3321: The second communication service module 322 of the server 300 receives the status data.

S3322: The second communication service module 322 of the server 300 transparently transmits the status data to the first communication service of the server 300. Service module 321.

Step S35: The server 300 sends the status data to the remote monitoring device 100, including:

S351: The first communication service module 321 of the server 300 sends the status data to the remote monitoring device 100.

In some embodiments, step S3321 and step S3322 and step S351 can each be implemented by the third processor 30. That is to say, the third processor 30 is further used to:

The second communication service module 322 of the control server 300 receives the status data;

The second communication service module 322 of the control server 300 transparently transmits the status data to the first communication service module 321 of the server 300; and

The first communication service module 321 of the control server 300 transmits status data to the remote monitoring device 100.

Specifically, in conjunction with FIG. 31, the base station 200 transmits the status data of the drone 500 and then receives it from the second communication service module 322 and transparently transmits it to the first communication service module 321, and the first communication service module transmits the status data to the remote. Monitoring device 100. In this way, the user's grasp of the flight state of the drone 500 is achieved.

In some embodiments, the status data of the drone 500 includes at least one of flight parameters, attitude information, location information, pan/tilt attitude information, and power information.

As can be understood, the status data of the drone 500 that the user frequently pays attention to includes flight parameters, posture information, position information, pan/tilt posture information, power amount information, and the like. The server 300 forwards these status information to facilitate the user's grasp of the flight status of the drone 500.

Referring to FIG. 36, in some embodiments, the control method of the embodiment of the present invention further includes:

S36: The server 300 receives an image processing instruction of the image data sent by the remote monitoring device 100; and

S37: The server 300 sends an image processing instruction to the streaming server 400, and the image processing instruction is used to process the image data captured by the drone 500 buffered in the streaming server 400.

In some embodiments, step S37 and step S38 can be implemented by the third processor 30. That is to say, the third processor 30 is also used to:

An image processing instruction for receiving image data transmitted by the remote monitoring device 100; and

The image processing instructions are sent to the streaming server 400 for processing the image data captured by the drone 500 buffered in the streaming server 400.

It will be appreciated that the user may wish to process image data taken by the drone 500. For example, in some cases users would like to be able to store some of the more important data in the image data. In this manner, after receiving the image processing instruction sent by the remote monitoring device 100, the server 300 forwards the image processing instruction to the streaming server 400. The streaming server 400 performs corresponding processing on the image data in accordance with the image processing instructions.

In some embodiments, the image processing instructions include one or more of storing, classifying, watermarking, and extracting the target data.

In this way, image data can be processed in various ways to meet the user's needs.

Referring again to FIG. 31, in some embodiments, server 300 also includes database 34 and algorithm engine 36. The database 34 stores form information of the remote monitoring device 100, form information of the base station 200, routes, and the like. The algorithm engine 36 stores an algorithm program for controlling the cooperation or polling work of the plurality of drones 500. When the server 300 receives the control command sent by the remote monitoring device 100, the database 34 matches the remote monitoring device 100 and the corresponding base station 200 according to the control command to find the corresponding execution base station 200 of the control command. When the remote monitoring device 100 transmits a control command for controlling the simultaneous operation or polling operation of the plurality of drones 500, the algorithm engine 36 invokes a corresponding algorithm program to implement manipulation of the plurality of drones 500 according to the control command.

Referring to FIG. 37, FIG. 38 and FIG. 41 together, the control method of the embodiment of the present invention is applied to the streaming server 400. The streaming server 400 communicates with the base station 200 and the server 300, respectively. Control methods include:

S41: The streaming server 400 receives the image data captured by the drone 500 transmitted by the base station 200; and

S42: The streaming server 400 transmits an image acquisition address corresponding to the image data to the server 300.

Referring to FIG. 38 again, the control method of the embodiment of the present invention may be implemented by the streaming server 400 of the embodiment of the present invention. The streaming server 400 of the embodiment of the present invention includes a fourth processor 40. Step S41 and step S42 may be implemented by the fourth processor 40. That is to say, the fourth processor 40 is used to:

Receiving image data captured by the drone 500 transmitted by the base station 200; and

An image acquisition address corresponding to the image data is transmitted to the server 300.

It can be understood that the streaming media server 400 buffers the image data of the drone 500, and the remote monitoring device 100 can acquire the image data of the drone 500 according to the image acquisition address sent by the streaming server 400 forwarded by the server 300. In this way, remote monitoring of the drone 500 by the user is achieved.

In some embodiments, the streaming server 400 transmits an image acquisition address to the first communication service module 321 of the server 300, wherein the server 300 transmits the image acquisition address to the remote monitoring device 100 via the first communication service module 321 .

Referring to FIG. 39, in some embodiments, the streaming media server 400 receives the image data captured by the drone transmitted by the base station 200 in step S41.

S411: The streaming server 400 receives the image data captured by the drone 500 sent by the base station 200 according to a preset network protocol.

In some embodiments, step S411 can be implemented by fourth processor 40. That is to say, the fourth processor 40 is further configured to receive image data captured by the drone 500 transmitted by the base station 200 according to a preset network protocol.

It can be understood that the transmission of image data can be implemented between the streaming server 400 and the base station 200 according to various network protocols. The preset network protocol is one or more of a variety of network protocols. In this manner, the streaming server 400 communicates with the base station 200 according to a preset network protocol to ensure the accuracy of image data transmission.

In some embodiments, the preset network protocol includes an RTMP network protocol.

It can be understood that the RTMP network protocol is a real-time message transmission protocol. The streaming media server 400 can receive the image data in real time by using the RTMP network protocol to receive the image data, and the remote monitoring device 100 can also acquire the image data in real time.

In some embodiments, the step S42 that the streaming server 400 transmits the image acquisition address corresponding to the image data comprises transmitting an image acquisition address of one or more formats corresponding to the image data.

It can be understood that the remote monitoring device 100 may include various devices, such as devices of the IOS system, devices of the Android system, and the like, and thus the format of the image acquisition address between the remote monitoring device 100 and the streaming media server 400 is also various. In this way, the streaming media server 400 transmits the image acquisition addresses in multiple formats, so that the selection freedom of the remote monitoring device 100 is greater, that is, the various types of remote monitoring devices 100 can obtain the addresses according to the images of the corresponding formats. Get image data.

In a specific embodiment of the present invention, the format of the image acquisition address includes three formats: HLS, RTMP, and M3U8. Among them, the image acquisition address of the HLS format is applicable to devices of the IOS system, such as an Apple mobile phone, an Apple computer, and the like. Image acquisition addresses in RTMP and M3U8 formats are available for non-IOS devices such as Android phones, Windows computers, and more.

Referring to FIG. 40, in some embodiments, the control method of the embodiment of the present invention further includes:

S43: The streaming server 400 receives an image processing instruction of the image data sent by the server 300; and

S44: The streaming server 400 processes the image data according to the image processing instruction.

In some embodiments, step S43 and step S44 can be implemented by fourth processor 40. That is to say, the fourth processor 40 is also used to:

An image processing instruction that receives image data transmitted by the server 300; and

The image data is processed in accordance with image processing instructions.

It will be appreciated that the user may wish to process image data taken by the drone 500. For example, in some cases users would like to be able to store some of the more important data in the image data. In this manner, after receiving the image processing instruction forwarded by the server 300, the streaming server 400 can perform corresponding processing on the image data according to the image processing instruction.

In some embodiments, the image processing instructions include one or more of storing, classifying, watermarking, and extracting the target data.

In this way, image data can be processed in various ways to meet the user's needs.

Referring to FIG. 41, in summary, the control method for controlling the remote monitoring device 100, the base station 200, the server 300, and the streaming media server 400, respectively, of the embodiments of the present invention may implement the remote monitoring device 100 remotely from the drone 500. monitor. The dotted line shown in FIG. 40 indicates the data flow direction of the image data captured by the drone 500, and the solid line shown in FIG. 41 indicates the control command determined by the remote monitoring device 100 according to the user operation, the state data and image of the drone 100. The data flow of the image acquisition address corresponding to the processing instruction and the image data.

Specifically, when the user issues a control instruction for acquiring graphic data to the remote monitoring device 100, the remote monitoring device 100 sends a control command to the first communication service module 321, and the first communication service module 321 transparently transmits the control command to the second communication service module. 322. The second communication service module 322 sends a control command to the first client module 221, the first client module 221 transparently transmits the control command to the second client module 222, and the second client module 222 sends the control command to the first service module 24. Finally, the first service module 24 controls the drone 500 to transmit image data according to the control command. The image data is captured by the drone 500 and sent to the first service module 24, and then transmitted by the first service module 24 to the streaming server 400 for caching. At this time, the streaming server 400 transmits an image acquisition address corresponding to the image data to the server 300, and is forwarded by the server 300 to the remote monitoring device 100. As such, the remote monitoring device 100 can acquire image data from the streaming server 400 based on the image acquisition address.

In addition, the status data of the drone 500 is transmitted by the drone 500 to the first service module 24 and forwarded by the first service module 24 to the second client module 222, and the second client module 222 transparently transmits the status data to the first The client module 221 is sent by the first client module 221 to the second communication service module 322. The second communication service module 322 transparently transmits the status data to the first communication service module 321, and finally is sent by the first communication service module 321 to the remote. The monitoring device 100 is implemented to enable the user to grasp the flight state of the drone 500. When the user issues an image processing instruction for processing image data, the remote monitoring device 100 transmits an image processing instruction to the first communication service module 321, and the first communication service module 321 sends the image processing instruction to the streaming server 400, which is executed by the streaming server 400. The image processing instructions cache the image data into the object storage system 600. When the user issues a control command for controlling the plurality of drones 500 to work or poll the work or control the drone to execute the preset route, the remote monitoring device 100 sends a control command to the first communication service module 321, and the server 300 The corresponding algorithm program is called from the algorithm engine 36 to achieve the target operation.

The service module in the present invention may be an application program, a hardware module, or a combination of an application program and a hardware module.

In this way, even if the user is far away from the drone 500, the user can easily realize the manipulation of the drone 500, and is suitable for various application scenarios such as security monitoring and out-of-office duty, and is convenient for the user to use.

In the description of the present specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiment", "example", "specific example", or "some examples", etc. Particular features, structures, materials or features described in the embodiments or examples are included in at least one embodiment or example of the invention. In the present specification, the schematic representation of the above terms does not necessarily mean the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in a suitable manner in any one or more embodiments or examples.

Any process or method description in the flowcharts or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for performing the steps of a particular logical function or process. And the scope of the preferred embodiments of the invention includes additional implementations, which may not be in the order shown or discussed. The functions are performed in a substantially simultaneous manner or in the reverse order depending on the functions involved, as will be understood by those skilled in the art to which the embodiments of the present invention pertain.

The logic and/or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for performing logical functions, and may be embodied in any computer readable medium, Used in conjunction with, or in conjunction with, an instruction execution system, apparatus, or device (eg, a computer-based system, a system including a processor, or other system that can fetch instructions and execute instructions from an instruction execution system, apparatus, or device) Or use with equipment. For the purposes of this specification, a "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport a program for use in an instruction execution system, apparatus, or device, or in conjunction with the instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer readable media include the following: electrical connections (electronic devices) having one or more wires, portable computer disk cartridges (magnetic devices), random access memory (RAM), Read only memory (ROM), erasable editable read only memory (EPROM or flash memory), fiber optic devices, and portable compact disk read only memory (CDROM). In addition, the computer readable medium may even be a paper or other suitable medium on which the program can be printed, as it may be optically scanned, for example by paper or other medium, followed by editing, interpretation or, if appropriate, other suitable The method is processed to obtain the program electronically and then stored in computer memory.

It should be understood that portions of the invention may be implemented in hardware, software, firmware or a combination thereof. In the above-described embodiments, multiple steps or methods may be performed by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if executed in hardware, as in another embodiment, it can be performed by any one of the following techniques or combinations thereof known in the art: having logic gates for performing logic functions on data signals Discrete logic circuits, application specific integrated circuits with suitable combinational logic gates, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

Those skilled in the art can understand that all or part of the steps carried in carrying out the above implementation method can be completed by a program to instruct related hardware, and the program can be stored in a computer readable storage medium, and the program is executed. Including one or a combination of the steps of the method embodiments.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above integrated modules can be executed in the form of hardware or in the form of software functional modules. The integrated modules, if executed in the form of software functional modules and sold or used as separate products, may also be stored in a computer readable storage medium.

The above mentioned storage medium may be a read only memory, a magnetic disk or an optical disk or the like. Although the embodiments of the present invention have been shown and described, it is understood that the above-described embodiments are illustrative and are not to be construed as limiting the scope of the invention. The embodiments are subject to variations, modifications, substitutions and variations.

Claims (100)

A method for controlling a remote monitoring device, wherein the remote monitoring device communicates with a server, wherein the control method comprises: The remote monitoring device detects a user's operation; The remote monitoring device determines a control instruction for controlling the drone according to an operation of the user; and The remote monitoring device transmits a control command of the drone to the server to enable the drone to acquire the control command. The control method according to claim 1, wherein after the remote monitoring device sends the control command to the server, the control method further includes: The remote monitoring device acquires the working data of the drone according to the control instruction. The control method according to claim 2, wherein said work data comprises image data captured by said drone, said control command comprising first control instructions for acquiring said image data, said server Communicating with the streaming server that caches the image data, the remote monitoring device acquiring the working data of the drone according to the control instruction includes: The remote monitoring device acquires the image data from the streaming server according to the first control instruction. The control method according to claim 3, wherein the unmanned aerial vehicle communicates with the base station, and the remote monitoring device acquires the image data from the streaming media server according to the first control instruction, including: Receiving, by the remote monitoring device, an image acquisition address sent by the server, the image acquisition address being sent to the server when the streaming media server buffers the image data sent by a base station communicating with the drone Sent; and The remote monitoring device acquires the image data from the streaming server according to the image acquisition address. The control method according to claim 4, wherein the image acquisition address comprises one or more formats; The acquiring, by the remote monitoring device, the image data from the streaming media server according to the image obtaining address includes: The remote monitoring device acquires the image data from the streaming server according to one of the image acquisition address or the image acquisition address of a plurality of formats. The control method according to claim 2, wherein said work data includes status data of said drone, said control command includes second control command for acquiring said status data, said remote monitoring Obtaining the working data of the drone according to the control instruction includes: The remote monitoring device acquires the status data from the server according to the second control instruction. The control method according to claim 6, wherein said remote monitoring device is in accordance with said second control The obtaining the status data from the server includes: The remote monitoring device receives the status data sent from the server by using a preset network protocol according to the second control instruction. The control method according to claim 7, wherein the preset network protocol comprises a Websocket network protocol. The control method according to claim 6, wherein the state data of the drone includes at least one of flight parameters, attitude information, position information, pan/tilt attitude information, and power amount information of the drone . The control method according to claim 1, wherein said control command comprises a third control command for controlling a plurality of said drones to fly, said third control command for controlling said plurality of said none The human machine works and/or polls at the same time. The control method according to claim 1, wherein said control command comprises a fourth control command for controlling flight of said drone, said fourth control command for controlling said drone to execute Set the route. The control method according to claim 11, wherein the preset route includes a route stored in the server and/or a route input through the remote monitoring device. The control method according to claim 6, wherein the remote monitoring device acquires the status data from the server according to the second control instruction, including: The remote monitoring device receives one or more of all of the status data or the status data according to the second control instruction. The control method according to claim 1, wherein the control method further comprises: The remote monitoring device determines an image processing instruction of the image data according to an operation of the user; and The remote monitoring device sends the image processing instruction to the server, and the image processing instruction is configured to process image data captured by the drone cached in the streaming server. The control method according to claim 14, wherein the image processing instruction comprises processing, storing, classifying, watermarking, and extracting one or more of the target objects. A method for controlling a base station, wherein the base station communicates with a drone and a server, respectively, wherein the control method includes: Receiving, by the base station, a control instruction sent by the server for controlling the drone; and The base station controls the drone according to the control instruction. The control method according to claim 16, wherein the base station comprises a communication client module and a first service module, and the base station receiving the control command sent by the server for controlling the drone comprises: The communication client module of the base station receives the control command sent by the server; and The communication client module of the base station sends the control command to the first service module; The controlling, by the base station, the drone according to the control instruction includes: The first service module of the base station controls the drone according to the control instruction. The control method according to claim 17, wherein said communication client module comprises a first client module and a second client module, and said communication client module of said base station receives said control sent by said server Instructions include: Receiving, by the first client module of the base station, the control instruction sent by the server; and The first client module of the base station transparently transmits the control command to the second client module; The sending, by the communication client of the base station, the control command to the first service module includes: The second client module of the base station sends the control command to the first service module. The control method according to claim 16, wherein the controlling, by the base station, the drone according to the control instruction comprises: The base station controls the drone to send work data according to the control instruction; The control method further includes: The base station receives the work data; and The base station transmits the work data. The control method according to claim 19, wherein said work data comprises image data captured by said drone, said control command comprising first control command for acquiring said image data, said base station Communicating with the streaming server, the base station controlling, according to the control instruction, the sending of the working data by the drone includes: The base station controls the drone to transmit the image data according to the first control instruction; The receiving, by the base station, the working data includes: The base station receives the image data; The sending, by the base station, the working data includes: The base station transmits the image data to the streaming server. The control method according to claim 20, wherein the transmitting, by the base station, the image data to the streaming media server comprises: Sending, by the base station, the image data to the streaming media server by using a preset network protocol according to the first control instruction. The control method according to claim 21, wherein said preset network protocol comprises an RTMP network protocol. The control method according to claim 19, wherein said work data includes status data of said drone, said control command includes second control command for acquiring said status data, said base station The controlling instruction controlling the drone to send the working data includes: The base station controls the drone to send the status data according to the second control instruction; The receiving, by the base station, the working data includes: The base station receives the status data; The sending, by the base station, the working data includes: The base station sends the status data to the server. The control method according to claim 23, wherein the sending, by the base station, the status data to the server comprises: The first service module of the base station sends the status data to the second client module; The second client module of the base station transparently transmits the status data to the first client module; and The first client module of the base station sends the status data to the server. The control method according to claim 23, wherein the sending, by the base station, the status data to the server comprises: The base station sends the status data to the server by using a preset network protocol according to the second control instruction. The control method according to claim 25, wherein the preset network protocol comprises a Websocket network protocol. The control method according to claim 23, wherein the state data of the drone includes at least one of flight parameters, attitude information, position information, pan/tilt attitude information, and power amount information. The control method according to claim 23, wherein said receiving, by said base station, said status data is performed periodically; The transmitting, by the base station, the status data to the server is performed periodically. The control method according to claim 16, wherein the receiving, by the base station, the control command sent by the server for controlling the drone comprises: The base station receives, according to a preset network protocol, a control command sent by the server for controlling the drone. The control method according to claim 29, wherein the preset network protocol comprises a Websocket network protocol. The control method according to claim 16, wherein said control command comprises a third control command for controlling a plurality of said drones to fly, said third control command for controlling said plurality of said none The human machine works and/or polls at the same time. The control method according to claim 16, wherein said control command includes a fourth control command for controlling flight of said drone, said fourth control command for controlling said drone to execute Set the route. The control method according to claim 32, wherein the preset route includes a route stored in the server and/or a route input through the remote monitoring device. A server control method, wherein the server communicates with a base station of a drone and a remote monitoring device, respectively, wherein the control method comprises: Receiving, by the server, a control instruction sent by the remote monitoring device for controlling the drone; and The server sends the control command to the base station. The control method according to claim 34, wherein the server comprises a communication service module, and the server receiving the control instruction sent by the remote monitoring device for controlling the drone comprises: The communication service module of the server receives the control instruction; The sending, by the server, the control instruction to the base station includes: The communication service module of the server sends the control command to the base station. The control method according to claim 35, wherein the communication service module comprises a first communication service module and a second communication service module, and the communication service module of the server receiving the control instruction comprises: The first communication service module of the server receives the control instruction; and The first communication service module of the server transparently transmits the control instruction to the second communication service module; The sending, by the communication service module of the server, the control command to the base station includes: The second communication service module of the server sends the control command to the base station. The control method according to claim 34, wherein the control method further comprises: The server receives the work data of the drone according to the control instruction. The control method according to claim 37, wherein said work data comprises an image acquisition address of image data captured by said drone, said control command comprising first control for acquiring said image acquisition address An instruction, the server is in communication with a streaming server that caches the image data, and the server receiving the working data of the drone according to the control instruction includes: Receiving, by the server, an image acquisition address corresponding to the image data sent by the streaming media server according to the first control instruction; The control method further includes: The server sends the image acquisition address to the remote monitoring device. The control method according to claim 38, wherein the receiving, by the server, the image acquisition address corresponding to the image data sent by the streaming media server according to the first control instruction comprises: Receiving, by the server, an image acquisition address of one or more formats corresponding to the image data sent by the streaming media server according to the first control instruction; The sending, by the server, the image obtaining address to the remote monitoring device includes: The server transmits the image acquisition address in one or more formats to the remote monitoring device. The control method according to claim 37, wherein said work data includes a shape of said drone The control data includes a second control instruction for acquiring the status data, and the server receiving the working data of the drone according to the control instruction includes: The server receives the status data according to the second control instruction; The control method further includes: The server sends the status data to the remote monitoring device. The control method according to claim 40, wherein the receiving, by the server, the status data according to the second control instruction comprises: The second communication service module of the server receives the status data according to the second control instruction; The second communication service module of the server transparently transmits the status data to the first communication service module of the server; The sending, by the server, the status data to the remote monitoring device includes: The first communication service module of the server sends the status data to the remote monitoring device. The control method according to claim 40, wherein the status data of the drone includes at least one of flight parameters, attitude information, position information, pan/tilt attitude information, and power amount information. The control method according to claim 34, wherein the control method further comprises: Receiving, by the server, an image processing instruction of the image data sent by the remote monitoring device; and And the server sends the image processing instruction to the streaming media server, where the image processing instruction is used to process image data captured by the drone cached in the streaming media server. The control method according to claim 43, wherein said image processing instruction comprises one or more of storing, classifying, watermarking, and extracting the target object. A method for controlling a streaming media server, wherein the streaming media server communicates with a base station and a server of a drone, respectively, wherein the control method comprises: Receiving, by the streaming media server, image data captured by the drone sent by the base station; and The streaming server transmits an image acquisition address corresponding to the image data to the server. The control method according to claim 45, wherein the receiving, by the streaming server, image data captured by the drone by the base station comprises: And the streaming media server receives image data captured by the base station and sent by the base station according to a preset network protocol. The control method according to claim 46, wherein said preset network protocol comprises an RTMP network protocol. The control method according to claim 45, wherein the streaming media server transmitting an image acquisition address corresponding to the image data comprises transmitting an image acquisition location of one or more formats corresponding to the image data site. The control method according to claim 45, wherein the control method further comprises: The streaming server receives an image processing instruction of the image data sent by the server; and The streaming server processes the image data in accordance with the image processing instructions. The control method according to claim 49, wherein said image processing instruction comprises one or more of storing, classifying, watermarking, and extracting said image data. A remote monitoring device, wherein the remote monitoring device is in communication with a server, wherein the remote monitoring device comprises a processor, and the processor is configured to: Detecting user operations; Determining a control instruction for controlling the drone according to an operation of the user; and Sending a control command of the drone to the server to enable the drone to acquire the control command. The remote monitoring device of claim 51, wherein the processor is further configured to: Obtaining the working data of the drone according to the control instruction. A remote monitoring device according to claim 51, wherein said work data comprises image data captured by said drone, said control command comprising first control instructions for acquiring said image data, said The server is in communication with a streaming server that caches the image data, the processor further configured to: Acquiring the image data from the streaming server according to the first control instruction. The remote monitoring device of claim 53, wherein the drone communicates with a base station, the processor further configured to: Receiving an image acquisition address transmitted by the server, the image acquisition address being transmitted to the server when the streaming server caches the image data transmitted by a base station in communication with the drone; and Acquiring the image data from the streaming server according to the image acquisition address. A remote monitoring device according to claim 54, wherein said image acquisition address comprises one or more formats; The obtaining the image data from the streaming media server according to the image obtaining address includes: The image data is acquired from the streaming server according to one of the image acquisition address of one format or the image acquisition address of a plurality of formats. A remote monitoring device according to claim 51, wherein said work data includes status data of said drone, said control command including a second control command for acquiring said status data, said process Also used to: The status data is obtained from the server according to the second control instruction. The remote monitoring device of claim 56, wherein the processor is further configured to: Receiving the status data transmitted from the server by a preset network protocol according to the second control instruction. The remote monitoring device of claim 57, wherein the predetermined network protocol comprises a Websocket network protocol. The remote monitoring device according to claim 56, wherein the state data of the drone includes at least one of flight parameters, posture information, position information, pan/tilt posture information, and power amount information of the drone Kind. A remote monitoring device according to claim 51, wherein said control command includes a third control command for controlling a plurality of said drones to fly, said third control command for controlling said plurality of said The drone works and/or polls at the same time. A remote monitoring device according to claim 51, wherein said control command includes a fourth control command for controlling flight of said drone, said fourth control command for controlling execution of said drone Preset route. A remote monitoring device according to claim 61, wherein said predetermined route comprises a route stored in said server and/or a route entered through said remote monitoring device. The remote monitoring device according to claim 56, wherein the obtaining the status data from the server according to the second control instruction comprises: Controlling, by the second control instruction, the remote monitoring device to receive one or more of all of the status data or the status data. The remote monitoring device of claim 51, wherein the processor is further configured to: Determining an image processing instruction of the image data according to an operation of the user; and Sending the image processing instruction to the server, the image processing instruction for processing image data captured by the drone cached in the streaming server. The remote monitoring device of claim 64, wherein the image processing instructions comprise processing, storing, sorting, watermarking, and extracting one or more of the target objects. A base station, wherein the base station communicates with a drone and a server, respectively, wherein the base station includes a processor, and the processor is configured to: Receiving a control instruction sent by the server for controlling the drone; and The drone is controlled according to the control command. The base station according to claim 66, wherein the base station further comprises a communication client module and a first service module, the processor further configured to: Controlling, by the communication client module of the base station, the control instruction sent by the server; Controlling, by the communication client module of the base station, the control command to send the first service module; and The first service module controlling the base station controls the drone according to the control instruction. The base station according to claim 67, wherein said communication client module comprises a first client module a block and a second client module, the processor further for: Controlling, by the first client module of the base station, the control instruction sent by the server; Controlling, by the first client module of the base station, the control command to transparently transmit the control command to the second client module; and Controlling, by the second client module of the base station, the control command to the first service module. The base station according to claim 66, wherein said processor is further configured to: Controlling, by the control instruction, the drone to send work data according to the control instruction; Receiving the work data; and Send the work data. The base station according to claim 69, wherein said work data comprises image data captured by said drone, said control command comprising first control instructions for acquiring said image data, said base station and Streaming media server communication, the processor is further configured to: Controlling, by the first control instruction, the drone to transmit the image data; Receiving the image data; Sending the image data to the streaming server. The base station according to claim 70, wherein said processor is further configured to: And transmitting the image data to the streaming media server by using a preset network protocol according to the first control instruction. The base station according to claim 71, wherein said predetermined network protocol comprises an RTMP network protocol. The base station according to claim 69, wherein said work data includes status data of said drone, said control command includes second control instructions for acquiring said status data, said processor further Used for: Controlling, by the second control instruction, the drone to send the status data; Receiving the status data; The status data is sent to the server. The base station according to claim 73, wherein the processor is further configured to: Controlling, by the first service module of the base station, the status data to send to the second client module; Controlling, by the second client module of the base station, transparently transmitting the status data to the first client module; and The first client module controlling the base station transmits the status data to the server. The base station according to claim 73, wherein said processor is further configured to: And transmitting the status data to the server by using a preset network protocol according to the second control instruction. The base station according to claim 75, wherein said predetermined network protocol comprises a Websocket network protocol. The base station according to claim 73, wherein the status data of the drone includes at least one of flight parameters, attitude information, position information, pan/tilt attitude information, and power amount information. The base station according to claim 73, wherein said receiving said status data is performed periodically; The transmitting the status data to the server is performed periodically. The base station according to claim 66, wherein said processor is further configured to: Receiving, by the preset network protocol, a control instruction sent by the server for controlling the drone. The base station according to claim 79, wherein said predetermined network protocol comprises a Websocket network protocol. The base station according to claim 66, wherein said control command includes a third control command for controlling a plurality of said drones to fly, said third control command for controlling said plurality of said unmanned persons The machine works and/or polls at the same time. The base station according to claim 66, wherein said control command includes a fourth control command for controlling flight of said drone, said fourth control command for controlling said drone to execute a preset route. The base station according to claim 82, wherein said preset route comprises a route stored in said server and/or a route input through said remote monitoring device. A server, wherein the server communicates with a base station and a remote monitoring device of the drone, respectively, wherein the server includes a processor, and the processor is configured to: Receiving a control instruction sent by the remote monitoring device for controlling the drone; and Sending the control command to the base station. The server of claim 84, wherein the server further comprises a communication service module, the processor further configured to: A communication service module controlling the server receives the control instruction; and A communication service module controlling the server transmits the control command to the base station. The server according to claim 85, wherein said communication service module comprises a first communication service module and a second communication service module, said processor being further configured to: Receiving, by the first communication service module of the server, the control instruction; Controlling, by the first communication service module of the server, the control command to transparently transmit the control command to the second communication service module; and A second communication service module controlling the server transmits the control command to the base station. The server of claim 84, wherein the processor is further configured to: Receiving work data of the drone according to the control instruction. A server according to claim 87, wherein said work data includes an image acquisition address of image data captured by said drone, said control command including a first control command for acquiring said image acquisition address The server is in communication with a streaming server that caches the image data, and the processor is further configured to: Receiving, according to the first control instruction, an image acquisition address corresponding to the image data sent by the streaming media server; and Sending the image acquisition address to the remote monitoring device. The server according to claim 88, wherein the receiving an image acquisition address corresponding to the image data sent by the streaming server according to the first control instruction comprises: Receiving, according to the first control instruction, the image acquisition address of one or more formats corresponding to the image data sent by the streaming media; The sending the image acquisition address to the remote monitoring device includes: The image acquisition address in one or more formats is sent to the remote monitoring device. A server according to claim 87, wherein said work data includes status data of said drone, said control command includes second control instructions for acquiring said status data, said processor further Used for: Receiving the status data according to the second control instruction; and Sending the status data to the remote monitoring device. The server of claim 90, wherein the processor is further configured to: The second communication service module controlling the server receives the status data according to the second control instruction; Controlling, by the second communication service module of the server, the state data to transparently transmit to the first communication service module of the server; and A first communication service module controlling the server transmits the status data to the remote monitoring device. The server according to claim 90, wherein the status data of the unmanned aerial vehicle comprises at least one of flight parameters, attitude information, position information, pan/tilt attitude information, and power amount information. The server of claim 84, wherein the processor is further configured to: Receiving an image processing instruction of the image data transmitted by the remote monitoring device; and And sending the image processing instruction to the streaming server, where the image processing instruction is used to process image data captured by the drone cached in the streaming server. The server of claim 94, wherein the image processing instructions comprise one or more of storing, classifying, watermarking, and extracting the target object. A streaming media server, wherein the streaming media server communicates with a base station and a server of a drone, respectively, wherein the streaming media server includes a processor, and the processor is configured to: Receiving image data captured by the drone transmitted by the base station; and An image acquisition address corresponding to the image data is transmitted to the server. The streaming media server of claim 95, wherein the processor is further configured to: Receiving image data captured by the base station transmitted by the base station according to a preset network protocol. The streaming server of claim 96, wherein the predetermined network protocol comprises an RTMP network protocol. The streaming server according to claim 95, wherein said transmitting an image acquisition address corresponding to said image data comprises transmitting an image acquisition address of one or more formats corresponding to said image data. The streaming media server of claim 95, wherein the processor is further configured to: An image processing instruction that receives image data transmitted by the server; and The image data is processed in accordance with the image processing instructions. The streaming server of claim 99, wherein the image processing instructions comprise one or more of storing, classifying, watermarking, and extracting the target object.

Images