WO2025127558A1 - Drone and drone control method therefor - Google Patents
Drone and drone control method therefor Download PDFInfo
- Publication number
- WO2025127558A1 WO2025127558A1 PCT/KR2024/019419 KR2024019419W WO2025127558A1 WO 2025127558 A1 WO2025127558 A1 WO 2025127558A1 KR 2024019419 W KR2024019419 W KR 2024019419W WO 2025127558 A1 WO2025127558 A1 WO 2025127558A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- drone
- situation
- motor
- neural network
- signal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- B64U20/00—Constructional aspects of UAVs
- B64U20/80—Arrangement of on-board electronics, e.g. avionics systems or wiring
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64C—AEROPLANES; HELICOPTERS
- B64C15/00—Attitude, flight direction, or altitude control by jet reaction
- B64C15/02—Attitude, flight direction, or altitude control by jet reaction the jets being propulsion jets
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64C—AEROPLANES; HELICOPTERS
- B64C39/00—Aircraft not otherwise provided for
- B64C39/02—Aircraft not otherwise provided for characterised by special use
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64C—AEROPLANES; HELICOPTERS
- B64C39/00—Aircraft not otherwise provided for
- B64C39/02—Aircraft not otherwise provided for characterised by special use
- B64C39/024—Aircraft not otherwise provided for characterised by special use of the remote controlled vehicle type, i.e. RPV
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- B64U50/00—Propulsion; Power supply
- B64U50/10—Propulsion
- B64U50/19—Propulsion using electrically powered motors
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/40—Control within particular dimensions
- G05D1/49—Control of attitude, i.e. control of roll, pitch or yaw
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- B64U2201/00—UAVs characterised by their flight controls
- B64U2201/10—UAVs characterised by their flight controls autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D2109/00—Types of controlled vehicles
- G05D2109/20—Aircraft, e.g. drones
- G05D2109/25—Rotorcrafts
Definitions
- the present embodiments relate to a drone for controlling an aircraft in a specific situation and a method for controlling the drone.
- attitude and speed are important factors that determine the stability and maneuverability of the drone.
- the present embodiments can provide a drone and a method for controlling the drone that stably controls the drone with low power and low latency in specific situations such as a gust of wind.
- the present embodiments provide a drone and a control method for the drone, which controls the drone to fly along a specific flight trajectory by converting a motor signal into a motor control signal in a general situation, recognizes a specific situation by measuring attitude based on sensor data using a learned spike neural network, and controls the drone to fly while maintaining attitude by adjusting the motor control signal in a specific situation.
- the present embodiments may provide a drone including a drone sensor unit that provides sensor data, a drone flight unit that provides motor signals, and a drone control unit that converts the motor signals into the motor control signals in general situations to control the drone to fly along a specific flight trajectory, measures attitude based on the sensor data using a learned spike neural network to recognize a specific situation, and controls the drone to fly while maintaining attitude by adjusting the motor control signals in specific situations.
- the present embodiments can provide a method for controlling a drone, including a first step of controlling the drone to fly along a specific flight trajectory by converting a motor signal into a motor control signal in a general situation, and a second step of controlling the drone to recognize a specific situation by measuring an attitude based on sensor data using a learned spike neural network, and to control the drone to fly while maintaining the attitude by adjusting the motor control signal in the specific situation.
- the drone can be stably controlled with low power and low delay in specific situations such as a gust of wind.
- Figure 1 is a front view of a drone according to one embodiment.
- Figure 2 is a configuration diagram of the drone of Figure 1.
- Figure 3 is a schematic diagram of a portion of a spike neural network included in the drone control unit of Figure 2.
- Figure 4 is a configuration diagram of a drone control unit of a drone according to another embodiment.
- Figure 5 is a configuration diagram of a drone control unit of a drone according to another embodiment.
- Figure 6 is a conceptual diagram of the flight control transfer process of the conventional control microprocessor and the gust situation analog computer of Figure 5.
- Figure 7 illustrates input/output signals of the analog computer of Figure 6.
- Figure 8 is a flow chart of a drone control method according to another embodiment.
- FIG. 9 illustrates in detail the drone control processes of the second stage of Figure 8.
- first, second, A, B, (a), (b), etc. may be used. These terms are only intended to distinguish the components from other components, and the nature, order, sequence, or number of the components are not limited by the terms.
- temporal chronological relationship or the chronological flow relationship when the temporal chronological relationship or the chronological flow relationship is described as “after”, “following”, “next to”, or “before”, it can also include cases where it is not continuous, as long as “immediately” or “directly” is not used.
- the numerical value or its corresponding information may be interpreted as including an error range that may occur due to various factors (e.g., process factors, internal or external impact, noise, etc.).
- Figure 1 is a front view of a drone according to one embodiment.
- a drone (100) can fly using a propeller (110) and an electric motor (120).
- the electric motor (120) is used to convert electric power into electrical energy to rotate the propeller (110). This rotational motion pushes out air, creating a force that propels the drone (100) upward.
- the propeller (110) helps to move the air using this rotational motion, and the drone can fly using this principle.
- Drones (100) are equipped with various sensors, cameras, GPS, communication systems, etc. and are used for flight control and data collection.
- the drone (100) has multiple (e.g., four in FIG. 1) propellers (110), and each propeller (110) can be controlled to perform ascent, descent, forward, backward, left and right movement, rotation, etc. To this end, the direction and height of the drone (100) are controlled by controlling the rotation speed of each propeller (110).
- the drone (100) has a built-in system for controlling flight. This system controls the speed of each electric motor (120) and adjusts the attitude to move in the desired direction according to the command entered by the user.
- a drone (100) stably controls attitude and speed without drifting, tilting, crashing or losing control with low power and low latency in specific situations such as gusts of wind.
- Figure 2 is a configuration diagram of the drone of Figure 1.
- a drone (100) includes a drone sensor unit (130) that provides sensor data, a drone flight unit (140) that provides a motor signal, and a drone control unit (150) that converts a motor signal into a motor control signal in a general situation and controls the drone to fly along a specific flight trajectory, measures attitude based on the sensor data using a learned spike neural network (SNN) to recognize a specific situation, and controls the drone to fly while maintaining attitude by adjusting the motor control signal in a specific situation.
- SNN learned spike neural network
- a specific situation is exemplified by a gust situation as described above, but is not limited thereto, and generally includes all cases other than a general situation in which a drone (100) may drift or tilt, crash, or lose control. That is, this specification defines two situations in which a drone (100) may perform a general control operation in a first situation such as a general situation, and perform a special control operation in a second situation such as a specific situation. As described below, the drone (100) may perform a software control operation or a hardware control operation in the general situation and the special situation.
- the drone sensor unit (130) includes a gyro sensor (132) and an acceleration sensor (134), and the sensor data may be, but is not limited to, angular velocity sensed by the gyro sensor (132) and acceleration sensed by the acceleration sensor (134).
- the gyro sensor (132) can measure angular velocity.
- An example of the gyro sensor (132) may be a gyroscope that measures angular velocity.
- the acceleration sensor (134) is a sensor that measures acceleration.
- the acceleration sensor (132) can measure acceleration in the x-axis, y-axis, and z-axis directions, for example.
- the drone flight unit (140) can convert the tuned motor control signal using a learned spike neural network (SNN) into a specific PWM signal that adjusts the motor speed and direction.
- SNN learned spike neural network
- the drone flight unit (140) converts the motor control signal generated from the spike neural network (SNN) into a corresponding PWM signal that adjusts the motor speed and direction.
- the control logic of the drone flight unit (140) compensates for wind disturbance and maintains the desired flight trajectory using aerodynamic and flight dynamics models.
- the drone flight unit (140) provides the converted PWM signal to the electric motor (120) and controls the electric motor (120), thereby inducing the flight of the drone (200).
- the drone control unit (150) includes a microprocessor (152), and the drone (100) may additionally include a memory (not shown).
- the memory stores various sensor data and various programs.
- the memory may be a volatile memory (e.g. SRAM, DRAM) or a nonvolatile memory (e.g. NAND Flash).
- the microprocessor (152) can control the flight along a specific flight trajectory by converting a motor signal into a motor control signal in a general situation.
- the microprocessor (152) can measure the attitude based on sensor data using a learned spike neural network (SNN) to recognize a specific situation, and control the flight while maintaining the attitude by adjusting the motor control signal in a specific situation.
- SNN learned spike neural network
- a microprocessor (152) included in a drone control unit (150) may control the drone to fly along a specific flight trajectory by converting a motor signal into a motor control signal in a general situation, store a learned spike neural network (SNN) in a memory, and execute it in the microprocessor (152) to measure attitude based on sensor data using the spike neural network (SNN), recognize a specific situation, and control the drone to fly while maintaining attitude by adjusting a motor control signal in a specific situation.
- SNN learned spike neural network
- the drone control unit (150) may include separate hardware that measures attitude based on sensor data using a learned spike neural network (SNN) to recognize a specific situation, and controls flight while maintaining attitude by adjusting motor control signals in a specific situation.
- the separate hardware may be implemented as an analog circuit or a semiconductor chip, and may be a digital circuit or an analog circuit. In this regard, it will be described later with reference to FIGS. 4 and 5.
- a neural network is an artificial intelligence technique that imitates the neural network of the human brain and is applied in various industries such as data mining, language recognition, image processing, and signal processing.
- convolutional neural network techniques were widely used, but since image data must be extracted, compared, and all data must be stored, it requires a large amount of memory and high power consumption, making it difficult to apply to small systems. Therefore, a spiking neural network (SNN), which is similar to the human brain's learning and information processing method and uses a small amount of data, has emerged.
- SNN spiking neural network
- Figure 3 is a schematic diagram of a portion of a spike neural network included in the drone control unit of Figure 2.
- the spike neural network (SNN) included in the drone control unit (150) includes a plurality of layers (Layers, 210), and each layer (210) contains a plurality of neurons.
- the layer (210) represents a group of a plurality of neurons having the same directionality.
- Neurons existing in different layers (210) are connected and signals are transmitted by synapses (230).
- the neuron before a synapse (230) is a presynaptic neuron (220)
- the neuron after the synapse (230) that is connected to the presynaptic neuron (220) through the synapse (230) is a post-synaptic neuron (250).
- a signal generated from a presynaptic neuron (220) and transmitted to a post-synaptic neuron (250) through the synapse (230) is called an input spike (240).
- An output spike (260) is a signal generated from a post-synaptic neuron (250).
- SNN Spiking Neural Network
- the post-synaptic neuron (250) multiplies each spike input from the pre-synaptic neuron (220) by a synaptic weight, and when the sum of all these values exceeds a certain threshold, it fires to the next layer (210).
- STDP Sespiking Timing Dependent Plasticity
- each synapse (230) has a synapse weight.
- the synapse weight is an arbitrary value that is multiplied by the input spike (240) to amplify or attenuate the input spike (240).
- the input spikes (240) multiplied by the synapse weight are added and compared with the neuron threshold of the output to control the output. Therefore, an optimized synapse weight design for controlling the output is required.
- STDP Spiking Timing Dependent Plasticity
- STDP is a function that calculates the amount of change in the synapse weight by using the difference between the time when the input spike (240) occurs and the time when the output spike (260) occurs.
- the weight of the synapse is adjusted by the difference between the occurrence time of the input spike (240) in the post-synaptic neuron (250) and the occurrence time of the output spike (260) in the pre-synaptic neuron (220).
- the drone control unit (150) applies an algorithm for learning and classifying a spike neural network (SNN).
- the artificial neuron analyzes the attitude information of the drone (100) through the generation of a small number of spikes at multiple points simultaneously.
- the spike neural network (SNN) analyzes the attitude information of the drone (100) and generates a motor control signal.
- a control signal can be generated based on the attitude information of the drone (100) analyzed by the spike neural network (SNN). Through this, the spike neural network (SNN) can quickly control the attitude of the drone (100).
- a drone (100) solves this problem by using a spiking neural network (SNN) to adjust motor control signals based on sensor data, compensate for gust effects, and maintain a desired flight trajectory.
- SNN spiking neural network
- Spike neural networks have the advantage of low power and low latency compared to traditional networks such as deep learning and recurrent neural networks in terms of spatiotemporal information processing. These advantages include low power consumption, high parallel processing, and real-time operation. This is very important in drone control systems where low latency and low power consumption are essential for long-term and safe operation.
- a drone (100) provides a spike neural network-based drone optimized for low power and low latency operation in specific situations, such as gust conditions, for example.
- the drone (100) according to one embodiment can provide efficient and powerful control while minimizing power consumption by integrating sensor data, spike neural network, and motor control system as subsystems.
- the drone (100) according to one embodiment can optimize performance in various weather conditions through the adaptability and learning mechanism of the spike neural network.
- Figure 4 is a configuration diagram of a drone control unit of a drone according to another embodiment.
- a drone control unit (150) may include a first microprocessor (154) that converts a motor signal into a motor control signal in a general situation and controls the drone to fly along a specific flight trajectory, and a second microprocessor (156) that measures attitude based on sensor data using a learned spike neural network (SNN) to recognize a specific situation, and controls the drone to maintain the flight trajectory by adjusting the motor control signal in the specific situation.
- a first microprocessor that converts a motor signal into a motor control signal in a general situation and controls the drone to fly along a specific flight trajectory
- a second microprocessor (156) that measures attitude based on sensor data using a learned spike neural network (SNN) to recognize a specific situation, and controls the drone to maintain the flight trajectory by adjusting the motor control signal in the specific situation.
- SNN learned spike neural network
- the first microprocessor (154) may be a general microprocessor, and the second microprocessor (156) may be a dedicated microprocessor implementing a spike neural network (SNN).
- the second microprocessor (156) may be a gust situation control analog computer (158) that processes analog signals as described with reference to FIG. 5, but may also be a digital microprocessor that processes digital signals in the same manner as the first microprocessor (154).
- Figure 5 is a configuration diagram of a drone control unit of a drone according to another embodiment.
- a drone control unit (150) may include a general control microprocessor (157) that converts a motor signal into a motor control signal in a general situation and controls the drone to fly along a specific flight trajectory, and a gust situation control analog computer (158) that measures attitude based on sensor data using a learned spike neural network (SNN) to recognize a specific situation, and controls the drone to fly while maintaining attitude by adjusting the motor control signal in a specific situation.
- a general control microprocessor that converts a motor signal into a motor control signal in a general situation and controls the drone to fly along a specific flight trajectory
- a gust situation control analog computer 158) that measures attitude based on sensor data using a learned spike neural network (SNN) to recognize a specific situation, and controls the drone to fly while maintaining attitude by adjusting the motor control signal in a specific situation.
- SNN learned spike neural network
- control microprocessor (157) may be implemented as a von Neumann computer-based microprocessor system capable of performing flight and desired tasks of the drone (100) in general situations or normal situations rather than specific situations such as gusts of wind.
- the drone control unit (150) recognizes a specific situation, such as a gust of wind, and transfers the drone's flight control to the SNN-based gust of wind situation control analog computer (158) based on the recognition.
- the spike neural network (SNN) utilized in the gust situation control analog computer (158) is an artificial neural network that imitates the operating principles of biological neurons, and the spike neural network (SNN) is used to detect and analyze the attitude of the drone (100) and generate a control signal.
- the gust situation control analog computer (158) can implement a drone attitude control system with low power and low latency by applying the spike neural network (SNN) capable of processing analog signals.
- the analog computer (158) for controlling the gust situation applies an algorithm for learning and classifying a spike neural network (SNN).
- the artificial neuron analyzes the attitude information of the drone (100) through the occurrence of a small number of spikes at multiple points simultaneously.
- the spike neural network (SNN) analyzes the attitude information of the drone (100) and generates a motor control signal.
- a device is required to generate a control signal based on the attitude information of the drone (100) analyzed by the spike neural network (SNN). Through this, the spike neural network (SNN) can quickly control the attitude of the drone (100).
- a drone (100B) utilizes a spike neural network (SNN) to quickly control and maintain the attitude of the aircraft in a gust of wind, and is designed using an analog computer-based integrated circuit for this purpose.
- SNN spike neural network
- the attitude of the drone is quickly detected and analyzed through a spike neural network (SNN) to immediately generate a control signal, and a spike neural network (SNN) structure with low power and low latency is used for this purpose.
- the drone (100B) according to another embodiment described above can quickly and stably control the attitude of the drone (100B) in a gust of wind, thereby enabling safe flight.
- Figure 6 is a conceptual diagram of the flight control transfer process of the conventional control microprocessor and the gust situation analog computer of Figure 5.
- the drone control unit (150) may transfer control of the aircraft to a gust of wind situation control analog computer (158), and when the gust of wind situation is over, the control of the aircraft may be transferred to a normal control microprocessor (157).
- the control of the aircraft is in the gust situation control analog computer (158)
- the determination of the degree of stabilization of the aircraft of the drone (100) is made through a spike neural network (SNN), and when the spike neural network (SNN) recognizes the end of the gust situation, the control is transferred to the existing conventional control microprocessor (157).
- SNN spike neural network
- a drone (100B) can more efficiently implement a spike neural network (SNN) using an analog computer-based integrated circuit.
- SNN spike neural network
- This is also called a neural processing unit or neuromorphic chip, and can more efficiently implement a drone attitude control system.
- Figure 7 illustrates the input/output signals of the gust situation control analog computer of Figure 6.
- the gust situation control analog computer (158) can receive a motor signal and sensor data, and output a gust situation judgment signal and a motor control signal that judge a gust situation using a learned spike neural network (SNN).
- SNN learned spike neural network
- the gust situation control analog computer receives motor signals and sensor data, measures attitude based on the sensor data using a learned spike neural network (SNN), recognizes a specific situation, and outputs a gust situation judgment signal and a motor control signal in a specific situation to control flight while maintaining attitude.
- SNN learned spike neural network
- the drone (100, 100A, 100B) can be recognized as superior to the conventional drone attitude control system. This is because it can quickly detect and analyze the surrounding environment and conditions, such as the attitude of the drone, the degree of influence due to wind, and prediction of future wind strength, using a spike neural network (SNN), and immediately generate a control signal while having low power and low latency, thereby showing more efficient performance than the conventional system.
- SNN spike neural network
- the drone (100, 100A, 100B) according to the above-described embodiments has the advantage of being able to quickly and stably control attitude in a gust of wind situation, thereby enabling safe flight. Therefore, the drone (100, 100A, 100B) according to the above-described embodiments provides a high level of safety, stability, and performance in the drone industry.
- Figure 8 is a flow chart of a drone control method according to another embodiment.
- a drone control method (300) includes a first step (S310) of controlling the drone to fly along a specific flight trajectory by converting a motor signal into a motor control signal in a general situation, and a second step (S320) of measuring attitude based on sensor data using a learned spike neural network to recognize a specific situation, and controlling the drone to fly while maintaining attitude by adjusting the motor control signal in the specific situation.
- the learned spike neural network can be used to convert the adjusted motor control signal into a specific PWM signal that adjusts the motor speed and direction.
- the sensor data may be, but is not limited to, angular velocity sensed by a gyro sensor and acceleration sensed by an acceleration sensor.
- the spike neural network includes a plurality of layers (210), and each layer (210) includes a plurality of neurons, and based on an arbitrary layer (210), includes a pre-synaptic neuron (220) and a post-synaptic neuron (250), and a synapse (230) connecting the pre-synaptic neuron (220) and the post-synaptic neuron (250), and the weight of the synapse can be adjusted based on the difference in the occurrence time of an output spike (260) in the post-synaptic neuron (250) and the occurrence time of an input spike (240) in the pre-synaptic neuron (220).
- the first microprocessor (154) illustrated in FIG. 4 converts the motor signal into a motor control signal in a general situation and controls the flight to a specific flight trajectory
- the second microprocessor (156) illustrated in FIG. 4 measures the attitude based on sensor data using a learned spike neural network (SNN) to recognize a specific situation, and controls the flight to maintain the attitude by adjusting the motor control signal in the specific situation.
- SNN learned spike neural network
- the conventional control microprocessor (157) illustrated in FIG. 5 converts a motor signal into a motor control signal in a general situation and controls the flight to a specific flight trajectory
- the gust situation control analog computer (158) illustrated in FIG. 5 measures the attitude based on sensor data using a learned spike neural network to recognize a specific situation, and controls the flight to maintain the attitude by adjusting the motor control signal in the specific situation.
- control of the aircraft may be transferred to a gust situation control analog computer (158), and if the gust situation has ended, control of the aircraft may be transferred to a normal control microprocessor (157).
- the gust situation control analog computer (158) can receive motor signals and sensor data, and output a gust situation judgment signal and a motor control signal that judge a gust situation using a learned spike neural network.
- FIG. 9 illustrates in detail the drone control processes of the second stage of Figure 8.
- the drone control right is transferred in a gust situation based on a motor signal (PWM) (S321), the input motor signal (PWM) and sensing data on the current state of the aircraft are input (S322), spike embedding and a spike neural network for the sensing data are input (S323), a PWM control signal suitable for each electric motor (220) is generated based on the input information and input to the electric motor (220) (S324), and after controlling the drone through the PWM control signal, the control right is terminated when the gust situation is resolved (S325).
- PWM motor signal
- control of the aircraft can be transferred to a gust situation control analog computer (158), and when the gust situation has ended, control of the aircraft can be transferred to a normal control microprocessor (157).
- the drone control method (300) can be recognized as superior to conventional drone attitude control systems. It can quickly detect and analyze the surrounding environment and conditions, such as the attitude of the drone, the degree of influence by the wind, and prediction of future wind strength, using a spike neural network (SNN), and generate a control signal immediately while having low power and low latency, so that it can exhibit more efficient performance than conventional systems.
- SNN spike neural network
- the drone control method (300) according to another embodiment described above has the advantage of being able to quickly and stably control attitude in a gust of wind situation, thereby enabling safe flight. Therefore, the drone control method (300) according to another embodiment described above provides a high level of safety, stability, and performance in the drone industry.
Landscapes
- Engineering & Computer Science (AREA)
- Aviation & Aerospace Engineering (AREA)
- Physics & Mathematics (AREA)
- Combustion & Propulsion (AREA)
- Chemical & Material Sciences (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Remote Sensing (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Automation & Control Theory (AREA)
- Radar, Positioning & Navigation (AREA)
- Microelectronics & Electronic Packaging (AREA)
- Mechanical Engineering (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Abstract
Description
본 실시예들은 특정 상황에서 기체를 제어하는 드론 및 그 드론의 제어방법에 관한 것이다. The present embodiments relate to a drone for controlling an aircraft in a specific situation and a method for controlling the drone.
드론 제어 시스템에서 자세와 속도는 드론의 안정성과 기동성을 결정하는 중요한 요소이다. In drone control systems, attitude and speed are important factors that determine the stability and maneuverability of the drone.
대부분의 드론에서 특정 상황, 예를 들어 돌풍의 영향을 받아 드론이 표류하거나 기울어 추락 또는 제어력 상실로 이어질 수 있는 위험성을 내포한다.Most drones pose a risk of drifting or tilting under certain conditions, such as when subjected to gusts of wind, which can lead to a crash or loss of control.
그러나, 일반적인 드론은 돌풍 상황과 같은 특정 상황에서 드론이 표류하거나 기울어 추락 또는 제어력을 상실하지 않도록 드론을 제어하는 방법이 없거나, 일반적인 제어 방법으로 돌풍 상황과 같은 특정 상황에서 드론을 제어할 경우 높은 전력 소비와 느린 연산 속도로 인해 비행 시간 및 안정성, 빠른 대응이 필요로 하는 돌풍 상황에서 적용되기에 적합하지 않았다.However, general drones do not have a way to control the drone so that it does not drift or tilt and crash or lose control in certain situations such as gusts, and when controlling the drone in certain situations such as gusts with a general control method, the high power consumption and slow computation speed make it unsuitable for application in gusts that require flight time, stability, and quick response.
본 실시예들은 돌풍 상황과 같은 특정 상황에서 저전력과 저지연으로 드론을 안정적으로 제어하는 드론 및 그 드론의 제어 방법을 제공할 수 있다.The present embodiments can provide a drone and a method for controlling the drone that stably controls the drone with low power and low latency in specific situations such as a gust of wind.
본 실시예들은 일반적인 상황에서 모터 신호를 모터 제어 신호로 변환하여 특정 비행 궤적으로 비행하도록 제어하고, 학습된 스파이크 신경망을 사용하여 센서 데이터를 기반으로 자세를 측정하여 특정 상황을 인지하고, 특정 상황에서 모터 제어 신호를 조정하여 자세를 유지하면서 비행하도록 제어하는 드론 및 그 드론의 제어방법을 제공한다.The present embodiments provide a drone and a control method for the drone, which controls the drone to fly along a specific flight trajectory by converting a motor signal into a motor control signal in a general situation, recognizes a specific situation by measuring attitude based on sensor data using a learned spike neural network, and controls the drone to fly while maintaining attitude by adjusting the motor control signal in a specific situation.
일 측면에서, 본 실시예들은 센서 데이터를 제공하는 드론 센서부, 모터 신호를 제공하는 드론 비행부 및 일반적인 상황에서 모터 신호를 상기 모터 제어 신호로 변환하여 특정 비행 궤적으로 비행하도록 제어하고, 학습된 스파이크 신경망을 사용하여 센서 데이터를 기반으로 자세를 측정하여 특정 상황을 인지하고, 특정 상황에서 모터 제어 신호를 조정하여 자세를 유지하면서 비행하도록 제어하는 드론 제어부를 포함하는 드론을 제공할 수 있다.In one aspect, the present embodiments may provide a drone including a drone sensor unit that provides sensor data, a drone flight unit that provides motor signals, and a drone control unit that converts the motor signals into the motor control signals in general situations to control the drone to fly along a specific flight trajectory, measures attitude based on the sensor data using a learned spike neural network to recognize a specific situation, and controls the drone to fly while maintaining attitude by adjusting the motor control signals in specific situations.
다른 측면에서, 본 실시예들은 일반적인 상황에서 모터 신호를 모터 제어 신호로 변환하여 특정 비행 궤적으로 비행하도록 제어하는 제1단계 및 학습된 스파이크 신경망을 사용하여 센서 데이터를 기반으로 자세를 측정하여 특정 상황을 인지하고, 특정 상황에서 모터 제어 신호를 조정하여 자세를 유지하면서 비행하도록 제어하는 제2단계를 포함하는 드론의 제어 방법을 제공할 수 있다.In another aspect, the present embodiments can provide a method for controlling a drone, including a first step of controlling the drone to fly along a specific flight trajectory by converting a motor signal into a motor control signal in a general situation, and a second step of controlling the drone to recognize a specific situation by measuring an attitude based on sensor data using a learned spike neural network, and to control the drone to fly while maintaining the attitude by adjusting the motor control signal in the specific situation.
본 실시예들에 따른 드론 및 그 드론의 제어 방법에 의하면, 돌풍 상황과 같은 특정 상황에서 저전력과 저지연으로 드론을 안정적으로 제어할 수 있다.According to the drone and the method for controlling the drone according to the present embodiments, the drone can be stably controlled with low power and low delay in specific situations such as a gust of wind.
도 1은 일 실시예에 따른 드론의 정면도이다. Figure 1 is a front view of a drone according to one embodiment.
도 2는 도 1의 드론의 구성도이다. Figure 2 is a configuration diagram of the drone of Figure 1.
도 3은 도 2의 드론 제어부에 포함되는 스파이크 신경망의 일부 개략도이다.Figure 3 is a schematic diagram of a portion of a spike neural network included in the drone control unit of Figure 2.
도 4는 다른 실시예에 따른 드론의 드론 제어부의 구성도이다.Figure 4 is a configuration diagram of a drone control unit of a drone according to another embodiment.
도 5는 또 다른 실시예에 따른 드론의 드론 제어부의 구성도이다.Figure 5 is a configuration diagram of a drone control unit of a drone according to another embodiment.
도 6은 도 5의 통상 제어 마이크로 프로세서와 돌풍 상황 아날로그 컴퓨터의 비행 제어권 천이 과정의 개념도이다. Figure 6 is a conceptual diagram of the flight control transfer process of the conventional control microprocessor and the gust situation analog computer of Figure 5.
도 7은 도 6의 아날로그 컴퓨터의 입출력 신호들을 도시하고 있다. Figure 7 illustrates input/output signals of the analog computer of Figure 6.
도 8은 또 다른 실시예에 따른 드론 제어방법의 흐름도이다.Figure 8 is a flow chart of a drone control method according to another embodiment.
도 9는 도 8의 제2단계의 드론 제어 과정들을 상세히 도시하고 있다.Figure 9 illustrates in detail the drone control processes of the second stage of Figure 8.
이하, 본 개시의 일부 실시예들을 예시적인 도면을 참조하여 상세하게 설명한다. 각 도면의 구성 요소들에 참조부호를 부가함에 있어서, 동일한 구성 요소들에 대해서는 비록 다른 도면상에 표시되더라도 가능한 한 동일한 부호를 가질 수 있다. 또한, 본 실시예들을 설명함에 있어, 관련된 공지 구성 또는 기능에 대한 구체적인 설명이 본 기술 사상의 요지를 흐릴 수 있다고 판단되는 경우에는 그 상세한 설명은 생략할 수 있다. 본 명세서 상에서 언급된 "포함한다", "갖는다", "이루어진다" 등이 사용되는 경우 "~만"이 사용되지 않는 이상 다른 부분이 추가될 수 있다. 구성 요소를 단수로 표현한 경우에 특별한 명시적인 기재 사항이 없는 한 복수를 포함하는 경우를 포함할 수 있다.Hereinafter, some embodiments of the present disclosure will be described in detail with reference to exemplary drawings. When adding reference numerals to components of each drawing, the same components may have the same numerals as much as possible even if they are shown in different drawings. In addition, when describing the present embodiments, if it is determined that a specific description of a related known configuration or function may obscure the gist of the technical idea of the present invention, the detailed description thereof may be omitted. When “includes,” “has,” “consists of,” etc. are used in this specification, other parts may be added unless “only” is used. When a component is expressed in the singular, it may include a case where the plural is included unless there is a special explicit description.
또한, 본 개시의 구성 요소를 설명하는 데 있어서, 제1, 제2, A, B, (a), (b) 등의 용어를 사용할 수 있다. 이러한 용어는 그 구성 요소를 다른 구성 요소와 구별하기 위한 것일 뿐, 그 용어에 의해 해당 구성 요소의 본질, 차례, 순서 또는 개수 등이 한정되지 않는다. Additionally, in describing components of the present disclosure, terms such as first, second, A, B, (a), (b), etc. may be used. These terms are only intended to distinguish the components from other components, and the nature, order, sequence, or number of the components are not limited by the terms.
구성 요소들의 위치 관계에 대한 설명에 있어서, 둘 이상의 구성 요소가 "연결", "결합" 또는 "접속" 등이 된다고 기재된 경우, 둘 이상의 구성 요소가 직접적으로 "연결", "결합" 또는 "접속" 될 수 있지만, 둘 이상의 구성 요소와 다른 구성 요소가 더 "개재"되어 "연결", "결합" 또는 "접속"될 수도 있다고 이해되어야 할 것이다. 여기서, 다른 구성 요소는 서로 "연결", "결합" 또는 "접속" 되는 둘 이상의 구성 요소 중 하나 이상에 포함될 수도 있다. In a description of the positional relationship of components, when it is described that two or more components are "connected", "coupled" or "connected", it should be understood that the two or more components may be directly "connected", "coupled" or "connected", but the two or more components and another component may be further "interposed" to be "connected", "coupled" or "connected". Here, the other component may be included in one or more of the two or more components that are "connected", "coupled" or "connected" to each other.
구성 요소들이나, 동작 방법이나 제작 방법 등과 관련한 시간적 흐름 관계에 대한 설명에 있어서, 예를 들어, "~후에", "~에 이어서", "~다음에", "~전에" 등으로 시간적 선후 관계 또는 흐름적 선후 관계가 설명되는 경우, "바로" 또는 "직접"이 사용되지 않는 이상 연속적이지 않은 경우도 포함할 수 있다.In the description of the temporal flow relationship related to components, operation methods, or manufacturing methods, for example, when the temporal chronological relationship or the chronological flow relationship is described as "after", "following", "next to", or "before", it can also include cases where it is not continuous, as long as "immediately" or "directly" is not used.
한편, 구성 요소에 대한 수치 또는 그 대응 정보(예: 레벨 등)가 언급된 경우, 별도의 명시적 기재가 없더라도, 수치 또는 그 대응 정보는 각종 요인(예: 공정상의 요인, 내부 또는 외부 충격, 노이즈 등)에 의해 발생할 수 있는 오차 범위를 포함하는 것으로 해석될 수 있다.Meanwhile, when a numerical value or its corresponding information (e.g., level, etc.) for a component is mentioned, even if there is no separate explicit description, the numerical value or its corresponding information may be interpreted as including an error range that may occur due to various factors (e.g., process factors, internal or external impact, noise, etc.).
이하 도면을 참조하여 실시예들을 상세히 설명한다. The embodiments are described in detail with reference to the drawings below.
도 1은 일 실시예에 따른 드론의 정면도이다 Figure 1 is a front view of a drone according to one embodiment.
도 1을 참조하면, 일 실시예에 따른 드론(100)은 프로펠러(110)와 전기모터(120)를 이용하여 비행할 수 있다. 전기모터(120)는 전력을 전기적 에너지로 변환하여 프로펠러(110)를 회전시키는데 사용된다. 이 회전운동은 공기를 밀어내어 드론(100)을 위쪽으로 추진하는 힘을 만들어낸다. 프로펠러(110)는 이 회전운동을 이용해 공기를 이동시키는데 도움을 주는데, 이런 원리를 이용해 드론이 비행할 수 있다. Referring to FIG. 1, a drone (100) according to one embodiment can fly using a propeller (110) and an electric motor (120). The electric motor (120) is used to convert electric power into electrical energy to rotate the propeller (110). This rotational motion pushes out air, creating a force that propels the drone (100) upward. The propeller (110) helps to move the air using this rotational motion, and the drone can fly using this principle.
드론(100)은 다양한 센서와 카메라, GPS, 통신 시스템 등을 탑재하여 비행 제어와 데이터 수집에 사용된다. Drones (100) are equipped with various sensors, cameras, GPS, communication systems, etc. and are used for flight control and data collection.
드론(100)은 여러 개(예를 들어 도 1의 4개)의 프로펠러들(110)를 가지고 있어 각각의 프로펠러(110)를 제어하여 상승, 하강, 전진, 후진, 좌우 이동, 회전 등을 수행할 수 있다. 이를 위해서는 각 프로펠러(110)의 회전 속도를 조절하여 드론(100)의 방향과 높이를 조절한다. The drone (100) has multiple (e.g., four in FIG. 1) propellers (110), and each propeller (110) can be controlled to perform ascent, descent, forward, backward, left and right movement, rotation, etc. To this end, the direction and height of the drone (100) are controlled by controlling the rotation speed of each propeller (110).
드론(100)에는 비행을 제어하기 위한 시스템이 내장되어 있다. 이 시스템은 사용자가 입력한 명령에 따라 각 전기모터(120)의 속도를 조절하고, 자세를 조절하여 원하는 방향으로 이동하도록 한다. The drone (100) has a built-in system for controlling flight. This system controls the speed of each electric motor (120) and adjusts the attitude to move in the desired direction according to the command entered by the user.
일 실시예에 따른 드론(100)은 돌풍 상황과 같은 특정 상황에서 저전력과 저지연으로 표류하거나 기울어 추락 또는 제어력을 상실하지 않고 자세와 속도를 안정적으로 제어한다. A drone (100) according to one embodiment stably controls attitude and speed without drifting, tilting, crashing or losing control with low power and low latency in specific situations such as gusts of wind.
도 2는 도 1의 드론의 구성도이다. Figure 2 is a configuration diagram of the drone of Figure 1.
도 2를 참조하면, 일 실시예에 따른 드론(100)은 센서 데이터를 제공하는 드론 센서부(130), 모터 신호를 제공하는 드론 비행부(140) 및 일반적인 상황에서 모터 신호를 모터 제어 신호로 변환하여 특정 비행 궤적으로 비행하도록 제어하고, 학습된 스파이크 신경망(Spike neural network(SNN))을 사용하여 센서 데이터를 기반으로 자세를 측정하여 특정 상황을 인지하고, 특정 상황에서 모터 제어 신호를 조정하여 자세를 유지하면서 비행하도록 제어하는 드론 제어부(150)를 포함한다.Referring to FIG. 2, a drone (100) according to one embodiment includes a drone sensor unit (130) that provides sensor data, a drone flight unit (140) that provides a motor signal, and a drone control unit (150) that converts a motor signal into a motor control signal in a general situation and controls the drone to fly along a specific flight trajectory, measures attitude based on the sensor data using a learned spike neural network (SNN) to recognize a specific situation, and controls the drone to fly while maintaining attitude by adjusting the motor control signal in a specific situation.
이하에서 특정 상황은 전술한 바와 같이 돌풍 상황을 예시적으로 설명하나, 이에 제한되지 않고 일반적으로 드론(100)이 표류하거나 기울어 추락 또는 제어력을 상실할 수 있는 일반적 상황이 아닌 모든 경우를 포함한다. 즉, 본 명세서는 드론(10)이 두개의 상황들을 정의하고, 일반적 상황과 같은 제1상황에서 일반적인 제어 동작을 수행하다 특정 상황과 같은 제2상황에서 특수한 제어 동작을 수행할 수 있다. 후술하는 바와 같이 드론(100)은 일반적 상황과 특수 상황에서 소프트웨어적인 제어 동작을 수행할 수도 있고, 하드웨어적인 제어 동작을 수행할 수도 있다. Hereinafter, a specific situation is exemplified by a gust situation as described above, but is not limited thereto, and generally includes all cases other than a general situation in which a drone (100) may drift or tilt, crash, or lose control. That is, this specification defines two situations in which a drone (100) may perform a general control operation in a first situation such as a general situation, and perform a special control operation in a second situation such as a specific situation. As described below, the drone (100) may perform a software control operation or a hardware control operation in the general situation and the special situation.
드론 센서부(130)는 자이로 센서(132)와 가속도 센서(134)를 포함하고, 센서 데이터는 자이로 센서(132)에서 센싱된 각속도와 가속도 센서(134)에서 센싱한 가속도일 수 있으나, 이에 제한되지 않는다. The drone sensor unit (130) includes a gyro sensor (132) and an acceleration sensor (134), and the sensor data may be, but is not limited to, angular velocity sensed by the gyro sensor (132) and acceleration sensed by the acceleration sensor (134).
자이로 센서(132)는 각속도를 측정할 수 있다. 자이로센서(132)의 일예로 각속도를 측정하는 자이로스코프일 수 있다. 가속도 센서(134)는 가속도를 측정하는 센서이다. 가속도 센서(132)는 예를 들어 x축과 y축, z축 방향의 가속도를 측정할 수 있다. The gyro sensor (132) can measure angular velocity. An example of the gyro sensor (132) may be a gyroscope that measures angular velocity. The acceleration sensor (134) is a sensor that measures acceleration. The acceleration sensor (132) can measure acceleration in the x-axis, y-axis, and z-axis directions, for example.
드론 비행부(140)는 학습된 스파이크 신경망(SNN)을 사용하여 조정된 모터 제어 신호를 모터 속도와 방향을 조정하는 특정 PWM 신호로 변환할 수 있다. The drone flight unit (140) can convert the tuned motor control signal using a learned spike neural network (SNN) into a specific PWM signal that adjusts the motor speed and direction.
드론 비행부(140)는 스파이크 신경망(SNN)에서 생성된 모터 제어 신호를 모터 속도와 방향을 조정하는 해당 PWM 신호로 변환한다. 드론 비행부(140)의 제어 로직은 공기역학 및 비행 역학 모델을 사용하여 바람의 방해를 보상하고 원하는 비행 궤적을 유지한다.The drone flight unit (140) converts the motor control signal generated from the spike neural network (SNN) into a corresponding PWM signal that adjusts the motor speed and direction. The control logic of the drone flight unit (140) compensates for wind disturbance and maintains the desired flight trajectory using aerodynamic and flight dynamics models.
드론 비행부(140)는 변환된 PWM 신호를 전기 모터(120)에 제공하고 전기 모터(120)를 제어하므로 드론(200)의 비행을 유도할 수 있다.The drone flight unit (140) provides the converted PWM signal to the electric motor (120) and controls the electric motor (120), thereby inducing the flight of the drone (200).
드론 제어부(150)는 마이크로 프로세서(152)를 포함하고 있고, 드론(100)은 메모리(미도시)를 추가로 포함할 수 있다. 메모리는 각종 센서 데이터와 각종 프로그램을 저장하고 있다. 메모리는 휘발성 메모리(e.g. SRAM, DRAM) 또는 비휘발성 메모리(e.g. NAND Flash)일 수 있다.The drone control unit (150) includes a microprocessor (152), and the drone (100) may additionally include a memory (not shown). The memory stores various sensor data and various programs. The memory may be a volatile memory (e.g. SRAM, DRAM) or a nonvolatile memory (e.g. NAND Flash).
일예로, 마이크로 프로세서(152)는 일반적인 상황에서 모터 신호를 모터 제어 신호로 변환하여 특정 비행 궤적으로 비행하도록 제어할 수 있다. 또한, 마이크로 프로세서(152)는 학습된 스파이크 신경망(Spike neural network(SNN))을 사용하여 센서 데이터를 기반으로 자세를 측정하여 특정 상황을 인지하고, 특정 상황에서 모터 제어 신호를 조정하여 자세를 유지하면서 비행하도록 제어할 수 있다. For example, the microprocessor (152) can control the flight along a specific flight trajectory by converting a motor signal into a motor control signal in a general situation. In addition, the microprocessor (152) can measure the attitude based on sensor data using a learned spike neural network (SNN) to recognize a specific situation, and control the flight while maintaining the attitude by adjusting the motor control signal in a specific situation.
다른 예로, 드론 제어부(150)에 포함된 마이크로 프로세서(152)가 일반적인 상황에서 모터 신호를 모터 제어 신호로 변환하여 특정 비행 궤적으로 비행하도록 제어하고, 학습된 스파이크 신경망(SNN)을 메모리에 저장하였다가 마이크로 프로세서(152)에서 실행하여 스파이크 신경망(SNN) 을 사용하여 센서 데이터를 기반으로 자세를 측정하여 특정 상황을 인지하고, 특정 상황에서 모터 제어 신호를 조정하여 자세를 유지하면서 비행하도록 제어할 수 있다. As another example, a microprocessor (152) included in a drone control unit (150) may control the drone to fly along a specific flight trajectory by converting a motor signal into a motor control signal in a general situation, store a learned spike neural network (SNN) in a memory, and execute it in the microprocessor (152) to measure attitude based on sensor data using the spike neural network (SNN), recognize a specific situation, and control the drone to fly while maintaining attitude by adjusting a motor control signal in a specific situation.
또 다른 예로 드론 제어부(150)는 학습된 스파이크 신경망(SNN)을 사용하여 센서 데이터를 기반으로 자세를 측정하여 특정 상황을 인지하고, 특정 상황에서 모터 제어 신호를 조정하여 자세를 유지하면서 비행하도록 제어하는 별도의 하드웨어를 포함할 수도 있다. 별도의 하드웨어는 아날로그 회로이거나 반도체 칩으로 구현될 수 있고, 디지털회로이거나 아날로그회로일 수 있다. 이와 관련하여 도 4 및 도 5를 참조하여 후술한다. As another example, the drone control unit (150) may include separate hardware that measures attitude based on sensor data using a learned spike neural network (SNN) to recognize a specific situation, and controls flight while maintaining attitude by adjusting motor control signals in a specific situation. The separate hardware may be implemented as an analog circuit or a semiconductor chip, and may be a digital circuit or an analog circuit. In this regard, it will be described later with reference to FIGS. 4 and 5.
신경망(neural network)이란 인간 두뇌의 신경망을 모방하여 데이터 마이닝, 언어 인식, 이미지 처리, 신호처리 등과 같은 여러 산업에서 적용하는 인공지능 기법이다. 종래에, 회선 신경망(Convolution neural network) 기법을 많이 사용하였으나, 이는 이미지 데이터를 추출하고 비교 후 데이터를 모두 저장하여야 하기 때문에 대용량의 메모리와 높은 소모 전력을 요구하여 소형 시스템에 적용이 어렵다. 이에 인간의 두뇌 학습 및 정보처리 방식과 유사하고 적은 양의 데이터를 사용하는 스파이크 신경망(SNN)이 대두되었다.A neural network is an artificial intelligence technique that imitates the neural network of the human brain and is applied in various industries such as data mining, language recognition, image processing, and signal processing. In the past, convolutional neural network techniques were widely used, but since image data must be extracted, compared, and all data must be stored, it requires a large amount of memory and high power consumption, making it difficult to apply to small systems. Therefore, a spiking neural network (SNN), which is similar to the human brain's learning and information processing method and uses a small amount of data, has emerged.
도 3는 도 2의 드론 제어부에 포함되는 스파이크 신경망의 일부 개략도이다.Figure 3 is a schematic diagram of a portion of a spike neural network included in the drone control unit of Figure 2.
도 3을 참조하면, 드론 제어부(150)에 포함되는 스파이크 신경망(SNN)은 다수의 레이어들(Layers, 210)을 포함하고, 각각의 레이어(210)에는 다수의 뉴런이 존재한다. 레이어(210)는 같은 방향성을 가진 다수의 뉴런의 집단을 나타낸다.Referring to FIG. 3, the spike neural network (SNN) included in the drone control unit (150) includes a plurality of layers (Layers, 210), and each layer (210) contains a plurality of neurons. The layer (210) represents a group of a plurality of neurons having the same directionality.
임의의 레이어를 기준으로 했을 때 프리 시냅틱 뉴런(Pre-Synaptic Neuron, 220)과 포스트 시냅틱 뉴런(Post-Synaptic Neuron, 250), 프리 시냅틱 뉴런(120)과 포스트 시냅틱 뉴런(250) 사이를 연결하는 시냅스(Synapse, 230)를 포함한다. Based on an arbitrary layer, it includes a pre-synaptic neuron (220) and a post-synaptic neuron (250), and a synapse (230) connecting the pre-synaptic neuron (120) and the post-synaptic neuron (250).
각각 다른 레이어(210)에 존재하는 뉴런을 연결하고 신호를 시냅스(Synapse, 230)가 전달한다. 뉴럴 네트워크에서는 시냅스(230)를 기준으로 이전의 뉴런이 프리 시냅틱 뉴런(220)이고, 시냅스(230)를 통해 프리 시냅틱 뉴런(220)과 연결되어 있는 시냅스(230) 이후의 뉴런이 포스트 시냅틱 뉴런(250)이다. 프리 시냅틱 뉴런(220)에서 발생하고 시냅스(230)를 통하여 포스트 시냅틱 뉴런(250)에 전달되는 신호를 입력 스파이크(240)라 한다. 출력 스파이크(260)는 포스트 시냅틱 뉴런(250)에서 발생하는 신호이다.Neurons existing in different layers (210) are connected and signals are transmitted by synapses (230). In a neural network, the neuron before a synapse (230) is a presynaptic neuron (220), and the neuron after the synapse (230) that is connected to the presynaptic neuron (220) through the synapse (230) is a post-synaptic neuron (250). A signal generated from a presynaptic neuron (220) and transmitted to a post-synaptic neuron (250) through the synapse (230) is called an input spike (240). An output spike (260) is a signal generated from a post-synaptic neuron (250).
SNN(Spiking Neural Network)은 입력 스파이크(240)의 합이 출력 뉴런의 발화 임계값 이상이 될 때만 출력 스파이크(260)가 발생하는 뉴럴 네트워크 방식이다.SNN (Spiking Neural Network) is a neural network method in which an output spike (260) occurs only when the sum of input spikes (240) exceeds the firing threshold of the output neuron.
즉, 포스트 시냅틱 뉴런(250)은, 프리 시냅틱 뉴런(220)으로부터 입력으로 들어오는 스파이크(Spike)에 각각 시냅스 가중치를 곱하고 이를 모두 더한 값이 특정 임계값을 넘게 되면 다음 레이어(210)로 발화한다.That is, the post-synaptic neuron (250) multiplies each spike input from the pre-synaptic neuron (220) by a synaptic weight, and when the sum of all these values exceeds a certain threshold, it fires to the next layer (210).
이에 특정 임계값을 넘을 수 있도록 최적화 가중치를 찾아야 하며, 가중치의 변화량을 조절하여 새로운 시냅스가중치를 구한 후 위와 같은 계산을 반복하여 출력을 제어한다.Therefore, we need to find the optimal weights so that they can exceed a certain threshold, and then adjust the amount of change in the weights to obtain new synaptic weights, and then repeat the above calculations to control the output.
SNN에서는 통상, STDP(Spiking Timing Dependent Plasticity)라는 비지도 학습 방식을 사용한다. STDP 학습법으로 시냅스 가중치를 최적화 하여 출력을 제어할 수 있다.In SNN, an unsupervised learning method called STDP (Spiking Timing Dependent Plasticity) is usually used. The STDP learning method can control the output by optimizing the synaptic weights.
SNN에서 입력 스파이크(240)가 시냅스(230)를 통하여 출력 뉴런(250)에 전달 될 때, 각 시냅스(230)에는 시냅스 가중치가 있다. 시냅스 가중치는 입력 스파이크 (240)와 곱해져 입력 스파이크(240)를 증폭 또는 감폭하는 임의의 값이다. 시냅스 가중치가 곱해진 입력 스파이크(240)들을 모두 더하여 출력의 뉴런 임계값과 비교하여 출력을 제어한다. 이에 출력을 제어하기 위한 최적화된 시냅스 가중치 설계가 필요하게 된다. STDP(Spiking Timing Dependent Plasticity)는 입력 스파이크(240)가 발생한 시간과 출력 스파이크(260)가 발생한 시간의 차를 이용하여 시냅스 가중치의 변화 량을 구하는 함수이다. In SNN, when an input spike (240) is transmitted to an output neuron (250) through a synapse (230), each synapse (230) has a synapse weight. The synapse weight is an arbitrary value that is multiplied by the input spike (240) to amplify or attenuate the input spike (240). The input spikes (240) multiplied by the synapse weight are added and compared with the neuron threshold of the output to control the output. Therefore, an optimized synapse weight design for controlling the output is required. STDP (Spiking Timing Dependent Plasticity) is a function that calculates the amount of change in the synapse weight by using the difference between the time when the input spike (240) occurs and the time when the output spike (260) occurs.
즉, 포스트 시냅틱 뉴런(250)에서의 입력 스파이크(240) 발생 시간과 프리 시냅틱 뉴런(220)에서의 출력 스파이크(260) 발생 시간 차이로 시냅스의 가중치를 조절한다. That is, the weight of the synapse is adjusted by the difference between the occurrence time of the input spike (240) in the post-synaptic neuron (250) and the occurrence time of the output spike (260) in the pre-synaptic neuron (220).
드론 제어부(150)는 스파이크 신경망(SNN)의 학습 및 분류를 위한 알고리즘을 적용한다. 주어진 데이터를 학습하고 분류하는 과정에서 인공 뉴런은 다수의 지점에서 동시에 소수의 스파이크 발생을 통해 드론(100)의 자세 정보를 분석한다. 이를 통해 스파이크 신경망(SNN)은 드론(100)의 자세 정보를 분석하여 모터 제어 신호를 생성한다.The drone control unit (150) applies an algorithm for learning and classifying a spike neural network (SNN). In the process of learning and classifying given data, the artificial neuron analyzes the attitude information of the drone (100) through the generation of a small number of spikes at multiple points simultaneously. Through this, the spike neural network (SNN) analyzes the attitude information of the drone (100) and generates a motor control signal.
스파이크 신경망(SNN)이 분석한 드론(100)의 자세 정보를 기반으로 제어 신호를 생성할 수 있다. 이를 통해 스파이크 신경망(SNN)은 드론(100)의 자세를 빠르게 제어할 수 있다.A control signal can be generated based on the attitude information of the drone (100) analyzed by the spike neural network (SNN). Through this, the spike neural network (SNN) can quickly control the attitude of the drone (100).
전술한 바와 같이 일반적인 드론은 돌풍 상황과 같은 특정 상황에서 드론이 표류하거나 기울어 추락 또는 제어력을 상실하지 않도록 드론을 제어하는 방법이 없거나, 일반적인 제어 방법으로 돌풍 상황과 같은 특정 상황에서 드론을 제어할 경우 높은 전력 소비와 느린 연산 속도로 인해 비행 시간 및 안정성, 빠른 대응이 필요로 하는 돌풍 상황에서 적용되기에 적합하지 않았다.As mentioned above, general drones do not have a way to control the drone in certain situations such as gusts to prevent it from drifting or tilting and crashing or losing control, and when controlling the drone in certain situations such as gusts with a general control method, the high power consumption and slow computation speed make it unsuitable for application in gusts that require flight time, stability, and quick response.
일 실시예에 따른 드론(100)은 스파이크 신경망(SNN)을 사용하여 센서 데이터를 기반으로 모터 제어 신호를 조정하고 돌풍 효과를 보상하고 원하는 비행 궤적을 유지함으로써 이 문제를 해결한다. A drone (100) according to one embodiment solves this problem by using a spiking neural network (SNN) to adjust motor control signals based on sensor data, compensate for gust effects, and maintain a desired flight trajectory.
스파이크 신경망(SNN)은 시공간 정보 처리와 관련하여 딥 러닝 및 반복 신경망과 같은 기존 네트워크에 비해 저전력, 저지연에 강점이 있다. 이러한 장점에는 낮은 전력 소비, 높은 병렬 처리 및 실시간 작동이 포함된다. 이는 낮은 대기 시간과 낮은 전력 소비가 장기적이고 안전한 작동에 필수적인 드론 제어 시스템에서 매우 중요하다.Spike neural networks (SNNs) have the advantage of low power and low latency compared to traditional networks such as deep learning and recurrent neural networks in terms of spatiotemporal information processing. These advantages include low power consumption, high parallel processing, and real-time operation. This is very important in drone control systems where low latency and low power consumption are essential for long-term and safe operation.
일 실시예에 따른 드론(100)은 돌풍 상황과 같은 특정 상황에서 예를 들어 돌풍 조건에서 저전력 및 저지연 작동에 최적화된 스파이크 신경망 기반 드론을 제공한다. 일 실시예에 따른 드론(100)은 센서 데이터, 스파이크 신경망 및 모터 제어 시스템을 하위 시스템을 통합하여 전력 소비를 최소화하면서 효율적이고 강력한 제어를 제공할 수 있다. 또한, 일 실시예에 따른 드론(100)은 스파이크 신경망의 적응성 및 학습 메커니즘을 통해 다양한 날씨 조건에서 성능을 최적화할 수 있다.A drone (100) according to one embodiment provides a spike neural network-based drone optimized for low power and low latency operation in specific situations, such as gust conditions, for example. The drone (100) according to one embodiment can provide efficient and powerful control while minimizing power consumption by integrating sensor data, spike neural network, and motor control system as subsystems. In addition, the drone (100) according to one embodiment can optimize performance in various weather conditions through the adaptability and learning mechanism of the spike neural network.
이하, 학습된 스파이크 신경망(Spike neural network(SNN))을 사용하여 센서 데이터를 기반으로 자세를 측정하여 특정 상황을 인지하고, 특정 상황에서 모터 제어 신호를 조정하여 자세를 유지하면서 비행하도록 제어하도록 별도의 하드웨어로 구현하는 예들을 도 4 및 도 5를 참조하여 설명한다. Hereinafter, examples of implementing separate hardware to measure attitude based on sensor data using a learned spike neural network (SNN) to recognize a specific situation, and control flight while maintaining attitude by adjusting motor control signals in a specific situation are described with reference to FIGS. 4 and 5.
도 4는 다른 실시예에 따른 드론의 드론 제어부의 구성도이다.Figure 4 is a configuration diagram of a drone control unit of a drone according to another embodiment.
도 4를 참조하면, 다른 실시예에 따른 드론(100A)에서 드론 제어부(150)는 일반적인 상황에서 모터 신호를 모터 제어 신호로 변환하여 특정 비행 궤적으로 비행하도록 제어하는 제1마이크로 프로세서(154) 및 학습된 스파이크 신경망(SNN)을 사용하여 센서 데이터를 기반으로 자세를 측정하여 특정 상황을 인지하고, 상기 특정 상황에서 모터 제어 신호를 조정하여 비행 궤적을 유지하도록 제어하는 제2마이크로 프로세서(156)를 포함할 수 있다. Referring to FIG. 4, in a drone (100A) according to another embodiment, a drone control unit (150) may include a first microprocessor (154) that converts a motor signal into a motor control signal in a general situation and controls the drone to fly along a specific flight trajectory, and a second microprocessor (156) that measures attitude based on sensor data using a learned spike neural network (SNN) to recognize a specific situation, and controls the drone to maintain the flight trajectory by adjusting the motor control signal in the specific situation.
제1마이크로 프로세서(154)는 일반적인 마이크로 프로세서이고, 제2마이크로 프로세서(156)는 스파이크 신경망(SNN)을 구현한 전용 마이크로 프로세서일 수 있다. 제2마이크로 프로세서(156)는 도 5를 참조하여 설명하는 바와 같이 아날로그 신호를 처리하는 돌풍상황 제어 아날로그 컴퓨터(158)일 수도 있으나, 제1마이크로 프로세서(154)와 동일하게 디지털 신호를 처리하는 디지털 마이크로 프로세서일 수도 있다. The first microprocessor (154) may be a general microprocessor, and the second microprocessor (156) may be a dedicated microprocessor implementing a spike neural network (SNN). The second microprocessor (156) may be a gust situation control analog computer (158) that processes analog signals as described with reference to FIG. 5, but may also be a digital microprocessor that processes digital signals in the same manner as the first microprocessor (154).
도 5은 또 다른 실시예에 따른 드론의 드론 제어부의 구성도이다.Figure 5 is a configuration diagram of a drone control unit of a drone according to another embodiment.
도 6을 참조하면, 또 다른 실시예에 따른 드론(100B)에서 드론 제어부(150)는 일반적인 상황에서 모터 신호를 모터 제어 신호로 변환하여 특정 비행 궤적으로 비행하도록 제어하는 통상 제어 마이크로 프로세서(157) 및 학습된 스파이크 신경망(SNN)을 사용하여 센서 데이터를 기반으로 자세를 측정하여 특정 상황을 인지하고, 특정 상황에서 모터 제어 신호를 조정하여 자세를 유지하면서 비행하도록 제어하는 돌풍상황 제어 아날로그 컴퓨터(158)를 포함할 수 있다. Referring to FIG. 6, in a drone (100B) according to another embodiment, a drone control unit (150) may include a general control microprocessor (157) that converts a motor signal into a motor control signal in a general situation and controls the drone to fly along a specific flight trajectory, and a gust situation control analog computer (158) that measures attitude based on sensor data using a learned spike neural network (SNN) to recognize a specific situation, and controls the drone to fly while maintaining attitude by adjusting the motor control signal in a specific situation.
통상 제어 마이크로 프로세서(157)는 돌풍 상황과 같은 특정 상황이 아닌 일반적 상황 또는 통상 상황에서 드론(100)의 비행 및 요구하고자 하는 작업을 수행할 수 있는 폰 노이만 컴퓨터 기반 마이크로프로세서 시스템으로 구현될 수 있다. Typically, the control microprocessor (157) may be implemented as a von Neumann computer-based microprocessor system capable of performing flight and desired tasks of the drone (100) in general situations or normal situations rather than specific situations such as gusts of wind.
드론 제어부(150)는 돌풍 상황과 같은 특정 상황을 인지하고 인지에 따라 드론의 비행 제어권을 SNN 기반 돌풍상황 제어 아날로그 컴퓨터(158)로 이전한다. The drone control unit (150) recognizes a specific situation, such as a gust of wind, and transfers the drone's flight control to the SNN-based gust of wind situation control analog computer (158) based on the recognition.
전술한 바와 같이, 돌풍상황 제어 아날로그 컴퓨터(158)에 활용한 전술한 스파이크 신경망(SNN)은 생물학적 뉴런의 작동 원리를 모방한 인공 신경망으로, 스파이크 신경망(SNN)은 드론(100)의 자세를 감지하고 분석하여 제어 신호를 생성하는데 사용된다. 돌풍상황 제어 아날로그 컴퓨터(158)는 아날로그 신호를 처리할 수 있는 스파이크 신경망(SNN)을 적용함으로써 저전력 및 저지연성을 갖는 드론 자세 제어 시스템을 구현할 수 있다.As described above, the spike neural network (SNN) utilized in the gust situation control analog computer (158) is an artificial neural network that imitates the operating principles of biological neurons, and the spike neural network (SNN) is used to detect and analyze the attitude of the drone (100) and generate a control signal. The gust situation control analog computer (158) can implement a drone attitude control system with low power and low latency by applying the spike neural network (SNN) capable of processing analog signals.
돌풍상황 제어 아날로그 컴퓨터(158)는 스파이크 신경망(SNN)의 학습 및 분류를 위한 알고리즘을 적용한다. 주어진 데이터를 학습하고 분류하는 과정에서 인공 뉴런은 다수의 지점에서 동시에 소수의 스파이크 불 발생을 통해 드론(100)의 자세 정보를 분석한다. 이를 통해 스파이크 신경망(SNN)은 드론(100)의 자세 정보를 분석하여 모터 제어 신호를 생성한다.The analog computer (158) for controlling the gust situation applies an algorithm for learning and classifying a spike neural network (SNN). In the process of learning and classifying given data, the artificial neuron analyzes the attitude information of the drone (100) through the occurrence of a small number of spikes at multiple points simultaneously. Through this, the spike neural network (SNN) analyzes the attitude information of the drone (100) and generates a motor control signal.
스파이크 신경망(SNN)이 분석한 드론(100)의 자세 정보를 기반으로 제어 신호를 생성하는 장치가 필요하다. 이를 통해 스파이크 신경망(SNN)은 드론(100)의 자세를 빠르게 제어할 수 있다.A device is required to generate a control signal based on the attitude information of the drone (100) analyzed by the spike neural network (SNN). Through this, the spike neural network (SNN) can quickly control the attitude of the drone (100).
전술한 또다른 실시예에 따른 드론(100B)은 돌풍 상황에서 기체의 자세를 빠르게 제어하고 유지하기 위하여 스파이크 신경망(SNN)을 활용하며, 이를 위해 아날로그 컴퓨터 기반의 집적회로를 이용하여 설계한다. A drone (100B) according to another embodiment described above utilizes a spike neural network (SNN) to quickly control and maintain the attitude of the aircraft in a gust of wind, and is designed using an analog computer-based integrated circuit for this purpose.
이를 위해 스파이크 신경망(SNN)을 통해 드론의 자세를 빠르게 감지하고 분석하여 즉각적으로 제어 신호를 생성하며, 이를 위해 저전력 및 저지연성을 갖는 스파이크 신경망(SNN) 구조를 이용한다. 이에 따라, 전술한 또다른 실시예에 따른 드론(100B)은 돌풍 상황에서 빠르고 안정적으로 드론(100B)의 자세를 제어할 수 있어 안전하게 비행을 수행할 수 있게 한다.To this end, the attitude of the drone is quickly detected and analyzed through a spike neural network (SNN) to immediately generate a control signal, and a spike neural network (SNN) structure with low power and low latency is used for this purpose. Accordingly, the drone (100B) according to another embodiment described above can quickly and stably control the attitude of the drone (100B) in a gust of wind, thereby enabling safe flight.
도 6은 도 5의 통상 제어 마이크로 프로세서와 돌풍 상황 아날로그 컴퓨터의 비행 제어권 천이 과정의 개념도이다. Figure 6 is a conceptual diagram of the flight control transfer process of the conventional control microprocessor and the gust situation analog computer of Figure 5.
도 5 및 도 6을 참조하면, 또 다른 실시예에 따른 드론(100B)에서 드론 제어부(150)는 돌풍 상황을 인지한 경우 기체의 제어권을 돌풍 상황 제어 아날로그 컴퓨터(158)로 천이하고, 돌풍 상황이 종료된 경우 통상 제어 마이크로 프로세서(157)로 기체의 제어권을 천이할 수 있다. Referring to FIGS. 5 and 6, in a drone (100B) according to another embodiment, when a gust of wind situation is recognized, the drone control unit (150) may transfer control of the aircraft to a gust of wind situation control analog computer (158), and when the gust of wind situation is over, the control of the aircraft may be transferred to a normal control microprocessor (157).
구체적으로 기체의 제어권이 돌풍상황 제어 아날로그 컴퓨터(158)에 있는 경우 드론(100)의 기체의 안정화 정도에 대한 판단은 스파이크 신경망(SNN)을 통해 이루어지며, 스파이크 신경망(SNN)에서 돌풍 상황 종료를 인지하여 제어권을 기존의 통상 제어 마이크로프로세서(157)로 이전한다.Specifically, when the control of the aircraft is in the gust situation control analog computer (158), the determination of the degree of stabilization of the aircraft of the drone (100) is made through a spike neural network (SNN), and when the spike neural network (SNN) recognizes the end of the gust situation, the control is transferred to the existing conventional control microprocessor (157).
또 다른 실시예에 따른 드론(100B)은 스파이크 신경망(SNN)을 아날로그 컴퓨터 기반의 집적회로를 이용하여 보다 효율적으로 구현할 수 있다. 이는 신경망 집적 회로(Neural Processing Unit) 또는 뉴로모픽스 칩으로도 명칭되며, 이를 통해 드론 자세 제어 시스템을 더욱 효율적으로 구현할 수 있다.A drone (100B) according to another embodiment can more efficiently implement a spike neural network (SNN) using an analog computer-based integrated circuit. This is also called a neural processing unit or neuromorphic chip, and can more efficiently implement a drone attitude control system.
도 7은 도 6의 돌풍 상황 제어 아날로그 컴퓨터의 입출력 신호들을 도시하고 있다. Figure 7 illustrates the input/output signals of the gust situation control analog computer of Figure 6.
도 7을 참조하면, 돌풍 상황 제어 아날로그 컴퓨터(158)는 모터 신호 및 센서 데이터를 수신하고, 학습된 스파이크 신경망(SNN)을 이용하여 돌풍 상황을 판단하는 돌풍 상황 판단 신호와 모터 제어 신호를 출력할 수 있다. Referring to FIG. 7, the gust situation control analog computer (158) can receive a motor signal and sensor data, and output a gust situation judgment signal and a motor control signal that judge a gust situation using a learned spike neural network (SNN).
돌풍상황 제어 아날로그 컴퓨터(158)는 모터 신호 및 센서 데이터를 수신하고 학습된 스파이크 신경망(SNN)을 사용하여 센서 데이터를 기반으로 자세를 측정하여 특정 상황을 인지하고, 특정 상황에서 돌풍 상황 판단 신호와 모터 제어 신호를 출력하여 자세를 유지하면서 비행하도록 제어할 수 있다.The gust situation control analog computer (158) receives motor signals and sensor data, measures attitude based on the sensor data using a learned spike neural network (SNN), recognizes a specific situation, and outputs a gust situation judgment signal and a motor control signal in a specific situation to control flight while maintaining attitude.
전술한 실시예들에 따른 드론(100, 100A, 100B)은 종래의 드론 자세 제어 시스템과 비교하여 우수함을 인정받을 수 있다. 이는 스파이크 신경망(SNN)을 이용하여 드론의 자세와 바람으로 인해 영향 받은 정도, 앞으로의 바람 강도에 대한 예측 등 주위 환경과 상태를 빠르게 감지하고 분석하여 즉각적으로 제어 신호를 생성하면서 저전력 및 저지연성을 갖는다는 점에서 기존의 시스템보다 효율적인 성능을 낼 수 있다. The drone (100, 100A, 100B) according to the above-described embodiments can be recognized as superior to the conventional drone attitude control system. This is because it can quickly detect and analyze the surrounding environment and conditions, such as the attitude of the drone, the degree of influence due to wind, and prediction of future wind strength, using a spike neural network (SNN), and immediately generate a control signal while having low power and low latency, thereby showing more efficient performance than the conventional system.
전술한 실시예들에 따른 드론(100, 100A, 100B)은 돌풍 상황에서 빠르고 안정적으로 자세를 제어할 수 있으며, 이를 통해 안전하게 비행을 수행할 수 있게 한다는 것을 장점으로 갖는다. 따라서, 전술한 실시예들에 따른 드론(100, 100A, 100B)은 드론 산업에서 높은 수준의 안전성, 안정성 및 성능을 제공한다.The drone (100, 100A, 100B) according to the above-described embodiments has the advantage of being able to quickly and stably control attitude in a gust of wind situation, thereby enabling safe flight. Therefore, the drone (100, 100A, 100B) according to the above-described embodiments provides a high level of safety, stability, and performance in the drone industry.
도 8은 또 다른 실시예에 따른 드론 제어방법의 흐름도이다.Figure 8 is a flow chart of a drone control method according to another embodiment.
도 8을 참조하면, 또 다른 실시예에 따른 드론 제어방법(300)은 일반적인 상황에서 모터 신호를 모터 제어 신호로 변환하여 특정 비행 궤적으로 비행하도록 제어하는 제1단계(S310) 및 학습된 스파이크 신경망을 사용하여 센서 데이터를 기반으로 자세를 측정하여 특정 상황을 인지하고, 특정 상황에서 모터 제어 신호를 조정하여 자세를 유지하면서 비행하도록 제어하는 제2단계(S320)를 포함한다.Referring to FIG. 8, a drone control method (300) according to another embodiment includes a first step (S310) of controlling the drone to fly along a specific flight trajectory by converting a motor signal into a motor control signal in a general situation, and a second step (S320) of measuring attitude based on sensor data using a learned spike neural network to recognize a specific situation, and controlling the drone to fly while maintaining attitude by adjusting the motor control signal in the specific situation.
제2단계(S320)에서, 학습된 스파이크 신경망을 사용하여 조정된 상기 모터 제어 신호를 모터 속도와 방향을 조정하는 특정 PWM 신호로 변환할 수 있다. In the second step (S320), the learned spike neural network can be used to convert the adjusted motor control signal into a specific PWM signal that adjusts the motor speed and direction.
전술한 바와 같이, 센서 데이터는 자이로 센서에서 센싱된 각속도와 가속도 센서에서 센싱한 가속도일 수 있으나, 이에 제한되지 않는다. As described above, the sensor data may be, but is not limited to, angular velocity sensed by a gyro sensor and acceleration sensed by an acceleration sensor.
도 3을 참조하여 전술한 바와 같이, 스파이크 신경망(SNN)은 다수의 레이어들(210)을 포함하고, 각각의 레이어(210)에는 다수의 뉴런이 존재하고, 임의의 레이어(210)를 기준으로 했을 때 프리 시냅틱 뉴런(220)과 포스트 시냅틱 뉴런(250), 프리 시냅틱 뉴런(220)과 포스트 시냅틱 뉴런(250) 사이를 연결하는 시냅스(230)를 포함하고, 포스트 시냅틱 뉴런(250)에서의 출력 스파이크(260) 발생 시간과 프리 시냅틱 뉴런(220)에서의 입력 스파이크(240) 발생 시간 차이로 상기 시냅스의 가중치를 조절할 수 있다. As described above with reference to FIG. 3, the spike neural network (SNN) includes a plurality of layers (210), and each layer (210) includes a plurality of neurons, and based on an arbitrary layer (210), includes a pre-synaptic neuron (220) and a post-synaptic neuron (250), and a synapse (230) connecting the pre-synaptic neuron (220) and the post-synaptic neuron (250), and the weight of the synapse can be adjusted based on the difference in the occurrence time of an output spike (260) in the post-synaptic neuron (250) and the occurrence time of an input spike (240) in the pre-synaptic neuron (220).
일 예로, 제1단계(S310)에서, 도 4에 도시한 제1마이크로 프로세서(154)가 일반적인 상황에서 상기 모터 신호를 모터 제어 신호로 변환하여 특정 비행 궤적으로 비행하도록 제어하고, 제2단계(320)에서, 도 4에 도시한 제2마이크로 프로세서(156)가 학습된 스파이크 신경망(SNN)을 사용하여 센서 데이터를 기반으로 자세를 측정하여 특정 상황을 인지하고, 특정 상황에서 모터 제어 신호를 조정하여 자세를 유지하면서 비행하도록 제어할 수 있다. For example, in the first step (S310), the first microprocessor (154) illustrated in FIG. 4 converts the motor signal into a motor control signal in a general situation and controls the flight to a specific flight trajectory, and in the second step (320), the second microprocessor (156) illustrated in FIG. 4 measures the attitude based on sensor data using a learned spike neural network (SNN) to recognize a specific situation, and controls the flight to maintain the attitude by adjusting the motor control signal in the specific situation.
다른 예로, 제1단계(S310)에서, 도 5에 도시한 통상 제어 마이크로 프로세서(157)가 일반적인 상황에서 모터 신호를 모터 제어 신호로 변환하여 특정 비행 궤적으로 비행하도록 제어하고, 제2단계(S320)에서, 도 5에 도시한 돌풍상황 제어 아날로그 컴퓨터(158)가 학습된 스파이크 신경망을 사용하여 센서 데이터를 기반으로 자세를 측정하여 특정 상황을 인지하고, 특정 상황에서 상기 모터 제어 신호를 조정하여 자세를 유지하면서 비행하도록 제어할 수 있다. As another example, in the first step (S310), the conventional control microprocessor (157) illustrated in FIG. 5 converts a motor signal into a motor control signal in a general situation and controls the flight to a specific flight trajectory, and in the second step (S320), the gust situation control analog computer (158) illustrated in FIG. 5 measures the attitude based on sensor data using a learned spike neural network to recognize a specific situation, and controls the flight to maintain the attitude by adjusting the motor control signal in the specific situation.
제2단계(S320)에서, 도 6에 도시한 바와 같이, 돌풍 상황을 인지한 경우 기체의 제어권을 돌풍 상황 제어 아날로그 컴퓨터(158)로 천이하고, 돌풍 상황이 종료된 경우 통상 제어 마이크로 프로세서(157)로 기체의 제어권을 천이할 수 있다. In the second step (S320), as illustrated in FIG. 6, if a gust situation is recognized, control of the aircraft may be transferred to a gust situation control analog computer (158), and if the gust situation has ended, control of the aircraft may be transferred to a normal control microprocessor (157).
도 7을 참조하여 전술한 바와 같이, 돌풍 상황 제어 아날로그 컴퓨터(158)는 모터 신호 및 센서 데이터를 수신하고, 학습된 스파이크 신경망을 이용하여 돌풍 상황을 판단하는 돌풍 상황 판단 신호와 모터 제어 신호를 출력할 수 있다.As described above with reference to FIG. 7, the gust situation control analog computer (158) can receive motor signals and sensor data, and output a gust situation judgment signal and a motor control signal that judge a gust situation using a learned spike neural network.
도 9는 도 8의 제2단계의 드론 제어 과정들을 상세히 도시하고 있다.Figure 9 illustrates in detail the drone control processes of the second stage of Figure 8.
도 9을 참조하면, 도 8을 참조하여 전술한 제2단계(S320)에서, 모터 신호(PWM) 기반 돌풍 상황에서 드론 제어권을 천이하고(S321), 입력된 모터 신호(PWM) 및 현재 기체의 상태에 대한 센싱 데이터를 입력하고(S322), 센싱 데이터에 대한 스파이크 임베딩 및 스파이크 신경망 네트워크를 입력하고(S323), 입력된 정보를 바탕으로 각 전기 모터(220)에 적합한 PWM 제어 신호를 생성하고 전기 모터(220)에 입력하고(S324), PWM 제어 신호를 통해 드론 제어 이후에 돌풍 상황 해소시 제어권을 종료한다(S325). Referring to FIG. 9, in the second step (S320) described above with reference to FIG. 8, the drone control right is transferred in a gust situation based on a motor signal (PWM) (S321), the input motor signal (PWM) and sensing data on the current state of the aircraft are input (S322), spike embedding and a spike neural network for the sensing data are input (S323), a PWM control signal suitable for each electric motor (220) is generated based on the input information and input to the electric motor (220) (S324), and after controlling the drone through the PWM control signal, the control right is terminated when the gust situation is resolved (S325).
도 6을 참조하여 전술한 바와 같이, 돌풍 상황을 인지한 경우 기체의 제어권을 돌풍 상황 제어 아날로그 컴퓨터(158)로 천이하고, 돌풍 상황이 종료된 경우 통상 제어 마이크로 프로세서(157)로 기체의 제어권을 천이할 수 있다. As described above with reference to FIG. 6, when a gust situation is recognized, control of the aircraft can be transferred to a gust situation control analog computer (158), and when the gust situation has ended, control of the aircraft can be transferred to a normal control microprocessor (157).
전술한 또 다른 실시예에 따른 드론 제어방법(300)은 종래의 드론 자세 제어 시스템과 비교하여 우수함을 인정받을 수 있다. 이는 스파이크 신경망(SNN)을 이용하여 드론의 자세와 바람으로 인해 영향 받은 정도, 앞으로의 바람 강도에 대한 예측 등 주위 환경과 상태를 빠르게 감지하고 분석하여 즉각적으로 제어 신호를 생성하면서 저전력 및 저지연성을 갖는다는 점에서 기존의 시스템보다 효율적인 성능을 낼 수 있다. The drone control method (300) according to another embodiment described above can be recognized as superior to conventional drone attitude control systems. It can quickly detect and analyze the surrounding environment and conditions, such as the attitude of the drone, the degree of influence by the wind, and prediction of future wind strength, using a spike neural network (SNN), and generate a control signal immediately while having low power and low latency, so that it can exhibit more efficient performance than conventional systems.
전술한 또 다른 실시예에 따른 드론 제어방법(300)은 돌풍 상황에서 빠르고 안정적으로 자세를 제어할 수 있으며, 이를 통해 안전하게 비행을 수행할 수 있게 한다는 것을 장점으로 갖는다. 따라서, 전술한 또 다른 실시예에 따른 드론 제어방법(300)은 드론 산업에서 높은 수준의 안전성, 안정성 및 성능을 제공한다.The drone control method (300) according to another embodiment described above has the advantage of being able to quickly and stably control attitude in a gust of wind situation, thereby enabling safe flight. Therefore, the drone control method (300) according to another embodiment described above provides a high level of safety, stability, and performance in the drone industry.
이상의 설명은 본 개시의 기술 사상을 예시적으로 설명한 것에 불과한 것으로서, 본 개시가 속하는 기술 분야에서 통상의 지식을 가진 자라면 본 기술 사상의 본질적인 특성에서 벗어나지 않는 범위에서 다양한 수정 및 변형이 가능할 것이다. 또한, 본 실시예들은 본 개시의 기술 사상을 한정하기 위한 것이 아니라 설명하기 위한 것이므로 이러한 실시예에 의하여 본 기술 사상의 범위가 한정되는 것은 아니다. 본 개시의 보호 범위는 아래의 청구범위에 의하여 해석되어야 하며, 그와 동등한 범위 내에 있는 모든 기술 사상은 본 개시의 권리 범위에 포함되는 것으로 해석되어야 할 것이다. The above description is merely an example of the technical idea of the present disclosure, and those skilled in the art to which the present disclosure pertains may make various modifications and variations without departing from the essential characteristics of the technical idea of the present disclosure. In addition, the present embodiments are not intended to limit the technical idea of the present disclosure but to explain it, and therefore the scope of the technical idea of the present disclosure is not limited by these embodiments. The protection scope of the present disclosure should be interpreted by the claims below, and all technical ideas within a scope equivalent thereto should be interpreted as being included in the scope of the rights of the present disclosure.
CROSS-REFERENCE TO RELATED APPLICATIONCROSS-REFERENCE TO RELATED APPLICATION
본 특허출원은 2023년 12월 12일 한국에 출원한 특허출원번호 제 10-2023-0179317 호에 대해 미국 특허법 119(a)조 (35 U.S.C § 119(a))에 따라 우선권을 주장하며, 그 모든 내용은 참고문헌으로 본 특허출원에 병합된다. 아울러, 본 특허출원은 미국 이외에 국가에 대해서도 위와 동일한 이유로 우선권을 주장하면 그 모든 내용은 참고문헌으로 본 특허출원에 병합된다.This patent application claims the benefit of priority under 35 U.S.C. § 119(a) to Korean patent application No. 10-2023-0179317, filed December 12, 2023, the entire contents of which are incorporated herein by reference. In addition, this patent application claims priority for other countries than the United States for the same reasons, the entire contents of which are incorporated herein by reference.
Claims (16)
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| KR10-2023-0179317 | 2023-12-12 | ||
| KR1020230179317A KR20250089721A (en) | 2023-12-12 | 2023-12-12 | Drone and Method for controlling Drone |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2025127558A1 true WO2025127558A1 (en) | 2025-06-19 |
Family
ID=96057936
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/KR2024/019419 Pending WO2025127558A1 (en) | 2023-12-12 | 2024-12-02 | Drone and drone control method therefor |
Country Status (2)
| Country | Link |
|---|---|
| KR (1) | KR20250089721A (en) |
| WO (1) | WO2025127558A1 (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN120523227B (en) * | 2025-07-24 | 2025-09-19 | 山东龙翼航空科技有限公司 | Control system and control method of vector bypass unmanned aerial vehicle |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180253980A1 (en) * | 2017-03-03 | 2018-09-06 | Farrokh Mohamadi | Drone Terrain Surveillance with Camera and Radar Sensor Fusion for Collision Avoidance |
| JP2019059314A (en) * | 2017-09-26 | 2019-04-18 | 日本電信電話株式会社 | Flight control device, method and program |
| KR102204107B1 (en) * | 2020-07-16 | 2021-01-18 | 세종대학교산학협력단 | Spiking Neural Network with STDP apllied to the threshold of neuron |
| KR102313115B1 (en) * | 2021-06-10 | 2021-10-18 | 도브텍 주식회사 | Autonomous flying drone using artificial intelligence neural network |
| KR20230149071A (en) * | 2022-04-19 | 2023-10-26 | 백석대학교산학협력단 | System and method for predicting dangerous driving behavior using spiking neural network |
-
2023
- 2023-12-12 KR KR1020230179317A patent/KR20250089721A/en active Pending
-
2024
- 2024-12-02 WO PCT/KR2024/019419 patent/WO2025127558A1/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180253980A1 (en) * | 2017-03-03 | 2018-09-06 | Farrokh Mohamadi | Drone Terrain Surveillance with Camera and Radar Sensor Fusion for Collision Avoidance |
| JP2019059314A (en) * | 2017-09-26 | 2019-04-18 | 日本電信電話株式会社 | Flight control device, method and program |
| KR102204107B1 (en) * | 2020-07-16 | 2021-01-18 | 세종대학교산학협력단 | Spiking Neural Network with STDP apllied to the threshold of neuron |
| KR102313115B1 (en) * | 2021-06-10 | 2021-10-18 | 도브텍 주식회사 | Autonomous flying drone using artificial intelligence neural network |
| KR20230149071A (en) * | 2022-04-19 | 2023-10-26 | 백석대학교산학협력단 | System and method for predicting dangerous driving behavior using spiking neural network |
Also Published As
| Publication number | Publication date |
|---|---|
| KR20250089721A (en) | 2025-06-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Pérez et al. | Ground vehicle detection through aerial images taken by a UAV | |
| CN113848984B (en) | Unmanned aerial vehicle cluster control method and system | |
| WO2025127558A1 (en) | Drone and drone control method therefor | |
| Quigley et al. | The open vision computer: An integrated sensing and compute system for mobile robots | |
| WO2021230463A1 (en) | Method for optimizing on-device neural network model by using sub-kernel searching module and device using the same | |
| WO2021241983A1 (en) | Method and apparatus for semi-supervised learning | |
| CN114153166B (en) | Integrated aircraft system based on modularized application | |
| WO2020231080A1 (en) | Method for generating intersection point pattern recognition model using sensor data of mobile robot and intersection point pattern recognition system | |
| WO2022075543A1 (en) | Anomaly detection method using multi-modal sensor, and computing device for performing same | |
| WO2022055099A1 (en) | Anomaly detection method and device therefor | |
| CN107065909A (en) | A kind of flight control system based on BCI | |
| Andersen et al. | Event-based navigation for autonomous drone racing with sparse gated recurrent network | |
| Rajathanakodi et al. | MAD-STORM: Maneuverable Autonomous Drone with Sensing Technologies for Observing Rainfall and Meteorology in Northern Ireland | |
| WO2020213755A1 (en) | Robot and map updating method using same | |
| WO2021066275A1 (en) | Electronic device and method for controlling same | |
| WO2022211310A1 (en) | Systems and methods for adaptive robotic motion control | |
| WO2023128654A1 (en) | Method for optimizing learning model for target apparatus and system therefor | |
| CN113724295A (en) | Unmanned aerial vehicle tracking system and method based on computer vision | |
| WO2019245191A1 (en) | Apparatus and method for analyzing image | |
| WO2024063589A1 (en) | System and method for intelligent user localization in metaverse | |
| CN117830879A (en) | Indoor-oriented distributed unmanned aerial vehicle cluster positioning and mapping method | |
| WO2022139109A1 (en) | Method, system, and non-transitory computer-readable recording medium, for monitoring object | |
| WO2023158075A1 (en) | Autonomous robot capable of moving obstacle avoidance | |
| US20250036915A1 (en) | Npu and system for switching neural network models | |
| WO2022149906A1 (en) | Apparatus and method for identifying floor on which user is located, by using barometric pressure and magnetic field |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 24904123 Country of ref document: EP Kind code of ref document: A1 |