CN116611481A - Object motion trend sensing method and system based on impulse neural network - Google Patents

Abstract

The application relates to an object motion trend sensing method and system based on a pulse neural network, comprising the steps of obtaining an index of a current area of an object; generating pulse information by binary coding the region index; the FPGA converts the pulse information into frame information and sends the frame information to the PAICORE2.0 through a PCIe interface; and the PAICORE2.0 analyzes the frame information and performs reasoning to generate the position information of the object at the next moment, wherein the PAICORE2.0 is a brain-like chip deployed with a pulse neural network model and is used for realizing the pulse neural network function. The method solves the problems of long calculation time, high power consumption of a hardware system and low utilization rate of time-space relevance of information in the traditional algorithm.

Description

A method and system for object movement trend perception based on spiking neural network

technical field

The invention relates to the field of new computer technologies, in particular to a method and system for sensing object movement trends based on a pulse neural network.

Background technique

Most of the existing object motion trend perception systems are realized by combining the traditional target detection algorithm and tracking algorithm. Traditional target detection algorithms mainly include background difference method, inter-frame difference method, optical flow method and block-based motion estimation and compensation method in dynamic background. extracted from. The traditional tracking algorithm first expresses the target effectively, then uses the similarity measurement algorithm to match the moving target and the frame image, and finally uses the search algorithm to match the best position. Common prediction algorithms include Kalman filter and particle filter methods.

However, the prior art has the following disadvantages: 1. The calculation time of traditional target detection and prediction algorithms is long. The hardware implementation of traditional algorithms takes hundreds of milliseconds and cannot be used for trend prediction and perception of high-speed moving targets. 2. The hardware system consumes a lot of power. Traditional algorithm deployment does not use dedicated chips, which consumes a lot of resources and consumes a lot of power. 3. The utilization rate of information spatio-temporal relevance is low. The realization of the traditional object motion trend perception system relies on continuous single calculation, without considering the temporal correlation of information.

Contents of the invention

In order to solve at least one of the problems mentioned in the above-mentioned background technology, the present invention proposes a method and system for sensing the movement trend of an object based on a pulse neural network. The PC judges the current area index of the object through event information, and generates pulse information through binary coding. As a converter of pulse information and frames required by PAICORE2.0, FPGA sends frames to PAICORE2.0 through PCIe interface (2.5GT/s). PAI CORE2.0 analyzes the frame information, sends the pulse information to the internally built pulse neural network for inference, and obtains the position information of the object at the next moment. Send this information to the FPGA through the frame and then to the PC, analyze and obtain the corresponding position index, and realize the perception of the movement trend of the object.

For realizing the above object, the technical scheme adopted in the present invention comprises:

A spiking neural network-based object motion trend perception method, comprising:

Get the index of the area where the object is currently located;

Binary coding the region index to generate pulse information;

The FPGA converts the pulse information into frame information and sends it to PAICORE2.0 through the PCIe interface;

The PAICORE 2.0 analyzes the frame information and performs inference to generate the position information of the object at the next moment.

Further, the PAICORE2.0 is a brain-inspired chip deployed with a spiking neural network model, which is used to realize the function of the spiking neural network.

Further, the acquiring the index of the area where the object is currently located includes: judging the index of the area where the object is currently located by the PC through event information, wherein the method for acquiring the event information includes acquiring by using a sensor.

Further, the method further includes: sending the location information back to the PC through the FPGA, and analyzing to obtain a location index corresponding to the location information.

Further, the method further includes: using DQN to train a neural network, and converting the neural network into the spiking neural network.

Further, the PAICORE2.0 supports the LIF neuron model.

The present invention also relates to an object motion trend perception system based on a pulse neural network, comprising:

The input module is used to obtain the index of the area where the object is currently located;

Binary coding the region index to generate pulse information;

The FPGA converts the pulse information into frame information and sends it to PAICORE2.0 through the PCIe interface;

The processing module is used for the PAICORE2.0 to analyze the frame information and perform inference to generate the position information of the object at the next moment.

The present invention also relates to a computer-readable storage medium, on which a computer program is stored, and the above-mentioned method is realized when the computer program is executed by a processor.

The present invention also relates to an electronic device, comprising a memory, a processor, and a computer program stored in the memory and operable on the processor, and the processor implements the above-mentioned method when executing the computer program.

The present invention also relates to a computer program product, including computer programs and/or instructions, characterized in that, when the computer programs and/or instructions are executed by a processor, the steps of the above method are realized.

The beneficial effects of the present invention are:

1. Compared with the calculation delay of hundreds of milliseconds in the traditional algorithm, the response speed of 10ms/each feedback is realized by using the characteristics of PAICORE2.0 and event information processing.

2. The hardware power consumption is low. When the object movement trend perception system is running, the power consumption of PAICORE2.0 is within 1W.

3. Utilizing the event correlation of information, the way of bio-like processing information, the event stream is binary coded into pulse information, and the pulse neural network uses the temporal and spatial correlation of pulses to reason and speed up perception.

Description of drawings

FIG. 1 is a schematic flow chart of the object movement trend perception method based on the spiking neural network of the present invention.

Fig. 2 is a schematic diagram of a simulated scene in a preferred embodiment of the present invention.

Fig. 3 is a flow chart of the table tennis interception experiment program of the preferred embodiment of the present invention.

Fig. 4 is a schematic diagram of constraints on the way of intercepting movement of the slider in the preferred embodiment of the present invention.

Fig. 5 is a structural diagram of a neural network in a preferred embodiment of the present invention.

Fig. 6 is a schematic diagram of the principles of the DQN algorithm in the preferred embodiment of the present invention.

Fig. 7 is a flowchart of neural network training in a preferred embodiment of the present invention.

Fig. 8 is a schematic diagram of an abstract process from a biological neuron to a spiking neuron in a preferred embodiment of the present invention.

Fig. 9 is an abstracted spiking neuron model of the preferred embodiment of the present invention.

FIG. 10 is a schematic diagram of a comparison of calculation modes between ANN and SNN in a preferred embodiment of the present invention.

FIG. 11 is a schematic structural diagram of the object movement trend perception system based on the spiking neural network of the present invention.

Fig. 12 is a simplified wiring diagram of the input module of the preferred embodiment of the present invention.

Fig. 13 is a program flow chart of the input module of the preferred embodiment of the present invention.

Fig. 14 is a flow chart of the processing module program in the preferred embodiment of the present invention.

Fig. 15 is a flow chart of the output module program in the preferred embodiment of the present invention.

Fig. 16 is the final measured system parameter diagram of the present invention.

Detailed ways

In order to more clearly illustrate the technical solutions of the embodiments of the present specification, the following briefly introduces the drawings that need to be used in the description of the embodiments. Apparently, the accompanying drawings in the following description are only some examples or embodiments of this specification, and those skilled in the art can also apply this specification to other similar scenarios. Unless otherwise apparent from context or otherwise indicated, like reference numerals in the figures represent like structures or operations.

It should be understood that "system", "device", "unit" and/or "module" as used herein is a method for distinguishing different components, elements, parts, parts or assemblies of different levels. However, the words may be replaced by other expressions if other words can achieve the same purpose.

As indicated in the specification and claims, the terms "a", "an", "an" and/or "the" are not specific to the singular and may include the plural unless the context clearly indicates an exception. Generally speaking, the terms "comprising" and "comprising" only suggest the inclusion of clearly identified steps and elements, and these steps and elements do not constitute an exclusive list, and the method or device may also contain other steps or elements.

The flowchart is used in this specification to illustrate the operations performed by the system according to the embodiment of this specification. It should be understood that the preceding or following operations are not necessarily performed in the exact order. Instead, various steps may be processed in reverse order or simultaneously. At the same time, other operations can be added to these procedures, or a certain step or steps can be removed from these procedures.

The first aspect of the present invention relates to an object motion trend perception method based on a pulse neural network as shown in Figure 1, including:

Get the index of the area where the object is currently located;

Binary coding the region index to generate pulse information;

The FPGA converts the pulse information into frame information and sends it to PAICORE2.0 through the PCIe interface;

Specifically, in the embodiment of the present application, the PC judges the index of the area where the object is currently located based on the event information, where the event information can be obtained by a sensor, such as an event camera. The area where the object is located is judged by the number of events in a certain area, and the index of the area is recorded in real time, such as (6,5), (4,12). Binary encoding is performed on the region index to generate pulse information. As a converter of pulse information and frames required by PAICORE2.0, FPGA sends frames to PAICORE2.0 through PCIe interface (2.5GT/s).

The PAICORE 2.0 analyzes the frame information and performs inference to generate the position information of the object at the next moment.

Specifically, in the embodiment of the present application, python programming is used to construct a scene, simulating the interception of a table tennis ball in reality. Ping-pong balls are shot into the acrylic board from the outside at any angle and speed. The slider predicts the movement trend and final arrival position of the ping-pong ball, and intercepts the ping-pong ball at the corresponding predicted position, as shown in Figure 2.

In this process, pygame.init() is mainly used to initialize pygame; pygame.display.se t_mode([1280,800]) sets the size of the background curtain to 1280×800, and the size of the curtain is consistent with the resolution of the event camera during hardware deployment; pygame.Rect(1280/2-80,800-160,160,160) creates a square table tennis ball with a width and height of 160; pygame.Rect(-80,0,320,40) creates a rectangular slider with a width of 320 and a height of 40; pygame. font.Font() sets the font; pygame.time.Clock() sets the clock; pygame.draw.ellipse() and pygame.draw.rect() draw table tennis and sliders on the background respectively, and the ellipse() command can Draw the table tennis ball as a circle, the rect() command draws the slider as a bar; pygame.display.flip() updates the text texture; pygame.image.save() saves the current screen; pygame.display.update() updates The whole scene; cv2.imread() reads the image information.

After building the basic scene, it is necessary to move the ping-pong ball and the slider in the scene to complete the task of intercepting the ping-pong ball with the slider. First of all, it is necessary to set a set of reward rules so that when the slider can reach the predicted position in advance and finally predict successfully, and intercept the ping pong ball, the agent will be given a large reward; When the movement trend of , give the agent a certain penalty. This set of rules provides the basis for DQN reinforcement learning, and has three elements: state, action, and income.

Use the pygame library and the cv2 library to program the above-mentioned table tennis and slider movement and reward and punishment rules. The program flow chart is shown in Figure 3.

In the intercepting table tennis task, this embodiment gives it 100 chances, that is, ejecting table tennis balls from the outside to the slider 100 times, when the slider intercepts the table tennis ball each time, it is regarded as a success, so that the interception success rate can be calculated in the future . At the beginning, the ping-pong ball is ejected at a random angle and speed, and then the ping-pong ball and the slider move sequentially according to the constraint conditions. After each movement of the ping-pong ball and the slider, rules are used to reward and punish. The specific rules are: Whether there is a collision between the slider or the bottom edge, if there is no collision, then the distance between the current table tennis ball and the slider needs to be calculated, using reward_1=1-(abs(ball.centerx-paddle.centerx)+abs(ball.centery-paddle. centery))/1820 statement calculation, that is, the closer the distance between the two, the greater the reward_1, the maximum value of reward_1 is 1, and the minimum value is 0. If there is a collision, it is necessary to judge whether the ping-pong ball collides with the slider or the bottom edge. If the ping-pong ball collides with the slider, the agent gets extra points. The extra points are based on reward=-1/24*abs(ball. centerx-paddle.centerx)+10 statement calculation, the maximum value of reward is 10, and the minimum value is 0, that is, when the slider collides with the table tennis ball, the closer the two centers are, the better the collision effect is, and it is hoped that the slider and the table tennis ball will collide They will be able to collide in this posture in the future. If the ping-pong ball collides with the bottom edge, the agent gets a minus point, which is calculated using the reward=-5 statement. Then calculate the total score according to reward_all=reward/10.0*0.7+reward_1*0.3. This completes the acquisition of a set of <state, action, benefit>.

The main constraints of the model focus on the motion form of the table tennis ball. In addition, this embodiment also imposes some constraints on the interception motion of the slider. First of all, it is necessary to constrain the process of each ejection of the table tennis ball. At the beginning, the ping pong ball is ejected at a random angle and speed, but it is necessary to ensure that the ping pong ball does not move out of the scene during the whole process, so the ejection angle and speed are limited here. The constraints made are shown in Figure 4. Every time a ping-pong ball is ejected, regardless of whether the slider intercepts it successfully or not, the ping-pong ball will be refreshed at a random initial position. Ball.centerx=random.randint(80,1200), ball.centery=720 two statements constrain this process. Taking table tennis ball ② as an example, if the initial position is refreshed to position ②, the allowable range of ejection angle is between ②→④ and ②→⑥, the minimum ejection angle is zeta1, and the maximum ejection angle is zeta2. Use zeta1=math.atan2((80-ball.centery), (80-ball.centerx)), zeta2=math.atan2((80-ball.centery), (1200-ball.centerx)), speed_direction= The three statements random.uniform(zeta1, zeta2) describe the process of random angle selection. In order to be suitable for the deployment of the table tennis interception system, here let the ejection speed be arbitrarily selected from 6 to 18, and mapped to the real scene is 24cm/s to 128cm/s, using the speed=random.uniform(6,18) sentence constraint.

After the ping pong ball is launched, it will move in a straight line on the acrylic board. In reality, the table tennis ball will be affected by friction during the movement on the acrylic board, and the speed will continue to decrease slightly. However, because the movement distance of the table tennis ball on the acrylic board is very small, the speed reduction of the table tennis ball is limited. Therefore, this patent believes that The form of motion of table tennis can be regarded as uniform linear motion. According to the above discussion, use ball.centerx+=math.cos(speed_direction)*speed and ball.centery+=math.sin(speed_direction)*speed to constrain the motion form of the table tennis ball to ensure that the motion of the table tennis ball is a uniform linear motion. During the movement, the movement angle and speed of the table tennis ball on the background curtain board are consistent with the angle and speed of each ejection.

In reality, the movement form of the slider is the synthesis of three kinds of movements. The whole movement process (controlled by the servo drive) undergoes accelerated linear motion, uniform linear motion and decelerated linear motion, and finally reaches another position from one position. However, since the movement speed of the slider is much faster than that of the table tennis ball, it needs to move from one position to another at an extremely fast speed, so this embodiment regards this movement as a one-dimensional jumping movement . The slider moves once, and the instantaneous displacement is 160. Because the width of the curtain is 1280, the slider is equivalent to jumping back and forth at 8 positions in the horizontal direction. This embodiment calls this movement form "eighth jump".

In the DQN reinforcement learning, the Q-table needs to be replaced by a neural network, so a neural network needs to be constructed. The structure of the neural network in this embodiment is shown in FIG. 5 .

The entire network consists of 7 layers of fully connected layers and 6 layers of activation layers. ReLu() is the activation function; QuantTrainLinear() is the quantized fully connected layer, and the two parameters are input feature quantities and output feature quantities.

The input of the network is the pulse information converted from the index. The index is converted from binary code into 11-bit binary abscissa, 10-bit binary ordinate, and 11-bit abscissa of the slider, a total of 32-bit pulse information, and this pulse information is copied 4 times to generate 128-bit pulse information as a network input of. The output of the network is 3 feature quantities, which respectively represent the slider moving to the right once, the slider is still and the slider is moving to the left once.

Use the DQN algorithm for the training of neural networks. The principle of the DQN algorithm is shown in Figure 6.

Input the current pulse sequence and the next pulse sequence into the prediction network and the actual network respectively, calculate the Q values of the three actions respectively, take the largest Q value in the prediction network and the actual network as the mean square error, and use this mean square error As a loss function, perform backpropagation, update the parameters of the prediction network, and update the parameters of the prediction network to the real network at regular intervals. According to the principle of the DQN algorithm, write a program to train the neural network. The program flow chart is shown in Figure 7.

The amount of training the neural network using DQN each time includes 5 million state updates, and the state update refers to the process of updating the parameters involved in one movement of the slider and the table tennis ball for one round. In order to fully traverse various states, it is not only necessary to use the neural network to judge the action of the slider after the table tennis movement (here the action refers to the three different states of the slider moving to the left once, standing still, and moving to the right once), but also Add random actions in the early stage of training, and randomly make the slider move to the left once or stay still or move to the right once, so as to avoid the deadlock of DQN reinforcement learning at some point in the local optimal solution. Of course, in this embodiment, it is hoped that the operation of randomly assigning slider actions will only occur at the initial stage of training, so random. Whether to give the slider a random action. If it is not necessary to give the slider a random action, the current state needs to be sent to the neural network, and the action of the slider can be obtained through reasoning. Then operate the slider movement according to this action, operate the table tennis movement according to the rules, and generate a state for standby. Finally, the current state, next state, action, and income need to be sorted and stored in an array, and this information needs to be used when iterating the parameters of the neural network. Finally, it is necessary to constantly judge whether the current score is the highest. If a new highest score is found, the current model needs to be stored and used as the local optimal model.

In this embodiment, the steps for updating the parameters of the prediction network are as follows: First, any piece of data in the array (including current state, next state, action, and income) is taken. This piece of data includes 32 groups<current state, next state, action, income>, that is to say, the neural network training process is carried out with 32 groups <current state, next state, action, income> as the minimum unit. Then calculate the Q value of 32 sets of prediction network and real network, which are called q and q_truth respectively, where q=net1(b_s).gather(1,b_a), q_next=net2(b_s_).detach().max( 1)[0].reshape(b_size, 1), q_truth=b_r+gama*q_next*(1-b_done). The loss function is the minimum mean square error of q and q_truth, and then backpropagation is performed to update and predict network parameters. After the training, save the trained prediction network model as the current optimal model. During the training process, the parameters of the prediction network must be assigned to the real network every once in a while. The real network does not participate in the training process, and only the prediction network uses the gradient descent method to perform backpropagation to update network parameters.

After the network is trained, it is necessary to convert the neural network into a spiking neural network. The spiking neural network is composed of spiking neuron connections. The spiking neuron model is abstracted from biological neurons. The dendrites of biological neurons receive the neurotransmitter released by the synapse of the previous neuron, generate electrical signal form stimulation, and transmit it to the cell body. The action potential is generated, and the action potential is transmitted to the synapse through the axon, and continues to release neurotransmitters to the next neuron, completing the transmission process of electrical signal-chemical signal conversion. The abstract process from biological neuron to spiking neuron is shown in Figure 8.

The abstracted spiking neuron model is shown in Figure 9, where the input is a pulse sequence, and multiple pulse inputs are accumulated with corresponding weights to make the neuron cell body generate a membrane potential. When the membrane potential exceeds a certain threshold, the cell body emits a pulse.

PAICORE2.0 supports the LIF neuron model, and its model can be expressed by the following formula:

Where t represents the time step, n represents the layer of the network, w _ij represents the weight from the jth neuron in the front layer to the i-th neuron in the back layer in the network, o _j represents the output of the jth neuron, and the value is A value of 1 indicates that a pulse is issued, and a value of 0 indicates that no pulse is issued. u _i represents the membrane potential of the ith neuron. Among them, f(·) is the decay function, and g(·) is the activation function, which respectively represent the process of leaking and emitting pulses exceeding the threshold. After the pulse is delivered, the membrane potential returns to zero. If there is no pulse, the membrane potential leaks naturally over time.

The calculation mode comparison of ANN (artificial neural network) and SNN (spiking neural network) is shown in Figure 10. ANN and SNN have the same network structure, weight and bias, the difference is that the input and output of ANN are single floating point numbers, The input and output of SNN are a series of pulse sequences, so if you need to convert ANN to SNN, you need to select the appropriate timestep and Vthr for SNN. The larger the timestep, the closer the precision of the converted SNN is to the original ANN, but considering the efficiency, the timestep cannot be very large. Vthr can choose the maximum value that the membrane potential has ever reached, or a (0<a<1) times the maximum value of the membrane potential, and the best Vthr can be selected through multiple attempts.

In the conversion scheme of the embodiment, the timestep is selected as 64, and Vthr is determined in an iterative manner, and Vthr is the value when the reasoning result of each layer of ANN is most similar to the reasoning result of each layer of SNN. Then through the process of creating a quantizer, creating a converter, using the transformer's init_dag method to parse the ANN, using the transformer's quantize method to quantify, and using the transformer's generate_snn method to generate the SNN, the SNN is finally generated. The specific mapping of the network on PAICORE2.0 can be obtained by converting SNN into a calculation graph and then into an on-chip network.

Deploying the mapping to PAICORE2.0 to complete network configuration requires sending configuration frames to PAICORE2.0. After completing the network configuration, input data needs to be sent to PAICORE2.0, and the input accepted by PAICORE2.0 is in the form of working frames. The result after each inference needs to be converted into a frame and sent to the FPGA, and sent to PAICORE2.0 through the PCIe interface. After completing an input, PAICORE2.0 analyzes the frame and performs inference, and then outputs the working frame and sends it back through the FPGA. This cycle can be realized The operation of the motion trend perception system.

This application can be applied to various scenarios of object motion trend perception. Taking the table tennis interception scene as an example, the event camera is used as the sensor to input event information. Based on the consideration of versatility, the pulse neural network can be deployed on the existing common platforms. , such as Jetson Xavier NX.

Another aspect of the present invention also relates to an object movement trend perception system based on a pulse neural network, its structure is shown in Figure 2, including:

The input module is used to obtain the index of the area where the object is currently located;

Binary coding the region index to generate pulse information;

The FPGA converts the pulse information into frame information and sends it to PAICORE2.0 through the PCIe interface;

The processing module is used for the PAICORE2.0 to analyze the frame information and perform inference to generate the position information of the object at the next moment.

The input module is also used for judging the index of the area where the object is currently located by the PC through the event information, wherein the method for obtaining the event information includes using a sensor to obtain.

The system further includes an output module, configured to transmit the location information back to the PC through the FPGA, and analyze to obtain a location index corresponding to the location information.

Specifically, in this embodiment, the entire system is mainly divided into three modules, which carry out information transmission and processing in sequence, namely an input module, a processing module, and an output module. The system is used to predict the movement trend of the table tennis ball (the table tennis ball is shot into the acrylic board from the outside towards the slide rail, and makes a two-dimensional plane uniform linear motion on the acrylic board), and uses the slider to intercept the table tennis ball in motion.

The input module is an event camera. The input module is mainly used to obtain the real-time position of the table tennis ball and the slider on the acrylic board, and input the event flow information to the AI edge computing device; the processing module is the AI edge computing device. The processing module is mainly used to process the event flow input by the input module and convert it into a pulse flow. Through the calculation of the pulse neural network deployed on it, the trajectory of the table tennis ball and the interception position of the slider are predicted, and the interception position of the slider is sent to the output module. Command; the output module is composed of single-chip microcomputer, servo driver, servo motor, slide rail and slider, and switching power supply. The single-chip microcomputer receives the slider interception position instruction sent by the processing module, forms a control signal and sends it to the servo driver in real time, and the servo driver drives the servo motor to drive the slider to move to the corresponding interception position. The simplified wiring diagram of the system is shown in Figure 12.

The flow chart of the input module program is shown in Figure 13.

The program flow chart of the processing module is shown in Figure 14.

The program flow chart of the output module is shown in Figure 15.

The final measured system parameters are shown in Figure 16.

By using the system, it is possible to execute the above-mentioned arithmetic processing method and achieve corresponding technical effects.

Embodiments of the present invention also provide a computer-readable storage medium capable of implementing all the steps in the method for perceiving object movement trends based on the impulse neural network in the above-mentioned embodiments, and a computer program is stored on the computer-readable storage medium. When the computer program is executed by the processor, all the steps of the method for sensing the movement trend of an object based on the spiking neural network in the above-mentioned embodiments are realized.

An embodiment of the present invention also provides an electronic device for performing the above method. As an implementation device of the method, the electronic device is at least equipped with a processor and a memory, and in particular, the memory stores data required for executing the method. and related computer programs, etc., all the steps of the method are realized by calling the data in the memory by the processor, and the program is executed, and corresponding technical effects are obtained.

Preferably, the electronic device may include a bus architecture, and the bus may include any number of interconnected buses and bridges, and the bus may link together various circuits including one or more processors and memories. The bus may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, all of which are well known in the art and therefore will not be further described herein. The bus interface provides an interface between the bus and the receiver and transmitter. The receiver and transmitter can be the same element, a transceiver, providing means for communicating with various other systems over a transmission medium. The processor is responsible for managing the bus and general processing, while memory may be used to store data that the processor uses when performing operations.

Additionally, the electronic device may further include components such as a communication module, an input unit, an audio processor, a display, and a power supply. The processor (or called controller, operation control) adopted by it may include a microprocessor or other processor devices and/or logic devices, which receive input and control the operation of various components of the electronic equipment; the memory may be One or more of buffer memory, flash memory, hard drive, removable media, volatile memory, non-volatile memory or other suitable devices, which can store the above-mentioned relevant data information, and can also store execution-related information program, and the processor can execute the program stored in the memory to realize information storage or processing, etc.; the input unit is used to provide input to the processor, such as a button or a touch input device; the power supply is used to provide power to electronic equipment ; The display is used for displaying display objects such as images and text, for example, it may be an LCD display. A communication module is a transmitter/receiver that sends and receives signals via an antenna. A communication module (transmitter/receiver) is coupled to the processor to provide input signals and receive output signals, which may be the same as in conventional mobile communication terminals. Based on different communication technologies, multiple communication modules, such as a cellular network module, a bluetooth module and/or a wireless local area network module, may be provided in the same electronic device. The communication module (transmitter/receiver) is also coupled to a speaker and a microphone via an audio processor to provide audio output via the speaker and receive audio input from the microphone for usual telecommunication functions. Audio processors may include any suitable buffers, decoders, amplifiers, etc. In addition, the audio processor is also coupled to the central processing unit, so that the recording on the machine can be made through the microphone, and the sound stored on the machine can be played through the speaker.

Those skilled in the art should understand that the embodiments of the present invention may be provided as methods, systems, or computer program products. Accordingly, the present invention can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a A system for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising a system of instructions, the The system implements the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby The instructions provide steps for implementing the functions specified in the flow chart or blocks of the flowchart and/or the block or blocks of the block diagrams. While preferred embodiments of the present invention have been described, additional changes and modifications can be made to these embodiments by those skilled in the art once the basic inventive concept is appreciated. Therefore, it is intended that the appended claims be construed to cover the preferred embodiment as well as all changes and modifications which fall within the scope of the invention.

The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto, any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention etc. should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be determined by the protection scope of the claims.

Claims (10)

1. A method for sensing the motion trend of an object based on a pulse neural network, characterized in that it comprises: Get the index of the area where the object is currently located; Binary coding the region index to generate pulse information; The FPGA converts the pulse information into frame information and sends it to PAICORE2.0 through the PCIe interface; The PAICORE 2.0 analyzes the frame information and performs inference to generate the position information of the object at the next moment. 2. The method according to claim 1, wherein the PAICORE2.0 is a brain-inspired chip deployed with a spiking neural network model for realizing the spiking neural network function. 3. The method according to claim 1, wherein said obtaining the index of the area where the object is currently located comprises: judging the index of the area where the object is currently located by the PC through event information, wherein the method for acquiring the event information includes using a sensor Get it. 4. The method according to claim 2, further comprising: sending the location information back to the PC through the FPGA, and analyzing to obtain a location index corresponding to the location information. 5. The method according to any one of claims 1 to 4, further comprising: using DQN to train a neural network, and converting the neural network into the spiking neural network. 6. The method according to claim 2, wherein the PAICORE2.0 supports the LIF neuron model. 7. An object motion trend perception system based on a pulse neural network, characterized in that it comprises: The input module is used to obtain the index of the area where the object is currently located; Binary coding the region index to generate pulse information; The FPGA converts the pulse information into frame information and sends it to PAICORE2.0 through the PCIe interface; The processing module is used for the PAICORE2.0 to analyze the frame information and perform inference to generate the position information of the object at the next moment. 8. A computer-readable storage medium, wherein a computer program is stored on the storage medium, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented. 9. A computer device, comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, characterized in that, when the processor executes the computer program, any one of claims 1 to 6 is realized. method described in the item. 10. A computer program product, comprising computer programs and/or instructions, characterized in that, when the computer programs and/or instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.