WO2021057811A1 - Network node processing method, device, storage medium, and electronic apparatus - Google Patents

Abstract

Embodiments of the present application disclose a network node processing method, a device, a storage medium, and an electronic apparatus. The method comprises: triggering and using a neural network model to perform inference computation, and determining a second computation processing unit corresponding to a target network node of at least one network node; if a first computation processing unit is different from the second computation processing unit, deleting the target network node from a first linked list, modifying connection relationships of a previous network node and a next network node of the target network node in the first linked list, and allocating the target network node to the second computation processing unit; acquiring the next network node of the target network node, setting the next network node to be the target network node, and performing determination on the second computation processing unit corresponding to the target network node of the at least one network node; and if it is determined that there is no next network node, generating an inference computation result. The embodiments of the present application improve efficiency of inference computation using a neural network model.

Description

Network node processing method, device, storage medium and electronic equipment Technical field

This application relates to the field of computer technology, and in particular, to a method, device, storage medium, and electronic equipment for processing network nodes.

Background technique

There are many neural network models suitable for mobile terminals on today's smart phone platforms, and it is usually necessary to use appropriate data structures to represent the network nodes of the neural network. For example, Xiaomi's MACE, Tencent's NCNN, Alibaba's MNN, etc., they all use containers to represent the network nodes of the neural network.

When triggering the inference calculation using the neural network model, it is necessary to dynamically determine which calculation processing unit each network node runs on according to the current idle state of each calculation processing unit (such as CPU, GPU, DSP) of the mobile phone. The network structure of the neural network model is modified, that is, the network node in the container needs to be deleted or the network node needs to be added to the container.

Summary of the invention

The embodiments of the present application provide a network node processing method, device, storage medium, and electronic equipment, which only need to modify the connection relationship between the previous network node and the next network node of the network node, without frequent shift operations, and save The adjustment time of the network structure can be improved, which can improve the efficiency of inference calculation using the neural network model. The technical solution is as follows:

In the first aspect, an embodiment of the present application provides a method for processing a network node, and the method includes:

Trigger using a neural network model including at least one network node to perform inference calculations, and determine a second calculation processing unit corresponding to the target network node in the at least one network node, and the at least one network node is pre-stored in the first linked list in the form of a first linked list. A calculation processing unit, where the target network node runs on the second calculation processing unit;

When the first calculation processing unit is different from the second calculation processing unit, delete the target network node from the first linked list and modify the previous network of the target network node in the first linked list The connection relationship between the node and the next network node, assigning the target network node to the second calculation processing unit;

Acquiring the next network node of the target network node, determining the next network node as the target network node, and executing the step of determining the second calculation processing unit corresponding to the target network node in the at least one network node;

When it is determined that there is no next network node, the inference calculation result is generated.

In the second aspect, an embodiment of the present application provides a network node processing device, the device including:

The node determination module is configured to trigger the use of a neural network model including at least one network node to perform inference calculations, to determine a second calculation processing unit corresponding to the target network node in the at least one network node, and the at least one network node uses the first linked list Is pre-stored in the first calculation processing unit, and the target network node runs on the second calculation processing unit;

A node allocation module, configured to delete the target network node from the first linked list and modify the target in the first linked list when the first calculation processing unit is different from the second calculation processing unit The connection relationship between the previous network node and the next network node of the network node, assigning the target network node to the second calculation processing unit;

The node cycle module is used to obtain the next network node of the target network node, determine the next network node as the target network node, and execute the determination of the second corresponding to the target network node among the at least one network node Steps of calculation processing unit;

The result generation module is used to generate the inference calculation result when it is determined that there is no next network node.

In a third aspect, an embodiment of the present application provides a computer storage medium, the computer storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the above method steps.

In a fourth aspect, an embodiment of the present application provides an electronic device, which may include a processor and a memory; wherein the memory stores a computer program, and the computer program is adapted to be loaded by the processor and execute the above method steps .

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work.

FIG. 1 is a schematic flowchart of a method for processing a network node according to an embodiment of the present application;

2 is a schematic diagram of an example of a linked list structure provided by an embodiment of the present application;

FIG. 3 is a schematic diagram of an example of deleting a network node in a first linked list according to an embodiment of the present application;

4 is a schematic diagram of an example of a linked list structure after deleting a network node in a first linked list according to an embodiment of the present application;

FIG. 5 is a schematic diagram of an example of adding a network node to a second linked list according to an embodiment of the present application;

6 is a schematic diagram of an example of the structure of a second linked list after adding a network node according to an embodiment of the present application;

FIG. 7 is a schematic flowchart of a method for processing a network node according to an embodiment of the present application;

FIG. 8 is a schematic structural diagram of a network node processing device provided by an embodiment of the present application;

FIG. 9 is a schematic structural diagram of a network node processing device provided by an embodiment of the present application;

FIG. 10 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

detailed description

In order to make the purpose, technical solutions, and advantages of the present application clearer, the following will further describe the embodiments of the present application in detail with reference to the accompanying drawings.

When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings indicate the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

In the description of this application, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above-mentioned terms in this application can be understood under specific circumstances. In addition, in the description of this application, unless otherwise specified, "plurality" means two or more. "And/or" describes the association relationship of the associated objects, indicating that there can be three types of relationships, for example, A and/or B, which can mean: A alone exists, A and B exist at the same time, and B exists alone. The character "/" generally indicates that the associated objects before and after are in an "or" relationship.

In the process of implementing the deletion or addition of the above network nodes, every time a network node is deleted or added to the container, all network nodes behind it in the container need to be shifted, especially when the number of network nodes is large, every time Network structure adjustments require a long time, which reduces the efficiency of inference calculations using neural network models.

In order to solve the above technical problems, the network node processing method provided by the embodiments of the present application will be introduced in detail below in conjunction with FIG. 1 to FIG. 7. The method can be realized by relying on a computer program, and can be run on a network node processing device based on the von Neumann system. The computer program can be integrated in the application or run as an independent tool application. Among them, the network node processing device in the embodiment of the present application may be a user terminal, and the user terminal includes, but is not limited to: a smart phone, a personal computer, a tablet computer, a handheld device, a vehicle-mounted device, a wearable device, a computing device, or Other processing equipment for wireless modems, etc.

Refer to FIG. 1, which is a schematic flowchart of a method for processing a network node according to an embodiment of this application. As shown in Figure 1, the method of the embodiment of the present application may include the following steps:

S101. Trigger using a neural network model including at least one network node to perform inference calculations, and determine a second calculation processing unit corresponding to a target network node in the at least one network node, and the at least one network node pre-exists in the form of a first linked list A first calculation processing unit, where the target network node runs on the second calculation processing unit;

Neural network is a complex network system formed by a large number of simple processing units (neurons) widely connected to each other. Neural networks have large-scale parallelism, distributed storage and processing, self-organization, self-adaptation, and self-learning capabilities. They are particularly suitable for processing inaccurate and fuzzy information processing problems that need to consider many factors and conditions at the same time. The neural network model is described based on the mathematical model of neurons.

Among them, each neuron is a network node, also called an operator.

When the neural network model needs to be used for inference calculations, each network is dynamically determined according to the load rate F of each calculation processing unit (CPU, GPU, DSP, etc.) and the execution time t of the network node in each calculation processing unit. Which processing unit the node needs to be allocated to run, therefore, the network structure of the neural network model needs to be modified. In at least one network node, any network node is the target network node, and all network nodes can be allocated in the same manner.

Specific modifications can include: dividing the network and distributing network nodes to various computing processing units; according to the characteristics of the computing processing unit, a certain network node needs to be deleted before entering a certain processing unit; according to the characteristics of the computing processing unit, Combine certain network nodes, etc.

It should be noted that, in the embodiment of the present application, before triggering the neural network model to perform inference calculation, each network node of the neural network model is pre-stored in the first calculation processing unit, and is pre-stored in the form of a first linked list.

The first calculation processing unit may be any processing unit of the user terminal, which is pre-configured by the system, such as a CPU.

A linked list is a non-contiguous, non-sequential storage structure on a physical storage unit, and the logical order of data elements is achieved through the link order of pointers in the linked list. The linked list is composed of a series of nodes (each element in the linked list is called a network node), and the network node can be dynamically generated at runtime.

Figure 2 shows a schematic diagram of a linked list structure, which includes multiple network nodes. Each network node includes two parts: one is the data field data storing data elements, and the other is the pointer that stores the address of the next node. Domain next. Since it is not necessary to store in order, the linked list can reach O(1) complexity when inserting.

Currently, it is necessary to determine to which second calculation processing unit each network node on the first calculation processing unit is specifically allocated to run. The second calculation processing unit and the first calculation processing unit may be the same or different. The second calculation processing unit may include at least one. For example, if the user terminal includes three calculation processing units: CPU, GPU, and DSP, the CPU is the first calculation processing unit. The processing unit, the second calculation processing unit may be a CPU, a GPU, or a DSP.

It is understandable that for multiple network nodes included in the neural network model, the second calculation processing unit to which each network node belongs can be determined first, and then node allocation can be performed, or the second calculation described by one network node can be determined first The processing unit then allocates and allocates the next network node in the same manner after the allocation of the network node is completed. Among them, the second calculation processing unit to which each network node belongs is the carrier (second calculation processing unit) that each network node is about to run. In the embodiment of the present application, description is made by taking the sequential allocation of each network node as an example.

S102: When the first calculation processing unit is different from the second calculation processing unit, delete the target network node from the first linked list and modify the upper limit of the target network node in the first linked list. The connection relationship between one network node and the next network node, assigning the target network node to the second calculation processing unit;

When the first calculation processing unit is the same as the second calculation processing unit, it indicates that the node is still on the original calculation processing unit and does not need to be moved to another calculation processing unit. When the first calculation processing unit is different from the second calculation processing unit, the network node needs to be deleted from the first calculation processing unit and added to the second calculation processing unit.

If the target network node is q, and it is determined that the network node needs to be allocated to a second calculation processing unit different from the first calculation processing unit, then q needs to be deleted from the first linked list. For example, as shown in Figure 3, the value of q The previous node is p and the next node is s. By setting p->next=q->next; free q, you can make p point to s and delete q, while other network nodes remain unchanged. The structure of the first linked list is shown in Figure 4.

Of course, it is also necessary to insert the deleted q into the second calculation processing unit. The network nodes are also stored in the form of a linked list (second linked list) on the second calculation processing unit. When q is the first network node allocated to the second calculation processing unit, q can be directly placed in the first node in the second linked list. When q needs to be inserted between two network nodes on the second linked list, as shown in Figure 5, if q needs to be inserted between node o and node r on the second linked list, set q->next= r->next; o->next=q, so that q is inserted, while other network nodes remain unchanged, and the structure of the second linked list after insertion is shown in Figure 6.

S103. Obtain the next network node of the target network node, determine the next network node as the target network node, and execute the determination of the second calculation processing unit corresponding to the target network node in the at least one network node. step;

For example, as shown in Figure 3, if the target network node is q, then the next network node of the target network node is s. After s is determined as the target network node, it is processed in the same way as q, and processed accordingly The remaining network nodes.

S104: When it is determined that there is no next network node, a reasoning calculation result is generated.

When it is determined that there is no next network node, it indicates that all network nodes stored in the first linked list have been allocated. Therefore, the first calculation processing unit and the second calculation processing unit can be used to run each network node at the same time, and the inference calculation is obtained result. Or it can be understood that after each network node is allocated, run the network node immediately, and use the operation result as the input of the next network node, until the last network node on each processing unit is completed, and the operation results are combined to generate The final inference calculation result.

For example, the neural network model includes 30 network nodes. These network nodes are numbered from 1 to 30 in order. If 1 to 10 are allocated to the CPU, 11 to 20 are allocated to the GPU, and 21 to 30 are allocated to the DSP. You can run the node after the allocation of node 1 is completed, then allocate node 2, and use the operation result of node 1 as the input of node 2, and so on, output the first inference calculation result after node 10 is completed , Run 11-20 and 21-30 respectively in the same way, output the second inference calculation result after the 20 operation is completed, and output the third inference calculation result after the 30 operation is completed, and then according to the first inference calculation result and the second inference calculation result The calculation result and the third inference calculation result obtain the final inference settlement result. At this time, the operation of the neural network model ends.

For another example, the neural network model includes 30 network nodes. These network nodes are numbered from 1 to 30 in sequence. If 1 to 10 are allocated to the CPU, 11 to 20 are allocated to the GPU, and 21 to 30 are allocated to the DSP. , After all the nodes are allocated, the nodes on different calculation processing units are run at the same time, and finally the combination on each calculation processing unit generates the final inference settlement result.

In the embodiment of the present application, the neural network model including at least one network node is used to perform inference calculations, and the second calculation processing unit corresponding to the target network node in the at least one network node is determined, and the at least one network node adopts the first The form of the linked list is pre-stored in the first calculation processing unit and the target network node runs on the second calculation processing unit. When the first calculation processing unit is different from the second calculation processing unit, the target network The node deletes and modifies the connection relationship between the previous network node and the next network node of the target network node in the first linked list from the first linked list, and assigns the target network node to the second calculation process Unit, and process other network nodes in the same way. When all network nodes are processed, the inference calculation result is generated. Each network node is expressed in the form of a linked list, that is, each network node in the neural network model is connected in a linked list. Every time the network structure is adjusted, when the network node is frequently inserted and deleted, even if the number of network nodes is large, only The connection relationship between the previous network node and the next network node of the network node needs to be modified without frequent shift operations, which saves time for network structure adjustment, and can improve the efficiency of inference calculations using neural network models.

Refer to FIG. 7, which is a schematic flowchart of a method for processing a network node according to an embodiment of this application. In this embodiment, the network node processing method is applied to a smart phone as an example. The network node processing method may include the following steps:

S201: Trigger to use a neural network model including at least one network node to perform inference calculation, and the at least one network node pre-stores in the first calculation processing unit in the form of a first linked list;

For details, please refer to S101, which will not be repeated here.

S202: Obtain the current load rate of each calculation processing unit and the execution time of the target network node on each calculation processing unit.

The load factor F refers to the ratio of the actual load of the transformer to its capacity, which is used to reflect the load-bearing capacity of the transformer and whether its operating curve lies between the optimal 75-80%. The current load rate is the load rate of each calculation processing unit when the neural network model including at least one network node is used for inference calculations.

Assuming that the computing processing units of a smartphone include CPU, GPU, and DSP, the execution time of the target network node on these three computing processing units is t1, t2, and t3, respectively, and the current load rate of each computing processing unit is F1, F2, respectively And F3.

S203, based on the current load rate and the execution time, calculate the respective execution expectations of the target network node in the respective calculation processing units;

Implementation expectation E(device)=epsilon*F/t, where epsilon is the correction coefficient, which is a constant that can be modified according to experimental conditions.

According to the above formula, the execution expectations of the target network node on the CPU, GPU, and DSP are: E1(device)=epsilon*F1/t1, E2(device)=epsilon*F2/t2, E3(device)=epsilon* F3/t3.

S204: Determine a minimum execution expectation among the execution expectations of each calculation processing unit, and determine the calculation processing unit indicated by the minimum execution expectation as a second calculation processing unit, where the second calculation processing unit includes a second linked list;

Specifically, find the smallest value among E1, E2, and E3. If E1 is the smallest, determine that the device corresponds to the CPU, use the CPU as the second calculation processing unit, and allocate the target network node to the CPU. Of course, if the first calculation processing unit is also a CPU, it indicates that the target network node still needs to run on the first calculation processing unit.

Optionally, if the second calculation processing unit does not support the target network node, the target network node needs to be replaced with another equivalent network node.

S205: When the first calculation processing unit is different from the second calculation processing unit, control the previous network node of the target network node in the first linked list to point to the next network node of the target network node, And release the target network node;

Pointing along the linked list, the network node that the target network node points to is the next network node, and the network node that points to the target network node is the previous network node. As shown in Figure 3, q is the target network node, s is the next network node, and p is the previous network node.

Setting p->next=q->next; free q can make p point to s, thereby releasing q.

S206, controlling the target network node in the second linked list to point to the next network node of the target network node, and controlling the previous network node of the target network node to point to the target network node;

As shown in Figure 5, if q needs to be inserted between node o and node r on the second linked list, set q->next=r->next; o->next=q.

S207: Execute the target network node in the second calculation processing unit;

After inserting q into the second linked list, q can be executed to generate the corresponding running result.

Optionally, before executing the target network node, according to the characteristics of the second calculation processing unit, auxiliary nodes can be added and/or deleted to the second linked list, combined to generate a new network node, and then run.

For example, the neural network model includes 30 network nodes. These network nodes are numbered from 1 to 30 in order. If 1 to 10 are allocated to the CPU, 11 to 20 are allocated to the GPU, and 21 to 30 are allocated to the DSP. After the allocation of node 1 is completed, if you need to add auxiliary nodes to node 1 according to the characteristics of the CPU, and combine them into a new network node to run, and then allocate node 2.

S208. Acquire the next network node of the target network node, determine the next network node as the target network node, and execute the acquisition of the current load rate of each calculation processing unit, and the target network node is located in each of the target network nodes. The steps of calculating the execution time on each processing unit;

For details, refer to S103, which will not be repeated here.

S209: When it is determined that there is no next network node, add and/or delete auxiliary nodes to the first linked list, and/or add and/or delete auxiliary nodes to the second linked list, and generate a reasoning calculation result.

When it is determined that the next network node does not exist, it indicates that all network nodes stored in the first linked list are allocated. It is necessary to add or delete auxiliary nodes to the first linked list and the second linked list according to the characteristics of the calculation processing unit, or to add or delete some Several network nodes are merged to form a new network node to run the neural network model. After the addition or deletion of auxiliary nodes is completed, the first calculation processing unit and the second calculation processing unit can be used to run each network node and obtain Inference calculation results.

For example, the neural network model includes 30 network nodes. These network nodes are numbered from 1 to 30 in order. If 1 to 10 are allocated to the CPU, 11 to 20 are allocated to the GPU, and 21 to 30 are allocated to the DSP. 1 to 10 form the first linked list, 11 to 20 form the second linked list, and 21 to 30 form the third linked list. After all network nodes are allocated, auxiliary nodes are added or deleted from the three linked lists, or several nodes are merged. Then each network node on the three linked lists is executed separately to generate the final inference calculation result.

In the embodiment of the present application, the neural network model including at least one network node is used to perform inference calculations, and the second calculation processing unit corresponding to the target network node in the at least one network node is determined, and the at least one network node adopts the first The form of the linked list is pre-stored in the first calculation processing unit and the target network node runs on the second calculation processing unit. When the first calculation processing unit is different from the second calculation processing unit, the target network The node deletes and modifies the connection relationship between the previous network node and the next network node of the target network node in the first linked list from the first linked list, and assigns the target network node to the second calculation process Unit, and process other network nodes in the same way. When all network nodes are processed, the inference calculation result is generated. Express each network node in the form of a linked list, that is, connect each network node in the neural network model in a linked list manner, which can support dynamic device switching, optimize the corresponding network structure storage method, and frequently perform network structure adjustments every time. When inserting and deleting network nodes, even if the number of network nodes is large, only the connection relationship between the previous network node and the next network node of the network node needs to be modified, without frequent shift operations, which saves time for network structure adjustment. Thereby, the efficiency of inference calculation using the neural network model can be improved.

The following are device embodiments of this application, which can be used to execute the method embodiments of this application. For details not disclosed in the device embodiment of this application, please refer to the method embodiment of this application.

Please refer to FIG. 8, which shows a schematic structural diagram of a network node processing apparatus provided by an exemplary embodiment of the present application. The network node processing device can be implemented as all or a part of the user terminal through software, hardware or a combination of the two. The device 1 includes a node determination module 10, a node allocation module 20, a node circulation module 30, and a result generation module 40.

The node determining module 10 is configured to trigger the use of a neural network model including at least one network node to perform inference calculations, and determine the second calculation processing unit corresponding to the target network node in the at least one network node, and the at least one network node adopts the first The form of a linked list is pre-stored in the first calculation processing unit, and the target network node runs on the second calculation processing unit;

The node allocation module 20 is configured to delete the target network node from the first linked list and modify the target network node when the first calculation processing unit is different from the second calculation processing unit. The connection relationship between the previous network node and the next network node of the target network node, assigning the target network node to the second calculation processing unit;

The node cycle module 30 is configured to obtain the next network node of the target network node, determine the next network node as the target network node, and perform the determination of the first network node corresponding to the target network node in the at least one network node 2. Steps of calculating the processing unit;

The result generation module 40 is used to generate the inference calculation result when it is determined that there is no next network node.

Optionally, the node determining module 10 is specifically configured to:

Acquiring the current load rate of each calculation processing unit and the execution time of the target network node on each calculation processing unit;

Based on the current load rate and the execution time, a second calculation processing unit corresponding to the target network node is determined in each calculation processing unit.

Optionally, the node determining module 10 is specifically configured to:

Based on the current load rate and the execution time, calculating the execution expectations of the target nodes in the respective calculation processing units;

The minimum execution expectation among the execution expectations of the calculation processing units is determined, and the calculation processing unit indicated by the minimum execution expectation is determined as the second calculation processing unit.

Optionally, the node allocation module 20 is specifically configured to:

Control the previous network node of the target network node in the first linked list to point to the next network node of the target network node, and release the target network node.

Optionally, the node allocation module 20 is specifically configured to:

The target network node is added to the second linked list of the second calculation processing unit, and the connection relationship between the previous network node and the next network node of the target network node in the second linked list is modified.

Optionally, the node allocation module 20 is specifically configured to:

Control the target network node in the second linked list to point to the next network node of the target network node, and control the previous network node of the target network node to point to the target network node.

Optionally, as shown in FIG. 9, the device further includes an auxiliary node increase/decrease module 50, configured to:

Add and/or delete auxiliary nodes to the first linked list; and/or,

Add and/or delete auxiliary nodes to the second linked list

Optionally, as shown in FIG. 9, the device further includes a node execution module 60, configured to:

The target network node is executed in the second calculation processing unit.

It should be noted that, in the network node processing method of the network node processing device provided in the above embodiment, only the division of the above functional modules is used as an example for illustration. In actual applications, the above functions can be allocated to different functional modules according to needs. Complete, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above. In addition, the network node processing device provided in the foregoing embodiment and the network node processing method embodiment belong to the same concept, and the implementation process is detailed in the method embodiment, which will not be repeated here.

The serial numbers of the foregoing embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

In the embodiment of the present application, the neural network model including at least one network node is used to perform inference calculations, and the second calculation processing unit corresponding to the target network node in the at least one network node is determined, and the at least one network node adopts the first The form of the linked list is pre-stored in the first calculation processing unit and the target network node runs on the second calculation processing unit. When the first calculation processing unit is different from the second calculation processing unit, the target network The node deletes and modifies the connection relationship between the previous network node and the next network node of the target network node in the first linked list from the first linked list, and assigns the target network node to the second calculation process Unit, and process other network nodes in the same way. When all network nodes are processed, the inference calculation result is generated. Express each network node in the form of a linked list, that is, connect each network node in the neural network model in a linked list manner, which can support dynamic device switching, optimize the corresponding network structure storage method, and frequently perform network structure adjustments every time. When inserting and deleting network nodes, even if the number of network nodes is large, only the connection relationship between the previous network node and the next network node of the network node needs to be modified, without frequent shift operations, which saves time for network structure adjustment. Thereby, the efficiency of inference calculation using the neural network model can be improved.

The embodiment of the present application also provides a computer storage medium, the computer storage medium may store a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the method steps of the embodiment shown in FIG. 1 to FIG. 7 above. For the specific execution process, please refer to the specific description of the embodiment shown in FIG. 1 to FIG. 7, which will not be repeated here.

Please refer to FIG. 10, which provides a schematic structural diagram of an electronic device according to an embodiment of the present application. As shown in FIG. 10, the electronic device 1000 may include: at least one processor 1001, at least one network interface 1004, a user interface 1003, a memory 1005, and at least one communication bus 1002.

Among them, the communication bus 1002 is used to implement connection and communication between these components.

The user interface 1003 may include a display (Display) and a camera (Camera), and the optional user interface 1003 may also include a standard wired interface and a wireless interface.

Among them, the network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

The processor 1001 may include one or more processing cores. The processor 1001 uses various excuses and lines to connect various parts of the entire electronic device 1000, and executes by running or executing instructions, programs, code sets, or instruction sets stored in the memory 1005, and calling data stored in the memory 1005. Various functions and processing data of the electronic device 1000. Optionally, the processor 1001 may adopt at least one of digital signal processing (Digital Signal Processing, DSP), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA), and Programmable Logic Array (Programmable Logic Array, PLA). A kind of hardware form to realize. The processor 1001 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), a modem, and the like. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is used to render and draw the content that needs to be displayed on the display screen; the modem is used to process wireless communication. It is understandable that the above-mentioned modem may not be integrated into the processor 1001, but may be implemented by a chip alone.

The memory 1005 may include random access memory (RAM) or read-only memory (Read-Only Memory). Optionally, the memory 1005 includes a non-transitory computer-readable storage medium. The memory 1005 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 1005 may include a program storage area and a data storage area, where the program storage area may store instructions for implementing the operating system and instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), Instructions used to implement the foregoing method embodiments, etc.; the storage data area can store data and the like involved in the foregoing method embodiments. Optionally, the memory 1005 may also be at least one storage device located far away from the foregoing processor 1001. As shown in FIG. 10, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a network node processing application program.

In the electronic device 1000 shown in FIG. 10, the user interface 1003 is mainly used to provide an input interface for the user to obtain data input by the user; and the processor 1001 can be used to call a network node processing application stored in the memory 1005, and Specifically perform the following operations:

Trigger using a neural network model including at least one network node to perform inference calculations, and determine a second calculation processing unit corresponding to the target network node in the at least one network node, and the at least one network node is pre-stored in the first linked list in the form of a first linked list. A calculation processing unit, where the target network node runs on the second calculation processing unit;

When the first calculation processing unit is different from the second calculation processing unit, delete the target network node from the first linked list and modify the previous network of the target network node in the first linked list The connection relationship between the node and the next network node, assigning the target network node to the second calculation processing unit;

Acquiring the next network node of the target network node, determining the next network node as the target network node, and executing the step of determining the second calculation processing unit corresponding to the target network node in the at least one network node;

When it is determined that there is no next network node, the inference calculation result is generated.

In an embodiment, the processor 1001 specifically executes the following operations when determining the second calculation processing unit corresponding to the target network node in the at least one network node:

Acquiring the current load rate of each calculation processing unit and the execution time of the target network node on each calculation processing unit;

Based on the current load rate and the execution time, a second calculation processing unit corresponding to the target network node is determined in each calculation processing unit.

In an embodiment, the processor 1001 specifically executes the second calculation processing unit corresponding to the target network node in the calculation processing units based on the current load rate and the execution time. The following operations:

Based on the current load rate and the execution time, calculating the execution expectations of the target nodes in the respective calculation processing units;

The minimum execution expectation among the execution expectations of the calculation processing units is determined, and the calculation processing unit indicated by the minimum execution expectation is determined as the second calculation processing unit.

In one embodiment, the processor 1001 deletes the target network node from the first linked list and modifies the previous network node and the next network node of the target network node in the first linked list. For the connection relationship, perform the following operations:

Control the previous network node of the target network node in the first linked list to point to the next network node of the target network node, and release the target network node.

In an embodiment, the processor 1001 specifically executes the following operations when executing the allocation of the target network node to the second calculation processing unit:

The target network node is added to the second linked list of the second calculation processing unit, and the connection relationship between the previous network node and the next network node of the target network node in the second linked list is modified.

In one embodiment, the processor 1001 adds the target network node to the second linked list of the second calculation processing unit, and modifies the previous one of the target network node in the second linked list. When connecting a network node and the next network node, perform the following operations:

Control the target network node in the second linked list to point to the next network node of the target network node, and control the previous network node of the target network node to point to the target network node.

In an embodiment, the processor 1001 further performs the following operations before generating the inference calculation result:

Add and/or delete auxiliary nodes to the first linked list; and/or,

Adding and/or deleting auxiliary nodes to the second linked list.

In an embodiment, the processor 1001 further executes the following operations after executing the allocation of the target network node to the second calculation processing unit:

The target network node is executed in the second calculation processing unit.

In the embodiment of the present application, the neural network model including at least one network node is used to perform inference calculations, and the second calculation processing unit corresponding to the target network node in the at least one network node is determined, and the at least one network node adopts the first The form of the linked list is pre-stored in the first calculation processing unit and the target network node runs on the second calculation processing unit. When the first calculation processing unit is different from the second calculation processing unit, the target network The node deletes and modifies the connection relationship between the previous network node and the next network node of the target network node in the first linked list from the first linked list, and assigns the target network node to the second calculation process Unit, and process other network nodes in the same way. When all network nodes are processed, the inference calculation result is generated. Express each network node in the form of a linked list, that is, connect each network node in the neural network model in a linked list manner, which can support dynamic device switching, optimize the corresponding network structure storage method, and frequently perform network structure adjustments every time. When inserting and deleting network nodes, even if the number of network nodes is large, only the connection relationship between the previous network node and the next network node of the network node needs to be modified, without frequent shift operations, which saves time for network structure adjustment. Thereby, the efficiency of inference calculation using the neural network model can be improved.

A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing relevant hardware through a computer program. The program can be stored in a computer readable storage medium, and the program can be stored in a computer readable storage medium. During execution, it may include the procedures of the above-mentioned method embodiments. Wherein, the storage medium can be a magnetic disk, an optical disc, a read-only storage memory or a random storage memory, etc.

The above-disclosed are only preferred embodiments of this application, and of course the scope of rights of this application cannot be limited by this. Therefore, equivalent changes made in accordance with the claims of this application still fall within the scope of this application.

Claims (20)

A method for processing network nodes, characterized in that the method includes: Trigger using a neural network model including at least one network node to perform inference calculations, and determine a second calculation processing unit corresponding to the target network node in the at least one network node, and the at least one network node is pre-stored in the first linked list in the form of a first linked list. A calculation processing unit, where the target network node runs on the second calculation processing unit; When the first calculation processing unit is different from the second calculation processing unit, delete the target network node from the first linked list and modify the previous network of the target network node in the first linked list The connection relationship between the node and the next network node, assigning the target network node to the second calculation processing unit; Acquiring the next network node of the target network node, determining the next network node as the target network node, and executing the step of determining the second calculation processing unit corresponding to the target network node in the at least one network node; When it is determined that there is no next network node, the inference calculation result is generated. The method according to claim 1, wherein the determining a second calculation processing unit corresponding to a target network node in the at least one network node comprises: Acquiring the current load rate of each calculation processing unit and the execution time of the target network node on each calculation processing unit; Based on the current load rate and the execution time, a second calculation processing unit corresponding to the target network node is determined in each calculation processing unit. The method according to claim 2, wherein the determining, in each of the calculation processing units, a second calculation processing unit corresponding to the target network node based on the current load rate and the execution time, comprises : Based on the current load rate and the execution time, calculating the execution expectations of the target nodes in the respective calculation processing units; The minimum execution expectation among the execution expectations of the calculation processing units is determined, and the calculation processing unit indicated by the minimum execution expectation is determined as the second calculation processing unit. The method according to claim 1, wherein said deleting said target network node from said first linked list and modifying the previous network node and the next one of said target network node in said first linked list The connection relationship of network nodes includes: Control the previous network node of the target network node in the first linked list to point to the next network node of the target network node, and release the target network node. The method according to claim 1, wherein the allocating the target network node to the second calculation processing unit comprises: The target network node is added to the second linked list of the second calculation processing unit, and the connection relationship between the previous network node and the next network node of the target network node in the second linked list is modified. The method according to claim 5, wherein the adding the target network node to a second linked list of the second calculation processing unit, and modifying the target network node in the second linked list The connection relationship between the previous network node and the next network node includes: Control the target network node in the second linked list to point to the next network node of the target network node, and control the previous network node of the target network node to point to the target network node. The method according to claim 6, characterized in that, before said generating the inference calculation result, it further comprises: Add and/or delete auxiliary nodes to the first linked list; and/or, Adding and/or deleting auxiliary nodes to the second linked list. The method according to claim 1, wherein after the allocating the target network node to the second calculation processing unit, the method further comprises: The target network node is executed in the second calculation processing unit. The method according to claim 1, wherein the method further comprises: If the second calculation processing unit does not support the target network node, obtain an equivalent network node of the target network node, and use the equivalent network node to replace the target network node. A network node processing device, characterized in that the device includes: The node determination module is configured to trigger the use of a neural network model including at least one network node to perform inference calculations, to determine a second calculation processing unit corresponding to the target network node in the at least one network node, and the at least one network node uses the first linked list Is pre-stored in the first calculation processing unit, and the target network node runs on the second calculation processing unit; A node allocation module, configured to delete the target network node from the first linked list and modify the target in the first linked list when the first calculation processing unit is different from the second calculation processing unit The connection relationship between the previous network node and the next network node of the network node, assigning the target network node to the second calculation processing unit; The node cycle module is used to obtain the next network node of the target network node, determine the next network node as the target network node, and execute the determination of the second corresponding to the target network node among the at least one network node Steps of calculation processing unit; The result generation module is used to generate the inference calculation result when it is determined that there is no next network node. The device according to claim 10, wherein the node determining module is specifically configured to: Acquiring the current load rate of each calculation processing unit and the execution time of the target network node on each calculation processing unit; Based on the current load rate and the execution time, a second calculation processing unit corresponding to the target network node is determined in each calculation processing unit. The device according to claim 11, wherein the node determining module is specifically configured to: Based on the current load rate and the execution time, calculating the execution expectations of the target nodes in the respective calculation processing units; The minimum execution expectation among the execution expectations of the calculation processing units is determined, and the calculation processing unit indicated by the minimum execution expectation is determined as the second calculation processing unit. The device according to claim 101, wherein the node allocation module is specifically configured to: Control the previous network node of the target network node in the first linked list to point to the next network node of the target network node, and release the target network node. The device according to claim 10, wherein the node allocation module is specifically configured to: The target network node is added to the second linked list of the second calculation processing unit, and the connection relationship between the previous network node and the next network node of the target network node in the second linked list is modified. The device according to claim 14, wherein the node allocation module is specifically configured to: Control the target network node in the second linked list to point to the next network node of the target network node, and control the previous network node of the target network node to point to the target network node. The device according to claim 15, wherein the device further comprises an auxiliary node increase or decrease module, configured to: Add and/or delete auxiliary nodes to the first linked list; and/or, Adding and/or deleting auxiliary nodes to the second linked list. The device according to claim 10, wherein the device further comprises a node execution module for: The target network node is executed in the second calculation processing unit. The device according to claim 10, wherein the device further comprises a node replacement module for: If the second calculation processing unit does not support the target network node, obtain an equivalent network node of the target network node, and use the equivalent network node to replace the target network node. A computer storage medium, wherein the computer storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the method steps according to any one of claims 1-9. An electronic device, comprising: a processor and a memory; wherein the memory stores a computer program, and the computer program is adapted to be loaded by the processor and executed as claimed in any one of claims 1-9. Method steps.

Images