WO2021235312A1 - Information processing device, and information processing method - Google Patents

Abstract

An information processing device (100) is applied to an information processing system which utilizes an inferred result obtained by an inferrer employing a neural network, and is provided with an acquiring unit (113), a calculating unit (114), and a determining unit (115). The acquiring unit (113) acquires the overall resource usage amount of the information processing system. The calculating unit (114) calculates, on the basis of the resource usage amount, a target resource usage amount to be allocated to at least a portion of the calculations in the inference processing performed by the inferrer. The determining unit (115) determines a calculation method corresponding to the target resource usage amount.

Description

Information processing equipment and information processing method

This disclosure relates to an information processing device and an information processing method.

Conventionally, when an increase in data processing load is predicted, technologies for selectively removing input data, delaying data processing, and offloading data processing to other systems have been proposed.

In recent years, machine learning algorithms have rapidly evolved by using artificial neural networks such as DNN (Deep Neural Network), and processing using DNN includes image recognition, voice recognition, and artificial intelligence. Widely applied in the field.

Japanese Unexamined Patent Publication No. 2012-43409

However, while the processing using DNN has high accuracy, the processing load related to the calculation is large, and a large amount of resources may be required. Therefore, it is conceivable to apply the above-mentioned conventional technique to the system that operates the DNN process so that the resource usage of the entire system does not exceed the maximum amount allowed in the system. That is, by applying the above-mentioned conventional technique, when it is predicted that the resource usage of the entire system exceeds the maximum amount, it is conceivable to perform selective removal / delay / offload in data processing. However, problems such as deterioration of real-time performance and output quality of data processing itself may occur. Therefore, it is difficult to select the application of the above-mentioned prior art for the purpose of adjusting the resource usage of the entire system so as not to exceed the maximum amount allowed in the system.

Therefore, in the present disclosure, we propose an information processing device and an information processing method that can adjust the amount of resources used for DNN processing so as not to cause an excess of the amount of resources used.

In order to solve the above problems, the information processing device of one form according to the present disclosure is an information processing device applied to an information processing system that uses an inference result by an inference device using a neural network, and is an acquisition unit. , A calculation unit, and a determination unit. The acquisition unit acquires the total resource usage of the information processing system. The calculation unit calculates the target resource usage to be allocated to at least a part of the calculation of the inference processing by the inference device based on the resource usage. The decision unit determines the calculation method corresponding to the target resource usage.

It is a figure which shows an example of information processing which concerns on embodiment of this disclosure. It is a figure which shows the schematic structure example of the information processing system which concerns on embodiment of this disclosure. It is a figure which shows the structural example of the DNN model which concerns on embodiment of this disclosure. It is a figure which shows the outline of the DNN partial inference device which concerns on embodiment of this disclosure. It is a figure which shows the outline of the DNN partial inference device which concerns on embodiment of this disclosure. It is a figure which shows the outline of the DNN inference device which concerns on embodiment of this disclosure. It is a functional block diagram which shows an example of the functional structure of the monitoring / adjustment module which concerns on embodiment of this disclosure. It is a figure which shows the outline of the monitoring / adjustment module which concerns on embodiment of this disclosure. It is a figure which shows the outline of the monitoring / adjustment module which concerns on embodiment of this disclosure. It is a figure which shows the outline of the determination method of the calculation method based on the preliminary analysis result which concerns on embodiment of this disclosure. It is a figure which shows the outline of the notification operation which concerns on embodiment of this disclosure. It is a flowchart which shows an example of the processing procedure by the monitoring / adjustment module which concerns on embodiment of this disclosure. It is a figure which shows the outline of the determination method of the calculation method based on the preliminary analysis result which concerns on a modification. It is a figure which shows the structural example of the DNN inference device which concerns on the modification. It is a figure which shows an example of information processing which concerns on a comparative example. It is a figure which shows an example of the information processing by the DNN inference device which concerns on embodiment of this disclosure. It is a figure which shows the relationship between the elapsed time and the resource use amount about the information processing which concerns on a comparative example. It is a figure which shows the relationship between the elapsed time and the resource use amount about the information processing by the DNN inference device which concerns on embodiment of this disclosure. It is a figure which shows an example of information processing by an existing technology.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In each of the following embodiments, duplicate explanations may be omitted by assigning the same numbers or reference numerals to the same parts. Further, in the present specification and the drawings, a plurality of components having substantially the same functional configuration may be distinguished by adding a different number or reference numeral after the same number or reference numeral.

In addition, the present disclosure will be described according to the order of items shown below.
1. 1. An example of information processing according to the embodiment of the present disclosure 2. Configuration example of information processing system 3. DNN model configuration example 4. Example of functional configuration of monitoring / adjustment module 4-1. Overview of monitoring / adjustment module 4-2. Operation of monitoring / adjustment module (1)
4-3. Operation of monitoring / adjustment module (2)
5. Example of processing procedure of monitoring / adjustment module 6. Modification example 6-1. Generation of correspondence information by machine learning 6-2. Reduction of resource switching overhead 7. Others 7-1. Application to robot systems 7-2. Application to game consoles 8. Conclusion

<< 1. An example of information processing according to an embodiment of the present disclosure >>
FIG. 1 is a diagram showing an example of information processing according to the embodiment of the present disclosure. As shown in FIG. 1, the monitoring / adjusting module 100 (an example of an information processing apparatus) according to the embodiment of the present disclosure is applied to an information processing system 1 including a DNN model 30 and a system module 50.

The information processing system 1 operates the inference of the DNN model 30 in parallel with the processing in the system module 50.

The DNN model 30 is composed of a trained model machine-learned using DNN, which is an artificial neural network having a plurality of layers such as an input layer, an output layer, and a plurality of hidden layers (intermediate layers). The DNN model 30 is composed of a plurality of layers such as an input layer, an output layer, and a plurality of hidden layers, as well as a plurality of elements such as a channel and a matrix. The DNN model 30 performs calculation processing of input data by a calculation method given for each element from the monitoring / adjustment module 100, and outputs a calculation result.

The system module 50 executes various processes using the inference result of the DNN model 30. The system module 50 is provided with, for example, a module that controls the operation of the robot based on the recognition result of the camera image by the DNN model 30, and a UI (User Interface) function that uses the recognition result of voice and hand gesture by the DNN model 30. It corresponds to a module that executes the processing of the game machine.

In such an information processing system 1, when the resource usage of the entire system is predicted to exceed the maximum amount, if selective removal / delay / offload is performed in the data processing, the real-time property and output of the data processing itself are performed. Problems such as deterioration of quality may occur. That is, if time-series data is selectively removed in data processing, important data at a certain time may be overlooked, and if processing is delayed, acquisition of processing results is delayed, and a system that uses processing results Affects the real-time nature of other functions.

For example, in a system such as a robot system where real-time performance from recognition to action is important, the delay in action decision due to oversight due to selective removal or processing delay is serious. Further, depending on the recognition process, there is a task that requires about 10 milliseconds for one data, but the control of the electric unit may be controlled at intervals of several tens to several hundreds of microseconds. In this case, even if the resource usage amount of the recognition processing is switched at the timing when the data to be recognized processing is switched, it is not possible to follow the fluctuation of the resource usage amount of the electric unit.

Further, if the system is equipped with a UI agent that operates in parallel with the game software, there may be a problem that the operation by the user cannot be recognized at all while the processing load of the game software itself is high.

In view of these problems, in the present disclosure, in a system in which DNN inference is operated in parallel with a certain process, it is possible to adjust the amount of resources used for DNN processing so as not to cause an excess of resource usage. We propose a monitoring / adjustment module 100.

The monitoring / adjustment module 100 according to the embodiment of the present disclosure can instantly adjust the amount of resources used for inference processing by DNN, for example, while monitoring the amount of resources used in the entire system. Each time the monitoring / adjusting module 100 calculates a part of elements such as layers and maps constituting the DNN, the resource usage of the entire system is acquired, and the maximum amount of resources allowed to be used in the system is exceeded. Determine the amount of resources that may be used in the next factor so as not to cause. By reducing the time interval (fineness of elements) that determines the amount of resources, it becomes possible to instantly adjust a part of the resources used for inference processing by DNN even during processing. Due to the surplus created as a result of the adjustment, the amount of resources of the entire system does not exceed the limit, and the system continues to operate without abnormal termination, abnormal operation, excess latency, etc.

Further, as shown in Reference 1, DNN has a property that the recognition accuracy of, for example, a camera image is unlikely to be lowered even if the calculation accuracy is lowered and the processing is performed with a small number of resources. Reference 1 proposes a method of suppressing the influence on the recognition accuracy when the number of quantization bits of DNN is increased or decreased. According to this method (PACT), even if the number of quantization bits is reduced in some object recognition models, it can be suppressed by a certain decrease in recognition accuracy.
Reference 1: C.I. Jungwook, “PACT: Parameterized Clipping Activation for Quantized Neural Networks”, Computer Vision and Pattern Recognition, 2018

The function realized by the monitoring / adjusting module 100 according to the embodiment of the present disclosure includes a technique for adjusting the resource usage in the middle between the elements during the inference processing by DNN. As mentioned above, DNN needs to stop inference by DNN while considering the resource usage of the entire system by paying attention to the property that the accuracy does not easily decrease even if the calculation accuracy (the amount of resources allocated to DNN) is reduced. It is possible to realize an inference device without.

Further, the monitoring / adjustment module 100 according to the embodiment of the present disclosure has a mechanism for considering that the accuracy of the DNN inference processing is not lowered as much as possible when determining the resource usage amount of each element. This is realized by analyzing in advance the relationship between the amount and the accuracy (inference accuracy) and the element that reduces the resources required for processing in the DNN model. With this technology, data processing can be performed with extremely high accuracy when there is a margin in the resource usage of the entire system, and even in a situation where there is no margin in the usage, with a certain degree of processing (recognition) accuracy. It can be guaranteed that data processing can be continued.

The outline of the information processing by the monitoring / adjustment module 100 according to the embodiment of the present disclosure will be described below.

First, the monitoring / adjustment module 100 acquires the total resource usage of the information processing system 1 (step S1). The total resource usage of the information processing system 1 corresponds to, for example, power consumption, memory usage, processing time, and the like.

Subsequently, the monitoring / adjustment module 100 calculates the target resource usage amount to be allocated to at least a part of the calculation of the inference processing by the DNN model 30 based on the total resource usage amount of the information processing system 1 (step S2). The calculation of the inference process by the DNN model 30 is composed of, for example, a series of calculations of a plurality of layers composed of multiple stages. Therefore, the monitoring / adjustment module 100 calculates the target resource usage amount for each layer constituting the inference processing of the DNN model, for example, based on the resource surplus of the information processing system 1.

After calculating the target resource usage amount, the monitoring / adjustment module 100 determines a calculation method corresponding to the target resource usage amount (step S3), and outputs the calculation method to the DNN model 30. Specifically, the monitoring / adjustment module 100 determines the calculation method based on the correspondence information between the resource usage amount and the calculation method pre-analyzed for each resource usage amount. The calculation method is determined based on "quantization bit number", "Pruning ratio", etc., which are control information indicating how the calculation of the inference process by the DNN model 30 is performed.

In this way, the monitoring / adjustment module 100 calculates the target resource usage amount to be allocated to at least a part of the calculation of the inference processing by the DNN model 30 based on the total resource usage amount of the information processing system 1. Then, the monitoring / adjustment module 100 determines a calculation method corresponding to the calculated target resource usage amount. Thereby, the monitoring / adjusting module 100 can adjust the resource amount used for the inference processing of the DNN model 30 so as not to cause the resource usage amount to be exceeded in the information processing system 1.

<< 2. Information processing system configuration example >>
Hereinafter, a configuration example of the information processing system 1 according to the embodiment of the present disclosure will be described. FIG. 2 is a diagram showing a schematic configuration example of an information processing system according to an embodiment of the present disclosure.

As shown in FIG. 2, the information processing system 1 communicates with a processor 11, a main storage device 12, an auxiliary storage device 13, a peripheral circuit 14, an input device 15, an output device 16, and a peripheral device 17. A device 18 is provided. The processor 11, the main storage device 12, the auxiliary storage device 13, the peripheral circuit 14, and the communication device 18 are connected to each other via the internal bus 20.

The processor 11 is realized by, for example, a processor such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or a GPU (Graphics Processing Unit). The processor 11 executes arithmetic processing and operation control in the information processing system 1.

The main storage device 12 is realized by a semiconductor memory element such as a RAM (Random Access Memory). The auxiliary storage device 13 is realized by a semiconductor memory element such as a ROM (Read Only Memory) or a storage device such as a hard disk or an optical disk.

The peripheral circuit 14 is realized by an A / D converter, a timer, a signal processing circuit, or the like. The peripheral circuit 14 processes various signals and data of the input device 15, the output device 16, and the peripheral device 17.

The input device 15 is realized by, for example, a mouse, a keyboard, a touch panel, buttons, switches, levers, and the like. Further, the input device 15 can also be realized by a voice input device such as a remote controller or a microphone capable of transmitting a control signal using infrared rays or other radio waves.

The output device 16 can be realized by a display device such as a CRT (Cathode Ray Tube), an LCD (Liquid Crystal Display), or an organic EL, and an audio output device such as a speaker or a headphone. Further, the output device 16 can also be realized by a device such as a printer, a mobile phone, or a facsimile that can visually or audibly notify the user of the acquired information.

The peripheral device 17 is another device mounted on the information processing system 1 other than the input device 15 and the output device 16. The peripheral device 17 can be realized by, for example, various sensors such as an acceleration sensor and an angular velocity sensor, an inertial measurement unit, a ToF (Time of Flight) sensor, a GPS (Global Positioning System), an actuator, a camera, a speaker, a battery, and the like.

The communication device 18 can be realized by a wireless module such as a NIC (Network Interface Card), various communication modems, Bluetooth (registered trademark) and Wi-Fi (registered trademark).

In the various processes by the monitoring / adjusting module 100 shown in FIG. 1, the processor 11 shown in FIG. 2 executes various programs and the like stored in the auxiliary storage device 13 shown in FIG. 2 with the main storage device 12 and the like as a work area. It can be realized by. Further, the inference by the DNN model 30 can be realized by the processor 11 shown in FIG. 2 executing various programs and the like stored in the auxiliary storage device 13 shown in FIG. 2 with the main storage device 12 and the like as a working area. .. Further, various processes by the system module 50 are realized by the processor 11 shown in FIG. 2 executing various programs and the like stored in the auxiliary storage device 13 shown in FIG. 2 with the main storage device 12 and the like as a work area. obtain. That is, the processor 11, the main storage device 12, and the auxiliary storage device 13 cooperate with software (various programs) to perform various functions (for example, acquisition unit 113 to notification unit 116) of the monitoring / adjustment module 100 described below. Processing function) can be realized. As the various programs executed by the processor 11 shown in FIG. 2, a program downloaded from an external device such as a server can also be used via the communication device 18.

<< 3. DNN model configuration example >>
Hereinafter, the outline of the DNN model 30 according to the embodiment of the present disclosure will be described. FIG. 3 is a diagram showing a configuration example of the DNN model according to the embodiment of the present disclosure. 4 and 5 are diagrams showing an outline of the DNN partial inference device according to the embodiment of the present disclosure.

As shown in FIG. 3, the DNN model 30 is composed of _{a plurality of DNN partial inferiors 31 (31 m} to 31 _{m + n) divided into predetermined particle sizes based on predetermined conditions.} The DNN model 30 is composed of a plurality of elements including layers, channels, and matrices. The DNN partial reasoner 31 is configured by being divided by a particle size based on at least one element of the layers, channels, and matrices constituting the DNN model 30, respectively. The particle size for dividing the DNN model 30 is determined in advance by the administrator of the monitoring / adjustment module 100, which will be described later, based on the cost required for the preliminary analysis and the required resource usage adjustment time interval.

As shown in FIG. 4, the DNN partial inferior 31 calculates and outputs the result of a certain element (block) of the divided DNN by, for example, the calculation method given by the monitoring / adjusting module 100. Therefore, the input is the input data of DNN inference or the output (activity value: Activation) of the DNN partial inference device 31 in the previous stage, and the output is the result of the element (block) of the DNN in charge. That is, as shown in FIG. 3, _{by connecting and operating a plurality of DNN partial inference devices 31 (31 m} to 31 _{m + n} ), it is possible to realize a function of returning the result of DNN inference for the input data. .. Depending on the structure of the DNN model 30 to be inferred, a branched structure may be required, but it can be configured in the same manner.

In the embodiment of the present disclosure, the DNN partial inference device 31 receives the calculation method Om (control information) which is the output of the monitoring / adjusting module 100 as an input, as shown in FIG. The DNN partial inference device 31 simultaneously acquires the input data and the calculation method Om (control information), performs a calculation on the input data by the acquired (based on the control information) calculation method, and as a result of the DNN element (block) in charge. Returns the output. Specifically, the number of quantization bits and the quantization method of the weight and the activation value, the Pruning ratio of the Pruning method, the calculation formula to be used, the weight parameter, and the like can be changed.

<< 4. Function configuration example of monitoring / adjustment module >>
Hereinafter, an example of the functional configuration of the monitoring / adjustment module according to the embodiment of the present disclosure will be described. FIG. 6 is a diagram showing an outline of the DNN inference device according to the embodiment of the present disclosure. FIG. 7 is a functional block diagram showing an example of the functional configuration of the monitoring / adjustment module according to the embodiment of the present disclosure.

As shown in FIG. 6, the DNN inference device 70 can be realized by combining the DNN partial inference device 31 (31 _m to 31 _{m + n} ) and the monitoring / adjustment module 100 (100 _m to 100 _{m + n).} In the DNN inference device 70 shown in FIG. 6, a monitoring / adjustment module 100 (100 _m to 100 _{m + n} ) is provided for each DNN partial inference device 31. According to the DNN inferior 70 shown in FIG. 6, the monitoring / adjustment module 100 (100 _m to 100 _{m + n} ) acquires the resource usage of the entire system every time the decomposed element of the DNN is calculated, and puts it in the system state. You can switch to the matching calculation method each time. As a result, the resource usage can be adjusted instantly, and the DNN inference device 70 that does not require interruption of inference can be realized. The "block" appearing in the following description is synonymous with the DNN partial inference device 31.

As shown in FIG. 7, the monitoring / adjustment module 100 includes a resource usage information storage unit 111, a correspondence information storage unit 112, an acquisition unit 113, a calculation unit 114, a determination unit 115, and a notification unit 116. Be prepared. The monitoring / adjusting module 100 realizes or executes the functions and operations of the monitoring / adjusting module 100 described below by each of these parts.

Each block (resource usage information storage unit 111 to notification unit 116) constituting the monitoring / adjustment module 100 is a functional block indicating the function of the monitoring / adjustment module 100, respectively. These functional blocks may be software blocks or hardware blocks. For example, each of the above-mentioned functional blocks may be one software module realized by software (including a microprogram), or may be one circuit block on a semiconductor chip (die). Of course, each functional block may be one processor or one integrated circuit. The method of configuring the functional block is arbitrary. The monitoring / adjustment module 100 may be configured in a functional unit different from the above-mentioned functional block.

The resource usage information storage unit 111 stores information on the resource usage used in the calculation of each block of DNN (DNN partial inference device 31). The resource usage information stored in the resource usage information storage unit 111 is stored by the calculation unit 114, which will be described later.

The correspondence information storage unit 112 stores the correspondence information between the resource usage amount and the calculation method pre-analyzed for each resource usage amount. The calculation method for each resource usage is acquired, for example, by prior analysis of the operator who is the administrator of the monitoring / adjustment module 100.

The acquisition unit 113 acquires the total resource usage of the information processing system 1. The acquisition unit 113 acquires the total power consumption and memory usage of the information processing system 1 from, for example, the system module 50.

The calculation unit 114 calculates the target resource usage amount to be allocated to at least a part of the calculation of the inference processing by the DNN inference device 70 based on the total resource usage amount of the information processing system 1 acquired by the acquisition unit 113. For example, the calculation unit 114 calculates the target resource usage amount to be allocated for each calculation of the inference process by the DNN partial inference device 31 (see FIG. 6) divided into predetermined particle sizes based on predetermined conditions. The calculation unit 114 calculates the target resource usage amount based on, for example, the resource surplus of the information processing system 1.

The determination unit 115 determines the calculation method of the DNN partial inference device 31 corresponding to the target resource usage calculated by the calculation unit 114. The calculation method is control information indicating how to calculate an element (block) of a certain DNN model 30, that is, a DNN partial inference device 31 (see FIG. 6) that performs inference processing next, and is "quantization bit". It corresponds to "number" and "Pruning ratio". The determination unit 115 can determine the calculation method based on the control information composed of tuples, such as combining the "quantization bit number" and the "Pruning ratio". The determination unit 115 outputs the determined calculation method to the corresponding DNN partial inference device 31.

The notification unit 116 notifies that the resource usage allocated to the DNN partial inferior 31 will decrease. For example, when the target resource usage amount calculated by the calculation unit 114 is equal to or less than a predetermined standard, the notification unit 116 may reduce the accuracy of the inference processing result (inference result) by the DNN inference device 70. Notify.

<4-1. Overview of monitoring / adjustment module>
Hereinafter, the outline of the monitoring / adjustment module 100 will be described. FIG. 8 is a diagram showing an outline of the monitoring / adjustment module according to the embodiment of the present disclosure.

As shown in FIG. 8, the monitoring / adjusting module 100 determines the calculation method Om of a certain element (block) constituting the DNN model 30 from the total resource usage Im of the information processing system 1 given as an input. And output. An element (each block: DNN partial inferior 31) may be a part of a layer, a channel (Map), a matrix (Tensor), or a combination thereof. The administrator of the monitoring / adjusting module 100 divides the DNN model 30 into a plurality of blocks in advance, and assigns a block number bi to each block. The degree of fine division is determined based on the cost required for pre-analysis and the required resource usage adjustment time interval.

The resource usage amount of the entire system given as an input to the monitoring / adjustment module 100 can be given the resource usage amount such as the power consumption, memory usage amount, and processing time of the entire system.

Note that the monitoring / adjustment module 100 may acquire a predicted value of future usage using a simple machine learning model instead of the measured value as the total resource usage of the information processing system 1. By doing so, it is possible to cope with a sudden change in the amount of resources used in the information processing system 1. Further, the monitoring / adjustment module 100 may acquire a taple that combines a plurality of different types of usage, such as power consumption and memory usage, as the total resource usage of the information processing system 1. good. By doing so, it is possible to calculate an appropriate output according to the amount of each resource used.

Furthermore, the monitoring / adjustment module 100 can specify the DNN block number bi as an input. This is used to determine which block calculation method is to be determined and output.

In addition, the monitoring / adjustment module 100 can receive the resource usage amount Rdn'allocated to each block before the block number bi. This is used to grasp the calculation status of the entire model and determine the calculation method Om that can improve the accuracy of the entire DNN model.

The calculation method Om, which is the output of the monitoring / adjustment module 100, is control information indicating how to calculate the block with the block number bi. For example, the number of quantization bits for adjusting the accuracy of numerical calculation, the Pruning ratio indicating what percentage of DNN elements are omitted as represented by the Pruning method, the network structure and parameters to be used, and the like. As the module output format, when control information composed of multiple elements such as the number of quantization bits and the Pruning ratio is used, the number of quantization bits Qb and the Pruning ratio Pr are used, and Om = (Qb, Pr). It can be expressed in tuples.

The calculation method Om, which is the output of the monitoring / adjustment module 100, determines the calculation method for one DNN block, but one block includes a plurality of channels: Channel and a matrix: Tensor. There is. In this case, the number of quantization bits Qb may be configured to have a plurality of quantization bits so that the number of quantization bits or the like can be specified for each of the elements in the block.

<4-2. Operation of monitoring / adjustment module (1)>
Hereinafter, the operation of the monitoring / adjusting module 100 will be described. FIG. 9 is a diagram showing an outline of the monitoring / adjustment module according to the embodiment of the present disclosure.

As shown in FIG. 9, the determination of the calculation method Om by the monitoring / adjustment module 100 is performed in two stages. First, in the first stage, the monitoring / adjusting module 100 determines the target resource usage Rdnn used in the block of the block number bi from the resource usage Im of the entire system given as an input. The operation of the first stage is realized, for example, by the function of the calculation unit 114 shown in FIG. 7.

Next, in the second stage, the monitoring / adjustment module 100 selects and determines the calculation method Om that does not reduce the DNN calculation accuracy as much as possible from among the calculation methods that achieve the target resource usage Rdnn. The operation of the second stage is realized, for example, by the function of the determination unit 115 shown in FIG. 7. That is, the acquisition unit 113 of the monitoring / adjustment module 100 acquires the resource usage amount each time the calculation method is determined by the determination unit 115. Subsequently, the monitoring / adjustment module 100 calculation unit 114 calculates the target resource usage amount of the DNN partial inference device 31 that next calculates the inference processing each time the resource usage amount is acquired by the acquisition unit 113. Then, each time the determination unit 115 of the monitoring / adjustment module 100 calculates the target resource usage amount by the calculation unit 114, the determination unit 115 determines the calculation method of the DNN partial inference device that next calculates the inference process.

As a simple calculation method of the target resource usage amount Rdnn determined by the operation of the first stage described above, there is a method of allocating the resource surplus of the system to the calculation of DNN. For example, it can be calculated by the following equation (1) using the input resource usage amount Im of the entire system, the maximum resource amount Rmax that can be supplied by the system, and the constant margin Epsilon.
"Target resource usage Rdnn" = "Maximum resource usage Rmax"-"Resource usage Im"-"Constant margin Epsilon" ... (1)

In the second step described above, it is necessary to determine the calculation method Om from the target resource usage amount Rdnn that may be used in the block with the block number bi and the resource usage amount Rdnn'used in the block before the block number bi. However, there are various combinations of the number of quantization bits and the Pruning ratio, and the required calculation accuracy differs for each DNN part, so it is difficult to determine the calculation method Om. On the other hand, in the method of simply allocating the calculation resources evenly based on Rdnn for each block, the accuracy of the DNN inference processing is greatly reduced. Therefore, as an example of a method of instantly determining the calculation method Om from a plurality of target resource usage Rdnn, a determination method based on the preliminary analysis result is shown. FIG. 10 is a diagram showing an outline of a method for determining a calculation method based on the preliminary analysis result according to the embodiment of the present disclosure.

FIG. 10 shows an example of a method of listing in advance calculation methods with less decrease in accuracy of inference processing for various combinations of resource usage. It is assumed that the input Im is one type of resource such as power consumption and memory usage, and the range is from 0% to 100%. In this case, for example, for 11 interpolation points of Im = 0%, 10%, 20%, ..., 100% between 0% and 100%, the optimum calculation with little deterioration in the accuracy of the inference processing. Method Om is analyzed in advance. This analysis is performed for all combinations of the target resource usage Rdnn of the block to be determined by the calculation method and the resource usage Rdn'used in the previous block.

Then, the monitoring / adjustment module 100 acquires and acquires a plurality of calculation methods associated with the resource usage amount close to the target resource usage amount Rdnn based on the correspondence information stored in the correspondence information storage unit 112. Determine the calculation method based on multiple calculation methods. Specifically, the monitoring / adjustment module 100 can determine the calculation method Om by searching for points (interpolation points) close to the input Im from the pre-analysis results during operation and interpolating from them. If the number of interpolation points is increased to 11 or more in the pre-analysis, a more accurate calculation method Om can be determined. Further, even if the input Im is for a plurality of resources, it can be dealt with by similarly analyzing the interpolation points in advance.

The operation of the monitoring / adjustment module 100 described above is only an example, and as an input Im, the output of the monitoring / adjustment module 100 in front (if the operating subject is the monitoring / adjustment module 100 _{m + 1} , the monitoring / adjustment module 100 _m ). The calculation method Om is also given. From this, it is possible to know the change over time in the resource usage of the entire system over a long period of time and the degree of calculation accuracy assigned to which element in the DNN. Therefore, it is possible to obtain a calculation method Om that is more stable and improves the accuracy of the entire DNN process with respect to the resource usage amount with large fluctuation.

In the operation of the monitoring / adjustment module 100 described above, by replacing "block" with "DNN partial inference device", calculation is performed for each of the DNN partial inference devices 31 constituting the DNN inference device 70 shown in FIG. The operation of outputting the method Om can be realized.

<4-3. Operation of monitoring / adjustment module (2)>
The monitoring / adjusting module 100 can operate so as to notify the information processing system 1 that the resource usage allocated to the DNN partial inferencer 31 is decreasing. Such an operation is realized by the function of the notification unit 116 shown in FIG. 7. FIG. 11 is a diagram showing an outline of the notification operation according to the embodiment of the present disclosure.

As shown in FIG. 11, the information processing system 1 includes a user notification module 90. The user notification module 90 visualizes and notifies the user of the information processing system 1, for example, of the processing state of the information processing system 1. The processing state corresponds to, for example, a decrease in recognition accuracy in the case of recognition processing such as image recognition or voice recognition, and a decrease in responsiveness in the case of UI response processing.

The monitoring / adjustment module 100 executes notification to the user notification module 90, for example, when the target resource usage amount Rdnn allocated to the DNN partial inference device 31 is equal to or less than a predetermined standard.

Upon receiving the notification from the monitoring / adjusting module 100, the user notification module 90 visualizes information indicating that the responsiveness of the information processing system 1 is deteriorated, for example, on the operation device 91 operated by the user. As a visualization method, for example, a method of lighting the light emitting unit provided in the operation device 91 with a predetermined color, or a method of blinking the light emitting unit can be considered.

Note that the monitoring / adjustment module 100 may notify the system module 50 when the target resource usage amount Rdnn allocated to the DNN partial inference device 31 is equal to or less than a predetermined standard. Upon receiving the notification from the monitoring / adjusting module 100, the system module 50 can change the operation of the system so as to improve the safety of the user according to the decrease in the accuracy of the inference result by the DNN inference device 70. For example, if the information processing system 1 is a transport robot system, it is conceivable to change the operation so as to select a route that does not damage the environment or people even at the expense of transport time. Alternatively, if the information processing system 1 is a pet-type robot, it is conceivable to change the operation so as to take a resting gesture, for example, by closing the eyes in accordance with the decrease in the accuracy of the response processing.

<< 5. Example of processing procedure by monitoring / adjustment module >>
Hereinafter, the processing procedure by the monitoring / adjusting module 100 according to the embodiment of the present disclosure will be described. FIG. 12 is a flowchart showing an example of a processing procedure by the monitoring / adjusting module according to the embodiment of the present disclosure. The processing procedure shown in FIG. 12 is repeatedly executed while the information processing system 1 is in operation.

As shown in FIG. 12, the acquisition unit 113 acquires the input Im, which is the total resource usage of the information processing system 1 (step S101).

The calculation unit 114 calculates the target resource usage amount Rdnn of the DNN partial inference device 31 based on the input Im, and stores it in the resource usage amount information storage unit 111 (step S102).

The determination unit 115 uses the block number bi assigned to the current block (DNN partial inferior 31) to be processed as a key, and the resource usage amount Rdn'used in the blocks before the current block as the resource usage amount. Obtained from the information storage unit 111 (step S103).

The determination unit 115 determines the calculation method Om based on the target resource usage amount Rdnn and the resource usage amount Rdnn'used in the previous block (step S104).

The determination unit 115 outputs the determined calculation method Om to the current block to be processed (step S105), and returns to the processing procedure of step S101.

In the above-described embodiment, by giving the predicted value of the future resource usage instead of the current resource usage as the input Im of the monitoring / adjustment module 100, it becomes easy to avoid the excess of the resource usage. , The DNN inferior 70 can be configured. Simple machine learning models and Kalman filters can be used to predict future resource usage.

Further, for example, the particle size of the DNN model 30 to be finely divided and the pre-analysis to configure the DNN inferior 70 depends on how much time required for the pre-analysis and the cost of computational resources can be tolerated, and the amount of resources used. It depends on how fine the time interval you want to switch between. In order to reduce the switching time interval, it is necessary to evaluate more combinations of calculation methods in the preliminary analysis, which requires a large implementation time and calculation resources. In addition, since the timing at which switching is possible for the entire model increases, the processing time of the entire model may increase due to the switching overhead.

When using this technology, it is necessary to determine two particle sizes. The first is, for example, the particle size of dividing the DNN model 30 into blocks, and the second is how finely the elements in the block are analyzed. The balance of these is adjusted with these particle sizes.

First, the DNN inference is the first "particle size for dividing the DNN model 30 into blocks", and the DNN model 30 can be divided into a plurality of DNN partial inference devices 31. For example, in the pre-analysis method shown in the example shown in FIG. 10 and the like, since the calculation method is searched for this particle size switching, the minimum unit in which the resource usage is switched during DNN inference is this particle size. If not much time has passed after processing one block divided by this particle size, the next block is continuously executed with the same resource usage without switching the resource usage, etc. Blocks can be combined by doing so. However, the resource usage cannot be switched in units smaller than the block.

The second "grain size of how finely the elements in a block are viewed" is the finer elements in the block, such as channels and matrices, in order to protect the resource usage of the block and maximize the accuracy. How to allocate the number of bits and computing power to tensor). As shown in Reference 1 above, the decrease in accuracy can be suppressed by making this finer, but it is necessary to balance it with the cost of pre-analysis, which is expected to increase by making it finer.

Further, in the above embodiment, as an inference device, a DNN that performs inference processing by connecting calculations by a plurality of layers in multiple stages has been described, but similarly, the above-mentioned monitoring / adjustment module is also used for a neural network composed of a single layer. Processing by 100 can be applied. In this case, the calculation method corresponding to the target resource usage is determined based on the result of pre-analysis of the calculation method corresponding to the resource usage for each element such as the channel and the matrix constituting the layer of the target neural network. can.

<< 6. Modification example >>
<6-1. Generation of correspondence information by machine learning ＞
The correspondence information stored in the correspondence information storage unit 112 is not particularly limited when it is acquired by prior analysis by the operator, and may be acquired as a result of machine learning, for example.

For example, a DNN model that uses reinforcement learning to infer the calculation method Om as the output for these inputs, with the inputs as the resource usage Rdn', the target resource usage Rdnn, and the block number bi used in the previous block. To learn. In this case, the state of reinforcement learning is Rdnn'(the amount of resources used in the previous block), Rdnn (the amount of target resources used), the taple of the block number bi, the output is the calculation method Om, and the reward is calculated. It may be the difference between the accuracy of the inference processing that can be achieved by the method Om (in the case of the recognition processing, the recognition accuracy) and the accuracy when the inference processing is performed without limiting the resource usage. An efficient calculation method can be determined by using the reinforcement learning model.

<6-2. Resource switching overhead reduction>
In the operation (1) of the monitoring / adjustment module described above, the operation of the second stage described above can be modified in order to reduce the overhead of resource switching. FIG. 13 is a diagram showing an outline of a method for determining a calculation method based on a preliminary analysis result according to a modified example. FIG. 14 is a diagram showing a configuration example of the DNN inference device according to the modified example.

For example, as shown in FIG. 13, the monitoring / adjustment module 100 calculates all the blocks after the block number bi with respect to the input resource usage amount Rdn', target resource usage amount Rdnn, and block number bi. It may be output.

Further, as shown in FIG. 14, a determination mechanism 71 (determining whether to operate the monitoring / adjustment module 100 to output a new calculation method or to use the previously determined calculation method as it is before calculating a certain block ( 71 _m to 71 _{m + n-1} ) may be newly provided in the DNN inferior 70. The determination mechanism 71 shown in FIG. 14 measures the passage of time from the previous operation of the monitoring / adjusting module 100, and when a certain time has elapsed, operates the monitoring / adjusting module 100 to output a new calculation method. .. On the other hand, if a certain time has not passed since the previous operation of the monitoring / adjusting module 100, the determination mechanism 71 uses the previously determined calculation method as it is. In this way, the overhead of resource switching can be reduced rather than determining the calculation method each time the resource usage of the entire system is acquired.

For example, when the determination mechanism 71 _{m + n-1 determines the calculation method of the partial inference device 31 m + n} to which the block number bi = n is assigned, the determination mechanism 71 m + n-1 determines the calculation method of the block number bi = n-1 for a certain period of time. Determines if has passed. When a certain period of time has passed, the determination mechanism 71 _{m + n-1} causes the monitoring / adjustment module 100 _{m + n to determine the calculation method of the partial inference device 31 m + n} to which the block number bi = n is assigned, and a new calculation method is used. Is output. On the other hand, when a certain period of time has not elapsed, the determination mechanism 71 _{m + n-1} outputs the calculation method previously determined for the _{partial inference device 31 m + n} to which the block number bi = n is assigned, as it is.

<< 7. Others >>
<7-1. Application to robot systems>
The DNN inference device 70 including the DNN partial inference device 31 and the monitoring / adjustment module 100 described above can be applied to a robot system that processes a camera image by a DNN. Most robots that operate outdoors have a maximum power consumption contract because they are driven by a battery. In addition, electric parts with high power consumption such as actuators are mounted, and recognition processing and communication processing for determining actions are also performed at the same time, so that the power consumption fluctuates depending on the situation. If the system is operated without considering the power consumption of the entire system, the usage amount may exceed the maximum supply amount, causing abnormal operation or the inability to control the posture.

In the robot system, the maximum power consumption is supplied by executing the recognition process of recognizing the camera image by the DNN by using the DNN inference device 70 composed of the DNN partial inference module 31 and the monitoring / adjustment module 100. It is possible to continue to execute the recognition process while adjusting so that it does not exceed. The current power consumption is given to the input of the monitoring / adjusting module 100, and the Pruning ratio of the next DNN element is determined as the output. As a result, when the power consumption of the entire system becomes large, the pruning ratio becomes large and the calculation of some DNN elements is omitted. By omitting some calculations, the utilization rate of the circuit per hour decreases, the dynamic power consumption of DNN processing is kept low, a surplus is generated, and power is generated in places where a large amount of power consumption is required, such as actuators. Can be supplied. In addition, since the pruning ratio is determined by prior analysis and the recognition process itself by DNN can be continuously performed, it is possible to minimize the omission of recognition of the animal body captured by the camera and the delay in action decision. In a situation where the power consumption of the actuator etc. increases and the Pruning ratio must be significantly increased, the recognition accuracy of the recognition process by DNN decreases, and the detection rate of the animal body decreases to some extent, so that the surroundings It is also possible to take unsafe actions against the environment and people. In such a case, if it is a transport robot, select a route that does not damage the environment or people even if the transport time is sacrificed, or if it is a pet-type robot used at home, close your eyes. The safety can be further enhanced by making the user look like it is.

<7-2. Application to game consoles>
The above-mentioned DNN inference device 70 can be applied to a game machine having a UI function for recognizing voice, hand gesture, and the like by processing by DNN. The game machine has a predetermined memory capacity. In order to improve the quality of the scene drawing process, it is necessary to expand a large amount of geometry and material information in the memory. The recognition process by DNN operates in parallel, and when the memory is used to some extent, the drawing quality is lowered to some extent. However, due to the progress of the game, there are scenes in which the player wants to see the scene by improving the drawing quality even if the operability is given up to some extent in the movie scene or the like. In addition, in indoor scenes and screens where only the UI is displayed, the amount of memory used for drawing processing is small. Therefore, in such a case, it is desired to allocate the surplus memory amount to the recognition process by DNN to improve the recognition accuracy.

By applying the DNN inference device 70 described above to a game machine, it is possible to meet the above requirements. The memory usage of the drawing process is given to the input of the monitoring / adjusting module 100 so that the number of quantization bits is determined as the output. By doing so, when it becomes necessary to secure a large memory area for the drawing process, the number of quantization bits is changed instantly, and the recognition process by the DNN can be performed with a small amount of memory usage. It is possible to improve the drawing quality with the surplus memory capacity created as a result. On the other hand, there is a concern that the responsiveness of the UI function using DNN will decrease, but the game developer can carefully select the scene such as using it when he wants to draw the player into the movie scene, and the responsiveness. It is possible to take measures such as visualizing the decrease in the game to a controller or the like (see FIG. 11 or the like), or instructing the user to interrupt the game and use the UI function.

<< 8. Conclusion >>
The monitoring / adjustment module 100 (an example of an information processing apparatus) according to the embodiment of the present disclosure is applied to an information processing system 1 that uses an inference result by an inference device using a neural network, and is applied to an information processing system 1 and has an acquisition unit 113 and a calculation unit 114. And a determination unit 115. The acquisition unit 113 acquires the total resource usage of the information processing system 1. The calculation unit 114 calculates the target resource usage amount to be allocated to at least a part of the calculation of the inference processing by the DNN inference device 70 (for example, the calculation of the inference processing by the DNN partial inference device 31) based on the resource usage amount. The determination unit 115 determines a calculation method (calculation method of inference processing by the DNN partial inference device 31) corresponding to the target resource usage amount. Therefore, the monitoring / adjustment module 100 can adjust the resource amount used for the inference processing of the DNN model 30 so as not to cause the resource usage amount to be exceeded in the information processing system 1.

FIG. 15 is a diagram showing an example of information processing according to a comparative example. FIG. 16 is a diagram showing an example of information processing by the DNN inference device according to the embodiment of the present disclosure. FIG. 17 is a diagram showing the relationship between the elapsed time and the resource usage amount of the information processing according to the comparative example. FIG. 18 is a diagram showing the relationship between the elapsed time and the resource usage amount of the information processing by the DNN inferior according to the embodiment of the present disclosure.

As shown in FIG. 15, the function of adjusting the resource usage of processing based on the resource usage of the entire system may be realized by a technique called code switching. In FIG. 15, two codes having the same purpose are prepared, that is, the code used in the DNN inferior is the "normal code" when there is a resource surplus and the "excess code" when the resource is exceeded. It is assumed that the code can be switched according to the resource usage. In this mechanism, as shown by the arrow in FIG. 17, the opportunity to adjust the resource usage is limited before executing the inference by DNN such as the DNN inference for the data 1 and the DNN inference for the data 2. Therefore, when the resource usage of the entire system suddenly increases, the resource usage of the DNN inferior cannot be instantly accommodated and adjusted. As a result, the resource usage of the entire system may be exceeded and the system may be stopped, or DNN inference may be stopped for safety.

On the other hand, in the embodiment of the present disclosure, in the DNN inferior 70, as described above, the DNN calculation is a stack of calculation for each element (calculation of each DNN partial inference device 31), and the calculation accuracy is changed for each element. Even if this is the case, the property that the output accuracy of the entire DNN does not easily decrease (see Reference 1 above) is utilized. In the DNN inference device 70 according to the embodiment of the present disclosure, _{even if it becomes necessary to adjust the resource usage in the DNN partial inference device 31 m + 1} as shown in FIG. 16, as shown by the arrow in FIG. 18, for each element. Since there is an opportunity to adjust the resource usage, it is possible to make adjustments in sequence.

FIG. 19 is a diagram showing an example of information processing by the existing technology. The purpose of adjusting the resource usage of processing based on the resource usage of the entire system is to appropriately allocate the limited resources that can be used in the system in DNN processing and other processing. As a function for realizing such an object, for example, it is conceivable to apply the mechanism shown in the prior art document (Japanese Patent Laid-Open No. 2012-43409) described above. That is, when the data processing load (resource usage) is predicted to increase, time-series input data is selectively removed, data processing is delayed, or data processing is offloaded to other systems. It can be realized by reading the data processing as the DNN inference processing in the data processing system. However, with such a mechanism, when the resource usage of other processing increases, data that cannot be DNN inferred is generated, or the processing result is acquired with a delay, and the real-time property and quality of DNN inference are improved. There is a risk that it will drop significantly.

On the other hand, in the DNN inference device 70 according to the embodiment of the present disclosure, for example, by utilizing the property that even if the calculation accuracy is lowered during the DNN calculation, the accuracy of the result is not easily affected (see Reference 1 above), for example. As described with reference to FIG. 6 and the like, it is possible to construct a data processing system that does not skip data or increase delay while adjusting the amount of resources used.

Further, the DNN inference device 70 (an example of an inference device) is divided into a plurality of DNN partial inference devices 31 (an example of a partial inference device) with a predetermined particle size based on a predetermined condition. Then, the calculation unit 114 calculates the target resource amount to be allocated to the inference processing of the DNN partial inference device 31 to be processed, and the determination unit 115 calculates the target resource amount of the DNN partial inference device 31 to be processed based on the target resource amount. The calculation method in the inference processing is determined and output to the DNN partial inference device 31 to be processed. As a result, the monitoring / adjustment module 100 can allocate an appropriate amount of resources to each DNN partial inferencer 31 from the resources that can be allocated in the system.

Further, the DNN inferior 70 is divided into a plurality of DNN partial inferiors 31 based on the cost required for analyzing the calculation method for each resource usage or the time interval required for adjusting the resource usage. As a result, the monitoring / adjustment module 100 enables flexible resource management according to the purpose of the system administrator.

Further, the DNN inferior 70 is composed of a plurality of elements including a layer, a channel, and a matrix. The DNN partial reasoner 31 is then divided at a particle size based on at least one element of the layer, channel, and matrix. As a result, the monitoring / adjustment module 100 can adjust the resource usage amount at an arbitrary granularity according to the purpose of the system administrator.

Further, the determination unit 115 determines the calculation method based on the control information composed of tuples. Thereby, the monitoring / adjustment module 100 can specify the calculation method by a plurality of elements.

Further, the calculation unit 114 calculates the target resource usage amount based on the resource surplus in the information processing system 1. As a result, the monitoring / adjustment module 100 can easily calculate the target resource amount.

In addition, the calculation unit 114 calculates the target resource usage amount based on the resource surplus and the predetermined margin. As a result, the monitoring / adjustment module 100 can increase the availability of the system.

Further, the monitoring / adjustment module 100 includes a correspondence information storage unit 112 (an example of a storage unit) that stores correspondence information between the resource usage amount and the calculation method pre-analyzed for each resource usage amount. The determination unit 115 determines the calculation method based on the correspondence information. As a result, the monitoring / adjusting module 100 can quickly adjust the amount of resources for each element of the DNN while maintaining the accuracy of the processing by the DNN in the system.

Further, the correspondence information storage unit 112 stores the correspondence information obtained as a result of machine learning. This makes it possible to improve the accuracy of resource adjustment.

Further, the determination unit 115 acquires a plurality of calculation methods associated with the resource usage amount close to the target resource usage amount based on the correspondence information, and determines the calculation method based on the acquired plurality of calculation methods. .. This makes it possible to balance the adjustment speed of the resource amount and the adjustment accuracy.

Further, the monitoring / adjusting module 100 includes a notification unit 116 for notifying that the target resource usage amount is equal to or less than a predetermined standard in the information processing system 1. Thereby, the safety of the operation of the information processing system 1 can be enhanced.

Further, the acquisition unit 113 acquires the resource usage amount each time the determination unit 115 determines the calculation method, and the calculation unit 114 next processes the resource usage amount each time the acquisition unit 113 acquires the resource usage amount. The target resource usage amount of the DNN partial inference device 31 is calculated, and the determination unit 115 calculates the target resource usage amount of the DNN partial inference device 31 to be processed next each time the target resource usage amount is calculated by the calculation unit 114. To determine. Thereby, the resource amount adjustment of the DNN partial inference device 31 can be realized according to the state of the system as much as possible.

The acquisition unit 113 acquires the resource usage amount each time the determination unit 115 determines the calculation method, and the calculation unit 114 is subject to subsequent processing each time the resource usage amount is acquired by the acquisition unit 113. The target resource usage of all the DNN partial inference devices 31 is calculated, and each time the calculation unit 114 calculates the target resource usage of the determination unit 115, the determination unit 115 of all the DNN partial inference devices 31 to be processed thereafter. Determine the calculation method. This can reduce the overhead associated with resource switching in the system.

Although the embodiments of the present disclosure have been described above, the technical scope of the present disclosure is not limited to the above-described embodiments as they are, and various changes can be made without departing from the gist of the present disclosure. In addition, components spanning different embodiments and modifications may be combined as appropriate.

Further, the effects described in the present specification are merely explanatory or exemplary and are not limited. That is, the techniques of the present disclosure may have other effects apparent to those of skill in the art from the description herein, in addition to, or in lieu of, the above effects.

The technology of the present disclosure can be configured as follows, assuming that it belongs to the technical scope of the present disclosure.
(1)
It is an information processing device applied to an information processing system that uses the inference result by an inference device using a neural network.
An acquisition unit that acquires the total resource usage of the information processing system,
A calculation unit that calculates the target resource usage to be allocated to at least a part of the calculation of the inference processing by the inference device based on the resource usage.
An information processing device including a determination unit that determines a calculation method corresponding to the target resource usage.
(2)
The inference device is divided into a plurality of partial inference devices with a predetermined particle size based on predetermined conditions.
The calculation unit calculates the target resource usage amount to be allocated to the inference processing of the partial inference device that next calculates the inference processing.
The information processing apparatus according to (1), wherein the determination unit determines a calculation method in the inference processing of the partial inference device that next calculates the inference processing based on the target resource usage amount.
(3)
The inferior is divided into a plurality of partial inferiors based on the cost required for analyzing the calculation method for each resource usage or the time interval required for adjusting the resource usage (2). The information processing device described in.
(4)
The inferencer is composed of a plurality of elements including layers, channels, and matrices.
The information processing apparatus according to (3) above, wherein the partial inference device is divided by a particle size based on at least one element of a layer, a channel, and a matrix.
(5)
The decision-making part
The information processing apparatus according to (1) above, wherein the calculation method is determined based on control information composed of tuples.
(6)
The calculation unit
The information processing apparatus according to any one of (1) to (5) above, which calculates the target resource usage amount based on the resource surplus in the information processing system.
(7)
The calculation unit
The information processing apparatus according to (6), wherein the target resource usage amount is calculated based on the resource surplus and a predetermined margin.
(8)
It also has a storage unit that stores the correspondence information between the resource usage and the calculation method pre-analyzed for each resource usage.
The decision-making part
The information processing apparatus according to any one of (1) to (7), wherein the calculation method is determined based on the corresponding information.
(9)
The storage unit is
The information processing device according to (8) above, which stores the corresponding information obtained as a result of machine learning.
(10)
The decision-making part
Based on the correspondence information, a plurality of the calculation methods associated with the resource usage amount close to the target resource usage amount are acquired, and the calculation method is determined based on the acquired plurality of the calculation methods. (8) or the information processing apparatus according to (9) above.
(11)
The information processing apparatus according to any one of (1) to (10) above, further comprising a notification unit for notifying that the target resource usage amount is equal to or less than a predetermined standard in the information processing system.
(12)
The acquisition unit
Every time the calculation method is determined by the determination unit, the resource usage amount is acquired.
The calculation unit
Each time the resource usage amount is acquired by the acquisition unit, the target resource usage amount of the partial inference device to be processed next is calculated.
The decision-making part
The information processing apparatus according to (2), wherein the calculation method of the partial inference device to be processed next is determined each time the target resource usage amount is calculated by the calculation unit.
(13)
The acquisition unit
Every time the calculation method is determined by the determination unit, the resource usage amount is acquired.
The calculation unit
Every time the resource usage amount is acquired by the acquisition unit, the target resource usage amount of all the partial inference devices to be processed thereafter is calculated.
The decision-making part
The information processing apparatus according to (2) above, wherein each time the target resource usage amount is calculated by the calculation unit, the calculation method of all the partial inference devices to be processed thereafter is determined.
(14)
The processor of the information processing device applied to the information processing system that uses the inference result by the inference device using the neural network
Acquire the total resource usage of the information processing system,
Based on the resource usage, the target resource usage to be allocated to the inference processing of the inference device is calculated.
An information processing method that determines a calculation method in the inference processing of the inference device based on the target resource usage.

1 Information processing system 11 Processor 12 Main storage device 13 Auxiliary storage device 14 Peripheral circuit 15 Input device 16 Output device 17 Peripheral device 18 Communication device 20 Internal bus 30 DNN model 31 DNN partial inference device 50 System module 70 DNN inference device 90 User notification Module 91 Operation device 100 Monitoring / adjustment module 111 Resource usage information storage unit 112 Corresponding information storage unit 113 Acquisition unit 114 Calculation unit 115 Decision unit 116 Notification unit

Claims (14)

It is an information processing device applied to an information processing system that uses the inference result by an inference device using a neural network.
An acquisition unit that acquires the total resource usage of the information processing system,
A calculation unit that calculates the target resource usage to be allocated to at least a part of the calculation of the inference processing by the inference device based on the resource usage.
An information processing device including a determination unit that determines a calculation method corresponding to the target resource usage. The inference device is divided into a plurality of partial inference devices with a predetermined particle size based on predetermined conditions.
The calculation unit calculates the target resource usage amount to be allocated to the inference processing of the partial inference device that next calculates the inference processing.
The information processing apparatus according to claim 1, wherein the determination unit determines a calculation method in the inference processing of the partial inference device that next calculates the inference processing based on the target resource usage amount. The inference device is divided into a plurality of partial inferencers based on the cost required for analyzing the calculation method for each resource usage amount or the time interval required for adjusting the resource usage amount according to claim 2. The information processing device described. The inferencer is composed of a plurality of elements including layers, channels, and matrices.
The information processing apparatus according to claim 3, wherein the partial inference device is divided by a particle size based on at least one element of a layer, a channel, and a matrix. The decision-making part
The information processing apparatus according to claim 1, wherein the calculation method is determined based on control information composed of tuples. The calculation unit
The information processing apparatus according to claim 1, wherein the target resource usage amount is calculated based on the resource surplus in the information processing system. The calculation unit
The information processing apparatus according to claim 6, wherein the target resource usage amount is calculated based on the resource surplus and a predetermined margin. It also has a storage unit that stores the correspondence information between the resource usage and the calculation method pre-analyzed for each resource usage.
The decision-making part
The information processing apparatus according to claim 1, wherein the calculation method is determined based on the corresponding information. The storage unit is
The information processing apparatus according to claim 8, which stores the corresponding information obtained as a result of machine learning. The decision-making part
A claim that acquires a plurality of the calculation methods associated with the resource usage amount close to the target resource usage amount based on the correspondence information, and determines the calculation method based on the acquired plurality of the calculation methods. Item 8. The information processing apparatus according to Item 8. The information processing apparatus according to claim 1, further comprising a notification unit for notifying that the target resource usage amount is equal to or less than a predetermined standard in the information processing system. The acquisition unit
Every time the calculation method is determined by the determination unit, the resource usage amount is acquired.
The calculation unit
Each time the resource usage is acquired by the acquisition unit, the target resource usage of the partial inference device that next calculates the inference process is calculated.
The decision-making part
The information processing apparatus according to claim 2, wherein the calculation method of the partial inference device that next calculates the inference process is determined each time the target resource usage amount is calculated by the calculation unit. The acquisition unit
Every time the calculation method is determined by the determination unit, the resource usage amount is acquired.
The calculation unit
Every time the resource usage is acquired by the acquisition unit, the target resource usage of all the partial inference devices for which the inference processing is calculated thereafter is calculated.
The decision-making part
The information processing apparatus according to claim 2, wherein every time the target resource usage amount is calculated by the calculation unit, the calculation method of all the partial inference devices for which the inference processing is calculated thereafter is determined. The processor of the information processing device applied to the information processing system that uses the inference result by the inference device using the neural network
Acquire the total resource usage of the information processing system,
Based on the resource usage, the target resource usage to be allocated to at least a part of the calculation of the inference processing by the inference device is calculated.
An information processing method that determines a calculation method corresponding to the target resource usage.

Images