CN113052301A - Neural network generation method and device, electronic equipment and storage medium - Google Patents
Neural network generation method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113052301A CN113052301A CN202110334296.0A CN202110334296A CN113052301A CN 113052301 A CN113052301 A CN 113052301A CN 202110334296 A CN202110334296 A CN 202110334296A CN 113052301 A CN113052301 A CN 113052301A
- Authority
- CN
- China
- Prior art keywords
- neural network
- channel group
- network
- information
- channel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
本公开提供了一种神经网络生成方法、装置、智能行驶方法、设备、电子设备及计算机可读存储介质，其中，本公开将神经网络中各个网络层的通道分别进行分组，并基于网络规模限制信息和通道组的重要程度信息，剔除不重要的通道组，得到的目标神经网络通道数满足对应移动终端对通道数的特定要求，不需要后续对通道数进行增加或减少，能够直接部署到移动终端上。利用通道组获取最终的目标神经网络的方式，速度快，效率高；同时不会损失剪裁得到的目标神经网络的精度。
The present disclosure provides a neural network generation method, device, intelligent driving method, device, electronic device, and computer-readable storage medium, wherein the present disclosure groups the channels of each network layer in the neural network respectively, and based on network scale restrictions The information and the importance of the channel group, excluding the unimportant channel groups, the obtained target neural network channel number meets the specific requirements of the corresponding mobile terminal for the number of channels, and does not need to increase or decrease the number of channels subsequently, and can be directly deployed to the mobile terminal. on the terminal. The method of obtaining the final target neural network by using the channel group is fast and efficient; at the same time, the accuracy of the target neural network obtained by tailoring will not be lost.
Description
Technical Field
The present disclosure relates to the field of model compression technologies, and in particular, to a neural network generation method, an intelligent driving device, an electronic device, and a computer-readable storage medium.
Background
Neural networks are used in more and more fields such as autopilot. When the neural network is deployed on a mobile terminal such as a vehicle-mounted device, the large-scale neural network needs to be tailored due to the limited computing capability of the mobile terminal. The mobile terminal has a requirement on the number of channels of each layer of the neural network deployed thereon, for example, the number of channels of each layer of the neural network deployed thereon is required to be multiple of 8 by the mobile terminal, and the directly cut neural network hardly meets the requirement and cannot be directly deployed on the mobile terminal.
Currently, this requirement is generally met by simply adding or subtracting channels from the trimmed neural network. However, simply adding or subtracting channels may cut out some important channels, resulting in a reduced accuracy of the neural network, or in a situation where the added channels cause the neural network to exceed the load capacity of the mobile terminal.
Disclosure of Invention
The embodiment of the disclosure at least provides a neural network generation method, a neural network generation device, an intelligent driving method, equipment, electronic equipment and a computer-readable storage medium.
In a first aspect, an embodiment of the present disclosure provides a neural network generation method, including:
grouping channels included in each network layer in a neural network to be trained respectively to obtain at least one channel group;
determining importance degree information corresponding to each channel group;
and based on the network scale limit information and the importance degree information of each channel group, removing at least one channel group in the neural network to be trained to obtain the target neural network.
In the aspect, channels of each network layer in the neural network are grouped, unimportant channel groups are removed based on network scale limitation information and important degree information of the channel groups, the obtained number of the channels of each network layer in the target neural network meets the specific requirement of the mobile terminal on the number of the channels of each layer, subsequent adjustment such as increase or reduction of the number of the channels is not needed, and the channels can be directly deployed on the mobile terminal. In the aspect, the final target neural network mode is determined by utilizing the channel group, so that the speed is high and the efficiency is high; and meanwhile, the precision of the target neural network obtained by cutting is not lost.
In a possible implementation manner, the determining the importance information corresponding to each channel group includes:
constructing a loss function of the neural network to be trained by utilizing the gamma values corresponding to the channels in each channel group; the gamma value is used for representing the importance degree of the corresponding channel;
training the neural network to be trained by using a sample image until a training cut-off condition is met;
determining the optimized value of the gamma value corresponding to each channel according to the value of the loss function of the neural network when the training is finished;
and determining importance degree information corresponding to each channel group based on the optimized value of the gamma value corresponding to each channel.
According to the embodiment, the loss function is constructed by utilizing the gamma values corresponding to the channels in each channel group, then, the optimized values of the gamma values corresponding to the channels are obtained through training, and the importance degree of each channel group can be accurately determined by utilizing the obtained optimized values.
In a possible embodiment, the determining importance information corresponding to each channel group based on the optimized value of the gamma value corresponding to each channel includes:
calculating the mean value of the optimized gamma values corresponding to the channels included in a channel group aiming at the channel group;
and taking the calculated average value as the importance degree information of the channel group.
In this embodiment, the average value of the optimized gamma values corresponding to each channel in the channel group can reflect the importance of the corresponding channel group more accurately.
In a possible embodiment, the constructing the loss function of the neural network to be trained by using the gamma values corresponding to the channels in each channel group includes:
and constructing a loss function by using the number of channels in one channel group and the gamma values corresponding to the channels in each channel group.
According to the embodiment, the loss function constructed by the number of the channels in the channel group and the gamma values corresponding to the channels can be trained to obtain the optimized value of the gamma value which can reflect the importance degree of each channel more accurately.
In a possible implementation manner, the removing at least one channel group in the neural network to be trained based on the network scale constraint information and the importance information of each channel group to obtain a target neural network includes:
acquiring network scale information corresponding to a channel group;
and based on the network scale limiting information, the network scale information and the importance degree information of each channel group, removing at least one channel group in the neural network to be trained to obtain a target neural network.
According to the embodiment, the channel group is cut based on the network scale limit information, so that the cut neural network is suitable for the load capacity of the mobile terminal to be deployed, even if the cut neural network can be deployed on the mobile terminal; when the channel group is cut, the unimportant channel group can be cut by combining the important degree information of the channel group, and the precision of the cut neural network is ensured.
In a possible embodiment, the rejecting at least one channel group in the neural network to be trained based on the network size limitation information, the network size information, and the importance degree information of each channel group includes:
determining the target number of the channel group to be eliminated based on the network scale limiting information and the network scale information;
and based on the importance degree information, rejecting the target number of channel groups.
In this embodiment, based on the network scale limit information and the network scale information corresponding to the channel groups, the number of the channel groups to be cut, that is, the target number, can be determined more accurately, and it is ensured that the cut neural network can be deployed on the mobile terminal.
In a possible implementation manner, the rejecting the target number of channel groups based on the importance information includes:
sequencing each channel group according to the sequence of the importance degrees from small to large;
and rejecting the channel groups with the sorting order less than or equal to the target number.
According to the embodiment, the channel groups are sorted according to the importance degree, and the channel groups with the sorting order less than or equal to the target number are removed, so that the precision of the cut neural network can be ensured, and the cut neural network can be deployed on the mobile terminal.
In one possible embodiment, the network size limitation information comprises at least one of:
network parameter amount limit information; network operation time consuming restriction information.
In a second aspect, an embodiment of the present disclosure provides an intelligent driving method, including:
detecting the road image by using a target neural network generated by using the neural network generation method provided by the first aspect of the present disclosure or any one of the embodiments of the first aspect to obtain a target object;
and controlling the intelligent driving equipment based on the target object obtained by detection.
In a third aspect, an embodiment of the present disclosure provides a neural network generation apparatus, including:
the grouping module is used for grouping the channels included in each network layer in the neural network to be trained respectively to obtain at least one channel group;
the importance determining module is used for determining importance degree information corresponding to the channel group;
and the channel removing module is used for removing at least one channel group in the neural network to be trained based on the network scale limiting information and the importance degree information of the channel group to obtain a target neural network corresponding to the neural network to be trained.
In a fourth aspect, an embodiment of the present disclosure provides an intelligent driving apparatus, including:
the acquisition module is used for acquiring a road image;
a detection module, configured to detect the road image by using a target neural network generated by using the neural network generation method provided in any one of the first aspect and the embodiments of the first aspect of the present disclosure, so as to obtain a target object;
and the control module is used for controlling the intelligent driving equipment based on the target object obtained by detection.
In a fifth aspect, an embodiment of the present disclosure further provides an electronic device, including: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating via the bus when the electronic device is running, the machine-readable instructions when executed by the processor performing the steps of the first aspect described above, or any one of the possible implementations of the first aspect, or the steps of the implementation of the second aspect.
In a sixth aspect, this disclosed embodiment also provides a computer readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to perform the steps in the first aspect, or any one of the possible implementation manners of the first aspect, or the steps in the implementation manner of the second aspect.
For the description of the effects of the neural network generating device, the electronic device, and the computer-readable storage medium, reference is made to the description of the neural network generating method, and details are not repeated here.
In order to make the aforementioned objects, features and advantages of the present disclosure more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly described below, and the drawings herein incorporated in and forming a part of the specification illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the technical solutions of the present disclosure. It is appreciated that the following drawings depict only certain embodiments of the disclosure and are therefore not to be considered limiting of its scope, for those skilled in the art will be able to derive additional related drawings therefrom without the benefit of the inventive faculty.
Fig. 1 illustrates a flow chart of a neural network generation method provided by an embodiment of the present disclosure;
fig. 2 shows a schematic diagram of a neural network generating device provided by an embodiment of the present disclosure;
fig. 3 shows a schematic diagram of an electronic device provided by an embodiment of the present disclosure.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be described clearly and completely with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only a part of the embodiments of the present disclosure, not all of the embodiments. The components of the embodiments of the present disclosure, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present disclosure, presented in the figures, is not intended to limit the scope of the claimed disclosure, but is merely representative of selected embodiments of the disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the disclosure without making creative efforts, shall fall within the protection scope of the disclosure.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
The term "and/or" herein merely describes an associative relationship, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, C, and may mean including any one or more elements selected from the group consisting of A, B and C.
In order to solve the problems that in the prior art, the requirement of a mobile terminal on the number of channels is difficult to meet by a tailored neural network, and the precision of the neural network is reduced or the neural network exceeds the load capacity of the mobile terminal after the number of channels is increased or reduced, the disclosure provides a neural network generation method and device, electronic equipment and a computer-readable storage medium. The method groups the channels of each network layer in the neural network, eliminates unimportant channel groups based on network scale limitation information and important degree information of the channel groups, obtains the number of target neural network channels which meets the specific requirement of the corresponding mobile terminal on the number of channels, does not need to increase or reduce the number of the channels subsequently, and can be directly deployed on the mobile terminal. The target neural network which can meet the requirements of equipment can be directly obtained through the channel group, the cutting speed and efficiency of the neural network are improved, and meanwhile, the precision of the target neural network obtained by cutting cannot be lost.
The following describes a neural network generation method, an apparatus, an electronic device, and a storage medium according to the present disclosure with specific embodiments.
As shown in fig. 1, the embodiment of the present disclosure discloses a neural network generation method, which can be applied to a server for cutting a large-scale neural network to obtain a target neural network satisfying both the scale and the precision. Specifically, the neural network generation method may include the steps of:
s110, grouping the channels of each network layer in the neural network to be trained to obtain at least one channel group.
Before grouping channels in a neural network to be trained, firstly, a mobile terminal to be deployed by a target neural network obtained by cutting needs to be determined, the requirement on the number of the channels in the target neural network is met, and then the channels of each network layer in the neural network to be trained are grouped based on the number of the channels required by the mobile terminal. For example, the number of channels of the target via the neural network required to be deployed on the mobile terminal is a multiple of 8, and then, when the channels are grouped, each channel group obtained by grouping may include 8 channels, 16 channels, 24 channels, and the like.
After channels of each network layer in the neural network to be trained are grouped, when the channels are cut, no matter a plurality of channel groups are cut, the rest channels can meet the specific requirement of the mobile terminal on the number of the channels.
And S120, determining importance degree information corresponding to each channel group.
The importance degree information can reflect the influence degree of the corresponding channel group on the detection precision of the target neural network, namely the importance degree information can reflect the importance degree of the corresponding channel group.
The importance information of each channel group can be determined by training the neural network to be trained. Or training the neural network to be trained, determining the importance degree information of each channel in the channel group, and determining the importance degree information of the corresponding channel group according to the importance degree information of each channel.
S130, based on the network scale limiting information and the importance degree information of each channel group, at least one channel group in the neural network to be trained is removed, and the target neural network is obtained.
The network size limitation information may include limitation information of network parameters of the target neural network by the mobile terminal (or referred to as network parameter limitation information) and/or limitation information of network operation time consumption of the target neural network (or referred to as network operation time consumption limitation information). The network parameter limiting information ensures that the load capacity of the tailored target neural network is smaller than that of the mobile terminal, the tailored target neural network can be deployed and operated on the mobile terminal, and the network time consumption limiting information ensures the detection efficiency of the tailored target neural network.
In a specific implementation, the network parameter limiting information may include a maximum parameter H, and the network time consumption limiting information may include a maximum network operation consumptionTime T. For example, the neural network to be trained comprises N network layers, and the number of channels in each network layer is Ci(i ═ 1, 2.., N), the number of channels in each network layer in the target neural network obtained after clipping is Ci' (i ═ 1, 2.., N). Wherein, C is more than or equal to 0i'≤CiWhen channel groups are rejected, a (C) needs to be satisfied1',C'2,...,C'N)≤H，b(C1',C'2,...,C'N)≤T，a(C1,C2,...,CN) To calculate the function of the neural network parameters, b (C)1,C2,...,CN) To calculate the function that the neural network takes to operate.
The network size limitation information may further include floating-point operation per second (FLOPS ) limitation information that specifies limitation information on the computation power of the target neural network obtained by the clipping.
In this case, the unimportant channel groups are eliminated based on the network size restriction information and the importance degree information of the channel groups. The target neural network obtained after the elimination can meet the limitation of the network scale limitation information on the network scale, namely the target neural network obtained after the elimination is slightly smaller than or equal to the load capacity of the mobile terminal and can be deployed on the mobile terminal. Meanwhile, the obtained target neural network can meet the requirements on computing power and detection efficiency of the target neural network.
If the channels of a layer are all clipped, it means that the layer is degraded to an identity transform layer without parameters.
In the embodiment, the channels of each network layer in the neural network are grouped, and the unimportant channel group is removed based on the network scale limitation information and the importance degree information of the channel group, so that the obtained number of the channels of each network layer in the target neural network meets the specific requirement of the corresponding mobile terminal on the number of the channels of each layer, and the number of the channels does not need to be increased or reduced subsequently and the like, and the target neural network can be directly deployed on the mobile terminal. The final target neural network mode is determined by utilizing the channel group, the speed is high, and the efficiency is high; and meanwhile, the precision of the target neural network obtained by cutting is not lost.
In some embodiments, the importance information corresponding to each channel group may be determined by the following steps:
and 11, constructing a loss function of the neural network to be trained by utilizing the gamma values corresponding to the channels in each channel group.
Here, the gamma value is used to characterize the importance of the corresponding channel. In particular implementation, the loss function may be constructed using the following formula:
in the formula, L represents a loss function, L (·) represents a main loss function, f (·) represents a function represented by a neural network to be trained, x represents an input of the neural network to be trained, i.e., a sample image, y represents a standard value corresponding to the sample image, W represents a parameter in the neural network to be trained, M represents the number of gamma values in one channel group, i.e., the number of channels in one channel group, g represents the number of channel groups, γ represents a gamma value corresponding to a channel, and λ represents a preset parameter. The loss function can ensure that the parameters in the same channel group can be synchronously changed in the training process.
Step 12, training the neural network to be trained by using the sample image until a training cut-off condition is met; and determining the optimized value of the gamma value corresponding to each channel according to the loss function value of the neural network when the training is finished.
The above formula utilizes the number of channels in each channel group and the gamma values corresponding to the channels in the channel group to construct a loss function, and can train to obtain an optimized value of the gamma values which can reflect the importance degree of each channel more accurately.
The optimized value of the gamma value can represent the importance degree of the corresponding channel, the greater the optimized value of the gamma value is, the higher the importance degree of the corresponding channel is, and conversely, the smaller the optimized value of the gamma value is, the lower the importance degree of the corresponding channel is.
And step 13, determining importance degree information corresponding to each channel group based on the optimized value of the gamma value corresponding to each channel.
After the optimized value of the gamma value corresponding to each channel is obtained, the importance degree information of a certain channel group is determined according to the following modes: and calculating the mean value of the optimized gamma values corresponding to all the channels included in the channel group, and taking the mean value as the importance degree information of the channel group.
The average value of the optimized gamma value corresponding to each channel in the channel group can reflect the importance degree of the corresponding channel group more accurately. The larger the mean value, the higher the importance of the corresponding channel group, and the smaller the mean value, the lower the importance of the corresponding channel group.
In addition, in the implementation, constraints may be added to the parameters of the convolution kernels corresponding to the channels to construct the loss function, or constraints may be added to the convolution layers to construct the loss function. And then, training the neural network to be trained by using the constructed loss function to obtain a corresponding optimized value, and determining the importance degree information of the corresponding channel group by using the optimized value, wherein the method is the same as or similar to the method and is not repeated here.
In the embodiment, the loss function is constructed by using the gamma values corresponding to the channels in each channel group, then, the optimized values of the gamma values corresponding to the channels are obtained through training, and the importance degree of each channel group can be more accurately determined by using the obtained optimized values.
After the importance information of each channel group is obtained, at least one channel group in the neural network to be trained can be specifically removed by using the following steps to obtain a target neural network:
and step 21, obtaining network scale information corresponding to one channel group.
Here, the network size information of all the channel groups is substantially the same, and therefore the network size information of any one channel group can be acquired. The network scale information corresponding to a certain channel group may include the network parameters and/or the operation time of the channel group.
And step 22, based on the network scale limiting information, the network scale information and the importance degree information of each channel group, removing at least one channel group in the neural network to be trained to obtain a target neural network.
As can be seen from the above description, the network size limitation information includes network parameter limitation information and/or network operation time consumption limitation information, and the network size information corresponding to the channel group may include the network parameter and/or operation time consumption of the channel group, so that the target number of the channel group to be removed can be determined by using the network size limitation information and the network size information corresponding to the channel group, and then the least important target number of channel groups may be removed based on the importance information.
Based on the network scale limit information and the network scale information corresponding to the channel groups, the number of the channel groups to be cut, namely the target number, can be determined more accurately, and the neural network obtained by cutting can be deployed on the mobile terminal.
After determining the target number of the channel groups to be removed, the above-mentioned removing the least important target number of channel groups based on the importance information may be specifically implemented by using the following steps:
firstly, sorting each channel group according to the sequence of the importance degrees corresponding to the importance degree information from small to large; and then eliminating the channel groups with the sorting order less than or equal to the target number.
The channel groups are sorted according to the importance degree, and the channel groups with the sorting order less than or equal to the target number are removed, so that the precision of the cut neural network can be ensured, and the cut neural network can be deployed on the mobile terminal.
In a specific implementation, the determining, by using the network scale constraint information and the network scale information corresponding to the channel group, the target number of the channel group to be removed may be:
firstly, determining the number of channel groups which should be included in a target neural network by using network scale limit information and network scale information corresponding to the channel groups; and then, subtracting the number of the channel groups which should be included in the target neural network from the number of the channel groups in the neural network to be trained to obtain the target number of the channel groups which need to be eliminated.
The network size limitation information includes maximum network size information of the target neural network, and the number of the channel groups to be included in the target neural network can be obtained by dividing the maximum network size information by the network size information corresponding to the channel groups.
The channel group is cut based on the network scale limit information, so that the cut neural network is smaller than or equal to the load capacity of the mobile terminal to be deployed, even if the cut neural network can be deployed on the mobile terminal; when the channel group is cut, the unimportant channel group can be cut by combining the important degree information of the channel group, and the precision of the cut neural network is ensured.
In the embodiment, when the loss function is constructed, the constraint is added to the channel group to cut the channel group, the cut of the convolutional layer can be realized by adding the constraint to the convolutional layer by using a method similar to the method, and the cut of other network layers can be realized by adding the constraint to other network layers.
The embodiment of the present disclosure further provides an intelligent driving method, including:
detecting the road image by using a target neural network generated by using the neural network generation method provided by the first aspect of the present disclosure or any one of the embodiments of the first aspect to obtain a target object;
and controlling the intelligent driving equipment based on the target object obtained by detection.
The intelligent driving device may include an autonomous vehicle, a robot, or a vehicle equipped with an advanced driver assistance system, among others.
Corresponding to the neural network generation method, the embodiment of the present disclosure further discloses a neural network generation device, and each module in the device can implement each step in the neural network generation method of each embodiment, and can obtain the same beneficial effect, and therefore, the description of the same part is omitted here. Specifically, as shown in fig. 2, the neural network generating apparatus includes:
the grouping module 210 is configured to group channels included in each network layer in the neural network to be trained, respectively, to obtain at least one channel group.
The importance determining module 220 is configured to determine importance information corresponding to the channel group.
And a channel removing module 230, configured to remove at least one channel group in the neural network to be trained based on the network scale limitation information and the importance information of the channel group, to obtain a target neural network corresponding to the neural network to be trained.
In some embodiments, the importance determining module 220, when determining the importance degree information corresponding to each channel group, is configured to:
constructing a loss function of the neural network to be trained by utilizing the gamma values corresponding to the channels in each channel group; the gamma value is used for representing the importance degree of the corresponding channel;
training the neural network to be trained by using a sample image until a training cut-off condition is met;
determining the optimized value of the gamma value corresponding to each channel according to the value of the loss function of the neural network when the training is finished;
and determining importance degree information corresponding to each channel group based on the optimized value of the gamma value corresponding to each channel.
In some embodiments, the importance determining module 220, when determining the importance information corresponding to each channel group based on the optimized value of the gamma value corresponding to each channel, is configured to:
calculating the mean value of the optimized gamma values corresponding to the channels included in a channel group aiming at the channel group;
and taking the calculated average value as the importance degree information of the channel group.
In some embodiments, the importance determining module 220, when constructing the loss function of the neural network to be trained by using the gamma values corresponding to the channels in each channel group, is configured to:
and constructing a loss function by using the number of channels in one channel group and the gamma values corresponding to the channels in each channel group.
In some embodiments, the channel culling module 230, when culling at least one channel group in the neural network to be trained based on the network scale constraint information and the importance information of each channel group to obtain a target neural network, is configured to:
acquiring network scale information corresponding to a channel group;
and based on the network scale limiting information, the network scale information and the importance degree information of each channel group, removing at least one channel group in the neural network to be trained to obtain a target neural network.
In some embodiments, the channel culling module 230, when culling at least one channel group in the neural network to be trained based on the network size limitation information, the network size information, and the importance information of each channel group, is configured to:
determining the target number of the channel group to be eliminated based on the network scale limiting information and the network scale information;
and based on the importance degree information, rejecting the target number of channel groups.
In some embodiments, the channel culling module 230, when culling the target number of channel groups based on the importance information, is configured to:
sequencing each channel group according to the sequence of the importance degrees from small to large;
and rejecting the channel groups with the sorting order less than or equal to the target number.
In some embodiments, the network size restriction information comprises at least one of:
network parameter amount limit information; network operation time consuming restriction information.
The embodiment of the present disclosure further provides an intelligent driving device, including:
the acquisition module is used for acquiring a road image;
a detection module, configured to detect the road image by using a target neural network generated by using the neural network generation method provided in any one of the first aspect and the embodiments of the first aspect of the present disclosure, so as to obtain a target object;
and the control module is used for controlling the intelligent driving equipment based on the target object obtained by detection.
Corresponding to the above neural network generation method, an embodiment of the present disclosure further provides an electronic device 300, as shown in fig. 3, which is a schematic structural diagram of the electronic device 300 provided in the embodiment of the present disclosure, and includes:
a processor 31, a memory 32, and a bus 33; the storage 32 is used for storing execution instructions and includes a memory 321 and an external storage 322; the memory 321 is also referred to as an internal memory, and is used for temporarily storing the operation data in the processor 31 and the data exchanged with the external memory 322 such as a hard disk, the processor 31 exchanges data with the external memory 322 through the memory 321, and when the electronic device 300 operates, the processor 31 communicates with the memory 32 through the bus 33, so that the processor 31 executes the following instructions:
grouping channels included in each network layer in a neural network to be trained respectively to obtain at least one channel group; determining importance degree information corresponding to the channel group; based on network scale limit information and the importance degree information of the channel groups, removing at least one channel group in the neural network to be trained to obtain a target neural network;
or cause the processor 31 to execute the following instructions:
detecting the road image by using a target neural network generated by the neural network generation method provided by the embodiment of the method to obtain a target object;
and controlling the intelligent driving equipment based on the target object obtained by detection.
The disclosed embodiments also provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, performs the steps of the neural network generation method in the above method embodiments, or performs the steps of the intelligent driving method in the above method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.
The computer program product of the neural network generation method provided in the embodiments of the present disclosure includes a computer-readable storage medium storing a program code, where instructions included in the program code may be used to execute the steps of the neural network generation method in the above method embodiments, or execute the steps of the intelligent driving method in the above method embodiments, which may be referred to in the above method embodiments specifically, and are not described herein again. The computer program product may be embodied in hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied in a computer storage medium, and in another alternative embodiment, the computer program product is embodied in a Software product, such as a Software Development Kit (SDK), or the like.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, apparatus, and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present disclosure may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present disclosure. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
Finally, it should be noted that: the above-mentioned embodiments are merely specific embodiments of the present disclosure, which are used for illustrating the technical solutions of the present disclosure and not for limiting the same, and the scope of the present disclosure is not limited thereto, and although the present disclosure is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive of the technical solutions described in the foregoing embodiments or equivalent technical features thereof within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present disclosure, and should be construed as being included therein. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.
Claims (13)
1. A neural network generation method, comprising:
grouping channels included in each network layer in a neural network to be trained respectively to obtain at least one channel group;
determining importance degree information corresponding to each channel group;
and based on the network scale limit information and the importance degree information of each channel group, removing at least one channel group in the neural network to be trained to obtain the target neural network.
2. The method according to claim 1, wherein the determining the importance information corresponding to each channel group comprises:
constructing a loss function of the neural network to be trained by utilizing the gamma values corresponding to the channels in each channel group; the gamma value is used for representing the importance degree of the corresponding channel;
training the neural network to be trained by using a sample image until a training cut-off condition is met;
determining the optimized value of the gamma value corresponding to each channel according to the value of the loss function of the neural network when the training is finished;
and determining importance degree information corresponding to each channel group based on the optimized value of the gamma value corresponding to each channel.
3. The method of claim 2, wherein determining the importance information corresponding to each channel group based on the optimized gamma value corresponding to each channel comprises:
calculating the mean value of the optimized gamma values corresponding to the channels included in a channel group aiming at the channel group;
and taking the calculated average value as the importance degree information of the channel group.
4. The method according to claim 2 or 3, wherein the constructing the loss function of the neural network to be trained by using the gamma values corresponding to the channels in each channel group comprises:
and constructing the loss function by using the number of channels in one channel group and the gamma value corresponding to the channel in each channel group.
5. The method according to any one of claims 1 to 4, wherein the rejecting at least one channel group in the neural network to be trained to obtain a target neural network based on the network size limitation information and the importance degree information of each channel group comprises:
acquiring network scale information corresponding to a channel group;
and based on the network scale limiting information, the network scale information and the importance degree information of each channel group, removing at least one channel group in the neural network to be trained to obtain a target neural network.
6. The method according to claim 5, wherein the rejecting at least one channel group in the neural network to be trained based on the network size limitation information, the network size information, and the importance degree information of each channel group comprises:
determining the target number of the channel group to be eliminated based on the network scale limiting information and the network scale information;
and based on the importance degree information, rejecting the target number of channel groups.
7. The method of claim 6, wherein said culling the target number of channel groups based on the importance information comprises:
sequencing each channel group according to the sequence of the importance degrees from small to large;
and rejecting the channel groups with the sorting order less than or equal to the target number.
8. The method according to any of claims 1 to 7, wherein the network size restriction information comprises at least one of:
network parameter amount limit information; network operation time consuming restriction information.
9. An intelligent driving method, comprising:
acquiring a road image;
detecting the road image by using a target neural network generated by the neural network generation method of any one of claims 1 to 8 to obtain a target object;
and controlling the intelligent driving equipment based on the target object obtained by detection.
10. A neural network generating apparatus, comprising:
the grouping module is used for grouping the channels included in each network layer in the neural network to be trained respectively to obtain at least one channel group;
the importance determining module is used for determining importance degree information corresponding to the channel group;
and the channel removing module is used for removing at least one channel group in the neural network to be trained to obtain the target neural network based on the network scale limit information and the importance degree information of the channel group.
11. An intelligent travel apparatus, characterized by comprising:
the acquisition module is used for acquiring a road image;
a detection module, configured to detect the road image by using a target neural network generated by the neural network generation method according to any one of claims 1 to 8, so as to obtain a target object;
and the control module is used for controlling the intelligent driving equipment based on the target object obtained by detection.
12. An electronic device, comprising: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating over the bus when the electronic device is running, the machine-readable instructions, when executed by the processor, performing the steps of the neural network generating method of any one of claims 1 to 8, or performing the steps of the intelligent driving method of claim 9.
13. A computer-readable storage medium, characterized in that a computer program is stored thereon, which computer program, when being executed by a processor, performs the steps of the neural network generating method according to any one of claims 1 to 8, or performs the steps of the intelligent driving method according to claim 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110334296.0A CN113052301B (en) | 2021-03-29 | 2021-03-29 | Neural network generation method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110334296.0A CN113052301B (en) | 2021-03-29 | 2021-03-29 | Neural network generation method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113052301A true CN113052301A (en) | 2021-06-29 |
| CN113052301B CN113052301B (en) | 2024-05-28 |
Family
ID=76515992
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110334296.0A Active CN113052301B (en) | 2021-03-29 | 2021-03-29 | Neural network generation method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113052301B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109242092A (en) * | 2018-09-29 | 2019-01-18 | 深圳市商汤科技有限公司 | Network acquisition and image processing method and device, electronic equipment, storage medium |
| US20190065884A1 (en) * | 2017-08-22 | 2019-02-28 | Boe Technology Group Co., Ltd. | Training method and device of neural network for medical image processing, and medical image processing method and device |
| CN109671020A (en) * | 2018-12-17 | 2019-04-23 | 北京旷视科技有限公司 | Image processing method, device, electronic equipment and computer storage medium |
| CN109740554A (en) * | 2019-01-09 | 2019-05-10 | 宽凳（北京）科技有限公司 | A kind of road edge line recognition methods and system |
| CN111860557A (en) * | 2019-04-30 | 2020-10-30 | 北京市商汤科技开发有限公司 | Image processing method and device, electronic equipment and computer storage medium |
| WO2021022685A1 (en) * | 2019-08-08 | 2021-02-11 | 合肥图鸭信息科技有限公司 | Neural network training method and apparatus, and terminal device |
-
2021
- 2021-03-29 CN CN202110334296.0A patent/CN113052301B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190065884A1 (en) * | 2017-08-22 | 2019-02-28 | Boe Technology Group Co., Ltd. | Training method and device of neural network for medical image processing, and medical image processing method and device |
| CN109242092A (en) * | 2018-09-29 | 2019-01-18 | 深圳市商汤科技有限公司 | Network acquisition and image processing method and device, electronic equipment, storage medium |
| CN109671020A (en) * | 2018-12-17 | 2019-04-23 | 北京旷视科技有限公司 | Image processing method, device, electronic equipment and computer storage medium |
| CN109740554A (en) * | 2019-01-09 | 2019-05-10 | 宽凳（北京）科技有限公司 | A kind of road edge line recognition methods and system |
| CN111860557A (en) * | 2019-04-30 | 2020-10-30 | 北京市商汤科技开发有限公司 | Image processing method and device, electronic equipment and computer storage medium |
| WO2021022685A1 (en) * | 2019-08-08 | 2021-02-11 | 合肥图鸭信息科技有限公司 | Neural network training method and apparatus, and terminal device |
Non-Patent Citations (1)
| Title |
|---|
| 朱金铭;邰阳;邹刘磊;范洪辉;朱洪锦;: "基于深度可分离卷积与通道裁剪的YOLOv3改进方法", 江苏理工学院学报, no. 02 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113052301B (en) | 2024-05-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2023024407A1 (en) | Model pruning method and apparatus based on adjacent convolutions, and storage medium | |
| CN111008631B (en) | Image association method and device, storage medium and electronic device | |
| CN114492799B (en) | Convolutional neural network model pruning method and device, electronic device, and storage medium | |
| CN106650928A (en) | Neural network optimization method and device | |
| CN112328715A (en) | Visual positioning method and related model training method and related devices and equipment | |
| CN113204714A (en) | User portrait based task recommendation method and device, storage medium and terminal | |
| CN116416561A (en) | Video image processing method and device | |
| CN116075821A (en) | Form convolution and acceleration | |
| CN111831894B (en) | Information matching method and device | |
| CN113554084A (en) | Vehicle re-identification model compression method and system based on pruning and light-weight convolution | |
| CN110568445A (en) | A Lightweight Convolutional Neural Network LiDAR and Vision Fusion Perception Method | |
| CN112232477B (en) | Image data processing method, device, equipment and medium | |
| CN111340223A (en) | Neural network compression method, target detection method, driving control method and device | |
| CN111221827B (en) | Database table connection method and device based on graphic processor, computer equipment and storage medium | |
| CN112465141A (en) | Model compression method, model compression device, electronic device and medium | |
| CN113344181B (en) | Neural network structure searching method and device, computer equipment and storage medium | |
| CN113971734A (en) | Target object detection method, device, electronic device and storage medium | |
| CN107563324A (en) | A kind of hyperspectral image classification method and device of the learning machine that transfinited based on core basis | |
| CN115439848A (en) | Scene recognition method, device, equipment and storage medium | |
| CN113052301A (en) | Neural network generation method and device, electronic equipment and storage medium | |
| CN113159297A (en) | Neural network compression method and device, computer equipment and storage medium | |
| CN117392386B (en) | Classification training method and device for superside mask generation network based on instance segmentation | |
| CN111325343B (en) | Neural network determination, target detection and intelligent driving control method and device | |
| CN111967579B (en) | Method and device for performing convolution calculation on image using convolutional neural network | |
| CN116204613A (en) | Vector matching method, device and storage medium for text travel question-answering system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |