CN116304685A - Method and device for generating countermeasure sample and electronic equipment - Google Patents

Abstract

The present application provides a method, device and electronic equipment for generating an adversarial sample. The method includes: acquiring training data, and a target neural network model trained according to the training data; using the training data as an input value, respectively calculating the average value of the output value of each layer of the target neural network model, and The average value of each layer is used as the output reference data of the layer; according to the output reference data, the target data is obtained through iterative calculation; in response to determining that a predetermined condition is met, the target data is used as an adversarial example. Through the method, the extent to which the generated adversarial samples are far from the normal sample manifold is limited, and the possibility of the adversarial samples being detected is reduced.

Description

A method, device, and electronic device for generating an adversarial sample

technical field

The present application relates to the technical field of artificial intelligence, and in particular to a method, device and electronic equipment for generating an adversarial example.

Background technique

Adversarial examples refer to input samples formed by deliberately adding subtle disturbances in the data set, causing the model to give a wrong output with high confidence. Using adversarial examples to train the target neural network model can improve the robustness of the target neural network.

In related technologies, a single loss function is generally used to measure the gap between the output label of an adversarial example and the real label, which limits and affects the generation of adversarial examples.

Contents of the invention

In view of this, the purpose of this application is to solve the limitations and problems affecting the generation of adversarial samples proposed in the background technology, and propose a method, device and electronic equipment for generating adversarial samples.

Based on the above purpose, this application provides a method for generating an adversarial example, including:

Acquiring training data, and a target neural network model trained according to the training data;

Using the training data as an input value, calculate the average value of the output value of each layer of the target neural network model respectively, and use the average value of each layer as the output reference data of this layer;

Obtaining target data through iterative calculation according to the output benchmark data;

In response to determining that a predetermined condition is met, the target data is used as an adversarial example.

Optionally, the obtaining the target data through iterative calculation according to the reference value includes performing the following steps iteratively until the predetermined condition is reached:

In response to determining to perform a first round of iterative calculation, using the training data as input data;

In response to determining that the iterative calculation is not performed in the first round, using the target data obtained in the previous round of iterative calculation as input data;

calculating a loss function value according to the output benchmark data and the input data;

Calculate and obtain temporary target data according to the loss function value and a predetermined disturbance coefficient;

calculating a disturbance value according to the temporary target data;

The disturbance value is superimposed on the input data to obtain the target data of the current round of iterative calculation.

Optionally, the calculation of the loss function value according to the output reference data and the input data includes:

determining the true category corresponding to the input data;

According to the input data, through the target neural network model, the output value and classification category of each layer of the target neural network model are obtained;

According to the output value of each layer of the target neural network model, the output reference data, the real category, the classification category and the input data, the loss function value is calculated by the following formula:

表示所述目标神经网络模型第i层的所述输出值与所述输出基准数据的欧式距离，/>

表示所述目标神经网络模型第i层的输出值，/>

表示所述目标神经网络模型第i层的输出基准数据。Among them, θ represents the parameters of the target neural network model, x represents the input data, y represents the true category, L(θ,x,y) represents the loss function value, C(θ,x,y) Represent the cross-entropy loss function value of the true category and the classification category, k represents the number of layers of the target neural network model, _ζi represents the mean square loss term coefficient,

Indicates the Euclidean distance between the output value of the i-th layer of the target neural network model and the output benchmark data, />

Indicates the output value of the i-th layer of the target neural network model, />

Indicates the output benchmark data of the i-th layer of the target neural network model.

Optionally, the calculating and obtaining temporary target data according to the loss function value and a predetermined disturbance coefficient includes:

According to the loss function value, the gradient value is obtained by calculating the backpropagation algorithm;

Superimpose the product of the disturbance coefficient and the gradient value on the input data to obtain temporary target data.

Optionally, the mean square loss term coefficient can be obtained by the following method:

Obtain the predetermined initial mean square loss term coefficient;

In response to determining to perform a first round of iterative calculation, the mean square loss term coefficient is the initial mean square loss term coefficient;

In response to determining to perform non-first round of iterative calculations, according to the output value and output reference data of each layer of the target neural network model in the previous round of iterative calculations, the target neural network in the previous round of iterative calculations is calculated by the following formula: The output distance of each layer of the network model:

表示计算上一轮次计算中所述所述目标神经网络模型的第i层输出值与第i层所述输出基准数据的差值的绝对值，所述/>

表示上一轮次得到的目标数据通过所述目标神经网络模型得到的第i层输出值，所述/>

表示所述输出基准数据；Among them, Dis _i represents the output distance of the i-th layer,

Indicates the absolute value of the difference between the output value of the i-th layer of the target neural network model in the previous round of calculation and the output reference data of the i-th layer, the />

Represents the i-th layer output value obtained by the target data obtained in the previous round through the target neural network model, the />

represents said output reference data;

Determine the layer with the largest output distance of the target neural network model in the previous round of iterative calculation, and set the coefficient of the mean square loss term of this layer to the initial mean square loss term coefficient of the preset value, and set the mean square loss term coefficient of other layers to The term coefficient is set to the initial mean squared loss term coefficient.

Optionally, the predetermined mean square loss term coefficient can also be obtained by the following method:

In response to determining to execute the third and above rounds of iterative calculations, the distances between the total output values of the target neural network model in the previous two rounds of iterative calculations are respectively calculated by the following formula:

Wherein, Dis _i represents the output distance of the i-th layer, and k represents the number of layers of the target neural network model;

In response to determining that the total output value distance calculated by the previous round of iteration is greater than the total output value distance calculated by the previous round of iteration, the mean square loss term coefficient is calculated by the following formula:

Among them, ζ _{i, t} represents the coefficient of the mean square loss term of the i-th layer of the current round, ζ _{i, t-1} represents the coefficient of the mean square loss term of the i-th layer of the previous round, Dis _i represents the previous round The output distance of the i-th layer of the target neural network model in the calculation, Dis _min represents the minimum value of the output distances of all layers of the target neural network model in the previous round of calculation.

Optionally, the calculation of the disturbance value according to the temporary target data includes:

According to the training data and the temporary target data, the disturbance value is obtained through the following calculation formula:

Among them, ε' is the disturbance, p is the temporary target data; ε is the preset maximum disturbance value; p _o is the input data.

Optionally, the predetermined condition includes that iterative calculation rounds reach a predetermined number of times.

Based on the same inventive concept, this application also provides an adversarial sample generation device, including:

An acquisition module configured to acquire training data and a target neural network model trained according to the training data;

The output reference data calculation module is configured to use the training data as an input value, respectively calculate the average value of the output value of each layer of the target neural network model, and use the average value of each layer as the output reference data of the layer ;

The target data calculation module is configured to obtain the target data through iterative calculation according to the output benchmark data;

A generating module configured to use the target data as an adversarial example in response to determining that a predetermined condition is met.

Based on the same inventive concept, the present application also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and operable on the processor. When the processor executes the program, any one of the above-mentioned The adversarial example generation method described in Item .

It can be seen from the above that the adversarial sample generation method provided by this application calculates the disturbance value of the adversarial sample by comprehensively considering the loss value between the output data of the target neural network and the target classification result, thereby obtaining the adversarial sample. The adversarial samples obtained by the above method are closer to the manifold geometry structure of the training data, so it has the advantage of being closer to the normal manifold geometry structure and not easily recognized by the detector based on manifold detection.

Description of drawings

In order to more clearly illustrate the technical solutions in the present application or related technologies, the following will briefly introduce the accompanying drawings that need to be used in the description of the embodiments or related technologies. Obviously, the accompanying drawings in the following description are only for this application Embodiments, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

FIG. 1 is a schematic flowchart of a method for generating an adversarial example in one or more embodiments of the present application;

FIG. 2 is a schematic structural diagram of an adversarial sample generation device in one or more embodiments of the present application;

FIG. 3 is a schematic diagram of a hardware structure of an electronic device according to one or more embodiments of the present application.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application shall have the usual meanings understood by those skilled in the art to which the present application belongs. "First", "second" and similar words used in the embodiments of the present application do not indicate any order, quantity or importance, but are only used to distinguish different components. "Comprising" or "comprising" and similar words mean that the elements or items appearing before the word include the elements or items listed after the word and their equivalents, without excluding other elements or items. Words such as "connected" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "Down", "Left", "Right" and so on are only used to indicate the relative positional relationship. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

A manifold is a space that locally has the properties of a Euclidean space, and is used in mathematics to describe geometric shapes. In physics, the phase space of classical mechanics and the four-dimensional pseudo-Riemannian manifold that constructs the space-time model of general relativity are examples of manifolds. One of the applications of manifolds is to reduce the dimensionality of data from high-dimensional space to low-dimensional space without loss of information by comprehensively considering the distance and topology of data.

Deep learning is to use the neural network model to discover the essence of the data by learning the internal laws and representation levels of the sample data, and then perform data analysis to realize functions such as target detection. But since their internal structures are mostly built based on linear blocks, the overall functions they implement are proven to be highly linear in some experiments. These linear functions are easy to optimize, but are also susceptible to rapid changes in the input data.

In order to improve the robustness of the target neural network, it is proposed in related technologies to use adversarial examples to train the target neural network.

As mentioned in the background, current methods for generating adversarial samples generally use a single loss function to measure the distance between the output value of the adversarial sample and the real label, and then correct the training data to obtain the adversarial sample. However, the adversarial samples generated by the above method fail to consider the manifold geometry between the training data, resulting in the output of the middle layer of the target neural network model will gradually deviate from the manifold of the training data, so that the flow of adversarial samples and training data There is a deviation between the graphs, entering the manifold of the target label. The adversarial examples generated by the above methods not only lose the information of the original training data, but also are difficult to pass the detection of the detector based on manifold detection. Therefore, the above methods limit and affect the generation of adversarial examples.

Therefore, this application proposes a method for generating adversarial samples, which not only considers the output value of each output layer of the target neural network model, the loss value between the model output value and the real label, but also considers the flow of adversarial samples and training data. In order to find the coupling area between the training data manifold and the adversarial sample manifold, it is determined that the adversarial sample enters the coupling area during the iterative calculation process.

Hereinafter, the technical solution of one or more embodiments in this specification will be described in detail through specific embodiments.

Referring to Figure 1, the method for generating an adversarial example in one or more embodiments of the present application includes the following steps:

Step S101: Obtain training data, and a target neural network model trained according to the training data.

In some embodiments, the training data may be images, text sequences and other data. In some embodiments, the original dataset may be an existing public dataset, such as CIFAR-10, MNIST, etc., or a custom dataset. In some embodiments, the target neural network model may be a neural network model such as ResNet34 or Inception v3.

Step S102: Using the training data as an input value, calculate the average value of the output value of each layer of the target neural network model, and use the average value of each layer as the output reference data of the layer.

In the process of implementing this application, the applicant found that the reason why the neural network model has advantages in the field of target recognition is that it has completed data dimensionality reduction on complex data sets, and will have complex manifold geometric structures, high-dimensional Dimensionality reduction of data with a variety of feature labels into simple manifold geometry, low-dimensional data with a small number of feature labels. In the process of data dimensionality reduction, the less data information loss, the higher the accuracy of its output value. In the process of data dimensionality reduction, the data dimension and data manifold geometry will change at each layer of the neural network model. Therefore, the adversarial example generation method proposed in this application comprehensively considers the output value of each layer of the target neural network model, in order to obtain an adversarial example with the least information loss.

In this step, each layer of the target neural network model will obtain its output data. Calculate the average value of the output data of each layer as the output benchmark data of this layer. In an embodiment of the present application, the target neural network model is a convolutional neural network, and the input data is at least one image data of 32*32 pixels. After the image data is input into the convolutional neural network, each intermediate layer of the convolutional neural network will output at least one output value, and the average value of the output value of each layer is calculated as the output benchmark of this layer data. Taking the first layer as an example, the output value of each output image data in this layer is 6 feature maps of 28*28 pixels. Therefore, this layer includes multiple feature maps of 28*28 pixels, the average value of each pixel is calculated, and the 28*28 pixel image composed of all the average values is used as the data reference data of this layer.

Step S103: Obtain target data through iterative calculation according to the output reference data.

According to the above content, in this step, based on the output benchmark data in step S102, the characteristics of the data and the geometric structure of the manifold are comprehensively considered, and the target data is obtained through iterative calculation.

In some embodiments, first determine the input data for iterative calculation; then calculate the loss value and distance value of the input data; finally calculate the disturbance value according to the loss value and distance value, and add the disturbance value to the Adversarial examples are obtained on the input data.

Specifically, in some embodiments, in response to determining to perform the first round of iterative calculations, the training data is used as input data; in response to determining to perform non-first round of iterative calculations, the target data obtained in the previous round of iterative calculations is used as Input data. That is, in some embodiments, when performing the first iterative calculation, the training data is used as the input data; in the subsequent iterative calculation, the target data obtained in the last round is used as the input data of the current round.

In some embodiments, the loss value of the input data is calculated by a cross-entropy function. The cross-entropy loss function is used to represent the difference between the probability distribution of output benchmark data and the probability distribution of output data in this round. In some embodiments, the distance value between the input data and the output reference data is calculated by Euclidean distance. Manifold is a space with local Euclidean space properties, and Euclidean distance can be used for distance calculation. In some embodiments, the loss function value is calculated by combining the above loss value and the distance value through the following formula:

表示所述目标神经网络模型第i层的所述输出值与所述输出基准数据的欧式距离，/>

表示所述目标神经网络模型第i层的输出值，/>

表示所述目标神经网络模型第i层的输出基准数据。Among them, θ represents the parameters of the target neural network model, x represents the input data, y represents the true category, L(θ,x,y) represents the loss function value, C(θ,x,y) Represent the cross-entropy loss function value of the true category and the classification category, k represents the number of layers of the target neural network model, _ζi represents the mean square loss term coefficient,

Indicates the Euclidean distance between the output value of the i-th layer of the target neural network model and the output benchmark data, />

Indicates the output value of the i-th layer of the target neural network model, />

Indicates the output benchmark data of the i-th layer of the target neural network model.

In some embodiments, according to the loss function value, the gradient value of the target neural network model is calculated by using a backpropagation algorithm.

In some embodiments, the product of the gradient value, the input data, and the disturbance coefficient is used as temporary target data.

In some embodiments, the perturbation system is acquired by predetermined means. In some embodiments, an initial disturbance coefficient is predetermined, and the disturbance coefficient is obtained according to the initial disturbance coefficient. In some embodiments, the disturbance coefficient is determined by the following method:

Obtain the predetermined initial mean square loss term coefficient;

In response to determining to perform a first round of iterative calculation, the mean square loss term coefficient is the initial mean square loss term coefficient;

In response to determining to perform non-first round of iterative calculations, according to the output value and output reference data of each layer of the target neural network model in the previous round of iterative calculations, the target neural network in the previous round of iterative calculations is calculated by the following formula: The output distance of each layer of the network model:

表示计算上一轮次计算中所述所述目标神经网络模型的第i层输出值与第i层所述输出基准数据的差值的绝对值，所述/>

表示上一轮次得到的目标数据通过所述目标神经网络模型得到的第i层输出值，所述/>

表示所述输出基准数据；Among them, Dis _i represents the output distance of the i-th layer,

Indicates the absolute value of the difference between the output value of the i-th layer of the target neural network model in the previous round of calculation and the output reference data of the i-th layer, the />

Represents the i-th layer output value obtained by the target data obtained in the previous round through the target neural network model, the />

represents said output reference data;

Determine the layer with the largest output distance of the target neural network model in the previous round of iterative calculation, and set the mean square loss term coefficient of this layer to 1.05 times the initial mean square loss term coefficient, and set the mean square loss term coefficients of other layers to Set to the initial mean squared loss term coefficient.

In some embodiments, in response to determining to perform the third and above rounds of iterative calculations, the distances between the total output values of the target neural network model in the previous two rounds of iterative calculations are respectively calculated by the following formula:

Wherein, Dis _i represents the output distance of the i-th layer, and k represents the number of layers of the target neural network model;

In response to determining that the total output value distance calculated by the previous round of iteration is greater than the total output value distance calculated by the previous round of iteration, the mean square loss term coefficient is calculated by the following formula:

Among them, ζ _{i, t} represents the coefficient of the mean square loss term of the i-th layer of the current round, ζ _{i, t-1} represents the coefficient of the mean square loss term of the i-th layer of the previous round, Dis _i represents the previous round The output distance of the i-th layer of the target neural network model in the calculation, Dis _min represents the minimum value of the output distances of all layers of the target neural network model in the previous round of calculation.

In some embodiments, the disturbance value is obtained according to the training data and the temporary target data through the following calculation formula:

Among them, ε' is the disturbance, p is the temporary target data; ε is the preset maximum disturbance value; p _o is the input data.

In some embodiments, the disturbance value is superimposed on the input data to obtain the target data of the current round of iterative calculation. In some embodiments, after the disturbance value is added to the input data, a maximum and minimum value check is performed on the obtained value, and the data is distributed within a predetermined value range through the maximum and minimum value check. In some embodiments, the maximum and minimum value checks are performed by the following formula:

Among them, p is the input data; γ _max is the predetermined maximum value; γ _min is the predetermined minimum value.

Step S104: In response to determining that a predetermined condition is met, use the target data as an adversarial example.

In some embodiments, the predetermined condition is that the number of iterations reaches a predetermined value.

In some embodiments, the predetermined value is predetermined according to past experimental results. The ultimate goal of an adversarial example is that its data manifold enters the coupled region of the training data manifold. In some embodiments, the average number of iterations for the manifold of adversarial samples entering the coupling region is determined through past experiments, and the predetermined value is set according to the average number of iterations.

It should be noted that the method in the embodiment of the present application may be executed by a single device, such as a computer or a server. The method of this embodiment can also be applied in a distributed scenario, and is completed by cooperation of multiple devices. In the case of such a distributed scenario, one of the multiple devices may only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete all described method.

It should be noted that some embodiments of the present application are described above. Other implementations are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in an order different from those in the above-described embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. Multitasking and parallel processing are also possible or may be advantageous in certain embodiments.

Based on the same inventive concept, the present application also provides an adversarial example generation device corresponding to the method in any of the above embodiments.

Referring to Figure 2, the device for generating an adversarial example includes:

An acquisition module configured to acquire training data and a target neural network model trained according to the training data;

The output reference data calculation module is configured to use the training data as an input value, respectively calculate the average value of the output value of each layer of the target neural network model, and use the average value of each layer as the output reference data of the layer ;

The target data calculation module is configured to obtain the target data through iterative calculation according to the output benchmark data;

A generating module configured to use the target data as an adversarial example in response to determining that a predetermined condition is met.

For the convenience of description, when describing the above devices, functions are divided into various modules and described separately. Of course, when implementing the present application, the functions of each module can be realized in one or more pieces of software and/or hardware.

The device in the above embodiment is used to implement the corresponding adversarial sample generation method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, and will not be repeated here.

Based on the same inventive concept, and corresponding to the method in any of the above embodiments, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and operable on the processor, the processor When the program is executed, the method for generating an adversarial example described in any one of the above embodiments is realized.

FIG. 3 shows a schematic diagram of a more specific hardware structure of an electronic device provided by this embodiment. The device may include: a processor 1010 , a memory 1020 , an input/output interface 1030 , a communication interface 1040 and a bus 1050 . The processor 1010 , the memory 1020 , the input/output interface 1030 and the communication interface 1040 are connected to each other within the device through the bus 1050 .

The processor 1010 may be implemented by a general-purpose CPU (Central Processing Unit, central processing unit), a microprocessor, an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute related programs to realize the technical solutions provided by the embodiments of this specification.

The memory 1020 may be implemented in the form of ROM (Read Only Memory, read only memory), RAM (Random Access Memory, random access memory), static storage device, dynamic storage device, and the like. The memory 1020 can store operating systems and other application programs. When implementing the technical solutions provided by the embodiments of this specification through software or firmware, the relevant program codes are stored in the memory 1020 and invoked by the processor 1010 for execution.

The input/output interface 1030 is used to connect the input/output module to realize information input and output. The input/output/module can be configured in the device as a component (not shown in the figure), or can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, and the like.

The communication interface 1040 is used to connect a communication module (not shown in the figure), so as to realize the communication interaction between the device and other devices. The communication module can realize communication through a wired method (such as USB, network cable, etc.), or can realize communication through a wireless method (such as mobile network, WIFI, Bluetooth, etc.).

Bus 1050 includes a path that carries information between the various components of the device (eg, processor 1010, memory 1020, input/output interface 1030, and communication interface 1040).

It should be noted that although the above device only shows the processor 1010, the memory 1020, the input/output interface 1030, the communication interface 1040, and the bus 1050, in the specific implementation process, the device may also include other components. In addition, those skilled in the art can understand that the above-mentioned device may only include components necessary to implement the solutions of the embodiments of this specification, and does not necessarily include all the components shown in the figure.

The electronic device in the foregoing embodiments is used to implement the corresponding adversarial sample generation method in any of the preceding embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

Those of ordinary skill in the art should understand that: the discussion of any of the above embodiments is exemplary only, and is not intended to imply that the scope of the application (including claims) is limited to these examples; under the thinking of the application, the above embodiments or The technical features in different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of the different aspects of the embodiments of the application as described above, which are not provided in details for the sake of brevity.

In addition, to simplify illustration and discussion, and so as not to obscure the embodiments of the present application, well-known power supply/connection circuits associated with integrated circuit (IC) chips and other components may or may not be shown in the provided figures. ground connection. Furthermore, devices may be shown in block diagram form in order to avoid obscuring the embodiments of the present application, and this also takes into account the fact that details regarding the implementation of these block diagram devices are highly dependent on the implementation of the embodiments of the present application to be implemented. platform (ie, the details should be well within the purview of those skilled in the art). Where specific details (e.g., circuits) have been set forth to describe example embodiments of the present application, it will be apparent to those skilled in the art that reference may be made without or with variation from these specific details. Implement the embodiment of the present application below. Accordingly, these descriptions should be regarded as illustrative rather than restrictive.

Although the application has been described in conjunction with specific embodiments thereof, many alternatives, modifications and variations of those embodiments will be apparent to those of ordinary skill in the art from the foregoing description. For example, other memory architectures such as dynamic RAM (DRAM) may use the discussed embodiments.

The embodiments of the present application are intended to embrace all such alternatives, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent replacements, improvements, etc. within the spirit and principles of the embodiments of the present application shall be included within the protection scope of the present application.

Claims (10)

1. A method of challenge sample generation, comprising:

acquiring training data and training a target neural network model according to the training data;

taking the training data as input values, respectively calculating the average value of the output values of each layer of the target neural network model, and taking the average value of each layer as output reference data of the layer;

obtaining target data through iterative computation according to the output reference data;

in response to determining that a predetermined condition is reached, the target data is taken as an challenge sample.

2. The challenge sample generation method according to claim 1, wherein the obtaining target data by iterative calculation from the reference value includes iteratively performing the steps of until the predetermined condition is reached:

in response to determining to perform a first round of iterative computations, taking the training data as input data;

in response to determining to perform non-first round of iterative computation, taking target data obtained by the previous round of iterative computation as input data;

calculating a loss function value according to the output reference data and the input data;

calculating to obtain temporary target data according to the loss function value and a preset disturbance coefficient;

calculating to obtain a disturbance value according to the temporary target data;

and superposing the disturbance value to the input data to obtain target data of the iterative calculation of the round.

3. The challenge sample generation method of claim 2, wherein the calculating a loss function value from the output reference data and the input data comprises:

determining the real category corresponding to the input data;

according to the input data, obtaining an output value and a classification category of each layer of the target neural network model through the target neural network model;

according to the output value, the output reference data, the real category, the classification category and the input data of each layer of the target neural network model, a loss function value is calculated according to the following formula:

wherein θ represents a parameter of the target neural network model, x represents the input data, y represents the real class, L (θ, x, y) represents the loss function value, C (θ, x, y) represents a cross entropy loss function value of the real class and the classification class, k represents the number of layers of the target neural network model, ζ _i Represents the mean square loss term coefficient of the i-th layer,

representing the Euclidean distance between the output value of the ith layer of the target neural network model and the output reference data,/I>

Representing the output value of the ith layer of the target neural network model,/for>

And the output reference data of the ith layer of the target neural network model is represented.

4. A challenge sample generation method according to any one of claims 2 or 3, wherein the calculating temporary target data from the loss function value and a predetermined disturbance factor comprises:

according to the loss function value, calculating to obtain a gradient value through a back propagation algorithm;

and superposing the product of the disturbance coefficient and the gradient value with the input data to obtain temporary target data.

5. The challenge sample generation method of claim 4, wherein the mean square loss term coefficient is obtained by:

acquiring a preset initial mean square loss term coefficient;

performing a first round of iterative computations in response to determining that the mean square loss term coefficient is the initial mean square loss term coefficient;

in response to determining to execute the non-first round of iterative computation, according to the output value and the output reference data of each layer of the target neural network model in the previous round of iterative computation, calculating the output distance of each layer of the target neural network model in the previous round of iterative computation through the following formula:

wherein Dis _i Indicating the output distance of the i-th layer,

representing the calculation described in the previous round of calculationThe absolute value of the difference between the i-th layer output value of the target neural network model and the i-th layer output reference data, the +.>

Representing the output value of the ith layer of the target data obtained in the previous round through the target neural network model, wherein +.>

Representing the output reference data;

and determining a layer with the largest output distance of the target neural network model in the previous iteration calculation, setting the mean square loss term coefficient of the layer as a preset value times of the previous mean square loss term coefficient, and setting the mean square loss term coefficients of other layers as the previous mean square loss term coefficient.

6. The challenge sample generation method of claim 5, wherein the predetermined mean square loss term coefficient is further obtainable by:

in response to determining to perform the third and more rounds of iterative computations, respectively computing a total output value distance of the target neural network model in the previous two rounds of iterative computations by the following formula:

wherein Dis _i Representing the output distance of an ith layer, wherein k represents the number of layers of the target neural network model;

in response to determining that the total output value distance of the previous iteration calculation is greater than the total output value distance of the previous iteration calculation, the mean square loss term coefficient is calculated by the following formula:

wherein ζ _i,t The mean square loss term coefficient, ζ, representing the ith layer of the present round _i,t-1 The mean square loss term coefficient, dis, representing the ith layer of the previous round _i Representing the output distance of the ith layer of the target neural network model in the previous calculation, dis _min Representing the minimum of the output distances of all layers of the target neural network model in the previous round of computation.

7. The challenge sample generation method according to any one of claims 5 or 6, wherein the calculating a disturbance value from the temporary target data includes:

obtaining the disturbance value according to the training data and the temporary target data through the following calculation formula:

wherein epsilon' is disturbance and p is temporary target data; epsilon is a preset disturbance maximum value; p is p _o For input data.

8. The challenge sample generation method of claim 2, wherein the predetermined condition comprises a round of iterative computation reaching a predetermined number of times.

9. An challenge sample generating device, comprising:

the acquisition module is configured to acquire training data and a target neural network model obtained by training according to the training data;

an output reference data calculation module configured to calculate an average value of output values of each layer of the target neural network model using the training data as input values, respectively, and use the average value of each layer as output reference data of the layer;

the target data calculation module is configured to obtain target data through iterative calculation according to the output reference data;

a generation module configured to take the target data as a challenge sample in response to determining that a predetermined condition is reached.

10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any one of claims 1 to 8 when the program is executed by the processor.

Images