Generating adversarial examples for DNN using pooling layers

Abstract

Deep Neural Network is an application of Big Data, and the robustness of Big Data is one of the most important issues. This paper proposes a new approach named PCD for computing adversarial examples for Deep Neural Network (DNN) and increase the robustness of Big Data. In safety-critical applications, adversarial examples are big threats to the reliability of DNNs. PCD generates adversarial examples by generating different coverage of pooling functions using gradient ascent. Among the 2707 input images, PCD generates 672 adversarial examples with L_∞ distances less than 0.3. Comparing to PGD (state-of-art tool for generating adversarial examples with distances less than 0.3), PCD finds 1.5 times more adversarial examples than PGD (449) does.

Keywords

Deep neural network robustness coverage big data

1 Introduction

Recent researches show that Machine Learning (ML) algorithms based on Big Data, especially Deep Neural Networks (DNN), have achieved great success in solving complex tasks with near (or beyond) human-intelligence (with acceptable accuracy). Powered by massive amount of data, DNNs are increasingly being utilized in several domains, including speech interpreting [1, 2], image recognition [3, 4] and game playing [5, 6]. The average accuracy of modern Deep Neural Networks are pretty high, which allows them to apply in real-world applications and tasks.

But for some of the security (safety)-critical tasks, including autonomous driving [7 –9], malware detection [10 –12], and cyber-security [13 –15], there are more concerns need to be addressed other than accuracy.

One of the main problems in these areas is the robustness of DNN towards adversarial examples. Great effort have been made to build reliability DNNs, including [16 –20]. With less differences from the original input, the adversarial examples are more likely to active the same part of neurons in DNNs. Therefore, the weight matrices of the active neurons are more accurate and robust.

There are also several approaches to generate adversarial examples, such as gradient based attacks [21] spatial based attack [22] and attacks from real world (brightness and contrast) [23].

Here, this paper proposes a new adversarial attack PCD to challenge adversarial trained DNN proposed by Madry et al. in paper [18]. In the paper, Mardy et al. proposed PGD attack and generate adversarial examples with L_∞ less than 0.3 using MNIST data-set.

In the new approach, PCD first generates coverage of pooling functions and guides the generating of adversarial examples with the coverage. PCD uses gradient ascent to generate coverage for pooing functions.

PCD attacks the adversarial-trained DNN in Mardy et al. paper [18] and follows the same constraint established in that paper, such that the L_∞ distance between adversarial examples and original images should be less than 0.3. With the constraint, PCD is able to compute more useful adversarial examples for the DNN in [18] than PGD does.

PCD selected 2707 images as testing benchmark, PCD found 672 images with adversarial examples, which is 1.5 times more than PGD does (449).

There are three main contributions in the new proposed approach. Firstly, PCD finds more adversarial examples than PGD. With the 2707 images, PCD PCD finds 672 images with adversarial examples, while PGD finds 449 images with adversarial examples.

Secondly, PCD first uses the coverage of pooling functions in DNNs. Pooling Layers and pooling functions are widely used in DNNs, and the coverage of pooling functions are easily generated in PCD. With the coverage of pooling functions, PCD generates adversarial examples.

Thirdly, PCD generates multiple adversarial examples for a single input image, which is helpful to increase the robustness of adversarial trained DNNs.

By generating different coverage for different pooling functions in the DNN, PCD generates multiple adversarial examples for one image without randomization.

2 Related work

There are three other work that also use coverage in adversarial examples generating. DeepXplore [23] uses neuron coverage to generate adversarial examples, DeepGauge [24] aims at covering a range of values using activated neurons, and DeepCover [25] aims at generating a MC/DC coverage for a layer in DNN.

DeepXplore tries to increase the neuron value of a select neuron. Different from PCD, DeepXplore only choose one neuron to cover, and it only tries to increase the value that selected neuron. While PCD tries to increase the value of neurons in the input set of a pooling functions.

DeepGauge tries to cover a value range with activated neurons. It finds the neurons that are closet either to the higher or lower boundary of the range, and either increase or decrease the value of neurons. Comparing to PCD, DeepGauge has to decide the value of range with some strategies and generates the loss function, while PCD is able to generate the loss function based on each pooling functions automatically.

DeepCover tries to generate a MC/DC coverage for a layer in DNN. It tries to find a neuron n (c, i) such that v (c + 1,) * v (c, i) <0 and for all j ≠ i, v (c, j) * (v (c + 1, j) >0. DeepCover finds the coverage for layers, and PCD tries to find the coverage for pooling functions.

In the paper that proposed PGD attacks, the DNN N used in PCD was proposed in Madry, Aleksander et al. And N is also the target DNN of PGD attacks. In the paper, N was trained using a min-max optimization to find a large number of adversarial examples, and add them to the training set of N. In this way, N was adversarial trained and the robustness of N towards adversary is better than other DNN.

3 Preliminaries and approaches

3.1 Preliminaries

The DNN considered in this paper is composed of an input layer, an output layer and multiple hidden layers. A layer is composed of multiple neurons, connected to the neurons in the previous layer with a set of weights and activation functions. The set of weights is well-trained in the training phase. There are three types of activation functions used in this paper: convolution functions, pooling functions and Relu functions. There are also three types of hidden layers: convolution layers, pooling layers and fully connected layers.

Given a determined input image x, DNN N is able to recognize i as a digit number from 0 to 9. The output of N with input x is denoted as N (x).

Suppose there are n layers in N, then layer m₁ is the input layer, and layer m_n is the output layer, layers m₂ . . . . m_n-1, are hidden layers. The m_n-1 layer is denoted as the logit layer. For a 2D-matrix m, there are only two dimensions (columns and rows) in m. A Multi-Dimensional Matrix m′ is an extension of 2D-matrix and use additional subscripts for indexing. If there are N subscripts in m′, then m′ is a ND-matrix.

An input image x is a 2D-matrix, a layer m_i in N is a 4D-matrix, the weight matrix w₁ of m_i is a 4D-matrix.

For a 4D-matrix m, PCD uses m [x, y, z, n] to denote an element in m.

Definition 1 [Neuron]

For a layer c in a given DNN Ninnn, if a neuron is the i-th neuron in layer c, then the neuron is denoted asn (c, i).

A neuron n (c, i) is a 3D-matrix. The weight matrix w (c, i) of n (c, i) is a 3D-matrix.

PCD uses v (c, i) to denote the value of neuron n (c, i) with original input image x.

PCD uses v (c, i, k) _x (short as v (c, i, k) if x is not specified) to denote the value of n (c, i) with image x as input.

In the DNN N used in this paper, there are 10 neurons in layer m_n-1. Neuron n (n - 1, i) is called the i_tk label inN, v (n - 1, i) shows the possibility that N distinguishes the input image as the i_tk digit number. There are also 10 neurons in m_n-2, the values of neurons in m_n-1 is the softmax results of values of neurons in m_n-2. v (n - 1, i) is greater than v (n - 1, j) if and only if v (n - 2, i) is greater thanv (n - 2, j).

For an input image x, the neuron (label) that has the highest value in n - 2 layer is n (n - 2, l) , and the neuron (label) that has the second highest value in n-2 layer is n(n - 2, s). The value difference between n (n - 2, l) and n (n - 2, s) isg (x) .

Layers are composed of neurons, for example, in a hidden layer m_c, suppose the size of m_c is 28 * 28 * 16 * 32, then m_c[0 : 28, 0 : 28, 0 : 16, i] is the matrix of neuron n (c, i). In other words, the matrix m of n (c, i) is a 3D-sub-matrix of m_c.

For an image x′, x′ is a k - mutated image of x if the L_∞ distance between x and x’ is less than k. In this paper, k equals to 0.3.

Definition 2 [k-Adversarial Example]

For an input image x, and a k - mutated image x′, it N (x) is different from N (x′) , then x′ is a k-Adversarial Example of x.

3.2 Approaches

3.2.1 Pooling functions

In Deep Neural Network, Pooling Functions are used to adjust the scale and complexity of the DNN. The pooling functions used in this paper are 2 * 2 pooling functions. There are two pooling layers m_p1 and m_p1 with pooling functions in it. PCD tries to generate a coverage for each pooling function. Each neuron value v (p, i) [x, y, z] in a pooling layer p is the output of a pooling function pf. The input of pf is a 2 * 2 matrix, represented as v (p, i - 1) [x : x + e, y : y + e, z], where 0 < e < 2. In other words:

pf : v (p, i - 1) [x : x + e, y : y + e, z] → v (p, i)[x, y, z] , and v (p, i) [x, y, z] : = max (v (p, i - 1) [x : x + e, y : y + e, z]) . Suppose v (p, i - 1) [m_a, m_b, z]= = v (p, i, z), then m_a and m_b are called the maximal indices of v (p - 1, i) [x : x + e, y : y + e, z] .

3.2.2 Coverage of pooling functions

For an input v(p-1, i)[x:x+e, y:y+e] of a pooling function pf, if there exists another k * k matrix v′ (p - 1, i) [x : x + e, y : y + e] such that the maximal indices of v (p - 1, i) [x : x + e, y : y + e] and v′ (p - 1, i) [x : x + e, y : y + e] are different, then v′ (p - 1, i) [x : x + e, y : y + e] covers a different output situation of pf, and PCD considers it as the coverage of v (p - 1, i) [x : x + e, y : y + e] . For an input image x, and a k-Mutated image x′ of $x, if v (p - 1, i) [x : x + e, y : y + e] _x′ is a coverage of v (p - 1, i) [x : x + e, y : y + e] _x′, then x′ is a pooling coverage of x . x′ might be a k -Adversarial Example of x.

3.2.3 Gradient ascent based coverage generation

For a pooling function pf, an input matrix v(p-1, i)[x:x+e, y:y+e], and the maximal indices m_a, m_b,PCD tries to decrease the value of v (p - 1, i) [m_a, m_b] and increase the value of other elements in the input matrix to generate a coverage for the input. The loss function is the sum of element values in the input matrix minus two times the value of v (p - 1, i) [m_a, m_b].

PCD uses the gradient function in tensorflow to compute the gradient of input layer towards the loss value. PCD also uses the clip function in numpy package to guarantee that the changes in the input image x is less than k (L_∞ distance). In this way, PCD generates k-Mutated image x′ of an original input image x. PCD then feeds the DNN N with x’ and decide whether x’ is a k-Adversarial Example of x or not. If N (x) ≠ N (x′), then x′ is a k-Adversarial Example, otherwise, x′ is not.

The detail algorithms in PCD of generating a k -Adversarial Example is presented in Algorithm 1.

In Algorithm 1, the input is a pooling function p f with n (p - 1, n) [x : x + e, y : y + e, z] as input. At Line 2, v computes the maximum value among the input using the pooling function. Starting at Line 3, PCD finds the maximum indices of the maximum by searching the input one by one. At Line 7 and the following line, PCD computes the loss function to generate mutated images. The loss function is the sum of element value among the input matrix minus 2 times the value of the maximum value (m at Line 2). If the generated k-Mutated image x′ has a different output from x with N. Then x′ is returned as a k-Adversarial Example. Otherwise, the algorithm returns null.

3.2.4 PCD is useful for all DNN with pooling functions

PCD uses pooling functions to generate adversarial examples of a given DNN. Therefore, PCD is applicable to all DNNs with Pooling Layers. Moreover, Pooling Layers are widely used in Deep Neural Network, therefore, PCD is useful for most of the current DNN model.

4 Experimental results and evaluation

4.1 Experiments on finding adversarial examples

4.1.1 DNN and testing images set

PCD uses the adversarial trained DNN N in Mardy’s paper and generates adversarial examples of Ninnn. Comparing to other DNNs, N is more reliable towards adversarial attacks [18].

In Mardy et al. paper [18], all adversarial examples find by PGD are 0.3-Adversarial Example, since any adversary that is allowed to perturb each pixel by more than 0.5 can construct a uniformly gray image, thus fooling any classifier [18]. PCD also applies the same constraint and only finds 0.3-Adversarial Example for the input images. The runtime limit of PCD is 1 hour and the memory limit of PCD is 32G.

As defined in Section 3, g (x) is the difference between the maximal neuron value v (n - 2, l) and the second maximal neuron value v (n - 2, s) in m_n-2 layer. The range of g(x) is (0 + ∞)

The testing images set T of PCD is a subset of testing images in MNIST with 2707 images, such that for every xinT, g (x) <7. For image x′ such that g (x′) > =7, PCD normally ran out of the computing resources to generate adversarial examples of x′.

P is the set of images that adversarial examples find by PGD. P is a subset of T, the number of images in P is 449.

PCD separates T and P into 7 different groups.

The first and second columns show the ID of the groups and the range of g (x′) in each group. The third and last columns show the number of images that belongs to the groups in P and T respectively.

4.1.2 Adversarial examples find by PCD

Figure 1 shows the original images and adversarial examples that are found by PCD and PGD.

Fig.1

Adversarial examples generated by PGD and PCD.

Less pixels are changed by PCD to generate an adversarial example than PGD does. It means that the gradient used in PCD is more efficient than that used in PGD.

Figure 2 shows the original images, the adversarial examples that PCD finds and the images that PGD generates (not adversary).

Fig.2

Images with adversarial examples only generated PCD.

There are minor differences between the images generated using PCD and PGD in the green rectangle.

PGD can not find an adversarial example from the images because the gradients it used were rigid, and PCD is able to increase the flexibility of gradients by using the coverage of pooling functions.

4.1.3 Number of images with adversarial examples

Table 2 shows the number of images with adversarial examples find in the experiments using PCD or PGD.

Table 1
Groups of testing data in MNIST

Group Range of g(x) P T

1 (0,1] 78 129

2 (1,2] 100 191

3 (2,3] 93 236

4 (3,4] 86 358

5 (4,5] 46 523

6 (5,6] 34 469

7 (6,7] 12 801

Group	Range of g(x)	P	T
1	(0,1]	78	129
2	(1,2]	100	191
3	(2,3]	93	236
4	(3,4]	86	358
5	(4,5]	46	523
6	(5,6]	34	469
7	(6,7]	12	801

Table 2

Adversarial examples in each group

Group	Only PCD	Only PGD	All PCD	All PGD	All images
1	26	0	99	78	129
2	59	1	154	100	191
3	58	1	151	93	236
4	37	0	123	86	358
5	22	0	68	46	523
6	19	0	54	34	469
7	11	0	23	12	801

The first column is the ID of the group, the second and third columns show the number of images with adversarial examples only find by PCD and PGD respectively. The fourth and fifth columns show the number of images with adversarial examples find by PCD and PGD respectively. The last column shows the number of images in each group.

Comparing to PGD, PCD is able to find 232 more adversarial examples in T, and PGD is able to find 2 more adversarial examples than PCD in T.

PCD finds 1.5 times more adversarial examples than PGD does.

For the set of P, PCD finds 287 images with adversarial examples without adding the xent value to loss function. After adding xent value to loss function, PCD finds 447 images with adversarial examples.

PCD finds more adversarial examples in all the 7 groups in T and PGD only finds 2 more adversarial examples than PCD does in Group 2 and 3.

Also, PCD is able to generate multiple adversarial examples for one image without the help of randomization. There are 3.24e + 4 more adversarial examples generated by PCD with the 679 images with adversarial examples in T.

Relations Between g (x) and Adversary

In Table 2, the number of images with adversarial examples find by either tools decreases dramatically in group 5 (i.e., images that g (x) is greater than 4), it means that for both PCD and PGD, find adversarial examples becomes harder when g (x)is greater than 4.

PCD is still able to find 1.57 times more adversarial examples than PGD for g (x) greater than 4. It means that thought it becomes harder to find adversarial examples for images with g (x) greater than 4, the advantage for PCD is not weaken by the increasing of g (x).

5 Conclusion

In this paper, PCD proposed a new approach to generate adversarial examples with adversarial trained DNN.

Experiments show that for the selected 2707 images, PCD is able to generate around 3.2e + 4 adversarial examples from 672 images. PCD generates 222 more adversarial examples than PGD does, and the advantage of finding more adversarial examples is not weaken by the increasing of g (x).With more adversarial examples generated, the robustness of N will be increased.

PCD only generates one test case for a pooling function (neuron), actually, for a k * k pooling function (neuron), there are also k * k possibilities of maximum neurons in the previous layers. From complete testing coverage perspective, all the k * k possible output situations should be covered by different test cases.

References

Xiong

, Droppo

and Huang

, Toward Human Parity in Conversational Speech Recognition, IEEE/ACM Transactions on Audio, Speech, and Language Processing 2017 25(12) (2017), 2410–2423.

Hinton

, Deng

and Yu

, Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups, IEEE Signal processing magazine 29(6) (2012), 82–97.

, Zhang

and Ren

, Deep residual learning for image recognition, CVPR (12) (2016), 770–778.

Shin

Hoo Chang

, Roth , Holger

, Gao

Mingchen

, et al., Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset, IEEE Transactions on Medical Imaging 35(5) (2016), 1286–1298.

Silver

, Aja

and Chris

, Mastering the game of Go with deep neural networks and tree search, Nature 529(7587) (2016), 484–489.

Silver

, Schrittwieser

and Simonyan

, Mastering the game of Go without human knowledge, Nature 550(7676) (2017), 354–359.

Tian

, Pei

and Jana

, DeepTest: Automated testing of deep-neural-network-driven autonomous cars, ICSE (2018), 303–314.

Sallab

A.E.

, Abdou

, Perot

and Yogamani

, Deep reinforcement learning framework for autonomous driving, Electronic Imaging (2017), 70–76.

Zhang

, Zhang

and Zhang

, DeepRoad: GAN-based Metamorphic Autonomo Driving System Testing, arXiv preprint arXiv: 1802.02295 (2018).

10.

Huang

and Stokes

J.W.

, MtNet: A multi-task neural network for dynamic malware classification, DIMVA (2018), 399–418.

11.

Tobiyama

, Yamaguchi

and Shimada

, Malware detection with deep neural network using process behavior, COMPSAC 2 (2016), 577–582.

12.

, Zhang

and Jayasena

, Hadm: Hybrid analysis for detection of malware, IntelliSys, IEEE (2016), 702–724.

13.

Rosenberg

, Sicard

and David

E.O.

, DeepAPT: Nation-state APT attribution using end-to-end deep neural networks, ICANN (2017), 91–99.

14.

Norton

A.P.

and Qi

, Adversarial-Playground: A visualization suite showing how adversarial ex- amples fool deep learning, VizSec, IEEE (2017), 1–4.

15.

Bezemskij

, Loukas

, Gan

, et al., Detecting cyber-physical threats in an au- tonomous robotic vehicle using Bayesian Networks, iThings, IEEE (2017), 98–103.

16.

Ruan

, Huang

and Kwiatkowska

, Reachability analysis of deep neural networks with provable guarantees, arXiv preprint arXiv:1805.02242 (2018).

17.

Carlini

and Wagner

, Towards evaluating the robustness of neural networks, IEEE Symposium on Security and Privacy (5) (2017), 39–57.

18.

Madry , Aleksander , Makelov

Aleksandar

and Schmidt

Ludwig

, Towards deep learning models resistant to adversarial attacks, ICLR (2018).

19.

Selsam

, Liang

and Dill

D.L.

, Developing bug-free machine learning systems with formal mathematics, arXiv preprint arXiv:1706.08605 (2017).

20.

Katz

, Barrett

and Dill

D.L.

, Reluplex: An efficient SMT solver for verifying deep neural networks, CAV pp. 97–117 (2017).

21.

Elsayed

G.F.

, Shankar

and Cheung

, Adversarial examples that fool both human and computer vision, arXiv preprint arXiv:1802.08195 (2018).

22.

Xiao

, Zhu

J.Y.

and Li

, He, Spatially transformed adversarial examples, arXiv preprint arXiv:1801.02612 (2018).

23.

Pei

, Cao

, Yang

and Jana

, Deepxplore: Automated whitebox testing of deep learning systems, Symposium on Operating Systems Principles (2017), 1–18.

24.

Xie

and Ma

, Coverage-Guided Fuzzing for Deep Neural Networks, arXiv preprint arXiv:1809.01266 (2018).

25.

Wang

and Pei

, Whitehouse, Formal Security Analysis of Neural Networks using Symbolic Intervals, arXiv preprint arXiv:1804.10829 (2018).

26.

Sun

, Wu

and Ruan

, Concolic Testing for Deep Neural Networks, arXiv preprint arXiv:1805.00089 (2018).

27.

Sun

, Huang

and Kroening

, Testing deep neural networks, arXiv preprint arXiv:1803.04792 (2018).

Generating adversarial examples for DNN using pooling layers

Abstract

Keywords

1 Introduction

2 Related work

3 Preliminaries and approaches

3.1 Preliminaries

3.2 Approaches

3.2.1 Pooling functions

3.2.2 Coverage of pooling functions

3.2.3 Gradient ascent based coverage generation

3.2.4 PCD is useful for all DNN with pooling functions

4 Experimental results and evaluation

4.1 Experiments on finding adversarial examples

4.1.1 DNN and testing images set

4.1.2 Adversarial examples find by PCD

Table 1 Groups of testing data in MNIST Group Range of g(x) P T 1 (0,1] 78 129 2 (1,2] 100 191 3 (2,3] 93 236 4 (3,4] 86 358 5 (4,5] 46 523 6 (5,6] 34 469 7 (6,7] 12 801

References

Table 1
Groups of testing data in MNIST

Group Range of g(x) P T

1 (0,1] 78 129

2 (1,2] 100 191

3 (2,3] 93 236

4 (3,4] 86 358

5 (4,5] 46 523

6 (5,6] 34 469

7 (6,7] 12 801