See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/330041745

Performance Review of Convolutional Neural Network Models in

Classification and Adversarial Attacks

Thesis · December 2018

DOI: 10.13140/RG.2.2.28682.31682

CITATIONS READS

0 32

1 author:

Narmada M Balasooriya
University of Peradeniya
4 PUBLICATIONS   0 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

A Vision Based Obstacle Avoidance And Safety System for a Quadcopter (Hardware Implementation) View project

All content following this page was uploaded by Narmada M Balasooriya on 02 January 2019.

The user has requested enhancement of the downloaded file.

 PERFORMANCE REVIEW OF CONVOLUTIONAL
NEURAL NETWORK MODELS IN CLASSIFICATION
AND ADVERSARIAL ATTACKS

B.M.NARMADA MADHUWANTHI BALASOORIYA

M.Sc. 2018
 PERFORMANCE REVIEW OF CONVOLUTIONAL
NEURAL NETWORK MODELS IN CLASSIFICATION
AND ADVERSARIAL ATTACKS

A PROJECT REPORT PRESENTED BY

B.M.NARMADA MADHUWANTHI BALASOORIYA

to the Board of Study in Statistics and Computer Science of the

POSTGRADUATE INSTITUTE OF SCIENCE

in partial fulfillment of the requirement

for the award of the degree of

MASTER OF SCIENCE IN COMPUTER SCIENCE

of the

UNIVERSITY OF PERADENIYA
SRI LANKA
2017
 DECLARATION

I do hereby declare that the work reported in this project report/thesis was exclusively
carried out by me under the supervision of Dr.Ruwan Nawarathna. It describes the
results of my own independent research except where due reference has been made in the
text. No part of this project report/thesis has been submitted earlier or concurrently
for the same or any other degree.

Date: ............................. ...................................................

Signature of candidate

Certified by :

Supervisor (Name):................................................... Date: ..................................

Signature:.........................................

PGIS Stamp:

i
 Abstract

B.M.Narmada Madhuwanthi Balasooriya

Postgraduate Institute of Science, University of Peradeniya, Peradeniya, Sri Lanka

Department of Statistics and Computer Science, University of Peradeniya,
Peradeniya, Sri Lanka

Convolutional Neural Network models are the state of the art algorithms used in
numerous object classification and identification tasks and research. Due to their higher
accuracy and advanced image processing capability, these models outrun traditional
computer vision algorithms. As these models are widely used, it is necessary to
make sure that these architectures are robust and precise in doing the necessary
computations.

When using convolutional neural network models for a different dataset, it is required
to fine-tune or train the network to the particular dataset. This requires additional
effort and computations even though similar images are used to pretrain the models.

In this project, the performance of three selected models which are pretrained on
ImageNet dataset is evaluated on a similar but a different dataset called Caltech-101.
The approach is to test the predictions of the selected models on randomly selected
images from the new dataset.

From the results obtained, it is evident that the selected models extensively require
fine-tuning to the caltech-101 dataset to obtain expected performance. It is concluded
that this an issue related to most CNN models that are pretrained on a very large
dataset like ImageNet.

ii
CONTENTS

Declaration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i
Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii
Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv
List of Abbreviations . . . . . . . . . . . . . . . . . . . . . . . . . . . v
Chapter 1 INTRODUCTION 1
1.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 VGGNet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.4 ResNet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.5 DenseNet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.6 ImageNet Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.7 Performance of VGG19, Resnet152, and DenseNet161 on ImageNet
Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Chapter 2 PERFORMANCE REVIEW OF VGG-19, RESNET-
152 AND DENSENET-161 5
2.1 The dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Chapter 3 LITERATURE REVIEW ON ADVERSARIAL
ATTACKS 11
Chapter 4 FUTURE IMPROVEMENTS 13
Chapter 5 CONCLUSION 14
Chapter 6 APPENDIX 15
6.1 Python Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

iii
LIST OF FIGURES

Figure 1.1 General architecture of a convolutional neural network . . . . . 1

Figure 2.1 sample images from the dataset . . . . . . . . . . . . . . . . . . 5

Figure 2.2 Results set 1: pred1-ResNet152, pred2-DenseNet161, pred3-
VGG19 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Figure 2.3 Results set 2: pred1-ResNet152, pred2-DenseNet161, pred3-
VGG19 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Figure 2.4 Results set 3: pred1-ResNet152, pred2-DenseNet161, pred3-
VGG19 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Figure 2.5 Results set 4: pred1-ResNet152, pred2-DenseNet161, pred3-
VGG19 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Figure 2.6 Results set 5: pred1-ResNet152, pred2-DenseNet161, pred3-
VGG19 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Figure 2.7 Results set 6: pred1-ResNet152, pred2-DenseNet161, pred3-
VGG19 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Figure 2.8 Results set 7: pred1-ResNet152, pred2-DenseNet161, pred3-
VGG19 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Figure 2.9 Results set 8: pred1-ResNet152, pred2-DenseNet161, pred3-
VGG19 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Figure 3.1 Image from the article ‘Explaining and Harnessing Adversarial
Examples’ [1] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

iv
LIST OF ABBREVIATIONS

CNN CONVOLUTIONAL NEURAL NETWORK

ICLR INTERNATIONAL CONFERENCE ON LEARNING REPRESENTATIONS

v
Chapter 1 INTRODUCTION

1.1 Overview

The idea of this project is to identify the problems associated with deep learning
algorithms especially convolutional neural network models. Convolutional neural networks
are higly used in object identification and classification tasks related to various applications.
Here the objective is to identify the different types of issues that might occur when using
convolutional neural networks in object classification, discuss and suggest solutions.

The intended approach has two phases. I try to evaluate the performance of popular
network models on a different dataset without fine-tuning.

The state of the art object identification and classification architecture used in most
present day research is convolutional neural network. Basically, a CNN has three
main layer types as Convolutional, Pooling and Fully-connected layers. The input can
be a RGB image or a set of images with three colour channels or Grayscale with a
single colour channel. The convolutional layer computes the output of the neurons
by running a dot product between the weight and the regions where the neurons are
connected to the input. If the width and the height of the input image is [64 ×
64] and if 8 filters are used, the output of the convolutional layer is [64 × 64 × 8].
The pooling layer downscales the input by a given factor. The fully-connected layer
computes the class scores resulting the outputs in an array. A simple CNN architecture
is shown in Figure.1.1. Similar architectures are built by different researchers for various
object classification tasks. In addition to the main layers, some optimization layers and
dropout layers are added to address several issues related to the training and testing
of those models.

Figure 1.1: General architecture of a convolutional neural network

1
Due to its advanced image processing capability and faster prediction rate, CNN models
are popularly used to identify objects in autonomous navigation, obstacle avoidance,
industrial applications etc. But in [2], several researchers find a property related to
neural networks that enables external programs to fool the neural network model.

One of the objectives of this project is to evaluate the robustness of three CNN models
VGG19[3], ResNet[4] and DenseNet[5] in classifying images from a dataset different
to the dataset used for training. The other objective is to do a literature review on
adversarial attacks on deep learning algorithms, test and run several identified attacks
on the previously mentioned CNN models to compute their robustness.

The upcoming content of the report is organized as follows. Sections 1.2, 1.3, 1.4,
1.5, 1.6, and 1.7 contain the objectives of the project, VGGNet, ResNet, DenseNet,
ImageNet dataset, and the performance of the VGGNet, ResNet and DenseNet on
ImageNet dataset respectively. Sections 2.1, 2.2, 2.3, and 2.4 reviews the performance
of VGGNet, ResNet and DenseNet models on Caltech-101 dataset. Chapter 3 discusses
adversarial examples and several articles related to it. Chapter 4 contains the improvements
that are to be made in the future. Chapter 5 consists of the conclusion about the
project. Chapter 6 contains all the relevant computer codes used throughout the
project.

1.2 Objectives

1. Review the performance of VGG19, ResNet152 and DenseNet161 on a sample

dataset

2. Literature review on research in adversarial attacks

3. Test some identified black-box attacks on VGG19, ResNet152 and DenseNet161

4. Develop a image perturbation algorithm to fool the above models

1.3 VGGNet

VGGNet[3] is introduced by Karen Simonyan and Andrew Zisserman from University

of Oxford at ICLR in 2015. The authors investigate the effect of convolutional network

2
depth on accuracy in a large-scale image recognition task. In their architecture, the
depth of the network is increased using very small convolutional filters of size (3 ×
3). Their findings secured a first place in the ImageNet Challenge in 2014 under the
localization track. The input to this architecture is a fixed-size (224×224) RGB image.
Throughout their architecture, they incorporated very small filter sizes such as 3 × 3
and 1 × 1. The results of their findings can be found in [3]. Here in this project, I use
VGG-19 architecture.

1.4 ResNet

ResNet or Deep Residual Learning for Image Recognition[4] is introduced by four

researchers at Microsoft in 2015. In their approach, the layers are reformulated as
learning residual functions with reference to the layer inputs instead of unreferenced
functions. The architecture with a depth of 152 layers is tested on ImageNet dataset
and achieved 3.75% error. The architecture won the first place on the tasks of ImageNet
detection, ImageNet localization, COCO detection and segmentation. The full description
of the network can be found in [4]. In this project I use ResNet-152 architecture.

1.5 DenseNet

DenseNet or Densely Connected Convolutional Network[5] introduced in 2018 connects

each layer to every other layer in a feed-forward manner. In this network, the feature
maps from all preceding layers are input the current layer. The proposed model is
evaluated on four datasets namely CIFAR-10, CIFAR-100, SVHN, and ImageNet.
The full description and the results can be found in their publication[5]. Here I use
DenseNet-161 architecture.

1.6 ImageNet Dataset

ImageNet dataset is considered as the largest dataset of more than 15 million labeled
high resolution images in over 22000 categories[6, 7]. It is the widely used dataset to
train numerous deep learning algorithms. The models used for review in this project
are pretrained on ImageNet dataset as this dataset has images in over 22,000 categories.

3
1.7 Performance of VGG19, Resnet152, and DenseNet161 on ImageNet
Dataset

As mentioned in the PyTorch[8] documentation, VGG-19 has Top-1 error of 27.62

and Top-5 error of 9.12, ResNet-152 has 21.69 Top-1 error and 5.94 Top-5 error,
and DenseNet-161 has Top-1 error of 22.35 and Top-5 error of 6.20 on ImageNet test
dataset. According to [8], the mentioned models are the best performing architectures
in classification using CNN. I use VGG-19, ResNet-161, and DenseNet-161 on a dataset
other than ImageNet, but, with similar type of images. It is assumed that the used
images are not present in the ImageNet dataset.

4
Chapter 2 PERFORMANCE REVIEW OF VGG-19, RESNET-
152 AND DENSENET-161

2.1 The dataset

The dataset used in this project is called Caltech-101 introduced by Computational

Vision Lab at Caltech University, USA in 2003[9]. Pictures belonging to 101 categories
and around 40 to 800 images per each category are available in this dataset. The size
of the images are roughly 300 × 200 pixels. Though the dataset can be regarded as
outdated, it contains an excellent collection of old images that can be tested on the
mentioned best performing models. Some sample images from the dataset are shown
in Figure2.1

Figure 2.1: sample images from the dataset

2.2 Methodology

The three models used in evaluation are obtained from PyTorch Torchvision[8] built-in
models pretrained on ImageNet dataset. The reason for selecting the mentioned models
is their performance over the ImageNet test dataset and ImageNet challenge. For the
purpose of this project, a completely different dataset described in Section2.1 is used.

5
The images from the program are loaded into a numpy array and the class names are
loaded into string arrays. Images are reshaped before testing on the mentioned models.
Several random images are generated, tested for the predicted labels and displayed.

The program is developed using Python programming. Additionally, PyTorch Deep

Learning API, Numpy, PIL, and Matplotlib are used to handle the models, to load the
images and to run the evaluations.

2.3 Results

Sample results from the performance evaluation are shown in Figures 2.2, 2.3, 2.4, 2.5,
and 2.6. The predictions of ResNet-152 are listed as pred1, predictions of DenseNet-161
are listed as pred2, and predictions of VGG-19 are listed as pred3.

Figure 2.2: Results set 1: pred1-ResNet152, pred2-DenseNet161, pred3-VGG19

6
Figure 2.3: Results set 2: pred1-ResNet152, pred2-DenseNet161, pred3-VGG19

Figure 2.4: Results set 3: pred1-ResNet152, pred2-DenseNet161, pred3-VGG19

7
 Figure 2.5: Results set 4: pred1-ResNet152, pred2-DenseNet161, pred3-VGG19

Figure 2.6: Results set 5: pred1-ResNet152, pred2-DenseNet161, pred3-VGG19

Though most of the predictions are incorrect according to the above displayed results,
there were correct predictions for a selected number of images. The results for selected
9 images can be shown as below when tested three times.

8
Figure 2.7: Results set 6: pred1-ResNet152, pred2-DenseNet161, pred3-VGG19

Figure 2.8: Results set 7: pred1-ResNet152, pred2-DenseNet161, pred3-VGG19

9
 Figure 2.9: Results set 8: pred1-ResNet152, pred2-DenseNet161, pred3-VGG19

2.4 Discussion

The results shows that the three models, ResNet-152, DenseNet-161 and VGG-19
delivered unsatisfactory outcomes to images from Caltech-101 dataset. As the three
models are trained and weights of the models are fine-tuned to the images of the
ImageNet dataset, it must have resulted in poor performance to different type of images.
It can be argued that these models need to be fine-tuned to the Caltech-101 dataset to
deliver better performance. But the argument I suggest here is that since the majority
of images in Caltech-101 are similar to images from ImageNet dataset, the models
should be able to correctly classify the Caltech-101 images.

Another issue with the program implemented for this performance review is the inability
to use performance metrics such as accuracy and precision. The reason for this is the use
of completely different labeling formats in Caltech-101 dataset and ImageNet dataset.
In order to match the class label names, natural language processing algorithms must
be used as some labels contain words with similar meanings.

10
Chapter 3 LITERATURE REVIEW ON ADVERSARIAL ATTACKS
This chapter discusses adversarial examples and few articles on adversarial attacks.

Adversarial examples are inputs to a a neural network model that results in an incorrect
output from the network. For an example, take an image of a chair with the ground
truth label as ‘chair’ and a particular model predicts the label of the image as a ‘chair’
with a confidence of 60%. Consider if a carefully constructed noise is added to the image
that cannot fool human eye and the image with the noise is input to the particular
model. The model will predict the label of the new image something other than ‘chair’.
A similar example is shown in Figure3.1.

Figure 3.1: Image from the article ‘Explaining and Harnessing Adversarial Examples’
[1]

Adversarial examples are first discussed in [2], where the authors discuss the blind spots
in neural networks and the existence of adversarial negatives which is a contradiction
to the neural network’s ability to achieve high performance.

In [1], the authors provide examples for adversarial training to reduce the test set error
of a maxout network. The authors discuss an intriguing aspect of adversarial examples
which is an example generated for a particular model can often mislead other models
with different architectures or trained on disjoint training sets. They also explains that
the adversarial examples can be a property of high-dimensional dot products and are
results of too linear models. They introduce a family of fast methods for generating
adversarial examples.

Adversarial attacks are generation of inputs or modification of inputs such that they
will result in misclassification from neural network models. There are two types of
adversarial attacks. In white-box attacks, the attacker has the knowledge of the model

11
used, the architecture of the model and access to the weights of the model when it is
computing a certain image. It is easy to generate white-box attacks if the necessary
information is available. On the other hand, black-box attacks are where no information
on the model or the architecture of the model is available to the attacker. The attacker
only sees the output from a particular network. In [10], a local substitute DNN is
trained with a synthetic dataset where the inputs are synthetic and generated by the
adversary. The outputs are the labels assigned by the target DNN and observed by the
adversary. The adversarial examples are crafted from the only parameters available to
the attacker. The attack is validated by targeting a remote DNN hosted by MetaMind
where 84.24% of the adversarial examples are misclassified. Other research are done
on computing adversarial attacks in articles [11] and [12].

Adversarial examples are a threat to many applications of image classification using

neural network models. It is identified as a problem that should be addressed in
autonomous navigation which uses computer vision and deep learning.

12
Chapter 4 FUTURE IMPROVEMENTS
In the section2.4, it is mentioned that class label names cannot be matched without
using natural language processing due to the presence of similar words. Therefore, as
a future improvement natural language processing algorithms will be incorporated to
match label names precisely.

As the models ResNet-152, DenseNet-161, and VGG-19 did not perform well to the
given dataset without fine-tuning the weights to the new dataset, it is required to
develop a more robust and accurate CNN model for image classification task. It is
required that this model must be trained on all the available datasets. This is a rather
difficult task as it is computationally expensive in time and cost.

13
Chapter 5 CONCLUSION
It can be concluded that the described pretrained CNN models perform poorly to a
new dataset with similar type of images without fine-tuning the weights to the new
dataset. This can be decided as a major drawback in the current CNN models though
they performed well in ImageNet test dataset. Another issue pointed by this project
is the inability to match the class label names when two different numeric labeling
systems are used in the training dataset and the test dataset. It is required that the
ground truth labels of the test dataset to have the same numeric labeling system. It is
suggested to use natural language processing algorithms to address this problem.

14
Chapter 6 APPENDIX

6.1 Python Program

1 """
2 -----------------------------------------------------
3 M . Sc . in Computer Science - Independent Study
4 Performance Review of ResNet152 , DenseNet161 and VGG19
5 Author : B . M . Narmada Madhuwanthi Balasooriya
6 Copyrights reserved
7 -----------------------------------------------------
8 """
9
10 from __future__ import print_function , division
11
12 import pickle
13 import numpy as np
14 import matplotlib . pyplot as plt
15 from PIL import Image
16 from matplotlib import cm
17 import cv2
18 import glob
19
20 import torchvision . models as models
21 import torch
22 import torchvision
23 import torchvision . transforms as transforms
24
25 # #################
26 # Preprocessing ##
27 # #################
28 normalize = transforms . Normalize ( mean =[0.485 , 0.456 ,
0.406] , std =[0.229 , 0.224 , 0.225])

15
29
30 transform = transforms . Compose (
31 [ transforms . RandomSizedCrop (224) ,
32 transforms . RandomHorizontalFlip () ,
33 transforms . ToTensor () ,
34 normalize ])
35
36 # ###############################
37 # # loading Caltech 101 images ##
38 # ###############################
39
40 directories = glob . glob ( ’ ./ ObjectCategories_101 /* ’)
41 image_list = []
42 class_list_original = []
43 class_list = []
44
45 for direct in directories :
46 for f in glob . glob ( direct + ’ /*. jpg ’) :
47 image_list . append ( f )
48 class_list_original . append ( direct )
49
50 images = []
51
52 # # Loading images
53 for im in image_list :
54 images . append ( cv2 . imread ( im ) )
55
56 # # Loading ground truth labels
57 for name in class_list_original :
58 name = name . split ( ’ ./ ObjectCategories_101 / ’ , 1)
59 name = name [1]
60 class_list . append ( name )

16
61
62 # # Loading imageNet class names
63 # File1
64 with open ( ’ imagenet_synsets . txt ’ , ’r ’) as f :
65 synsets = f . readlines ()
66
67 synsets = [ x . strip () for x in synsets ]
68 splits = [ line . split ( ’ ’) for line in synsets ]
69 key_to_classname = { spl [0]: ’ ’. join ( spl [1:]) for spl in
splits }
70
71 with open ( ’ imagenet_classes . txt ’ , ’r ’) as f :
72 class_id_to_key = f . readlines ()
73
74 class_id_to_key = [ x . strip () for x in class_id_to_key ]
75
76 # File2
77 with open ( " imagenet1000_clsid_to_human . txt " ) as f :
78 label = eval ( f . read () )
79
80 # #############################
81 # # Loading the Models ##
82 # #############################
83
84 # Cuda device selection
85 torch . cuda . device ( 1 )
86 device = torch . device ( " cuda :0 " if torch . cuda .
is_available () else " cpu " )
87 print ( device )
88
89 # Resnet -152 model
90 net1 = models . resnet152 ( pretrained = True )

17
 91 net1 . eval ()
92 net1 . to ( device )
93
94 # DenseNet -161 model
95 net2 = models . densenet161 ( pretrained = True )
96 net2 . eval ()
97 net2 . to ( device )
98
99 # VGG -19 model
100 net3 = models . vgg19 ( pretrained = True )
101 net3 . eval ()
102 net3 . to ( device )
103
104 # # ## # ## ## # ## ## # ## # ## ## # ## # ## ## # ## ## # ## # ## ## # #
105 # # Evaluating the models for random images ##
106 # # ## # ## ## # ## ## # ## # ## ## # ## # ## ## # ## ## # ## # ## ## # #
107
108 num_plot = 3
109 f , ax = plt . subplots ( num_plot , num_plot )
110
111 for m in range ( num_plot ) :
112 for n in range ( num_plot ) :
113
114 idx = np . random . randint (0 , len ( images ) )
115
116 image_n = images [ idx ]
117 im = cv2 . resize ( image_n , dsize =(224 , 224) )
118 im = Image . fromarray ( im )
119 input_img = transform ( im )
120 input_img = input_img . float () . to ( device )
121 input_img = input_img . view (1 , 3 , 224 , 224)
122

18
123 # ResNet -152 results
124 output1 = net1 ( input_img )
125 _ , predicted1 = torch . max ( output1 ,1)
126 predict1 = predicted1 [0]. cpu () . numpy ()
127 predict1 = int ( predict1 )
128 class_key1 = class_id_to_key [ predict1 ]
129 classname1 = key_to_classname [ class_key1 ]
130
131 # DenseNet -161 results
132 output2 = net2 ( input_img )
133 _ , predicted2 = torch . max ( output2 ,1)
134 predict2 = predicted2 [0]. cpu () . numpy ()
135 predict2 = int ( predict2 )
136 class_key2 = class_id_to_key [ predict2 ]
137 classname2 = key_to_classname [ class_key2 ]
138
139 # VGG -19 results
140 output3 = net3 ( input_img )
141 _ , predicted3 = torch . max ( output3 ,1)
142 predict3 = predicted3 [0]. cpu () . numpy ()
143 predict3 = int ( predict3 )
144 class_key3 = class_id_to_key [ predict3 ]
145 classname3 = key_to_classname [ class_key3 ]
146
147 # Original ground truth label
148 original = class_list [ idx ]
149
150 # Plotting the images with the labels
151 ax [m , n ]. set_title ( ’ ori : ’ + original + ’\ n ’ +
’ pred1 : ’ + classname1 + ’\ n ’ + ’ pred2 : ’ +
classname2 + ’\ n ’ + ’ pred3 : ’ + classname2 )
152 ax [m , n ]. imshow ( image_n )

19
153
154 ax [m , n ]. get_xaxis () . set_visible ( False )
155 ax [m , n ]. get_yaxis () . set_visible ( False )
156
157 f . subplots_adjust ( hspace =1)
158 f . subplots_adjust ( wspace =0.2)
159
160 plt . show ()
161
162 # # End of Program ##
163 # - - - - - - - - - - - - - - - - - -#

20
Bibliography
[1] Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and
harnessing adversarial examples. In International Conference on Learning
Representations, 2015.

[2] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru
Erhan, Ian J. Goodfellow, and Rob Fergus. Intriguing properties of neural
networks. CoRR, abs/1312.6199, 2013.

[3] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for
large-scale image recognition. CoRR, abs/1409.1556, 2014.

[4] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning
for image recognition. CoRR, abs/1512.03385, 2015.

[5] Gao Huang, Zhuang Liu, and Kilian Q. Weinberger. Densely connected
convolutional networks. CoRR, abs/1608.06993, 2016.

[6] J. Deng, W. Dong, R. Socher, L. Li, Kai Li, and Li Fei-Fei. Imagenet: A large-
scale hierarchical image database. In 2009 IEEE Conference on Computer Vision
and Pattern Recognition, pages 248–255, June 2009.

[7] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification
with deep convolutional neural networks. In F. Pereira, C. J. C. Burges, L. Bottou,
and K. Q. Weinberger, editors, Advances in Neural Information Processing
Systems 25, pages 1097–1105. Curran Associates, Inc., 2012.

[8] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang,
Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer.
Automatic differentiation in pytorch. In NIPS-W, 2017.

[9] R. Fergus L. Fei-Fei and P. Perona. Learning generative visual models from
few training examples: an incremental bayesian approach tested on 101 object
categories. In editor, editor, IEEE. CVPR 2004, Workshop on Generative-Model
Based Vision.

21
 [10] Nicolas Papernot, Patrick D. McDaniel, Ian J. Goodfellow, Somesh Jha, Z. Berkay
Celik, and Ananthram Swami. Practical black-box attacks against deep learning
systems using adversarial examples. CoRR, abs/1602.02697, 2016.

[11] Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. Deepfool:

a simple and accurate method to fool deep neural networks. CoRR,
abs/1511.04599, 2015.

[12] Nina Narodytska and Shiva Prasad Kasiviswanathan. Simple black-box adversarial
perturbations for deep networks. CoRR, abs/1612.06299, 2016.

22

View publication stats