Go to www.menti.

com and use the code 53 39 35

CS230: Lecture 5
Attacking Networks with Adversarial Examples
-
Generative Adversarial Networks
Kian Katanforoosh

Kian Katanforoosh, Andrew Ng, Younes Bensouda Mourri

 Today’s outline

I. Attacking NNs with Adversarial Examples

II. Generative Adversarial Networks

Kian Katanforoosh, Andrew Ng, Younes Bensouda Mourri

 I. Adversarial examples

Discovery (2014): several machine learning models, including state-of-the-art neural networks,
are vulnerable to adversarial examples

A. How to build adversarial examples and attack a network?

B. Examples
C. How to defend against adversarial examples?

[Ian J. Goodfellow, Jonathon Shlens & Christian Szegedy (2015): Explaining and harnessing adversarial examples] Kian Katanforoosh, Andrew Ng, Younes Bensouda Mourri
 I. A. How to build adversarial examples and attack a network?
Goal: Given a pretrained network on ImageNet, find an example that is not a
iguana but will be classify as an iguana.
⎛ 0.04 ⎞ “car”
“car”
⎜ !0.85⎟ “iguana
⎜ ⎟
Neural network ⎜ !0.02 ⎟ “tomato”
⎜ ⎟
x (pretrained on
ImageNet)
⎜ 0.07 ⎟
⎜ 0.81
⎜
0.01⎟
⎟
“bike”
“bike”

“cat”
“cat”

⎜! ⎟
⎜⎝ 0.07
0.02⎟⎠ “crab”
“crab”

⎛ 0⎞
1. Rephrasing what we want: ⎜1⎟ 2. Defining the loss function 3. Optimize the image
⎜ ⎟
⎜! ⎟
⎜ ⎟ 1
x
Network

Find x such that: ŷ(x) = yiguana = ⎜ 0⎟ L( ŷ, y) = ŷ(W ,b, x) − yiguana

2
(pretrained on L( ŷ, y)
2 2 ImageNet)

⎜ 0⎟
⎜ ⎟ After many iterations

⎜! ⎟
⎜⎝ 0⎟⎠ ∂L ∂L
? ∂x
x = x −α
∂x

[Ian J. Goodfellow, Jonathon Shlens & Christian Szegedy (2015): Explaining and harnessing adversarial examples] Kian Katanforoosh, Andrew Ng, Younes Bensouda Mourri
 I. A. How to build adversarial examples and attack a network?

Question: Will the learned image x look like an iguana?

Space of possible Space of images classified

256 32×32×3
≈ 10 7400
input images as iguanas

Space of real images

Kian Katanforoosh, Andrew Ng, Younes Bensouda Mourri

 I. A. How to build adversarial examples and attack a network?
Goal: Given a pretrained network on ImageNet, find an example that is a cat but
will be classify as an iguana.
⎛ 0.04 ⎞ “car”
“car”
⎜ !0.85⎟ “iguana
⎜ ⎟
Neural network ⎜ !0.02 ⎟ “tomato”
⎜ ⎟
(pretrained on ⎜ 0.07 ⎟
“bike”
“bike”

⎜ 0.81
0.01⎟
ImageNet) ⎜ ⎟
“cat”
“cat”

⎜! ⎟
⎜⎝ 0.07
0.02⎟⎠ “crab”
“crab”

1. Rephrasing what we want: ⎛ 0⎞ 2. Defining the loss function 3. Optimize the image
⎜1⎟
⎜ ⎟
⎜! ⎟
⎜ ⎟ 1
x
2 Network
Find x such that: ŷ(x) = yiguana = ⎜ 0⎟ L( ŷ, y) = ŷ(W ,b, x) − yiguana (pretrained on L( ŷ, y)
⎜ 0⎟ 2 2 ImageNet)
⎜ ⎟
⎜! ⎟ 2
⎜⎝ 0⎟⎠ + λ x − xcat 2
After many iterations

And: x = xcat
∂L ∂L
x = x −α
? ∂x ∂x

[Ian J. Goodfellow, Jonathon Shlens & Christian Szegedy (2015): Explaining and harnessing adversarial examples] Kian Katanforoosh, Andrew Ng, Younes Bensouda Mourri
I. A. How to build adversarial examples and attack a network?

92% Cat 94% Iguana

Kian Katanforoosh, Andrew Ng, Younes Bensouda Mourri

[Alexey Kurakin, Ian J. Goodfellow, Samy Bengio (2017): Adversarial examples in the physical world] Kian Katanforoosh, Andrew Ng, Younes Bensouda Mourri
 I. B. How to defend against adversarial examples?

Types of attacks: Knowledge of the attacker:

• Non-targeted attacks • White-box

• Targeted attacks • Black-box

Solution 1
x=
• Train on correctly labelled adversarial examples
Solution 2
y = cat

• Adversarial training Lnew = L(W ,b, x, y) + λ L(W ,b, xadv , y)

2
• Adversarial logit pairing Lnew = L(W ,b, x, y) + λ f (x;W ,b) − f (xadv ;W ,b)
2

Do neural networks actually understand the data?

[Harini Kannan et al. (2018): Adversarial Logit Pairing] Kian Katanforoosh, Andrew Ng, Younes Bensouda Mourri
 II. Generative Adversarial Networks (GANs)

A. Motivation


B. G/D Game


C. Practical tips to train/evaluate GANs


D. Interesting results

[Ian J. Goodfellow, Jonathon Shlens & Christian Szegedy (2015): Explaining and harnessing adversarial examples] Kian Katanforoosh, Andrew Ng, Younes Bensouda Mourri
 II.A - Motivation

Motivation: endowing computers with an understanding of our world.

Goal: collect a lot of data, use it to train a model to generate similar data from scratch.

Intuition: number of parameters of the model << amount of data

[The Gan Zoo (2017)] Kian Katanforoosh, Andrew Ng, Younes Bensouda Mourri
 II.A - Motivation

Probability distributions:

Samples from the “true data distribution” “true data distribution”

Matching distributions

Goal
Image space

“generated distribution”
Samples from the “generated distribution”

Image space

Image space

[Andrej Karpathy et al. (2016): Generative Models, OpenAI blog] Kian Katanforoosh, Andrew Ng, Younes Bensouda Mourri
 II.B - G/D Game
100-d (64,64,3)
random code generated image

≠
⎛ 0.47 ⎞
⎜ ⎟ Generator “G”
⎜ ⎟
⎜ ⎟ (Neural Network)
⎜⎝ 0.19 ⎟⎠

How can we train G to generate images from the true data

distributions?

[Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaogang Wang, Xiaolei Huang, Dimitris Metaxas (2017): StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks] Kian Katanforoosh, Andrew Ng, Younes Bensouda Mourri
 II.B - G/D Game
100-d (64,64,3)
random code generated image

Run Adam simultaneously on two minibatches

⎛ 0.47 ⎞
⎜ ⎟ (true data / generated data)
Generator “G”
⎜ ⎟
⎜ ⎟ (Neural Network)
⎜⎝ 0.19 ⎟⎠

z Gradients Binary classification

⎧ y = 0 if x = G(z)
⎪
x Discriminator “D”
(Neural Network) ⎨
⎪⎩ y = 1 otherwise

Probability distributions

Real images
(database)

Image space

Kian Katanforoosh, Andrew Ng, Younes Bensouda Mourri

 II.B - G/D Game
100-d (64,64,3)
random code generated image

⎛ 0.47 ⎞
⎜ ⎟ Generator “G”
⎜ ⎟
⎜ ⎟ (Neural Network)
⎜⎝ 0.19 ⎟⎠

z Gradients Binary classification

⎧ y = 0 if x = G(z)
End goal: G is outputting ⎪
images that are x Discriminator “D”
(Neural Network) ⎨
indistinguishable from real ⎪⎩ y = 1 otherwise
images for D
Probability distribution

Real images
(database)

Image space

Kian Katanforoosh, Andrew Ng, Younes Bensouda Mourri

 II.B - G/D Game

⎧⎪ yreal is always 1
Training procedure, we want to minimize: Labels: ⎨
y
⎪⎩ gen is always 0
• The loss of the discriminator
mreal mgen
1 1
J ( D)
=− ∑
mreal i=1
yreal .log(D(x )) −
(i) (i)
∑
mgen i=1
(i) (i)
(1− ygen ).log(1− D(G(z )))
!####"####$ !###### #"####### $
cross-entropy 1: cross-entropy 2:
“D should correctly label real data as 1” “D should correctly label generated data as 0”

• The loss of the generator

mgen
1
J (G )
= −J ( D)
= ∑
mgen i=1
(i)
log(1− D(G(z ))) “G should try to fool D: by minimizing the
opposite of what D is trying to minimize”

Kian Katanforoosh, Andrew Ng, Younes Bensouda Mourri

 II.C - Training GANs

Saturating cost ⎡ 1 mgen ⎤ ⎡ 1

mgen
⎤ ⎡ 1
mgen
⎤
min ⎢ ∑ log(1− D(G(z ))) ⎥ ⇔ max ⎢
(i)
∑ log(D(G(z ))) ⎥ ⇔ min ⎢ −
(i)
∑ (i)
log(D(G(z ))) ⎥
for the generator: ⎢⎣ mgen i=1 ⎥⎦ ⎢⎣ mgen i=1 ⎥⎦ ⎢⎣ mgen i=1 ⎥⎦

5
Non-saturating cost
mgen
0 1
(G )
J (G )
=− ∑
mgen i=1
(i)
log(D(G(z )))
J
Saturating cost
mgen
1
J (G )
= ∑
mgen i=1
(i)
log(1− D(G(z )))

-20
0 D(G(z)) 1

[Ian Goodfellow (2014): NIPS Tutorial: GANs] Kian Katanforoosh, Andrew Ng, Younes Bensouda Mourri
 II.C - Training GANs

⎡ 1 mgen ⎤ ⎡ 1
mgen
⎤ ⎡ 1
mgen
⎤
Note that: min ⎢ ∑ log(1− D(G(z ))) ⎥ ⇔ max ⎢
(i)
∑ log(D(G(z ))) ⎥ ⇔ min ⎢ −
(i)
∑ (i)
log(D(G(z ))) ⎥
⎢⎣ mgen i=1 ⎥⎦ ⎢⎣ mgen i=1 ⎥⎦ ⎢⎣ mgen i=1 ⎥⎦

New training procedure, we want to minimize:

mreal mgen
1 1
J ( D)
=− ∑
mreal i=1
yreal .log(D(x )) −
(i) (i)
∑
mgen i=1
(i) (i)
(1− ygen ).log(1− D(G(z )))
!####"####$ !###### #"####### $
cross-entropy 1: cross-entropy 2:
“D should correctly label real data as 1” “D should correctly label generated data as 0”

mg
1
J (G )
=− ∑
mg i=1
(i)
log(D(G(z ))) “G should try to fool D: by minimizing this”

Kian Katanforoosh, Andrew Ng, Younes Bensouda Mourri

 II.C - Training GANs

Simultaneously training G/D?

5
Non-saturating cost
mg
0 1
(G )
J (G )
=− ∑
mg i=1
(i)
log(D(G(z )))
for num_iterations:
J
Saturating cost
for k iterations:
mg
update D 1
update G
J (G )
= ∑
mg i=1
(i)
log(1− D(G(z )))

-20
0 D(G(z)) 1

[Ian Goodfellow (2014): NIPS Tutorial: GANs] Kian Katanforoosh, Andrew Ng, Younes Bensouda Mourri
 II.C - Training GANs

BatchNorm with GANs:

Generated images
(batch 1)

Generated images
(batch 2)

Kian Katanforoosh, Andrew Ng, Younes Bensouda Mourri

 II.C - Training GANs

BatchNorm with GANs:

Assume no batchnorm Assume batchnorm

(64,64,3) (64,64,3)
Code 1 generated image generated images

⎛ 0.12⎞ Code 1 Code 2

⎜ ⎟
⎜ ⎟ Generator “G”
⎜ ⎟ (Neural Network) 1 ⎛ 0.12⎞ ⎛ 0.47 ⎞ 1
⎜ ⎟
⎝ 0.92⎠
⎜ ⎟ ⎜ ⎟
⎜ ⎟ ⎜ ⎟ Generator “G”
⎜ ⎟ ⎜ ⎟ (Neural Network)
⎜ ⎟ ⎜⎝ 0.19 ⎟⎠
⎝ 0.92⎠
(64,64,3)
2
Code 2 generated image

⎛ 0.47 ⎞
⎜ ⎟ Generator “G”
⎜ ⎟ 2
⎜ ⎟ (Neural Network)
⎜⎝ 0.19 ⎟⎠

Kian Katanforoosh, Andrew Ng, Younes Bensouda Mourri

 II.C - Training GANs

BatchNorm with GANs:

BatchNorm Reference BatchNorm Virtual BatchNorm

Z = {z ,...., z
(1) ( m)
} R = {r ,....,r
(1) ( m)
} R = {r ,....,r(1) ( m)
}
1 m Z = {z ,...., z
(1) ( m)
} Z = {z ,...., z
(1) ( m)
}
µB = ∑ z (i)
1 m
m i=1 µB = ∑ r (i)

m m i=1 For k = 1….m

1
σ = ∑ (z −µ B )
2 (i) 2
1 m 1 ⎛ ( k ) m (i) ⎞
B
z + ∑r ⎟
m i=1 σ = ∑ (r −µ B )
2
B
(i) 2 µB = ⎜
m + 1⎝ ⎠
z − µB
(i) m i=1 i=1

1 ⎛ m
⎞
z (i)
norm
= z − µB
(i) σB =
2
⎜ (z − µ B ) + ∑ (r −µ B ) ⎟
(k ) 2 (i) 2

σ +ε 2 m + 1⎝ ⎠
B z (i)
norm
= i=1

σ +ε 2
z(k ) − µB
(i)
=γ z (i)
+β =
(k )
z! B z norm
norm σ B2 + ε
z! (i)
=γ z (i)
norm
+β
z! (k )
=γ z (k )
norm
+β

Kian Katanforoosh, Andrew Ng, Younes Bensouda Mourri

 II.C - Training GANs

5 Non-1 mg

0 J (G ) = − ∑ log(D(G(z (i)
)))
Recap: GANs’ training tips J (G ) mg i=1

Satura
1
mg

J =
m
∑
(G )
log(1− D(G(z (i)
)))
g i=1

• Use the non-saturated cost function
 -0 D(G(z))

1
• Keep D up-to-date with respect to G (k update for D / 1 update for G)


• Use Virtual Batchnorm


• (not presented but important) One-sided label smoothing

And a lot more, GANs are hard to train!

[Soumith et al. (2016): GanHacks] Kian Katanforoosh, Andrew Ng, Younes Bensouda Mourri
 II.D - Interesting results

Operation on codes
(64,64,3)
Code 1 generated image

⎛ 0.12⎞
⎜ ⎟
⎜ ⎟ Generator “G”
⎜ ⎟ (Neural Network) 1
⎜ ⎟
⎝ 0.92⎠

(64,64,3)
Code 2 generated image
Code 1 Code 2 Code 3
⎛ 0.47 ⎞
⎜ ⎟ Generator “G” ⎛ 0.12⎞ ⎛ 0.47 ⎞ ⎛ 0.42 ⎞
⎜ ⎟ 2 ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ Generator “G”
⎜ ⎟ (Neural Network)
⎜⎝ 0.19 ⎟⎠
⎜
⎜
⎟
⎟
- ⎜
⎜
+
⎟
⎟
⎜
⎜
⎟
⎟
(Neural Network)
⎜ ⎟ ⎜⎝ 0.19 ⎟⎠ ⎜ ⎟
⎝ 0.92⎠ ⎝ 0.07 ⎠
(64,64,3)
Code 3 generated image

⎛ 0.42 ⎞
⎜ ⎟
⎜ ⎟ Generator “G” Man with glasses - man + woman = woman with glasses
⎜ ⎟ (Neural Network) 2
⎜ ⎟
⎝ 0.07 ⎠

[Radford et al. (2015): UNSUPERVISED REPRESENTATION LEARNING WITH DEEP CONVOLUTIONAL GENERATIVE ADVERSARIAL NETWORKS] Kian Katanforoosh, Andrew Ng, Younes Bensouda Mourri
 II.D - Interesting results

Image Generation:
Samples from the “generated distribution”

[Zhang et al. (2017): StackGAN++] Kian Katanforoosh, Andrew Ng, Younes Bensouda Mourri
 II.D - Interesting results

Pix2Pix:

https://affinelayer.com/pixsrv/

[Isola et al. (2017): Image-to-Image Translation with Conditional Adversarial Networks] Kian Katanforoosh, Andrew Ng, Younes Bensouda Mourri
 II.D - Interesting results

Super-resolution image:

[Ledig et al. (2016): Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network] Kian Katanforoosh, Andrew Ng, Younes Bensouda Mourri
 II.D - Interesting results

CycleGANs:

[Jun-Yan Zhu et al. (2017): Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks] Kian Katanforoosh, Andrew Ng, Younes Bensouda Mourri
 II.D - Interesting results

Face2ramen
CycleGANs:

https://hardikbansal.github.io/CycleGANBlog/

[Jun-Yan Zhu et al. (2017): Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks] Kian Katanforoosh, Andrew Ng, Younes Bensouda Mourri
 Announcements

For Tuesday 05/01, 9am:


C2M3
• Quiz: Hyperparameter tuning, Batch Normalization, Programming
Frameworks
• Programming assignment: Tensorflow

C3M1 and C3M2

• Quiz: Bird recognition in the city of Peacetopia (case study)
• Quiz: Autonomous driving (case study)

Check out the project

For Friday 02/16, 9am:
 example code!
(cs230-stanford.github.io)
• Hands-on session this Friday

Meet with your mentor (TA), you’ll receive a Calendly invite.

Kian Katanforoosh, Andrew Ng, Younes Bensouda Mourri