Scribd
Upload
17 views54 pages

Generative Adversarial Networks: Biplab Banerjee

The document discusses generative adversarial networks (GANs) including implicit density estimation, how GANs work by training a generator and discriminator in an adversarial manner, common loss functions, techniques to address vanishing gradients and mode collapse, and extensions like conditional GANs and InfoGAN to provide disentangled representations.

Uploaded by

Atul Verma
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
17 views54 pages

Generative Adversarial Networks: Biplab Banerjee

The document discusses generative adversarial networks (GANs) including implicit density estimation, how GANs work by training a generator and discriminator in an adversarial manner, common loss functions, techniques to address vanishing gradients and mode collapse, and extensions like conditional GANs and InfoGAN to provide disentangled representations.

Uploaded by

Atul Verma
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 54

Generative adversarial

networks
Biplab Banerjee
Implicit density estimation problem
• What if we are only interested to sample from the complex high
dimensional intractable data distribution without caring to know
P(data) at all?
Magic of GAN
Magic of GAN
What about
Training discriminator
Training generator
Loss functions
• For generator

• For discriminator
Loss function

Generator minimizes it Discriminator maximizes it


Training a vanilla GAN
Tackling vanishing gradients
Optimal D
• The value function

• What is the gradient of L with respect to D?


Global optima

Which is equal to the JSD between real and


fake distributions
DC GAN
Latent vectors capture meaningful
information
Some tips for GAN network design
• Replace pooling by strided convolution and unpolling by fractional
strided convolution
• Use batch-norm
• Avoid FC layers in deeper architectures
• Use ReLU in all the generator layers except output where
use tanh
• Use leaky-ReLU in the discriminator layers

** See GAN-hacks (by soumith chintala) for more details


Problems in training a GAN
Non convergence of GAN training
Problem in obtaining the equilibrium
Mode collapse
Mode collapse in MNIST
Feature matching
Mini-batch discrimination
Supervision with labels
An alternate view of GAN
Energy based GAN
One issue so far
• How can we ensure the generation of images with a specific
characteristics?
• Since the input to G is noise, we do not have explicit control over
which images are generated.
• C-GAN solves to some extent by conditioning on the class labels
• A learning based unsupervised way is called INFOGAN
• It provides disentangled representations
Disentangled representations
The evolution so far
Idea of mutual information
InfoGAN
InfoGAN
Example (CGAN)
Examples (InfoGAN)
Calculate KL, JS for non-overlapping
distributions

Always constant
EM distance
Wasserstein distance

Kantorovich rubinson duality

You might also like