Scribd
Upload
2 views5 pages

Deep Learning Unit 3 To 5 Notes

The document covers deep learning concepts from Units III to V, focusing on Convolutional Networks, Recurrent Neural Networks, and Deep Generative Models. It discusses various convolution operations, RNN architectures, and generative models like VAEs and GANs. Key topics include efficient algorithms, bidirectional processing, and the use of probabilistic models for data generation.

Uploaded by

adityabundela072001
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
2 views5 pages

Deep Learning Unit 3 To 5 Notes

The document covers deep learning concepts from Units III to V, focusing on Convolutional Networks, Recurrent Neural Networks, and Deep Generative Models. It discusses various convolution operations, RNN architectures, and generative models like VAEs and GANs. Key topics include efficient algorithms, bidirectional processing, and the use of probabilistic models for data generation.

Uploaded by

adityabundela072001
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 5

Deep Learning Notes: Unit III to V

UNIT-III: CONVOLUTIONAL NETWORKS

1. CONVOLUTION OPERATION

Convolution is a mathematical operation used to extract features from data like images.

1D Convolution:

y(t) = (x * w)(t) = sum_{a=-inf}^{inf} x(a) * w(t - a)

2D Convolution (used in images):

Y(i, j) = sum_m sum_n X(i - m, j - n) * W(m, n)

Variants include stride, padding, dilation.

2. VARIANTS OF CONVOLUTION FUNCTIONS

- Tied weights

- 1x1 convolutions

- Dilated convolution

- Transposed convolution

3. STRUCTURED OUTPUTS

- Fully convolutional networks (FCNs)

- Skip connections

- U-Net

4. EFFICIENT CONVOLUTION ALGORITHMS


- FFT-based convolution

- Depthwise separable convolutions

- Grouped convolutions

5. RANDOM OR UNSUPERVISED FEATURES

- Autoencoders

- PCA

- K-means

6. LeNet

- Early CNN model with 2 conv + pooling layers.

- Used on MNIST dataset.

7. AlexNet

- Used ReLU, dropout, overlapping pooling.

- Won ImageNet 2012 challenge.

UNIT-IV: RECURRENT NEURAL NETWORKS

1. UNFOLDING COMPUTATIONAL GRAPHS

RNNs are unfolded across time steps.

Each time step shares weights and passes hidden state.

2. RECURRENT NEURAL NETWORKS (RNNs)

Process sequences by passing hidden state.


Formulas:

h_t = f(W_h h_{t-1} + W_x x_t + b)

y_t = g(W_y h_t + b_y)

3. BIDIRECTIONAL RNNs

Process sequence in both directions.

Useful for full-context sequence tasks.

4. ENCODER-DECODER SEQUENCE TO SEQUENCE ARCHITECTURES

Encoder maps input to fixed vector.

Decoder generates output step-by-step.

5. DEEP RECURRENT NEURAL NETWORKS

Stack RNN layers to model complex sequences.

Improves depth and representation.

6. CONDITIONAL GENERATIVE MODELS

Model output y conditioned on input x: P(y|x)

Used in machine translation, text generation.

UNIT-V: DEEP GENERATIVE MODELS

1. INTRODUCTION

Generative models learn to generate data similar to training data.


Categories:

- Variational Autoencoders (VAEs)

- Generative Adversarial Networks (GANs)

- Autoregressive Models

- Normalizing Flows

- Self-Organizing Maps (SOMs)

2. VARIATIONAL AUTOENCODERS (VAEs)

Probabilistic encoder-decoder model.

Uses ELBO loss:

L = E_q(z|x)[log p(x|z)] - KL(q(z|x) || p(z))

3. GENERATIVE ADVERSARIAL NETWORKS (GANs)

Two networks:

- Generator: creates fake samples

- Discriminator: classifies real vs. fake

Minimax loss:

min_G max_D V(D,G) = E[log D(x)] + E[log(1 - D(G(z)))]

4. AUTOREGRESSIVE MODELS

Model p(x) as a product of conditionals.

Examples: PixelRNN, PixelCNN, WaveNet.

5. NORMALIZING FLOWS

Learn invertible functions to map noise to data.

Allows exact log-likelihood computation.


6. SELF-ORGANIZING MAPS (SOMs)

Unsupervised learning for data visualization.

Competitive learning updates nodes based on input.

Forms topological maps of data.

You might also like