Unleash the Potential of Deep Learning

PG Level Advanced Certification Programme in

Deep Learning
Foundations and Applications

For Practicing and Aspiring AI/ML Professionals

Programmme Partner

iisc.talentsprint.com/deeplearning
 Curriculum

2
2
Module 1

Mathematical Preliminaries and Data Visualization (Preparatory Module)

1. Linear Algebra

Eigenvalues and eigenvectors, singular value decomposition, linear system of equations, rank and null
space, pseudo inverse, matrix factorization, matrix calculus.

2. Probability and Statistics

Random variables, Probability distributions, Marginal probability, Conditional probability, the chain rule
of conditional probabilities, Independence and conditional independence, Expectation, variance and
covariance, Common probability distributions, Multivariate distributions, Bayes rule.

3. Parameter estimation

Bias, consistency, mean squared error, maximum likelihood estimator; Hypothesis testing: likelihood
ratio tests, Neyman-Pearson lemma; Bayesian inference: prior and posterior distribution; Bayesian point
estimates: MAP estimator.

4. Numerical Optimization for ML

Introduction to Numerical Optimization, Modeling --- variables, criteria, constraints, Gradient descent,
stochastic gradient descent, Step-size selection, Linear Least squares, Second order methods(Newton,
Quasi-newton), Unconstrained and Constrained Optimization, Case study: Support Vector machines.

5. Data Visualization

The value of visualization; Visualization design: data types, visualization pipeline, marks and channels,
color; Visualization techniques: charts, graph layout, treemaps; Visualization systems.

3
Module 2

Paradigms of Machine Learning

1. Supervised Learning

Introduction to SL, How to formulate a SL problem, linear classification, linear regression, loss function,
regularization, gradient algorithms, features, non-linear classification, tutorial on using ML packages.

2. Unsupervised Learning

Introduction to UL, Parameter Estimation, EM Algorithm, learning, mixtures

Module 3

Deep Learning and its applications

1. Deep Learning Architectures

Logistic regression, feedforward neural networks, backpropagation, deep networks, optimization

algorithms for training deep networks, regularization, dropout, batch normalization, convolutional
neural networks (CNN), recurrent neural networks (RNN), long short term memory (LSTM) networks.
Attention in neural networks. Understanding deep networks. Case study: Different networks and training
strategies for MNIST.

2. Deep Learning for Speech and Audio Processing

Audio representations for deep learning, models in speech recognition, speaker and language recognition
models. End-to-end deep networks, sequence-to-sequence encoder-decoder attention models. Audio
event detection models. Case study: End-to-end speech recognition

4
3. Deep Learning for Computer Vision

Popular CNN architectures (AlexNet, VGG, ResNet, GoogleNet), transfer learning, autoencoders, object
detection (FasterRCNN/Yolo), image segmentation (U-Net, DeepLab), generative adversarial networks
(GAN) and variants, RNN and LSTM for image captioning/video Case studies: CNN training example on
CIFAR-10 dataset, transfer learning from pre-trained network on ImageNet to cats/dogs, training Yolo
for pedestrian detection.

4. Deep Learning for Natural Language Processing

Introduction, Distributed word representations, Recurrent neural networks for language modeling,
Long short-term memory networks, Convolutional neural networks for Text, Transformer networks,
Contextual word representations, Modeling attention, Applications – Sentiment analysis, Machine
translation, Reading comprehension, Textual entailment, Text classification.

5. Deep Reinforcement Learning

Introduction to sequential decision making under uncertainty, Markov Decision Process (MDP), Value
function and Q-functions, Finite and Infinite Horizon problems, Value and policy iteration, Policy
gradients, implementing RL algorithms with deep neural networks.

6. Deep Learning for IoT/Edge Devices

Overview of various ML hardware for IoT/Edge devices, Energy-efficiency, speed trade-off for the
IoT/Edge devices, Optimization techniques to compress the ML model for Edge devices, Use cases of
ultra-light ML algorithms on Edge devices.

5
Module 4

Representation Learning

1. Deep Generative Models I

Introduction to Generative models, Latent variable models, Variational inference and Auto Encoders

2. Deep Generative Models II

Adversarial Learning, Generative Adversarial networks and variants, Normalizing Flows, Applications
for image-to-image translation, image inpainting and attribute manipulation, Dataset Augmentation,
Wav2Vec

3. Semi and Self-supervised Learning I

Consistency Regularization, Weakly supervised learning methods, Domain Adaptation, transfer learning
and Domain generalisation, Applications on learning with scarce data

4. Semi and Self-supervised Learning II

Self-supervised and Contrastive Representation Learning, Contrastive losses, Hard negative mining,
Applications in learning with unlabelled data

Disclaimer: This is the outline of the program in general. Topics covered in the program can be
changed at the discretion of the IISc faculty based on the class dynamics and industrial trends

6
For any query, Contact

Fasahath +91 73373 39785

Write to

[email protected]

To apply visit

iisc.talentsprint.com/deeplearning

Programmme Partner