Uka Tarsadia University

B. Tech.
Semester VI

DEEP LEARNING
AI5014

EFFECTIVE FROM June-2023

Syllabus version: 1.00
 Teaching Scheme
Subject
Subject Title Hours Credits
Code
Theory Practical Theory Practical

AI5014 Deep Learning 3 2 3 1

Theory Practical
Subject Examination Examination Total
Subject Title Marks Marks
Code Marks
Internal External CIE

AI5014 Deep Learning 40 60 50 150

Objectives of the course:

 To expose the student of this course from fundaments of neural networks to most
advanced topics of the deep learning architectures.

Course outcomes:
Upon completion of the course, the student shall be able to,
CO1: Understand the types of neural networks with their topologies.
CO2: Understand and apply training of deep neural network architectures.
CO3: Understand and apply Radial Basis Function Networks.
CO4: Understand and apply Restricted Boltzmann Machines and recurrent networks.
CO5: Understand and apply Convolutional architectures and reinforcement learning.
CO6: Elaborate advanced topics like transformer and generative adversarial
architectures.

Sr. No. Topics Hours

Unit – I

1 Introduction to Neural Networks and Learning with Shallow 8

Networks:
The basic architecture of neural networks, Training a neural
network with back-propagation, Practical issues in neural network
training, Common neural network architectures, Neural
architectures for binary classification models, Neural architectures
for multiclass models, Backpropogated saliency for feature
selection, Matrix factorization with Autoencoders, Word2Vec,
Neural networks for graph embeddings.
 Unit – II
2 Training Deep Neural Networks and Teaching Deep Learners 8
to Generalize:
Back-propagation, Setup and initialization issues, The vanishing
and exploding gradient problems, Gradient-descent strategies,
Batch normalization, The bias-variance trade-off, Generalization
issues in model tuning and evaluation, Penalty based
regularization, Ensemble methods, Early stopping, Unsupervised
pretraining, Continuation and Curriculum learning, Parameter
sharing.
Unit – III
3 Radial Basis Function (RBF) Networks: 7
Training an RBF network, Variations and special cases of RBF
networks, Relationship with kernel methods.
Unit – IV
4 Restricted Boltzmann Machines (RBM): 8
Hopfield networks, The Boltzmann machine, Restricted Boltzmann
Machines, Applications of Restricted Boltzmann Machines, Using
RBMs beyond binary data types, Stacking RBMs.

Recurrent Neural Networks (RNNs):

The architecture of RNN, The challenges in training recurrent
networks, Echo-state networks, Long Short-Term Memory (LSTM),
Gated Recurrent Units (GRUs), Applications of RNN.
Unit – V
5 Convolutional Neural Networks (CNN): 8
The basic structure of Convolutional network, Training a
Convolutional network, Case studies of Convolutional
architectures, Visualization and unsupervised learning,
Applications of CNN.

Deep Reinforcement Learning:

Stateless algorithms, The basic framework of reinforcement
learning, Bootstrapping for value function training, Policy gradient
methods, Monte Carlo tree search, Case studies of reinforcement
learning.

Unit – VI
6 Advanced Topics in Deep Learning: 6
Attention mechanism, Neural networks with external memory,
Generative Adversarial Networks (GANs), Competitive Learning,
Limitations of neural networks.
 Sr. No. Deep Learning (Practicals) Hours
1 To implement and analyze back-propagation neural network 2
architecture training with handwritten character datasets.
2 To implement RBF. 4
3 To implement and train RBM. 2
4 To implement RNN architecture. 2
5 To implement and train RNN for sequence prediction task. 4
6 To implement and train simple CNN on CIFAR100 dataset. 4
7 Hands-on with pre-trained state-of-art CNN models. 4
8 To implement deep reinforcement learning models. 4
9 Hands-on with pre-trained state-of-art transformer models. 4

Text book:
1. Charu C. Aggarwal, “Neural Networks and Deep Learning”, Springer.

Reference books:
1. Ian Goodfelllow, Yoshua Benjio, Aaron Courville, “Deep Learning”, The MIT Press.

2. Richard O. Duda, Peter E. Hart, David G. Stork, “Pattern Classification”, John Wiley &
Sons Inc.
3. Richard S. Sutton, Andrew G. Barto, and Francis Bach, “Reinforcement Learning: An
Introduction”, MIT Press, ISBN 978-0262039246.
4. Marc Peter Deisenroth, A. Aldo Faisal, and Cheng Soon Ong, “Mathematics For
Machine Learning”, Cambridge University Press, ISBN 978-1108455145.
5. Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani, and Jonathan Taylor,
“An Introduction to Statistical Learning: with Applications in Python”, Springer
Publication, ISBN 978-3031387463.

Course objectives and Course outcomes mapping:

 To expose the student of this course from fundaments of neural networks to most
advanced topics of the deep learning architectures: CO1, CO2, CO3, CO4, CO5, and
CO6.

Course units and Course outcomes mapping:

Unit Course Outcomes
No. Unit Name
CO1 CO2 CO3 CO4 CO5 CO6
1 Introduction to Neural Networks
and Learning with Shallow ✓
Networks
2 Training Deep Neural Networks and
Teaching Deep Learners to ✓
Generalize
3 Radial Basis Function (RBF) ✓
 Networks

4 Restricted Boltzmann Machines

(RBM) and Recurrent Neural ✓
Networks (RNNs)
5 Convolutional Neural Networks
(CNN) and Deep Reinforcement ✓
Learning
6 Advanced Topics in Deep Learning ✓

Programme outcomes:

PO 1: Engineering knowledge: An ability to apply knowledge of mathematics,

science, and engineering.
PO 2: Problem analysis: An ability to identify, formulates, and solves engineering
problems.
PO 3: Design/development of solutions: An ability to design a system, component,
or process to meet desired needs within realistic constraints.
PO 4: Conduct investigations of complex problems: An ability to use the techniques,
skills, and modern engineering tools necessary for solving engineering
problems.
PO 5: Modern tool usage: The broad education and understanding of new
engineering techniques necessary to solve engineering problems.
PO 6: The engineer and society: Achieve professional success with an
understanding and appreciation of ethical behavior, social responsibility, and
diversity, both as individuals and in team environments.
PO 7: Environment and sustainability: Articulate a comprehensive world view that
integrates diverse approaches to sustainability.
PO 8: Ethics: Identify and demonstrate knowledge of ethical values in non-
classroom activities, such as service learning, internships, and field work.
PO 9: Individual and team work: An ability to function effectively as an individual,
and as a member or leader in diverse teams, and in multidisciplinary settings.
PO 10: Communication: Communicate effectively on complex engineering activities
with the engineering community and with society at large, such as, being able
to comprehend and write effective reports and design documentation, make
effective presentations, and give/receive clear instructions.
PO 11: Project management and finance: An ability to demonstrate knowledge and
understanding of the engineering and management principles and apply
these to one’s own work, as a member and leader in a team, to manage
projects and in multidisciplinary environments.
PO 12: Life-long learning: recognition of the need for, and an ability to engage in life-
long learning.

Programme outcomes and Course outcomes mapping:

Programme Course Outcomes

Outcomes
CO1 CO2 CO3 CO4 CO5 CO6

PO1 ✓ ✓

PO2 ✓ ✓ ✓

PO3 ✓ ✓ ✓

PO4

PO5 ✓ ✓ ✓

PO6

PO7

PO8

PO9

PO10

PO11

PO12