Syllabus_ECL17
Syllabus_ECL17
R-20
COURSE OBJECTIVES:
This course will enable students to
1. Understand supervised and unsupervised learning paradigms
2. Understand the basic ideas and principles of Neural Networks
3. Learn the fundamentals of back propagation algorithm.
4. Learn basic principles of deep networks.
COURSE OUTCOMES:
After successful completion of the course, the students are able to
1. Compare supervised and unsupervised learning algorithms.
2. Identify neural network architectures and apply unconstrained optimization techniques.
3. Apply simple neural network based algorithms.
4. Illustrate the principles of deep networks.
Introduction to Neural Networks: What is a Neural Network?, Models of a Neuron, Neural networks
viewed as directed graphs, Feedback, Network architectures, Rules for knowledge representation,
Learning Processes: Error-Correction Learning, Memory based Learning and Hebbian Learning,
Unconstrained optimization techniques - Method of Steepest Descent, Newton's method and
Gauss-Newton method, Linear Least Squares filters.
Single-Layer Perceptrons: Least Mean Square algorithm, Perceptron, Perceptron convergence theorem,
Multi-Layer Perceptrons: Architectural graph, Back propagation algorithm, XOR problem, Output
representation and Decision Rule, Feature detection, Back propagation and Differentiation, Hessian
Matrix, Cross Validation.
Deep feedforward networks - Learning XOR, Stochastic Gradient Descent algorithm, Gradient based
learning - Cost functions, Output Units - Linear Units and Sigmoid Units, Hidden Units - Rectified Linear
Units and their generalizations, Regularization for Deep Learning: Parameter Norm Penalties - Ridge
Regression, Parameter sharing, Sparse representations, Bagging, Challenges in Neural Network
Optimization, Convolutional Networks: The Convolution Operation, Motivation, Pooling
LEARNING RESOURCES:
TEXT BOOK(s):
1. Neural Networks: A Comprehensive Foundation by Simon Haykin, 2nd edition, Pearson Education.
2. Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville, MIT Press, 2016.
REFERENCE BOOK(s):
1. Jacek M. Zurada, "Introduction to Artificial Neural Systems", JAICO Publications. ISDN
9788172246501
2. B.Yegnanarayana, "Artificial Neural Networks", PHI, New Delhi.
3. Neural Network Design by Martin T. Hagan and Howard B. Demuth, 2nd Edition