IT549: Deep Learning

Lecture - 01

A Primer for Deep Learning

[Definition, Tasks, and Case Studies]

Arpit Rana
2nd January 2025
 Course Logistics
Syllabus and Evaluation Scheme
Course Logistics

Dr. Arpit Rana

Instructor Room-3105, Faculty Block-3
Email: [email protected] (with a prior appointment only)

Himanshu Beniwal ([email protected]) - PMRF TA

TA Contact Info
DA-IICT, TBD

Prerequisites Programming in Python, Machine Learning

Eligibility B.Tech. VI Semester, M.Tech. II Semester, and Ph.D. Students

Course Logistics

Credit Weighting 4 Credits (3-0-2)

Lectures Monday, Thursday: 12:00 - 13:00 hrs. and Friday: 08:00 -

[CEP-108] 09:00 hrs.

Lab
Thursday, 14:00 – 16:00
[LT-02]

Private Study At least 4 hrs per week

● Learn how to solve Data-driven Decision-Making

Problems;
● Learn how to work on structured and unstructured
Potential Outcome
(e.g., text, image) data;
● Targeted Jobs: Data Scientist, ML Engineer,
Research Engineer, AI Engineer
Course Logistics

● In-Sem Exams: 30% (15% + 15%)

● End-Sem Exam: 30%
Assessment ● Course Project:: 45%
Extra Credits: ML Challenges, Participate on Google Stream

Skip lectures; avoid private study; cram just before the exam;
How to Fail expect the exam to be a memory test; copy project assignments;
be inactive on the course stream

Attend lectures; summarize the notes; expect a problem-solving

How to Pass exam; do your project yourself; be active and accurate in the
class and on the course stream; and do group study
Preliminary Schedule

Week Lecture Lab Project

Week-1 Course Admin; Fundamentals of Predictive Analytics:
– No lab – –
[1 Jan 2025] Representation, Evaluation, and Optimization

– No lab –
Week-2 Introduction to Neural Networks: Neurons, Activation,
(Group formation and domain –
[6 Jan 2025] Layers, Architecture, and Examples
finalization through Google form)

Training Deep Neural Networks: Backpropagation,

Week-3 Vanishing Gradient Problem: Definition, Better Linguistic Preprocessing, Text Project
[13 Jan 2025] Weight Initialization, Non-Saturating Activation Vectorization, and Embeddings Assigned
Functions, and Batch-Normalization

Week-4 Overfitting: Reducing the network size, Weight or

Pre-training and Fine-tuning –
[20 Jan 2025] Max-norm regularization, Dropout, Early stopping

Week-5 100-Minute ML Development

Applications in Natural Language Processing –
[27 Jan 2025] Challenge

Week-6 First In-Semester Examination

–
[3 Feb 2025] (7th Feb - Friday to 11th Feb - Tuesday)
Preliminary Schedule

Convolution Neural Networks (CNNs): Introduction,

Week-6 Motivation, Images and Rank-3 Tensors, Convolution Image Vectorization or Image Tensors, Progress
[10 Feb 2025] Layers, Filters, Feature Maps, and Stacking Using Pre-trained CNNs, Fine-tuning, etc. Check I
Convolution Layers

Training CNNs: Memory requirements, overfitting,

Week-7 Object Detection and Image
data augmentation, Pre-trained CNNs, Transfer –
[17 Feb 2025] Segmentation
Learning,

Recurrent Neurons and Layers; Training RNNs;

Week-8 Working with Sequences or Time-Series
Forecasting a Time Series using Simple RNN, Deep –
[24 Feb 2025] Data
RNN; Handling Long Sequences: LSTM;

Week-9 Encoder-Decoder Network: Bidirectional RNN, Beam

100-Minute ML Development Challenge –
[3 Mar 2025] Search; Attention Mechanism:

Week-10 In-semester Break

-
[10 Mar 2025] (Entire week)

Week-11 The Transformer Architecture — Training of Progress

Training Transformers
[17 Mar 2025] Transformer Check II

Week-12 Second In-Semester Examination

–
[24 Mar 2025] (22 Mar Saturday to 26th Mar Wednesday)
nd
Preliminary Schedule

Deep learning for unsupervised learning;

Unsupervised Learning using
Week-13 Architecture design of AE: Linear, Stacked,
Auto-encoders: An application for –
[31 Mar 2025] Convolutional; Recurrent; Denoising, Sparse; and
demonstration
Variational; Generative Learning using VAE.

Generative modeling as a game-theoretic approach;

Week-14 Training GANs: An application for
Architecture design of GAN; Training methodology -
[7 Apr 2025] demonstration
of GAN; Applications.

Recommendation Systems: Definition, objectives,

components, approaches, evaluation, and challenges.
Week-15
Models: Matrix Factorization, Neural Matrix 100-Minute ML Development Challenge -
[14 Apr 2025]
Factorization, and Collaborative Denoising
Auto-Encoders;

Week-16 Last date of Classes, Labs, and Tutorials Progress

[21 Apr 2024] (23rd Apr 2025) Check III

Week-17 End-Semester Examination

[29 Apr 2024] (24 Apr 2025 to 02 May 2025)

Course Project Evaluation

(05 May 2025 to 07 May 2025)
Course Policy

Student Groups

● The course project will be allocated to groups of three/ four members. Each group will
also be involved in 100-minute ML development challenges.
● 45% of your marks will be based on team efforts, so choose your members wisely.
● Teams will remain unchanged throughout the semester once registered. No requests for
changes will be entertained.
● Team registration will open the second week after classes start, and you must register
your team via a Google form within three days of the announcement.
● During lab hours, there will be three machine-learning development challenges. The
three most proficient teams shall be acknowledged with a bonus of up to 3% on their
respective scores.
● Every team member must understand the concepts, code, and claims they submit, as
any member may be asked questions about their project.
Course Policy

Course Project
● There is only one-course project, an End-to-End ML application.
● Student groups must select a thematic domain: Finance, E-commerce, Healthcare,
Pharma, Sports, Entertainment, Renewable Energy, Oil & Gas, Automobile, Agriculture,
FMCG, Security, Social Media, Supply Chain, or any other exciting and valuable domain.
● Each group will define the problem in their selected domain and collect dataset(s) from
reliable sources, including publicly available ones (no two groups can work on the same
dataset and not more than two groups can work on the same domain). You are
encouraged to gather additional data to enhance your dataset and better address the
problem.
● Each group must develop a multimodal machine-learning application and select a
dataset with all the necessary modalities. You must add a novel contribution to your
project and compare yours with the existing baselines.
● Three progress checks are scheduled to ensure incremental progress, not a last-week
effort.
● The project guideline document will provide information on domain allocation, general
instructions, evaluation criteria, and other protocols.
Course Policy

Submission
● One group member will submit the project report and the code on Google Classroom.
Submission instructions will be provided in the project guideline document.
● Evaluation will primarily be online, reviewing your code. Any group member may be
asked questions about anything in the assignment.
● Late submissions (up to 24 hours) will incur a 20% penalty.
● Plagiarism includes:
○ Copying any segment of code from any source.
○ Submitting code not written by you personally.
● Suspected plagiarism will result in a ZERO for the assignment.
Introduction
Definition and Tasks
What is Data (Knowledge) Mining?

The process of automatically (or

semi-automatically) discovering interesting
patterns from large amounts of data.

● Implicit (somewhat hidden),

● Non-trivial (not obvious),

● Previously unknown (novel), and

● Potentially useful (for consumers /

sellers / stakeholders)

Image Source: https://www.investopedia.com/terms/d/datamining.asp

Data (Knowledge) Mining: Knowledge Discovery from Data

Databases Extracting
Flat files Interesting Patterns Novel
Data Warehouses using Actionable
and so on. Intelligent Methods Useful

Data Post-
Data Data Mining Knowledge
Preprocessing processing

Textual data
Cleaning Evaluation w.r.t.
e.g. text, blogs
Reduction
- Interestingness
Multimedia data Transformation
- Completeness
e.g. image, video Discretization
- Optimality
Sequential data Selection etc.
e.g., gene sequence
Spatial data
e.g. maps,
and so on.

DIKW Pyramid: https://www.ontotext.com/knowledgehub/fundamentals/dikw-pyramid/

Data Mining vs. Machine Learning

The process of automatically (or Machine learning (ML) is focused on

semi-automatically) discovering interesting understanding and building methods that
patterns from large amounts of data. 'learn'.

It uses methods at the intersection of It leverages data to improve performance

machine learning, statistics, and database on some set of tasks.
systems.
E.g.: A spam filter (an ML program)
E.g., customer churn
Data Mining Tasks

Data Mining Tasks

The actual data mining task is the semi-automatic or automatic
analysis of large quantities of data to extract interesting patterns.

Descriptive Predictive
Find human-interpretable patterns Use some variables to predict future
that describe the data. or unknown values of other variables.

● Cluster Analysis ● Regression

● Outlier Analysis ● Classification
● Association Rule Mining
● Sequence Pattern Mining

In Machine Learning terminology, these In Machine Learning terminology, these

tasks are categorised as “Unsupervised tasks are categorised as “Supervised
Learning”. Learning”.
Data Mining Tasks

Data Mining Tasks

The actual data mining task is the semi-automatic or automatic
analysis of large quantities of data to extract interesting patterns.

Descriptive Predictive
Find human-interpretable patterns Use some variables to predict future
that describe the data. or unknown values of other variables.

● Cluster Analysis ● Regression

● Outlier Analysis ● Classification
● Association Rule Mining
● Sequence Pattern Mining

In Machine Learning terminology, these In Machine Learning terminology, these

tasks are categorised as “Unsupervised tasks are categorised as “Supervised
Learning”. Learning”.
Machine Learning: Definition

Machine Learning is

● the science (and art) of programming computers

● so they can learn from data. AI

ML
– Aurelien Geron, Google
DL

Gen
-AI
Machine Learning: Example

A Spam Filter,
● a Machine Learning Program, given
○ examples of “spam” emails (e.g. flagged by
users), and
○ examples of “ham” (i.e. regular) emails
● can learn to flag spam
Machine Learning: A New Programming Paradigm

Data Rules Data Answers

Traditional
Programming Machine
Learning
(Symbolic AI)

Answers Rules

● A long list of complex (hard coded) rules ● Automatically learns which words or
phrases are good predictors of spam
● Keep writing new rules as the new
phrases are introduced by spammers
Machine Learning: Definition Revisited

Machine Learning is the training of a model from data that generalises a decision against a
performance measure.

● Training a model suggests training examples. Data Answers

● A model suggests state acquired through experience.

● Generalises a decision suggests the capability to make a

decision based on inputs and anticipating unseen inputs in
the future for which a decision will be required. Machine
Learning
● against a performance measure suggests a targeted need and
directed quality to the model being prepared.

Model
Learning = Representation + Evaluation + Optimization

Representation
Choosing a representation of the learner: the hypotheses
space or the model class — the set of models that it can
possibly learn.

Evaluation
Choosing an evaluation function (also called objective
function, utility function, loss function, or scoring
function) is needed to distinguish good classifiers from
bad ones.

Optimization
��
Choosing a method to search among the models in the
hypothesis space for the highest-scoring one.
Learning = Representation + Evaluation + Optimization

✔
✔ ✔ ✔
✔ ✔ ✔

✔ ✔
✔ ✔ ✔
✔
✔

✔
 Business Case Studies
Fakespot, GoKwik, and Intello Labs
Case Study - I: Fakespot

Problem Identified
● Nearly 93% consumers read reviews before any kind of purchasing decision.
● Out of these, around 91% of 18–34 year olds trust reviews as much as a
recommendation from a friend!
● Over 30% of reviews are found to be fake.

Target Audience
All e-commerce businesses that allow users to write
reviews.

Data-driven Solution
Fakespot reports provide an Adjusted Rating that
weighs reviews based on authenticity and then Courtesy: Fakespot
recalculates it.

For more information, go to Fakespot Blogs: https://www.fakespot.com/posts

Case Study - II: GoKwik

Problem Identified
● In e-commerce, more than 30% of orders are returned to origin (RTO, i.e. shipped back
to the warehouse) in India.

Target Audience
All e-commerce businesses

Data-driven Solution
● Mostly, CoD orders are converted to
RTO.
● So, analyzing customer behavioural
patterns and disable CoD option for Courtesy: Gokwik
those showing high-risk RTO
behaviour.

For more information, go to GoKwik Blogs: https://www.gokwik.co/blog/

Case Study - III: Intello Labs

Problem Identified
● One-third of the food produced in the world for human consumption every year gets
lost or wasted.
● Mainly (in some countries) at the early stages of the food value chain.

Target Audience
From growers to packers, from
exporters to food services

Data-driven Solution
Smart, scalable solutions to digitize
food quality, achieve fair pricing and
reduce food wastage. Using AI, ML, and Computer Vision technology

For more information, go to Intello Labs Blogs: https://blogs.intellolabs.com/en

Next lecture
Primer for Deep Learning Contd…
3rd January 2025

Note: Study first three chapters of Data Mining and

Machine Learning by Mohammed J. Zaki. Second Edition.