Introduction to

Machine Learning
Ketan Rajawat
IIT Kanpur
What is ML?
• Humans: learn from experience

• Computers (machines): follow instructions

• Smarter machines: learn from experience

• Better or worse than humans
• Humans do not know how they learn
• How to learn from experience?
ML Workflow

Data (Imaging, Sensors, Algorithms Perform task “like” a

Observations) (e.g.: Deep Learning) human
Example: deepface

Manual tagging is slow

Facebook AI (deepface) trained on 4 million face photos
Not 100% accurate (~97%)
Example: Recommendation Engines
Example: Chat Bots
Understanding ML
Introduction to Machine Learning
Example: picking tasty mangoes

• I love mangoes
• Too busy to pick out the best ones myself
• Design a robot that can
• learn my taste preferences and
• choose tasty mangoes for me
• Key Idea: Robot should predict whether a mango will taste good
based on easily measurable features
Features
• Potential features
• Variety (type of mango)
• Ripeness level
• Color
• Season of harvest

• Let us start small and focus on only two features

• Color (green to yellow to spotted scale)
• Firmness (how hard or soft the mango feels)
Data Collection
• Gather a set of mangoes
• For each mango, record its color and firmness
• Taste Test: determine which mangoes are “tasty” according to my
preferences (this is our label).
• Training Set: each mango is a data point
• Data point: (Color, Firmness) → Tasty/Not
Tasty.
• Use this data to teach the robot how to
predict taste
Taste prediction
• Create a rule to predict taste

Firm with
yellowish color

Decision tree
classifier

• Goal: Predict the taste of a new mango (without actually tasting it)
How to improve accuracy?
• More data
• larger training sets usually help
• Unless data is poor quality (e.g. labeling errors)

• More features
• Sometimes (e.g. image and speech recognition)
• But not necessarily (e.g.: day of the week, medical diagnostics)

• More complicated model

• Sometimes (e.g. LLMs, Deep Learning)
• But not necessarily (e.g. credit scoring in finance)
Terminology: AI, ML, DL, DS
Types of Machine Learning
• Supervised Learning: regression, classification

• Unsupervised Learning: clustering, dimensionality reduction

• Reinforcement Learning

•…
 Supervised Learning

Area Perimeter Height Width Label

0.5 3 0.87 1 Triangle
1 4 1 1 Square
1 6 1.73 2 Triangle

https://www.javatpoint.com/supervised-machine-learning
Examples of Supervised Learning
Unsupervised Learning

• Which ones have something in common?

• Finding patterns in unlabeled data
Examples of Unsupervised Learning
Reinforcement Learning
• Analogy: training your dog
• Reward good behavior
• Penalize bad behavior

• Similar: train a computer to play Chess/Go

• Award points for winning
• Deduct points for losing
• Find an algorithm that maximizes points
 Quiz: supervised or unsupervised?

Digiyatra recognizes your face when entering the airport

Amazon suggest what you should purchase

Email identified as spam

AI Terminology: Deep Learning
• The models are encapsulated in Neural Networks (NNs)
• Convolutional NN (CNN) – for images, videos
• Recurrent NN (RNN) – for audio, text
• Deep learning involves very large NNs
• Each node is a parameter that must be learned during training

• Eg: ChatGPT 4o has

1 trillion parameters
Generative AI

• “Generate” intelligent content

• Sophisticated algorithms created by combination of
• Supervised learning
• Unsupervised learning
• Reinforcement learning
Generative Adversarial Network
Categorical Features
• Categorical: Gender, Country, City, Food
• How to represent?
Categorical Features
• Numerical valued data:
• Continuous: temperature, length
• Discrete: rating (1-5), age

• Depends on dataset
 Classification vs. Regression
• Label is either categorical or numerical

• What is the price of

• What is this food?
bitcoin tomorrow?
• What is this animal?
• What will be the
• Who is this person?
temperature tomorrow?
Classification vs. Regression
Model
Feature 1
Feature 2
Model Output (Prediction)
Feature M

• How do we know if the model is good?

 Example: Diabetes prediction

Labels
Rows are
Single data
features
point

https://www.kaggle.com/datasets/mathchi/diabetes-data-set
How are models trained?
• Start with a basic (untrained) model, e.g. always outputs 1

𝑥𝑖
Model output 𝑦ො𝑖 = 𝑓 𝑥𝑖
2 115 0 0 0 35.3 0.134 29 Model 1

Loss ℓ𝑖 = 𝑦ො𝑖 − 𝑦𝑖 2

0
Actual label 𝑦𝑖
Optimization Fine-tune
the model

Ideally model output = actual label for every data point

Total loss = 0
Generalization ability
• Suppose we train a model that perfectly predicts diabetes for
every data point (total loss = 0)
• Are we done?
• Recall original goal: predict well for new (unseen) data
• Eg: New patient comes, is she likely to get diabetes?

• Would our model predict well over new & unseen data?
• How to assess the ability of our model to generalize on new data?
 Training and testing
• Learn model over training data
(minimize training loss)
• Calculate loss over test data
• Which model is better?

Model A B C D
Training loss 0.9 0.8 1 0.001
Test loss 0.8 0.9 0.7 0.91

Best model
• Can use other loss functions or other metrics like accuracy

https://www.linkedin.com/pulse/traintest-split-versus-cross-fold-validation-william-monroe/
 Working example
• Predict if house price is high or low

• Features: Median income, Average age, # rooms, # bedrooms,

district population, occupancy, latitude, longitude

• The last column contains the house price normalized from 1-5.
We can consider price >=2 to be high and <2 to be low.

https://colab.research.google.com/drive/1ge4xx5A6n3orwM0pBLbquUsceLYHUxBH?usp=sharing
 Try out other models
model Training accuracy Test accuracy
model = LogisticRegression() 80.6 81.6
model = SVC(kernel='linear', C=1.0) 83.3 83.8
model = DecisionTreeClassifier() 100 84.4
model = RandomForestClassifier() 100 89
model = KNeighborsClassifier() 77.5 62.5
model = GaussianNB() 76.4 76.7
model = GradientBoostingClassifier() 89 88.2
model = MLPClassifier() 76 76
model = LogisticRegression(penalty='l1', solver='liblinear', 83.2 83.9
max_iter=1000) # L1 regularization
Homework
• Study more about decision tree classifier

https://www.youtube.com/watch?v=ZVR2Way4nwQ
https://scikit-learn.org/stable/modules/tree.html

• Run the code for different values of max_depth parameter of

decision tree classifier
Thank You
• Longer introduction to ML
https://www.youtube.com/watch?v=i_LwzRVP7bg

• Book on basics of ML
https://www.microsoft.com/en-us/research/uploads/prod/2006/01/Bishop-
Pattern-Recognition-and-Machine-Learning-2006.pdf