Scribd
Upload
9 views30 pages

Classification: Prof. Gheith Abandah

This document discusses classification algorithms. It introduces the MNIST dataset and explores training binary classifiers. It then covers performance measures like accuracy, confusion matrices, and precision and recall. Multiclass classification algorithms like one-vs-all and one-vs-one are explained. Finally, multilabel classification is defined.

Uploaded by

rana
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
9 views30 pages

Classification: Prof. Gheith Abandah

This document discusses classification algorithms. It introduces the MNIST dataset and explores training binary classifiers. It then covers performance measures like accuracy, confusion matrices, and precision and recall. Multiclass classification algorithms like one-vs-all and one-vs-one are explained. Finally, multilabel classification is defined.

Uploaded by

rana
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 30

Classification

Prof. Gheith Abandah

Reference: Hands-On Machine Learning with Scikit-Learn and


TensorFlow by Aurélien Géron (O’Reilly). Copyright 2017 Aurélien
Géron, 978-1-491-96229-9.

1
Introduction
• YouTube Video: Machine Learning - Supervised
Learning Classification from Cognitive Class

https://youtu.be/Lf2bCQIktTo

2
Outline
1. MNIST dataset
2. Training a binary classifier
3. Performance measures
4. Multiclass classification
5. Multilabel classification
6. Exercise

3
1. MNIST Dataset

• MNIST is a set of 70,000


small images of
handwritten digits.
• Available from
mldata.org
• Scikit-Learn provides
download functions.

4
1.1. Get the Data

5
1.2. Extract Features and Labels

There are 70,000 images, and each image has 784


features. This is because each image is 28×28 pixels,
and each feature simply represents one pixel’s
intensity, from 0 (white) to 255 (black).

6
1.3. Examine One Image

7
1.4. Split the Data

• The MNIST dataset is actually already split into a training


set (the first 60,000 images) and a test set (the last 10,000
images).
• You need to shuffle the training set to guarantee that all
cross-validation folds are similar.

8
Outline
1. MNIST dataset
2. Training a binary classifier
3. Performance measures
4. Multiclass classification
5. Multilabel classification
6. Exercise

9
2. Training a Binary Classifier
• A binary classifier can classify two classes.
• For example, classifier for the number 5, capable of
distinguishing between two classes, 5 and not-5.

True for all 5s, False for all


other digits.

Stochastic Gradient
Descent (SGD) classifier
10
2. Training a Binary Classifier
• Note that a better classifier for this problem is the
Random Forest Classifier.

11
Outline
1. MNIST dataset
2. Training a binary classifier
3. Performance measures
4. Multiclass classification
5. Multilabel classification
6. Exercise

12
3. Performance Measures

• Accuracy: Ratio of correct predictions


• Confusion matrix
• Precision and recall
• Precision/recall tradeoff

13
3.1. Accuracy

Example how to find the


accuracy.

Using the cross_val_score() function


to find the accuracy on three folds

14
3.2. Confusion Matrix

15
3.2. Confusion Matrix
• Scikit Learn has a function for finding the confusion
matrix.

• The first row is for the non-5s (the negative class):


• 53,272 correctly classified (true negatives)
• 1,307 wrongly classified (false positives)
• The second row is for the 5s (the positive class):
• 1,077 wrongly classified (false negatives)
• 4,344 correctly classified (true positives)

16
3.3. Precision and Recall

Precision Recall

The precision and recall are smaller than the accuracy.


Why? 17
3.4. Precision/Recall Tradeoff
• Increase the decision threshold to improve the
precision when it is bad to have FP.
• Decrease the decision threshold to improve the
recall when it is important not to miss FN.

18
3.4. Precision/Recall Tradeoff

19
3.4. Precision/Recall Tradeoff
• The function cross_val_predict() can return
decision scores instead of predictions.

• When using larger decision threshold, we increase


the precision and decrease the recall.

20
Outline
1. MNIST dataset
2. Training a binary classifier
3. Performance measures
4. Multiclass classification
5. Multilabel classification
6. Exercise

21
4. Multiclass Classification
• Multiclass classifiers can distinguish between more
than two classes.
• Some algorithms (such as Random Forest classifiers
or Naive Bayes classifiers) are capable of handling
multiple classes directly.
• Others (such as Support Vector Machine classifiers
or Linear classifiers) are strictly binary classifiers.
• There are two main strategies to perform multiclass
classification using multiple binary classifiers.

22
4.1. One-versus-All (OvA) Strategy

• For example, classify the digit images into 10


classes (from 0 to 9) to train 10 binary classifiers,
one for each digit (a 0-detector, a 1-detector, a 2-
detector, and so on).
• Then to classify an image, get the decision score
from each classifier for that image and select the
class whose classifier outputs the highest score.

23
4.2. One-versus-One (OvO)
Strategy
• Train a binary classifier for every pair of digits.
• If there are N classes, need N × (N – 1) / 2 classifiers.
For MNIST, need 45 classifiers.
• To classify an image, run the image through all 45
classifiers and see which class wins the most duels.
• The main advantage of OvO is that each classifier only
needs to be trained on a subset of the training set.
• OvO is preferred for algorithms (such as Support Vector
Machine) that scale poorly with the size of the training
set.

24
4.3. Scikit Learn Support of
Multiclass Classification
• Scikit-Learn detects when you try to use a binary
classification algorithm for a multiclass
classification task, and it automatically runs OvA
(except for SVM classifiers for which it uses OvO).

25
4.3. Scikit Learn Support of
Multiclass Classification
• Note that the multiclass task is harder than the
binary task.
• Binary task:

• Multiclass task:

26
Outline
1. MNIST dataset
2. Training a binary classifier
3. Performance measures
4. Multiclass classification
5. Multilabel classification
6. Exercise

27
5. Multilabel Classification
• Classifiers that output multiple classes for each
instance.

Popular algorithm

28
Summary
1. MNIST dataset
2. Training a binary classifier
3. Performance measures
4. Multiclass classification
5. Multilabel classification
6. Exercise

29
Exercise
• Try to build a classifier for the MNIST dataset that
achieves over 97% accuracy on the test set. Hint:
the KNeighborsClassifier works quite well for this
task; you just need to find good hyperparameter
values (try a grid search on the weights and
n_neighbors hyperparameters).

30

You might also like