Subject: Machine Learning

Module I: Supervised Learning

Mr. K.V.Satya Kumar
Assistant Professor
Department of EECE
GITAM Institute of Technology (GIT)
Visakhapatnam – 530045
Email: [email protected]
9 January 2023 Department of EECE, GIT 19EEC334A: MACHINE LEARNING 1
Unit-I
Supervised Learning (Regression/Classification): Basic methods: Distance-based
methods, Nearest-Neighbors, Decision Trees, Naive Bayes, Linear models:
Linear Regression, Logistic Regression, Support Vector Machines, Nonlinearity
and Kernel Methods, Beyond Binary Classification: Multi-class
Unit-II
Unsupervised Learning: Clustering: K-means, Dimensionality Reduction: PCA
and kernel PCA, Generative Models (Gaussian Mixture Models and Hidden
Markov Models)
Unit-III
Evaluating Machine Learning algorithms, Model Selection, Ensemble Methods
(Boosting, Bagging, Random Forests)
Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 2
LEARNING
Unit-IV
Modeling Sequence/Time-Series Data, Deep Learning (Deep generative
models, Deep Boltzmann Machines, Deep auto-encoders, Applications of
Deep Networks) and Feature Representation Learning

Unit-V
Scalable Machine Learning (Online and Distributed Learning) Semi-supervised
Learning, Active Learning, Reinforcement Learning, Inference in Graphical
Models, Introduction to Bayesian Learning and Inference

Department of EECE, GIT 19EEC334A: MACHINE

9 January 2023 3
LEARNING
Text Book(s):
1. Kevin Murphy, Machine Learning: A Probabilistic Perspective, MIT Press,
2012.
2. Trevor Hastie, Robert Tibshirani, Jerome Friedman, The Elements of
Statistical Learning, Springer, 2017.
3. Jiawei Han, Micheline Kamber, Jian Pei , Data Mining: Concepts and
Techniques, 3/e, Morgan Kaufmann, 2016.
4. Christopher Bishop, Pattern Recognition and Machine Learning, Springer,
2016.

Department of EECE, GIT 19EEC334A: MACHINE

9 January 2023 4
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 5
LEARNING
 A(AI) Making machine to think, analyze and make
decision

Department of EECE, GIT 19EEC334A: MACHINE

9 January 2023 6
LEARNING
A(AI) Making machine to think, analyze and make decision

Department of EECE, GIT 19EEC334A: MACHINE

9 January 2023 7
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 8
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 9
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 10
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 11
LEARNING
 Machine Learning (ML)

Department of EECE, GIT 19EEC334A: MACHINE

9 January 2023 12
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 13
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 14
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 15
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 16
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 17
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 18
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 19
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 20
LEARNING
9 January 2023 Department of EECE, GIT 19EEC334A: MACHINE LEARNING 21
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 22
LEARNING
9 January 2023 23
Department of EECE, GIT 19EEC334A: MACHINE LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 24
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 25
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 26
LEARNING
A Typical Supervised Learning Workflow (for Classification)

Department of EECE, GIT 19EEC334A: MACHINE

9 January 2023 27
LEARNING
A Typical Supervised Learning Workflow (for Classification)

Department of EECE, GIT 19EEC334A: MACHINE

9 January 2023 28
LEARNING
A Typical Supervised Learning Workflow (for Classification)

Department of EECE, GIT 19EEC334A: MACHINE

9 January 2023 29
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 30
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 31
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 32
LEARNING
A Typical Unsupervised Learning Workflow (for Clustering)

Department of EECE, GIT 19EEC334A: MACHINE

9 January 2023 33
LEARNING
A Typical Unsupervised Learning Workflow (for Clustering)

Department of EECE, GIT 19EEC334A: MACHINE

9 January 2023 34
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 35
LEARNING
A Typical Reinforcement Learning Workflow

Department of EECE, GIT 19EEC334A: MACHINE

9 January 2023 36
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 37
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 38
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 39
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 40
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 41
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 42
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 43
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 44
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 45
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 46
LEARNING
9 January 2023 Department of EECE, GIT 19EEC334A: MACHINE LEARNING 47
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 48
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 49
LEARNING
9 January 2023 Department of EECE, GIT 19EEC334A: MACHINE LEARNING 50
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 51
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 52
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 53
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 54
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 55
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 56
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 57
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 58
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 59
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 60
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 61
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 62
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 63
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 64
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 65
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 66
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 67
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 68
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 69
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 70
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 71
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 72
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 73
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 74
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 75
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 76
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 77
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 78
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 79
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 80
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 81
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 82
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 83
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 84
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 85
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 86
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 87
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 88
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 89
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 90
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 91
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 92
LEARNING
Linear Regression Line using least square method

On Board:

Solve Example

Goodness of Fit (Performance metric)

R Square value

R2 =
 ( yp − y ) 2

On Board:

MSE
 ( y − y) 2

Solve R square
Give assignment of MSE
MSE =
 ( yp − y ) 2

n
Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 93
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 94
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 95
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 96
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 97
LEARNING
 Y^= Predicted
Y^= Predicted

Y= Actual
Y= Actual

Department of EECE, GIT 19EEC334A: MACHINE

9 January 2023 98
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 99
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 100
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 101
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 102
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 103
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 104
LEARNING
9 January 2023 Department of EECE, GIT 19EEC334A: MACHINE LEARNING 105
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 106
LEARNING
 Y^= Predicted

Y= Actual

Department of EECE, GIT 19EEC334A: MACHINE

9 January 2023 107
LEARNING
 Y^= Predicted

Y= Actual

Department of EECE, GIT 19EEC334A: MACHINE

9 January 2023 108
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 109
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 110
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 111
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 112
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 113
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 114
LEARNING
 Y^= Predicted

Y= Actual

Department of EECE, GIT 19EEC334A: MACHINE

9 January 2023 115
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 116
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 117
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 118
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 119
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 120
LEARNING
Applications of Linear Regression
Some popular applications of linear regression are:

• Analyzing trends and sales estimates

• Salary forecasting
• Real estate prediction
• Arriving at ETAs in traffic.

Department of EECE, GIT 19EEC334A: MACHINE

9 January 2023 121
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 122
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 123
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 124
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 125
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 126
LEARNING
Dividing data set into two subsets:
➢ Training set—a subset to train a model.
➢ Test set—a subset to test the trained model.

Department of EECE, GIT 19EEC334A: MACHINE

9 January 2023 127
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 128
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 129
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 130
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 131
LEARNING
➢ The sigmoid function is a mathematical function used to map the predicted values to probabilities.
➢ It maps any real value into another value within a range of 0 and 1.
➢ The value of the logistic regression must be between 0 and 1, which cannot go beyond this limit, so
it forms a curve like the "S" form. The S-form curve is called the Sigmoid function or the logistic
function.
➢ In logistic regression, we use the concept of the threshold value, which defines the probability of
either 0 or 1. Such as values above the threshold value tends to 1, and a value below the threshold
values tends to 0.

Department of EECE, GIT 19EEC334A: MACHINE

9 January 2023 132
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 133
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 134
LEARNING
 The math behind Logistic regression

Department of EECE, GIT 19EEC334A: MACHINE

9 January 2023 135
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 136
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 137
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 138
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 139
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 140
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 141
LEARNING
Type of Logistic Regression:
On the basis of the categories, Logistic Regression can be classified into three
types:

• Binomial: In binomial Logistic regression, there can be only two possible

types of dependent variables, such as 0 or 1, Pass or Fail, etc.

• Multinomial: In multinomial Logistic regression, there can be three or more

possible unordered types of the dependent variable, such as "cat", "dogs", or
"sheep“

• Ordinal: In ordinal Logistic regression, there can be three or more possible

ordered types of dependent variables, such as "low", "Medium", or "High".

Department of EECE, GIT 19EEC334A: MACHINE

9 January 2023 142
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 143
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 144
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 145
LEARNING
 Y^= Predicted

Y^= Predicted

Y= Actual

Y= Actual

Department of EECE, GIT 19EEC334A: MACHINE

9 January 2023 146
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 147
LEARNING