Subject: Machine Learning

Module I: Supervised Learning

Mr. K.V.Satya Kumar
Assistant Professor
Department of EECE
GITAM Institute of Technology (GIT)
Visakhapatnam – 530045
Email: [email protected]
9 January 2023 Department of EECE, GIT 19EEC334A: MACHINE LEARNING 1
Unit-I
Supervised Learning (Regression/Classification): Basic methods: Distance-based
methods, Nearest-Neighbors, Decision Trees, Naive Bayes, Linear models:
Linear Regression, Logistic Regression, Support Vector Machines, Nonlinearity
and Kernel Methods, Beyond Binary Classification: Multi-class
Unit-II
Unsupervised Learning: Clustering: K-means, Dimensionality Reduction: PCA
and kernel PCA, Generative Models (Gaussian Mixture Models and Hidden
Markov Models)
Unit-III
Evaluating Machine Learning algorithms, Model Selection, Ensemble Methods
(Boosting, Bagging, Random Forests)
Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 2
LEARNING
Unit-IV
Modeling Sequence/Time-Series Data, Deep Learning (Deep generative
models, Deep Boltzmann Machines, Deep auto-encoders, Applications of
Deep Networks) and Feature Representation Learning

Unit-V
Scalable Machine Learning (Online and Distributed Learning) Semi-supervised
Learning, Active Learning, Reinforcement Learning, Inference in Graphical
Models, Introduction to Bayesian Learning and Inference

Department of EECE, GIT 19EEC334A: MACHINE

9 January 2023 3
LEARNING
Text Book(s):
1. Kevin Murphy, Machine Learning: A Probabilistic Perspective, MIT Press,
2012.
2. Trevor Hastie, Robert Tibshirani, Jerome Friedman, The Elements of
Statistical Learning, Springer, 2017.
3. Jiawei Han, Micheline Kamber, Jian Pei , Data Mining: Concepts and
Techniques, 3/e, Morgan Kaufmann, 2016.
4. Christopher Bishop, Pattern Recognition and Machine Learning, Springer,
2016.

Department of EECE, GIT 19EEC334A: MACHINE

9 January 2023 4
LEARNING
9 January 2023 Department of EECE, GIT 19EEC334A: MACHINE LEARNING 5
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 6
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 7
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 8
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 9
LEARNING
9 January 2023 Department of EECE, GIT 19EEC334A: MACHINE LEARNING 10
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 11
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 12
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 13
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 14
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 15
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 16
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 17
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 18
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 19
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 20
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 21
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 22
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 23
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 24
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 25
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 26
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 27
LEARNING
λ is the regularization parameter
What it does: It decides how much weight
we must give to the miss classification maximum margin, minimum miss
classification
Manually we should assign some value for λ
to minimize miss classification
Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 28
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 29
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 30
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 31
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 32
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 33
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 34
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 35
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 36
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 37
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 38
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 39
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 40
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 41
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 42
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 43
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 44
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 45
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 46
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 47
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 48
LEARNING
Kernel Method
Kernel Method
Kernel Functions
Kernel Functions
Kernel Functions
Kernel Functions
Kernel Functions
Multiclass
• In machine learning, multiclass classification is the problem of
classifying instances into one of three or more classes.

• Classifying instances into one of the two classes is called binary

classification.

• Support vector machines can be constructed only when the

dataset has only two class labels and is linearly separable.

• Two methods are generally used to handle such cases known by

the names ”One-against-all" and “one-against-one”.
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 57
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 58
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 59
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 60
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 61
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 62
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 63
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 64
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 65
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 66
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 67
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 68
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 69
LEARNING
“One-against-all”
“One-against-all”
“One-against-one”
“One-against-one”
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 74
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 75
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 76
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 77
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 78
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 79
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 80
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 81
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 82
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 83
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 84
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 85
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 86
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 87
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 88
LEARNING
 K-Nearest Neighbors

Department of EECE, GIT 19EEC334A: MACHINE

9 January 2023 89
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 90
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 91
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 92
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 93
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 94
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 95
LEARNING
 n= 9, √n = √9 = 3 K = 3

Department of EECE, GIT 19EEC334A: MACHINE

9 January 2023 96
LEARNING
 n= 9 √n = √9 = 3 K = 3

Department of EECE, GIT 19EEC334A: MACHINE

9 January 2023 97
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 98
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 99
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 100
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 101
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 102
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 103
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 104
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 105
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 106
LEARNING
Bayes theorem

Department of EECE, GIT 19EEC334A: MACHINE

9 January 2023 107
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 108
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 109
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 110
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 111
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 112
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 113
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 114
LEARNING
 Humans ? Or Mammals ?

Department of EECE, GIT 19EEC334A: MACHINE

9 January 2023 115
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 116
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 117
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 118
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 119
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 120
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 121
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 122
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 123
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 124
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 125
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 126
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 127
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 128
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 129
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 130
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 131
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 132
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 133
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 134
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 135
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 136
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 137
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 138
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 139
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 140
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 141
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 142
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 143
LEARNING
9 January 2023 Department of EECE, GIT 19EEC334A: MACHINE LEARNING 144
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 145
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 146
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 147
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 148
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 149
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 150
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 151
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 152
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 153
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 154
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 155
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 156
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 157
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 158
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 159
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 160
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 161
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 162
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 163
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 164
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 165
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 166
LEARNING
 Department of EECE, GIT 19EEC334A: MACHINE
9 January 2023 167
LEARNING