SNS COLLEGE OF TECHNOLOGY

(AN AUTONOMOUS INSTITUTION)

Approved by AICTE & Affiliated to Anna University
Accredited by NBA & Accrediated by NAAC with ‘A+’ Grade,
Recognized by UGC saravanampatti (post), Coimbatore-641035.

Department of Artificial Intelligence and

Machine learning
Vision Tit 2 Vision Title 3

Course Name: 19AMT301- Deep Learning

III Year : VI Semester

Unit I –INTRODUCTION

Topic : Support Vector Machine

19AMT301/Deep learning/Unit 1/Mr. Karthik G.L./AP/BME

1
 SVM

• SVM is a powerful supervised algorithm that works well with

smaller data set but with complex one
•Used for both regression and classification but preferred as a
Vision Tit 2 Vision Title 3
classifier
•Famous since 1990s and more effective with tuning

19AMT301/Deep learning/Unit 1/Mr. Karthik G.L./AP/BME

2
 How SVM works?

•It is a supervised machine learning problem where we try to

find a hyperplane that best separates the two classes
•Logistic regression Vs SVM-Vision
Both Tit 2are similar without Kernel
Vision Titlebut
3

with Kernel SVM is preferred

•For Small and complex dataset, go-to is SVM

19AMT301/Deep learning/Unit 1/Mr. Karthik G.L./AP/BME

3
 SVM Types

• Linear SVM- Perfectly linearly separable data, can be

classified as 2 classes (2D)
• Non-Linear SVM- NOT linearly
Vision Tit 2 Vision Title 3
separable data, kernel tricks
needed, 2 Classes not possible (2D)

19AMT301/Deep learning/Unit 1/Mr. Karthik G.L./AP/BME

4
SVM graph

Vision Tit 2 Vision Title 3

19AMT301/Deep learning/Unit 1/Mr. Karthik G.L./AP/BME

5
 Important terms in SVM
• Support Vectors-These are the points that are closest to the
hyperplane. A separating line will be defined with the help of these
data points.
Vision Tit 2 Vision Title 3
• Margin-it is the distance between the hyperplane and the
observations closest to the hyperplane (support vectors). In SVM large
margin is considered a good margin. There are two types of
margins hard margin and soft margin.

19AMT301/Deep learning/Unit 1/Mr. Karthik G.L./AP/BME

6
 SVM working with example

•Lets take an example that the dataset has two classes(Blue and
Green). Now we want to classify whether the new dataset is
either Blue or Green Vision Tit 2 Vision Title 3
•2D- Decision boundary or straight line or Separating line
•3D- Hyperplane

19AMT301/Deep learning/Unit 1/Mr. Karthik G.L./AP/BME

7
 Selection of Hyperplane
The best hyperplane is that plane that has the maximum
distance from both the classes, and this is the main aim of SVM.
This is done by finding different hyperplanes which classify the
labels in the best way then it will choose the one which is
farthest from the data points orTitthe
Vision 2 one which has a maximum
Vision Title 3

margin.

19AMT301/Deep learning/Unit 1/Mr. Karthik G.L./AP/BME

8
Optimal Hyperplane

Vision Tit 2 Vision Title 3

19AMT301/Deep learning/Unit 1/Mr. Karthik G.L./AP/BME

9
 Mathematical intuition
behind SVM
Dot Product:
The dot product can be defined as the projection of one vector
along with another, multiply by the product of another vector.
Vision Tit 2 Vision Title 3

A . B = |A| cosθ * |B|

A.B = |A| cosθ * unit vector of B
19AMT301/Deep learning/Unit 1/Mr. Karthik G.L./AP/BME
10
 Use of Dot product in
SVM
Consider a random point X and we want to know whether it lies
on the right side of the plane or the left side of the plane
(positive or negative).
Vision Tit 2 Vision Title 3

19AMT301/Deep learning/Unit 1/Mr. Karthik G.L./AP/BME

11
 Projection of X on W

“C” is distance of vector w from origin to decision boundary

Vision Tit 2 Vision Title 3

19AMT301/Deep learning/Unit 1/Mr. Karthik G.L./AP/BME

12
Dot product of X and W

Vision Tit 2 Vision Title 3

19AMT301/Deep learning/Unit 1/Mr. Karthik G.L./AP/BME

13
 Margin in SVM

• The equation of a hyperplane is w.x+b=0 where w is a vector

normal to hyperplane and b is an offset.

Vision Tit 2 Vision Title 3

19AMT301/Deep learning/Unit 1/Mr. Karthik G.L./AP/BME

14
 Margin in SVM

Decision Rule:

Vision Tit 2 Vision Title 3

19AMT301/Deep learning/Unit 1/Mr. Karthik G.L./AP/BME

15
 Calculate W and d such
that margin has a
maximum distance
To calculate ‘d’ we need the equation of L1 and L2. For this, we
Vision Tit 2 Vision Title 3
will take few assumptions that the equation of L1 is w.x+b=1 and
for L2 it is w.x+b=-1.

19AMT301/Deep learning/Unit 1/Mr. Karthik G.L./AP/BME

16
 Calculate W and d such
that margin has a
maximum distance
Let equation of hyperplane be 2x+y=2
Vision Tit 2 Vision Title 3

19AMT301/Deep learning/Unit 1/Mr. Karthik G.L./AP/BME

17
Margin for the
hyperplane

Vision Tit 2 Vision Title 3

19AMT301/Deep learning/Unit 1/Mr. Karthik G.L./AP/BME

18
 Optimization and its
constraints
• In order to get our optimization function, there are few
constraints to consider. That constraint is that “We’ll calculate
the distance (d) in such a way that no positive or negative
Vision Tit 2 Vision Title 3
point can cross the margin line”. Let’s write these constraints
mathematically:

19AMT301/Deep learning/Unit 1/Mr. Karthik G.L./AP/BME

19
 Condition for correct
classification
• Lets make
• Y=1 for positive class
• Y=-1 for negative class
Vision Tit 2 Vision Title 3
• Condition for correct classification of every point is,

19AMT301/Deep learning/Unit 1/Mr. Karthik G.L./AP/BME

20
Projection of w on x2-x1

Vision Tit 2 Vision Title 3

19AMT301/Deep learning/Unit 1/Mr. Karthik G.L./AP/BME

21
• Since x2 and x1 are support vectors and they lie on the
hyperplane, hence they will follow yi* (2.x+b)=1 so we can
write it as:
Vision Tit 2 Vision Title 3

19AMT301/Deep learning/Unit 1/Mr. Karthik G.L./AP/BME

23
• Putting equations (2) and (3) in equation (1) we get:

Vision Tit 2 Vision Title 3

19AMT301/Deep learning/Unit 1/Mr. Karthik G.L./AP/BME

24
Hence the equation which we have to maximize is:

Vision Tit 2 Vision Title 3

19AMT301/Deep learning/Unit 1/Mr. Karthik G.L./AP/BME

25
 Soft Margin

For linearly non seperable data we modify the equation for Soft
Margin as,

Vision Tit 2 Vision Title 3

19AMT301/Deep learning/Unit 1/Mr. Karthik G.L./AP/BME

26
To make a soft margin equation we add 2 more terms to this
equation which is zeta and multiply that by a hyperparameter ‘c’

Vision Tit 2 Vision Title 3

19AMT301/Deep learning/Unit 1/Mr. Karthik G.L./AP/BME

27
 SVM Error

d1,d2,d3 are zeta value for every misclassification and “0” for
correct classification
SVM Error = Margin Error + Classification Error.
Vision Tit 2 Vision Title 3

19AMT301/Deep learning/Unit 1/Mr. Karthik G.L./AP/BME

28
Which Model is the best model?
Vision Tit 2 Vision Title 3

19AMT301/Deep learning/Unit 1/Mr. Karthik G.L./AP/BME

29
 Kernels

Vision Tit 2 Vision Title 3

19AMT301/Deep learning/Unit 1/Mr. Karthik G.L./AP/BME

30
THANK YOU !!!
Vision Tit 2 Vision Title 3

19ECT303/Artificial Intelligence and Machine learning/Unit 1/

Mr. Karthik G.L./AP/BME 31