Scribd
Upload
36 views26 pages

Support Vector Machines

Support Vector Machines (SVMs) are a supervised machine learning algorithm used for classification and regression analysis. SVMs find the optimal separating hyperplane that maximizes the margin between the different classes of data points. The goal of an SVM is to determine this hyperplane using support vectors and kernels to perform linear or nonlinear classification. SVMs are effective for classification problems and have been successfully applied in domains like bioinformatics, text recognition, and image analysis.

Uploaded by

Zain Ul Haq
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
36 views26 pages

Support Vector Machines

Support Vector Machines (SVMs) are a supervised machine learning algorithm used for classification and regression analysis. SVMs find the optimal separating hyperplane that maximizes the margin between the different classes of data points. The goal of an SVM is to determine this hyperplane using support vectors and kernels to perform linear or nonlinear classification. SVMs are effective for classification problems and have been successfully applied in domains like bioinformatics, text recognition, and image analysis.

Uploaded by

Zain Ul Haq
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 26

Support Vector Machines

Dr Muhammad Atif Tahir


Professor
School of Computer Science
National University of Computing & Emerging Sciences
Karachi Campus
Classification Techniques

 Decision Tree based Methods


 Rule-based Methods
 Memory based reasoning
 Neural Networks
 Naïve Bayes and Bayesian Belief Networks
 Support Vector Machines
Support Vector Machines
 Theoretically well motivated algorithm:
developed from Statistical Learning Theory
(Vapnik & Chervonenkis) since the 60s

 Empirically good performance: successful


applications in many fields (bioinformatics, text,
image recognition, . . . )
Support Vector Machines
 Centralized website: www.kernel-machines.org.
 Several textbooks, e.g. ”An introduction to
Support Vector Machines” by Cristianini and
Shawe-Taylor is one.
 A large and diverse community work on them:
from machine learning, optimization, statistics,
neural networks, functional analysis, etc
Support Vector Machine

 The goal of a support vector machine is to


find the optimal separating hyperplane
which maximizes the margin of the training data
 Support Vector Machine” (SVM) is a supervised
machine learning algorithm which can be used
for both classification or regression challenges
 However, it is mostly used in classification
problems
Support Vector Machines

 Find a linear hyperplane (decision boundary) that will separate the data
Support Vector Machines
B1

 One Possible Solution


Support Vector Machines

B2

 Another possible solution


Support Vector Machines

B2

 Other possible solutions


Support Vector Machines
B1

B2

 Which one is better? B1 or B2?


 How do you define better?
Support Vector Machines
B1

B2

b21
b22

margin
b11

b12

 Find hyperplane maximizes the margin => B1 is better than B2


What is a Hyperplane
An hyperplane is a generalization of a plane

• in one dimension, an hyperplane is called a point

• in two dimensions, it is a line

• in three dimensions, it is a plane

• in more dimensions you can call it an hyperplane


Identify the right hyper-plane (Scenario-1)

Select the hyper-plane which segregates the two classes better”. In this
scenario, hyper-plane “B” has excellently performed this job
Identify the right hyper-plane (Scenario-2)

• A, B and C are all good hyperplanes

• Here, maximizing the distances between nearest data point (either class) and
hyper-plane will help us to decide the right hyper-plane

• This distance is called as Margin


Identify the right hyper-plane (Scenario-2)

• Margin for hyper-plane C is high as compared to both A and B


• Hence, we name the right hyper-plane as C
• Another lightning reason for selecting the hyper-plane with higher
margin is robustness
• If we select a hyper-plane having low margin then there is high
chance of miss-classification
Identify the right hyper-plane (Scenario-3)

• Hyper-plane B may be selected due to higher margin compared to A

• But, here is the catch, SVM selects the hyper-plane which classifies
the classes accurately prior to maximizing margin

• Here, hyper-plane B has a classification error and A has classified all


correctly. Therefore, the right hyper-plane is A
Identify the right hyper-plane (Scenario-4)

• Outliers ignored by SVM


Identify the right hyper-plane (Scenario-5)

• z=x^2+y^2
Support Vector Machines
• The support vectors are indicated
by the circles around them

• Datapoints in this subset are


called “support vectors”

• It will be useful computationally


if only a small fraction of the
data points are support vectors,

• Since, we use the support vectors to decide which


side of the separator a test case is on
Support Vector Machines
 What if the problem is not linearly separable?
Nonlinear Support Vector Machines
 What if decision boundary is not linear?
Nonlinear Support Vector Machines
 Transform data into higher dimensional space
Nonlinear Support Vector Machines
 Kernel Trick
• SVM has a technique called the kernel trick

• These are functions which takes low dimensional input


space and transform it to a higher dimensional space

• i.e. it converts not separable problem to separable


problem, these functions are called kernels

• It is mostly useful in non-linear separation problem

• http://crsouza.com/2010/03/17/kernel-functions-for-
machine-learning-applications/
References

 Introduction to Data Mining by Tan, Steinbach, Kumar


(Lecture Slides)
 https://www.analyticsvidhya.com/blog/2015/10/understaing-
support-vector-machine-example-code/
 http://www.svm-tutorial.com/2014/11/svm-understanding-
math-part-1/
Questions!

You might also like