Scribd
Upload
17 views18 pages

Support Vector Machine

The document provides an overview of Support Vector Machine (SVM), a popular supervised learning algorithm used primarily for classification tasks. It explains key concepts such as hyperplanes, support vectors, and the distinction between linear and non-linear SVMs, along with their respective applications. Additionally, it covers important terminologies like margin, kernel trick, and decision functions, illustrating how SVM works with examples.

Uploaded by

Shehzad Aman
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
17 views18 pages

Support Vector Machine

The document provides an overview of Support Vector Machine (SVM), a popular supervised learning algorithm used primarily for classification tasks. It explains key concepts such as hyperplanes, support vectors, and the distinction between linear and non-linear SVMs, along with their respective applications. Additionally, it covers important terminologies like margin, kernel trick, and decision functions, illustrating how SVM works with examples.

Uploaded by

Shehzad Aman
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 18

Support vector

Machine
INSTRUCTOR: MUHAMMAD HASAAN MUJTABA
GMAIL: [email protected]
Support Vector Machine Algorithm

► Support Vector Machine or SVM is one of the most popular


Supervised Learning algorithms, which is used for Classification as
well as Regression problems. However, primarily, it is used for
Classification problems in Machine Learning.

► The goal of the SVM algorithm is to create the best line or decision
boundary that can segregate n-dimensional space into classes so
that we can easily put the new data point in the correct category in
the future. This best decision boundary is called a hyperplane.
Support Vector Machine Algorithm

► SVM chooses the extreme points/vectors that help in creating the


hyperplane. These extreme cases are called as support vectors, and
hence algorithm is termed as Support Vector Machine. Consider the
below diagram in which there are two different categories that are
classified using a decision boundary or hyperplane:
Support Vector Machine Algorithm
Example

► SVM can be understood with the example that we have used in the
KNN classifier. Suppose we see a strange cat that also has some
features of dogs, so if we want a model that can accurately identify
whether it is a cat or dog, so such a model can be created by using
the SVM algorithm. We will first train our model with lots of images of
cats and dogs so that it can learn about different features of cats
and dogs, and then we test it with this strange creature. So as
support vector creates a decision boundary between these two
data (cat and dog) and choose extreme cases (support vectors), it
will see the extreme case of cat and dog. On the basis of the
support vectors, it will classify it as a cat. Consider the below
diagram:
Example
Types

► Linear SVM: Linear SVM is used for linearly separable data, which
means if a dataset can be classified into two classes by using a
single straight line, then such data is termed as linearly separable
data, and classifier is used called as Linear SVM classifier.
► Non-linear SVM: Non-Linear SVM is used for non-linearly separated
data, which means if a dataset cannot be classified by using a
straight line, then such data is termed as non-linear data and
classifier used is called as Non-linear SVM classifier.
Terminologies

► Support Vectors: These are the data points that are closest to the decision
boundary. They are critical in defining the position and orientation of the
hyperplane and influence the margin of the classifier.
► Hyperplane: A hyperplane is a decision boundary that separates different
classes in the feature space. In a two-dimensional space, it’s a line; in
higher dimensions, it's a plane or manifold.
► Margin: The margin is the distance between the hyperplane and the
closest data points (support vectors) from either class. SVM aims to
maximize this margin to improve generalization.
► Kernel Trick: The kernel trick is a technique used in SVM to transform data
into a higher-dimensional space where it may become more easily
separable by a hyperplane. Common kernels include linear, polynomial,
radial basis function (RBF), and sigmoid.
Terminologies

► Soft Margin: A soft margin SVM allows some misclassification of data


points (i.e., some points can be within the margin or on the wrong
side of the hyperplane) to achieve a better generalization. It
introduces a penalty for misclassification, controlled by the
regularization parameter 𝐶C.
► Hard Margin: A hard margin SVM does not allow any data points
within the margin or on the wrong side of the hyperplane. This is less
flexible and is only suitable for linearly separable data.
► Decision Function: The decision function is the output of the SVM
that determines which side of the hyperplane a new data point falls
on, thus assigning it to a particular class.
How does SVM works?

► Linear SVM:
► The working of the SVM algorithm can be understood by using an
example. Suppose we have a dataset that has two tags (green and
blue), and the dataset has two features x1 and x2. We want a
classifier that can classify the pair(x1, x2) of coordinates in either
green or blue. Consider the below image:
Linear SVM

► So as it is 2-d space so by just using a straight line, we can easily


separate these two classes. But there can be multiple lines that
can separate these classes. Consider the 2 image:
Svm

► Hence, the SVM algorithm helps to find the best line or decision
boundary; this best boundary or region is called as a hyperplane.
SVM algorithm finds the closest point of the lines from both the
classes. These points are called support vectors. The distance
between the vectors and the hyperplane is called as margin. And
the goal of SVM is to maximize this margin. The hyperplane with
maximum margin is called the optimal hyperplane.
SVM
Non linear

► If data is linearly arranged, then we can separate it by using a


straight line, but for non-linear data, we cannot draw a single
straight line. Consider the below image:
Non linear

► So to separate these data points, we need


to add one more dimension. For linear data,
we have used two dimensions x and y, so
for non-linear data, we will add a third
dimension z. It can be calculated as:
► z=x2 +y2
► By adding the third dimension, the sample
space will become as below image:
Non linear

► So now, SVM will divide the datasets into classes in the following way. Consider the
below image:
Non linear

► Since we are in 3-d Space, hence it is looking like a plane parallel to the x-axis. If
we convert it in 2d space with z=1, then it will become as:

Hence we get a circumference of radius 1 in case of non-linear data.


Any Question

You might also like