Scribd
Upload
66 views11 pages

Support Vactor Machine Final

Support Vector Machine (SVM) is a supervised machine learning algorithm that can be used for both classification and regression problems. The objective of SVM is to find a hyperplane in a multidimensional space that distinctly separates data points into classes. For data with only two input features, the hyperplane is a line, while for three features it is a 2D plane. The optimal hyperplane is the one that represents the largest separation or margin between the two classes. SVM can handle outliers and non-linearly separable data by using soft margins and kernel functions to transform data into higher dimensions where it may become linearly separable. Common kernel functions used in SVM include linear, polynomial, sigmoid, and Gaussian kernels.

Uploaded by

sesixad248
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
66 views11 pages

Support Vactor Machine Final

Support Vector Machine (SVM) is a supervised machine learning algorithm that can be used for both classification and regression problems. The objective of SVM is to find a hyperplane in a multidimensional space that distinctly separates data points into classes. For data with only two input features, the hyperplane is a line, while for three features it is a 2D plane. The optimal hyperplane is the one that represents the largest separation or margin between the two classes. SVM can handle outliers and non-linearly separable data by using soft margins and kernel functions to transform data into higher dimensions where it may become linearly separable. Common kernel functions used in SVM include linear, polynomial, sigmoid, and Gaussian kernels.

Uploaded by

sesixad248
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 11

Support Vector Machine Algorithm

 Support Vector Machine(SVM) is a supervised machine learning


algorithm used for both classification and regression. Though we say
regression problems as well its best suited for classification. The
objective of SVM algorithm is to find a hyperplane in an N-dimensional
space that distinctly classifies the data points. The dimension of the
hyperplane depends upon the number of features. If the number of
input features is two, then the hyperplane is just a line. If the number of
input features is three, then the hyperplane becomes a 2-D plane. It
becomes difficult to imagine when the number of features exceeds
three.

 Let’s consider two independent variables x1, x2 and one dependent


variable which is either a blue circle or a red circle.

 From the figure above its very clear that there are multiple lines (our
hyperplane here is a line because we are considering only two input
features x1, x2) that segregates our data points or does a classification
between red and blue circles. So how do we choose the best line or in
general the best hyperplane that segregates our data points.
 Selecting the best hyper-plane:
 One reasonable choice as the best hyperplane is the one that
represents the largest separation or margin between the two classes.
 So we choose the hyperplane whose distance from it to the nearest
data point on each side is maximized. If such a hyperplane exists it is
known as the maximum-margin hyperplane/hard margin. So from the
above figure, we choose L2.
 Let’s consider a scenario like shown below

 Here we have one blue ball in the boundary of the red ball. So how
does SVM classify the data? It’s simple! The blue ball in the boundary
of red ones is an outlier of blue balls. The SVM algorithm has the
characteristics to ignore the outlier and finds the best hyperplane that
maximizes the margin. SVM is robust to outliers.
 So in this type of data points what SVM does is, it finds maximum
margin as done with previous data sets along with that it adds a
penalty each time a point crosses the margin. So the margins in these
type of cases are called soft margin. When there is a soft margin to the
data set, the SVM tries to minimize (1/margin+∧(∑penalty)). Hinge loss
is a commonly used penalty. If no violations no hinge loss.If violations
hinge loss proportional to the distance of violation.
 Till now, we were talking about linearly separable data(the group of
blue balls and red balls are separable by a straight line/linear line).
What to do if data are not linearly separable?

 Say, our data is like shown in the figure above. SVM solves this by
creating a new variable using a kernel. We call a point x i on the line
and we create a new variable y i as a function of distance from origin
o.so if we plot this we get something like as shown below
 In this case, the new variable y is created as a function of distance
from the origin. A non-linear function that creates a new variable is
referred to as kernel.
SVM Kernel:
The SVM kernel is a function that takes low dimensional input space and
transforms it into higher-dimensional space, ie it converts non separable
problem to separable problem. It is mostly useful in non-linear separation
problems. Simply put the kernel, it does some extremely complex data
transformations then finds out the process to separate the data based on the
labels or outputs defined.
Advantages of SVM:
 Effective in high dimensional cases
 Its memory efficient as it uses a subset of training points in the
decision function called support vectors
 Different kernel functions can be specified for the decision functions
and its possible to specify custom kernels

Major Kernel Functions in Support


Vector Machine (SVM)
 Kernel Function is a method used to take data as input and transform
it into the required form of processing data. “Kernel” is used due to a
set of mathematical functions used in Support Vector Machine
providing the window to manipulate the data. So, Kernel Function
generally transforms the training set of data so that a non-linear
decision surface is able to transform to a linear equation in a higher
number of dimension spaces. Basically, It returns the inner product
between two points in a standard feature dimension.
 Standard Kernel Function Equation :

 Gaussian Kernel: It is used to perform transformation when there


is no prior knowledge about data.

 Gaussian Kernel Radial Basis Function (RBF): Same as above


kernel function, adding radial basis method to improve the
transformation.

Gaussian Kernel Graph


 Sigmoid Kernel: this function is equivalent to a two-layer,
perceptron model of the neural network, which is used as an
activation function for artificial neurons.

 Polynomial Kernel: It represents the similarity of vectors in the


training set of data in a feature space over polynomials of the
original variables used in the kernel.

Polynomial Kernel Graph


Applications of support vector machine:
1. Face observation – It is used for detecting the face according to
the classifier and model.
2. Text and hypertext arrangement – In this, the categorization
technique is used to find important information or you can say
required information for arranging text.
3. Grouping of portrayals – It is also used in the Grouping of
portrayals for grouping or you can say by comparing the piece of
information and take an action accordingly.
4. Bioinformatics – In is also used for medical science as well like in
laboratory, DNA, research, etc.
5. Handwriting remembrance – In this, it is used for handwriting
Recognization.

Linear support vector machine example :

In Linear SVM, the two classes were linearly separable, i.e a single straight
line is able to classify both the classes. But imagine if you have three
classes, obviously they will not be linearly separable. Therefore, Non-linear
SVM’s come handy while handling these kinds of data where classes are not
linearly separable.

You might also like