Scribd
Upload
19 views9 pages

Support Vector Machine

A Support Vector Machine (SVM) is a machine learning algorithm used for classification and regression that identifies the optimal hyperplane to separate data into groups while maximizing the margin between them. SVM can handle both linear and non-linear data using kernel tricks, making it effective in high-dimensional spaces and robust against overfitting. However, it can be computationally expensive, difficult to tune, and less effective with noisy data.

Uploaded by

Antonio A. Dizon
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
19 views9 pages

Support Vector Machine

A Support Vector Machine (SVM) is a machine learning algorithm used for classification and regression that identifies the optimal hyperplane to separate data into groups while maximizing the margin between them. SVM can handle both linear and non-linear data using kernel tricks, making it effective in high-dimensional spaces and robust against overfitting. However, it can be computationally expensive, difficult to tune, and less effective with noisy data.

Uploaded by

Antonio A. Dizon
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 9

What is a Support Vector Machine(SVM)?

A Support Vector Machine (SVM) is a machine learning algorithm used for classification and
regression. It finds the best line (or hyperplane) to separate data into groups, maximizing the
distance between the closest points (support vectors) of each group. It can handle complex data
using kernels to transform it into higher dimensions.

Types of Support Vector Machine (SVM) Algorithms


• Linear SVM: When the data is perfectly linearly separable only then we can use Linear
SVM. Perfectly linearly separable means that the data points can be classified into 2
classes by using a single straight line(if 2D).
• Non-Linear SVM: When the data is not linearly separable then we can use Non-Linear
SVM, which means when the data points cannot be separated into 2 classes by using a
straight line (if 2D) then we use some advanced techniques like kernel tricks to classify
them. In most real-world applications we do not find linearly separable datapoints hence
we use kernel trick to solve them.

Support Vector Machine (SVM)


1. Maximizes the Margin:
SVM focuses on finding the decision boundary that maximizes the margin (the distance
between the boundary and the closest data points of each class). This makes it more
robust to new data.
2. Handles Non-Linear Data:
SVM can handle non-linear data using the “kernel trick,” which transforms the data into a
higher-dimensional space where it becomes easier to separate.
3. Effective in High Dimensions:
SVM works well even when the number of features (dimensions) is much larger than the
number of samples, making it suitable for complex datasets.
4. Robust to Overfitting:
By focusing on the points closest to the boundary (support vectors), SVM is less likely to
overfit, especially in smaller datasets.
5. Requires Tuning:
SVM requires careful tuning of parameters (like the choice of kernel and regularization) to
achieve optimal performance, which can be time-consuming.

How Does Support Vector Machine Algorithm Work?


SVM is defined such that it is defined in terms of the support vectors only, we don’t have to worry
about other observations since the margin is made using the points which are closest to the
hyperplane (support vectors), whereas in logistic regression the classifier is defined over all the
points. Hence SVM enjoys some natural speed-ups.
Let’s understand the working of SVM using an example. Suppose we have a dataset that has two
classes (green and blue).
To classify these points, we can have many decision boundaries.

The best hyperplane is that plane that has the maximum distance from both the classes, and this
is the main aim of SVM. This is done by finding different hyperplanes which classify the labels in
the best way then it will choose the one which is farthest from the data points or the one which
has a maximum margin.

Advantages of Support Vector Machine


1. Works well with complex data: SVM is great for datasets where the separation between
categories is not clear. It can handle both linear and non-linear data effectively.
2. Effective in high-dimensional spaces: SVM performs well even when there are more
features (dimensions) than samples, making it useful for tasks like text classification or
image recognition.
3. Avoids overfitting: SVM focuses on finding the best decision boundary (margin) between
classes, which helps in reducing the risk of overfitting, especially in high-dimensional data.
4. Versatile with kernels: By using different kernel functions (like linear, polynomial, or radial
basis function), SVM can adapt to various types of data and solve complex problems.
5. Robust to outliers: SVM is less affected by outliers because it focuses on the support
vectors (data points closest to the margin), which helps in creating a more generalized
model.
Disadvantages of Support Vector Machine
1. Slow with large datasets: SVM can be computationally expensive and slow to train,
especially when the dataset is very large.
2. Difficult to tune: Choosing the right kernel and parameters (like C and gamma) can be
tricky and often requires a lot of trial and error.
3. Not suitable for noisy data: If the dataset has too many overlapping classes or noise, SVM
may struggle to perform well because it tries to find a perfect separation.
4. Hard to interpret: Unlike some other algorithms, SVM models are not easy to interpret or
explain, especially when using non-linear kernels.
5. Memory-intensive: SVM requires storing the support vectors, which can take up a lot of
memory, making it less efficient for very large datasets.

Example 1

Must convert it into numerical format.


• Sex: M → 1, F → -1
• Course: CS 3 → 1, CS 2 → -1
• Subject: Data Mining → 1, Programming → -1
• Remarks (Target Variable): Passed → 1, Failed → -1
The objective of SVM is to find a hyperplane:

where w (weight vector) and b(bias) define the decision boundary.


SVM Solve

subject to:

where yi are labels (1 for passed, -1 for failed).

Find Support Vectors


From the dataset, possible support vectors are:

For the python code:


Example 2:

You might also like