Scribd
Upload
24 views16 pages

Support Vector Machines 1639601280

Uploaded by

9m8cr5k72j
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
24 views16 pages

Support Vector Machines 1639601280

Uploaded by

9m8cr5k72j
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 16

Support vector machines (SVM)

Week 12

Middlesex University Dubai;


CST4050; Instructor: Dr. Ivan Reznikov
SVM concept
Support vector machine (SVM) is a
simple algorithm that produces
significant accuracy with relatively low
computation power. SVM can be used
for regression problems, but more often
used to solve classification tasks.

For the purpose of today’s session,


we’ll slightly modify our usual dataset.
Let’s include a level column, that will
be 1 for Overall Rating >= 70, else 0.
2
(a) 1Dimensional Data
What classifier would you pick?

3
(b) 1Dimensional Data
What classifier would you pick now?

4
(c) 1Dimensional Data
What classifier would you pick this time?

5
Soft margins
For 1D we only need a
point to set threshold.
It is considered as 0-
dimensional hyperplane

Soft margins
are more flexible and
allow misclassification.

The thresh may be


found empirically using
cross-validation
6
2Dimensional Data
What classifier would you pick?

We can look for Support


Vectors – border points
of certain class.

After, we look for the


maximum margin size

7
2Dimensional Data
For 2Dimensional Data
we need a line to set
threshold. It is
considered as 1-
dimensional hyperplane.

Similar situation we
can imagine for higher
dimensions.

8
SVM: Linear Kernel
What classifier would you pick?

9
SVM: Polynomial Kernel
We can plot our data in
the following coordinates:
X = x
Y = x2

Now we can look for a


linear support vector
classifier

The algorithm of
transforming data and
finding SVC is called
Support Vector Machines 10
SVM: Polynomial Kernel
In order to classify a new
data point, we’ll need to
classify it’s x-x2 position.

We could’ve used x3, xd or


other formula. These are
called kernel functions

Also, rather than


transforming data we can
calculate pair relationships
between data points.
This is known as kernel trick
11
SVM: Types of Kernels
Linear kernel

Sigmoid kernel

Radial kernel

Polynomial kernel

12
SVM: Multi-Class
Besides binary
classification (2
classes) SVM can
handle tasks
classifying more
than 2 classes
(multi-class).

Strategies involved:
● 1 vs all
● 1 vs 1
13
SVM: Multi-Class: 1 vs all
Creating training sets

With the updated data


we now calculate prob of
each class. The class
with highest score wins. 14
SVM: Multi-Class: 1 vs 1
Green vs Purple
Winner: Green

Red vs Purple:
Winner: Red Compared to 1 vs all, 1
vs 1 compares whether
the data point should
be classified as either
Green vs Red: class. The data point is
Winner: Red assigned to class with
most “wins”. 15
Result: Red
Support Vector Machines

●
Works well on datasets with many ●
Require high training time, so not
features recommended for large datasets.
●
Provides a clear separation margin ●
Very sensitive to outliers.
●
Effective for datasets where the
number of features are greater
than the number of data points
●
Possible to specify different kernel
functions to make a proper
decision boundary 16

You might also like