Machine Learning

Sunbeam Infotech www.sunbeaminfo.com

 -
de! I predict

↓ -
classification
Regression -

I
re I
crage malig,"
y mx + c
=

-> kNN - load data calculate dist

neighbours
-

BuXn +find
y B0+ B,X,+..
+

coes.
intercept
data +
find
-> sunp -
load the

the sightplane
--

*
formula
&
10
gin
--
-2
91
SVRP

sure
T

hyperplane
~

Support Vector Machine

-

itymetric - distance

Sunbeam Infotech www.sunbeaminfo.com

Overview

§ It is a supervised machine learning algorithm that can be used for both classification and regression
-

§ However, it is mostly used in classification problems

-

§ The objective of the support vector machine algorithm is to find a hyperplane in an N-dimensional
m e a n
space(N — the number of features) that distinctly classifies the data points
-

irint"
unknown
data
is -
↳O

in partie

Sunbeam Infotech www.sunbeaminfo.com

How does it work ?

§ SVM separates the classes using hyperplane

-

§ In two dimensional space this hyperplane is a line dividing a plane in two parts where in each class
e n te

lay in either side

-

§ To separate the two classes of data points, there are many possible hyperplanes that could be chosen
- -

§ Our objective is to find a plane that has the maximum margin, i.e the maximum distance between data
-
points of both classes
-

hyperplane

It
hyperplane
-

&⑱
↑

Sunbeam Infotech www.sunbeaminfo.com

Hyperplane

Sunbeam Infotech www.sunbeaminfo.com

Scenario 1 :no or minimum missclassification

§ Here, we have three hyper-planes (A, B and C)

-

§ Now, identify the right hyper-plane to classify star and circle

-

§ You need to remember a thumb rule to identify the right hyper-plane

e
missclassification)
§ Select the hyper-plane which segregates the two classes better
-
(withouthaving any

§ In this scenario, hyper-plane “B” has excellently performed this job

-

#
!-

som

o
Sunbeam Infotech www.sunbeaminfo.com
Scenario 2 ·maximum margin

§ Here, we have three hyper-planes (A, B and C) and all are segregating the classes well
-

§ Now, How can we identify the right hyper-plane?

e

§ Here, maximizing
-
the distances between nearest data point (either class) and hyper-plane will help us
to decide the right hyper-plane. This distance is called as Margin.
- -
-

⑰narsie
Sunbeam Infotech www.sunbeaminfo.com
Scenario 3

§ Use the rules as discussed in previous section to identify the right hyper-plane
- -

§ Some of you may have selected the hyper-plane B as it has higher margin compared to A.
-
§ But, SVM selects the hyper-plane which classifies the classes accurately prior to maximizing margin
-

§ Here, hyper-plane B has a classification error and A has classified all correctly
-

§ Therefore, the right hyper-plane is A.

-

**
Sunbeam Infotech www.sunbeaminfo.com
Scenario 4

§ Unable
-
to segregate the two classes using a straight
-
line, as one of star lies in the territory of
other(circle) class as an outlier
-

§ SVM has a feature to ignore outliers and find the hyper-plane that has maximum margin
-

§ Hence, we can say, SVM is robust to outliers.

-

↑ar A 0

Sunbeam Infotech www.sunbeaminfo.com

Scenario 5 el -> K-SUMp
-
-

§ In the scenario below, we can’t have linear

- -
§ SVM solves this problem by introducing
-
hyper-plane between the two classes
-
2n
z=x^2+y^2
-

4 4

⑮
⑰

⑧
-

Sunbeam Infotech www.sunbeaminfo.com

Scenario 5

§ Points to consider are:

-

§ All values for z would be positive always because z is the squared sum of both x and y
-

§ In the original plot, red circles appear close to the origin of x and y axes, leading to lower value of z and star
-
-

relatively away from the origin result to higher value of z.

-

§ In SVM, it is easy to have a linear hyper-plane between these two classes, but for such scenarios,
--
SVM uses a trick called as Kernel.
-

§ These
-
are functions which takes low dimensional input
-
space and transform it to a higher dimensional
space, i.e. it converts non separable problem to separable problem
-

§-
It is mostly useful in non-linear separation problem
§-
Simply put, it does some extremely complex data transformations,
then find out the process to separate the data based on the
-

labels or outputs you’ve defined.

-

Sunbeam Infotech www.sunbeaminfo.com

Tuning Parameters - Kernels
--

§ The learning of the hyperplane in linear SVM is done by transforming the problem using some linear
-
algebra. This is where the kernel plays role. -

Sunbeam Infotech www.sunbeaminfo.com

Tuning Parameters - Regularization
--

§ The Regularization parameter tells the SVM optimization how much you want to avoid misclassifying
-entere

-
each training example
§ For large values of C, the optimization will choose a smaller-margin hyperplane if that hyperplane
- e n e r

does a better job of getting all the training points classified correctly
-

§ Conversely, a very small value of C will cause the optimizer to look for a larger-margin separating
-

-
hyperplane, even if that hyperplane misclassifies more points

Low regularization High regularization

% -
-

Sunbeam Infotech www.sunbeaminfo.com

Tuning Parameters - Gamma -
-

§ The gamma parameter defines how far the influence of a single training example reaches, with low
-
e
values meaning ‘far’ and high values meaning ‘close’
-

§ In other words
§ With low gamma, points far away from plausible separation line are considered in calculation for the
separation line
§ Where as high gamma means the points close to plausible line are considered in calculation.

&
0
-
-
-

-
·
2

Sunbeam Infotech www.sunbeaminfo.com

Tuning Parameters - Margin -
=-

§ A margin is a separation of line to the closest class points

-

§ A good margin is one where this separation is larger for both the classes
-

/ -
-

Sunbeam Infotech www.sunbeaminfo.com