Scribd
Upload
41 views14 pages

Support Vector Machines Stat 390

Support vector machines (SVM) are a type of supervised machine learning algorithm used for classification and regression analysis. An SVM finds the optimal separating hyperplane in a multidimensional space that distinctly classifies data points of separate categories by maximizing the margin between the categories. Key aspects of SVMs include defining hyperplanes as decision boundaries, using support vectors to maximize the margin between classes, employing kernel functions to handle nonlinear data, and preprocessing data for scaling before classification.

Uploaded by

MaariaJahangir
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
41 views14 pages

Support Vector Machines Stat 390

Support vector machines (SVM) are a type of supervised machine learning algorithm used for classification and regression analysis. An SVM finds the optimal separating hyperplane in a multidimensional space that distinctly classifies data points of separate categories by maximizing the margin between the categories. Key aspects of SVMs include defining hyperplanes as decision boundaries, using support vectors to maximize the margin between classes, employing kernel functions to handle nonlinear data, and preprocessing data for scaling before classification.

Uploaded by

MaariaJahangir
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 14

SUPPORT VECTOR

MACHINES
(SVM)

MAARIA JAHANGIR
STAT 390
WHAT IS SVM?

Possible
• Uses: Regression and Classification hyperplanes

tasks
• Aim: find the hyperplane in an N-
dimensional space that best classifies
data
• N=# of features
• Best hyperplane maximizes distance
between data classes
HYPERPLANES & SUPPORT VECTORS
• Hyperplanes: Decision boundaries
used to classify data
• N=2 -> Hyperplane is a line
• N=3 -> Hyperplane becomes 2-
D plane

• Support Vectors: used to maximize margin


• Influence position & orientation of hyperplane
LOSS FUNCTION

• Hinge-Loss: Error due to misclassification


• Regularization Term (also seen as C)
• with λ as regularization coefficient
KERNEL FUNCTIONS

• Set of algorithms used in pattern analysis


• RBF Kernel good 1st choice if N is small
• Has similar performance to linear, sigmoid kernels for certain parameters
• Less hyperparameters than polynomial kernel
NONLINEAR DATA

• Solution: Map data to higher dimension – computationally intense


• Kernel Trick: Uses dot product to obtain nonlinear classifier
• No data transformations
• Also used with other classifiers, logistic regression
Pros Cons
Kernal
Memory
Parameters
Efficient
Selection

High Non-
Dimensionality Probabilistic

Versatile
DATA PREPROCESSING

• Convert categorical attributes into numeric data

• Scaling – important when attributes have different numeric ranges


• Larger values cause issues with linear, polynomial kernels
MULTI-CLASS SVMS

• OVA Classification: One-versus-all

• One-versus-one classifiers: choose class selected by most classifiers

• Directed Acyclic Graph Support Vector Machines (DAGSVM)


CODING

• Package: e1071
• y2 <- factor(y)
• svm( y2 ~ .,
data=<dataframe>,
kernel=“linear, radial”,
“polynomial”, “sigmoid”,
cost=<#>, scale=< T or F>
• plot( <svm model>,
<dataframe>)
CODING - NONLINEAR

• library(kernlab)
• ksvm( x, y, data=<dataframe>, scaled=<T or F>, type=<see list below>, kernel=<listed below>
• Types: Other types available for regression and novelty detection
• “C-svc” – C classification
• “nu-svc”,”C-bsvc” – nu classification
• Many more types – Hyperbolic Tangent, Laplacian, Bessel, Spline, String, or ANOVA RBF
kernel
• Can also specify “gamma”
REFERENCES

• HTTPS://TOWARDSDATASCIENCE.COM/SUPPORT-VECTOR-MACHINE-INTRODU
CTION-TO-MACHINE-LEARNING-ALGORITHMS-934A444FCA47
• HTTPS://MONKEYLEARN.COM/BLOG/INTRODUCTION-TO-SUPPORT-VECTOR-M
ACHINES-SVM/
• HTTPS://WWW.DATACAMP.COM/COMMUNITY/TUTORIALS/SUPPORT-VECTOR-
MACHINES-R
• HTTP://UC-R.GITHUB.IO/SVM
• HTTPS://NLP.STANFORD.EDU/IR-BOOK/HTML/HTMLEDITION/MULTICLASS-SV
MS-1.HTML
• HTTPS://WWW.CSIE.NTU.EDU.TW/~CJLIN/PAPERS/GUIDE/GUIDE.PDF
• HTTPS://DATA-FLAIR.TRAINING/BLOGS/SVM-KERNEL-FUNCTIONS/

You might also like