Scribd
Upload
19 views15 pages

Detailed SVM Presentation

Support Vector Machine (SVM) is a supervised machine learning algorithm used for classification and regression by finding the optimal hyperplane that separates data into classes. It employs concepts like hard and soft margins, kernel tricks for non-linear data, and requires hyperparameter tuning for optimal performance. SVM is effective in high-dimensional spaces and has applications in various fields such as image classification, text categorization, and bioinformatics.

Uploaded by

harshitajain8755
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
19 views15 pages

Detailed SVM Presentation

Support Vector Machine (SVM) is a supervised machine learning algorithm used for classification and regression by finding the optimal hyperplane that separates data into classes. It employs concepts like hard and soft margins, kernel tricks for non-linear data, and requires hyperparameter tuning for optimal performance. SVM is effective in high-dimensional spaces and has applications in various fields such as image classification, text categorization, and bioinformatics.

Uploaded by

harshitajain8755
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 15

Support Vector Machine (SVM) in

Machine Learning
A Comprehensive Guide with
Formulas and Examples
Your Name | Date
Introduction to SVM
• Support Vector Machine (SVM) is a powerful
supervised machine learning algorithm used
for classification and regression tasks.
• The key idea is to find a decision boundary
(hyperplane) that best separates data into
different classes.
SVM Intuition
• Imagine plotting your data in an n-dimensional
space (where n is the number of features).
• SVM finds the hyperplane that has the
maximum margin, i.e., the largest distance to
the nearest data point of any class.
Mathematical Formulation
• Hyperplane Equation: w · x + b = 0
• For binary classification:
• - Class +1: w · x + b ≥ 1
• - Class -1: w · x + b ≤ -1
• Optimization Goal: Minimize ½ ||w||² subject
to the above constraints.
Understanding the Margin
• The margin is the distance between the
hyperplane and the closest data points
(support vectors).
• Formula: Margin = 2 / ||w||
• Support vectors are the points that lie closest
to the decision boundary.
Hard Margin vs Soft Margin
• Hard Margin: Assumes data is perfectly
separable.
• Soft Margin: Introduces slack variables ξᵢ to
allow for misclassification.

• Soft Margin Optimization:


• Minimize: ½ ||w||² + C Σ ξᵢ
• Subject to: yᵢ(w · xᵢ + b) ≥ 1 - ξᵢ, ξᵢ ≥ 0
Kernel Trick
• To handle non-linear data, SVM uses kernels
to map input data into higher dimensions.
• Common Kernels:
• - Polynomial: K(x, x') = (x · x' + 1)^d
• - RBF (Gaussian): K(x, x') = exp(-||x - x'||² /
(2σ²))
• - Sigmoid: K(x, x') = tanh(κx · x' + θ)
Numerical Example: Linear SVM
• Dataset:
• (2,2) → 1, (4,4) → 1, (4,0) → -1, (0,0) → -1
• Assume w = [1,1], b = -3
• For x = [2,2], f(x) = sign(1×2 + 1×2 - 3) = sign(1)
= +1 (Correct)
• For x = [0,0], f(x) = sign(0 + 0 - 3) = sign(-3) = -1
(Correct)
Numerical Example: RBF Kernel
• Points:
• A = (1,2), B = (2,3), C = (8,8), σ = 1
• K(A, C) = exp(-((1-8)² + (2-8)²)/2) = exp(-49/2) ≈
1.5e-11 (Almost zero)
• This means A and C are very far apart in the
transformed space.
SVM Optimization (Dual Form)
• Lagrangian Dual Problem:
• Maximize: Σ αᵢ - ½ ΣΣ αᵢαⱼyᵢyⱼ(xᵢ · xⱼ)
• Subject to: Σ αᵢyᵢ = 0 and 0 ≤ αᵢ ≤ C
• Support vectors are the samples with non-zero
αᵢ.
Hyperparameter Tuning
• Important parameters:
• - C (regularization): Controls trade-off
between margin size and misclassification
• - Kernel type and parameters (e.g., σ for RBF)
• - Use grid search or cross-validation to select
best parameters
Pros and Cons
• Pros:
• - Effective in high-dimensional spaces
• - Works well for small to medium datasets
• - Robust to overfitting with proper C and
kernel

• Cons:
• - Not suitable for very large datasets
• - Kernel selection can be tricky
Applications of SVM
• SVM is used in various domains such as:
• - Image classification (e.g., facial recognition)
• - Text categorization (e.g., spam detection)
• - Bioinformatics (e.g., gene classification)
• - Handwriting recognition (e.g., MNIST
dataset)
Conclusion
• SVM is a versatile and powerful algorithm for
classification and regression.
• With the right kernel and parameters, it can
handle both linear and non-linear problems
effectively.
References
• - Bishop, C. M., Pattern Recognition and
Machine Learning
• - scikit-learn.org SVM documentation
• - Stanford CS229 Lecture Notes
• - Coursera ML Courses by Andrew Ng

You might also like