Classification and

Regression techniques
using SVM
SVM(Support Vector Machines)
• Support Vector Machines (SVM) are a set of supervised learning
methods used for classification, regression, and outlier detection.
• SVMs are known for their effectiveness in high-dimensional spaces
and their ability to handle cases where the number of dimensions
exceeds the number of samples.
• We discuss the theory behind SVMs for both classification and
regression, focusing on three kernel techniques: linear, polynomial,
and radial basis function (RBF).
o Support Vector Machine
are powerful models and
perform well on a variety
of datasets.
SVM technique is easy to
understand and is
generally useful for
unknown distribution data
and
nonregular data.
In this algorithm, each
data item is plotted as a
point in n-dimensional
space, where n is
number of attributes in the
dataset.
Classification using Svm
Classification with SVM aims to find the optimal hyperplane that separates data points of different classes with the
maximum margin.
Key Concepts
1. Hyperplane:
1. In n-dimensional space, a hyperplane is a flat affine subspace of dimension n-1. For a 2D space, it's a line, and for a 3D space, it's
a plane.
2. Support Vectors:
1. Support vectors are the data points closest to the hyperplane. They are critical in defining the position and orientation of the
hyperplane.
3. Margin:
1. The margin is the distance between the hyperplane and the nearest support vectors. SVM aims to maximize this margin, leading
to better generalization.
4. Kernel Trick:
1. Kernels are functions that transform the input data into a higher-dimensional space where a linear separation is possible.
Common kernels include:
1. Linear
2. Polynomial
3. Radial Basis Function (RBF)
3 techniques used for classification

•Linear Kernel:
A linear kernel is used when the data is linearly separable.
•Polynomial Kernel:
A polynomial kernel can create non-linear decision boundaries.
•Radial Basis Function (RBF) Kernel:
An RBF kernel maps the input space into an infinite-dimensional space,
making it suitable for complex non-linear relationships.
 Code for Classification using Svm

import numpy as np
import matplotlib.pyplot as plt
from sklearn import datasets
from sklearn.svm import SVC
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report,
confusion_matrix
from mlxtend.plotting import plot_decision_regions
# Load the Iris dataset
iris = datasets.load_iris()
X = iris.data[:, :2]
# We take the first two features for visualization
purposes
y = iris.target
# Split the data into training and testing sets
X_train, X_test, y_train, y_test =
train_test_split(X, y, test_size=0.3,
random_state=42)
# Create SVM models with different kernels
svm_linear = SVC(kernel='linear', C=1.0)
svm_poly = SVC(kernel='poly', degree=3, C=1.0)
svm_rbf = SVC(kernel='rbf', gamma=0.7, C=1.0
 Train the models on the training data
svm_linear.fit(X_train, y_train)
svm_poly.fit(X_train, y_train)
svm_rbf.fit(X_train, y_train)
# Make predictions on the testing data
y_pred_linear = svm_linear.predict(X_test)
y_pred_poly = svm_poly.predict(X_test)
y_pred_rbf = svm_rbf.predict(X_test)
# Evaluate the models
print("Linear Kernel")
print(classification_report(y_test,
y_pred_linear))
print(confusion_matrix(y_test,
y_pred_linear))
print("Polynomial Kernel")
print(classification_report(y_test,
y_pred_poly))
print(confusion_matrix(y_test,
y_pred_poly))
print("RBF Kernel")
print(classification_report(y_test,
y_pred_rbf))
print(confusion_matrix(y_test, y_pred_rbf))
# Plot the decision boundaries
plt.figure(figsize=(18, 6))
plt.subplot(1, 3, 1)
plot_decision_regions(X, y, clf=svm_linear,
legend=2)
plt.title('SVM with Linear Kernel')
plt.subplot(1, 3, 2)
plot_decision_regions(X, y, clf=svm_poly,
legend=2)
plt.title('SVM with Polynomial Kernel')
plt.subplot(1, 3, 3)
plot_decision_regions(X, y, clf=svm_rbf,
legend=2)
plt.title('SVM with RBF Kernel')
plt.show()
Regression using Svm

Regression with SVM, known as Support Vector Regression (SVR), aims to find a function
that approximates the data within a certain margin of tolerance.
Linear Kernel:
A linear kernel is used when the data can be approximated with a linear function.
Polynomial Kernel:
A polynomial kernel can model more complex relationships by fitting a polynomial to the data.
Radial Basis Function (RBF) Kernel:
An RBF kernel maps the input space into an infinite-dimensional space, making it
suitable for complex non-linear relationships.
 Code for Regression using Svm
# Make predictions on the testing data
y_pred_linear = svr_linear.predict(X_test)
y_pred_poly = svr_poly.predict(X_test)
y_pred_rbf = svr_rbf.predict(X_test)
# Evaluate the models
mse_linear = mean_squared_error(y_test, y_pred_linear)
mse_poly = mean_squared_error(y_test, y_pred_poly)
mse_rbf = mean_squared_error(y_test, y_pred_rbf)
print(f"Linear Kernel Mean Squared Error:
{mse_linear}")
print(f"Polynomial Kernel Mean Squared Error:
{mse_poly}")
print(f"RBF Kernel Mean Squared Error: {mse_rbf}")
# Plot the results
plt.figure(figsize=(18, 6))
plt.subplot(1, 3, 1)
plt.scatter(X, y, color='darkorange', label='data')
plt.plot(X_test, y_pred_linear,
color='cornflowerblue', label='Linear model')
plt.title('SVR with Linear Kernel')
plt.xlabel('X')
plt.ylabel('y')
plt.legend()
plt.subplot(1, 3, 2)
plt.scatter(X, y, color='darkorange',
label='data')
plt.plot(X_test, y_pred_poly,
color='cornflowerblue', label='Polynomial
model')
plt.title('SVR with Polynomial Kernel')
plt.xlabel('X')
plt.ylabel('y')
plt.legend()
plt.subplot(1, 3, 3)
plt.scatter(X, y, color='darkorange',
label='data')
plt.plot(X_test, y_pred_rbf,
color='cornflowerblue', label='RBF model')
plt.title('SVR with RBF Kernel')
plt.xlabel('X')
plt.ylabel('y')
plt.legend()
plt.show()
Applications of SVM Applications of SVM
in classification in regression

• Text Classification • Financial Forecasting

• Effective in High-Dimensional Spaces • Robust to Outliers