Scribd
Upload
193 views22 pages

21 SVR

Uploaded by

Sahith Krishna 21BDS0078
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
193 views22 pages

21 SVR

Uploaded by

Sahith Krishna 21BDS0078
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 22

Support Vector

Regression
Introduction to Support Vector
Regression (SVR)
• Support Vector Regression (SVR) is an extension of the Support Vector Machine
(SVM) algorithm, primarily used for regression tasks.
• Like SVM, it attempts to find the best hyperplane that separates data points, but in
SVR, the goal is to predict a continuous value rather than classifying data.
• SVR works by mapping input data into high-dimensional space and then finding
the hyperplane that best fits the data within a specified margin of tolerance ϵ.
• The core idea is to maintain a balance between fitting the data and ensuring that
the model remains generalized, avoiding overfitting.
• SVR is particularly useful when you need accurate regression predictions while
managing model complexity and avoiding overfitting.
Support Vector Regression (SVR)
• Objective: SVR aims to find a function that predicts a continuous target variable
while maximizing the margin between the predicted values and the actual data
points.
• Margin: SVR identifies a “margin” around the predicted regression line, and its
goal is to fit the line within this margin while minimizing the prediction error.
• Support Vectors: In SVR, data points that are closest to the regression line and
define the margin are known as “support vectors.” These points play a crucial
role in determining the regression model.
• Kernel Trick: SVR can use various kernel functions (e.g., linear, polynomial,
radial basis function) to transform the feature space, making it possible to
model non-linear relationships between input features and the target variable.
Support Vector Regression (SVR)
• Hyperparameters: SVR requires tuning hyperparameters, such as the regularization
parameter (c ) and kernel parameters, to achieve the best model performance.
• Loss Function: SVR typically uses an epsilon-insensitive loss function that allows for
some errors within a defined range (epsilon), and it penalizes errors outside this
range more heavily.
• Complexity Control: The regularization parameter (C ) in SVR controls the trade-off
between maximizing the margin and minimizing the prediction error. A smaller C
leads to a wider margin with more errors allowed, while a larger C results in a
narrower margin with fewer errors allowed.
• Robustness: SVR is robust to outliers, as it primarily focuses on the data points
close to the margin (support vectors) and doesn’t heavily rely on all data points.
Support Vector Regression (SVR)
• Approach of defining acceptable margins, slacks, and a
regularizing function that combines smoothness and
error is adopted from regression.
• Given f (x) = wT x + w0
• We know e2 (rt ,f(xt )) = [rt − f (xt)]2
• In support vector regression, we use the ε-sensitive loss
function:
Support Vector Regression (SVR)
• Analogous to the soft margin hyperplane, we introduce
slack variables to account for deviations out of the ε-zone
and we get (Vapnik 1995)

• where we use two types of slack variables, for positive and


negative deviations, to keep them positive
Advantages of SVR
1.Effective with High-Dimensional Data: SVR can handle
high-dimensional feature spaces and works well when there are
many input variables.
2.Robust to Outliers: SVR focuses on finding a margin for most
data points and ignores small variations, making it robust to
noise and outliers.
3.Flexibility in Choosing Kernels: SVR supports different
kernel functions (linear, polynomial, RBF, etc.), which allows it to
model complex relationships between input and output.
4.Good Generalization: SVR provides good generalization
performance as it controls model complexity through
regularization (parameter C).
Disadvantages of SVR:

1.Complexity and Computation Time: SVR can be


computationally expensive, especially with large datasets, due to
its dependence on quadratic optimization.
2.Sensitive to Parameter Tuning: Performance is highly
dependent on choosing the right kernel and tuning parameters
such as C, ϵ, and kernel-specific parameters.
3.Scaling Required: Features need to be scaled before applying
SVR as it is sensitive to the relative magnitude of different features.
4.Not Well-Suited for Large Datasets: SVR may struggle with
very large datasets, as it can be slow and memory-intensive.

You might also like