Scribd
Upload
8 views3 pages

MLT Content

The document discusses various regression techniques, including Ridge (L2) and Lasso (L1) regression, highlighting their roles in mitigating overfitting and performing feature selection, respectively. It also covers Gaussian Naïve Bayes for probabilistic predictions, Random Forest Classifier as an ensemble method for improved performance, and Support Vector Machine (SVM) for creating optimal decision boundaries in classification tasks. The key differences between Ridge and Lasso are noted, particularly in terms of penalty methods and feature selection capabilities.

Uploaded by

Suba Lakshmi
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
8 views3 pages

MLT Content

The document discusses various regression techniques, including Ridge (L2) and Lasso (L1) regression, highlighting their roles in mitigating overfitting and performing feature selection, respectively. It also covers Gaussian Naïve Bayes for probabilistic predictions, Random Forest Classifier as an ensemble method for improved performance, and Support Vector Machine (SVM) for creating optimal decision boundaries in classification tasks. The key differences between Ridge and Lasso are noted, particularly in terms of penalty methods and feature selection capabilities.

Uploaded by

Suba Lakshmi
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 3

Ridge Regression (L2 Regularization)

• L2 Regularization, also called a ridge regression, adds the


“squared magnitude” of the coefficient as the penalty term to the
loss function.
 In classification, overfitting occurs when the model learns the
noise in the training data rather than the underlying pattern.
Ridge regression helps mitigate overfitting by adding a penalty
for large coefficients, which smoothens the decision boundary
and makes the model less sensitive to noise.
 By controlling the size of the coefficients, Ridge helps to create
a more generalized model that performs better on unseen data,
which is crucial for classification tasks.
 Ridge regression adds a penalty equivalent to the square of the
magnitude of coefficients to the loss function, aiming to shrink
the coefficients of less important features closer to zero but not
exactly zero.
Lasso Regression:(L1 Regularization)
• L1 Regularization, also called a lasso regression, adds the
“absolute value of magnitude” of the coefficient as a penalty
term to the loss function.
 Lasso regression performs feature selection by driving some
coefficients to zero, effectively removing irrelevant or less
important features.
 Lasso regression introduces a penalty equivalent to the absolute
value of the magnitude of coefficients, which can shrink some
coefficients to zero, effectively performing feature selection.
Difference between ridge and lasso regression:
 Penalty: Ridge uses the sum of squared coefficients, while Lasso
uses the sum of absolute values.
 Feature Selection: Lasso can set some coefficients to zero,
effectively selecting features; Ridge does not remove features but
shrinks them.
 Behaviour: Ridge is better for models with many small/medium-
sized effects, while Lasso is more suited when you expect few
important variables.
Gaussian Naïve Bayes:
 Provides probabilistic predictions, which can be useful for
decision-making processes
 Naive Bayes provides probabilistic outputs, offering insights
into the confidence level of predictions.
 In a clinical context, knowing the probability of a heart attack
can aid in decision-making.
 Quick initial assessment of heart attack risk in emergency
settings, rather than binary predictions.
Random Forest Classifier:
 The Random Forest algorithm is an ensemble method used
mainly for classification and regression tasks.
 It builds multiple decision trees using random subsets of
features.
 Ensemble methods combine multiple models to improve overall
performance, reduce overfitting.
 The ensemble methods are particularly effective in complex
datasets, where different models can capture varying aspects of
the data.
 For example, some of the ensemble methods are bagging,
boosting and stacking.
Support Vector Machine:
 Support Vector Machine (SVM) is a powerful supervised
machine learning algorithm widely used for classification,
regression tasks.
 The goal of the SVM algorithm is to create the best line or
decision boundary that can segregate n-dimensional space
into classes so that we can easily put the new data point in the
correct category in the future.
 The best decision boundary is called a hyperplane.
 We should create a hyperplane that has a maximum margin.

You might also like