Scribd
Upload
10 views5 pages

Aml 1

The document discusses key concepts in machine learning, focusing on bias, variance, and their tradeoff, which are crucial for model performance. It covers various regression techniques, including linear regression, ridge, lasso, and elastic net, as well as dimensionality reduction through PCA and classification methods like Naive Bayes and SVM. Additionally, it highlights the importance of regularization to prevent overfitting and improve model accuracy.

Uploaded by

praveencertificates11
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
10 views5 pages

Aml 1

The document discusses key concepts in machine learning, focusing on bias, variance, and their tradeoff, which are crucial for model performance. It covers various regression techniques, including linear regression, ridge, lasso, and elastic net, as well as dimensionality reduction through PCA and classification methods like Naive Bayes and SVM. Additionally, it highlights the importance of regularization to prevent overfitting and improve model accuracy.

Uploaded by

praveencertificates11
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 5

Reading Material for AML

1. Bias in a Machine Learning Model


Definition: Bias refers to the error introduced when a model makes overly
simplistic assumptions about the data. It prevents the model from capturing
the true relationships within the data.
Impact: High bias often leads to underfitting, where the model performs
poorly on both training and test data.
Example: Using a straight line to fit a complex curve.
Key Point: Bias measures how far off the model’s predictions are from the
actual data.

2. High Variance in a Model


Definition: Variance measures the sensitivity of a model to changes in the
training data. High variance often leads to overfitting, where the model
performs well on training data but poorly on new, unseen data.
Impact: Models with high variance are too complex and memorize training
data instead of generalizing.
Key Point: High variance indicates overfitting.

3. High Bias and Low Variance


Scenario: When a model is too simple (high bias) and has low variance, it
fails to learn the underlying patterns in the data.
Outcome: This situation is referred to as underfitting.
Key Point: Balance between bias and variance is necessary for good model
performance.

4. Bias-Variance Tradeoff
Goal: Achieve a balance between bias and variance to minimize total error.
Key Concept: A good model has neither too much bias (underfitting) nor
too much variance (overfitting).

1
5. Reducing Variance
Methods:
• Simplifying the model (e.g., reducing complexity).
• Collecting more training data.
• Using techniques like regularization.
Key Point: Increasing the dataset size can help reduce variance.

6. Simple Linear Regression Formula


Formula: y = mx + c
• y: Target variable.
• x: Predictor variable.
• m: Slope of the line.
• c: Intercept.
Purpose: Predicts the relationship between one independent variable and a
dependent variable.

7. R-Squared Value in Linear Regression


Definition: The R-squared value represents the proportion of variance in
the dependent variable that is explained by the independent variables.
Range: Between 0 and 1. A higher R-squared value indicates a better fit.
Key Point: Measures how well the model fits the data.

8. Assumptions in Linear Regression


Linear regression requires:
• A linear relationship between variables.
• Independence of residuals.
• Homoscedasticity (constant variance of residuals).
Not Required: Predictors don’t need to follow a normal distribution.

2
9. Multicollinearity in Linear Regression
Definition: Multicollinearity occurs when independent variables are highly
correlated with each other.
Impact: It can make the coefficient estimates unstable and unreliable.
Key Point: Address multicollinearity using techniques like ridge regression.

10. Evaluating Regression Accuracy


Metric: Mean Squared Error (MSE).

• Measures the average squared difference between actual and predicted


values.

• Lower MSE indicates better model performance.

11. Ridge Regression


Penalty Term: Adds λ w2 to the loss function, where w are the model’s
P
coefficients.
Use Case: Addresses multicollinearity by shrinking coefficients.
Regularization Type: L2 regularization.

12. Lasso Regression


P
Penalty Term: Adds λ |w| to the loss function.
Key Feature: Performs feature selection by shrinking some coefficients to
zero.
Use Case: Preferred when feature selection is required.

13. Elastic Net


Definition: Combines L1 (lasso) and L2 (ridge) regularization, balancing
feature selection and regularization.
Advantage: Effective when features are highly correlated.

3
14. Principal Component Analysis (PCA)
Purpose: Reduces the number of features (dimensionality reduction) while
retaining the most variance in the data.
Steps:
• Scale the features (essential).

• Calculate the principal components.

• Retain components based on explained variance.

15. Naive Bayes


Based On: Bayes’ theorem.
Assumption: Features are conditionally independent.
Variants:
• Gaussian Naive Bayes: Handles continuous data with normal dis-
tribution.

• Multinomial Naive Bayes: Used for text classification.

16. Decision Tree


Leaf Node: A terminal node where no further splits occur; represents a
prediction.
Gini Impurity: Measures the homogeneity of a dataset. Lower values
indicate purer nodes.
Pruning: Reduces tree depth to prevent overfitting and reduce variance.

17. Support Vector Machines (SVM)


Parameters:
• C: Controls the tradeoff between bias and variance.

• Gamma: Determines the influence of individual data points.


Kernels:
• Linear Kernel: Suitable for linearly separable data.

4
• RBF Kernel: Effective for non-linear data by mapping to higher di-
mensions.

18. Regularization
Purpose: Reduces overfitting by adding a penalty term to the model’s loss
function.
Types:

• L1 (Lasso): Shrinks some coefficients to zero (feature selection).

• L2 (Ridge): Shrinks coefficients without setting them to zero.

You might also like