Scribd
Upload
5 views6 pages

Regression Bayesian SVM Notes

Uploaded by

avantika27271
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
5 views6 pages

Regression Bayesian SVM Notes

Uploaded by

avantika27271
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 6

1.

REGRESSION

Linear Regression:

- Definition:

It is a statistical method to predict the value of one variable (dependent) based on another

(independent).

The relationship is linear, meaning it forms a straight line.

- Key Points:

- Used for predicting continuous values like house prices, salaries, etc.

- The goal is to find the best-fit line that minimizes the error.

- Error is calculated using the Least Squares Method (difference between actual and predicted

values).

- Formula:

y = mx + c

Where:

- y: Predicted output (dependent variable).

- x: Input (independent variable).

- m: Slope (how steep the line is).

- c: Intercept (where the line crosses the y-axis).

- Example:

Predicting the price of a house based on its size:

- Input: Size of the house.

- Output: Price.

- The line might look like:

Price = 500 * Size + 20,000


Logistic Regression:

- Definition:

Used when the output is a categorical value (e.g., Yes/No, 0/1). Instead of predicting a continuous

number, it predicts probabilities.

- Key Points:

- Uses the sigmoid function to map values between 0 and 1.

- Thresholding (e.g., 0.5) is used to classify the result.

- Example use cases: Fraud detection, email spam classification.

- Formula:

The sigmoid function is:

P = 1 / (1 + e^(-z))

Where z = mx + c.

- How it works:

- Converts the output of linear regression to probabilities.

- If P > 0.5, classify as 1 (e.g., Pass).

- If P <= 0.5, classify as 0 (e.g., Fail).

- Example:

Predicting whether a student will pass:

- Input: Hours of study.

- If P = 0.8, the student has an 80% chance of passing.

2. BAYESIAN LEARNING

Bayes' Theorem:

- Definition:

A formula used to find the probability of an event occurring based on prior knowledge.

- Formula:
P(A|B) = (P(B|A) * P(A)) / P(B)

Where:

- P(A|B): Probability of A given B.

- P(B|A): Probability of B given A.

- P(A): Prior probability of A.

- P(B): Prior probability of B.

- Example:

If 1% of people have a disease and a test is 99% accurate, what's the probability you have the

disease if you test positive?

Concept Learning:

- Definition:

The process of learning a general rule from examples. Bayesian methods use probabilities to pick

the most likely rule (hypothesis).

- Example:

If it rains, people use umbrellas 80% of the time. From past data, the model learns the concept:

- Hypothesis: "Rain -> Umbrella usage."

Bayes Optimal Classifier:

- Definition:

Combines all possible hypotheses and chooses the one with the highest probability to classify

data.

- Key Point:

It's computationally expensive because it evaluates every hypothesis.

Naïve Bayes Classifier:

- Definition:
A simpler version of the Bayes classifier that assumes all features are independent.

- Formula:

P(H|D) = (P(D|H) * P(H)) / P(D)

- Example:

Email spam detection:

- If an email contains the word "free" and "win," classify as spam.

Bayesian Belief Networks:

- Definition:

A graphical model showing the relationships (dependencies) between variables.

- Example:

Weather -> Traffic -> Late to work.

Each variable affects the next one.

EM Algorithm (Expectation-Maximization):

- Definition:

An iterative algorithm to find missing/hidden data in problems.

- Steps:

1. E-step: Estimate missing values.

2. M-step: Update model parameters to maximize the likelihood.

3. SUPPORT VECTOR MACHINE (SVM)

Introduction:

- SVM is used for classification and regression.

- It tries to find the best boundary (hyperplane) that separates data points into classes.
Hyperplane:

- Definition:

A decision boundary that separates different classes.

- Example:

A straight line separates red dots and blue dots.

Support Vectors:

- The points closest to the hyperplane that help define its position.

Types of Kernels in SVM:

1. Linear Kernel:

- Suitable for linearly separable data.

- Example: Distinguishing male and female heights.

2. Polynomial Kernel:

- For non-linear data.

- Maps data to higher dimensions.

3. Gaussian Kernel (RBF):

- For very complex data.

- Example: Handwritten digit recognition.

Properties of SVM:

- Focuses on maximizing the margin between classes.

- Works well for small datasets.

- Effective in high-dimensional spaces.

Issues in SVM:

1. Computationally expensive for large datasets.


2. Choosing the right kernel is challenging.

3. Sensitive to noise in data.

You might also like