Scribd
Upload
6 views16 pages

8.2 Regression

Regression analysis is a statistical method used to model the relationship between independent and dependent variables for prediction and analysis. It includes various types such as simple linear, curvilinear, multiple, and logistic regression, each serving different purposes. The method of least squares is commonly used to estimate the parameters of linear regression models.

Uploaded by

osama7abx
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
6 views16 pages

8.2 Regression

Regression analysis is a statistical method used to model the relationship between independent and dependent variables for prediction and analysis. It includes various types such as simple linear, curvilinear, multiple, and logistic regression, each serving different purposes. The method of least squares is commonly used to estimate the parameters of linear regression models.

Uploaded by

osama7abx
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 16

8.

2_Regression
Regression

Introduction
Regression analysis is a statistical method used to examine the relationship
between one or more independent variables (predictors) and a dependent variable
(response).
The goal of regression is to model this relationship mathematically to predict,
analyze, or understand how changes in the predictors affect the response.
Regression is widely applied across fields such as economics, biology, machine
learning, finance, and social sciences for prediction, trend analysis, and
hypothesis testing.

Definition
Regression : is a mathematical measure of the average relationship between two
or more variables in terms of original units of the data.

Regression analysis serves three main purposes :


the first is description .
the second is control .
the third is prediction .

Types of Regression Models :

1. Simple Linear Regression :


One independent variable is used to predict a dependent variable .
If the relation between the dependent and independent variable follows a straight
line pattern then the regression is called as linear regression.

2. Curvilinear (Non-linear or Polynomial) Regression :


One independent variable is used to predict a dependent variable .
if the relation is expressed in the form of a curve then the type of regression is called
as curvilinear regression .

3. Multiple Regression :

Multiple independent variables are used at the same time .

4. Logistic Regression :

Used when the dependent variable is categorical .

Knowing The Regression Type Using a Scatter Diagram :


If the dots of a scatter diagram concentrate around a certain curve then the
regression is curvilinear regression and the corresponding relation is called as the
regression curve .
If the points of a scatter diagram lie on a straight line then the line is called as the
line of regression and the type of regression is simple linear regression .

Example :

Ex 1 :
Ex 2 :

It is clear that the data in this example takes the form of linear distribution, as it is
distributed around that line.
Ex 3 :
There is a linear relationship :

Ex 4 :
Simple Linear Regression
Mathematical Formulation :
The simple linear regression model is expressed as:

Y = aX i + b + ϵ i

Where:
Y : Dependent variable (response variable) .
X : Independent variable (predictor variable) .
b ​: Intercept parameter (value of Y when X = 0 ).
a ​: Slope parameter (change in Y for a one-unit change in X ) .
ϵi : Error term (captures variability not explained by the model ) with mean
2 2
E[ϵ i ] = 0 and variance σ (ϵ i ) = σ .
Error Terms are uncorrelated , so that their covariance is zero ,
Cov(ϵ i , ϵ j ) = 0 for all i ≠ j .

Objective :
^
Estimate b ​and a such that the difference between observed Y and predicted Y
values is minimized.
This is typically achieved using the Least Squares Method.

Assumptions of Linear Regression :


Linearity :

The relationship between X and X is linear.

Independence :
Observations are independent.

Homoscedasticity :

Constant variance of error terms.

Normality :
Error terms are normally distributed.

The Method Of Least Squares :


This method it used to determine the parameters a and b of a linear model to compute
the model equation .
n n n
n∑ xi yi − ∑ xi ∑ yi
i=1 i=1 i=1
^ = ^ ^x̄
a = a , β = b = ȳ − a
n n 2
2
n∑ x − (∑ xi )
i=1 i i=1

^ ^ ^
Y = aX + b

Where :

^
Y : is the predicted variable .
x̄ : the average of X values .
ȳ : the average of y values .

The Inverse prediction


Often, a regression model of Y on X is used to predict the value of X that a new
¯ , which is known as
observation has arisen from, based on the predicted variable Y
inverse prediction .

^ ^
Y − b
X =
^
a

The Equations of The Reversed Case :

On the other hand, the regression line equation as X is the response variable and
Y is the predictor variable can be written as follows :

c and d are the parameters


2 2
ζi is the error term ( σ (ζ i ) = σ )

X i = cY i + d + ζ i

n n n
n∑ xi yi − ∑ xi ∑ yi
i=1 i=1 i=1
^= ^ ^ȳ
λ = c , δ = d = x̄ − a
n n 2
2
n∑ y − (∑ yi )
i=1 i i=1

^ ^
^ Y + d
X = c.

Where :

^
X : is the predicted variable .
x̄ : the average of X values .
ȳ : the average of y values .

Examples :
Ex 1 :
Ex 2 :
Ex 3 : On the Inverse Prediction

You might also like