Dr. K. R.

Sekhar

REGRESSION ANALYSIS
What is regression?
 Regression Analysis is a statistical model which gives the relationship

between the dependent variables and independent variables.

 Regression analysis is used in many fields like machine learning, artificial

intelligence, data science, economics, finance, real estate, healthcare,

marketing, business, science, education, psychology, sports analysis,

agriculture, and many more.

 The main aim of the regression analysis is to give the relationship between

the variables, nature, and strength among the variables, and make

predictions based on the model.

(OR)

Regression Analysis is a supervised learning analysis where supervised

learning is the analyzing or predicting the data based on the previously available

data or past data.

For supervised learning, we have both train data and test data. Regression analysis

is one of the statistical methods for the analysis and prediction of the data.

Regression analysis is used for predictive data or quantitative or numerical data.

Dr. K. R. Sekhar
 Dr. K. R. Sekhar

TYPES OF REGRESSION ANALYSIS

1. Simple Linear Regression

 It is one of the basic and linear regression analysis. In this simple linear

regression there is only one dependent and one independent variable.

 This linear regression model only one predictor.

 This linear regression model gives the linear relationship between the

dependent and independent variables.

 Simple linear regression is one of the most used regression analyses.

 This simple linear regression analysis is mostly used in weather

forecasting, financial analysis, market analysis.

y=ax+b

 where y is a dependent variable

 x is a independent variable

 a, b are the regression coefficients

 Dr. K. R. Sekhar

2. Multiple Linear Regression

 Multiple linear regression analysis gives the relationship between the two

or more independent variables and a dependent variable.

 Multiple linear regression can be represented as the hyperplane in

multidimensional space. It is also a linear type regression analysis.

 It is almost similar to the linear regression but the major difference is the

number of independent variables is different.

 Multi linear regression analysis is used in the fields of real estate, finance,

business, public healthcare etc.

y=a0x1+a1x2+a2x3+....+b

 where y is a dependent variable

 x1,x2,x3 .... are the independent variables

 a0,a1,a2,... are the constants /coeffiecients of independent variables, b is

the intercept.
 Dr. K. R. Sekhar

3. Polynomial Regression

 Polynomial regression analysis is a nonlinear regression analysis.

 Polynomial regression analysis helps for the flexible curve fitting of the

data, involves the fitting of polynomial equation of the data.

 Polynomial regression analysis is the extension of the simple linear

regression analysis by adding the extra independent variables obtained by

raising the power.

 The mathematical expression for the polynomial regression analysis is

shown below.

y=a0+a1x+a2x^2+...........+anx^n

 where y is dependent variable

 x is independent variable

 a0,a1,a2 are the coefficeients of independent variable.

 Dr. K. R. Sekhar

4. Exponential Regression

 Exponential regression is a nonlinear type of regression.

 Exponential regression can be expressed in two ways.

 Exponential regression can be used in finance, biology, physics etc fields.

Let us look the mathematical expression for the exponential regression

with example.

y=ae^(bx)

 where y is dependent variable

 x is independent variable

 a , b are the regression coefficients.

5. Logistic Regression

Logistic regression analysis can be used for classification and regression. We can

solve the logistic regression equation by using the linear regression

representation. The mathematical equation of the logistic regression can be

denoted in two ways as shown below.

y=a+b*ln(x)

where y is dependent variable

 x is independent variable

 β0 , β1 .... are the constants/regression coefficients

 Dr. K. R. Sekhar

APPLICATIONS OF REGRESSION ANALYSIS

Regression Analysis has various applications in many fields like economics,

finance, real estate, healthcare, marketing, business, science, education,

psychology, sport analysis, agriculture and many more.

Let us now discuss about the few applications of regression analysis.

1. Regression analysis is used for the prediction of stock price based on the

past data, analyzing the relationship between the interest rate and consumer

spending.

2. It can be used for the analysis of the impact of price changes on product

demand and for predicting the sales based on expenditure of advertisement.

3. It can be used in real estate for predicting the value of property based on

the location.

4. Regression is also used in the weather forecasting.

5. It is also used for the prediction of crops yield based on the weather

conditions, impact of fertilizers and irrigation the plant.

6. It can be used in the analysis of the product quality and also gives the

relationship between the manufacturing variables and product quality.

7. It can be used for the prediction of performance of the sports players based

on the historical data and impact of coaching strategies on team success.

 Dr. K. R. Sekhar

Regression analysis describes how the independent variable(s) is (are) related to

the dependent variable i.e. regression analysis measures the average relationship

between independent variables and dependent variable.

The literal meaning of regression is “stepping back towards the average” which

was used by British Biometrician Sir Francis Galton (1822-1911) regarding the

height of parents and their offspring’s.

Regression analysis is a mathematical measure of the average relationship

between two or more variables. There are two types of variables in regression

analysis:

1. Independent variable

2. Dependent variable

The variable which is used for prediction is called independent variable. It is also

known as regressor or predictor or explanatory variable.

The variable whose value is predicted by the independent variable is called

dependent variable. It is also known as regressed or explained variable.

 Dr. K. R. Sekhar

LINES OF REGRESSION

Regression lines are the lines of best fit which express the average relationship

between variables. Here, the concept of lines of best fit is based on principle of

least squares.

1. Let X be the independent variable and Y be the dependent variable.

Let the equation of the line of regression of y on x is y=a+bx.

𝑟𝜎𝑦
If regression line of y on x is (𝑦 − 𝑦̅) = (𝑥 − 𝑥̅ ) and it is denoted by 𝑏𝑦𝑥
𝜎𝑥

𝑟𝜎𝑦
Where 𝑏𝑦𝑥 =
𝜎𝑥

Note: (𝑦 − 𝑦̅) = 𝑏𝑦𝑥 (𝑥 − 𝑥̅ )

𝑪𝒐𝒗(𝒙,𝒚)
Slope 𝒃𝒚𝒙 = ; 𝑪𝒐𝒗(𝒙, 𝒚) = 𝒓𝝈𝒙 𝝈𝒚
𝝈𝟐𝒙

∑(𝒙 − 𝒙
̅)(𝒚 − 𝒚
̅)
𝒃𝒚𝒙 =
∑(𝒙 − 𝒙̅)𝟐

2. Let Y be the independent variable and X be the dependent variable.

Let the equation of the line of regression of y on x is x=a+by.

𝑟𝜎𝑥
If regression line of y on x is (𝑥 − 𝑥̅ ) = (𝑦 − 𝑦̅) and it is denoted by 𝑏𝑥𝑦
𝜎𝑦

𝑟𝜎𝑥
Where 𝑏𝑥𝑦 =
𝜎𝑦

Note: (𝑥 − 𝑥̅ ) = 𝑏𝑥𝑦 (𝑦 − 𝑦̅)

 Dr. K. R. Sekhar
𝑪𝒐𝒗(𝒙,𝒚)
Slope 𝒃𝒙𝒚 = ; 𝑪𝒐𝒗(𝒙, 𝒚) = 𝒓𝝈𝒙 𝝈𝒚
𝝈𝟐𝒚

∑(𝒙 − 𝒙
̅)(𝒚 − 𝒚
̅)
𝒃𝒙𝒚 =
∑(𝒚 − 𝒚̅)𝟐

Properties of Regression Coefficients:

1. Geometric mean of the regression coefficients is correlation coefficient.

i.e; 𝑟 = ∓√𝑏𝑦𝑥 𝑏𝑥𝑦

2. If one of the regression coefficients is greater than one, then other must be less

than one.

1
i.e; 𝑟 2 ≤ 1 then 𝑏𝑦𝑥 ≤ <1
𝑏𝑥𝑦

3. It is expressed in the form of an original unit of data.

4. The Arithmetic means (A.M) of both regression coefficients is equal to or

greater than the coefficient of correlation.

 (byx + bxy)/2= equal or greater than r.

5. If 𝑚1 and 𝑚2 are the slopes of two lines and 𝜃be the angle between them

𝑚1 −𝑚2
therefore 𝑡𝑎𝑛𝜃 = | |
1+𝑚1 𝑚2

1−𝑟 2 𝜎𝑥 𝜎𝑦
𝜃 = 𝑡𝑎𝑛−1 { (𝜎2 +𝜎2)}
𝑟 𝑥 𝑦
 Dr. K. R. Sekhar

NOTE:

𝜋
1. If 𝑟 = 0 i.e., variables are uncorrelated then 𝑡𝑎𝑛𝜃 = ∞ ⇒ 𝜃 =
2

In this case lines of regression are perpendicular to each other.

2. If 𝑟 = ∓ i.e. variables are perfect positive or negative correlated then

𝑡𝑎𝑛𝜃 = 0 ⇒ 𝜃 = 0 𝑜𝑟 𝜋.

In this case, regression lines either coincide or parallel to each other.

3. There are two angles between regression lines whenever two lines intersect

each other, one is acute angle and another is obtuse angle.

𝜋
The tan θ would be greater than zero if θ lies between 0 and then θ is called
2

acute angle.

𝜋
The tan θ would be less than zero if θ lies between and π then θ is called obtuse
2

angle.