Regression

Analysis
Unit III
Part B
Why Is It Called Regression?
• The term regression in the literary sense is also referred as ‘moving
backward’.

• Although there is some debate about the origins of the name, the
statistical technique described above most likely was
termed "regression" by Sir Francis Galton in the 1877
• To describe the statistical feature of biological data (such as heights of
people in a population) to regress to some mean level. In
other words, while there are shorter and taller people, only
outliers are very tall or short, and most people cluster somewhere
around (or "regress" to) the average.
What Is the Purpose of Regression?
• In statistical analysis, regression is used to identify the associations
between variables occurring in some data.
• It can show both the magnitude of such an association and also
determine its statistical significance (i.e., whether or not
the association is likely due to chance).
• Regression is a powerful tool for statistical inference and has also
been used to try to predict future outcomes based on past
observations.
Regression Analysis
• Definition: The statistical technique that expresses the relationship
between two or more variables in the form of an equation to
estimate the value of a variable, based on the given value of
another variable, is called regression analysis.

• The variable whose value is estimated using the algebraic equation is

called dependent (or response) variable.

• The variable whose value is used to estimate this value is called

independent (regressor or predictor) variable.
Regression Analysis
• The linear algebraic equation used for expressing a dependent
variable in terms of independent variable is called linear regression
equation.
• Regression equation an its geometrical representation is called a
Regression curve.
Example of How Regression Analysis Is Used
in Finance
• Regression is often used to determine how many specific factors such
as the price of a commodity, interest rates, particular industries,
or sectors influence the price movement of an asset. The
aforementioned Capital Asset pricing model (CAPM) is based on
regression, and it is utilized to project the expected returns for
stocks and to generate costs of capital. A stock's returns are
regressed against the returns of a broader index, such as the
S&P 500, to generate a beta for the particular stock.
• Beta is the stock's risk in relation to the market or index and is
reflected as the slope in the CAPM model. The return for the stock
in question would be the dependent variable Y, while the
independent variable X would be the market risk premium.
Regression analysis is a branch of statistical theory that is widely used
in almost all the scientific disciplines. In economics it is the basic
techniques for measuring or estimating the relationship among
economic variables that constitute the essence of economic theory and
economic life.
For example, if we know that two variables price (X) and demand (Y)
are closely related we can find out the most probable value of X for a
given value of Y or the most probable value of Y for a given value of X.
Similarly, if we know that the amount of tax and the rise in the price of
a commodity are closely related we can find the expected price for a
certain amount of tax levy. The regression analysis helps
Differences between correlation and
regression analysis
Regression Analysis Correlation analysis
• Developing an algebraic equation • While measuring the strength (or
between two variables from sample degree) of the relationship between
data and predicting the value of one two variables is referred as
variable, given the value of the correlation analysis. The sign of
other variable is referred to as correlation coefficient indicates the
regression analysis. nature (direct or inverse) of
relationship between two variables,
while the absolute value of
correlation coefficient indicates the
extent of relationship.
Differences between correlation and
regression analysis
Regression Analysis Correlation analysis

• Correlation analysis •Regressionanalysis, in contrast

an association between two
determines to correlation, determines the
variables x and y but not that cause-and-effect relationship
they have a cause-and-effect between x and y, that is, a change
relationship. in the value of independent
variable x causes a
corresponding change (effect) in
the value of dependent variable y
if all other factors that affect y
remain unchanged.
Differences between correlation and
regression analysis
Regression Analysis Correlation analysis

• In linear regression analysis one • In correlation analysis

variable is considered as
both variables are considered to be
dependent variable and other as independent.
independent variable.
 Regression: Two Regression Lines
includes Regression Equation and
Regression coefficient.
Regression line Y on X Regression line X on Y
• Y on X, • X on Y,
where, where,
X = independent variable or Y = independent variable or
regressor or predictor regressor or predictor
Y = dependent variable or X = dependent variable or
responses variable. responses variable.
 Regression Equations
The regression equation of Y on X The regression equation of X on Y

• The regression equation of Y on • The regression equation of X on

X is the equation of the Y is the equation of the
best- fitting straight line in the best- fitting straight line in the
form form
Y = a + bX, X = a + bY,
• When deviation taken from • When deviation taken from

𝑌 − 𝑌ത = 𝑏𝑦𝑥 𝑋 − 𝑋ത 𝑋 − 𝑋ത = 𝑏𝑥𝑦 𝑌 − 𝑌ത
mean then model is mean then model is
Normal
equations
The regression equation of Y on X The regression equation of X on Y

𝑌 = 𝑎 + 𝑏𝑋, such as X = 𝑎 + 𝑏𝑌, such as

• Two normal equations for model • Two normal equations for model

•σ 𝑦 = 𝑛𝑎 + 𝑏 σ 𝑥 •σ 𝑥 = 𝑛𝑎 + 𝑏 σ 𝑦
•σ 𝑥𝑦 = 𝑎 σ 𝑥 + 𝑏 σ 𝑥2 •σ 𝑥𝑦 = 𝑎 σ 𝑦 + 𝑏 σ 𝑦2
 Regression
Equations
The regression equation of Y on X The regression equation of X on Y

where, where,
X = independent variable or Y = independent variable or
regressor or predictor. regressor or predictor.
Y = dependent variable or X = dependent variable or
responses variable. responses variable.
a = intercept term. a = intercept term.
b = slope or regression b = slope or regression
coefficient. coefficient.
 Regression
coefficient
𝑏 𝑏𝑥
𝑦
Regression coefficient Regression coefficient
𝑥 of Y on X is 𝑦 of X on Y is
represented by 𝑏𝑦𝑥 , is represented by 𝑏𝑥𝑦 , is
• The regression coefficient • The regression coefficient
mathematically 𝜎𝑦 represented as
𝑏𝑦 = r 𝑏𝑥 = r
mathematically represented as
𝜎 𝜎
𝑥 is −∞ 𝑡𝑜 + • Range of regression𝑦 is −∞ 𝑡𝑜 +
𝜎𝑦
𝑥 𝑥

∞ ∞.
• Range of regression

• When the deviation are taken for the • When the deviation are taken for the
assumed mean assumed mean
𝑏𝑦𝑥
𝑛
σ 𝑖= σ 𝑛
𝑥𝑖𝑦𝑖 σ 𝑥 𝑦 σ
1𝑛 𝑥 = 𝑖=1 𝑖 𝑖
σ 𝑖=1𝑥𝑦 𝑛 𝑥𝑦
= � =
σ 𝑦 σ 𝑖=1 𝑖 σ
𝑖
𝑥2 𝑦2
= �
𝑥 2 𝑦2
 Regression
Regression coefficientcoefficient
𝑏 𝑦 𝑏𝑥
Regression coefficient
𝑥 𝑦
• Where, • Where,
• r = Correlation coefficient • r = Correlation coefficient
• 𝜎𝑦 = Standard deviation of • 𝜎𝑦 = Standard deviation of
Y Y
• 𝜎𝑥 = Standard deviation of •𝑥𝜎𝑥 =𝑋
=
Standard deviation of
− 𝑋ത
•𝑦
X X
•𝑥 = 𝑋 − 𝑋ത 𝑌
=
•𝑦 = 𝑌 − 𝑌ത − 𝑌ത
Properti
es
• The correlation is • Arithmetic mean of
the geometric
coefficient mean of the regression coefficient is greater

𝑟=
regression coefficient. i.e. than the correlation coefficient.
𝑏𝑋𝑌𝑏
• If one± of 𝑌𝑋
• Regression coefficient are
the regression independent of change of
coefficient is greater than origin but not of scale.

− ∞ 𝑡𝑜 + ∞.
unity then the other is less than • Range of regression coefficient is
unity.
 Field: Shelf Space (x) and Field: Spice Sales (y) appear highly correlated.
90

80 f(x) = 0.134431137724551 x + 29.6706586826347

70

60
Spice Sales (y)

50

40

30

20

10

0
150 200 250 300 350 400 450

Shelf Space (x)

Numerical
• Q1 From the data given below find : (a) The two regression
coefficients. (b) The two regression equations. (c) The coefficient
of correlation between the marks in Economics and Statistics.
(d) The most likely marks in Statistics when marks in Economics
are 30.
Numerical
• Q2 The lines of regression of a bivariate population are :

• The variance of x is 9. Find (i) The mean values of x and y; (ii)

Correlation coefficient between x and y; (iii) Standard deviation of y.
Numerical for practice
 X 57 58 59 59 60 61 62 64
Y 67 68 65 68 72 72 69 71

Calculate the correlation and find the two lines of regression from the above data.