Birinder Singh, Assistant Professor, PCTE

REGRESSION
ANALYSIS
REGRESSION
 Regression Analysis measures the nature and
extent of the relationship between two or more
variables, thus enables us to make predictions.

Birinder Singh, Assistant Professor, PCTE

 Regression is the measure of the average
relationship between two or more variables.
UTILITY OF REGRESSION
 Degree & Nature of relationship
 Estimation of relationship

Birinder Singh, Assistant Professor, PCTE

 Prediction

 Useful in Economic & Business Research

DIFFERENCE BETWEEN CORRELATION &
REGRESSION
 Degree & Nature of Relationship
 Correlation is a measure of degree of relationship
between X & Y

Birinder Singh, Assistant Professor, PCTE

 Regression studies the nature of relationship
between the variables so that one may be able to
predict the value of one variable on the basis of
another.
 Cause & Effect Relationship
 Correlation does not always assume cause and effect
relationship between two variables.
 Regression clearly expresses the cause and effect
relationship between two variables. The independent
variable is the cause and dependent variable is effect.
DIFFERENCE BETWEEN CORRELATION &
REGRESSION
 Prediction
 Correlation doesn’t help in making predictions

Birinder Singh, Assistant Professor, PCTE

 Regression enable us to make predictions using
regression line
 Symmetric
 Correlation coefficients are symmetrical i.e. rxy = ryx.
 Regression coefficients are not symmetrical i.e. bxy ≠ byx.

 Origin & Scale

 Correlation is independent of the change of origin and
scale
 Regression coefficient is independent of change of origin
but not of scale
TYPES OF REGRESSION ANALYSIS
 Simple & Multiple Regression
 Linear & Non Linear Regression

Birinder Singh, Assistant Professor, PCTE

 Partial & Total Regression
SIMPLE LINEAR REGRESSION

Birinder Singh, Assistant Professor, PCTE

Simple
Linear
Regression

Regression Regression Regression

Lines Equations Coefficients
REGRESSION LINES
 The regression line shows the average relationship
between two variables. It is also called Line of Best Fit.
If two variables X & Y are given, then there are two

Birinder Singh, Assistant Professor, PCTE


regression lines:
 Regression Line of X on Y
 Regression Line of Y on X

 Nature of Regression Lines

 If r = ±1, then the two regression lines are coincident.
 If r = 0, then the two regression lines intersect each other at
90°.
 The nearer the regression lines are to each other, the greater
will be the degree of correlation.
 If regression lines rise from left to right upward, then
correlation is positive.
REGRESSION EQUATIONS
 Regression Equations are the algebraic
formulation of regression lines.

Birinder Singh, Assistant Professor, PCTE

 There are two regression equations:
 Regression Equation of Y on X
 Y = a + bX
 Y – 𝑌 = 𝑏𝑦𝑥 (𝑋 − 𝑋 )

σ𝑦
 Y – 𝑌 = 𝑟. (𝑋 − 𝑋)
σ𝑥
 Regression Equation of X on Y
 X = a + bY
 X – 𝑋 = 𝑏𝑥𝑦 (𝑌 − 𝑌 )

σ𝑥
 X – 𝑋 = 𝑟. (𝑌 − 𝑌)
σ𝑦
REGRESSION COEFFICIENTS
 Regression coefficient measures the average
change in the value of one variable for a unit
change in the value of another variable.

Birinder Singh, Assistant Professor, PCTE

 These represent the slope of regression line

 There are two regression coefficients:

σ𝑦
 Regression coefficient of Y on X: byx = 𝑟. σ𝑥
σ𝑥
 Regression coefficient of X on Y: bxy = 𝑟. σ𝑦
PROPERTIES OF REGRESSION
COEFFICIENTS
 Coefficient of correlation is the geometric mean of
the regression coefficients. i.e. r = 𝑏𝑥𝑦 . 𝑏𝑦𝑥

Birinder Singh, Assistant Professor, PCTE

 Both the regression coefficients must have the
same algebraic sign.
 Coefficient of correlation must have the same sign
as that of the regression coefficients.
 Both the regression coefficients cannot be greater
than unity.
 Arithmetic mean of two regression coefficients is
equal to or greater than the correlation
𝑏𝑥𝑦+𝑏𝑦𝑥
coefficient. i.e. ≥r
2
 Regression coefficient is independent of change of
origin but not of scale
OBTAINING REGRESSION EQUATIONS

Regression

Birinder Singh, Assistant Professor, PCTE

Equations

Using
Using Normal
Regression
Equations
Coefficients
REGRESSION EQUATIONS IN INDIVIDUAL
SERIES USING NORMAL EQUATIONS
 This method is also called as Least Square Method.
 Under this method, regression equations can be
calculated by solving two normal equations:

Birinder Singh, Assistant Professor, PCTE

 For regression equation Y on X: Y = a + bX
 Σ𝑌 = 𝑁𝑎 + 𝑏Σ𝑋
 Σ𝑋𝑌 = 𝑎Σ𝑋 + 𝑏Σ𝑋2

 Another Method
𝑁 .Σ𝑋𝑌 − Σ𝑋.Σ𝑌 &
 byx = a = 𝑌 − b𝑋
𝑁.Σ𝑋 2 −(Σ𝑋)2
 Here a is the Y – intercept, indicates the minimum
value of Y for X = 0
 & b is the slope of the line, indicates the absolute
increase in Y for a unit increase in X.
PRACTICE PROBLEMS
Q1: Calculate the regression equation of X on Y using
method of least squares: X = 0.5 + 0.5Y

Birinder Singh, Assistant Professor, PCTE

X 1 2 3 4 5
Y 2 5 3 8 7

Q2: Given the following data:

N = 8, ƩX = 21, ƩX2 = 99, ƩY = 4, ƩY2 = 68, ƩXY = 36
Using the values, find:
o Regression Equation of Y on X Y = – 1.025 + 0.581X
o Regression Equation of X on Y X = 2.432 + 0.386Y
o Value of Y when X = 10 Y = 4.785
o Value of X when Y = 2.5 X = 3.397
Birinder Singh, Assistant Professor, PCTE
REGRESSION EQUATIONS USING
REGRESSION COEFFICIENTS

Birinder Singh, Assistant Professor, PCTE

Methods

Using Actual Using deviations Using deviations

Values of from Actual from Assumed Using r, σx, σy
X&Y Means Means
REGRESSION EQUATIONS USING REGRESSION
COEFFICIENTS (USING ACTUAL VALUES)
 Regression Equation of Y on X
𝑁 .Σ𝑋𝑌 − Σ𝑋.Σ𝑌
 Y – 𝑌 = byx (X – 𝑋) where byx = 𝑁.Σ𝑋 2 −(Σ𝑋)2

Birinder Singh, Assistant Professor, PCTE

 Regression Equation of X on Y
𝑁 .Σ𝑋𝑌 − Σ𝑋.Σ𝑌
 X – 𝑋 = bxy (Y – 𝑌) where bxy = 𝑁.Σ𝑌 2 −(Σ𝑌)2

Q3: Calculate the regression equation of Y on X & X on Y

Y = 1.3X + 1.1, X = 0.5 + 0.5Y
REGRESSION EQUATIONS USING REGRESSION
COEFFICIENTS (USING DEVIATIONS FROM
ACTUAL VALUES)
 Regression Equation of Y on X
Σ𝑥𝑦
 Y – 𝑌 = byx (X – 𝑋) where byx =

Birinder Singh, Assistant Professor, PCTE

Σ𝑥 2
 Regression Equation of X on Y
Σ𝑥𝑦
 X – 𝑋 = bxy (Y – 𝑌) where bxy = 2
Σ𝑦

Q4: Calculate the regression equation of Y on X & X on Y

using method of least squares: Y = 0.26X + 3.2, X = 4.75 + 0.45Y

X 2 4 6 8 10 12
Y 4 2 5 10 3 6
REGRESSION EQUATIONS USING REGRESSION
COEFFICIENTS (USING DEVIATIONS FROM
ASSUMED MEAN)
 Regression Equation of Y on X
𝑁 .Σ𝑑𝑥𝑑𝑦 − Σ𝑑𝑥 Σ𝑑𝑦
 Y – 𝑌 = byx (X – 𝑋) where byx =

Birinder Singh, Assistant Professor, PCTE

𝑁.Σ𝑑𝑥 2 −(Σ𝑑𝑥)2
 Regression Equation of X on Y
𝑁 .Σ𝑑𝑥𝑑𝑦 − Σ𝑑𝑥 Σ𝑑𝑦
 X – 𝑋 = bxy (Y – 𝑌) where bxy =
𝑁.Σ𝑑𝑦 2 −(Σ𝑑𝑦)2

Q5: Calculate the regression equation of Y on X & X

Y = 1.212 X + 34.725

X 78 89 97 69 59 79 68 61

Y 125 137 156 112 107 136 124 108

Birinder Singh, Assistant Professor, PCTE
REGRESSION EQUATIONS USING REGRESSION
COEFFICIENTS (USING STANDARD DEVIATIONS)
 Regression Equation of Y on X
σ𝑦
 Y – 𝑌 = byx (X – 𝑋) where byx = 𝑟.

Birinder Singh, Assistant Professor, PCTE

σ𝑥
 Regression Equation of X on Y
σ𝑥
 X – 𝑋 = bxy (Y – 𝑌) where bxy = 𝑟. σ𝑦

Q6: Estimate Y when X = 9 as per the following information:

Y = 15.88

X Y
Arithmetic Mean 5 12
Standard Deviation 2.6 3.6
Correlation Coefficient 0.7
PRACTICE PROBLEMS
Q7: If 𝑋 = 25, 𝑌 = 120, bxy = 2. Estimate the value of X
when Y = 130. X = 45

Birinder Singh, Assistant Professor, PCTE

Q8: If σ𝑥 2 = 9, σ𝑦 2 = 1600, obtain bxy. bxy = 0.04

Q9: Given two regression equations:

3X + 4Y = 44
5X + 8Y = 80
Variance of X = 30.
Find 𝑋, 𝑌, r and σ𝑦 8,5,– 0.91, 3.7
SHORTCUT METHOD OF CHECKING
REGRESSION EQUATIONS
 Suppose two regression equations are as follows:
 a1x + b1y + c1 = 0

Birinder Singh, Assistant Professor, PCTE

 a2x + b2y + c2 = 0

Case 1: If a1b2 ≤ a2b1 (in magnitude, ignoring negative), then

 a1x + b1y + c1 = 0 is the regression of Y on X
 a2x + b2y + c2 = 0 is the regression of X on Y

Case 2: If a1b2 > a2b1 (in magnitude, ignoring negative), then

 a1x + b1y + c1 = 0 is the regression of X on Y
 a2x + b2y + c2 = 0 is the regression of Y on X
STANDARD ERROR OF ESTIMATE
 Standard error of estimate helps us to know that
to what extent the estimates are accurate.

Birinder Singh, Assistant Professor, PCTE

 It shows that to what extent the estimated values
by regression line are closer to actual values
 For two regression lines, there are two standard
error of estimates:
 Standard error of estimate of Y on X (Syx)
 Standard error of estimate of X on Y (Sxy)
FORMULAE FOR SE (Y ON X)
2
Σ 𝑌 −𝑌𝑐
 Syx = Y = Actual Values,
𝑁
Yc = Estimated Values

Birinder Singh, Assistant Professor, PCTE

Σ𝑌2 −𝑎Σ𝑌 −𝑏Σ𝑋𝑌
 Syx = Here a & b are to be
𝑁
obtained from normal equations

 Syx = σy 1 − 𝑟2
PRACTICE PROBLEMS – SE
Q10: Find the Standard error of estimates if σx = 4.4,
σy = 2.2 & r = 0.8 Ans: 1.32, 2.64

Birinder Singh, Assistant Professor, PCTE

Q11: Given: ƩX = 15, ƩY = 110, ƩXY = 400, ƩX2 = 250,
ƩY2 = 3200, N = 10. Calculate Syx Ans: 13.21

Q12: Compute regression equation Y on X. Hence, find Syx

Ans: Y = 11.9 – 0.65X, 0.79
X 6 2 10 4 8
Y 9 11 5 8 7