Chapter No.

14
SIMPLE REGRESSION AND CORRELATION
Discuss the term regression.
The dependence of one variable on one or more other variables is called regression.
OR
The process in which we estimate one variable on the basis of another variable is called regression.
Note: The term regression was introduced by the English biometrician, Sir Francis Galton in 1885.
The term regression means “to step back or to regress”.
Define regression analysis.
In regression analysis, we obtain an equation which can be sued to estimate the values of the
dependent variable on the basis of independent variable whose values are known.
OR
The technique used to develop the equation and provide the estimates is called regression analysis.
Explain the simple linear regression model.
The simple linear regression model is 𝑌 = 𝛼 + 𝛽(𝑋), where X is independent Y is dependent variables.
In this model α is the y - intercept and β is the slope of line or regression coefficient.
Define independent variable or Regressor.
A variable whose values do not depend on any other variable is called independent variable or
regressor. OR
The variable that provide the basis of estimation or prediction is called as independent variable or
regressor.
Define regressand or dependent variable.
A variable whose values depend on the values of another variable is called dependent variable or
regressand. OR
The variable which we want to estimate or predict on the basis of independent variable is
called dependent variable or regressand.
What is scatter diagram?
The graphical representation of the paired observations (𝑥𝑖 , 𝑦𝑖 ) is called scatter diagram.
What is meant by y-intercept or intercept?
Y-intercept or intercept is the value of y against the value of x = 0.
Define slope or regression coefficient for the simple regression line.
What is regression coefficient?
Slope or regression coefficient is the rate of change in the dependent variable as per unit change in the
independent variable. It is denoted by β.
Give the properties of regression coefficient.
1. Regression Coefficients (𝑏𝑦𝑥 , 𝑏𝑥𝑦 ) always have the same sign.

2. If regression coefficient 𝑏𝑦𝑥 is greater than 1, then 𝑏𝑥𝑦 will be less than 1.

3. Regression coefficient is independent of origin but dependent on scale.

4. Two regression coefficients are not symmetric with respect to x and y. 𝑏𝑦𝑥 ≠ 𝑏𝑥𝑦

5. The correlation coefficient is the geometric mean of two regression coefficients. r = ± √𝑏𝑦𝑥 × 𝑏𝑥𝑦
What do you understand by simple linear regression?
In simple linear regression, the dependent variable y is expressed as a linear function
of one independent variable is called simple linear regression.
OR
If the dependent variable depends on a single independent variable is called simple linear
regression modal. Simple linear regression modal is 𝑌 = 𝑎 + 𝑏𝑋 + 𝑒𝑖
Where 𝑌 = Dependent variable;
𝑋 = Independent variable
𝑎 = Intercept i.e. average value of “Y” when “X= 0”
𝑏 = Regression coefficient or coefficient of independent variable Slope of regression line
𝑒𝑖 = Random error
Enlist the properties of regression line.

1. The least square regression line always passes through the mean values i.e. (𝑋 , 𝑌).

2. The sum of observed values and estimated values is equal. i.e. ∑ 𝑌= ∑ 𝑌̂

3. The sum of the residual error is always zero. i.e. ∑ 𝑒𝑖 = ∑( 𝑌 − 𝑌̂ ) = 0
∑𝑌 ∑ 𝑌̂
4. The mean of the observed and estimated values are equal. i.e. =
𝑛 𝑛

5. The sum of square deviation of observed values from the estimated values is minimum,
2
i.e. ∑(𝑌 − 𝑌̂) = Minimum
Define principal of least square for fitting a regression line.
OR
State the principal of least square.
The principal of least squares states that the sum of squared deviations of the observed
values from the estimated values should be least or minimum.
OR
The principal of the method of least squares consists of determining the value of the unknown
parameters that will minimize the sum of squares of errors or residuals.
Define intercept of straight line or regression line.
In regression modal the average value of dependent variable when there is no association
is called intercept. In simple linear regression modal is 𝑌 = 𝑎 + 𝑏𝑋 + 𝑒𝑖
𝑎 = Intercept i.e. average value of “Y” when “X= 0”.

What is the role of independent and dependent variables in the regression?

The values of the independent variable are fixed in advance and it is used to estimate
the dependent variable, whereas dependent variable is used for the purpose of estimation.
Distinguish between positive and negative regression.
When the regression line slopes upwards to the right and its value will be positive. It
indicates the amount of increase in y for each unit increase in X, which is called positive
regression.
When the regression line slopes downwards to the right and its value will be negative. It
indicates the amount of decrease in y for each unit increase in X, which is called negative
regression.
Define the term residual.
The difference between observed and estimated value is called residual or error. i.e.
∑ 𝑒𝑖 = ∑( 𝑌 − 𝑌̂ ) = 0
Write the Normal equation of regression line Y on X.
The Normal equations of regression line Y on X are
𝑌 = 𝑎 + 𝑏𝑋
∑ 𝑌 = 𝑛𝑎 + 𝑏 ∑ 𝑋…………………. (i)
∑ 𝑋𝑌 = 𝑎 ∑ 𝑋 + 𝑏 ∑ 𝑋 2 ……………. (ii)
What is the intercept and slope of the line in the equation?
The intercept and slope of the line is
𝑛 ∑ 𝑋𝑌 − ( ∑ 𝑋 )( ∑ 𝑌 ) ∑(𝑋 − 𝑋)(𝑌 − 𝑌)
𝑏𝑦𝑥 = 2 2
𝑂𝑅 𝑏𝑦𝑥 = 2
𝑛 ∑ 𝑋 − (∑ 𝑋) ∑(𝑋 − 𝑋)
∑ 𝑌 − 𝑏𝑦𝑥 (∑ 𝑋)
𝑎𝑦𝑥 = 𝑂𝑅 𝑎𝑦𝑥 = 𝑌 − 𝑏𝑦𝑥 (𝑋)
𝑛
Explain the term correlation.
The measure the strength or closeness of linear relationship between two variables is called
simple linear correlation.
Examples
i) The height and weight of children correlated with age.
ii) Supply and demands of goods correlated with price.

Give two examples of correlation.

Examples of correlation are height and weight of children, ages of husband and ages of wives
at the time of their marriage, temperature and length of copper wire.
Define positive correlation.

The correlation is said to be positive if the two random variables tend to move in the same
direction i.e. increase or decrease simultaneously.

Example: The length of iron bar increases as temperature increases.

Elaborate the term negative correlation.

The correlation is said to be negative if the two random variables tend to move in opposite
direction i.e. one random variable increases as the other random variable decreases.

Example: The demand of items increases as price of items decreases.

Describe perfect positive.

The correlation is said to be perfect positive if the relationship between the two variables is
perfectly linear with positive slop i.e. 𝑟 = +1.
Define perfect negative.
The correlation is said to be perfect negative if the relationship between the two variables is
perfectly linear with negative slops i.e. 𝑟 = −1.
Differentiate between direct and inverse correlation.
Direct Correlation: The correlation is said to be direct if the two random variables tend to
move in the same direction i.e. increase or decrease simultaneously.
Inverse Correlation: The correlation is said to be inverse if the two random variables tend to
move in opposite direction i.e. one random variable increases as the other random variable
decreases.
Define the term correlation coefficient.
A numerical value which measures the degree of strength in the linear relationship between
any two variables is called correlation coefficient. It is denoted by "𝑟" . It is given as
𝑛 ∑ 𝑋𝑌−(∑ 𝑋)(∑ 𝑌)
𝑟𝑋𝑌 = 𝑟𝑌𝑋 = OR 𝑟𝑋𝑌 = 𝑟𝑌𝑋 =
√[𝑛 ∑ 𝑋 2 −(∑ 𝑋)2 ][𝑛 ∑ 𝑌 2 −(∑ 𝑌)2 ]

∑(𝑋−𝑋)(𝑌−𝑌)
2 2
√∑(𝑋−𝑋) × ∑(𝑌−𝑌)

What is the rage of correlation coefficient?

The rage of correlation coefficient is −1 ≤ 𝑟 ≤ +1 .
If 𝒓 = +𝟏 , 𝒓 = −𝟏 , 𝒓 = 𝟎. what does it show?
𝑟 = +1, It shows that there is perfect positive correlation.
𝑟 = −1, It shows that there is perfect negative correlation.
𝑟 = 0, It shows that there is no correlation i.e. the variables are independent.
Sate any five properties of correlation coefficient.
The properties of correlation coefficient are as fellows
i). Correlation Coefficient is symmetrical with respect to x and y i.e. 𝑟𝑥𝑦 = 𝑟𝑦𝑥
ii). Correlation Coefficient lies between -1 to +1 inclusive i.e. −≤ 𝑟 ≤ +1
iii). Correlation Coefficient is the G.M of two regression co-efficients i.e. 𝒓 =
± √𝒃𝒙𝒚 × 𝒃𝒚𝒙
iv). The signs of Regression coefficients and of correlation coefficient remain same.
v). Correlation Coefficient is independent of change of origin and unit scale i.e. 𝑟𝑥𝑦 = 𝑟𝑢𝑣
What do you understand by zero – correlation?
If the change in one variable does not affect the other variable then there will no correlation
or zero correlation.
What is the relationship between regression co – efficient and correlation coefficient?
The relationship between regression co – efficient and correlation coefficient is the geometric
mean of the two regression coefficients 𝒃𝒚𝒙 and 𝒃𝒙𝒚 i.e. i.e. 𝒓 = ± √𝒃𝒙𝒚 × 𝒃𝒚𝒙 .

Write any two formulas of correlation coefficient.

 Sxy Sxy
r= , r= , r = ± √bxy × byx
Sx . Sy
√Sx2 . Sy2

Given 𝐫𝐱𝐲 = 𝟎. 𝟗𝟕, explain or interpret it.

There is a high positive correlation between the variables.

𝟐 𝟐
Given ∑(𝐗 − 𝑿) (𝐘 − 𝒀) = 𝟗𝟐, ∑(𝐗 − 𝑿) = 𝟏𝟕𝟎 , ∑(𝐘 − 𝒀) = 𝟏𝟒𝟎 𝒂𝒏𝒅 𝒏 = 𝟏𝟎.

Find the value of regression coefficients and correlation coefficient.

Solution:
2 2
∑(X − 𝑋) (Y − 𝑌) = 92, ∑(X − 𝑋) = 170 , ∑(Y − 𝑌) = 140 , 𝑛 = 10 , bxy = ?, byx =

?, rxy = ?

∑(X − 𝑋) (Y − 𝑌) 92
byx = 2 = = 0.54
∑(X − 𝑋) 170

∑(X − 𝑋) (Y − 𝑌) 92
bxy = 2 = = 0.66
∑(Y − 𝑌) 140

∑(X − 𝑋) (Y − 𝑌) 92
rxy = = = 0.60
2 2 √170 × 140
√∑(Y − 𝑌) × ∑(Y − 𝑌)

Given 𝐛𝐱𝐲 = 𝟎. 𝟖𝟐 , 𝐫𝐱𝐲 = 𝟎. 𝟗𝟕. Find 𝐛𝐲𝐱 =?

Solution:

bxy = 0.82 , rxy = 0.97 , byx =?

rxy = ± √bxy × byx

Taking both side square

2
rxy = byx × bxy

(0.97)2 = byx (0.82)

0.9409 = byx (0.82)

 0.9409
byx =
0.82

byx = 1.15

If 𝐛𝐱𝐲 = 𝟎. 𝟐𝟕 , 𝐛𝐲𝐱 = 𝟎. 𝟔𝟎 Find 𝐫𝐲𝐱 =?

Solution: bxy = 0.27 , byx = 0.60 , rxy =?

rxy = ± √bxy × byx

rxy = √0.27 × 0.60 = √0.162 = 0.402

Given 𝐒𝐱 = 𝟒 , 𝐫𝐱𝐲 = 𝟎. 𝟖 , 𝐒𝐱𝐲 = 𝟐𝟎. Find 𝐒𝐲 =?

Solution: Sx = 4 , rxy = 0.8 , Sxy = 20 , Sy =?

Sxy
rxy =
Sx . Sy

20
(0.8) =
(4) . Sy

20
Sy = = 6.25
(4). (0.8)

Given ∑ 𝑿𝒀 = 𝟑𝟓𝟎, 𝒏 = 𝟏𝟎 , 𝑿 = 𝟓 , 𝒀 = 𝟔 , 𝑺𝟐𝒙 = 𝟒 , 𝑺𝟐𝒚 = 𝟗. Find 𝐫𝐱𝐲 =?

Solution: ∑ 𝑋𝑌 = 350, 𝑛 = 10 , 𝑋 = 5 , 𝑌 = 6 , 𝑆𝑥2 = 4 , 𝑆𝑦2 = 9. Find rxy =?

∑ XY − nXY
rxy =
nSx . Sy

350 − (10)(5)(6)
rxy =
(10)(2)(3)

350 − 300
rxy =
60

50
rxy =
60

rxy = 0.8333
Given 𝐒𝐱 = 𝟐. 𝟏𝟐 , 𝐒𝐲 = 𝟐. 𝟑𝟒 , 𝐫𝐱𝐲 = 𝟎. 𝟔𝟎𝟓 , ∑(𝐗 − 𝑿) (𝐘 − 𝒀) = 𝟐𝟒. Compute the

number of pairs.

Solution: Sx = 2.12 , Sy = 2.34 , rxy = 0.605 , ∑(X − 𝑋) (Y − 𝑌) = 24 , n =?

∑(X − 𝑋) (Y − 𝑌)
rxy =
√n 𝑆𝑥2 . 𝑆𝑦2

24
0.605 =
√n(2.12)2 (2.34)2

24
0.605 =
√(4.49)(5.47)n

24
0.605 =
4.95 √n

24
√n =
4.95 × 0.605

√n = 9.60

Taking both side square

n = 92.16 ≅ 92

Prepared by: Darshan jee

Lecturer: Govt. Khawaja Fareed Graduate College R.Y.K
(03081845584)