CORRELATION & REGRESSION

Dr. Ashutosh Srivastava

Geoinformatics Department
 CORRELATION

Finding the relationship between two quantitative variables

without being able to infer causal relationships

Correlation is a statistical technique used to determine the

degree to which two variables are related
 Scatter diagram

• Rectangular coordinate
• Two quantitative variables
• One variable is called independent (X) and the
second is called dependent (Y)
• Points are not joined Y
* *
• No frequency table *
X
 Example

Wt. 67 69 85 83 74 81 97 92 114 85
(kg)
SBP 120 125 140 160 130 180 150 140 200 130
mmHg)
 Wt. 67 69 85 83 74 81 97 92 114 85
(kg)
SBP 120 125 140 160 130 180 150 140 200 130
(mmHg)

SCATTER DIAGRAM OF WEIGHT AND SYSTOLIC

BLOOD PRESSURE
Scatter diagram of weight and systolic blood pressure
 SCATTER PLOTS

The pattern of data is indicative of the type of

relationship between your two variables:
➢ positive relationship

➢ negative relationship
➢ no relationship
POSITIVE RELATIONSHIP
 18

16

14

12
Height in CM

10

0
0 10 20 30 40 50 60 70 80 90
Age in Weeks
 NEGATIVE RELATIONSHIP

Reliability

Age of Car
NO RELATION
 CORRELATION COEFFICIENT

Statistic showing the degree of relation between two variables

SIMPLE CORRELATION COEFFICIENT (R)

➢ It is also called Pearson's correlation or product moment correlation

coefficient.

➢ It measures the nature and strength between two variables of

the quantitative type.
➢The sign of r denotes the nature of
association

➢while the value of r denotes the strength of

association.
➢ If the sign is +ve this means the relation is direct (an increase
in one variable is associated with an increase in the
other variable and a decrease in one variable is associated with
a decrease in the other variable).

➢ While if the sign is -ve this means an inverse or indirect

relationship (which means an increase in one variable is
associated with a decrease in the other).
 ➢ The value of r ranges between ( -1) and ( +1)
➢ The value of r denotes the strength of the association
as illustrated by the following diagram.

strong intermediate weak weak intermediate strong

-1 -0.75 -0.25 0 0.25 0.75 1

indirect Direct
perfect perfect
correlation correlation
no relation
➢ Ifr = Zero this means no association or correlation
between the two variables.

➢ If 0 < r < 0.25 = weak correlation.

➢ If 0.25 ≤ r < 0.75 = intermediate correlation.

➢ If 0.75 ≤ r < 1 = strong correlation.

➢ If r = l = perfect correlation.
How to compute the simple correlation coefficient (r)

 x y
 xy −
n
r=
 ( x) 2
 ( y) 2

 x −
2 .  y −
2 
 n  n 
  
 EXAMPLE:
A sample of 6 children was selected, data about their age in
years and weight in kilograms was recorded as shown in the
following table . It is required to find the correlation
between age and weight.

serial Age (years) Weight (Kg)

No
1 7 12
2 6 8
3 8 12
4 5 10
5 6 11
6 9 13
These 2 variables are of the quantitative type, one
variable (Age) is called the independent and
denoted as (X) variable and the other (weight)
is called the dependent and denoted as (Y)
variables to find the relation between age and
weight compute the simple correlation coefficient
using the following formula:

 xy −  x y
r = n

  x2 −
(  x) 2

.  y 2 −
(  y) 2


 n  n 
  
 Age Weight
Serial
(years) (Kg) xy X2 Y2
n.
(x) (y)
1 7 12 84 49 144
2 6 8 48 36 64
3 8 12 96 64 144
4 5 10 50 25 100
5 6 11 66 36 121
6 9 13 117 81 169
Total ∑x= ∑y= ∑xy= ∑x2= ∑y2=
41 66 461 291 742
 41  66
461 −
r= 6
 (41)2   (66)2 
291 − .742 − 
 6  6 

r = 0.759
strong direct correlation
EXAMPLE: RELATIONSHIP BETWEEN ANXIETY
AND TEST SCORES
Anxiety Test score X2 Y2 XY
(X) (Y)
10 2 100 4 20
8 3 64 9 24
2 9 4 81 18
1 7 1 49 7
5 6 25 36 30
6 5 36 25 30
∑X = 32 ∑Y = 32 ∑X2 = 230 ∑Y2 = 204 ∑XY=129
CALCULATING CORRELATION COEFFICIENT

(6)(129) − (32)(32) 774 − 1024

r= = = −.94
(6(230) − 32 )(6(204) − 32 )
2 2
(356)(200)

r = - 0.94

Indirect strong correlation

SPEARMAN RANK CORRELATION COEFFICIENT (RS)

➢ It is a non-parametric measure of correlation.

➢ Thisprocedure makes use of the two sets of ranks that
may be assigned to the sample values of x and Y.
➢ Spearman Rank correlation coefficient could be
computed in the following cases:
➢ Both variables are quantitative.
➢ Both variables are qualitative ordinal.
➢ One variable is quantitative and the other is
qualitative ordinal.
PROCEDURE:

➢ Rank the values of X from 1 to n where n is the numbers of pairs of

values of X and Y in the sample.

➢ Rank the values of Y from 1 to n.

➢ Compute the value of di for each pair of observation by subtracting the

rank of Yi from the rank of Xi

➢ Square each di and compute ∑di2 which is the sum of the squared
values.
5. Apply the following formula

6 (di) 2
rs = 1 −
n(n 2 − 1)

➢ The value of rs denotes the magnitude and

nature of association giving the same
interpretation as simple r.
 EXAMPLE
In a study of the relationship between level education
and income the following data was obtained. Find the
relationship between them and comment.

sample level education Income

numbers (X) (Y)
A Preparatory. 25
B Primary. 10
C University. 8
D secondary 10
E secondary 15
F illiterate 50
G University. 60
ANSWER:
Rank Rank di di2
(X) (Y) X Y
A Preparatory 25 5 3 2 4

B Primary. 10 6 5.5 0.5 0.25

C University. 8 1.5 7 -5.5 30.25
D secondary 10 3.5 5.5 -2 4
E secondary 15 3.5 4 -0.5 0.25
F illiterate 50 7 2 5 25
G university. 60 1.5 1 0.5 0.25

∑ di2=64
 6  64
rs = 1 − = −0.1
7(48)

Comment:
There is an indirect weak correlation between level of education
and income.
EXERCISE
 REGRESSION ANALYSES

➢ Regression:technique concerned with predicting

some variables by knowing others

➢ The process of predicting variable Y using variable

X
 REGRESSION

➢ Uses a variable (x) to predict some outcome variable (y)

➢ Tells you how values in y change as a function of changes

in values of x
 CORRELATION AND REGRESSION

➢ Correlation
describes the strength of a linear relationship
between two variables

➢ Linear means “straight line”

➢ Regression tells us how to draw the straight line described

by the correlation
 REGRESSION
➢ Calculates the “best-fit” line for a certain set of data
The regression line makes the sum of the squares of the
residuals smaller than for any other line
Regression minimizes residuals
By using the least squares method (a procedure that
minimizes the vertical deviations of plotted points
surrounding a straight line) we are
able to construct a best fitting straight line to the scatter
diagram points and then formulate a regression equation in
the form of:

ŷ = a + bX

 x y
 xy −
ŷ = y + b(x − x) bb1 = n
(  x) 2
 x 2
−
n
 REGRESSION EQUATION

➢ Regression equation describes the regression line mathematically

 Intercept
 Slope
SBP(mmHg)
220

200

180

160

140

120

100

80
Wt (kg)
60 70 80 90 100 110 120
LINEAR EQUATIONS

ŷ = a + bX
Y = bX + a
Change
b = Slope in Y
Change in X
a = Y-intercept
X
HOURS STUDYING AND GRADES
REGRESSING GRADES ON HOURS


Linear Regression


90.00 Final grade in c ourse = 59.95 + 3 .17 * study
R-Squa re = 0.88
Final grade in course




 
80.00

 
70.00

2.00 4.00 6.00 8.00 10.00

Number of hours spent studying

Predicted final grade in class =

59.95 + 3.17*(number of hours you study per week)
 Predicted final grade in class = 59.95 + 3.17*(hours of study)

PREDICT THE FINAL GRADE OF…

 Someone who studies for 12 hours

 Final grade = 59.95 + (3.17*12)
 Final grade = 97.99

 Someone who studies for 1 hour:

 Final grade = 59.95 + (3.17*1)
 Final grade = 63.12
 EXERCISE

A sample of 6 persons was selected the value of their age ( x variable)

and their weight is demonstrated in the following table. Find the
regression equation and what is the predicted weight when age is 8.5
years.
Serial no. Age (x) Weight (y)
1 7 12
2 6 8
3 8 12
4 5 10
5 6 11
6 9 13
ANSWER

Serial no. Age (x) Weight (y) xy X2 Y2

1 7 12 84 49 144
2 6 8 48 36 64
3 8 12 96 64 144
4 5 10 50 25 100
5 6 11 66 36 121
6 9 13 117 81 169

Total 41 66 461 291 742

 41 66
x= = 6.83 y= = 11
6 6

41  66
461 −
b= 6 = 0.92
2
(41)
291 −
6

Regression equation

ŷ (x) = 11 + 0.9(x − 6.83)

 ŷ (x) = 4.675 + 0.92x

ŷ (8.5) = 4.675 + 0.92 * 8.5 = 12.50Kg

ŷ (7.5) = 4.675 + 0.92 * 7.5 = 11.58Kg

 12.6

Weight (in Kg)

12.4
12.2
12
11.8
11.6
11.4
7 7.5 8 8.5 9
Age (in years)

we create a regression line by plotting two estimated

values for y against their X component, then extending
the line right and left.
 EXERCISE 2
Age B.P Age B.P
(x) (y) (x) (y)
20 120 46 128
43 128 53 136
The following are the age
(in years) and systolic 63 141 60 146
blood pressure of 20 26 126 20 124
apparently healthy 53 134 63 143
adults. 31 128 43 130
58 136 26 124
46 132 19 121
58 140 31 126
70 144 23 123
➢Find the correlation between age and
blood pressure using simple and
Spearman's correlation coefficients,
and comment.

➢Find the regression equation?

➢What is the predicted blood pressure

for a man aging 25 years?
Serial x y xy x2
1 20 120 2400 400
2 43 128 5504 1849
3 63 141 8883 3969
4 26 126 3276 676
5 53 134 7102 2809
6 31 128 3968 961
7 58 136 7888 3364
8 46 132 6072 2116
9 58 140 8120 3364
10 70 144 10080 4900
Serial x y xy x2
11 46 128 5888 2116
12 53 136 7208 2809
13 60 146 8760 3600
14 20 124 2480 400
15 63 143 9009 3969
16 43 130 5590 1849
17 26 124 3224 676
18 19 121 2299 361
19 31 126 3906 961
20 23 123 2829 529
Total 852 2630 114486 41678
  x y
 xy −
n 114486 −
852  2630
b1 = = 20 = 0.4547
( x) 2 852 2

 −
x 2

n
41678 −
20

ŷ =112.13 + 0.4547 x

for age 25
B.P = 112.13 + 0.4547 * 25=123.49 = 123.5 mm hg
 MULTIPLE REGRESSION

Multiple regression analysis is a straightforward

extension of simple regression analysis which allows
more than one independent variable.
VARIANCE

Variance is the average squared deviation from the mean of

a set of data. It is used to find the standard deviation.
VARIANCE

Mean
VARIANCE

2
-
VARIANCE

2
-

2
-
VARIANCE

1
---------------- ……… 2 2
+ - + - + ………
No. of Data
Points
VARIANCE FORMULA
 STANDARD DEVIATION

[ standard deviation = square root of the variance ]

VARIANCE (2D)
VARIANCE (2D)
VARIANCE (2D)
VARIANCE (2D)
VARIANCE (2D)

Variance doesn’t explore

relationship between variables
COVARIANCE
COVARIANCE
COVARIANCE
COVARIANCE
COVARIANCE
COVARIANCE

Positive
Relation
COVARIANCE
COVARIANCE
COVARIANCE
COVARIANCE
COVARIANCE

Negative
Relation
COVARIANCE
COVARIANCE

No
Relation
COVARIANCE MATRIX
COVARIANCE MATRIX
COVARIANCE MATRIX

➢ Covariance is a real symmetric positive semi-definite matrix.

❖ All eigenvalues must be real
❖ Eigenvectors corresponding to different eigenvalues are
orthogonal
❖ All eigenvalues are greater than or equal to zero
❖ Covariance matrix can be diagonalized,
i.e. Cov = PDPT
 COVARIANCE AND CORRELATION
➢ Covariance measures how two variables move with respect to each other and is an
extension of the concept of variance (which tells about how a single variable
varies). It can take any value from -∞ to +∞.
➢ Correlation is a step ahead of covariance as it quantifies the relationship between
two random variables. In simple terms, it is a unit measure of how these variables
change with respect to each other (normalized covariance value).

Covariance is a measure indicating the Correlation is a statistical measure that

extent to which two random variables indicates how strongly two variables
change in tandem. are related.

Lie between -∞ and +∞ Lie between -1 and +1

Unit Unit free measure