k 8

e e
W

Copyright© Dorling Kindersley India Pvt. Ltd

Correlation and Simple
Linear Regression
Analysis

Correlation and Simple Linear Regression Analysis 1

 Learning Objectives

Upon completion of this chapter, you will be able to:

Copyright© Dorling Kindersley India Pvt. Ltd

 Use the simple linear regression equation
 Compute the coefficient of correlation and understand its
interpretation.
 Understand the concept of measures of variation, coefficient of
determination, and standard error of the estimate
 Understand and use residual analysis for testing the
assumptions of regression
 Understand statistical inference about slope, correlation
coefficient of the regression model, and testing the overall
model

Correlation and Simple Linear Regression Analysis 2

 Measures of Association
 Measures of association are statistics for measuring the strength of
relationship between two variables.

Copyright© Dorling Kindersley India Pvt. Ltd

 Correlation measures the degree of association between two
variables.
 Karl Pearson’s coefficient of correlation is a quantitative measure
of the degree of relationship between two variables. Suppose these
variables are x and y, then Karl Pearson’s coefficient of correlation
is defined as

 The coefficient of correlation lies in between +1 and –1.

Correlation and Simple Linear Regression Analysis 3

Figure 15.1: Interpretation of correlation coefficient

Copyright© Dorling Kindersley India Pvt. Ltd

Correlation and Simple Linear Regression Analysis 4
 Example 15.1

Table 15.2 shows the sales revenue and advertisement expenses of a

Copyright© Dorling Kindersley India Pvt. Ltd

company for the past 10 months. Find the coefficient of correlation
between sales and advertisement.

Correlation and Simple Linear Regression Analysis 5

Table 15.3 : Calculation of correlation coefficient between sales
and advertisement

Copyright© Dorling Kindersley India Pvt. Ltd

Correlation and Simple Linear Regression Analysis 6
Figure 15.9: Five examples of correlation coefficient

Copyright© Dorling Kindersley India Pvt. Ltd

Correlation and Simple Linear Regression Analysis 7
 Using MS Excel, Minitab and SPSS for
Computing Correlation Coefficient
 Ch 15 Solved Examples\Excel\Ex 15.1.xls

Copyright© Dorling Kindersley India Pvt. Ltd

 Ch 15 Solved Examples\Minitab\Ex 15.1.MPJ
 Ch 15 Solved Examples\SPSS\Ex 15.1.sav
 Ch 15 Solved Examples\SPSS\Output Ex 15.1.spv

Correlation and Simple Linear Regression Analysis 8

 Introduction to Simple Linear Regression
 Regression analysis is the process of developing a statistical
model, which is used to predict the value of a dependent variable

Copyright© Dorling Kindersley India Pvt. Ltd

by at least one independent variable.
 In simple linear regression analysis, there are two types of
variables. The variable whose value is influenced or to be
predicted is called dependent variable and the variable which
influences the value or is used for prediction is called
independent variable.
 In regression analysis, independent variable is also known as
regressor or predictor, or explanatory while the dependent
variable is also known as regressed or explained variable. In a
simple linear regression analysis, only a straight line relationship
between two variables is examined.

Correlation and Simple Linear Regression Analysis 9

A Deterministic and Probabilistic Model

Copyright© Dorling Kindersley India Pvt. Ltd

ε is the error of the regression line in fitting the points of the
regression equation. If a point is on the regression line, the
corresponding value of ε is equal to zero. If the point is not on the
regression line, the value of ε measures the error.
It can be noticed that in the deterministic model, all the points are
assumed to be on the regression line and hence, in all the cases
random error ε is equal to zero. Probabilistic model includes an error
term which allows the value of y to vary for any given value of x.

Correlation and Simple Linear Regression Analysis 10

 Assumptions

Copyright© Dorling Kindersley India Pvt. Ltd

• Assumption #1: Your dependent variable should be
measured at the continuous level (i.e., it is either
an interval or ratio variable). Examples of continuous
variables include revision time (measured in hours),
intelligence (measured using IQ score), exam
performance (measured from 0 to 100), weight
(measured in kg), and so forth.
• Assumption #2: Your independent variable should
also be measured at the continuous level (i.e., it is
either an interval or ratio variable). See the bullet
above for examples of continuous variables.

Correlation and Simple Linear Regression Analysis 11

 Assumptions

Copyright© Dorling Kindersley India Pvt. Ltd

• There needs to be a linear relationship between
the two variables. Whilst there are a number of
ways to check whether a linear relationship exists
between your two variables, we suggest creating a
scatterplot using SPSS Statistics where you can
plot the dependent variable against your
independent variable and then visually inspect the
scatterplot to check for linearity. Your scatterplot
may look something like one of the following:

Correlation and Simple Linear Regression Analysis 12

 Copyright© Dorling Kindersley India Pvt. Ltd
Correlation and Simple Linear Regression Analysis 13
 Copyright© Dorling Kindersley India Pvt. Ltd
• There should be no significant outliers. An outlier
is an observed data point that has a dependent
variable value that is very different to the value
predicted by the regression equation. As such, an
outlier will be a point on a scatterplot that is
(vertically) far away from the regression line
indicating that it has a large residual, as highlighted
below:

Correlation and Simple Linear Regression Analysis 14

 Copyright© Dorling Kindersley India Pvt. Ltd
Correlation and Simple Linear Regression Analysis 15
 Copyright© Dorling Kindersley India Pvt. Ltd
This table provides the R and R2 values. The R value represents the simple
correlation and is 0.873 (the "R" Column), which indicates a high degree of
correlation. The R2 value (the "R Square" column) indicates how much of the total
variation in the dependent variable, Price, can be explained by the independent
variable, Income. In this case, 76.2% can be explained, which is very large.

Correlation and Simple Linear Regression Analysis 16

 Copyright© Dorling Kindersley India Pvt. Ltd
This table indicates that the regression model predicts the dependent variable significantly
well. How do we know this? Look at the "Regression" row and go to the "Sig." column. This
indicates the statistical significance of the regression model that was run. Here, p < 0.0005,
which is less than 0.05, and indicates that, overall, the regression model statistically
significantly predicts the outcome variable (i.e., it is a good fit for the data).

Correlation and Simple Linear Regression Analysis 17

 Copyright© Dorling Kindersley India Pvt. Ltd
Correlation and Simple Linear Regression Analysis 18
Figure 15.10: Error in simple regression

Copyright© Dorling Kindersley India Pvt. Ltd

Correlation and Simple Linear Regression Analysis 19
Figure 15.11: Summary of the estimation process for simple linear regression.

Copyright© Dorling Kindersley India Pvt. Ltd

Correlation and Simple Linear Regression Analysis 20
 Example 15.2
A cable wire company has spent heavily on advertisements. The sales
and advertisement expenses (in thousand rupees) for the 12 randomly
selected months are given in Table 14.2. Develop a regression model

Copyright© Dorling Kindersley India Pvt. Ltd

to predict the impact of advertisement on sales.

Correlation and Simple Linear Regression Analysis 21