Simple Linear Regression

www.kent.ac.uk/student-learning-advisory-service
1
 Linear Regression
• We will introduce simple linear regression, in particular we will:

• Learn when we can use simple linear regression

• Learn the basic workings involved in simple linear regression

• Linear Regression in SPSS

• This presentation is intended for students in initial stages of

Statistics. No previous knowledge is required.

2
 Linear Regression
• Regression is used to study the relationship
between two variables.
• How a change in one variable (e.g., someone’s
exercise habits) can predict the outcome of another
variable (e.g., general health).

• We can use simple regression if both the

dependent variable (DV) and the independent
variable (IV) are numerical.

• If the DV is numerical but the IV is categorical, it is

best to use ANOVA.
3
 Examples
The following are situations where we can use
regression:

• Testing if IQ affects income (IQ is the IV and income

is the DV).
• Testing if study time affects grades (hours of study
time is the IV and average grade is the DV).
• Testing if exercise affects blood pressure (hours of
exercise is the IV and blood pressure is the DV).

4
 Displaying the data
When both the DV and IV are numerical, we can
represent data in the form of a scatterplot.

5
 Displaying the data
It is important to perform a scatterplot because it
helps us to see if the relationship is linear.

In this example, the

relationship between
body fat % and chance
of heart failure is not
linear and hence it is
not sensible to use
linear regression.
 Linear Regression
• Straight line prediction model.

• As an independent variable
changes, what happens to the
dependent variable? I.e., as an
independent variable goes up
and down, does the dependent
variable go up and down?

• They could either move in the

same direction (positive
relationship) or opposite
direction (negative relationship)

7
 Linear Regression
• Straight line prediction model.

• As an independent variable
changes, what happens to the
dependent variable? I.e., as an
independent variable goes up
and down, does the dependent
variable go up and down?

• They could either move in the

same direction (positive
relationship) or opposite
direction (negative relationship)

8
Linear Regression

9
 Linear Regression
y = B0 + B1 * X + E

B1
grades

B0

study time 10
 Linear Regression
y = B0 - B1 * X + E

11
 Assumptions of regression
• The errors E are normally distributed.

This can be tested by plotting an histogram of the residuals of

the regression and checking that they all have a bell shape.

Alternatively, you could use the Shapiro-Wilk test for

normality.

12
 Assumptions of regression
• There are no clear outliers
This can be checked by performing the scatterplot. The
outliers (circled in red in the figure) can simply be removed
from the analysis.

13
 Hypothesis testing
Regression tests the null hypothesis:

H0 : There is no effect of X on Y.

versus the alternative hypothesis:

H1 : There is an effect of X on Y.

If the null hypothesis is rejected, we reject the hypothesis that there is no

relationship and hence we conclude that there is a significant relationship
between X and Y.

14
 Hypothesis testing

How do we know if we should reject the null

hypothesis?

We perform regression in SPSS and look at the p-value

of the coefficient b.

If the p-value is less than 0.05, we reject the null

hypothesis (the variable is significant), otherwise, we
do not reject the null hypothesis (the variable is not
significant).
15
 Linear Regression in SPSS
1. Go to: www.openup.co.uk/spss
2. Click the link to access the support website
3. Under ‘Data Files & Exercises’ click ‘Data Files’
4. Download the survey.zip file

Then follow along!

To do this, you will need to install SPSS:

https://www.kent.ac.uk/software/spss-
statistics

16
 Linear Regression in SPSS
Our variables of interest:
IV - Total mastery (tmast): total score on the Mastery
Scale. High scores indicate higher levels of perceived
control over events and circumstances.
DV - Total perceived stress (tpstress): total score on the
Perceived Stress Scale. High scores indicate high levels of
stress.

Question:
How well does our measure of control (i.e., mastery)
predict perceived stress?
17
 Regression in SPSS
After producing a scatterplot to check for linearity…

1. Click on Analyze, Regression, Linear.

2. Click on your dependent variable (tpstress). Move

this into the box marked Dependent by clicking
on the arrow button.

3. Click on your independent variable (tmast). Move

this into the box marked Independent(s).

4. Click on the OK button.

https://statistics.laerd.com/spss-tutorials/linear-regression-using-spss-statistics.php 18
 Regression in SPSS
• In the Linear Regression box, transfer the DV
(tpstress) to the Dependent box and the IV (tmast)
to the Independent(s): box

Then click on OK

https://statistics.laerd.com/spss-tutorials/linear-regression-using-spss-statistics.php 19
 Regression in SPSS
Interpreting the output:
1. The first table that we’re interested in is the Model Summary.
• The R value represents the simple correlation. This indicates a strong
degree of correlation between our two variables.
• The R2 value indicates how much of the total variation in the dependent
variable (perceived stress) can be explained by the independent variable
(mastery). In this case, 37.3% can be explained.

https://statistics.laerd.com/spss-tutorials/linear-regression-using-spss-statistics.php 20
 Regression in SPSS
Interpreting the output:
2. The next table is the ANOVA table, which shows us how well the
regression equation fits the data (i.e., predicts the dependent
variable).
• The regression predicts the dependent variable significantly well (p < .001).

https://statistics.laerd.com/spss-tutorials/linear-regression-using-spss-statistics.php 21
 Regression in SPSS
Interpreting the output:
3. The Coefficients table gives us the information that we need to
predict stress from mastery, as well as determine whether mastery
contributes statistically significantly to the model.

Y = B0 + B1 * X

Total perceived stress = 46.32 + (-.9*Total Mastery)

22
 Regression in SPSS
Example results write-up:
A simple linear regression was carried out to test if total mastery
significantly predicted total perceived stress. The results of the regression
indicated that the model explained 37.3% of the variance and that the
model was significant F (1, 431) = 257.63, p < .001. It was found that total
mastery significantly predicted total perceived stress (B1 = -.9, p < .001).
The final predictive model was:
total perceived stress = 46.32 + (-9*total mastery)

https://www.open.ac.uk/socialsciences/spsstutorial/files/tutorials/simple-linear-regression.pdf 23
 Regression in SPSS
Results are
usually
rounded to two
decimal places

24
 Multiple Regression
• Simple linear regression (i.e., with one IV) allows us
to study the relationship between two variables only.

• But in reality, we do not believe that only a single

variable explains all the variation of the dependent
variable.

• For example, in the scenario of IQ and income, we

do not expect IQ only to explain income, but that
there are also other variables, such as years in
education, to explain income.
25
Upcoming workshops…

maths/stats appointments
available!

www.kent.ac.uk/student-learning-advisory-service

26