Republic of the Philippines

City of Olongapo
GORDON COLLEGE
Olongapo City Sports Complex, Donor St., East Tapinac, Olongapo City
www.gordoncollege.edu.ph

Quantitative Methods
Module 4: Regression Models

Name (LN,FN,MN): ________________________________ Program/Yr/Block: ___________

I. Introduction
Regression analysis is a very valuable tool for today’s manager. Regression has been used
to model such things as the relationship between level of education and income, the price of a
house and the square footage, and the sales volume for a company relative to the dollars spent
on advertising. When businesses are trying to decide which location is best for a new store or
branch office, regression models are often used. Cost estimation models are often regression
models. The applicability of regression analysis is virtually limitless.
There are generally two purposes for regression analysis. The first is to understand the
relationship between variables such as advertising expenditures and sales. The second purpose
is to predict the value of one variable based on the value of the other. Because of this, regression
is a very important forecasting technique.
In this course, we will introduce you to the following:

• Scatter Diagram
• Simple Linear Regression
• Measuring the Fit of the Regression Model
• Using Computer Software for Regression
• Assumptions of the Regression Model
• Testing the Model for Significance

We will use computers and spreadsheet models to demonstrate regression model.

Software such as QM for Windows, and Excel is often used for regression calculations. We will
rely on Excel for most of the calculations in the rest of this module.

II. Learning Objectives

After completing this module, you should be able to:

1. Identify variables and use them in a regression model.

2. Develop simple linear regression equations from sample data and interpret the slope and
intercept.
3. Compute the coefficient of determination and the coefficient of correlation and interpret
their meanings.
4. Interpret the F test in a linear regression model.
5. List the assumptions used in regression and use residual plots to identify problems.

Prepared by: Ms. Liezel T. Armada 1

 Republic of the Philippines
City of Olongapo
GORDON COLLEGE
Olongapo City Sports Complex, Donor St., East Tapinac, Olongapo City
www.gordoncollege.edu.ph

Topics and Key Concepts

In regression model, the variable to be predicted is called the dependent variable,
sometimes called the response variable. The value of this variable depends on the value of the
independent variable, sometimes called the explanatory or predictor variable.

A. Scatter Diagram

◼ Graphing is a helpful way to investigate the relationship between variables

◼ A scatter diagram or scatter plot is often used
◼ The independent variable is normally plotted on the X axis
◼ The dependent variable is normally plotted on the Y axis

Example: Triple A Construction

◼ Triple A Construction renovates old homes

◼ They have found that the dollar volume of renovation work is dependent on the
area payroll

Prepared by: Ms. Liezel T. Armada 2

 Republic of the Philippines
City of Olongapo
GORDON COLLEGE
Olongapo City Sports Complex, Donor St., East Tapinac, Olongapo City
www.gordoncollege.edu.ph

B. Simple Linear Regression

◼ Regression models are used to test if there is a relationship between variables (predict
sales based on payroll)
◼ There is some random error that cannot be predicted

Y =  0 + 1X + 

◼ True values for the slope and intercept are not known so they are estimated using
sample data
Yˆ = b0 + b1 X

Example: Triple A Construction

◼ Triple A Construction is trying to predict sales based on area payroll

Y = Sales
X = Area payroll

◼ The line chosen in scatter diagram is the one that minimizes the errors

Error = (Actual value) – (Predicted value)

e = Y − Yˆ

Prepared by: Ms. Liezel T. Armada 3

 Republic of the Philippines
City of Olongapo
GORDON COLLEGE
Olongapo City Sports Complex, Donor St., East Tapinac, Olongapo City
www.gordoncollege.edu.ph

Least Squared Regression

◼ Errors can be positive or negative so the average error could be zero even though
individual errors could be large.
◼ Least squares regression minimizes the sum of the squared errors.

Example: Triple A Construction

◼ For the simple linear regression model, the values of the intercept and slope can
be calculated using the formulas below

Yˆ = b0 + b1 X

Prepared by: Ms. Liezel T. Armada 4

 Republic of the Philippines
City of Olongapo
GORDON COLLEGE
Olongapo City Sports Complex, Donor St., East Tapinac, Olongapo City
www.gordoncollege.edu.ph

C. Measuring the Fit of the Regression Model

◼ Regression models can be developed for any variables X and Y

◼ How do we know the model is actually helpful in predicting Y based on X?
o We could just take the average error, but the positive and negative errors would
cancel each other out
◼ Three measures of variability are
o SST – Total variability about the mean
o SSE – Variability about the regression line
o SSR – Total variability that is explained by the model

o Sum of the squares total SST =  (Y − Y )2

o Sum of the squared error SSE =  e 2 =  (Y − Yˆ )2
o Sum of squares due to regression SSR =  (Yˆ − Y )2
o An important relationship SST = SSR + SSE
• SSR – explained variability
• SSE – unexplained variability

Prepared by: Ms. Liezel T. Armada 5

 Republic of the Philippines
City of Olongapo
GORDON COLLEGE
Olongapo City Sports Complex, Donor St., East Tapinac, Olongapo City
www.gordoncollege.edu.ph

Coefficient of Determination

◼ The proportion of the variability in Y explained by regression equation is called the

coefficient of determination
◼ The coefficient of determination is r2

SSR SSE
r2 = = 1−
SST SST
Example: Triple A Construction

15.625
r2 = = 0.6944 About 69% of the variability in Y is explained
22.5 by the equation based on payroll (X)

Correlation Coefficient

◼ The correlation coefficient is an expression of the strength of the linear relationship

◼ It will always be between +1 and –1
◼ The correlation coefficient is r
r = r2
Example: Triple A Construction

r = 0.6944 = 0.8333

Prepared by: Ms. Liezel T. Armada 6

 Republic of the Philippines
City of Olongapo
GORDON COLLEGE
Olongapo City Sports Complex, Donor St., East Tapinac, Olongapo City
www.gordoncollege.edu.ph

D. Using Computer Software for Regression

Prepared by: Ms. Liezel T. Armada 7

 Republic of the Philippines
City of Olongapo
GORDON COLLEGE
Olongapo City Sports Complex, Donor St., East Tapinac, Olongapo City
www.gordoncollege.edu.ph

Prepared by: Ms. Liezel T. Armada 8

 Republic of the Philippines
City of Olongapo
GORDON COLLEGE
Olongapo City Sports Complex, Donor St., East Tapinac, Olongapo City
www.gordoncollege.edu.ph

E. Assumptions of the Regression Model

◼ If we make certain assumptions about the errors in a regression model, we can

perform statistical tests to determine if the model is useful
1. Errors are independent
2. Errors are normally distributed
3. Errors have a mean of zero
4. Errors have a constant variance
◼ A plot of the residuals (errors) will often highlight any glaring violations of the
assumption
◼ A random plot of residuals

◼ Nonconstant error variance

o Errors increase as X increases, violating the constant variance assumption

◼ Nonlinear relationship
o Errors consistently increasing and then consistently decreasing indicate that
the model is not linear

Prepared by: Ms. Liezel T. Armada 9

 Republic of the Philippines
City of Olongapo
GORDON COLLEGE
Olongapo City Sports Complex, Donor St., East Tapinac, Olongapo City
www.gordoncollege.edu.ph

Estimating the Variance

◼ Errors are assumed to have a constant variance ( 2), but we usually don’t know this
◼ It can be estimated using the mean squared error (MSE), s2
SSE
s 2 = MSE =
n− k −1
where,
n = number of observations in the sample
k = number of independent variables

Example: For Triple A Construction

SSE 6.8750 6.8750

s 2 = MSE = = = = 1.7188
n − k − 1 6 − 1− 1 4

◼ We can estimate the standard deviation, s

◼ This is also called the standard error of the estimate or the standard deviation of
the regression
s = MSE = 1.7188 = 1.31

F. Testing the Model for Significance

◼ When the sample size is too small, you can get good values for MSE and r2 even if
there is no relationship between the variables
◼ Testing the model for significance helps determine if the values are meaningful\
◼ We do this by performing a statistical hypothesis test
◼ We start with the general linear model
Y =  0 + 1X + 
◼ If  1 = 0, the null hypothesis is that there is no relationship between X and Y
◼ The alternate hypothesis is that there is a linear relationship (1 ≠ 0)
◼ If the null hypothesis can be rejected, we have proven there is a relationship
◼ We use the F statistic for this test
◼ The F statistic is based on the MSE and MSR

SSR
MSR = where,
k k = number of independent variables in the model

◼ The F statistic is
This describes an F distribution with
MSR
F= degrees of freedom for the numerator = df1 = k
MSE degrees of freedom for the denominator = df2 = n – k – 1

Prepared by: Ms. Liezel T. Armada 10

 Republic of the Philippines
City of Olongapo
GORDON COLLEGE
Olongapo City Sports Complex, Donor St., East Tapinac, Olongapo City
www.gordoncollege.edu.ph

◼ If there is very little error, the MSE would be small and the F-statistic would be large
indicating the model is useful
◼ If the F-statistic is large, the significance level (p-value) will be low, indicating it is
unlikely this would have occurred by chance
◼ So, when the F-value is large, we can reject the null hypothesis and accept that there
is a linear relationship between X and Y and the values of the MSE and r2 are
meaningful

Steps in a Hypothesis Test

1. Specify null and alternative hypotheses

2. Select the level of significance (). Common values are 0.01 and 0.05
3. Calculate the value of the test statistic using the formula
MSR
F=
MSE
4. Make a decision using one of the following methods
a. Reject the null hypothesis if the test statistic is greater than the F-value from
the table. Otherwise, do not reject the null hypothesis:

Prepared by: Ms. Liezel T. Armada 11

 Republic of the Philippines
City of Olongapo
GORDON COLLEGE
Olongapo City Sports Complex, Donor St., East Tapinac, Olongapo City
www.gordoncollege.edu.ph

b. Reject the null hypothesis if the observed significance level, or p-value, is less
than the level of significance (). Otherwise, do not reject the null
hypothesis:

Example: Triple A Construction

Step 1: H0: 1 = 0 (no linear relationship between X and Y)

H1: 1 ≠ 0 (linear relationship exists between X and Y)

Step 2: Select  = 0.05

Step 3: Calculate the value of the test statistic

Step 4: Reject the null hypothesis if the test statistic is greater than the F-
value in the table.

The value of F associated with a 5% level of significance and with

degrees of freedom 1 and 4 is found in the table.

Prepared by: Ms. Liezel T. Armada 12

 Republic of the Philippines
City of Olongapo
GORDON COLLEGE
Olongapo City Sports Complex, Donor St., East Tapinac, Olongapo City
www.gordoncollege.edu.ph

r2 Coefficient of Determination

◼ The F-test determines whether or not there is a relationship between the

variables.
◼ r2 (coefficient of determination) is the best measure of the strength of the
prediction relationship between the X and Y variables.
o Values closer to 1 indicate a strong prediction relationship.
o Good regression models have a low significance level for the F-test
and high r2 value.

Coefficient Hypothesis

◼ Statistical tests of significance can be performed on the coefficients.

◼ The null hypothesis is that the coefficient of X (i.e., the slope of the line) is 0 i.e.,
X is not useful in predicting Y
◼ P values are the observed significance level and can be used to test the null
hypothesis.
o Values less than 5% negate the null hypothesis and indicate that X is
useful in predicting Y
◼ For a simple linear regression, the test of the regression coefficients gives the
same information as the F-test.

Prepared by: Ms. Liezel T. Armada 13

 Republic of the Philippines
City of Olongapo
GORDON COLLEGE
Olongapo City Sports Complex, Donor St., East Tapinac, Olongapo City
www.gordoncollege.edu.ph

Analysis of Variance (ANOVA) Table

◼ ANOVA shows the observed significance level (p-value) for the calculated F
value.
◼ This can be compared to the level of significance ( ) to make a decision.

◼ Because this probability is less than 0.05, we reject the null hypothesis of no
linear relationship and conclude there is a linear relationship between X and Y

Teaching and Learning Materials Resources

• PC Computer | Laptop | Android Phone
• GC LAMP
• Google Meet
• Facebook Messenger

Prepared by: Ms. Liezel T. Armada 14

 Republic of the Philippines
City of Olongapo
GORDON COLLEGE
Olongapo City Sports Complex, Donor St., East Tapinac, Olongapo City
www.gordoncollege.edu.ph

Learning Tasks
A. Explore

a. Use MS Excel with the data given below to find the regression model. Take a
screenshot of the output and paste it inside the box. (10pts)

Sales (Y) Advertising (X)

11 5
6 3
10 7
6 2
12 8

b. From the output, is there a relationship between X (advertising) and Y (sales)?

Explain. (10 pts)

__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________

Prepared by: Ms. Liezel T. Armada 15

 Republic of the Philippines
City of Olongapo
GORDON COLLEGE
Olongapo City Sports Complex, Donor St., East Tapinac, Olongapo City
www.gordoncollege.edu.ph

B. Explain

a. Why do we use regression analysis? (10 pts)

__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
_____________________________________________

C. Engage

a. Answer the following:

Judith Thompson runs a florist shop on the Gulf Coast of Texas, specializing
in floral arrangements for weddings and other special events. She advertises
weekly in the local newspapers and is considering increasing her advertising
budget. Before doing so, she decides to evaluate the past effectiveness of these
ads. Five weeks are sampled, and the advertising dollars and sales volume for each
of these is shown in the following table.

Sales ($1000) Advertising ($100)

11 5
6 3
10 7
6 2
12 8

1. Develop a regression equation that would help Judith evaluate her

advertising. (15 pts)
2. Find the coefficient of determination for this model. (15pts)

Prepared by: Ms. Liezel T. Armada 16

 Republic of the Philippines
City of Olongapo
GORDON COLLEGE
Olongapo City Sports Complex, Donor St., East Tapinac, Olongapo City
www.gordoncollege.edu.ph

References
1. Render, Stair, Hannah. 2012. “Quantitative Analysis for Management Global Edition
11th Edition. Retrieve from:
https://wps.pearsoned.co.uk/ema_ge_render_qam_11/202/51951/13299709.cw/-
/t/index.html
2. MooMooMath and Science (20218, March 7). How to draw line of best fit
(Scatterplot) [Video]. YouTube.
https://www.youtube.com/watch?v=l2BOZDosuIk&feature=emb_logo
3. Babbie, Earl R. The Practice of Social Research. 12th ed. Belmont, CA: Wadsworth
Cengage, 2010; Muijs, Daniel. Doing Quantitative Research in Education with SPSS.
2nd edition. London: SAGE Publications, 2010.

Prepared by: Ms. Liezel T. Armada 17