Statistics for

Business and Economics

8th Global Edition

Chapter 11

Two Variable Regression Analysis

Copyright © 2013 Pearson Education Ch. 11-1

 Chapter Goals
After completing this chapter, you should be
able to:
 Explain the simple linear regression model
 Obtain and interpret the simple linear regression
equation for a set of data
 Describe R2 as a measure of explanatory power of the
regression model
 Understand the assumptions behind regression
analysis
 Explain measures of variation and determine whether
the independent variable is significant
Copyright © 2013 Pearson Education Ch. 11-2
 Chapter Goals
(continued)
After completing this chapter, you should be
able to:
 Calculate and interpret confidence intervals for the
regression coefficients
 Use a regression equation for prediction
 Form forecast intervals around an estimated Y value
for a given X
 Use graphical analysis to recognize potential problems
in regression analysis
 Explain the correlation coefficient and perform a
hypothesis test for zero population correlation
Copyright © 2013 Pearson Education Ch. 11-3
11.1
Overview of Linear Models

 An equation can be fit to show the best linear

relationship between two variables:

Y = β0 + β1X

Where Y is the dependent variable and

X is the independent variable
β0 is the Y-intercept
β1 is the slope
Copyright © 2013 Pearson Education Ch. 11-4
 Least Squares Regression
 Estimates for coefficients β0 and β1 are found
using a Least Squares Regression technique
 The least-squares regression line, based on sample
data, is
yˆ  b0  b1x
 Where b1 is the slope of the line and b0 is the y-
intercept:

Cov(x, y)  s y 
b1  2
 r   b0  y  b1x
sx  sx 
Copyright © 2013 Pearson Education Ch. 11-5
 Introduction to
Regression Analysis

 Regression analysis is used to:

 Predict the value of a dependent variable based on
the value of at least one independent variable
 Explain the impact of changes in an independent
variable on the dependent variable
Dependent variable: the variable we wish to explain
(also called the endogenous variable)
Independent variable: the variable used to explain
the dependent variable
(also called the exogenous variable)

Copyright © 2013 Pearson Education Ch. 11-6

11.2
Linear Regression Model

 The relationship between X and Y is

described by a linear function
 Changes in Y are assumed to be influenced
by changes in X
 Linear regression population equation model

y i  β0  β1x i  ε i

 Where 0 and 1 are the population model

coefficients and  is a random error term.

Copyright © 2013 Pearson Education Ch. 11-7

 Simple Linear Regression
Model
The population regression model:
Population Random
Population Independent Error
Slope
Y intercept Variable term
Coefficient
Dependent
Variable

y i  β0  β1x i  ε i
Linear component Random Error
component

Copyright © 2013 Pearson Education Ch. 11-8

 Linear Regression Assumptions

 The true relationship form is linear (Y is a linear function

of X, plus random error)
 The error terms, εi are independent of the x values
 The error terms are random variables with mean 0 and
constant variance, σ2
(the uniform variance property is called homoscedasticity)
2
E[ε i ]  0 and E[ε i ]  σ 2 for (i  1, , n)
 The random error terms, εi, are not correlated with one
another, so that
E[ε iε j ]  0 for all i  j

Copyright © 2013 Pearson Education Ch. 11-9

 Simple Linear Regression
Model
(continued)

Y Yi  β0  β1Xi  ε i
Observed Value
of Y for xi

εi Slope = β1
Predicted Value Random Error
of Y for xi
for this Xi value

Intercept = β0

xi X
Copyright © 2013 Pearson Education Ch. 11-10
 Simple Linear Regression
Equation
The simple linear regression equation provides an
estimate of the population regression line
Estimated Estimate of Estimate of the
(or predicted) the regression regression slope
y value for intercept
observation i
Value of x for

yˆ i  b0  b1x i observation i

The individual random error terms ei have a mean of zero

ei  ( y i - yˆ i )  y i - (b0  b1x i )

Copyright © 2013 Pearson Education Ch. 11-11

11.3
Least Squares
Coefficient Estimators

 b0 and b1 are obtained by finding the values

of b0 and b1 that minimize the sum of the
squared residuals (errors), SSE:
n
min SSE  min  ei2
i 1

 min  (y i yˆ i )2

 min  [y i  (b0  b1x i )]2

Differential calculus is used to obtain the
coefficient estimators b0 and b1 that minimize SSE
Copyright © 2013 Pearson Education Ch. 11-12
 Least Squares
Coefficient Estimators
(continued)

 The slope coefficient estimator is

n

 (x  x)(y  y)
i1
i i
Cov(x, y) sy
b1  n
 2
r
2 sx sx
 i
(x
i1
 x)

 And the constant or y-intercept is

b 0  y  b1x

 The regression line always goes through the mean x, y

Copyright © 2013 Pearson Education Ch. 11-13

 Computer Computation of
Regression Coefficients

 The coefficients b0 and b1 , and other

regression results in this chapter, will be
found using a computer
 Hand calculations are tedious
 Statistical routines are built into Excel
 Other statistical analysis software can be used

Copyright © 2013 Pearson Education Ch. 11-14

 Interpretation of the
Slope and the Intercept

 b0 is the estimated average value of y

when the value of x is zero (if x = 0 is
in the range of observed x values)

 b1 is the estimated change in the

average value of y as a result of a one-
unit change in x

Copyright © 2013 Pearson Education Ch. 11-15

 Simple Linear Regression
Example

 A real estate agent wishes to examine the

relationship between the selling price of a home
and its size (measured in square feet)

 A random sample of 10 houses is selected

 Dependent variable (Y) = house price in $1000s

 Independent variable (X) = square feet

Copyright © 2013 Pearson Education Ch. 11-16

 Sample Data for
House Price Model
House Price in $1000s Square Feet
(Y) (X)
245 1400
312 1600
279 1700
308 1875
199 1100
219 1550
405 2350
324 2450
319 1425
255 1700

Copyright © 2013 Pearson Education Ch. 11-17

 Graphical Presentation

 House price model: scatter plot

450
400
House Price ($1000s)

350
300
250
200
150
100
50
0
0 500 1000 1500 2000 2500 3000
Square Feet

Copyright © 2013 Pearson Education Ch. 11-18

 Excel Output
(continued)
Regression Statistics
Multiple R 0.76211 The regression equation is:
R Square 0.58082
Adjusted R Square 0.52842 house price  98.24833  0.10977 (square feet)
Standard Error 41.33032
Observations 10

ANOVA
df SS MS F Significance F
Regression 1 18934.9348 18934.9348 11.0848 0.01039
Residual 8 13665.5652 1708.1957
Total 9 32600.5000

Coefficients Standard Error t Stat P-value Lower 95% Upper 95%

Intercept 98.24833 58.03348 1.69296 0.12892 -35.57720 232.07386
Square Feet 0.10977 0.03297 3.32938 0.01039 0.03374 0.18580

Copyright © 2013 Pearson Education Ch. 11-19

 Graphical Presentation
 House price model: scatter plot and
regression line
450
400
House Price ($1000s)

350 Slope
300
250
= 0.10977
200
150
100
50
Intercept 0
= 98.248 0 500 1000 1500 2000 2500 3000
Square Feet

house price  98.24833  0.10977 (square feet)

Copyright © 2013 Pearson Education Ch. 11-20

 Interpretation of the
Intercept, b0

house price  98.24833  0.10977 (square feet)

 b0 is the estimated average value of Y when the

value of X is zero (if X = 0 is in the range of
observed X values)
 Here, no houses had 0 square feet, so b0 = 98.24833
just indicates that, for houses within the range of
sizes observed, $98,248.33 is the portion of the
house price not explained by square feet

Copyright © 2013 Pearson Education Ch. 11-21

 Interpretation of the
Slope Coefficient, b1

house price  98.24833  0.10977 (square feet)

 b1 measures the estimated change in the

average value of Y as a result of a one-
unit change in X
 Here, b1 = .10977 tells us that the average value of a
house increases by .10977($1000) = $109.77, on
average, for each additional one square foot of size

Copyright © 2013 Pearson Education Ch. 11-22

 11.4
Explanatory Power of a
Linear Regression Equation

 Total variation is made up of two parts:

SST  SSR  SSE
Total Sum of Regression Sum Error (residual)
Squares of Squares Sum of Squares

SST   (y i  y)2 SSR   (yˆ i  y)2 SSE   (y i  yˆ i )2

where:
y = Average value of the dependent variable
yi = Observed values of the dependent variable
ŷi = Predicted value of y for the given xi value
Copyright © 2013 Pearson Education Ch. 11-23
 Analysis of Variance

 SST = total sum of squares

 Measures the variation of the yi values around their
mean, y
 SSR = regression sum of squares
 Explained variation attributable to the linear
relationship between x and y
 SSE = error sum of squares
 Variation attributable to factors other than the linear
relationship between x and y

Copyright © 2013 Pearson Education Ch. 11-24

 Analysis of Variance
(continued)
Y Unexplained
yi variation 
 2 y
SSE = (yi - yi )
_
SST = (yi - y)2
Explained

 _2
y variation
_ SSR = (yi - y) _
y y

xi X
Copyright © 2013 Pearson Education Ch. 11-25
 Coefficient of Determination, R2
 The coefficient of determination is the portion
of the total variation in the dependent variable
that is explained by variation in the
independent variable
 The coefficient of determination is also called
R-squared and is denoted as R2
SSR regression sum of squares
2
R  
SST total sum of squares

2
note:
0 R 1
Copyright © 2013 Pearson Education Ch. 11-26
 Examples of Approximate
r2 Values

Y
r2 = 1

Perfect linear relationship

between X and Y:
X
r2 = 1
Y 100% of the variation in Y is
explained by variation in X

X
r2 =1
Copyright © 2013 Pearson Education Ch. 11-27
 Examples of Approximate
r2 Values
Y
0 < r2 < 1

Weaker linear relationships

between X and Y:
X
Some but not all of the
Y
variation in Y is explained
by variation in X

X
Copyright © 2013 Pearson Education Ch. 11-28
 Examples of Approximate
r2 Values

r2 = 0
Y
No linear relationship
between X and Y:

The value of Y does not

X depend on X. (None of the
r2 = 0
variation in Y is explained
by variation in X)

Copyright © 2013 Pearson Education Ch. 11-29

 Excel Output
Regression Statistics 2SSR 18934.9348
R    0.58082
Multiple R 0.76211 SST 32600.5000
R Square 0.58082
Adjusted R Square 0.52842 58.08% of the variation in
Standard Error 41.33032 house prices is explained by
Observations 10
variation in square feet
ANOVA
df SS MS F Significance F
Regression 1 18934.9348 18934.9348 11.0848 0.01039
Residual 8 13665.5652 1708.1957
Total 9 32600.5000

Coefficients Standard Error t Stat P-value Lower 95% Upper 95%

Intercept 98.24833 58.03348 1.69296 0.12892 -35.57720 232.07386
Square Feet 0.10977 0.03297 3.32938 0.01039 0.03374 0.18580

Copyright © 2013 Pearson Education Ch. 11-30

11.6
Prediction

 The regression equation can be used to

predict a value for y, given a particular x

 For a specified value, xn+1 , the predicted

value is
yˆ n1  b0  b1x n1

Copyright © 2013 Pearson Education Ch. 11-31

 Predictions Using
Regression Analysis
Predict the price for a house
with 2000 square feet:

house price  98.25  0.1098 (sq.ft.)

 98.25  0.1098(2000)

 317.85
The predicted price for a house with 2000
square feet is 317.85($1,000s) = $317,850
Copyright © 2013 Pearson Education Ch. 11-32
 Relevant Data Range
 When using a regression model for prediction,
only predict within the relevant range of data

Relevant data range

450
400
House Price ($1000s)

350
300
250
200
150 Risky to try to
100
extrapolate far
50
0
beyond the range
0 500 1000 1500 2000 2500 3000 of observed x
Square Feet values
Copyright © 2013 Pearson Education Ch. 11-33