Slides Prepared by

JOHN S. LOUCKS
St. Edward’s University

© 2002 South-Western/Thomson Learning Slide

1
 Chapter 14
Simple Linear Regression
 Simple Linear Regression Model
 Least Squares Method
 Coefficient of Determination
 Model Assumptions
 Testing for Significance
 Using the Estimated Regression Equation
for Estimation and Prediction
 Computer Solution
 Residual Analysis: Validating Model Assumptions
 Residual Analysis: Outliers and Influential
Observations

Slide
2
 The Simple Linear Regression Model

 Simple Linear Regression Model

y =  0 +  1x + 
 Simple Linear Regression Equation
E(y) = 0 + 1x
 Estimated Simple Linear Regression Equation
^
y = b 0 + b 1x

Slide
3
 Least Squares Method

 Least Squares Criterion

min  (y i  y i ) 2

where:
yi = observed value of the dependent variable
for the ith observation
^
yi = estimated value of the dependent variable
for the ith observation

Slide
4
 The Least Squares Method

 Slope for the Estimated Regression Equation

 xi y i  (  xi  y i ) / n
b1  2 2
 xi  (  xi ) / n
 y-Intercept for the Estimated Regression Equation
_ _
b 0 = y - b 1x
where:
xi = value of independent variable for ith observation
y_i = value of dependent variable for ith observation
x_ = mean value for independent variable
y = mean value for dependent variable
n = total number of observations
Slide
5
 Example: Reed Auto Sales

 Simple Linear Regression

Reed Auto periodically has a special week-long sale.
As part of the advertising campaign Reed runs one or
more television commercials during the weekend
preceding the sale. Data from a sample of 5 previous
sales are shown below.
Number of TV Ads Number of Cars Sold
1 14
3 24
2 18
1 17
3 27

Slide
6
 Example: Reed Auto Sales

 Slope for the Estimated Regression Equation

b1 = 220 - (10)(100)/5 = 5
24 - (10)2/5
 y-Intercept for the Estimated Regression Equation
b0 = 20 - 5(2) = 10
 Estimated Regression Equation
^
y = 10 + 5x

Slide
7
 Example: Reed Auto Sales

 Scatter Diagram

30
25
20
Cars Sold

y = 5x + 10
15
10
5
0
0 1 2 3 4
TV Ads

Slide
8
 The Coefficient of Determination

 Relationship Among SST, SSR, SSE

SST = SSR + SSE
 ( y i  y )   ( y^i  y )   ( y i  y^i )
2 2 2

 Coefficient of Determination
r2 = SSR/SST
where:
SST = total sum of squares
SSR = sum of squares due to regression
SSE = sum of squares due to error

Slide
9
 Example: Reed Auto Sales

 Coefficient of Determination

r2 = SSR/SST = 100/114 = .8772

The regression relationship is very strong

since 88% of the variation in number of cars sold can
be explained by the linear relationship between the
number of TV ads and the number of cars sold.

Slide
10
 The Correlation Coefficient

 Sample Correlation Coefficient

rxy  (sign of b1 ) Coefficient of Determination

rxy  (sign of b1 ) r 2

where:
b1 = the slope of the estimated regression
equationyˆ  b0  b1 x

Slide
11
 Example: Reed Auto Sales

 Sample Correlation Coefficient

rxy  (sign of b1 ) r 2
The sign of b1 in the equation yˆ  10  5 x is “+”.

rxy = + .8772

rxy = +.9366

Slide
12
 Model Assumptions

 Assumptions About the Error Term 

• The error  is a random variable with mean of
zero.
• The variance of  , denoted by 2, is the same for
all values of the independent variable.
• The values of  are independent.
• The error  is a normally distributed random
variable.

Slide
13
 Testing for Significance

 To test for a significant regression relationship, we

must conduct a hypothesis test to determine whether
the value of 1 is zero.
 Two tests are commonly used
• t Test
• F Test
 Both tests require an estimate of 2, the variance of 
in the regression model.

Slide
14
 Testing for Significance

 An Estimate of  2
The mean square error (MSE) provides the estimate
of  2, and the notation s2 is also used.
s2 = MSE = SSE/(n-2)
where:
SSE   ( yi  yˆ i ) 2   ( yi  b0  b1 xi ) 2

Slide
15
 Testing for Significance

 An Estimate of 
• To estimate  we take the square root of 2.
• The resulting s is called the standard error of the
estimate.
SSE
s  MSE 
n2

Slide
16
 Testing for Significance: t Test

 Hypotheses
H 0:  1 = 0
H a:  1 = 0
 Test Statistic
b1
t
sb 1
 Rejection Rule
Reject H0 if t < -tor t > t

where t is based on a t distribution with

n - 2 degrees of freedom.

Slide
17
 Example: Reed Auto Sales

 t Test
• Hypotheses H 0:  1 = 0
H a:  1 = 0
• Rejection Rule
For  = .05 and d.f. = 3, t.025 = 3.182
Reject H0 if t > 3.182
• Test Statistics
t = 5/1.08 = 4.63
• Conclusions
Reject H0

Slide
18
 Confidence Interval for 1

 We can use a 95% confidence interval for 1 to test the

hypotheses just used in the t test.
 H0 is rejected if the hypothesized value of 1 is not
included in the confidence interval for 1.

Slide
19
 Confidence Interval for 1

 The form of a confidence interval for 1 is:

b1  t / 2 sb1

where b1 is the point estimate

t / 2 sb1 is the margin of error
t / 2 is the t value providing an area
of /2 in the upper tail of a
t distribution with n - 2 degrees
of freedom

Slide
20
 Example: Reed Auto Sales

 Rejection Rule
Reject H0 if 0 is not included in the confidence
interval for 1.
 95% Confidence Interval for 1
b1  t / 2 sb1
= 5 +/- 3.182(1.08) = 5 +/- 3.44
or 1.56 to 8.44
 Conclusion
Reject H0

Slide
21
 Testing for Significance: F Test

 Hypotheses
H 0:  1 = 0
H a:  1 = 0
 Test Statistic
F = MSR/MSE
 Rejection Rule
Reject H0 if F > F

where F is based on an F distribution with 1 d.f. in

the numerator and n - 2 d.f. in the denominator.

Slide
22
 Example: Reed Auto Sales

 F Test
• Hypotheses H 0:  1 = 0
H a:  1 = 0
• Rejection Rule
For  = .05 and d.f. = 1, 3: F.05 = 10.13
Reject H0 if F > 10.13.
• Test Statistic
F = MSR/MSE = 100/4.667 = 21.43
• Conclusion
We can reject H0.

Slide
23
 Some Cautions about the
Interpretation of Significance Tests
 Rejecting H0: 1 = 0 and concluding that the
relationship between x and y is significant does not
enable us to conclude that a cause-and-effect
relationship is present between x and y.
 Just because we are able to reject H0: 1 = 0 and
demonstrate statistical significance does not enable
us to conclude that there is a linear relationship
between x and y.

Slide
24
 Using the Estimated Regression Equation
for Estimation and Prediction
 Confidence Interval Estimate of E(yp)
y p  t  /2 s y p

 Prediction Interval Estimate of yp

yp + t/2 sind

where the confidence coefficient is 1 -  and

t/2 is based on a t distribution with n - 2 d.f.

Slide
25
 Example: Reed Auto Sales

 Point Estimation
If 3 TV ads are run prior to a sale, we expect the mean
number of cars sold to be:
y^ = 10 + 5(3) = 25 cars
 Confidence Interval for E(yp)
95% confidence interval estimate of the mean number
of cars sold when 3 TV ads are run is:
25 + 4.61 = 20.39 to 29.61 cars
 Prediction Interval for yp
95% prediction interval estimate of the number of
cars sold in one particular week when 3 TV ads are
run is: 25 + 8.28 = 16.72 to 33.28 cars
Slide
26
 Residual Analysis

 Residual for Observation i

yi – ^
yi

 Standardized Residual for Observation i

y i  y^i
syi  y^i

where: syi  y^i  s 1  hi

Slide
27
 Example: Reed Auto Sales

 Residuals
Observation Predicted Cars Sold Residuals
1 15 -1
2 25 -1
3 20 -2
4 15 2
5 25 2

Slide
28
 Example: Reed Auto Sales

 Residual Plot

TV Ads Residual Plot

3
2
Residuals

1
0
-1
-2
-3
0 1 2 3 4
TV Ads

Slide
29
 Residual Analysis

 Detecting Outliers
• An outlier is an observation that is unusual in
comparison with the other data.
• Minitab classifies an observation as an outlier if its
standardized residual value is < -2 or > +2.
• This standardized residual rule sometimes fails to
identify an unusually large observation as being
an outlier.
• This rule’s shortcoming can be circumvented by
using studentized deleted residuals.
• The |i th studentized deleted residual| will be
larger than the |i th standardized residual|.

Slide
30
End of Chapter 14

Slide
31