Scribd
Upload
10 views29 pages

Lecture 8 Regression Analyses

Uploaded by

Sayeeda Jahan
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
10 views29 pages

Lecture 8 Regression Analyses

Uploaded by

Sayeeda Jahan
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 29

Introduction to

Statistics

Sayeeda Jahan

Spring 2021
Chapter 14
Simple Linear Regression
n Simple Linear Regression Model
n Least Squares Method
n Coefficient of Determination
n Model Assumptions
n Testing for Significance
The Simple Linear Regression Model

n Simple Linear Regression Model


y = 0 + 1 x + 

n Simple Linear Regression Equation


E(y) = 0 + 1x

n Estimated Simple Linear Regression Equation


y^ = b0 + b1x
Least Squares Method

n Least Squares Criterion

min  (y i  y i ) 2

where:
yi = observed value of the dependent variable
for the ith observation
y^i = estimated value of the dependent variable
for the ith observation
The Least Squares Method

n Slope for the Estimated Regression Equation

 xi y i  (  xi  y i ) / n
b1  2 2
 xi  (  xi ) / n
n y-Intercept for the Estimated Regression Equation
_ _
b0 = y - b1x
where:
xi = value of independent variable for ith observation
yi = value of dependent variable for ith observation
_
x = mean value for independent variable
_
y = mean value for dependent variable
n = total number of observations
Example: Reed Auto Sales

n Simple Linear Regression


Reed Auto periodically has a special week-long sale.
As part of the advertising campaign Reed runs one or
more television commercials during the weekend
preceding the sale. Data from a sample of 5 previous
sales are shown below.
Number of TV Ads Number of Cars Sold
1 14
3 24
2 18
1 17
3 27
Example: Reed Auto Sales

n Slope for the Estimated Regression Equation


b1 = 220 - (10)(100)/5 = 5
24 - (10)2/5
n y-Intercept for the Estimated Regression Equation
b0 = 20 - 5(2) = 10
n Estimated Regression Equation
y^ = 10 + 5x
Using Excel to Develop a Scatter Diagram
and Compute the Estimated Regression Equation

n Formula Worksheet (showing data)

A B C D
1 Week TV Ads Cars Sold
2 1 1 14
3 2 3 24
4 3 2 18
5 4 1 17
6 5 3 27
7
Using Excel to Develop a Scatter Diagram
and Compute the Estimated Regression Equation

n Producing a Scatter Diagram


Step 1 Select cells B1:C6
Step 2 Select the Chart Wizard
Step 3 When the Chart Type dialog box appears:
Choose XY (Scatter) in the Chart type list
Choose Scatter from the Chart sub-type display
Select Next >
Step 4 When the Chart Source Data dialog box appears
Select Next >
… continued
Using Excel to Develop a Scatter Diagram
and Compute the Estimated Regression Equation

n Producing a Scatter Diagram


Step 5 When the Chart Options dialog box appears:
Select the Titles tab and then
Delete Cars Sold in the Chart title box
Enter TV Ads in the Value (X) axis box
Enter Cars Sold in the Value (Y) axis box
Select the Legend tab and then
Remove the check in the Show Legend box
Select Next >
… continued
Using Excel to Develop a Scatter Diagram
and Compute the Estimated Regression Equation

n Producing a Scatter Diagram


Step 6 When the Chart Location dialog box appears:
Specify the location for the new chart
Select Finish to display the scatter diagram
Using Excel to Develop a Scatter Diagram
and Compute the Estimated Regression Equation

n Adding the Trendline


Step 1 Position the mouse pointer over any data
point and right click to display the Chart
menu
Step 2 Select the Add Trendline option
Step 3 When the Add Trendline dialog box appears:
On the Type tab select Linear
On the Options tab select the Display
equation on chart box
Click OK
Using Excel to Develop a Scatter Diagram
and Compute the Estimated Regression Equation
n Scatter Diagram

30

25
20
Cars Sold

y = 5x + 10
15

10
5

0
0 1 2 3 4
TV Ads
The Coefficient of Determination

n Relationship Among SST, SSR, SSE


SST = SSR + SSE
 ( y i  y )   ( y^i  y )   ( y i  y^i )
2 2 2

n Coefficient of Determination
r2 = SSR/SST
where:
SST = total sum of squares
SSR = sum of squares due to regression
SSE = sum of squares due to error
Example: Reed Auto Sales

n Coefficient of Determination

r2 = SSR/SST = 100/114 = .8772

The regression relationship is very strong since


88% of the variation in number of cars sold can be
explained by the linear relationship between the
number of TV ads and the number of cars sold.
Using Excel to Compute
the Coefficient of Determination
n Producing R2
Step 1 Position the mouse pointer over any data
point in the scatter diagram and right click
Step 2 When the Chart menu appears:
Select the Add Trendline option
Step 3 When the Add Trendline dialog box appears:
On the Options tab, select the Display R-
squared value on chart box
Click OK
Using Excel to Compute
the Coefficient of Determination
n Value Worksheet (showing R2)

30

25
20
Cars Sold

y = 5x + 10
15
2
R = 0.8772
10
5

0
0 1 2 3 4
TV Ads
The Correlation Coefficient

n Sample Correlation Coefficient

rxy  (sign of b1 ) Coefficient of Determinat ion

rxy  (sign of b1 ) r 2

where:
b1 = the slope of the estimated regression
equation yˆ  b0  b1 x
Example: Reed Auto Sales

n Sample Correlation Coefficient

rxy  (sign of b1 ) r 2
The sign of b1 in the equation yˆ  10  5x is “+”.

rxy = + .8772
rxy = +.9366
Model Assumptions

n Assumptions About the Error Term 


• The error  is a random variable with mean of
zero.
• The variance of  , denoted by  2, is the same for
all values of the independent variable.
• The values of  are independent.
• The error  is a normally distributed random
variable.
Testing for Significance

n To test for a significant regression relationship, we


must conduct a hypothesis test to determine whether
the value of 1 is zero.
n Two tests are commonly used
• t Test
• F Test
n Both tests require an estimate of  2, the variance of 
in the regression model.
Testing for Significance

n An Estimate of  2
The mean square error (MSE) provides the estimate
of  2, and the notation s2 is also used.
s2 = MSE = SSE/(n-2)
where:

SSE   ( yi  yˆ i ) 2   ( yi  b0  b1 xi ) 2
Testing for Significance

n An Estimate of 
• To estimate  we take the square root of  2.
• The resulting s is called the standard error of the
estimate.
SSE
s  MSE 
n2
Testing for Significance: t Test

n Hypotheses
H 0 : 1 = 0
H a : 1 = 0
n Test Statistic
b1
t
sb 1
n Rejection Rule
Reject H0 if t < -t or t > t
where t is based on a t distribution with
n - 2 degrees of freedom.
Example: Reed Auto Sales

n t Test
• Hypotheses H 0 : 1 = 0
H a : 1 = 0
• Rejection Rule
For  = .05 and d.f. = 3, t.025 = 3.182
Reject H0 if t > 3.182
• Test Statistics
t = 5/1.08 = 4.63
• Conclusions
Reject H0
Testing for Significance: F Test

n Hypotheses
H 0 : 1 = 0
H a : 1 = 0
n Test Statistic
F = MSR/MSE
n Rejection Rule
Reject H0 if F > F
where F is based on an F distribution with 1 d.f. in
the numerator and n - 2 d.f. in the denominator.

© 2003 South-Western/Thomson Learning™


Slide 26
Example: Reed Auto Sales

n F Test
• Hypotheses H 0 : 1 = 0
H a : 1 = 0
• Rejection Rule
For  = .05 and d.f. = 1, 3: F.05 = 10.13
Reject H0 if F > 10.13.
• Test Statistic
F = MSR/MSE = 100/4.667 = 21.43
• Conclusion
We can reject H0.

© 2003 South-Western/Thomson Learning™


Slide 27
Some Cautions about the
Interpretation of Significance Tests

n Rejecting H0: 1 = 0 and concluding that the


relationship between x and y is significant does not
enable us to conclude that a cause-and-effect
relationship is present between x and y.
n Just because we are able to reject H0: 1 = 0 and
demonstrate statistical significance does not enable
us to conclude that there is a linear relationship
between x and y.
End of Chapter 14

You might also like