Quantitative Methods

Analysis of Variance
and Goodness of Fit

Introduction to Linear Regression

Analysis of Variance (ANOVA)

Yi
Y n
Yˆi  b̂0  b̂1X i SSE   (Yi  Yˆi )2
ˆi
Y i1
SST
n
SSR   (Yˆi  Y )2
i 1
Y
SSE + SSR = SST
n
SST   (Yi  Y )2
b̂0 i 1

© Kaplan, Inc.
X 1

1
 Analysis of Variance and Goodness of Fit

Mean Square Regression and Error

SSR
mean square regression (MSR) =
k
SSE
mean square error (MSE) =
(n – k – 1)
For simple regression, k = 1 (number of independent variables)

SSR SSE
MSR = MSE =
1 (n – 2)

2
© Kaplan, Inc.

Analysis of Variance and Goodness of Fit

ANOVA Table for k = 1

Source of Df Sum of Squares Mean Square
Variation
Regression 1 SSR SSR/1 = MSR
(explained)
Error n-2 SSE SSE/(n-2) = MSE
(unexplained)
Total n-1 SST
These columns sum vertically.

3
© Kaplan, Inc.

2
 Analysis of Variance and Goodness of Fit

Using the ANOVA Table for ABC Data: Example

Source of Df Sum of Squares Mean Square

Variation
Regression ? 0.0076 ?
(explained)
Error ?-2 0.0406 ?
(unexplained)
Total ?-1 ?

ABC data: N = 3 years of monthly return observations 4

© Kaplan, Inc.

Analysis of Variance and Goodness of Fit

ANOVA Table for ABC: Solution

Source of Df Sum of Mean Square

Variation Squares
Regression 1 0.0076 SSR 0.0076
(explained)
Error 34 0.0406 SSE 0.0012
(unexplained)
Total 35 0.0482 SST

5
© Kaplan, Inc.

3
 Analysis of Variance and Goodness of Fit

The Coefficient of Determination (R2)

R2 measures the percentage of total variation in Y variable explained

by the variation in X (SSR / SST).

An R2 of 0.63 means the variation in X can explain 63% of

the variation in Y.

For simple regression R 2 = correlation2xy

6
© Kaplan, Inc.

Analysis of Variance and Goodness of Fit

R2 for the ABC Regression: Example

Coefficient of determination (R2) = SSR / SST
Total variation (SST) = explained (SSR) + unexplained (SSE)

Given SSR = 0.0076, SSE = 0.0406

SSR 0.0076
  0.1577  15.8%
SST 0.0406  0.0076

7 -1
© Kaplan, Inc.

4
 Analysis of Variance and Goodness of Fit

Standard Error of the Estimate (SEE)

Standard error of the estimate (SEE) measures the accuracy of
predicted values from the regression equation.

SEE is the standard deviation of the error term.

SSE
S EE = = MSE
(n  2)

Lower the SEE, the better the model fit

8
© Kaplan, Inc.

Analysis of Variance and Goodness of Fit

Standard Error of the Estimate (SEE)

Calculate the SEE based on the ANOVA table for ABC:

0.0406
SEE = = 0.0012  0.035
34

9 -1
© Kaplan, Inc.

5
 Analysis of Variance and Goodness of Fit

The F-Statistic
Tests whether the independent variables explain variation in the
dependent variable (i.e., test of overall model significance)
H0: slope coefficient = 0
HA: slope coefficient ≠ 0
 One-tailed test (significance is right-hand tail probability)

 Reject H0 if F-statistic exceeds critical value

 Critical F determined by two degrees of freedom

(numerator and denominator)

10
© Kaplan, Inc.

Analysis of Variance and Goodness of Fit

The F-Statistic

SSR
k MSR
F= =
SSE MSE
(n – k – 1)

with k = 1

numerator df = 1
denominator df = n –2

11
© Kaplan, Inc.

6
 Analysis of Variance and Goodness of Fit

The F-Statistic
Perform F-test for 36 observations at 5% significance:
SSR = 0.0076, SSE = 0.0406 H0: b1 = 0 vs. Ha: b1 ≠ 0
MSR = SSR / k = 0.0076
MSE = SSE / (n – 2) = 0.0406 / (36 – 2) = 0.0012

MSR 0.0076 Critical F-value for 1 and

F   6.333 34 df at 5% is 4.13
MSE 0.0012

Test stat 6.33 > critical stat 4.13; therefore, reject H0

12 -6
© Kaplan, Inc.

Analysis of Variance and Goodness of Fit

Regression Coefficient t-Test

A test of H0 : b1  0 vs. Ha : b1  0 is a t-test with (n – 2) degrees of

freedom for a simple regression.

bˆ 1 – b1 estimate  hypothesized bˆ 1 – 0 slope

tb1 =   
sbˆ standard error sbˆ standard error
1 1

SEE
Standard error of the slope coefficient = sb̂1 
  Xi – X 
n 2

i=1
13
© Kaplan, Inc.

7
 Analysis of Variance and Goodness of Fit

Regression Coefficient t-Test

Estimated slope for ABC (b1) = 0.64 with an SE = 0.26
Assuming n = 36, determine if slope is significant at 5% level:

Degrees of freedom (df)

0.64 – 0
tb1 =  2.46 are 36 – 2 = 34
0.26

Critical two-tailed t-stat (34 df Computed stat 2.46 > critical

and 5% significance) is 2.03 stat 2.03; therefore, null is
rejected
14 -4
© Kaplan, Inc.