1.017/1.010 Class 19 Analysis of Variance: Concepts and Definitions

1.017/1.
010 Class 19
Analysis of Variance
Concepts and Definitions
Objective: Identify factors responsible for variability in observed data
Specify one or more factors that could account for variability (e.g.
location, time, etc.). Each factor is associated with a particular set of
populations or treatments (e.g. particular sampling stations, sampling
days, etc.). One-way analysis of variance (ANOVA) considers only a
single factor.
Suppose a random sample [xi1, xi2, ..., xiJ] is obtained for treatment i.
There are i =1,..., I treatments (e.g. each treatment may correspond to a
different sampling location).
Arrange data in a table/array -- rows are treatments, columns are

replicates:
[x11, x12, ..., x1J]

[x21, x22, ..., x2J]
.
.
[xI 1, xI 2 , ..., xIJ]
Here we assume each treatment has same number of replicates J. The

ANOVA procedure may be generalized to allow different number of
replicates for each treatment.
Each random sample has a CDF Fxi(xi). The different Fxi(xi) are assumed
identical except for their means, which may differ. Classical ANOVA also
assumes that all data are normally distributed.
Each random variable xij is decomposed into several parts, as specified by

the following one-factor model:
xij = µi+ eij =µ + ai + eij
µi = E[xij] is unknown mean of xi (for all j).

µ = unknown grand mean (average ofµi's ).
ai = µi -µ = unknown deviation of treatment mean from grand
mean (often called an effect)
eij = random residual for treatment i, replicate j
E[eij ] = 0, Var[eij ] = σ2, for all i, j
1
Objective is to estimate/test values of ai's, which are the unknown
distributional parameters of the Fxi(xi)'s.
Formulating the Problem as a Hypothesis Test
If the factor does not affect variability in the data then all ai's = 0. Use
hypothesis test:
H0: a1 = a2 = .... = aI = 0
It is better to test all ai simultaneously than individually or in pairs. Test

that sum-of-squared ai's = 0.
I
H0 : ∑ ai2 = 0
i =1
Derive a test statistic based on sums-of-squares of data.
Sums-of-Squares Computations
Define the sample treatment and grand means:
J
1
m xi =
J ∑ xij = xi.
j =1
I J
1
mx =
IJ ∑ ∑ xij = x..
i =1 j =1
The total sum-of-squares SST measures variability of xij around mx:
I J
SST = ∑∑ ( x ij − m x ) 2
i =1 j =1
I J I J
= ∑∑ 2
( x ij − m xi ) + ∑∑ (m xi − m x ) 2
i =1 j =1 i =1 j =1
= SSE + SSTr
SST can be divided into error sum-of-squares SSE and treatment sum-
of-squares SSTr.
SSE measures variability of xij around mxi, within treatments:
2
I J
SSE = ∑∑ ( x ij − m xi ) 2
i =1 j =1
SSTr measures variability of mxi around mx, across treatments:
I J
SSTr = ∑∑ (m xi − m x ) 2
i =1 j =1
Error and treatment mean squared values:
SSE
MSE =
I ( J − 1)
SSTr
MSTr =
I −1
E[ MSE ] = σ 2
I
J
E[ MSTr ] = σ 2 +
I −1 ∑ ai2
i =1
MSE is an unbiased estimate of σ2, even if ai's are not zero.

MSTr is an unbiased estimate of σ2, only if all ai's are zero.
Test Statistic
Use ratio MSTr /MSE as a test statistic:
MSTr
F ( MSE , MSTr ) =
MSE
When H0 is true and xij's are normally distributed this statistic follows F
distribution with νT r = I - 1 and νE = I(J-1) degrees of freedom. Check
normality by plotting (xij - mxi) with normplot.
One-sided rejection region (rejects only if MSTr is large):
R0 : F ( MSE, MSTr ) ≥ FF-1,ν ,ν [α ]

Tr E
3
One-sided p-value:
p = 1 − FF , ν ,ν [F ( MSE, MStr )]
Tr E
Unbalanced ANOVA problems with different sample sizes for different

treatments can be handled by modifying formulas slightly (see Devore,
Section 10.3).
Single Factor ANOVA Tables
Above calculations are typically summarized in an ANOVA table:
Source SS df MS F p
Treatments SSTr νTr = I-1 MSTr = F = MSTr/MSE p=
SSTr/νTr 1-FF ,νTr,νE(F )
MSE =
Error SSE νE = I(J-1)
SSE/νE
MST =
Total SST νT = IJ-1
SST/νT
Example -- Effect of Season on Oxygen Level
Consider following set of dissolved oxygen concentration data ( xij )

obtained in 4 different seasons/treatments (rows), 6 replicates per season
(columns):
5.62 6.12 6.62 6.21 7.08 5.36

7.70 8.31 8.80 8.24 7.87 7.44
2.52 5.44 4.94 2.99 4.39 4.44
6.77 6.65 6.01 6.26 7.09 6.05
Use a single factor ANOVA to determine if season has a significant impact

on oxygen variability.
The MATLAB anova1 function derives the error and treatment sums of
squares and computes p value. When using anova1 be sure to
transpose the data array (MATLAB requires treatments in columns and
replicates in rows).
Results are presented in this standard single factor ANOVA table:
4
Source SS df MS=SS/df F p
Treatments 47.1642 3 15.7214 29.8 1.4E-7
Error 10.5518 20 0.5276
Total 57.716 23
The very low p value indicates that seasonality is highly significant in this
case. Note that MSTr, which depends on the ai's, is much larger than MSE
F CDF, νTr = 3, νE = 5
Copyright 2003 Massachusetts Institute of Technology

Last modified Oct. 8, 2003

1.017/1.010 Class 19 Analysis of Variance: Concepts and Definitions

Uploaded by

Copyright:

Available Formats

1.017/1.010 Class 19 Analysis of Variance: Concepts and Definitions

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

1.017/1.010 Class 19 Analysis of Variance: Concepts and Definitions

Uploaded by

Copyright:

Available Formats

1.017/1.

Concepts and Definitions

Objective: Identify factors responsible for variability in observed data

Arrange data in a table/array -- rows are treatments, columns are

[x11, x12, ..., x1J]

Here we assume each treatment has same number of replicates J. The

Each random variable xij is decomposed into several parts, as specified by

xij = µi+ eij =µ + ai + eij

µi = E[xij] is unknown mean of xi (for all j).

Formulating the Problem as a Hypothesis Test

It is better to test all ai simultaneously than individually or in pairs. Test

Derive a test statistic based on sums-of-squares of data.

Define the sample treatment and grand means:

The total sum-of-squares SST measures variability of xij around mx:

SSE measures variability of xij around mxi, within treatments:

SSTr measures variability of mxi around mx, across treatments:

Error and treatment mean squared values:

MSE is an unbiased estimate of σ2, even if ai's are not zero.

Use ratio MSTr /MSE as a test statistic:

One-sided rejection region (rejects only if MSTr is large):

R0 : F ( MSE, MSTr ) ≥ FF-1,ν ,ν [α ]

Unbalanced ANOVA problems with different sample sizes for different

Single Factor ANOVA Tables

Above calculations are typically summarized in an ANOVA table:

Example -- Effect of Season on Oxygen Level

Consider following set of dissolved oxygen concentration data ( xij )

5.62 6.12 6.62 6.21 7.08 5.36

Use a single factor ANOVA to determine if season has a significant impact

Results are presented in this standard single factor ANOVA table:

Copyright 2003 Massachusetts Institute of Technology

You might also like