Statistics 514: Experiments with Random Effects

Lecture 13: Experiments with Random Effects

Montgomery, Chapter 12 or 13

Spring , 2008
Page 1
Statistics 514: Experiments with Random Effects

Example 1

A textile company weaves a fabric on a large number of looms. It would like the looms to
be homogeneous so that it obtains a fabric of uniform strength. A process engineer
suspects that, in addition to the usual variation in strength within samples of fabric from the
same loom, there may also be significant variations in strength between looms. To
investigate this, she selects four looms at random and makes four strength determinations
on the fabric manufactured on each loom. The layout and data are given in the following.

Observations

looms 1 2 3 4

1 98 97 99 96

2 91 90 93 92

3 96 95 97 95

4 95 96 99 98

Spring , 2008
Page 2
Statistics 514: Experiments with Random Effects

Random Effects vs Fixed Effects

• Consider factor with numerous possible levels

• Want to draw inference on population of levels
• Not concerned with any specific levels
• Example of difference (1=fixed, 2=random)
1. Compare reading ability of 10 2nd grade classes in NY
Select a = 10 specific classes of interest. Randomly choose n students from
each classroom. Want to compare τi (class-specific effects).
2. Study the variability among all 2nd grade classes in NY
Randomly choose a = 10 classes from large number of classes. Randomly
2
choose n students from each classroom. Want to assess στ (class to class
variability).

• Inference broader in random effects case

• Levels chosen randomly → inference on population

Spring , 2008
Page 3
Statistics 514: Experiments with Random Effects

Random Effects Model (CRD)

• Similar model (as in the fixed case) with different assumptions


 i = 1, 2 . . . a
yij = µ + τi + ǫij
 j = 1, 2, . . . ni

µ - grand mean
τi - ith treatment effect (random)
ǫij ∼ N(0, σ 2 )
P
• Instead of τi = 0, assume
τi ∼ N(0, στ2 )
{τi } and {ǫij } independent

• Var(yij ) = στ2 + σ 2

στ2 and σ 2 are called variance components

Spring , 2008
Page 4
Statistics 514: Experiments with Random Effects

Statistical Analysis

• The basic hypotheses are:

H0 : στ2 = 0 vs. H1 : στ2 > 0

• Same ANOVA table (as before)

Source SS DF MS F0
MStr
Between SStr a−1 MStr F0 = MSE

Within SSE N −a MSE

Total SST = SStr + SSE N −1
2
– E(MSE )=σ
2
– E(MStr )=σ + nστ2
• Under H0 , F0 ∼ Fa−1,N −a
• Same test as before
• Conclusions, however, pertain to entire population

Spring , 2008
Page 5
Statistics 514: Experiments with Random Effects

Estimation

• Usually interested in estimating variances

• Use mean squares (known as ANOVA method)

σ̂ 2 = MSE
σ̂τ2 = (MStr − MSE )/n
If unbalanced, replace n with
Pa
n2
P 
1 a
n0 = a−1 i=1 ni − i=1
Pa i
i=1 ni

• Estimate of στ2 can be negative

- Supports H0 ? Use zero as estimate?

- Validity of model? Nonlinear?

- Other approaches (MLE, Bayesian with nonnegative prior)

Spring , 2008
Page 6
Statistics 514: Experiments with Random Effects

Confidence intervals

• σ 2 : Same as fixed case

(N − a)MSE 2
∼ χN −a
σ2

(N − a)MSE 2 (N − a)MSE
≤ σ ≤
χ2α/2,N −a χ21−α/2,N −a

• στ2 : Linear combination of χ2

(a − 1)MStr 2
∼ χa−1
σ 2 + nστ2

σ̂τ2 = (M Str − M SE )/n, so

σ 2 + nστ2 2 σ2
σ̂τ2 ≈ χa−1 − χ2N −a
n(a − 1) n(N − a)
No closed form expression for this distribution. Can use approximation

Spring , 2008
Page 7
Statistics 514: Experiments with Random Effects

Recall
M Str /(σ 2 + nστ2 )
∼ Fa−1,N −a
M SE /σ 2
• For στ2 /σ 2 :
στ2
L≤ 2 ≤U
σ
   
MStr MStr
L= MSE Fα/2,a−1,N −a
− 1 /n and U = MSE F − 1 /n
1−α/2,a−1,N −a

• For στ2 /(σ 2 + στ2 ) :

L στ2 U
≤ 2 ≤ , or
L+1 σ + στ2 U+1

F0 − Fα/2,a−1,N −a στ2 F0 − F1−α/2,a−1,N −a

≤ 2 ≤
F0 + (n − 1)Fα/2,a−1,N −a σ + στ2 F0 + (n − 1)F1−α/2,a−1,N −a

Spring , 2008
Page 8
Statistics 514: Experiments with Random Effects

Loom Experiment (continued)

Source of Sum of Degrees of Mean F0

Variation Squares Freedom Square

Between 89.19 3 29.73 15.68

Within 22.75 12 1.90

Total 111.94 15

Highly significant result (F.05,3,12 = 3.49)

σ̂τ2 = (29.73 − 1.90)/4 = 6.98

78.6% (=6.98/(6.98+1.90)) is attributable to loom differences

Time to improve consistency of the looms

Spring , 2008
Page 9
Statistics 514: Experiments with Random Effects

Loom Experiment

Confidence Intervals

• 95% CI for σ 2
SSE SSE
≤ σ2 ≤ = c(22.75/23.34, 22.75/4.40)
χ2.025,12 χ2.975,12
= (0.97, 5.17)

• 95% CI for στ2 /(στ2 + σ 2 )

 
15.68 − 4.47 15.68 − (1/14.34)
, = (0.385, 0.982)
15.68 + (4 − 1)4.47 15.68 + (4 − 1)(1/14.34)

F0.025,3,12 = 4.47, F.975,3,12 = 1/14.34

using property that

F1−α/2,v1 ,v2 = 1/Fα/2,v2 ,v1

• the variability between looms is not negligible.

Spring , 2008
Page 10
Statistics 514: Experiments with Random Effects

Using SAS
options nocenter ps=35 ls=72;

data example;
input loom strength;
datalines;
1 98
1 97
1 99
1 96
.
4 98
;
proc glm;
class loom;
model strength=loom;
random loom;
output out=diag r=res p=pred;

proc plot;

Spring , 2008
Page 11
Statistics 514: Experiments with Random Effects

plot res*pred;

proc varcomp method = type1;

class loom;
model strength = loom;

proc mixed CL;

class loom;
model strength = ;
random loom;
run;

Spring , 2008
Page 12
Statistics 514: Experiments with Random Effects

SAS Output
Dependent Variable: strength

Sum of
Source DF Squares Mean Square F Value Pr > F
Model 3 89.1875000 29.7291667 15.68 0.0002
Error 12 22.7500000 1.8958333
Corrected Total 15 111.9375000

Source DF Type I SS Mean Square F Value

loom 3 89.18750000 29.72916667 15.68

Source Type III Expected Mean Square

loom Var(Error) + 4 Var(loom)
--------------------------------------------------------------------
Variance Components Estimation Procedure
Dependent Variable: strength
Sum of
Source DF Squares Mean Square
loom 3 89.187500 29.729167

Spring , 2008
Page 13
Statistics 514: Experiments with Random Effects

Error 12 22.750000 1.895833

Corrected Total 15 111.937500 .

Source Expected Mean Square

loom Var(Error) + 4 Var(loom)
Error Var(Error)

Variance Component Estimate

Var(loom) 6.95833
Var(Error) 1.89583

Spring , 2008
Page 14
Statistics 514: Experiments with Random Effects

SAS Output (Continued)

The Mixed Procedure

Iteration History
Iteration Evaluations -2 Res Log Like Criterion
0 1 75.48910190
1 1 63.19303249 0.00000000
Convergence criteria met.

Covariance Parameter Estimates

Cov Parm Estimate Alpha Lower Upper

loom 6.9583 0.05 2.1157 129.97
Residual 1.8958 0.05 0.9749 5.1660

Fit Statistics
-2 Res Log Likelihood 63.2
AIC (smaller is better) 67.2
AICC (smaller is better) 68.2
BIC (smaller is better) 66.0

Spring , 2008
Page 15
Statistics 514: Experiments with Random Effects

A Measurement Systems Capability Study

A typical gauge R&R experiment is shown below. An instrument or gauge is used to

measure a critical dimension of certain part. Twenty parts have been selected from the
production process, and three randomly selected operators measure each part twice with
this gauge. The order in which the measurements are made is completely randomized, so
this is a two-factor factorial experiment with design factors parts and operators, with two
replications. Both parts and operators are random factors.

Parts Operator 1 Operator 2 Operator 3

1 21 20 20 20 19 21
2 24 23 24 24 23 24
3 20 21 19 21 20 22
. . . . . . .
19 25 26 25 24 25 25
20 19 19 18 17 19 17

Variance components equation: σy2 = στ2 + σβ2 + στ2β + σ 2

Total variability=Parts + Operators + Interaction + Experimental Error
=Parts + Reproducibility + Repeatability

Spring , 2008
Page 16
Statistics 514: Experiments with Random Effects

Statistical Model with Two Random Factors


 i = 1, 2, . . . , a


yijk = µ + τi + βj + (τ β)ij + ǫijk j = 1, 2, . . . , b


k = 1, 2, . . . , n


τi ∼ N(0, στ2 ) βj ∼ N(0, σβ2 ) (τ β)ij ∼ N(0, στ2β )

• Var(yijk ) = σ 2 + στ2 + σβ2 + στ2β
• Expected MS’s similar to one-factor random model
2 2
E(MSE )=σ ; E(MSA ) = σ + bnστ2 + nστ2β
2
E(MSB ) = σ + anσβ2 + nστ2β ; E(MSAB ) = σ 2 + nστ2β
• EMS determine what MS to use in denominator
H0 : στ2 = 0 → MSA /MSAB
H0 : σβ2 = 0 → MSB /MSAB
H0 : στ2β = 0 → MSAB /MSE

Spring , 2008
Page 17
Statistics 514: Experiments with Random Effects

Estimating Variance Components

• Using ANOVA method

σ̂ 2 = MSE
σ̂τ2 = (MSA − MSAB )/bn
σ̂β2 = (MSB − MSAB )/an
σ̂τ2β = (MSAB − MSE )/n
• Sometimes results in negative estimates
• Proc Varcomp and Proc Mixed compute estimates
• Can use different estimation procedures
ANOVA method - Method = type1

RMLE method - Method = reml (default)

• Proc Mixed
Variance component estimates

Hypothesis tests and confidence intervals

Spring , 2008
Page 18
Statistics 514: Experiments with Random Effects

Gauge Capability Example in Text 13-2

options nocenter ls=75;

data randr;
input part operator resp @@;
cards;
1 1 21 1 1 20 1 2 20 1 2 20 1 3 19 1 3 21
2 1 24 2 1 23 2 2 24 2 2 24 2 3 23 2 3 24
3 1 20 3 1 21 3 2 19 3 2 21 3 3 20 3 3 22
4 1 27 4 1 27 4 2 28 4 2 26 4 3 27 4 3 28
..........................................
20 1 19 20 1 19 20 2 18 20 2 17 20 3 19 20 3 17
;

proc glm;
class operator part;
model resp=operator|part;
random operator part operator*part / test;
test H=operator E=operator*part;
test H=part E=operator*part;

Spring , 2008
Page 19
Statistics 514: Experiments with Random Effects

proc mixed cl maxiter=20 covtest method=type1;

class operator part;
model resp = ;
random operator part operator*part;

proc mixed cl maxiter=20 covtest;

class operator part;
model resp = ;
random operator part operator*part;
run;
quit;

Spring , 2008
Page 20
Statistics 514: Experiments with Random Effects

Dependent Variable: resp

Sum of
Source DF Squares Mean Square F Value Pr > F
Model 59 1215.091667 20.594774 20.77 <.0001
Error 60 59.500000 0.991667
CorreTotal 119 1274.591667

Source DF Type III SS Mean Square F Value Pr > F

operator 2 2.616667 1.308333 1.32 0.2750
part 19 1185.425000 62.390789 62.92 <.0001
operator*part 38 27.050000 0.711842 0.72 0.8614

Source Type III Expected Mean Square

operator Var(Error) + 2 Var(operator*part) + 40 Var(operator)
part Var(Error) + 2 Var(operator*part) + 6 Var(part)
operator*part Var(Error) + 2 Var(operator*part)

Tests of Hypotheses Using the Type III

MS for operator*part as an Error Term

Source DF Type III SS Mean Square F Value Pr > F

Spring , 2008
Page 21
Statistics 514: Experiments with Random Effects

operator 2 2.616667 1.308333 1.84 0.1730

part 19 1185.425000 62.390789 87.65 <.0001

Tests of Hypotheses for Random Model Analysis of Variance

Dependent Variable: resp

Source DF Type III SS Mean Square F Value Pr > F

operator 2 2.616667 1.308333 1.84 0.1730
part 19 1185.425000 62.390789 87.65 <.0001
Error 38 27.050000 0.711842
Error: MS(operator*part)

Source DF Type III SS Mean Square F Value Pr > F

operator*part 38 27.050000 0.711842 0.72 0.8614
Error: MS(Error) 60 59.500000 0.991667

Spring , 2008
Page 22
Statistics 514: Experiments with Random Effects

The Mixed Procedure

Type 1 Analysis of Variance

Sum of
Source DF Squares Mean Square
operator 2 2.616667 1.308333
part 19 1185.425000 62.390789
operator*part 38 27.050000 0.711842
Residual 60 59.500000 0.991667

Type 1 Analysis of Variance

Error
Source Expected Mean Square Error Term DF
operator Var(Residual) + 2 Var(operator*part) MS(operator*part) 38
+ 40 Var(operator)
part Var(Residual) + 2 Var(operator*part) MS(operator*part) 38
+ 6 Var(part)
operator*part Var(Residual) + 2 Var(operator*part) MS(Residual) 60
Residual Var(Residual) .

Spring , 2008
Page 23
Statistics 514: Experiments with Random Effects

Source F Value Pr > F

operator 1.84 0.1730
part 87.65 <.0001
operator*part 0.72 0.8614

Covariance Parameter Estimates

Standard Z
Cov Parm Estimate Error Value Pr Z Alpha Lower Upper
operator 0.0149 0.0330 0.45 0.6510 0.05 -0.0497 0.0795
part 10.2798 3.3738 3.05 0.0023 0.05 3.6673 16.8924
operator*part -0.1399 0.1219 -1.15 0.2511 0.05 -0.3789 0.0990
Residual 0.9917 0.1811 5.48 <.0001 0.05 0.7143 1.4698

Spring , 2008
Page 24
Statistics 514: Experiments with Random Effects

The Mixed Procedure

Estimation Method REML
Iteration History
Iteration Evaluations -2 Res Log Like Criterion
0 1 624.67452320
1 3 409.39453674 0.00003340
2 1 409.39128078 0.00000004
3 1 409.39127700 0.00000000
Convergence criteria met.

Covariance Parameter Estimates

Standard Z
Cov Parm Estimate Error Value Pr Z Alpha Lower Upper
operator 0.0106 0.03286 0.32 0.3732 0.05 0.001103 3.7E12
part 10.2513 3.3738 3.04 0.0012 0.05 5.8888 22.1549
operator*part 0 . . . . . .
Residual 0.8832 0.1262 7.00 <.0001 0.05 0.6800 1.1938

Spring , 2008
Page 25
Statistics 514: Experiments with Random Effects

Confidence Intervals for Variance Components

• Can use asymptotic variance estimates to form CI

• Known as Wald’s approximate CI

• Mixed: option CL=WALD or METHOD=TYPE1

Use standard normal → 95% CI uses 1.96

σ̂β2 ± 1.96(.0330) = (−0.05, 0.08)

σ̂τ2 ± 1.96(3.3738) = (3.67, 16.89)

• In general Proc Mixed uses Satterthwaite CI

Default method - REML

Versions < 6.12 computed Wald CI

Current uses Satterthwaite’s Approximation

Will discuss this CI construction later on.

Spring , 2008
Page 26
Statistics 514: Experiments with Random Effects

Rules For Expected Mean Squares

• In models so far, EMS fairly straightforward

• Could calculate EMS using brute force method
• For mixed models, good to have formal procedure
• Montgomery describes procedure for restricted model
0 Write the error term in the model as ǫ(ij..)m , where m represents the replication
subscript
1 Write each variable term in model as a row heading in a two-way table
2 Write the subscripts in the model as column headings. Over each subscript write F
if factor fixed and R if random. Over this, write down the levels of each subscript
3 For each row, copy the number of observations under each subscript, providing the
subscript does not appear in the row variable term
4 For any bracketed subscripts in the model, place a 1 under those subscripts that
are inside the brackets
5 Fill in remaining cells with a 0 (if subscript represents a fixed factor) or a 1 (if
random factor).

Spring , 2008
Page 27
Statistics 514: Experiments with Random Effects

6 To find the expected mean square of any term (row), cover the entries in the
columns that contain non-bracketed subscript letters in this term in the model. For
those rows with at least the same subscripts, multiply the remaining numbers to
get coefficient for corresponding term in the model.

Spring , 2008
Page 28
Statistics 514: Experiments with Random Effects

Two-Factor Fixed Model:

yijk = µ + τi + βj + (τ β)ij + ǫijk

F F R

a b n
term i j k EMS
2 bnΣτi2
τi 0 b n σ + a−1
2 anΣβj2
βj a 0 n σ + b−1
2 nΣΣ(τ β)2ij
(τ β)ij 0 0 n σ + (a−1)(b−1)
2
ǫ(ij)k 1 1 1 σ

Spring , 2008
Page 29
Statistics 514: Experiments with Random Effects

Two-Factor Random Model:

yijk = µ + τi + βj + (τ β)ij + ǫijk

R R R

a b n
term i j k EMS

τi 1 b n σ 2 + nστ2β + bnστ2
βj a 1 n σ 2 + nστ2β + anσβ2
(τ β)ij 1 1 n σ 2 + nστ2β
ǫ(ij)k 1 1 1 σ2

Spring , 2008
Page 30
Statistics 514: Experiments with Random Effects

Two-Factor Mixed Model (A Fixed):

yijk = µ + τi + βj + (τ β)ij + ǫijk

F R R

a b n
term i j k EMS
2 bnΣτi2
τi 0 b n σ + nστ2β + a−1
βj a 1 n σ2 + anσβ2
(τ β)ij 0 1 n σ 2 + nστ2β
ǫ(ij)k 1 1 1 σ2

Spring , 2008
Page 31
Statistics 514: Experiments with Random Effects

Three-Factor Mixed Model (A Fixed):

yijkl = µ + τi + βj + γk + (τ β)ij + (τ γ)ik + (βγ)jk + (τ βγ)ijk + ǫijkl

F R R R

a b c n
term i j k l EMS
2 bcnΣτi2
τi 0 b c n σ + cnστ2β + bnστ2γ + nστ2βγ + a−1

βj a 1 c n σ 2 + anσβγ
2
+ acnσβ2
γk a b 1 n σ 2 + anσβγ
2
+ abnσγ2
(τ β)ij 0 1 c n σ 2 + nστ2βγ + cnστ2β
(τ γ)ik 0 b 1 n σ 2 + nστ2βγ + bnστ2γ
(βγ)jk a 1 1 n σ 2 + anσβγ
2

(τ βγ)ijk 0 1 1 n σ 2 + nστ2βγ
ǫijkl 1 1 1 1 σ2

Spring , 2008
Page 32
Statistics 514: Experiments with Random Effects

Two-Factor Mixed Effects Model

yijk = µ + τi + βj + (τ β)ij + ǫijk

• Assume A fixed and B random
τi = 0 and β ∼ N(0, σβ2 )
P
1 usual assumptions
2 (τ β)ij ∼ N(0, (a − 1)στ2β /a) (a − 1)/a simplifies EMS
P
3 (τ β)ij = 0 for β level j added restriction

• Due to added restriction

– Not all (τ β)ij indep, Cov((τ β)ij , (τ β)i′ j ) = − a1 στ2β
– Cov(yijk , yi′ jk′ ) = σβ2 − a1 στ2β , i 6= i′ .
• Known as restricted mixed effects model
• This model coincides with EMS algorithm
2
E(MSE )=σ
2
τi2 /(a − 1) + nστ2β
P
E(MSA ) = σ + bn
2
E(MSB ) = σ + anσβ2
2
E(MSAB ) = σ + nστ2β

Spring , 2008
Page 33
Statistics 514: Experiments with Random Effects

Hypotheses Testing and Diagnostics

• Testing hypotheses:
H0 : τ1 = τ2 = . . . = 0 → MSA /MSAB
H0 : σβ2 = 0 → MSB /MSE
H0 : στ2β = 0 → MSAB /MSE

• Variance Estimates ( Using ANOVA method)

σ̂ 2 = MSE
σ̂β2 = (MSB − MSE )/an
σ̂τ2β = (MSAB − MSE )/n

• Diagnostics
– Histogram or QQplot
Normality or Unusual Observations

– Residual Plots
Constant variance or Unusual Observations

Spring , 2008
Page 34
Statistics 514: Experiments with Random Effects

Multiple Comparisons for Fixed Effects

yijk = µ + τi + βj + (τ β)ij + ǫijk

y i.. = µ + τi + β . + (τ β)i. + ǫi..

Var(y i.. ) = σβ2 /b + (a − 1)στ2β /ab + σ 2 /bn

y i.. − y i′ .. = τi − τi′ + (τ β)i. − (τ β)i′ . + ǫi.. − ǫi′ ..

2 2
Var(y i.. − y i′ .. ) = 2στ β /b + 2σ /bn
= 2(nστ2β + σ 2 )/bn
• Need to plug in variance estimates to compute Var(y i.. )
• What are the DF?
• For pairwise comparisons, use estimate 2MSAB /bn
• Use dfAB for t-statistic or Tukey’s method.

Spring , 2008
Page 35
Statistics 514: Experiments with Random Effects

Gauge Capability Example in Text 12-3

options nocenter ls=75;

data randr;
input part operator resp @@;
cards;
1 1 21 1 1 20 1 2 20 1 2 20 1 3 19 1 3 21
2 1 24 2 1 23 2 2 24 2 2 24 2 3 23 2 3 24
3 1 20 3 1 21 3 2 19 3 2 21 3 3 20 3 3 22
4 1 27 4 1 27 4 2 28 4 2 26 4 3 27 4 3 28
.........................................
20 1 19 20 1 19 20 2 18 20 2 17 20 3 19 20 3 17
;
proc glm;
class operator part;
model resp=operator|part;
run;
==================================================
Dependent Variable: resp
Sum of

Spring , 2008
Page 36
Statistics 514: Experiments with Random Effects

Source DF Squares Mean Square F Value Pr > F

Model 59 1215.091667 20.594774 20.77 <.0001
Error 60 59.500000 0.991667
CorrTotal 119 1274.591667

Source DF Type I SS Mean Square F Value Pr > F

operator 2 2.616667 1.308333 1.32 0.2750
part 19 1185.425000 62.390789 62.92 <.0001
operator*part 38 27.050000 0.711842 0.72 0.8614

Spring , 2008
Page 37
Statistics 514: Experiments with Random Effects

Gauge Capability Example

• H0 : τ1 = τ2 = τ3 = 0:
M SA 1.308
F0 = = = 1.84
M SAB 0.712
P-value based on F2,38 : 0.173.

• H0 : σβ2 = 0:
M SB 62.391
F0 = = = 62.89
M SE 0.992
P-value based on F19,60 : 0.000

• H0 : στ2β = 0:
M SAB 0.712
F0 = = = 0.72
M SE 0.992
P-value based on F38,60 : 0.86

• Variance components estimates:

62.39 − 0.99
σ̂β2 = = 10.23
(3)(2)

Spring , 2008
Page 38
Statistics 514: Experiments with Random Effects

0.71 − 0.99
σ̂τ2β = = −.14(≈ 0)
2
σ̂ 2 = 0.99

• Pairwise comparison for τ1 , τ2 and τ3 .

Spring , 2008
Page 39
Statistics 514: Experiments with Random Effects

Unrestricted Mixed Model

• SAS uses unrestricted mixed model in analysis

• yijk = µ + τi + βj + (τ β)ij + ǫijk
τi = 0 and βj ∼ N(0, σβ2 )
P

(τ β)ij ∼ N(0, στ2β )

• Expected mean squares:
2
E(MSE )=σ
2
τi2 /(a − 1) + nστ2β
P
E(MSA ) = σ + bn
2
E(MSB ) = σ + anσβ2 + nστ2β
2
E(MSAB ) = σ + nστ2β
• random statement in SAS also gives these results
• Differences
– E(MSB )
– Test H0 : σβ2 = 0 using MSAB in denominator
2
– Cov(yijk , yi′ jk′ )=σβ , i 6= i′ .

Spring , 2008
Page 40
Statistics 514: Experiments with Random Effects

• Connection
(τ¯β).j = (
X
(τ β)ij )/a
i

yijk = µ + τ + (βj + (τ¯β).j ) + ((τ β)ij − (τ¯β).j ) + ǫijk

Check the model above satisfies the conditions of restricted mixed model

• Restricted model is slightly more general.

Spring , 2008
Page 41
Statistics 514: Experiments with Random Effects

Gauge Capability Example (Unrestricted Model)

options nocenter ls=75;

data randr;
input part operator resp @@;
cards;
1 1 21 1 1 20 1 2 20 1 2 20 1 3 19 1 3 21
2 1 24 2 1 23 2 2 24 2 2 24 2 3 23 2 3 24
3 1 20 3 1 21 3 2 19 3 2 21 3 3 20 3 3 22
4 1 27 4 1 27 4 2 28 4 2 26 4 3 27 4 3 28
.
.
;

proc glm;
class operator part;
model resp=operator|part;
random part operator*part / test;
means operator / tukey lines E=operator*part;
lsmeans operator / adjust=tukey E=operator*part tdiff stderr;

Spring , 2008
Page 42
Statistics 514: Experiments with Random Effects

proc mixed alpha=.05 cl covtest;

class operator part;
model resp=operator / ddfm=kr;
random part operator*part;
lsmeans operator / alpha=.05 cl diff adjust=tukey;
run;
quit;

Spring , 2008
Page 43
Statistics 514: Experiments with Random Effects

Approximate F Tests

• For some models, no exact F-test exists

• Recall 3 Factor Mixed Model (A - fixed)

• No exact test for A based on EMS

Assume a = 3, b = 2, c = 3, n = 2 and following MS were obtained

Spring , 2008
Page 44
Statistics 514: Experiments with Random Effects

Source DF MS EMS F P
2 2
A 2 0.7866 12φA + 6σAB + 4σAC ? ?
2
+2σABC + σ2
2 2
B 1 0.0010 18σB + 6σBC + σ2 0.33 .622
2 2
AB 2 0.0056 6σAB + 2σABC + σ2 2.24 .222
2 2
C 2 0.0560 12σC + 6σBC + σ2 18.87 .051
2 2
AC 4 0.0107 4σAC + 2σABC + σ2 4.28 .094
2
BC 2 0.0030 6σBC + σ2 10.00 .001
2
ABC 4 0.0025 2σABC + σ2 8.33 .001

Error 18 0.0003 σ2
• Possible approaches:
– Could assume some variances are negligible, not recommended without
“conclusive” evidence
– Pool (insignificant) means squares with error, also risky, not recommended when df
for error is already big.

Spring , 2008
Page 45
Statistics 514: Experiments with Random Effects

Satterthwaite’s Approximate F-test

τi2 = 0.
P
• H0 : effect = 0, e.g., H0 : τ1 = · · · = τa = 0 or equivalently H0 :
No exact test exists.

• Get two linear combinations of mean squares

M S ′ = MSr ± . . . ± MSs

M S ′′ = MSu ± . . . ± MSv
′ ′′
such that 1) M S and M S do not share common mean squares; 2)
E(M S ′ ) − E(M S ′′ ) is a multiple of the effect.
M S′ MSr ±...±MSs
– approximate test statistic F : F = M S ′′
= MSu ±...±MSv
≈ Fp,q
(MSr ±...±MSs )2 (MSu ±...±MSv )2
where p = MS2 2 and q = MS2 2
r /fr +...+MSs /fs u /fu +...+MSv /fv

– fi is the degrees of freedom associated with MSi

– p and q may not be integers, interpolation is needed. SAS can handle noninteger
dfs.

– Caution when subtraction is used

Spring , 2008
Page 46
Statistics 514: Experiments with Random Effects

Example: 3-Factor Mixed Model (A Fixed)

H0 : τ1 = τ2 = τ3 = 0

M S ′ = MSA

M S ′′ = MSAB + MSAC − MSABC

′ ′′ Στi2
E(M S − M S ) = 12φA = 12 3−1

MSA .7866
F = = = 57.0
MSAB + MSAC − MSABC .0107 + .0056 − .0025

.01382
p=2 q= = 4.15
.01072 /4 + .00562 /2 + .00252 /4

• Interpolation needed

P(F2,4 > 57) = .0011 P(F2,5 > 57) = .0004

P = .85(.0011) + .15(.0004) = .001

Spring , 2008
Page 47
Statistics 514: Experiments with Random Effects

• SAS can be used to compute P-values and quantile values for F and χ2 values with
noninteger degrees of freedom.

Upper Tail Probability: probf(x,df1,df2) and probchi(x,df)

Quantiles : finv(p,df1,df2) and cinv(p,df)

data one;
p=1-probf(57,2.0,4.15);
f=finv(.95,2.0,4.15);
c1=cinv(.025,18.57);
c2=cinv(.975,18.57);
proc print data=one;

OBS P F C1 C2
1 .00096 6.7156 8.61485 32.2833

Spring , 2008
Page 48
Statistics 514: Experiments with Random Effects

Another Approach to Testing H0 : τ1 = τ2 = τ3 = 0

M S ′ = MSA + MSABC

M S ′′ = MSAB + MSAC

E(M S ′ − M S ′′ ) =?
MSA + MSABC .7866 + .0025
F = = = 48.41
MSAB + MSAC .0107 + .0056

.78912 .01632
p= = 2.01 q= = 6.00
.78662 /2 + .00252 /4 .01072 /4 + .00562 /2

P-value= P (F > 48.41) = 0.002

• This is again found significant

• Avoid subtraction,summation should be preferred.

Spring , 2008
Page 49
Statistics 514: Experiments with Random Effects

Approximate Confidence Intervals

2
Suppose we are interested in σx .

• Case 1: there exists a mean square M Sx with dfx such that E(M Sx ) = σx2 . Then
σ̂x2 = M Sx , and
dfx M Sx 2
2
∼ χ (dfx )
σx
df M S
Exact 100(1-α)% CI: χ2x x
≤ σx2 ≤ df x M Sx
χ2
α/2,df x 1−α/2,df x

• Case 2: there exist

M S ′ = MSr + . . . + MSs and, M S ′′ = MSu + . . . + MSv
′ ′′ 2
such that E(M S − M S ) = kσx . Then

2
M S ′ − M S ′′ dfx σ̂x
σ̂x2 = , and ≈ χ2
(dfx )
k σx2
where

(σ̂x2 )2 (MSr + · + MSs − MSu − · − MSv )2

dfx = P M S =
i
k2 fi
MS2r /fr + · + MS2s /fs + MS2u /fu + · + MS2v /fv

Spring , 2008
Page 50
Statistics 514: Experiments with Random Effects

Approximate 100(1-α)% CI:

df x σ̂x2 2 df x σ̂x2
≤ σx ≤ 2
χ2α/2,df x χ1−α/2,df x

Spring , 2008
Page 51
Statistics 514: Experiments with Random Effects

Gauge Capability Example (Both Factors are Random)

Dependent Variable: RESP

Source DF Sum of Squares Mean Square F Value Pr > F

Model 59 1215.09166667 20.594774 20.77 0.0001
Error 60 59.50000000 0.991667
CorrecTotal 119 1274.59166667

Source DF Type III SS Mean Square F Value Pr > F

OPERATOR 2 2.61666667 1.308333 1.32 0.2750
PART 19 1185.42500000 62.390789 62.92 0.0001
OPERATOR*PART 38 27.05000000 0.711842 0.72 0.8614
M SA −M SAB
σ̂τ2 = bn
= (62.39 − 0.71)/6 = 10.28

(62.39 − 0.71)2
df = 2 2
= 18.57
62.39 /19 + 0.71 /38

CI: (18.57(10.28)/32.28, 18.57(10.28)/8.61) = (5.91, 22.17)

Spring , 2008
Page 52
Statistics 514: Experiments with Random Effects

M SB −M SAB
σ̂β2 = an
= (1.31 − 0.71)/40 = 0.015

(1.31 − 0.71)2
df = = .413
1.312 /2 + 0.712 /38
−8
CI: (.413(.015)/3.079, .413(.015)/2.29 × 10 ) = (.002, 270781)

Spring , 2008
Page 53
Statistics 514: Experiments with Random Effects

General Mixed Effect Model

• In terms of linear model

Y = Xβ + Zδ + ǫ

β is a vector of fixed-effect parameters

δ is a vector of random-effect parameters
ǫ is the error vector
• δ and ǫ assumed uncorrelated
– means 0
– covariance matrices G and R (allows correlation)

• Cov(Y ) = ZGZ ′ + R
• If R = σ 2 I and Z = 0, back to standard linear model
• SAS Proc Mixed allows one to specify G and R
• G through RANDOM, R through REPEATED
• Unrestricted linear mixed model is default
Spring , 2008
Page 54