Post Hoc Analysis(Tukey’s test)

Dr. A. Ramesh
DEPARTMENT OF MANAGEMENT STUDIES
IIT ROORKEE

1
 Lecture Objectives
After completing this lecture, you should be able to:
• Use Tukey’s test and LSD Test to identify specific differences between
means

2
 Designing engineering experiments

• Experimental design methods are also useful in engineering design

activities, where new products are developed and existing ones are
improved
• By using designed experiments, engineers can determine which subset of
the process variables has the greatest influence on process performance

3
 Designing engineering experiments

• The results of an experiment can lead to

1. Improved process yield
2. Reduced variability in the process and closer conformance to nominal
or target requirements
3. Reduced design and development time
4. Reduced cost of operation

4
 Designing engineering experiments

• Every experiment involves a sequence of activities:

1. Conjecture—the original hypothesis that motivates the experiment
2. Experiment—the test performed to investigate the conjecture
3. Analysis—the statistical analysis of the data from the experiment
4. Conclusion—what has been learned about the original conjecture
from the experiment. Often the experiment will lead to a revised
conjecture, and a new experiment, and so forth

5
 The completely randomized single-factor experiment
example
• A manufacturer of paper that is used for making
grocery bags is interested in improving the tensile
strength of the product
• Product engineer thinks that tensile strength is a
function of the hardwood concentration in the
pulp and that the range of hardwood
concentrations of practical interest is between 5
and 20%.

6
 The completely randomized single-factor experiment
example
• A team of engineers responsible for the study decides to investigate four
levels of hardwood concentration: 5%, 10%, 15%, and 20%.
• They decide to make up six test specimens at each concentration level,
using a pilot plant.
• All 24 specimens are tested on a laboratory tensile tester, in random order.
The data from this experiment are shown in Table

7
 The completely randomized single-factor experiment
example
• Tensile Strength of Paper (psi)
Hardwood Observations Total Avg
Concentration (%) 1 2 3 4 5 6

5 7 8 15 11 9 10 60 10.00
10 12 17 13 18 19 15 94 15.67
15 14 18 19 17 16 18 102 17.00
20 19 25 22 23 18 20 127 21.17
383 15.96

8
The completely randomized single-factor experiment
example

9
Typical Data for Single Factor Experiment

Treatment Observations Totals Averages

---
1 y11 y12 ... y1n y1. y1.
---
2 y 21 y 23 ... y2n y 2. y 2.
. . . ... . . .
. . . ... . . .
. . . ... . . .
---
a y a1 ya 2 ... y an ya. ya.
---
y .. y ..

10
 Sum of Squares

a n --
Total sum of squares  SST   (yij - y..)2
i 1 j 1
a --- ---
Treatment sum of squares  SSTreatments  n  ( y i.  y ..)2
i 1
a n ---
Error sum of Squares  SSE   (yij  y j. ) 2
i 1 j 1

11
 ANOVA with Equal Sample Sizes

a n 2
y ..
SST   y 
2
ij
i 1 j 1 N
1 a 2 y 2 ..
SSTreatments   yi. 
n i 1 N

N = an = No. of Treatments x no. of sample size = Total no. of Sample Size

12
 ANOVA with unequal Sample Sizes

a n 2
y ..
SST   y i j 2

i 1 j 1 N
a
yi.2 y 2 ..
SSTreatments   
i 1 ni N

N = an = No. of Treatments x no. of sample size = Total no. of Sample Size

13
 Problem: Analysis of variance

• Consider the paper tensile strength experiment described.

• We can use the analysis of variance to test the hypothesis that different
hardwood concentrations do not affect the mean tensile strength of the
paper.
• The hypotheses are
• H0:  1   2   3   4  0
• H1:  i  0 for at least one i

14
 Problem: Analysis of variance

• We will use a = 0.01.

• The sums of squares for the analysis of variance are computed are as
follows:

15
 ANOVA Table

Sources of Sum of Squares Degrees of Mean Square F

Variation Freedom

Treatments SS Treatments a-1 MS Treatments MS Treatments

/ MSE
Error SSE a(n-1) MSE

Total SST an-1

16
 Problem: Analysis of variance

• The ANOVA is summarized as follow

Source of Sum of Degrees Mean F0 P-value
Variation Squares of Square
freedom

Hardwood 382.79 3 127.6 19.6 3.59 E-6

concentrati
on
Error 130.17 20 6.51
Total 512.96 23

17
 Problem: Analysis of variance

• Since f0.01,3,20 = 4.94, we reject H0 and conclude that hardwood

concentration in the pulp significantly affects the mean strength of the
paper

18
Problem: Analysis of variance

19
Jupyter code

20
Jupyter code

21
Jupyter code

22
Jupyter code

23
 Multiple Comparisons Following the ANOVA

• When the null hypothesis is rejected in the ANOVA, we know that some of
the treatment or factor level means are different
• ANOVA doesn’t identify which means are different
• Methods for investigating this issue are called multiple comparisons
methods

24
 Fisher’s least significant difference (LSD) method

• The Fisher LSD method compares all pairs of means with the null
hypotheses H0:i   j (for all i ≠ j) using the t-statistic

yi*  y j*
t0 
2 MS E
n

25
 Fisher’s least significant difference (LSD) method

• Assuming a two-sided alternative hypothesis, the pair of means i and j

would be declared significantly different if

yi*  y j*  LSD
where LSD, the least significant difference, is
2MS E
LSD  ta /2,a ( n 1)
n

26
 Fisher’s least significant difference (LSD) method

• If the sample sizes are different in each treatment, the LSD is defined as

1 1
LSD  ta /2, N  a MS E (  )
ni n j

27
 Problem : LSD method

• We will apply the Fisher LSD method to the hardwood concentration

experiment. There are a = 4 means, n = 6, MSE = 6.51, and t0.025,20 = 2.086.
The treatment means are

28
 Problem : LSD method

• The value of LSD is:

2MS E 2(6.51)
LSD  t0.025,20  2.086  3.07
n 6

• Therefore, any pair of treatment averages that differs by more than 3.07
implies that the corresponding pair of treatment means are different.

29
Jupyter code

30
 Problem : LSD method

• The comparisons among the observed treatment averages are as follows:

31
The Tukey-Kramer Test for Post Hoc analysis

• Tells which population means are significantly different

• Done after rejection of equal means in ANOVA
• Allows pair-wise comparisons
• Compare absolute mean differences with critical range

32
 The Tukey-Kramer Test for Post Hoc analysis

• Determine is there any significant difference between the means

• is μ1 = μ2 ≠ μ3

x
μ1 = μ 2 μ3

33
 Tukey-Kramer Critical Range

MSW  1 1 
Critical Range  QU 
2  n j n j' 


where:
QU = Value from Studentized Range
Distribution with c and n - c degrees of freedom for
the desired level of a
MSW = Mean Square Within
nj and nj’ = Sample sizes from groups j and j’
34
 Problem: Tukey- Kramer test

• Tensile Strength of Paper (psi)

Hardwood Observations Total Avg
Concentratio 1 2 3 4 5 6
n (%)
5 7 8 15 11 9 10 60 10.00
10 12 17 13 18 19 15 94 15.67
15 14 18 19 17 16 18 102 17.00
20 19 25 22 23 18 20 127 21.17
383 15.96

35
 The Tukey-Kramer Procedure
1. Compute absolute mean differences:

x1  x 2  10.00  15.67  5.67

x1  x 3  10.00  17.00  7
x 2  x 3  15.67  17.00  1.33
x1  x 4  10.00  21.17  11.17
x 2  x 4  15.67  21.17  5.5
x 3  x 4  17.00  21.17  4.17

36
 The Tukey-Kramer Procedure

2. Find the QU value from the table with c = 4 and (n – c) = (24 – 4) = 20

degrees of freedom for the desired level of a (a = .05 used here):

QU  3.96

37
• Q table: The critical values
for q corresponding to
alpha = .05 (top) and
alpha = .01 (bottom)

38
 The Tukey-Kramer Procedure

Source of Sum of Degrees Mean F0 P-value

Variation Squares of Square
freedom

Hardwood 382.79 3 127.6 19.6 3.59 E-6

concentrati
on
Error 130.17 20 6.51
Total 512.96 23

39
 The Tukey-Kramer Procedure
3. Compute Critical Range:
MSW  1 1  6.51  1 1 
Critical Range  Q U     3.96     4.124
2  n j n j'  2 6 6

4. Compare: x1  x 2  10.00  15.67  5.67

x1  x 3  10.00  17.00  7
x 2  x 3  15.67  17.00  1.33
x1  x 4  10.00  21.17  11.17
x 2  x 4  15.67  21.17  5.5
x 3  x 4  17.00  21.17  4.17

40
 The Tukey-Kramer Procedure
5. Other then x 2  x 3 , all of the absolute mean differences are greater than
critical range. Therefore there is significant difference between each pair of
means, except 10% concentration and 15% concentration at the 5% level of
significance.

41
Jupyter code

42
 Problem 2

• Following table shows observed tensile

strength (lb/in square) of different clothes
having different weight percentage of cotton.
• Check whether having different weight
percentage of cotton, plays any role in tensile
strength (lb/in square) of clothes.

43
 Problem 2

Weight Observed tensile strength (lb/in square) Total Average

Percentage
of cotton

1 2 3 4 5
15 7 7 15 11 9 49 9.8
20 12 17 12 18 18 77 15.4
25 14 18 18 19 19 88 17.6
30 19 25 22 19 23 108 21.6
35 7 10 11 15 11 54 10.8
Grand Grand
total=376 mean=
15.004

44
• SSA = 5 (9.8 – 15.04)2 + 5 (15.4 – 15.04)2 + 5
(17.6 – 15.04)2 +5( 21.6-15.04)2+ 5(10.8-
15.04)2 = 475.76
SST = 636.96
SSE = 636.96 - 475.76=161.20

Sources of Sum of Degrees of Mean square F-value

variation squares freedom

Cotton weight 475.76 4 118.94 14.76

percentage
Error 161.20 20 8.06
Total 639.96 24

45
 Problem 2

• When alpha =.05, F(0.05,4,20) =2.87

• Reject Ho

46
• Q table: The critical values
for q corresponding to
alpha = .05 (top) and
alpha = .01 (bottom)

47
 Problem 2

MS E
Ta  qa (c, n  c )
n
a  0.05

q0.05 (5, 20)  4.23

8.06
T0.05  4.23  5.37
5

48
 Problem 2

Any pair of treatment averages that differ in absolute value by more

than 5.37 would imply that the corresponding pair of population means
are significantly different.

49
 Problem 2
__ __
y1.  y2.  9.8  15.4  5.6*
__ __
y1.  y3.  9.8  17.6  7.8*
__ __ Starred values indicate pairs of means
y1.  y4.  9.8  21.6  11.8 *
that are significantly different.
__ __
y1.  y5.  9.8  10.8  1

__ __ __ __
y2.  y3.  15.4  17.6  2.2 y3.  y4.  17.6  21.6  4
__ __ __ __
y2.  y4.  15.4  21.6  6.2* y3.  y5.  17.6  10.8  6.8*
__ __ __ __
y2.  y5.  15.4  10.8  4.6 y4.  y5.  21.6  10.8  10.8*

50
Jupyter code

51
Jupyter Code

52
Thank you

53