Prepared by: Dr.

Shazarel
BDA 24103 (Engineering Statistics)

Chapter 6
DESIGN OF EXPERIMENTS
(DOE)
 Prepared by: Dr. Shazarel

Learning Objectives
At the end of this chapter, the students should be able to:
1. Design and conduct engineering experiments involving a single factor
2. Explain the concepts of completely randomized designed single-factor
experiment
3. Perform an analysis of variance from the experimental data
4. Analyse the differences between treatment means by multiple
comparison techniques
5. Perform model adequacy checking with the residual plots

2
 Prepared by: Dr. Shazarel

Table of content :
6.1 Introduction: Designing engineering experiment
6.2 Completely randomized designed single-factor experiment
6.3 Analysis of variance (ANOVA)
6.4 Multiple comparison techniques
6.5 Residual analysis & model adequacy checking

3
 Prepared by: Dr. Shazarel

▪ Benefits of DOE:

1) Reduce time to design/develop new products & processes

2) Improve performance of existing processes
3) Improve reliability and performance of products
4) Achieve product & process robustness
5) Evaluation of materials, design alternatives, setting
component & system tolerances, etc.

4
 Prepared by: Dr. Shazarel

DOE Terms and Concepts

▪ Refer to Table 1:
1. Factors = inputs to the process and can be classified as either controllable or
uncontrollable variables. E.g.: Machine & Feed rate
2. Response (one) = output of the experiment. E.g.: Process capability
Table 1
Process capability Cp
Feed rate (mm/sec.)
Machine 0.2 0.4 0.6
USA 2.1, 2.2 1.7, 1.6 1.2, 1.3
Japan 0.9, 1.0 0.7, 0.6 0.4, 0.3

▪ Conclusion: using USA machine at lowest feed rate will result in max. process
capability.

5
 Prepared by: Dr. Shazarel

Table 2
Run A B C Run A B C
No Temp. Machine Switch No Temp. Machine Switch
1 10 Japan On 13 30 Japan On
2 10 Japan Off 14 30 Japan Off
3 10 USA On 15 30 USA On
4 10 USA Off 16 30 USA Off
5 10 UK On 17 30 UK On
6 10 UK Off 18 30 UK Off
7 20 Japan On 19 40 Japan On
8 20 Japan Off 20 40 Japan Off
9 20 USA On 21 40 USA On
10 20 USA Off 22 40 USA Off
11 20 UK On 23 40 UK On
12 20 UK Off 24 40 UK Off

6
 Prepared by: Dr. Shazarel

Factor (input variable), refer to Table 2 for an

example:
(1) Types
Quantitative: Temperature, time: They can be set at any value.
Qualitative : Machines, operators: They have different categories
Switch: It has different status

(2) Levels
Temperature (oC):10, 20, 30, 40 [4 levels]
Machine: made by Japan, by USA, by UK [3 levels]
Switch: on, or off [2 levels]

7
 Prepared by: Dr. Shazarel

(3) Run: A run is one set of levels for all factors in an experiment. In below
experiment (refer Table 2), there are three factors:
▪ (A) temperature, (B) machine and (C) switch
▪ The total number of runs is, R = 4 x 3 x 2 = 24
▪ For example, [1] A = 10, B = Japan-made, C = on
..........
[24] A = 40, B = UK-made, C = off
▪ For single factor experiments, run is equivalent to level of the factor. The
number R of runs is equal to the number a of levels.

(4)Replicates (n): number of observations at each run. A larger value of n

increases the accuracy of the experimental results, but makes the experiment more
expensive.

8
 Prepared by: Dr. Shazarel

▪ Response (or output, observation)

• Response is dependent on the values of the factors. Types of response:

Larger the better: product yield, process capability, student’s mark

Smaller the better: product cost, number of errors
Nominal the best: diameter of a shaft

• By DOE, we can find out which factors have effects on the response, and
the direction of the influence (i.e., whether the increase of a factor
will increase or decrease the response ?)

9
 Prepared by: Dr. Shazarel

▪ What Makes Designed Experiment (DOE) Special?

1) It conducts the experiments in a systematic and efficient way. Reduce
design and development time, as well as the cost.
2) It presents experiment results in the simplest and clearest way.
3) It extracts the maximum amount of information from a given set of
experiments.
4) It draws the right conclusions, despite the variability presented in the data.

Examples:
▪ Increase process capability
▪ Improve students’ study
▪ Reduce output variability

10
 Prepared by: Dr. Shazarel

6.2 SINGLE FACTOR EXPERIMENTS

Introduction
▪ Single factor experiment can be used in the situation where the effect (influence)
of a single factor on the response is dominant. The effects of all other factors are
negligible.

▪ The procedures and formulae developed for the single factor

experiments can be easily modified, and then used for the several factors
experiments.

11
 Prepared by: Dr. Shazarel

Example 1:
Table 5: Tensile strength of paper
Hardwood Observations
Concen. (%) 1 2 3 4 5 6 Total Avg
5 7 8 15 11 9 10 60 10.00
10 12 17 13 18 19 15 94 15.67
15 14 18 19 17 16 18 102 17.00
20 19 25 22 23 18 20 127 21.17
383 15.96

• Objective: To study the effect (influence) of hardwood concentration on the

tensile strength of paper. Results are given in Table 5.

12
 Prepared by: Dr. Shazarel

(1) One factor: hardwood concentration.

(2) Four levels: 5%, 10%, 15%, 20%: a = 4.
(3) Response is the tensile strength of paper.
(4) Each level has six observations or replicates: n = 6.
(5) Total number of observations: N = (a)(n)= (6)(4)=24.

• It is desired to apply the Randomization in taking observations from different

levels of hardwood concentration, in order to balance out the effects of
any nuisance variables (e.g., warm up effect) that may influence the
response.

13
 Prepared by: Dr. Shazarel

Table 6: Randomization in taking observations

Hardwood Observations Concen.

Concen. (%) 1 2 3 4 5 6 Total Avg
5 712 84 157 112 915 1021 60 10.00
10 1219 171 1313 1814 1917 156 94 15.67
15 1410 1820 1911 1724 1622 185 102 17.00
20 199 2516 2223 233 188 2018 127 21.17
383 15.96

• The subscript attached to each observation in Table 6 indicates the

run order (order of taking the observation).

14
 Prepared by: Dr. Shazarel

Paper strength

Hardwood
10 20 30 Concen. (%)

Figure 6.1: Paper strength vs Hardwood Concen. (%)

15
 Prepared by: Dr. Shazarel

Table 7: General data table for a single factor experiment

Level Observation Total Average

1 y11 y12 . . y1n y1. y1.

2 y21 y22 . . y2n y2. y2.

. . . . . .

. . . . . .

a ya1 ya2 . . yan

ya. 𝑦ҧ a.
y.. 𝑦ҧ ..

yij = the jth observation taken under the ith level of the factor.

16
 Prepared by: Dr. Shazarel

yi. the total of the observations at the ith level

(6.1)

ȳi. the average of the observations at the ith level

ȳi.. = yi. / n (6.2)
y.. the grand total of all observations

(6.3)
ȳ.. the grand average of all observations.
ȳ.. = y.. / N (6.4)
N = an is the total number of observations

17
 Prepared by: Dr. Shazarel

6.3 The Analysis of Variance (ANOVA)

▪ The Analysis of Variance (ANOVA) - ANOVA is the most important statistical tool used in
DOE. Its task is to decide whether and which factors or interactions have significant
effect (influence) on the response.

Model of ANOVA
Yij =  +  i + ij = i + ij (6.5)

 the overall mean.  ≈ ȳ.. (mean  average)

i the ith level mean.  i ≈ ȳi.
i the ith level effect.  i = i −   ȳi. − ȳ..
 ij the random error.  ij = yij − i  yij − yi.

18
 Prepared by: Dr. Shazarel

Machining Example (1)

Study the effect of machine on the process capability

Table 8
Machine Process capability Cp
USA 2.1, 2.0, 1.9
Japan 0.9, 1.0, 1.1

1) One factor: machine (qualitative).

2) Two levels: USA-made, Japan-made, a = 2
3) Response is the process capability.
4) Each level has 3 observations or replicates: n = 3
5) Total number of observations: N = (a)(n) = (2) (3) = 6

19
 Prepared by: Dr. Shazarel

y1. = 2.1 + 2.0 + 1.9 = 6.0, ȳ1. = y1. / 3 = 2.0

y2. = 0.9 + 1.0 + 1.1 = 3.0, ȳ2. = y2. / 3 = 1.0

y.. = (2.1 + 2.0 + 1.9) + (0.9 + 1.0 + 1.1) = 9.0

ȳ.. = 9.0 / 6 = 1.5

1  ȳ1. = 2.0, 2  ȳ 2. =1.0

  ȳ.. = 1.5

1 = 1 −   ȳ1. − ȳ.. = 2.0 −1.5 = 0.5

 2 = 2 −  ȳ2. − ȳ.. = 1.0 −1.5 = −0.5

20
 Prepared by: Dr. Shazarel

Cont.
Evaluation of Errors
• For instance, if i = 1, j = 1, yij = y11 = 2.1
ij = yij − i
11 = y11 − 1  y11 − ȳ1. = 2.1 − 2.0 = 0.1

• Errors for the machining example (1)

Table 8.1
Machine Errors
USA 0.1, 0.0, -0.1
Japan -0.1, 0.0, 0.1

21
 Prepared by: Dr. Shazarel

Figure 6.2

22
 Prepared by: Dr. Shazarel

• Hypothesis Test: to test if a factor has effect on the response.

• If a factor has NO effect on the response, the response is always the

same, regardless the change of the factor,
1 = 2 = • • • = a = 
(1 − ) = (2 − ) = • • • = (a − ) = 0
1 =  2 = • • •  a = 0

• If a factor has effect, the value of the response will be different,

along with the change of the factor. The more the response
differs, the greater the effect of the factor is.

i = 0 for at least one i

23
 Prepared by: Dr. Shazarel

Null: Ho: the factor has no effect on the response

Alternative: H1: the factor has effect on the response

Ho : 1 = 2 = • • • = a = 0 (|i| is small, no effect)

H1:  i = o for at lease one i (|i| is larger, has effect) (6.6)

The larger the value of any |i|, the more effective the factor will be. Here,
the magnitude of i, rather than its sign, makes sense.

Type I error: the null hypothesis Ho is rejected when it is actually true

(The factor is concluded effective, when it is not)
Type II error: the null hypothesis is accepted when it is actually
false (The factor is concluded ineffective, when it is effective).

24
 Prepared by: Dr. Shazarel

Machining Example (2)

Table 9
Machine Process capability Cp
USA 1.6, 1.5, 1.4
Japan 1.4, 1.5, 1.6

1  1.5, 2 1.5   1.5

1 = 1 −  = 1.5 − 1.5 = 0
 2 = 2 −  = 1.5 − 1.5 = 0

Conclusion: in this example, the factor (machine) has no effect

on the process capability. Because, when using different
machines, the response is always the same, on average.

25
 Prepared by: Dr. Shazarel

Figure 6.4

26
 Prepared by: Dr. Shazarel

5 Major Steps of Analysis of Variance (ANOVA):

(1) Calculate sum of squares (SS).

(2) Determine degrees of freedom (D.O.F.).
(3) Calculate mean square (MS).
(4) Calculate the test ratio F0.
(5) Make conclusion.

27
 Prepared by: Dr. Shazarel

(1) Calculate sum of squares (SS)

Three types of differences:
[1] Difference between an individual observation and the grand
average:
yij - ȳ..
[2] Difference between a level average and the grand average:
ȳi. - ȳ.. = I
[3] Difference between an individual observation and
the corresponding level average:
yij - ȳi. = ij

Note: yij - ȳ.. = ( ȳi. - ȳ.. ) + ( yij - ȳi )

28
 Prepared by: Dr. Shazarel

Sum of squares is the sum of squares of the three differences

SST The total sum of squares, which is a measure of total variability in the
data.

(6.7)

SSF the sum of squares due to the factor, that is the sum of squares of differences
between factor level averages and the grand average (difference between levels).

(6.8)

29
 Prepared by: Dr. Shazarel

SSE the sum of squares due to error, that is the sum of squares of
differences between observations and their level averages (difference
within levels).

(6.9)

SST = SSF + SSE (6.10)

30
 Prepared by: Dr. Shazarel

Alternative Formulae of Calculating the Sum of Squares

(6.11)

(6.12)

(6.13)

Formulae (6.7) to (6.9) are used to describe the underlying ideas.

Formulae (6.11) to (6.13) are used for actually computation.

31
 Prepared by: Dr. Shazarel

From Machining Example (1)

Table 10
Machine Process capability Cp
USA 2.1, 2.0, 1.9
Japan 0.9, 1.0, 1.1

a = 2, n=3
y1 = 6 y2 = 3 y = 9
ȳ1 = 2 ȳ2 = 1 ȳ = 1.5

SST = (2.12 + 2.02 + 1.92 + 0.92 + 1.02 + 1.12 ) − 92 / 6 = 1.54

SSF = (62 + 32 ) / 3 − 92 / 6 = 1.50
SS E = 1.54 − 1.50 = 0.04

32
 Prepared by: Dr. Shazarel

Cont.
Machining Example (2)
Table 9
Machine Process capability Cp
USA 1.6, 1.5, 1.4
Japan 1.4, 1.5, 1.6

a = 2, n=3
y1 = 4.5 y2 = 4.5 y = 9
ȳ1 = 1.5 ȳ2 = 1.5 ȳ = 1.5

SST = (1.62 + 1.52 + 1.42 + 1.42 + 1.52 + 1.62 ) − 92 / 6 = 13.54 – 13.5 = 0.04
SSF =(4.52+ 4.52)/3 - 92 / 6 = 13.5-13.5=0
SS E = 0.04− 0= 0.04

33
 Prepared by: Dr. Shazarel

Cont.
• If SSF is large, it is due to differences among the means at the
different factor levels. See equation (6.12), a large SSF means that  i (or |i|)
is great. It is an indication that the factor has significant effect on the
response.
• Usually, SSF is standardized by taking SSF / SSE . By comparing SSF to
SSE, we can see how much variability is due to changing factor levels
and how much is due to error.

Machining Example

Table 11
Effective ? SSF/SSE
Example 1 Yes (1.50) 37.5(1.50/0.04)
Example 2 No (0) 0 (0/0.04)

34
 Prepared by: Dr. Shazarel

(2) Determine degrees of freedom (D.O.F.).

total number of observation N: an

degrees of freedom of SST: N - 1 = an - 1 (6.14)

degrees of freedom of SSF: a-1 (6.15)
degrees of freedom of SSE: R (n - 1) = a (n – 1) (6.16)

where, (n - 1) is the degrees of freedom of errors in a run.

Equation of DOF
an - 1 = (a - 1) + a (n - 1)
DOF of SST = DOF of SSF + DOF of SSE

35
 Prepared by: Dr. Shazarel

Example:
From Machining Example (1)
Table 8
Machine Process capability Cp
USA 2.1, 2.0, 1.9
Japan 0.9, 1.0, 1.1

a = 2, n = 3
total number of observations N: 23=6
degrees of freedom of SST : N–1 =6–1=5
degrees of freedom of SSF : a–1 =2–1=1
degrees of freedom of SSE : a (n-1) = 2  (3 - 1) = 4
Check equation : 5=1+4

36
 Prepared by: Dr. Shazarel

(3) Calculate mean square (MS).

The ratio between a sum of squares and its DOF.

(6.17)

(6.18)

There is no need to calculate MS for the SST

37
 Prepared by: Dr. Shazarel

Cont.
(4) Calculate the test ratio F0.

(6.19)

If H0 is true, F0 follows a theoretical F distribution, which is completely

determined by the two parameters 1 and 2.

1 = a - 1 the D.O.F. of MSF (numerator)

2 = a(n - 1) the D.O.F. of MSE (denominator)

1 = 2-1=1
2 = 2(3-1)=4

38
 Prepared by: Dr. Shazarel

Cont.

Figure 6.5
The density function curve of an example of the F distribution

39
 Prepared by: Dr. Shazarel

Cont.
(5) Make conclusion.

F0 is equivalent to (the standardized SSF) except for a constant Q.

40
 Prepared by: Dr. Shazarel

Cont.
If F0 is large → is large → SSF is large → i2 is large →

|i| is large → the factor has significant effect on the response.

Control limit: F ,1 ,2

❖ 1 = a - 1 and 2 = a(n - 1) are the two parameters of
an F distribution
❖ is the probability of Type I error, which means concluding
that the factor has significant effect on the response when it in
fact has no effect. Usually, set  = 1% or 5%

If Fo < F ,1 ,2 conclude that the factor has no effect.

If Fo > F ,1 ,2 conclude that the factor has significant effect.

41
 Prepared by: Dr. Shazarel

Cont.

F0

Sometimes, p value is used to indicate the result of ANOVA. p is equal to

the area under the F probability curve and beyond F0.

If p is small → F0 is large → the factor has significant effect on the response

42
 Prepared by: Dr. Shazarel

Table 12 Control Limit F ,1 ,2 found from the F Table

F ,1 ,2

43
 Prepared by: Dr. Shazarel

Cont.
Table 13

Analysis of Variance for a Single-Factor Experiment

Source of Sum of Degree of Mean

F0
Variation Squares Freedom Square
MSF
Factor SSF a-1 MSF
MSE
Error SSE a(n-1) MSE
Total SST an-1

44
 Prepared by: Dr. Shazarel

Example 1:
From Machining Example (1) (a = 2, n = 3, N = 6), refer to Table 8

Table 14

Source of Sum of Degrees of Mean

Fo
Variation Squares Freedom Square

Machine 1.50 1 (= 1) 1.50 150

Error 0.04 4 (= 2) 0.01
Total 1.54 5

Specify  = 0.05, Control limit: F ,1 ,2 = F0.05, 1,4 = 7.71

Since F0 > F ,1 ,2 machines have significant effect on the process capability.
Sum of squares, 1.50 + 0.04 = 1.54
Degree of freedom, 1 + 4 = 5

45
 Prepared by: Dr. Shazarel

Example 2:
From Machining Example (2) (a = 2, n = 3, N = 6), refer to Table 9

Table 15

Source of Sum of Degree of Mean

F0
Variation Squares Freedom Square
Factor 0 1(=v1) 0 0
Error 0.04 4(=v2) 0.01

Total 0.04 5

Specify α = 0.05, Control limit: Fα,v1 ,v2 = F0.05, 1,4 = 7.71

Since F0 < F ,1 ,2 machines do not have significant effect on the process
capability.

46
 Prepared by: Dr. Shazarel

Tutorial:
Table 16 : Tensile strength of paper

Hardwood Observations
Concen. (%) 1 2 3 4 5 6 Total Avg
5 7 8 15 11 9 10 60 10.00
10 12 17 13 18 19 15 94 15.67
15 14 18 19 17 16 18 102 17.00
20 19 25 22 23 18 20 127 21.17
383 15.96
a = 4, n = 6, N = an = 24
Please find :

(i) SSF, SSE and SST

(ii)degrees of freedom of SSF and degrees of freedom of SSE
(iii)MSF and MSE
(iv)Fo
47