Introduction to Design of Experiments

Dr. Lotfi K. Gaafar

The American University in Cairo

© 2003 QA Publishing, LLC

This presentation uses information from Paul A. Keller of QA Publishing, LLC .
Lotfi K. Gaafar 2004
Lotfi K. Gaafar 2004A. Keller
By Paul Introduction to DOE 1
Overview
Controllable factors

Input Output
Process

© 2003 QA Publishing, LLC

Lotfi K. Gaafar 2004
Uncontrollable factors
By Paul A. Keller Introduction to DOE 2
Designed Experiment Terminology
• Response:
– Mfg: Yield of a Process
– Service: Customer Satisfaction
• Controlled Factors: set to predefined levels for DOE
– Mfg: Furnace Temp., Fill Pressure, Material Moisture
– Service: Process Design, Follow-up
• Uncontrollable Factors: factors that cannot be
controlled in actual operations, but may be controlled
during experimentation.
Mfg: Humidity, air pollution
Service: Arrival rate, efficiency
© 2003 QA Publishing, LLC
By Paul A. Keller Introduction to DOE 3
 Designed vs. Traditional Experiments
• Traditional: vary one factor at a time
• Factor Response is deviation from “base”
– How do you maximize the result?
– What is Effect of each Factor?

© 2003 QA Publishing, LLC

By Paul A. Keller Introduction to DOE 4
One factor at a time
•Ignores effect of Interaction

Trial 3
Trial 2

© 2003 QA Publishing, LLC

By Paul A. Keller Introduction to DOE 5
Implications of Interaction
• We may think a factor is unimportant if
we don’t vary other factors at the same
time.
• We may improve the process, but it only
works if other factors remain constant.
• We may be able to reduce the effect of a
factor by minimizing variation of another.

© 2003 QA Publishing, LLC

By Paul A. Keller Introduction to DOE 6
 Designed Experiments Vs.
Historical Data
• Designed
– Designed to detect specific factors and
interactions (orthogonal)
– Relatively short period of time
– Casual Factors observed and/or controlled
– Recorded anomalies
• Historical
– May be incapable of detecting interactions
– May lack range to detect factor significance
– Unrecognized biases
– Changing environment
© 2003 QA Publishing, LLC
By Paul A. Keller Introduction to DOE 7
 DOE: Objectives

• Determine influential variables (factors)

• Determine where to set influential factors
to optimize response
• Determine where to set influential factors
to minimize response variability
• Determine where to set influential factors
to minimize the effect of the uncontrollable
factors

© 2003 QA Publishing, LLC

Lotfi K. Gaafar 2004
By Paul A. Keller Introduction to DOE 8
 DOE: Applications in Process Development

• Improve process yield

• Reduce variability
• Reduce development time
• Reduce overall costs

© 2003 QA Publishing, LLC

Lotfi K. Gaafar 2004
By Paul A. Keller Introduction to DOE 9
 DOE: Applications in Design

• Evaluate and compare alternatives

• Evaluate material alternatives
• Product robustness
• Determine key design parameters

© 2003 QA Publishing, LLC

Lotfi K. Gaafar 2004
By Paul A. Keller Introduction to DOE 10
 DOE: Basic Principles

• Replication
– Error estimation
– Accuracy
• Blocking
– Unimportant significant factor
– Precision
• Randomization
– Independence
– Even out uncontrollable factors
© 2003 QA Publishing, LLC
Lotfi K. Gaafar 2004
By Paul A. Keller Introduction to DOE 11
 DOE Steps
• Problem statement
• Choice of factors, levels, and ranges
• Choice of response variable(s)
• Choice of experimental design
• Performing the experiment
• Statistical analysis
• Conclusions and recommendations

© 2003 QA Publishing, LLC

Lotfi K. Gaafar 2004
By Paul A. Keller Introduction to DOE 12
 Resource Allocation
• Don’t commit all resources to one design
– Start with Screening design
– Only 25% of resources on any one experiment
• Learn from each design
– What did you do wrong?
• Excluded factors, wrong conditions, etc.
– What to do next?
• Sometimes next stage of improvement isn’t worth the
cost of another experiment
© 2003 QA Publishing, LLC
Lotfi K. Gaafar 2004
By Paul A. Keller Introduction to DOE 13
Selecting Factors
• For each response, brainstorm likely factors
• For screening, if more than 5-7 factors:
– Reduce factor list through ranking
• Nominal Group Technique, Prioritization Matrix
– Hold some factors constant
• ex: raw material type/supplier

© 2003 QA Publishing, LLC

By Paul A. Keller Introduction to DOE 14
 Selecting Factor Level Values
• Spanning entire region likely to yield the
most understanding.
– If factor's levels are close, measured effect may
be statistically insignificant
• Moving off current operating points
presents a risk.
– Probing techniques: Response Surface Analysis
– Evolutionary Operation (EVOP): converge on
best solution
© 2003 QA Publishing, LLC
By Paul A. Keller Introduction to DOE 15
 Effects of Aliasing: Confounding
• Aliased parameters are CONFOUNDED
– Cannot be estimated independently of one
another
– Estimates are linear combination of confounded
parameters
• Aliasing creates other confounded pairs
– If ABC = D, then A = BCD; B = ACD; C = ABD;
AB = CD; AC = BD; AD = BC;

© 2003 QA Publishing, LLC

By Paul A. Keller Introduction to DOE 16
Desirable Designs
(ref: Box, G.E.P. and N.R. Draper. Robust Designs. Biometrika 62 (1975):347-352)

• Provide sufficient distribution of

information throughout region of interest
• Provide model that predicts the response,
as close as possible to true response, at all
points w/in region of interest
• Provide ability to detect model lack of fit

© 2003 QA Publishing, LLC

By Paul A. Keller Introduction to DOE 17
Desirable Designs (cont.)
(ref: Box, G.E.P. and N.R. Draper. Robust Designs. Biometrika 62 (1975):347-352)

• Allow blocking
• Allow sequential buildup of design
• Provides internal estimate of error
variance
• Provide simple means of calculating
estimates of coefficients

© 2003 QA Publishing, LLC

By Paul A. Keller Introduction to DOE 18
 Design Performance
Considerations
• Number of Runs
– minimal best
• Design Resolution
– indicates which, if any, interactions can be
independently estimated
• Minimum Detectable Effect
• Orthogonality & Balance
• Other: D-Optimal, A-Optimal & G-Optimal

© 2003 QA Publishing, LLC

By Paul A. Keller Introduction to DOE 19
Design Resolution
• Resolution III
– Estimates of Main factor effects only; all
interactions may be confounded with one
another and MF may be confounded with
interactions.
• Resolution IV
– Estimates of MF are not confounded with 2-
factor interactions but may be confounded with
higher order interactions. Two factor
interactions may be confounded with one
another and with higher order interactions.
© 2003 QA Publishing, LLC
By Paul A. Keller Introduction to DOE 20
 Design Resolution (continued)
• Resolution V
– Estimates of MF and 2-factor effects are not
confounded with one another but may be
confounded with higher-order interactions.
Three-factor and higher interactions may be
confounded.
• Resolution VI
– Estimates of MF and 2-factor effects are not
confounded with each other or with 3-factor
interactions. Three-factor and higher interactions
may be confounded with one another.
© 2003 QA Publishing, LLC
By Paul A. Keller Introduction to DOE 21
 Design Resolution (continued)
• Resolution VII
– Estimates of MF, 2-factor and 3-factor
effects are not confounded with one another
but may be confounded with higher order
interactions. Four-factor and higher
interactions may be confounded.
• Resolution vs. Number of Trials

© 2003 QA Publishing, LLC

By Paul A. Keller Introduction to DOE 22
 Orthogonality
• Orthogonality refers to the property of a
design that assures that all specified
parameters may be estimated
independently of any other
– If sum of factors’ columns in standard
format equal 0, then design is orthogonal
• Some writers lump balance as part of
orthogonality.

© 2003 QA Publishing, LLC

By Paul A. Keller Introduction to DOE 23
 Balance
• Balance implies data is properly distributed over
design space.
– uniform physical distribution
– an equal number of levels of each factor.
• Some designs sacrifice balance to achieve better
distribution of variance or predicted error
– Ex: Central Composite.
• Balance may be sacrificed by avoiding extreme
combinations of factors
– Ex: Box-Behnken
© 2003 QA Publishing, LLC
By Paul A. Keller Introduction to DOE 24
 Sample Designs
• Box Behnken
• Plackett Burman
• 2k designs (fractional, confounding, fold over,
projection)
• 3k designs
• Mixed level designs
• Latin Squares
• Central Composite (with axial points)
• John’s ¾

© 2003 QA Publishing, LLC

Lotfi K. Gaafar 2004
By Paul A. Keller Introduction to DOE 25
 Sample Designs
• Nested Designs
• Split Plots
• Simplex lattice design
• Simplex centroid design
• D- Optimal
• A- Optimal

© 2003 QA Publishing, LLC

Lotfi K. Gaafar 2004
By Paul A. Keller Introduction to DOE 26
 General Guidelines
1. Good understanding of the problem
Research has shown that one of the key reasons for an
industrial experiment to be unsuccessful is due to lack of
understanding of the problem itself. The success of any
industrially designed experiment will heavily rely on the
nature of the problem at hand. The success of the experiment
also requires team effort.

© 2003 QA Publishing, LLC

From:http://www.qualityamerica.com/knowledgecente/articles/ANTONYdoe1.htm
Lotfi K. Gaafar 2004
By Paul A. Keller Introduction to DOE 27
 General Guidelines
2. Conduct a thorough and in-depth Brainstorming Session
The successful application of DOE requires a mixture of
statistical, planning, engineering, communication and teamwork
skills. Brainstorming must be treated as an integral part in the
design of effective experiments. It is advised to consider the
following key issues while conducting brainstorming session:

•Identification of the process variables, the number of levels of each process variable
and other relevant information about the experiment
•Development of team spirit and positive attitude in order to assure greater participation
of the team members.
•How well does the experiment simulate users’ environment?
•Who will do what and how?
•How quickly does the experimenter need to provide the results to management?
© 2003 QA Publishing, LLC
From:http://www.qualityamerica.com/knowledgecente/articles/ANTONYdoe1.htm
Lotfi K. Gaafar 2004
By Paul A. Keller Introduction to DOE 28
 General Guidelines
3. Select the appropriate response or quality characteristic
A response is the performance characteristic of a product which is
most critical to customers and often reflects the product quality. It
is important to choose and measure an appropriate response for
the experiment. The following tips may be useful to engineers in
selecting the quality characteristics for industrial experiments.

•Use responses that can be measured accurately.

•Use responses which are directly related to the energy transfer associated with the
fundamental mechanism of the product or the process.
•Use responses which are complete, i.e., they should cover the input-output relationship
for the product or the process.

© 2003 QA Publishing, LLC

From:http://www.qualityamerica.com/knowledgecente/articles/ANTONYdoe1.htm
Lotfi K. Gaafar 2004
By Paul A. Keller Introduction to DOE 29
 General Guidelines
4. Choose a suitable design for the experiment
The choice of an experimental design will be dependent upon the
following factors:

•Number of factors and interactions (if any) to be studied

•Complexity of using each design
•Statistical validity and effectiveness of each design
•Ease of understanding and implementation
•Nature of the problem
•Cost and time constraints

© 2003 QA Publishing, LLC

From:http://www.qualityamerica.com/knowledgecente/articles/ANTONYdoe1.htm
Lotfi K. Gaafar 2004
By Paul A. Keller Introduction to DOE 30
 General Guidelines
5. Perform a screening experiment
A screening experiment is useful to reduce the number of process
variables to a manageable number and thereby reduce the number
of experimental runs and costs associated with the entire
experimentation process. For example, one may be able to study
seven factors using just eight experimental trials. It is advisable
not to invest more than 25% of the experimental budget in the first
phase of any experimentation such as screening. Having identified
the key factors, the interactions among them can be studied using
full or fractional factorial experiments (Box et al., 1978).

© 2003 QA Publishing, LLC

From:http://www.qualityamerica.com/knowledgecente/articles/ANTONYdoe1.htm
Lotfi K. Gaafar 2004
By Paul A. Keller Introduction to DOE 31
 General Guidelines
6. Use Blocking Strategy to increase the efficiency of
experimentation
Blocking can be used to minimize experimental results being
influenced by variations from shift-to-shift, day-to-day or machine-
to-machine. The blocks can be batches of different shifts, different
machines, raw materials and so on.

© 2003 QA Publishing, LLC

From:http://www.qualityamerica.com/knowledgecente/articles/ANTONYdoe1.htm
Lotfi K. Gaafar 2004
By Paul A. Keller Introduction to DOE 32
 General Guidelines
7. Perform Confirmatory trials/experiments
It is necessary to perform a confirmatory experiment/trial to verify
the results from the statistical analysis. Some of the possible
causes for not achieving the objective of the experiment are:
wrong choice of design for the experiment
inappropriate choice of response for the experiment
failure to identify the key process variables which affect the
response
inadequate measurement system for making measurements
lack of statistical skills, and so on.

© 2003 QA Publishing, LLC

From:http://www.qualityamerica.com/knowledgecente/articles/ANTONYdoe1.htm
Lotfi K. Gaafar 2004
By Paul A. Keller Introduction to DOE 33