Planning and Conducting Experiments:

Introduction to
Design of Experiments
ENGR. JUICY CORDERO-VALDUEZA

Learning objectives and outcomes

Upon completion of this lesson, you should be able to do the
following:
> understand the issues and principles of Design of
Experiments (DOE),
 > understand experimentation is a process,
 > list the guidelines for designing experiments, and
> recognize the key historical figures in DOE.

Introduction
 > The term experiment is defined as the systematic procedure
carried out under controlled conditions in order to discover an
unknown effect, to test or establish a hypothesis, or to illustrate a
known effect. When analyzing a process, experiments are often
used to evaluate which process inputs have a significant
impact on
the process output, and what the target level of those inputs
should be to achieve a desired result (output).
 > Experiments can be designed in many different ways to collect
this information.
Design of Experiments (DOE) is also referred to as Designed
Experiments or Experimental Design - all of the terms have the
same meaning.
 > Experimental design can be used at the point of greatest
leverage to reduce design costs by speeding up the design process,
reducing late engineering design changes, and reducing product
material and labor complexity.
 > Designed Experiments are also powerful tools to achieve
manufacturing cost savings by minimizing process variation and
reducing rework, scrap, and the need for inspection.

What is the Scientific Method?

 Do you remember learning about this back in high school or junior
high even? What were those steps again?
 Decide what phenomenon you wish to investigate. Specify how
you can manipulate the factor and hold all other conditions fixed, to
insure that these extraneous conditions aren't influencing the
response you plan to measure.
 Then measure your chosen response variable at several (at least
two) settings of the factor under study. If changing the factor causes
the phenomenon to change, then you conclude that there is indeed
a cause-and-effect relationship at work.
 How many factors are involved when you do an experiment? Some
say two -perhaps this is a comparative experiment? Perhaps there is
a treatment group and a control group? If you have a treatment
group and a control group then in this case you probably only have
one factor with two levels.

How many of you have baked a cake? What are the factors involved
to ensure a successful cake? Factors might include preheating the
oven, baking time, ingredients, amount of moisture, baking
temperature, etc.-- what else? You probably follow a recipe so there
are many additional factors that control the ingredients - i.e., a
mixture. In other words, someone did the experiment in advance!
What parts of the recipe did they vary to make the recipe a
success? Probably many factors, temperature and moisture, various
ratios of ingredients, and presence or absence of many additives.
Now, should one keep all the factors involved in the experiment at a
constant level and just vary one to see what would happen? This is a
strategy that works but is not very efficient.

A Quick History of the Design of Experiments (DOE)

"All experiments are designed experiments, it is just that some are

poorly designed
and some are well-designed."

Engineering Experiments
If we had infinite time and resource budgets there probably
wouldn't be a big fuss made over designing experiments. In
production and quality control we want to control the error and learn
as much as we can about the process or the underlying theory with
the resources at hand. From an engineering perspective we're trying
to use experimentation for the following purposes:
> reduce time to design/develop new products & processes
> improve performance of existing processes
> improve reliability and performance of products
> achieve product & process robustness
>perform evaluation of materials, design alternatives, setting
component & system
tolerances, etc.

 > We always want to fine tune or improve the process. In today's

global world this
drive for competitiveness affects all of us both as consumers and
producers.
 > Robustness is a concept that enters into statistics at several
points. At the analysis stage robustness refers to a technique that
isn't overly influenced by bad data. Even if there is an outlier or bad
data you still want to get the right answer. Regardless of who or
what is involved in the process - it is still going to work.

Every experiment design has inputs. Back to the cake baking

example: we have our ingredients such as flour, sugar, milk, eggs,
etc. Regardless of the quality of these ingredients we still want our
cake to come out successfully. In every experiment there are inputs
and in addition there are factors (such as time of baking,
temperature, geometry of the cake pan, etc.), some of which you
can control and others that you can't control. The experimenter
must think about factors that affect the outcome. We also talk about
the output and the yield or the response to your experiment. For the
cake, the output might be measured as texture, flavor, height, size,
or flavor.

Four Eras in the History of DOE

Notes:
 A lot of what we are going to learn in this course goes back to
what Sir Ronald Fisher developed in the UK in the first half of the
20th century. He really laid the foundation for statistics and for
design of experiments. He and his colleague Frank Yates developed
many of the concepts and procedures that we use today. Basic
concepts such as orthogonal designs and Latin squares began there
in the 20's through the 40's. World War II also had an impact on
statistics, inspiring sequential analysis, which arose from World War
II as a method to improve the accuracy of long-range artillery guns.
 Immediately following World War II the first industrial era marked
another resurgence in the use of DOE. It was at this time that Box
and Wilson (1951) wrote the key paper in response surface
designs thinking of the output as a response function and trying to
find the optimum conditions for this function. George Box died
early in 2013. And, an interesting fact here - he married Fisher's
daughter! He worked in the chemical industry in England in his early
career and then came to America and worked at the University of
Wisconsin for most of his career.

The Second Industrial Era - or the Quality Revolution

The Modern Era

Around 1990 Six Sigma, a new way of representing CQI,
became popular. Now it is a company and they employ a technique
which has been adopted by many of the large manufacturing
companies. This is a technique that uses statistics to make decisions
based on quality and feedback loops. It incorporates a lot of the
previous statistical and management techniques.
Clinical Trials
Montgomery omits in this brief history a major part of design of
experimentation that evolved - clinical trials. This evolved in the
1960's when medical advances were previously based on anecdotal
data; a doctor would examine six patients and from this wrote a
paper and published it. The incredible biases resulting from these
kinds of anecdotal studies became known. The outcome was a move
toward making the randomized double-blind clinical trial the gold
standard for approval of any new product, medical device, or
procedure. The scientific application of the statistical procedures
became very important.

The Basic Principles of DOE

Randomization - this is an essential component of any
experiment that is going to have validity. If you are doing a
comparative experiment where you have two treatments, a
treatment and a control for instance, you need to include in your
experimental process the assignment of those treatments by some
random process. An experiment includes experimental units. You
need to have a deliberate process to eliminate potential biases from
the conclusions, and random assignment is a critical step.

Blocking - is a technique to include other factors in our

experiment which contribute to undesirable variation. Much of the
focus in this class will be to creatively use various blocking
techniques to control sources of variation that will reduce error
variance. For example, in human studies, the gender of the subjects
is often important factor. Age is another factor affecting the
response. Age and gender are often considered nuisance factors
which contribute to variability and make it difficult to assess
systematic effects of a treatment. By using these as blocking
factors, you can avoid biases that might occur due to differences
between the allocation of subjects to the treatments, and as a way
of accounting for some noise in the experiment. We want the
unknown error variance at the end of the experiment to be as small
as possible. Our goal is usually to find out something about a
treatment factor (or a factor of primary interest), but in addition to
this we want to include any blocking factors that will explain
variation.
 Confounding - is something that is usually considered bad!
Here is an example. Let's say we are doing a medical study with
drugs A and B. We put 10 subjects on drug A and 10 on drug B. If we
categorize our subjects by gender, how should we allocate our drugs
to our subjects? Let's make it easy and say that there are 10 male
and 10 female subjects. A balanced way of doing this study would
be to put five males on drug A and five males on drug B, five
females on drug A and five females on drug B. This is a perfectly
balanced experiment such that if there is a difference between male
and female at least it will equally influence the results from drug A
and the results from drug B.

 An alternative scenario might occur if patients were randomly

assigned treatments as they came in the door. At the end of the
study they might realize that drug A had only been given to the
male subjects and drug B was only given to the female subjects. We
would call this design totally confounded. This refers to the fact that
if you analyze the difference between the average response of the
subjects on A and the average response of the subjects on B, this is
exactly the same as the average response on males and the
average response on females. You would not have any reliable
conclusion from this study at all. The difference between the two
drugs A and B, might just as well be due to the gender of the
subjects, since the two factors are totally confounded.
 Confounding is something we typically want to avoid but when we
are building complex experiments we sometimes can use
confounding to our advantage. We will confound things we are not
interested in order to have more efficient experiments for the things
we are interested in. This will come up in multiple factor
experiments later on. We may be interested in main effects but not
interactions so we will confound the interactions in this way in order
to reduce the sample size, and thus the cost of the experiment, but
still have good information on the main effects.

Steps for Planning, Conducting and Analyzing an Experiment

The practical steps needed for planning and conducting an
experiment include: recognizing the goal of the experiment, choice
of factors, choice of response, choice of the design, analysis and
then drawing conclusions. This pretty much covers the steps
involved in the scientific method.
Components of Experimental Design (Example)
Consider the following diagram of a cake-baking process (Figure 1).
There are three aspects of the process that are analyzed by a
designed
experiment:
Factors, or inputs to the process. Factors can be classified as
either controllable or uncontrollable variables. In this case, the
controllable factors are the ingredients for the cake and the oven
that the cake is baked in. The controllable variables will be referred
to throughout the material as factors. Note that the ingredients list
was shortened for this example - there could be many other
ingredients that have a significant bearing on the end result (oil,
water, flavoring, etc). Likewise, there could be other types of
factors, such as the mixing method or tools, the sequence of mixing,
or even the people involved. People are generally considered a
Noise Factor - an uncontrollable factor that causes variability under
normal operating conditions, but we can control it during the
experiment using blocking and randomization.

Levels, or settings of each factor in the study. Examples

include the oven temperature setting and the particular amounts of
sugar, flour, and eggs chosen for evaluation.

Response, or output of the experiment. In the case of cake

baking, the taste, consistency, and appearance of the cake are
measurable outcomes potentially influenced by the factors and their
respective levels. Experimenters often desire to avoid optimizing the
process for one response at the expense of another. For this reason,
important outcomes are measured and analyzed to determine the
factors and their settings that will provide the best overall outcome
for the critical-to-quality characteristics - both measurable variables
and asses sable attributes.

Purpose of Experimentation
Designed experiments have many potential uses in improving
processes and products,
including:

Experiment Design Guidelines

The Design of an experiment addresses the questions outlined
above by stipulating the following:
 The factors to be tested.
 The levels of those factors.
 The structure and layout of experimental runs, or conditions.
A well-designed experiment is as simple as possible - obtaining the
required information in a cost effective and reproducible manner.

When designing an experiment, pay particular heed to four

potential traps that can create experimental difficulties:
 If the data are analyzed without knowledge of the operation, a
false conclusion could be reached that low production rates cause
orange peel. In fact, both low production rates and orange peel are
caused by excessive absenteeism - when regular spray booth
operators are replaced by employees with less skill. This example
highlights the importance of factoring in operational knowledge
when designing an experiment. Brainstorming exercises and Fish
bone Cause & Effect Diagrams are both excellent techniques
available through the Toolbox to capture this operational knowledge
during the design phase of the experiment. The key is to involve the
people who live with the process on a daily basis.
Experiment Design Process