Method of

Research for
Research Design Mechanical
Engineering
 variables or parameters to
be measured and evaluated
or analyzed

Research
Design: treatments to be used and
their layout

Focus
experimental procedures and
design
Experimental Design

Experimental design is the process of carrying out research in an objective and controlled fashion so that precision is
maximized, and specific conclusions can be drawn regarding a hypothesis statement. Generally, the purpose is to
establish the effect that a factor or independent variable has on a dependent variable.

An experiment is a type of research method in which you manipulate one or more independent variables and measure
their effect on one or more dependent variables. Experimental design means creating a set of procedures to test a
hypothesis.

A good experimental design requires a strong understanding of the system you are studying. By first considering the
variables and how they are related, you can make predictions that are specific and testable.

How widely and finely you vary your independent variable will determine the level of detail and the external validity of
your results. Your decisions about randomization, experimental controls, and between- vs within-subjects designs will
determine the internal validity of your experiment.
 Experimental design is a way to carefully plan
experiments in advance so that your results
are both objective and valid. The terms
“Experimental Design” and “Design of
What is Experiments” are used interchangeably and
mean the same thing.
Experimental
Design? However, the medical and social sciences tend
to use the term “Experimental Design” while
engineering, industrial and computer sciences
favor the term “Design of experiments.
 Describe how participants are allocated to experimental groups. A common
method is completely randomized design, where participants are assigned to
groups at random. A second method is randomized block design, where
participants are divided into homogeneous blocks (for example, age groups) before
being randomly assigned to groups.

Ideally, your Minimize or eliminate confounding variables, which can offer alternative

experimental explanations for the experimental results.

design Allow you to make inferences about the relationship between independent

should:
variables and dependent variables.

Reduce variability, to make it easier for you to find differences in treatment

outcomes.
Design of Experiments, DOE (for Engineering)

The term is generally associated with

Aims at predicting the outcome by The change in one or more independent
experiments in which the design
The design of experiments aims to introducing a change of the variables is generally hypothesized to
introduces conditions that directly affect
describe and explain the variation of preconditions, which is represented by result in a change in one or more
the variation but may also refer to the
information under conditions that are one or more independent variables, also dependent variables, also referred to as
design of quasi-experiments, in which
hypothesized to reflect the variation. referred to as "input variables" or "output variables" or "response
natural conditions that influence the
"predictor variables." variables."
variation are selected for observation.

Experimental design involves not only

the selection of suitable independent, There are multiple approaches for
The experimental design may also
dependent, and control variables, but determining the set of design points Main concerns in experimental design
identify control variables that must be
planning the delivery of the experiment (unique combinations of the settings of include the establishment of validity,
held constant to prevent external factors
under statistically optimal conditions the independent variables) to be used in reliability, and replicability.
from affecting the results.
given the constraints of available the experiment.
resources.

For example, these concerns can be

partially addressed by carefully choosing
the independent variable, reducing the
risk of measurement error, and ensuring
that the documentation of the method is
sufficiently detailed. Related concerns
include achieving appropriate levels of
statistical power and sensitivity.
Design of experiments involves:

The systematic collection of A focus on the design itself,

data rather than the results

Planning changes to
Ensuring results are valid,
independent (input) variables
easily interpreted, and
and the effect on dependent
definitive.
variables or response variables
 • Randomization:
• the assignment of study components by a
completely random method, like simple random
sampling. Randomization eliminates bias from
the results
DOE • Replication:
important • the experiment must be replicable by other
researchers.
principles: • This is usually achieved with the use of statistics
like the standard error of the sample mean or
confidence intervals.
• Blocking:
• controlling sources of variation in the
experimental results.
 In between subject's design, separate groups are created for
each treatment.
What is
Between This type of experimental design is sometimes called
independent measures design because each participant is
Subjects assigned to only one treatment group.

Design? For example, you might be testing a new depression

medication:

• one group receives the actual medication and the other receives a placebo.
• Participants can only be a member of one of the groups (either the treatment
or placebo group).
• A new group is created for every treatment. For example, if you are testing
two depression medications, you would have: Group 1 (Medication 1), Group
2 (Medication 2), Group 3 (Placebo).
Advantages of • Between subject's design is one of the simplest types of
Subjects experimental design setup. Other advantages include:
• Multiple treatments and treatment levels can be
Design. tested at the same time.
• This type of design can be completed quickly.
 • A major disadvantage in this type of experimental design is that as
each participant is only being tested once, the addition of a new
treatment requires the formation of another group. The design can
become extremely complex if more than a few treatments are being
tested. Other disadvantages include:
• Differences in individuals (i.e. age, race, sex) may skew results
Disadvantages and are almost impossible to control for in this experimental
design.
of Subjects • Bias can be an issue unless you control for this factor using
experimental blinds (either a single blind experiment–where the
Design. participant doesn’t know if they are getting a treatment or
placebo–or a double blind, where neither the participant nor the
researcher know).
• Generalization issues means that you may not be able to
extrapolate your results to a wider audience.
• Environmental bias can be a problem with between subjects
design. For example, let’s say you were giving one group of
college students a standardized test at 8 a.m. and a second
group the test at noon. Students who took the 8 a.m. test may
perform poorly simply because they weren’t awake yet.
Completely Randomized Experimental Design

A completely randomized design (CRD) is an experiment where the treatments are assigned at random.

Every experimental unit has the same odds of receiving a particular treatment.

This design is usually only used in lab experiments, where environmental factors are relatively easy to control
for; it is rarely used out in the field, where environmental factors are usually impossible to control.

When a CRD has two treatments, it is equivalent to a t-test.

Completely Randomized
Experimental Design
• completely randomized design is generally implemented by:
• Listing the treatment levels or treatment combinations.
• Assigning each level/combination a random number.
• Sorting the random numbers in order, to produce a
random application order for treatments.
• However, you could use any method that completely
randomizes the treatments and experimental units, as long
as you take care to ensure that:
• The assignment is truly random.
• You have accounted for extraneous variables.
Completely Randomized Design Example

• Let’s suppose you were conducting

an experiment to see how a type
of fertilizer (you have 4 different
ones) affects the growth rate of 16
tomato plants in a greenhouse. 1 5 9 13
• The first step is to list the
treatment levels. You have four 2 6 10 14
fertilizers, so let’s call these ABCD.
You have 16 plant locations, 3 7 11 15
labeled 1-16.
4 8 12 16
 1 5 9 13
Completely Randomized
Design Example
A C B D
2 6 10 14

• First, write the numbers 1-16 in 16 pieces

of equal sized paper and place them into a
C B A C
bowl.
• Next, write the letters A B C D on 16 3 7 11 15
separate pieces of paper (i.e. you’ll have 4
x As, 4 x Bs, 4 Cs and 4 Ds) and place them
in another bowl. B B D D
• Select one piece of paper from the first
bowl and one from the second to get a 4 8 12 16
location and a treatment.
A C A D
Completely Randomized
Design Example
• Some errors that could cause serious design flaws
could include randomly assigning columns to
treatments so that it’s easier to apply fertilizer to an
entire column (for example, column 1 has fertilizer B,
column 2 has fertilizer D and so on).
• While this may be convenient, you run the risk that
the plants in row A and D have more access to
sunlight as they are on the outside of the space.
Completely
Randomized Design
with Subsampling.

• This subset of CRD is usually used

when experimental units are
limited. Subsampling might
include several branches of a
particular tree, or several
samples from an individual plot.
What is a Factorial Design
For example, let’s say a researcher
wanted to investigate components for The researcher plans to manipulate
A factorial experimental design is increasing SAT Scores. The three each of these independent variables.
used to investigate the effect of two components are: Each of the independent variables is
or more independent variables on
• SAT intensive class (yes or no). called a factor, and each factor has
one dependent variable. • SAT Prep book (yes or no). two levels (yes or no).
• Extra homework (yes or no).

As this experiment has 3 factors with

In some experiments where the
2 levels, this is a 2 x 2 x 2 = 23
number of level/factor combinations
factorial design. An experiment with 3
Most factorial experiments only have are unmanageable, the experiment
factors and 3 levels would be a 33
two levels. can be split into parts (for example,
factorial design and an experiment
by half), creating a fractional
with 2 factors and 3 levels would be a
experimental design.
32-factorial design.
 A null outcome is when the
experiment’s outcome is the same
regardless of how the levels and
Factorial factors were combined.
Design:
Null In the above example, that would
Outcome. mean no amount of SAT prep (book
and class, class and extra homework
etc.) could increase the scores of
the students being studied.
Factorial Design:
Main Effect and Interaction Effect.
The main effect is the effect of an
Two types of effects are considered
independent variable (in this case, SAT For a main effect to exist, you’d want to
when analyzing the results from a
prep class or SAT book or extra see a consistent trend across the
factorial experiment: main effect and
homework) on the dependent variable different levels.
interaction effect.
(SAT Scores).

For example, one group of students who

took the SAT class and used the SAT
For example, you might conclude that
prep book showed an increase in SAT
students who took the SAT prep class
scores while the students who took the
scored consistently higher than students
class but did not use the book didn’t
who did not. An interaction effect occurs
show any increase. You could infer that
between factors.
there is an interaction between the SAT
class and use of the SAT prep book.
What is Matched Pairs Design?

Matched pairs design is a special case of randomized block design.

In this design, two treatments are assigned to homogeneous groups (blocks) of

subjects.

The goal is to maximize homogeneity in each pair. In other words, you want the
pairs to be as similar as possible.

The blocks are composed of matched pairs which are randomly assigned a
treatment (commonly the drug or a placebo).
Matched Pairs Design:
Examples Pair Placebo Drug
• For example, an experiment to test a new drug may
1 1 1
have blocks of 200 males and 200 females.
2 1 1
• Each block contains 100 pairs, who are matched
according to some criteria other than sex (like age,
other medications, or health conditions).
3 1 1
• Each pair is then treated like a block, with each 4 1 1
randomly assigned to receive the drug or a placebo.
• The following table shows experiment, where pair 1 … … …
could represent two healthy women age 29, pair 2
could represent two women age 29 with liver 199 1 1
disease, pair 3 could contain two healthy women
age 39, pair 4 could contain two women age 39 with
liver disease, and so on.
200 1 1
 • You can think of matched pair design as a type of stacked
randomized block design.
• With either design, your goal is to control for some variable
that’s going to skew your results.
• In the above experiment, it isn’t just age that could account for
differences in how people respond to drugs, several other
Stacking in confounding variables could also affect your experiment.
• The purpose of the blocks is to minimize a single source of
Matched variability (for example, differences due to age).
• When you create matched pairs, you’re creating blocks within
Pairs Design. blocks, enabling you to control for multiple sources of
potential variability.
• You should construct your matched pairs carefully, as it’s often
impossible to account for all variables without creating a huge
and complex experiment.
• Therefore, you should create your blocks starting with which
candidates are most likely to affect your results.
What is Randomized Block Design?
• In randomized block design, the researcher divides experimental subjects
into homogeneous blocks.
• Treatments are then randomly assigned to the blocks. The variability within
blocks should be greater than the variability between blocks.
• In other words, you need to make sure that the blocks contain subjects that
are very similar.
• For example, you could put males in one block and females in a second
block. This method is practically identical to stratified random
sampling(SRS), except the blocks in SRS are called “strata.”
• Randomized block design reduces variability in experiments.
Randomized Block Design: Example

• For example, you might run an

experiment to find out the efficacy of a
new drug. Age Placebo Drug
• According to the Merck Manual, one
factor that can affect how a patient 18-29 100 100
responds to a drug is age.
• Therefore, you run the risk that your
results might be affected by age as a
30-39 100 100
confounding variable.
• A Solution is to set up randomized block 40-49 100 100
design so that different age groups are
spread across equally sized blocks.
50-59 100 100
• The table below shows a randomized
block design for a hypothetical
experiment that tests a new drug on 60-69 100 100
1,000 people:
Randomized Block Design: Example
• This randomized block design contains equal blocks of 200 people from each age group, where they are assigned
randomly to either the placebo or the real drug. Therefore, age is removed as a potential source of variability.
• Age isn’t the only potential source of variability. Other blocking factors that you could consider for this type of
experiment include:
• Consumption of certain foods.
• Use of over the counter food supplements.
• Adherence to dosing regimen.
• Differences in metabolism due to genetic differences, liver or kidney issues, race, or sex.
• Coexistence of other disorders.
• Use of other drugs.
• Randomized block experimental design is sometimes called randomized complete block experimental design,
because the word “complete” makes it clear that all subjects are included in the experiment, not just a sample.
However, the setup of the experiment usually makes it clear that all subjects are included, so most people will drop
the word complete.
Descriptive Study
• A descriptive study is one in which information is collected without changing the environment (i.e., nothing
is manipulated).
• Sometimes these are referred to as “ correlational ” or “ observational ” studies.
• A descriptive study as “Any study that is not truly experimental.” In human research, a descriptive study can
provide information about the naturally occurring health status, behavior, attitudes or other characteristics
of a particular group. Descriptive studies are also conducted to demonstrate associations or relationships
between things in the world around you.
• Descriptive studies can involve a one-time interaction with groups of people ( cross-sectional study ) or a
study might follow individuals over time ( longitudinal study ).
• Descriptive studies, in which the researcher interacts with the participant, may involve surveys or interviews
to collect the necessary information.
• Descriptive studies in which the researcher does not interact with the participant include observational
studies of people in an environment and studies involving data collection using existing records (e.g., medical
record review).
Case example of a descriptive study

Descriptive studies are usually the best methods for collecting information
that will demonstrate relationships and describe the world as it exists.

These types of studies are often done before an experiment to know what
specific things to manipulate and include in an experiment.

Bickman and Rog (1998) suggest that descriptive studies can answer
questions such as “what is” or “what was.”

Experiments can typically answer “why” or “how.”