EXPERIMENTAL

RESEARCH DESIGN
Dr.Nor Syazwani Binti Mat Salleh & Dr. Azlin Iryani Binti
Mohd Noor
What is it and when
should it be used?
Experimental Research
Learning Objectives
• Define experiment
• Identify the core features of true experimental designs
• Describe the difference between an experimental group and a control
group
• Identify and describe the various types of true experimental designs
DEFINE EXPERIMENT
• Experiments are an excellent data collection strategy for social workers wishing
to observe the effects of a clinical intervention or social welfare program.
• An experiment is a method of data collection designed to test hypotheses
under controlled conditions.
• …the researcher selects participants and divides them into two or more
groups having similar characteristics and, then, applies the treatment(s) to
the groups and measures the effects upon the groups
• Students in research methods classes often use the term experiment to
describe all kinds of research projects, but in social scientific research, the
term has a unique meaning and should not be used to describe all research
methodologies.
 Steps in Conducting
Experimental Research
 Decide if an experiment addresses the research problem
 Form hypotheses to test cause-effect relationships

 Select an experimental treatment and introduce it

 Identify study participants choose a type of experimental
design
 Conduct the experiment

 Organize and analyze the data

 Develop an experimental research report

True experimental
design
 Experimental
Design

Pre- Quasi- True-

Statistical
Experimental Experimental Experimental

One Shot Pre-test-post-

Experimental Group : R Case
O1 X O2 Time test Control Researcher is able to eliminate
Study
Control Group : O3
Series O4 Group effect of extraneous variables
from experimental and control
Experimental Group : R XPre
One Group O1 Multiple Time Post-test-only group
Randomized
Control Group test-post
: testO2 Series Control Group Blocks

Experimental Group 1 : R O1 X O2 Solomon Four

Static Group Latin Square
Control Group 1 : O3 O4 Group
Experimental Group 2 : R X O1
Control Group 2 : O2 R=
Randomization
O=
Factorial
Measurement
X=
Treatment
The core features of true
experimental designs
• In general, designs considered to be true experiments contain three
key features: independent and dependent variables, pretesting and
posttesting, and experimental and control groups.
• In a true experiment, the effect of an intervention is tested by
comparing two groups: one that is exposed to the intervention
(the experimental group, also known as the treatment group) and
another that does not receive the intervention (the control group).
What is True Experimental
• Is a type of experimental design and is used to establish cause and
effect relationships.
• There are three criteria that must be met in a true experiment
1. Control group and experimental group
2. Researcher-manipulated variable
3. Randomization (1. Random assignment; 2. Random Selection)
Control group and experimental group

• Control group: which is a group of research participants that

resemble the experimental group but do not receive the
experimental treatment.
• Experimental group: is the group of research participants who
receive the experimental treatment.

* True experiments must have atleast 1 group and 1 experimental

group, though it is possible to have more than one experimental group.
Researcher- Manipulated
Variable
• The researcher has to change or manipulated the variable that is hypothesized to
affect the outcome variable that is being studied.
• Independent Variable: the variable that researcher has control over (treatment)
eg. experimental variable, active variable, cause, treatment
• Predictor Variable: is also independent variable because it is the presumed cause
of the differences in the outcome variable.
• Dependent Variable: The outcome of effect of research. eg. Assigned variable,
effect, outcome, posttest
• Outcome Variable: is also dependent variable because it is outcome that the
research is studying
*The researcher does not manipulate the dependent variable.
1. Random Assignment

• Random assignment means that every individual who is participating in the

experiment has an equal chance of being assigned to any of the experimental or
control groups.
• Researcher participants have to be randomly assigned to the sample groups.
• Each researcher participant must have an equal chance of being assigned to each
sample group
• Three things occur with random assignments of subjects:
1) It takes place before the experiment begins
2) Process of assigning the groups takes place
3) Groups should be equivalent
2. Random Selection
• Random selection means that every member of a population has an
equal chance of being selected to be a member of the sample.
 The difference between an
experimental group and a control
group
Positive and Negative Controls

• Positive and negative controls are two other types of control groups:
• Positive control groups are control groups in which the conditions
guarantee a positive result. Positive control groups are effective to
show the experiment is functioning as planned.
• Negative control groups are control groups in which conditions
produce a negative outcome. Negative control groups help identify
outside influences which may be present that were not unaccounted
for, such as contaminants.
The various types of true
experimental designs
• Pre test- post test control group
• Post-test only control group
• Solomon four group, Six-study design
Classic experimental design
Pre test- post test control group
 Experimental group tested before and after treatment
exposure
 Control group tested at same two times without exposure to
experimental treatment
 Includes random assignment to groups.
 Pretest raises the possibility of a pretest treatment interaction
threat
Example of a Randomized Pretest- Posttest
Control Group Design
Post-test only control group
 Experimental group tested after treatment exposure

 Control group tested at the same time without exposure to

experimental treatment

 Includes random assignment to groups

 Threats to internal validity – mortality, attitudinal,

implementation, data collector bias, location and history
A Randomized Posttest-Only Control
Group Design
Solomon four group, Six-study
design
 Combines pretest-posttest with control group design and the
posttest-only with control group design.

 Provides means of controlling the interactive test effect and

other sources of extraneous variation.

 Does include random assignment.

 Weakness: requires a large sample.

Example of a Randomized Solomon Four-
Group Design
• Solomon four-group designs are challenging to implement in the real world
because they are time- and resource-intensive.
• Researchers must recruit enough participants to create four groups and
implement interventions in two of them.
• Overall, true experimental designs are sometimes difficult to implement in a
real-world practice environment.
• It may be impossible to withhold treatment from a control group or randomly
assign participants in a study.
• In these cases, pre-experimental and quasi-experimental designs can be used.
• However, the differences in rigor from true experimental designs leave their
conclusions more open to critique.
Glossary
• Classic experimental design- uses random assignment, an experimental and control group, as well as pre- and post-
testing
• Comparison group- a group in quasi-experimental designs that receives “treatment as usual” instead of no treatment
• Control group- the group in an experiment that does not receive the intervention
• Experiment- a method of data collection designed to test hypotheses under controlled conditions
• Experimental group- the group in an experiment that receives the intervention
• Posttest- a measurement taken after the intervention
• Posttest-only control group design- a type of experimental design that uses random assignment, and an experimental
and control group, but does not use a pretest
• Pretest- a measurement taken prior to the intervention
• Random assignment-using a random process to assign people into experimental and control groups
• Solomon four-group design- uses random assignment, two experimental and two control groups, pretests for half of
the groups, and posttests for all
• Testing effects- when a participant’s scores on a measure change because they have already been exposed to it
• True experiments- a group of experimental designs that contain independent and dependent variables, pretesting
and post testing, and experimental and control groups
References
• McCoy, S. K., & Major, B. (2003). Group identification moderates
emotional response to perceived prejudice. Personality and Social
Psychology Bulletin, 29, 1005–1017.
• Babbie, E. (2010). The practice of social research (12th ed.). Belmont,
CA: Wadsworth; Campbell, D., & Stanley, J. (1963). Experimental and
quasi-experimental designs for research. Chicago, IL: Rand McNally.
Pre-experimental
and quasi-
experimental
design
Pre-experimental Quasi – True experimental
design experimental design
design

•One shot case design •Non randomized block •Post –test only control
•One group pretest- design design
posttest design •Time series design •Pre –test– posttest
•FEATURES •FEATURES control group
•Manipulation of •Manipulation of design
independent variables independent variable •Factorial design
•Limited control over •Absence of either •Randomized block
the extraneous randomization/ control design
variables group •Cross over design
•No randomization and •FEATURES
control group •Manipulation of
independent variable
•Presence of control
group
•Randomization
Learning Objectives
• Identify and describe the various types of quasi-experimental designs
• Distinguish true experimental designs from quasi-experimental and
pre-experimental designs
• Identify and describe the various types of quasi-experimental and pre-
experimental designs
• As we discussed in the previous section, time, funding, and ethics may
limit a researcher’s ability to conduct a true experiment.
• When true experiments are not possible, researchers often use quasi-
experimental designs.
• Quasi-experimental designs are similar to true experiments, but they
lack random assignment to experimental and control groups.
 Types of pre experimental
design
 The One-Shot Case Study
 A single measure is recorded after the treatment
in administered.
 Study lacks any comparison or control of
extraneous influences.
 To remedy this design, a comparison could be
made with another group.
 Diagrammed as:
 Experimental
Design Do not use randomization
procedure to control
extraneous variable.
Pre- Quasi- True-
Statistical
Experimental Experimental Experimental

Pre-test-post- Completely
One Shot Case Treatment (X ) and measurement
Time Series test Control of effect (O) on
Study Group
Randomized X O1
dependent e
variabl

One Group Pre Measurement

Multiple Time before (O1) and afterRandomized
Post-test-only (O2) the
test-post test Series Control(X)
Group O1 X O2
Treatment
Blocks

Group 1 – X O1 Solomon Four

Static Group Latin Square
Group 2 – O2 Group

O=
FactorialMeasurement
X=
Treatment
 Pre experimental
design
Advantages Disadvantages
Very simple Weak design to establish
Convenient to conduct in casual relationship between
natural settings independent and dependent
variable
Suitable for beginners
Very little control over the
research

Higher threat to internal

validity
 Characteristic of quasi
experimental research design

 Manipulation of independent variable

 Lack of one / two essential character of true experimental design
 Quasi independent variable used instead of true independent
variable.
 Experimental
Design

Pre- Quasi- True-

Statistical
Experimental Experimental
Do not use randomizationxperimental
procedure to control
extraneous variable. Pre-test-post-
One Shot Case
Time Series O1 O2 O3 O4 X O5 O6 O7 O8
testCompletely
Control
Study
Group
Randomized

One Group Pre Multiple Time Experimental Group : O1 O2 O3 O4 X O5 O6 O7 O8

Post-test-only
test-post test Series Randomized
Control Group : O1 O2 O3 O4 O5 O6 O7 O8
Control Group Blocks

Solomon Four
Static Group Latin Square
Group

O=
FactorialMeasurement
X=
Treatment
 STATISTICAL
Completely Randomized Design
Randomized Block Design
Latin Square Design
Factorial
 Experimental
Design
It allows for statistical
control and analysis of
external variable.
Pre- Quasi- True-
Statistical
Experimental Experimental Experimental
•Used to investigate effect of one independent
variableOneonShot
dependent
Case variable. Pre-test-post-
Time Completely
Study Randomized
•Independent Variable should be nominal scale Series
• test-post test Series test
Control Group Control
Group Randomized
• Example: Blocks
• Measurement of sales for different price level
One Group Pre Multiple Time Post-test-
Solomon Four
Static Group
only
Price Level : Low Medium High Group Latin Square
Stores : A B C

Use ANOVA technique to measure effect

Factorial
 Experimental
Design

Pre- Quasi- True-

Limitation of Completely Randomized Design: Extraneous Variables are Statistical
Experimental Experimental Experimental
assumed to be constant

Oneminimize
Shot Case influence of one extraneous
Pre- t-
Using Randomized Blocks to Completely
test-pos
variable Time l
Randomized
Study
Serie
s
One Group Pre Multiple Time Post-test-only Randomized
test-post test Series Control
test Group Blocks
Contr
o
Grou Four
Solomon
Static Group pGroup Latin Square

Factorial
 SMALL SIZE

STORES 1 2 3 4 5 6 7 8 9 10 11 12

PRICE LOW MEDIUM HIGH

MEDIUM SIZE

STORES 1 2 3 4 5 6 7 8 9 10 11 12

PRICE LOW MEDIUM HIGH

LARGE SIZE

STORES 1 2 3 4 5 6 7 8 9 10 11 12

PRICE LOW MEDIUM HIGH

 Experimental
Design

Used to separate influencePre- Quasi-

of 2 Extraneous Variables True-
Statistical
Experimental Experimental
Experimental
Example: Sales Would be influenced by 2 extraneous variables.
Pre-test-post-
One Shot Case Completely
Time Series test Control
Study Randomized
Packaging Group
Store Size
One IGroup Post-test-only Randomized
II Multiple TimIeI
Pre test-post Series Control Group Blocks
test
1 (Small) X1 X2 X3
Solomon Four
2 (Medium) X3 X1 Group Latin Square
StaXt2ic
Group
3 (Large) X3 X1 X2 Here, X1=Low price, X2=Medium, X3=High
Factorial
 Experimental
Design

Pre- Quasi- True-

Used to measure effectExperimental
of 2 or more independent variable. Experimental Statistical
Experimental

Allows interaction between 2 variables. Pre-

One Shot Case t-
Completely
test-pos Randomized
Ex. An individual may like mango as well as ice cream but may not like
Study
Time l
Mango Ice Cream Serie
s
One Group PStores Multiple Time Post-test-only Randomized
test-post test Series Control
test Group Blocks
Price Contr
Small (B1) Big (B2) o
Grou Four
Solomon
Low (A1) A1B2 pGroup Latin Square
StatAi1Br1oup
Medium (A2) A2B1 A2B2

High (A3) A3B1 A3B2 Factorial

SUMMARY
• Research Design is a plan specifying sources and types of information

• Categories of research design – Sampling, Observational, Statistical,

operational

• A variable is defined as anything that has a quantity or quality that

varies.

• Independent Variable influences change in dependent variable.

• Extraneous Variables - Variables that are not related to the purpose of the
study but may affect the dependent variable

• Control is used to minimize the effect of extraneous independent

variables.
• Hypothesis is a predictive statement that relates an independent variable to a
dependent variable.

• Research in which independent variable is manipulated – Experimental

• Research in which independent variable is not manipulated – Non

Experimental

• Conditions under which groups are put is called as Treatments

• Process of examining truth is called as ‘Experiment’

• Exploratory Design - Objective is to explore and obtain clarity

about problem
situation.
• Descriptive Design - Concerned with describing the characteristics of
a particular individual, group, situation, problem etc.

• Cross Sectional - Selection of current subdivision of population

and study the nature of variables.

• Longitudinal - Study of a group over a stretched period of time

• Experiment - It is used to infer causality

• Experimental Design – Pre experimental, Quasi experimental, True

Experimental, Statistical
Key Takeaways
• Quasi-experimental designs do not use random assignment.
• Comparison groups are often used in quasi-experiments.
• Matching is a way of improving the comparability of experimental and
comparison groups.
• Quasi-experimental designs and pre-experimental designs are often
used when experimental designs are impractical.
• Quasi-experimental and pre-experimental designs may be easier to
carry out, but they lack the rigor of true experiments.
Glossary
• Aggregate matching- when the comparison group is determined to be similar to the experimental group along important
variables
• Ex post facto control group- a control group created when a researcher matches individuals after the intervention is
administered
• Individual matching- pairing participants with similar attributes for the purpose of assignment to groups
• Natural experiments- situations in which comparable groups are created by differences that already occur in the real world
• Nonequivalent comparison group design- a quasi-experimental design similar to a classic experimental design but without
random assignment
• One-group pre-/posttest design- a pre-experimental design that applies an intervention to one group but also includes a
pretest
• One-shot case study- a pre-experimental design that applies an intervention to only one group without a pretest
• Pre-experimental designs- a variation of experimental design that lacks the rigor of experiments and is often used before a
true experiment is conducted
• Quasi-experimental design- designs lack random assignment to experimental and control groups
• Static group design- uses an experimental group and a comparison group, without random assignment and pretesting
• Time series design- a quasi-experimental design that uses multiple observations before and after an intervention
References
• Rubin, C. & Babbie, S. (2017). Research methods for social work (9th edition).
Boston, MA: Cengage.
• Stratmann, T. & Wille, D. (2016). Certificate-of-need laws and hospital quality.
Mercatus Center at George Mason University, Arlington, VA. Retrieved from:
https://www.mercatus.org/system/files/mercatus-stratmann-wille-con-hospital
-quality-v1.pdf

• Baicker, K., Taubman, S. L., Allen, H. L., Bernstein, M., Gruber, J. H., Newhouse, J.
P., ... & Finkelstein, A. N. (2013). The Oregon experiment—effects of Medicaid
on clinical outcomes. New England Journal of Medicine, 368(18), 1713-1722.
• Campbell, D., & Stanley, J. (1963). Experimental and quasi-experimental designs
for research. Chicago, IL: Rand McNally.
References
• Bailey, R. A. (2008). Design of Comparative Experiments. Cambridge
University Press. ISBN 978-0-521-68357-9.
• Chaplin, S. (2006). "The placebo response: an important part of
treatment". Prescriber: 16–22. doi:10.1002/psb.344
• Hinkelmann, Klaus; Kempthorne, Oscar (2008). Design and Analysis of
Experiments, Volume I: Introduction to Experimental Design (2nd ed.).
Wiley. ISBN 978-0-471-72756-9.