Topic 5
Topic 5
Topic 5
Topic 5
Experimental design
Acknowledgement
of Country
We acknowledge the Binjareb people of the Noongar nation as
the traditional custodians of this country and its waters and that
Murdoch University stands on Noongar country.
Labs
surfaces after you have Reporting contact with a confirmed case,
used them. stay home and let your unit
coordinator know.
Required reading:
• Oates Ch 9
Recommended reading:
• Basili, V. R. & Zelkowitz, M. V. (2007) Empirical studies to build a science of
computer science. Communications of the ACM 50(11) 33-37.
4
Learning objectives
5
Experiments are a research strategy or approach
6
What is an experiment?
7
Treatment
https://stattrek.com/statistics/dictionary.aspx?definition=treatment
8
REVISION - Hypotheses
9
Causality versus correlation
10
Question
11
Characteristics of experiments
Cause Effect
Independent Dependent
variable variable
12
Control variables and the environment
Ways of controlling factors that might affect the outcomes of an experiment include:
• Eliminate the factor via exclusion criteria
• Hold the factor constant via inclusion criteria (if you cannot eliminate it)
• Use random selection/allocation
• Use control group - the only difference from treatment group is the change in independent variable.
Unwanted
factor
Validity is concerned with the study's success at measuring what the researchers
set out to measure. Researchers should be concerned with both external and
internal validity.
Internal validity relates to the rigour with which the study was conducted. An
experiment has good internal validity if the results are due to the manipulations of
the independent variable, and not to any other factors.
External validity refers to the extent to which the results of a study are
generalizable or transferable – to different times, places, technologies etc.
14
Internal validity threats
15
External validity threats
16
Types of experiment
17
Example experimental designs
18
Example experimental designs
R X 0
R 0
Assumes that randomisation removes the need for
pretesting, and because subjects are only tested once
reduces threats due to testing and instrumentation
19
Example experimental designs
20
Causality
21
Designing and conducting an experiment
22
Field Experiment
• Can be used for both theory testing and theory building. Field experimentation is
of value because it enables the development of causal models based on
information gathered in natural, and hence more generalizable settings, settings
for studying phenomena that could not easily be replicated in a laboratory setting.
23
Problems with conducting field experiments
24
Problems with conducting field experiments
25
Advantages
Some advantages of experiments as a research strategy
include:
• Well-established method
• The only way to demonstrate cause-effect relationships
• Ability to repeat experiments and extend results across people, times &
situations
26
Disadvantages
Some disadvantages include:
• Artificiality – can limit ability to generalise
• Often impossible to control all the parameters
• Difficult to recruit representative samples
• Ethical considerations – control group not receiving treatment believed to
be valuable
27
Learning objectives revisited
• What is an experiment?
• Why would a control group be used in an experiment?
• What is the difference between a true experiment and a
quasi-experiment?
• What is an experimental design?
• What is the main advantage of experimentation as a
research strategy?
28
Thank you