Scribd
Upload
235 views20 pages

Lecture 5 - Threats To Validity

This document discusses threats to validity in experimental research, including internal and external validity. It identifies eight factors that threaten internal validity: history, maturation, testing, instrumentation, regression, selection, mortality, and interaction effects. Threats to external validity include interaction of testing, selection biases, reactive experimental settings, and multiple treatments. The document emphasizes controlling variables while balancing internal and external validity.

Uploaded by

thinagaran
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
235 views20 pages

Lecture 5 - Threats To Validity

This document discusses threats to validity in experimental research, including internal and external validity. It identifies eight factors that threaten internal validity: history, maturation, testing, instrumentation, regression, selection, mortality, and interaction effects. Threats to external validity include interaction of testing, selection biases, reactive experimental settings, and multiple treatments. The document emphasizes controlling variables while balancing internal and external validity.

Uploaded by

thinagaran
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 20

THREATS TO VALIDITY

VALIDITY
• There are two (2) important classification
of validity in experimental research:
– Internal validity
– External validity
Internal Validity
• How valid the findings are within, or
internal to, the study.
• Did the treatments cause the outcome to
occur, or did other extraneous factors
cause the outcome?
Internal Validity
• Eight (8) factors that threat the internal validity
of a study:
1. History
2. Maturation
3. Testing
4. Instrumentation
5. Statistical regression
6. Selection
7. Experimental mortality
8. Interaction of Selection and Maturation or History
History
• Specific things that happen while
conducting the research study that affect
the final scores of the participants in
addition to the effect of the experimental
treatment.
– The effects of linear vs. nonlinear
periodization models for in-season volleyball
players
– Resistance training vs. aerobic training for
physical fitness in middle-aged men.
Maturation
• Subjects grow older during the course of
an experiment – this may affect the
outcome of the study
– The time-course for training-induced
increases in strength for adolescents during a
25-week resistance training program
– The effects of 12 weeks of aerobic training on
VO2max (from September to December)
Testing
• The “learning” effect
– Subjects get better at scoring well on a test
the more times they take it.
• The effects of 2 days of resistance training on
muscle strength and power output.
Instrumentation
– Unreliable measures can invalidate a study
– Possible causes
• Faulty equipment, inconsistent instructions to study
participants, unreliable training of examiners,
fatigue or boredom of examiners, or examiners
becoming more skilled at doing the test
Statistical Regression
• The tendency for groups with extremely
high or low scores on one measure to
score closer to the mean score of the
population on a second measure.
– Can be mistaken for a treatment effect
– Example: Low-IQ and high-IQ groups are
formed. A treatment is applied to both groups
for twelve weeks. A physical ability test is
administered to both groups. It is found that
both groups are equal in physical ability.
Selection
• The selection of subjects and their
assignments to groups can bias the
outcome of a study.
– Subjects that are not representative of the
population
• Kinesiology majors vs. all college-aged males and
females
– Groups that are not equal in their abilities
prior to the experiment
• VO2max and muscle strength
Experimental Mortality
• Subject attrition
– Excessive loss of subjects throughout the
course of a study
– Sample of subjects is no longer
representative of the population
Interaction of Selection and
Maturation or History
• The maturation effect or history effect is
not the same for all groups selected for the
research study – may affect the outcome.
– Groups that differ in age or background may
respond differently to the same treatment
– If a group starts out unequal in ability due to
maturation or history, they may not all have
the same potential for improvement or
change.
– Disproportional experimental mortality
External Validity
• The degree to which findings in a research study
can be inferred or generalized to other
populations, settings, or experimental
treatments.
• Can the findings of a particular study be inferred
to the entire population?
• Are the findings unique to the participants of the
study, or do they apply to other groups?
– Examples: A study using diabetic patients. Good
internal validity.
External Validity
• Four threats to external validity
1. Interaction effect of testing
2. Interaction effects of selection bias and
experimental treatment
3. Reactive effects of experimental setting
4. Multiple-treatment interference
Interaction Effect of Testing
• This effect occurs when the pre-test itself
changes the group’s response to the
treatment.
– Example: A pre-test may identify deficiencies
that the subject may try to correct during the
course of the treatement
Interaction Effects of Selection Bias
and Experimental Treatment
• The participant or groups selected in a
biased manner respond to the
experimental treatment in a unique way so
they are not representative of any
population.
– Novice vs. intermediate vs. elite weight lifters
– This threat generally occurs as a result of a
convenient group is tested and an attempt is
made at generalizing the results
Reactive Effects of Experimental
Settings
• The experimental setting is not the same
as a subject’s ambient environment – this
may affect the study outcome.
– Subjects may respond differently around
investigators
– Laboratory vs. natural settings
– Subjects may think that a particular treatment
is supposed to do “something” and respond
like they are “supposed to.”
Multiple-Treatment Interference
• Multiple treatments administered at the
same time on the same group of subjects.
– Subject involved in two studies at the same
time
– Competing influences of treatments
Control of Variables
• Control of all variables operating in an
experimental study is highly desirable, but
seldom (if ever) accomplished.
– Sleep, food, exercise, etc.
• Too much control can harm external validity
• Not enough control can harm internal validity
Validity in Summary
• In many cases, internal and external
validity cannot be obtained.
– Must decide which is most important
• Good internal validity is necessary for
external validity
– Internal validity is the basic minimum for an
experimental design
• Investigators must consider the threats to
validity when designing research studies.

You might also like