Introduction to Validity
What is Validity?
● The best available approximation to the
truth or falsity of a given inference,
proposition, or conclusion
● A set of standards by which research
can be judged
The Causal Context
Theory
Observation
The Causal Context
Theory
Cause
construct
Observation
The Causal Context
Theory
Cause
construct
Observation
The Causal Context
Theory
Cause cause-effect construct
Effect
construct construct
Observation
The Causal Context
Theory What you think
Cause cause-effect construct
Effect
construct construct
Observation
The Causal Context
Theory What you think
Cause Cause-Effect Construct
Effect
construct construct
Operationalize
Program
Observation In this study
The Causal Context
Theory What you think
Cause Cause-effect Construct
Effect
construct construct
Operationalize Operationalize
Program Observations
Observation In this study
The Causal Context
Theory What you think
Cause Cause-effect Construct
Effect
construct construct
Operationalize Operationalize
Program Observations
What you do What you see
Observation In this study
The Causal Context
Theory What you think
Cause Cause-effect Construct
Effect
construct construct
Operationalize Operationalize
Program Program-outcome Relationship
Observations
What you do What you see
Observation What you test
In this study
Conclusion Validity
Is there a relationship between...
● What you did and what you saw?
● Your program and your observations?
Program Program-outcome Relationship
Observations
What you do What you see
Observation In this study
Internal Validity
Is the relationship causal between...
• What you did and what you saw?
• Your program and your observations?
Alternative
cause
Alternative
cause
Program Program-outcome Relationship
Observations
What you do What you see
Alternative
cause Alternative
Observation cause In this study
Construct Validity
Theory What you think
Cause Cause-effect Construct
Effect
construct construct
Can we generalize to the constructs?
Program Program-outcome Relationship
Observations
What you do What you see
Observation
External Validity
Theory What you think
Cause Cause-effect Construct
Effect
construct construct
Can we generalize to other persons, places, times?
Program program-outcome relationship
Observations Program program-outcome relationship
Observations
What You Do What You See What You Do What You See
Observation Observation
Program program-outcome relationship
Observations Program program-outcome relationship
Observations
What You Do What you see What You Do What You See
Observation Observation
The Validity Questions Are Cumulative...
The Validity Questions Are Cumulative...
Is there a relationship between
In this study the cause and effect?
The Validity Questions Are Cumulative...
In this study Is the relationship causal?
Is there a relationship between
Conclusion the cause and effect?
The Validity Questions Are Cumulative...
Can we generalize to
In theory the constructs?
Internal Is the relationship causal?
Is there a relationship between
Conclusion the cause and effect?
The Validity Questions Are Cumulative...
Can we generalize
In theory to other persons,
places, times?
Construc Can we generalize to
the constructs?
t
Internal Is the relationship causal?
Is there a relationship between
Conclusion the cause and effect?
The Validity Questions are cumulative...
Validity
Can we generalize
External to other persons,
places, times?
Can we generalize to
Construct the constructs?
Internal Is the relationship causal?
Is there a relationship between
Conclusion the cause and effect?
Threats to Validity
You Want to Make an Inference...
● There is a relationship between
the cause and effect.
● The relationship is causal.
● You can generalize to the
constructs.
● You can generalize to other
persons, places, and times.
Threats to Validity
How could you be wrong in the inference?
Conclusion Validity
● There is a relationship, but you don’t
see it.
● There is no relationship, but you do
see one.
Threats to Validity
How could you be wrong in the inference?
Internal Validity
● There is a causal relationship, but you
don’t see it.
● There is no causal relationship, but
you do see one.
Threats to Validity
How could you be wrong in the inference?
Construct Validity
● You can generalize to constructs, but
you conclude you can’t.
● You can’t generalize to constructs, but
you conclude you can.
Threats to Validity
How could you be wrong in the inference?
External Validity
● You can generalize to other contexts,
but you conclude you can’t
● You can’t generalize to contexts, but
you conclude you can
Threats to Validity
● How you can be wrong in making your
inference
● Specific factors that can bias or distort
your conclusions
● A list of common threats to the quality of
your study
● A “checklist” you can use in planning
your study
