Reliability vs. Validity: What’s The Difference?

Reliability and validity are concepts used to evaluate the quality of research. They indicate how
well a method, technique or test measures something. Reliability is about the consistency of a
measure, and validity is about the accuracy of a measure.

It’s important to consider reliability and validity when you are creating your research design,
planning your methods, and writing up your results, especially in quantitative research.

Reliability vs. Validity

Reliability Validity

What does it tell The extent to which the results can be The extent to which the results really
you? reproduced when the research is repeated measure what they are supposed to
under the same conditions. measure.

How is it assessed? By checking the consistency of results By checking how well the results
across time, across different observers, correspond to established theories and
and across parts of the test itself. other measures of the same concept.

How do they relate? A reliable measurement is not always A valid measurement is generally reliable:
valid: the results might be reproducible, if a test produces accurate results, they
but they’re not necessarily correct. should be reproducible.

Understanding Reliability vs. Validity

Reliability and validity are closely related, but they mean different things. A measurement can be
reliable without being valid. However, if a measurement is valid, it is usually also reliable.

What is Reliability?
Reliability refers to how consistently a method measures something. If the same result can be
consistently achieved by using the same methods under the same circumstances, the
measurement is considered reliable.
Example:

A community worker uses a survey questionnaire to identify a community’s condition. Several

community workers use the same questionnaire with the same community but give different
analysis. This indicates that the questionnaire has low reliability as a measure of the condition.

What is Validity?
Validity refers to how accurately a method measures what it is intended to measure. If research
has high validity that means it produces results that correspond to real properties,
characteristics, and variations in the physical or social world.

High reliability is one indicator that a measurement is valid. If a method is not reliable, it probably
isn’t valid.

Example:

If an analysis questionnaire results in a reliable finding when answered at different times and
with different community workers, this indicates that it has high validity as a measurement of
the community condition.
However, reliability on its own is not enough to ensure validity. Even if a test is reliable, it may
not accurately reflect the real situation.

A group of participants take a test designed to measure working memory. The results are reliable,
but participants’ scores correlate strongly with their level of reading comprehension. This
indicates that the method might have low validity: the test may be measuring participants’
reading comprehension instead of their working memory.
Validity is harder to assess than reliability, but it is even more important. To obtain useful
results, the methods you use to collect your data must be valid: the research must be
measuring what it claims to measure. This ensures that your discussion of the data and
the conclusions you draw are also valid.

How are Reliability and Validity Assessed?

Reliability can be estimated by comparing different versions of the same measurement. Validity
is harder to assess, but it can be estimated by comparing the results to other relevant data or
theory. Methods of estimating reliability and validity are usually split up into different types.
Types of Reliability
Different types of reliability can be estimated through various statistical methods.

Types of Reliability

Type of Reliability What Does it Assess? Example

Test-retest The consistency of a measure across A group of participants complete a

time: do you get the same results when questionnaire designed to measure
you repeat the measurement? personality traits. If they repeat the
questionnaire days, weeks or months
apart and give the same answers, this
indicates high test-retest reliability.

Interrater The consistency of a measure across Based on an assessment criteria

raters or observers: do you get the same checklist, five examiners submit
results when different people conduct substantially different results for the
the same measurement? same student project. This indicates that
the assessment checklist has low inter-
rater reliability (for example, because the
criteria are too subjective).

Internal The consistency of the measurement You design a questionnaire to measure

consistency itself: do you get the same results from self-esteem. If you randomly split the
different parts of a test that are designed results into two halves, there should be
to measure the same thing? a strong correlation between the two
sets of results. If the two results are very
different, this indicates low internal
consistency.
Types of Validity
The validity of a measurement can be estimated based on three main types of evidence. Each
type can be evaluated through expert judgement or statistical methods.

Types of Validity

Type of Validity What Does it Assess? Example

Construct The adherence of a measure to existing A self-esteem questionnaire could be

theory and knowledge of the concept assessed by measuring other traits known
being measured. or assumed to be related to the concept
of self-esteem (such as social skills and
optimism). Strong correlation between
the scores for self-esteem and associated
traits would indicate high construct
validity.

Content The extent to which the A test that aims to measure a class of
measurement covers all aspects of the students’ level of Spanish contains
concept being measured. reading, writing and speaking
components, but no listening
component. Experts agree that listening
comprehension is an essential aspect of
language ability, so the test lacks content
validity for measuring the overall level of
ability in Spanish.

Criterion The extent to which the result of a A survey is conducted to measure the
measure corresponds to other valid political opinions of voters in a region. If
measures of the same concept. the results accurately predict the later
outcome of an election in that region, this
indicates that the survey has high
criterion validity.
How to Ensure Validity and Reliability in your Research
The reliability and validity of your results depends on creating a strong research design, choosing
appropriate methods and samples, and conducting the research carefully and consistently.

Ensuring Validity
If you use scores or ratings to measure variations in something (such as psychological traits, levels
of ability or physical properties), it’s important that your results reflect the real variations as
accurately as possible. Validity should be considered in the very earliest stages of your research,
when you decide how you will collect your data.

 Choose appropriate methods of measurement

Ensure that your method and measurement technique are high quality and targeted to measure
exactly what you want to know. They should be thoroughly researched and based on existing
knowledge.

For example, to collect data on a personality trait, you could use a standardized questionnaire
that is considered reliable and valid. If you develop your own questionnaire, it should be based
on established theory or findings of previous studies, and the questions should be carefully and
precisely worded.

 Use appropriate sampling methods to select your subjects

To produce valid generalizable results, clearly define the population you are researching (e.g.
people from a specific age range, geographical location, or profession). Ensure that you have
enough participants and that they are representative of the population.

Ensuring Reliability
Reliability should be considered throughout the data collection process. When you use a tool or
technique to collect data, it’s important that the results are precise, stable and reproducible.

 Apply your methods consistently

Plan your method carefully to make sure you carry out the same steps in the same way for each
measurement. This is especially important if multiple researchers are involved.

For example, if you are conducting interviews or observations, clearly define how specific
behaviours or responses will be counted, and make sure questions are phrased the same way
each time.
  Standardize the conditions of your research

When you collect your data, keep the circumstances as consistent as possible to reduce the
influence of external factors that might create variation in the results.

For example, in an experimental setup, make sure all participants are given the same information
and tested under the same conditions.

Where to Write About Reliability and Validity in a Thesis

It’s appropriate to discuss reliability and validity in various sections of your thesis or dissertation.
Showing that you have taken them into account in planning your research and interpreting the
results makes your work more credible and trustworthy.

Reliability and Validity in a Thesis

Section Discuss

Literature review What have other researchers done to devise and improve methods that are reliable
and valid?

Methodology How did you plan your research to ensure reliability and validity of the measures
used? This includes the chosen sample set and size, sample preparation, external
conditions and measuring techniques.

Results If you calculate reliability and validity, state these values alongside your main results.

Discussion This is the moment to talk about how reliable and valid your results actually were.
Were they consistent, and did they reflect true values? If not, why not?

Conclusion If reliability and validity were a big problem for your findings, it might be helpful to
mention this here.