Validity and Reliability in

Education
Schools all over the country are beginning to develop a culture
of data, which is the integration of data into the day-to-day
operations of a school in order to achieve classroom, school, and
district-wide goals. One of the biggest difficulties that comes with
this integration is determining what data will provide an accurate
reflection of those goals.

Such considerations are particularly important when the goals of

the school aren’t put into terms that lend themselves to cut and
dry analysis; school goals often describe the improvement of
abstract concepts like “school climate.”  

Schools interested in establishing a culture of data are advised to

come up with a plan before going off to collect it. They need to
first determine what their ultimate goal is and what
achievement of that goal looks like. An understanding of the
definition of success allows the school to ask focused questions to
help measure that success, which may be answered with the
data.

For example, if a school is interested in increasing literacy, one

focused question might ask: which groups of students are
consistently scoring lower on standardized English tests? If a
school is interested in promoting a strong climate of
inclusiveness, a focused question may be: do teachers treat
different types of students unequally?

These focused questions are analogous to research questions

asked in academic fields such as psychology, economics, and,
unsurprisingly, education. However, the question itself does not
always indicate which instrument (e.g. a standardized test,
student survey, etc.) is optimal.
If the wrong instrument is used, the results can quickly become
meaningless or uninterpretable, thereby rendering them
inadequate in determining a school’s standing in or progress
toward their goals.

Differences Between
Validity and Reliability
When creating a question to quantify a goal, or when deciding on
a data instrument to secure the results to that question, two
concepts are universally agreed upon by researchers to be of
pique importance.

These two concepts are called validity and reliability, and they
refer to the quality and accuracy of data instruments.

WHAT IS VALIDITY?
The validity of an instrument is the idea that the instrument
measures what it intends to measure.

Validity pertains to the connection between the purpose of the

research and which data the researcher chooses to quantify that
purpose.

For example, imagine a researcher who decides to measure the

intelligence of a sample of students. Some measures, like physical
strength, possess no natural connection to intelligence. Thus, a
test of physical strength, like how many push-ups a student could
do, would be an invalid test of intelligence.

WHAT IS RELIABILITY?
Reliability, on the other hand, is not at all concerned with intent,
instead asking whether the test used to collect data produces
accurate results. In this context, accuracy is defined by
consistency (whether the results could be replicated).

The property of ignorance of intent allows an instrument to be

simultaneously reliable and invalid.

Returning to the example above, if we measure the number of

pushups the same students can do every day for a week (which, it
should be noted, is not long enough to significantly increase
strength) and each person does approximately the same amount
of pushups on each day, the test is reliable. But, clearly, the
reliability of these results still does not render the number of
pushups per student a valid measure of intelligence.

Because reliability does not concern the actual relevance of the

data in answering a focused question, validity will generally
take precedence over reliability. Moreover, schools will often
assess two levels of validity:

1. the validity of the research question itself in quantifying the

larger, generally more abstract goal
2. the validity of the instrument chosen to answer the research
question

Conclusion
3. Validity and reliability are meaningful measurements that
should be taken into account when attempting to evaluate
the status of or progress toward any objective a district,
school, or classroom has.
4. If precise statistical measurements of these properties are
not able to be made, educators should attempt to evaluate
the validity and reliability of data through intuition, previous
research, and collaboration as much as possible.
5. An understanding of validity and reliability allows educators
to make decisions that improve the lives of their students
both academically and socially, as these concepts teach
 educators how to quantify the abstract goals their school or
district has set.

Validity and reliability of assessment methods are considered the two most important
characteristics of a well-designed assessment procedure.
Validity refers to the degree to which a method assesses what it claims or intends to assess. The
different types of validity include:

Validity Definition
the assessment method matches the content
content
of the work

relates to whether the assessment method is

criterion explicit in terms of procedures correlating
with particular behaviours

relates to whether scores reflect the items

construct
being tested.5,13

Performance based assessments are typically viewed as providing more valid data than
traditional examinations because they focus more directly on the tasks or skills of practice.2
Reliability refers to the extent to which an assessment method or instrument measures
consistently the performance of the student. Assessments are usually expected to produce
comparable outcomes, with consistent standards over time and between different learners and
examiners.  However, the following factors impede both the validity and reliability of assessment
practices in workplace settings:

 inconsistent nature of people

 reliance on assessors to make judgements without bias
 changing contexts/conditions
 evidence of achievement arising spontaneously or incidentally.2,13

Explicit performance criteria enhance both the validity and reliability of the assessment process.
Clear, usable assessment criteria contribute to the openness and accountability of the whole
process.  The context, tasks and behaviours desired are specified so that assessment can be
repeated and used for different individuals.  Explicit criteria also counter criticisms of
subjectivity.13

As mentioned in Key Concepts, reliability and validity are closely related.  To better
understand this relationship, let's step out of the world of testing and onto a bathroom
scale.
 
  If the scale is reliable it tells you the same weight every time you  step
on it as long as your weight has not actually changed.   However, if the
scale is not working properly, this number may not  be your actual
weight.  If that is the case, this is an example of a  scale that is reliable,
or consistent, but not valid.  For the scale to  be valid and reliable, not
only does it need to tell you the same weight every time you step on the
scale, but it also has to measure your actual weight.
 
Switching back to testing, the situation is essentially the same.  A test can
be reliable, meaning that the test-takers will get the same score no matter
when or where they take it, within reason of course.  But that doesn't mean
that it is valid or measuring what it is supposed to measure.  A test can be
reliable without being valid. However, a test cannot be valid unless it is
reliable.