EDUR 7130

Educational Research On-Line

Practice Exercise for Validity

Instructions

Answer the questions listed below each example. Note that each example
represents one of the following types of validity: concurrent, construct, content,
and predictive.

1. The Goodly Test of Basic Third-Grade Math has been around a long time, but it
takes 60 minutes to administer. Being pressed for teaching time, you develop
another test that takes only 20 minutes to administer and call it the Shorter and
Better Test of Basic Third Grade Math. You send it off to a test publisher with
visions of fame and wealth.

What is the easiest and most logical method for establishing it’s validity? If this
example represents either concurrent or predictive validity, then what is the
criterion?

2. A high school teacher is interested in determining which students, as they enter

high school, are likely to do well in a calculus class. Upon entering high school
students are required to take a high school scholastic aptitude test (HSSAT). Just
prior to graduation, students are also required to take a comprehensive
departmental (mathematics) examination which covers a variety of mathematical
concepts. Using scores from the aptitude test and the comprehensive examination,
the teacher determines whether a relationship exists between the two tests. If one
does exist, the teacher plans to use HSSAT scores to help in selecting students for
calculus.

What type of validity does the process described above represent? If this example
represents either concurrent or predictive validity, then what is the criterion?

3. A school principal decides to begin implementing a new test of achievement in

ninth-grade mathematics. However, to insure that this test is an adequate
representation of the material taught in nine-grade mathematics courses, the
principals asks the math teachers to take a look at the test and provide some
feedback. They tell him that several concepts taught in their courses are ignored in
the test, and others are present which are not in their courses.

What type of validity is being evaluated? If this example represents either

concurrent or predictive validity, then what is the criterion?
4. A guidance counselor uses a vocational/occupational test to help students
determine the type of occupation for which they would be best suited.

What type of validity is most important with this test of test? If this example
represents either concurrent or predictive validity, then what is the criterion?

5. A researcher develops a new test of anxiety. The researcher finds that scores
from this anxiety test correlate as expected with scores from a number of other
variables (e.g., self-efficacy, test scores, etc.).

Based on the evidence she provides, what type of validity does she claim? If this
example represents either concurrent or predictive validity, then what is the
criterion?

6. A teacher develops a test of ignorance. The teacher wishes to use this test, but
feels that he needs to establish that it actually does measure ignorance. So he looks
through the research literature using ERIC and finds that some researcher at Insane
State has also developed a measure of ignorance that has gone through extensive
testing and has proved to be valid. So the teacher obtains a copy of the test, and
administers it to some students. He finds that the scores on his test are highly
correlated with the test scores from the Insane State test.

What type of validity is established above? If this example represents either

concurrent or predictive validity, then what is the criterion?

7. A researcher wishes to test the notion that creative children will be able to state
more meanings of a word than will non-creative children. To see if this is correct,
the researcher decides to do a study. After listening to children for a few weeks, the
researcher believes he found traits that clearly distinguish creative from non-
creative children. Based on these traits, the researcher assigns rankings to the
children in terms of their creativity. Next, the researcher develops a test of
creativity based upon the traits he observed. The test is administered to the children
and scores are obtained. The researcher finds that his test is highly correlated with
the rankings he assigned to the children. Next he asks the children to provide
meanings to several words and records the number of meanings provided. He finds
that creativity, based on his test, is related to the number of meanings provided.

What type of validity was established for the creativity test developed above? If
this example represents either concurrent or predictive validity, then what is the
criterion?

8. A teacher develops a test of history. When creating the test, the teacher was
careful to follow the objectives and material he taught in class. The test was
administered, and students did as expected.
What type of validity does the teacher’s actions imply? If this example represents
either concurrent or predictive validity, then what is the criterion?

Answers

1. The Goodly Test of Basic Third-Grade Math has been around a long time, but it
takes 60 minutes to administer. Being pressed for teaching time, you develop
another test that takes only 20 minutes to administer and call it the Shorter and
Better Test of Basic Third Grade Math. You send it off to a test publisher with
visions of fame and wealth.

What is the easiest and most logical method for establishing it’s validity? If this
example represents either concurrent or predictive validity, then what is the
criterion?

Answer: Concurrent validity would work here--a newly developed instrument and
an previously established instrument. The criterion is the previously established
instrument, Goodly Test.

2. A high school teacher is interested in determining which students, as they enter

high school, are likely to do well in a calculus class. Upon entering high school
students are required to take a high school scholastic aptitude test (HSSAT). Just
prior to graduation, students are also required to take a comprehensive
departmental (mathematics) examination which covers a variety of mathematical
concepts. Using scores from the aptitude test and the comprehensive examination,
the teacher determines whether a relationship exists between the two tests. If one
does exist, the teacher plans to use HSSAT scores to help in selecting students for
calculus.

What type of validity does the process described above represent? If this example
represents either concurrent or predictive validity, then what is the criterion?

Answer: Predictive validity is appropriate for this situation. The question is

whether the HSSAT can be used to predict performance on the comprehensive
examination, and if it can accurately predict performance on the comprehensive
examination, then the HSSAT will be used for selecting students. The criterion in
this example is the comprehensive examination.

3. A school principal decides to begin implementing a new test of achievement in

ninth-grade mathematics. However, to insure that this test is an adequate
representation of the material taught in nine-grade mathematics courses, the
principals asks the math teachers to take a look at the test and provide some
feedback. They tell him that several concepts taught in their courses are ignored in
the test, and others are present which are not in their courses.

What type of validity is being evaluated? If this example represents either

concurrent or predictive validity, then what is the criterion?

Answer: Content validity was used here. No empirical data were collected to
correlate with a criterion or other variable; only logical assessment was used.
Almost anytime reference is made to an achievement test, content validity will be a
likely candidate for the method used to establish validity.

4. A guidance counselor uses a vocational/occupational test to help students

determine the type of occupation for which they would be best suited.

What type of validity is most important with this test of test? If this example
represents either concurrent or predictive validity, then what is the criterion?

Answer: Predictive validity is appropriate here. The issue with this study is
whether the vocational/occupation test does a good job of predicting appropriate
job categories for individuals. For this example, the criterion would be some
measure of job success in the future.

5. A researcher develops a new measure of verbal acuity. The researcher finds that
scores from this verbal acuity instrument correlate as expected with scores from a
number of other variables thought to be related to verbal acuity (e.g., intelligence,
creativity, self-confidence).

Based on the evidence she provides, what type of validity does she claim? If this
example represents either concurrent or predictive validity, then what is the
criterion?

Answer: Construct validity appears to be the appropriate choice. Scores from an

instrument are correlated with scores from variables expected to be related to
acuity.

6. A teacher develops a test of ignorance. The teacher wishes to use this test, but
feels that he needs to establish that it actually does measure ignorance. So he looks
through the research literature using ERIC and finds that some researcher at Insane
State has also developed a measure of ignorance that has gone through extensive
testing and has proved to be valid. So the teacher obtains a copy of the test, and
administers it to some students. He finds that the scores on his test are highly
correlated with the test scores from the Insane State test.

What type of validity is established above? If this example represents either

concurrent or predictive validity, then what is the criterion?
Answer: Concurrent validity because scores from the test of ignorance are
correlated with scores for an instrument that measures a similar construct (Insane
State test). So, there are two instruments designed to measure the same thing. The
criterion in this example is the Insane State test.

7. A researcher wishes to test the notion that creative children will be able to state
more meanings of a word than will non-creative children. To see if this is correct,
the researcher decides to do a study. After listening to children for a few weeks, the
researcher believes he found traits that clearly distinguish creative from non-
creative children. Based on these traits, the researcher assigns rankings to the
children in terms of their creativity. Next, the researcher develops a test of
creativity based upon the traits he observed. The test is administered to the children
and scores are obtained. The researcher finds that his test is highly correlated with
the rankings he assigned to the children. Next he asks the children to provide
meanings to several words and records the number of meanings provided. He finds
that creativity, based on his test, is related to the number of meanings provided.

What type of validity was established for the creativity test developed above? If
this example represents either concurrent or predictive validity, then what is the
criterion?

Answer: Construct validity applies here. The researcher develops a measure of

creativity. Scores from this measure are correlated with two behaviors that should
be highly related--creativity rankings and the number of meanings provided.

8. A teacher develops a test of history. When creating the test, the teacher was
careful to follow the objectives and material he taught in class. The test was
administered, and students did as expected.

What type of validity does the teacher’s actions imply? If this example represents
either concurrent or predictive validity, then what is the criterion?

Answer: Content validity. The teacher shows that test items match objectives and material
covered.