Chapter 7: Instrumentation

Activity 7.1: Major Categories of Instruments and Their Uses

Activity 7.2: Which Types of Instrument Is Most Appropriate?

Activity 7.3: Types of Scales

Activity 7.4: Norm-Referenced vs. Criterion-Referenced Instruments

Activity 7.5: Developing a Rating Scale

Activity 7.6: Design an Instrument

Activity 7.1:
Major Categories of Instruments and Their Uses

Match the letter of the instrument from Column B with its most likely use listed in
Column A.

Column A: Purpose Column B: Instrument

1. __g__ A researcher wishes to observe a. Questionnaire

and record the behavior of an individual
over time. b. Interview schedule

2. __a__ A researcher wishes to survey a c. Performance checklist

large group of individuals.
d. Achievement test
3. __d__ A researcher wants to find out
how much someone knows about the e. Attitude scale
causes of the French Revolution.
f. Rating scale
4. _f___A researcher wants to evaluate the
quality of a new microwave oven. g. Anecdotal record

5. _b___A researcher wishes to get in-depth

information from a small group of
people.

6. __e__ A researcher wants to gain some

idea of how students in a graduate
program in teacher education feel about
their student teaching experience.
Activity 7.2:
Which Type of Instrument is Most Appropriate?

For each of the items listed below, indicate whether it would be most likely to be
measured by an aptitude test (AT), a questionnaire (Q), an interview (I), a rating scale
(RS), a tally sheet (TS), or a performance checklist (PC).

1. _Q__ a person’s self-concept

2. __AT__ readiness for kindergarten

3. __I__ a person’s experiences in high school

4. __PC__ assessing paramedic skills

5. __RS_ quality of a college application

6. __RS__ ability to work with others on a research project

7. _I__ educational experiences of exceptional teachers

8. __AT__ potential of high school seniors for college work

9. __TS__ type of questions asked by students in a chemistry class

10. __TS_ prevalence of different kinds of errors in baseball

11. _RS__ student evaluations of instructor competence

12. __I__ how a particular student feels about poetry

13. __Q__ public reactions to a recently announced plan to raise property taxes

14. __TS__ ability to use a calculator

15. _TS__ who participates -- and how much -- in the discussions that occur in an
advanced-placement twelfth-grade American government class
Activity 7.3:
Types of Scales

Match the letter of the type of measurement scale from Column B with the example listed
in Column A to which the scale applies.

Column A: Practice Column B: Measurement Scale

1. __d_ Type of scale that possesses a true a. Nominal scale

zero point
b. Ordinal scale
2. _d_ Type of scale that possesses all of the
characteristics of the other scales c. Interval scale

3. __b__ Type of scale that indicates only d. Ratio scale

relative standing among individuals

4. __a__ Type of scale in which a researcher

simply assigns numbers to different
categories in order to show differences

5. __d_ Type of scale that is rarely

encountered in educational research

6. __a__ Type of scale that cannot be used to

measure quantitative variables

7. __c__ Type of scale in which all of the

distances between the points on the scale
are equal, but does not have a true zero
point

8. __n__ The simplest type of scale that

provides the least information

9. __c__ Type of scale that assumes that

equal differences between scores really
mean equal differences in the variable
being measured
Activity 7.4:
Norm-Referenced vs. Criterion-Referenced Instruments

For each of the items listed below, indicate whether Norm-referenced (N) or Criterion-
referenced (C) instrument is described.

1. _c__ Provides a clear-cut goal to work toward

2. __c____ Indicates that an individual was able to run a mile in at least 12 minutes

3. ___n___ Compares an individual’s scores with the scores of a group

4. ___c___ Focuses more directly on instruction than the other type

5. ___n___ Indicates how an individual did compared to other members in his or her
class

6. ___c___ Is almost always easier to use than the other type

7. ____n__ Desired difficulty level is at or about 50 percent

8. __n____ Generally is inferior than the other type for research purposes

9. __n____ Generally will provide more variability in scores

Activity 7.5:
Developing a Rating Scale

In one of our research classes, a student designed a study to investigate the following
hypothesis:
“The more open the classroom, the higher the amount of student motivation.”
As part of the process, she developed a rating scale to assess both the degree of
openness and the level of student motivation within a particular classroom. Only by
having some kind of measurement of each of these variables could she determine if a
relationship existed between them.
Her first step was to produce a number of items related to the idea of “openness.”
She began by listing various things that could be taken as indicators of openness. Certain
groupings began to emerge from the list of indicators. For example, a number of
indicators seemed to concern the physical arrangement of the classroom, so that
constituted one grouping. Listed below are some of the indicators and categories she
formulated:

Physical environment: Are the desks placed in rows? Are there specific learning centers
for subjects? Are classes sometimes held outdoors? Is there a general meeting area for
students in the classroom? Are any other types of furniture used besides desks (e.g.,
sofas, rocking chairs, etc.) used? How many adults (teaches, paraprofessionals, etc.) are
there in the classroom?

Curriculum: What amount of time does the teacher spend on planning? On evaluation?
Does the teacher have a list of overall objectives she tries to attain? How much time is
spent on the academic curriculum? On arts and crafts? On discussions or problem
solving? Do the students direct their own planning? Devise their own curriculum? Are
affective objectives included in the curriculum? Are students taught to express their
feelings? Are grades given?

Teacher-student relationships: How often does the teacher give directions? Help
students? How often do students initiate activities? Can students leave the classroom on
their own, or must they request permission? Does the teacher work with students
individually? Work in small groups? Teach the entire class? Do students and teachers
jointly evaluate student work? Are class meetings student or teacher-directed? Can
students set their own free time?

Materials: Are students assigned specific materials to use? What kinds of materials are
available to students? How much time is spent on workbook assignments? Are there
manipulative materials available? Are the materials easily accessible, or must students
request them? Is the use of materials directed by teacher or students? Are art materials
available? What sorts of books do students use? What other materials exist?

Social environment: Are students encouraged to help one another? To tutor others? Are
students free to talk with others in class? How often? Do students work alone, or may
they work with others? Do students group themselves, or is this done by the teacher? Do
students share in room cleanup? How many times must the teacher ask for quiet? Does
physical aggression ever occur between students? If so, how often? Who handles
aggression, teacher or students?

Parental participation: Are parents allowed in the classroom? Observers? Others? Are
parents and others free to enter the classroom when they wish, or is there a formal
procedure they must go through? Is there a volunteer parent-aide program? What kinds of
tasks do parents perform? How often are parents present?

Once she felt that she had a sufficient number of items within each of these
categories, she worked to refine and clarify the indicators in each of the categories and
then to convert them into items for a rating scale. Shown below is her completed rating
scale:

RATING SCALE FOR CLASSROOM OPENNESS

1. Students do not move without 12345 1. Students may move in or out of

teacher permission class without permission

2. All students work at the 12345 2. A great variety of tasks are

same task at the same time performed at the same time

3. The teacher is the only 12345 3. Several human resources other

resource in the classroom than the teacher are in the
Classroom

4. Human resources are only 12345 4. Human resources interact with

clerical or housekeeper aides students or with small groups
to the teacher

5. Furniture is permanently 12345 5. Furniture is spontaneously

arranged arranged

6. Everyone works at own desk 12345 6. There are many floating study
centers

7. Desks, tables, and chairs are 12345 7. There is a complete variety of

arranged traditionally furniture in a variety of
arrangements

8. Students cannot interact 12345 8. Students are free to interact

without direct permission with others in any way they
of the teacher desire

9. The teacher initiates all 12345 9. Students also initiate activities

the activities
10. The teacher teaches the class 12345 10. The teacher works with small
as a group groups or individual students

11. The teacher is addressed 12345 11. The teacher is addressed

formally (e.g., Mrs. X; hands informally (first name, etc.)
are raised; etc.)

12. Reprimands are punitive 12345 12. No reprimands or only friendly

reminders are given

13. No feelings are verbally 12345 13. Feelings are expressed verbally

14. The textbook is closely 12345 14. No formally prepared materials

followed are used in class

What differences do you notice between the original list of indicators and the final rating
scale above?
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
Activity 7.6:
Design an Instrument

In this activity, you are to design an instrument (see Chapter 7 in the text) on a topic of
interest.

1. Prepare either a rating scale or a questionnaire and describe it on the back of this
page.

2. In the spaces below, describe how you checked for:

instrument validity _______________________________________________________

_______________________________________________________________________

_______________________________________________________________________

instrument reliability ______________________________________________________

_______________________________________________________________________

_______________________________________________________________________

3. Administer the instrument to a group (at least five) of your friends. Summarize the
results here. __________________________________________________________

________________________________________________________________________

________________________________________________________________________

________________________________________________________________________

________________________________________________________________________

4. What problems did you encounter? _______________________________________

_______________________________________________________________________

_______________________________________________________________________

5. What could you do better next time to avoid such problems? ___________________

_______________________________________________________________________

________________________________________________________________________
Problem Sheet 7: Instrumentation

1. Describe the types of instrument(s) you plan to use for your study (e.g., interview
protocol, attitudinal survey, achievement test, observation scale, questionnaire,
focus group protocol, etc.). ____________________________________________
___________________________________________________________________

2. Is it a preexisting instrument or one you plan to develop?

___________________________________________________________________
___________________________________________________________________

3. If preexisting, state the name of the instrument. Also, why did you decide to use this
particular instrument? _________________________________________________
___________________________________________________________________

4. What is the instrument supposed to measure or assess?

___________________________________________________________________
___________________________________________________________________

5. How many items will the instrument contain?

___________________________________________________________________
___________________________________________________________________

6. How will the instrument be scored or interpreted?

___________________________________________________________________
___________________________________________________________________