Selecting and Constructing Test Items and Tasks

TEST TYPES

1. Supply Type-Completion, Short Answer, Essay-Restricted, Essay-Expanded

2. Selection Type- Binary Choice, Multiple-Choice, Matching

3. Performance tasks- Paper-pencil, Identification, Work sample, Simulated Performance

RELATING TEST TYPES WITH LEVELS OF LEARNING OUTCOMES

A review of curricular frameworks of educational systems across various countries shows

common integral domains that govern their content and performance standards in different subject
areas. Basic in all are Knowledge, Skills and Processes, Understanding, Product, Affect (i.e. attitudes and
values). The assessment of these domains is realized by the appropriate selection and alignment of test
types and assessment tasks to their respective foci of learning.

A. Measuring Knowledge and Simple Understanding

Knowledge, as it appears in cognitive taxonomies (Bloom, 1956, Anderson & Krathwol, 2004) as
the simplest and lowest level, is categorized further into what thinking process is involved in learning.
Knowledge involves remembering or recalling specific facts, symbols, details, elements of events and
principles to acquire new knowledge. The revision of Bloom's taxonomy (Anderson & Krathwol, 2004)
recognizes how remembering can be viewed not only as being able to recall but also as being necessary
in learning interrelationships among basic elements and in learning methods, strategies and procedures.
These two types of learning make up the early phase of understanding requiring lower-order thinking
such as remembering, comprehending and applying. McMillan (2007) refers to the latter two as simple
understanding requiring comprehension of "concepts, ideas and generalizations" known as declarative
knowledge and application of skills and procedures learned in new situations, referred to as procedural
knowledge, They represent how knowledge is represented in the minds of learners either as a learned
concept or as a learned way of doing things.

B. Measuring Deep Understanding

Beyond knowledge and simple understanding level comes deep understanding which require
more complex thinking processes. McMillan (2007) utilizes a Knowledge/Understanding continuum to
illustrate the relative degree of understanding from knowledge to simple understanding to deep
understanding. Knowledge and simple understanding involve the first three cognitive levels, i.e.
remembering, comprehending and applying, while deep understanding requires the three higher
cognitive levels, i.e. analyzing, evaluating and creating.

CONSTRUCTING OBJECTIVE SUPPLY TYPE OF ITEMS

 The item types falling under Supply type require the learners to construct a response to a
question or directive. The sub-types however, differ in terms of the structure of the response needed to
answer the item:

1. Completion Type-An item structure consists of a stimulus which defines the question or problem, and
a response which defines what is to be provided or constructed by the learner. For a completion item,
an incomplete statement with a blank is often used as stimulus and the response is a constructed word,
symbol, numeral of phrase to complete the statement.

Gap -filling--another term for this variant as the student fills several gaps in a discourse depending on
the target outcome. Language teachers often utilize this form for integrative testing where more than
one type of skill (ex. vocabulary and comprehension skills) are needed to fill in the gaps.

Guidelines in the construction of Completion Items:

1. There should only be one correct response to complete a statement.

2. The blank should be placed at the end or towards the end of the incomplete statement.

3. Avoid providing unintended clues to the correct answer.

2. Short Answer Items- Instead of supplying words to complete statements, relatively short answers are
constructed as direct answers to questions.

Guidelines in writing a short -item test:

1. State the item so that only one answer is correct.

2. State the item so that the required answer is brief. Requiring a long response would not be necessary
and it can limit the number of items students can answer within the allotted period of time.

3. Do not use questions verbatim from textbooks and other instructional materials. This will give undue
disadvantage to students not familiar with the materials since it can become a memory test instead of
comprehension.

4. Designate units required for the answer. This frequently occurs when the constructed response
requires a definite unit to be considered correct. Without designating the unit, a response may be
rendered wrong because of differing mind-set

5. State the item succinctly with words students understand. This is true for all types of tests. Validity of
classroom-based test is at risk when students cannot answer correctly, not because they do not know,
but could be due to the messy wording of the question.

*****The two supply types, completion and short answer items, share common points:
  Appropriate for assessing learning outcomes involving understanding knowledge and simple
understanding.

 Capable of assessing both declarative and procedural knowledge.

 Both are easy and simple to construct.

 Both are objectively scored since a key to correction can be prepared in advance. Both need
ample number of items to assess a learning outcome. A single completion or short-answer item
is not sufficient to test mastery of a competency.

CONSTRUCTING NON-OBJECTIVE SUPPLY TYPE

Essay Type

Essay type likewise belongs to the Supply category for the simple reason that the required
response is to be fully constructed by the students. This format therefore is appropriate for testing deep
understanding and reasoning. Some of the thinking processes to satisfactorily answer essay questions
involve comparison Induction, deduction, abstracting, analyzing perspectives, decision-making, problem
solving constructing support and experimental inquiry (Marzano, et al (1993). They actually involve
higher order thinking skills.

Types of Essay Items

A. Restricted-response Type

The question or directive given suggests a specification in constructing the response. It can be in
terms of limited coverage of content, specified length of response, expected form of response, or
definite perspective or mind-set to be used.

B. Extended-response Type

The question or directive does not suggest any form of restriction in the construction of the
response. The students are free to organize and expound on their ideas freely.

Suggestions for constructing essay questions:

1. Restrict the use of essay questions to those learning outcomes that cannot be measured satisfactorily
by objective items.

2. Construct questions that will call forth the skills specified in the learning standards.

3. Phrase the question so that the student's task is clearly defined.

4. Indicate an approximate time limit for each question.

5. Avoid the use of optional questions.

Suggestions to improve the reliability of scoring responses to essay questions:

1. Prepare an outline of the expected answer in advance.

2. Use the scoring rubric that is most appropriate.

3. Decide how to handle factors that are irrelevant to the learning outcomes being measured

4. Evaluate all responses to one question before going on to the next one.

5. When possible, evaluate the answers without looking at the student's name.

6. If especially important decisions are to be based on the results.

CONSTRUCTING SELECTED-RESPONSE TYPES

The greatest challenge for this item format is the construction of plausible options or distracters
se not one stands out as attractively correct. There are three sub-types of the selected-response format
depending on the number of given options:

a. alternate form or binary choice provides only two options.

b. multiple-choice type offers 3 to 5 options or solutions to a problem, and

c. matching type gives a set of problems or premises and a set of options which will be appropriately
paired.

a. Binary Choice or alternate form

Varieties of Binary Choice:

-True-False, Yes-No, Right-Wrong

Suggestions given to construct good binary choice items:

1. Write the item so that the answer options are consistent with the logic in the sentence (Align your
options with the logic of your proposition, e.g. if after truth or falsehood, better not use yes-no or
agree/disagree options)

2 Focus on a single fact or idea in the item (Adding more than one idea in the statement can make the
item ambiguous. One idea may be correct and the other one is incorrect)

3. Avoid unnecessarily long and wordy statements that obscure the significant idea.

4. Avoid insignificant or trivial facts or words. (Students commit errors not because they do not know
but due to unnecessary facts).

5. Avoid negative statements. (Statements with not or no are confusing to young readers.)
6 Avoid inadvertent clues to the answer. (Items using such words as never, always, all the time, all, etc.
are most of the time false and are recognized by test-wise students.)

7. Avoid using vague adjectives and adverbs, Students interpret differently such adjectives and adverbs
as typically, usually occasionally, quite, etc. It often becomes a best of vocabulary when done)

b. Multiple Choice Items

The wide choice for this format in classroom testing is mainly due to its versatility to assess
various levels of understanding tum knowledge and simple understanding to deep understanding,
McMillan (2007) asserts that multiple choice can assess whether students can use reasoning as a skill
similar to binary-choice terms and to use students' knowledge and skills in performing problem solving,
decision-making are other reasoning task.

Writing good multiple-choice items requires clarity in stating the problem in the stem and the
plausibility or attractiveness of the distracters. Test experts agree on a set of guidelines to achieve this
purpose:

Stem-an interrogative statement or direct question that ends in a question mark, incomplete statement

1. All the words of the stem should be relevant to the task. It means stating the problem succinct and
clear so students understand what is expected to be answered.

2. Stem should be meaningful by itself and should fully contain the problem. This should especially be
observed when the stem uses an incomplete statement format. Consider this stem:

The constitution is ____________________________________.

A stem worded this way does not make definite the conceptual knowledge being assessed. One
does not know what is being tested. Is it after the definition of the term its significance or its history? To
test whether a stem is effectively worded is to be able to answer it without the distracters. This stem
can be improved by changing its format to a direct question or adding more information in the
incomplete statement like:

What does the constitution of an organization provide? (Direct-question format)

The constitution of an organization provides__________________ (Incomplete statement format)

This way, the test writer determines what knowledge competence to focus on and what appropriate
distracters to use.

3. The stem should use a question with only one correct or clearly best answer. Ambiguity sets in when
the stem allows for more than one best answer. Students will likely base their answer on personal
experience instead of on facts.

Distracters
1. All distracters should appear plausible to uninformed test takers. This is the key to making the item
discriminating and therefore valid.

Guidelines in making distracters should appear plausible:

1. Use the students' most common errors.

2. Use important-sounding-words (e.g. significant, accurate) that are relevant to the item stem. But do
not overdo it)

3. Use words that have verbal associations with the item stem (e.g. politician, political)

4. Use textbook language or other phraseology that has the appearance of truth.

5. Use incorrect answers that are likely to result from student misunderstanding or carelessness (e.g.
forgets to convert feet to yards)

2. Randomly assign correct answers to alternative positions. Item writers have a tendency to assign the
correct answer to the third a as they run short of incorrect alternatives. Students then who have been
used to taking multiple-choice tests choose wily option C when guessing for greater chance of being
correct. No deliberate order should be followed in assigning the correct answers (e.g. ABCDARC or
AACCBBDD) for ease in scoring. As much as possible have as equal number of correct answers
distributed randomly in each of the distracters.

3 Avoid using "All-of-the-above" or "None-of-the-above" as distracters. Item writers think that using
them adds difficulty to item since it is a way to test reasoning ability. However students without much
thinking, will tend to choose these "of-the-above distracters haphazardly when they see at least two
distracters as correct or incorrect without considering the remaining ones. When forced to come up
with a fourth plausible option and there seem to be none available except All-of-the-above" or "None-
of-the-above do not make them the correct answer.

c. Matching Items

Of the three general selected-response item formats, matching items appear differently consists
of two parallel lists of words or phrases the students are tasked to pair. The first list which is to be
matched is referred to as premises while the other list from which to choose its match based on a kind
of association is the options.

Guidelines in constructing matching items:

1. Keep the list of premises and the list of options homogenous or belonging to a category.

2. Keep the premises always in the first column and the options in the second column.

3. Keep the lists in the two columns unequal in number.

4. Test directions always describe the basis for matching.

5. Keep the number of premises not more than eight (8).

6. Ambiguous lists should be avoided.