Chapter 6 Item Analysis and Validation

Lesson 6.1 Item Analysis: Difficulty Index and Discrimination Index

________________________________________________________________________

Item analysis is a statistical technique which is used for selecting and rejecting the items
of the test on the basis of their difficulty value and discriminated power
The item analysis will provide information that will allow the teacher to decide whether to
revise, retain or replace an item (item revision phase).

There are two important characteristics:

These are (a) Item difficulty and (b) discrimination index.
The item difficulty is defined as the number of students who are able to answer the item
correctly divided by the total number of students. Thus:

Item difficulty – number of students with correct answer

total number of students

Note: The item difficulty is usually expressed in percentage.

Example 1: What is the item difficulty index of an item if 25 students are unable to answer
it correctly while, 75 answered it correctly?
Here, the total number of students is 100, hence the item difficulty index is 75/100 or 75%.
Example 2: 25 students answered the item correctly while 75 students did not. The total
number of students is 100 so the difficulty index is 25/100 or 25 which is 25%.

A high percentage indicates an easy item/question while a low percentage

indicates a difficult item.
How do we decide on the basis of this index whether the item is too difficult or too easy?
The following arbitrary rule is often used in the literature:

Range of Difficulty Index Interpretation Action

0 – 0.25 Difficult Revise or discard
0.26 – 0.75 Moderate (right difficulty) Retain
0.76 – above Easy Revise or discard

Difficult items tend to discriminate between those who know and those who do not know
the answer. Conversely, easy items cannot discriminate between these two groups of
students. We are therefore interested in deriving a measure that will tell us whether an
item can discriminate between these two groups of students. Such a measure is called
an index of discrimination.
An easy way to derive such a measure is to measure how difficult an item is with respect
to those in the upper 25% of the class and how difficult it is with respect to those in the
lower 25% of the class.

If the upper 25% of the class found the item easy yet the lower 25% found it difficult, then
the item can discriminate properly between these two groups. Thus:

Index of discrimination = DU – DL (U – Upper group; L – Lower group)

Example 1: Obtain the index of discrimination of an item if the upper 25% of the class
had a difficulty index of 0.60 (i.e. 60% of the upper 25% got the correct answer) while the
lower 25% of the class had a difficulty index of 0.20. Here, DU = 0.60 while DL = 0.20,
thus index of discrimination = 0.60 – 0.20 = 0.40.

Discrimination index is the difference between the proportion of the top scorers
who got an item correct and the proportion of the lowest scorers who got the item
right.

The discrimination index range is between -1 and +1. The closer the discrimination index
is to +1, the more effectively the item can discriminate or distinguish between the two
groups of students.

Types of Discrimination Index:

1. Positive discrimination index means that an item is correctly answered by upper group
and is not answered correctly by the lower group.

2. A negative discrimination index means more from the lower group got the item correctly.
The last item is not good and so must be discarded.

3. Zero discrimination or no discrimination means if the proportion of students who got the
item right in the upper performing group and low performing group are equal.

Theoretically, the index of discrimination can range from -1.0 (when DU =0 and
DL = 1) to 1.0 (when DU = 1 and DL = 0).

When the index of discrimination is equal to -1, then this means that all of the
lower 25% of the students got the correct answer while all of the upper 25% got
the wrong answer. In a sense, such an index discriminates correctly between the
two groups but the item itself is highly questionable. Why should the bright ones
get the wrong answer and the poor ones get the right answer?
On the other hand, if the index of discrimination is 1.0, then this means that all of
the lower 25% failed to get the correct answer while all of the upper 25% got the
correct answer. This is a perfectly discriminating item and is the ideal item that
should be included in the test.
From these discussions, let us agree to discard or revise all items that have
negative discrimination index for although they discriminate correctly between
the upper and lower 25% of the class, the content of the item itself may be highly
dubious or doubtful. As in the case of the index of difficulty, we have the following
rule of thumb:
Index Range Interpretation Action
-1.0 - -0.50 Can discriminate but item is questionable Discard
-0.55 - 0.45 Non-discriminating Revise
0.46 - 1.0 Discriminating item Retain

Example 2: Total Number of students = 20

20 x 25% = 5
UG 25% = 5 Right Upper = 5 (5/5 = 1.0)
LG 25% = 5 Right Lower = 1 (1/5 = 0.2)
Index of discrimination = DU – DL
1.0 – 0.20 = 0.80 (Retain the Item)

Example 3: Total Number of students = 30

30 x 25% = 7.5 or 8
UG 25% = 8 Right Upper = 5 (5/8 = 0.625)
LG 25% = 8 Right Lower = 2 (2/8 = 0.25)
Index of discrimination = DU – DL
0.625– 0.25 = 0.375 (Revise the Item)

Example 4: Total Number of students = 40

40 x 25% = 10
UG 25% = 10 Right Upper = 4 (4/10 = 0.40)
LG 25% = 10 Right Lower = 10 (10/10 = 1.0)
Index of discrimination = DU – DL
0.40– 1.0 = - 0.60 (Revise the Item)
Example 5: Consider a multiple choice type of test of which the following data were
obtained:
Item A B C D
0 40 20 20 Total
1
0 15 5 0 Upper 25%a

do not 0 5 10 5 Lower 25%the answer. Conversely, easy

items cannot discriminate be

The correct response is B. Let us compute the difficulty index and index of
discrimination:
Difficulty Index = no. of students getting correct response/total = 40/100 = 40%,
within range of a "good item"

Discrimination index:
80 x 25% = 20
UG 25% = 20 Right Upper = 15 (15/20 = 0.75)
LG 25% = 20 Right Lower = 5 (5/20 = 0.25)

Index of discrimination = DU – DL
0.75– 0.25 = 0.50 (Retain the Item)

Thus, the item also has a "good discriminating power."

It is also instructive to note that the distracter A is not an effective distracter since
this was never selected by the students. It is an implausible distracter. Distracters
C and D appear to have good appeal as distracters. They are plausible
distracters.

Index of Difficulty (Percentage)

𝑅𝑢 + 𝑅𝐿
P= X 100
𝑇

Where:
RU – The number in the upper group who answered the item correctly
RL - The number in the lower group who answered the item correctly
T – The total number who tried the item

𝟔+𝟐
P= 𝒙 𝟏𝟎𝟎 = 𝟒𝟎%
𝟐𝟎
The smaller the percentage figure the more difficult the item.
Index of item Discriminating Power
𝑹𝒖 + 𝑹𝑳
D= 𝟏
𝑻
𝟐

Where:
P- percentage who answered the item correctly (index of difficulty)
R- number who answered the item correctly
T- total number who tried the item

𝑹𝒖 − 𝑹 𝑳 𝟔−𝟐
D= 𝟏 = = 𝟎. 𝟒𝟎
𝑻 𝟏𝟎
𝟐

The discriminating power of an item is reported as a decimal fraction;

maximum discriminating power is indicated by an index of 1.00.

Maximum discrimination is usually found at the 50 % level of difficulty

0.00 – 0.20 very difficult
0.21 – 0.80 moderately difficult
0.81 – 1.00 very easy

SUMMARY:
The Item-Analysis Procedure for Norm-provides the following information:
1. The difficulty of the item;
2. The discriminating power of the item, and
3. The effectiveness of each alternative•
Some benefits derived from Item Analysis are:
1: It provides useful information for class discussion of the test.
2. It provides data which help students improve their learning.
3. It provides insights and skills that lead to the preparation for better tests in the
future.
The INDEX OF DISCRIMINATION is the difference between the proportion of the
upper group who got an item right and the proportion of the lower group who got
the item right. This index is dependent upon the difficulty of an item. It may reach
a maximum value of 100 for an item with an index of difficulty of 50; that is, when
100% of the upper group and none of the lower group answer the item correctly.
For items of less than or greater than 50 difficulty, the index of discrimination has
a maximum value of less than 100.

More Sophisticated Discrimination Index

Item discrimination refers to the ability of an item to differentiate among students
on the basis of how well they know the material being tested. Various, hand
calculation procedures have traditionally been used to compare item responses to
total test scores using high and low scoring groups of students. Computerized
analyses provide more accurate assessment of the discrimination power of items
more accurate assessment of the discrimination power of items because they take
into account responses of all students rather than just high and low scoring groups.

The item discrimination index provided by ScorePak® is a Pearson Product

Moment correlation between student responses to a particular item and total
scores on all other items on the test. This index is the equivalent of a point-biserial
coefficient in this application. It provides an estimate of the degree to which an
individual item is measuring the same thing as the rest of the items.
Because the discrimination index reflects the degree to which an item and the test
as a whole are measuring a unitary ability or attribute, values of the coefficient will
tend to be lower for tests measuring a wide range of content areas than for more
homogeneous tests. Item discrimination indices must always be interpreted in the
context of the type of test which is being analyzed. Items with low discrimination
indices are often ambiguously worded and should be examined. Items with
negative indices should be examined to determine why a negative value was
obtained. For example, a negative value may indicate that the item was mis-keyed,
so that, students who knew the material tended to choose an unkeyed, but correct,
response option.
Tests with high internal consistency consist of items with mostly positive
relationships with total test score. In practice, values of the discrimination index
will seldom exceed.50 because of the differing shapes of item and total score
distributions. ScorePak® classifies item discrimination as "good" if the index is
above .30; "fair" if it is between 0. 10 and 0.30; and "poor" if it is below 0.10.
A good item is one that has good discriminating ability and has sufficient level of
difficult (not too difficult nor too easy). At the end of the Item Analysis report, test
items are listed according to their degrees of difficulty (easy, medium, hard) and
discrimination (good, fair, poor). These distributions provide a quick overview of
the test, and, can be used to identify items which are not performing well and
which can perhaps be improved or discarded.

tween these two groups of students. We are therefore interested in deriving a measure
that will tell us whether an item can discriminate between these two groups of
students. Such a measure is called an index of discrimination. An easy way to derive
such a measure is to measure how difficult an item is with respect to those in the
upper 25% of the class and how difficult it is with respect to those in the lower 25%
of the class. If the upper 25% of the class found the item easy yet the lower 25% found
it difficult, then the item can discriminate properly between these two groups. Thus:
Index of discrimination = DU - DL (U - Upper group; L - Lower group) Example: Obtain
the index of discrimination of an item if the upper 25% of the class had a difficulty
index of 0.60 (i.e. 60% of the upper 25% got the correct answer) while the lower 25%
of the class had a difficulty index of 0.20. Here, DU = 0.60 while DL = 0.20, thus index
of discrimination = .60 - 20 - 40. Discrimination index is the difference between the
proportion of the top scorers who got an item correct and the proportion of the lowest
scorers who got the item right. The discrimination index range is between -1 and +1.
The closer the discrimination index is to +1, the more effectively the item can
discriminate or distinguish between the two groups of students. A negative
discrimination index lower group to answer the item correctly even if the question is
easy. How do we decide on the basis of this index whether the item is too difficult or
too easy? The following arbitrary rule is often used in the literature: Range of Difficulty
Index 0 - 0.25 Interpretation Difficult Action Revise or discard 0.26 - 0.75 Right
difficulty Retain 0.76 - above Easy Revise or discard