0% found this document useful (0 votes)
13 views43 pages

Item Analysis - Group 5

Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
13 views43 pages

Item Analysis - Group 5

Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 43

ENGLISH LANGUAGE TEACHING AND LEARNING

ASSESSMENT AND EVALUATION

ITEM
ANALYSIS
CHABIB
ELFRIDA
GROUP 5 NAFTALIA
TRI SETYA
LIST OF CONTENTS

Introduction Analyzing Text


Items

Revising Text Implications for


Items MBKM Program
INTRODUCTION

Test item analysis is a way to help


teachers gain more insight into their
students’ learning. Item analysis is the
act of analyzing responses to individual
test questions, or items, to make sure
that their difficulty level is appropriate.
FUNCTIONS OF ITEM ANALYSIS

Increase the efficacy of the exams by testing knowledge


accurately.
Inform course content and curriculum.
When it comes to item difficulty, it’s important to note
whether errors indicate a misunderstanding of the question
or of the concept the item addresses.
It could be that the material may have to be reviewed in
class, possibly with a different learning approach.
RESEARCH ON TEST ITEM ANALYSIS

Hartati & Yogi, 2019 Karim, et al., 2019

Item analysis has proven effective in the effort


it is highly recommended for
of testing and improving the quality of multiple-
teachers as the test developers and
choice items. By
other stakeholders in the
conducting item analysis, the quality of the
educational assessment to utilize
stems, the answer keys, and also distractors
the test item analysis to improve the
can be tested, which in turns, may assist to
existing teacher-made tests.
prepare better test in the future to make sure
that the assessment reaches its true goal.
PROCEDURES OF ITEM ANALYSIS Gronlund (1993)

ARRANGE SELLECT UPPER GROUP LOWER GROUP Project


ARRANGE ALL
FROM UPPER AND IS 1/3 NUMBERS IS 1/3 NUMBERS installation
NUMBERS OF
HIGHEST TO LOWER OF HIGHEST OF LOWEST
TEST PAPERS SCORES
THE LOWEST GROUP SCORES
DEFINITION OF ITEM ANALYSIS

Item analysis is a statistical procedure


used to help make decisions about which
items are good, which items need to be
revised and which items should be
discarded (Cohen & Swerdlik, 2005)

Notes : When you want to see “how your


question are performing, you can use
item analysis.
THE PURPOSE OF
ITEM ANALYSIS
Examine each question
item to get quality
questions

Knowing students' Can improve the quality of


diagnostic information test items through revision
regarding understanding or removing ineffective
of the material that has (invalid) questions
been taught
Discrimination Power : helps
IMPORTANCE OF ITEM determine how well individual test items
ANALYSIS IN EVALUATING discriminate between high-performing
TEST QUALITYNG TEXT ITEMS and low-performing test-takers

Reliability Assessment : By
Validity Check : Item analysis can analyzing item statistics, can
reveal whether test items are assess the internal consistency or
measuring what they are intended reliability of the test
to measure.
Identification of Problematic Items
: It helps in identifying items that may
need to be revised or removed from
the test.
ADVANTAGES OF ITEM ANALYSIS

help test users in evaluating the quality of the tests used

suitable for preparing informal tests, such as tests prepared by


teachers for students

supports writing effective and quality questions

increasing the validity and reliability of the questions so that


high-quality questions will be created
IN ITEMS ANALYSIS WE CAN GET
THREE THINGS
Item Diffic ulty Item Discrim ination Distractor Analysis

measurement used in item measure used in item analysis the process used in the
analysis to determine how to assess the extent to which context of multiple choice tests
difficult a question or item is an item or question in a test to evaluate the answer options
on a test can differentiate between high- or choices provided for each
achieving test takers from low- test item
achieving test takers
ITEM DIFFICULTY
The item difficulty index is one of the most useful,
and most frequently reported, item analysis
statistics. It is a measure of the proportion of
examinees who answered the item correctly.
Simply it states the level ofdifficulty in answering the
questions by therespondents during their test

FUNCTION
Determining the overall difficulty of the test
Adjusting the test difficulty
Identify items that need revision
Ensuring test equivalency
TO CALCULATE SIMPLY (ITEM DIFFICULTY)
INTERPRETATION

INDEX RANGE DIFFICULTY LEVEL


Example

0.00 – 0.20 VERY


Among 75 test takers, only 45 students got the right DIFFICULT
answer. Solve the index of difficulty of an item?
0.21 – 0.40 DIFFICULT

Answer
0.41 – 0.60 MODERATE
Formula : DIFFICULT
Di = r/N
= 45/75 0.61 – 0.80 EASY
= 0,60
Note: 0.81 – 1.00 VERY EASY

r = number of students who got the item right


N = Total Number of the students
ITEM DISCRIMINATION

Item discrimination is a measure


used in item analysis to assess
the extent to which an item or
question in a test can
differentiate between high-
achieving test takers from low-
achieving test takers. Items that
have good discrimination will
show a positive correlation with
the overall test score.
TO CALCULATE SIMPLY (ITEM DISCRIMINATION)
EXAMPLE
ITEM CHOICES UPPER 27% LOWER 27%

#15 A 1 9

B 0 3

C* 26 15

D 1 1

E 0 0
TO SOLVE FOR THE INDISCRIMINATION INDEX,
WE USE FORMULA

Formula
Ds = Pu-PI
N

Where :
Pu = Proportion of the students in the upper group who got the correct
response.
PL = Proportion of students in the lower group who got the correct response.
N = Total number of takers
TO SOLVE FOR THE INDISCRIMINATION INDEX, WE USE
FORMULA
Formula
Ds = Pu-PI
N

= 26-15
56
= 0.20

26 = The total number of student who get the correct response from the upper group.
15 = The total number of student who get the correct response from the lower group.
We divide it to 56 which is the sum of the test takers from UG or Upper Group and
the LG or Lowered group. Which is 28 + 28 that makes it 56.
N =28 + 28 = 56
INTERPRETATION

DISCRIMINATION INDEX ITEM EVALUATION


VERY DISCRIMINATING OR
0.41 – 1.00 VERY GOOD ITEM

0.31 - 0.40 DISCRIMINATING OR GOOD ITEM

0.21 - 0.30 MODERATE ITEM

NON-DISCRIMINATING, FAIR ITEM


0.11 - 0.20 & NEEDS REVISION

BELOW 0.10 POOR OR QUESTIONABLE ITEM,


CAN BE REJECTED OR DISCARDED
INDEX DIFFICULTY
INTERPRETATION

INDEX RANGE DIFFICULTY LEVEL


Answer VERY
0.00 – 0.20 DIFFICULT
Formula :
Di = r/N
0.21 – 0.40 DIFFICULT
= 41/56
= 0,73
0.41 – 0.60 MODERATE
r = 26+15 = 41 DIFFICULT

Note: 0.61 – 0.80 EASY


r = number of students who got the item right
N = Total Number of the students 0.81 – 1.00 VERY EASY
DISTRACTOR ANALYSIS
is a process used in the context of multiple choice tests to
evaluate the answer options provided for each test item.
Distractor analysis is an important part in the development
and evaluation of multiple choice tests, especially in
measuring participants' ability to understand and apply
certain concepts or knowledge.

PURPOSE
Assessing the quality of distractors
Improving test validity and reliability
Identify conceptual problems
Refinement of instructions
FOR EXAMPLE

TICK THE CORRECT ANSWER !

1. WHEN THE SUM OF THE MEASURES OF TWO ANGLES IS 90


DERAJAT, THE ANGLES ARE CALLED?
A. COMPLEMENTARY ANGELS
B. ADJACENT ANGELS
C. LINER PAIR SUPPOSE 110 STUDENTS APPEARED IN
D. SUPLEMENTARY ANGELS THIS ACHIEVEMENT TEST. FIRST WE
ARRANGE THE OBTAINED SCORES OF
2. WHAT IS THE SUPPLEMENTARY OF AN ANGLE 110 STUDENTS IN DESCENDING ORDER.
MEASURING 54 NOW WE HAVE TO SELECT HIGHEST
A.36 SCORER, UPPER 27% STUDENTS I.E., 30
B. 126
C.156 STUDENTS AND LOWEST SCORER 27%
D.46 STUDENTS
I.E. 30 STUDENTS FOR ITEM ANALYSIS
Options

Item 1

A B C D

Number in
22 3 2 3
Upper group (30)
HOW TO CALCULATE ?

Number in
9 7 8 6
Lower Group (30)

Total (60) 31 10 10 9

% of
53% 16,5 % 16,5 % 15%
selection
FORMULA

Difficulty Index = Ru + RL
T
= Ru+ RL = 22 + 9 = 0.52
60

NOTES
RU = THE NUMBER IN THE UPPER GROUP WHO ANSWERED THE ITEM CORRECTLY
RL = THE NUMBER IN THE LOWER GROUP WHO ANSWERED THE ITEM CORRECTLY
T = THE NUMBER WHO TRIED THE ITEM
INTERPRETATION
INDEX DIFFICULTY
INDEX RANGE DIFFICULTY LEVEL

0.00 – 0.20 VERY


DIFFICULT

0.21 – 0.40 DIFFICULT

0.41 – 0.60 MODERATE


DIFFICULT

0.61 – 0.80 EASY

0.81 – 1.00 VERY EASY


FORMULA

Descrimination Index = Ru - RL
1/2T
= Ru+ RL = 22 - 9 = 0.43
30

NOTES
RU = THE NUMBER IN THE UPPER GROUP WHO ANSWERED THE ITEM CORRECTLY
RL = THE NUMBER IN THE LOWER GROUP WHO ANSWERED THE ITEM CORRECTLY
T = THE NUMBER WHO TRIED THE ITEM
INTERPRETATION
DISCRIMINATION INDEX

INDEX DIFFICULTY DISCRIMINATION

0.86 ABOVE VERY EASY TO BE DISCARDED

0.71- 0.85 EASY TO BE REVISED

0.30 -0.70 VERY GOOD ITEM


MODERATE

0.15 - 0.29 DIFFICULT TO BE REVISED

0.14 BELOW VERY DIFFICULT TO BE DISCARDED

NOW COMPARING IT WITH THE TABLE WE CAN CONCLUDED THAT, THE


ITEM NO 1 IS MODERATE AND VERY
GOOD ITEM, HENCE WE CAN SELECT IT.
Options

Item 2

A B C D

Number in
11 28 0 0
Upper group (30) HOW TO CALCULATE ?

Number in
5 12 0 4
Lower Group (30)

Total (60) 16 40 0 4

% of
26,5 % 67 % 0% 6,5 %
selection
FORMULA

Difficulty Index = Ru + RL
T
= Ru+ RL = 28 + 12 = 0.67
60

NOTES
RU = THE NUMBER IN THE UPPER GROUP WHO ANSWERED THE ITEM CORRECTLY
RL = THE NUMBER IN THE LOWER GROUP WHO ANSWERED THE ITEM CORRECTLY
T = THE NUMBER WHO TRIED THE ITEM
INTERPRETATION
INDEX DIFFICULTY
INDEX RANGE DIFFICULTY LEVEL

0.00 – 0.20 VERY


DIFFICULT

0.21 – 0.40 DIFFICULT

0.41 – 0.60 MODERATE


DIFFICULT

0.61 – 0.80 EASY

0.81 – 1.00 VERY EASY


FORMULA

Discrimination Index = Ru - RL
1/2T
= Ru-RL = 28 - 12 = 0.53
30

NOTES
RU = THE NUMBER IN THE UPPER GROUP WHO ANSWERED THE ITEM CORRECTLY
RL = THE NUMBER IN THE LOWER GROUP WHO ANSWERED THE ITEM CORRECTLY
T = THE NUMBER WHO TRIED THE ITEM
INTERPRETATION
DISCRIMINATION INDEX

INDEX DIFFICULTY DISCRIMINATION

0.86 ABOVE VERY EASY TO BE DISCARDED

0.71- 0.85 EASY TO BE REVISED

0.30 -0.70 VERY GOOD ITEM


MODERATE

0.15 - 0.29 DIFFICULT TO BE REVISED

0.14 BELOW VERY DIFFICULT TO BE DISCARDED

NOW COMPARING IT WITH THE TABLE, WE CAN CONCLUDE THAT, ITEM


NO 2 IS MODERATE AND VERY GOOD ITEM, HENCE WE CAN SELECT IT
REVISING TEST ITEMS

Revising test items refers to the process of reviewing and modifying the
questions or tasks used in a test or assessment.
This is done to ensure that the items are clear, accurate, and aligned with the
intended learning outcomes.

Revising test items may involve making changes to the wording, format, or
content of the questions to improve their validity and reliability.
IMPORTANCE OF REVISING TEST ITEMS

1. Ensuring clarity (clear and easily understood by the test takers)


2. Enhancing validity (questions or tasks are aligned with the intended
learning outcomes and are assessing the desired knowledge or skills)
3. Improving reliability (help to eliminate any potential sources of bias or
error)
4. Promoting fairness (fair and unbiased)
5. Enhancing engagement (engage test takers and maintain their interest and
motivation/Meaningful test)
TECHNIQUES FOR REVISING TEST ITEMS

1. Expert review : doing by expert

2. Pilot testing : doing by group and they will be give frrdback

3. Item analysis : identify that it is too esay or too difficult

4. Cognitive analysis : identify any issues with item wording, clarity, or

comprehension

5. Alignment with standard : Test items should be aligned with the relevant

standards
EXAMPLE OF REVISING TEST ITEMS

ORIGINAL TEST ITEM:

WHICH OF THE FOLLOWING


IS THE CAPITAL OF REVISED TEST ITEM:
FRANCE?
WHICH OF THE FOLLOWING IS
A. LONDON THE CAPITAL OF FRANCE?
B. PARIS
C. ROME A. LONDON
D. BERLIN B. PARIS
C. ROME
D. MADRID

In this example, the original test item had the correct answer as option b) Paris.
However, upon review, it was determined that option d) Berlin was too similar to
option b) and could potentially confuse test takers.
Therefore, the item was revised by replacing option d) with option d) Madrid,
which is the capital of Spain.
IMPLICATIONS FOR MBKM PROGRAM

Importance of analyzing and Potential benefits of analyzing and Challenges and limitations of
revising test items in the context of revising test items for improving analyzing and revising test items
MBKM program student learning outcomes in the context of MBKM program
IMPORTANCE OF ANALYZING Discrimination Power : helps
AND REVISING TEST ITEMS IN determine how well individual test items
THE CONTEXT OF MBKM discriminate between high-performing
PROGRAM and low-performing test-takers

Reliability Assessment : By
Validity Check : Item analysis can analyzing item statistics, can
reveal whether test items are assess the internal consistency or
measuring what they are intended reliability of the test
to measure.
Identification of Problematic Items
: It helps in identifying items that may
need to be revised or removed from
the test.
POTENTIAL BENEFITS OF ANALYZING AND REVISING TEST ITEMS
FOR IMPROVING STUDENT LEARNING OUTCOMES

T h e i m p r o v e m e n t o f l e a r n i n g o u t c o m e s a n d d e e p er comprehension

T h e d e v e l o p m e n t o f t h e c r i t i c a l t h i n k i n g s k i l l s a n d knowledge needed for


future career success

F a i r a n d u n b i a s e d a s s e s s m e n t s , e s p e c i a l l y f o r d i v erse student populations

H e l p s i n s t r u c t o r s e l i m i n a t e i n e f f e c t i v e m e a s u r e s of
student learning outcomes

G u i d e l i n e s s f o r i n s t r u c t o r s t o a d a p t a s s e s s m e n t s to evolving student needs


and learning objectives
POTENTIAL BENEFITS OF ANALYZING AND REVISING TEST ITEMS
FOR IMPROVING STUDENT LEARNING OUTCOMES

i d e n t i f y g a p s i n k n o w l e d g e a n d h e l p s t u d e n t s l e a r n more from the next


study episode

e n c o u r a g e s s t u d e n t s t o s t u d y a n d p r o v i d e s f e e d b ack to instructors

b e t t e r o r g a n i z a t i o n o f k n o w l e d g e a n d i m p r o v e t r a n sfer of knowledge to new


contexts

f a c i l i t a t e r e t r i e v a l o f m a t e r i a l t h a t w a s n o t t e s t e d and improve metacognitive


monitoring
CHALLENGES AND LIMITATIONS OF ANALYZING AND REVISING
TEST ITEMS IN THE CONTEXT OF MBKM PROGRAM

difficult to ensure that test items are aligned with the goals
of the MBKM curriculum and accurately measure student
learning outcomes
time-consuming and require a
significant amount of effort continuous process that requires
ongoing effort and resources

there may be resistance to change


from instructors who are used to challenging to ensure that test items
traditional assessment methods are valid and reliable measures of
student learning outcomes

difficulty to ensure that assessments


ensuring test item accessibility for
are fair and unbiased, particularly in
all students can be challenging
the context of a diverse student
population
designing accurate test items for
these skills can be challenging
THANK YOU
DISCUSSION
REFERENCES
Gronlund, N.E. (1993). How to make achievement tests and assessments. University of Michigan.

Hartati, N., & Yogi, H. P. S. (2019). Item analysis for better quality test. English Language in Focus, 2(1), 59-70.

Karim, S. A., Sudiro, S., & Sakinah, S. (2021). Utilizing test items analysis to examine the level of difficulty and
discriminating power in a teacher-made test. EduLite: Journal of English Education, Literature, and Culture, 6
(2), 256-269. http://dx.doi.org/10.30659/e.6.2.256-269

You might also like