ASSESSMENT
PLANNING THE CLASSROOM
TEST
Presented by Dr. Z. Dedovets
12.08.2014
The Power of Testing
Just as medical test help diagnose and treat
patients, rigorous and meaningful education
assessments can help ensure the academic
health of all students
McGiffert
Gandal &
Test
Measurement
Evaluation
Testing Terminology
Test: a formal, systematic, usually paper-and-pencil
procedure for gathering information.
Test is the presentation of a standard set of questions to
be answered.
Measurement: the process of quantifying or assigning a
number to performance.
Using observation, rating scales, or any other devise that
allows us to obtain information in a quantitative form is
measurement.
Testing Terminology
Evaluation is the determination of congruence
between performance and objectives.
Evaluation : a systematic process for the collection
and use of information from many sources to be
applied in interpreting the results and in making value
judgments and decisions.
Purposes of Classroom Tests
Purposes of Classroom Tests
Judging students mastery of skills and knowledge
Measuring growth over time
Ranking students in terms of their achievement
Diagnosing student difficulties
Evaluating the teachers instructional method
Encouraging good study habits
Motivating students
Purposes of measurement and evaluation
Measurement and Evaluation help the teacher
The major role of the school is to facilitate
learning. Kinds of changes we wish to obtain in
students are commonly referred to as objectives,
or goals. The method we employ to help students
realize the objectives constitute educational
experiences or instruction. The evaluation
procedures are the means of determining the
extent to which the instruction has been effective.
There is a definite relationship among instruction,
objectives, and evaluation.
Schematically,
we can represent this relationship as follows:
Objectives
Educational
Experiences
Evaluation
Procedures
Objectives determine the instructional procedures and the
method used to evaluate both educational experiences and
evaluation procedures. At the same time, evaluation and
educational experience help clarify the objectives, and the
learning experiences help determines the evaluative procedure
to be used.
Evaluation procedures aid the teacher; (1) they help in providing
knowledge concerning the students entry behaviors; (2) they
help in setting, refining, and clarifying realistic goals for each
student; (3) they help in evaluating the degree to which the
objectives have been achieved; and (4) they help in
determining,
evaluating, and refining the instructional techniques.
Measurement and Evaluation help the student
Measurement and evaluation aid the student by (1)
communicating the teachers goals, (2) increasing
motivation, (3) encouraging good study habits, and (4)
providing feedback that identifies strengths and
weaknesses.
It is important to point out that we never measure or
evaluate students. We measure or evaluate characteristics
or properties of students : their scholastic potential,
knowledge of algebra, honesty, perseverance, ability to
learn, and etc.
Classification
of classroom assessment
Formative
Summative
http://www.youtube.com/watch?v=u0nSW3W4MW0
Formative and Summative Assessment
Formative Assessment
Assessment for learning
How can students improve?
How can they close or bridge the gap?
Weeden, Winter and Broadfoot (2002)
In a formative evaluation (assessment for learning),
teachers use information about student
achievement to monitor progress and plan further
instruction.
Formative Assessment
Short and simple tasks;
Not assigned marks or grades;
Monitoring learning progress;
Pinpoint learning difficulties;
Focus is on growing and developing skills.
Summative Assessment
Assessment of learning
What is students current performance?
Weeden, Winter and Broadfoot (2002)
In a summative evaluation (assessment of learning),
teachers make judgments about student achievement at
the completion of the learning process. Teachers use
the results of summative evaluation to determine final
grades.
Summative Assessment
Given at the end of a unit, term or years;
Grades the attainment of learners;
Determines the extent to which objectives were
achieved;
Contributes evidence to the effectiveness of courses
and teachers.
Planning Classroom Test
1. Specify the instructional objectives
Blooms Taxonomy of objectives
2. Preparing the Table of Specifications
Bloom's Taxonomy
Before we examine question types, it's beneficial if we establish
a common language for discussing cognitive processes. If we
hope to assess specific levels of student mastery of course
content, we will need a framework for describing those levels.
Bloom's taxonomy is a system created to improve testing
precision by categorizing cognitive functioning into distinct
levels. Appropriate questions could then be developed to
assess the desired level. Psychologist Benjamin Bloom
developed this system at the University of Chicago in the late
1940s. His goal in formulating this classification was to
increase precision in the discussion of educational goals
among teachers, administrators, and researchers.
Bloom's levels are:
Knowledge
Low
Comprehension
Application
Analysis
Synthesis
Evaluation
High
"Taxonomy" is simply a system of categorizing and
organizing. In this case, the taxonomy is hierarchical;
each level is subsumed by the higher levels. In other
words, a student functioning at the "application" level
has also mastered the material at the "knowledge" and
"comprehension" levels.
Level
Knowledge
Learner Action
Question Cues
Recall content in the exact form
List, define, label,
that it was presented.
identify, name.
Memorization of definitions,
formulas, or procedures are
examples of knowledge-level
functioning.
Comprehension Restate material in their own
Application
Describe, associate,
words, or can recognize
previously unseen examples of
a concept.
categorize,
summarize.
Apply rules to a problem,
without being given the rule or
formula for solving the problem.
Apply, calculate,
illustrate, solve.
Level
Learner Action
Question Cues
Analysis
Break complex concepts or
Analyze, compare, separate,
situations down into their
order, explain.
component parts, and analyze
how the parts are related to
one another.
Synthesis
Evaluation
Rearrange component parts to Combine, modify, rearrange,
form a new whole.
"what-if.
Evaluate or make judgments
Assess, decide, grade,
on the worth of a concept,
recommend, explain, judge.
object, etc. for a purpose.
Types of Test Question
Multiple-choice Questions
Matching Questions
True/False Questions
Short-Answer Questions
Essay Questions
Multiple-choice Questions
Multiple-choice item consists of two parts. The first is the
stem, which can be a question or an incomplete statement.
The second part consists of several options from which to
select the correct answer. The options should include one,
and only one, correct response and several incorrect
options, or distractors.
Advantages:
Multiple-choice items allow the instructor to sample a
large amount of content in a single test.
These items can be scored easily and objectively. Scores
on multiple-choice test are less influenced by guessing
than are scored on true-false tests. These items are
versatile because they can measure understanding at
several cognitive levels in the taxonomy categories.
Multiple-choice Questions
Disadvantages:
Writing good items with plausible distractors can be very
time consuming for the instructor.
This item type takes more time for the student to read and
understand.
These items may discriminate the creative, verbal student.
Scored can be affected by students reading ability and
the instructors writing style.
This item type can raise the score of the student who can
recognize rather than produce the correct answer.
Multiple-choice Questions
Blooms Levels:
Knowledge
Comprehension
Application
Analysis
Multiple-choice Questions
An incomplete statement followed by several answer
choices.
Example 1
The first president of the United States, _____, was
known as the
Father of his country.
a.
b.
c.
d.
Thomas Jefferson
Abraham Lincoln
George Washington
Theodore Roosevelt
Multiple-choice Questions
Example 2.
______ is a country in South America.
a. Russia
b. Mexico
c. Japan
d. none of the above
Matching Questions
Matching questions involve paired lists that require
students to correctly identify, or match, the relationship
between the items.
Advantages:
Matching items can assess a large amount of information
in a confined space on the exam page; if developed
carefully, the probability of guessing is low. To decrease
that probability further, avoid equal-sized lists by including
a few distractor items in the second (answer) column.
Matching Questions
Disadvantages:
Matching assesses recognition rather than recall of
information.
Most Appropriate For:
Assessing student understanding of related information.
Examples of related items include states and capitals,
terms and definitions, tools and uses, and events and
dates.
Blooms Levels:
Knowledge
Comprehension
Matching Questions
Example 1:
Directions: On the line next to each author in Column A, place
the letter of the type of writing in Column B for which the
author is best known. Answers in Column B may be used
once, more than once, or not at all.
Column A
______1. Agatha Christie
______2. Isaac Asimov
______3. Erma Bombeck
______4. Walt Whitman
______5. Stephen King
______6. James Michener
Column B
A. History
B. Horror
C. Humor
D. Mystery
E. Poetry
F. Science Fiction
G. Tragedy
Matching Questions
Example 2:
Column A
______1. And
______2. Dog
______3. Jump
______4. She
______5. Quickly
Column B
A. Adjective
B. Adverb
C. Conjunction
D. Noun
E. Preposition
F. Pronoun
G. Verb
True/False Questions
True/false questions present a statement, and prompt the
student to choose whether the statement is truthful. Students
typically have a great deal of experience with this type of
question.
Advantages:
True/false questions are among the easiest to write, and can be
scored electronically.
Disadvantages:
True/false questions are limited in what kinds of student
mastery they can assess. They have a relatively high probability
of student guessing the correct answer (50%). True/false also
assesses recognition of information, as opposed to recall.
True/False Questions
Most Appropriate For:
Factual information and naturally dichotomous information
(information with only two plausible possibilities).
Dichotomous information is "either/or" in nature. Examples
include male/female, analog/digital, and internal/external.
Bloom's Levels:
Knowledge
Comprehension
True/False Questions
Example 1
T/F
A virus is the smallest known organism.
T/ F
An atom is the smallest particle of matter.
Yes / No
In the equation 2x + 5 = 9; x equals 3.
Yes / No
Acid turns litmus paper red.
True/False Questions
Example 2:
The University is centrally located in the city with sufficient
student parking.
According to the President, teachers in rural areas should
be first to receive salary increases.
Short-Answer Questions
Short-answer questions are constructed-response, or
open-ended questions that require students to create an
answer. Short-answer items typically require responses of
one word to a few sentences. Fill in the blank and
completion questions are examples of short-answer
question types.
Advantages:
Short-answer questions assess unassisted recall of
information, rather than recognition. Compared to essay
questions, they are relatively easy to write.
Short-Answer Questions
Disadvantages:
Short-answer items are only suitable for questions that can be
answered with short responses. Additionally, because students
are free to answer any way they choose, short-answer questions
can lead to difficulties in scoring if the question is not worded
carefully. Its important when writing short-answer questions
that the desired student response is clear.
Most Appropriate For:
Questions that require student recall over recognition. Examples
include assessing the correct spelling of items, or in cases when it
is desirable to ensure that the students have committed the
information to memory (medical students, for example, will require
recall of information more than recognition by the nature of their
jobs).
Blooms Levels:
Knowledge
Comprehension
Application
Short-Answer Questions
Example 1:
What is the name of the author of Moby Dick?
(Herman
Melville)
What is the formula for hydrochloric acid?
(HCL)
What is the value of x in the equation 2x + 5 = 9?
(2)
Short-Answer Questions
Example 2:
Define the vocabulary words listed below:
_________________________________Vegetarian
_____________________________________Vegan
_____________________________________Carnivore
_____________________________________ Herbivore
Essay Questions
Essay items, like short-answer, are constructed- response
questions. However, essay answers are typically much
longer than those of short-answer, ranging from a few
paragraphs to several pages.
Advantages:
Essay questions are the only question type that can
effectively assess all six levels of Blooms Taxonomy. They
allow students to express their thoughts and opinions in
writing, granting a clearer picture of the level of student
understanding. Finally, as open-ended questions, they
assess recall over recognition.
Essay Questions
Disadvantages:
There are two main disadvantages to essay questions:
time requirements and grading consistency. Essays are
time-consuming for students to complete, and require
careful instructions on the part of the test writer. Scoring
can be difficult because of the variety of answers, as well
as the halo effect (students rewarded for strong writing
skills as opposed to demonstrated mastery of the
content).
Essay Questions
Most appropriate for:
Assessment that cannot be accomplished with other question
types. Because essays are the only question types that can
effectively assess the highest levels of student mastery, they
are the only option if the goal of testing is the assessment of
synthesis and evaluation levels.
Blooms Levels:
Knowledge
Comprehension
Application
Analysis
Synthesis
Evaluation
Essay Questions
Examples:
Education comes not from books but from practical
experience.
An understanding of the past is necessary for
solving the problems of the present.
Types of Question
Multiple-choice
Blooms Levels
Knowledge
Comprehension
Application
Matching
Analysis
Knowledge
Comprehension
True/False
Knowledge
Comprehension
Short-Answer
Knowledge
Comprehension
Application
Knowledge
Comprehension
Application
Analysis
Synthesis
Evaluation
Essay
What is a Table of Specification (TOS)
A two-way grid that lists major areas of content to be
covered by the test and major kinds of abilities to be
developed and tested.
A grid which outlines the plan or blueprint of the test
to ensure that it is consistent with the teaching goals
both in content and levels of analyses expected.
Why use a table of specifications?
To help teachers focus their instruction (plan a unit of
work);
To ensure a fair and representative sample of questions
(balanced test);
To allow the teachers to construct a test which focuses on
key areas and weights those different areas based on their
importance (score and emphasis);
To identify the achievement domains being measured;
To assist students develop better understanding of material
and improve their study skills.
A Step by Step Example of the
Development of a Table of Specification
(TOS)
Step 1: List the contents
For example
The students are to be tested on a story they
have read. The questions will relate to:
The story
The characters
Step 2: List the objectives
For example
General objective: the students will understand
the story.
Specific Objectives:
1. They will state certain facts about the plot.
2. They will explain in their own words certain
underlined words and phrases.
3. They will state certain facts about the
characters.
4. They will write characters sketches.
5. They will compare two characters.
6. They will evaluate the characters.
Step 3: Classify the objectives
Using Blooms classification for the above
objectives, they relate to:
Knowledge
Comprehension
Synthesis
Analysis
Evaluation
Step 4: Prepare a grid
1. Place your content areas in column 1, and your
classifications in row 1.
2. Extend your grid one space below contents,
and one space beyond your classifications to be
used for totals. (See Table 1.1.)
3. Enter each objective in the appropriate box
according to its content and classification. (See
Table 1.2.)
Table 1.1: The Empty Grid
Classification
Contents
Story
Character
Knowledge
Comprehension
Analysis
Synthesis
Evaluation
Table 1.2: Grid with Objectives Entered
Classification
Contents
Story
Knowledge
Compreh
ension
Analysis
Synthesis
Evaluation
Obj._____
Obj___
Obj___
Obj.____
Obj.____
(i)
(ii)
(iv)
(vi)
(iii)
Character
(v)
___
Step 5: Decide on the relative importance of each level,
and weight it accordingly
This should reflect the emphases you placed in your teaching
and the overall importance of tasks at each level. Weighting
can be done either by using percentages or score points. Use
whichever you are more at ease with.
My example will use score points. For this particular test I
might decide that I stressed knowledge more than the other
levels, then placed some emphasis on comprehension and
analysis, but very little on synthesis and evaluation. This might
lead me to decide that, out of 50 score points for the entire
test, I will allot
1. 20 points to knowledge
2. 10 points to comprehension
3. 10 point to analysis
4. only 5 points to synthesis
5. only 5 points to evaluation
I now enter these figures in the final row of my table in the
appropriate columns (Table1.3).
Table 1.3: Grid with Weightings for Each Level
Knowledge Compre Analysis
Classifihension
cation
Contents
Story
Character
Obj._____
Obj___
(i)
(ii)
(iii)
20
Obj___
(v)
10
10
Synthesis
Evaluation
Obj.____
Obj.____
(iv)
(vi)
___
50
Step 6: Weighting objectives
In order to decide how many points to give to each
objective, I now examine how many objectives fall in each
column, and share the score points between them. This
should be done to leave whole numbers, or else the
scoring of the final test will be very complicated. For
example, in the knowledge column there are two
objectives, which will therefore receive 10 points each; if
there were three objectives here, it would be better to allot
scores 7,7, and 6, or perhaps 6,6,8 rather than 6.666. In
such a case, the test constructor would have to decide
which objective merited the higher score.
Enter these points on your grid (Table 1.4).
Table 1.4: Grid with Objectives Weighted
Knowledge Compre Analysis
Classifihension
cation
Contents
Story
Character
Obj._____
Obj___
Synthesis
Evaluation
Obj___
Obj.____
Obj.____
(v)
10
(iv) 5 (vi) 5
10
(i) 10 (ii)
(iii) 10
20
10
10
50
Step 7: Total the scores for the separate content areas
In the final column of the grid add up the total number of
points
you have given each content area. Ask yourself whether
these totals are a reasonable distribution of the scores in
relation to your teaching. If you decide so, then your grid is
complete.
If you decide that the distribution is inaccurate, you must
adjust your plans at this time. You may have to change the
weighting of one or more of the classifications, or you may
even decide to test more objectives. If your test is to have a
proper balance, you must sort out these problems now.
Remember there is no magic formula for deciding how many
points go where. It is up to you as a teacher and test
constructor to arrive at a reasonable solution. Table 1.5
shows a completed grid.
Table 1.5: The Finished Grid
Knowledge Compre Analysis
Classifihension
cation
Contents
Story
Character
Obj._____
Obj___
Obj___
Synthesis
Evaluation
Obj.____
Obj.____
(i) 10 (ii)
(iii) 10
20
10
10
20
(v)
10
(iv) 5 (vi) 5 30
10
50
References
1.
Brown, Frederick, G. (1971). Measurement and
Evaluation. Itasca, Ill.: F.E. Peacock
2. Krathwohl, David R., Bengamin S. Bloom, and Bertram B.
Mesia (1964). Taxonomy of Educational Objectives (two
vols: The Affective Domain & The Cognitive Domain). New
York: David McKay.
3. Wolansky, William, D. (1985). Evaluating Student
Performance in Vocational Education. Ames, Iowa: Iowa
State University Press.
4. William A. Mehrens, Irvin J. Lehmann (1991)
Measurement and Evaluation in education and
Psychology, Michegan State University
THANK YOU
for
YOUR ATTENTION
