Module 5
Module 5
01 07-April-2021
QUALITATIVE EVALUATION
Qualitative and quantitative information are valuable forms of evidence about student outcomes.
Quantitative evidence consists of data that are represented numerically. For instance, performance
on a test or responses to a questionnaire may be scored so that a number represents the degree to
which an individual performed or agreed/disagreed with a certain concept. Because quantitative
data are expressed in numbers, they can be compared directly or subjected to statistical analysis,
and they can enable the researcher to make certain assumptions when comparing one data point to
another. Quantitative data also may permit one to express numerically meaningful changes in
performance (given certain conditions). One may claim, for instance, that a change in a test score
from 50 to 60 represents a 10-point or a 20 percent gain in an individual’s performance, expressed
as a percentage of his or her original score. Quantitative data, therefore, are valued for the ease with
which calculations and comparisons can be made, and for the easily understandable representations
of performance that they produce.
Qualitative evidence typically comes in two forms. The first form involves simple categorization
of individuals into discrete groups (e.g., employed or unemployed; participates in athletics or does
not participate in athletics). The second form of qualitative evidence is data expressed in prose or
narrative. A question is asked of an individual and he or she responds in a free-form manner,
expressing, for instance, an idea, opinion, or evaluation. Because of their non-numerical nature,
qualitative data cannot be subjected directly to statistical analyses, nor can easy direct comparisons
be made without engaging in an intervening process to categorize or interpret the data.
Qualitative data, however, can be “richer” than quantitative data, because they provide a more
extensive variety of information related to a particular learning goal. Many faculty members, for
instance, use the numerical scores (quantitative data) from their teaching evaluations to make
overall judgments of their own performance, but they value the qualitative, narrative comments
A common misconception is that qualitative assessments are not as reliable, valid, or objective as
quantitative ones. This is not necessarily the case. There are well-designed and statistically reliable
means of interpreting and analyzing qualitative data and numerous resources for learning to use
qualitative methods. For example, an instructor might assess the same learning goals using a
multiple-choice test or an essay test. Similarly, an instructor might grade a senior project
presentation quantitatively with a standard set of evaluation criteria (i.e., a rubric). Alternatively,
he or she might provide the student with a prose evaluation, in a non-scaled format, citing the
strengths and weaknesses of the presentation. However, it is best if this evaluation is organized
around a standard set of criteria that were shared with the student beforehand.
Most beginning assessment initiatives are likely to rely more heavily on quantitative, rather than
qualitative, forms of assessment for several reasons. Quantitative data are easier to collect and are
in the form of a readily-analyzable numeric score. In contrast, qualitative data must be sorted,
categorized, and interpreted (most often by humans rather than by computer programs) before a
final judgment can occur. Methods of ensuring the reliability of qualitative data are time-
consuming. For instance, to ensure that portfolio assessment is reliable, at least two raters are used
to review each portfolio, providing a form of “inter-rater” reliability. Focus groups, another
commonly used form of qualitative data collection, require large investments of time to gather data
from comparatively few students.
A good use of qualitative evaluation is to help develop quantitative evaluation criteria (rubrics). For
instance, one might conduct focus groups for the purpose of designing questions for a satisfaction
questionnaire or use a scoring rubric for portfolios to determine what characteristics of students’
writing might be evaluated. For assessing student learning, Characteristics encourages the use of
multiple approaches—both quantitative and qualitative—but it does not require the use of both
approaches. Institutions and faculty members in different programs should be thoughtful about
which approach, or combination of approaches, best suits the student outcomes that are being
assessed in each unique situation.
Quantitative Qualitative
Focus on numbers/numeric values Focus on text/narrative form
Who, what, where, when Why, how
Match with outcomes about knowledge and Match with outcomes about application,
comprehension (define, classify, recall, analysis, synthesis, evaluate
recognize) Seeks to explain and understand
Allows for measurement of variables Ability to capture “elusive” evidence of student
Uses statistical data analysis learning and development
May be generalize to greater population with
larger samples
Easily replicated
Qualitative findings may be presented alone or in combination with quantitative data. At the
simplest level, a questionnaire or interview that asks both fixed-choice (closed) questions and open-
ended questions is an example of how quantitative measurement and qualitative inquiry are often
combined.
The quality of qualitative data depends to a great extent on the methodological skill, sensitivity, and
integrity of the evaluator. Systematic and rigorous observation involves far more than just being
present and looking around. Skillful interviewing involves much more than just asking questions.
Content analysis requires considerably more than just reading to see what's there. Generating useful
and credible qualitative findings through observation, interviewing, and content analysis requires
discipline, knowledge, training, practice, creativity, and hard work.
LEARNING ACTIVITY 1
Answer this:
1. What kind of test item is the given? Qualitative or Quantitative? Why?
“Students, I’d like you to decide whether this scene from the novel is about the theme of social
injustice. Be sure to back up your statement with examples from the text.”
2. If a science teacher wants to see how much a student remembers about cell parts and their specific
functions, which test method is he/she likely to choose? A quantitative or qualitative? Why?
3. If an English teacher wants to measure a student’s ability to understand two selections from
different periods of literature, which test method is he/she likely to choose? A quantitative or
qualitative? Why?
CONSTRUCTIVE FEEDBACKING
In order for student assessment information to improve curriculum, instruction, and student
learning, accurate analysis of assessment results must be followed by feedback to the student and
teacher.
Constructive feedback is providing useful comments and suggestions that contribute to a positive
outcome, a better process or improved behaviors.
It provides encouragement, support, corrective measures and direction to the person receiving it.
Knowing how to give constructive feedback is a valuable skill.
Constructive feedback can be positive (letting someone know they're doing well), negative (letting
people know about ways in which things could be improved), or neutral (just an objective
observation).
Feedback at the beginning, during, and at the end of the instructional process are all necessary to
provide students the best learning opportunities.
The goal of feedback is to increase individual knowledge, skills, and achievement. For
students to learn from their confusion or miscues, they must be aware of what they have
accomplished as well as what they need to work on. Providing descriptive feedback, either
Feedback at the end of instruction informs the student, parents, teacher, and school
system how well the goals of instruction have been met. For the student, reporting of
summative assessments ranges from a grade on a single assignment to earning a credit for a
class. For the teacher, progress of the individual student as well as effectiveness of their
teaching can be measured. Analysis of trends in student performance informs teachers as
they ensure all students master a guaranteed and viable curriculum.
2. It is specific rather than general. To be told that one is “dominating” will probably not be as useful
as to be told that “in the conversation that just took place, you did not appear to be listening to what
others were saying, and I felt forced to accept your arguments.”
3. It is focused on behavior rather than on the person. It is important that we refer to what a person
does rather than to who we think or imagine they are. Thus, we might say that a person “talked more
than anyone else in this meeting” rather than that they are a “loud-mouth.” The former allows for the
possibility of change: the latter implies a fixed personality trait.
4. It takes into account the needs of both the receiver and giver of feedback. Feedback can be
destructive when it serves only our own needs and fails to consider the needs of the person on the
receiving end. It should be given to help, not to hurt. We too often give feedback because it makes us
feel better or gives us a psychological advantage.
5. It is directed toward behavior, which the receiver can do something about. Frustration is only
increased when a person is reminded of some shortcoming over which they have no control.
6. It is solicited rather than imposed. Feedback is most useful when the receiver has formulated the
kind of question which those observing can answer, or when they actively seek feedback.
7. It is well-timed. In general, feedback is most useful at the earliest opportunity after the given
behavior. The reception and use of feedback involves many possible emotional reactions. Excellent
feedback presented at an inappropriate time may do more harm than good.
8. It involves sharing of information rather than giving advice. By sharing information, we leave a
person free to decide for themselves, in accordance with their own goals and needs. When we give
advice, we tell them what to do, and to some degree take away their freedom to decide for themselves.
9. It involves the amount of information the receiver can use rather than the amount we would like
to give. To overload a person with feedback is to reduce the possibility that they may be able to use
what they receive effectively. When we give more than can be used, we are more often than not
satisfying some need of our own rather than helping the other person.
10. It concerns what is said and done, or how, not why. The “why” takes us from the observable to
the inferred and involves assumptions regarding motive or intent. Telling a person what their
motivations or intentions are more often than not tends to result in a defensive response.
FORMS OF FEEDBACK
1. Oral and Written Feedback
Oral feedback is usually given during a lesson while written feedback tends to be given after a
task.
Oral feedback is sometimes underestimated because it is less formal, but it can be a very
powerful and effective tool as it can be provided easily in the ‘teachable moment’ and in a timely
way.
Effective written feedback provides students with a record of what they are doing well, what
needs improvement and suggested next steps.
Effective written feedback also needs to be timely, written in a manner that is understandable to
the student and actionable so that the student can make revisions.
Written feedback needs to include information about where the student has met the learning
intentions and/or success criteria and where improvement is still required.
looks like’ so that they can begin to take on the responsibility of self-assessing and self-
correcting.
LEARNING ACTIVITY 2
Answer this:
1. Why is constructive feedbacking important in classroom assessment? Give some advantages.
2. As a student, which do you think is better, an oral feedback or written feedback? Justify your
answer.
WHAT IS SELF-ASSESSMENT?
Student self-assessment involves students describing and evaluating the processes and products of
their learning.
Students evaluate the work they have produced and reflect on processes, actions and activities that
contributed to the production of the work.
Student self-assessment can support students to regulate their own learning by requiring them to
monitor how they are working, and encouraging them to modify their approach if something is not
working as well as it could.
Self-assessment can also enhance students’ self-efficacy (a student’s belief that they are able to
succeed at a given task) and support ownership of learning. The capacity for self-regulation is linked
to higher achievement as well as improved motivation and engagement.
Effective learners tend to self-assess the quality of their work more regularly than less effective
learners and are also better able to incorporate evidence from other sources to make judgments on
their learning.
Self-assessment is a key part of Assessment for Learning where reflection during the flow of
learning is used to improve learning and teaching.
2. Explicitly teach students how to self-assess and provide ongoing support. Many students do not
innately know how to self-assess. Teachers need to directly teach students how to self-assess and
provide opportunities for students to practice self-assessment and receive feedback and assistance.
Students, particularly lower-achieving students, tend to need support to reflect on their learning and
often benefit from guidelines or instructions that they can follow. Providing students with feedback on
their progress enhances the impact of self-assessment. Also, providing opportunities for students to
engage in group discussions and evaluation supports the accuracy and impact of self-assessment.
3. Allow students to contribute to creating standards or criteria. When students contribute to the
creation of standards or criteria, they tend to be more engaged and more invested in the self-assessment
process. They also will spend more time, and are more rigorous, in undertaking their own self-
assessment.
4. Create a positive, trusting classroom culture. Students are more likely to accurately selfassess when
they feel their classroom is a safe space and that they will not be judged. Consider keeping self-
assessments private.
6. Consider using a rubric. Research has found greater learning results from self-assessment when
more complex judgments are used. Rubrics are one way of achieving this. However, students tend to
require explicit instruction around how to effectively interpret and use rubrics. Combining criteria and
performance standards with specific examples of the final product, or worked examples, can also
support the impact of rubrics.
2. Over- or under- estimating one’s ability. Most people have a tendency to be unrealistically
optimistic about our own abilities and to believe that we are above average. In general, higher-
performing students tend to be more humble in self-assessment than lower-performing students. And
in certain cultures, giving oneself a positive self-assessment can be considered boastful.
4. Students may not be old enough. Younger students might be less able to accurately self-assess.
Research has found that secondary school and university students appear to be able to judge more
accurately how they have done in formal tests and assessments.
5. Students’ willingness to self-assess. Students sometimes are unwilling to self-assess. This might be
because they feel they lack the necessary skills to accurately judge their work, they are afraid of being
wrong or they prefer an expert to assess their progress and work. Research also suggests that students
often do not understand the benefits of self-assessment or do not see it as a valuable exercise and so
do not put much effort into the self-assessment process.
2. Scripts
Scripts consist of specific questions that are structured into a clear progression of steps, to guide
learners in how best to achieve a task. They explicitly detail the ‘self-talk’ that accompanies working
through a task. A script is useful for students to follow throughout a task, but it can also be used to
analyze the final outcome. A script can help students to assess whether they are on the right track to
completing the task, and supports them to adjust their learning behavior according to the directions of
the scripted questions. You can also ask students to use scripts to analyze their or their peers’ work
after production, e.g. by reviewing whether each step was satisfactorily completed.
3. Journals
A learning journal is a place for students to reflect in writing about how their learning is going,
what they need help with, and the effectiveness of different strategies for learning. Teachers need to
provide regular, short periods of time for writing in the journal, with guiding questions to support self-
assessment, goal setting, and metacognitive knowledge (understanding their thinking). For example,
you might ask students to respond to questions such as these in their journals:
• Think about your work in relation to the success criteria and write in your journals what you
noticed you were able to do well and two things you need to work on next.
• How well are different strategies working? How does what you are doing help you?
• What one thing will you remember long after this lesson and why?
• What question are you still trying to understand and why might this be?
4. Portfolios
A portfolio is a student-managed collection of work which demonstrates the student’s learning
and developing competence. Portfolios help students in self-assessment skills such as understanding
progress through record keeping, documenting interests and strengths, and identifying choices and
preferences. Portfolios can form a shared focus to assessment discussions and goal setting with
teachers. Some ways you could support students to build up their portfolios include:
• Reminding students, every lesson, that they can decide whether to include their work from
this lesson in their portfolio.
• Sharing with students the outcomes and success criteria they are expected to achieve and then
give them the responsibility to determine their own evidence.
• Providing a list of what students need to add to their portfolio each term: one piece of writing,
one mathematics test or activity, one project, one artwork, and so on.
• Asking students to consider what others (such as parents, employers and other educational
institutions) would need for proof of meeting learning goals.
• Providing a range of examples – such as taking photos, videoing themselves undertaking
tasks, screenshots of internet search histories, a progression of drafts and peer feedback
reports. You can ask students “Which work will you choose to show what you know?” or you
can ask them to collect a set of work that shows different stages in development.
Portfolios can be digital, rather than paper-based, which solves issues of storage and the risk
of loss or damage. Students should engage in summarizing and reflecting upon their evidence; for
example, identifying areas of strength or of growth, what they enjoyed or were most proud of, where
they have made progress and why, and what else they could do to improve. You can provide prompts
for self-evaluation and goal setting, in the form of an evaluation sheet. For example:
• Did I ensure that each paragraph had a topic sentence?
• Did I include an introduction, main body and conclusion in my story/report?
• Did I proofread my story/report carefully, e.g. by checking the spelling of difficult words and
making sure I used the correct punctuation?
• Did I avoid repeating the same words?
• What did I like best about my story/report? Why?
• What would I do next time to improve my writing? Students are likely to enjoy amassing a
collection of their work, and will begin to assume increasing responsibility for collecting,
reflecting, and organizing the work.
5. Traffic lights
Traffic lights are a simple system for students to use to indicate their perceived understanding
of particular work. Using a traffic light icon, students label their work green, yellow, or red according
to whether they think they have good (green), partial (yellow), or little (red) understanding. This can
be useful for determining groupings or activities for the next part of the lesson. The teacher might pair
the green students with the yellow students while he or she works with the reds. Or students can select
a follow-up activity matched to their traffic light color.
You can also give students a list of key concepts or topics within the lesson content so far, and
ask them to reflect on where they feel their learning is secure (green), and where they feel they need
to concentrate their efforts (yellow or red). Traffic lights might also be used for peer assessment.
One word of caution: traffic lights provide a very visible way of communicating learning
progress. Students need to feel safe to share their learning progress with the class without feeling
threats to their self-esteem and peer relationships. Teachers might need to work hard to ensure that red
lights are perceived positively (as part of learning and as opportunities for further instruction) rather
than negatively associated with ability.
A similar strategy is ‘coloring squares’. This is a self-assessment technique in which students
color goal statements or sections of a rubric according to the level of confidence they feel in achieving
that goal.
6. Exit cards
An exit card is a small piece of card or a post-it given to each student at the end of a lesson, on
which they write a comment to self-assess what they have learned in the lesson and what they need
more help with. This helps in planning the next lesson. This only takes a few minutes for the student
to complete and is handed to the teacher as the students leave the class.
You can ask students to summarize the lesson or, alternatively, ask a generic question such as
“What are the three most important bits of information you have learned today?” You can then check
how well these responses match the learning objectives for the lesson. Or you might choose to ask a
question related to the learning objective, such as “Why are historians concerned about bias in
historical sources?” You can also ask “What do you think we need to spend more time on because you
haven’t quite understood the topic?” or “What questions do you have?”
Exit cards might be pre-printed, like this:
3
1.
2.
3.
The two parts of the lesson I most enjoyed were:
2
1.
2.
1
1.
7. Paired marking
In paired marking, pairs of students interchange and assess work. This might mean using a
rubric, or applying success criteria to each other’s work. Paired marking gives student opportunities to
practice assessment skills, and opens up dialogue about the purpose of assessment, what information
is useful as feedback from an assessor, and for reflection on success criteria and what they mean. These
discussions can take place between the paired students or with the whole class after a paired marking
session. The objectivity and assessment skills developed in peer assessment can help in improving the
quality of students’ self-assessments. Students might be given scripts and exemplars to support their
peer-assessments or they might be offered formulas for providing feedback, such as ‘two stars and a
wish’ (two positive comments and one suggestion to improve it), or one specific thing that met the
criteria, and one question.
For students that are very confident giving and receiving improvement feedback, consider a
‘Four Faults and Fixes table’:
FAULT FIX
1. 1.
2. 2.
3. 3.
An extension of paired marking is group marking, where students are put into groups of four
and assess each other’s work, giving feedback on a post-it. The post-it notes are collated (see the
template below) and the teacher reviews, moderates and feeds back on the feedback (not the original
work). The following template is titled “Accept or amend?”. Here the student reviews and reflects
upon the feedback given to them and decides whether to accept the comments and make changes to
their work or to amend the comments, and provide reasons.
LEARNING ACTIVITY 3
Do this:
1. Give a classroom situation where self and peer assessment can be implemented.
2. Choose two assessment tools for self and peer assessment and present example in your field of
specialization.
SUMMARY
Here are some pointers to remember:
Qualitative evaluation focuses on text or narrative form. It answers the questions why and how. It
matches with outcomes about application, analysis, synthesis, and evaluate. It seeks to explain and
understand. It has also the ability to capture elusive evidence of student learning and development.
Qualitative methods include three kinds of data collection: interview, direct observation, and
written documents.
Constructive feedback is providing useful comments and suggestions that contribute to a positive
outcome, a better process or improved behaviors. It provides encouragement, support, corrective
measures and direction to the person receiving it. Knowing how to give constructive feedback is a
valuable skill.
Feedback can take many forms such as oral, written, informal, formal, descriptive, evaluative, peer
and self-assessed feedback.
Peer assessment or peer review provides a structured learning process for students to critique and
provide feedback to each other on their work. It helps students develop lifelong skills in assessing
and providing feedback to others, and also equips them with skills to self-assess and improve their
own work.
The tools for peer and self-assessment are: rubrics, scripts, journals, portfolios, traffic lights, exit
cards, paired marking, and dedicated Improvement and Reflection Time (DIRT).
REFERENCES
Chappuis, J. (2012, September). Leadership for Learning: How am I doing? Educational Leadership,
70(1), 36-41.