Module 8.1
Module 8.1
1. Validity
An assessment can only be valid if it measures what it is supposed to measure. The assessment must match
the elements and performance criteria of the unit of competence.
2. Reliability (Consistency)
A good test will be a reliable test. This means that if the same students are given the same test on two
different occasions, (and marked by the same or different markers), they should achieve the same or very
similar scores.
3. Washback
Any assessment piece must have positive washback. This means that the effect of the test on the teaching
must be beneficial. Otherwise, what’s the point of the assessment?
It’s essential to ensure a test is not constructed in a way that candidates can achieve high marks by merely
learning material off-by-heart. This type of test is unlikely to assess genuine language skills and would not,
therefore, provide positive washback.
4. Construct Validity
Construct validity is an element within validity. It relates specifically to the construct or trait being measured.
If your test aims to assess your students’ listening skills, then it must test listening and not speaking, reading,
writing or memory skills.
5. Fairness
Fairness means using the same assessment process for all candidates. Fairness is achieved mainly by:
ensuring assessment methods and procedures are not developed or implemented in such a way that
they exclude or limit individuals due to race, age, gender, disability, employment status, social or
educational background
ensuring candidates understand the assessment process, assessment methods and expected
outcomes
ensuring that candidates can perform at their best, e.g. the examination room is not too hot, too cold,
too noisy, etc.
6. Sufficiency
Sufficiency means that there is sufficient evidence or examples of work to make an assessment decision.
7. Flexibility
Flexibility means using a variety of methods of assessment.
8. Practicability
Practicability means that assessments must be capable of being carried out both efficiently and
costeffectively, and to do this, there needs to be adequate resources and time.
8.5.3. Standardisation
Standardisation is used here to refer to a process that aims to ensure that:
One of the primary aims of standardisation is, where possible, to improve reliability. When valid decisions are
made with a tolerable level of reliability, assessment judgements could be said to have been standardised.
How can assessors’ judgements be standardised (made consistent and reliable)?
Assessors make judgements about the competence of students by reference to written standards. They base
their decisions on a variety of evidence accumulated, e.g. observing the student, questioning them and
judging an activity carried out by the students.
The tasks that each student performs and the other evidence they provide to prove that they are competent
may be different for each assessment judgement.
For assessment to be reliable, each assessor’s judgements must be consistent for various students and tasks
and consistent with the judgements of other assessors.
Achieving standardisation is not easy for inexperienced teacher assessors. The following activities help to
cement this skill:
1. Some assessors – and lead assessors and internal verifiers – tend to believe that standardisation is the
process of ensuring that all assessors follow the same assessment procedure and that such
standardisation ensures that assessors make reliable judgements.
Of course, it is difficult to entirely separate assessment decisions as they are part of the assessment
process. But standardising the assessment process and operationalising the internal verification
system is certainly not the same as standardising consistency in assessment decisions.
2. Assessment teams are sometimes unsure how much disagreement/inconsistency is tolerable. For
example, with assessment decisions on the borderline between competent and not yet competent, the
teams would have to establish an acceptable level of agreement.
3. Consistency of assessment judgements may be affected by the diversity of evidence that students
can present.
1. Select a unit of learning which many of the team assess and ask each to bring along two examples of
their completed assessments.
2. The completed assessments are passed around the team, and each assessor completes an assessment
feedback form as if he/she is assessing.
3. The feedback forms are collected and evaluated by the internal verifier and feedback is given to
individual assessors, confidentially, at a later time.
4. These exercises should be carried out frequently until standardisation is cemented within the assessor
team.
Ensure you don’t just focus on speaking skills. The communicative approach integrates all skills, as we have
stressed several times.
All skills need to be tested.
Make the question and requirements unambiguous and in a language appropriate for the candidates.
Avoid giving clues in the choices/options which help to identify which response is the correct one.
Provide around four choices to reduce guessing. Ensure that the distractors (options which look like
they may be correct, but they aren’t) are close to the correct response so that the candidate will
consider them. That is, make the options realistic.
Keep the choices around the same length; ensure the correct answer is not much longer than the
others, where possible.
Avoid giving grammatical clues. For example, the word an in the stem requires an option that begins
with a vowel; the word is in the stem requires an option written in the singular.
Avoid using textbook language in the correct choice only. Learners can easily spot that this is the
correct answer.
Ensure that the questions include significant learning rather than just a simple recalling of facts and
figures.
Embed the nature of the issue in the stem (the question) of the item, ensuring that the stem is
meaningful in itself.
Ensure that the stem focuses on as much of the item as possible, with no irrelevancies.
Keep the readability levels low.
Ensure that all the options are reasonably plausible, so that guessing of the only possible option is
avoided.
Avoid the possibility of candidates making the correct choice through incorrect reasoning.
They can be completed relatively quickly by a competent question writer, enabling many questions to
be asked which, in turn, enables good coverage of each skill area, thereby increasing reliability and
validity.
There is limited writing, so candidates’ writing skills (or their lack of these) do not impede
demonstration of knowledge or skill.
The opportunities for errors or biases in marking can be reduced.
Multiple-choice testing may demean and reduce the complexity of knowledge, learning and education
to the trivial, atomised (fragmented) and low level.
They have little diagnostic or formative potential.
Scores may be inflated through informed guessing.
However, if you follow our guidelines above, you will construct valid multiple-choice assessments.
Multiple-choice activity
Have a look at these test questions. Is each of them valid and reliable – or not? Please read them and
add Yes or No beside the numbers below and comment on why you think the question is valid and reliable – or
not. Then check what we think, below the questions.
Q1. Who has scored the most goals in one season for Real Madrid, and in which year?
Q2. Which state was the 16th state to join the Union in the USA?
a. A. Tennessee
b. B. Hawaii
c. Vancouver
d. Alaska
A. refers to any drug effect other than the one the doctor wanted the drug to have
B. is the chain effect
C. is an additional benefit
D. is the key effect
Q4. Each organism is made of cells, and every cell comes from another cell. This is the:
A. Relativity Theory
B. Evolution Theory
C. Heat Theory
D. Cell Theory
Q6. A test which may be scored merely by counting the correct responses is called an
_______________ test.
A. consistent
B. objective
C. stable
D. standardised
Q7. What is the effective number of questions in a test to ensure it is a true test of competence?
A. less than 10
B. less than 20
C. more than 40
D. more than 50
A. A only
B. B only
C. C only
D. both A and B
Answers
Well done! We wonder if you came to the same conclusion as we have: they are all unacceptable as test items.
1. Not acceptable. The answers contain 2 pieces of data. However, it is possible to answer this, knowing
only one piece of data. Let’s say the answer is D. If we know that the highest scorer in recent times is
Juanmi, then we can answer the question without knowing this occurred in 2014.
Similarly, if we know the most goals were scored in 2014, then we can pick Juanmi even although we
did not know at all that he was the highest goal-scorer.
2. Not acceptable. Providing they know that Vancouver is in Canada and Alaska and Hawaii were the 49th
and 50th states – the most recent states which many people (around the world) remember – to join
the Union, then they have too good a chance.
3. Not acceptable. The long answer in A gives a clue that this is likely to be the correct answer.
4. Not acceptable. There is a clue to the answer D in the stem of the question – the word cell.
5. Not acceptable. The intended answer C contains wording which is clearly textbook language and
candidates will likely identify this as the answer.
6. Not acceptable. The word an in the stem of the question indicates that the next word must start with a
vowel – thus, the answer is obviously B.
7. Not acceptable. This is heavily flawed. If the answer is meant to be A, then B is also correct as any
number that is less than 10 is also less than 20. Similarly, if the answer is meant to be D, then C is also
correct as any number that is more than 90 is also more than 80.
8. Not acceptable. Bolivia is one of the few countries which has two capitals. This is easy to remember.
Therefore, this can be answered easily by choosing D without knowing the names of the cities.
So, we can see that writing valid and reliable multiple-choice tests requires a lot of thinking.
8.6 Supplementary Materials (150)
Congratulations! You have passed this lesson.
Supplementary materials are worksheets and other materials you can use in addition to the classroom text if
there is one. They include skills development materials, grammar, vocabulary and phonology practice
materials, collections of communicative activities and your own resource materials.
Supplementary materials may also come from authentic sources. Examples of these authentic materials are
newspaper and magazine articles, pictures, and videos.
You can use supplementary materials to:
overcome the lack of materials when you are teaching with minimal resources
replace unsuitable material in the classroom text, if there is one
fill gaps in the classroom text, if there is one
provide appropriate material for learners’ specific needs and interests
give learners extra language or skills practice
add variety to your teaching
Making your supplementary materials may seem daunting at first, but it’s not if you know what to do. Teacher-
made materials can be very effective, assuming that they are relevant and personalised and answer the needs
of the learners in a way no other materials can.
Our key focus in this Section will be to consider two particular situations:
where you are teaching with minimal resources and cannot depend on pre-made
supplementary materials being readily available or appropriate
where you find that the coursebook, if you have one, lacks the types of materials you wish to
have
In these situations, you must resolve this yourself and get or make them.
In the next two Sections, we will focus on worksheets, workcards, and flashcards.
You would typically create these yourself for all kinds of tasks and situations.
These are typically small, laminated cards, about the size of an index card. You would laminate them so
they can be used repeatedly with different learners. However, if there’s no laminator, keep them in
plastic files.
The learners complete their activity on a separate blank sheet or in their notebooks, not on the
workcard.
They are typically for short tasks – individual, pair or group.
Different learners may be working with varying workcards at the same time.
Depending on your choice you can colour them and put little pictures on them.
Recycling activities where, for example, some individuals need more practice with some specific item
Giving out to more able learners who have finished ahead of the others who are still completing the
whole class activity you set
Giving the class a break from learning after they have all been working hard. Quizzes, small puzzles and
riddles can all be entered on these cards for these relaxing moments.
1. These could be used for a discussion between pairs or groups, for a short, written piece, or a student
presentation to the whole class:
2. This next type could be used for relaxation, and perhaps a little prize could be given to the winning
group. You could encourage them to do some dictionary work for any difficult words, where
dictionaries are available. Remember this: Don’t use examples of animals or things which don’t exist in
their culture. We used this with Spanish students.
It is, of course, the Iberian Wolf. Depending on student levels, you could increase or decrease the level of
difficulty.
Some practical ideas for workcards
1. Stamps
Put different postage stamps on a card. Ask the pairs to identify what countries the stamps are from. A word
bank can be supplied. Higher performing students can be asked to add the capital of the country, the name of
the language spoken, etc.
3. Things In Common
Give a list of 3-4 words. The students write what the words have in common. For example, beginners could
get dog, cat, bear: animals. Older learners could be challenged with pint, silver, width. They may take some
time to work out that no other English words rhyme with these words!
4. Matching
You make up two lists, one of countries and the other of capitals. The students match the country with its
capital city.
5. Words That Sound The Same
You present two lists of words that sound the same but have different meanings, e.g. right, write. The
students must match the words that sound alike. This can also be done with opposite words and words that
rhyme.
8.6.2 Flashcards
A flashcard is a laminated picture of, say, a house, garden or kitchen with the name below it or on the back of
the picture that you can hold up for all to see.
Ensure these are culturally appropriate. For example, if all your students come from the inner city,
there may not be many gardens around.
You could make these Letter/A4 size. You could also make smaller versions so that, say, each pair can have
their own to look at. Gradually, through time, you’ll be able to miss out the picture and just show the word.
It’s good to intersperse showing the word with also writing it on the board, to get their minds thinking further
that spoken words can be written down.
It’s a wise idea to colour your flashcards from the start if you can, e.g. nouns in pink, adjectives in blue, verbs
in green, etc. You would typically start with nouns. Of course, you won’t use the metalanguage with them. You
would just call it a thing or an animal.
When you move on to say, adjectives, you’d point out that this card is not in pink like the other card (nouns)
and this will help them a bit to understand categories and differences through time.
With ‘showing’ activities like holding up flashcards, always plan to do some other activity after this to
consolidate the learning. For example, once they have learned some simple words, they can practise in pairs,
reading words to their partners.
Remember! Become familiar with what supplementary materials are available in your school.
When you arrive, draw up a materials needs analysis checklist at the beginning of the course to find out what
you will want to add to the classroom text, if there is one, when you are planning your lessons. Plan ahead!
The coursebook and accompanying materials produced for the global EFL market are too generic.
Often, they are not geared to specific groups of learners.
Often, they are not aimed at any specific educational or cultural context.
Often, they contain functions and speech outputs based on situations that the majority of foreign
language learners may never be in.
They are often Anglo-centric in their construction and production and therefore do not reflect any
local varieties of English. ‘Anglo-centric’ typically means centred on or giving priority to England or
things English.
Thus, many teachers find it much, much better to make their own materials to make them fit the specific
educational and cultural context.
creating or adapting materials to the individual needs and learning preferences of students
creating or modifying materials that take into account the learners’ first language and the learners’
culture and personal experiences
creating or adapting materials at the right level for specific learners, to ensure the materials present an
appropriate challenge and degree of success
changing the often-repetitive model and organisation in the coursebook to add more variety within the
classroom to meet developing needs (e.g. using whatever is appropriate to achieve the goal – topics,
situations, functions, etc.)
4. Lack Of Finance
A large number of schools may not have the budget to supply modern resources for the TEFL classroom.
Instead of moaning and groaning, most teachers step up to the plate and design their own materials as a
matter of course. The school and your learners will be very appreciative of this.
Guidelines for Designing Your Materials
Remember the words of the Cheshire Cat in Alice in Wonderland: Alice: ‘Would you tell me, please, which way
I ought to go from here?’
‘That depends a good deal on where you want to get to,’ said the Cat.
‘I don’t much care where —’ said Alice.
‘Then it doesn’t matter which way you go,’ said the Cat.
If you haven’t designed materials, you must have a definite route to follow; otherwise, things may not turn
out as planned, and you and your learners may miss out on learning opportunities.
Here are the practical points you need to consider when planning the materials, while constructing
your materials and when reviewing your materials after completion.
Time to reflect
What practical points do you think you would need to consider when planning, constructing
and reviewing YOUR materials?
Try and do this without looking at the next Section.
Take some time out to make a drink or sandwich/biscuit and reflect on this.
Then you can return to check your thoughts with what we think below.
Well done!
Here’s what to do:
5. Consider Copyright
Ensure you consider copyright. Yes, you must, not just for your sake but also for the sake of who is employing
you. Unless an artist, writer or producer clearly states that the material can be freely used within your
classroom, then you need to take care.
If you use copyrighted material in your class, which then goes down so well that it’s included in the school
brochure or on the school website, this could cause a problem for the school.
If, say, a writer or producer says you can use their material in class, that’s fine. If they say you need to ask
permission to do so, then you should follow this up and seek approval. If a writer or producer says you cannot
use the material, then steer clear of it.
In general, an idea cannot be copyrighted, so you may see something that sparks your interest, and you can
make up your own material based on that idea. However, the simple lifting of photos and text as they were
initially produced is not generally allowed.
Re pictures and images, numerous sites offer these without any copyright, but at a price,
e.g. https://www.shutterstock.com/
14. Impress
Ensure you make a good impression. Ensure your materials look as good as they can, with consistency in the
layout.
Also, ensure you don’t cause anxiety or confusion for the learner. For example, when designing an information
gap or cloze activity ensure there is adequate room for the learners to write their answer.
Ensure they know whether the hatched lines in the gap represent the number of letters or not. Ensure they
know whether to write or print their answer.
In addition, it’s always wise to consider whether you will be using these materials again. If they are materials
which will be used frequently, get them laminated (where possible) so that they still look fresh when used
again and again.
Realia pieces can help to bring the language classroom to life. A carefully assembled collection of materials,
such as restaurant menus, classified ads, flyers, and travel brochures can be excellent supplements to the
primary resources you use. Collect these before you go. Here are some other examples of realia:
Audio/visual
These pieces of realia can be used as the raw materials or input data for a task, or they can be used to provide
cultural background, to assist in explaining new vocabulary and as a stimulus for a range of learner language
activity.
Good job, as they say.! Another Module completed! What you have learned here will help you better
understand your teacher role and the practical skills and techniques needed to ensure you are
competent and effective at all times.
And there are more skills and techniques to come!
Time for a little break. Then, come back and have a go at Quiz 8.
It’s not taxing. Good luck!
After that, we’ll move on to Module 9, where we will focus indepth on effective classroom
management. It’s vital that you manage your class well, so absorb all the points and put these into
action in your classroom.
End of Module 8 (150)
Congratulations! You have passed this lesson's quiz achieving 93%
There is only one correct answer for each question. Select the correct answer.
Although you will not fail the whole course if you score below 40% in the End Of Module Quiz it will count
towards your final assessment. You will be able to retake this quiz. When you are finished be sure to look
over the questions again and take note of any errors and use this to learn, you may be asked similar questions in
your final assessment.
With regard to commercially made materials, which one of the following statements is
generally untrue?1
They often contain functions and speech outputs based on situations that the majority of
foreign language learners will be in.
They are often not geared to any specific educational or cultural context.
They are often Anglo-centric in their construction and production and therefore do not
reflect any local varieties of English.
Grade: 1
2. Which of the following is unlikely to be a successful body language tip?1
Try to be specific.
You need to remember that good spoken communication relies purely on the words
spoken.
You need to stretch your learners’ knowledge, understanding and application skills.
Grade: 1
5. When an assessment measures what it is supposed to measure, it is:1
fair
reliable
sufficient
valid
Incorrect - Right Answer: valid
6. Always be aware that in the EFL classroom effective communication with an
appropriate response is often more important than grammatical accuracy.1
True
False
Grade: 1
7. Constructive feedback means only giving positive feedback.1
True
False
Grade: 1
8. Which of the following is unlikely to be regarded as summative assessment activity?1
You omit words from a passage and your learners have to choose, from a list, the correct
words to fill the gaps.
You give the learners a sentence and they rewrite it, incorporating a change of expression
but preserving the basic meaning.
You dictate a passage and the learners write down a set of words.
Grade: 1
9. One of the primary aims of standardisation is, where possible, to improve:1
results
reliability
practicability
flexibility
Grade: 1
10. Concept checking:1
can be achieved by asking students if they understand the point you are making
True
False
Grade: 1
12. Which of the following statements about eliciting is invalid?1
organise the students into pairs and then give out the instructions
Grade: 1
14. Feedback:1
is always verbal
is always given to individuals, not groups
Grade: 1
15. When using the board, which one of the following actions would need to be improved?1
Keeping quiet as you write so that they can focus on what you are writing