Assessing

Speaking
 Speaking is a productive skill that
can be directly and empirically
observed, those observations are
invariably colored by the accuracy
and effectiveness of a test-taker’s
listening skill, which necessarily
compromises the reliability and
validity of an oral production test.
Basic Types of Speaking
1. Imitative.
At one end of a continuum of types of speaking
performance is the ability to simply parrot back (imitate)
a word or phrase or possibly a sentence.
- a purely phonetic level of oral production, a number
of prosodic, lexical, and grammatical properties of
language may be included in the criterion performance.
Example:

We are interested only in

what is traditionally labeled
“pronunciation”.
2. Intensive
A second type of speaking frequently
employed in assessment contexts is the
production of short stretches of oral language
designed to demonstrate competence in a
narrow band of grammatical, phrasal, lexical,
or phonological relationships (such as
prosodic elements– intonation, stress, rhythm,
juncture).
Example:
Directed response tasks, reading
aloud, sentence and dialogue
completion; limited picture-cued
tasks including simple sequences;
and translation up to the simple
sentence level.
3. Responsive
It includes the interaction and
test comprehension but at the
somewhat limited level of very short
conversations, standard greetings and
small talk, simple requests and
comments, and the like.
 The stimulus is almost always a spoken
prompt (in order to preserve authenticity),
with perhaps only one or two follow-up
questions or retorts:
A. Mary: Excuse me, do you have the time?
Doug: Yeah, Nine-fifteen.
B. T: What is the most urgent environmental problem
today?
S: I would say massive deforestation.

C. Jeff: Hey, Stef, how’s it going?

Stef: Not bad, and yourself?
Jeff: I’m good.
Stef: Cool. Okay, gotta go.
4. Interactive
The difference between responsive
and interactive speaking is in the length
and complexity of the interaction, which
sometimes includes multiple exchanges
and/or multiple participants.
 Interaction can take the two forms
of transactional language, which has
the purpose of exchanging the
specific information, or interpersonal
exchanges, which have the purpose
of maintaining the social
relationships.
Example:
In the three dialogues cited above, A and B
were transactional, and C was interpersonal.
In interpersonal exchanges, oral production
can become pragmatically complex with the
need to speak in a casual register and use
colloquial language, ellipses, slang, humor, and
other sociolinguistic conventions.
5. Extensive
It include speeches, oral
presentations, and story-telling,
during which the opportunity for oral
interaction from listeners is either
highly limited (perhaps to nonverbal
responses) or ruled out altogether.
Example:
My vacation in the mountains.
A recipe for outstanding pasta
primavera.
Recounting the plot of a novel or
movie.
Micro and Macro of Speaking

The micro-skills refer to producing the smaller

chunks of language such as phonemes, morphemes,
words, collocations, and phrasal units.

The macro-skills imply the speaker’s focus on the

larger elements: fluency, discourse, function, style,
cohesion, nonverbal communication, and strategic
options.
Micro Skills
1. Produce differences among English phonemes and
allophones.
2. Produce chunks of language of different lengths.
3. Produce English stress patterns, words in stressed
and unstressed positions, rhythmic structure and
intonation contours.
4. Produce reduced forms of words and phrases.
5. Use an adequate number of lexical units (words) to
accomplish pragmatic purposes.
6. Produce fluent speech at different rates of
delivery.
7. Monitor one’s own oral production and use
various strategic devices-pauses, fillers, self-
corrections, back tracking- to enhance the clarity of
the message.
8. Use grammatical word classes (nouns, verbs,
etc.), systems (eg. Tense, agreement, pluralization)
word order, patterns, rules, and elliptical forms.
9. Produce speech in natural constituents:
inappropriate phrases, pause groups and
sentence constituents.
10. Express a particular meaning in
different grammatical forms.
11. Use cohesive devices in spoken
discourse.
Macro Skills
12. Appropriately accomplish communicative
functions according to situations, participants, and
goals.
13. Use appropriate styles, registers, implicature,
redundancies, pragmatic conventions, conversation
rules, floor-keeping and yielding, interrupting, and
other sociolinguistic features in face-to-face
conversations.
14. Convey links and connections between events and
communicate such relations as focal and peripheral ideas,
events and feelings, new information and given information,
generalization and exemplification.
15. Convey facial features, kinesics, body language, and other
nonverbal cues along with verbal Lg.
16. Develop and use a battery of speaking strategies, such as
emphasizing key words, rephrasing, providing a context for
interpreting the meaning of words, appealing for help, and
accurately assessing how well your interlocutor is
understanding .
Three Important Issues in designing tasks for assessing spoken language

1. No speaking task is capable of isolating the single skill of oral production.

Concurrent involvement of the additional performance of aural
comprehension, and possibly reading, is usually necessary.
2. Eliciting the specific criterion you have designated for a task can be tricky
because beyond the word level, spoken language offers a number of
productive options to test-takers. Make sure your elicitation prompt
achieves its aims as closely as possible.
3. Because of the above two characteristics of oral production assessment, it
is important to carefully specify scoring procedures for a response so that
ultimately you achieve as high a reliability index as possible.
Designing Assessment Tasks: Imitative Speaking

In a simple repetition task, test takers repeat the stimulus, whether it is a pair
of words, a sentence, or perhaps a question (to test for intonation production)
1. Pronunciation.
2. Suprasegmentals
Test-takers hear: repeat after me.
● beat/bit bat/vat
● I bought a boat yesterday.
● The glow of the candle is growing.
Test-takers repeat the stimulus
Scoring scale for repetition tasks:

2 Acceptable pronunciation
1 Comprehensible, partially, correct pronunciation
0 Silence, seriously, incorrect pronunciation

The longer the stretch of language, the more possibility for

error and therefore the more difficult it becomes to assign a
point system to the text. In such case, it may be impressive to
score only the criterion of the task.
Versant (Formerly called Phone pass)

Among a number of speaking tasks on the test, repetition of sentences

occupies an important role.
It has high construct validity. (not just for a test taker’s phonological
ability but also for discourse and overall production ability
It elicits computer-assisted oral production over a telephone. Test-takers
read aloud, repeat sentences, say words, and answer questions.
● Computerized scoring (by speech-recognition tech.)
- More validation.
- High reliabilities.
- Scoring reported back to the test-taker within minutes
TEST SECTIONS
No. PART ACTIVITY RELATED WITH IMITATIVE SPEAKING

1 A READ ALOUD SELECTED SENTENCES

2 B REPEAT SENTENCES

3 C ANSWER QUESTIONS WITH 2 OR 3 WORDS

4 D HEAR WORDS IN RANDOM ORDER AND CORRECTION

5 E 30 SECONDS TO TALK ABOUT OPINION

Designing Assessment Tasks: Intensive Speaking

At the intensive level, test-takers are prompted to produce

short stretches of discourse through which they demonstrate
linguistic ability at a specific level of language.
It may also be described as limited response tasks
(Madsen, 1983), Mechanical tasks (Underhill, 1987), or what
classroom pedagogy would label controlled responses.
Designing Assessment Task: Intensive Speaking

- Directed Response Tasks

- Read-Aloud tasks
- Sentence/Dialogue Completion Tasks and Oral
Questionnaires
- Picture Cued Tasks
- Translation (of Limited Stretches of Discourse)
Designing Assessment tasks: Intensive Speaking

At the intensive level, test-takers are prompted

to produce short stretches of discourse (no more
than a sentence) through which they demonstrate
linguistic ability at a specified level of language.
Many tasks are “cued” tasks in that they lead the
test-taker into a narrow band of possibilities.
1. Directed Responsive Tasks

Directed response Elicits a particular grammatical

form/transformation of a sentences. Such a tasks are mechanical and not
communicative.
2. Read-Aloud Tasks
This technique is easily administered by
selecting a passage that incorporates test specs
and by recording the test-takers’ output.
Intensive read-aloud tasks include reading
beyond the sentence level up to paragraph or
two.
The scoring scales are:

Pronunciation Points:
0.0—0.4 frequent errors and unintelligible
0.5---1.4 occasionally unintelligible
1.5--- 2.4 some errors but intelligible
2.5---3.0 occasional errors but always intelligible
Fluency

0.0-0.4 slow, hesitant, and unintelligible

0.5-1.4 non-native pauses and flow that interferes
with intelligibility
1.5-2.4 non-native pauses but the flow is
intelligible
2.5-3.0 smooth and effortless
 Underhill suggested some variations on the task
of simply reading a short passage:

a. reading a scripted dialogue

b. reading sentences containing minimal pairs.
Ex. Try not to heat/hit the pan too much.
c. reading information from a table or chart
Sentence/Dialogue Completion Task and Oral
Questionnaires
Another technique for targeting intensive
aspects of language requires test-takers to read
dialogue in which one speaker’s lines have been
omitted. Test-takers are first given time to read
through the dialogue to get its gist and to think
about appropriate lines to fill in.
 Underhill (1987) described another
technique that is useful for controlling
the test-taker’s output: form-filling, or
oral questionnaire. The test-taker see a
questionnaire that asks for certain
categories of information and supplies
the information orally.
Picture-cued Task
A. Picture-cued stimulus requires a description from the test-taker. It may elicit a
word, a phrase, a story or incident.

The types are:

▪ Picture-cued elicitation of minimal pairs

▪ Picture-cued elicitation of comparatives
▪ Picture-cued elicitation of future tense
▪ Picture-cued elicitation of nouns, negative responses, numbers, location
▪ Picture-cued elicitation of responses and description
▪ Picture-cued elicitation of giving directions
▪ Picture-cued elicitation of multiple choice description for two tests takers
Scoring for Intensive tasks

2 comprehensible acceptable target form

1 comprehensible; partially correct target
form
0 silence, or seriously incorrect target form
Translation

Translation is a communicative device in contexts where

English is not a native language. English can be called on to
be interpreted as a second language. Conditions may vary
from an instant translation of a native word, phrase, or
sentence to a translation of longer texts.

Advantages: the control of the output and easily specified

scoring.
Designing Assessment Tasks: Responsive
Speaking

1. Question & Answer

- It can consist of one or two questions
from an interviewer or they can make up a
portion of a whole battery of questions and
prompts in an oral interview.
2. Giving Instructions and Directions
- The technique is simple: the administrator poses the
problem, and test taker responds. Scoring is based
primarily on comprehensibility, and secondary on other
specified grammatical or discourse categories. The
choice of topics needs to be familiar enough so that the
test is not general knowledge but linguistic competence.
Finally, the task should require the test-taker to produce
at least five or six sentences.
3. Paraphrasing
The test-takers read or hear a short story or
description with a limited number of sentences
(perhaps two or five) and produce a paraphrase of
the story. The advantages is the elicit short
stretches of output and perhaps tap into test takers
to practice the conversational art of conciseness by
reducing the output/input ratio.
Paraphrasing a phone message

Test-takers hear:
Please tell Jeff that I’m tied up in traffic so I’m going to
be about a half hour late for the nine o’clock meeting. And
ask him to bring up our question about the employee
benefits plan. If he wants to check in with me on my
cellphone, have him call 415-338-3095. Thanks.

Test-takers respond with two or three sentences.

Test of Spoken English (TSE Test)

The TSE is a 20 minute audio-taped test of oral language ability

within an academic or professional environment.

It is designed to elicit oral production in various discourse

categories rather than in selected phonological, grammatical, or
lexical targets.
The following content specifications for the TSE represent the
discourse and pragmatic contexts assessed in each administration:
1. Describe something physical.
2. Narrate from presented material.
3. Summarize information of the speaker’s own choice.
4. Give directions based on visual materials.
5. Give instructions
6. Give an opinion
7. Support an opinion
8. Compare/Contrast
9. Hypothesize
10. Function “interactively”
11. Define
Using these specifications, Lazaraton and Wagner (1996) examined 15 different specific
tasks in collecting background data from native and non-native speakers of English.

1. Giving a personal description

2. Describing a daily routine
3. Suggesting a gift and supporting one’s choice
4. Recommending a place to visit and supporting one’s choice
5. Giving directions
6. Describing a favorite movie and supporting one’s choice
7. Telling a story from pictures
8. Hypothesizing about future action
9. Hypothesizing about a preventative action
10. Making a telephone call to the dry cleaner
11. Describing an important news event
12. Giving an opinion about animals in the zoo
13. Defining a technical term
14. Describing information in a graph and speculating about its
implication
15. Giving details about a trip schedule

From their findings, the researchers were able to report on the

validity of the tasks, especially the match between the intended task
functions and the actual output of both native and non-native
speakers.
Designing Assessment Tasks:
Interactive Speaking

It is more interpersonal described and

more transactional focused speech event.
INTERVIEW
Oral production assessment that is words. This effective interview
contains a number of mandatory stages (Michael Canale, 1984).

The steps are:

1. Warm up
2. Level check
3. Probe
4. Wind-down
1. Warm Up

a. The interviewer direct mutual introductions, helps

the test-taker become comfortable with the situation,
appraises the test-taker of the format and allays
anxieties.
b. No scoring takes place

.
2. Level Check

a. The interviewer stimulates the test-taker to respond using expected

or predicted forms and functions.
b. Question are design to elicit grammatical categories, discourse
structure, vocabulary usage and/or sociolinguistic factors.
c. This stage give the interviewer a picture of the test-taker’s
extroversion. Readiness to speak and confidence.
d. Linguistic target criteria are scored in this phase
3. Probe

a. Challenge test-takers to go to the heights of their ability, to extend beyond

the limits of the interviewer’s expectation through increasingly difficult
questions.
b. Through probe items, the interviewer discovers the ceiling or limitation of
the test-taker proficiency.
c. At the lower level of proficiency: It may simply demand higher range of
vocabulary and grammar.
d. At the higher level of proficiency: It will typically ask the test taker to give
an opinion or a value judgment, to discuss his/her field of specialization, to
recount a narrative, or to respond to questions that are wondered in complex
form.
4. Wind-down

a. The interviewer encourages the test-taker

to relax with some easy questions.

b. This part is not scored.

Role Play
A popular pedagogical activity in communicative language-
teaching classes.
As an assessment device, role play opens some windows of
opportunity for test-takers to use discourse that might otherwise be
difficult to elicit.

Role play can be controlled or “guided” by the interviewer, this

technique takes test-takers beyond simple intensive and responsive
levels to a level of creativity and complexity that approaches real-
world pragmatics.
Discussions and Conversations

As formal assessment devices, discussion and conversation with and

among students are difficult to specify and even more difficult to score.

Discussion may be especially appropriate tasks through which elicit and

observe such abilities:
- Topic nomination, maintenance, and termination
- Attention getting, interrupting, floor holding, control
- Clarifying, questioning, paraphrasing
- Comprehension signals (nodding, “uh, huh”, “hmm”)
- Negotiating meaning
- Intonation patterns for pragmatic effect
- Kinesics, eye contact, proxemics, body
language
- Politeness, and other sociolinguistics factors
Games
Among informal assessment devices are a variety of games that
directly involve language production. Consider the following types:

1. “Tinkertoy” game: A Tinkertoy (or Lego block) structure is built

behind a screen. One or two are allowed to view the structure.
2. Crossword puzzles are created in which the names of all members
of a class are clued by obscure information about them.
3. Information gap grids are created such that class members
must conduct mini-interviews of other classmates to fill in
boxes, e.g, “born in July”, “plays the violin” etc.
4. City maps are distributed to class members. Predetermined
map directions are given to one student who, with a city map
in front of him or her, describes the route to a partner, who
must then trace the route and get to the correct final destination
ORAL PROFICIENCY INTERVIEW (OPI)

Originally known as Foreign Service Institute (FSI) test, the OPI

is the result of a historical progression of revisions under the
auspices of several agencies, including the Educational Testing
Service and the American Council on Teaching Foreign Languages
(ACTFL).
Specifications for the OPI approximate those delineated above
under the discussion of oral interviews in general. It is carefully
designed to elicit pronunciation, fluency and integrative ability,
sociolinguistic and cultural knowledge, grammar and vocabulary.
Designing Assessment Task: Extensive Speaking
Extensive speaking tasks involve complex, relatively lengthy
stretches of discourse. They are frequently variations on monologues,
usually with minimal verbal inter-action.

Oral Presentation
A checklist or grid is a common means of scoring or evaluation.
Holistic scores are tempting to use for their apparent practicality, but
they may obscure the variability of performance across several
subcategories, especially the two major components of content and
delivery.
Picture-cued Story Telling
One of the most common techniques for
eliciting oral production is through visual pictures,
photographs, diagrams, and charts. We have
already looked at this elicitation device for
intensive tasks, but at this level we consider a
picture or a series of pictures as a stimulus for a
longer story or description.
Retelling a Story, News Event
In this type of task, test-takers hear or read a story or news event that they
are asked to retell.

Translation (of Extended Prose)

Translation of words, phrases, or short sentences was mentioned under the
category of intensive speaking. Here, longer texts are presented for the test-
taker to read in the native language and then translate into English.
The advantage of translation is in the control of content, vocabulary, and to
some extent, the grammatical and discourse features.
The disadvantage is that translation of longer texts is a highly specialized
skill for which some individuals obtain post-baccalaureate degrees.
 THANK YOU
AND
GOD BLESS YOU
ALL!!!