Assessing Speaking

Challenges of the testing speaking:

1- The interaction of speaking and listening
2- Elicitation techniques
3- Scoring
BASIC TYPES OF SPEAKING
1.Imitative: (parrot back) Testing the ability to imitate a word, phrase,
sentence. Pronunciation is tested. Examples: Word, phrase, sentence
repetition
2. Intensive: The purpose is producing short stretches of oral language. It
is designed to demonstrate competence in a narrow band of grammatical,
phrasal, lexical, phonological relationships (stress / rhythm / intonation)

3.Responsive: (interacting with the interlocutor) include interaction and

test comprehension but somewhat limited level of very short conversations,

standards greetings, small talk, simple requests and comments, and the like.
4. Interactive: Difference between responsive and interactive speaking
is length and complexity of interaction, which includes multiple exchanges
/or multiple participant.
5. Extensive (monologue) : Extensive oral production tasks include
speeches, oral presentations, story-telling, during which the opportunity for
oral interaction from listeners is either highly limited (perhaps to nonverbal
responses) or ruled out together.

Micro- and Macroskills of Speaking

microskills of speaking refer to producing small chunks of language
such as phonemes, morphemes, words and phrasal units. The
macroskills include the speakers' focus on the larger elements such
as fluency, discourse, function, style cohesion, nonverbal
communication and strategic options.
Macroskills
1.Apropriately accomplish communicative functions according to
situations, participants,and goals.
2.Use appropriate styles, registers, implicative, redundancies,
pragmatic conventions, conversation rules, floor-keeping and
yielding, interrupting, and other sociolinguistic features in face-toface conversations.
3.Convey links and connections between events and communicative
such relations as focal and peripheral ideas, events and feelings,
new information and given information, generalization and
exemplification.
4.Convey facial features, body language, and other nonverbal cues
along with verbal language.
5.Develop and use a battery of speaking strategies, such as
emphasizing key words, rephrasing, providing a context for
interpreting the meaning of words, appealing for help, and
accurately assessing how well your interlocutor is understanding

Microskills:
1.Produce differences among English phonemes and allophonic
variants.
2.Produce chunks of language of different lengths.
3.Produce English stress patterns, words in stressed and unstressed
positions, rhytmic structure, and intonation contours.
4.Produce reduced forms of words and phrases.
5.Use adequate number of lexical units(words) to accomplish
pragmatic purposes
6.Produce fluent speech at different rates of delivery.
7.Monitor ones own oral production and use various devices-pauses,
fillers, self-corrections, backtracking- to enhance the clarity of the
message.
8.Use grammatical word classes (nouns,verbs,etc.),systems (tense,
agreement, pluralization), word order, patterns, rules, and elliptical
forms.
9.Produce speech in natural constituents: in appropriate phrases,
pause groups,breath groups, and sentence constituents.
10.Express a particular meaning in different grammatical forms.
11.Use cohesive devices in spoken discourse.

Three important issues as you set out to design tasks;

1.No speaking task is capable of isolating the single skills of

oral production. Concurrent involvement of the additional
performance of aural comprehension, and possibly reading, is
usually necessary.

2.Eliciting the specific criterion you have designated for

a task can be tricky because beyond the word level, spoken

language offers a number of productive options to test-takers. Make
sure your elicitation prompt achieves its aims as closely as possible.

3.It is important

to carefully specify scoring procedures for

a response so that ultimately you achieve as high a reliability index
as possible.
interaction between speaking and listening or reading is
unavoidable.
Interaction effect:impossibility of testing speaking in isolation
Elicitation techniques:to elicit specific criterion we expect from
test takers.
Scoring:to achieve reliability

Designing Assessment Tasks: Imitative Speaking

paying more attention to pronunciation, especially suprasegmentals,
in attemptto help learners be more comprehensible.
Repetition tasks are not allowed to occupy a dominant role in an
overall oral production assessment, and as long as avoid a negative
washback effect.
In a simple repetition task, test-takers repeat the stimulus, whether it
is a pair of words, a sentence, or perhaps a question ( to test for
intonation production.)
Word repetition task:
Scoring specifications must be to avoid reliability breakdowns. A
common form of scoring simply indicates 2 or 3 point system for
each response
Scoring scale for repetition tasks:
2 acceptable pronunciation
1 comprehensible, partially correct pronunciation
0 silence, seriously incorrect pronunciation
The longer the stretch of language, the more possibility for error and
therefore the more difficult it becomes to assign a point system to

PHONEPASS TEST
The phonepass test has supported the construct validity of its repetition
tasks not just for discourse and overall oral production ability.
The PhonePass tests elicits computer-assisted oral production over a
telephone.
Test-takers read aloud, repeat sentences, say words, and answer questions.
Test-takers are directed to telephone a designated number and listen for
directions.
The test has five sections.
Part A Testee read aloud selected sentences forum among printed on the test
sheet.
Part B Testee repeat sentences dictated over the phone.
Part C Testee answer questions with a single word or a short phrase of 2 or 3
words.
Part D Testee hear 3 word groups in random order and link them in correctly
ordered sentence
Part E Testee have 30 seconds to talk about their opinion about some topic
that is dictated over phone.
Scores are calculated by a computerized scoring template and reported back
to the test-taker within minutes.
Pronunciation, reading fluency, repeat accuracy and fluency, listening
vocabulary are the sub-skills scored
The scoring procedure has been validated against human scoring with
extraordinary high reliabilities and correlation statistics.

Designing Assessment Tasks: Intensive Speaking

test-takers are prompted to produce short stretches of discourse (no more
then a sentence) through which they demonstrate linguistic ability at a
specified level lang
Intensive tasks may also be described aslimitedresponse tasks,
ormechanicaltasks, or what classroom pedagogy would label
ascontrolledresponses.
Directed Response Tasks
Administrator elicits a particular grammatical form or a transformation of a
sentence.
Such tasks are clearlymechanical and not communicative(possible
drawbacks),but they do requireminimal processing of meaning in order
to produce the correct grammatical output.(practical advantages
Read Aloud Tasks (to improve pronunciation and fluency)
include beyond sentence level up to a paragraph or two. It is easily
administered by selecting a passage that incorporates test specs and bye
recording testee output;the scoring is easy because all of the testtakerss oral production is controlled.
If reading aloud shows certain practical adavantages (predictable output,
practicality, reliability in scoring), there are several drawbacks
Reading aloud is somewhatinauthenticin that we seldom read anything
aloud to someone else in the real world, with exception of a parent reading to
a child.

Sentence / Dialogue Completion Tasks and Oral

Questionnaries
( to produce omitted lines, words in a dialogue appropiriately)
Test-takers read dialogue in which one speakers lines have been
omitted. Test-takers are first given time to read through the dialogue
to get its gist and to think about appropriate lines to fill in.
An advantage of this technique lies in its moderate control of the
output of the test-taker (practical advantage).
One disadvantage of this technique is itsreliance on literacyand
anability to transfer easily from written to spoken English.
(possible drawback)
Another disadvantage is contrived, inauthentic nature of this task.
(drawback.)
Picture Cued Tasks (to elicit oral production by using
pictures)
One of more popular ways to elicit oral language performance at
both intensive and extensive levels is a picture-cued stimulus that
requires a destcription from the test-taker.
Assessment of oral production may be stimulated through a more
elaborate picture. (practical advantages)
Maps are anothervisual stimulusthat can be used to assess the
language forms needed to give directions and specify locations.

Scoring may be problematic depending on the expected

performance.
Scoring scale for intensive tasks
2 comprehensible; acceptable target form
1 comprehensible; partially correct target form
0 silence, or seriously incorrect target form
Translation (of Limited Stretches of Discourse) (To translate
from target language to native language)
The test-takers are given a native language word, phrase, or
sentence and are asked to translate it.
As an assessment procedure, the advantages of translation lie in its
control of the output of the test-taker, which of course means
thatscoring is more easily specified.

Designing Assessment Tasks: Response Speaking

Assessment involves brief interactions with an interlocutor, differing
from intensive tasks in the increased creativity given to the test-taker
and from interactive tasks by the somewhat limited length of
utterances.
Question and Answer
Question and answer tasks can consist of one or two questions from
an interviewer, or they can make up a portion of a whole battery of
questions and prompts in an oral interview.
The first question is intensive in its purpose; it is adisplay
questionintended to elicit a predetermined correct response.
Questions at the responsive level tend to be genuinereferential
questionsin which the test-taker is given more opportunity to
produce meaningful language in response.
Test-takers respond with a few sentences at most.
Test-takers respond with questions.
A potentially tricky form of oral production assessment involves more
than one test-taker with an interviewer. With two students in an
interview contxt, both test-takers can ask questions of each other.

Giving Instruction and Directions

The technique is simple : the administrator poses the problem, and
the test-taker responds. Scoring is based primarily on
comprehensibility and secondarily on other specified grammatical or
discourse categories.
Eliciting instructions or directions
Paraphrasing
read or hear a number of sentences and produce a paraphrase of the
sentence.
Advantages they elicit short stretches of output and perhaps tap into
testee ability to practice conversation by reducing the output/input
ratio.
If you use short paraphrasing tasks as an assessment procedure, its
important to pinpoint objective of task clearly. In this case, the
integration of listening and speaking is probably more at stake than
simple oral production alone.
TEST OF SPOKEN ENGLISH (TSE)
The TSE is a 20 minute audio-taped test of oral language ability
within an academic or Professional environment.
The scores are also used for selecting and certifying health
professionals such as physicians, nurses, pharmacists, physical
therapists,
and
The tasks on
theveterinaries.
TSE are designed to elicit oral production in various
discourse categories rather than in selected phonological,

Designing Assessment Tasks: Interactive Speaking

Tasks include long interactive discourse ( interview, role plays, discussions,
games).
nterview
A test administrator and a test-taker sit down in a direct face-to-face
Exchange and proceed through a protocol of questions and directives. The
interview is then scored on accuracy in pronunciation and/or grammar,
vocabulary usage, fluency, pragmatic appropriateness, task
accomplishment, and even comprehension.
Placement interviews, designed to get a quick spoken sample from a student
to verify placement into a course,
Four stages:

1.Warm-up: (small talk) interviewer directs matual introductions, helps

testee become comfortable, apprises testee, anxieties.(No scoring)

2.Level check: interviewer stimulates testee to respond using
expected - predicted forms and functions. This stage give interviewer a
picture of testees extroversion, readiness to speak, confidence.Linguistic
target criteria are scored in this phase.
3.Probe:Probe questions and prompts challenge testee to go heights of
their ability, to extend beyond limits of interviewers expectation through
difficult questions.
4.Wind-down:This phase is a short period of time during which
interviewer encourages testee to relax with easy questions, sets testees

The scussess of an oral interview will

depend on;
*clearly specifying administrative procedures of
the assessment(practicality)
*focusing the q and probes on the purpose of the
assessment(validity)
*appropriately eliciting an optimal amount and
quality of oral production from the test-taker.
(biased for best performance)
*creating a consistent, workable scoring system
(reliability).

Role Play
Role playing is a popular pedagogical activity in communicative
language teaching classes.
Within constraints set forth by guidelines, it frees students to be
somewhat creative in their linguistic output.
While role play can be controlled or guided by the interviewer, this
technique takes test-takers beyond simple intensive and responsive
levels to a level of creativity and complexity that approaches realworld pragmatics.
Scoring presents the usual issues in any task that elicits somewhat
unpredictable responses from test-takers.
Discussions and Conversations
As formal assessment devices, discussions and conversations with
and among students are difficult to specify and even more difficult to
score.
But as informal techniques to assess learners, they offer a level of
authenticity and spontaneity that other assessment techniques may
not provide.
Assessing the performance of participants through score or checklists
should be carefully designed to suit the objectives of the observed
discussion.
Discussion is a integrative task, and so it is also advisable to give
some cognizance to comprehension performance in evaluating

Games
Among informal assessment devices are a variety of games that
directly involve language production.
Assessment games:
1.Tinkertoy game (Logo block)
2.Crossword puzzles
3.Information gap grids
4.City maps
ORAL PROFICIENCY INTERVIEW (OPI)
The best-known oral interview format is the Oral Proficinecy
Interview.
OPI is the result of historical progression of revisions under the
auspices of several agencies, including the Educational Testing
Service and American Council on Teaching Foreign Language
(ACTFL).
The OPI is carefully designed to elicit pronunciation, fluency and
integrative ability, sociolinguistic and cultural knowledge, grammar,
and vocabulary.
Performance is judged by the examiner to be at one of ten possible
levels on the ACTFL-designated proficiency guidelines for speaking:
Superior; Advanced-high, mid, low; Intermediate-high, mid,low;
Novice-high, mid,low.

Designing Assessments : Extensive Speaking

involves complex, relatively lengthy stretches of discourse.
They are variations on monologues, with minimal verbal interaction.
Oral Presentations
it would not be uncommon to be called on to present a report, a paper, a
marketing plan, a sales idea, a design of new product, or a method.

Once again the rules for effective assessment must be invoked:

a- specify the criterion,
b-set appropriate tasks,
c- elicit optimal output,
d-establish practical, reliable scoring procedures.
Scoring is the key assessment challenge.
Picture Cued Story-Telling
techniques for eliciting oral production is through visual pictures,
photographs, diagrams, and charts.

consider a picture or series of pictures as a stimulus for a longer or

description.
Criteria for scoring need to be clear about what it is you are hoping to

Retelling a Story, News Event

In this type of task, test-takers hear or read a story or news event
that they are asked to retell.
The objectives in assigning such a task vary from listening
comprehension of the original to production a number of oral
discourse features (communicating sequences and relationships of
events, stress and emphasis patterns, expression in the case of a
dramatic story), fluency, and interaction with the hearer.
Scoring should meet the intended criteria
Translation (of Extended Prose)
Longer texts are presented for test-taker to read in NL and then
translate into English (dialogues, directions for assembly of a
product, synopsis of a story or play or movie, directions on how to
find something on map, and other genres).
The advantage of translation is in the control of the content,
vocabulary, and to some extent, the grammatical and discourse
features.
The disadvantage is that translation of longer text is a highly
specialized skill for which some individuals obtain postbaccalaureate.
Criteria for scoring should take into account not only purpose in
stimulating a translation but possibility of errors that are unrelated to
oral production ability