1

TESTING AND EVALUATION_3

Subjective testing techniques – communicative testing

Answer the following questions:

1. Here are the characteristics of methods of scoring tests. Classify each one as objective (O) or
subjective (S).

a. each answer is either right or wrong

b. scoring is a matter of judgement
c. scoring can be carried out mechanically
d. scoring may vary from person to person
e. scoring may vary from time to time
f. scoring may vary from place to place
g. scoring has good reliability

2. Which type of testing (O) or (S) is generally more suitable for each of the following types of test?

a. tests of grammar
b. tests of vocabulary
c. tests of reading
d. tests of writing
e. tests of listening
f. tests of speaking

3. Modern testing (third generation testing) is usually described as employing two different types of
techniques:
i) integrative + objective
ii) integrative + improved subjective
Which of the following test items are examples of i) and which are examples of ii)

a. dictation
b. cloze procedure
c. role play
d. oral interview
e. letter writing
f. using the telephone
g. group discussion to solve a problem

4. Although much modern testing is loosely called "communicative testing," not all techniques can
approximate to real language use in the real world. Mark each of these items C (= communicative,
i.e. approximating the real-world language use) or NC (= non-communicative) for non-ESP
students.

a. dictation
b. cloze
 2

c. role play
d. oral interview
e. letter writing
f. using the telephone
g. group discussion to solve a problem

5. So far communicative meant "approximating to real life language use." But what is
communicative to one learner may not be communicative to another learner. Mark each of these test
items C or NC.

a. dictation (for a general learner)

b. dictation (for a secretarial student)
a. reading aloud (for a general learner)
b. reading aloud ( for an airport announcer)
a. translating a text (for a general learner)
b. translating a text ( for a trainee interpreter)
a. writing 200 words about a holiday
b. writing a postcard to a friend about a holiday
a. writing about your country (for a general learner)
b. writing about your country (for a travel guide)
a. correcting mistakes in a letter (for a general learner)
b. correcting mistakes in a letter (for a secretarial student)

From your answers in 5. it is possible to identify another important element in the definition of
communicative. What do all the b. examples contain and all the a. examples lack (one word).
_____________

With English learners increasingly demanding to be taught the spoken form of the language,
traditional testing became irrelevant. So in the mid-60s different attempts to devise communicative
tests were made (e.g. the tests of ARELS – Association of Recognised English Language Schools –
or Wilkins's concept of language functions.

Some examples of communicative tests

1. The PLAB test

The Professional and Linguistic Assessment Board's test was devised in the mid-70s as a way of
assessing the proficiency of doctors from non-English speaking countries. Strongly influenced by
Wilkins but ignoring his insight that the tests involved would inevitably be largely tests of
integrated rather than isolated skills, the devisers of the PLAB test merely substituted language
functions for grammatical or lexical items, testing them in a discrete-point way.

E.G. Here are four test items from the listening component of the PLAB test. As a way of
examining how it was constructed, match each (1-4) to the appropriate rubric (A-D). First solve 1-
4 :) :) :)

1. 'Show me where it hurts.' 3. 'When did you cut your hand?'

 3

a) seeking information a) your hand and not your foot

b) expressing surprise b) what time did it happen?
c) making a statement c) when did you cut it rather than bruise it?
d) making a suggestion d) I know when your mother cut hers

2. 'I'm just going to pop in a stitch or two.' 4. 'I was in the kitchen getting lunch and I cut
a) serious my finger.'
b) authoritative a) you've bruised it, have you?
c) excited b) was this last night?
d) reassuring c) where were you at the time?
d) with a knife, was it?

A. You will hear 20 recorded sentences. Choose the correct meaning given by the
placing of stress.
B. You will hear 20 recorded sentences. Choose the most suitable response.
C. You will hear 20 recorded sentences. Decide what function the speaker is trying to
perform.
D. You will hear 20 recorded sentences. Choose which of the four adjectives best
describes the attitude or mood conveyed by the speaker's interaction.

2. Palmer's COMTEST

It was evident that the PLAB test was still a discrete-point test and that it did not succeed in
evaluating the integrated skills required in communication. An earlier attempt to devise a test which
was both oral (the demand of the ARELS teachers) and integrative (Wilkins's demand) was Adrian
Palmer's COMTEST (1972). This was perhaps the first test to incorporate what is now seen as one
of the fundamentals of communicative teaching and testing – the 'information gap' – that is one
person has to communicate 'new' or unknown information to the other. In the COMTEST the
learner select one picture from a series of similar but slightly different pictures (at least 4 pics).

.........
The assessor has all of the pictures but does not know which one the learner has chosen. The learner
then describes his/her picture to the assessor, and when the latter is satisfied that (s)he/ can identify
the learner's picture, (s)he points to it. The interesting part was the way that it was scored: in order
to achieve an objective score of communicative effectiveness, he times each learner – the shorter
the time taken to identify the picture, the higher the score.

Palmer's COMTEST was an interesting and innovative experiment that established many of the
principles of oral communicative testing. In certain aspects however, it was not successful. Here is a
list of some of the features of the test. Tick those you think were successful and have been adopted
in subsequent communicative tests; cross those which you think were unsuccessful and (if you can)
indicate why.
 4

1. oral communication is essential

2. communication is a two-way process
3. there should be an 'information gap'
4. conveying one's message is the most important feature of communication
5. linguistic inaccuracy is unimportant if it does not interfere with the message
6. the meaning and message are negotiated between speaker and listener
7. repair strategies ( repetitions, clarifications, corrections, etc.) are signs of this
negotiation, not evidence of linguistic inadequacy
8. the oral interaction should have a purpose or outcome
9. the speaker should be able to use any appropriate language structures and not be
restricted to particular ones
10. the assessment should be objective
11. time is a good measure of communicative efficiency
12 learners should be given smth. to talk about during the oral discussion

Unsuccessful features Reason

______ ________________________________________
______ ________________________________________

3. The RSA CUEFL test

The first real attempt to produce what we now recognise as a communicative test of wide scope and
application grew out of the Morrow report commissioned by the Royal Society of Arts in 1977 as a
direct result of Wilkins's challenge.

Morrow begins his report by surveying the approaches of the first two generations of testing,
rejecting them as 'uncommunicative.' What he means by communicative becomes evident when
examining his criticisms of former practitioners (Lado and Oller).

Look at these criticisms of Lado and Oller, and work out the kind of communicative test that
Morrow is advocating.

uncommunicative tests communicative tests

1. Lado's discrete-point approach depends on Communicative tests will be integrative
the assumption that knowledge of the rather than discrete-point; they will test
elements of a lg. is equivalent to knowledge knowledge of the lg. rather than
of the lg. knowledge of the elements of the lg.
2. Lado, as a behaviourist, believed that
learning a lg. consisted of learning to make
responses. Each response was either right or
wrong, so that in lg. tests each answer is
either right or wrong.
3. In Lado's objective tests, select alternatives
rather than produce lg., so that only their
ability to recognize appropriate lg. forms is
tested.
4. In Lado's tests, the learner's responses are
based on the examiner's lg.; the learner has
 5

no opportunity to use his/her lg.

5. Oller's alternatives to Lado's discrete-point
tests (cloze and dictation) offer the learner
no opportunity for spontaneous production
of lg.
6. Oller's tests offer no possibility for oral
production.
7. Both Lado's and Oller's are tests of
underlying ability (competence) rather than
actual performance.

On the basis of the Morrow report, the RSA developed a series of tests known as the
Communicative Use of English as a Foreign Language (CUEFL). These tests were to become the
prototypes of communicative tests as we know them today and they attempt to incorporate ideas,
especially ideas on methodology, derived from lg. teachers; views of lg. as communication. As
such, they reflect lg. teachers' objections to earlier generations and techniques of testing:

objection test construction scoring

teaching integrative subjective
(Morrow, 1977)

Morow (1982) defines communicative tests as those which test the learner's ability "to translate the
competence (or lack of it) ... into actual performance 'in ordinary situations,' i.e. actually using the
lg. to read, write, speak or listen in ways and contexts which correspond to real life." Three aspects
of this definition are worse examining in more detail – Morrow's concept of 'situation,' his
insistence on using the lg. in all four skills, and his idea of 'corresponding to real life.'

Marrow says that lg. should be tested in ways which sample or simulate 'ordinary situations' or
'communicative situations.' He lists seven features of lg. in these situations which, he argues, should
be measured in a communicative test – all of them features which are absent from traditional tests.

These features are listed below(1-7). Underneath are seven definitions (A-G). Match each feature
to its definition.
1. interaction-based
2. unpredictability
3. context
4. purpose
5. performance
6. authenticity
7. behaviour-based

A. One feature of lg. should be noted, namely that with rare exceptions it is not
simplified to take account of the linguistic abilities of the addressee. An
important feature of communicative ability is precisely the capacity to come to
terms with what is unknown.
B. The success or failure of an interaction is judged by it participants on the basis of
outcomes. Thus an interaction in which we invite smb. to dinner at 8 o'clock on
 6

Saturday will be judged a success if the person in fact arrives at 8 o'clock on

Saturday feeling hungry. Strictly speaking no other criteria are relevant, and in a
test of communicative ability none should be applied.
C. The lg. forms used on any occasion will vary in accordance with features of role,
status and formality. These variations can be subsumed under the heading of
appropriateness. A communicative test should concern itself with the extent to
which and the ways in which lg. is appropriate.
D. Even tasks such as letter writing which may seem to be solitary activities involve
an addressee, whose expectations will be taken into account by the writer. These
expectations will often affect both the content of the message and the way in
which it is expressed.
E. Linguistic investigation has largely been concerned with competence, leaving out
of account 'such grammatically irrelevant conditions as memory limitation,
distractions, shifts of attention and interest, and errors' (Chomsky, 1965). Such
conditions may or may not be 'grammatically relevant' but they certainly exist
and in a lg. use situation a learner must formulate strategies for dealing with
them.
F. A rather obvious feature of communication is that every utterance is made for a
reason. Clearly a communicative test should measure the ability of the candidate
to recognise and use these functions of utterances.
G If participants could foresee how the interaction would develop, and what its
outcome would be, there would after all be rather little point in holding it. The lg.
user needs to process the data he receives in an extremely short time in order to
formulate an appropriate response. Time pressure in processing lg. and
formulating an appropriate response on the basis of it is thus likely to be an
extremely relevant feature of lg. use.

For further info and practical solutions for testing the four skills (L, S, R, W)::

John Brian Heaton, Writing English Language Tests, Longman, 1988

Andrew Harrison, A Language Testing Handbook, Macmillan, 1983
John Brian Heaton, Language Testing, Macmillan, 1982