The Design of Discrete Vocabulary Tests

Discrete tests most commonly focus on vocabulary knowledge: whether the

test takers know the meaning or use of a selected set of content words in the target
language. They may also assess particular strategies o vocabulary learning or
vocabulary use. Such tests are to be distinguished from broader measures of
vocabulary ability that are embedded in the assessment of learner's performance of
language use task. Vocabulary tests may be used discretely to judge the effectiveness
of a vocabulary learning programme, or as embedded measures of a language
education programme with broader objectives. Use of tests or this purpose has some
of the characteristics of research, in that the study should be well designed to yield
reliable and valid conclusion.

One of important foundations for test validity is the defining of the construct
we are intending to measure. There are two approaches to construct definition:
syllabus-based and theory-based. A syllabus based definition is appropriate when
vocabulary assessment takes place within a course o study, so that the lexical items
and the vocabulary skills to be assessed can be specified in relation to the learning
objective of the course. The test covers the vocabulary learning that the students are
supposed to have achieved during the course and provided feedback on areas of
weakness. For research purposes and in proficiency testing, the definition of the
construct needs to be based on theory. Issues such as what we mean by a 'word',
whether vocabulary includes multi-word lexical items, what it means to 'know' a
word and how word are influenced by context all arise when we seek to define what
any particular vocabulary test is measuring.

The design of test task in the next step in test development. In the design, test
makers needs to carefully consider about the characteristics of the test input, one if
which is selection of target words that is how to choose the target words. First, it is
necessary to make a general point about the frequency distribution of words. It is a
well established fact that a small proportion of the total stock of lexical items in the
language occurs very frequently. For teaching and learning purposes, a broad three
way division can be made into high frequency, low frequency, and specialised
vocabulary. The high frequency category in English consists of 2000 word families,
which form the foundation of the vocabulary knowledge that all proficient users of
the language must have acquired. These items figure prominently in general word
lists compiled for use in language teaching, because teachers and learners at the
beginning to intermediate levels can be confident that time devoted to developing a
good knowledge of these words is well spent.

Low frequency vocabulary as a whole is of much less value to learners. A

large proportion of the items are ones that hardly anyone knows or uses, especially
those words given labels like 'rare', 'obsolete', or 'dialectal' in the dictionary. The
selection of specific low frequency words for testing purpose depends in
communicative values for the learners for whom the test is designed, in relation to
their shared educational, occupational, or personal need. This is where specific
vocabulary comes in. It comprises technical terms and other lexical items that occur
relatively frequently in particular registers of the language. It is of relevance in
language courses for specific purposes and in language across the curriculum
programmes. Specialised vocabulary is likely to be better acquired through content
instruction by a subject teacher than through language teaching. Correspondingly, it
may be more validly assessed in subject area examinations than in general language
tests.

Within the categories, there is no standard approach to the choice of target

words for vocabulary testing. For some specific kind of test, a random selection
procedure from a word list is appropriate. For most test, the test writer must exercise
judgment in choosing the lexical items, having regarded to the teaching/learning
objective and the purpose of the assessment. In classroom test, the teachers normally
make a selection from the words that the learners have recently been studying.
Coursebooks that include word lists for each unit simplify the selection task for the
teacher and at the same time specify for the learners the items that they need to learn.
In some major achievement tests, published vocabulary lists are available that specify
not just the words to be tested directly but also those that are likely to be included in
the reading and listening texts. In tests of the vocabulary size of second language
learners, word-list frequency play a significant role. The general assumption here is
that the more frequently a word occurs in the language, the more likely it is that
learners will know it. In achievement and proficiency tests where vocabulary is to be
tested in context, the starting point for selection is likely to be a particular text rather
than a list of words. The choice of text is influenced by whether it contains the kind
of lexical items for testing purpose.

Once target words have been selected, there is often a decision to be made as
to whether they should be presented to the test takers in isolation or in some form of
context. The question of how to present selected words to the test takers needs to be
related to the purpose of the assessment. In systematic vocabulary learning, students
apply memorising techniques to sets of target words and their meanings. They are
assessed on their ability to supply the meaning when given the target word, or vice
versa, whether it be research purpose or for monitoring of progress in classroom
learning. In tests of vocabulary size, words are often presented isolation. This allows
the test designers to simplify the test takers' task and cover a large sample of words in
the test. In research on incidental vocabulary learning, the learners encounter the
target words in context during reading or listening activitis, but in the test afterward
the words are presented separately.

For other purposes, the presentation of target words in some context is

desirable or necessary. In discrete, selective tests, the context most commonly
consists of a sentence in which the target word occurs, but it can also be a paragraph
or a longer text containing a whole series of target words. One function of sentence
context in traditional discrete-point items is to signal the particular meaning or use of
a high frequency target word which the test designer wishes to test. If the target word
is a low-frequency one that the learners are not expected to know, the ability to infer
its meaning on the basis of contextual clues may be precisely what is to be assessed.
In recall-type tests of productive vocabulary, in which the test takers are expected to
supply the target word rather than being presented with it directly, a sentence context
is one means of eliciting the words. A whole passage offers greater opportunities to
assess aspects of word knowledge in addition to meaning, for example, the grammar
of the word, its inflectional and derivational forms, collocational possibilities,
stylistic appropriateness to the context. One test format that draws on these kinds of
knowledge is a selective deletion cloze, in which content words are deleted from the
text and the task is to write a suitable in each blank.

One last design consideration concerns the language of the test itself.
Whereas monolingual test format only the target language is used, a bilingual one
employs both the target language and the learner's own language. This aspect of test
design involves more than just characteristics of the expected response. The question
of whether a bilingual format should be used has much in common with the debate
over the relative merits of monolingual and bilingual dictionaries for language
learners. There are both practical considerations and issues of pedagogical involved.

On a practical level, it obviously makes a difference whether the learners have

a common first language and whether the teacher is proficient in that language. In
English speaking countries teachers o English are typically working with learners
from a variety of language backgrounds, which makes it impractical for them to use
bilingual test items. On the other hand, most teacher of English in secondary schools
in European countries share with their students a national language and the same
applies to foreign language teachers. These teachers routinely use the national
language as a tool in foreign language teaching and testing. Another practical
consideration is the proficiency of the learners. L1 is likely tp play a greater role in
assessing the vocabulary of lower proficiency learners than those who are more
advanced.

On a broader level there is ongoing debate about the appropriate role of L1 in

the teaching of English to speakers of other languages. Native speaking ESOL
teachers, who are all too often ignorant of their learner's first language, have tended
to make a virtue out of their monolingualism and to assume that the most effective
way of teaching the language is through the exclusive use of English in their
classrooms. If teachers take this position, there is presumbaly no place for bilingual
vocabulary tests. The issue of translation equivalence between languages is also
relevant here. It is unwise to encourage in learner's belief that any word in L2 has a
directly synonymous word or phrase in L1.

Thus, although much vocabulary testing is monolingual L2, especially as

learners advance in their proficiency, there is certainly a case for using the first
language when it is practicable and when L1 provides a better means for the test
takers their understanding of the target vocabulary.