Center for Advanced Language Proficiency Education and Research (CALPER)

The Pennsylvania State University

Investigating Assessment Perceptions and

Practices in the Advanced Foreign
Language Classroom

Elana Shohamy, Ofra Inbar-Lourie and Matthew E. Poehner

Report No. 1108

Research funded by a grant from the United State Department of Education (CFDA 84.229, P229A020010).
However, the contents do not necessarily represent the policy of the Department of Education, and one should not
assume endorsement by the Federal Government.
© CALPER 2008. All rights reserved.
 Investigating Assessment Perceptions and Practices
in the Advanced Foreign Language Classroom

Center for Advanced Language Proficiency Education and Research

The Pennsylvania State University

Report No. 1108

Elana Shohamy, Tel Aviv University

Ofra Inbar-Lourie, Tel Aviv University
Matthew E. Poehner, The Pennsylvania State University

Research funded by a grant from the United States Department of Education (CFDA 84.229,
P229A020010). However, the contents do not necessarily represent the policy of the Department of
Education, and one should not assume endorsement by the Federal Government.

© CALPER 2008. All rights reserved. No part of this report may be reproduced or utilized, stored in
a retrieval system, or transmitted in any form or by any means. Electronic, mechanical, or
otherwise, without the written permission of CALPER, The Pennsylvania State University, 5 Sparks
Building, University Park, PA 16802, USA.

Please cite as: Shohamy, E., Inbar-Lourie, O., & Poehner, M.E. (2008). Investigating assessment
perceptions and practices in the advanced foreign language classroom (Report No. 1108).
University Park, PA: Center for Advanced Language Proficiency Education and Research.
Contents
ACKNOWLEDGEMENTS 3
INTRODUCTION 4
BACKGROUND 5
1. Current views on the ALP construct 5
a. A focus on content 5
b. A focus on multilingualism 5
c. A focus on interactiveness 6
d. A focus on culture 6
e. A focus on the multiple ways of being advanced 6
2. Current views on classroom assessment 7
METHODOLOGY 8
1. The data collection Tool 9
2. The sample 9
a. Teachers’ Educational and Language Background 9
b. Teaching Experience 10
ANALYSIS 11
RESULTS 11
1. Perception of ALP 11
a. Understanding ALP 11
b. ALP as formal accuracy 14
c. Determining ALP: standards and learner seat-time 14
d. ALP, learner goals, and L1-FL differences 14
e. Teacher perceptions of ALP learners 16
2. Perceptions of ALP assessment 17
3. Assessment Practices 21
4. Differences in assessment perceptions and practices by groups 24
a. Assessment Practice, Instructional Context, and Level of Experience 24
DISCUSSION AND CONCLUSION 25
NOTES 28
REFERENCES 29
APPENDIX 32
Survey on Assessment Practices of Teachers of Advanced Language Learners 32
LIST OF TABLES
Table 1 Perceptions of ALP competencies 11
Table 2 Assessment purposes in ALP classes 18
Table 3 The importance of activities for assessing ALP 18
Table 4 Teachers' level of training in using assessment tools 19
Table 5 Effect of external guidelines and standards on teachers’ assessment 20
Table 6 Frequency of administration of assessment procedures 21
Table 7 Contribution of type of assessments to final grade 22
Table 8 Assessment procedures and their foci (N=413) 22
Table 9 Focus of portfolio assessment 23

2
ACKNOWLEDGEMENTS

This report would not have been possible without the significant contributions of
Gabriela Appel, CALPER Program Coordinator and Arlo Bensinger, CALPER
Technology Coordinator, and Robert Schrauf, Associate Professor of Applied Linguistics.
Design and editorial assistance were provided by Sungwoo Kim, Tracy Beckett and
Meredith Doran.

3
INTRODUCTION

This report is contextualized within the work of the Center for Advanced Language
Proficiency Education and Research (CALPER), which examines theory, research and
practices relating to advanced language learners. One of the CALPER initiatives is the
Assessment Project, which considers various aspects involved in the assessment of
Advanced Language Proficiency (ALP). The activities conducted within the framework
of this project included the following: a) conceptualizing of the ALP construct with
regard to assessment; b) administering a series of workshops for practicing foreign
language teachers on the use of performance- and content-based tasks for assessing ALP
(Shohamy & Inbar-Lourie, 2006a; Shohamy & Inbar-Lourie, 2006b); and c) carrying out
research on the perceptions and practices of assessing ALP learners by practicing
teachers in secondary schools and universities. This research can contribute to the
creation of a body of knowledge that is instrumental for conceptualizing and developing
effective practices for assessing ALP, and that can ultimately be useful in
professionalizing ALP teachers.

This paper reports on the research results of a survey conducted in 2005-06 to examine
issues relating to ALP and its assessment in classroom settings. Specifically, the present
report focuses on the views of language teachers' of varied backgrounds (in Foreign
Languages, and English as a Second Language, at both high schools and universities) on
assessment of ALP, and on these teachers’ actual assessment practices.

Assessment is known to consist of two major components, 'the what' (e.g., how the
language is perceived and conceptualized) and the 'how', (e.g., the very methods used to
assess this knowledge); the focus of this report is on these two elements. Thus, the results
that will be presented here focus on two principal dimensions: the perceptions of teachers
about ALP (the ‘what’), and the teachers’ reports of the practices they use in assessing
ALP (the ‘how’). The interest in teachers' perceptions and practices originates from a
number of current research strands. First, there is a growing recognition of the need to
link theories of second language learning to the application of testing as part of learning
(Bachman & Cohen, 1998). Second, in the area of classroom and formative assessment,
there is growing attention to the importance of integrating learning and assessment and to
the teacher’s dual role as instructor and assessor (Leung, 2004; Rea-Dickens, 2008).
Third, there is growing interest in defining the nature of the ALP construct (Byrnes,
2006). The study we report here brings together these emerging research areas to draw
connections between theories and concepts of ALP on the one hand, and teachers'
perceptions and assessment practices on the other.

The paper is divided into three components; in the first, we survey the current thinking
about ALP and its assessment in classroom instruction. In the second section, we report
results of the survey we created that assessed testing perceptions and practices of ALP
among a large sample of language teachers at the secondary school and university level.
In the third section, we draw conclusions from the theory and research we have reviewed
to offer recommendations as to the direction needed in both research and practices
relating to the assessment of ALP.

4
BACKGROUND

1. Current views on the ALP construct

Defining ALP is of great importance, as it is only when the 'what' is clearly defined that
we can think of 'how' to assess it. Below is a condensed list of some of the features of
ALP we have identified based on our thinking in the CALPER project on this subject (a
more elaborated description is provided in Shohamy, 2006).

a. A focus on content
We begin by claiming that ALP is a complex and rich construct with multiple
components, resulting in mixes and hybrids and endless varieties in terms of accents,
lexicon, and syntax. 'Advancedness' varies in terms of a speaker’s personal style as
well as in terms of his or her choices with regards to content, words, and intonations.
A further feature is that language is viewed here as serving as a means for expressing
and interpreting rich and meaningful content through interactions and negotiations
(Byrnes, 2002a; 2002b). This perspective stands in contrast to language being viewed
as secondary in relation to content and knowledge used for interpretation, ideas, and
concepts. Indeed, while in beginning and intermediate classes, language is often
perceived as the goal and target of instruction, in advanced levels it is considered
mostly as a tool, rather than as the goal. Weigle and Jensen (1997) describe a range of
content-based programs that differ in their primary focus and in the relative
importance of language versus content. Among them are theme-based programs, in
which language is organized around a single topic or theme; here, language
instruction and acquisition are the primary focus, and the content serves as a vehicle
for learning the language. At the other end of the spectrum, we can identify programs
that give equal weight to content and language, or programs that give priority to
content with little attention being given to language. Typically, we claim, a stronger
focus on content characterizes more advanced levels of instruction.

b. A focus on multilingualism
Traditionally, the construct of ALP has been based on a monolingual construct of
language where ‘other’ languages, and especially the L1 of the learners, had no place
and constituted an unacceptable intrusion. It has been further assumed that the higher
the proficiency level, the less learners engaged in L1 as they reached toward the goal
of being ‘native speaker-like.' This assumption has persisted in spite of research that
points to the important role that L1 plays in L2 proficiency (Bialystok, 2001). In
studies of immigrant students, for example, it has been shown that the use of two
languages improves academic achievement in math (Solanos-Flore, 2003). The view
that L1 has no place in the L2 classroom thus has no basis in multilingual contexts. In
addition, the field of language and literacy studies increasingly views multiple
expressive modalities as central to literacy practices, and highlights the ways that
non-linguistic components such as gestures, visuals, images, music, and diverse signs
interact with and inform understandings of linguistic features (Kress 2003; Kress and
van Leeuwen 1996). All of these communicative modalities are thus important
components to consider in definitions of ALP as well.

5
 c. A focus on interactiveness
This feature views language as a social and interactive tool between readers and
writers, readers and readers, speakers and listeners, and among a variety of agents co-
constructing meanings in different ways by using dialogical devices (Swain, 2001).
Such a view is in sharp contrast with earlier positions, which focused on the learner as
an independent and autonomous agent and user of a given language. Bachman (1990)
discusses interactional ability in testing between the language user, the context, and
the discourse. This perspective on interaction as multilayered is particularly related to
content-based language learning.

d. A focus on culture
While culture has a long history in FL learning (Kramsch, 1995), it has traditionally
been treated in terms of a dichotomy between language and culture. By contrast,
Lantolf (2006) supports a unified approach to the study of language and culture, and
proposes the term languaculture “to signal the re-introduction of meaning into
language study” (see also Agar, 1994). Among the consequences of such a stance is
this: meaning becomes much more interesting and complex because it is seen as
entailing knowledge of different concepts, encoded in such features as conceptual
metaphors, lexical networks, lexico-grammatical structures, and schemas that
represent different ways of organizing the world and our experiences in it. Within
this view, then, ALP is not only about the mastery of linguistic forms in a formal
sense, but also about understanding how such forms can be used to interpret and to
create meaning in specific ‘languacultural’ contexts.

e. A focus on the multiple ways of being advanced

The emphasis here is on the many ways of being 'advanced,’ taking into account the
richness and complexity of language. Indeed, if language is seen as a means to
deliver specific content, then language proficiency cannot be locked into uniform and
systematic ways of producing language, and certainly cannot be judged by abstracted
‘native speaker’ criteria. Instead, it should be evaluated in terms of various content-
and context- related criteria. These multiple ways of conceiving proficiency also
include multimodal communicative resources involving hybrids and fusions of a
number of languages in multilingual communities (Shohamy, 2006, Canagarajah,
2007). In this view, ALP implies a variety of ways of knowing in line with the open
nature of languages, cultures, multilingualism, and social interaction. With this
perspective in mind, it is therefore difficult to see how language proficiency
guidelines that emphasize homogeneity and linearity, such as those of ACTFL (the
American Council for the Teaching of Foreign Languages) and the CEFR (Common
European Framework of Reference) can capture these diverse and complex notions.

Having identified some of the key features for conceptualizing ALP, we now turn to
questions about whether and how these characteristics are incorporated into the 'what' and
the 'how' of classroom assessments.

6
2. Current views on classroom assessment

Despite the growing understanding of the importance of classroom assessment in the

learning process, testing researchers have generally overlooked the area of classroom-
based assessments (Leung, 2004; Davison, 2004). While large-scale standardized
language tests are receiving strong attention in the literature and research, relatively little
interest has been directed at what teachers actually do in the classroom to assess learners.
This lack of interest in classroom assessments can be explained by the fact that large-
scale tests have a major impact on the lives of test-takers and institutions, and are
therefore often considered more significant for testing institutions and researchers to
study. In most situations, classroom-based language assessment practices are perceived
as less high-stakes than standardized language tests, though this is a view that is being
increasingly challenged (see special issue of Language Testing, 2004). In this regard, the
recent work by Rea-Dickins and colleagues (Rea-Dickins, 2004; Rea-Dickins & Gardner,
2000) has been especially enlightening. These researchers argue that classroom
assessments are indeed high-stakes, as they often inform decisions such as program
admission, placement, and promotion, as well as access to instructional resources and
learning support services.

The move to integrate assessment and teaching (Black & Wiliam, 1998), and the growing
awareness of the centrality of teacher-based assessment in the learning process, have
brought about an increased interest in language teachers’ perceptions of assessment
practices. Most of the existing studies in this area have focused on English language
teaching, whether as a Second or a Foreign Language (Brindley, 2001; Davison, 2004;
Leung, 2004; Cheng, Rogers & Hu, 2004; Rea-Dickens, 2001, 2004; Shohamy, 1998).
These studies have surveyed teachers’ assessment practices and beliefs, and the impact of
external norm-setting and tests on these practices. In addition, research has been
conducted on the extent to which the use of different assessment tools (e.g., portfolios,
self-assessment, projects, tasks) is incorporated into language learning
(see http://calper.la.psu.edu/assessment). Findings on teachers’ assessment practices also
point to great diversity with regard to assessment practices and beliefs.

The growing recognition of the importance of assessment practices in the FL classroom

has led to concern over the validity and reliability of classroom assessments. The work
of Torrance and Pryor (1998) reports on formative assessment practices among language
arts teachers in primary schools. The researchers conclude that while teachers were
familiar with various types of assessment practices (e.g., cloze tests, performance
assessments, etc.), they did not have a clear framework for implementing assessments
that would reflect and support student learning. Questions are also raised with regards to
the quality and appropriacy of reliability and validity as measures, as they are based on
procedures used for standardized tests; and whether they are applicable to classroom
assessment (Leung & Lewkowicz, 2008).

Yet, the role of teachers is crucial in the use of assessment as part of learning, as
expressed by Rea-Dickins (2004). She notes that foreign language teachers are in reality
agents of assessment, in that they routinely design and implement assessments and

7
interpret student performance resulting from those assessments. She suggests that
teachers may feel torn by what they see as competing demands, namely to function as a
“facilitator” of students’ efforts to learn the FL, but also as a “judge” of learner
performance (p. 253). In a similar vein, Edelenbos and Kubanek-German (2004) have
proposed the construct diagnostic competence to further conceptualize the unique
features of classroom assessment by teachers.

Though some research has been conducted on advanced foreign language learners,
especially in academic contexts (Byrnes, 2002; Norris, 2002), little is known about the
specific assessment needs and practices in advanced learner classrooms (Norris, 2006).
As noted above, advanced learner classes incorporate unique language teaching
components, such as emphasis on content and genre, and often require multifaceted
learner performances. It is not known if and to what extent these aspects are reflected in
the teachers’ assessments, nor even how teachers conceptualize advanced language
proficiency. Since teachers have such a significant role in conducting classroom
assessment, understanding of their perceptions and assessment practices is vital.

A first step in this direction is to discover how foreign language teachers understand ALP
and how their views are instantiated in concrete, day-to-day assessment practices in the
FL classroom. Questions are therefore raised as to the extent to which teachers perceive
and implement the new notions of ALP and apply them to assessment practices as part of
instruction. It is in the context of the new ways of thinking about ALP and the growing
focus on classroom assessment that we posed the following research questions:

1. What are the perceptions of teachers of ALP?

2. What are their perceptions of assessing ALP?
3. What the assessment practices of teachers with regards to ALP?
4. Are there differences among teacher groups with regards to perceptions and
practice of ALP? Differences will be examined in relation to the following
factors:
a. Training
b. Teaching experience
c. Language teaching type (e.g., ESL vs. foreign languages)
d. Level of teaching (e.g., advanced vs. other levels)
e. Type of institution (e.g., university vs. secondary school)
f. Teaching focus (e.g., content-based vs. general)

METHODOLOGY

We undertook an investigation of teachers’ perceptions of ALP, and of how these

teachers carry out classroom-based assessments of ALP. We developed an assessment
survey which was administered online with a commercial survey (Survey
Monkey www.surveymonkey.com) tool from November 10, 2005 until April 10, 2006 (a
total of one hundred and fifty days).

8
1. The data collection Tool
The survey consisted of forty-eight items aimed at investigating the perceptions and
practices of language teachers (see Appendix). The questionnaire included three key
components:

a. Background information (e.g., language of teaching, institution, teaching

experiences, and level of teaching)
b. Assessment practices (e.g., frequency of assessment, type of tools,
contribution of tools to final grade, types of assessment used, the specific
language components per tool, emphasis on content vs. language, impact
of standards and guidelines)
c. Perceptions regarding skills and abilities of ALP (e.g., competencies of
ALP).

Most of the items were in the form of multiple choice or Likert type scales (with either 4
or 5 options). In addition, two open-ended questions asked respondents to list additional
ALP competencies and additional ideas on assessing ALP.

2. The sample
In total, 467 individuals completed the entire survey: approximately one-third (33%) of
the respondents were teachers in secondary school1 (SEd) settings, while more than one-
half (>50%) taught in higher education (HEd), that is, at two and four-year colleges,
community colleges, and universities. The respondents consisted primarily of foreign
language teachers (275), some ESL teachers (22) and a small number that taught both.
The questions that dealt with teacher backgrounds and their instructional context revealed
considerable diversity among the respondents, and helped to ground interpretations of
their assessment practices and their views of ALP. Before discussing the teachers’
assessment of ALP, a more detailed description of the respondents themselves is in order.

a. Teachers’ educational and language background

The majority of respondents held advanced degrees, with approximately forty percent
(40%) having earned a Master’s degree, and another forty percent (40%) a doctorate.
Nearly all (96%) received formal training as FL teachers, and over half (>50%) did
some of this in the form of college or university coursework. However, there was a
good deal of variability with regard to the extent of their teacher preparation. For
instance, only thirteen percent (13%) of the respondents had specialized in education
during their studies, while roughly half (≈50%) had focused on foreign languages and
literatures. This trend would suggest that the teachers may have had far more expert
knowledge of the object of instruction (i.e., the FL) than of theories of learning,
pedagogical principles, methods of instruction, and approaches to assessment.
Furthermore, it is not clear that teachers are able to supplement their formal academic
backgrounds with pedagogical training after they have begun teaching: only twenty-
five percent (25%) reported having participated in in-service professional
development workshops and seminars.

9
In addition to their educational credentials, the teachers had generally achieved expert
or native proficiency in at least two languages. Over ninety percent considered
themselves to be native or expert speakers of English. Spanish (9.8%) and Chinese
(9.0%) were also common native languages, while Spanish (29.0%) and French
(24.1%) were listed as the two most popular “expert proficiency” languages after
English. The predominance of Spanish and French as expert proficiency languages
may be explained by the fact that these were also the two most commonly taught
foreign languages in these teachers’ institutional contexts. Spanish was by far the
most widespread foreign language taught, with 37.2% of respondents identifying
themselves as teachers of this language, compared with 23.3% French, 12.3%
German, 10% Japanese, 9.7%, Chinese, and 14% ‘other’ languages. Although
Spanish and French accounted for the lion’s share of languages taught, even these
teachers’ experiences were far from uniform. Their length of their teaching
experience and the extent to which they had worked with ALPs were also important
factors, and these are considered next.

b. Teaching experience
Determination of respondents’ length of teaching experience yielded the following
three groups: veteran teachers (more than twenty years experience), experienced
teachers (those having taught between eleven and twenty years), and junior teachers
(those with ten or fewer years of experience). In this pool of respondents, thirty-
seven percent (37%) were veteran teachers, thirty-three percent (33%) experienced
teachers, and twenty-nine percent (29%) junior teachers. The picture of teachers’
experiences is further refined by focusing specifically on the amount of their teaching
at advanced levels. More than half of the respondents (58%) had fewer than eleven
years of experience teaching ALP students and more than one-third (38.2%) had only
taught at the advanced level between one and five years. Of course, this may be due
to the fact that there are typically far fewer learners at advanced levels of language
study compared with intermediate and beginning levels. Indeed, when asked about
the percentage of their current teaching load devoted to ALP, nearly half (47%) of the
respondents teach advanced courses just twenty-five percent (25%) of the time and
only a minority (5%) report working exclusively with ALP students.

Regardless of the level of the courses they taught, respondents generally described
their syllabi as following either a skills-based or a content-based instructional
approach. These curricular orientations should be kept in mind during the discussion
of teachers’ views of effective assessment methods discussed below.2 Before turning
to the matter of assessment practice, however, it is worth considering how teachers
understand what it is they are assessing, and indeed what it is they are helping
learners develop. That is, a principal goal of the survey was to shed light on teachers’
thinking about ALP. With a clear view of teachers’ conceptualization of ALP, we will
be better positioned to appreciate how teachers’ understanding of the construct leads
to their expectations for ALP and informs their approach to assessment.

10
ANALYSIS

For all survey items, a double-level approach was applied, whereby responses were
analyzed for the entire population and then analyzed again according to subgroups of
respondents. Subgroups were formed based on respondents’ answers to initial
demographic items on the survey, and teachers were categorized according to the
following criteria: whether they had formal training as language educators (trained vs.
untrained); their level of teaching experience (less than ten years, between ten and twenty
years, or more than twenty years); whether the language they taught was a foreign
language, English as a Second Language, or both; their institutional context (secondary
school vs. college or university); whether they focused their instruction on language,
content or both; and finally, whether they worked primarily with advanced language
learners. For all items, frequency counts and percentages were first determined for the
entire population as well as for the subgroups, and initial descriptive statistics were run,
i.e., means and standard deviations. Repeated-measures ANOVAs were run to ascertain
differences between and among groups for multiple option items, and Tukey Post-hoc
tests were employed to establish which of these differences were significant. For two-
option survey items (e.g., the ‘yes-no’ format of item 23), chi-squares were used to
compare groups’ responses, and for items in which continuous variables (such as the
percentage that an assessment counts towards the overall grade) were presented as
multiple options (as in items 25 and 28). The means were analyzed using either a Mann-
Whitney U Test (to compare two subgroups) or a Kruskal-Wallis ANOVA (to compare
three groups).

A grounded content analysis approach was taken with the open-ended survey items. A
constant comparative method, as outlined by Bogdan and Biklin (1998), was employed to
situate each individual’s comments relative to those of other respondents and to identify
recurring themes and patterns in the data. Through an iterative process of data reduction
and verification (see Miles & Huberman, 1984), the various themes were then grouped
according to larger categories that accounted for the teachers’ beliefs about the nature of
advanced language proficiency, their views of effective classroom assessment, and their
actual assessment practice. Data from the open-ended items helped to illuminate the
meanings and conceptualizations behind the teachers’ responses to other survey items.

RESULTS

1. Perception of ALP
a. Understanding ALP
Teachers’ perceptions of ALP were based on the responses they gave to both the
open- ended and closed questions. The six closed questions focused on whether
teachers view L2 ALP in similar ways to L1 proficiency, and then on ALP
competencies, e.g., writing academic papers, giving presentations on academic topics,
understanding humor, metaphors and complex language structure. The open-ended
question asked respondents to provide other abilities that they perceived as
characterizing ALP learners.
11
With regard to the first question (on the relationship between L2 advanced
proficiency and L1 proficiency), results show that a majority of respondents (46.1%)
felt that ALP students should be able to do perform the same language functions in
their FL as in their L1, implying that at the advanced level, learners should have
attained a proficiency comparable to that of native speakers. This approach was
partially corroborated in the responses provided to a list of functions in various
language domains in the form of ‘advanced language learners can do’ statements.
Teachers were asked to rate to what extent they believe ALP learners should be able
to perform these functions on a Likert scale ranging from strongly agree (5) to
strongly disagree (1). Results indicated that most respondents agree that ALP learners
should be able to carry out the competencies described, though to varying degrees.
The competence that was rated the highest was ‘participate in conversation on current
events’ (x=4.44) and the lowest was ‘read professional journals’ (x=3.35); these
differences were found to be significant. Thus it is evident that teachers believe that
multiple competencies are required in various language domains. These results are
presented in Table 1 below.

Table 1
Perceptions of ALP competencies

Competencies Means for the whole Standard

sample (n=396) Deviation
Read a work of popular fiction 4.10 .77
Read newspapers 4.38 .68
Read professional journals 3.35 1.06
Write academic papers 3.61 1.05
Participate in a discussion on an 3.98 .84
academic topic
Give presentations on academic topics 3.92 .85
Participate in conversations on current 4.44 .62
events
Understand humor 3.83 .90
Use and understand metaphors 3.83 .85
Use complex language structures 3.77 .80
accurately
Explain grammar to other learners 3.81 .89
Demonstrate intercultural competence 4.13 .74
Describe complex processes 3.54 .87
Construct narratives 4.22 .67

12
Interestingly, respondents stressed everyday language activities over academic ones.
Specifically, the three activities teachers most commonly believed ALLs (Advanced
Language Learners) should be able to accomplish were: participating in conversations
about current events, reading newspapers, and constructing narratives to describe
events. The least common responses were writing academic papers, describing
complex processes, and reading professional journals.

As explained above, analysis of the two open-ended items followed an iterative

process in which initial themes were identified by comparing responses across
individuals and verifying, rejecting, or reformulating the themes to account for each
new set of comments. This approach allows for a detailed examination of the content
of responses without imposing a priori conceptual categories.

Based on the two open-ended questions, two broad perspectives of ALP emerged:
Advanced Language Proficiency as Communicative Competence and ALP as Formal
Accuracy. In fact, many of the teachers’ descriptions of ALP identified multiple
dimensions of communicative competence such as various communicative skills,
intercultural competence and language variations.

Many teachers explicitly mentioned “communication” in their statements about ALP,

often emphasizing “functionality” over formal “accuracy.” One teacher captured this
perspective succinctly in stating that ALP involves the awareness that one can be
“free to commit errors and still participate” in the activity at hand.

Some of the teachers further defined ALP by linking it to communicative acts,

understood both broadly and more specifically. For example, some stated simply that
ALP concerned the ability “to live abroad and cope” with the everyday demands of
the FL environment while others mentioned the capacity to express one’s views and
“to argue and persuade.” Teachers felt that such abilities involve “a more subtle
understanding of the language” requiring learners “to choose expressions and
vocabulary to express things not concrete.”

A number of respondents mentioned “intercultural competence” as a defining feature

of ALP. One teacher explained this as follows:
[An ALP learner is] not only proficient at understanding a second or foreign
language, but is capable of explaining rather complex or subtle aspects of the
target culture. This might involve knowledge of target culture life patterns and
value systems as well as literary / artistic works (i.e. small 'c' as well as large 'C'
culture).

Similarly, other respondents referred to a form of sociolinguistic competence,

explaining that ALP connotes appreciation of linguistic variation across speech
communities and that “the ALL [Advanced Language Learner] values the learning of
the multiple languages with the Language [sic].”

13
b. ALP as formal accuracy
In contrast to communicatively-oriented definitions, others characterized ALP in
more formal terms, emphasizing mastery of grammar, vocabulary, and fluency. For
instance, some stressed “automaticity” as a defining feature of ALP, explaining that
rapidity of speech suggests that learners have “internalized the rules of the language.”
Still others conceptualized ALP according to increased language “output,” which one
respondent attributed to “more refined listening skills.” The following sub-sections
address various factors teachers believed influence the determination of ALP.

c. Determining ALP: Standards and learner seat-time

For the majority of respondents, application of the label “advanced” to particular
individuals was unproblematic. Two common criteria for establishing learners’ level
of proficiency were their performance relative to standards established by
professional organizations such as ACTFL, and the number of semesters or years they
had been studying the FL. With regard to external standards, however, there was
variation concerning which level qualified as “advanced.” In the case of
Interlanguage Roundtable (ILR) standards, some teachers suggested Level 3 as a
benchmark for ALP while others insisted that learners should test at a Level 2, 3+, or
even 4. ACTFL standards were similarly open to interpretation. For one respondent,
only a rating of Superior warranted designation of ALP, but for others ALP ranged
from Advanced to Distinguished. Another formal indicator was the Advanced
Placement (AP) exam, but here again there was little agreement among teachers, with
some suggesting Passing as a cut-off and others requiring Advanced. A similar
phenomenon occurred among respondents who linked the concept of ALP to learner
seat time. Some viewed students in their third (or fourth) year of university-level
language study as advanced and therefore having achieved ALP, while others
reserved the term ALP to describe graduate students “studying [in the FL] to be
professionals in their academic discipline.” Still others suggested that ALP may need
to be defined differently if one is referring to secondary school students, university
undergraduates, or graduate and professional students. This notion that the meaning
of the label “advanced” is not stable, but varies according to learning contexts and the
individuals to whom it is applied, was taken up by many teachers, as explained below.

d. ALP, learner goals, and L1-FL differences

Many respondents rejected the idea that ALP can be defined in general terms or by a
set of benchmarks that are applicable to all students in all instructional contexts. For
these teachers, definitions of ALP must also take account of other variables, including
learners’ goals, the language of study, and learners’ first language. For example, one
respondent explained that ALP relates to:
…a situated learning context, not general proficiency. So,understanding
metaphors and humor [in the FL] would be relative to the professional and
personal needs of a student.

14
From this teacher’s perspective, then, it would not be appropriate to include the
capacity to understand humor or to use metaphors as criteria for determining ALP,
because these abilities may be relevant to some learners and not others. This remark
suggests that ALP criteria need to be determined differently for individual learners in
particular contexts, and that while there would undoubtedly be some overlap in what
constitutes ALP, one should also expect variation in learners’ abilities.

According to some respondents, the FL itself must be considered during discussions

of proficiency level. In some cases, this view reflected a conviction that certain
features of language are more important to the study of some languages than others.
For example, one teacher contrasted Korean with other languages, arguing that the
study of Korean must focus on grammar and that control over grammatical structures
is central to ALP in Korean but that it might not be for other languages. Another
respondent similarly emphasized the importance of grammatical concepts for
successful study of Russian, noting differences between Russian and other languages
and concluding that:

If they [learners of Russian] really do not understand that the basic parts of
speech are indicated by case markers, and are still relying entirely on word order
or intuition about who did what to whom, then they are simply guessing about
what is really going on and have no system that they can build on for more
abstract topics, or for that matter, even concrete events.

These teachers’ remarks indicate that even as accepted an indicator of language

ability as knowledge of grammar need not be included – or always equally weighted –
in definitions of ALP. That is, the centrality of grammar is not a given, but rather its
value must be established, and for these teachers that depends in part on the
importance of grammatical concepts for mediating learning of a particular FL. This
may also account for the conceptualization of ALP in formal terms on the part of
some teachers. Additional research in this area might consider the relationship
between how instructors understand the particular FL they teach and their general
attitude towards ALP.

Interestingly, a number of respondents indicated that in their view definitions of ALP

must include not only features of the FL, but also learners’ L1. For these teachers,
some languages are simply “harder” to learn than others if they differ in important
ways from learners’ L1. From this perspective, learners whose L1 is English will find
it easier to achieve ALP in Spanish or French than in Asian or Middle Eastern
languages, which do not share a common alphabet with English. One respondent
referred to these as “languages that take longer to learn,” noting that there are usually
fewer expectations for students of these languages at advanced levels. As one teacher
reasoned,

A student of 'Advanced Spanish' and one of 'Advanced Arabic' can cope with very
different quantities of text and require different kinds and extent of
scaffolding/ancillary materials to find authentic content accessible. This does not

15
 mean that a course in the latter is in any way less 'advanced' because student
competence is in some ways more limited.

In other words, even though the learner of Arabic cannot perform in the same way as
the learner of Spanish, this does not disqualify the former from having achieved ALP.
For this teacher, the two learners had different linguistic distances to traverse from the
beginning, and evaluation of their achievements must take account of this difference.
Another teacher expressed a similar preference for understanding learners’
achievements relative to the challenges they face rather than a set of standard criteria
for establishing ALP. This individual advocated that Chinese instructors should
consider whether a student is an L1 English speaker, or an L1 speaker of a language
more closely related to Chinese, before evaluating ALP.

Before concluding this section, it is important to point out that, regardless of how they
understood the construct, there was general agreement among respondents as to how
to increase the number of learners in the U. S. who achieve ALP in their FL.
Prevalent among their recommendations was the conclusion that language instruction
at advanced levels should shift from its current focus on literature to an emphasis on
pragmatics, metaphor, and culture. In addition, many noted the difficulty of
achieving ALP during secondary school and university study. Respondents advocated
strongly that even for undergraduate majors, there was insufficient time to
realistically develop ALP and that those who did achieve such levels typically had
spent considerable time in an environment where the FL is spoken. These teachers
stressed the importance of study abroad experiences, and some also referred to the
practice in many nations of beginning FL instruction during elementary school,
suggesting that this would ultimately increase the number of U. S. students who
develop ALP. One teacher noted that this would effectively require a cultural shift in
how FL education is regarded in the U. S., and suggested that this might be achieved
if the need for ALP individuals in government and commercial contexts were clearly
articulated and FL curricula were explicitly linked to this need. These views of ALP
and the state of FL education in the U. S. were more fully fleshed out when teachers
were asked to reflect upon their own experiences teaching ALP classes, and it is to
this that we now turn.

e. Teacher perceptions of ALP learners

Respondents tended to view ALP learners as “good students.” Descriptions of ALP
learners typically emphasized their motivation for studying the language and culture,
and their “hard-working” approach to completing course assignments. In addition,
many teachers observed ALP learners' willingness to assume “ownership” or
responsibility for their own learning by taking the initiative to actively “seek out
opportunities to use language” in contexts outside the classroom. Moreover,
respondents noted that ALP learners assign great value to mastery of a FL. As one
teacher put it, ALP learners view FL learning as integral to their “cognitive,
emotional, and social development.”

16
 Similarly, a high degree of self-awareness and awareness of the learning process was
attributed to ALP learners. Students were believed to have achieved their advanced
status by being “independently strategic” in their approach to studying the FL.
Interestingly, teachers portrayed these qualities as inherent in ALP learners rather
than as features that can be brought out through appropriate pedagogies. Indeed, the
term “intrinsic motivation” was frequently applied to ALP learners to emphasize that
these individuals are driven less by grades than by “a strong desire to explore social
and cultural similarities and differences.” A number of respondents offered their
opinion that ALP learners appreciated “the lifetime aspect of second language
learning,” and that they recognized that they might never be “competitive” with
native speakers. This statement is striking, particularly in light of the tension some
teachers felt in their expectations of ALP learners and the validity of holding them to
native speaker standards (described above).

To summarize the discussion to this point, teachers generally found ALP learners to
be successful as a result of their intrinsic motivation for FL study and their
recognition that developing ALP is an enriching, lifelong process. However, what
exactly constitutes ALP was difficult for these respondents to define. On the one
hand, some searched for objective criteria, such as the duration of an individual’s
language study or performance on a standardized assessment measure like the
ACTFL's OPI (Oral Proficiency Interview). On the other hand, many respondents
referred to various competencies, including intercultural and pragmatic awareness.
For these teachers, effective communication was central to ALP, and the litmus test
for an ALP learner was his/her ability to function abroad in a FL environment.
Determining ALP was further complicated, according to some teachers, because
certain languages require more time to reach proficiency, particularly if one’s L1
differs substantially from the FL. Nevertheless, respondents were clear that if more
learners in the U. S. are to attain ALP in the years ahead, FL education will need to be
rethought. More precisely, beginning FL instruction earlier and supporting study
abroad programs were frequently mentioned means of helping learners develop ALP.
We will now examine teachers' perceptions of how to assess ALP.

2. Perceptions of ALP assessment

Teachers were first asked to indicate their perceptions regarding the importance of
various assessment formats for their particular ALP contexts. Responses provided on a 4
point scale (ranging from ‘not important’ to ‘very important’) indicate a strong preference
for using assessment for formative rather than summative purposes. In other words,
teachers were more interested in diagnosing ALP learners’ abilities in order to determine
areas in need of further support than in assigning grades. In fact, when asked to rate the
value of various assessment purposes, tracking learners’ 'on-going progress' was the most
frequent response, followed by 'appraising learners’ difficulties'. These results are
displayed in Table 2 below.

17
 Table 2
Assessment purposes in ALP classes

Purposes Mean Score Standard

(N=403) Deviation
To place into a program or course 2.60 1.06
To diagnose difficulties 3.46 .69
To assign grades 2.98 .89
To assess on-going learning progress 3.66 .59
To assess end of semester (or year) 3.34 .72
achievements

Teachers were then asked to rank the importance of various assessment activities in their
ALP teaching context. As can be seen from the results displayed in Table 3, “evaluating
written essays and compositions” was viewed as the most important activity (3.46 out of
a 4 point scale), followed by ”evaluating oral presentations” (3.39). The assessment
activity reported to be the least important was ”evaluating portfolios” (2.19). These
differences were significant as well

Table 3
The importance of activities for assessing ALP

Assessment activities Mean score Standard

[N=405 ] Deviation
Evaluating portfolios 2.19 1.22
Developing rubrics 2.88 1.03
Evaluating oral presentations 3.39 .81
Evaluating written essays and compositions 3.46 .76
Constructing classroom tests and quizzes 3.06 .91
Creating performance-tasks 3.12 1.00

These findings coincide with the amount of training reported for using different
assessment formats: the lowest amount of training reported was for using portfolios
(almost 50% stated they had received no training in this area), and the highest for
evaluating compositions and essays (3.06 on a 4 point scale). It is also important to note,
as can be seen in Table 4, that overall the level of training for all the assessment tools is
not high, ranging from ‘minimal’ to ‘some’.

18
 Table 4
Teachers' level of training in using assessment tools

Assessment tool Mean score Standard

(N=410) Deviation
Using portfolios 2.20 1.00
Developing rubrics 2.82 .98
Evaluating oral presentations 2.76 .88
Evaluating written essays and 3.06 .89
compositions
Constructing classroom tests or quizzes 3.16 .86
Diagnostic assessment 2.60 1.02
Using standards in assessment 2.98 .93
Assessing oral interviews 3.03 .88
Linking assessment to the curriculum 3.00 .93

The open-ended responses to the assessment practices provided more information on the
teachers’ perceptions regarding ALP assessment. Assigning grades was consistently
among the least important reasons for assessing. Even more interesting is that
respondents were specific regarding assessment procedures that further these ends and
those that do not. Quizzes and tests were criticized on a number of grounds, including the
fact that they “produce anxiety and diminish performance” by placing learners in high-
stake situations where a one-time, accuracy-based “measure” can determine their future
language learning opportunities.

An even more trenchant critique was that tests and quizzes construct language in an
“easy-to-measure” fashion that privileges knowledge of vocabulary and grammatical
structures and consequently presents a distorted picture of learners’ abilities. A number of
respondents referred to this approach to assessment as an inappropriate way not only of
viewing language, but of understanding language abilities. Rather than examples of what
learners are able to do in the FL, tests and quizzes aim to determine whether learners are
“grammatically accurate in speech or writing.” As one teacher pointed out, this is a
standard few “native speakers” live up to, and so “holding non-native speakers to that
standard is unfair.” Another teacher observed an inherent contradiction in using
decontextualized test questions to determine proficiency over everyday functionality in
the language: “Americans have very high standards for non-native speakers [of English],
but the reverse is not true for Americans learning a foreign language.”

In contrast to traditional tests and quizzes, so-called “alternative” forms of assessment,

including portfolios and performance-based assessments, were seen as invaluable when
working with ALP learners. The advantage of alternative assessments over traditional
assessments was attributed to three factors: 1) the integration of language and content,
particularly at advanced levels of study; 2) the range of language abilities that are
required for performing assessment tasks, and 3) the changing population of language
learners in the U. S.

19
As has already been mentioned in the description of the sample, the majority of
respondents reported following a syllabus characterized as either a skills-based or
content-based approach to language instruction. It is therefore not entirely surprising to
find that three-quarters (75%) of the respondents targeted both language and content with
their assessments. As one teacher observed, at the advanced level learners are likely to
engage with ideas and problems through the FL in ways that “aren’t regularly considered
in proficiency guidelines.” This view as to the inadequacy of the guidelines to reflect
ALP was corroborated in findings which pertain to the impact of external assessment
guidelines and standards on teachers’ perceptions. Respondents were asked to report on
the impact of external guidelines and standards on their assessment. The question related
to 5 types of commonly used standards and scales and teachers were asked to note the
extent to which they were influenced by these external measures on a 4 point Likert scale
ranging from ‘not at all’ (1) to ‘a great extent’ (4). Results point at a relatively limited
impact of these measures on teacher assesment practices, with means ranging from 1.73
for School Board Standards to 2.67 for the ACTFL Proficiency Guidelines (Table 5).

Table 5
Effect of external guidelines and standards on teachers’ assessment

Guidelines/Standards Mean score Standard

(N=397) Deviation
State Standards 2.06 1.16
ACTFL Proficiency Guidelines 2.67 1.03
School Board Standards 1.73 1.08
ILR Proficiency Scale 1.76 1.19
National Standards in Foreign Language 2.49 1.13
Education

Alternative assessments were viewed as providing ALP learners with an opportunity to

display the full extent of their development – including the intercultural, pragmatic, and
sociolinguistic aspects of competence mentioned earlier – through various genres and
modalities that are typically not possible using conventional 'paper-and-pencil tests'. This
position is even more striking when one considers that respondents also reported that they
had greater experience and training employing traditional forms of assessment. In other
words, even though teachers were more comfortable using tests and quizzes in terms of
their own training, they believed that alternative assessments better suited their
instructional goals and learners’ needs.

This interpretation is further supported by recurring comments expressing the need for
teachers of ALP to become more adept at employing a variety of assessment “tools” to
help them understand the impact of instruction, study abroad, and computer technology
on ALP development. In addition, one respondent commented that relying on multiple
sources of information about ALP learners rather than a single test decreases the
possibility of underestimating the abilities of certain populations, namely immigrant and
minority students as well as returning adult learners, who may under-perform on

20
traditional measures. Here again, teachers expressed the desire to obtain greater training
in the use of alternative assessment approaches.

3. Assessment Practices

The divergence between desirable and actual assessment practices was evident in terms
of the assessment procedures teachers reported to employ, the frequency with which they
assessed learners, and the importance attributed to various assessments. Results showed
that quizzes were the most common assessment measures employed by teachers of ALP
learners, with respondents administering an average of just over four (4) per semester.
Other common assessments included essays and compositions, performance tasks, tests,
and simulations. The frequency of each of these assessments (measure by number of
times per semester) is presented in Table 6.

Table 6
Frequency of administration of assessment procedures

Assessment Administrations/Semester * Standard

Deviation
Quizzes 4.06 1.64
Essays / Compositions 3.44 1.23
Performance Tasks 3.36 1.81
Tests 3.23 1.44
Simulations (e.g. role 3.03 1.38
plays)
Presentations 2.78 1.65
Oral Interviews 2.63 1.43

*Figures are given as an average number of administrations per semester

In terms of the relative contributions of each of these assessments to the calculation of

ALP learners' final grades, written tests were the most important. This is in contrast to
the criticism teachers voiced of tests, as reported earlier. Nevertheless, tests on average
comprised 29% of learners’ course grades, followed by essays and oral interviews (see
Table 7).

21
 Table 7
Contribution of type of assessments to final grade

Type of assessment Average % of final

grade
Written (including Midterm and/or final exams) 28.94
Essays (or compositions) 18.60
Oral interviews (including midterm and/or final 17.81
exams)
Quizzes 15.82
Other 15.34
Projects 15.07
Attendance 8.96
Portfolios 7.80

Although teachers described ALP in terms of communicative ability and the capacity to
function in everyday situations, suggesting an integration of language-related abilities,
their assessment practices broke down these abilities according to modalities and often
into discrete skills. For instance, tests and quizzes were most frequently used to assess
writing and reading abilities while presentations, oral interviews, and performance tasks
overwhelmingly focused on speaking. These abilities, in turn, tended to be
conceptualized as knowledge of vocabulary and grammar rather than in terms of the
intercultural or discourse competencies that teachers had reported were crucial to ALP
(see Table 8).

Table 8
Assessment Procedures and Their Foci (N=413)*

Tests Quizzes Essays/ Presen- Oral Perf.

Compositions tations Interviews Tasks

Vocabulary 38.4 34.6 9.0 11.8 6.2 38.4

Grammar 30.0 31.2 17.4 17.8 3.6 30.0
Intercultural 24.3 21.2 28.1 23.6 2.7 24.3
Competence
Discourse 28.2 30.2 21.1 18.4 2.1 28.2
Competence

*Figures are given as a percentage of total respondents

22
Thus, while intercultural and discourse competencies played a greater role in
presentations, oral interviews, and performance tasks, their overall importance was
eclipsed by the predominance of vocabulary and grammar in all forms of ALP
assessment. Furthermore, as mentioned, formal tests focusing on grammar and
vocabulary were by far the most significant determiner of ALP learners' final grades.

While tests and quizzes were tied explicitly to the evaluation of ALP learners'
grammatical and lexical knowledge, some of the less traditional assessments were
concerned with “non-linguistic” aspects of classroom performance. For instance, while
projects most likely require ALP learners to engage with the FL, 60% of respondents
used projects to assess students’ research skills and 50% added that projects were useful
for evaluating students’ ability to collaborate with peers. Self-assessment, too, was less
clearly linked to a particular language modality but was instead primarily viewed as a
tool for prompting ALP learners' to reflect upon their overall learning. Portfolios also
were extremely flexible with regard to the range of knowledge and abilities they assessed.
As summarized in Table 9, portfolios, used by approximately one third of the respondents,
were used extensively to assess everything from writing to subject matter knowledge and
research skills.

Table 9
Focus of portfolio assessment

Ability/Knowledge Response %*
Writing 85.8
Integrated skills 71.7
Subject matter knowledge 69.8
Vocabulary 62.3
Grammar 52.8
Intercultural competence 51.9
Speaking 46.2
Collaboration 37.7
Research skills 37.7
Discourse 34.9
Other (please specify) 20.8

*Figures given are as a percentage of total respondents

In spite of their recognition of the potential of portfolios to provide these insights into
ALP learners’ development, the majority of teachers (61%) counted them for less than
twenty-percent (25%) of students’ final grades, and less than two percent (2%) of
teachers reported using portfolios for more than fifty percent (50%) of the grades.

23
4. Differences in assessment perceptions and practices by groups

Examining the subgroups of the samples and looking at differences between the various
teacher groups provides further insight into the data. We focused on different groups of
teachers: a) those who teach content vs. those who do not; b) secondary school teachers
vs. University; c) teachers who teach advanced learners vs. those who teach other levels;
and d) experienced teachers vs. inexperienced ones. (The sample size of English as a
Second Language [ESL] teachers was small and hence did not allow for comparison with
Foreign Language [FL] teachers).

a. Assessment Practice, Instructional Context, and Level of Experience

In general, teaching context affected the amount and type of assessment activity teachers
engaged in while teachers’ level of experience played a role in how they chose to use
various forms of assessment. Teachers in Secondary Education (SEd) settings tended to
assess far more frequently than HEd (Higher Education) instructors, and this was
particularly striking in the use of quizzes, which were much more common in SEd
schools than in HEd institutions (M=4.37 vs. 3.19). Self-assessment and peer assessment,
in contrast, were more widely used by HEd teachers (61.05 % vs 38.95 and 62.79 vs.
37.21, respectively). HEd teachers also reported more frequent use of portfolios, projects,
self and peer assessment (p<.05). In addition, standards and guidelines were rated as
more influential by SEd teachers than by HEd teachers. In fact, while SEd teachers
reported that their practices were greatly influenced by State Standards (44%), teachers
working in HEd institutions reported that the ACTFL proficiency guidelines affected
their practice to the greatest extent (34%) with State Standards deemed less influential
chosen by only 8% of the sample. SEd teachers also tended to have specific training in
how to relate their assessments to standards and to their curriculum.

In terms of the use of various forms of assessment, the main effects were found between
using different assessment instruments and years of teaching experience (p< .0001).
Specifically, veteran teachers were found to use tests and quizzes less frequently in
comparison with junior teachers, who also gave greater weight to tests and quizzes in
determining learners’ final grades. Moreover, junior teachers differed from their more
experienced colleagues in how they used tests and quizzes; for junior teachers these were
a means of evaluating grammar and vocabulary knowledge, while veteran teachers noted
that these can also be used effectively to assess learners’ intercultural and discourse
competencies. Veteran teachers reported experimenting a great deal with project-based
assessments, peer assessment and self-assessment. In fact, they not only used these
assessments more than junior teachers but they also used them in more innovative ways.
For example, when less experienced teachers employed portfolios, it was almost
exclusively to evaluate FL writing, while veteran teachers recognized portfolios as a
valuable approach to assessing integrated skills, discourse, and learner collaborations.
This trend was paralleled in the use of peer assessment: veteran teachers engaged learners
in peer assessment not only of compositions and essays, but also of student presentations,
projects, and portfolios. The difference in practice between more and less experienced
teachers was perhaps most striking in the area of performance assessment, which junior
teachers understood as a method for assessing speaking but veteran teachers viewed as an

24
opportunity to evaluate all language modalities. Similarly comparison between ESL and
FL teachers in terms of their use of a variety of assessment tools, showed the latter use an
array of assessment procedures more than their counterparts (F=2.830).

Interestingly, some of this variability in the application of assessment procedures may be

explained, in part, by the relative focus of teachers at different levels of experience.
Veteran teachers more often emphasized culture in their assessments. Furthermore,
repeated measure analysis of variance showed significant effect for differences among
veteran and junior teachers in terms of degree of training. Veteran teachers reported
greater training in assessment than their junior colleagues, perhaps simply as a result of
having more time to fulfill professional hours necessary for re-certification (F=2.445).

In terms of defining ALL (Advanced Language Learner) abilities, instructional context

played a major role in teachers’ expectations of the skills an ALL should have. HEd
teachers, as one might expect, were more likely to assert that ALLs should be able to read
professional journals than were SEd teachers, who prioritized ALLs’ ability to explain
grammatical concepts and to use language structures accurately (F=1.763). Greater
differences were found by sorting responses according to the proportion of respondents’
teaching load at the advanced level. As noted earlier, teachers varied regarding the
amount of contact they had with ALLs. Expectations of ALLs differed between
respondents who taught at the advanced level fifty percent or more (>50%) of their time
and those who taught at the advanced level only a quarter (25%) of the time. Important
differences between these two groups were found in several areas. For instance, teachers
who worked primarily with ALLs placed greater importance upon writing academic
papers (3.94) and discussing academic topics (4.28) than their counterparts who taught
less classes at the advanced level (3.55 and 3.92 respectively, F=3.210; p<.05).

DISCUSSION AND CONCLUSION

While considerable resources have been devoted to developing large-scale standardized

test, assessment in the foreign language classroom remains far less well understood,
though it is recognized as being very instrumental for learning. The survey reported in
this paper presents information that can be useful in understanding classroom assessment
of ALP, as it identifies both teachers’ perceptions of the construct of ALP and how to
assess it, as well as teachers’ actual assessment practices.

In terms of how teachers view the construct of ALP, it was found that teachers perceive it
as a complex construct, consisting of multiple dimensions and competencies. Such
competencies include various academic abilities, intercultural competencies, construction
of narratives, interactions, argumentations, subtle understanding of language, and the
ability to draw on learners' L1 knowledge. ALP is thus not viewed in simple terms. In
terms of perceptions of assessment, teachers believe that this complex ALP construct can
only be assessed through the use of multiple assessment procedures such as portfolios,
performance tasks, essays, and rubrics and that assessment is always an on-going process
with a formative dimension.

25
Yet, in terms of the actual uses and practices of assessing ALP, teachers continue to use
mostly traditional forms of assessment such as quizzes and tests. Even when they use
more innovative procedures such as portfolios, they tend to focus on traditional language
elements such as vocabulary and grammar. In calculating the final grade, written exams
contribute the highest percentage to the grade. Thus, while the majority of respondents
were critical of conventional testing, tests and quizzes were still the predominant forms of
assessment used by teachers for understanding their students’ progress, and these were
assigned great importance in determining grades. In a similar manner, most teachers
advocated communicative or functional definitions of ALP, but knowledge of grammar
and vocabulary continued to be privileged indicators of learners’ achievements.

As the responses to this survey indicated, teachers’ pedagogical preparation very often
fails to offer them the broad, theoretical perspectives on assessment and its relation to
teaching and learning – that is, the necessary ‘scientific‘ knowledge – that they need to
connect to their everyday practice. In fact, many teachers believed that improving FL
education involves pursuing assessment initiatives beyond traditional tests and quizzes,
and that over time these practices can change the FL educational culture in the U. S.

Thus, the challenge to those involved in pre-service and in-service teacher professional
development is to offer resources and materials, as well as training opportunities, that
familiarize FL teachers with theoretically-driven principles (i.e., the components of ALP)
for creating assessment procedures that are appropriate for classroom assessment. It is
especially important to provide teachers with opportunities to engage in dialogue and
reflection about how to connect these principles to their own instructional contexts.
Rather than a ‘how-to’ model of assessment, such programs should offer teachers
systematic ways of thinking about the meaning of ALP in relation to assessment and of
subsequently connecting assessment with teaching and learning. Such training needs to
be understood in ways that will offer teachers the flexibility to create assessment
procedures and materials that will to fit their specific teaching and learning contexts.

In this final section, we would like to connect the theoretical views of ALP as we
described at the beginning of the paper with recommendations for how these features can
be assessed; i.e., bringing together the ‘what’ and the ‘how’ (see a more detailed list of
recommendations in Shohamy, 2006).

Since language is open and flexible, assessment, too, must have open boundaries and
flexible definitions of correctness. For example, ALP can often be multilingual,
incorporate the learner’s L1, and allow for different forms of language use. It is not
possible to apply uniform criteria of progression as there are different ways of ‘being’
and ‘becoming’ 'advanced'. In addition, assessment of ALP needs to be based on tasks
which are interactive, meaning that they should aim at capturing the contextual,
culturally embedded, and socially mediated nature of language. In terms of multiple
approaches to assessment, over the past two decades, a variety of alternative modes of
assessment have been introduced, among them performance assessment, task based
assessment, and portfolios, because the broad definition of ALP requires the testing
community to acknowledge that no single test can capture its complexity. If ALP draws

26
on a variety of knowledge sources, including other languages and multi-modal sources,
then tests must reflect that reality and include tasks that require such broad use of
language varieties. This type of test can be created by incorporating diverse forms of
expression and opportunities for conveying, among other things, nuanced meanings,
humor, metaphorical language use and symbolic competencies (Kramsch, 2004).

Multiple and diverse ways of knowing refer as well to the various ways of
demonstrating such knowledge, including utilizing visuals, music, and gestures, as well
as mixing several languages simultaneously, or creatively negotiating multilingualism, as
suggested by Canagarajah (2006). Furthermore, if no single test can be comprehensive
enough to probe the vastness of possible content, then selection of a specific assessment
device depends largely on the uses to which assessment will be put, that is, the type of
decisions that will rest on it. Methods may include dynamic assessment, performance-
based, and content- and task-based assessment.

In line with such views, dynamic assessment has come to be seen as a viable alternative
approach to many forms of static testing, as it affirms the key assessment principle that
language must serve as a medium for communication (Lantolf & Poehner, 2004). Based
on Vygotsky’s sociocultural and interactional views, dynamic assessment attempts to
erase the separation of testing and learning. At present, testing is usually an isolated
event that takes place before learning, as in placement tests, or during or after learning, as
with tests for certification or achievement. By contrast, dynamic assessment proposes to
integrate and connect learning and assessment. One current program that is developing
methods for dynamic assessment is at CALPER, led by James P. Lantolf and Matthew E.
Poehner, and it builds on many of Vygotsky’s insights to integrate teaching and
assessment into a single activity aimed at supporting FL learners’ development.

The focus on content has important implications for the design and selection of
appropriate assessment procedures. Assessment practices will need to delineate
beforehand the relationship between language and content and incorporate that
specification into test design in the ALP context (Weigle and Jensen, 1997).

In terms of connecting teaching, research, and assessment the gap that was found
between how teachers perceive ALP and its assessment in relation to how they practice
assessment should lead us to view assessment of ALP as a very urgent need. Clearly
these results could lead us to develop more realistic and useful assessment policies,
enhance teacher training, and inform more appropriate pedagogies and training. The
insights obtained are relevant not only for assessment practices but to understand the
important connection and close relationship between language constructs, curriculum,
instruction, learning goals and outcomes. These results can provide important feedback to
the testing community so that researchers can develop more sophisticated answers about
the relation of assessment instruments and strategies and learning. Finally, valid test
development means getting engaged with classroom teachers and school programs in
order to become informed about existing practices of learning and assessment. This is of
prime importance given the responsibility of language testers, and given the enormous
power of tests in affecting how language knowledge is defined, perceived and practiced.

27
NOTES

1. In addition to foreign language instructors in higher education and

secondary school settings, a small number of respondents were teaching in
primary (2.6%) and middle (3.8%) schools when they completed the
survey. Analysis revealed that their beliefs and practices closely matched
those of secondary school teachers, and so for convenience these groups
were collapsed and are described here as SEd.

2. Respondents were not asked whether they design their own syllabi,
develop them in collaboration with other colleagues, or are compelled to
follow syllabi prepared by others. Therefore, the preponderance of
content-based and skills-based syllabi may not necessarily reflect the
respondents’ own orientation to education in all instances. However, the
trends evident in teachers’ responses to other survey, and particularly their
remarks concerning the importance of assessing both language and content
and the potential of alternative assessments to fulfill this purpose, would
suggest that the preference for this type of syllabus was indeed generally
motivated by teachers’ own educational beliefs.

28
REFERENCES

Agar, M. (1994). Language shock: Understanding the culture of conversation. New York:
William Morrow and Company, Inc.
Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford: Oxford
University Press.
Bachman, L. & Cohen, A. D. (1998) (Eds.). Interfaces between second language
acquisition and language testing research. Cambridge: Cambridge University
Press.
Bialystok, E. (2001). Bilingualism in development. Cambridge University Press
Black, P. & Wiliam, D. (1998). Inside the Black Box: Raising standards through
classroom assessment. Phi Delta Kappa International.
URL : http://www.pdkintl.org/kappan/kbla9810.htm
Bogdan, R. & Biklin, S.K. (1998). Qualitative research for education: An introduction to
theory and methods. (3rd ed.). Boston, MA: Allyn and Bacon.
Brindely, G. (2001). Outcomes-based assessment in practice: Some examples and
emerging insights. Language Testing, 18, 393-408.
Byrnes, H. (2000). Languages across the curriculum – intradepartmental curriculum
construction: Issues and options. In M. R. Kecht & K. von Hammerstein (Eds.),
Languages across the curriculum: Interdisciplinary structure and
internationalized education (pp. 151-75). Columbus, OH: National East Asian
Languages Resource Center at The Ohio State University.
Byrnes, H. (2002). The role of task and task-based assessment in a content-oriented
collegiate foreign language curriculum. Language Testing, 19, 419-37.
Byrnes, B., Weger-Guntharp, H.D. & Sprang, K. (2006) (Eds.). Educating for advanced
foreign language capacities. Washington D.C.: Georgetown University Press.
Canagarajah, S. (2007). The ecology of global English. International Multilingual
Research Journal, 1, 89-100.
Canagarajah, S. (2006). Changing communicative needs, revised assessment objectives:
Testing English as an international language. Language Assessment Quarterly, 3,
229-242.
Cheng, L, Rogers, T. & Hu, H. (2004). ESL/EFL instructors' classroom assessment
practices: Purposes, methods, and procedures. Language Testing, 21, 360 - 389.
Chalhoub-Deville, Micheline. (2003). Second language interaction: Current perspectives
and future trends. Language Testing, 20, 369-83.
Council of Europe. (2001). Common European Framework of Reference for Languages:
Learning, teaching, assessment. Cambridge: Cambridge University Press.
Davison, C. (2004). The contradictory culture of teacher-based assessment: ESL teacher
assessment practices in Australian and Hong Kong secondary schools. Language
Testing, 21, 305-334.
Edelenbos, P. & Kubanek-German, A. (2004). Teacher assessment: The concept of
diagnostic competence. Language Testing, 21, 259-283.
Kramsch, C. (1995). The cultural component of language teaching. Language, Culture
and Curriculum, 8, 83-92.
Kress, G. (2003). Literacy in the new media age. London: Routledge.

29
Kress, G., & van Leeuwen, T. (1996). Reading images – the grammar of visual design.
London: Routledge.
Lantolf, J. P. (2006). Re(de)fining language proficiency in light of the concept of
“languaculture.” In H. Byrnes (Ed.), Advanced language learning: The
contribution of Halliday and Vygotsky (pp. 72-94). London: Continuum.
Lantolf, J. & Poehner, M. (2004). Dynamic assessment in the language classroom.
CALPER Professional Development Document (CPDD) 0411.
URL: http://calper.la.psu.edu/publications.php
Leung, C. (2004). Developing formative teacher assessment: Knowledge, practice and
change. Language Assessment Quarterly, 1, 19-41.
Leung, C. & Lewkowicz, J. (2006). Expanding horizons and unresolved conundrums:
Language testing and assessment. TESOL Quarterly, 40, 211-234.
Miles, M.B., & Huberman, A.M. (1984). Qualitative data analysis. Beverly Hills, CA:
Sage Publications.
Norris, J. M. (2002). Interpretations, intended uses, and designs in task-based language
assessment: Introduction to the special issue. Language Testing, 19, 337-46.
Norris, J.M. (2006). Assessing advanced foreign language learning and learners: From
measurement constructs to educational uses. In H. Byrnes, H. D. Weger-Guntharp
& K. Sprang (Eds.). Educating for Advanced Foreign Language Capacities (pp.
167-187). Washington D.C.: Georgetown University Press.
Poehner, M. & Lantolf, J. P. (2003). Dynamic assessment of L2 development: Bringing
the past into the future. CALPER Working Papers, no. 1, October 2003. URL:
http://calper.la.psu.edu/publications.php
Rea-Dickins, P. (2004). Understanding teachers as agents of assessment. Language
Testing, 21, 249-258.
Rea-Dickins, P. (2008). Classroom-based language assessment. In N. Hornberger & E.
Shohamy. (Eds.), Encyclopedia of language and education (pp. 257-272). New
York: Springer.
Rea-Dickins, P. & Gardner, S. (2000). Snares and silver bullets: Disentangling the
construct of formative assessment. Language Testing, 17, 215-243.
Shohamy, E. (1998). Evaluation of learning outcomes in second language acquisition: A
multiplism perspective. In H. Byrnes (Ed.), Learning foreign and second
languages: Perspectives in research and scholarship (pp. 238-61). New York:
MLA.
Shohamy, E. (2004). Assessment in multicultural societies: Applying democratic
principles and practices to language testing. In B. Norton & K. Toohey (Eds.),
Critical pedagogies and language learning (pp. 72-93). Cambridge: Cambridge
University Press.
Shohamy, E. (2006). Rethinking assessment for advanced language proficiency. In H.
Byrnes, H. D. Weger-Guntharp & K. Sprang (Eds.). Educating for advanced
foreign language capacities (pp. 188-208). Washington D.C.: Georgetown
University Press.
Shohamy, E. & Inbar-Lourie, O. (2006a). Assessment of advanced language proficiency:
Why performance-based tasks? CALPER Professional Development Document
(CPDD) 0603. URL: http://calper.la.psu.edu/publications.php

30
Shohamy, E. & Inbar-Lourie, O. (2006b). The Language Assessment Process: A
“Multiplism” Perspective. CALPER Professional Development Document
(CPDD) 0605. URL: http://calper.la.psu.edu/publications.php
Solano-Flores, G. & Trumbull, E. (2003). Examining language in context: The need for
new research and practice paradigms in the testing of English-language-learners.
Educational Researcher, 32, 3-13.
Swain, M. (2001). Examining dialogue: Another approach to content specification and to
validating inferences drawn from test scores. Language Testing , 18, 275-302.
Torrance, H. & Pryor, J. (1998). Investigating formative assessment: Teaching, learning
and assessment in the classroom. Buckingham: Open University Press.
Weigle, S. C., & Jensen, L. (1997). Issues in assessment for content-based instruction. In
M. A. Snow & D. M. Brinton (Eds.), The Content-based classroom: Perspectives
on integrating language and content (pp. 201-212). White Plains, NY: Longman.

31
APPENDIX
Survey on Assessment Practices of Teachers of Advanced Language
Learners

(Participant Recruitment Letter)

Dear Language Educator,

We would like to ask your help in supplying information about how you assess the
proficiency of your advanced foreign language learners and what some of your
perceptions are about advanced language proficiency.

The survey is part of a project conducted by the Center for Advanced Language
Proficiency Education and Research (CALPER), a National Foreign Language
Resource Center at the Pennsylvania State University.

Thank you very much for your time and cooperation.

Sincerely,
Elana Shohamy, Ph.D.
Ofra Inbar, Ph.D.

CALPER
The Pennsylvania State University
5 Sparks Building
University Park, PA 16802-5203
Tel.: (814) 863-1212

32
IMPLIED INFORMED CONSENT FOR SOCIAL SCIENCE RESEARCH
The Pennsylvania State University

Title of Project: “Assessment Practices of Teachers of Advanced Language Learners”

Principal Investigator: Center for Advanced Language Proficiency Education and
Research (CALPER)

The purpose of this survey is to gather information on the practices and procedures of
teachers to assess foreign language learners. Results of this survey might provide
better understanding of how advanced language learners are assessed.

You will be asked 48 questions.

Your participation in this online survey is voluntary, usernames or remote computer

names will not be retained, stored, or used in data analysis or reporting by those
involved in the data analysis and reporting. You can stop at any time by exiting
the survey.

The survey does not ask for any information that would identify who the responses
belong to. Your confidentiality will be kept to the degree permitted by the
technology used. No guarantees can be made regarding interception of data sent
via the Internet by any third party.

The Office for Research Protections and the Social Sciences Institutional Review Board
of the Pennsylvania State University may review records related to this project.

You can ask questions about this research. Contact Gabriela Appel at email:
[email protected] or phone (814) 863-1212. If you have questions about your
rights as a research participant, contact the Pennsylvania State University’s Office
for Research Protections at (814) 865-1775.

You must be 18 years of age or older to take part in this on-line survey.

Clicking the “I agree” button below, implies that you have read the information and
consent to take part in this research.

This informed consent form was reviewed and approved by the Office of Research
Protections (IRB #21705) at the Pennsylvania State University on 10-17-2005. It
will expire on 10-04-2006 (DWM).

*1. I agree I disagree

________________________________________________________________

33
2. Are you

Male Female
________________________________________________________________

* 3. Do you currently teach in the U.S.?

Yes, I currently teach in the United States.

No, I currently teach in another country.
No, I am not currently teaching (on leave, sabbatical, retired etc.)

________________________________________________________________

* 4. If you do not teach in the US, in which country do you teach?

___________________________

________________________________________________________________

* 5. When was the last time you taught in the United States?

# of years ago:

1-2 years ago

3-5 years ago
Longer than 5 years ago
________________________________________________________________

* 6. Which language(s) are you a NATIVE speaker of?

Chinese
English
French
German
Japanese
Korean
Russian
Spanish
Other (please specify) ____________________

________________________________________________________________

* 7. Besides your native language(s), which language(s) do you consider yourself an

EXPERT speaker of?

Chinese

34
 English
French
German
Japanese
Korean
Russian
Spanish
Other (please specify) ____________________

________________________________________________________________

* 8. Which language(s) do you teach?

Chinese
English
French
German
Japanese
Korean
Russian
Spanish
Other (please specify) ____________________

________________________________________________________________

* 9. Which best describes the institution at which you teach?

Elementary school
Middle school
High school
Community college
2-year college
4-year college/university
Adult education
Commercial language school
Government
Other (please specify) ____________________

______________________________________________________________________

* 10. Have you had any training in language teaching?

Yes
No

______________________________________________________________________

35
* 11. How did you receive MOST of your training in language teaching?

Coursework at a College/University
Certificate Course/Program in FL Pedagogy
Intensive Courses at a Government Agency
Inservice Training Program at a College/University
Professional Development Workshops, Seminars, and/or Intensive Institutes
Other (please specify) ____________________

________________________________________________________________

* 12. Do you have an academic degree?

Yes
No
________________________________________________________________

* 13. What is the highest academic degree you earned?

B.A. or B.S.
M.A. or M.S.
M.Ed.
Ph.D.
Ed.D.
Other (please specify) ____________________

________________________________________________________________

* 14. Which of the following is the major area of studies related to your highest
degree?

Applied Linguistics and/or Linguistics

Secondary Education
Elementary Education
Foreign Languages / Literatures
Other (please specify) ____________________

________________________________________________________________
* 15. How many years have you been teaching foreign languages?

0 -1 years
2 -5 years
6 -10 years
11 -15 years
16 - 20 years
More than 20 years

36
________________________________________________________________

* 16. How many years have you been teaching ADVANCED foreign language
learners?

0 -1 years
2 -5 years
6 -10 years
11 -15 years
16 - 20 years
More than 20 years

________________________________________________________________

* 17. Typically during the year, what percentage of your teaching load do you teach
each of the following levels? [Note: Sum should not exceed 100%]

% of
Time

100% 75% 50% 25% 0%

Advanced
Intermediate
Beginning

________________________________________________________________

* 18. During the last year, how many courses have you taught where the
curriculum/syllabus can be labeled as ….?

0 1 2 3 4 5 more than 5

Skills-based
Literature-based
Content-based

________________________________________________________________

* 19. Have you ever taught an advanced course with a focus on ….

Yes No

International law
Engineering
Business
Health profession

37
________________________________________________________________

In the following section, we would like you to tell us about your assessment practices.

* 20. Please estimate how many times you typically use each of the following
procedure to assess your advanced students during a SEMESTER.

never 1 -2 3-4 5-6 7-8 more than 8

Performance Tasks
Oral Interviews
Tests
Essays / Compositions
Simulations (e.g. Role Plays)
Presentations
Quizzes

________________________________________________________________

21. Think of ONE of your advanced language courses and tell us how you calculate
your final grade for the course. Enter numerical values between 0 - 100 (for
percentages) and note that the sum cannot exceed 100.

____ Written Tests (incl. midterm and/or final exams)

____ Oral Tests (incl. midterm and/or final exams)
____ Quizzes
____ Essays (or Compositions
____ Portfolios
____ Projects
____ Attendance
____ Other

________________________________________________________________

22. Can you tell us the title of that course?

_________________________________________

________________________________________________________________

* 23. Do you use “portfolios” to assess your advanced learners?

Yes No

________________________________________________________________

38
* 24. What do you typically assess with a “portfolio?”
[check all that apply]

Subject matter knowledge

Integrated skills
Research skills
Grammar
Discourse
Vocabulary
Intercultural competence
Speaking
Writing
Collaboration
Other (please specify) ____________________

________________________________________________________________

* 25. Think of one of the courses in which you use “portfolios”. What percentage
does the portfolio count towards the final grade in that course?

0-5%
5 - 10%
10 - 20%
20 - 30%
30 - 40%
40 - 50%
more than 50%

________________________________________________________________

* 26. Do you use “projects” to assess your advanced learners?

Yes No

________________________________________________________________

* 27. What do you typically assess with a “project”?

[check all that apply]

Speaking
Integrated skills
Collaboration
Research skills
Subject matter knowledge
Listening
Reading

39
 Writing
Presentational skills
Other (please specify) ____________________

________________________________________________________________

* 28. Think of one of the courses in which you use “projects”. What percentage does
the project count towards the final grade in the class?

0-5%
5 - 10%
10 - 20%
20 - 30%
30 - 40%
40 - 50%
more than 50%

________________________________________________________________

* 29. Do you engage your students in “self-assessment?”

Yes
No

________________________________________________________________

* 30. When your students engage in “self-assessment”, can you tell us what your
students typically assess?
[click all that apply]

Their research skills

Their oral presentation
Their progress in learning the language
Their projects
Their portfolios
Their essays and compositions
Other (please specify) ____________________

________________________________________________________________

* 31. Do you engage your students in “peer assessment?”

Yes
No

________________________________________________________________

40
* 32. When your students engage in “peer assessment,” can you tell us what your
students typically assess?
[check all that apply]

Presentations of peers
Projects of peers
Portfolios of peers
Essays and compositions of peers
Other (please specify) ____________________

_________________________________________________________________

* 33. Which skills, do you typically assess with the following assessment procedures?
[check all that apply]

listening reading speaking writing do not use

Quizzes
Tests
Oral Interviews
Compositions or Essays
Performance Tasks
Presentations

________________________________________________________________

* 34. What (vocabulary, grammar, discourse or intercultural competence) do you

typically assess with the following procedures?
[check all that apply]

do not use intercultural competence discourse grammar vocabulary

Quizzes
Tests
Presentations
Compositions or Essays
Performance Tasks
Oral Interviews

________________________________________________________________

* 35. Consider the dichotomy between “language” and “content”: If you think of
your assessment practices in your advanced courses, what would you say you
generally focus on?

I generally focus on assessing Content

41
 I generally focus on assessing Language
I generally focus on assessing Content and Language
I cannot answer that question
_______________________________________________________________

* 36. What best describes the level of training you had in each of the following.

extensive training some training minimal training no training

Diagnostic assessment
Linking assessment to the curriculum
Evaluating written essays and compositions
Constructing classroom tests or quizzes
Developing rubrics
Using standards in assessment
Assessing oral interviews
Evaluating oral presentations
Using portfolios

________________________________________________________________

* 37. Indicate how IMPORTANT each one of the following is for your own
advanced language teaching context.

very important important somewhat important not important

Developing rubrics
Evaluating oral presentations
Constructing classroom tests and quizzes
Creating performance-tasks
Evaluating written essays and compositions
Evaluating portfolios

________________________________________________________________

* 38. Rate how important each of the following assessment PURPOSES is for the
advanced learning context in which you teach.

very important important somewhat important not important

To assess on-going learning progress

To diagnose difficulties
To place into a program or course
To assess end of the semester (or year) achievements
To assign grades
________________________________________________________________

42
* 39. To what extent do external standards, guidelines and/or scales affect your
classroom assessment practices?

to a great extent to some extent to a limited extent not at all

ILR Proficiency scale

National Standards in Foreign Language Education
School Board Standards
ACTFL Proficiency Guidelines
State Standards

40. If ANY of the above Standards or Guidelines affect your assessment, please
specify in what way.

___________________________________________________________

________________________________________________________________

* 41. “Advanced language learners should be able to do the same things in their
second/foreign language as in their first language.”

strongly agree agree neutral disagree strongly disagree

________________________________________________________________

* 42. “An advanced language learner can ….”

strongly agree agree neutral disagree strongly disagree

Read a work of popular fiction

Read newspapers
Read professional journals

________________________________________________________________

* 43. “An advanced language learner can ….”

strongly agree agree neutral disagree strongly disagree

Write academic papers

Participate in a discussion on an academic topic
Give presentations on academic topics

________________________________________________________________
* 44. “An advanced language learner can ….”

43
 strongly agree agree neutral disagree strongly disagree

Understand humor
Use and understand metaphors
Participate in conversations on current events

______________________________________________________________________

* 45. “An advanced language learner can ….”

strongly agree agree neutral disagree strongly disagree

Use complex language structures accurately

Explain grammar to other learners

______________________________________________________________________

* 46. “An advanced language learner can ….”

strongly agree agree neutral disagree strongly disagree

Demonstrate intercultural competence

Describe complex processes
Construct narratives
________________________________________________________________

47. Are there other abilities that in your opinion characterize an “advanced”
learner/user of a language?

Other ____________________
and ____________________
and ____________________
and ____________________
and ____________________
and ____________________
and ____________________

________________________________________________________________

* 48. If you would like to add additional comments about assessment of advanced
learners and/or your conceptualization of what characterizes an advanced
language learner, please feel free to do so.
________________________________________________________________

Thank you very much for your time and cooperation!

44
Results of this survey will become available at the website of the Center for Advanced
Language Proficiency Education and Research (CALPER) once the research is completed.

45