Title English as an International language perception scale : development, validation,

and application
Sub Title 「国際語としての英語」認知評価スケール : 開発, 妥当性検証と応用
Author 中村, 優治(Nakamura, Yuji)
Lee, Ju Seong (John)( )
Lee, Kilryoung
Publisher 慶應義塾大学日吉紀要刊行委員会
Publication year 2018
Jtitle 慶應義塾大学日吉紀要. 言語・文化・コミュニケーション (Language, culture and
communication). No.50 (2018. ) ,p.189- 208
JaLC DOI
Abstract
Notes
Genre Departmental Bulletin Paper
URL https://koara.lib.keio.ac.jp/xoonips/modules/xoonips/detail.php?koara_id=AN10032
394-20181231-0189
慶應義塾大学学術情報リポジトリ(KOARA)に掲載されているコンテンツの著作権は、それぞれの著作者、学会または出版社/発行者に帰属し、その
権利は著作権法によって保護されています。引用にあたっては、著作権法を遵守してご利用ください。

The copyrights of content available on the KeiO Associated Repository of Academic resources (KOARA) belong to the respective authors, academic
societies, or publishers/issuers, and these rights are protected by the Japanese Copyright Act. When quoting the content, please follow the
Japanese copyright act.

Powered by TCPDF (www.tcpdf.org)

English as an International Language Perception Scale:
Development, Validation, and Application

Yuji Nakamura
Ju Seong ( John) Lee
Kilryoung Lee

The issue of English as an international language (EIL) perception has received

considerable attention among EIL researchers and practitioners. However, several
studies in the EIL perception have not provided validity and reliability information
pertaining to the instrument (e.g. a questionnaire), which may have affected the
accuracy and interpretation of data. Further, there is currently no validated EIL
Perception Scale (EILPS) that EIL researchers and practitioners can utilize in an
accessible manner. To address these concerns, this study develops EILPS that is
theoretically underpinned and empirically validated through Exploratory and
Confirmatory Factor Analyses with two large samples of Korean EFL students (N =
277, N = 278). Results provide support for a four-factor solution that encompasses 1)
Current Status of English (CSE); 2) Varieties of English (VE); 3) Strategies for
Multilingual/Multicultural Communication (SMC); and 4) English Speakers’ Identity
(ESI). This study contributes to helping EIL researchers and practitioners employ a
validated instrument for measuring EIL perception in an approachable fashion and,
ultimately, allowing generalization across settings. Research and pedagogical
recommendations — e.g. how EILPS can be applied in research and practice settings
— are also made.

Keywords: English as an International Language (EIL); EIL Perception Scale

(EILPS); Scale Development and Validation; Exploratory Factor Analysis;
Confirmatory Factory Analysis

189
 Introduction: Testing and Measurement Context

As research into English as an international language (EIL) has been increasing

in response to the heterogeneity and pluricentricity of English, the last two decades
have particularly witnessed a huge growth in EIL perceptions and teaching EIL
(TEIL). For example, EIL researchers have conducted studies on perceptions vis-à-
vis different varieties of English in various EFL contexts (e.g. Bernaisch and Koch
2016). At the instructional level, EIL practitioners have also begun to develop and
integrate a variety of EIL activities as instructional interventions in an effort to
enhance EFL learners’ perceptions toward EIL (see Matsuda 2012, 2017).

Measurement Problem Encountered and Purpose

Having reviewed the current research vis-à-vis EIL perceptions, however, one
gap seems particularly noticeable; to date, there is no validated EIL Perception Scale
(EILPS) that EIL researchers and practitioners can utilize in an accessible manner.
This methodological concern needs to be urgently addressed because several
studies in EIL perceptions have not provided validity and reliability information
pertaining to the instrument (e.g. a questionnaire), which may have affected the
accuracy and interpretation of data and, therefore, could limit the applicability to
other studies. Although Nakayama (2015) attempted to resolve this measurement
issue by developing EILPS such as EIL Feasibility Questionnaire, it was narrowly
focused in scope in terms of dimensions of EIL perception (i.e. native speaker myths
and identity) and research participants (i.e. EFL students from one Japanese
university). Further, the instrument is only available in Japanese, which can hardly
be accessible to and usable by other EIL researchers and practitioners.
This research gap has served as the motivation to develop and validate
multidimensional aspects of EILPS, which can be readily used by other researchers
and practitioners for measuring students’ EIL perceptions. From a practical
perspective, this study will allow EIL practitioners to implement EIL pedagogy in the
classroom, administer EILPS as pre- and post-tests, and investigate whether their
students’ EIL perception has improved as a result of their EIL instruction. Hence, it
will help prepare EFL learners become more competent users of English in today’s
multilingual and multicultural contexts (Matsuda 2017). From a research perspective,
it will enable future research to (a) use a parallel type of EILPS, (b) make comparisons
across studies, and (c) advance our understanding of EIL perception on a more

190
 English as an International Language Perception Scale

global scale perspective (e.g. students’ EIL perception levels across different
contexts), which will, in the long run, contribute to a global body of EIL knowledge.

Review of Literature

English as an International Language (EIL)

The concept of EIL has been the subject of research over the last four decades.
Larry Smith first defined EIL as an “international language…used by people of
different nations to communicate with one another” (Smith 1976, 38). He emphasized
that every English user, regardless their “nativeness,” needed to learn English for
international communication, which made English language ‘de-nationalized.’
Parallel to Smith’s concept, Kachru (1976, 1986) challenged the orthodoxies of
English language (e.g. Practor’s [1968] article “The British heresy in TESL”) and
proposed a pluralistic and dynamic (or creative) view of English (e.g. three concentric
circles of English), which led him to pioneering World Englishes (WE) as a paradigm
of research in the study of English. Particularly, Kachru (1985) challenged the
(unchallenged) hegemony of the native speaker model and questioned the
conventional role of native speakers. Scholars working under the paradigm of WE,
who also embraced Kachru’s (1990) ‘Liberation Linguistics,’ perceived the monolithic
and static views of English as a matter of social injustice and make efforts to transform
this phenomenon for the speakers of the ‘other tongue’ (Bhatt 2010). Although the
Kachruvian WE perspective attracted criticism and opposition by provoking heated
debate about standard language, codification, and hegemony (see Kachru 1990;
Quirk 1990), the wind of change has slowly permeated into the field of TESOL, which
influence other related issues of EIL/WE:
• Current status of English (e.g. Crystal 1997, 2010; Graddol 1997, 2006)
• Varieties of English (e.g. Jenkins 2007; Seidlhofer 2011)
• Strategies for Multilingual/Multicultural Communication (e.g. Canagarajah
2007, 2013)
• English Speakers’ Identities (e.g. Phillipson 2009; Widdowson 2003)
EIL/WE1) scholarship influenced the other related fields or ‘critically oriented
scholarship,’ such as Global Englishes (GE), English as a Global Language (EGL),
and English as a lingua franca (ELF), which have also attempted to re-conceptualize

1) The terms EIL (used by Larry Smith) and WE (by Braj Kachru) may seem different (e.g. different
terminologies) but in this study are not considered as mutually exclusive, as both notions share much
in common in terms of their visions and principles. For example, Smith and Kachru established the
International Association for World Englishes (IAWE) and its quarterly journal World Englishes.

191
the traditional approach to ELT and, by extension, raise pedagogical implications for
incorporating an EIL (or the other ‘critically oriented’) perspective into ELT (Matsuda
2017; Selvi 2017).
More recently, to encompass such diverse perspectives under different
terminologies (e.g. GE, EGL, ELF), Matsuda (2017) put forward the broad term EIL
as “a function that English performs in international, multilingual contexts, to which
each speaker brings a variety of English that they are most familiar with, along with
their own cultural frames of reference, and employs various strategies to communicate
effectively” (xiii) (also see Selvi 2017).

EIL perception and its measurement

With an increase in research on EIL, the past two decades have witnessed a
particular surge in non-interventional research on perceptions vis-à-vis different
varieties of English in Outer (Bernaisch and Koch 2016; Bernaisch 2012; Hundt,
Zipp, and Huber 2015; Tan and Tan 2008) and Expanding Circle (Ahn 2014, 2015;
Ahn and Kang 2017; Chiba, Matsuura, and Yamamoto 1995; Matsuda 2003; Ren,
Chen and Lin 2016; Sasayama 2013; Wang 2015; Yu 2010) contexts. On the whole
these studies have shown a strong ‘favoritism’ toward standard variety in comparison
to non-standard or other local varieties; that is, English learners or users in both
Outer and Expanding Circle countries tend to recognize American or British English
as the ‘notion of correctness,’ thereby attaching high prestige to both forms (Ahn
2014). These studies have generally reported that national policies (e.g. high-stakes
English tests) and pedagogical practices (e.g. English teachers’ preference toward
standard English as an appropriate teaching model) have been known to contribute
to this phenomenon.
At the instructional level, an increasing number of EIL researchers and
practitioners have begun to develop and incorporate a variety of EIL activities as
interventions in an effort to enhance students’ perceptions of EIL (see Matsuda
2012, 2017). Although some similar studies did not explicitly explain their theoretical
framework, at the discretion of the authors they seemed deeply grounded in
Matsuda’s (2017) conceptualization of teaching EIL (TEIL) as “preparing our
students to become competent users of English in international contexts” (xvii).
These TEIL undertakings, such as in-class activities (Galloway 2013; Galloway and
Rose 2013), extracurricular projects (Author 2018; Galloway and Rose 2014; Hino
2012, 2017), and technology-enhanced cross-cultural activities (Author et al. 2017; Ke
and Suzuki 2011; Ke and Cahyani 2014) have generally proved to be conducive to
greater exposure to varieties of English and EIL communication.

192
 English as an International Language Perception Scale

Having reviewed the above-cited EIL research, however, one serious concern
has been raised: surprisingly, most studies have failed to provide validity and reliability
information pertaining to the instrument used to assess students’ perceptions of
English language or EIL. Some studies, such as Ahn (2015) and Bernaisch and Koch
(2016), seemed to adopt the questionnaire that was developed by other researchers,
but still did not provide reliability and validity evidence related to the instrument that
was used for their own studies. To compensate for this potential methodological
issue, some studies (Author et al. 2017; Ahn 2014, 2015; Galloway 2013; Matsuda
2003; Ke and Cahyani 2014) have used a mixed-methods approach to investigate the
phenomenon in question. However, it is worth noting that this measurement issue,
especially the use of a questionnaire, may have skewed the accuracy and interpretation
of data and, therefore, limited the applicability to other studies.
Nakayama (2015) attempted to develop the EIL Perception Scale (EILPS) to
address this measurement issue. For instance, in light of Jenkins’s (2005) and
Golombek and Jordan’s (2005) theoretical frameworks, Nakayama (2015) developed
a reliable and validated EIL Feasibility Questionnaire (EILFQ) consisting of three
factors with nine items — namely, native speaker myths (4 items), identity (3 items),
and EIL awareness (2 items). Although the statistical data supported the psychometric
usefulness of EILFQ, all items in the scale were devised in reference to only two
dimensions of EIL (i.e. native speaker myths and identity).
Additionally, the instrument is only available in Japanese language, which can
hardly be accessible to and usable by other EIL researchers and practitioners. To
address this gap, the present study aims to develop and validate EILPS through
Exploratory and Confirmatory Factor Analyses. It will also aim to offer insights into
how EILPS can be applied in research and practice settings, which will enable other
researchers and practitioners to employ EILPS for measuring students’ EIL
perceptions in an approachable fashion.

Methodology

Participants and contexts

This study was conducted in 19 EFL classes of three separate Korean universities
— named with the pseudonyms West University, East University, and Center University
— based on a convenience sampling technique (Bryman 2004). As summarized in
Table 1, two groups of non-overlapping Korean undergraduate students participated
in the study; a total of 277 participants (Group 1) voluntarily provided usable surveys
for the validation of EILPS (EFA); 278 participants (Group 2) provided usable surveys

193
for the cross-validation of EILPS (CFA). All were older than 18 years of age (age
mean = 21.37, ranging from 19 to 30).

Table 1. Demographic features of the participants for EFA and CFA

EFA CFA Total
Category Description Group 1 (N = 277) Group 2 (N = 278) (N = 555)
N % N % N %
University West University 92 33.2 148 53.2 240 43.2
East University 139 50.2 8 2.9 147 26.5
Center University 46 16.6 122 43.9 168 30.3
Gender Female 166 59.9 165 59.4 331 59.6
Male 111 40.1 113 40.6 224 40.4
Grade Freshman 104 37.5 169 60.8 273 49.2
Sophomore 102 36.8 69 24.8 171 30.8
Junior 38 13.8 13 4.7 51 9.2
Senior 33 11.9 27 9.7 60 10.8
Major English 111 40 135 48.6 246 44.3
Humanities 59 21.3 95 34.2 154 27.7
Engineering 78 28.2 47 16.8 125 22.5
Others 29 10.5 1 0.4 30 5.5

Procedure
Data collection occurred from fall 2015 to fall 2017 semester in the 8-step
sequence (DeVellis 2012). Here, we will describe the overall steps for the development
and validation of EILPS in great details as follow:

Step 1: Focus of Measurement

Since no previous study established a validated EILPS (cf. Nakayama 2015), the
authors developed a structured questionnaire as a first step. More specifically, an
extensive literature search was carried out in order to identify potential components
of EIL perception. The second author and a senior librarian (who worked in his
affiliation) carried out the search in EBSCO, Scopus, MLA International Bibliography,
online library catalogs and Google Scholar. The search terms and keywords included
“English as an international language,” “EIL,” “Teaching EIL,” and “EIL perception.”
A manual search of articles in journals (such as Journal of Multilingual and
Multicultural Development, World Englishes, Asian Englishes, The Modern Language
Journal, TESOL Quarterly, and RELC Journal) published over the past four decades
(1976–2017), was also conducted. This time, seminal works2) in the other relevant

2) Seminal works represented through often-cited articles that made a significant impact on the field.

194
 English as an International Language Perception Scale

fields (e.g. GE, EGL, ELF, and WE) were also added based in consultation with
several EIL researchers. As a result, hundreds of papers and books were identified
from these processes. Then, initial screening of titles and abstracts and additional
screening of main texts were conducted, which led us to selecting 31 relevant articles,
book chapters and books on the subject of EIL perception. Some of the major
references are briefly summarized in the Literature Review section.

Step 2: Item Pool Generation

Relevant studies were reviewed to produce an initial pool of 35 EILPS items
(constructed by the authors) that encompassed multi-dimensional scope of the EIL
perception.

Step 3: Format for Measurement

A closed-ended, 5-point Likert scale, ranging from strongly disagree (1) to strongly
agree (5), was chosen because it was widely used for measuring abstract concepts
such as attitudes and beliefs (Rasinger 2008). For aesthetic reasons and the ease of
administration, the online questionnaire software (i.e. Google Forms) was employed.
The responders could easily tick boxes (or answers) on the horizontal layout of the
Likert scales.

Step 4: Expert Review of Item Pool

The modified Delphi technique was employed for an initial development of EILPS
and examination of its content validity. It is a practical method for achieving a
consensus of opinion on a particular topic (e.g. EIL perception) among an expert
group (e.g. EIL researchers and practitioners and measurement specialists) (Hsu
and Sandford 2007). First, the original version of EILPS, which was produced during
the step one to three, was presented to two EIL researchers, seven EIL practitioners,
and three language testing experts to review its items and provide constructive
feedback on EILPS3). Based on their suggestions, we chose to include the most
appropriate items in EILPS. Subsequently, the revised items were distributed to the
panelists repeatedly until reaching a high level of consensus concerning EILPS
among the Delphi participants. As a result of the modified Delphi study processes,
three items were removed, resulting in a total of 32 question items. Then, we
continued revising and improving the EILPS items based on Dörnyei and Csizér’s

3) The criteria for selecting the Delphi participants included individuals who had relevant backgrounds
and experiences on EIL issues and could offer useful and timely feedback in the process of
constructing the questionnaire items (Pill 1971).

195
suggestions (2012, 78): 1) Aim for short and simple items, 2) use simple and natural
language, 3) avoid ambiguous or loaded words and sentences, 4) avoid negative
constructions, and 5) avoid double-barreled questions.

Step 5: Development Administration

Prior to the pilot testing, a quantitative SLA researcher scrutinized the readability of
the EILPS. Through this process, five poorly worded items were deleted, leaving a
total of 28 items. With reference to Dörnyei and Csizér’s (2012) guidelines on pilot
studies (see page 79 for details), two pilot testing sessions were administered in the
fall of 2015 with 186 ESL and EFL university students from Indonesia, Brazil, Japan,
Korea and the USA4). Item analysis was conducted by comparing the means between
groups on each item in consultation with two language testing researchers; if the
p-value was below .02, it was considered as a good item for discriminating between
high and low EIL perception group. If the p-value was above .1, it was considered as
a bad item. Based on the item analysis, ten items (out of 28) were retained, four items
deleted, and the rest of the items were revised, resulting in 24 EILPS items.

Step 6: Further Item Improvement

In the spring of 2016, three EIL researchers, three EFL teachers and five EFL
undergraduate students (i.e. target research participants) were asked to evaluate
how well EILPS items would measure the EIL perception (Hughes 1995). Using a
qualitative content validity method, these groups judged the content validity in terms
of whether the items could actually measure the constructs (or characteristics) of
the EIL perception for the EFL students. They also offered suggestions on the
questionnaire in terms of comprehensibility and clarity of each question item. Some
question items were rephrased based on the feedback and achieved an acceptable
level of content validity.
Additionally, the quantitative SLA researcher suggested that EILPS should
include reverse-coding items (i.e. 24 main items + 24 reversed items = 48 items) to
mitigate response bias. So, we included both positive and negative items in the
questionnaire. For instance, the question Item 2, “Many non-native English-speaking
countries use English as their official or working language today” was revised to,
“Few non-native English-speaking countries use English as their official or working
language today,” which was added to the questionnaire. However, it turned out that
during the pilot tests respondents experienced fatigue due to a longer questionnaire,
4) Forward- and backward-translation technique (e.g. from English to Korean) was applied to ensure
clarity and precision in the translated Korean versions.

196
 English as an International Language Perception Scale

which led to increasing inattention and confusion during the survey. Thus, we
decided not to use the reverse wording in the EILPS. In addition, an Institutional
Review Board (IRB) committee asked us to revise a few question items as some EFL
participants might have difficulty in understanding unfamiliar EIL jargon. In the
subsequent version of EILPS, the questions containing potentially problematic terms
(e.g. Outer Circle countries) were rephrased as plain language (e.g. Hong Kong
English, Indian English, and Singaporean English). Finally, an English editing
specialist turned the syntactically complex items into questions with simple and
concise language to make the final version of EILPS.
In the early summer of 2016, the final pilot testing session was administered
with a total of 217 university EFL students from four Expanding Circle countries
such as Japan (N = 20), Indonesia (N = 40), Brazil (N = 49), and Taiwan (N = 108).
Based on Fabrigar and Wegener’s (2012) recommendations for adequate item
loadings and theoretical alignment, 10 problematic items were removed, which
yielded a total of 14 EILPS items (Author et al. 2017).

Step 7: Exploratory Scale Testing and Development

After going through step one to six, we undertook EFA to extract possible constructs
from EILPS and then CFA to validate the underlying factors (identified by EFA).
From fall 2016 to fall 2017 semester, data were collected from 555 Korean EFL
university students enrolling in 19 EFL classes of three separate Korean universities.
Prior to collecting data, instructors and the second author explained the study (e.g.
the goal, procedures, benefits/risks, security of participants’ data) to the prospective
participants in their classes. If the prospective participants agreed to voluntarily
participate in the study, they were given the online survey URL. Then, they read the
consent form, and, if consented, they completed 14 five-point Likert-scale items (see
Appendix) in the classroom using their digital devices (e.g. laptop, computer, or
smartphone). Cases with missing data were not included in both EFA and CFA.
For EFA analysis, data from Group 1 were used (see Table 1). The sample size
(N = 277) for EFA was considered large (Gorsuch 1974), acceptable (Cattell 1978),
and appropriate (MacCallum, Widaman, Zhang, and Hong 1999). As shown in Table
2, the data were normally distributed because there was no extreme skewness (>2)
and kurtosis (>2) (Field 2009). The sample was also factorable in that Kaiser-Meyer-
Olkin (KMO) measure of sampling adequacy was .89, and Bartlett’s test of sphericity
was significant ( χ2 = 2363.58; df = 91; p < .001) (Kaiser 1960).
Subsequently, EFA was conducted to extract latent constructs and investigate
interrelationships among the items using IBM SPSS Statistics 24. Principal

197
Table 2. Descriptive statistics for EFA (N = 277)
Item Min. Max. M SD Skewness Kurtosis
1 1 5 4.28 0.88 -1.10 0.73
2 1 5 3.97 0.92 -0.70 0.12
3 1 5 4.07 0.89 -0.84 0.38
4 1 5 3.59 1.01 -0.25 -0.52
5 1 5 3.45 1.03 -0.28 -0.33
6 1 5 3.50 0.99 -0.19 -0.30
7 1 5 3.29 1.03 -0.20 -0.37
8 1 5 3.26 1.01 -0.17 -0.32
9 1 5 3.05 1.07 -0.08 -0.48
10 1 5 3.85 0.93 -0.53 -0.02
11 1 5 3.43 0.94 -0.19 -0.13
12 1 5 3.74 0.99 -0.49 -0.18
13 1 5 3.30 1.10 -0.15 -0.70
14 1 5 3.77 0.99 -0.41 -0.54

components analysis with promax rotations was applied because it was expected that
the factors would be interrelated. Eigenvalues higher than 1, examination of the
scree plot, and correlations between four factors were used to determine the number
of factors. Communalities (h2; above .5), primary factor loading (above .5), and
internal consistency (using Cronbach’s alpha) were used as criteria for selecting
items. Additionally, in reference to Fornell and Larcker (1981), convergent and
discriminant validity were assessed through the Composite Reliability (CR; above .8)
and the Average Variance Extracted (AVE; above .5). These results are presented in
Table 3 and 4, respectively.

Step 8: Confirmatory Scale Testing and Refinement

As a final step, CFA was conducted to validate the underlying factors (identified by
EFA) using IBM SPSS Amos 22.0 (a structural equation modeling program). Data
from Group 2 (N = 278) were used for CFA analysis. Values for skewness and kurtosis
were less than 2, indicating the data were normally distributed (Field 2009). Chi-
square test ( χ2/df ratio), the comparative fit index (CFI), goodness of fit index (GFI),
the root mean square error of approximation (RMSEA), and adjusted goodness of fit
index (AGFI) were used to assess the model fit in CFA tests. Generally, χ2/df less
than 3, CFI and GFI above .9, RMSEA between .05 and .08, and AGFI above .85
indicate adequate fit (Baumgartner and Hombur 1996; Bollen 1989; Hu and Bentler
1999). Goodness-of-fit indices for the CFA model are displayed in Table 5.

198
 English as an International Language Perception Scale

Table 3. Results of EFA (N = 277)

Factor structure coefficient (>.5)
Item h2 (>.5)
1 2 3 4
1 .85 .20 .21 .21 .85
2 .87 .17 .18 .16 .84
3 .84 .24 .23 .18 .84
4 .30 .77 .26 .10 .76
5 .21 .74 .27 .08 .67
6 .21 .83 .18 .17 .79
7 .03 .70 .17 .38 .67
8 .20 .18 .85 .08 .81
9 .11 .24 .83 .06 .77
10 .39 .28 .57 .25 .61
11 .18 .22 .83 .20 .80
12 .37 .24 -.03 .68 .65
13 .05 .15 .36 .80 .80
14 .22 .15 .10 .82 .75
Eigenvalues 6.57 1.54 1.31 1.16
Explained variable (%) 46.93 10.97 9.35 8.30
Cumulative variable (%) 46.93 57.90 67.25 75.55 (Total)
Retained items per factor 3 4 4 3 14
Factor name CSE VE SMC ESI
Cronbach’s α (>.7) .91 .85 .87 .78 .91
CR (>.8) .89 .85 .86 .81
AVE (>.5) .73 .58 .61 .59
Note: Primary factor loadings > 0.5 are in bold; CSE = Current Status of English, VE = Varieties of English, SMC =
Strategies for Multilingual/Multicultural Communication, ESI = English Speakers’ Identity; h2 = Communalities, CR =
Composite Reliability, AVE = Average Variance Extracted.

Findings

In this section, the results of EFA will first be presented, which will be followed
by those of CFA.

Explorator y Factor Analysis

EFA on the remaining 14 items yielded the four-factor solution, meeting factor
extraction criteria mentioned above. As shown in Table 3, four eigenvalues (Factor 1
= 6.57, Factor 2 = 1.54, Factor 3 = 1.31, Factor 4 = 1.16) were greater than one. The

199
scree plot (Figure 1) showed an abrupt break before Factor 5 (with eigenvalue <1),
indicating that only first four factors were meaningful to be retained (Cattell 1966). A
four-factor solution was also supported by percentage of the cumulative variance (i.e.
the sum of the four values for variance) accounted for by the set of retained factors
(75.55 %). All communalities (i.e. the sum of the squared loadings for each row) also
scored higher than .5. The rotated factor loadings also reached above .5.
Interpreting the factor loadings, Factor 1 encompasses three items with factor
loadings ranging from .84 to .87. It depicts L2 learners’ perception about the current
status of English in terms of current English usage and users, as well as the effects
of the global spread of English. Thus, Factor 1 is labeled as Current Status of English
(CSE). Factor 2 consists of four items with factor loadings ranging from .70 to .83.
Factor 2 is labeled as Varieties of English (VE), as the four items are about perceptions
toward English in Outer and Expanding Circle countries and English diversity in the
classroom. Factor 3 consists of four items with factor loadings ranging from .57 to
.85, and is labeled Strategies for Multilingual/Multicultural Communication (SMC),
referring to the perceptions toward multilingual and multicultural English users and
strategic competence in multilingual and multicultural contexts. Lastly, Factor 4
includes three items with factor loadings ranging from .68 to .82. Factor 4 is labeled
English Speaker’s Identity (ESI), as the three items are about perceptions toward the
native speaker-nonnative speaker dichotomy and ownership of the English language.

Scree Plot

6
Eigenvalue

0
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Component Number

Figure 1. Scree plot and eigenvalues for sample correlation matrix

Internal consistency (Cronbach’s α) was also satisfactory: .91 for Factor 1 (CSE),

200
 English as an International Language Perception Scale

.85 for Factor 2 (VE), .87 for Factor 3 (SMC), .78 for Factor 4 (ESI), and .91 on the
global scale. Values of CR exceeded the threshold of .8, ranging from .81 to .89,
providing evidence of convergent validity (Fornell and Larcker 1981). AVE also
exceeded the cutoff threshold of .5, ranging from .58 to .73, supporting convergent
validity. Given the high level of reliability, corresponding items of each factor were
merged and averaged for a correlation analysis. As exhibited in Table 4, the square
root of AVE for each factor exceeded the inter-factor correlations (or its correlation
with any other factor), verifying discriminant validity for all factors. In other words,
the inter-correlations between factors were weak so that these factors were
discriminable. On the grounds of these results, the four factors were found to be
valid and reliable with items loaded onto the same factor measuring the same
construct.

Table 4. Correlation matrix and squared correlations between factors (N = 277)

CSE VE SMC ESI
CSE .73 .51** .52** .49**
VE .51** .58 .57** .50**
SMC .52** .57** .61 .44**
ESI .49** .50** .44** .59
The square roots of each AVE value are shown in bold type on the diagonal.
**. Correlation is significant at the 0.01 level (2-tailed).

Confirmator y Factor Analysis

At the last stage, CFA was conducted on the factors obtained in EFA in order to
cross-validate between the EFA constructs and a new sample (i.e. Group 2; N = 278),
as summarized in Table 1.

Table 5. Goodness-of-fit indices for the CFA model (N = 278)

χ2/df CFI GFI RMSEA AGFI
Model 1.85 .94 .95 .06 .90
Recommended value ≤3 ≥ .9 ≥ .9 .05 – .08 ≥ .85
Result Good Good Good Good Good

Table 5 presents the goodness-of-fit indices of the model from the CFA. The
table shows a good model-fit within recommended cut-off values, representing χ2/df
(1.85), CFI (.94), GFI (.95), RMSEA (.06), and AGFI (.90). Consequently, all indices
achieved adequate cut-off levels. Hence, the CFA results for the four-factor model

201
with the 14 items based on the existing EILPS demonstrated a good model fit. Figure
2 illustrates the fitted model for the four-factor structure of EILPS.

.44 CSE1
.67
CSE CSE2
.58
CSE3
.73
.46 VE1
.44
VE2
.76 VE .58
VE3
.62
VE4
.51 .80
.58 SMC1

.37 SMC2
.80
SMC .75
SMC3
.69

.73 SMC4

.56 ESI1
.46
ESI ESI2
.33
ESI3

Figure 2. Factor structure of EILPS

Insights Gained

The current study aimed to develop a scale to measure EIL perception (i.e.
EILPS) that can be used by EIL researchers and practitioners in an accessible
fashion. The results of EFA generated a 14-item scale comprising four factors.
Additionally, EILPS had good psychometric properties (i.e. internal consistency and
convergent and discriminant validity) with satisfying values for , CR and AVE. Results
of CFA confirmed the four-factor structure of EFA. Specifically, the first factor (i.e.
CSE) gauges perceptions toward the current status of English. We speculate that
informants who score high on this factor are highly aware of current English uses
and users, plus the effects of the global spread of English. The second factor (i.e. VE)
measures perceptions toward varieties of English. We posit that respondents who
score high on this factor are likely to have positive perceptions toward English in
Outer and Expanding Circle countries and English diversity in the classroom. The
third factor (i.e. SMC) assesses perceptions toward strategies for multilingual and

202
 English as an International Language Perception Scale

multicultural communication. We speculate that individuals who score high on this

factor tend to have positive perceptions toward multilingual and multicultural English
users and strategic competence in multilingual and multicultural contexts. Finally,
the fourth factor (i.e. ESI) measures perceptions of English speakers’ identity. We
postulate those who score high on this factor are often positioned against the native
speaker-nonnative speaker dichotomy and take an ownership of his or her English.

Conclusion: Implications for Measurement Users

Informed by these results, EILPS can be considered a reliable and valid measure
of EIL perception, which can be of much use in research and practice settings. From
a research perspective, the study provides a practical and validated instrument for
objectively measuring EIL perception. It is practical due to its easy administration.
For instance, based on our anecdotal experience, participants could complete the
test within five minutes by employing a given URL link (e.g. using Google Forms)
using their own digital devices. Students’ responses were then automatically saved
into a Google Sheets spreadsheet in which researchers and teachers were able to
download and analyze the data.
In addition, a valid and reliable EILPS will allow EIL researchers to gauge EIL
perceptions in a scientifically convincing way. For example, EIL researchers, who
used to rely on or self-made or other researchers’ questionnaires without testing
validity and reliability, can employ EILPS to assess students’ EIL perceptions and
thus obtain more accurate data for better analysis of the results. Moreover, as Ren et
al. (2016) have indicated, a comparative study about students’ perceptions of EIL in
cross-cultural contexts is quite limited in the current literature. In this regard, this
study could be a feasible starting point to stimulate such comparative studies. Putting
differently, if EILPS is validated in different cross-cultural contexts, it can make it
possible to conduct a comparative study regarding how similarities or differences in
four dimensions of EIL perception play out across different contexts. For example,
the author and co-author (2018) investigated EIL perceptions of Taiwanese and
Korean non-English majors by means of EILPS. It was found that both groups
generally held positive perceptions toward all four aspects of EIL (i.e. CSE, VE, SMC,
and ESI). However, significant statistical differences existed in VE and SMC between
two groups. The authors attributed this phenomenon to different sociopolitical
context and context-specific educational factors. Therefore, a comparative study
using a validated EILPS can help us understand this issue from more diverse
perspectives and, ultimately, allow generalization across contexts.

203
 From a practical perspective, EILPS can be useful for EIL practitioners to gauge
students’ EIL-related components (e.g. EIL awareness and strategy) that contribute
to (or undermine) its development. Specifically, those who implement EIL pedagogy
in the classroom can administer the EILPS as pre- and post-tests and investigate
whether their students’ EIL perception has improved as a result of their EIL
instruction, which will, consequently, help prepare EFL learners to become more
competent users of English in today’s multilingual and multicultural contexts
(Matsuda 2017). For example, our previous project (author et al. 2017) measured the
pedagogical effectiveness of the videoconference-embedded classroom on enhancing
students’ EIL perception. Although a mixed-methods approach (i.e. questionnaire,
class evaluations, in-class observation) was applied, validity and reliability of the
questionnaire was not reported in that study. Future research of its kind may employ
EILPS as a viable way to improve the accuracy and interpretation of data. Finally,
unlike Nakayama’s (2015) study, the EILPS questionnaire is available in English with
four dimensions of EIL (see Appendix), which can be readily accessible to and usable
by other EIL researchers and practitioners. When reporting such findings using the
validated EILPS, it will also enrich our understanding of the status of EIL perception
on a more global scale perspective, which will contribute to a global body of EIL
knowledge.
Despite its novelty and empirical evidence regarding the reliability and validity
of EILPS, there are limitations that should be acknowledged. Although a sample of
555 is considered a large number of participants when conducting factor analysis
(Kline 2000), these participants were all Korean ethnicities, mostly consisting of
freshman and sophomore (N = 444, 80%). Non-random, homogeneous selection of
samples or convenience samples may lose its representativeness of Korean EFL
students, which could ultimately affect generalizability of results (Bryman 2004).
Additionally, due to this particularity of the participants, the results of this study may
not be entirely applicable to other EFL contexts. Thus, a future research may benefit
from recruiting Korean EFL students from other grade levels as well as participants
with diverse linguistic and cultural backgrounds.
To conclude, we can measure changes in the language or linguistic ability of
students (e.g. speaking ability or listening ability) using standard or commercialized
tests such as TOEFL iBT or IELTS. However, the non-language aspects of the students
such as perception or attitude change have been rarely examined in a scientifically
convincing fashion. Therefore, EILPS can be of a great use for classroom language
teachers as well as researchers who need or require objective data. As more and more
EIL researchers and educators engage in EIL research and the amount of such

204
 English as an International Language Perception Scale

research is accumulating quickly, we hope the current study contributes to theory-

building process in the area of EIL by offering a validated EILPS.

References
Ahn, H. 2014. “Teachers’ attitudes towards Korean English in South Korea.” World Englishes 33 (2): 195–
222.
Ahn, H. 2015. “Awareness of and attitudes to Asian Englishes: A study of English teachers in South Korea.”
Asian Englishes 17 (2): 132–151.
Ahn, S. -Y., and H. -S. Kang. 2017. “South Korean University students’ perceptions of different English
varieties and their contribution to the learning of English as a foreign language.” Journal of
Multilingual and Multicultural Development. 38(8): 712–725.
Baumgartner, H., and H. Christian. 1996. “Applications of structural equation modeling in marketing and
consumer research: A review.” International Journal of Research in Marketing 13: 139–161.
Bernaisch, T. 2012. “Attitudes towards Englishes in Sri Lanka.” World Englishes 31 (3): 279–291.
Bernaisch, T., and C. Koch. 2016. “Attitudes towards Englishes in India.” World Englishes 35 (1): 118–132.
Bhatt, R. 2010. “World Englishes, globalization, and the politics of conformity.” In Contending with
globalization in World Englishes, edited by M. Saxena and T. Ominiyi, 93–112. London: Multilingual
Matters.
Bollen, K. A. 1989. Structural equations with latent variables. New York: Wiley.
Bryman, A. 2004. Social research methods. Oxford: Oxford University Press.
Canagarajah, S. 2007. “Lingua franca English, multilingual communities, and language acquisition.” The
Modern Language Journal 91: 923–939.
Canagarajah, S. 2013. Translingual practice: Global Englishes and cosmopolitan relations. London:
Routledge.
Cattell, R. B. 1966. “The scree test for the number of factors.” Multivariate Behavioral Research 1: 245–276.
Cattell, R. B. 1978. The scientific use of factor analysis in behavioral and life sciences. New York: Plenum.
Chiba, R., H. Matsuura, and A. Yamamoto. 1995. “Japanese attitudes toward English accents.” World
Englishes 14 (1): 77–86.
Crystal, D. 1997. English as a global language. Cambridge: Cambridge University Press.
Crystal, D. 2010. The Cambridge encyclopedia of language. Cambridge: Cambridge University Press.
DeVellis, R. F. 2012. Scale development: Theory and applications. 3rd ed. Thousand Oaks: Sage.
Dörnyei, Z., and K. Csizér. 2012. “How to design and analyze surveys in SLA research?” In Research
methods in second language acquisition: A practical guide, edited by A. Mackey and S. Gass, 74–94.
Malden: Wiley-Blackwell.
Fabrigar, L. R., and D. T. Wegener. 2012. Exploratory factor analysis (Understanding Statistics). New York:
Oxford University Press.
Field, A. 2009. Discovering statistics using SPSS. London: Sage.
Fornell, C., and D. F. Larcker. 1981. “Structural equation models with unobservable variables and
measurement error: Algebra and statistics.” Journal of Marketing Research 18 (3): 382– 388.
Galloway, N. 2013. “Global Englishes and English Language Teaching (ELT) - Bridging the gap between
theory and practice in a Japanese context.” System 41: 786–803.
Galloway, N., and H. Rose. 2013. “They envision going to New York, not Jakarta”: The differing attitudes
toward ELF of students, teaching assistants, and instructors in an English-medium business program
in Japan.” Journal of English as a Lingua Franca 2(2): 229–253.

205
Galloway, N., and H. Rose. 2014. “Using listening journals to raise awareness of global Englishes in ELT.”
ELT Journal 68 (4): 386–396.
Golombek, P., and S. Jordan. 2005. “Becoming “black lambs” not “parrots”: A poststructuralist orientation
to intelligibility and identity.” TESOL Quarterly 39 (3): 513–533.
Gorsuch, R. L. 1974. Factor analysis. Philadelphia: Saunders.
Graddol, D. 1997. The future of English. London: British Council.
Graddol, D. 2006. English next: Why global English may mean the end of ‘English as a foreign language’.
London: British Council.
Hino, N. 2012. “Participating in the community of EIL users through real-time news: Integrated Practice
in Teaching English as an International Language (IPTEIL).” In Principles and practices of teaching
English as an international language, edited by A. Matsuda, 183–200. Bristol: Multilingual Matters.
Hino, N. 2017. “Training graduate students in Japan to be EIL teachers.” In Preparing teachers to teach
English as an international language, edited by A. Matsuda, 87–100. Bristol: Multilingual Matters.
Hsu, C., and B. A. Sandford. 2007. “The Delphi technique: Making sense of consensus.” Practical
Assessment, Research & Evaluation 12 (10): 1–8.
Hu, L., and P. M. Bentler. 1999. “Cutoff criteria for fit indexes in covariance structure analysis: Conventional
criteria versus new alternatives.” Structural Equation Modeling 6 (1): 1–55.
Hughes, A. 1995. Testing for language teachers. Cambridge: Cambridge University Press.
Hundt, M., L. Zipp, and A. Huber. 2015. “Attitudes in Fiji towards varieties of English.” World Englishes 34
(4): 688–707.
Jenkins, J. 2005. “Implementing an international approach to English pronunciation: The role of teacher
attitudes and identity.” TESOL Quarterly 39 (3): 535–543.
Jenkins, J. 2007. English as a lingua franca: Attitude and identity. Oxford: Oxford University Press.
Kachru, B. B. 1976. “Models of English for the third world: White man’s linguistic burden or language
pragmatics?” TESOL Quarterly 10 (2): 221–239.
Kachru, B. B. 1985. “Standards, codification and sociolinguistic realism: The English language in the
outer circle.” In English in the world: Teaching and learning the language and literatures, edited by R.
Quirk and H. Widdowson, 11–30. Cambridge: Cambridge University Press.
Kachru, B. B. 1986. The alchemy of English: The spread, functions, and models of non-native Englishes. New
York: Pergamon Institute of English.
Kachru, B. B. 1990. “Liberation linguistics and the quirk concern.” English Today 7 (1): 1–13.
Kaiser, H. F. 1960. “The application of electronic computers to factors analysis.” Educational and
Psychological Measurement 20: 141–151.
Ke, I. C., and H. Cahyani. 2014. “Learning to become users of English as a Lingua Franca (ELF): How ELF
online communication affects Taiwanese learners’ beliefs of English.” System 46: 28–38.
Ke, I-C., and T. Suzuki. 2011. “Teaching global English with NNS-NNS online communication.” The
Journal of Asia TEFL 8 (2): 169–188.
Kline, P. 2000. The handbook of psychological testing, 2nd ed. London: Routledge.
MacCallum, R. C., K. F. Widaman, S. Zhang, and S. Hong. 1999. “Sample size in factor analysis.”
Psychological Methods 4: 84–99.
Matsuda, A. 2003. “The ownership of English in Japanese secondary schools.” World Englishes 22 (2):
483–496.
Matsuda, A., ed. 2012. Principles and practices of teaching English as an international language. Bristol:
Multilingual Matters.
Matsuda, A., ed. 2017. Preparing teachers to teach English as an international language. Bristol: Multilingual
Matters.

206
 English as an International Language Perception Scale

Nakayama, T. 2015. “An EIL awareness questionnaire: Development and validity in a Japanese context.”
実践女子大学文学部紀要（The Faculty of Letters in Jissen Women’s University Annual Reports of
Studies) 57: 11–19.
Phillipson, R. 2009. Linguistic imperialism continued. New York: Routledge.
Pill, J. 1971. “The Delphi method: Substance, context, a critique and an annotated bibliography.” Socio-
Economic Planning Science 5: 57–71.
Prator, C. 1968. “The British heresy in TESL.” In Language problems in developing nations, edited by J.
Fishman, C. Ferguson, and J. Das Gupta, 11–30. New York: John Wiley & Sons.
Quirk, R. 1990. “Language varieties and standard language.” English Today 21: 3–10.
Rasinger, S. M. 2008. Quantitative research in linguistics: An introduction. New York: Bloomsbury.
Ren, W., Y. S. Chen, and C. Y. Lin. 2016. “University students’ perceptions of ELF in mainland China and
Taiwan.” System 56: 13–27.
Seidlhofer, B. 2011. Understanding English as a lingua franca. Oxford: Oxford University Press.
Selvi, A. F. 2017. “Preparing teachers to teach English as an international language: Reflections from
Northern Cyprus.” In Preparing teachers to teach English as an international language, edited by A.
Matsuda, 114–124. Bristol: Multilingual Matters.
Smith, L. E. 1976. “English as an international auxiliary language.” RELC Journal 7 (2): 38–43.
Tan, P. K. W., and D. K. H. Tan. 2008. “Attitudes towards non-standard English in Singapore.” World
Englishes 27 (3/4): 465–479.
Wang, W. 2015. “Teaching English as an international language in China: Investigating university teachers’
and students’ attitudes towards China English.” System 53: 60–72.
Widdowson, H. G. 2003. Defining issues in English language teaching. Oxford: Oxford University Press.
Yu, Y. 2010. “Attitudes of Learners toward English: A Case of Chinese College Students.” PhD diss., Ohio
State University, USA

Appendix
Factors and Items of EIL Perception Scale
Current Status of English (CSE)
(CSE 1) English is used today as an international language to communicate effectively with people from
around the world.
(CSE 2) Many non-native English-speaking countries currently use English as their official or working
language.
(CSE 3) English is the language of business, culture, and education around the world today.

Varieties of English (VE)

(VE 1) Different varieties of English, such as Hong Kong English, Indian English, and Singaporean
English, are acceptable today.
(VE 2) Teachers can use English listening materials that are recorded by people who have different kinds
of English accents.
(VE 3) Different varieties of English, such as Indonesian English, Taiwanese English, and Japanese
English, are acceptable today.
(VE 4) Teachers can include the interaction between non-native and non-native English speakers (e.g.,
Indonesian-Japanese speakers) in English listening materials.

Strategies for Multilingual/Multicultural Communication (SMC)

(SMC 1) I can adjust my conversational style according to my interactions with people of other cultural

207
backgrounds.
(SMC 2) I can explain my own culture and customs clearly in English to people from other cultures.
(SMC 3) I am open-minded about accepting speaking/pronunciation patterns that are different from
those of my home country.
(SMC 4) I can behave appropriately according to English users I speak with.

English Speakers’ Identity (ESI)

(ESI 1) English teachers should not push me to speak like a “native” English speaker.
(ESI 2) I don’t mind if people laugh at my English accent when I speak because it is my own English.
(ESI 3) It is unnecessary to speak like American or British English speakers as long as my English is
intelligible (or understandable) to others.

208