See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/326300862

ELT Materials Evaluation: A System and Criteria

Article  in  Theory and Practice in Language Studies · July 2018

DOI: 10.17507/tpls.0807.11

CITATIONS READS

0 6,730

1 author:

Ali Işık
Istinye Universitesi
25 PUBLICATIONS   164 CITATIONS   

SEE PROFILE

All content following this page was uploaded by Ali Işık on 31 July 2018.

The user has requested enhancement of the downloaded file.

ELT Materials Evaluation: A System and Criteria
Ali
Istinye University, Istanbul, Turkey

Abstract This study aims at proposing a 3-stage materials evaluation system and separate checklists for
screening and detailed analysis and testing their effectiveness. The checklists were developed at two stages:
First they were developed by 113 trainees as the requirement of the materials evaluation training program and
evaluated by 43 ELT teachers. Later both the system and the checklists themselves were piloted and tested in
real-life by 11 ELT teachers when adopting ELT materials for their students. The data was collected through
5-point Likert scale questionnaires. The results indicated that the system allows for the quick disqualification
of the inappropriate ones and saves enough time to focus on the candidate ones in detail. The checklists
themselves are found to be both reliable and valid.

Index Terms ELT materials, materials evaluation checklists, language education

I. INTRODUCTION
ELT materials establish the backbone for English language education (Tomlinson & Masuhara, 2017). Generally
speaking in many language programs they function as the de facto syllabus and dictate what to teach, in what order to
teach, even the density and pace of teaching (AbdelWahab, 2013; Allen, 2015; Garton, & Graves, 2014; Mishan &
Timmis, 2015). Thus, they provide a common ground for both learners and teachers in an institution and besides
determining the content they also control what to and how to teach and test. Not only do they regulate content but also
how to implement and realize language education (Harwood, 2014; McGrath, 2013). Each ELT material is based on a
language teaching philosophy (approach) which guides how to realize its principles in actual use via materials (Richards,
2006). In other words, ELT materials are more than content, they impose a teaching methodology on both teachers and
aching
methodology get adopted together with them, too. It is highly likely to say that the degree to which the methodology is
in congruence with the overall goal of a language program, decides the success of a language education program
(Harwood, 2010). Likewise, they are of a great help for especially novice teachers (Garton & Graves, 2014; López-
Medina, 2016). If they are based on a sound methodology, they are likely to guide and scaffold teachers to be efficient
teachers. The tasks and teaching tips help them get to be equipped with the knowledge and skills to be self-sufficient,
even maybe, exploratory teachers. They also orient learners about how to learn a second/foreign language through the
tasks they provide. Even some ELT materials have specific tasks on language learning strategies to train learners to
become successful learners. Furthermore, they are the main source of language input for learners. Nowadays learners
can have access to ample amount of input on the internet (Allehyani, Burnapp & Wilson, 2017) however, being exposed
to the relevant materials at the appropriate level of cognitive and linguistic difficulty is a problem. The ELT materials,
in that sense, ensures to provide learners with the appropriate materials. ELT materials are accompanied by the
periphery; audios, videos, workbooks, assessment packs, projects packs, I-tools, etc. which all create a variety of texts
and tasks and enrich languages activities both in and out of the classroom. This variety also helps to address different
needs and interests and individualize activities especially outside the class. I-tool applications engage students and
facilitate comprehension of the texts and tasks. Together with the I-tools, online applications take learners beyond the
boundaries of schools and integrate them with the rest of the world (Allen, 2015; Kukulska-Hulme, Norris & Donohue,
2015). Moreover, they pave the way for creating a global community in which learners get engaged in real, authentic
interaction with others. Some materials, in addition, are conducive for self-regulated learning. They create online
platforms for both teachers and learners by which teachers follow what activities their students do, how much time they
spend doing those and how successful they are in task completion. The same feedback is also given to learners
themselves. Those platforms also provide progress checks at the defined intervals. Both the activity reports and progress
checks provide invaluable feedback to teachers and learners to diagnose their weaknesses and strengths and behave
accordingly (Tomlinson & Masuhara, 2017). To sum up, ELT materials are essential for any language education
programs and they set the ground for the language teaching/learning process (McGrath; 2006, 2016; Tomlinson, 2008).
The effort to emphasize the pivotal role of materials evaluation has abounded (Tomlinson, 2013). It has aimed to
guide the stake holders to consider it a data-driven, disciplined and informed decision-making process. The pioneers in
the field not only set the theoretical ground but also carried out research and came up with practical implications, even
with materials evaluation checklists (Brown, 1995; Cunningsworth, 1995; Dubin, & Olshtain, 1986; Ellis, 1997; Graves,
1996, 2000; Grossman, & Thompson, 2008; Guyer, & Peterson, 1988; Hargreaves,1989;Harmer, 2003, Hirsch, 1988;
Hutchinson& Waters, 1987; Johnson, 1989; Krahnke, 1987; Letter, 2000; Littlejohn & Windeatt,1989; McDonough &
-Amato, 1988; Sheldon, 1987; Skierso, 1991;
Tomlinson, 2003; Willis, 2000). The theoretical and practical knowledge and data about materials have been
accumulated and raised the awareness of stake holders about the issue. That knowledge has spread out and contributed
to make the materials evaluation a global concern (McGrath, 2016; McDonough, et al., 2013; Tomlinson & Masuhara,
2017). In turn the research on it has also flourished globally and researchers all around the world have contributed to the
ever-growing body of knowledge in the field (Garton & Graves, 2014). That concern even gave birth to the birth of a
professional association, The Materials Development Association (MATSDA) which has offered courses, organized
conferences, and provide services. The knowledge accumulated in the field, however, needed to be streamlined and put
into practice to meet local needs.
As the knowledge about materials evaluation has developed and evolved, so has materials development (Garton &
Graves, 2014; McGrath, 2016). ELT materials are either developed locally or adopted from among the ones which are
on the market developed by the international publishers. The ideal one is developing materials unique for each program;
custom-made ones arise from the needs analysis and context analysis to realize the goal(s) of a particular language
education program. However, it is a huge task requiring expertise, time, energy, money, and team work. Since materials
development is to be a data-driven, disciplined and systematic process, setting the goal(s) of a program, carrying out
need and context analysis, preparing interim goals, objectives, determining content and related tasks and following the
necessary steps to develop materials, first of all, necessitate expertise (McGrath, 2006; 2016). Materials developers are
to be equipped with required knowledge and skills about the materials development and language teaching. It is a
demanding task a single person cannot handle, thus, it requires a team. Moreover, they are needed to be accompanied by
a team of experts, psychologists, applied linguists, pedagogues, information technology specialists, designers, script
writers, directors, etc. All these obviously call for a budget. In short, it is a long process which is time, money and
energy consuming and much above the shoulder of a teacher. The logical one is to adopt ELT materials and adapt them
for the context in which they are implemented. The adopting process is not an easy one, it necessitates a data-driven,
disciplined decision making. Teachers are to collect data through needs and context analysis and then implement
evaluation criteria to choose the most appropriate materials (Kostka & Bunning, 2016). Since the ELT materials
developed by publishers, especially international ones, are developed for a broad spectrum of learners, comprehensive,
well-designed criteria get in action to choose the best. Evaluation criteria need to be customized as well to serve the
specific needs of specific contexts. Obviously, evaluation criteria include items of global use as well as local one; in
other words, the criteria must be customized for local use
In Turkey, as in other countries in the world, English is the key for a job, promotion, more income, and pursuing an
academic career. There is an English fever (Krashen, 2006) and a great demand to master English. In formal education,
the English education starts at the kindergarten and continues at the tertiary level. Besides, there are plenty of private
institutions offering English courses. In short, huge amount of time, money, and energy is spent on English, thus,
learners are to be provided with appropriate materials. Naturally it can be provided with a sound evaluation process. To
make sound, informed, disciplined decisions there is a need for both comprehensive evaluation criteria and a systematic
approach for materials evaluation. An approach to evaluate the materials are important because there are plenty of ELT
materials in the market and it is impossible to evaluate all the possible materials in detail. To be practical the candidate
materials are needed to be screened from among the possible ones and evaluated in detail.
To the knowledge of the researcher, there are only few studies on proposing ELT materials evaluation checklists and
testing their efficiency in
-Medina (2016) developed ELT materials evaluation checklists.
Reinders & Lewis (2006) developed a checklist to evaluate self-access materials and López-Medina (2016) for content

by almost any ELT materials. There is definitely a need for a comprehensive checklist as well. In addition, its validity
and reliability must be verified and data about its real-life use is to be provided. To sum up, the literature review has
indicated the pivotal role of materials evaluation to come up with the most effective ones for each context. Although
materials evaluation has aroused global concern, the research on materials evaluation checklists and proposing new
checklists is not so common. Besides, since each context is unique it is mandatory to customize the evaluation process
for each specific context. Moreover, the proposed checklists are no different from one another and a systematic
approach to the ELT materials evaluation process has not been offered to make it more efficient and save time to focus
more on the possible candidate ELT materials. In short, this study aims at proposing a 3-stage ELT materials evaluation
system to create an efficient evaluation process and suggest detailed comprehensive checklists that can be exploited
while forming ones for each specific ELT context.

II. METHODOLOGY

A. Aim
The aim of the study is to develop comprehensive materials evaluation checklist in a two-stage fashion and to test its
effectiveness.
B. References
 Three groups of subjects participated in the study. The first group consisting of 113 trainees prepared the items for
the criteria. The second group consisting of 43 English teachers tested the effectiveness of the items in the criteria. The
third group, 11 teachers piloted the checklists.
C. Treatment
A 12-week training program was designed for the trainees who met three hours a week (see Appendix 1). In the first
three weeks they study the articles on ELT materials to have a general perspective about the topic. The following three
weeks they focused on materials evaluation and examined checklists. Each week, the trainer divided the class in groups
and each group raised a real-life issue about topics of the week and offered solutions for them. In the 7th and 8th weeks
they worked in groups to develop their own checklists. The trainer functioned as a member of each group and provided
continuous feedback. At the end of the 8th week the groups shared their checklists with the others online and came to
class evaluating them in the 9th week. In that week each group presented their checklists to the class and first other
trainees (classmates) and then the trainer gave feedback about them. Upon receiving the feedback the group revised
their checklists during the week after the class. In the 10th week in the class, the same groups started to evaluate the
ELT materials used in their schools. The trainer again worked with each group. The process continued in the 11th week.
At the end of the week, each group posted their evaluation to the whole class to get feedback about their evaluation. In
the 12th week each group presented their evaluations and received feedback from their friends and teachers. Based on
the feedback they received, they were required to revise their projects and submit them to the trainer within two weeks.
The training program lasted two years with the participation of different trainees.
D. The Materials Evaluation System
A three-stage materials evaluation process is adopted; screening stage and a thorough detailed analysis stage. There
are numerous ELT materials on the market it is too time- and energy consuming to evaluate each book in detail. The
two-stage evaluation helps to ease the burd
books to determine the candidate ELT materials for furher detailed analysis. In other words, this stage eliminates the
unqualified ones and delimits the number of ELT materials to be evaluated in the second stage. The evaluation approach
is unique in a sense that it spares enough time to focus on the candidate books in detail in the second stage. The
screening (quick evaluation) checklist is employed in the first, detailed evaluation checklist is used in the second stage.
E. The Item Development Process
The aim is to come up with two checklists, one for the screening to determine the candidate materials, and a
comprehensive one for further detailed evaluation. The subjects started to work with a trainer who has evaluated,
adapted, and developed materials, worked as a freelance materials evaluator for international publishers since 1989, and
offered courses on ELT materials since 1999. After discussing the materials in the course pack in order to form the
theoretical basis of materials evaluation, the trainees grouped in the groups of three or four. Each group was required to
come up with two sets of items one for the quick evaluation checklist one for the comprehensive one. The tasks were
carried out both in and out of the classroom and the trainer acted as a member of each group and provided feedback
when necessary on the spot. The groups worked on their checklists for two weeks and revised them. Finally, they
submitted their checklist to the trainer. The trainer worked on each checklist and created one checklist for screening
(quick evaluation) and one for comprehensive detailed evaluation composed of different subcategories such as external
evaluation, internal evaluation, face validity, approach, teacher-related factors, etc.
After the trainer had come with the checklists, judgmental evaluation was carried out by 43 English teachers who
evaluated the quality of each item in the checklists using a 5 Likert scale. With an open-ended item in the questionnaire
they are also asked what other items they would add to the checklist. In addition, to elicit their ideas about the general
evaluation of the checklist another questionnaire was administered to the teachers. Depending on the feedback from the
teachers, the reliability and validity of the checklists was carried out item by item. Some items were deleted, revised and
added in the checklists. They were revised considering the feedback and suggestions given by the teachers and they
were finalized. In other words, they got ready for piloting- the actual use (see Appendix 2).
During the judgmental evaluation teachers also answered the open-ended item. One teacher indicated that the
deliberate focus on target language culture had to be looked for in materials evaluation. In the same vein, three teachers
demanded native speaker standards in pronunciation. Since English as a Lingua Franca (ELF) approach has been widely
adopted and it is hard to determine which target culture and dialect (American, British, Australian, etc.) are to be taught,
that suggestion was realized in the item related with ELF. Two teachers suggested and item about the size of the
coursebooks and offered A4 size so that they could fit in a student school bag. It was thought about and decided that
size is important and that suggestion was revised and added in the checklist. One teacher offered a hardcover-related
item. Since hardcover might add extra cost for the materials and it is unlikely to find hardcover ones on the market, that
suggestion was disqualified. One teacher offered that materials should have an extra booklet in which the parts in the
materials students are required to write would be transferred there and student materials could be kept clean and used
again and again. As providing compact materials is more convenient, that suggestion was not put into practice while
developing the checklist.
F. Data Collection and Data Analysis
 Use The data were collected at two stages, initial evaluation and piloting. The first step was judgmental and the
second practical. The first stage was carried out with 43 teachers to learn their ideas about the checklists and the
approach. The second stage was actually a real-life one as it was implemented to choose course materials by eleven
teachers. The teachers were supposed to adopt new course materials for the following academic year and they exploited
the approach and the checklists. For both stages the evaluation criteria were given to the teachers and the data was
collected through a five-point Likert. SPSS was used to analyze the data.
G. Validity of the Questionnaires

checklist application in the second investigation. In addition, the expert panel was considered an integral design element
to demonstrate content relevance and representativeness along with the substantive and consequential aspects of validity.
H. Reliability of the Questionnaires
Internal consistency
-item
correlation.

III. RESULTS
First trainees were asked to evaluate the training program and the results are as follows.

TABLE I
THE RESULTS OF THE SURVEY O N THE TRAINING PROGRAM
Highly unsatisfactory Unsatisfactory Neutral Satisfactory Highly satisfactory
Ideas about the training 100%

The data about the checklists were collected at two different stages, judgmental evaluation and practical (piloting)
evaluation.
A. Judgmental Evaluation
The results obtained from the teachers who evaluated the effectiveness of the checklist are summarized in the tables
below:
Screening: Table 2 summarizes how teachers evaluated the screening checklist.
TABLE II
RESULTS OF THE SCREENING
Mean Min. Max. Var. Cronbach Alpha
Screening 3.98 3.51 4.09 0,02 0,93

The reliability for the screening component of the proposed system is found to be within the acceptable range.
B. Detailed Evaluation
The results of the detailed evaluation are presented in two subcategories, external evaluation and internal evaluation.
C. External Evaluation
In Table 3 the external evaluation subcategory results indicate that teachers found it effective. The Cronbach Alpha
values about each external evaluation subcategory are also very high, the reliability of this subcategory is quite high.
TABLE III
THE PROPOSED SYSTEM EVALUATION RESULTS
Highly unsatisfactory Unsatisfactory Neutral Satisfactory Highly satisfactory
It is practical 3% 12% 42% 43%
It is effective 49% 51%
It is time-saving 6% 43% 51%

D. Internal Evaluation
Table 4 indicates that the detailed internal evaluation part is found to be effective by the teachers. The reliability
values of the subcategories of internal evaluation are within the acceptable range.
 TABLE IV
RESULTS OF THE INTERNAL EVALUATION
Mean Min. Max. Var. Cronbach Alpha
Aim 3.86 3.23 4.31 0.95 0.89
Approach 3.13 2.92 3.47 0.03 0.91
Syllabus 3.63 3.17 4.12 0.08 0.83
Linguistic Aspect 3.93 3.76 4.39 0.15 0.91
Teacher-related Factors 3.76 3.39 3.10 0.04 0.93
Student-related Factors 3.50 3.03 4.06 0.08 0.91
Classroom Organization 3.77 3.41 4.04 0.05 0.96
Instructions 4.09 2.93 3.23 0.02 0.93
Content 3.69 2.79 3.56 0.16 0.96
Culture 3.60 2.71 4.36 0.03 0.89
Lexis 3.89 3.37 4.43 0.09 0.91
Skills 3.15 2.95 3.65 0.03 0.87
Unit Format 3.57 3.23 4.12 0.07 0.86
Measurement and Evaluation 4.10 3.87 4.56 0.19 0.84
Software 3.77 3.56 4.09 005 00.93

E. The Overall Evaluation

The teachers are asked to evaluate the materials evaluation system (screening and detailed evaluation) in general. The
results are as follows:
TABLE V
OVERALL EVALUATION RESULTS
Highly unsatisfactory Unsatisfactory Neutral Satisfactory Highly satisfactory
It is practical 6% 46% 48%
It is effective 58% 42%
It is time-saving 3% 9% 52% 36%

F. Practical Evaluation (Piloting) Results

Table 6 presents the results of piloting. It indicates a high reliability of the checklist in general:
TABLE VI
PRACTICAL EVALUATION RESULTS
Cronbach Alpha
Screening 0.93
External evalaution 0.87
Internal evalauation 0.89

G. Evaluation of the Proposed System

The teacher who piloted the checklists during their materials adaptation process were also asked given the
questionnaire to elicit their overall all impression about the materials evaluation approach. Table 7 summarizes the
results.
TABLE VII
THE PROPOSED SYSTEM EVALUATION RESULTS
Highly unsatisfactory Unsatisfactory Neutral Satisfactory Highly satisfactory
It is practical 3% 12% 42% 43%
It is effective 49% 51%
It is time-saving 6% 43% 51%

IV. DISCUSSION
Trainees seem to be satisfied with the content of the training and the tasks they carried out. Since ELT materials are
vital in the language process, participating in such a training and fulfilling relevant tasks fostered their involvement.
The data obtained from the judgmental evaluation indicate that the Cronbach Alpha value for the screening checklist
is very high and the reliability related with the screening is satisfactory. The results of the detailed evaluation are no
different. The data related with the external evaluation indicated that the reliability values range from 0.88 to 0.94. They
are within the acceptable range. The results of the internal evaluation show that the reliability values are also very high,

options and fall above the accepted value. 6% of the teachers are neutral about the practicality of the materials

-
 Practical Evaluation (Piloting) Results indicate that the proposed system is reliable. The reliability related with the
screening checklist is 0.93, external evaluation 0.87, and internal evaluation 0.89. Overall evaluation results show that
generally the teachers either go for the

f effectiveness, 49% of the teachers found the system

-saving quality of the system, while 6% were remained neutral.

and AbdelWahab (2013) tested the validity and reliability of their checklists and revised them in relation to the data as it
was done in this study. Adopting the most appropriate materials in English language is a must, and to do so developing
reliable and valid checklists is a must as well. The research and literature about materials development issue are
flourishing, yet the checklist development, which is one of the vital means of that process, requires more emphasis.

V. CONCLUSION
First of all, the checklists are the brain child of 156 ELT teachers, not a single expert. They collaborated to create the
most sound, practical checklists that can be employed to evaluate ELT materials. This collaboration helped synthesize
different perspectives in two-stage evaluation criteria which could be employed by other practitioners in the field.
In addition, the process through which the trainees study ELT materials, materials evaluation and checklist
preparation and develop checklist for evaluation is unique in itself. The trainees had no ELT materials evaluation
background. First they received theoretical instruction about the topic, got familiar with approaches and discussions and
examined checklists. After getting equipped with enough theoretical knowledge they came up with their own and
employed it to evaluate ELT materials used in their schools. Some teacher may do the same task without having enough
insights into the topic. Thus the training helps trainees have a wider perspective about the topic guides them to make
informed decisions while developing their checklists and evaluating their own materials. Hence, the first implication of
the study is that raising the awareness of teachers about the topic of interest can be an effective strategy to elicit reliable
data about it.
Furthermore, the research indicates that the proposed materials evaluation system and related checklists can be used
effectively for materials evaluation. The results about the reliability are found to be high and both the categories and
subcategories of the checklists are employed dependably when evaluating materials. Thus, the research helps realize a
reliable system for materials evaluation, concise screening checklist and the comprehensive detailed checklist for
materials evaluation in language teaching.
Moreover, generally speaking teachers initiate the materials evaluation process directly with the materials evaluation
checklist. In other words, it forms the first step of the process. The proposed 3-stage system for materials evaluation
presents a unique perspective. First it directs the attention of teachers to context-related factors. It emphasizes the
pivotal role of language teaching context in the materials evaluation process. Namely it raises the awareness of English

learners. The overall goal determines the syllabus (knowledge base and language content), the language skills, academic
skills, etc. Likewise, the teaching-learning context is to be emphasized as well. It provides invaluable information about
the role of English, whether it is a medium of instruction or one of the courses in the syllabus, length and frequency of
the English class hours, class size, availability of course materials, technological equipment of classrooms, international
trend, national policy, and expectations and attitudes of parents towards English. Furthermore, learner-related factors
necessitate special attention. The demographic information about learners, their education background, their needs and
interests, their linguistic and cognitive level, their language learning goals affect both affective and cognitive
involvement of learners in the language education process. Finally, teacher-related factors need to be taken into
consideration. Thus, the research highlights the vital role of context-related factors and reminds that it is the first
inescapable step of the materials evaluation checklist development process.

Figure 1. Materials evaluation checklist development process

 The proposed three-stage system is also found to be practical and time saving. Since there are numerous ELT
materials on the market, it is not practical to evaluate each and every in detail. Hence, the system and the screening
(quick evaluation) checklist speeds up disqualifying the inappropriate ones and saves enough time to focus on the
qualified ones in depth.
Additionally, the detailed analysis checklist is composed of two main categories (external and internal) and nineteen
subcategories, that is, it offers an extremely comprehensive checklist which pave the way for a thorough evaluation of
ELT materials. To the knowledge of the researcher it is the most inclusive one developed so far. In other words, not
only does the study propose a new perspective for materials evaluation, but a detailed checklist that may serve the ones
who are in search of a thorough tool when evaluating their materials.
Finally, the detailed analysis checklist may appear to be too exhaustive and hard to implement. Because of the
screening checklist, only few set of materials are to be evaluated and they deserve enough time and focus to adopt the
most appropriate ones. Besides, it can function as a resource for teachers to choose from among to create their own
checklists.

APPENDIX A. MATERIALS EVALUATION IN EFL

Objectives: This training aims to introduce prospective foreign language teachers to theoretical issues in materials
evaluation. Trainees are required to evaluate the ELT materials for a specific group of learners. More specifically, they
will
a. gain insights into ELT materials and fundamental concepts and principles about them.
b. gain insights into ELT materials evaluation
Products:
a. Trainees will complete the real-life tasks specified for each week
b. Trainees will come up with a materials evaluation checklist
c. Trainees will evaluate materials
Course Outline
Week 1: Introduction
Week 2: ELT Materials
Nation and Macalister, 2010, Chapter I
Graves, 1996, Chapter 2
Long and Doughty, 2011, Chapter 17
Week 3: ELT Materials
McDonough & Shaw, 2005, Chapter 4
Byrd, 2002
Tomlinson, 2011, Chapter 1
Week 4: Materials Evaluation
Tomlinson, 2011, Chapter 8

Graves, 2000, Chapter 9

Week 5: Materials Evaluation
McDonough & Shaw, 2005, Chapter 5
Graves, 2000, Chapter 4
Graves, 2000, Chapter 8
Week 6: Materials Evaluation
McDonough & Shaw, 2005, Chapter 3
Tomlinson, 2011, Introduction
Week 7: Developing Materials Evaluation Checklists
Week 8: Developing Materials Evaluation Checklists
Week 9: Feedback on Checklists
Week 10: Evaluating ELT Materials Using the Developed Checklists
Week 11: Evaluating ELT Materials Using the Developed Checklists
Week 12: Feedback on ELT Materials Evaluation

APPENDIX B. MATERIALS E VALUATION CHECKLIST

I. SCREENING (QUALIFYING EVALUATION)
 completely unsatisfactory unsatisfactory neutral satisfactory completely satisfactory
Methodology
Relevance of content
Publication date
Congruence with the aim
of the course
Age level
Linguistic level
Time (length of the course)
Periphery
Recent, innovative ideas
Appropriacy to the context
Software/internet support
Universality
Availability
Technological requirements
The quality and reliability of
the publisher
Price
Compact nature of the
learner book.

II. DETAILED EVALUATION

a. EXTERNAL EVALUATION
 strongly disagree disagree neutral agree strongly disagree
The information given in the front and
back cover is informative enough
The materials writer(s) is a well-
known an expert
The materials are quantitatively
sufficient for the language program
Table of contents provides enough
information about the materials.
Face validity
The face validity of the front and
back cover is good
Page design offers a colorful variety
Variety in page design is provided
throughout the materials
Page design is not squeezed, clear
Drawings, pictures, illustrations
are used effectively
The font is appropriate
The use of colors is successful
The size of the book does not disturb
learners.
The periphery
They offer a learner book

They offer a workbook

They offer realia
They offer an assessment
tool/component
They offer a project file
They offer a resources file for
teachers
They offer extra authentic documents
They offer online applications
They offer an access code for teachers
and learners for the internet resources
They provide an international online
platform through which teachers and
learners interact
Learners
They are relevant to learner needs
(needs, wants, lacks)
They are relevant to learner cultural
orientation
They are learner-centered.
Context-related factors
They are practical and
applicable in classroom
The text/task/classroom
organization requirements
are met by the classroom
context
In-class time required to
cover the coursebook is
appropriate
Out-class time required to
cover the coursebook is
appropriate
Classroom meets the
technological requirements
of the materials

b. Internal
strongly disagree disagree neutral agree strongly disagree
Approach
They are based on an up-to-date
language teaching/learning
philosophy
Adopted language
teaching/learning philosophy is
observed systematically in the
tasks
They transfer recent scientific
findings to materials
development
They aim at language
acquisition not teaching of pre-
determined linguistic structures
They use language as means
of learning, interaction
Aim
They meet the overall goal of
the program
They address learning needs of
learners
They meet the general
expectations of all stake holders
They lead learners to the
goal in a carefully planned
step-by-step fashion
They provide interim goals
Syllabus
They are cross-curricular
They teach language through
content
They reflect the adopted
language teaching/learning
philosophy
They have enough number of
texts to realize program goals
They have a carefully-knitted
smooth organization
They are organized around
multiple-intelligences
They are real-life oriented
They employ variety of tasks
They employ variety of texts
(from different genres)
They require active participation
of learners
They have a holistic approach to
language
They avoid stereotypes and
discrimination
They favor world citizenship
They provide ample amount of
input
They do not force learners to
produce at the early stages of
learning
The modules/units are organized
around the same goal
Each unit has objectives to fulfill
the ultimate goal
There is a logical organization
among the modules/units
There is a transition between
modules/units
They exploit bodily-kinesthetic
tasks
They emphasize patterns (daily
social language)
They appreciate what learners
bring to the class

They foster cooperative learning

They foster problem-based
learning
They favor experiential learning
They are designed
developmentally in terms of
skills, content, and tasks
If it is an ESP course
They focus on specific language
skills
They focus on specific content
They provide specific texts
They provide specific tasks
They exploit specific documents
They provide specific
terminology
They teach specific symbols and
acronyms
They have real-life simulations
They meet real-life workplace
task demands
Linguistic aspect
They comply with CEFR
They have appropriate intensity
They have appropriate pace
They are based on corpus
They are appropriate to the
linguistic level of learners
They are divorced from
traditional order of grammar
items. The content determines
the language forms not vice
versa.
They are language-awareness-
oriented. They do not attempt to
teach language forms in a
classical sense.
They present language with a
context.
They employ meaningful (not
mechanical) tasks
They are discourse-oriented.
Teacher-related factors
They empower and guide
teachers
They help teacher development
They provide extra tasks
They provide extra materials
They provide extra ideas
They have an interleaved

Teacher

to teachers, does not strictly

control them.
Learner-related factors
They exploit what learners bring
to the learning context
They are learner friendly
They are easy to use
They guide learners successfully
about what to do
They motivate learners
They meet affective needs of
learners
They address learner differences
Their cognitive load is
manageable
Their content is not too
demanding
They give choice to learners
through a variety of texts/tasks
They provide strategies for
learning
They help learners develop
academic/study skills
They make learners responsible
for their own learning.
They foster learners to question
and investigate
They orient learners about how
to use the coursebook and the
periphery
They enlarge learner schemata
They have an holistic approach
to learners
They are resourceful for learner
differences (slow/fast, level,
personality)
Classroom management
They employ a variety of
classroom organization
(individual, pair, group, class)
They help class socialization
through different classroom
organization.
They require active participation
of learners in classroom
organization.
Instructions
They set the scene, explain the
activity and its goal.
They are learner-friendly.
They do not have an
authoritarian voice.
They clear and easy to follow.
The symbols used are systematic
and guiding learners
Content
They present language in a
simple, meaningful manner
They aim at real daily language
All tasks are transferred from
real life
They focus on daily speech
patterns (social language)
They are functional
The people and topics are not
hypothetical
The topics are interesting
The topics and people are up-to-
date
They expand learner schemata
They provide content knowledge
from relevant areas
They employ a variety of topics
They are of optimal level of
cognitive difficulty
They address learners of
different interest
Each content has a reference (to
show where it is taken and
learners refer to further
information)
There is a fine balance between
content and content related tasks
The content between the units
and within the same unit is
developmentally organized
The unit sticks to the same topic,
does not jump from one to
another
They are age-appropriate
The quantity of texts is enough
Literature is exploited
effectively
The length of texts is appropriate
The density of texts is
appropriate
The texts are authentic
They exploit texts from different
genres
Culture
They adopt English as Lingua
Franca (ELF) approach
They do not aim at imposing
target culture
They exploit local (our) and
international culture
They try to raise cultural-
awareness
They are not in conflict with our
culture
Lexis
They have a lexical approach to
teach new lexical items
They emphasize collocations
They emphasize associations
They provide terminology from
different relevant areas
The load of lexical items are
appropriate
They recycle previously taught
target lexical items
systematically
They present new items within a
context
Skills
They have an integrated-skills
approach
They arrange the balance of the
receptive vs productive skills
depending on linguistic level
They have a systematic plan to
develop subskills of each skill.
Format of a unit
At the end beginning of each
unit the objectives are stated
At the beginning of each unit
there is a pre-test about the
content of the unit
Warm-up
The unit starts with a warm-up
The warm-up sets the scene.
Explain what is done so far and
what will be done
The warm-up aims at affective
preparation
The warm-up aims at cognitive
preparation (activates or build
up schemata)
The warm-up aims at pre-
teaching key lexis and grammar
(if there is any)
The warm-up gives a purpose to
do the tasks in the unit, it creates
an inner need
The warm-up does something
striking (exploits a short text and
task, facts, data, visuals) to
capture the attention of learners
Body
The body focuses on the content
The content of each texts is
exploited enough
The body employs enough
number of texts
-
up, content-related tasks,

The task type is relevant to text

type
The body offers relevant tasks to
learners
The body exploits game-like,
enjoyable tasks
All tasks stem from the content
covered
The content of the task
(cognitive load) is manageable
The tasks have a gradual level of
difficulty
The tasks are not piled up, there
is a logical organization
The tasks are organized from
simple to complex
The tasks are organized from
general to specific
There are relevant (related with
tasks learners are expected to
carry out in their education or
career)
The tasks are meaningful,
purposeful, not designed for the
sake of carrying out tasks
All tasks are integrated
There is smooth transition from
one task to another
Tasks are age-appropriate
Tasks foster cognitive
involvement of learners
Tasks foster affective
involvement of learners
Follow up
They allow to put what is
covered to put into practice
They favor to carry out the tasks
in different relevant contexts
They allow field-trips
They employ real-life tasks
They employ simple
experiments, practices
They provide extension of the
content (personalization and
localization related to the
content)
They let learners choose from
among different tasks
They provide contextualized
lexical practice stemmed from
the content
They provide contextualized
grammar practice stemmed from
the content
They provide tasks to be done
after the class
They guide self-interest groups
for extra activities after the class
They provide a summary of the
module/unit
They provide self-evaluation
They provide a post-test
Measurement and evaluation
They favor both alternative and
traditional assessment
They provide feedback to both
teachers and learners to facilitate
learning
They have valid assessment
procedures.
They facilitate language
program evaluation
Software
Software supports the
coursebook (program goals)
Software exposes learners to
comprehensible samples of
language
Software provides rich extra
materials/tasks for learners of
different needs and interests
 Software offers a variety of
materials/tasks for learners of
different learning styles and
types
Software provides engaging and
compelling materials/tasks
Software is easy to use
Software exploits multimedia
Software has an interactive
nature
Software provides glocal
interaction with other learners
Software provides immediate
feedback.
Software provides evaluation for
learner performance

III. OVERALL EVALUATION

are completely Are need a comprehensive are satisfactory but are completely satisfactory but need
unsatisfactory unsatisfactory adaptation need adaptation minor adaptation
The materials

REFERENCES
[1] AbdelWahab, M. M. (2013). Developing an English language textbook evaluative checklist. IOSR Journal of Research &
Method in Education 1.3, 55-70.
[2] Allen, C. (2015). Marriages of convenience? Teachers and coursebooks in the digital age. ELT Journal 69.3, 249-263.
[3] Allehyani, B., Burnapp, D. & Wilson, J. (2017). A comparison of teaching materials (school textbooks vs authentic materials)
from the perspective of English teachers and educational supervisors in Saudi Arabia. International Journal of English
Language and Linguistics Research 5.2, 1-14.
[4] Angell, J., DuBravac, S. & Gonglewski, M. (2008). Thinking Globally, Acting Locally: Selecting Textbooks for College-Level
Language Programs. Foreign Language Annals 41.3, 562-573.
[5] Brown, J. D. (1995). The elements of language curriculum. Boston, Massachusetts: Newbury House.
[6] Cunningsworth, A. (1995). Choosing your coursebook. Oxford: Heienemann.
[7] Dubin, F. & Olshtain, E. (1986). Course design. New York: Cambridge: Cambridge University Press.
[8] Ellis, R. (1997). The empirical evaluation of language teaching materials. ELT Journal 51, 36-42.
[9] Garton, S. & Graves, K. (2014). Identifying a research agenda for language teaching materials. The Modern Language Journal
98.2, 654-657.
[10] Graves; K. (1996). Teachers as course developers. Cambridge: Cambridge: Cambridge University Press.
[11] Graves, K. (2000). Designing language courses. Cambridge: Cambridge: Cambridge University Press.
[12] Grossman, P. & Thompson, C. (2008). Learning from curriculum materials: Scaffolds for new teachers? Teaching and Teacher
Education, 24, 2014 2026.
[13] Guyer, E. & Peterson, P. W. (1988). Language and/or content? Principles and procedures for materials development in an
adjunct course. In S. Benesch (ed.), Ending Remediation: Linking ESL and content in higher education. Washington, DC:
TESOL Publications, 67-90.
[14] Hargreaves, P. (1989). DES-IMPL-EVALU-IGN: an evaluator's checklist . In R. K. Johnson (ed.), The second language
curriculum. Cambridge: Cambridge University Press, 35-47.
[15] Harmer, J. (2003). The practice of English language teaching. Essex: Longman.
[16] Hart, I. (2003). The outsider's gaze: a learner-centred approach to language-teaching materials. Educational Media
International 40.3-4, 287-292.
[17] Harwood, N. (ed.) (2010). Materials in ELT: Theory and practice. Cambridge: Cambridge University Press.
[18] Harwood, N.(ed.) (2014). English language teaching textbooks: Content, consumption, production. Basingstoke: Palgrave
Macmillan.
[19] Hirsch, Linda. (1988). Language across the curriculum: a model for ESL students in the content courses In S. Benesch (ed.),
Ending Remediation: Linking ESL and content in higher education. Washington, DC: TESOL Publications, 67-90.
[20] Hutchinson, T & Waters, A. (1987). English for specific purposes. Cambridge: Cambridge University Press.
[21] Hacettepe
40, 256-266.
[22] ELT materials chosen in high schools? Some suggestions.
Dergisi 28, 28-1.
[23] Education Journal 2. 3, 98-
107.
[24] & Al
Dergisi 23.1, 221-238.
[25] Johnson, R. K. (ed.) (1989). The second language curriculum. Cambridge: Cambridge University Press.
[26] Kostka, I. & Bunning, L. (2016). Curriculum design in language teaching. Washington, DC: TESOL Publications.
View publication stats

[27] Krahnke, K. (1987). Approaches to syllabus design for foreign language teaching. Washington, DC: Center for Applied
Linguistics.
[28] Krashen, S. D. (2006). English Fever. Taipeh City: Crane Publishing Company.
[29] Kukulska-Hulme, A., Norris, L. & Donohue, J. (2015). Mobile pedagogy for English language teaching: A guide for teachers.
London: BritishCouncil.
[30] Letter, W. (2000). A Coursebook Evaluation. Birmingham: University of Birmingham Centre for English Language Studies.
[31] Littlejohn, A. & Windeatt, S. (1989). Beyond language learning: perspectives on materials design. In R. K. Johnson (ed.), The
second language curriculum. Cambridge: Cambridge University Press, 155-175.
[32] López-Medina, B. (2016). Developing a CLIL textbook evaluation checklist. LACLIL, 9.1, 159-173.
doi:10.5294/laclil.2016.9.1.7.
[33] McDonough, J. & Shaw, C. (1993). Materials and methods in ELT. Oxford: Oxford University Press.
rd
[34] McDonough, J., Shaw, C. & ed.). London:
Blackwell.
[35] McGrath, I. (2002). Materials evaluation and design for language teaching. Edinburgh: Edinburgh University Press.
[36] ELT Journal 60. 2, 171-180.
[37] McGrath, I. (2013). Teaching materials and the roles of EFL/ESL teachers: Practice and theory. London: Bloomsbury.
[38] McGrath, I. (2016). Materials evaluation and design for language teaching (2nd ed.). Edinburgh: Edinburgh University Press.
[39] Mishan, F. & Timmis, I. (2015). Materials development for TESOL. Edinburgh: Edinburgh University Press.
[40] . Why use textbooks? English Language Teaching Journal 36, 104-111.
[41] Purgoson, K. B. (1991). Planning lesons and units. In M. Celce-Murcia (ed.), Teaching English as a Second or Foreign
Language. Boston, Massachusetts: Heinle & Heinle, 419-431.
[42] Reinders, H., & Lewis, M. (2006). An evaluative checklist for self-access materials. ELT Journal 60. 3, 272-278.
[43] Richard-Amato, P. (1988). Making it happen. New York. Longman.
[44] Richards, J. C. (2006). Materials Development and Research Making the Connection. RELC 37.1, 5-26.
[45] Shave, J. (2010). A Teacher friendly process for evaluating and selecting ESL/EFL coursebooks. The Internet TESL Journal 16,
11.
[46] Sheldon, L. E. (ed.) (1987). ELT textbooks and materials: Problems in evaluation and development. London: Modern English
Publications.
[47] Skierso, A. (1991). Planning lessons and units. In M. Celce-Murcia (ed.), Teaching English as a second or foreign language.
Boston, Massachusetts: Heinle & Heinle, 432-453.
[48] Tomlinson, B. (2003). Developing materials for language learning. London: Continuum.
[49] Tomlinson, B. (ed.) (2008). English language teaching materials: a critical review. London: Continuum.
[50] Tomlinson, B. (2010). What do teachers think about EFL coursebooks? Modern English Teacher
[51] Tomlinson, B. (ed.) (2013). Developing materials for language teaching (2nd ed.). London: Continuum Press.
[52] Tomlinson, B. & Masuhara, H. (2017). The complete guide to the theory and practice of materials development for language
learning. Hohoken, NJ: John Wiley & Sons.
[53] Willis, D. (2000). Syllabus and materials. Birmingham: The Center for English Language Studies.

graduated from Bogazici University (Istanbul, Turkey) Education Faculty Foreign Language
Education Program in 1989. He completed his MA in 1993 and Ph.D. 1999 at the same university and
department. He completed his second MA at Gazi University (Ankara, Turkey) Educational Administration
Program in 2006.
Between 1999-2016 he worked as a part-time lecturer at Bogazici University, Middle East Technical
University, Hacettepe University, Yeditepe University, and Bogaziçi University consecutively. He is currently
the ELT program and materials development advisor and teacher trainer at Istinye University, Istanbul,
Turkey. His research interests include language teaching methodology, teacher training, content-based
instruction, materials development, measurement and evaluation.