UNIVERSITY OF SCIENCE AND TECHNOLOGY OF

SOUTHERN PHILIPPINES
COLLEGE OF SCIENCE AND TECHNOLOGY EDUCATION
C.M. Recto Avenue, Lapasan, Cagayan de Oro City
Office of Student Affairs

Math 322: Assessment and Evaluation in Mathematics

Instructor: Dr. Jennifer Parcutilo
Group 4: Topic 4
Leader:
Dejeto, lorlyn
Members:
Anabieza, Janessa
Jayme, Gemma Rose
Nacaitona, Clyde Cyrrle
Pacana, Sheila
Ramones, Clarisse
Salvador, Rachel
Suazo, Marje Guen

Interactions of Large-Scale and Classroom Assessment Intended

Learning Outcomes:

▪ Discuss the impact of large-scale assessment to classroom assessment.

▪ Discuss ways on how to make use of assessment results in large-scale assessments and classroom
assessments.

INTRODUCTION
(Lorlyn Dejeto)

• Assessment is a part of the procedure of making inferences, some of which are about students, some about
curricula, and some about instruction (Wiliam 2015).

• Assessment is always a process of reasoning from evidence. By its very nature, moreover, assessment is
imprecise to some degree.

• Assessment results are only estimates of what a person knows and can do (Pellegrino et al. 2001, p. 2).

• Assessment results in mathematics education are and have been used in a variety of ways, particularly when
we examine the impact of large-scale assessment on policy, curriculum, classroom practice, and individual
student’s careers.

• When large-scale assessments focus on monitoring, they are at the system level and some might suggest that
there is no direct impact upon teachers and learners. Such large-scale assessments are perceived as having
little to say about individuals, because they do not deliver sufficiently reliable results for individuals, but only
 on higher levels of aggregation.
However, in some countries (France, Belgium, Netherlands, Norway, etc.) large-scale assessments
include national examinations that all students must take in order to progress to further studies. Such large
scale assessments are exit assessments, whereby students cannot leave secondary school without passing
the national exams.

The question arises as to what extent are large-scale assessments for accountability for
teachers and students used and how might such use influence the nature of classroom
instruction?

2.4.1 Assessments Driving Reform

(Lorlyn Dejeto)

One issue is whether the results of large-scale assessment should drive curriculum and instructional
reform (Barnes et al. 2000). Standings on international assessments of mathematics often drive political and
educational agendas and there have been several examples of this in situations in recent years (c.f. Klieme et
al. 2003). In particular, if countries have nationally organised exit examinations, these may drive (or hinder)
reform.
Some suggest that externally imposed assessments have been used in attempts to drive reform in
some cases (Earl and Torrance 2000; Mathematical Sciences Education Board [MSEB] and National Research
Council [NRC] 1993).
For instance, in many countries the OECD PISA results have affected curriculum in such a way to focus
it more specifically on particular topics, such as problem solving (De Lange 2007).
‘‘All of these efforts were based on the idea that assessment could be used to sharpen the focus of
teachers by providing a target for their instruction’’ (Graue and Smith 1996, p. 114).

Impact of Large-Scale Assessment on Classroom Assessment

(Marje Guen Suazo)

• The nature and design of assessment tasks and the results of assessments often have an enormous influence
on the instruction orchestrated by teachers.

• Swan and Burkhardt (2012) note that the influence of assessments is not just on the content of instruction but
on the types of tasks that students often experience (e.g., multiple-choice versus more open or non-routine
problems). If students and teachers are judged based on the results of large scale assessments, in particular in
countries with national exit examinations, there is a need for students to experience some classroom
assessments that are related to the types of tasks used on such assessments so the enacted curriculum is
aligned with the assessed curriculum and so that results of the assessments provide useful and reliable
evidence to the educational system.

• Some have questioned whether large-scale assessments that are used for accountability measures can be used
or designed to generate information useful for intervention by teachers (Care et al. 2014).

• Also, as indicated by Swan and Burkhardt and Klieme et al., there is a need for piloting of assessment tasks or
empirical study of competence levels before these are used in or applied to large-scale assessments. Piloting
provides opportunities for trying tasks in classroom contexts.
• As Obersteiner et al. (2015) discuss, when items are developed to assess competence at different levels and
then students’ solutions are analysed for potential misconceptions from a psychological perspective, teachers
 are able to use competency models in ways that can guide instruction and help students move forward in
their mathematical understanding.

• However, there is some caution about the use of large-scale assessment types in classroom assessment.

If teachers teach to tests constructed using a psychometric model, they are likely to teach in a
superficial manner that has students learn in small chunks with performance demonstrated in isolated and
disconnected segments of knowledge.

A study by Walcott and colleagues challenges a practice of preparing students for large-scale multiple
choice assessments by having the majority of classroom assessments take a multiple choice format. The study
results show that, in fact, the students in classrooms where the teachers are using a variety of assessments
seem to have higher achievement (Walcott et al. 2015). Their results do not necessarily demonstrate a causal
relationship but provide evidence that a constant use of multiple-choice tests as classroom assessments may
not necessarily lead to high achievement.

Brunner and colleagues provide further evidence of the effects of teaching to the test. They report
that, when students were taught using PISA materials, their performance on the PISA assessment was similar
to the control group in the study (Brunner et al. 2007).

Others have found that assessments that accompany published materials teachers use in classrooms
often fail to address important process goals for mathematics, such as the ability to engage in reasoning and
proof, communicate about mathematics, solve non-routine problems, or represent mathematical concepts in
multiple ways (e.g., Hunsader et al. 2014; Sears et al. 2015).

2.4.3 “Teaching to the Test”

(Sheila Pacana)

• Many might argue or complain that assessments force too many teachers to teach to the test. Some would
argue against large-scale assessments driving classroom instruction, as often the nature of the large-scale
assessment might narrow the curriculum.
• Swan and Burkhardt argue that if the assessment is well designed and aligns with curriculum goals then
‘teaching to the test’ is a positive practice.
• They suggest:
❖ To make progress, it must be recognised that high-stakes assessment plays three roles:
A. Measuring performance across the range of task-types used.
B. Exemplifying performance objectives in an operational form that teachers and students
understand.
C. Determining the pattern of teaching and learning activities in most classrooms (p. 4).

Swan and Burkhardt note that large-scale assessments provide messages to classroom teachers about what is
valued and have an impact on both instructional and assessment activities within a classroom.

Their work suggests a need to develop assessments that pay attention to their implications and align not only
with mathematics content but also with mathematical processes and actions.
However, Swan and Burkhardt note that if the assessments reflect what is important for students to learn
about mathematics, both in terms of content and performance, then “teachers who teach to the test are led to
deliver a rich and balanced curriculum” (2012, p. 5, italics in original).

They argue that if an assessment is perfectly balanced and represents what is important to learn, then
perhaps ‘teaching to the test’ can be a good thing.
 This scheme of ‘teaching to the test’ has been put forward as a potential tool for innovation and has also
been articulated by other scholars (Van den Heuvel-Panhuizen and Becker 2003).

As De Lange noted in (1992): …[throughout the world] the teacher (or school) is judged by how well the
students perform on their final exam. This leads to test-oriented learning and teaching. However, if the test is made
according to our principles, this disadvantage (test-oriented teaching) will become an advantage.

The problem then has shifted to the producers of tests since it is very difficult and time consuming to produce
appropriate ones. (as cited in Van den Heuvel-Panhuizen and Becker, p. 691)

An important question, then, is whether the assessment adheres to design principles that engage students
with tasks that provide opportunities to engage with important mathematical processes or practices. However, even
the best-designed assessments cannot account for issues such as test anxiety.

When assessment activities mirror instructional activities, that anxiety might be reduced and such
assessments might be a more accurate indicator of student achievement.

2.4.4 Examples of Positive Interaction Between Large-Scale Assessment and Classrooms

(Janessa Anabieza & Clarisse Ramones)

If the enacted curriculum of the classroom and the assessed curriculum are to inform each other and to
enhance student learning in positive and productive ways, then large-scale external assessments cannot operate in
isolation from the class room.

A number of researchers from different parts of the globe have documented the interplay of these two
assessment contexts.
• Shimizu (2011) discusses bridges within the Japanese educational system between large-scale external
assessments and actual classrooms. As in many countries, he notes tensions between the purposes of
external assessments and those of classroom assessments and that teachers are concerned about the
influences of external assessments on their classroom instructional practices and assessment. To
assist teachers in using external assessments to inform their classroom instruction, sample items are
released with documentation about the aims of the item as well as student results on the item, and
potential lesson plans that would enable such items to be used in the classroom.
• For instance, on multiple-choice items, information can be provided that suggests the types of thinking
in which students may be engaged based on answer choice, thus assisting teachers to understand the
nature of misconceptions in students’ thinking based upon the incorrect answer selected.
- As a result, external assessment tasks provide students with opportunities for learning. • Given the expectation
that the assessed curriculum reflects what should be taught within the classroom, Shalem et al. (2012) describe
the benefits of having South African teachers engage in curriculum mapping of large-scale assessments. The
authors acknowledge that assessment developers often engage in such activities, but that classroom teachers
rarely do. For a given test item, teachers identified the concept being assessed, identified and justified the
relevant assessment standard, and then determined if the necessary content was taught explicitly or not and at
what grade. The results of the curriculum mapping helped to highlight
discrepancies between the intended curriculum assumed by assessment designers and the actual enacted
curriculum of the classroom.

• Specifically, there were three important outcomes for teachers:

✓ teachers developed an understanding of the content assessed at a particular grade and at

what cognitive demand;
✓ teachers could reflect on their own practice as they identified whether the content was
 actually taught in clasrooms; and
✓ teachers developed a more robust understanding of the curriculum.

All three of these outcomes provided an important link between the external assessment and actual
classroom practice that could inform future instruction.

Brodie (2013) discusses an extension of the work of Shalem, Sapire, and Huntley in the South African context.
In the Data Informed Practice Improvement Project, teachers work in professional learning communities to analyse
test items, interview students, map curriculum to assessments, identify concepts underlying errors, and then read
and discuss relevant research.

Within school based learning communities, teachers use school level data to design and then reflect on
lessons to address issues within the data analysis. As a final representative example of how external assessments
might be used to inform classroom instruction and assessment, we consider the work of Paek (2012). Paek suggests
that learning trajectories are one means “to make an explicit and direct connection of high-level content standards,
what is measured on high-stakes large-scale assessments, and what happens in classrooms” (p. 6712). She argues
that development of such trajectories “requires deep understanding of the content, how students learn, and what to
do when students are struggling with different concepts” (p. 6712). Knowledge of such trajectories can be used in the
development of assessment items but also can help to inform teachers about how students learn particular concepts.
In her project, Paek works with researchers, mathematics educators, and national consultants to develop resources
for teachers that connect learning trajectories to big ideas within and across grades. It is hoped that by becoming
more familiar with learning trajectories, teachers can consider developing classroom assessment items that focus on
different aspects of the learning contin uum or that provide opportunities for transferring of skills or that enable
student work to be collected to monitor progress along the continuum.

2.4.5 Making Use of Assessment Results

(Gemma Rose Jayme & Clyde Cyrrle Nacaitona)

Making use of assessment data is strongly connected to assessment literacy.

• Webb (2002) defines assessment literacy as:

▪ the knowledge of means for assessing what students know and can do, how to interpret the
results from these assessments, and how to apply these results to improve student learning
and program effectiveness (p. 4).
This definition of assessment literacy can apply to the interpretation of results from both classroom and large-scale
assessments. In fact, the challenges of making use of assessment results span both classroom assessment results and
large-scale assessment results.

• Making sense of classroom assessment results is often seen as teachers’ work.

Wyatt-Smith et al. (2014) acknowledge that, along with the task of designing assessments, teachers need to
know how to use assessment evidence to identify the implications for changing teaching. However, interpreting
classroom assessment results is also the work of parents, students, and school administrators. In terms of students,
including students in the assessment process has been shown to provide them with a stronger sense of what is being
assessed, why it is being assessed, and ways they can improve, thus allowing them to make better use of assessment
results (Bleiler et al. 2015; Tillema 2014).

• Developing students’ ability to self-assess is an important aspect of this work (Fan 2011). Once students have a
strong assessment literacy, they can assist their parents in interpreting assessment results, but this should be
supported with solid and frequent communication between schools, teachers, students, and parents in order
to enhance the assessment literacy of all who are involved in understanding assessment results.
• Making sense of large-scale assessment results is often seen as the purview of many education stakeholders,
including teachers, administrators, school district officials, state or provincial policy makers, national policy
makers or administrators, as well as the general public.
 • Each group has its own perspective on the results, sometimes multiple contrasting perspectives, which can
complicate interpretation. To the extent that large-scale assessment results are useful for more than
conversations across constituent groups, the value will come in the application of suitable interpretations to
educational policy and practice.
• The question arises as to what do educators need to know to make productive use of assessment results? • The
first priority when examining test results is to consider the purpose of the test and to view the results in light of
the assessment’s purpose. Those viewing the results should also have some assessment literacy to be able to
know what they can and cannot infer from the results of the assessment.
• Rankin (2015) places some of this responsibility on the assessment developer and suggests that, in order to
assist educators in making use of assessment results, those responsible for conveying the results should
consider how they are organized and what information is provided to those using the results. She suggests
that data analysis problems often occur because the data are not presented in an organized and meaningful
manner. Her work highlights several common types of errors that are made in the interpretation of
assessment results and she cautions educators to pay close attention to the assessment frameworks and
literature published in concert with the assessment’s results (often on a website). She presents a series of
recommendations for which she feels educators should advocate that include improvements to data systems
and tools that will facilitate easier and more accurate use of data.
• See also Boudett et al. (2008) and Boudett and Steele (2007) for other strategies in helping teachers interpret
results.

2.4.6 Suggestions for Future Work

(Rachel Salvador)

There is no doubt that much can be learned from large-scale assessment.

Assessments help to highlight achievement gaps, point to areas where adaptations to instruction might be
required, and can lead to reforms in curriculum and teaching. International assessments have helped to globalize
mathematics education and have created a forum for mathematics educators to share their knowledge and
experiences, and work together to resolve common issues.

• As shown, large-scale and classroom assessment interact with one another in different ways. Because they
rest on similar principles of sound assessment, ultimately having coherence between the two would help to support
all students to be successful.
• Wyatt-Smith et al. (2014) suggest “better education for young people is achievable when educational policy
and practice give priority to learning improvement, thereby making assessment for accountability a
related, though secondary, concern” (p. 2).
This section has raised many issues for future research.
Some questions to consider include:
• What are the challenges and or issues in attempting to bring all stakeholders for assessment to the
table—developers of large-scale national or international assessments, textbook publishers or developers, classroom
teachers?

• What are issues in attempting to ensure that important mathematical practices/processes are evident in
assessments?

• What training or skills do teachers need to develop to be able to analyse/critique assessments?

• Should policy decisions be made based on large-scale assessments, given evidence that some assessments may not
reflect students’ achievement when the assessment has little impact on students’ educational lives (e.g., some
students do not give their best effort on the assessment)?