See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/261080807

Assessing conceptual understanding in mathematics

Conference Paper in Proceedings - Frontiers in Education Conference · October 2013

DOI: 10.1109/FIE.2013.6685135

CITATIONS READS
14 13,127

3 authors, including:

Hyesun You
University of Iowa
28 PUBLICATIONS 228 CITATIONS

SEE PROFILE

All content following this page was uploaded by Hyesun You on 13 February 2017.

The user has requested enhancement of the downloaded file.

Assessing Conceptual Understanding in Mathematics
Austin, Texas
Audrey De Zeeuw, Tara Craig and Hye Sun You
[email protected], [email protected],
Department of Curriculum and Instruction [email protected]
University of Texas at Austin

Abstract— Modern mathematics careers are now requiring conceptual understanding of mathematics could result in
conceptual skills such as, critical thinking, modeling, and alternative instructional materials and methodologies.
application of the content. This change in skillset needed NetLogo is a modeling computer language used to create
for careers has strongly impacted mathematics curriculum an integrated environment. Students, acting as the modelers,
and assessment. Meaningful assessment involves are given tools to represent basic elements of a system. Then
examining students’ ability to inquire, to reason on the program provides a way to simulate the interactions
targeted questions or tasks, and to promote conceptual between these elements. With NetLogo, students write rules
understanding, not just focusing on discreet facts and for hundreds or thousands of these basic elements, specifying
principles. Mathematics assessment tools still focus solely how they should behave and interact with one another. These
on this procedural side of understanding mathematics individual elements are referred to as turtles [8]. Students
instead of the equally important conceptual aspect of exploring within NetLogo have been able to develop their
learning mathematics. Given that math is an active process understanding of rates of change through modeling scientific
that encourages higher-order thinking and problem phenomena [2;9].
solving, an assessment focusing on the growth of
II. PURPOSE OF RESEARCH
conceptual understanding is required. The proposed
research focuses on the development of a tool that will be The purpose of this research is to transform a modeling unit
used to assess current U.S. calculus students' ability to within the NetLogo agent-based modeling program to assess
apply their conceptual understanding of a mathematic students’ conceptual understanding of a key mathematics
concept to scientific phenomena through modeling. The concept. Students that are successful in high school calculus
assessment using Netlogo would be very useful for math are likely to choose STEM fields for their college studies [10].
educators to have good tools to assess students’ conceptual Students on this STEM path should be prepared to apply their
understanding of various Calculus concepts to their STEM
understanding, as well as to develop instructional
fields. Therefore, current calculus students should have the
strategies used.
ability to connect and apply their understanding of rate of
change to another STEM domain. This leads to several
Keywords—STEM; procedural knowledge; conceptual
questions: 1. How can NetLogo be re-designed as a tool to
knowledge; NetLogo; Assessment
assess students’ conceptual understanding of ‘rate of change’?
I. INTRODUCTION 2. Is students’ procedural understanding of rate of change a
positive predictor of students’ conceptual understanding of rate
Calculus is the capstone of K-12 mathematics curriculum. of change?
Within the domain of calculus, rate of change is a key concept
that is introduced to students in algebra 1 and is foundational to III. Background
calculus. Conceptual understanding of the reflectivity,
A. Assessment
generativity, and implicatively of rate of change is
fundamental to students’ study of calculus [1,2]. Procedural Current STEM curricula are beginning to promote the
mathematics content knowledge focuses only on reciting inclusion of higher-order thinking skills [11]. Higher-order
algorithms and facts, whereas conceptual mathematics thinking skills, related to sophisticated cognition, are difficult
knowledge emphasizes students’ ability to interconnect to measure with the use of multiple-choice tests. Standard,
mathematics across disciplines, critically think about the paper and pencil tests do not accurately measure 21st century
content, and communicate key components of mathematics skills. These tests are usually composed of low-level facts and
[3.4]. recipe-like procedures [7]. This is not the only shortcoming of
the current educational climate. State curriculum standards in
Mathematics assessment tools often focus solely on this each discipline are neither inter-related nor prioritized to
procedural side of understanding mathematics instead of the emphasize core understandings and performances all students
equally important conceptual aspect of learning mathematics will need to succeed in the 21st century, despite the call for
[5,6]. This is due to the restrictive nature of traditional interrelated curriculums [11]. There is also a call for students
multiple-choice assessments [7]. An alternative assessment to exhibit higher-order thinking skills, since a successful
tool that could quantify both students’ procedural and problem solver must be able to apply different approaches to a

978-1-4673-5261-1/13/$31.00 ©2013 IEEE

problem [12]. Students must demonstrate unusual, original, familiar with a type of program by using it, then modifying it a
and creative solutions to various real-life problems, which is bit to make it their own, and finally creating their own program
difficult on a multiple-choice test. from scratch. In this process, students are applying
computational thinking to solve problems and using higher-
As a consequence of this misalignment between call for order thinking skills.
reforms and current assessments, teachers are using weak but
rapid instructional methods to pass these limited scope Modeling provides a new perspective of the methodology
assessments instead of preparing students for 21st century of math [16]. A detailed analysis of modeling provides
citizenship. These summative, “drive by” tests provide characterization of problem solving as a coherent system for
retrospective feedback, rather than immediate feedback that constructing models in any mathematical domains. Modeling
could influence future instruction [7]. begins with a physical situation with which we wish to
understand [17]. Understanding comes from making,
Some International tests, such as Program for International analyzing and evaluating a model for the situation. The
Student Assessment (PISA), make attempts to emphasize core situation may be presented to us in any form; for example,
ideas and measure higher-order thinking skills. US students are predicting the path of a projectile in the study of quadratic
scoring below many of their International peers on this test, but functions. From examining the situation, our first task is to
these test do not have the high stakes and accountability come up with a suitable model in diverse ways. Sometimes,
standards that state mandated standardized tests posses [7]. we can simply adapt a familiar model to the situation. In any
Therefore, these lower scores do not impact curriculum and case, students’ understanding of what constitutes a model is a
instruction. Modern technology could provide a platform to useful guide for what to do. The first step of this process is to
improve US standardized tests to include assessment of higher identify the system [17], then specify precisely what it is we
order thinking skills, but a lack of resources, as well as, a fear want to model, and then identify the properties of interest and
of academic dishonesty has resulted in banning technology specify variables to represent them. When a model has been
from testing. If the goal of assessment is to measure particular constructed, it must be analyzed to understand its structure and
expectations, and the current call for students is to exhibit implications. The final step of analysis is validation, which is
higher-order thinking skills, the assessments themselves need the process of assessing the adequacy of a model to represent a
to change. particular system and situation. A full assessment of the
B. Modeling model’s validity will include an assessment of what is
Research suggests that modeling tasks “provide neglected in the model as well as the accuracy of the
opportunities for greater diversity of thinking than is usually representation.
found in problem-solving tasks and that the development of III. RESEARCH METHODOLOGY
students ideas occurs along multiple pathways” [12]. Students
develop conceptual understanding of various mathematical A design-based approach will be used to answer the
concepts when they are engaged in modeling activities that question, How can NetLogo Hotlink Replay be re-designed as
require they research, create, and revise models [12,13]. a tool to assess students’ conceptual understanding of ‘rate of
change’?
Modeling is “nominally generative,” meaning students are Instructional designers will begin with the NetLogo
asked to use a prescribed model (single pathway) to create
modeling activity for rate of change in which students model
computed outcomes that are often mechanically compared to
the effects of introducing an infection into a population and
the actual data collected [15]. Ideal modeling occurs when the
learners create a range of models and use them to generate manipulate various components (i.e. initial number infected,
model outcomes. In an agent-based modeling approach- identifying the infected). While exploring with this activity,
student can model at the individual (elemental) level, rather students are able to develop a conceptual understanding of the
than an aggregate approach [8]. Modeling tasks need to be calculus topic of rate of change [18]. Researchers will use a
composed of multiple-pathways and multiple-outcome tasks. design-based approach to repurpose this activity into an
The underlying constructs of othese modeling tasks should be assessment of students’ conceptual understanding of rate of
mathematically or scientifically significant [15]. change, using an extension of Hotlink replay [9]. Year 1 of
the project will focus on development of this tool through
Central to computational thinking is the idea that, given a
iterations of design, utilize, and revise.
complex problem, which consists of many different,
Programmers will incorporate prompts and challenges into
interconnected parts, one can meaningfully and sensibly
decompose the problem into more solvable sub-problems [11]. the Hotlink replay interface. For example, students may be
Breaking down a problem is not a haphazard process. Deep prompted to create a model of an infected agent entering a
understanding of the systems involved and how they interrelate population and moving randomly throughout the population
is required in order to meaningfully break up the problem into after viewing the visualization of their model, students may be
pieces that still make sense in relation to the whole. Students asked to change the components such as making the graph
that are successful in high school calculus are likely to choose steeper.
STEM fields for their college studies [10]. Student participants will be chosen from either algebra 1
classes or calculus classes at the end of the year. The
Computational thinking and computational problem solving participants will complete the first iteration of the assessment,
involve a process of revision and iteration. “Use-modify-
then they will be interviewed on the topic of rate of change
create” is a pedagogical model that allows students to become
with an interview protocol that focuses on procedural (i.e. the NetLogo to assist student conceptual development of rate of
use of slope formula to calculate slope) and conceptual change. This closer look at instruction using NetLogo will
(applying rate of change to graphical representations of inform new forms of assessments designed to measure
biology concepts to make conjectures) understanding of rate students’ conceptual understanding.
of change. Qualitative analysis of the transcriptions will be
REFERENCES
done to identify evidence of conceptual and procedural
understanding of rate of change. The same students will then
follow a think-aloud protocol while taking iterations of the [1] Carlson, M., N. Smith, and J. Persson. "Developing and Connecting
Calculus Students' Notions of Rate of Change and Accumulation: The
assessment. Over year 1, three iterations of design, utilize, Fundamental Theorem of Calculus." Paper presented at the Proceedings
and revise will be used to transform the modeling activity into of the 2003 Joint Meeting of PME and PEMENA, Honolulu, Hawaii,
an assessment of procedural and conceptual student 2003.
understanding. [2] Stroup, W. January 2005. “Learning the basics with calculus,” Journal
of Computers in Mathematics and Science Teaching. Vol. 24, pp 179-
Once the assessment has been revised and the research 196.
suggests that the score associated actually matches the [3] Hiebert, J. and Lefevere, P., “Conceptual and procedural knowledge,” in
qualitative analysis of students’ conceptual understanding of J. Hiebert (ed.) 1986. Conceptural and procedural knowledge: The case
‘rate of change,’ then scores on the assessment will be used to of mathematics. Hillsdale, New Jersey: Lawrence Erlbaum Associates,
answer the second research question: Is students’ procedural pp. 1-27.
understanding of rate of change a positive predictor of [4] Linn, R. 1994. “Perfomance assessment: Policy promises and technical
measurement standards,” Educational Researcher, Vol. 23, pp. 4-14.
students’ conceptual understanding of rate of change?
[5] A. Baroody, “The development of adaptive expertise and flexibility: the
During year 2, the algebra 1 and calculus student integration of conceptual and procedural knowledge,” in A. Baroody and
participants will be assessed using the designed NetLogo A. Dowker (eds.) 2003. The development of Arithmetic concepts and
modeling assessment as well as taking traditional, multiple- skills. Mahway, New Jersey: Elbaum, pp.1-31.
choice assessment on the topic of rate of change. This [6] Rittle-Johnson, B. & Alibali, M, 1999. “Conceptual and procedural
knowledge of mathematics: Does one lead to the other?” Journal of
traditional assessment will be designed mirroring the types of educational psychology. Vol. 91, pp.175-189.
multiple-choice questions on traditional standardized [7] Clarke-Midura J. &Dede C., 2010. “Assessment, technology, and
assessments that test understanding of ‘rate of change’. change,” Journal of research on technology in education. Vol. 42, pp.
The teacher participants will assign each student 4-digit 309-329
number to de-identify the student scores. If collected data [8] U. Wilensky & K. Reisman, 2006. “Thinking like a wolf, a sheep or a
show normal distribution, a series of paired t-test for within firefly: Learning biology through constructing and testing computational
theories—an embodied modeling approach,” Cognition and Instruction
group comparison will be performed to determine if Vol. 24, pp. 171-209.
significant difference existed between procedural scores [9] M. Wilkerson-Jerdel, & U.Wilensky,“Restructuring change, interpreting
versus conceptual scores. When the normality assumption is changes: The deltatick modeling and analysis toolkit.” Paper presented
violated nonparametric method like the Wilcoxon signed at the, Proceedings of Contstructionism, p.97 Paris, France, 2010, pp.
ranks test will be used. We will use SPSS 21 version for 98.
analysis. The SPSS data output will demonstrate basic [10] F. Spielhagen, 2006. “Closing the Achievement Gap in Math: The long-
term effects of eight-grade algebra,” Journal of advanced academics
descriptive statistics for samples and variables, correlation Vol. 18 pp. 34-59.
table, and the matched mean difference between the pairs. [11] National Research Council. “A framework for K-12 science education:
Practices, cross cutting concepts, and core ideas.” Washington, DC:
IV. IMPLICATIONS National Academies Press. 2012.
Research suggests that the ideal classroom environment is [12] H.Doerr & L. English. “A Modeling perspective on students’
mathematical reasoning about data,” Journal for Research in
the use of technology to assess student learning and inform Mathematics Education Vol. 34 pp. 110-136.
math instructors’ ability to prepare students for STEM career
[13] L. English. “Mathematical Modeling in the Primary School: Children’s
paths. One of the primary objectives of this proposed work is Construction of a Consumer Guide.” Educational Studies in
to redesign a NetLogo simulation to assess students’ Mathematics. Vol. 63 pp. 303-323.
conceptual understanding of mathematics. This assessment [14] National council of teachers of mathematics (2000). “Principles and
will assist the educational community to establish students’ Standards for school mathematics.” Reston, VA.
understanding of rate of change and potentially design a new [15] J. Zawojewski & R. Lesh, 2007, “Problem solving and modeling” in F.
math curriculum. Our vision is that this assessment could Lester (ed.) 2007. Second handbook of research on mathematics
result in a change in the mathematics curriculum. With the teaching and learning NCTM. Charlotte, NC: Information Age
Publishing, pp. 763-804.
possibility of teaching students concepts earlier or have
[16] R. Lesh, L. Carmona & M. Hjalmarson.“Models and modeling”.
students take algebra 1 in 8th grade. Working group at PME-NA Proceedings, Merida, 2006, p.1-4.
This project can potentially create a tool that will assess a [17] D. Hestenes. “Modleing Methodology for Physics Teachers.” Paper
skill that previous tools could not assess in mathematics. Since presented at the proceedings of the International Confernece on
Undergraduate Physics Education, college Park, 1996.
mathematics is key for engaging students in STEM careers and
[18] W. Stroup & U. Wilensky. “Networked gridlock: Students enacting
majors, having a tool that identifies the best and brightest will complex dynamical phenomena with the HubNet architecture.” Paper
impact the future of the STEM field in the United States. The presented at the proceedings of the 4th Annual International Confernece
most innovative aspect of the proposed work is the use of of the Learning Sciences . Ann Arbor, MI, 2006.

View publication stats