Computer Science Education

ISSN: (Print) (Online) Journal homepage: https://www.tandfonline.com/loi/ncse20

A Learning Trajectory for Variables Based in

Computational Thinking Literature: Using Levels of
Thinking to Develop Instruction

Kathryn M. Rich , Diana Franklin , Carla Strickland , Andy Isaacs & Donna
Eatinger

To cite this article: Kathryn M. Rich , Diana Franklin , Carla Strickland , Andy Isaacs &
Donna Eatinger (2020): A Learning Trajectory for Variables Based in Computational Thinking
Literature: Using Levels of Thinking to Develop Instruction, Computer Science Education, DOI:
10.1080/08993408.2020.1866938

To link to this article: https://doi.org/10.1080/08993408.2020.1866938

Published online: 23 Dec 2020.

Submit your article to this journal

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=ncse20
COMPUTER SCIENCE EDUCATION
https://doi.org/10.1080/08993408.2020.1866938

A Learning Trajectory for Variables Based in Computational

Thinking Literature: Using Levels of Thinking to Develop
Instruction
Kathryn M. Rich a, Diana Franklinb, Carla Stricklandc, Andy Isaacsc
and Donna Eatingerc
a
College of Education, Michigan State University, East Lansing, MI, USA; bComputer Science Department,
University of Chicago, Chicago, IL, USA; cUChicago STEM Education, University of Chicago, Chicago, IL, USA

ABSTRACT ARTICLE HISTORY

Background and Context: We explored how learning trajectories Received 27 November 2019
(LTs) might be used to design variables instruction. Accepted 17 December 2020
Objective: We aimed to develop an LT for variables and use it to KEYWORDS
guide curriculum development for fourth graders working in Variables; K-8 education;
Scratch in an integrated mathematics+CS curriculum. curriculum development;
Method: We synthesized learning goals (LGs) and levels of thinking learning trajectories
about variables from existing research, then designed instruction to
support progress along the levels of thinking while addressing
a subset of the LGs.
Findings: Our literature synthesis articulated four levels of thinking
and eight LGs about variables. Curriculum development allowed us
to support movement through the levels of thinking while addres­
sing five LGs in an integrated math+CS course for fourth grade
using Scratch.
Implications: Our model of curriculum development around levels
of thinking can support the creation of customized instructional
sequences.

Variables are an important and challenging topic in computer science (CS) education
(Grover et al., 2019; Kohn, 2017). As such, it is important to develop innovative ways for
teaching variables to students in a variety of contexts. In this paper, we describe our
process for synthesizing existing studies in CS education, which took place in a variety of
settings and with students with a variety of backgrounds, into levels of thinking about
variables and a set of variables-related learning goals (LGs). We describe how we designed
curriculum around the levels of thinking, incorporating instruction on a subset of the LGs,
to create hypothesized learning trajectory (LT) for Variables. We discuss how the LT’s focus
on levels of thinking about variables, which are related to, but distinct from, facts and skills
about variables, contributed to the flexibility in the ways in which we believe instruction
on variables might be developed. That flexibility can help to serve the needs of students
and teachers working under various conditions and constraints.

CONTACT Kathryn M. Rich [email protected] College of Education, Michigan State University, 620 Farm Ln,
Room 513, East Lansing, MI 48824
© 2020 Informa UK Limited, trading as Taylor & Francis Group
2 K. M. RICH ET AL.

Background
CS education research on variables
Variables are an important topic in CS education, as evidenced by attention to them in
curricula and educational programs (e.g. Grover et al., 2015; Pasternak & Vahrenhold,
2012; Peluso & Sprechini, 2012), teacher professional development (Liu et al., 2011; 2014),
and frameworks (Angeli et al., 2016). A rich understanding of variables serves as an
important foundation for several other CS concepts, including abstraction and looping
(Grover et al., 2019).
At least three lines of research suggest variables are a challenging topic. First, several
studies cataloguing common misconceptions in CS education highlighted issues related
to variables. Students often think variables hold more than one value at a time, believe
assignment statements can be interpreted in both directions (i.e. “let A = 2” is equivalent
to “let 2 = A”), or expect computers to interpret or assign values to variables based on the
semantic meaning of variable names (Du Boulay, 1986; Kohn, 2017; Putnam et al., 1986).
Second, research on block-based languages showed these languages do not prevent
students from developing misconceptions about variables, and can produce new mis­
conceptions not seen with text-based languages (Grover & Basu, 2017). Lastly, an analysis
of students’ programming projects produced in informal settings generally showed low
use of variables (Maloney et al., 2008), and the overall number of projects on the Scratch
website that use variables is also low (Meerbaum-Salant et al., 2013). These trends suggest
students do not tend to learn variables without explicit instruction.
Together, these three ideas – the importance of variables in CS, the common mis­
conceptions about variables, and the need for instruction targeting variables – establish
a need for new approaches for teaching students about variables, especially as CS
education is becoming increasingly common in K-12. Some research has focused on
challenging existing approaches to variables instruction. For example, Hermans et al.
(2018) compared assessment results for children who were taught about variables using
the traditional metaphor of a variable as a box and children who were taught about
variables as labels. They found children who learned the box metaphor showed better
general understanding of variables, but were more likely to have the misconception that
variables can hold more than one value. Other researchers have examined the implica­
tions of relating CS variables to the ways variables are used in other disciplines. Kohn
(2017) argued many of the misconceptions held by tenth graders could be explained by
them carrying understandings of variables they had from mathematics classes into their
interpretations of programs. Buffum et al. (2018) argued relating CS variables to indepen­
dent and dependent variables in science may be more productive than relating CS
variables to mathematics.
Other lines of work focused on developing new ways to organize variables instruction.
Grover et al. (2019) explored the potential of using non-programming activities to support
students in grasping variables concepts before attempting to use variables constructs.
Middle-school students who completed non-programming activities before they used
variables in code showed better use of variables in their programs than a control group.
Sajaniemi et al. (2006) developed a set of roles that variables commonly play in programs.
For example, stepper variables step through a predictable sequence of values, whereas
 COMPUTER SCIENCE EDUCATION 3

a most-wanted holder variable holds the most appropriate value for a purpose found so
far. These researchers claimed these roles are consistent across multiple programming
paradigms and so they serve as a useful framework for students to think about variables.
In an empirical study, they found students who learned about roles of variables and used
an animation tool that highlighted roles produced higher-level descriptions of programs,
which is evidence of better understanding of variables.
In this paper, we explore an additional approach to organizing variables instruction
guided by the theory of learning trajectories, which we discuss next.

Learning trajectories
Learning trajectories (LTs) are hypothetical routes along which learning about a topic
might proceed (Simon, 1995). Actual routes to learning for particular students are not
knowable in advance and often involve students moving both forward and backward
among levels of thinking (Battista, 2011). Hence, an LT is not a description of the learning
experience children must go through. Rather, LTs are descriptions of idealized, hypothe­
tical routes based on existing literature (Sarama et al., 2011).
Instructional activities are an integral part of an LT (Battista, 2011; Sarama et al., 2011).
Indeed, LTs have three parts: an overall learning goal, one or more conjectured pathways
from children’s intuitive knowledge to the overall goal (often referred to as a progression),
and a set of instructional activities hypothesized to help students move along the path­
ways (Clements & Sarama, 2004). While studying developmental progressions and the
impact of instructional sequences can be valid research goals in and of themselves, “the
power and uniqueness of the learning trajectories construct stems from the inextricable
interconnection between these two aspects” (Clements & Sarama, 2004, p. 83). On one
hand, instructional activities are designed with the intent of helping students move along
the progression – the progression influences the activities. On the other hand, while
movement along the progression is intended to be facilitated by the designed instruc­
tional experiences, student responses to the activities can vary and produce evidence
suggesting changes to the progression. In this manner, studies of instructional activities
can influence the progression.
The progressions inherent to LTs are not hierarchical orderings of facts and skills, but
rather descriptions of increasingly sophisticated ways of thinking about a topic, or levels of
thinking (Sarama et al., 2011). Levels of thinking cannot be described by simply referen­
cing the definitions, concepts, or procedures that students know. Rather, levels of thinking
describe how students think about a topic (Sarama et al., 2011). More specifically, these
levels specify the objects that are the focus of students’ thinking and the cognitive actions
students apply to those objects. For example, in Sarama et al.’s (2011) LT for length
measurement, students operating at the “length unit relater and repeater” (p. 670) level
can construct a mental length unit and iterate it along a length they can see. Children may
move a single physical unit as they measure, but can maintain the mental image of each
placement of the unit.
The interaction between an LT’s focus on levels of thinking and its dependence on
instructional activities poses a challenge for curriculum developers using an LT approach.
On one hand, the goal is to develop classroom activities that help students progress
through levels of thinking – and consequently, not merely to help them master particular
4 K. M. RICH ET AL.

facts and skills. On the other hand, although most teachers will be eager to help students
develop their thinking, teachers are also accountable for teaching students particular facts
and skills. Thus, it is important for curriculum developers to consider how instructional
activities can address particular facts and skills, which may be specific to context, in ways
that also help students progress in their levels of thinking. In this paper, we present an LT
for Variables (as defined and used in CS) developed through processes of literature
synthesis and curriculum development for students in fourth grade. The levels of thinking
we derive from literature are general and could be applied across contexts, but the
complete LT – including its activities – are specific to our context of fourth graders
working in an integrated math + CS curriculum based in Scratch.
Other research teams have developed learning progressions for CS concepts. Dwyer
et al. (2014) used focus groups with fourth-grade students to identify lower anchor points
for a progression on algorithm development. Werner et al. (2012) discussed the design of
an assessment tool for Scratch programs that categorizes programs by their levels of
computational learning, based on an implicit progression. Seiter and Foreman (2013)
created a Progression of Early Computational Thinking (PECT) model for assessing stu­
dents’ computational thinking (CT) competencies based on their programming projects.
This paper builds on, but differs from, this work by focusing on variables. Moreover, by
using the LT construct, rather than only learning progressions, we are able to consider the
role of instructional activities on students’ movement along a progression.

Research questions
(1) What facts, skills, and levels of thinking about variables are present in existing CS
education literature?
(2) How might activities addressing related facts and skills be designed to support
movement toward higher levels of thinking about variables in elementary
students?

Methods
Our four-step process was adapted from Confrey et al. (2014). Step 1 involved necessary
preparations to conduct analyses to answer the research questions. Steps 2 and 3 were
aimed at answering research question 1. Step 4 was aimed at answering research ques­
tion 2.

Step 1: review existing CS education literature to extract and cluster LGs

Confrey et al. (2014) began by grouping mathematics standards into clusters of related
goals that could be organized into LTs. Although standards and frameworks for K-8 CS
education are available (e.g. Angeli et al., 2016), most are organized by grade band rather
than grade level. As we began this work, we knew of no standards articulated in fine
enough detail to allow for the kinds of clustering and organization conducted by Confrey
and colleagues. We therefore conducted a review of CS education literature with an eye
toward extracting information about LGs that researchers and experts had claimed or
demonstrated K-8 students could accomplish. This literature search did not focus
 COMPUTER SCIENCE EDUCATION 5

specifically on variables, but rather encompassed CT broadly. This study is part of a larger
project aimed at defining LTs for multiple CT concepts. Our aim during this step was not
to focus on a specific topic, but rather to cast a wide net to gain a better sense of the ideas
discussed within CS education literature.
We gathered articles by conducting keyword searches of the Educational Research
Information Center database, which we chose as a starting point due to its focus on
education and its wide scope to include journal publications, conference papers, and
government documents. The keywords were “computational thinking,” “computer
science education,” “computer science domains,” “computer science activities,” and
“computer science pedagogy,” with the additional signifiers “K- 5,” “K-8,” and “K-12”
appended to each preceding term. Via examination of abstracts, we retained articles
that (1) discussed definitions of CS and CT and attempted to break it into components, (2)
discussed or described tools and interventions that focus on CS education in K-8, and/or
(3) were empirical studies of either of the preceding.
We next applied the same search criteria to the prior 10 years (2006–2016) of the
proceedings from the Special Interest Group on Computer Science Education (SIGCSE)
Technical Symposium and the Innovation and Technology in Computer Science
Education (ITiCSE) conference, discarding articles that did not address K-8 or include
direct work with students. We focused on empirical studies in this part of the search to
concentrate our efforts on understanding the ways student thinking develops.
Based on these searches, we read the full text of 108 articles. Two researchers inde­
pendently read each article and documented the LGs it contained. We defined an LG as
any explicit statement or implicit endorsement of what students can or should be able to
do in relation to CS. Example heuristics for identifying LGs are in Table 1.
When possible, researchers recorded the LGs verbatim. However, minor edits in word
form or tense were often made to make sure the resulting text read like a goal and could
be understood out of context.
After the two independent reviews of each article, readers met to discuss the LGs and
resolve discrepancies. In rare cases where the readers could not agree on an LG, a third
reader read the article and the disagreement was resolved through discussion. The
average agreement on LGs (across topics) before discussion was 67%, and 100% agree­
ment was reached through discussion. This process resulted in 671 LGs extracted from the
articles.
To gather LGs related to particular ideas, we conducted keyword searches on the text
of the LGs. For variables, these keywords were: variable, input, list, string, numeric,
Boolean, value, assign, load, store, hold, update, parameter, data, initial*, and declaration.
These searches resulted in 70 LGs. Ten were discarded for being too general to reflect
a focus on variables (e.g. Attend to precision in strings and commands) or too advanced to
be reasonably addressed in K-12 (e.g. Ascribe the appropriate level of “semantic meaning to

Table 1. Heuristics for identifying LGs (Rich, Strickland, & Franklin, 2017).
What students can do What students should be able to do
Explicit List of assessment topics students Description of a taxonomy of CS concepts to be
Statement successfully completed taught in K-12
Implicit Aggregate analyses of student work samples Description of the most important or foundational
Endorsement aspects of CS
6 K. M. RICH ET AL.

variable declarations”), leaving a collection of 60 variables-related LGs that form the basis
of the rest of this paper.

Step 2: synthesizing LGs to identify skill- and fact-based goals

Confrey et al. (2014) conceptualized their task as mapping mathematics standards onto
levels of thinking derived from research literature. As such, their next two steps involved
first elaborating the clusters of standards, and then relating those to the literature.
Because we relied on research literature for both deriving levels of thinking and standard-
like goals, steps 2 and 3 of our process each involved synthesizing the LGs from the
literature, with different aims in mind.
First, we focused on articulating a representative collection of the facts and skills
articulated within the LGs. Three researchers, two with curriculum development expertise
and one with CS expertise, independently grouped the LGs according to the knowledge
or skill expressed in the LG. They did not use a list of predetermined categories, but
nonetheless several common categories emerged, likely because of similarity in wording
across many LGs. For example, all three researchers created a group focused on initializa­
tion based on the following LGs, which all contain a direct reference to initialization:

● Learn “the importance of initial conditions and parameters.” (Gregg et al., 2012)
● “Abstract from the problem the need to initialize and synchronize costume with
motion.” (Seiter, 2015)
● Attend to “initialization” with respect to attributes including “size,” orientation,
background, “visibility,” “costumes,” and “position.” (Franklin et al., 2013)
● Initialize the properties of a sprite. (Seiter, 2015)
● Understand what “initialization is and why it is important.” (Grover et al., 2015)

Groups that were identified by at least two researchers were retained. Groups identi­
fied by only one researcher were discussed by the three researchers to determine if the
groups should be retained, combined with another group, or eliminated. This process
resulted in eight groups of LGs, each focused on one idea related to variables. Four were
identified by all three researchers, two were identified by two researchers, and two were
originally identified by only one researcher but retained after discussion. For each group
of LGs, the first author then articulated a consensus goal (CG) that captured the shared
ideas of the LGs in the group. The other two researchers then reviewed these CGs to
comment on how well they reflected the ideas from the literature. These comments were
incorporated when all three researchers agreed the changes better reflected the litera­
ture. Disagreements were resolved through discussion.

Step 3: developing a progression in levels of thinking

To derive levels of thinking, we sorted the LGs according to the ways in which they
required students to think about variables. This interpretive sort required additional steps.
Many LGs were stated in general terms of what students do or know about variables,
offering only minimal information about the kinds of thinking students were intended to
undertake. For example, Grover et al. (2015) articulated the following LG: “Students will learn
 COMPUTER SCIENCE EDUCATION 7

how variables are created, used, assigned values, and updated” (p. 209). To infer and articulate
how researchers hoped students would think about variables as they achieved these LGs, we
returned to the source literature to gain a sense of the context for the learning. For descrip­
tions and studies of curricula and workshops, we documented the activities with which
students engaged. This allowed us to infer a sense of how students might think about
variables as mental objects, and how they might cognitively operate on variables, as they
completed the activities. For post-hoc analyses of student programming projects, we looked
for relationships among LGs within the articles. For example, if many student work samples
showed evidence of successfully achieving one LG, and only a few work samples showed
evidence of students achieving a related LG, we considered whether this might indicate that
some students had moved to a new way of thinking about variables. Table 2 shows two
examples of annotated LGs, one from a workshop study and one from an analysis of student
work.
Using these annotated LGs, two researchers with curriculum development expertise
looked for themes that could indicate different levels of thinking, first working indepen­
dently and then collaboratively. For example, the quiz question from the first annotation
in Table 2 suggests the authors hoped students would begin to think about variables as
holding places for data. LGs stemming from instructional interventions that also aimed at
this kind of thinking about variables were placed in a group. The difference in how
variables were used by second and fourth graders, described in the second annotation,
suggested that there might be different levels of thinking relating to using variables and
creating them. Other annotated LGs suggesting the same idea were grouped together.
Based on these conversations around the themes in the annotated LGs, the two research­
ers collaboratively developed descriptions of levels of thinking, returning to the source
literature as needed to clarify the evidence. No interrater reliability was tracked because
the process was highly iterative and collaborative. One additional researcher with CS

Table 2. Sample annotated LGs.

Learning Goal and Source Annotations
Achieve a basic understanding of “input, variables, and LG comes from the description of the final lesson of the
conditionals (if/then/else)” (Franklin et al., 2013) camp. Lesson description: “Lesson 5: Interactive
This is a study of an interdisciplinary 2-week summer Conversation was taught after the students had already
camp with middle schoolers. started the culminating project, and consisted of a series
of warm-ups introducing input, variables, and
conditionals (if/then/else). . . . We used a written quiz-like
assessment (Table 4) for the concepts taught in this
lesson.” (p. 373) This quiz question relates to variables:
“How can you keep track of how many times the bat has
been hit by a banana?” [answer is variable]
Use variables within the chosen design patterns. (Seiter Variables are discussed as an “evidence variable” –
& Foreman, 2013) a primary construct used within design patterns.
This paper reports on a pilot test of the PECT model Each evidence variable has levels of basic, developing,
for detecting sophistication of CT within Scratch and proficient. For variables, these are using existing
projects. Scratch variables, creating new variables, and creating
new lists, respectively.
Variables are also included within the “User Interaction”
pattern (using an existing variable) and the “Maintain
score” pattern (creating a new variable). User interaction
was used by G2 and above. Maintain score was used by
G4 and above.
8 K. M. RICH ET AL.

expertise then commented on the levels of thinking. Comments were incorporated into
the levels when the suggested changes were consistent with the literature base.

Step 4: utilizing the progression and CGs to develop an LT

Next we designed instructional activities to facilitate moving students along the levels of
thinking. We aimed to create activities that included external objects similar to the
cognitive objects we wanted students to develop, and opportunities to externally manip­
ulate those objects in ways that paralleled the mental actions we hoped students would
develop (Sarama & Clements, 2014). In unplugged activities, the objects were concrete
and the actions were physical. In Scratch activities, the objects were sprites and the
actions involved programming.
As we developed activities, we took into account the fact- and skill-based CGs we had
articulated in Step 2 and how they could be incorporated into activities designed to
facilitate movement through the levels of thinking. A number of considerations influ­
enced which and how many fact and skills goals we addressed, and at what point in the
activity sequence:

● Limited instructional time. We could address only a subset of the facts and skills given
we were developing a curriculum to address multiple CS topics.
● Age level and context. We focused specifically on fourth graders and integration of CS
into compatible mathematics.
● Programming environment. Scratch had particular affordances and constraints that
influenced the activities and the choice of knowledge and skills they addressed.

When activity development was complete, we incorporated information about the

activities and the CGs they addressed into a diagram modeling how the activities
related to progression through the levels of thinking. This mapping coordinated the
three elements of an LT: an overall learning goal (fluent use of variables), a pathway
to the learning goal (through four levels of thinking), and activities to facilitate
movement along the path. Our LT served as a proof-of-concept for how the litera­
ture-based levels of thinking and CGs could be flexibly intertwined to support
variables instruction tailored to a particular context.
We piloted the activities during two school years. In 2018–2019, four classrooms of
fourth grade students in three schools piloted the variables-related instructional activities,
along with additional integrated mathematics+CT activities addressing other CT topics. In
2019–2020, six classes of fourth graders in two schools piloted the activities. Information
about the schools and number of participants is in Table 3.
We collected completed student pages, which included questions about the
meaning of the Scratch code, how changes to the code affected the project, and
how students might plan and implement particular tasks in Scratch. While
a thorough report of student and teacher experiences is beyond the scope of this
paper, we present limited teacher and student data in the results to demonstrate
that students were able to successfully engage with the activities and produce some
tentative evidence of student learning.
 COMPUTER SCIENCE EDUCATION 9

Table 3. Pilot school demographics.

Consented % Passing Math State % Free-Reduced
Pilot Year School Type Classrooms Students Assessment Lunch
2018–2019 Public, 1 20 62.5 42.9
Suburban
2018–2019 Public, Urban 1 28 57.9 14.7
2018–2019 Private, Urban 2 22 Unavailable Unavailable
2019–2020 Public, 3 53 62.3 48.1
Suburban
2019–2020 Public, 3 70 85.9 11.3
Suburban

Table 4. Fact- and skill-based consensus goals.

Consensus Goal Source articles Sample LG
Types. Know variables are categorized in Liu et al., 2011; Liu et al., 2014; “[K]now how to create, change, and use
terms of the type of data they hold Peluso & Sprechini, 2012 three basic types of variables:
and operations that can be performed numbers, strings, and Booleans” (Liu
on them, and when and why different et al., 2011)
data types might be used.
Updating. Know variables hold one Angeli et al., 2016; Grover et al., Understand how “variables are created,
value at a time, and the value of 2015; Taylor et al., 2014; used, assigned values, and updated ”
a variable may be updated in a variety Werner et al., 2014 (Grover et al., 2015, emphasis added)
of ways for a variety of purposes.
User Input. Know how and when to Franklin et al., 2013; Grover Know “how to accept and use data and
accept user input, and why this might et al., 2015; Liu et al., 2011; Liu keystrokes supplied by the user” (Liu
be useful. Define and use a variable et al., 2014; Werner et al., et al., 2011)
for the purpose of holding user input. 2014
(3 additional)
Assignment. Know the distinction Taylor et al., 2014 Correctly interpret “the assignment
between assignment of values and operator” (not “as an equality
checking for equivalence of values comparison operator”) (Taylor et al.,
and how to apply each in code. 2014)
Initialization. Know when and why it is Franklin et al., 2013; Gregg et al., Learn “the importance of initial
important to specify initial conditions 2012; Grover et al., 2015; conditions and parameters”. (Gregg
or initial values for variables. Seiter, 2015; Seiter & et al., 2012)
Foreman, 2013
Variables & Flow of Control. Know Grover et al., 2015; Liu et al., “[U]se simple Boolean expressions to
how variables in Boolean expressions 2011; Liu et al., 2014; Seiter & affect control flow.” (Liu et al., 2014)
can define the flow of control through Foreman, 2013
a program.
Parameters. Know the purpose and role Gregg et al., 2012; Peluso & Analyze and “determine values for the
of parameters in defining the Sprechini, 2012; Werner et al., parameters.” (Werner et al., 2012)
behaviors of methods. 2014; Werner et al., 2012
(2 additional)
List Variables. Know the purpose of list Liu et al., 2014; Werner et al., Use “list variables (student created)”
variables, and use list variables within 2014; Werner et al., 2012 (Werner et al., 2014)
a program.

Results
We present three sets of results: (1) the list of fact- and skill-based CGs derived from
literature; (2) the literature-based progression of levels of thinking; and (3) the LT we
created for our context, as an elaborated example of the potential use of (1) and (2).
10 K. M. RICH ET AL.

Fact- and skill-based goals for variables

Table 4 presents the eight fact- and skill-based CGs about variables we derived from
literature. The middle column lists most of the articles that contained an LG that was
included as support for the CG. For space considerations, we omitted articles that con­
tained support for just one CG and were not cited elsewhere in this paper. The right
column gives the text of one of the LGs.

Progression in levels of thinking for variables

Table 5 shows the four levels of thinking derived from the literature, presented in the
order we theorize students arrive at them. The first level relates to data generally, and the
rest represent gradually more sophisticated ways of thinking about variables. The right
column contains information about the literature from which these levels were derived.
We provide the text of two LGs for each level that we judged to best illustrate why we
specified each level as students having particular cognitive objects (the first LG) and
undertaking particular cognitive manipulations (the second LG). Then we list how many
LGs support the level in total and the number of articles from which the LGs were drawn.
We placed the Data User level first because we considered it important to start the
Variables LT with attention to the meaning of data as pieces of manipulable information,

Table 5. Levels of thinking.

Level Name and Description LGs and Source Articles
Data User. Students at this level have mental objects Understand “what data are, and how data is used in
representing general pieces of information, and can a program” (Grover et al., 2015)
operate on them in ways that mimic how a program Produce and manipulate game content to discover the
might use the data. Students at this level may role of variables. (Isomöttönen et al., 2011)
manipulate the data they are using, but do not label or Total: 2 LGs from 2 articles
store it for later use.
Data Storer. Students at this level have mental objects Understand that “variables are an abstraction or
representing variables. They can mentally place objects representation of data in the program” (Grover et al.,
representing data into the variables, and view the data 2015)
stored in their variables. These mental actions mimic “[L]oad a picture into a variable and” “work with such
the actions completed by a computer when it stores the variables” (Pasternak & Vahrenhold, 2012)
data in variables and reads the values. In other words, Total: 13 LGs from 11 articles
they can trace instructions that involve the
manipulation of data but may not be able yet to
produce or modify code that manipulates data.
Variable User. Students at this level continue to have Use variables within chosen design patterns. (Seiter &
mental objects for variables and data, but begin Foreman, 2013)
operating on them from the perspective of “[K]now how to . . . use three basic types of variables”.
a programmer rather than a computer. Students at this (Liu et al., 2011)
level can understand and modify code that manipulates Total: 3 LGs from 3 articles
variables, but they may not yet be able to coordinate
the considerations required to create new variables in
code.
Variable Creator. Students at this level can take more Effectively implement “abstraction and modeling
highly specified cognitive actions on variables, such as constructs” (including “variables”) such that they are
distinguishing between initialization and updating, and “reachable” and do not “cause abnormal program
associate those cognitive actions to programming termination”. (Werner et al., 2012)
commands. They can coordinate multiple operations. “[P]erform variable manipulation and proper type
usage.” (Liu et al., 2014)
Total: 6 LGs from 6 articles
 COMPUTER SCIENCE EDUCATION 11

to form a foundation for students to conceptualize the relationship between data and
variables in CS.
Next is the Data Storer level, where students conceptualize variables as storage spaces.
Pasternak and Vahrenhold (2012) introduced variables as storage spaces, choosing spe­
cifically to store images due to their connection to concrete objects. They theorized this
would support students’ move from concrete to abstract thinking. We considered this
a way of identifying a first level of thinking about variables as labeled places to store data.
Other articles also included LGs emphasizing that variables store data, without referen­
cing why data might be stored (e.g. Angeli et al., 2016; Grover et al., 2015).
At the next level, Variable User, students think about and manipulate individual
characteristics of existing variables from the perspective of a programmer. This level is
included because several of the reviewed articles suggested students are able to use and
interpret existing variables before they successfully create their own variables (Seiter &
Foreman, 2013; Werner et al., 2014). Additionally, other source articles highlighted con­
cerns involved with using variables correctly, including attention to type (Liu et al., 2014;
Peluso & Sprechini, 2012) and assignment and updating (Taylor et al., 2014). We inter­
preted this evidence as suggesting students can think about and manipulate basic
characteristics of variables (e.g. types), perhaps in isolation, before they can coordinate
multiple characteristics when they create new variables.
The last level is Variable Creator, which includes attention to thinking about how to
declare, initialize, and update a new variable. Unlike papers that included attention to
basic, isolated aspects of variables, Werner et al. (2012) highlighted the need for variables
to be included in code in ways that do not cause termination. This pointed to a level of
thinking beyond the ability to conduct simple operations on variables. This level involves
coordinating those actions in ways that allow for the creation of new variables.
Because the Data User and Data Storer levels are viewed from the perspective of
a computer (a computer is the agent that uses and stores data) rather than
a programmer, students can be encouraged to adopt these ways of thinking through
unplugged and embodied activities. This is another reason they come early in the
progression. In the latter two levels, Variable User and Variable Creator, students take
the perspective of a programmer (a programmer is the agent that uses and creates
variables).

Elaborated trajectory for a grade 4 mathematics + CS curriculum in Scratch

As a proof-of-concept of how the progression and CGs might be elaborated into an LT for
a particular context, we used these tools as guides for developing activities for our
integrated mathematics + CS curriculum, which uses Scratch and is targeted to grades
3 and 4 (Strickland et al., 2021). Our curriculum aimed to address six areas of CS (sequence,
conditionals, iteration, decomposition, debugging, and variables). Thus, the variables
activities are one piece of a larger picture. In this paper, we present only the five activities
that address variables, which all appear in grade 4.
We first present descriptions of each activity and how they reflect our efforts to
facilitate movement through the levels of thinking. Then we discuss which of the fact
and skill goals we found easiest to address while focusing on levels of thinking and
consider how elements of our context influenced which goals were addressed.
12 K. M. RICH ET AL.

Moving from data user to data storer: robot boxes

Robot Boxes is an unplugged activity introducing variables. In accordance with the
early levels of the trajectory, students do not program a computer; rather they act
from the perspective of a computer to simulate how data and actions flow through
a program. Students work with six labeled boxes (variables) called message, length,
width, action, measure, and answer. There are three roles: Filler, Calculator, and Robot.
The Filler chooses values for message (something the Robot will say), length and width
(numbers of units), action (something the Robot will do, like jump), and measure
(either area or perimeter). The Filler records the values on index cards and places them
in designated boxes, similar to the way a computer stores a value in a variable. The
Calculator checks the value in the measure box to determine if the Filler wants the
area or perimeter, performs the calculation based on the values in length and width,
and places that value in the answer box. Finally, the Robot follows an algorithm to
perform a silly dance using the values from all six boxes, encountering and using
variable values the way a computer might. Students switch roles and play the game
again. Working with pieces of information on index cards throughout the game
reinforces their understanding of data as general pieces of information (the Data
User level), which we also assume they bring to the activity from their experiences
with manipulating dimensions to calculate area and perimeter in past lessons. At the
end of the lesson, students participate in a whole-group discussion that introduces
variable as vocabulary and connects it to the Robot’s boxes, supporting their transition
to the Data Storer level. They also discuss whether the value in the message box can
be switched with the value in the length box, providing an introduction to variable
types.

Moving from data storer to variable user: math chat, ambling animals, and slicing
sandwiches
In the following lessons, students shift to the perspective of a programmer and are introduced
to variable blocks in Scratch, which supports their transition to the Variable User level. In Math
Chat, the idea of “having a chat” is used to help students think about input and output.
Teachers and students inspect an example program that calculates the area of a rectangle by
using a computation block to multiply two hard-coded numbers. Students consider how the
program could be changed to calculate the area of a different rectangle. After realizing the
limited utility of a program whose code must be changed for every rectangle, students then
examine a different program that asks for user input for a rectangle’s dimensions, stores the
values in variables, and outputs the result of a calculation with them. They modify the
program to output the area of the rectangle with the inputted dimensions.
In Ambling Animals, a Scratch program randomly generates two fractions between 0 and
1, places them on their own number lines marked with a frog and a crab, and displays the
value stored in a variable called GreaterAnimal (see Figure 1). Students run the program
several times and determine what is being stored in the variable (the name of the animal
marking the greater fraction). Students input a guess at which animal is on the greater
fraction and observe what the computer outputs in response. When students examine the
code, they attend to the use of variables to store both the name of the animal marking the
greater fraction and the user input. They modify the code so the computer will compare
the user input to the correct answer and provide feedback on the user’s answer.
 COMPUTER SCIENCE EDUCATION 13

Figure 1. The Ambling Animals activity.

In Slicing Sandwiches, students use a Scratch program to compare two fractions with
the same denominator. They run the project several times to generate pairs of fractions to
compare. Using the program’s output, they generate rules for comparing two fractions
with the same denominator based on the values of the numerators. The programming
tasks ask students to build a series of conditionals to display the correct inequality symbol
based on comparisons of numerator values stored in variables.

Moving from variable user to variable creator: adding fractions

In Adding Fractions, students manually add a variable into code for the first time, beginning
their transition to the Variable Creator level. Students use code that shows a bird making
“hops” on a number line to represent a sum of fractions with the same denominator. The
denominator value appears in multiple places in the code, leading students to consider how
they could replace each instance with a variable so they do not have to change the
denominator in many places when they want to show a new problem. The student pages
walk students through placing the variable in the code. When they run the program, if they
have not assigned an initial value to the variable, the program detects this and reports an
error.

Fact and skill goals addressed through our activities

Figure 2 shows the LT along which our activities are intended to direct students. The
figure shows the five fact- and skills-based CGs we addressed in some form in our five
instructional activities and which activities address each goal. We acknowledge we cannot
14 K. M. RICH ET AL.

Figure 2. Variables LT for fourth grade, Scratch-based, integrated CS+math curriculum.

claim to have taught these concepts in their entirety, even within the context of Scratch.
Rather, our claim is that we were able to create activities that both (1) provided exposure
to these facts and skills related to variables, and (2) have the potential to support students’
advancement along the levels of thinking in the LT. We describe the placement of each
goal below, with reference to which of our three constraints most strongly affected each
placement: the Scratch environment (for Types), integration with mathematics (for Flow
of Control and User Input), or limited instructional time (for Flow of Control, User Input,
Updating, and Initialization).
As noted in the description of Robot Boxes, we included attention to Types in an activity
designed to help students shift from Data User to Data Storer. We placed Types in Robot
Boxes due to its the unplugged context. Scratch does not obviously distinguish among
types. For example, the environment allows for text and numbers to be held in the same
variable. Thus, when working in Scratch, it is difficult to discuss variable types in
a programming project. By contrast, the unplugged context allowed us to “define”
variables that are clearly intended to hold different types of data. Our observations
suggested students understood the idea of Types in an unplugged context, but because
the particular nature of our plugged activities made it difficult to address Types in
a programming activity, we cannot speculate about how much attention to Types
impacted a shift in students’ levels of thinking. This is why there is a dotted arrow from
Types in Figure 2.
As shown in Figure 2, we were able to address goals related to user input and control
flow in our activities designed to support a move from the Data Storer level to the Variable
User level. This was facilitated by our context of integrating CS into an existing mathe­
matics curriculum. There were situations within the mathematics curriculum that fit well
with the CS ideas of user input and control flow. For example, Math Chat connects user
input to using length and width to calculate areas of rectangles. Ambling Animals and
Slicing Sandwiches connect using a conditional with a Boolean variable to direct the flow
of control to making comparisons based on mathematical properties.
To incorporate these connections into our curriculum while keeping in mind the
limited instructional time available, we focused students’ attention on using and inter­
preting the behavior of Scratch programs that used variables to hold user input and
control the flow of the programs. Math Chat, Ambling Animals, and Slicing Sandwiches all
provide opportunities for students to think about user input or control flow offline, then
 COMPUTER SCIENCE EDUCATION 15

use a program that utilizes variables and these concepts, then make some changes to the
program. The activities therefore address the User Input and Variables & Flow of Control
CGs while also helping students transition from thinking about variables from the per­
spective of a computer to thinking about variables from the perspective of a programmer.
Lastly, we found that our focus on having students modify code, rather than create
new code, left few opportunities to focus on the process of updating a variable. Instead, it
came up more naturally in Adding Fractions, where students add a variable into their code
for the first time. Here, although students do not create (or define) the variable them­
selves, they move beyond using variables that already appeared in the code, and thus we
felt we should place the Updating goal as supporting a shift beyond the Variable User
level. However, we acknowledge that we did not address creating variables nor coordi­
nating multiple ideas about variables, as described in the Variable Creator level. Thus our
curriculum can support, at best, a partial transition to the Variable Creator level.

Preliminary teacher and student data

Here we provide information about teacher evaluations and student work to establish
that the activities were accessible to fourth graders and provide early suggestions of
student learning.
Robot Boxes has only a brief record sheet for students to complete, so we have minimal
student work from this activity. However, during the second year of implementation, the
five participating teachers (one of whom had two classes of students) completed
a feedback form for each lesson. We read through the forms for Robot Boxes, pulling
out any comment relating to students’ understanding of variables. These comments are
included below and indicate teachers found the activity engaging and felt it helped
students conceptualize variables:

● “The students were able to understand the variables that go into each code and that
the outcome can be impacted by the variables.”
● “It went pretty well and students understood what to do” but a challenge was
“students not understanding to change the variable at first.”
● “I heard some students say ‘Oh. I get what variables are now.’ There were also
comments like ‘I thought the only variables in computer coding are zeros and one.’”

Math Chat included the questions in Figure 3. The percent of students that were able to
answer each of these questions correctly was 86%, 57%, and 62%, respectively, suggest­
ing students were beginning to make sense of variables embedded in Scratch code on
their first exposure during this activity. Ambling Animals contained a similar set of ques­
tions asking students to identify the purpose of variables-related blocks, this time with
attention to storing user input in a variable (Figure 4). The percent of students who
answered each of these questions correctly was 69%, 63%, and 68%, respectively. Thus,
more than half of students seemed to successfully trace how user input is stored in the
answer variable and then used by the computer. Lastly, Adding Fractions contains the
question in Figure 5, which explicitly asks students to identify the variable block as serving
a specific purpose. About 62% of students answered this question correctly. We do not
make any definitive claims about student learning based on this data, as the student
pages and teacher evaluations do not directly measure student learning. We merely
16 K. M. RICH ET AL.

Figure 3. Math Chat student questions 6–8.

Figure 4. Ambling Animals student questions 6a-6d.

Figure 5. Adding Fractions student question 9a.

present this data as evidence that our proof-of-concept attempt at using a literature-
based progression and a process of activity development to create a context-specific LT
for variables produced a curriculum that is accessible to students and feasible for use in
classrooms.

Discussion
We presented four levels of thinking about variables and eight fact- and skill-based goals
related to variables derived from CS education literature (Tables 4 and 5). Further, we
described how we used levels of thinking, combined with constraints of our context, to
develop a sequence of activities aimed at developing students’ ways of thinking about
variables while addressing a limited number of facts and skills that fit most easily into our
context. We presented preliminary evidence that students were able to successfully
engage with the activities and teachers were able to use them in their classroom contexts.
 COMPUTER SCIENCE EDUCATION 17

We hope our process can serve as a model for how instruction on variables might be
developed that is suited for a variety of contexts and age groups.
In this section, we first discuss the four levels of thinking in more detail, highlighting
both how they influenced our activity design and implications for other curriculum
designers. Then we discuss the generality of the levels, providing an example of how
they might be applied in courses that use other programming languages. Finally, we share
some reflections on our activity development process and piloting and how our experi­
ences suggest directions for future work.

Characterizing the levels of thinking

LTs are based in a constructivist view of learning (Simon, 1995), where students construct
their own knowledge as they make sense of their experiences. As such, our curriculum
design process foregrounds the tasks that students will undertake, while our support
materials focus on helping teachers become facilitators of that activity. The levels of
thinking were an important conceptual tool we used to develop student tasks, as they
helped us consider the perspective from which we wanted students to consider the
content and the connections we hoped they would make between ideas.
The progression begins at the Data User level. Students at this level can think about
individual bits of information and how those pieces of information might be used and
related to each other. High-school and middle-school students may already have cogni­
tive objects for data based on their previous experiences. For our elementary audience,
we designed Robot Boxes to directly represent pieces of data on index cards to help
students develop cognitive objects for pieces of data. To help them move to the Data
Storer level and develop cognitive actions of labeling and storing data in designated
places, we had them take on the role of a robot – in a sense, a personified computer – and
physically move, label, store, and retrieve the index cards. This approach is similar to
Papert’s (1980) notion of “playing Turtle”. In these first two levels, students are not taking
the perspective of a programmer, or someone telling a computational agent to handle
data. Rather, they are the agent handling the data. This is parallel to how students
“playing Turtle” are the Turtle rather than the person controlling the Turtle.
In our context, we felt having students physically act out the execution of a program
would support them in taking the perspective inherent to the Data User and Data Storer
levels. Educators of older students may develop ways to achieve the same aims through
mentally tracing instructions. The Data Storer level, in particular, seems highly compatible
with thinking about roles of variables. Sajaniemi and Kuittinen (2005) noted,
[I]n the traditional approach [to teaching variables], a variable has no special meaning by
itself but is only the object of some—to a novice apparently more or less incidental—
assignments. Role knowledge turns this situation upside down: a variable is an active subject
taking care of some specific task. (p. 79)

These researchers see roles of variables as a way for students to think of variables as active
agents. They help students “be the variable,” which is related – although not identical – to
“being the computer” at the Data Storer level or “playing Turtle” for Papert (1980).
At the Variable User level, students shift their perspective to that of a programmer.
Rather than directly manipulating and storing data, they begin to view variables as tools
18 K. M. RICH ET AL.

they can use to direct a computational agent to do the manipulation and storage. As we
designed activities to support this level of thinking, we focused on selecting contexts that
highlighted the purposes for which a programmer might use a variable – for example, to
store user input or control flow. Inspired by Sajaniemi and Kuittinen (2005) roles, we argue
that the idea of developing a similar set of variable purposes, and using them to organize
instruction at the Variable User level, is worthy of further investigation.
Students maintain the programmer perspective as they progress to the Variable
Creator level, but they expand the why thinking they do at the Variable User level to
include more how thinking. This progression aligns with Grover et al.’s (2019) concepts-
before-constructs approach. At the Variable User level, students can think about why
a programmer might need a variable and what it will do from a practical perspective. At
the Variable Creator level, students can coordinate this thinking with how a programmer
might implement a variable and what it will do from a technical perspective. This suggests
curriculum developers might view the transition from Variable User to Variable Creator as
a place to relate purposes of variables to Sajaniemi and Kuittinen (2005) roles of variables.

Generality of the levels of thinking

Much of the K-8 literature is based on the Scratch programming language, which can
affect learning goals and their orders for specific concepts. For example, the repeat loop
allows a single integer to specify the number of iterations in Scratch, whereas most
commonly-used text-based languages require the use of variables and conditionals for
even a simple loop. Given these potential dependencies on language, a pertinent ques­
tion is, how dependent on language are the levels of thinking at the core of the Variables
LT? To explore this question, we ask whether learning tasks aligned to the levels of
thinking could be created using other, potentially text-based, languages. We propose
that the answer is yes. At the Data User level, one might ask a student to evaluate a simple
expression, expressed in any language, with data but not variables. At the Data Storer
level, a student could be asked to declare and initialize a variable for later use or evaluate
an expression with a variable in it. A Variable User can modify the variable many times,
graduating to a more complicated code trace. Finally, the Variable Creator level could be
assessed through asking students to implement a function that requires variables. While
Scratch has many differences compared to other languages, these examples for all four
categories would be equally useful in any language in common use for CS instruction,
such as Python, Java, C, or Scheme.

Directions for future work on the levels of thinking

Our experiences with developing and piloting activities based on the four levels of
thinking about variables suggest at least two directions for future research. First, we
note that because we could only fit a handful of activities involving variables into
one year of curriculum, we focused on moving students through the first three levels
of thinking. However, we do not view the levels as strictly hierarchical. The levels of
thinking in an LT progress in sophistication, but moving through the levels does not
preclude a student from “falling back” to take the perspective engendered by a previous
level (Battista, 2011). Indeed, such changes in level can be helpful for making sense of
 COMPUTER SCIENCE EDUCATION 19

new contexts. For example, as students encounter more complex uses of variables in
programs, they may find it useful to “walk through” a program at the Data Storer level,
even though they have manipulated simpler programs at the Variable User level.
Indeed, explicitly encouraging students to take these shifts in perspective may be
a useful instructional strategy. Future research could focus on developing levels-based
curricula across a longer time period and investigate the benefits and costs of deliber­
ately asking students to shift backward among levels of thinking as they develop their
understanding of variables.
Second, our activity development process suggested that there may be value in
articulating and testing an additional level of thinking between the Variable User and
Variable Creator levels. As shown in Figure 2, we view three of our instructional activities
as supporting a shift beyond the Data User level: Math Chat, Ambling Animals, and Slicing
Sandwiches. In all of these activities, students interpret and modify code containing
a variable, effectively shifting from the computer’s perspective (as in Robot Boxes) to the
programmer’s perspective. This was an important element of the distinction between the
Data Storer and Variable User levels. However, these activities do not meaningfully
address concepts critical to helping students implement variables as programmers,
such as initialization or updating. Rather, they focus on the purposes for which
a programmer might use a variable, with particular attention to storing user input and
directing control flow.
While Adding Fractions addresses another particular purpose for variables – namely,
using the same value multiple times – the main conceptual leap promoted by the lesson is
thinking about how introducing a variable into code creates a need for initialization and
updating. Thus, there seems to be a shift in the ways of thinking about variables in this
activity, but it does not push all the way to the issue of coordination of all the skills needed
to create a new variable. Thus, we argue that additional activities are needed to support
students’ transitions to the Variable Creator level. Future research could examine whether
an intermediate level might exist between Variable User and Variable Creator, wherein
students are able to operate on new variables with respect to one attribute at a time but
not coordinate all the attributes necessary to create a variable from scratch. We have
tentatively named such a level the Variable Implementer level. Future research could
examine whether the level represents a meaningful distinction in student thinking about
variables that has utility for curriculum development and instruction.

Limitations
The levels of thinking and CGs were based on a broad review of research in K-12 CS
education from 2006–2016. Thus, they may not include ideas studied in more recent
research. Additionally, because the literature review was not specifically focused on
variables, it is possible we missed important work. We hope that our progression in levels
of thinking serves as a starting point for future work that incorporates more existing work
on variables as well as the results of new studies sparked by this work.
Although we piloted our variables activities and collected student work, we did
not collect data specifically designed to capture students’ levels of thinking or their
content knowledge. Thus, we can make few claims related to student thinking or
learning. Rather, we share how the process of developing the instructional activities
20 K. M. RICH ET AL.

was influenced by the progression at the core of our LT and also influenced how
details of the LT were elaborated.

Conclusion
This paper makes two contributions. First, we presented a sequence of levels of thinking
about variables that can serve as a new conceptual tool, perhaps alongside other frame­
works (e.g. Sajaniemi et al., 2006), for designing instruction addressing this important and
challenging topic. Second, we described our process of developing activities aligned with
the levels of thinking and suited to our context as proof-of-concept of a process for using
the levels flexibly in curriculum development. This process illustrates how the levels of
thinking, and general process of LT development around the levels, may be adaptable to
multiple ages and the needs of particular groups. For example, if the LT is used to design
a stand-alone CS high school course, different facts and skills may be better fits for
facilitating progress through levels of thinking, as the affordances and constraints of
such a course will differ from those in our context of integrating CS into fourth-grade
mathematics. We hope the levels of thinking and the general process may be useful for
creating instruction for all students.

Disclosure statement
The authors declare no conflict of interest.

Funding
This work was supported by the National Science Foundation under grant numbers 1742466 and
1542828. Any opinions, findings, and conclusions or recommendations expressed in this material
are those of the author(s) and do not necessarily reflect the views of the National Science
Foundation.

ORCID
Kathryn M. Rich http://orcid.org/0000-0003-3162-2980

References
Angeli, C., Voogt, J., Fluck, A., Webb, M., Cox, M., Malyn-Smith, J., & Zagami, J. (2016). A K-6
computational thinking curriculum framework: Implications for teacher knowledge. Journal of
Educational Technology & Society, 19(3), 47–57. https://www.researchgate.net/publication/
305140678
Battista, M. T. (2011). Conceptualizations and issues related to learning progressions, learning
trajectories, and levels of sophistication. The Mathematics Enthusiast, 8(3), 507–570. http://scho­
larworks.umt.edu/tme/vol8/iss3/5
Buffum, P. S., Price, K., Zheng, X., Boyer, K. E., Wiebe, E. N., Mott, B. W., . . . Lester, J. C. (2018, February).
Introducing the computer science concept of variables in middle school science classrooms. In
Proceedings of the 49th ACM Technical Symposium on Computing Science Education (pp. 906–911).
Baltimore, MD: ACM. https://doi.org/10.1145/3159450.3159545
 COMPUTER SCIENCE EDUCATION 21

Clements, D. H., & Sarama, J. (2004). Learning trajectories in mathematics education. Mathematical
Thinking and Learning, 6(2), 81–89. https://doi.org/10.1207/s15327833mtl0602_1
Confrey, J., Maloney, A. P., & Corley, A. K. (2014). Learning trajectories: A framework for connecting
standards with curriculum. ZDM - Mathematics Education, 46(5), 719–733. https://doi.org/10.1007/
s11858-014-0598-7
Du Boulay, B. (1986). Some difficulties of learning to program. Journal of Educational Computing
Research, 2(1), 57–73. https://doi.org/10.2190/3LFX-9RRF-67T8-UVK9
Dwyer, H., Hill, C., Carpenter, S., Harlow, D., & Franklin, D. (2014, March). Identifying elementary
students’ pre-instructional ability to develop algorithms and step-by-step instructions. In
Proceedings of the 45th ACM technical symposium on Computer science education (pp.
511–516). Atlanta, GA: ACM.
Franklin, D., Conrad, P., Boe, B., Nilsen, K., Hill, C., Len, M., . . . Laird, C. (2013, March). Assessment of
computer science learning in a scratch-based outreach program. In Proceedings of the 44th ACM
technical symposium on computer science education (pp. 371–376). Denver, CO: ACM.
Gregg, C., Tychonievich, L., Cohoon, J., & Hazelwood, K. (2012, February). EcoSim: A language and
experience teaching parallel programming in elementary school. In Proceedings of the 43rd ACM
technical symposium on computer science education (pp. 51–56). Raleigh, NC: ACM.
Grover, S., & Basu, S. (2017, March). Measuring student learning in introductory block-based
programming: Examining misconceptions of loops, variables, and Boolean logic. In Proceedings
of the 2017 ACM technical symposium on computer science education (pp. 267–272). Seattle, WA:
ACM.
Grover, S., Jackiw, N., & Lundh, P. (2019). Concepts before coding: Non-programming interactives to
advance learning of introductory programming concepts in middle school. Computer Science
Education, 29(2–3), 106–135. https://doi.org/10.1080/08993408.2019.1568955
Grover, S., Pea, R., & Cooper, S. (2015). Designing for deeper learning in a blended computer science
course for middle school students. Computer Science Education, 25(2), 199–237. https://doi.org/
10.1080/08993408.2015.1033142
Hermans, F., Swidan, A., Aivaloglou, E., & Smit, M. (2018, October). Thinking out of the box:
Comparing metaphors for variables in programming education. In Proceedings of the 13th work­
shop in primary and secondary computing education (pp. 1–8). Potsdam, Germany: ACM.
Isomöttönen, V., Lakanen, A. J., & Lappalainen, V. (2011, March). K-12 game programming course
concept using textual programming. In Proceedings of the 42nd ACM technical symposium on
computer science education (pp. 459–464). Dallas, TX: ACM.
Kohn, T. (2017). Variable evaluation: An exploration of novice programmers’ understanding and
common misconceptions. In Proceedings of the 2017 ACM SIGCSE Technical Symposium on
Computer Science Education (pp. 345–350). Seattle, WA: ACM. https://doi.org/10.1145/3017680.
3017724
Liu, J., Lin, C. H., Hasson, E. P., & Barnett, Z. D. (2011, March). Introducing computer science to K-12
through a summer computing workshop for teachers. In Proceedings of the 42nd ACM technical
symposium on computer science education (pp. 389–394). Dallas, TX: ACM.
Liu, J., Lin, C. H., Wilson, J., Hemmenway, D., Hasson, E., Barnett, Z., & Xu, Y. (2014, March). Making
games a snap with Stencyl: A summer computing workshop for K-12 teachers. In Proceedings of
the 45th ACM technical symposium on computer science education (pp. 169–174). Atlanta, GA:
ACM.
Maloney, J., Peppler, K., Kafai, Y. B., Resnick, M., & Rusk, N. (2008, March). Programming by choice:
Urban youth learning programming with scratch. In Proceedings of the 39th ACM technical
symposium on computer science education (pp. 367–371). Portland, OR: ACM.
Meerbaum-Salant, O., Armoni, M., & Ben-Ari, M. (Moti). (2013). Learning computer science concepts
with Scratch. Computer Science Education, 23(3), 239–264. https://doi.org/10.1080/08993408.
2013.832022
Papert, S. (1980). Mindstorms; children, computers and powerful ideas. New York: Basic Books.
Pasternak, A., & Vahrenhold, J. (2012, February). Design and evaluation of a braided teaching course
in sixth grade computer science education. In Proceedings of the 43rd ACM technical symposium
on computer science education (pp. 45–50). Raleigh, NC: ACM.
22 K. M. RICH ET AL.

Peluso, E. M., & Sprechini, G. (2012). The impact of Alice on the attitudes of high school students
toward computing. Journal for Computing Teachers, 7,(Winter 2012), 1–10.
Putnam, R. T., Sleeman, D., Baxter, J. A., & Kuspa, L. K. (1986). A summary of misconceptions of high
school basic programmers. Journal of Educational Computing Research, 2(4), 459–472. https://doi.
org/10.2190/FGN9-DJ2F-86V8-3FAU
Rich, K. M., Strickland, C., & Franklin, D. (2017, March). A literature review through the lens of
computer science learning goals theorized and explored in research. In Proceedings of the 2017
ACM SIGCSE Technical Symposium on Computer Science Education - SIGCSE ’17 (pp. 495–500).
Seattle, WA: ACM. https://doi.org/10.1145/3017680.3017772
Sajaniemi, J., Ben-Ari, M., Byckling, P., Gerdt, P., & Kulikova, Y. (2006). Roles of variables in three
programming paradigms. Computer Science Education, 16(4), 261–279. https://doi.org/10.1080/
08993400600874584
Sajaniemi, J., & Kuittinen, M. (2005). An experiment on using roles of variables in teaching intro­
ductory programming. Computer Science Education, 15(1), 59–82. https://doi.org/10.1080/
08993400500056563
Sarama, J., & Clements, D. H. (2014). Learning trajectories: Foundations for effective, research-based
education. In A. P. Maloney, J. Confrey, & K. H. Nguyen (Eds.), Learning over time: Learning
trajectories in mathematics education (pp. 1–30). Information Age Publishing.
Sarama, J., Clements, D. H., Barrett, J., Van Dine, D. W., & McDonel, J. S. (2011). Evaluation of
a learning trajectory for length in the early years. ZDM, 43(5), 667–680. https://doi.org/10.1007/
s11858-011-0326-5
Seiter, L. (2015, February). Using SOLO to classify the programming responses of primary grade
students. In Proceedings of the 46th ACM technical symposium on computer science education (pp.
540–545). Kansas City, MO: ACM.
Seiter, L., & Foreman, B. (2013, August). Modeling the learning progressions of computational
thinking of primary grade students. In Proceedings of the ninth annual international ACM con­
ference on international computing education research (pp. 59–66). San Diego, CA: ACM.
Simon, M. A. (1995). Reconstructing mathematics pedagogy from a constructivist perspective.
Journal for Research in Mathematics Education, 26(2), 114–145. https://doi.org/10.5951/jrese
matheduc.26.2.0114
Strickland, C., Rich, K. M., Eatinger, D., Lash, T., Isaacs, A., Israel, M., & Franklin, D. (2021, March). Action
Fractions: The design and pilot of an integrated math+CS elementary curriculum based on
learning trajectories. In Proceedings of the 2021 ACM SIGCSE Technical Symposium on Computer
Science Education. Online conference. ACM
Taylor, C., Zingaro, D., Porter, L., Webb, K. C., Lee, C. B., & Clancy, M. (2014). Computer science
concept inventories: Past and future. Computer Science Education, 24(4), 253–276. https://doi.org/
10.1080/08993408.2014.970779
Werner, L., Denner, J., & Campe, S. (2014). Children programming games: A strategy for measuring
computational learning. ACM Transactions on Computing Education (TOCE), 14(4), 24. http://dx.
doi.org/10.1145/2677091
Werner, L., Denner, J., Campe, S., & Kawamoto, D. C. (2012, February). The fairy performance
assessment: Measuring computational thinking in middle school. In Proceedings of the 43rd
ACM technical symposium on computer science education (pp. 215–220). Raleigh, NC: ACM.