Received 2 December 2016; revised 25 July 2017; accepted 28 July 2017.

Date of publication 1 August 2017; date of current version 11 March 2020.

Digital Object Identifier 10.1109/TETC.2017.2734818

Towards Data-Driven Learning Paths to Develop

Computational Thinking with Scratch
JESUS


MORENO-LEON

, GREGORIO ROBLES , AND MARCOS ROMAN-GONZ

ALEZ

n and G. Robles were with the Universidad Rey Juan Carlos, Madrid 28933, Spain
J. Moreno-Leo
M. Roma
n-Gonza
lez was with the Universidad Nacional de Educacio

n a Distancia, Madrid 28040, Spain
CORRESPONDING AUTHOR: J. MORENO-LEON
([email protected])

ABSTRACT With the introduction of computer programming in schools around the world, a myriad of
guides are being published to support educators who are teaching this subject, often for the first time. Most
of these books offer a learning path based on the experience of the experts who author them. In this paper
we propose and investigate an alternative way of determining the most suitable learning paths by analyzing
projects developed by learners hosted in public repositories. Therefore, we downloaded 250 projects of
different types from the Scratch online platform, and identified the differences and clustered them based on
a quantitative measure, the computational thinking score provided by Dr. Scratch. We then triangulated
the results by qualitatively studying in detail the source code of the prototypical projects to explain the
progression required to move from one cluster to the next one. The result is a data-driven itinerary that can
support teachers and policy makers in the creation of a curriculum for learning to program. Aiming to
generalize this approach, we discuss a potential recommender tool, populated with data from public reposi-
tories, to allow educators and learners creating their own learning paths, contributing thus to a personalized
learning connected with students’ interests.
INDEX TERMS Programming, computational thinking, learning paths, data-driven, Scratch

I INTRODUCTION others, Alice [5], AgentSheets [6] or Scratch [7]. A

In the last years several countries around the world have significant amount of didactic materials exists, too, but there
introduced computer programming in their primary and sec- is a lack of learning paths that clearly establish the steps to
ondary education curriculum, such as England, and others develop specific dimensions or aspects of these competen-
are planning to do so, as Finland [1]. Programming is seen ces [8]–[12].
as part of a movement that aims to promote Computational This paper addresses the challenge of finding a suitable
Thinking (CT) skills among young learners [2]. CT is the learning path for learners. In order to do so, we have focused
process involved in formulating a problem and expressing on the most used visual programming language in education,
its solution so that a computer can effectively carry it out. It Scratch.1 Scratch is being used both in formal educa-
is based on an iterative process based on three stages: for- tion environments [13]–[15], and in informal scenarios, such
mulation of a problem (abstraction), expression of a solu- as after-school clubs [16], summer camps [17] or promotion
tion (automation, which includes its implementation by events [18], as well as to train pre-service teachers [19],
means of programming) and execution and evaluation of a among other scenarios.
solution (analysis). The term CT was first used by Papert [3] Creations in Scratch are called projects. Projects that
in the 1980s, although it has been popularized recently by learners choose to share with other members of the commu-
Wing [4]. nity in the Scratch web repository are categorized under
Many educators are teaching this subject for the very first one or more project types, being the most common the fol-
time and they require support to help their students achieve lowing five types: games, animations, music, art and stories,
their learning goals, including tools and didactic materials.
Currently, a diversity of tools to learn computer programming 1
Scratch has more than 18 million users, according to https://projects/
and CT skills focused on young students exist, such as, among statistics/.
2168-6750 ß 2017 IEEE. Translations and content mining are permitted for academic research only.
Personal use is also permitted, but republication/redistribution requires IEEE permission.
VOLUME 8, NO. 1, JAN.-MAR. 2020 See ht_tps://www.ieee.org/publications/rights/index.html for more information. 193

n et al.: Towards Data-Driven Learning Paths to Develop Computational Thinking with Scratch
Moreno-Leo

although there are other categories such as tutorials, simula- were collected during an 18-month period. Although learners
tions or advertisements. did not receive formal instruction, the analysis of the pro-
To study whether different types of projects can be discrim- gramming concepts in the projects shows understanding of
inated and grouped according to their computational complex- user interaction, loops, conditionals, and communication and
ity, in this investigation we selected 250 projects, 50 for each synchronization. However, there are concepts that seem less
of the five main categories, and studied and clustered them easy to discover without guidance from mentors, such as var-
using i) Dr. Scratch [20], a static code analyzer for iables or Boolean logic.
Scratch programs, and ii) a qualitative, more detailed, man- A case study investigates the games developed by ten fifth
ual analysis of the prototypical projects of each cluster. graders in their science class: students were able to use differ-
Our aim is to establish a meaningful learning path which ent advanced programming concepts, although they required
guides the learner to start with simpler types of projects and help from teachers to use Boolean expressions [22].
then to move to more complex ones when he is more compe- 29 games created by 60 primary students working in
tent. We label this as a bottom-up itinerary, as it emerges groups during eight weeks are studied in [23]. The findings
from data obtained from publicly available projects, in con- show that children are able to learn basic programming con-
trast to what we could call a top-down approach, which is cepts, such as user interaction, loops and conditional state-
based on the intuition, experience and background of experts ments, although more advanced concepts, as the use of
that can be generally found in learning guides and books. random numbers, are less used in the projects.
Beneficiaries of our approach would be teachers, policy mak- In [24], the learning of 46 ninth graders (14-15 years old)
ers and educational resources designers, as well as the attending a computer science course in a formal education
experts who write learning guides and materials, as they environment is investigated. Authors state that most students
could create learning paths and strategies to develop specific learn important concepts of computer science, such as
CT and coding skills. bounded and conditional loops or message passing. Nonethe-
Based on this idea we present a proof of concept of a data- less, some problems are also reported with concepts such as
driven tool that allows learners to create their own, personal- initialization, variables and concurrency, in line with afore-
ized learning paths with virtually the number of phases of mentioned research.
their choice, as well as the types of projects to include in Even though these works are mainly focused on establish-
each of the levels. The tool would provide inspirational proj- ing whether Scratch is an interesting, viable platform to
ects from the Scratch repository with the appropriate level learn computer science concepts and CT skills, the fact that
of development of CT and from the selected types of projects several computational concepts or programming constructs
for each of the determined phases. The flexibility offered by are identified as more complex for learners than others will
this generalization of itineraries would enhance the connec- be useful to discuss if our learning path coincides with their
tion with the diverse range of learners’ interests, increasing conclusions.
their engagement with the curriculum. Adams and Webster [25] perform an investigation that, to
The structure of the paper is as follows. In Section II we some extent, is similar to our work. The authors organize
present a review of literature on the use of Scratch in edu- summer camps for middle and high school students aiming
cational environments to teach computer programming and to introduce learners to programming concepts. During
CT skills. In Section III we explain the selection process of 2003-2007 they use Alice to create storytelling projects.
the projects involved in the investigation, and outline the However, during 2008-2011 they changed to Scratch, and
methods and instruments utilized for assessment. We present learners program games and music video creations. In their
the findings in Section IV, where we measure differences investigation they compare the programming concepts that
between types of Scratch projects in terms of CT, create are used in the various types of projects, finding significant
clusters of projects based on this quantitative metric, and differences in the frequency of variables, if statements, loops,
study in detail the source code of the prototypical project of and dialog constructs. Given the resemblance with our goals,
each cluster. Based on these results, we propose a data-driven this investigation will be used as a baseline to compare our
learning path to develop CT with Scratch in Section V, results in the discussion section.
where we also present some threats to the validity of our With a different approach, the Progression of Early
work. Finally, in Section VI we summarize the main results Computational Thinking Model [26] is a framework to assess
of our investigation and present ideas for further research. CT skills of primary students by analyzing Scratch proj-
ects. The results of a pilot-test study with 150 projects pro-
II BACKGROUND grammed by students in grades one to six prove the potential
The analysis of artifacts created by young learners to evaluate of the model to detect differences in the development of CT
the viability of Scratch for learning computer science con- among students of different ages. The final goal of the paper
cepts and CT skills is an area that has been widely investi- is to contribute to establish age-appropriate curricula for stu-
gated in the last years. dents in primary education by providing evidence on which
In [21], 536 projects programmed by youths between 8 kind of activities can be challenging but not too difficult for
and 18 years old who participated in a Computer Clubhouse students of different grades. In the discussion section we

194 VOLUME 8, NO. 1, JAN.-MAR. 2020


n et al.: Towards Data-Driven Learning Paths to Develop Computational Thinking with Scratch
Moreno-Leo

TABLE 1. Level of development for each CT dimension assessed oriented approach led to high levels of intrinsic motiva-
by Dr. Scratch [38]. tion [34]. These remarks, consequently, can be used to sup-
port our proposal of learning paths through the remixing of
CT dimension Basic (1pt) Developing (2pt) Proficient (3pt)
prototypical projects in progressive stages of CT complexity.
Logical Thinking if if else logic operations
Data modifiers of variables lists
III METHODOLOGY AND SAMPLE
representation object properties
In this research we have analyzed 250 Scratch projects.
User interactivity green flag keyboard, mouse, webcam, input
ask and wait sound
The procedure for the selection of the projects has been as
follows: we visited the Scratch web repository to down-
Flow control sequence of repeat, forever repeat until
blocks load 75 projects of each of the five main categories in the
Abstraction more than one procedures use of clones
Explore section2: animations, art, games, music and stories.
and problem script and more (definition of Half of these projects were downloaded on February 28,
decomposition than one sprite own blocks) 2016, and the other half on October 19, 2016.
Parallelism two scripts on two scripts on two scripts on In order to designate a project as valid for the investigation
green flag key pressed or when I receive we established the following criteria: i) projects should not
on the same message, or be categorized under more than one category, and ii) projects
sprite clicked video or input
should not be empty—with no code whatsoever—and should
audio, or when
backdrop present a minimum level of elaboration or sophistication.
changes to We established the minimum level of sophistication for
Synchro- wait message wait until, when projects to be included in the investigation in 3 points in the
nization broadcast, backdrop score provided by Dr. Scratch,3 a free/libre static code
stop all, changes to, analyzer for Scratch projects, which is based on Hair-
stop program broadcast and ball [27] and inspired by Scrape [35]. As part of the vali-
wait
dation process of Dr. Scratch, its creators compared the
scores provided by the tool with classic software engineering
make use of these results to study similarities and differences complexity metrics [36] as well as with evaluations from
with the proposed learning path. experts in computer science education [37], finding positive,
On other hand, it is also possible to find research that is strong correlations between measurements in both cases.
supported by automatic tools that assess the computer sci- As summarized in Table 1, the mastery score that Dr.
ence concepts developed by learners. Using Hairball [27], Scratch assigns to projects is based on the degree of devel-
a static code analyzer for Scratch projects, the computer opment of seven dimensions of the CT competence: abstraction
science learning of 20 middle-schoolers in a two-week sum- and problem decomposition, logical thinking, synchronization,
mer camp is evaluated. Findings prove that students show parallelism, algorithmic notions of flow control, user interactiv-
competence with event-driven programming, initialization of ity and data representation. These dimensions are statically
state, message passing and synchronization [28]. evaluated by inspecting the source code of the analyzed proj-
Finally, since the learning path we propose is based on ect [38] giving each a score from 0 to 3, which results in a total
prototypical projects of clusters, we reviewed classic litera- evaluation (mastery score) that ranges from 0 to 21 when all
ture on prototype theory [29], [30]. This theory argues that seven dimensions are aggregated.
the natural way of categorizing and learning is through proto- Figure 1, which shows the source code of a Scratch
types, which are the most representative elements of a given project, can be used to illustrate the assessment of the tool.
category. The theory also states that categories contain both The project gets 1 point, a basic level of development, for
central and peripheral members and present vague bound- Data representation, as some properties of the objects are
aries, with gradual transitions between them. The analogy modified, such as its position and orientation; Abstraction
with our data-driven, bottom-up approach is clear and worth and problem decomposition, as the solution is implemented
noting. Furthermore, given that learners will use the proto- in several programs; Parallelism, as there are several pro-
typical projects to get inspiration and learn from them, we grams that start at the same event; as well as for Logic,
reviewed recent research investigating the impact of remix- because there is one if statement. In addition, the project
ing on computer programming learning. Two of these publi- receives 2 points for Flow control, because of a forever loop;
cations make use of a dataset containing public activity of and for User interactivity, as the character is controlled by
the first five years in the Scratch online community [31], means of the keyboard. Finally, the project is awarded with 3
and prove that learners who remix more often develop more points for Synchronization, as there is a wait until event
sophisticated projects [32], and that “exposure to computa- instruction. The total mastery score of the project is therefore
tional thinking concepts through remixing is associated with 11 points.
increased likelihood of using those concepts” [33]. In a dif-
ferent context, an intervention that collected the perceptions 2
See https://projects/explore/
of students during a 39-week activity shows that a remix- 3
Dr. Scratch is available at http://drscratch.org/

VOLUME 8, NO. 1, JAN.-MAR. 2020 195


n et al.: Towards Data-Driven Learning Paths to Develop Computational Thinking with Scratch
Moreno-Leo

FIGURE 2. Histogram of number of projects by mastery score, as

FIGURE 1. Source code of Maze-1, a Scratch project available at
calculated by Dr. Scratch. It shows a positively skewed, non-
https://scratch.mit.edu/projects/2629044/.
normal, distribution.
We have chosen 3 points as the minimum mastery score
required for projects to be included in the investigation since with projects between 3 (the threshold value) and 21 points.
working with empty or barely elaborated projects would be The mean score is 10.16, the standard deviation is 4.355, the
useless for studying differences in the levels of development mode is 7, the median is 9.5 and the IQR is 6.25. We use a
of CT. Once excluded the non-valid projects, we ordered the Kolmogorov-Smirnov test to check the normality of the dis-
remaining projects in descending order by mastery score and tribution; since Zk-s = .138 (p < .001) the normality null
selected the first 50 ones of each category.4 hypothesis is rejected and, therefore, subsequent statistical
In order to assess differences in terms of sophistication tests will be non-parametric.
between each of the five categories of projects under investi-
gation we used both quantitative and qualitative methods. In IV FINDINGS
terms of quantitative measurements we made use of the A QUANTITATIVE FINDINGS
aforementioned Dr. Scratch tool to study differences The goal of the quantitative analysis is to study the differen-
regarding the development of different aspects of CT skills. ces in the CT mastery score among categories of Scratch
Based on the results of this metric, we create clusters of proj- projects, and to identify clusters of projects using the CT
ects, grouping them in such a way that we preserve certain dimensions assessed by Dr. Scratch as the variables for
homogeneity in the characteristics of the projects within the analysis. We aim to determine a learning path with proj-
groups, but maintain the heterogeneity between projects of ects of increasing complexity.
different clusters. The results of the clustering process are the
basis for the learning path that we propose. 1) STATISTICAL ANALYSIS
However, aiming to study and detail the differences Figure 3 presents several violin plots comparing the median,
between clusters in terms of progression that would be interquartile range and distribution pattern of the mastery
required to move from one phase of the itinerary to the next score provided by Dr. Scratch for each of the categories
one—i.e., from one cluster to another—we manually of Scratch projects under investigation. There is a dispar-
inspected the source code of the prototypical projects, which ity in the mastery score between categories, with medians
are those projects that are closer to the center of the cluster. ranging from 7-8 points for art, music and animations, to 10
This qualitative analysis, in consequence, provides further points for stories and 15 points for games. We can also
insight to the findings of the quantitative approach. observe differences in terms of distribution, with higher lev-
els of concentration around the median for art and music
A CHARACTERISTICS OF THE STUDY SAMPLE
projects, and lesser ones for stories and games.
Figure 2 shows the frequency of mastery scores for the 250 The results of the non-parametric Kruskal-Wallis test show
projects analyzed, which presents a positively skewed distri- that there are statistically significant differences in the CT
bution. We can see a considerable diversity in the scores, mastery score depending on the type of project, as x2 (df=4) =
74.701 (p < .001). A Bonferroni post-hoc test confirms, with
4
a 99 percent confidence level, that significant differences (p-
The list (and URLs) of the selected and rejected projects is available in the
replication package of the paper: https://github.com/kgblll/ReplicationPack- value including Bonferroni correction < .01) between pairs
age-2017-TETC of means are present in the following couples of categories:

196 VOLUME 8, NO. 1, JAN.-MAR. 2020


n et al.: Towards Data-Driven Learning Paths to Develop Computational Thinking with Scratch
Moreno-Leo

FIGURE 4. Error bars for each category of Scratch projects and

for each CT dimension assessed by Dr. Scratch.

data representation the most notables ones between games

and the rest of categories.
FIGURE 3. Violin plots of the Dr. Scratch Mastery score for
each category of Scratch projects. In each violin plot: The A progression can be observed between categories, starting
white circle indicates the median of the data; the central box from art, music and animations, then moving to stories and
represents the interquartile range (IQR), and its whiskers finally to games. There are statistically significant differen-
extend to the minimum and maximum values (without consider- ces in terms of levels of development of CT among catego-
ing outliers); overlaid the box-and-whisker plot, the probability
ries of Scratch projects.
density of the data at different values is depicted.

games-art, games-music, games-animations, games-stories 2) CLUSTERING ANALYSIS

and stories-art. Since projects from different types present diverse levels of CT
Table 2 presents several non-parametric Kruskal-Wallis development, we have performed a cluster analysis, in which
tests of the seven CT dimensions among Scratch catego- the variables are the seven CT dimensions assessed by Dr.
ries. We can affirm with a 95 percent confidence level (a Scratch. Our aim with this analysis is to identify groups of
with Bonferroni correction = a / number of hypotheses = projects that can be used as the basis for establishing a learning
.05 / 7 = .007) that there are statistically significant differen- itinerary based on clusters with increasing complexity.
ces between categories of Scratch projects for each and In a first stage we performed a hierarchical cluster analysis
every of the CT dimensions, as p(x2) < .007 in all the cases. seeking to build a hierarchy of groups of projects. We fol-
Figure 4 shows the error bars, which represent the 95 per- lowed an aggregating approach based on similarity; thus,
cent confidence interval of the mean scores, for each category each of the 250 projects started in its own cluster, and pairs
of Scratch project and for each of the CT dimensions of clusters were merged when moving up the hierarchy.
assessed by Dr. Scratch. There are significant differences Figure 5, which presents the dendrogram of the analysis, can
in some of the dimensions, being logic, synchronization and be used to illustrate the arrangement of the groups, where the

TABLE 2. Kruskal-Wallis tests among categories of scratch projects, for each of the CT dimensions analyzed by Dr. Scratch.

Mean Rank x2 pðx2 Þ

Art Music Animations Stories Games
Abstraction 120.51 112.94 113.58 126.74 153.73 15.801 .003
Flow Control 112.14 108.38 122.86 124.78 159.34 21.135 .000
Parallelism 85.08 104.19 129.91 147.38 160.94 42.829 .000
Synchronization 71.45 123.50 115.49 140.36 176.70 60.584 .000
User Interactivity 134.42 117.10 77.38 117.06 181.54 71.810 .000
Logic 105.09 97.73 113.33 118.11 193.24 77.874 .000
Data Representation 107.11 97.26 112.05 116.24 194.84 82.158 .000

VOLUME 8, NO. 1, JAN.-MAR. 2020 197


n et al.: Towards Data-Driven Learning Paths to Develop Computational Thinking with Scratch
Moreno-Leo

FIGURE 5. Dendrogram of the hierarchical cluster analysis for the 250 Scratch projects under investigation.

bottom row of nodes represents the individual Scratch data representation; and 0 points in logic. Finally, the center
projects, while the remaining nodes represent the clusters to of Cluster 3 would be represented by a project with a score of
which the projects belong. The dendrogram suggests that the 2 points in all the CT dimensions but logic, synchronization
optimal approach could be either a two or a three cluster and flow control, which would be assessed with 3 points.
solution. After exploring both options, we chose to deepen Figure 7, which depicts the distance of each project to the
the solution with three clusters due to its higher explanatory center of its cluster, can be used as a visual representation of
and interpretative power that will be described in the follow- the tightness of the groups. As shown, the most homoge-
ing lines (conversely, the two clusters solution was rejected neous cluster is Cluster 3 (SD = .324), followed by Cluster 2
mainly because neither the animations nor the stories were (SD = .474), while Cluster 1 is the most heterogeneous group
clearly located in any of the clusters, so its explanatory and (SD = .540).
interpretative power was considered lower). We highlight the It must be noted that a Kruskall-Wallis test shows that this
three clusters in Figure 5: on the left (in color blue), there is a solution is effective to establish significant differences
big and heterogeneous cluster; in the middle (green) there is between clusters for all CT dimensions and for the overall
a second group, which is closer to cluster 1 than to cluster 3;
finally on the right (yellow), we see a more homogeneous
cluster, clearly differentiated from the other two. Conse- TABLE 3. Cluster centers for each of the three clusters of the
solution.
quently, a three cluster solution is explored in the following
stage of the analysis: a K-means clustering (K ¼ 3). Cluster
Figure 6 shows the distribution of projects by category
1 2 3
included in each of the three clusters of the optimal solution,
Logic 0 0 3
being Cluster 1 mainly formed by animations, art and music Abstraction 1 1 2
projects; Cluster 2 by stories; and Cluster 3 by games. This User Interactivity 1 1 2
grouping is in line with the differences in the mastery score Data Representation 1 1 2
already shown in Figure 3. The number of cases in all clus- Parallelism 1 2 2
Synchronization 1 2 3
ters is balanced, being 119 the number of projects included Flow Control 2 2 3
in Cluster 1, 72 in Cluster 2 and 59 in Cluster 3.
Mastery 7 9 17
Table 3 presents the center of each of the clusters for the
three groups of the solution. For Cluster 1, the center would
be a project with 0 points in logic, 2 points in flow control and
1 point in the rest of the CT dimensions. The center of Cluster
2 would have 2 points in parallelism, synchronization and
flow control; 1 point in abstraction, user interactivity and

FIGURE 7. Distance of each project to the center of its cluster.

FIGURE 6. Distribution of projects by category included in each The most homogeneous cluster is Cluster 3, followed by Cluster
of the three clusters of the solution. 2, while Cluster 1 is the most heterogeneous one.

198 VOLUME 8, NO. 1, JAN.-MAR. 2020


n et al.: Towards Data-Driven Learning Paths to Develop Computational Thinking with Scratch
Moreno-Leo

TABLE 4. Scores of the Prototypical Projects of the Clusters

Cluster
1 2 3
Logic 0 0 3
Abstraction 1 1 3
User Interactivity 1 1 2
Data Representation 1 1 2
Parallelism 1 3 3 FIGURE 9. Source code of project: Racoon going nomnomnom.
Synchronization 1 2 3 Prototypical project for Cluster 1.
Flow Control 2 2 2
Mastery 7 10 18

FIGURE 8. Screenshots of project: Racoon going nomnomnom.

Prototypical project for Cluster 1.

mastery score (x2 (df=2) = 203.872 (p < .001)). Furthermore,

this clustering produces a significant association between clus-
ter and type of Scratch project (Pearson x2 (df=8) = 60.662
(p < .001); Contingency Coefficient ¼.442 (p < .001)).
Three clusters of projects with increasing CT complexity FIGURE 10. Screenshots of project: When my Pencil Breaks
can be established. Cluster 1 is mainly formed by anima- remix. Prototypical project for Cluster 2.
tions, art and music projects; Cluster 2 by stories; and Clus-
its appearance, is accessed and modified by means of cos-
ter 3 by games.
tumes; algorithmic notions of flow control, as the program
includes forever loops; some notions of abstraction and
problem decomposition, as the code is divided into two pro-
B. QUALITATIVE, MANUAL FINDINGS
grams in two different entities, the sprite and the background;
In this section we aim to explain the results obtained with the
basic synchronization through wait instructions, and paral-
qualitative analysis by inspecting manually in detail the
lelism, as two programs start to run at the same time. The
source code of the prototypical projects of each cluster,
project, on the contrary, does not provide any evidence of
defined as the closest one to its center. Table 4 shows the
logic elements.
scores provided by Dr. Scratch for each of the prototypi-
cal projects analyzed in this section, and their distance with
2) CLUSTER 2: STORIES
the actual centers of the clusters can be studied by comparing
Figure 10 shows several screenshots of the prototypical project
these scores with those in Table 3.
for this cluster, which is When my Pencil Breaks remix.6 This
project makes use of several characters that appear in turns on
1) CLUSTER 1: ART, MUSIC AND ANIMATIONS the screen to tell a story.
The prototypical project for this cluster is racoon going nom- Studying the source code, we can see from Figure 11 that its
nomnom.5 Figure 8 shows some screenshots of the project in structure and blocks are very similar to the ones in the anima-
action, which consists of an animation of a single sprite that tion of Cluster 1. The same flow control loops and the same
changes its appearance while a song is played. instructions to modify properties of the objects are used, stat-
Figure 9 shows the source code of the two programs of the ing basic levels of data representation. The main difference,
project. Despite its simplicity, the project presents some basic in fact, is that there is more than one sprite in the project, and
form of user interactivity through the clicking of the green that sprites are synchronized by means of message passing,
flag; data representation, as one the properties of the object, which shows higher levels of development of parallelism and

5 6
https://projects/projects/99612654/ https://scratch.mit.edu/projects/124827092/

VOLUME 8, NO. 1, JAN.-MAR. 2020 199


n et al.: Towards Data-Driven Learning Paths to Develop Computational Thinking with Scratch
Moreno-Leo

FIGURE 11. Source code of project: When my Pencil Breaks

remix. Prototypical project for Cluster 2.

FIGURE 13. Source code of project: Infiltrate A Platformer. Proto-

typical project for Cluster 3.

structures, including nested loops. Furthermore, this game

incorporates instructions to store different types of informa-
tion, such as the score or the level, and consequently utilizes
variables, requiring higher levels of data representation.
Manual inspection of the source code of the prototypical
projects of each cluster allows to justify the differences in
terms of computational thinking (CT) between categories,
and contributes to explain the progression in CT skills
required to move between these groups.
FIGURE 12. Screenshots and sprites of project: Infiltrate A Plat-
former. Prototypical project for Cluster 3.
V DISCUSSION
synchronization. Nonetheless, given the linear structure of The results show that different types of projects can be used
stories and its limited user interactivity, this type of projects to develop different CT dimensions. In consequence, educa-
does not generally use logic instructions either. tors could make use of our results to establish appropriate
learning paths so that their students can progressively
3) CLUSTER 3: GAMES achieve different learning goals while remixing and develop-
The prototypical project in this cluster is Infiltrate A Plat- ing different kinds of prototypical projects.
former,7 a game that involves guiding a character to jump Considering the prototypical projects of the clusters pre-
and fall on platforms to avoid obstacles and advance through sented in Section IV, a possible, simple itinerary to intro-
levels. Some snapshots of the project, as well as the sprites duce different types of Scratch creations in the
window, are shown in Figure 12, where ten different objects classroom could start with animations, art and music proj-
can be identified. ects, then continue with stories, and finalize with games. In
Games usually involve higher levels of user interactivity the first phase, learners would start to develop basic
than animations, art and music creations (Cluster 1), or stories abstraction and problem decomposition, user interactivity,
(Cluster 2), as users have to control and guide their avatar in data representation, parallelism and synchronization skills,
the game. This is also the case in this project, as can be seen while achieving intermediate algorithmic notions of flow
from Figure 13. We can observe instructions to modify the ori- control abilities. In the next phase, usually by incorporating
entation and position of the avatar when certain keys are a bigger number of sprites that have to be organized on
pressed, which also imply the use of basic logic instructions. screen in turns, learners would develop their parallelism
Another typical element of games is some sort of collision and synchronization skills further. Finally, in the last phase
detection; in order to detect whether the avatar is touching ene- learners would need to work with logic instructions and
mies or obstacles, and to react in consequence, the program- operations, get to know more advanced flow control struc-
mer makes use of advanced logic operations and flow control tures, incorporate more user interactivity mechanisms, will
also require to store and update data, and will have to mas-
7
https://scratch.mit.edu/projects/99330902/ ter their synchronization skills.

200 VOLUME 8, NO. 1, JAN.-MAR. 2020


n et al.: Towards Data-Driven Learning Paths to Develop Computational Thinking with Scratch
Moreno-Leo

be more challenging and difficult to learn. In [21], variables

and Boolean logic are described as less easy to discover with-
out the help from mentors. Similar conclusions are obtained
in [22], where students required help from teachers to use
Boolean expressions. In our learning path both concepts,
which correspond to a developing level in data representation
and a master level in logical thinking are postponed to phase 3.
It has been reported that projects created by students rarely
make use of advanced concepts, such as random numbers [23].
Random numbers are a typical elements of games, as they
introduce chance; consequently, this aspect would also be
introduced in phase 3 of our learning path. Lastly, some prob-
lems are reported in [24] with learning concepts such as varia-
bles and concurrency. As mentioned, variables are introduced
in the third phase of our itinerary. Concurrency is related to
FIGURE 14. Recommendation of prototypical projects for each the concepts of parallelism and synchronization; in our learn-
phase of the learning itinerary. ing path students would start to develop basic skills of both
abilities in phase 1, continue to develop them in the second
Aiming to illustrate how educators and students could phase 2 and master synchronization skills in the third one.
make use of these results we have built a simple website that The study of the frequency of different design patterns
presents 6 prototypical projects for each of the phases of the in [26] indicates that some patterns are best suited for certain
learning path.8 Figure 14 shows a screenshot of the website, grades. Specifically, while patterns as Conversate, Animate
where learners could study and remix the prototypical proj- Looks and Animate Motion are uniformly used in all the
ects to develop the CT dimensions involved in each phase of classrooms, from first to sixth grade, some patterns, such as
the learning path. Collision and Scoring, that require advanced concepts such
as conditionals and variables, are only represented in fifth
A. COMPARISON WITH PREVIOUS RESEARCH and sixth grades. The parallelism of these results with our
As a baseline to compare our results in terms of skills that are learning path is evident: Animate Looks, Animate Motion and
developed with different types of projects we will use the Conversate patterns would be equivalent to our first phase,
conclusions in [25], where the learning from programming in which animations and art projects are introduced, and to
games, music video and storytelling projects is studied. It phase 2, dedicated to the creation of stories, while Collision
must be noted, nonetheless, that storytelling projects were and Scoring, which are typical elements of games, are clearly
programmed with another block-based language, Alice, related with our third phase.
not with Scratch.
In [25], significant differences are detected in the use of var- B. COMPARISON WITH ITINERARIES BY EXPERTS
iables between games, and both music video and storytelling In this section, we compare the learning path we propose
projects. In our investigation this is also the case, as there are with actual guides, authored by experts from education and
significant differences in the data representation dimension. software industry, that educators and learners are using now-
The same occurs when it comes to the use of if statements. adays to learn to program with Scratch.
Both in [25] and in our investigation, taking into account the Perhaps the guide that is most used worldwide is Creative
differences in the logic dimension, significant differences are computing [8], which has been translated into seven lan-
detected between games and music or storytelling. guages. This guide presents “a collection of ideas, strategies,
However, regarding the use of loops, our conclusions are and activities for an introductory creative computing experi-
not in line with the ones in [25]. Thus, while Adams and ence using the Scratch programming language”. As stated
Webster do not find differences between games and music in the introduction, “the activities are designed to support
video projects, our results show significant differences in the familiarity and increasing fluency with computational crea-
flow control dimension between these two categories. Dr. tivity and computational thinking. In particular, the activities
Scratch does, however, not consider the number of loops encourage exploration of key computational thinking con-
to assign its score, but the type of loops that appear in the cepts (sequence, loops, parallelism, events, conditionals,
code. In consequence we understand that the different results operators, data) and key computational thinking practices
can be attributed to the distinct analysis performed. (experimenting and iterating, testing and debugging, reusing
It is known from literature that studies whether Scratch and remixing, abstracting and modularization)”.
is a viable platform to learn coding and CT skills that several The structure of the guide matches the learning path we pro-
programming constructs or computational concepts seem to pose. Thus, after an introduction to the environment, the sec-
ond unit is focused on animation, art, and music creations to
8
The website is available at http://cpu03.libresoft.info/guided/ develop the computational concepts of loops, events, and

VOLUME 8, NO. 1, JAN.-MAR. 2020 201


n et al.: Towards Data-Driven Learning Paths to Develop Computational Thinking with Scratch
Moreno-Leo

TABLE 5. Order in which the contents related to each of the reactions based on events from the very beginning. Other
phases of our itinerary are included in the reviewed guides and classic structures of games, such as variables, are also pre-
books written by experts. sented very early, as they are required in most types of non-
Book Phase 1 Phase 2 Phase 3 simple games.
As we can see summarized in Table 5, some of the guides
Creative computing [8] 1st 2nd 3rd
Sams Teach Yourself in 24 Hours [9] 1st 2nd 3rd and books, although pursuing similar goals, use different
Learn to Program with Scratch [10] 3rd 1st 2nd approaches to develop skills and abilities with Scratch.
Scratch Programming in Easy S. [11] 1st 3rd 2nd These differences, which might be due to the different back-
Super Scratch Programming A. [12] 1st 3rd 2nd grounds and expertise of the authors, show that there is lack
of standarization in terms of learning paths that establish the
steps to develop specific dimensions of CT skills. Therefore,
parallelism. Then, the next unit encourages the creation of sto- we argue that investigations as the one presented in this
rytelling projects to learn to code conversations and shifting paper, which utilizes data extracted from the creations that
scenes, for which synchronization mechanisms are intro- users around the world are sharing in the Scratch project
duced. Finally, the next unit presents the main game mechan- repository, could be useful for curriculum and educational
ics, as score and levels, to develop advanced computational resources designers, since they could make use of our con-
concepts, such as variables, operators, and conditionals. clusions to choose different types of activities to develop and
Scratch 2.0 Sams Teach Yourself in 24 Hours [9] includes master concrete skills.
24 sessions of one hour that explain how to create anima-
tions, simulations, stories and games. The structure of this C. TOWARDS GENERALIZATION
book is also similar to the learning path we propose, as it Thus far we have placed the focus on the prototypical proj-
starts working with costumes and the stage, and then introdu- ects for each cluster or phase. But it is also possible to create
ces simple motion, looks and sound, and pen blocks, which art or music projects, animations or stories that show high
are the base for animations, art and music projects. The next levels of CT skills, and to create games with low CT com-
sessions explain how to work with events and control blocks, plexity. Indeed, the cluster analysis confirms that this is the
which are required to create stories. And it ultimately case in the Scratch repository, as we found several cases
presents blocks required to implement games, such as sens- for all of the categories that are assigned to the three clusters
ing, operators and data structures. of the solution (see Figure 6).
Learn to Program with Scratch: A Visual Introduction to We argue that it is feasible to progress from basic, simple
Programming with Games, Art, Science, and Math [10] is a initial projects to advanced, complex ones by developing all
collection of hands-on projects that are designed to learn con- kind of creations. Thus, if a sufficient number of projects
cepts to solve real-world programming problems. The struc- from the Scratch repository are analyzed, an application
ture of this book is different from our proposal, as it delays could be built where educators and learners could get recom-
to late chapters some basic concepts such as repetition mended examples by selecting the type of project that they
instructions, while advanced concepts, such as the possibility want to create and the level of development of CT that they
of using procedures by creating own blocks, are introduced wish to reach. The application would then propose projects
in early stages. In fact, this approach is highlighted in the from the repository with that particular level of CT that could
introduction of the book: “We jump into procedures here to be used as inspiration or as the solution for an exercise.
enforce good programming style from the beginning.” Based on the 250 projects under investigation we have
Scratch Programming in Easy Steps [11] is an introduction built a proof of concept to illustrate the basic working of
to Scratch to learn to create animations and games. After a such a tool.9 Figure 15 shows two screenshots of the tool in
basic introduction, the second chapter of this book is focused action; the one in the top shows the selection process, where
on the Scratch features to draw figures. In this chapter the user has selected the Games category and a CT score of
coordinates, position, direction and pen blocks are intro- 16 points; while the one in the bottom presents the list of
duced, as well as repeat loops. However, a very advanced Games stored in the database with that CT score.
construct (nested loops) is also presented in this introductory The data-driven tool could go one step further by allowing
unit. It is surprising that this advanced structure is introduced users the creation of a learning path with the number of
earlier than other, much simpler features like, for instance, phases of their choice, as well as the types of projects to
the use of the green flag. This is also the case with variables, include in each of the levels. The flexibility offered by this
another advanced concept that is presented in chapter 3, generalization of the itineraries enhances the chances to con-
while the basic possibilities of edition and animation of sprite nect to the diverse range of learners’ interests. For instance,
images are introduced in chapter 4. there is evidence that girls tend to prefer the development
Finally, Super Scratch Programming Adventure! [12] is story telling projects more than boys, who on the contrary
different from the previous analyzed books, as it is specifi- are more inclined to video games [25], [39]. So, in the same
cally focused on game development. In consequence, it
presents contents related to coordinates, movements and 9
The demo tool is available at http://cpu03.libresoft.info/wild/

202 VOLUME 8, NO. 1, JAN.-MAR. 2020


n et al.: Towards Data-Driven Learning Paths to Develop Computational Thinking with Scratch
Moreno-Leo

allows learners to create meaningful, challenging projects

that they would be proud to share with the community.
Our proposal has not been tested in real life scenarios with
actual learners. Nonetheless, both the prototype theory,
which argues that learning through prototypes is natural for
humans [29], [30], and recent research on the impact of
remixing on programming learning, which states that learners
who remix more often use larger repertoires of programming
commands [32], [33] and present higher levels of motivation
and self-efficacy in educational environments [34], seem to
support our proposal of learning through the remixing of pro-
totypical projects in progressive stages of CT complexity. As
will be stated in Section VI, we plan to address this issue in
the near future by carrying out an investigation involving
several schools.

VI CONCLUSIONS AND FUTURE RESEARCH

In this paper we analyzed 250 Scratch projects, 50 of each
of the five main categories of Scratch creations (games,
art, music, stories and animations), to study differences in
FIGURE 15. Recommendations of projects after selecting a cate- the CT dimensions that are developed by programming each
gory and a computational thinking score. Above: The selection type of project. Since we detected significant differences
process. Below: Games with 16 points. between categories for all of the CT dimensions of the analy-
sis, we clustered the projects obtaining a structure of three
way that using wrong examples when learning to program clusters with increasing CT complexity: Cluster 1 is mainly
can be harmful [40], we think that a wrong learning itinerary formed by animations, art and music projects; Cluster 2 by
increases the risk of the learners’ indifference. Enabling the stories; and Cluster 3 by games. Hence, we presented a learn-
possibility of creating a complete story-based learning itiner- ing path with three phases, wherein learners would develop
ary that includes the advanced development of all aspects of their CT skills by studying and remixing the prototypical
CT skills without having to jump to games—with the risk of projects of each cluster.
many girls losing interest—could contribute to not leaving However, we also found that for every category there are
students behind because of a lack of engagement. projects that show basic, intermediate and advanced CT
skills, proving that it would be possible to establish a itiner-
D. THREATS TO VALIDITY ary starting from simple projects and getting to complex ones
A threat to validity of our work is the limited number of proj- creating projects of all kinds. Thus, we presented a proof of
ects that we have analyzed, having been selected 50 projects concept that would allow users to set a learning path with the
of each of the 5 categories involved in the investigation, for a number of phases of their choice and the types of project to
total of 250 projects. The amount of projects under study is include in each level; the tool would provide a list of projects
low because their selection by category required human from the repository that could be used as inspiration.
inspection, being it labor intensive to scale to larger numbers. These results, in consequence, could be of interest for pri-
Future research, as detailed in Section VI, will extend the mary and secondary educators teaching computer program-
scope of this study by including more categories and a larger ming and CT skills, as well as for curriculum and educational
number of projects. resources designers, who may use them to create learning
On other hand it should be noted that the conclusions we paths of increasing difficulty, to prepare activities and chal-
get to might have a different, simpler explanation. For lenges focused on developing specific CT dimensions, learn-
instance, it could be the case that scratchers who have shared ing particular coding concepts or mastering different types of
art and music projects in the repository focused more on the projects before moving from one learning level or phase to a
art and the music rather than in CT skills while developing more advanced one, and do so by providing personalized
their projects. Or perhaps, people with limited CT skills examples that connect with learners’ interests.
chose animations or stories because they thought it would be In the near future we plan to extend the scope of this inves-
easier for them to create those types of projects rather than tigation by analyzing a much bigger number of projects, and
games. But, even if that was the case, this would even sup- by including in the study other categories of Scratch proj-
port our thesis, in the sense that those authors were proud ects, such as simulations or tutorials, among others. Such an
enough of their creations to share them publicly in the reposi- investigation will allow us to state stronger conclusions on
tory, and, in consequence, it is possible to establish a learning the different CT dimensions that each type of projects tend to
path with progressive difficulty in terms of CT skills that develop with more intensity, and will also allow to populate

VOLUME 8, NO. 1, JAN.-MAR. 2020 203


n et al.: Towards Data-Driven Learning Paths to Develop Computational Thinking with Scratch
Moreno-Leo

a database with a sufficient number of projects enabling the [15] B. Wohl, B. Porter, and S. Clinch, “Teaching computer science to 5–7 year-
olds: An initial study with Scratch, cubelets and unplugged computing,” in
creation of a meaningful service where educators and learn- Proc. Workshop Primary Secondary Comput. Edu., 2015, pp. 55–60.
ers can create their own learning paths. Since Dr. Scratch [16] N. Smith, C. Sutcliffe, and L. Sandvik, “Code club: Bringing programming
is able to detect certain bad habits of programming [41], [42], to uk primary schools through Scratch,” in Proc. 45th ACM Tech. Symp.
Comput. Sci. Edu., 2014, pp. 517–522.
also called code smells [43], future research could assess dif- [17] J. C. Adams, “Scratching middle schoolers’ creative itch,” in Proc. 41st
ferences in the presence of these smells by types of ACM Tech. Symp. Comput. Sci. Edu., 2010, pp. 356–360.
Scratch projects. [18] J. Moreno-Le
on and G. Robles, “The Europe code week (CodeEU) initia-
tive: Shaping the skills of future engineers,” in Proc. IEEE Global Eng.
Additionally, we plan to carry out an investigation in sev- Edu. Conf., Mar. 2015, pp. 561–566.
eral schools comparing the learning outcomes of different [19] J. Van
õcek, “Programming in Scratch Using Inquiry-Based Approach,” in
groups of students following distinct approaches. One group Proc. 8th Int. Conf. Informat. Schools Situation, Evol. Perspectives, 2015,
pp. 82–93.
will follow an structured, traditional class following the [20] J. Moreno-Le
on and G. Robles, “Dr. Scratch: A web tool to automatically
activities and itinerary included in the Creative computing [8] evaluate Scratch projects,” in Proc. Workshop Primary Secondary Com-
guide. Another group will follow our three-phase learning put. Edu., 2015, pp. 132–133.
[21] J. H. Maloney, K. Peppler, Y. Kafai, M. Resnick, and N. Rusk, “Program-
path working with the prototypical projects of each cluster. ming by choice: Urban youth learning programming with Scratch,” ACM
Participants in last group will create their own personal paths SIGCSE Bull., vol. 40, no. 1, pp. 367–371, Mar. 2008.
defining the number of phases and the category of projects to [22] A. Baytak and S. M. Land, “An investigation of the artifacts and process of
constructing computers games about environmental science in a fifth grade
use and remix. Such an intervention will enable the study of classroom,” Educational Technol. Res. Develop., vol. 59, no. 6, pp. 765–
potential differences in terms of both motivation and aca- 782, 2011.
demic results between groups. [23] A. Wilson, T. Hainey, and T. Connolly, “Evaluation of computer games
developed by primary school children to gauge understanding of program-
ming concepts,” in Proc. Eur. Conf. Games Based Learn., 2012, Art. no. 549.
ACKNOWLEDGMENTS [24] O. Meerbaum-Salant, M. Armoni, and M. M. Ben-Ari, “Learning com-
puter science concepts with Scratch,” Comput. Sci. Edu., vol. 23, no. 3,
This work has been funded in part by the Region of Madrid pp. 239–264, 2013.
under project “eMadrid - Investigaci
on y Desarrollo de [25] J. C. Adams and A. R. Webster, “What do students learn about program-
tecnolog
as para el e-learning en la Comunidad de Madrid” ming from game, music video, and storytelling projects?” in Proc. 43rd
ACM Tech. Symp. Comput. Sci. Edu., 2012, pp. 643–648.
(S2013/ICE-2715). [26] L. Seiter and B. Foreman, “Modeling the learning progressions of compu-
tational thinking of primary grade students,” in Proc. 9th Annu. Int. ACM
Conf. Int. Comput. Edu. Res., 2013, pp. 59–66.
REFERENCES
[27] B. Boe, C. Hill, M. Len, G. Dreschler, P. Conrad, and D. Franklin, “Hair-
[1] European Schoolnet, “Computing our future. Computer programming and ball: Lint-inspired static analysis of Scratch projects,” in Proc. 44th ACM
coding-Priorities, school curricula and initiatives across Europe,” Euro- Tech. Symp. Comput. Sci. Edu., 2013, pp. 215–220.
pean Schoolnet, 2015. [Online]. Available: http://www.eun.org/ [28] D. Franklin, et al., “Assessment of computer science learning in a Scratch-
publications/detail?publicationID=661 based outreach program,” in Proc. 44th ACM Tech. Symp. Comput. Sci.
[2] V. Barr and C. Stephenson, “Bringing computational thinking to K-12: Edu., 2013, pp. 371–376.
What is involved and what is the role of the computer science education [29] E. Rosch and C. B. Mervis, “Family resemblances: Studies in the internal
community?” ACM Inroads, vol. 2, no. 1, pp. 48–54, 2011. structure of categories,” Cogn. Psychology, vol. 7, no. 4, pp. 573–605,
[3] S. Papert, Mindstorms: Children, Computers, and Powerful Ideas. New 1975.
York, NY, USA: Basic Books, 1980. [30] E. Rosch, “Principles of categorization,” in Cognition and Categorization.
[4] J. M. Wing, “Computational thinking,” Commun. ACM, vol. 49, no. 3, Hilldale, NJ, USA: Lawrence Erlbaum, 1978, pp. 22–48.
pp. 33–35, 2006. [31] B. M. Hill and A. Monroy-Hern
andez, “A longitudinal dataset of five years
[5] S. Cooper, W. Dann, and R. Pausch, “Alice: A 3-D tool for introductory of public activity in the scratch online community,” Sci. Data, vol. 4, 2017,
programming concepts,” J. Comput. Sci. Colleges, vol. 15, no. 5, pp. 107– Art. no. 170002.
116, 2000. [32] J. Moreno-Le
on, G. Robles, and M. Rom
an-Gonz
alez, “Examining the
[6] A. Repenning, “Agentsheets: A tool for building domain-oriented visual relationship between socialization and improved software development
programming environments,” in Proc. INTERACT CHI Conf. Human Fac- skills in the scratch code learning environment,” J. Universal Comput.
tors Comput. Syst., 1993, pp. 142–143. Sci., vol. 22, no. 12, pp. 1533–1557, 2016.
[7] M. Resnick, et al., “Scratch: Programming for all,” Commun. ACM, vol. [33] S. Dasgupta, W. Hale, A. Monroy-Hern
andez, and B. M. Hill, “Remixing
52, no. 11, pp. 60–67, Nov. 2009. as a pathway to computational thinking,” in Proc. 19th ACM Conf. Com-
[8] K. Brennan, C. Balch, and M. Chung, Creative Computing. Cambridge, put.-Supported Cooperative Work Social Comput., 2016, pp. 1438–1449.
MA, USA: Harvard Graduate School of Education, 2014. [34] C.-C. Liu, W.-C. Chen, H.-M. Lin, and Y.-Y. Huang, “A remix-oriented
[9] T. Warner, Scratch 2.0 Sams Teach Yourself in 24 Hours. Indianapolis, IN, approach to promoting student engagement in a long-term participa-
USA: Sams Publishing, 2014. tory learning program,” Comput. Edu., vol. 110, pp. 1–15, 2017.
[10] M. Marji, Learn to Program with Scratch: A Visual Introduction to Pro- [Online]. Available: http://www.sciencedirect.com/science/article/pii/
gramming with Games, Art, Science, and Math. San Francisco, CA, USA: S0360131517300507
No Starch Press, 2014. [35] U. Wolz, C. Hallberg, and B. Taylor, “Scrape: A tool for visualizing the
[11] S. McManus, Scratch Programming in Easy Steps. Warwickshire, U.K.: In code of Scratch programs,” Poster Presented at 42nd ACM Tech. Symp.
Easy Steps Limited, 2013. Comput. Sci. Edu., Dallas, TX, USA, 2011.
[12] T. L. Project, Super Scratch Programming Adventure!, 2nd ed. San [36] J. Moreno-Le
on, G. Robles, and M. Rom
an-Gonz
alez, “Comparing
Francisco, CA, USA: No Starch Press, 2013. computational thinking development assessment scores with software
[13] F. Kalelio
glu, Y. G€ulbahar, S. Akçay, and D. Dogan, “Curriculum integra- complexity metrics,” in Proc. IEEE Global Eng. Edu. Conf., 2016,
tion ideas for improving the computational thinking skills of learners pp. 1040–1045.
through programming via Scratch,” in Local Proc. 7th Int. Conf. Informat. [37] J. Moreno-Le
on, M. Rom
an-Gonz
alez, C. Harteveld, and G. Robles, “On
Schools: Situation, Evol. Perspectives, 2014, pp. 101–112. the automatic assessment of computational thinking skills: A comparison
[14] Q. Burke and Y. B. Kafai, “The writers’ workshop for youth programmers: with human experts,” in Proc. CHI Conf. Extended Abstracts Human Fac-
Digital storytelling with Scratch in middle school classrooms,” in Proc. tors Comput. Syst., 2017, pp. 2788–2795. [Online]. Available: http://doi.
43rd ACM Tech. Symp. Comput. Sci. Edu., 2012, pp. 433–438. acm.org/10.1145/3027063.3053216

204 VOLUME 8, NO. 1, JAN.-MAR. 2020


n et al.: Towards Data-Driven Learning Paths to Develop Computational Thinking with Scratch
Moreno-Leo

[38] J. Moreno-Le
on, G. Robles, and M. Rom
an-Gonz
alez, “Dr. Scratch: Auto- GREGORIO ROBLES received the PhD degree,
matic analysis of Scratch projects to assess and foster computational think- in 2006 with a thesis on empirical software engi-
ing,” RED. Revista de Educaci
on a Distancia, vol. 15, no. 46, 2015. neering research on free/libre/open source software.
[39] C. Kelleher, R. Pausch, and S. Kiesler, “Storytelling alice motivates mid- He is an associate professor with the Universidad
dle school girls to learn computer programming,” in Proc. SIGCHI Conf. Rey Juan Carlos, Madrid. In addition to this line of
Human Factors Comput. Syst., 2007, pp. 1455–1464. research, he is active in the field of technology
[40] K. Malan and K. Halland, “Examples that can do harm in learning pro- enhanced learning, in particular, in the study and
gramming,” in Proc. Companion 19th Annu. ACM SIGPLAN Conf. evaluation of computational thinking.
Object-Oriented Programming Syst. Languages Appl., 2004, pp. 83–87.
[41] J. Moreno and G. Robles, “Automatic detection of bad programming hab-
its in Scratch: A preliminary study,” in Proc. IEEE Frontiers Edu. Conf.,
Oct. 2014, pp. 1–4.
[42] O. Meerbaum-Salant, M. Armoni, and M. Ben-Ari, “Habits of program- MARCOS ROMAN-GONZ


ALEZ received the
ming in Scratch,” in Proc. 16th Annu. Joint Conf. Innovation Technol. PhD degree in education. He is an assistant profes-
Comput. Sci. Edu., 2011, pp. 168–172. sor with the National University of Distance Educa-
[43] F. Hermans and E. Aivaloglou, “Do code smells hamper novice program- tion, Spain. His main research lines are related to
ming? A controlled experiment on scratch programs,” in Proc. IEEE 24th computational thinking, code-literacy and computer
Int. Conf. Program Comprehension, 2016, pp. 1–10. science education, with a special focus on their
assessment. His most recent publications include a
JESUS
MORENO-LEON
joined the Universidad paper in computers & education about the integra-
Rey Juan Carlos in 2012, after a decade as a Com- tion of visual programming languages across the
puter Science teacher, to perform research and pro- curriculum in elementary school.
motion of computational thinking. He co-founded
and runs Programamos, a non-profit organization
that promotes the development of computational
thinking from early ages, and collaborates with the
European Commission as Spanish ambassador for
EU Code Week.

VOLUME 8, NO. 1, JAN.-MAR. 2020 205