This article has been accepted for inclusion in a future issue of this journal.

Content is final as presented, with the exception of pagination.

IEEE TRANSACTIONS ON EDUCATION 1

A Systematic Literature Review on Teaching

and Learning Introductory Programming
in Higher Education
Rodrigo Pessoa Medeiros , Geber Lisboa Ramalho, and Taciana Pontual Falcão

Abstract—Contribution: This paper adds to the results of (Science, Technology, Engineering, and Mathematics). In this
previous systematic literature reviews by addressing a more con- paper, “introductory programming” (commonly called CS1 in
temporary context of introductory programming. It proposes the United States) refers to a course for novice students that
a categorization of introductory programming challenges, and
highlights key issues for a research roadmap on introductory typically covers problem-solving skills, basic programming
programming learning and teaching in higher education. concepts, the syntax and semantics of a programming lan-
Background: Despite the advances in methods and tools for guage, and the use of this programming language to formulate
teaching and learning introductory programming, dropout and solutions. Failing an introductory programming course impacts
failure rates are still high. Published surveys and reviews either various other courses taken [1].
cover papers only up to 2007, or focus on methods and tools for
teaching introductory programming. Despite the advances in methods and tools for teaching
Research Questions: 1) What previous skills and background and learning introductory programming [2], [3], and the grad-
knowledge are key for a novice student to learn programming? ual integration of programming fundamentals into high school
2) What difficulties do novice students encounter in learning curricula [2], dropout and failure rates are still high in intro-
how to program? 3) What challenges do teachers encounter in ductory programming courses [4], [5]. There is no consensus
teaching introductory programming?
Methodology: Following a formal protocol, automatic and man- on what the main challenges are [1], [3]–[5], nor a clear
ual searches were performed for work from 2010 to 2016. Of and complete categorization of them [6], [7]. Previous sur-
100 papers selected for data extraction, 89 were retained after veys and systematic literature reviews either cover papers up
quality assessment. to 2007, [1], [8]–[10], or focus on methods and tools for
Findings: The most frequently cited skills necessary for teaching introductory programming [11]. There is thus a need
learning programming were related to problem solving and math-
ematical ability. Problem solving was also cited as a learning for a systematic literature review that: (a) provides a better
challenge, followed by motivation and engagement, and difficul- understanding and categorization of introductory programming
ties in learning the syntax of programming languages. The main problems; (b) covers recent research; and (c) includes both
teaching challenges concern the lack of appropriate methods and learning and teaching perspectives.
tools, as well as scaling and personalized teaching. This paper is organized as follows: Section II presents
Index Terms—Achievement, faculty development, higher edu- some previous attempts to systematize the research results
cation, introductory programming, STEM, student experience, on introductory programming. Section III describes the sys-
systematic review. tematic method adopted, and Section IV details its use for
the work presented here. Section V presents and analyzes the
answers to the research questions. Section VI further analyzes
the answers, identifying possible key issues in learning and
I. I NTRODUCTION teaching introductory programming. Section VII makes some
NTRODUCTORY programming courses are part of vari-
I ous undergraduate curricula, particularly in STEM degrees
qualitative comparisons between this study and previous ones.
Conclusions and future work are described in Section VIII.

Manuscript received October 11, 2017; revised January 4, 2018

and July 3, 2018; accepted July 22, 2018. (Corresponding author: II. P REVIOUS S TUDIES
Rodrigo Pessoa Medeiros.)
R. P. Medeiros is with the Centro de Informática, Universidade Federal de Most of the surveys and systematic reviews on introduc-
Pernambuco, Recife 50670-901, Brazil, and also with the Instituto Federal tory programming teaching and learning do not cover research
da Paraíba, Departamento de Design Gráfico, Cabedelo 58103-772, Brazil
(e-mail: [email protected]). results after 2008 [1], [8]–[10], and focus more on teaching
G. L. Ramalho is with the Centro de Informática, Universidade Federal de methods and tools than on student problems. A more recent
Pernambuco, Recife 50670-901, Brazil, and also with CESAR School, Recife systematic review of introductory programming examined suc-
50030-220, Brazil (e-mail: [email protected]).
T. P. Falcão is with the Departamento de Computação, Universidade cessful teaching practices [11]. Evaluating tools, methods, and
Federal Rural de Pernambuco, Recife 52171-900, Brazil (e-mail: practices for teaching introductory programming is essential,
[email protected]). but given that dropout and failure rates are still high [4], [5],
Color versions of one or more of the figures in this paper are available
online at http://ieeexplore.ieee.org. and that the literature shows no consensus on the main prob-
Digital Object Identifier 10.1109/TE.2018.2864133 lems involved [1], [3], [4], there is a need for understanding
0018-9359 c 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

2 IEEE TRANSACTIONS ON EDUCATION

and categorization. Indeed, a better understanding of chal- Papers were excluded if they:
lenges in introductory programming may help to develop better 1) Did not address the research questions;
teaching tools, methods, and practices. 2) Were too short, such as workshop papers;
3) Were published in local conferences;
III. M ETHOD 4) Were written by the same research group with the same
The protocol adopted in this review follows the guide- data (in which case only the most recent was kept).
lines for systematic literature reviews presented by Because of the large number of papers retrieved in the
Kitchenham and Charters [12] and Petticrew and Roberts [13]. search, a two-step procedure was performed. First, a pre-
selection step applied the selection criteria on the basis of
A. Research Questions title, keywords and abstract. Next, the full text of the pre-
selected papers was analyzed, and duplicates (criterion 4) were
RQ1: What previous skills and background knowledge are
excluded.
key for a novice student to learn programming?
Two researchers carried out the selection process, both of
RQ2: What challenges do novice students encounter in
whom independently analyzed each paper. The reasons for
learning how to program?
inclusion or exclusion were carefully noted, and meetings were
RQ3: What challenges do teachers encounter in teaching
held to resolve any disagreements, with the help of a third
introductory programming?
researcher.
The boundaries between learning (RQ1 and RQ2) and
teaching (RQ3) are obviously fuzzy, especially in as com-
plex a subject as programming. Nevertheless, this review
keeps this distinction to provide a better understanding of D. Snowballing
challenges in introductory programming, already proposed in The results from both manual and automatic searches, after
a programming context [6], [7]. the application of the selection criteria, were “snowballed”,
that is, the bibliographical references of all the selected papers
B. Search Process were considered as potential studies, and were then analyzed
The search process started with a manual search in specific according to the inclusion and exclusion criteria.
journals chosen for their relevance to the subject: the ACM
Transactions on Computing Education (TOCE), the IEEE
Transactions on Education, and Computer Science Education. E. Quality Assessment
A fourth source of data entry for the manual search was a sys- The quality of the selected papers was assessed against
tematic literature review written in 2014 and often cited in the criteria selected and adapted from Kitchenham & Charters’
literature of the area [11]. guidelines [12], focusing on the methods adopted and their
A manual search is frequently completed by an auto- scientific rigor.
matic search on scientific databases using a search string. For papers based on field studies (qualitative and quantita-
Recurring key terms—such as “programming”, “program- tive approaches), the criteria were:
ming language”, “programming teaching”, “computer pro- 1) How well was the data collection carried out?
gramming”, “coding”, “CS1”, and “novice programmers” 2) How well was the approach to, and formulation of, the
—were identified in the papers from the manual search. The analysis conveyed?
search string for the automatic search was built from these 3) How well were the contexts and data sources retained
terms and, after preliminary simulations, was defined as: and portrayed?
(“learning programming” OR “teaching programming”) AND 4) How clear and coherent were the links between data,
(“novice programmers” OR “CS1”). interpretation, and conclusions?
The databases chosen for the automatic search were: For theoretical papers, the criteria were:
ACM Digital Library, IEEE Xplore Digital Library, 1) How well did the analysis address its original aims and
SpringerLink, ScienceDirect and ERIC (the Education objectives?
Resources Information Center, sponsored by the Institute of 2) How has knowledge or understanding been extended by
Education Sciences). Before doing the automatic search, the the research?
search string was validated by being used in the IEEE and 3) How well was diversity of perspective and context
ACM databases. The papers retrieved were cross-checked explored?
with the papers selected manually (by applying inclusion and The assessment was performed independently by two
exclusion criteria - see below) from IEEE and ACM journals researchers, and each paper was scored on the scale (for each
respectively. criterion): 0 - very poorly; 1 - poorly; 2 - reasonably; 3 - well;
4 - very well. For scores that differed between the researchers
C. Selection Criteria and Procedure by two or more points, a meeting was held with the third
Papers included in the review had to be written in English, researcher to resolve the conflict. A quality threshold was
and published in conferences or journals, or as book chapters, adopted to decide whether to keep the papers in the analy-
between 2010 and 2016, on the theme of teaching and learning sis. The scores obtained for each question were averaged to
introductory programming in higher education. give the final score for each paper.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

MEDEIROS et al.: SYSTEMATIC LITERATURE REVIEW ON TEACHING AND LEARNING INTRODUCTORY PROGRAMMING IN HIGHER EDUCATION 3

F. Data Collection and Analysis

The following data were extracted from each study and
recorded in the codebook (in spreadsheet format):
• Search mechanism (manual, automatic or snowballing)
and source (journal name, database name and article title,
respectively);
• Title, keywords, authors, publication venue, type (journal
or proceedings), and year;
• Objective and participants;
• Scientific method (qualitative and/or quantitative
approach, and specific methods adopted);
• Summary of the study;
• Answers to each research question;
• Quality assessment;
One researcher extracted the data, with the other check-
ing the extraction. Any disagreements were discussed until
resolved; if necessary, the third researcher was involved.
To analyze the answers to the research questions, a bottom-
up inductive approach was adopted. Firstly, key terms were
collected exactly as used in each paper. Different terms used
to refer to similar concepts were identified, showing that
authors choose words rather freely in discussing a topic,
rather than following a standard terminology (perhaps drawn
from prior experience or previous experience in program-
ming). Such synonyms were consolidated under the most
frequently-used terms. Then related concepts were grouped
into categories under each research question, to clarify the
presentation and discussion of the results, Section V. This
grouping also drew on the literature where applicable (for
example on theories on Computational Thinking [14], [15]),
as discussed in Section V. Some terms were used by some
authors with slightly different meanings; these are discussed
in Section V. The first two researchers performed the inductive
analysis together, then the third revised and validated it.

IV. M ETHOD A PPLICATION

A. Search and Selection Results
The search for papers relevant to the review followed
the process laid out in Fig. 1. The manual search pro-
vided 721 potentially relevant papers, from which 45 were
pre-selected. The automatic search yielded 5,870 potentially
relevant papers, from which 101 were pre-selected. This gave Fig. 1. Search process, indicating numbers of papers at each step.
a total of 146 pre-selected papers. In the second phase of
the selection process, 26 papers from the manual search and
62 papers from the automatic search were retained, giving
88 relevant papers in total. The snowballing process on these Generally, there was good agreement between raters: only
88 selected papers provided another 102 candidates, of which six of the 100 papers had scores for at least one of the crite-
12 were selected. The final total at the end of the whole pro- ria that differed by two or more points. In the histogram of the
cess was thus 100 papers. These are listed as Selected Papers quality assessment scores, Fig. 2, the x-axis labels the upper
in the Reference section of this paper [R01]–[R100]. border of each histobar range.
Most studies received a good score, which is to be expected
B. Quality Assessment Results since they were published in peer-reviewed scientific journals
To assess the quality of the selected studies (by answering and conferences. A few studies, however, obtained signif-
a series of questions—see Section V), the scientific meth- icantly low scores, and were not considered of acceptable
ods they adopted to support their arguments were collected, quality to be included in the analysis. The threshold score
Table I. This information per se can be of interest to the adopted was 1.81, which corresponds to one of the apparent
computer science education community. discontinuities in the histogram. Consequently, 11 papers (of
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

4 IEEE TRANSACTIONS ON EDUCATION

TABLE I
OVERVIEW OF M ETHODS

Fig. 2. Histogram of the quality assessment scores, showing the apparent

discontinuities in the threshold score adopted. X-axis labels give the upper
border of each histobar range, and the number of papers in that range.

TABLE II
S TUDENT S KILLS

the 100) were excluded. The results and discussion below only
consider the remaining 89 papers.

C. Protocol Limitations
V. R ESULTS
This review shares the most common limitations of the
systematic method: search coverage and possible biases intro- A. Skill and Background Knowledge (RQ1)
duced during study selection, data extraction, and analysis. The skills extracted from the papers were grouped into two
These limitations were addressed following the general recom- categories: Programming-Related and General Educational,
mendations for systematic reviews—using a combined manual Table II.
and automatic search complemented by a snowballing process, 1) Programming-Related Skills: Problem solving, which
and having two or more researchers selecting studies, assessing can be defined as understanding the context of a problem,
quality and extracting data [12], [13]. identifying key information, and making a plan to solve it [16],
The specific limitation of the study lies in the fact that was among the most cited skills for answering RQ1 (16 pub-
the stated research questions require answers that are not lications). It is considered a prerequisite for learning how
binary or objective. This review required a categorization of to program: “There are researchers that emphasize that the
terms used by different primary studies to characterize intro- computer programming requires the use of problem-solving
ductory programming challenges. However, as is typical of techniques. The lack of mastery of these techniques makes
education-related issues, the categories have fuzzy boundaries, it difficult to learn programming.” [R39, p. 02]; and even
so a given lexical term is used by different authors with the the main goal of introductory programming courses: “[the
same meaning. goal is] to familiarize the students to the art of problem
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

MEDEIROS et al.: SYSTEMATIC LITERATURE REVIEW ON TEACHING AND LEARNING INTRODUCTORY PROGRAMMING IN HIGHER EDUCATION 5

solving and to teach them how to express solutions in a sim- world to a more semantic and symbolic reality, formulat-
ple way” [R15, p. 68]; and “the long-term goals for the ing or representing a problem in a more abstract manner,
introductory programming course are that the students should less attached to the details and singular concrete elements.
eventually develop problem-solving ability and design compe- When a student faces difficulties while attempting to do this
tence” [R41, p. 130]. It is also important that students can abstraction, he/she is not capable of moving from reality
solve problems efficiently when working cooperatively with to a more generic language such as the notation used in
peers, teachers, and mentors [R39]. programming” [R28, p. 04].
Mathematical ability is cited as a necessary skill closely 2) General Educational Skills: Critical thinking and dis-
related to problem-solving ability. Fifteen studies argue that cussion skills were cited in four publications, associated with
deficient mathematical ability is one reason for the diffi- the students’ level of maturity at the university. They affect
culty encountered by novice learners of programming: “When the interpretation and analysis of problems, and figure in the
talking with programming teachers, most of them claim that low incidence of classroom debate.
students don’t know how to program because they don’t know Four studies comment on the importance of creativity in
how to solve problems and do not have enough mathematical programming, for example, “it is important to stress that these
background” [R07, p. 113]; “Moreover many students did not students also believe that mathematical dexterity is important
show basic mathematical skills that are expected when enter- as well as creativity, even though they state that programming
ing higher education (...). Students had difficulties in calculus; is basically reasoning” [R28, p. 04].
Students did not have enough basic mathematical concepts Five publications add that limited knowledge in English
concerning number theory; Students had difficulties to trans- is a barrier, as it is used in the syntax of all consolidated
form a textual problem into a mathematical formula that solves programming languages. Reference [R23] argues that, in the
it; (...) Students had weak abstraction levels; Students had lack course they analyzed, fluent English speaking students were
of logical reasoning” [R45, pp. 01–02]. However, the correla- more likely than non-fluent ones to be successful.
tion between mathematics and programming is not necessarily Finally, time management was also cited as an essential skill
a cause-effect one: “some teachers believe that mathematical by [R04] and [R81], in terms of planning use of time and the
ability is important for programming. However, they believe stages in the execution of a project.
that the positive correlation between them is due to more basic
cognitive functions that would be common to both domains.
Thus, when developing a “mathematical logic”, the student B. Difficulties Encountered by Students (RQ2)
acquires structures or cognitive abilities that facilitate or The first three categories in Table III are related to the
promote learning to program” [R28, p. 04]. three stages of computational thinking (problem formula-
Eleven publications discuss the impact of students’ pre- tion, solution expression and solution execution and evalu-
vious knowledge in programming; there is no consensus as ation) presented by Repenning, Basawapatna and Escherle
to its consequences. For instance, [R15] and [R29] consider [14, pp. 268–269]. The remaining category is behavior, related
previous knowledge in programming as a predictor of suc- to social, emotional and self-management aspects.
cess for novice learners in higher education: “Although a few 1) Problem Formulation: As expected, problem-solving
of the students have already learnt some programming lan- reappeared as a challenge faced by students, with 23 studies
guage before getting enrolled in Computer Science degree identifying this as a difficulty. These echo the arguments dis-
program at university level, and it is believed that prior pro- cussed with respect to RQ1 in Section V-A above: “As may
gramming has a positive impact on Computer Science studies be expected, the learners face more difficulties to establish
(...)” [R15, p. 64]; “Students who studied programming in the relations between different problem instances. Conceivably,
past tend to do well compared to those with no programming a student who has gained a little familiarity with this
experience.” [R29, p. 296]. problem-solving approach should provide logically correct
However, the associated benefits are not straightforward: answers” [R33, p. 164]; “Many of these [students] end up
“We normally think of the ability to code as the main benefit of dropping the course due to not being able to solve problems
having prior programming experience, but student interviews and therefore feeling inadequate.” [R26, p. 93].
suggest that the impact of this experience is more complex: Another problem cited, also discussed under RQ1, concerns
it affects students’ expectations, work habits, attitude and the abstract nature of programming, with similar arguments:
confidence, and perceptions of self and peers” [R06, p. 244]. “(...) students in learning programming need to imagine and
Finally, abstraction is cited in six studies. Wing [17] defines comprehend many abstract terms that do not have equivalents
abstraction as “the most important and high-level thought pro- in real life: how does a variable, a data type, or a memory
cess in computational thinking (...) used in defining patterns, address relate to a real-life object? These concepts are difficult
generalizing from specific instances, and parameterization (...) to grasp. Consequently, many students struggle to comprehend
to capture essential properties common to a set of objects”. even the most basic of programming concepts” [R49, p. 16];
Indeed, “a programmer needs to apply abstraction when “Students face diverse problems when they are learning pro-
analyzing computational problems, as well as to instantiate gramming (...) mainly because programming is dynamic and
abstract programming concepts and techniques to solve par- abstract” [R54, p. 414].
ticular computational problems” [R12, p. 624]; “It is expected Algorithmic reasoning, which appeared in three studies, is
that students will use it [abstraction] to pass from the concrete understood here as “a pool of abilities that are connected to
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

6 IEEE TRANSACTIONS ON EDUCATION

TABLE III
S TUDENT D IFFICULTIES this way, the student ‘no longer thinks as a person but starts
thinking as a computer, taking into account the machine’s
limited comprehension.”’ [R28, pp. 03–04].
2) Solution Expression: Once the problem is formulated,
students can move on to expressing a solution through pro-
gramming structures. The first problem within the solution
expression scope, with 20 occurrences, concerns the syntax
of programming languages, as reported by [R48, p. 02]: “The
overhead of learning the syntax and semantics of a language
at the same time, and difficulties in combining new and previ-
ous knowledge and developing their general problem-solving
skills”. According to [R49, p. 02], “even students, who have
adequate problem-solving skills and manage to phrase a solu-
tion to a programming problem in terms of a pseudo code, find
it difficult to turn the pseudo code into a syntactically correct
computer program.” In this context, it is important to consider
syntax error as part of the learning process and help stu-
dents learn to recover from them [R36], [R48]: “Syntax errors
are an important area of programming pedagogy research.
Experienced programmers rarely make syntax errors, and
when they do, they have clear strategies to correct them
very quickly. However, syntax errors are significant for novice
programmers; correcting them is a time-consuming process
and often leads to random debugging behavior, also influ-
enced by the fact that students do not understand compiler
messages” [R48, p. 16].
Control structures also appeared as a challenge (13 occur-
rences). Among them, conditional statements [R25], [R43],
loops [R14], [R25], [R27], [R43], [R48], [R65], [R86], [R90],
recursion [R18], [R27], [R90], and sequence, selection and
repetition [R21], [R30] were the most cited. Selecting the
appropriate structures for solving a problem (if, if/else, with,
for, while) was also cited [R21] as a difficulty.
Several papers also mention difficulties related to the use of
data structures. References [R21] and [R34] explore this topic
the most deeply, but only on the basis of previous literature,
and not on their findings. Data structures most often cited as
potentially difficult were arrays [R21], [R30], [R61], [R90].
For code structure, some papers mention the use of func-
tions/classes [R21], [R65], the relationship between classes
and objects [R25], and using language libraries [R34].
Finally, other aspects of solution expression cited were: point-
ers [R03], [R21], [R34], [R53], [R65], references [R34], [R65],
parameters [R25], [R34], [R65], variable scoping [R53] and
error handling [R34].
3) Solution Execution and Evaluation: Students should test
and analyze their code to identify and correct problems. In this
context, debugging (13 publications) and code tracing (nine
publications) were the most cited challenges.
Debugging is described as a complex activity, requiring sev-
constructing and understanding algorithms: to analyze given eral aspects to be mastered: “Debugging, one of the essential
problems; to specify a problem precisely; to find basic actions skills for successful programmers, is difficult for novices as it
that are adequate to the given problem; to construct a cor- requires the application of many new skills simultaneously.
rect algorithm to a given problem using the basic actions” Students must understand the problem domain, know rudi-
[18, p. 160]. According to teachers, “(...) it is this shift in mentary programming concepts and understand at least one
perception that allows them [the students] to see the whole programming language well enough to read and write instruc-
based on the parts that will allow the student to elaborate tions, comprehend the logic of the intended program, and be
the algorithm. In words of one of the teachers, when thinking able to track down and fix bugs” [R55, p. 390]. It demands
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

MEDEIROS et al.: SYSTEMATIC LITERATURE REVIEW ON TEACHING AND LEARNING INTRODUCTORY PROGRAMMING IN HIGHER EDUCATION 7

TABLE IV
much practice and should be explicitly supported by teach- FACULTY C HALLENGES
ers: “debugging should be explicitly taught. Assuming students
will simply “pick up” debugging skills as a by-product of
learning to program may lead them to develop some of the
ineffective strategies observed in this study” [R55, p. 395].
Reference [R02] states that students rarely test their code, per-
haps because they do not know how to formulate proper test
cases, or because they lack the discipline to test their code
systematically.
Finally, students should also be given the opportunity for
developing code tracing skills, as cited by nine papers, since
this helps them understand programming more holistically.
This activity may be related to debugging, but can also be
used to understand code passages. The authors argue that:
“code tracing activities exposed students’ not viable models
and, more importantly, provided a much clearer starting point
with which to discuss correct models” [R53, p. 562].
4) Behavior: This category encompasses social and emo-
tional aspects that can have an impact on student learning, such
as motivation, engagement, and confidence (including the per-
ception of programming as being difficult), as well as study
habits and time management.
The most discussed issue in this category (36 occurrences),
is the relationship between students’ motivation and engage-
ment, and positive learning outcomes [R14]: “For learners,
engagement correlates with improvements in specific desir-
able outcomes (...), such as general abilities and critical themselves acknowledge this difficulty: “Another constantly
thinking, cognitive development, improved grades and per- recurring theme in the free comments section was time man-
sistence” [R14, p. 02]. The perception of programming as agement issues. Many students reported that they have too
a complex discipline, coupled with the previously discussed many concurrent courses, all with high workload. Without
learning challenges, contributes to students’ demotivation, par- careful time management and prioritizing, there is not enough
ticularly in the first year of higher education [R10]: “(...) an time for deep-level learning in every study topic” [R44, p. 04].
unmotivated student will hardly succeed. This is aggravated Another issue is study skills (eleven occurrences) such as
by the fact that programming courses usually gain a reputa- organization and minimal work habits [R10], or the compre-
tion that passes from student to student of being difficult that hension of students’ own learning styles [R46], [R75], [R80]:
so many of them start already feeling defeated” [R12, p. 02]. “The acquisition of well-functioning learning strategies and
“The motivational dimension has a great impact on the indi- skills seems, in this specific context, to be affected by group
vidual’s cognitive development and is a determinant factor work strategies, extrinsic motivation, and some issues related
for success in the learning process (. . . ). It is motivation that to study habits. It is not always easy to establish a direction of
fosters in the student the disposition to want to progress and causality, but one important aspect in learning programming
reach the goals that were set, maintaining an adequate level seems to be the work that students are required to do on their
of volition to overcome the demands that are being dealt own time, outside the instructed learning sessions. For one
with” [R13, pp. 1–2]. reason or another, in too many cases students are not able to
Students’ confidence is another issue (nine occurrences). find effective ways of working (...)” [R80, p. 306].
According to [R10, p. 02], “some students may have moti-
vation but the self-confidence may be blocked, as such these C. Faculty Challenges (RQ3)
types of students need the teacher to work closer with them Challenges faced by faculty when teaching introductory
to develop self-esteem”. Similarly, [R98, p. 14] argues that programming are listed in Table IV. Obviously, learning chal-
“Student programmers who lack confidence are less able to lenges (RQ1 and RQ2) would naturally appear in RQ3 as
make independent progress with coding exercises. They fre- teaching ones. To avoid redundancy, only teaching chal-
quently become “stuck”, and will wait for assistance from lenges corresponding to the most-cited learning difficulties,
the instructor, rather than try an alternative approach on Table III, were included in the discussion here. Other teach-
their own”. Confidence in programming is expected to change ing challenges that have not yet been discussed are also
rapidly as the student gains experience [R98]. treated here.
Fifteen publications mention students’ poor time manage- 1) Revisited Challenges: Maintaining student motivation,
ment as another challenge: “most students indicated that they engagement and persistence is fundamental (17 occurrences),
regularly spent far less time studying for the unit than the rec- but teachers struggle to find strategies to attract students’
ommended eight hours [per week]” [R46, p. 125]. Students interest [R10]. Reference [R24, p. 499] argues that the
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

8 IEEE TRANSACTIONS ON EDUCATION

teacher should “be a motivator, not just a provider”, and in-class code tracing and coding activities, labs, small pro-
that “Not many teachers realize that it is also their respon- gramming homework exercises, larger programming projects,
sibility to motivate the students and arouse the students’ and other opportunities for students to develop valid mental
interests” [R24, p. 499]. Reference [R51] highlights the role models. We observed that some of these activities seemed to
of the instructors in helping students to persist: “Instructors hurt as much as they helped.”. On the other hand, [R53] reports
should thoroughly ground novices in foundational skills and an experiment using a trace-driven teaching approach that
simple problem solving before presenting problem-solving decreased the dropout and grade failures by 25.49% and 8.51%
plans, so that novices have the background needed to under- respectively. Reference [R47] comments on the need to priori-
stand and appreciate those plans. Doing so should increase tize and adjust teaching methods to improve problem-solving,
the persistence of less-advanced novices in introductory one of the most cited skills necessary for learning pro-
courses” [R51, p. 311]. A third concept closely related to this gramming (RQ2). Reference [R49] comments on the lack of
discussion is keeping student’s engagement: “(...) improving appropriate tools for teaching the object-oriented paradigm.
the student engagement can be different from case to case Reference [R19] comments that the choices of tools and the
and must be elaborated into practical approach and strate- initial programming language are important and challenging
gies” [R24, p. 502]; “Some of the factors that can result for teachers.
in un-engagement among students are: the current style of 3) Scale Problems: The scale topic is related to num-
education doesn’t appeal to everyone; some educators are ber and diversity of students in classes, and it involves
not sensitive to students’ responses to educational methods in various aspects: heterogeneity of students [R09], [R10],
class; students overwhelmed by all they have to do - ‘quality [R31], [R57], [R68], [R72], staff resource limitations and
of effort”’ [R14, p. 03]. classes’ size [R03], [R09], [R32], [R44], [R72], [R93], [R96]
Students’ inadequate mathematical background is also revis- and personalized teaching [R09], [R10], [R50].
ited; teachers should create specific activities and/or change Teachers struggle to develop problem-solving skills in
the pedagogical approach to deal with it: “(...) we propose classes with students with heterogeneous levels of knowledge,
the introduction of activities that foster the development of commitment and different learning styles [R10]. Furthermore,
mental models that are fit to the cognitive skills needed in it is extremely hard to address cognitive needs and individ-
programming, thus helping the students succeed in the course. ual difficulties in large groups [R10]. Staff is also essential
While problem based learning tries, by definition, to pro- since tutors and mentors can help increase the number of
mote problem solving abilities, we have verified that it is not activities and assessments [R03], [R32], [R44]. Using extrin-
enough when dealing with the students that present greater sic motivators to engage students also requires planning and
difficulties” [R28, p. 06]. extra effort [R03].
2) Methods and Tools for Teaching Programming: The The heterogeneity of students, cited by six publications,
main challenge in RQ3, cited by 33 publications, is related to relates to knowledge, motivation, commitment and learning
teaching methods and tools. This cross-cutting concern under- rhythm: “It is possible to explain concepts and examples,
lines various challenges discussed in RQ3. Various methods but it is very hard to promote the development of problem
are suggested: learning by doing and learning by examples; solving skills and address the variety of cognitive needs, learn-
problem-based learning; pre-recorded classes; active learning ing styles, difficulties and motivations present in a group
exercises and demonstrations; live coding, canned examples of students that is often very heterogeneous” [R10, p. 01];
and trace-driven teaching approach; extreme apprenticeship; “Therefore, it is difficult for the teacher to follow an approach
games, game-themed assignments and gamification; team suitable for every individual student. In an attempt to reach
based-learning; media computing; collaborative learning; col- all students, teachers often design lectures and activities to
laborative tutoring session; mentor support; peer instruction the ‘average student’, who may not even exist in the class.
and pair programming. However, the results of the application To improve this situation personalized support and guidance
of these methods are not yet conclusive [11]. Reference [R46] are necessary, so that individual needs and difficulties can be
reinforces the necessity of rethinking the current teaching addressed” [R09, p. 01].
methods: “This apparent mismatch in teacher expectations Five publications comment on staff resource limitations and
and student behavior might give universities the impetus to how this affects the teaching and learning process: “The gen-
rethink the type of educational experiences we provide for our eral problems covered contextual issues such as staff resource
students” [R46, p. 128]. limitations, university control systems (...). Changing a course
Some studies report applications of specific teaching meth- and increasing the number of student assignments required
ods in introductory programming in classes, but these do not extra resources, but a course revision should, in the long run,
always succeed, for reasons which are often unclear. One aim at optimizing the resource needs and uses” [R03, p. 23].
case of partial failure is reported in [R53, pp. 562–563]: In general, teachers say that having more people involved in
“Like many institutions, our introductory classes suffered from the teaching process, such as tutors and mentors, would help
low retention rates and poor student performance. These to make teaching more fluid, with better classroom dynamics
problems occurred despite our making a number of peda- and fewer differences in learning rhythms [R03], [R32], [R44].
gogical changes including adding active learning exercises 4) Teacher-Student Communication and Feedback: Another
and demonstrating how code features worked using live cod- challenge is student-teacher communication and the feedback
ing and canned examples. (...) The classes already included process (16 occurrences): “A good pedagogical theory for
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

MEDEIROS et al.: SYSTEMATIC LITERATURE REVIEW ON TEACHING AND LEARNING INTRODUCTORY PROGRAMMING IN HIGHER EDUCATION 9

teaching programming should focus on students’ learning, and process. (...) In general, a competent instructor may be able to
effective communication between teacher and student. This deliver a successful course without all the proposed systemic
could be achieved by clearly stating goals and keep the stu- frameworks” [R03, p. 23].
dents motivated” [R14, p. 104]. The way of evaluating and There also seems little research on instructional sequences
providing feedback to students can be a determining factor for teaching programming for novices in higher education, as
in their demotivation [R09]–[R12], [R14], [R29], particularly pointed out by [R51]: “(...) no study has directly compared
for women [R71]. However, giving quality feedback is not the effectiveness and efficiency of the instructional sequences
simple: “Give feedback, not only judgment. Assessment can used in presenting programming material to novices. Is it bet-
be classified into two categories: formative and summative. ter to begin with a strategic overview, to start with syntax
In formative assessment we can get the feedback of the cur- details and work upward, or to work through entire program-
rent teaching and learning condition and use the feedback to ming concepts one at a time? Knowing the effect of sequence
improve them. On the other hand, the summative intention is on instructional effectiveness can help instructional designers
solely to form a judgment” [R24, p. 499]. Besides its inherent avoid sequences that make learning unnecessarily difficult in
complexity, the feedback process is also affected by lack of an already difficult domain” [R51, p. 292].
time, quantity of students or format of courses.
More generally speaking, there are two reasons that teacher-
VI. G ENERAL D ISCUSSION
student communication seems to fail: too many students in one
class [R10]; and students’ resistance to discussing their errors Some issues for inclusion in the research roadmap on teach-
in exercises due to lack of rapport [R24]. There is a clear rela- ing and learning introductory programming have emerged
tionship between the communication and feedback challenges, from this review:
and the category of scale problems.
5) Choice of Programming Language: The most specific A. Better Understanding and Characterization of
introductory programming challenge is choosing the first pro- Problem-Solving in Programming
gramming language to be taught to students. Several program- Problem solving is a crucial concept in introductory pro-
ming languages can be used for introductory programming gramming. Some authors in this review [R09], [R15], [R41],
courses, as discussed by [R21, p. 02]: “The available pro- [R48], [R49] consider that the primary aim of an introduc-
gramming languages are numerous and selecting the one tory programming course is to develop problem-solving skills
that will be used is a multi-criteria decision”. This choice through algorithmic thinking and basic concepts of program-
has a direct impact on the development of novices’ pro- ming, rather than to teach the syntactic particularities of
gramming skills [R48] and can be key in facilitating the a specific programming language. Positive student percep-
teaching process [R05] and shaping programming style and tions have been reported from approaches aligned with this
coding technique [R15]. Reference [R47] discusses the choice perspective [20]. Computer Science Curricula 2013 [21] rec-
between commercial and academic languages: “The teacher ommends that introductory programming courses should avoid
is exposed to risks when teaching programming courses for conveying the idea to students that computer science is mainly
novices in which the emphasis is on the programming lan- about learning the specifics of a programming language, and
guage. One of the risks is that the novices focus their attention instead emphasize general concepts in computing within the
on syntax issues and not on the computational semantic power context of learning how to program.
of the language, which at the end of the day is what makes Although problem solving appears in several articles, its
it possible to build solutions using computer programs. This definitions are generic, lacking, or inconsistent across authors.
approach prevents novices from understanding that the main The reasons for students’ limitations in problem solving are
role of a programming language is to serve as a means to not detailed: is it because they do not understand the statement
express computational solutions proposed in the training exer- of the problem, do not know the strategy to solve it, or do not
cises. Moreover, choosing a standard de facto language in the know a step-by-step plan to implement the strategy? How is
industry offers the advantage of training the student to develop problem-solving related to algorithmic thinking, computational
skills that the market is looking for. However, it can also gen- thinking, or abstraction? Is problem-solving a monolithic
erate a bias of the concept of the student regarding the futility issue, or does it encompass various sub-problems? And if so,
of learning other languages and, besides, it can reduce the how are they related? Some current definitions of computa-
environment in which students develop the ability to learn to tional thinking [15] may help. Deeper investigations should be
learn” [R47, p. 02]. carried out to understand what problem-solving really means
6) Curriculum and Instructional Sequences: [R03] high- in programming.
lights the idea that the introductory programming course
should build more explicitly upon skills acquired in previous
courses. “At the curriculum level, departments should develop B. Improving Background Knowledge
explicit course sequences that build upon the skills acquired Students’ inadequate background knowledge (mainly of
in other courses to increase the student intrinsic motivation mathematics and English) is cited by introductory program-
to complete the courses as planned. The departments should ming teachers as a challenge, but how can they “solve” this
also track the performance of their courses to be able to act problem within the timeframe of an introductory program-
on problems early, since rehabilitating a course is a lengthy ming course? Efforts to teach programming to children and
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

10 IEEE TRANSACTIONS ON EDUCATION

teenagers at school [2] are expected to improve this situa- Therefore the creation of experimental standards for tasks,
tion for the future generations of programming students, but participant profiles and number, expected results, and evalua-
this is not certain. Methods and metrics should be estab- tion criteria should be encouraged. Benchmarks for evaluating
lished to evaluate the efficacy of this approach. In a broader methods and tools for introductory programming would be
perspective, it could be desirable to foster a productive and very welcome.
systematic exchange of information between school and uni-
versity teachers, with the involvement of policymakers who
define curricula at all levels. This would help bridge gaps in VII. C OMPARING R ESULTS W ITH P REVIOUS S TUDIES
students’ competencies between school and university. Such Studies covering papers published before 2010 [1], [8]–[10]
interaction should be encouraged. are surveys rather than systematic literature reviews. They
do not follow a rigid protocol in presenting results, so these
C. Better Specific Tools and Methods for Problem cannot be quantitatively compared with the results of this
Formulation and Solution Expression review. Nevertheless it is possible to qualitatively compare
Some teachers believe that learning to properly formulate some issues from the 1990s or 2000s. Remarkably, the solu-
a problem and express its solution in a specific programming tion expression problems (language syntax, control structures,
language, in the timeframe of an introductory programming and data structures) highlighted in the earlier studies are
course, is a great cognitive load for students. The idea of still relevant, but are now of similar importance to problem
separating problem formulation strategies and solution expres- formulation.
sion into two different courses is gaining ground, but this Several topics listed as novices’ difficulties by Robins et
assumes that problem framing and expression is one of the al. in their review from 2003 [9] were also identified in
roots of the learning problems in introductory programming. the present review, including: problem solving (strategies and
More empirical evidence is needed. skills); solution expression (abstraction, use of data structures
More methods and tools are also needed if these two like variables and arrays, use of control structures like loops,
phases are to be developed separately. Nowadays, block-based conditional statements, and recursions); and solution execution
programming environments for beginners help developing (tracing/tracking code, dedicating time to planning and test-
computational thinking, and are not tied to a professional pro- ing; and debugging). This 2003 review did not discuss issues
gramming language as taught in university courses. However, of choosing a programming language and learning syntax, or
the vast majority of these are designed for children (e.g., the importance of students’ mathematical background, algo-
Scratch, Alice, and others from code.org). Although they have rithmic and logical reasoning—all of which were identified in
been shown to help [2], [R61], most are too childish to be the present review.
transferable to the university level. Building similar tools Topics involving methods and tools for teaching and learn-
higher education should be considered. ing introductory programming were a significant concern
in 2003, and, as discussed in the present review, still are.
D. Improving Motivation Robins et al. [9] discussed the importance of fostering learning
of core principles of programming, focusing on student learn-
Motivation appears is a significant concern for teachers, ing rather than instructor teaching, linked to methods such
with high impact on the other learning problems. Approaches as learning-by-doing, problem solving-based methods, peer
are needed to help teachers maintain student motivation. and collaborative work, and lab-based learning (all of which
Many papers discuss motivation, but only one examines the should be combined with appropriate assessment and feedback
reasons for students’ demotivation [R07]. A more in-depth given through effective student-teacher communication). The
understanding of this phenomenon seems necessary. Several authors also discussed tools for teaching programming; cur-
promising teaching methods (such as gamification, problem- riculum and instructional sequences, but did not discuss scale
based learning, or unplugged computing) are available, but problems.
these are still not commonly adopted in higher education. From the 2014 review [11, p. 25] focused on evaluating
The barriers to adoption could be teaching culture, insufficient methods for teaching introductory programming: “What may
knowledge of the methods, the work required to put them into be missing however, are the reports on interventions that did
practice, or other unknowns. not yield an improvement. Thus, educators that have tried an
intervention but received poor results should also be encour-
E. More Empirical Basis and Standards aged and supported in reporting the results to create a more
The last point concerns the basis upon which the papers ana- stable picture of the field”.
lyzed deliver their conclusions. Several authors make claims Robins et al. [9] discuss the importance of engagement
mainly based on observations made by teachers in their daily and indicate motivation and confidence as potential factors for
classroom work. Although the importance of these obser- better understanding effective and ineffective novice learners.
vations cannot be denied, empirical data from more formal This confirms the finding of the present review of the growing
experimental projects would be welcome. When empirical interest in student engagement and motivation in recent years.
results exist, they are not comparable because the experimen- Further topics related to the behavior of novice programming
tal protocol, the number and profile of participants, and even learners discussed here, such as time management and study
the goals are too heterogeneous. skills, were not mentioned in [9].
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

MEDEIROS et al.: SYSTEMATIC LITERATURE REVIEW ON TEACHING AND LEARNING INTRODUCTORY PROGRAMMING IN HIGHER EDUCATION 11

VIII. C ONCLUSION [5] J. Bennedsen and M. E. Caspersen, “Failure rates in introductory

programming,” ACM SIGCSE Bull., vol. 39, no. 2, pp. 32–36, 2007.
A systematic literature review was performed on 100 papers [6] D. Teague, “Pedagogy of introductory computer programming: A
to move towards a better understanding of introductory pro- people-first approach,” M.S. thesis, Dept. Inf. Syst., Queensland Univ.
gramming problems. With respect to previous systematic Technol., Brisbane, QLD, Australia, 2011.
[7] P. Kinnunen, “Challenges of teaching and studying programming at
reviews, this study has as main contributions: (i) covering more a University of Technology—Viewpoints of students, teachers and the
recent papers; (ii) proposing novel research questions, includ- University,” Ph.D. dissertation, Dept. Comput. Sci. Eng., Helsinki Univ.
ing an original discussion on previous skills and background Technol., Espoo, Finland, 2009.
[8] T. Jenkins, “On the difficulty of learning to program,” in Proc. 3rd
knowledge; (iii) stimulating discussion of a categorization of HEA Conf. ICS Learn. Teach. Support Netw., Loughborough, U.K., 2002,
introductory programming challenges; (iv) suggesting poten- pp. 1–8.
tially key issues for a research roadmap on introductory [9] A. Robins, J. Rountree, and N. Rountree, “Learning and teaching pro-
programming learning and teaching. gramming: A review and discussion,” Comput. Sci. Educ., vol. 13, no. 2,
pp. 137–172, 2003.
For RQ1, on students’ background knowledge and skills, [10] A. Pears et al., “A survey of literature on the teaching of intro-
the most frequently cited issues were problem-solving abil- ductory programming,” in Proc. ITiCSE-WGR, Dundee, U.K., 2007,
ities and mathematical knowledge. For RQ2, on challenges pp. 204–223, doi: 10.1145/1345443.1345441.
[11] A. Vihavainen, J. Airaksinen, and C. Watson, “A systematic review of
faced by students in introductory programming courses, the approaches for teaching introductory programming and their influence
major issues cited were related to motivation and engagement, on success,” in Proc. ICER, Glasgow, U.K., 2014, pp. 19–26.
problem-solving, and the syntax of programming languages. [12] B. Kitchenham and S. Charters, “Guidelines for performing systematic
literature reviews in software engineering, version 2.3,” School Comput.
For RQ3, about the challenges faced by faculty, the need for Sci. Math., Keele Univ., Keele, U.K., Rep. EBSE-2007-01, 2007.
appropriate methods and tools for teaching programming at [13] M. Petticrew and H. Roberts, Systematic Reviews in the Social Sciences:
the introductory level were the most cited issue. A Practical Guide. Hoboken, NJ, USA: Blackwell, 2008, p. 352,
This review has also shown the need to: (i) clarify the doi: 10.1002/9780470754887.
[14] A. Repenning, A. Basawapatna, and N. Escherle, “Computational think-
concept of problem-solving in programming; (ii) foster the ing tools,” in Proc. IEEE Symp. Vis. Lang. Human-Centric Comput.
dialogue between the communities of primary and higher edu- (VL/HCC), Cambridge, U.K., 2016, pp. 218–222.
cation; (iii) create specific tools for the problem formulation [15] M. Berry, QuickStart Computing: A CPD Toolkit for Primary
Teachers, BCS, Swindon, U.K., 2015. [Online]. Available:
stage; (iv) understand why promising teaching methods are http://primary.quickstartcomputing.org/resources/pdf/qs_handbook.pdf
not frequently adopted in higher education; (v) stimulate the [16] S. Lehoczky and R. Rusczyk, The Art of Problem Solving, Volume 1:
creation of experimental standards. The Basics, 7th ed. Alpine, CA, USA: AoPS, 2006, p. 288.
[17] J. M. Wing, Computational Thinking—What and Why? Link Mag.,
In future work, the authors aim to pursue some of the issues Carnegie Mellon Univ., Pittsburgh, PA, USA, 2011, pp. 20–23. [Online].
raised in Section VI, deepening the investigation based on Available: http://link.cs.cmu.edu/files/11-399_The_Link_Newsletter-
a better understanding of problem-solving in programming, 3.pdf
following recent results on the categorization of computational [18] G. Futschek, “Algorithmic thinking: The key for understanding
computer science,” in Informatics Education—The Bridge Between
thinking [14], [15]. Another stream of future research, already Using and Understanding Computers. ISSEP 2006 (LNCS 4226),
under way, is an analysis of barriers to successful methods R. T. Mittermeir, Ed. Berlin, Germany: Springer-Verlag, 2006,
in basic education on computational thinking being used in pp. 159–168, doi: 10.1007/11915355_15.
[19] V. Trowler, Student Engagement Literature Review, Higher Educ. Acad.,
higher education. Finally, the authors intend to create a new York, U.K., 2010. [Online]. Available: https://www.heacademy.ac.uk/
systematic review devoted to the investigation of empirical system/files/studentengagementliteraturereview_1.pdf
methods, tools, and type of data employed in the identi- [20] S. Turner and G. Hill, “Robots in problem-solving and programming,”
in Proc. 8th Annu. Conf. Subject Centre Inf. Comput. Sci., Southampton,
fication of problems in learning and teaching introductory U.K., 2007, pp. 82–85, doi: 10.13140/2.1.2425.6800.
programming, as well as in the evaluation of solutions. [21] Joint Task Force on Computing Curricula, Association for Computing
Machinery (ACM) and IEEE Computer Society, Computer Science
Curricula 2013: Curriculum Guidelines for Undergraduate Degree
ACKNOWLEDGMENT Programs in Computer Science. New York, NY, USA: ACM, 2013,
doi: 10.1145/2534860.
The authors would like to thank Dr. G. Cabral and
Dr. P. Tedesco from Universidade Federal de Pernambuco
and C.E.S.A.R (Centro de Estudos e Sistemas Avançados do Selected Papers
Recife) for the support and assistance in the preparation of [R01] I. Utting et al., “A fresh look at novice programmers’ performance
this paper. and their teachers’ expectations,” in Proc. ITiCSE-WGR, Canterbury,
U.K., 2013, pp. 15–32.
[R02] A. N. Kumar, “A mid-career review of teaching computer science I,”
R EFERENCES in Proc. SIGCSE 44th ACM Tech. Symp. Comput. Sci. Educ., Denver,
CO, USA, 2013, pp. 531–536.
[1] E. Lahtinen, K. Ala-Mutka, and H. M. Järvinen, “A study of the dif- [R03] U. Nikula, O. Gotel, and J. Kasurinen, “A motivation guided
ficulties of novice programmers,” ACM SIGCSE Bull., vol. 37, no. 3, holistic rehabilitation of the first programming course,” ACM
pp. 14–18, 2005. Trans. Comput. Educ., vol. 11, no. 4, 2011, Art. no. 24,
[2] M. Resnick et al., “Scratch: Programming for all,” Commun. ACM, doi: 10.1145/2048931.2048935.
vol. 52, no. 11, pp. 60–67, Nov. 2009. [R04] S. Haatainen, A.-J. Lakanen, V. Isomöttönen, and V. Lappalainen, “A
[3] S. Mohorovičić and V. Strčić, “An overview of computer program- practice for providing additional support in CS1,” in Proc. Learn.
ming teaching methods,” in Proc. Central Eur. Conf. Inf., 2011, Teach. Comput. Eng. (LaTiCE), 2013, pp. 178–183.
pp. 47–52. [R05] J. Sorva, V. Karavirta, and L. Malmi, “A review of generic pro-
[4] C. Watson and F. W. B. Li, “Failure rates in introductory programming gram visualization systems for introductory programming education,”
revisited,” in Proc. Conf. Innov. Technol. Comput. Sci. Educ., Uppsala, ACM Trans. Comput. Educ., vol. 13, no. 4, Nov. 2013, Art. no. 15,
Sweden, 2014, pp. 39–44. doi: 10.1145/2490822.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

12 IEEE TRANSACTIONS ON EDUCATION

[R06] A. Tafliovich, J. Campbell, and A. Petersen, “A student per- [R25] T. Sirkiä and J. Sorva, “Exploring programming misconceptions: An
spective on prior experience in CS1,” in Proc. SIGCSE 44th analysis of student mistakes in visual program simulation exercises,” in
ACM Tech. Symp. Comput. Sci. Educ., Denver, CO, USA, 2013, Proc. Koli Calling, 2012, pp. 19–28, doi: 10.1145/2401796.2401799.
pp. 239–244. [R26] A. Vihavainen, M. Paksula, and M. Luukkaine, “Extreme
[R07] A. J. Gomes and A. J. Mendes, “A study on student performance apprenticeship method in teaching programming for begin-
in first year CS courses,” in Proc. ITiCSE, Ankara, Turkey, 2010, ners,” in Proc. SIGCSE, Dallas, TX, USA, 2011, pp. 93–98,
pp. 113–117. doi: 10.1145/1953163.1953196.
[R08] C. Gavan and M. Anderson, “Engaging undergraduate programming [R27] L. Ma, J. Ferguson, M. Roper, and M. Wood, “Investigating and
students: Experiences using lego mindstorms NXT,” in Proc. 13th improving the models of programming concepts held by novice pro-
Annu. Conf. Inf. Technol. Educ. (SIGITE), Calgary, AB, Canada, 2012, grammers,” Comput. Sci. Educ., vol. 21, no. 1, pp. 57–80, 2011,
pp. 139–144. doi: 10.1080/08993408.2011.554722.
[R09] A. Santos, A. Gomes, and A. Mendes, “A taxonomy of exercises to [R28] A. P. Ambrósio, F. M. Costa, L. Almeida, A. Franco, and
support individual learning paths in initial programming learning,” J. Macedo, “Identifying cognitive abilities to improve CS1 outcome,”
in Proc. Front. Educ. Conf. (FIE), Oklahoma City, OK, USA, 2013, in Proc. Front. Educ. Conf. (FIE), Rapid City, SD, USA, 2011,
pp. 87–93, doi: 10.1109/FIE.2013.6684794. pp. F3G-1–F3G-7, doi: 10.1109/FIE.2011.6142824.
[R10] A. Gomes and A. Mendes, “A teacher’s view about introductory [R29] L. Porter and D. Zingaro, “Importance of early performance in CS1:
programming teaching and learning: Difficulties, strategies and moti- Two conflicting assessment stories,” in Proc. SIGCSE, Atlanta, GA,
vations,” in Proc. Front. Educ. Conf. (FIE), Madrid, Spain, 2014, USA, 2014, pp. 295–300, doi: 10.1145/2538862.2538912.
pp. 1–8, doi: 10.1109/FIE.2014.7044086. [R30] H. Aris, “Improving students performance in introductory
[R11] S. Alhazbi, “ARCS-based tactics to improve students’ moti- programming subject: A case study,” in Proc. Comput.
vation in computer programming course,” in Proc. Comput. Sci. Educ. (ICCSE), Cambridge, U.K., 2015, pp. 657–662,
Sci. Educ. (ICCSE), Cambridge, U.K., 2015, pp. 317–321, doi: 10.1109/ICCSE.2015.7250328.
doi: 10.1109/ICCSE.2015.7250263. [R31] M. Konecki, N. Kadoić, and R. Piltaver, “Intelligent assistant for
[R12] M. A. Brito and F. de Sá-Soares, “Assessment frequency helping students to learn programming,” in Proc. Inf. Commun.
in introductory computer programming disciplines,” ACM Technol. Electron. Microelectron. (MIPRO), Opatija, Croatia, 2015,
Comput. Human Behav., vol. 30, pp. 623–628, Jan. 2014, pp. 924–928, doi: 10.1109/MIPRO.2015.7160406.
doi: 10.1016/j.chb.2013.07.044. [R32] M. Hertz and S. M. Ford, “Investigating factors of student learning
[R13] A. P. Ambrosio, S. Martins, L. Almeida, A. Franco, and F. Georges, in introductory courses,” in Proc. SIGCSE, Denver, CO, USA, 2013,
“Assessment of self-regulated attitudes and behaviors of introductory pp. 195–200, doi: 10.1145/2445196.2445254.
programming students,” in Proc. Front. Educ. Conf. (FIE), Seattle, [R33] C. Mirolo, “Learning (through) recursion: A multidimensional analy-
WA, USA, 2012, pp. 1–6, doi: 10.1109/FIE.2012.6462314. sis of the competences achieved by CS1 students,” in Proc. ITiCSE,
[R14] N. Eltegani and L. Butgereit, “Attributes of students engagement in Ankara, Turkey, 2010, pp. 160–164, doi: 10.1145/1822090.1822136.
fundamental programming learning,” in Proc. Comput. Control Netw. [R34] M. Piteira and C. Costa, “Learning computer programming: Study
Electron. Embedded Syst. Eng. (ICCNEEE), Khartoum, Sudan, 2015, of difficulties in learning programming,” in Proc. ISDOC, Lisbon,
pp. 101–106, doi: 10.1109/ICCNEEE.2015.7381438. Portugal, 2013, pp. 75–80, doi: 10.1145/2503859.2503871.
[R15] M. Ateeq, H. Habib, A. Umer, and M. U. Rehman, “C++ or Python? [R35] C. Watson, F. W. B. Li, and R. W. H. Lau, “Learning program-
Which one to begin with: A learner’s perspective,” in Proc. Teach. ming languages through corrective feedback and concept visuali-
Learn. Comput. Eng. (LaTiCE), Kuching, Malaysia, 2014, pp. 64–69, sation,” in Advances in Web-Based Learning—ICWL 2011 (LNCS
doi: 10.1109/LaTiCE.2014.20. 7048), H. Leung, E. Popescu, Y. Cao, R. W. H. Lau, and
[R16] Simon et al., “Can computing academics assess the difficulty of W. Nejdl, Eds. Heidelberg, Germany: Springer, 2011, pp. 11–20,
programming examination questions?” in Proc. Koli Calling, 2012, doi: 10.1007/978-3-642-25813-8_2.
pp. 160–163, doi: 10.1145/2401796.2401822. [R36] D. McCall and M. Kölling, “Meaningful categorisation of novice pro-
[R17] D. Horton, M. Craig, J. Campbell, P. Gries, and D. Zingaro, grammer errors,” in Proc. Front. Educ. Conf. (FIE), Madrid, Spain,
“Comparing outcomes in inverted and traditional CS1,” 2014, pp. 1–8, doi: 10.1109/FIE.2014.7044420.
in Proc. ITiCSE, Uppsala, Sweden, 2014, pp. 261–266, [R37] R. Matthews, H. S. Hin, and K. A. Choo, “Novice programming
doi: 10.1145/2591708.2591752. students’ perception of learning object,” in Proc. Informat. Creative
[R18] T. Wang, X. Su, P. Ma, Y. Wang, and K. Wang, “Ability- Multimedia (ICICM), Kuala Lumpur, Malaysia, 2013, pp. 292–297,
training-oriented automated assessment in introductory program- doi: 10.1109/ICICM.2013.67.
ming course,” Comput. Educ., vol. 56, no. 1, pp. 220–226, [R38] M. Yousoof and M. Sapiyan, “Optimizing instruction for learning
2011. computer programming—A novel approach,” in Intelligence in the
[R19] J. Holvikivi, “Conditions for successful learning of programming Era of Big Data. ICSIIT 2015 (Communications in Computer and
skills,” in Key Competencies in the Knowledge Society (IFIP Advances Information Science), vol. 516. R. Intan, C. H. Chi, H. Palit, and
in Information and Communication Technology (AICT)), vol. 324, L. Santoso, Eds. Berlin, Germany: Springer, 2015, pp. 128–139,
N. Reynolds and M. Turcsányi-Szabó, Eds. Berlin, Germany: Springer, doi: 10.1007/978-3-662-46742-8_12.
2010, pp. 155–164, doi: 10.1007/978-3-642-15378-5_15. [R39] O. D. L. Tavares, C. S. de Menezes, and R. A. de Nevado,
[R20] P. Kinnunen and B. Simon, “CS majors’ self-efficacy per- “Pedagogical architectures to support the process of teaching and
ceptions in CS1: Results in light of social cognitive the- learning of computer programming,” in Proc. Front. Educ. Conf. (FIE),
ory,” in Proc. ICER, Providence, RI, USA, 2011, pp. 19–26, Seattle, WA, USA, 2012, pp. 1–6, doi: 10.1109/FIE.2012.6462427.
doi: 10.1145/2016911.2016917. [R40] A. Shargabi, S. A. Aljunid, M. Annamalai, S. M. Shuhidan, and
[R21] S. Xinogalos, “Designing and deploying programming A. M. Zin, “Program comprehension levels of abstraction for novices,”
courses: Strategies, tools, difficulties and pedagogy,” Educ. in Proc. Comput. Commun. Control Technol. (I4CT), Kuching,
Inf. Technol., vol. 21, no. 3, pp. 559–588, Jul. 2014, Malaysia, 2015, pp. 211–215, doi: 10.1109/I4CT.2015.7219568.
doi: 10.1007/s10639-014-9341-9. [R41] N. Thota, “Programming course design: Phenomenographic
[R22] O. Seppälä, P. Ihantola, E. Isohanni, J. Sorva, and A. Vihavainen, “Do approach to learning and teaching,” in Proc. Teach. Learn.
we know how difficult the rainfall problem is?” in Proc. Koli Calling, Comput. Eng. (LaTiCE), Kuching, Malaysia, 2014, pp. 125–132,
2015, pp. 87–96, doi: 10.1145/2828959.2828963. doi: 10.1109/LaTiCE.2014.30.
[R23] D. Horton and M. Craig, “Drop, fail, pass, continue: Persistence [R42] D. Teague and R. Lister, “Programming: Reading, writing and
in CS1 and beyond in traditional and inverted delivery,” in reversing,” in Proc. ITiCSE, Uppsala, Sweden, 2014, pp. 285–290,
Proc. SIGCSE, Kansas City, MO, USA, 2015, pp. 235–240, doi: 10.1145/2591708.2591712.
doi: 10.1145/2676723.2677273. [R43] M. Yamamoto, T. Sekiya, and K. Yamaguchi, “Relationship between
[R24] B. Hartanto, “Enhancing the student engagement in an introductory programming concepts underlying programming skills,” in Proc. Inf.
programming: A holistic approach in improving the student grade Technol. Based Higher Educ. Train. (ITHET), Izmir, Turkey, 2011,
in the informatics Department of the University of Surabaya,” in pp. 1–7, doi: 10.1109/ITHET.2011.6018678.
Intelligence in the Era of Big Data. ICSIIT 2015 (Communications [R44] M. Apiola, N. Moisseinen, and M. Tedre, “Results from an action
in Computer and Information Science), vol. 516, R. Intan, C. H. Chi, research approach for designing CS1 learning environments in
H. Palit, and L. Santoso, Eds. Berlin, Germany: Springer, 2015, Tanzania,” in Proc. Front. Educ. Conf. (FIE), Seattle, WA, USA, 2012,
pp. 493–504, doi: 10.1007/978-3-662-46742-8_45. pp. 1–6, doi: 10.1109/FIE.2012.6462321.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

MEDEIROS et al.: SYSTEMATIC LITERATURE REVIEW ON TEACHING AND LEARNING INTRODUCTORY PROGRAMMING IN HIGHER EDUCATION 13

[R45] A. Gomes and A. J. Mendes, “Studies and proposals about initial pro- [R67] A. Lishinski, A. Yadav, R. Enbody, and J. Good, “The influence of
gramming learning,” in Proc. Front. Educ. Conf. (FIE), Washington, problem solving abilities on students’ performance on different assess-
DC, USA, 2010, pp. 1–6, doi: 10.1109/FIE.2010.5673426. ment tasks in CS1,” in Proc. 47th ACM Tech. Symp. Comput. Sci.
[R46] J. Sheard, A. Carbone, D. Chinn, and M.-J. Laakso, “Study Educ., Memphis, TN, USA, 2016, pp. 329–334.
habits of CS 1 students: What do they say they do?” in [R68] A. Luxton-Reilly, “Learning to program is easy,” in Proc. ACM
Proc. Learn. Teach. Comput. Eng. (LaTiCE), 2013, pp. 122–129, Conf. Innov. Technol. Comput. Sci. Educ., Arequipa, Peru, 2016,
doi: 10.1109/LaTiCE.2013.46. pp. 284–289.
[R47] S. M. M. Rubiano, O. López-Cruz, and E. G. Soto, “Teaching [R69] A. Petersen, M. Craig, J. Campbell, and A. Tafliovich, “Revisiting
computer programming: Practices, difficulties and opportunities,” in why students drop CS1,” in Proc. 16th Koli Calling Int. Conf. Comput.
Proc. Front. Educ. Conf. (FIE), El Paso, TX, USA, 2015, pp. 1–9, Educ. Res., 2016, pp. 71–80.
doi: 10.1109/FIE.2015.7344184. [R70] D. F. Shell, L.-K. Soh, A. E. Flanigan, and M. S. Peteranetz, “Students’
[R48] T. Koulouri, S. Lauria, and R. D. Macredie, “Teaching introduc- initial course motivation and their achievement and retention in college
tory programming: A quantitative evaluation of different approaches,” CS1 courses,” in Proc. 47th ACM Tech. Symp. Comput. Sci. Educ.,
ACM Trans. Comput. Educ., vol. 14, no. 4, 2014, Art. no. 26, Memphis, TN, USA, 2016, pp. 639–644.
doi: 10.1145/2662412. [R71] A. Lishinski, A. Yadav, J. Good, and R. Enbody, “Learning to
[R49] L. M. M. Giraffa, M. C. Moraes, and L. Uden, “Teaching object- program: Gender differences and interactive effects of students’ moti-
oriented programming in first-year undergraduate courses supported vation, goals, and self-efficacy on performance,” in Proc. ACM
by virtual classrooms,” in Proc. 2nd Int. Workshop Learn. Technol. Conf. Int. Comput. Educ. Res., Melbourne, VIC, Australia, 2016,
Educ. Cloud, 2013, pp. 15–26, doi: 10.1007/978-94-007-7308-0_2. pp. 211–220.
[R50] V. Isomöttönen and V. Tirronen, “Teaching programming by empha- [R72] C. Ott, A. Robins, and K. Shephard, “Translating principles of effective
sizing self-direction: How did students react to the active role required feedback for students into the CS1 context,” ACM Trans. Comput.
of them?” ACM Trans. Comput. Educ., vol. 13, no. 2, Jun. 2013, Art. Educ., vol. 16, no. 1, Jan. 2016, Art. no. 1, doi: 10.1145/2737596.
no. 6, doi: 10.1145/2483710.2483711. [R73] G. Silva-Maceda, P. D. Arjona-Villicaña, and F. E. Castillo-Barrera,
[R51] D. A. Kranch, “Teaching the novice programmer: A study of instruc- “More time or better tools? A large-scale retrospective comparison of
tional sequences and perception,” Educ. Inf. Technol., vol. 17, no. 3, pedagogical approaches to teach programming,” IEEE Trans., vol. 59,
pp. 291–313, 2011, doi: 10.1007/s10639-011-9158-8. no. 4, pp. 274–281, Nov. 2016, doi: 10.1109/TE.2016.2535207.
[R52] Á. Matthíasdóttir and H. J. Geirsson, “The novice problem in computer [R74] M. P. Uysal, “Improving first computer programming experiences: The
science,” in Proc. CompSysTech, Vienna, Austria, 2011, pp. 570–576, case of adapting a Web-supported and well-structured problem-solving
doi: 10.1145/2023607.2023702. method to a traditional course,” Contemporary Educ. Technol., vol. 5,
[R53] M. Hertz and M. Jump, “Trace-based teaching in early programming no. 3, pp. 198–217, 2014.
courses,” in Proc. SIGCSE, Denver, CO, USA, 2013, pp. 561–566, [R75] R. A. Alturki, “Measuring and improving student performance in an
doi: 10.1145/2445196.2445364. introductory programming course,” Informat. Educ., vol. 15, no. 2,
[R54] C. J. Costa, M. Aparicio, and C. Cordeiro, “Web-based graphic envi- pp. 183–204, 2014, doi: 10.15388/infedu.2016.10.
ronment to support programming in the beginning learning process,” [R76] K. Hartness and L.-J. Shannon, “Peer mentors and their impact for
in Entertainment Computing—ICEC 2012 (LNCS 7522), M. Herrlich, beginning programmers,” Inf. Syst. Educ. J., vol. 9, no. 6, pp. 21–29,
R. Malaka, and M. Masuch, Eds. Heidelberg, Germany: Springer, 2011.
2012, pp. 413–416, doi: 10.1007/978-3-642-33542-6_41. [R77] A. Hoskey and P. S. M. Maurino, “Beyond introductory programming:
[R55] S. Fitzgerald et al., “Debugging from the student perspec- Success factors for advanced programming,” Inf. Syst. Educ. J., vol. 9,
tive,” IEEE Trans., vol. 53, no. 3, pp. 390–396, Aug. 2010, no. 5, pp. 61–70, 2011.
doi: 10.1109/TE.2009.2025266. [R78] B. Hosack, B. Lim, and W. P. Vogt, “Increasing student performance
[R56] M. Haungs, C. Clark, J. Clements, and D. Janzen, “Improving first- through the use of Web services in introductory programming class-
year success and retention through interest-based CS0 courses,” in rooms: Results from a series of quasi-experiments,” J. Inf. Syst. Educ.,
Proc. 43rd ACM Tech. Symp. Comput. Sci. Educ. (SIGCSE), Raleigh, vol. 24, no. 4, pp. 373–383, 2012.
NC, USA, 2012, pp. 589–594. [R79] K. Sung et al., “Game-themed programming assignment modules:
[R57] J. Kurhila and A. Vihavainen, “Management, structures and tools to A pathway for gradual integration of gaming context into existing
scale up personal advising in large programming courses,” in Proc. introductory programming courses,” IEEE Trans. Edu., vol. 54, no. 3,
Conf. Inf. Technol. Educ. (SIGITE), New York, NY, USA, 2011, pp. 416–427, Aug. 2011, doi: 10.1109/TE.2010.2064315.
pp. 3–8. [R80] M. Apiola and M. Tedre, “New perspectives on the pedagogy of
[R58] P. Lasserre and C. Szostak, “Effects of team-based learning on a CS1 programming in a developing country context,” Comput. Sci. Educ.,
course,” in Proc. 6th Annu. Joint Conf. Innov. Technol. Comput. Sci. vol. 22, no. 3, pp. 285–313, 2012.
Educ. (ITiCSE), Darmstadt, Germany, 2011, pp. 133–137. [R81] T. B. Bati, H. Gelderblom, and J. van Biljonc, “A blended learning
[R59] L. Porter and B. Simon, “Retaining nearly one-third more majors with approach for teaching computer programming: Design for large classes
a trio of instructional best practices in CS1,” in Proc. SIGCSE, Denver, in Sub-Saharan Africa,” Comput. Sci. Educ., vol. 24, no. 1, pp. 71–99,
CO, USA, 2013, pp. 165–170. 2014.
[R60] C. F. Reilly and N. De La Mora, “The impact of real-world topic labs [R82] B. A. Becker et al., “Effective compiler error message enhancement for
on student performance in CS1,” in Proc. Front. Educ. Conf., Seattle, novice programming students,” Comput. Sci. Educ., vol. 26, nos. 2–3,
WA, USA, 2012, pp. 1–6, doi: 10.1109/FIE.2012.6462329. pp. 148–175, 2016.
[R61] M. Rizvi and T. Humphries, “A scratch-based CS0 course for at-risk [R83] J. Bennedsen and M. E. Caspersen, “Persistence of elementary pro-
computer science majors,” in Proc. Front. Educ. Conf., Seattle, WA, gramming skills,” Comput. Sci. Educ., vol. 22, no. 2, pp. 81–107,
USA, 2012, pp. 1–5, doi: 10.1109/FIE.2012.6462491. 2012.
[R62] G. Rößling and M. Mühlhäuser, “An unusual CS 1 with high standards [R84] M. C. Hughes, M. C. Jadud, and M. M. T. Rodrigo, “String format-
and confirming results,” in Proc. 15th Annu. Conf. Innov. Technol. ting considered harmful for novice programmers,” Comput. Sci. Educ.,
Comput. Sci. Educ., Ankara, Turkey, 2010, pp. 169–173. vol. 20, no. 3, pp. 201–228, 2010.
[R63] S. C. Shaffer and M. B. Rosson, “Increasing student success by modi- [R85] C. Ott, A. Robins, P. Haden, and K. Shephard, “Illustrating per-
fying course delivery based on student submission data,” ACM Inroads, formance indicators and course characteristics to support students’
vol. 4, no. 4, pp. 81–86, 2013, doi: 10.1145/2537753.2537778. self-regulated learning in CS1,” Comput. Sci. Educ., vol. 25, no. 2,
[R64] B. Simon, P. Kinnunen, L. Poter, and D. Zazkis, “Experience report: pp. 174–198, 2015.
CS1 for majors with media computation,” in Proc. 15th Annu. [R86] A. Robins, “Learning edge momentum: A new account of out-
Conf. Innov. Technol. Comput. Sci. Educ., Ankara, Turkey, 2010, comes in CS1,” Comput. Sci. Educ., vol. 20, no. 1, pp. 37–71,
pp. 214–218. 2010.
[R65] R. Cacceffo, S. Wolfman, K. S. Booth, and R. Azevedo, “Developing [R87] S. Shuhidan, M. Hamilton, and D. D’Souza, “Instructor perspec-
a computer science concept inventory for introductory programming,” tives of multiple-choice questions in summative assessment for novice
in Proc. 47th ACM Tech. Symp. Comput. Sci. Educ., Memphis, TN, programmers,” Comput. Sci. Educ., vol. 20, no. 3, pp. 229–259,
USA, 2016, pp. 364–369. 2010.
[R66] J. Figueiredo, N. Gomes, and F. J. García-Peñalvo, “Ne-course for [R88] S. Willman et al., “On study habits on an introductory course on
learning programming,” in Proc. 4th Int. Conf. Technol. Ecosyst. programming,” Comput. Sci. Educ., vol. 25, no. 3, pp. 276–291, 2015,
Enhancing Multiculturality, Salamanca, Spain, 2016, pp. 549–553. doi: 10.1080/08993408.2015.1073829.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

14 IEEE TRANSACTIONS ON EDUCATION

[R89] L. Beck and A. Chizhik, “Cooperative learning instructional meth- [R99] A. Yadin, “Reducing the dropout rate in an introductory programming
ods for CS1: Design, implementation, and evaluation,” ACM Trans. course,” ACM Inroads, vol. 2, no. 4, pp. 71–76, 2011.
Comput. Educ., vol. 13, no. 3, pp. 1–21, 2013. [R100] B. Simon, J. Parris, and J. Spacco, “How we teach impacts student
[R90] A. Berglund and R. Lister, “Introductory programming and the didactic learning: Peer instruction vs. lecture in CS0,” in Proc. 44th ACM Tech.
triangle,” in Proc. 12th Aust. Comput. Educ. Conf. (ACE), Brisbane, Symp. Comput. Sci. Educ., Denver, CO, USA, 2013, pp. 41–46.
QLD, Australia, 2010, pp. 35–44.
[R91] R. Cardell-Oliver, “How can software metrics help novice program-
mers?” in Proc. 13th Aust. Comput. Educ. Conf. (ACE), Perth, WA,
Australia, 2011, pp. 55–62.
[R92] P. Denny, A. Luxton-Reilly, E. Tempero, and J. Hendrickx, Rodrigo Pessoa Medeiros received the B.S. degree in Internet systems from
“Understanding the syntax barrier for novices,” in Proc. 16th Annu. Faculdade Marista, Brazil, in 2005 and the M.S. degree in digital arts and
Joint Conf. Innov. Technol. Comput. Sci. Educ., Darmstadt, Germany, technology from Universidade do Minho, Portugal, in 2009. He is currently
2011, pp. 208–212. pursuing the Ph.D. degree in computer science with the Universidade Federal
[R93] N. Hawi, “Causal attributions of success and failure made by under- de Pernambuco, Brazil.
graduate students in an introductory-level computer programming
course,” Comput. Educ., vol. 54, no. 4, pp. 1127–1136, 2010.
[R94] L. J. Höök and A. Eckerdal, “On the bimodality in an introductory
programming course: An analysis of student performance factors,” in
Proc. Int. Conf. Learn. Teach. Comput. Eng., Taipei, Taiwan, 2015,
pp. 79–86. Geber Lisboa Ramalho received the Ph.D. degree in computer science from
[R95] R. Ibrahim, R. C. M. Yusoff, H. M. Omar, and A. Jaafar, “Students the University of Paris VI in 1997. He is an Electronic Engineer and an
perceptions of using educational games to learn introductory program- Associate Professor with the Centro de Informática, Universidade Federal de
ming,” Comput. Inf. Sci., vol. 4, no. 1, pp. 205–216, 2011. Pernambuco, Brazil, where he is currently the Head of the Systems Engineer
[R96] V. Isomöttönen and V. Lappalaine, “CS1 with games and an emphasis Department and the Chair of the Board of CESAR, a Brazilian innovation
on TDD and unit testing: Pilling a trend upon a trend,” ACM Inroads, institute.
vol. 3, no. 3, pp. 62–68, 2012.
[R97] D. F. Shell et al., “Improving learning of computational thinking using
computational creativity exercises in a college CSI computer science
course for engineers,” in Proc. Front. Educ. Conf. (FIE), Madrid,
Spain, 2014, pp. 1–7, doi: 10.1109/FIE.2014.7044489. Taciana Pontual Falcão received the bachelor’s degree in computer sci-
[R98] K. Wood, D. Parsons, J. Gasson, and P. Haden, “It’s never too early: ence and the Ph.D. degree from the University of London in 2014. She is
Pair programming in CS1,” in Proc. 15th Aust. Comput. Educ. Conf. an Assistant Professor with the Departamento de Computação, Universidade
(ACE), Adelaide, SA, Australia, 2013, pp. 13–21. Federal Rural de Pernambuco, Brazil.