Python Versus C++: An Analysis of Student Struggle on Small

Coding Exercises in Introductory Programming Courses

Nabeel Alzahrani1, Frank Vahid1,3, Alex Edgcomb1,3, Kevin Nguyen1 and Roman Lysecky2,3
1
Computer Science and Engineering, University of California, Riverside
2
Electrical and Computer Engineering, University of Arizona
3
zyBooks, Los Gatos, California
[email protected], [email protected], [email protected], [email protected], [email protected]

ABSTRACT avoiding some of the complexities of compiling and then running.

Another factor is that Python’s syntax is simpler, thus preventing
Many teachers of CS 1 (introductory programming) have students from getting bogged down in syntax errors, and instead
switched to Python rather than C, C++, or Java. One reason is the allowing students to focus on higher-level programming concepts.
belief that Python’s interpreted nature plus simpler syntax and A third factor is that Python comes with graphics and other
semantics ease a student’s learning, but data supporting that belief libraries that can make introductory programming courses more
is scarce. This paper addresses the question: Do Python learners engaging for students, who can analyze real data, create graphics-
struggle less than C++ learners? We analyzed student submissions based programs like video games, etc. Other reasons include
on small coding exercises in CS 1 courses at 20 different increasing use of Python in industry, and studies showing fewer
universities, 10 courses using Python, and 11 using C++. Each lines of code and/or increased productivity among experienced
course used either the Python or C++ version of an online programmers. Key hopes by those who switch is to decrease
textbook from one publisher, each book having 100+ small coding attrition in CS 1 courses and to attract more people to computing
exercises, expected to take 2-5 minutes each. We considered 11 degrees.
exercises whose Python and C++ versions were nearly identical
and that appeared in various chapters. We defined struggle rate for But, many teachers disagree, and continue to teach C, C++, or
exercises, where struggle means a student spent excessive time or Java in CS 1. Reasons include a belief that new learners should
attempts on an exercise. Based on that rate, we found the learning think precisely about details like data types, that learners should
for Python was not eased; in fact, Python students had not rely so heavily on library functions, that C/C++/Java (or
significantly higher struggle rates than C++ students (26% vs. variations) are widely used in industry especially in domains like
13%). Higher rates were seen even when considering only classes mobile apps, and that learning Python after C/C++/Java is easier
with no prerequisites, classes for majors only, or classes for non- than the other way (strict to less strict being easier than less strict
majors only. We encourage the community to do further analyses, to strict). Some teachers indicate frustration that students who
to help guide teachers when choosing a CS 1 language. learn Python in CS 1 have trouble in later courses in C/C++/Java,
not wanting to be bothered by details. Some engineers state new
ACM Reference format:
hires don’t understand or respect underlying resources, which can
Nabeel Alzahrani, Frank Vahid, Alex Edgcomb, Kevin Nguyen
and Roman Lysecky. 2018. Python Versus C++: An Analysis of Student lead to problems on commonly-constrained platforms.
Struggle on Small Coding Exercises in Introductory Programming Furthermore, cloud-based coding systems for C/C++/Java, where
Courses. In SIGCSE ’18: 49th ACM Technical Symposium on Computer students code in a web window and press “Run”, reduce the
Science Education, Feb. 21–24, 2018, Baltimore, MD, USA. ACM, NY, interpreted/scripting benefit of Python for beginning students.
NY, USA, 6 pages.DOI: 10.1145/3159450.3160586
In previous work, Enbody [1, 2] used Python and C++ for groups
1 INTRODUCTION
in CS 1. They compared the Python-group and C++-group on
Python is growing in popularity in introductory programming three outcomes: final exam grade, programming projects scores,
classes (CS 1). Various factors are stated for switching from and course final grade, and found no significant differences.
languages like C, C++, or Java. One is that Python is interpreted Using progression analysis, they also found that programming
(also known as a scripting language), allowing students to interact language features had no effect on students’ performance in CS 2.
immediately by typing print statements or simple calculations, and For object-oriented programming (OOP), Goldwasser [3] reported
that students in CS 1 were overwhelmed by the syntax and
Permission to make digital or hard copies of all or part of this work for personal or
semantics of C++ and Java, and found Python provided a simple
classroom use is granted without fee provided that copies are not made or and consistent model to teach OOP. Prechelt [4] found
distributed for profit or commercial advantage and that copies bear this notice and programmers using a scripting language like Python can solve the
the full citation on the first page. Copyrights for components of this work owned by
others than ACM must be honored. Abstracting with credit is permitted. To copy same problem with less code and higher productivity vs. a system
otherwise, or republish, to post on servers or to redistribute to lists, requires prior language such as C++. Zelle [5] found that scripting languages are
specific permission and/or a fee. Request permissions from [email protected].
SIGCSE '18, February 21–24, 2018, Baltimore, MD, USA.
simpler, safer, and more flexible vs. system languages like C and
© 2018 Association for Computing Machinery. C++. Guo [6] stated that Python is the most used CS 1 language in
ACM ISBN 978-1-4503-5103-4/18/02…$15.00 top-ranked U.S. research universities. Such related research
https://doi.org/10.1145/3159450.3160586
suggests how Python can be helpful to teach programming skills.
Figure 1: Each coding activity (CA) includes instructions, an
We sought however to determine from the students’ experiences
example, a coding area, and a Run button. For this activity, only
whether Python led to less student struggle.
lines 8 - 10 are editable.
This paper describes our analysis of student struggle rates on
identical small coding exercises used in Python and C++ classes
across 20 universities. Struggle means spending excessive time or
making excessive attempts on such exercises. Some teachers (like
ourselves) believe struggle is an important metric because struggle
can lead to frustration, and excessive/repeated frustration can lead
to students giving up. Struggle is one way (but not the only way)
of estimating the learning curve of a language. This paper
introduces the small coding exercises, defines a struggle metric,
provides data comparing struggle rates for Python and C++ in CS
1 courses showing that struggle rates for Python are not lower
(and are actually higher), summarizes a manual investigation into
the student submissions to better understand those rates, discusses .
possible reasons that Python’s struggle rates are not lower and are Figure 2: A wrong student solution to the Figure 1 CA.
actually higher, and provides conclusions. Differences between student and expected output are
highlighted.
2 SMALL CODING EXERCISES
zyBooks [8] is a publisher that creates online textbooks for
introductory C, C++, Java, and Python, with those textbooks
being very similar but adapted to each language. Over 100 small
coding exercises are embedded at the end of sections throughout
those books. The exercises are cloud-based, meaning students
code directly in a web window and press “Run” to execute their
code, so no substantial difference exists between C++’s
compile/execute approach and Python’s interpreted approach. We
abbreviate those exercises in this paper as CA (coding activity).
Figure 1 provides some examples.
Each exercise has a coding window with a partial program, and
the student is instructed to complete the program to carry out a
particular task, like printing numbers by dividing a variable of
value 40 until reaching 1 as shown in Figure 1. The instructions
usually include a first test case (sample input and output). The
student can only edit the relevant portion of the existing program.
When the student presses “Run”, the program is executed with
various test cases that test for proper program output, and the
student is shown which test cases passed and which failed CA’s are summarized in Table 1, numbered as CA 1, 2, …, 11
(showing the difference in output for failed cases) as shown in (which differs from the numbering in the textbooks).
Figure 2. Each exercise is worth 2 points: 1 point for passing any 3 STRUGGLE RATE AS A METRIC
test case, and 1 point for passing all test cases. Students can
attempt such exercises as many times as desired. Most universities To calculate the struggle rate, we first needed to find the number
give students some homework points for completing those coding of struggling students for a particular CA. We define struggling
exercises. students using two parameters: number of attempts (number of
submissions), and time spent trying to solve the CA. Students may
The CA in Figure 1, titled “Basic while loop expression”, asks the submit multiple submissions before achieving the correct solution
students to print userNum divided by 2 until reaching 1 (given to a CA. We define a struggling student for a particular CA as a
userNum = 40). The shown code is the template. Students change student who has spent more than 5 minutes and spent more than
the comment “/* Your solution goes here */” with their code. double the Baseline time and attempted more than 3 times and
Our institution has taught both Python and C++ in our CS 1, so attempted more than double the Baseline attempts, or spent more
we wished to compare those languages. We noticed 11 of the than 15 minutes. The Baseline time is the time spent by the top
CA’s were nearly identical in the Python and C++ versions. Those 20% students in that class for that CA and the Baseline attempts is
the number of attempts by the top 20% students in that class for
 Table 1: List of the 11 CA’s with their titles and chapters. Table 2: A snapshot of the student 0xxx submissions for CA2:
Basic while loop expression.
CA CA title Ch # Chapter Title
# Time of User Answer Submitted solution
submission # correct
1 Tree Height 2 Vars / Assgnmt 3/10/2017 0xxx No while (userNum != 1){
1:20:44 PM cout << userNum << " ";
2 Basic while loop expression 4 Loops userNum = userNum / 2;
}
3 Simon says 4 Loops
... 0xxx No ...
4 Vector iteration: Sum excess 5 Arrays / Vectors 3/10/2017 0xxx No while (userNum !=0){
1:24:51 PM while (userNum != 1){
5 Function call in expression 6 User-Def Fcts cout << userNum << " ";
userNum = userNum / 2;
6 Function errors: Copying one 6 User-Def Fcts }
function to create another }
cout << userNum << " ";
7 Function with loop: Shampoo. 6 User-Def Fcts ... 0xxx No ...

8 Constructor overloading 7 Objs & Classes 3/10/2017 0xxx Yes while (userNum >= 1){
1:30:18 PM cout << userNum << " ";
9 Basic inheritance 10 Inheritance userNum = userNum / 2;
}
10 Derived class membr override 10 Inheritance
number of attempts to solve that CA, called “Baseline attempts”;
11 Recursive function: Writing 12 Recursion (3) if the student total time spent is greater than 15 minutes, then
the base case the student is a struggling student; or if the student total time spent
is greater than 5 minutes and the student total time spent is greater
than double the Baseline time and the student total number of
attempts is greater than 3 and the student total number of attempts
that CA. We defined a dynamic struggle rate (by referring to top
is greater than double the Baseline attempts, then the student is a
20% students) rather than a static struggle rate to account for the
struggling student. To calculate the struggle rate for a CA, we
class (students) background level in programming. A CA’s
divide the total number of struggling students for a CA by the
struggle rate is defined as the # of struggling students divided by
total number of students for that CA.
the # of students in that class. The following formulas summarize
a struggling student and the struggle rate. 4 RESULTS
Struggling student = ((time > 5 min.) AND (time We obtained anonymized student submission data for the 11
above-mentioned nearly-identical CAs, for C++ and Python
> 2 * Baseline time) AND (# attempts > 3) AND courses at dozens of universities. We chose 11 C++ courses and
(# attempts > 2 * Baseline attempts)) OR (time > 10 Python courses at 20 universities to represent a variety of
15 min.) institutions, including 4-year research institutions (none were
schools typically ranked in the top 20), non-research 4-year
Struggle rate = # struggling students / # students institutions, and 2-year institutions (community colleges). To
Changing the parameters increases or decreases the struggle rate; obtain roughly equal samples from both languages, we generally
we based these numbers on teaching experiences. Other struggle sought to match each C++ course with a Python course from an
rate metrics are possible. For our purposes, the raw % is less institution of the same type and roughly the same numbers of
important than is the comparison of rates for different languages. students. Table 3 shows the number of students in the 20 courses
As shown in Table 2, students can make multiple submissions for per language (C++ and Python). Obviously such matching can’t
each CA. Each submission consists of a timestamp, user id #, be perfect, but by attempting such matching, coupled with the
correctness, and the submitted code. To identify a struggling large numbers of students, we can have more confidence that the
student for a CA, we do the following: (1) get the student total two sample populations’ statistics can be meaningfully compared.
time spent and the student total number of attempts to solve that
Table 4 shows the struggle rates for the 11 coding exercises,
CA; (2) calculate the top 20% student average time to solve that
summarized for all 11 C++ courses and all 10 Python courses. For
CA, called “Baseline time”, and the top 20% student average
example, the first data row is for CA 1 (Coding Activity 1). 787
 Table 3: Each row is two similar schools using different Table 4: A comparison of struggle rates on 11 nearly-
languages. A Python-match for row 11 does not exist, but identical coding exercises for 11 C++ and 10 Python courses.
we kept the C++ offering to have data for no-prerequisite
and non-majors for C++ CA’s.

School Total # Total # students in

students Python
in C++

1 (Research universities) 153 105

2 (Community colleges) 13 33

3 (Teaching universities) 34 23 Table 5: Struggle rates considering only courses having no

computing-related prerequisites, meaning 4 C++ courses and
7 Python courses.
4 (Research universities) 277 176

5 (Same community college) 21 29

6 (Teaching universities) 48 35

7 (Research universities) 121 92

8 (Community colleges) 14 165

9 (Community colleges) 15 17 not have access to information about individual students, so we

examined the course descriptions. We considered that perhaps the
C++ courses were a second course, following a simpler
10 (Research universities) 167 195 programming course or some other computing-related course.
Thus, we excluded any courses that had a computing-related
11 (Teaching university) 194 N/A prerequisite, reducing the set to 4 C++ courses and 7 Python
courses. Table 5 shows results. The difference in struggle rates
continued: 14% for C++, 28% for Python.
Total number of students 1057 870
We considered that perhaps the C++ courses were taken by
majors and Python by non-majors. We thus divided the courses
into those intended for majors (per their course descriptions), and
C++ students across the 11 C++ courses attempted that CA, while those for non-majors. The effect was still seen: 10% vs. 26%
434 Python students did. Among those students, 5% of the C++ struggle for majors (2242 C++ and 383 Python students), and 18%
students struggled (39 out of 787), while 9% of the Python vs. 26% for non-majors (1783 C++ and 1834 Python students).
students struggled (41 out of 434). Such data is shown for each of
the 11 CA’s. On average, the C++ struggle rate was 13%, while Because students are anonymized and we have no data on the
the Python struggle rate 26%. The average was computed by students themselves, the above analyses should not be considered
dividing the total number of students for all the 11 CA’s by the as perfectly representative of the student populations. For
total number of struggling students for all the 11 CA’s. We did the example, the courses intended for non-majors may very well have
struggle rate analysis for one community college with similar had some majors. However, the analysis was intended merely to
numbers of students in both C++ (21) and Python (29), and we determine if the hypothesis is correct that Python students have a
found nearly identical results with Python students struggling substantially easier learning curve than C++ students. For that
more than C++ students. purpose, the data seems to suggest that belief is false (and in fact
the opposite may be true)
The data surprised us, so we further sought to see if perhaps the
effect was due to differences in the student populations. We did 5 ANALYSIS
 Table 6: % likelihood that any given student would consuming, we examined CA’s 1, 2, 7, and 10, seeking to spread
struggle more with Python than with C++ out the CA’s examined.
Table 7 illustrates our findings. For example, for CA 1, which
Table Students population p-value Struggle involved converting a math equation into an assignment
% statement, Python students struggled more with basic assignment
concepts, such as missing an = operator, confusing left and right
Table 4 All students. 11 C++ and 10 < 0.0001 12% sides, or assigning to an expression rather than a variable. For CA
Python courses and each 2, which involved writing a while loop to output a number halved
course's submissions has 11
until reaching 0, Python students struggled more on all aspects of
CA's.
the problem, including writing the loop condition, updating the
Table 5 No prerequisite. with 4 C++ < 0.0001 12% loop counter variable, or placing the output in the correct location.
and 7 Python courses.
7 DISCUSSION
Not No prerequisite and CS < 0.0001 18% Limitations: Other struggle metrics exist, such as measuring
shown majors. 2 C++ and 4 Python struggle on weekly programming assignments, surveying students,
courses. measuring performance on exams, etc.

Not No prerequisite and non CS < 0.0001 6% Our analysis involved 1,927 students at courses across 20
shown majors. 2 C++ and 3 Python universities. While those large numbers and the diversity of
courses. populations are strengths of the data and likely minimizes the
impact of one particular course’s policies or instructor’s teaching
style, also useful would be a controlled study at one university
(which is hard to carry out, since such random assignment is
rarely acceptable), or where the university switched from one
Tables 4 and 5 show Python students struggle more than C++
language to another across semesters (but other factors like
students. We want to account for the relative number of students
teacher and student population may confound results).
per CA because the struggle % is not consistent per CA. For
example, in Table 5, CA 8 had 247 C++ students but only 32 The Python/C++ textbooks use a standard approach. Other
Python students. We thus converted the difference of the average approaches, such as a media-based approach or objects-first
C++ and Python Z-scores to a % as follows. For each table, we: ordering, may yield different results.
1. Calculated the Z-score per CA: We used the mean and The perceived easier learning curve is just one reason some
standard deviation of C++ and Python struggle combined per teachers have switched to Python. Other reasons exist, such as
CA. built-in libraries. Thus, the above data relates to just one factor
2. Calculated the p-value for the whole table: We used a among many that influence a CS 1 language decision.
Student's t-test to compare the C++ Z-scored struggle to
Possible reasons: This study analyzed struggle rate, not the
Python Z-scored struggle
reasons. One possible reason for Python’s struggle rate not being
3. Calculated the percentage of the average difference: We
averaged the C++ Z-scores, and separately averaged the lower than for C++ is that learning core programming concepts
may overshadow syntax issues. The manual investigation of
Python Z-scores, then used a Z-score to percentile calculator
[7] to convert the difference in average to a percentile. student submissions seemed to support this reason; few students
struggled with syntax in either language. Instead, struggle was due
The final step gives the % likelihood that a given student would to programming concepts like creating a proper loop to solve a
struggle more with Python than C++. Table 6 shows that a student task. The case may be that college students can master the basic
is 12% more likely to struggle with Python than C++. syntax of C++ nearly as quickly as they master the slightly-easier
syntax of Python. Also, C++ teachers can choose whether or not
6 MANUAL INVESTIGATION to dwell on C++ syntax. The textbook in this study avoids
The analysis above suggests that the Python learning curve, based potentially-problematic aspects of C++, such as branches/loops
on the metric of struggle rate on small coding exercises, is not without braces (the book always uses braces), assignments in
easier than the C++ learning curve. In fact, the analysis suggests branch/loop expressions (the book avoids those), use of
(perhaps surprisingly) that the learning curve is actually harder. prefix/postfix increment operators (the book avoids except in a
To better understand, we manually examined student submissions for-loop header), etc., instead teaching a common and safer subset
to many of the CA’s. Because manual examination is very time of C++.
Python’s struggle rate was surprisingly higher. One possible students to think more precisely about language-independent
reason relates to programming requiring precision. From the problem-solving as well, like writing loop expressions that iterate
beginning, C++ requires precise thought about variable exactly as desired. Python’s forgivingness might breed a more
declarations, variable types, data types resulting from expressions, cavalier attitude that extends beyond syntax/semantics into
use of braces, use of = vs. ==, etc. This precision may prime problem solving as well. This of course is just conjecture; future
work may seek to test the idea.
Table 7: CA1, 2, 7, and 10 for all data and the reasons why
students struggled, as determined by manual We note that cloud-based programming is reducing the difference
investigation. between languages, eliminating (or postponing) the need to install
or even use an IDE.
CA C++ Python
8 CONCLUSIONS
Reasons Reasons One factor leading teachers to use Python in CS 1 courses is the
belief that Python has an easier learning curve. We analyzed
1 1-Using / instead of * 1-Using tan() instead of struggle rates for 11 nearly-identical short coding exercises in 11
2-Missing tan() for the math.tan() C++ and 10 Python courses, involving about 1,000 students in
angleElevation 2-Using / instead of * each language at 20 universities. We found the Python struggle
variable 3-Missing tan() for the
rate was not lower than C++. One possible reason is that the
3-Mistyping variable angleElevation variable
languages’ syntax differences are eclipsed by the difficulty of
names 3-Mistyping variable names
learning language-independent programming concepts, especially
4-Wrong assignment (using two
if C++ teachers don’t dwell on C++’s complex syntax options.
= symbols, assign to the wrong
variable, reverse assignment, In fact, our analysis showed Python’s struggle rate to be
etc.) significantly higher than C++. One possible reason is that C++’s
focus on precision translates to a more precise approach to
2 1-Wrong loop 1-Wrong loop condition programming. As for attrition, at our institution, we have found
condition 2-Wrong/missing loop counter
that a caring talented instructor with good class design, policies,
2-Wrong/missing loop update
and assignments -- appropriate homework/assignment points ratio,
counter update 3-Missing/wrong location
various help resources, encouragement of collaboration, flipped
3-Missing/wrong output statement
location output stmt 4-Indentation (few: just 5
lectures, interesting/relevant assignments -- seem far more
students) important than the language choice. In fact, in our most recent
offering of CS 1 in C++, students provided evaluations in the
7 1-Missing for-loop 1-Missing for-loop 95’th percentile for all courses in the university of 30,000
2-Missing counter 2-Wrong while-loop update students, while performing strongly on programming assignments
inside the for-loop (when using while-loop) and exams.
3-Wrong for-loop 3-Wrong for-loop condition
In any case, the analysis might help CS 1 teachers predict whether
counter initial value 4-Wrong print() argument
switching from C++ (or C or Java) to Python might yield the
4-Wrong for-loop inside the for-loop
condition 5-Missing for-loop condition
desired benefit of an easier learning curve. We encourage the
5-Wrong for-loop variable inside the for-loop community to perform more such analyses, so that teachers can be
location 6-Wrong for-loop location guided by data in making language decisions for CS 1 courses.
6-Wrong cout() arg
inside for-loop REFERENCES
[1] Richard J. Enbody, William F. Punch, and Mark McCullen. 2009. Python CS1 as
preparation for C++ CS2. ACM SIGCSE Bulletin 41, no. 1 (2009): 116-120.
10 1-Missing ; 1-Missing function definition [2] Richard J. Enbody, and William F. Punch. 2010. Performance of python CS1
2-Wrong cout() 2-Missing call to member students in mid-level non-python CS courses. In Proceedings of the 41st ACM
argument function technical symposium on Computer science education, pp. 520-523. ACM, 2010.
[3] Michael H. Goldwasser, and David Letscher. 2008. Teaching an object-oriented
3-Not complete cout() 3-Missing argument to call a CS1-: with Python. ACM SIGCSE bulletin, vol. 40, no. 3, pp. 42-46. ACM,
arguments member function 2008.
4-Wrong location to 4-Extra space when calling [4] Lutz Prechelt. 2003. Are scripting languages any good? A validation of Perl,
Python, Rexx, and Tcl against C, C++, and Java. Advances in Computers 57
call a member function print() (2003): 205-270.
5-Missing to call a 5-Wrong call to a member [5] John M. Zelle. 1999. Python as a first language. In Proceedings of 13th Annual
member function function Midwest Computer Conference, vol. 2, p. 145. 1999.
[6] Philip Guo. 2014. Python is now the most popular introductory teaching
6-Wrong format to call 6-Wrong print() statement language at top us universities. BLOG@ CACM, July (2014): 47.
a member function 7-Wrong function argument [7] Measuring U. Z-Score to Percentile Calculator. https://measuringu.com/pcalcz/,
7-Missing function def accessed Aug, 2017.
[8] zyBooks. https://www.zybooks.com/, accessed Aug, 2017.