See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/221538928

Open source projects in programming courses

Conference Paper in ACM SIGCSE Bulletin · March 2007

DOI: 10.1145/1227504.1227465 · Source: DBLP

CITATIONS READS
36 1,980

4 authors, including:

Manuel Oriol Andreas Pedroni

ABB University of Zurich
105 PUBLICATIONS 1,713 CITATIONS 37 PUBLICATIONS 1,574 CITATIONS

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Andreas Pedroni on 02 June 2014.

The user has requested enhancement of the downloaded file.

 Open Source Projects in Programming Courses

Michela Pedroni, Till Bay, Manuel Oriol Andreas Pedroni

Chair of Software Engineering Institute of Neuropsychology
ETH Zurich University of Zurich
8092 Zurich, Switzerland 8006 Zurich, Switzerland
{michela.pedroni,till.bay,manuel.oriol} [email protected]
@inf.ethz.ch

ABSTRACT directly into the classroom [4, 7]. In particular, open source
One of the main shortcomings of programming courses is software can be used to emphasize the importance of high
the lack of practice with real-world systems. As a result, quality software design, the role of design patterns, the need
students feel unprepared for industry jobs. In parallel, open of good documentation, and the relevance of social skills in
source software is accepting contributions even from inexpe- a real-world environment.
rienced programmers and achieves software that competes Over the last years, only a few instructors reported on
both in quality and functionality with industrial systems. their experimentation with open source software in the class-
This article describes: first, a setting in which students were room. Allen et. al. [1] used Dr. Java - an open source Java
required to contribute to existing open source software; sec- programming environment - to teach extreme programming
ond, the evaluation of this experience using a motivation techniques. Students of Fuhrman [3] freely chose an open
measuring technique; and third, an analysis of the efficiency source project which they inspected to propose corrections
and commitment of students over the time. The study shows in the design. Fuhrman did not require students to imple-
that students are at first afraid of failing the assignment, but ment their improvements, but many still did. Carrington [2]
end up having the impression of a greater achievement. It allowed students to choose from a list of open source projects
seems also that students are inclined to keep working on that are useful to software engineering and let them inspect,
the project to which they contributed after the end of the report, and extend the tools during his course. Except for
course. contributions by students to Dr. Java, these experiments
refrained from actively submitting changes, bug fixes, or im-
proved designs to the open source projects.
Categories and Subject Descriptors The assignment described here combines the freedom of
K.3.2 [Computer and Information Science Education]: choosing an arbitrary open source project with the ultimate
Computer science education goal of students contributing to it. Allowing students to
freely choose the project, on which they intend to work,
lets them adapt the assignment to their personal interests.
General Terms Requiring students to contribute back to the open source
Management, Experimentation, Human Factors project, increases the prestige of their work and the effort
that they are willing to put in the assignment. Students
Keywords experience all the tasks needed to adapt existing software:
understanding it, identifying a needed improvement, design-
Open source, Project, Assessment, Motivation, Communi- ing the solution, implementing, testing, and adapting to the
ties requirements for contribution. During this work, they need
to get socially involved with the other developers of the open
1. INTRODUCTION source project, having to deal with the communication prob-
We teach software engineering techniques, design patterns, lems that occur frequently with distributed development.
and project development models in our courses. Teaching This paper presents a thorough evaluation of the approach
these topics to students without actually exposing them to by using a questionnaire focusing on students’ motivation [6].
industry grade software may not only sound preachy, but The study compares the motivation of students working
may also leave no remaining impression. Open source soft- on an open source project to the motivation of a control
ware offers great opportunities to bring real-life experience group from another course working on a traditional (exer-
cise) project.
Section 2 reports on the course setup and describes the
project requirements imposed on the students. Section 3
Permission to make digital or hard copies of all or part of this work for provides a thorough evaluation based on a questionnaire
personal or classroom use is granted without fee provided that copies are from motivational psychology and shows the weaknesses and
not made or distributed for profit or commercial advantage and that copies strengths of using open source software. Section 4 extends
bear this notice and the full citation on the first page. To copy otherwise, to the motivational evaluation with results on students’ effi-
republish, to post on servers or to redistribute to lists, requires prior specific
permission and/or a fee.
ciency and commitment. After presenting the interpretation
SIGCSE’07, March 7–10, 2007, Covington, Kentucky, USA. of the results the article concludes in section 5.
Copyright 2007 ACM 1-59593-361-1/07/0003 ...$5.00.
2. COURSE SETUP
Table 1: Open source projects
This study focuses on two programming courses: an ad- Anteater: http://aft.sourceforge.net
vanced Java course (the experimental course, roughly 70 ArgoUML: http://argouml.tigris.org

Projects Suggested
students, master-level) and a course on concurrent object- JEiffel: http://se.ethz.ch/projects/benno baumgartner
oriented programming (the control course, roughly 30 stu- JSR’s: http://www.jcp.org
dents, master-level). Both courses are elective courses that Open Office: http://www.openoffice.org
Tomcat: http://tomcat.apache.org
can be chosen both by Bachelor and Master students, but XmlIO: http://www.bifrost.org/xmlio/index.shtml
also by post diploma students (e.g. PhD students). For the Azureus: http://azureus.sourceforge.net
project the experimental course required groups of students Eclipse: http://www.eclipse.org
(maximum 5 people) to contribute to a Java open source GPSylon: http://www.tegmento.org/gpsylon
project of their choice. The control course relied on a given JUnit: http://www.junit.org/index.htm

Projects Chosen
artificial project to be solved by groups of maximum 3 peo- Maven: http://maven.apache.org
CaCMS: http://cacms.sf.net
ple. The goal of the projects was in both cases to deepen

Other Projects
Columba: http://www.columbamail.org
the understanding of the lectures by putting the concepts FreeGuide TV: http://freeguide-tv.sourceforge.net
to work. Both projects spanned over 6 weeks and started Gham: http://www.hattrickitalia.org/gham
at the middle of the semester. For both courses the results Hunt for Gold: http://huntforgold.sourceforge.net
of the project influenced significantly the final grade of the JackSum: http://www.jonelo.de/java/jacksum
course (respectively 40% and 65% of the final grade). Jython: http://sourceforge.net/projects/jython
Tapestry: http://tapestry.apache.org
The following subsections describe each of the project as- WTflash: http://sourceforge.net/projects/wtflash
signments in more details.

2.1 Control course project

In line with the focus of the course the project required The assignment was to: (1) get an overview of the project,
the students to build a group of command line programs, (2) identify the parts to which they would contribute code,
that use a given concurrency framework and demonstrated (3) contribute, (4) write a report recalling their experience.
its capabilities. The second part of the project consisted in
extending the framework with a rendezvous synchronization 2.3 Outcomes
mechanism that did not exist previously. The students generally spent a lot of time and produced
The exact assignment was to: (1) program the required quality software. Their contributions include the world map
tasks, (2) answer questions, and (3) write a report includ- plug-in integrated in the latest versions of Azureus, vari-
ing a description of the code as well as the answers to the ous bugfixes for the Eclipse Maven plug-in, CSV-exporting
questions. Tapestry components, bugfixes and extension of the GPL
Hattrick Manager, bugfixes of JUnit, bugfixes and major
2.2 Open source project improvements of the Columba Mail Client, extensions of
At the beginning of the course, students were informed GPSylon, bugfixes of the FreeGuide TV, a GUI for Jack-
about the grading scheme and that they would have to form Sum, extensions of WTFlash, extensions to the game “Hunt
groups for the project. Students received the project de- for Gold”, and finally extensions to CaCMS to include Web-
scription only at the middle of the semester. At this oc- Dav support. These are only the projects producing code
casion, lecturers and students discussed the nature of the that is shipping in the products. The course’s wiki page1
community work that open source projects involve. The provides much more details.
discussion showed that very few of the seventy students had
actively participated or even contributed to an open source
project. Obviously, all of them had already used open source 3. EVALUATION OF MOTIVATION
software.
The project description included an initial selection of 3.1 Background
popular and active open source projects (see Table 1). The Students who are motivated by a task are more likely to
main criteria for selecting these projects were activity of the succeed in solving it. According to Pintrich, students’ moti-
community, existence of a project plan, existence of a bug vation is strongly correlated with their academic success [5].
tracking system, and the availability of the code through a In particular, Pintrich shows that students are more moti-
repository so that students are able to create branches on vated if:
which they can work.
Each description of the proposed projects included ref- 1. they believe that they are able to solve the task at
erences to the relevant web pages, wikis, mailing lists, and hand.
URLs of the bug tracking systems. It also included an infor-
mal evaluation of the projects’ organization such as specifics 2. they feel in control of their learning.
about the development process (e.g. was it bug-driven or
planned). This information provided a starting point when 3. they are personally or situationally interested in the
looking for possible contributions to the project. task.2
Besides this selection of projects students could also look 1
http://wiki.se.inf.ethz.ch/tjp 06/index.php/Project page
for other Java open source projects and assess their suitabil- 2
Personal interest describes a disposition of an individual
ity for contributions. As Table 1 shows, half of the students to be attracted to a particular activity or topic while situ-
chose one of the proposed projects and the other half se- ational interest describes a state of an individual where the
lected one of their own. interest results from the task itself.
 project. The control course had a similar setting. In the
person
(motives) rest of the section we use the following abbreviations:
QCMEt1 : questionnaires of the experimental group at the
beginning of the open source project

current
QCMEt2 : questionnaires of the experimental group at the
behavior
motivation end of the open source project
QCMCt1 : questionnaires of the control group at the begin-
ning of the project
situation
(potential
QCMCt2 : questionnaires of the control group at the end of
stimuli) the project
The responses QCMEt1 , QCMEt2 , QCMCt1 , and QCMCt2
Figure 1: Basic model of classical motivational psy- can be used to compare the two groups in two dimensions:
chology [6]
• Motivational differences between the experimental and
the control group (comparing QCMEt1 to QCMCt1 and
4. they believe what they are doing is valuable and useful QCMEt2 to QCMCt2 ).
for themselves. • Motivational changes over time for both of the groups
(comparing QCMEt1 to QCMEt2 and QCMCt1 to QCMCt2 ).
5. they have social and/or academic goals that they pur-
sue. Differences at the beginning. The 18 items of the
QCM were combined into measures for each of the four di-
By comparing the above motivational influences for the open
mensions. Based on this data, the analysis uses T-Tests4
source project approach and the traditional project approach,
for independent sets to obtain the factors that differ signifi-
one can expect the open source project to increase personal
cantly between the two groups for each of the two points in
and situational interest (3) and to give them the impression
time (means are gathered in Figure 2).
that the task is more valuable and useful (4) than the tra-
The comparison shows that QCMEt1 and QCMCt1 differ
ditional projects’. One can also expect that the traditional
significantly for the factors probability of success and anxiety
project approach is better at providing assurance of success
(see (∗a) respectively (∗b) in Figure 2). The interpretation
(1) and impression of control (2).
of such a result is that when students begin to work on
To assess the validity of using an open source project
an assignment requiring contributions to a real-world open
against using a traditional project, a good indicator is to
source project, they feel more uncertain about their suc-
compare the motivation associated to both projects. The
cess and therefore their fear of failure is higher than for a
Questionnaire on Current Motivation (QCM) [6] assesses
traditional small “toy” project. More surprisingly, students
the current motivation of students working on a specific task
are at this point in time not significantly more interested
and therefore is well suited for such a purpose. The QCM is
or more challenged by the open source project than by the
based on the ”classical” model of motivational psychology
traditional project.
[6] (see Figure 1) that states that personal and situational
Differences at the end. The comparison of QCMEt2
factors influence the current motivation which in turn influ-
and QCMCt2 shows that the factors probability of success
ences behavior i.e. learning. It also benefits from a whole
and anxiety do not differ significantly between the experi-
theoretical psychology framework that allows to easily com-
mental and the control group any more. Interestingly enough,
pare and aggregate questions.
the level of confidence is identical in both groups. The fac-
The QCM itself uses 18 items (questions) that measure
tor interest changed (with p = 0.065): the interest of the
four factors of current motivation: anxiety (fear of failure),
experimental group in their project grew while the interest
probability of success, interest, and challenge. The full ques-
of the control group diminished. The interpretation is that
tionnaire is available online3 and reproduced in the Ap-
the fascination of working on an open source project settles
pendix. The QCM items formulate statements for which
in only after the first hurdle of basic understanding and in-
students assign 1 to 7 points depending on how much they
volvement. Working on a traditional project is interesting
agree with the statements (1: totally disagree, 7: totally
in the initial design phase but looses fascination over time.
agree). As an example:
Another interesting outcome can be detected by compar-
ing individual statements. First, the statement “If I succeed
I enjoy problem solving tasks like the ones that emerge in
with the project, I will feel a little proud of my proficiency”
the project work.  1  2  3  4  5  6  7
was significantly higher at the end of the projects for the
experimental group than for the control group. Second, the
3.2 Results statement “I would also work on a project like that in my
free time” also produced a significantly higher result for the
To assess the motivational implications of the open source experimental group. This is consistent with the following
project assignment, the students of the Java programming intuitive interpretation. On one hand, contributing to a
course filled in the QCM twice: once just after receiving the
4
project description, and the second time at the end of the T-Tests allow to determine if the means of data differ sig-
nificantly. In general, this is assumed to be the case if the
3
http://se.ethz.ch/people/pedroni/qcm.html calculated value p < 0.05.
real-life project is very likely to make students proud and 4. COMPLEMENTARY ITEMS
may even make them wish to continue contributing after The previous section showed how an open source project
the mandatory work is completed. On the other hand, hav- impacts on students’ motivation. To estimate students’ ac-
ing completed a project specifically designed for a course is tivity, learning effect, and commitment additional items com-
not that rewarding. plement the QCM. These questionnaires were distributed to
the students of the experimental course at three occasions
during the course period - at the beginning, in the middle,
*a d and at the end.
7.0 * From over thirty additional items covering questions about
*c *e the course, the weekly assignments, and the project, three
6.0
*b are detailed here. These three items all concern the project
and represent a general trend emerging from these additional
5.0 questions on the project5 .
The first item assesses activity. This item addresses how
4.0 much time is spent on the open source project. The second
item - learning effect - quantifies the students’ perception on
3.0 the knowledge learned because of the open source project.
experimental group
control group
The third item is the commitment that results from the
2.0
project work. This third item was evaluated only at the
middle and at the end of the course. The results of these
Interest (t1)

Interest (t2)

Challenge (t1)

Challenge (t2)

Anxiety (t1)

Anxiety (t2)

Success (t1)

Success (t2)
complementary three items help to identify improvements
for a future course.

4.1 Results
The item addressing the activity was: “I am putting much
Figure 2: Mean and standard deviation of the four effort into the project”. Students assigned 1 to 5 points cap-
factors with a range of possible values from 1.0 to turing how much they agree with this statement (1: totally
7.0. (*) denotes significant differences (p < 0.05). disagree, 5: totally agree). Figure 3 shows that the mean
effort invested in the open source project drops towards the
Comparison over time for both groups. The second middle of the project’s lifespan.
part of the evaluation was done using T-Tests for paired sets
of data to obtain significant changes over time. These tests 4.6
show that for the experimental group the values for the fac-
tors interest, anxiety, and probability of success significantly 4.4
improved (see (∗c), (∗d), and (∗e) in Figure 2). It seems
that students working on the open source project first un- 4.2

derestimate their capabilities and then gain confidence. This

4.0
results in a significant increase of the probability of success
and a reduction of the anxiety. The increase of interest is
3.8
probably due to students beginning to understand the chal-
lenging and interesting sides of their programming project 3.6
while working on the open source project. For the control Activity
group no significant change occurred during the project. 3.4 Learning Effect
Commitment
3.3 Outcome 3.2
t1 t2 t3
The open source project results in a somewhat more un-
stable situation. Students start with a higher level of fear of
failure and a lower level of probability of success, while grad- Figure 3: Means of activity, learning effect and com-
ually gaining more confidence and finally showing a deeper mitment
interest in the subject. In particular, students feel more
proud of completing the open source project and are more The statement “I am learning much by doing the project”
likely to deepen their knowledge by continuing to work on it assessed the learning effect that students attribute to the
after they finished the official part of the work. This conclu- open source project. As for the activity, the felt learning
sion verifies the assumptions from section 3.1 which stated effect decreases in the middle of the course and rises again
that students working on the open source project value their towards the end (see Figure 3).
work more, but are less confident in their capabilities and The item to measure the commitment of students was in-
control. With the present data it is not possible to declare cluded only twice: at the middle and at the end of the course.
one of the approaches definitely better than another, but It stated “I feel responsible to make the project a success.”
using open source projects is as good as using traditional and the resulting averages show an increase towards the end
projects and helps students build self-esteem. of the course (see figure 3). Additional comments made by
students in the questionnaires support this observation: “It
5
For details see http://se.ethz.ch/people/pedroni/qcm.html
 was a lot of work, but very cool to have taken part in an their full cooperation and the students of Concurrent Object-
open source project.” Oriented Programming for acting as a control group.
The evaluation for the three additional items used a vari-
ance analysis with repeated measures to find out whether 7. REFERENCES
the averages between points in time differ significantly. This [1] E. Allen, R. Cartwright, and C. Reis. Production
was the case for all of the presented items. programming in the classroom. In SIGCSE ’03:
4.2 Outcome Proceedings of the 34th SIGCSE technical symposium
on Computer science education, pages 89–93, New
At first sight, the decrease of learning effect and activity in
York, NY, USA, 2003. ACM Press.
the middle of the course might be surprising. But it is impor-
[2] D. Carrington and S.-K. Kim. Teaching software design
tant to see the course and the project in the context of the
with open source software. Frontiers in Education,
other activities the students take part in. In fact, students
3(33):S1C– 9–14, November 2003.
were busy with midterm exams for other courses at this
point in time. To overcome the problem of multiplied pres- [3] C. P. Fuhrman. Appreciation of software design
sures, instructors need to consider both the project phases concerns via open-source tools and projects. In 10th
and the other obligations students have at university. In Workshop on Pedagogies and Tools for the Teaching
particular, during the time consuming phase of design and and Learning of Object Oriented Concepts, at 20th
implementation lectures could be reduced or transformed European Conference on Object Oriented Programming
into interactive lab sessions. (ECOOP), Nantes, FR, July 2006.
[4] K. J. O’Hara and J. S. Kay. Open source software and
computer science education. J. Comput. Small Coll.,
5. CONCLUSIONS AND FUTURE WORK 18(3):1–7, 2003.
The paper has two main contributions. Firstly, we show [5] P. R. Pintrich. A motivational science perspective on
that having students collaborate on open source projects the role of student motivation in learning and teaching
within the frame of a course is interesting and has advan- contexts. Journal of Educational Psychology,
tages over using an artificial project. Secondly, using a study 95(4):667–686, 2003.
backed up by research in psychology enables a scientifically
[6] F. Rheinberg, R. Vollmeyer, and B. D. Burns. QCM: A
sound evaluation of the approach.
questionnaire to assess current motivation in learning
This article described our first attempt at using open
situations. Diagnostica, 47:57–66, 2001.
source projects within a course. It showed that students
were obviously afraid at first but felt more proud of their [7] M. J. Wolf, K. Bowyer, D. Gotterbarn, and K. Miller.
achievement in the end. The next iterations of the course Open source software: intellectual challenges to the
will integrate this result and find solutions to cope with stu- status quo. In SIGCSE ’02: Proceedings of the 33rd
dents’ fear. A first approach is to give them a much more SIGCSE technical symposium on Computer science
detailed standard operating procedure to get started with education, pages 317–318, New York, NY, USA, 2002.
their projects. Such an approach could include the follow- ACM Press.
ing steps: (1) use the open source software as a tool, (2)
explore the code in search for programming patterns, (3) APPENDIX
identify weak points or adequate extensions that result in QCM questions, adapted from [6]:
a contribution, and (4) design, implement, and deliver the (I) I enjoy problem solving tasks like the ones that emerge in the
project work.
code. As a second measure, we plan to have several groups (S) I think I can tackle the difficulties of the tasks involved in the
of students work on the same project (but not the same sub- project assignment.
ject) so that they constitute a community within the open (S) Probably, I will fail solving the project assignment.
(I) What I like about the project assignment is the role of the re-
source community. This would then show the projects in a searcher that discovers connections.
more friendly and social way than currently. (A) I feel pressure having to perform well solving the project assign-
We plan to repeat the open source project and its assess- ment.
(C) This project assignment is a real challenge for me.
ment with undergraduate students to ensure that the results (I) After reading the instructions, the project assignment seemed very
also apply to this population. Our priority is to develop interesting to me.
further the methodology assessing teaching experiments by (C) I am very curious how well I will do in this project.
motivation analysis and identifying other settings in which it (A) I am a bit afraid of being embarrassed by my performance in the
project.
applies. In particular, we will experiment the impact of hav- (C) I am determined to work very hard for the project.
ing contests, written exercises, programming assignments, (I) I enjoy doing the project, I would not need any gratification.
in-class exercises, or in-class practical sessions on students (A) Failing the project assignment would embarrass me.
(S) I believe everyone can succeed in doing the project.
motivation. (S) I believe I won’t succeed in the project assignment.
(C) If I succeed with the project, I will feel a little proud of my pro-
ficiency.
6. ACKNOWLEDGMENTS (A) If I think about the project, I am a bit worried.
We thank Barbara Moser and Jessica Schnelle for their (I) I would also work on a project like that in my free time.
contributions to this work. Thanks also to the students of (A) The requirements of the project work paralyze me.
(C): Challenge
Techniques of Java Programming for participating in the (I): Interest
experiment and filling in the questionnaires. We also thank (S): Probability of success
Bertrand Meyer, Piotr Nienaltowski, and Volkan Arslan for (A): Anxiety

View publication stats