Software Engineering Lab – an Essential Component of a Software

Engineering Curriculum
Mira Balaban Arnon Sturm
Computer Science Software and Information Systems
Ben Gurion University of the Negev, Engineering
Israel Ben Gurion University of the Negev, Israel
[email protected] [email protected]

The goal of software engineering education is to teach theory

ABSTRACT and practice of software sciences, but the ultimate goal must be
The goal of Software Engineering (SE) education is to teach practical: Graduates of software engineering should study and
theory and practice of software sciences, with an ultimate practice production and management of good quality software
practical goal. Quite surprisingly, although standard SE [3] [4]. Observing the curriculum in other engineering fields,
programs include many hands-on courses, they do not include e.g., Chemical Engineering, we see that it includes besides
practical lab courses in software development and theory courses, lab courses where students get practical
maintenance, as common in other engineering disciplines. A experience and focused instruction in using engineering
capstone project course, that is standard in most SE programs, techniques. The importance of such labs in engineering
cannot function as an instructive SE-lab course since it does not education and their fundamental objectives are discussed in [5].
enable effective teaching and cannot replace a planned SE- In addition to these lab courses, there is a capstone project,
development experiment. where students are expected to independently initiate and
This paper describes an SE-lab course that creates lab deliver an engineering product. Lab courses should be
conditions, where students are faced with a deliberately distinguished from practice labs that are associated with
challenging, planned SE development and management tasks, courses and are task oriented, highly structured, and usually do
and the course staff provides team-based guidance. The paper not encourage broad abstract decision making.
analyzes the lab ideals, principles and goals, explains how they Quite surprisingly, although standard software engineering
are realized in the lab content, and presents its evaluation from study programs include many hands-on, studio, and project
the students and the instructors' viewpoints. based courses, they do not include practical lab courses in
software development and management, as common in other
engineering disciplines. Somehow, students are expected to
CCS CONCEPTS internalize, embed, apply and integrate all methods and turn
them into a practical body of knowledge, without practical
K.3.2 Computer and Information Science Education laboratory instruction. The standard software engineering
curriculum includes general background and foundation
KEYWORDS courses in Mathematics, Physics and Liberal Arts, Computer-
Software Engineering Education, Lab, Questionnaire Science foundation courses, core Software Engineering courses
and a final capstone project [3]. However, there are no
dedicated lab courses for practicing SE methods and
1 INTRODUCTION techniques. Indeed, Project Based Learning (PBL) [6] is based
on learning via practical experience, but it relies on the students
Software engineering (SE) is a practical paradigm, where
taking responsibility for their learning. Furthermore, when
practitioners apply software engineering techniques,
multiple tasks and deliverables are required (as in most
approaches, methods, ideas, tools, criteria and values, to create
software projects), close feedback is missing.
and manage software applications. The goal of software
One explanation for this unusual shortage lies in the self-
developers is to create software, which is sustainable,
embedded oxymoron of the mere concept of "Software
manageable, extendable, useful, and reusable (movable) [1].
Engineering instruction". While SE targets complex, large-scale
Permission to make digital or hard copies of all or part of this work for personal or systems that require stability, interoperability and reliability,
classroom use is granted without fee provided that copies are not made or distributed the teaching framework enforces restriction to small, short-
for profit or commercial advantage and that copies bear this notice and the full citation term, relatively simple systems, where it is hard to demonstrate
on the first page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, the essential role and value of using SE techniques. Moreover,
to post on servers or to redistribute to lists, requires prior specific permission and/or a SE techniques usually impose a "non-constructive" and
fee. Request permissions from [email protected]. delaying burden on the activity of software creation, while the
"direct programming" approach is straightforward,
ICSE-SEET'18, May 27-June 3, 2018, Gothenburg, Sweden constructive and fast rewarding. While students, after
© 2018 Association for Computing Machinery.
ACM ISBN 978-1-4503-5660-2/18/05…$15.00 mastering some programming skills, are enthusiastic on
https://doi.org/10.1145/3183377.3183395 producing software with observable performance, associated
software engineering activities like design (via modeling), instruction that is intended to trigger application and
testing and requirement analysis are met with a sigh. In integration of methods, sometimes in deliberately extreme
addition, SE methods require some programming experience conditions. For that purpose, there is a need for an SE-lab, for
and maturity that most students lack. These "built-in intentional investigation and usage of SE problems, processes
paradoxes" are summarized in Table 1 below. The difficulties and methods. Such a lab would provide opportunity for an
raised by the built-in paradoxes partly explain the lack of an SE- instructed integration of SE skills, methods, approaches, and
lab in the standard SE curriculum. technologies and would further allow discussion on software
engineering related decisions.
In order to address this need, we devised an SE-lab course,
Table 1. SE Practice and Education Built-in Paradoxes where the students cope with a natural SE task in a guided
SE Practice SE Education manner.
System  Complex  Relatively simple The lab mimics a natural SE-development task, where usage of
Characteristics  Large–scale  Small SE procedures is valuable and productive. It embodies the
 Stable  Relaxed stability following features:
 Interoperable  Usually isolated  Adopt an agile development approach.
 Reliable  Relaxed reliability  Maintenance challenges: Create a natural environment
Techniques and Systematic Feasibility of direct of successive changes due to complex evolution.
Methods design programming  Enable a thoughtful non-naïve customer (course staff),
Maturity Experienced Novice programmers that plan intentionally complex requirements.
developers  Insist on product validation.
 Enforce SE-systematic development, including models,
Another possible explanation is that the graduation capstone
validation, testing, requirements analysis and
project is often pointed at as the context where students are
reliability.
expected to exploit and integrate the overall plethora of tools
taught during previous courses. Indeed, capstone projects form  Cope with technology challenges.
an integral part of most SE curricula. The goal of such projects  Structured teamwork.
is to let students face the challenges of applying the theoretical The goal of the lab is to acquire experience in using SE
and technical knowledge, they gained in their studies, to solve methodologies, methods and technologies. The expected
realistic problems. In addition, such projects also enable learning outcomes of this course are experiencing the following
practicing other important capabilities such as teamwork, aspects of software development:
project management, and communication and presentation  Separate essential requirements from technical
skills. organization.
The SE curriculum at Ben-Gurion University of the Negev also  Analyze modes of usage, usability and needs.
includes a yearlong (two semesters) capstone project. The  Analyze requirements and select appropriate
projects are conducted by groups of 3-5 students that construct
architecture.
software solutions to problems of various kinds. We put special
emphasis on having external stakeholders in order to stress the  Analyze alternative design solutions.
work with realistic problems. Such projects have been  Separate logical architecture from technical
originated from various domains, including education, architecture.
machinery, mobile, and medical. The projects specification  Cope with technological requirements and select
imposes an iterative development, with emphasis on careful technology.
requirements analysis, modeling, prototyping, testing and  Plan evaluation and maintenance processes.
validation. In general, the resulting projects are impressive and
 Practice teamwork.
demonstrate high software development capabilities.
The contribution of this paper is in establishing an SE-lab
Nevertheless, our experience in leading this successful
course that adopts an instructive software engineering practice.
capstone project course for quite a few years shows that leaving
The paper further evaluates the course from both the students
all practical experience to the capstone project does not achieve
and the instructors' viewpoints.
the expected results and it is evident that much training is
We describe the evolution of the SE–lab course until it reached
required. The major limitation is the lack of practical
its current form. Next, we discuss the course content and
experience in applying SE processes and methods. Although the
structure, and finally present an evaluation and discussion of
SE curriculum includes a well-constructed core of courses that
the course from various perspectives.
teach SE theory, methods and techniques, there is no
opportunity for instructive practical integration of methods
into a working system. Consequently, quite frequently, the
effective SE-education is limited. We also see that development
2 HISTORY OF THE SE-LAB COURSE
and usage of associated SE artifacts are neglected in favor of The SE lab course has emerged from an initiative to introduce a
direct programming tasks. technology-tools lab. This initiative failed since many students
We believe that the capstone project course cannot replace the felt that at the end of their 3rd year they do not need an
need for an instructive SE-lab course. This framework does not instructed lab for studying tools, and also since the staff of
enable effective teaching, and besides, a capstone project as teaching assistants was not more knowledgeable on these
demanding as it might be, cannot replace planned lab matters than the students (sometimes it was the other way
around). An observation of this lab revealed that the students
missed a challenging context, i.e., an SE-challenging task, which difficult to faithfully grade the different teams and their
demands using technological tools. Just coping with tools was members.
not satisfactory. The major question that we asked ourselves involved the
The next stage in the evolution of this course was a common- contribution of this course, in view of having a final capstone
project team-based course, i.e., a course where all students project. We realized that we need a better structured, directed
develop the same project in individually instructed teams of 5 course, where we can plan the SE content and create extreme
students. The project framework was almost completely free, "lab-conditions" that call for thoughtful usage of SE methods.
apart of technological requirements that included multiple user The next, current stage, takes a structured-lab approach, with
interface requirements and a database. The project subject, changing prescribed team roles, and a declared software quality
almost from the beginning of this stage, involved software for moto: Abstraction before performance. The structured approach
constructing a social forum application. This task enables helps students in starting the development, the team roles
combination of design, maintenance, validation and clarify individual contributions, and the project moto clarifies
technological challenges. On the technology aspect, it requires that the emphasis is on a good design, rather than showing-off
usage of persistent storage, communication networks, multiple mastering of technologies. In particular, the "database first"
user interfaces and interactive behaviors. On the design and tendency, with automatically created application is not an
maintenance level, it requires thoughtful and flexible option any more. The current project topic is construction of a
architecture. On the validation aspect, it requires an overall forum-management system, i.e., a system for construction and
evaluation framework, including complex integration and maintenance of multiple, possibly inter-connected subject-
regression testing. specific human forums. The functionality of each forum is
This stage was a major advancement over the former tool- characterized by its policy rules that describe and constrain its
oriented lab. Yet, it posed several problems: structure and behavior.
1. Floundering: The unstructured, free development In the following, we provide a comprehensive description of the
framework caused some teams to flounder, not knowing SE lab and its management.
how to start (see a similar observation in [7]).
2. Bad solutions: Most teams started with building a
database, on top of which a thin application was developed. 3 DESCRIPTION OF THE SE-LAB COURSE
This led to a poorly structured application that could not The course starts with a couple of plenary (whole class)
cope with major requirement changes. For example, lousy lectures, intended for an overall scanning of the course ideals,
handling of communication logic (mixing technology with principles, goals, and organization, as well as for a condensed
logic), made addition of a new web-based interface more review and discussion of relevant SE methods. The overall
difficult. The expectations of the staff were that bad design project plan is dictated by the presented approach. Team-based
would show its problematics when facing hard work on the project starts in parallel, and is managed by bi-
weekly team-individual instructed sessions.
requirement changes. Practically, what happened, was that
The software ideals that we emphasize center around
since the developed systems are still relatively small abstraction in the software artifacts. We insist on including a
instructional size (recall the paradoxes from the short, abstract, platform and technology independent
Introduction section), most teams were able to cope with description of the project goal. For example, for the Forums
hard requirement change, in spite of the bad design. management application we have the specification: "A Forum
3. Unbalanced teamwork: Many teams suffered from uneven enables open, real-time discussions among its users, on a topic
experience of members. In many teams, there were one or of public interest. The Forums-maintenance system can
two students with some technology prior knowledge (due construct and maintain forums along time, and supports
to previous industry experience). These students simply different kinds of inter-relationships among forums, e.g.,
controlled the project development of the team, which subject-related or user connections". Usually, this specification
triggers a class discussion, as students expect mentioning web-
practically left the other team members paralyzed. In many
management, or explanation of terms like "subject-related".
cases, these "technology-masters" dictated the whole The class discussion helps clarifying the role of platform-
development. For example, students with web and independence and concept-abstraction.
database experience built a database system, used Another ideal that we emphasize is the essential distinction
database tools to creating an application with a web between what the software should achieve to how it is achieved.
interface. The dichotomy in such teams was between the This is a difficult issue for inexperienced students, that usually
members that actually built the project, to those that were tend to jump directly to the how challenge. This ideal implies
unable to contribute, due to lack of technological the "no direct jump to coding principle".
experience. Therefore, it was difficult to rightly evaluate The principles that we try to convey involve the primary status
the contribution of individual members. of software properties, over software writing. A natural
characteristic of novices (well, as humans) is to just jump ahead
4. Grading difficulties: Due to lack of process structure,
and code. We emphasize the principles of testable and traceable
uneven student experience, excessive usage of (sometimes requirements, and modifiable, and survivable code. These
unnecessary) technology, badly structured systems that principles imply emphasis on well-defined requirements; first-
achieved all listed functional and non-functional class status for tests – organization and integration; inter-
requirements, and lack of prescribed student roles, it was connected requirements, tests and code; and code abstraction,
traceability and alternatives.
The goals of this course are: (1) to enable students to technology independent, so to enable concentration on
experience and appreciate disciplined software development; analysis, design, management and validation. This decision was
(2) to demonstrate the importance of software design and taken following several years where the first version included
abstraction; (3) to let students select, practice, cope with and some heavy technology of client-server communication and
master new technologies; (4) to let students internalize, absorb persistent storage. This led most students to concentrate on
and adopt the self-discipline necessary for a planned thoughtful mastering standard layered architectures of web-based
software development and management; (5) to convey the applications, while neglecting analysis of the project under
necessity to balance between efforts invested in software development. An even worse situation was using dedicated
design to those invested in technology mastering; (6) to web-application technologies that dictated a data-base-
acknowledge the challenges in team work. oriented application structure. When this happens, we remind
Lab practices: The course organization is dictated by the goals. the project moto: "abstraction before performance", as such a
In order to create demanding SE conditions, the staff functions technology-dictated structure leaves little room for abstraction.
as a knowledgeable customer, that: (1) provides detailed well The version is split into two parts: Modeling and
written requirement documents; (2) plans the development implementation. For each part, the teams receive a
process; (3) specifies the expected artifacts; (4) enforces a requirements document, which specifies the subset of the
variety of extreme situations like technology failures. In requirements and the associated artifacts to be constructed. In
accordance with the course principles, there are no vague addition, this document describes the administrative individual
requirements, i.e., all requirements must be automatically roles of team members, and specifies the subject for the student
testable. For example, "a user-friendly GUI" is not automatically presentation in the next meeting. For each part, the teams have
testable, while concrete requirements about some GUI widgets roughly a two-week period in which they can independently
are. interact with their coach, and ends in a team meeting, as
In order to facilitate a supportive development environment, described above.
we encourage the students to team up to groups of five students The requirements document for the first part specifies a small
each. That way we reduce personal conflicts that may set of essential functional and non-functional requirements and
negatively affect the development effort. Each team has a coach constraints for starting the project. Teams are expected to
(mentor) and there are bi-weekly 2-hours team meetings. A hand-in: (1) detailed use-case analysis, using activity diagrams
team meeting consists of a short (10 minutes) oral presentation as a means for structured process specification, and with
of a technical subject, by a member of the team (serves the goal analysis of risks, interconnections and nuances; (2) an
of coping with new technologies); a summary of the architectural design of vertical and horizontal components for
achievements by the team member that functions as the version the necessary business logic; (3) intra and inter component
leader (serves the goal of teamwork and disciplined analysis and design of structure and processes; (4) planning of
development process), and a detailed planned discussion of the validation instrumentation, in the form of a testing machine,
version content (serves the goal of planning, design, validation the tests for this version and their architecture; (5) planning of
and traceability). The intention is to enable a constructive a traceability means which includes requirements, tests, and
discussion, in a pleasant and relaxed atmosphere, rather than a code. The tester and the leader of the first version are
test. Therefore, although each version and meeting are graded responsible for the design of the testing machine. The whole
by the coach, errors or bad decisions or malfunctioning can be group is responsible for the modeling artifacts.
corrected without penalty. Moreover, efforts to locate the The team meeting that follows this part mainly focuses on
individual contributions of team members are deliberately discussion of modeling decisions, their advantages and risks.
restricted. For example, use-case design for functional requirements is
The project is developed using an agile process of four versions. associated with a concrete specification of possibilities of
During the development, there are few plenary sessions for expected failures and implied consequences. These failures and
discussing version-related general issues. The work on each consequences must be reflected in the use-case associated tests.
version starts with a version-requirements document written The demand to use activity diagrams for describing the process
by the course staff. The version document is extracted from the of activities in a use-case is intended to enforce independent
original general overall project requirements document. study of a modeling language that is not taught in the Software
Version requirement documents usually leave some points Modeling course.
open, to be further augmented by the teams. For example, Inner component structure analysis is expected to include
specification of possible forum policies, password constraints, detailed class models with associated constraints and
or user rules. This is done in order to enable students to practice functionalities. Concrete architectural and structure decisions
requirement engineering. Yet, this element in the course must are discussed in order to make sure that they are thoughtful
be restricted, in order to preserve common control and tracking decisions rather than casual unplanned. For example, the
for all teams. implication of making a Security or a Forum-policy independent
In each version, team members change roles: There is a version vertical components are discussed using concrete examples of
leader who is responsible for the overall version results; interactions with other project elements like retrieval and
version client-tester who is responsible for the version verification of user passwords or forum policies.
acceptance (black-box) tests, and does not participate in the Architecture discussion covers also the need for a service layer
version implementation; a modeler role and version developers for use-case management in every component, so to enable
(the modeler and the leader are also developers). flexibility. Team coaches initiate discussions about changes in
First version: This version is the most important in the project functionality, like modification and removal of use-cases, and
construction since it lays an initial basis for the core logic the benefit in having a modular service layer. Traceability
component and its associated artifacts. It is deliberately functioning is reviewed by following various trace scenarios,
e.g., from a requirement to a use-case and a test, or from a test is first, to recall a modeling language that has been taught but
to a use-case and a requirement. The plan for a regression test, not much practiced, but mainly, to use the hierarchy feature of
using the testing machine is observed, as well as the plan of test statecharts, so to produce human understandable models that
architecture. are directly implemented in the software. This point is explicitly
The second part of the first version is mainly devoted to checked in the team meeting.
implementation of the first part artifacts, with addition of a few Third version: This version introduces the requirement to add
functionalities. The final result of the first version is a a web-based user interface. The interactive notification
standalone software that can be operated via an associated requirement is unidirectional for the web-based interface, i.e.,
testing component, management tools like logging and error- there is no requirement for pushing notifications to the web
recovery tracking, and a version document that includes its interface, due to the heavy technological load, but GUI
requirements, models and a requirement traceability tool. The notifications can be triggered by web-interface activities. The
central role of a testing component right from the first version risk with a new web-based interface is that students create a
eliminates reliance on technologies that do not lend themselves "duplicated" application, i.e., one for each interface that are
to heavy testing, as we have seen in the previous stages of this inter-connected via the common database. This really
course. Moreover, it conveys the importance of testing right happened in previous years, sometimes, with students not
from the beginning of a project (as writing tests is not a popular being fully aware of the bad architecture. In order to clarify and
activity, and usually requires an enforced development prevent this harmful duplication, we further introduce
procedure). The team meeting is devoted to demonstrating the requirements for application-based functionalities that do not
implemented functionality and the management and involve database. In particular, we have requirements for GUI
traceability capabilities. If time permits, the coach tries notifications that are triggered by user actions in the web
applying an ad hoc small requirement change. This is also tried interface, without involving database storage, e.g., proper
in later versions. operation while database connection mal functions, or
Second version: This version provides the first opportunity for management of non-persistent information, like temporary,
coping with technology, and inserts the need for balancing session-based IDs or even passwords. This version is meant to
technology with design. On the technology aspect, the version add a few more requirements, to add the new technology, and
introduces two technological requirements: A client-server to check the quality of the software constructed so far, its
communication between a GUI user interface, and persistent stability, flexibility and robustness. The new technology
storage. The main new logic functionality in this version requires careful design of its integration.
includes interactive connection between users and the Fourth version: This version wraps up the development. There
application. The interplay between the client-server is no new technology, and the requirements are meant to check
communication to the interactive connection raises important the features of the project. To check flexibility, there is a change
issues of abstraction and design. The insertion of technology of some previous requirements with the request to analyze and
raises issues of stability in case of technology failure. document the steps of the change application. To check
Teamwork on this version is preceded by a plenary session, robustness there are requirements for system load stability,
where we discuss lessons from the first version, the including stability under contradictory user operations.
technological functioning required for the second version, and Overall project summary: The project is a framework where
the interactive communication requirement. The new student experience instructed development of a demanding
technology requires teams to independently select persistent software. It introduces balance between design and technology,
storage technology and client server communication. Team abstraction and flexibility of requirements, and enforces
roles from the first version are switched. The new leader construction of a rich application (not a thin database client).
appoints a new tester, team members for studying and The instruction environment is based on team-individual
suggesting the necessary technology tools, a modeler and meetings, where conclusions of group discussions are
developers for the new functional requirements (developers implemented towards the next version, and enables backwards
can have multiple tasks, and the leader and modeler are also corrections.
developers). The time period for this version is three weeks, The SE-lab framework enables individual student experience
due to the relatively heavy requirements. through individual presentation, changing member roles, team
Interactive communication requires careful design of the administration, and independent study of models and
associated activities. The required interactive behavior technologies.
includes fine nuances. The requirement is for different kinds of Grading: Grading student projects is always challenging, due to
notifications to users, following a variety of triggers. The the variety of directions that a project can take. In the SE-lab,
students are directed to consult the Observer design pattern and since the planned content is identical for all teams, we have
its different versions. Coaches workout with their teams the developed a very detailed final-evaluation-scale that considers
need to separate the interactivity logic-management from the all aspects of the project, as they are realized in the final
communication technology. This is particularly important for version: Functionality, robustness, associated artifacts,
enabling multiple user interfaces. The requirements for back validation, and traceability. The final course grade is
notification (system pushes notifications to the user) are determined by the final-evaluation-scale, the version grades
planned to include a demand for notification that is not (including the backwards correction options), and the student
database driven, like notification for a user that opens too many presentations in the meetings.
simultaneous sessions, or delayed notifications. In order to reduce variability of team meetings, we prepare for
The introduction of a GUI interface calls for introduction of an every round of version-meetings, a meeting-plan sheet (beyond
appropriate modeling. We ask for statecharts modeling, with the student technical presentation and the version-leader
emphasis on appropriate level of hierarchy. The challenge here summary) and an evaluation sheet. During the meetings,
coaches are expected to follow the meeting-plan as much as mentioned that although being taught in various courses, the
possible (deviations are, sometimes, unavoidable). The actual execution and usage of software processes were minor.
evaluation-sheet and grades are given (right) after the As for tools, the students mentioned that training in tools like
meetings, so not to spoil the pleasant atmosphere. Therefore, GIT and Trello was hardly given or practiced. They mainly
the version-evaluation sheet cannot be as detailed as the final learned these by themselves. With respect to Experiencing
one. Every round of team-meetings is preceded and followed by Development in Teams, many students mentioned that in
staff meetings. In the pre-round meeting, we discuss the planed academic settings it is hard to coordinate among the students
requests, and in the post-round meeting, we discuss, compare and that there are free riders. Nevertheless, they claim that
and try to balance the staff experience and evaluations. when working together the development time is reduced and
Comparison with capstone projects: Both courses deal with more ideas and feedbacks are emerging. With respect to
development of an SE system. However, this is where the Experiencing Requirements Management, the students
comparison ends. While the SE-lab offers continuous coached mentioned that the subject did not get much of attention both
meetings with an expert in the project subject, capstone in theory and in practice. With respect to Experiencing Software
projects usually receive spurious coaching by a non-expert Design, the students acknowledge the subject of design
mentor. The SE-lab introduces extreme artificially created patterns, yet they think that there is a little emphasize on the
conditions while capstone projects might not have any software design itself. Non-functional properties were also put
demanding conditions of development. The SE-lab introduces aside.
a thoughtful software design that might be unnecessary for With respect to Experiencing Software Implementation, the
many capstone projects, e.g., projects with simple, well-defined students claim that not much of focus or feedback were given to
unchanged architecture, where difficulties lie in complex coding. Yet, they mentioned the background they got on code
algorithmic requirements. smell. With respect to Experiencing Software Quality, the
students claim that much focus was given on unit testing but
even those were not carefully checked. In their view, testing did
4 EVALUATION OF THE SE-LAB COURSE not get enough emphasis throughout their studies and that
To evaluate the effectiveness of the software engineering lab their capability is limited. Nevertheless, they mentioned that
and to check whether its intentions and objectives were they have been exposed to TDD and to integration and
successfully achieved, we adopt the survey technique and acceptance testing. With respect to Experiencing Training and
devise two questionnaires. One refers to the pre-lab experience Coaching, in general the students feel that the guidance or
and knowledge of the students before the lab and the other feedback regarding the design, coding, and testing is not
refers to post-lab experience. Both questionnaires address the sufficient.
following aspects: Experiencing Software Development In summary, as of the status before the SE-lab course, both the
Processes, Experiencing Development in Teams, Experiencing quantitative results and the students' comments indicate that
Requirements Management, Experiencing Software Design, further clarification, practicing, and guidance on software
Experiencing Software Implementation, Experiencing Software engineering is required.
Quality, and Experiencing Training and Coaching. Post-lab questionnaire: We were further interested in
Pre-lab questionnaire: The questionnaire consists of verifying that the SE-lab course indeed achieved its goals. Thus,
questions related to the above aspects and is aimed at checking we approached the two last cohorts of the program: The one
the experience and knowledge of the students in order to verify that took the course during 2016 and just graduate its studies
that there is a real need for the lab. We electronically (including the completion of the capstone project), and the
distributed the questionnaire to 97 students of the last year other that just completed the SE-lab course (2017). While
cohort, out of which we received 22 answers. For each talking with the 2016 cohort students, it seems that they
statement the students were required to refer to the extent to appreciate the SE-lab course to a large extent. They claim that:
which each statement holds (from 1- limited extent to 5 – large "it is a very good preparation for the capstone project", "it is a
extent). In the Pre SE-Lab column, Table 2 presents the critical course [in terms of importance]", "it is an important
averages of the students score with respect to the mentioned course in which all engineering aspects are integrated", "it
aspects. The "Tot" column reflects the overall score in that provides a good and right pilot to learn from mistakes before
aspect, the "Lrn" column reflects the average scores of the the capstone project takes place".
learning in that aspect (i.e., the students have learned new Having the generally good feedbacks, we were interested in
techniques) and the "Prc" reflects the score of the practical looking into the details and electronically distributed a post-lab
experience the students get (i.e., they used already known questionnaire to two cohorts (the one that respond to the pre-
techniques). It seems that actual experience or application of lab questionnaire and the cohort before, from which we got the
the taught theory and principles is quite limited (in all cases, above feedback). The questionnaire includes a detailed set of
beside teamwork, the scores are below 3 out of 5). questions related to the actual practices taken throughout the
In addition to ranking statements, we also asked the students to SE-lab (see the Appendix). With the 2017 cohort, we distributed
provide us with additional comments and insights regarding the questionnaire to 97 students and received 17 responses,
their knowledge and experience before the SE-lab. With respect whereas with the 2016 cohort we distributed the questionnaire
to Experiencing Software Development Processes, the students to 61 students are received 13 responses.

Table 2. Survey Results (means and standard deviations in brackets)

Software Engineering Pre SE Lab (2017) – in 5 scale Post SE Lab (2017) – in 7 scale Post SE Lab (2016) – in 7 scale
Aspects
 Tot Lrn Prc Tot Lrn Prc Tot Lrn Prc
Experiencing Software 3.05 4.05 2.71 4.29 5.38 4.14 4.56 5.84 4.38
Development Processes (0.69) (0.79) (0.34) (0.88) (0.71) (0.81) (0.88) (0.75) (0.75)
Experiencing 3.52 3.52 4.76 5.33 4.57 4.80 5.39 4.61
Development in Teams (0.47) (0.47) (0.68) (0.57) (0.62) (0.73) (0.39) (0.73)
Experiencing
2.76 3.05 2.61 5.27 5.75 5.12 5.13 5.72 4.96
Requirements
(0.35) (0.39) (0.27) (0.79) (0.25) (0.85) (1.09) (0.44) (1.18)
Management
Experiencing Software 3.02 3.32 2.71 4.58 4.75 4.53 4.31 4.59 4.23
Design (0.56) (0.6) (0.39) (0.74) (1.01) (0.70) (1.23) (1.75) (1.15)
Experiencing Software 3.07 3.73 2.94 4.83 4.38 4.91 4.52 4.66 4.49
Implementation (0.36) (0.94) (0.18) (0.56) (0.87) (0.50) (1.34) (0.70) (1.45)
Experiencing Software 2.87 2.87 4.59 5.06 4.54 4.52 4.74 4.49
Quality (0.46) (0.46) (1.15) (0.42) (1.21) (1.34) (0.61) (1.54)
Experiencing Training 3.09 3.77 2.75 4.83 4.83 4.08 4.08
and Coaching (0.59) (1.27) (0.03) (0.96) (0.96) (1.09) (1.09)

The two right columns in Table 2 summarize the average score With respect to Experiencing Software Quality, students
in each of the aspects described before. Here the columns of indicated that they felt that too much attention was paid to unit-
"Lrn" indicate either importance of aspect, lesson learnt, and test coverage (2017).
understanding of different viewpoints (i.e., of students and With respect to Experiencing Training and Coaching, students
instructors). In all aspects, the results of the students of both in both cohorts indicated that group meetups are generally
cohorts indicate an improvement in both learning and good, although guidance is sometimes limited to checking
practicing (above the mean (4) in the post questionnaire). In whether version requirements are met (2016, 2017). Several
general, the students were satisfied with the SE lab course. students suggested reducing the load (2017) in order to
However, as we asked them to provide ways to improve the provide more in-depth guidance (2016, 2017) and to increase
course, they provide us with issues that require further synchronization among course coaches.
attention. When examining in detail the various results for each
of the questions as appear in the Appendix, we had the
following observations. 5 DISCUSSION AND LESSONS LEARNED
With respect to development processes, the overall average We believe that the SE-lab course achieves its goals largely. In
score of the two cohorts was 4.21, yet the discussions on the all desired aspects, student responses indicate improvement, in
pros and cons of various processes and tools (average score = both learning and clarifying SE methods and in practicing these
3.48) can be improved. With respect to development in teams methods. Moreover, the instructors of the yearlong capstone
the overall average score was 4.59, yet, the students' results project course that follows the SE-lab note that they observe
indicated that the team work was not optimize (average score meaningful improvement of student knowledge and skills.
= 3.2). With respect to design the average score was 4.38, the Some aspects that constantly worry us, as course instructors,
results indicate that not much of design reviews was performed refer to teaching and evaluation. Although team meetings are
(average score = 2.7). With respect to implementation, the pre-planned and are associated with clear evaluation criteria,
average score was 4.44, however, the results of the 2016 cohort individual personality, character, training, experience and
indicate that not much of code reviews was performed (average knowledge of coaches still have their own impact. Naturally,
score = 2.5). With respect to software quality, the average score coaches tend to emphasize elements of their expertise. Some
was 4.52, yet, it seems that the unit testing was not planned in coaches spend more time on software models, some prefer
advance (average score = 3.24). checking coding or validation aspects, and others have
We next moved to read the student comments in order to get difficulties in conducting a discussion and lean towards an
further insights. It should be noted that these are anecdotal as undesirable test atmosphere, following the written meeting-
most students did not provide detailed comments. In the protocol. Clearly, it is impossible to expect that all coaches will
following, we refer to these comments divided into the relevant be ideal and identical: Knowledgeable, mature, self-confident,
aspects, as discussed before. In brackets we indicate the cohort open, and friendly. In the post-version staff meetings, we
from which the comments were issued. discuss relevant issues and try to reduce variability in version
With respect to Experiencing Development in Teams, some grading. In the future, we aim at having a more detailed meeting
students indicated that while teamwork is generally positive, it plan that precisely defines the content and procedure of team
allows free riders, since no tight monitoring of the actual work meetings. Yet, we must leave room for creativity, individual
was applied. Monitoring was mainly via team commit logs, and differences, and personal variety.
it might be better to determine the various roles within the The annoying phenomenon of free riders is still not solved. In
team more rigorously (2016). order to reduce the number of students with minimal or close
With respect to Experiencing Software Design, some students to zero contribution, we have conducted the procedure of self-
indicated that issues of planning and design received lesser organizing teams, assuming that at their third year of studies,
emphasis, as the focus was on the code (2017). They most students already belong to naturally created teams. This
commented that it would be beneficial to get more insight into procedure was positively accepted by the students, but there
actual usage of design patterns (2017).
are always teams that are administratively created, or students self-directed and self-motivated learning [10]. In [9] for
that are added to existing teams by the SE office. Besides, example, the studios are organized similarly to a capstone
students differ in their capabilities, devotion, and available projects. Furthermore, the SE principles according to which the
time. Efforts to pinpoint the individual contribution of team studios were designed are not explicated.
members sometimes create unpleasant, dense atmosphere in Barzilay at al [11], indicate that an advanced summary course
meetings. Some coaches have a softer character, and feel uneasy in software that synthesizes all aspect of SE is still missing. In
to embarrass students in public. Such activities spoil our their work they propose a course with attempts to address the
intention to create constructive friendly discussions in team fundamentals of SE, practices and tools, productization, and
meetings. Consequently, although we encourage coaches to try technology evolution. Nevertheless, it seems that the course
to locate free riders, we avoid explicit individual member refers only to a sub-set of the development lifecycle and less
testing. emphasizes the abstractions required is SE.
Considering the student comments, although the general
reaction is highly positive, there are several comments that
deserve attention. We discuss these following the aspects we 7 CONCLUSION
followed throughout the paper. We described an SE-lab course that fulfills the standard role of
Experiencing Software Development Processes: Some an engineering lab course: Offer lab experiments for practicing
students perceived that their actual engagement into a specific methods and techniques, in a planned environment that
development process was limited. enables integration and investigation, with team-based
Experiencing Development in Teams: The importance of guidance. We analyzed the ideals, principles and goals of the SE-
teamwork is appreciated by all. Yet, in practice, there were lab, and explained how they are realized in the lab content. The
cases in which the work was divided or performed by only few SE-lab evaluation shows that students consider it to be a highly
members of the team. This has caused frustration among the relevant core course in the program.
students as well as extra workload. We have already discussed We pointed to several aspects that deserve further attention,
the problematics of free riders. mainly with relation to balanced team instruction and to
Experiencing Requirements Management: Some students, in student grading. Another challenge involves continuing the SE-
particular from the last cohort, indicated that expected version lab ideal and principles in the capstone project. This goal
content was not well defined. Our conjecture is that in the last requires coordination with the capstone project instructors,
cohort, the course staff and the project theme have changed, and mainly further efforts during the SE-lab teaching, towards
and thus version contents were not clearly defined. embedding these software development values.
Experiencing Software Design: Some students expressed the
need for more in-depth focus on discussing design alternatives.
Experiencing Software Implementation: It seems that the ACKNOWLEDGMENTS
notion of code review was not sufficiently practiced, and there
is a need to look for ways to integrate that experience into the We thank Achiya Elyasaf and Avi Hayoun for their help in
course. managing the SE-lab and for their valuable insights.
Experiencing Software Quality: In general, it seems that the
issue of software quality turns to be addressed very well and
that the students absorb the importance of a quality software. REFERENCES
1. P. Bourque, R. E. (Dick) Fairley, SWEBOK 3.0, 2014.

6 RELATED WORK 2. IEEE, Systems and software engineering — Vocabulary,

INTERNATIONAL STANDARD ISO/IEC/IEEE 24765, 2017.
Many attempts have been done in order to increase the
professionalism and practice of SE graduate students. These 3. ACM, Software Engineering 2014 Curriculum Guidelines
attempts usually refer to project based learning of various types for Undergraduate Degree Programs in Software
including the capstone projects. Engineering,
For example, Chatley and Field [8] acknowledge the gap http://www.acm.org/binaries/content/assets/education
between academia and the required skills by the industry. They /se2014.pdf
indicated the there is a need for much more hands-on 4. A. Mishra and D. Mishra, "Industry Oriented Advanced
experience and for balancing between teaching, learning, and Software Engineering Education," Croatian Journal of
assessment. To address those needs they suggest adopting a Education, vol. 14, pp. 595-624, 2012.
lean learning, which means learn by doing, small and frequent
assessments, automation, and support by software engineering 5. L. D., Feisel, and A. J. Rosa, "The Role of the Laboratory in
practitioners. Their main claim is that small and frequent Undergraduate Engineering Education," Journal of
feedback cycles increase the learning outcome. Although, we Engineering Education, vol. 94, pp. 121–130. 2005.
acknowledge and adopt the short feedback cycles, small 6. J. A. Macias, "Enhancing Project-Based Learning in
assignments as we discussed in the introduction, might not be Software Engineering Lab Teaching Through an E-
sufficient to capture SE challenges. Portfolio Approach," in IEEE Transactions on Education,
Recently, the notion of studios has been introduced [7] [9]. vol. 55, no. 4, pp. 502-507, Nov. 2012.
Studios mainly refer to physical location in which students can
work and collaborate. Although studios further motivate 7. A. J. Lattanze, "Practice Based Studio," 2016 IEEE 29th
students to focus on their projects. It seems that only limited International Conference on Software Engineering
guidance can be provided to the students, as they are aimed at Education and Training (CSEET), Dallas, TX, 2016, pp. 1-7.
8. R. Chatley and T. Field, "Lean Learning - Applying Lean International Conference on Software Engineering,
Techniques to Improve Software Engineering Florence, 2015, pp. 389-397.
Education," 2017 IEEE/ACM 39th International Conference 10. S. R. Haynes and D. R. Mudgett, "A design studio course in
on Software Engineering: Software Engineering Education application development: Lessons learned," 2016 IEEE
and Training Track (ICSE-SEET), Buenos Aires, 2017, pp. Frontiers in Education Conference (FIE), Erie, PA, USA,
117-126. 2016, pp. 1-8.
9. J. Lee, G. Kotonya, J. Whittle and C. Bull, "Software Design 11. O. Barzilay, O. Hazzan and A. Yehudai, "A Multidimensional
Studio: A Practical Example," 2015 IEEE/ACM 37th IEEE Software Engineering Course," in IEEE Transactions on
Education, vol. 52, no. 3, pp. 413-424, Aug. 2009.

APPENDIX
Pre SE Lab Questions
Experiencing Software Development Processes Expose to principles of quality modeling (e.g., design patterns)
Learned about software development processes Implement principles of quality modeling
Implemented software development processes Experiencing Software Implementation
Integrated approaches and tools in software development processes Consider you implantation to include non-functional properties
Analyze the pros and cons of tools in the context of software development Decide upon the technology to implement the software
Experiencing Development in Teams Implement the software based of the preliminary modeling
Develop software in team larger than two students The implementation fitted the preliminary modeling
The work is team was a challenge Expose to principles of code quality (e.g., design patterns, bad
smell)
The work in team supported the software development Implement principles of code quality
Experiencing Requirements Management Experiencing Software Quality
Learn about requirements management Focus on testing development for the software you
implemented
Learn about development of a software that is partially defined Track the requirements vs. the code and the testing
Develop a software that is partially defined Check the software flexibility for introducing new requirements
Pay attention to the testability of the requirements Check the models you developed
Prioritize, decompose, and estimate the requirements Execute the testing on the code you developed
Use tool for requirements management Execute the testing for non-functional requirements
Experiencing Software Design Develop the testing in parallel to the code
Learn about design considerations in software development Experiencing Training and Coaching
Implement design considerations in software development Guide on the system you develop
Learn about design considerations originated from non-functional requirements Guidance in software development is important
Implement design considerations originated from non-functional requirements The actual guidance has helped you
Post SE Lab Questions
Experiencing Software Development Processes Experiencing Requirement Management
We examined various development processes We were not required for requirement analysis
The development process was defined in advance The project requirements were clear
We fully followed the chosen development process We analyzed the requirements
We partially followed the chosen development process We prioritize the requirements
We did not follow the chosen development process We took care of the requirements testability
We integrated various methods for software development The requirements were only used to define the project
We manage the requirement throughout the entire
We examined the pros and cons of the various techniques project
We examined the pros and cons of the various tools We used various tool to manage the requirements
Requirements traceability is important when these are
We did not examine the pros and cons of the various techniques changing
There is no need to trace the requirements as these are
We did not examine the pros and cons of the various tools changing frequently
We manage the relationship between the requirements
The importance of the development process was not clear and design
We manage the relationship between the requirements
It is important to have a development process and code
The development process was clear Experiencing Software Design
We used design consideration during the software
We adequately followed the development process development
Experiencing Development in Teams The design considerations consist of non-functional
requirements
We did not succeed to coordinate among the team members Modeling was the major means for specifying the design
There were team members with limited contribution Design patterns played an important role during the
design
We succeed to coordinate among the team members to a large extent The design was performed before coding
The teamwork helped us to fast advancing with the development Code organization is the design
The role division was effective The code was aligned with the design throughout the
entire project
The role division did not support the development We discussed design alternatives
All team members equally contribute to the development The project design was known at the beginning
We learn new things when working in a team From the view point of the students, the design is the
major part of the project
The teamwork improved my communication skills From the view point of the instructors, the design is the
major part of the project
The teamwork facilitated discussion on alternative solutions We performed design reviews
There was a need for a leader We followed software engineering principles throughout
the project design
The teamwork supported the due dates deliverable Experiencing Software Quality
Experiencing Software Implementation Unit testing were written before coding
We were required to select a technology for implementing the project Unit testing were written along with the coding
The code we had took into consideration the non-functional Unit testing were written after the coding
requirements
The code followed the models we had We had a high testing coverage
The code followed the design we had We had an exhaustive testing coverage
Design patterns were considered throughout the coding We had a low testing coverage
Bad smell were avoided throughout the coding We had integration testing
The code was reviewed by another team member We had system testing
The coding was done after determining coding rules We had automated testing
From the view point of the students, the code is the major part of the The testing were executed after each change
project automatically
From the view point of the instructors, the code is the major part of the The testing architecture was discuss in details
project
We performed code reviews The testing checked the system functionality with
respect to the requirements
We followed software engineering principles throughout the coding The testing checked non-functional requirements
The code was exhaustively documented The testing checked flexibility to changes
Experiencing Training and Coaching The testing checked reusability
The joint lectures discuss the various tools for executing the project The testing checked the user interface
The joint lectures discuss problems raise during the project From the view point of the students, the testing is the
major part of the project
The joint lectures contribute to the project From the view point of the instructors, the testing is the
major part of the project
The group meetups were used as an examination of the project
The group meetups were used as a focused forum for consulting and
guidance
In the group meetups we focused on understanding the requirements
In the group meetups we focused on testing
In the group meetups we focused on coding and problem solving
In the group meetups we focused on a discussion on alternatives for
design and implementation
In the group meetups we focused on the project management
The group meetups contribute to the project