Practical-03

Case Study of Conventional Process Model

Abstract: The actual benefits of model-driven approaches compared to code-centric development
have not been systematically investigated. This paper presents a case study in which functional
identical software was once developed in a code-centric, conventional style and once using
Eclipse-based model-driven development tools. In our specific case, the model-driven approach
could be carried in 11% of the time of the conventional approach, while simultaneously improving
code quality.

1. Introduction

During the last decade, model-driven development has made enormous progress and supporting
tools have improved significantly. Current tools form a second generation of instruments to support
model-driven development. However, there is a lack of comparisons of new, model-driven
development techniques with code-centric software development in terms of efficiency, code
quality, and time effort.

The contribution of this paper is a case study on code-centric and model-driven software
development. Software with the same functionality – an implementation of a software component
meta-model and a graphical editor for that meta-model – has been created twice: “by hand” and
with support of recent model-driven development tools. The case study has not been designed as
a controlled experiment (i.e, we could not control some factors influencing the results of our study).
Nonetheless, we found large differences among both approaches in terms of software
development efficiency, time effort, and quality of code. Those differences are worth reporting
despite the limitations of the case study (see section 4). This case study allows to draw
conclusions for other software development projects of the same domain (graphical editors for a
meta-modelled domain). By this, the paper tries to lower the lack of knowledge on the actual
benefits and limitations of specific approaches in software engineering.

The paper is structured as follows: In section 2, we present requirements of the developed

software, which includes a characterisation of the two compared software development projects. In
section 3, we compare these two projects and clarify to which extent they are comparable. Finally,
we summarize the results of the comparison and present future work in section 6.

2 The developed software systems

Both software development projects that being compared developed an implementation of the
Palladio Component Model (PCM, [Kro06], [BKR07]) and a corresponding graphical editor
(“Palladio Editor”).

The PCM is a research oriented meta-model to describe models of component-based software

architectures with focus on Quality of Service (QoS) aspects. The PCM has a large variety of
entities like components, interfaces, roles, and connectors. Requirements for an implementation of
the PCM include persistence, model observers, query methods, and model validations.

The Palladio Editor is a graphical editor supporting multiple views of the model. Those views can
be compared to the UML 2 component view (cf. [Obj05]), the UML 2 deployment view, and a view
to illustrate the behaviour of components. Additionally, containment relations need to be supported
to visualize composite component structures. Users of the PCM implementation and its editor are
mostly researchers and students, no industrial maturity level is required.

There are several constraints complicating the development. The PCM is constantly changing.
New research ideas shall be directly reflected by the PCM and its editor. As a result the PCM
changed frequently during the development process. Furthermore, the required PCM
implementation needed to be flexible to adapt for new requirements and extendible to integrate
new views, support many analysis tools, and changing validation routines. These requirements
had to be reflected by the architecture of the PCM implementation.

2.1 Conventional Development: Ride.NET

Overview The first software development project was performed by a student project group
“Ride.NET” at the University of Oldenburg, Germany [PGR05]. In total 13 graduate students were
involved in the project. The project started with a master thesis [Ufl05] that evaluated different
extensible architectures for large scale editors. The output of this work was a prototype of an editor
framework. Afterwards the main project group of 11 students started developing the editor for the
PCM. In parallel, one student assistant developed a PCM implementation.

The development was done conventionally (i.e., through writing the code by hand). The software
was developed in C# with the .NET framework 1.1. Besides the .NET framework only one
important larger library was used: Netron, a free graphical library [Pro]. The remaining software
was written from scratch.

The PCM and editor implementation finally were capable to support all requirements listed
in section 2. Additionally, the development included an import plugin that allowed to detect COM
components [Cor] from given C# code and an export plugin able to generate C# code skeletons
based on PCM specifications. During the development, often new requirements of the editor
directly initiated development at the PCM implementation. A considerable effort was spent on
debugging.
Time effort The project group developed one year in total. 11 students were involved in the
development. For each student, we assume a weekly time effort of about 20 hours1 . In total this
sums up to 11 students * 20 h/week * 52 weeks = 11.440 man hours for the members of the
project group. Additionally, one has to calculate the effort for the development of the PCM
implementation by the student assistant. The student assistant had 30 hours per month, summing
up to 30 h/month * 12 months = 360 man hours.

The project using conventional development took a total of about 11.800 man hours to write the
PCM implementation and the editor. The master thesis, which provided an initial prototype
architecture is not considered in the estimation, as it had a different focus.

2.2 Model-Driven Development: GMF / EMF

Overview From November 2005 to April 2006, a second project started. The aim of this project
was almost the same, but the chosen tools and frameworks were different. The project was a
master thesis [Kro06] of one student that previously was involved in Ride.NET. Instead of
hand-written code, the project completely focused on a model-driven approach to create an
implementation of the PCM and a corresponding editor. At the end, all requirements specified at
the beginning of section 2 could be fulfilled.

Development was done with IBM Rational Software Architect and the latest developer versions of
the Eclipse Modeling Project [Ecl06], including Eclipse Modelling Framework (EMF), Graphical
Editing Framework (GEF), and Graphical Modelling Framework (GMF).

Time effort If the same time measuring is used for the second project like for the first, one gets a
total of 1 student * 26 weeks * 50 h/week = 1.300 man hours.

3 Limitations
As already mentioned in the introduction, initially the projects were not designed to provide a
controlled study. Hence, the differences among both projects should be pointed out clearly. In
Ride.NET there was a requirements analysis phase that took about two months. Later, for the
model-driven approach the focus of requirements analysis laid on the PCM instead. The group of
12 students in the Ride.NET project caused a lot of communication overhead. Some time is
usually assumed to be spend on communication and coordination in large groups. In the Eclipse
based model-driven project the architecture was given. Hence it would not have been possible to
design the architecture according to very special needs. In particular, hand-written architecture and
code tend to be more customisable. For the needs of the PCM / Palladio Editor, the customisations
provided by EMF / GMF were mostly sufficient.

In the model-driven project, the focus was not on writing source code. Instead, the PCM
meta-model had to be defined, useful tools had to be explored (EMF / GMF were not set), useful
mappings and input models had to be defined – especially for GMF to generate the editor.
There was no time logging for the participants of both projects. Thus, all presented time efforts are
rough estimations.
The comparison applies only to the narrow domain of graphical editors for software models –
therefore the external validity of this comparison is limited to this domain. Although this section
lists several limitations, one can assume that model-driven development would bring more
efficiency to other graphical editor projects, too.

4 Results
We have presented and compared two projects aiming at developing an implementation of the
PCM and the Palladio Editor. It has been shown that developing functionally comparable software
took about nine-fold more time if the source code was written by hand, compared to model-driven
development. Although there are limitations of the presented comparison, the enormous win of
efficiency and productivity in model-driven development of the graphical editor shows a reasonable
domain where model-driven development is applicable.

The above description of both projects indicates that the successful deployment of modeldriven
approaches has to be supported by two pre-requisites: (a) the domain has to be well understood
(which was the case due to the knowledge on the requirements and the architecture) and (b) the
development has to be supported by powerful libraries and frameworks (in our case EMF, GEF,
and GMF).

Depending on the tool support of a domain, the time proportions of writing code and doing
meta-modelling change significantly. The existence of good templates is essential for productivity
as shown in the comparison. Another factor that influences the ratio of hand-written code is the
degree of customizations: Customizations that are not supported by templates still need
hand-written code.

For further research, the empirical investigation on the role of the influencing factors on the
success of model-driven development is required. In addition, the relation between generators and
transformations on the one hand and libraries and frameworks on the other hand and their
interplay has to be researched. There are some more challenges that come with the use of
model-driven development, raised by this comparison. What happens to model instances after the
evolution of a meta-model? How to manage the mix of generated and hand-written code after
major refactorings of meta-models? So far, there is no support for (meta-)model maintenance.
Future work will deal with these challenges