A Framework for Classifying and Comparing Architecture-Centric Software

Evolution Research
Pooyan Jamshidi 1, Mohammad Ghafari 2, Aakash Ahmad 1, Claus Pahl 1
1
Lero - The Irish Software Engineering Research Centre
School of Computing, Dublin City University, Ireland
2
DeepSE Group @ DEI - Politecnico di Milano, Italy
1
{pooyan.jamshidi|ahmad.aakash|claus.pahl}@computing.dcu.ie, 2 [email protected]

Abstract—Context: Software systems are increasingly required development to better understand requirements,
to operate in an open world, characterized by continuous systematically communicate with the stakeholders and
changes in the environment and in the prescribed objectively reason about qualities. Architecture models also
requirements. Architecture-centric software evolution (ACSE) help to crystallize design decisions and evaluate the
is considered as an approach to support software adaptation tradeoffs among them [1, 4]. This role of software
at a controllable level of abstraction in order to survive in the architecture also contributes to control the evolution [21] in
uncertain environment. This requires evolution in system order to avoid degradation as erosion [5], drifts [4] as well
structure and behavior that can be modeled, analyzed and as architecture pendency [16].
evolved in a formal fashion. Existing research and practices
Software architecture models not only facilitate
comprise a wide spectrum of evolution-centric approaches in
terms of formalisms, methods, processes and frameworks to
software development, but also extend their lifetime to run-
tackle ACSE as well as empirical studies to consolidate time to support dynamic software adaptation [30] and
existing research. However, there is no unified framework continuous verification [24]. This promotes dynamic
providing systematic insight into classification and architectures as a means to evolve software systems at
comparison of state-of-the-art in ACSE research. runtime [S44]1. Through reflective middleware, software
architecture models can be actively connected to the
Objective: We present a taxonomic scheme for a classification
running systems by providing facilities to change the
and comparison of existing ACSE research approaches,
leading to a reflection on areas of future research.
computation logic of a system.
We observed that existing studies for evidence-based
Method: We performed a systematic literature review (SLR), consolidation of ACSE research has focused on surveying
resulting in 4138 papers searched and 60 peer-reviewed [6] and characterizing [10] software architecture
papers considered for data collection. We populated the evolvability. They also focus on comparison of the support
taxonomic scheme based on a quantitative and qualitative for architecture evolution [17] and classification of
extraction of data items from the included studies.
dynamic aspects of architecture [S32] as well as taxonomic
Results: We identified five main classification categories: (i) frameworks [18]. These studies provide insight in the
type of evolution, (ii) type of specification, (iii) type of potential use of software architecture in software evolution.
architectural reasoning, (iv) runtime issues, and (v) tool However, we could not find any evidence to empirically
support. The selected studies are compared based on their synthesize the collective impact of existing literature.
claims and supporting evidences through the scheme. The objective of this paper is to systematically (i)
Conclusion: The classification scheme provides a critical view identify the focus of current research and (ii) classify the
of different aspects to be considered when addressing specific claims made for ACSE and available evidences for these
ACSE problems. Besides, the consolidation of the ACSE claims, and; (iii) provide a holistic comparison to analyze
evidences reflects current trends and the needs for future the potentials and limitations in current approaches that
research directions. (iv) outline hypotheses for future research.
To achieve this, we performed a systematic literature
Keywords- Architecture-Centric Software Evolution; Evidence- review. We adopted and tailored an integrated approach
Based and Empirical Study; Systematic Literature Review
[11, 26] to extract a coded schema by which we can
I. INTRODUCTION systematically review state-of-the-art of ACSE. Based on
this, we collected data from selected studies to answer five
Modern software systems are increasingly required to questions: (i) what types of evolutions are supported? (ii)
operate in an open world [27], characterized by frequent which formalisms are required? (iii) how architectural
and unpredictable change in the environment in which they reasoning is applied? (iv) which execution environments
are functioning and in the requirements they have to meet. are needed? (v) what tool support is available?.
Considering existing research [1, 6, 18] and practice [23, Having answered the research questions, we highlight
28], software architectures provide a sound basis to areas and hypotheses for future research. We report our
smoothly evolve software and dynamically adapt it to observations of the synthesis on the extracted data.
provide expected services. Architecture-centric software Extended materials that were used for the study comprising
evolution [1, 6, 10] allows an appropriate abstraction to a protocol, search results, quality assessments and lessons
model, analyze and execute software evolution in a learned as well as the extracted data are available at [25].
controllable and manageable fashion.
Traditionally, software architecture is considered as an
appropriate abstraction level in the early stages of software 1
Please note that notation [SN] refers to the primary studies in Table 5.
 The rest of the paper is structured as follows: Section II architecture description which should be treated by
compares and contrasts related work to justify the needs researchers and practitioners. Our paper extends some of
and scope of this review. Section III outlines the the findings related to evolution support in ADLs illustrated
methodology we adopted in this SLR. In Section IV we in their work. Mens et al. [1] also note the importance of
explain study planning. The first contribution of this paper, software architecture description to accommodate changes
a classification framework for ACSE approaches, is when there is a need to evolve critical software systems.
presented in Section V. This allows us to synthesize the They highlight key problem areas in ACSE and review the
data extracted from the primary studies and interpret this existing promising proposals to tackle them which helped
data answering the research questions in Section VI. us in the categorization process in our thematic mappings.
Section VII identifies future research directions based on
the results and it also discusses the limitations of our study. C. Taxonomies on Software Evolution
Finally, Section VII presents the conclusions. Although not directly related to the ACSE study
presented in this paper, some taxonomies of software
II. RELATED WORK change [18, 19] are proposed trying to answer the why,
In recent years, evidence-based and empirical research how, what, when and where of software evolution.
in software engineering gained a considerable momentum TABLE 1. SYSTEMATIC REVIEWS ON ACSE
[3]. In the context of ACSE, we observed that existing Study Reference Study Focus Change Time Number Years
of Studies
studies have focused on evolvability analysis [6], change Berivold et al. [6] Architecture evolvability analysis Design-Time 82 1992-2010
characterization [10] and classification of approaches Bradbury et al. [S32] Dynamic software architectures Run-Time 14 1992-2002
Williams et al. [10] Architecture change characterization Both 130 1976-2008
[S32], as summarized in Table 1. These are discussed Ahmad et al. [31] Architecture evolution reuse-knowledge Both 16 2004-2012
below in order to justify the needs for this review. Jamshidi et al. [25] Architecture-centric software evolution Both 60 1995-2011

A. Systematic Reviews on Software Architecture Evolution III. RESEARCH METHODOLOGY

Breivold [6] systematically reviews software In contrast to a non-structured review process, a
architecture evolvability analysis research. The objective is systematic review [26, 3] reduces bias and follows a precise
to obtain an overview of approaches in analyzing and and rigorous sequence of methodological steps. It relies on
improving software evolvability at architectural level, and a well-defined and evaluated review protocol [32], outlined
investigate impacts on research and practice. This survey in Figure 1. More specifically, we adopted the guidelines in
presents a synthesis of 82 primary studies. Their focus is on [17] for SLRs with a three step review process that
evolvability in general and analyzability, architectural includes: Planning, Conducting and Documenting. The
integrity and changeability in particular. Therefore, they review is complemented with an external evaluation for the
synthesize different sets of primary studies; our work outcome of each step, see Figure 1. We also extend the
focuses on the studies which formally specify software reporting of results so that it provides an explicit
architecture in order to enable a controllable evolution. taxonomical classification of the reviewed studies. To do
Bradbury et al. [S32] present a set of classification so, we take into account the recommended steps on
criteria for the comparison of dynamic software thematic analysis in software engineering [11]. This
architectures based on change type, process and formulates the foundation for a comparative analysis
infrastructure. They synthesize 14 formal specification among studies based on our defined data items that are also
approaches to discover similarities and differences. In subject to external evaluation prior to results reporting [25].
contrast to our proposal, they dedicatedly compare the
dynamic reconfigurations and architectural formalisms to
gain a deeper understanding of run-time software
architecture evolution, while not focusing on the other
dimensions such as design-time evolutionary aspects.
Williams and Carver [10] propose a change
characterization scheme and systematically classify
different approaches on how to distinguish and characterize
software architecture changes and software impact analysis.
This scheme works as a decision tree providing support for
developers to assess the impact of a proposed change and Figure 1. Overview of our research methodology
decide whether it is feasible to implement the change.
We also conducted an SLR [31] to classify studies A. Literature Extraction and Investigation
according to evidences that enable application and Four researchers were involved in the literature study.
acquisition of evolution reuse-knowledge in ACSE. We In review planning (Planning in Figure 1), a review
summarize the contribution of these studies to better protocol [32] was defined including the definition of
position the contribution of this paper in Table 1. research questions, the search strategy, and initial version
B. Comparative Studies on Description Languages of the classification scheme. In terms of search strategy, we
combined automatic with manual search. Automatic search
There are several surveys on architecture description was defined as a two-step process for which two categories
languages (ADLs) [8, 9, 17]. More recently, Medvidovic et of search strings (cf. Figure 2) were defined. The first
al. [12] look at recent developments and other aspects of
category selects the studies on architectural constraints A. Research Questions
which have been formally specified, and the second We formulated the general goal of the study through
category filters the studies on architecture-based evolution. PICOC (Population, Intervention, Comparison, Outcome
For the manual search, inclusion and exclusion criteria and Context) perspectives [22], summarized in Table 2.
were defined. The classification framework was The central research question translates to five concrete
subsequently defined. The definition of data items was questions:
based on information derived from literature sources, RQ1: What types of evolution are supported in ACSE?
specifically the works of [6, 10, 17, S32, 18], and from The aim is to get insight in what types of evolution are
experience with an earlier review [31]. For some data proposed by researchers following four perspectives of
items, additional attributes were introduced during a pilot evolution: “what”, “when”, “where”, and “why”.
run comprising of 10 papers to iteratively evaluate and RQ2: Which formalisms are required to enable an
improve the taxonomical scheme and synchronize ACSE approach? The aim is to get insight in the usage of
understanding of concepts between the researchers. The formal methods by researchers. This aims to assess which
protocol was cross-checked by an external reviewer and the languages and what levels of expressiveness have been
feedback was used to make small adaptations. used for modeling architectural constraints, verifying
Subsequently, we conducted the review (Conducting in properties and automation support.
Figure 1). One reviewer was responsible for automated RQ3: How architectural reasoning is adopted in
search. Manual search was performed by the other ACSE? The aim is to assess types of constraints and
researchers who checked each paper independently based architectural reasoning in existing ACSE.
on inclusion/exclusion criteria. Once the primary studies RQ4: Which execution environments and mechanisms
were selected, each study was read by one reviewer to are needed to enable run-time aspects of ACSE? The aim is
extract the data structured according to the scheme. to investigate execution environments or dynamic
Collected data items were crosschecked by the other reconfiguration functionalities used in ACSE.
reviewers. Finally, data derived from the primary studies RQ5: What tool supports is available for ACSE? The
was synthesized, collated, and classified to answer the aim is to investigate what automation facilities are utilized
research questions (Documenting in Figure 1). to provide support for ACSE concerns.
B. Data Validation and Synthesizing.
TABLE 2. PICOC CRITERIA TO DEFINE THE SCOPE AND GOALS OF THIS SLR
When reviewers entered study data into the scheme, Criteria RQ1
Type of Type of
RQ2 RQ3
Type of
RQ4
Runtime
RQ5
Tool Support
Population
they provided a short rationale why the paper should be in a Evolution Specification Reasoning Issues
Intervention Taxonomical classification; Internal/external validation; Extracting data and Synthesis
certain category. This rationale is used for internal Comparison A comparison by mapping the primary studies to the classification framework
validation purposes. The external validation was conducted Outcome A classification framework; A comparison framework; Hypotheses for future research
Context A systematic investigation to consolidate the peer-reviewed research
by an independent researcher outside the working group to
provide constructive feedback to the classification scheme B. Scope of the Study
and initial review data. The syntheses include the Once defined the entities and attributes of interest for
following: (i) classifying and comparing the primary the framework, the literature extraction process was driven
studies, (ii) analyzing of findings and reaching consensus, by the following search terms (Figure 2) structured into a
(iii) interpretation of the results and discussing potential logical expression. These search terms are combined by
hypotheses for future. using the Boolean OR and AND operators that resulted in
C. ACSE Taxonomical Classification 14*11=154 search strings in total.
No Name Results
We utilized a combination of existing ACSE 1 ACM 663
classification and thematic analysis to reduce the time 2 IEEE 1149
3 Science Direct 194
needed in developing the classification scheme. First, the 4 Web Of Science 480
reviewers read abstracts of the 10 selected papers for the 5 Springer Link 445
6 Pro-Quest 231
pilot run and look for segment of text, keywords and 7 Google Scholar 460
concepts that reflect the contribution of the papers. When 8 Wiley 516
Total 4138
this was done, the set of keywords from different papers
were labeled, overlaps reduced and combined. This helped
the reviewers to define a set of recurrent keywords Figure 2. A summary of search strings and results
representative of the underlying population. When abstracts
are insufficient to allow meaningful keywords to be chosen, After applying the 154 search strings on Google
reviewers studied also introduction or conclusion sections. Scholar, IEEE, ACM DL, Springer Link, Science Direct,
We then clustered the selected set of keywords to create a Wiley Inter-Science, Pro-Quest, and ISI Web of Science,
model of higher-order themes. we extracted 4138 manuscripts (Figure 2). Because we
used our search criteria on "title and abstract", the results
IV. AN OVERVIEW OF PLANNING AND CONDUCTING provided a relatively high number of irrelevant studies.
In this section, we summarize the key steps and Once the initial set of publications had been identified,
outcomes of the planning and conducting phases of the duplicate publications were removed. These publications
SLR as illustrated in Figure 1. were checked against some inclusion and exclusion criteria.
Irrelevant publications were removed and this resulted in
235 remaining publications. After further filtering by communities in particular. Column A in Table 5 shows the
reading titles and abstracts, 154 publications were left for counts of sub-criterions which characterize each paper. The
full text screening to assess their quality [25]. We ranked studies by year of publication are shown in Figure 3. The
them accordingly by investigating whether content focuses trend curve shows a steady number of publications from
on formal architecture description or formal analyses 1995 to 2003 followed by an increase from 2004 to 2008
related to architectural reasoning. We also checked whether and this trend soared in 2010. Note, that for 2011, the
the data analysis of the study is rigorous, based on evidence review only covers papers until September. While absolute
or theoretical reasoning instead of non-justified or ad-hoc numbers are relatively low, a recent trend indicates a
evaluations. Additionally, all references of the 154 studies significant increase of publications in ACSE area,
were checked to ensure no relevant paper was overlooked. especially in 2010 and 2011, which indicates that, as
Some authors also reported similar results in multiple systems are increasingly required to live in an open world
publications, which we eliminated to avoid bias of the [27], the crucial role of architecturally enabled evolution is
results. At the end, 60 studies were selected as primary being recognized [6].
studies with the highest ranks. We explicitly defined some
general and specific inclusion/exclusion criteria which we 12

described in detail in [32]. We describe the key criteria as 10

described in Table 3. 8
6
TABLE 3. INCLUSION AND EXCLUSION CRITERIA 4
2
Criteria Rationale
I1: The study must formalize We might have included studies which employ formal 0

1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
architecture description as well as methods, but do not actually employ them particularly
Inclusion

architectural properties. for evolution.

I2: The study must provide some We might have included studies which are in their
evaluation evidences or theoretical initial stages and mostly are based on trial examples.
reasoning. Figure 3. Number of papers by year of publication
E1: A study that is an editorial, These studies do not provide a reasonable amount of
abstract or a short paper. information.
Exclusion

E2: A study that focuses on formal These studies do not provide information regarding the TABLE 5. SELECTED PRIMARY STUDIES
methods themselves, rather than on research questions.
Id Study Title, Corresponding Author Year A
the use of formal methods for ACSE. S1 XSLT-based evolutions and analyses of design patterns, Dong et al. 2009 17
E3: A study that is a review paper. These studies do not propose any specific approach. S2 Behavior-preserving refinement relations between dynamic software architectures, Heckel et al. 2005 18
S3 Combining formal methods and aspects for specifying and enforcing architectural invariants, Kallel et al. 2007 18
S4 Evaluating pattern conformance of UML models a divide-and-conquer approach and case studies, Kim et al. 2008 16
C. Data Extraction and Synthesis S5 Formal analysis of architectural patterns, Caporuscio et al. 2004 20
S6 Modeling and enforcing invariants of dynamic software architectures, Kallel et al. 2010 19

The data extraction process was carried out by reading S7

S8
Style-based modeling and refinement of service-oriented architectures, Baresi et al.
A case study in re-engineering to enforce architectural control flow and data sharing, Abi-Antoun et al.
2006
2007
16
16

the 60 papers and extracting relevant data, managed S9

S10
A catalog of architectural primitives for modeling architectural patterns, Zdun et al.
Automated adaptations to dynamic software architectures by using autonomous agents, Wenpin et al.
2008
2004
17
17

through a bibliographic tool. In order to keep information S11

S12
A family of languages for architecture constraint specification, Tibermacine et al.
Architecture compliance checking at run-time, Ganesan et al.
2010
2008
20
17

consistent, data extraction for the 60 studies was driven by S13

A type-centric framework for specifying heterogeneous, large-scale, component-oriented, architectures, Jung
et al.
2010 12

the framework (Table 6). The included papers have been S14
S15
Analyzing architectural styles, Kim et al.
Changing attitudes towards the generation of architectural models, Castro et al.
2010
2011
12
12
initially reviewed and necessary information has been S16
S17
Classifying architectural constraints as a basis for software quality assessment, Giesecke et al.
A rule-based method to match software patterns against UML models, Ballis et al.
2007
2007
4
17
extracted and the framework populated with the extracted S18 Deriving detailed design models from an aspect-oriented ADL using MDD, Pinto et al. 2011 12
S19 Formal specification of the variants and behavioral features of design patterns, Bayley et al. 2010 12
information by one of the authors and then cross checked S20 Model-driven development for early aspects, Sánchez et al. 2010 14
S21 Pattern-based design evolution using graph transformation, Zhao et al. 2007 16
by the other authors. The results of the synthesis will be S22 PS-CoM building correct by design Publish Subscribe architectural styles with safe reconfiguration, Loulou e al. 2010 14
S23 Understanding the relevance of micro-structures for design patterns detection, Arcelli et al. 2011 9
described in the Section VI. S24 Using aspects for enforcing formal architectural invariants, Kallel et al. 2008 14
S25 Evolution styles to the rescue of architectural evolution knowledge, Le Goaer et al. 2008 16
S26 A model transformation approach for design pattern evolutions, Dong et al. 2006 13
TABLE 4. STUDY DISTRIBUTION PER PUBLICATION CHANNEL S27 A scalable approach to multi-style architectural modeling and verification, Wong et al. 2008 10

Publication Channel Count S28 A UML rule-based approach for describing and checking dynamic software architectures, Miladi et al. 2008 14
S29 Correct architecture refinement, Moriconi et al. 1995 12
Non-Pioneering Software Engineering Journals and Conferences 16
Preserving Architectural Choices throughout the Component-based Software Development Process,
Journal of Systems and Software (JSS) 8 S30 2005 15
Tibermacine et al.
Working IEEE/IFIP Conference on Software Architecture (WICSA) 4 S31 Style-based refinement of dynamic software architectures, Baresi et al. 2004 17
Workshop on Engineering of Computer-Based Systems (ECBS) 4 S32 A survey of self-management in dynamic software architecture specifications, Bradbury et al. 2004 20
International Conference on Software Engineering (ICSE) 3 S33 A contract place where architectural design and code meet together, Ubayashi et al. 2010 14
IEEE Transactions on Software Engineering (TSE) 3 S34 Design pattern solutions as explicit entities in component-based software development, Stepan et al. 2011 11
Lecture Notes in Computer Science (LNCS) 3 S35 Modeling architectural patterns using architectural primitives, Zdun et al. 2005 15
Journal of Information and Software Technology (IST) 3 S36 Simplifying transformation of software architecture constraints, Tibermacine et al. 2006 14

Software and System Modeling Journal (SOSYM) 2 A constraint architectural description approach to self-organizing component-based software systems,
S37 2004 16
Electronic Notes in Theoretical Computer Science (ENTCS) 2 Waewsawangwong et al.
S38 A constraint-oriented approach to software architecture design, Van den Berg et al. 2009 8
Workshop on Sharing and Reusing Architectural Knowledge (SHARK) 2
S39 A framework to specify incremental software architecture transformations, Barais et al. 2005 17
IEEE International Conference on Software Maintenance (ICSM) 1
An approach based on biographical reactive systems to check architectural instance conforming to its style,
International Conference on Quality Software (QSIC) 1 S40 2007 16
Chang et al.
Wiley Journal of Software: Practice and Experience (SPE) 1 An automated refactoring approach to design pattern-based program transformations in Java programs, Jeon
S41 2002 14
European Conference on Software Architecture (ECSA) 1 et al.
Springer Software Quality Journal (SQJ) 1 S42 Analyzing and comparing architectural styles, Levy et al. 1999 14
Journal of Visual Languages & Computing (JVLC) 1 S43 Architecting as Decision Making with Patterns and Primitives, Zdun et al. 2008 17

Workshop on Self-Managed Systems (WOSS) 1 S44 Architecture-based runtime software evolution, Oreizy et al. 1998 12

Workshop on Component-Oriented Programming (CompArch- WCOP) 1 S45 Capturing interactions in architectural patterns, Yadav et al. 2010 14
S46 Describing evolving dependable systems using co-operative software architectures, de Lemos et al. 2001 18
IEEE/IFIP Symposium on Theoretical Aspects of Software Engineering (TASE) 1
S47 Describing software architecture styles using graph grammars, Le Metayer et al. 1998 15
Future of Software Engineering (FOSE) 1
S48 Design pattern evolution and verification using graph transformation, Zhao et al. 2007 18
Total 60 S49 Evaluation of accuracy in design pattern occurrence detection, Pettersson et al. 2010 12
S50 Evolution styles foundations and tool support for software architecture evolution, Garlan et al. 2006 17
S51 Focus: a light-weight, incremental approach to software architecture recovery and evolution, Ding et al. 2001 14
D. Publication Channels and Trends S52 Formal specification of design patterns and their instances, Taibi et al. 2006 14
S53 Guiding architectural restructuring through architectural styles, Tamzalit et al. 2010 17

The selected papers are listed in Table 5. As indicated S54

S55
Safe integration of new concerns in a software architecture, Barais et al.
Scenario-based architectural design decisions documentation and evolution, Che et al.
2006
2011
16
13
in Table 4, the majority of these have been published in S56 Self-managed systems an architectural challenge, Kramer et al. 2007 13
S57 Style-based reuse for software architectures, Monroe et al. 1996 11
leading journals and conferences in software engineering in S58 Synthesizing approach for perspective based architecture design, Liang et al. 2003 10
S59 Towards a formal model for reconfigurable software architectures by bi-graphs, Chang et al. 2008 15
general and software architecture and evolution S60 Using UML2.0 and GG for describing the dynamic of software architectures, Kacem et al. 2005 18
 V. THE CLASSIFICATION FRAMEWORK taken down to apply a patch offline during maintenance
In this section, we propose a model consisting of 21 [17]. However, this scenario cannot satisfy the
analysis dimensions to characterize ACSE approaches. requirements of a specific class of software systems (e.g.
This model constitutes a foundation for evaluating ACSE mission-critical or safety-critical or business-critical) in
approaches that leads to a classification and comparison of which the system must operate continuously and remain
selected studies comprising those approaches. For each of capable of on-the-fly changes to the running system as a
the analysis dimensions (18 technical and 3 non-technical), result of the violation [S32]. Based on the
the model considers a set of standardized characterization classification in the context of Figure 4, we are interested in
options. These options resulted from combining the approaches which deal with reasoning about
classification attributes from recognized sources with those evolutionary aspects enabled by formal methods. Software
found in the set of papers that we selected. For instance, the architecture models therefore provide the required
set of options for the “Need for Evolution” was identified abstraction and facilitate the reasoning about the evolution
using the ISO/IEC 14764 Standard for Software of a system expressed by a formal model (e.g. Markov
Maintenance and, on the other hand, “Type of Formalism” model) [33]. We consider ACSE as a collection of
and “Description Language” were both mainly identified operational and analytical activities to evolve a software
using [S32] and [17] respectively. They, however, have system from an older version to a newer version, enabled
been improved over time by considering the new options in by architecture changes. Among approaches which use
the selected studies. As a driver for the selection of the architecture description to operationalize evolution [1], we
model parameters, Section A provides a rationale for the are interested in approaches which utilize formal theory to
proposed classification, while Section B outlines the details enable analytical reasoning to verify the system properties.
of various entities and attributes as a fine granular B. The Framework Entities and Attributes
representation of the classification framework.
Based on that discussion we understand that ACSE has
different dimensions. As conceptually outlined in Figure 4,
the architecture of the system needs to be specified (Type
of Specification). For analytical purposes, the properties of
systems are also specified as a part of the architecture
description as a number of constraints (Type of
Reasoning). An evolution mechanism (Type of Evolution)
can analyze consequences and apply changes either at
design-time or runtime (Run-time Issues). For scalability,
performance and economic issues, these activities require
automation (Tool Support). As far as the mentioned goals
are concerned, we propose the classification scheme as
presented in Table 6.
TABLE 6. THE ACSE CLASSIFICATION FRAMEWORK
Taxonomical Sub-
Coded Attributes
Classification classifications (Domain Knowledge, Standards, Keywords)
(Higher-Order Theme) (Sub-themes)
Need for Evolution ISO/IEC 14764 typology of change: Corrective, Perfective, Adaptive, Preventive, All applicable
Static: Transformation, Refactoring, Refinement, Restructuring, Pattern change; Dynamic:
Means of Evolution
Reconfiguration, Adaptation
Type of
Time of Evolution Design-Time, Run-Time
Evolution Change impact: Consistency checking, Impact analysis, Propagation; Change history: Evolution
Support Activity
analysis, Versioning
Stage of Evolution SDLC: Analysis/Design, Implementation, Integration/provisioning, Deployment, Evolution
Level of Formalism Lightweight, Formal
Modeling language: ADL, Programming lang., Domain-specific lang., Type systems, Archface,
Model-based; Formal models: Graph theory, Petri-net, Ontology, State machine, Constraint
Type of Formalism
automata, CHAM; Process algebra: FSP, CSP, π-calculus; Logic (Constraint language): OCL, CCL,
Type of FOL, Grammars, Temporal logics, Rules, Description logic, Z, Alloy, Larch
Specification Process algebra: Darwin, Wright, LEDA, PiLar; Standards: UML, Ex.-UML, SysML, AADL; Others:
Description Language
ACME, Aesop, C2, MetaH, Rapide, SADL, UniCon, Weaves, Koala, xADL, ADML, AO-ADL, xAcme
Figure 4. The conceptual framework UML Specification Static: Class, Component, Object; Dynamic: Activity, State, Sequence, Transition, Communication
Description Aspect Structural, Behavioral, Semantic
Structural: Pattern, Architectural style, Primitive, Metaphore, Micro-structure, Crosscutting
A. Rationale for the Classification Type of
Architectural
Constraint
concern, Clue; Invariant: Component-level invariant, Cross-component invariant, Pre-/Post-
condition, Temporal invariant; Relational constraints: Coding rules, Cardinality, Coordination,
Reasoning Meta-model, Variability
A causally connected architecture model, which is Intent of Reasoning
Type of Analysis
Specify, Preserve, Change, Enforce, Match, Discover, Analyze
Consistency checking, Model checking, Pattern conformance, Graph refinement, Enforcement
denoted by S, conceptually represents a running software Run-Time Runtime Environment
Component model: Fractal, KOALA, SOFA 2.0, CCM, OpenCOM, KobrA, Middleware: SIENA, OSGi,
.NET, MS-COM, EJB, JavaBeans, SOA middleware, Coordination model: Reo, Linda, MANIFOLD

system A embedded in an execution environment. Now let Issue Mechanism of

Runtime Evolution
Reflection mechanisms: Introspection, Constraint injection, Intercession, Reification, Causal
connection, Evolution enablers: Safe stopping, Runtime binding, State transfer
Need for Tool Support Architecture life-cycle: Business case, Creating architecture, Documenting, Analyzing, Evolving
D be the domain assumptions that are consistent with the S Tool Analysis Simulation, Dependence analysis, Model checking, Conformance testing, Interface consistency,
Support Usage of Tool Support Inspection and review-based
(Figure 4). Then it should be proved that they hold the Level of Automation Fully automated, Partially automated, Manual
A particular problem/challenge, Overview or Survey, Formalism for constraint specification,
requirements R. This can be formally expressed as Research
Research Motivation
Formalism for architectural analysis, Formalism for arch. evolution, Formalism for code generation
Development paradigm: SPL, OO, SOA, CBS; Traditional: Embedded, Real-time, Process-aware,
, where F is a function that maps the Method Application Domain Distributed, Event-based, Concurrent, Mechatronic, Mobile, Robotic, Grid computing; Emerging:
Cloud computing, Smart-*, Autonomic computing, *-critical, Ubiquitous

architecture specification into a semantic model. Software Evaluation Method Case study, Mathematical proof, Example application, Industrial validation

evolution deals with the violation of correctness criteria The groupings and their dimensions are shown in Table
after A is embedded in the environment and starts to 6. Note that the proposed framework is used for the
operate [24]. The violation may occur as a result of the comparison of ACSE approaches based on our focus which
divergence of (i) the system A from specification S, (ii) the was discussed in the previous section and should not be
environment behavior from specification D, (iii) the system adopted without a justification or verification. Firstly, we
goals from requirements R. Usually, a software system is deliberately did not include all aspects of software
architecture evolution. For example, the who question, (i) Need for evolution. According to the ISO/IEC
which identifies the stakeholders involved in software 14764 standard for software maintenance and Lehman’s
architecture change, is not covered in the current law of software evolution [34], the reason for evolution
classification because it reflects human aspects of evolution (purpose of change) can be categorized into four types:
and our focus of rigorous formal theory, analytical corrective, perfective, adaptive and preventive. Most
reasoning based on architectural constraints and runtime approaches are applicable for all purposes, but among them
issues aims to lessen human intervention. The where adaptive changes in the context of runtime evolution [S44]
question is also excluded because it has been thoroughly was explicitly mentioned in considerable number of
covered in [17] for specification-time evolution and in studies. (ii) Means of evolution. Among the different types
[S32] for runtime reconfiguration. Secondly, the proposed of evolution, transformation, reconfiguration, and pattern-
framework provides only one of the many ways in which based evolution gained significant attention. Refinement,
ACSE can be characterized. Most importantly, the restructuring, adaptation and architectural decision
framework is subject to continuous improvements, since evolution were explicitly adopted as a promising
the comparison attributes continue to mature due to mechanism for enabling evolution in the studies. (iii) Time
scientific and technological advances. A list of of evolution. Specification-time software architecture
improvement suggestions can be found in the protocol [32]. evolution has been more popular than runtime evolution.
The relative lack of contributions on runtime evolution
VI. RESULTS AND COMPARATIVE ANALYSIS results from a recent adoption of reflective component
We compare ACSE approaches along the dimensions models that provide architecture information to the
summarized in Table 6. The results of this analysis process mechanisms that enable the dynamic reconfiguration.
are summarized in Tables 7-12 below. Table 7-11 presents However, dynamic evolution recently gained a momentum.
the technical characterization of selected approaches based (iv) Support activities. ACSE related activities contain
on the model proposed in Section V. Table 12 classifies the analytic work to check relevant properties of architectures
studies based on their research method. Column P shows after the evolution takes place. For instance, consistency
the percentage of studies classified in each sub-criterion. checking aims to check whether the system architecture is
The last column shows the percentage of frequencies in consistent after the requested changes. Consistency
three yearly disjoint categories (1995-2005, 2006-2008, and checking is a popular architecture analysis, but other
2009-2011) for attributes that characterize 10 or more context-based analyses were also utilized. (v) Stage of
approaches. evolution. The focal stage of the software lifecycle was
A synthesized analysis of the studies is discussed in considered. Both early and late stages of the lifecycle were
each section and the investigated approaches are critically key areas of attention in comparison. Design and
summarized in subsequent tables. Mapping the studies to maintenance stages were the two stages in which evolution
the common scheme provides us with a unified framework mechanisms were active.
to group contributory artifacts of similar approaches. These
TABLE 7. COMPARATIVE ANALYSIS BASED ON “TYPE OF EVOLUTION”
categories provide a means to compare ACSE approaches. Sub-
Distribution %
Attributes Approaches P% N 95- 06- 09-
Moreover, the analysis of evidence based on a considerable Criteria
Corrective [50] 1
05 08 11

amount of research or lack of focus on specific attributes, Need for

Perfective
Adaptive
[12]
[1,9,40,42,46,50,53,56,59,60] 52
1
10 30 50 20
Evolution
differentiated by the count column in the subsequent tables, Preventive
All applicable
[45,46,52]
[2,3,5,6,8,10,11,17,37,39,41,44,47,48,49,51,54,55]
3
18 50 28 22
are the critical discussions of the next subsections - Transformation
Refactoring
[1,2,6,7,8,21,26,28,29,31,33,36,39,40,50,58]
[8,41,51]
16
3
31 50 19

structured by the sub-criteria. Each study may be Refinement

Restructuring
[2,7,13,22,29,31,43,51,58]
[8,13,50,53]
9
4
Means of
represented by multiple attributes in case of non-disjunct Evolution
Adaptation
Reconfiguration
[4,6,10,44,46,56]
[2,3,4,5,6,7,22,24,28,31,32,37,54,59,60]
83 6
15 40 47 13
attributes. Due to the limited amount of data, a statistical Pattern-based
Arch. decision
[1,9,17,21,26,34,35,41,42,45,48,49,52]
[11,12,25,47,55,57]
13
6
23 46 31

analysis was not feasible. Thus, we limit the discussions to Time of

Change operation
Design-Time
[4, 28]
[1,2,4,7,9,11,13,14,15,17,18-23,25-30,33-36,39,41-43,45-55,57,58]
95
2
43 27 35 37

results based on the table content, deriving future directions Evolution Run-Time
Consistency checking
[2,3,5-10,12,24,31,32,37,40,44,54,56,59,60]
[2,11,33,43,48,60]
19
6
42 53 5

in each area using quantitative summaries of the data. This Support

Activity
Evolution analysis
Change propagation
[5,7]
[3,6] 18
2
2
Versioning [50] 1
approach usually characterizes systematic mapping studies Impact analysis 0
[1,4,6,8,9,13-16,18-20,22,25,26,27,29,30,33-
[11]. At the same time, whilst addressing the reliability of Stage of
Analysis and Design
36,38,39,42,43,45,47,48,52,57,58]
32 25 34 31
Implementation [3,6,8,24,33,34] 92 6
the included studies, the research and evaluation method of Evolution
Evolution [3,5,6,10,12,17,21,23,28,31,37,40,41,44,46,49,50,51,53,54,56,59,60] 23 39 44 17
Integration and provisioning, Deployment 0
the approaches were also synthesized. This comparison
based on a common scheme reveals that e.g. a majority of As a concluding remark, the recent boost in autonomic
studies focuses on specific aspects of evolution or there is a computing [7] and specifically self-adaptive applications
lack of studies that point out gaps. [29] also reflects that reliable evolution of running software
has become a major challenge. This trend which is enabled
A. Type of Evolution mostly by architecture-based runtime software evolution
This category compares the included studies in terms of [S44] is expected to get more attention based on the current
a general taxonomy which focuses on the factors that publication trend on runtime evolution. Software systems in
characterize and position these mechanisms into a general open worlds [27] necessitate enabling run-time adaptability
purpose classification. The studies are further classified for systems to cope with environmental uncertainties,
into five sub-categories explained below, summarized in changing requirements, and failures.
Table 7.
B. Degree of Formalism and Expressiveness distributed. (v) Description aspect. We observed that both
This category compares the included studies in terms of structural and behavioral aspects have been addressed.
the degree of expressiveness and analyzability employed in However, semantic aspects have been neglected.
ACSE approaches. The design of languages and formalisms We observed advances in the fundamentals of
involves a trade-off between expressive power and architecture modeling, but a unifying formalism that
analyzability. The more formalism can express, the harder provides the basis for all architecture analysis requirements
it becomes to understand what instances of the formalism is unlikely [12]. Constraint specification languages are
say [12]. The degree of formalism is due to the ACSE growing due to the need to specify architectural properties
nature that requires utilizing analytical power of description that need to be verified against a system model (Figure 4).
languages, e.g., to check consistency or integrity after The coverage of topics points out a gap regarding
evolution or derive impact of primary changes or formalisms addressing architecture-centric evolution
quantitatively verify runtime changes for dynamic augmented with more semantic enabled reasoning. This is
adaptations. The language aspect has been exhaustively particularly challenging for specifying the system model
investigated in a comprehensive framework [17]. and reasoning based on a richer semantic model at runtime.
Therefore, we briefly review our observation in terms of Therefore, description languages need more integration
the attention of ACSE studies to the language aspect. Based with underlying formal theories to enable specification of
on this theme, the studies are further classified into five emerging architectural properties.
sub-categories explained below, summarized in Table 8. C. Type of Architectural Reasoning
TABLE 8. COMPARATIVE ANALYSIS BASED ON “TYPE OF SPECIFICATION” This category compares the included studies in terms of
Sub-Criteria Attributes Approaches P% N
Distribution %
95- 06- 09- constraints [S16] on architecture description to enable
05 08 11
Level of Lightweight [1,8,9,12,13,15,18,20,23,25,26,33-35,36,38,55]
88
17 8 46 46 automated reasoning. Architectural constraints are an
Formalism Formal [2-7,10,11,14,17,19,21,22,24,27-32,37,39-43,45-48,50,52-54,59,60] 36 38 42 20
Graph
Petri-net
[2,6,7,21,22,31,32,40,47,48,53,59,60]
[3,12,24]
13
3
38 39 23 inherent part of ACSE approaches since they facilitate
Model-based
Description logic
[1,2,4,5,6,7,9,11,13,15,18,19,20,26,28,35,55,56,60]
[29,32,41,52]
19
4
21 32 47 evaluating system properties. The studies are classified into
π-calculus
Prog. language
[10,45]
[8,3,17,41]
2
4
three sub-categories, see Table 9.
Alloy [14,27,37] 3
Type of
Formalism
OCL
CCL
[2,4,6,7,9,11,25,28,30,31,35,36,43,60]
[11,36]
83
14
2
36 50 14
TABLE 9. COMPARATIVE ANALYSIS BASED ON “TYPE OF REASONING”
State machine [5] 1 Distribution %
Sub-
Attributes Approaches P% N 95- 06- 09-
Z [3,6,22,24,28] 5 Criteria 05 08 11
C. Automata [39,46] 2 Pattern [1,5,11,16,17,19,21,23,26,34,35,39,41-43,45,48,49,52,57] 20 30 35 35
Process algebra [32,42] 2 Architectural style [7,10,12,25,36,37,38,40,42,43,44,46,47,50,51,53,57-60] 20 50 40 10
Temporal logics [5,19,50,52] 4 Primitive [1,9,26,35,39,41,43,54] 8
Archface [33] 1 Component-level invariant [30,39,60] 3
Ontology, Domain-Specific, Larch, Grammars, Type, FSP, CSP, CHAM, FOL, Rules 0 Cross-component invariant [2,3,9,20,25,30,36,39,42,44,50,56,58,60] 14 50 43 7
ACME [8,14,15,25,30,36,46,50] 7 Architectural
Crosscutting concern [18,39] 85 2
xAcme [30,36] 2 Constraint
Meta-model [4,7,9,43] 4
UML [1,5,17-21,26,30,31,33, 48,51,53,56,60] 16 31 31 38 Pre-/Post-condition [2,3,4,6,7,22,24,25,31,39,46,55,56,60] 14 36 43 21
Description Extended UML [2,4,6,7,9,11,25,28,35,39,43] 11 27 55 18 Coordination constraint [3,24] 2
58
Language AO-ADL [18] 1 Coding rules [11] 1
Darwin [37,56] 2 Cardinality, Metaphore, Micro-structure, Clue, Variability, Temporal 0
SafArchie [39,54] 2 Specify [5,9,11,13,14,15,24,25,27,28,35-37,39,43,45,47,50-56,58-60] 27 30 44 26
Aesop, SADL, Koala, xADL, ADML, AO-ADL, UniCon, Weaves, Wright, C2, Rapide, MetaH, AADL 0 Preserve [1,2,6-11,18-22,25,26,29-31,35,36,40,41,44,47,48,50,51,53-60] 35 37 37 26
Activity [2,28,43,53] 4 Intent Change [11,39,40,41,46,51,60] 7
State [2,5] 2 of Enforce [3,6,8,11,13,24,33,34,46,50] 97 10 10 40 50
Sequence [5,7,18,19,20,51] 6 Reasoning Match [4,12,17,40,42,49,50] 7
UML Class [1,4,7,8,9,11,17,19,21,25,26,31,33,35,39,48,51,52,60] 19 26 53 21 Discover [17,23,38,49] 4
55
Specification Component [2,6,7,8,18,20,28,30,35,43,46,50,51,53,54,57,60] 17 41 35 24 Analyze [3,5,6,11,14] 5
Object [2] 1 Consistency checking [1,6,9,10,11,12,21,23,30,39,40,46,48,60] 14 36 36 28
Communication [31,33] 2 Type of Model checking [2,3,5,22,24,27,29,31,32,37] 10 60 30 10
Transition [2,3] 2 Analysis Pattern conformance [4,17,41,54] 55 4
Structural [1-4,6-15,17,19-22,24-30,32-34,36,37,39-57,59,60] 52 28 42 30 Graph-based refinement [7,53] 2
Description
Behavioral [2,3,5,7,9,10,12,19,20,22,24,27,29,31-35,39-42,44-46,49-57] 93 34 41 32 27 Constraint enforcement [8,25,59] 5
Aspect
Semantic [43,58] 2

(i) Level of formalism. The main observation, which is (i) Means of constraint. Architectural styles, patterns,
biased because of our inclusion/exclusion criteria, is that component-level invariants, cross component invariants,
formal approaches dominate the ACSE area. (ii) Type of pre- and post-conditions which have an explicit description
formalism. Traditional modeling languages such as graph separable from the architecture of a system under
and model-based languages are the most popular. The consideration, are determinants of some characteristics of
majority of studies formulate the architectural constraints the system [13]. (ii) Intent of constraint. Specification,
corresponding to some properties of interest in OCL or a preservation and enforcement of constraints are the main
variation of logic as constraint specification language. We reasons behind constraints in the approaches. (iii) Analysis
observed that the majority of the studies either formulate of constraint. Most approaches are accompanied or
the properties of interest in the modeling language or they enabled by analytical mechanisms such as consistency
utilize a logic combined with a modeling language. checking, model checking, and conformance checking.
Recently, the creation of new languages has declined and We observed a trend towards quantitative verification
researchers tend to combine or extend existing languages of system properties based on constraints for correct
for their own purposes. (iii) Description language. UML adaptations, especially in self-adaptive systems which
extensions [12], especially component and class diagrams operate in uncertain environments that promote the use of
((iv) UML specification), are widely adopted by ACSE probabilistic modeling of systems.
approaches. Although UML has restricted architectural D. Run-time Issues
analysis support, its wide adoption is based on its support
for different domains and business contexts [12] and its This category compares the studies in terms of
standardization. Other description languages such as underlying environment and mechanism to enable dynamic
ACME, Darwin or SafArchie are approximately equally reconfiguration. It identifies the prerequisites for what is
needed to implement and utilize the often complex and or partially automated in either of them. Thus,
hypothetical conceptual contributions in the context of unsurprisingly, the majority of studies use tool support
reliable runtime infrastructures. The studies are classified either for proof of concept or automated analysis purposes.
into two sub-categories (Table 10).
(i) Environment of runtime evolution. A variety of F. Research methodology of the studies
execution environments has been proposed, ranging from This category differs from the other 5 purely technical
traditional platforms such as CORBA Component Model aspects and only reflects general aspects (Table 12).
(CCM) to the reflective and flexible Fractal and OpenCOM
to a new SOA and Cloud based platforms. (ii) Mean of TABLE 12. COMPARATIVE ANALYSIS BASED ON “RESEARCH METHOD”
Distribution %
Sub-
Attributes Approaches P% N
runtime evolution. To enable runtime evolution, these Criteria
Problem or Challenge [2,3,6,7,15,18,20,25,38,42,49,50,52,54,56-58] 17
95-
05
24
06-
08
41
09-
11
35
environments need to expose functions to provide the Overview or Survey
Unambiguous description
[8,16,32,38,49]
[4,9,11,14,19,22,26,27,30,35,45,47,52]
5
13 23 38 39
current state of a system at a specific time during its Motivation Analysis [4,5,9,12,14,17,23,28,33,35,40,49,55]
[1,6,10,13,18,20-22,24,26,28,29,31,33,34,36,37,39-41,44, 46-
98 13 15 46 39
Evolution 30 37 30 33
execution and a causal connection between the running 48,50,51,53,55,59,60]
A formalism to enable code generation 0

system and architectural information. Ideally, they provide SOA

OO
[1,2,7,31,43]
[1,3,8,17,21,26,41,42,48,49,51,57]
5
12 33 50 17

facilities to change architectural descriptions at runtime. Application

Domain
Distributed system
Event-based system
[3,24,46,60]
[5,22,44] 78
4
3
Component-based [6,10-13,22,24,25,28,30-37, 39,40,43,45-47,50,53-60] 32 38 34 28
SPL, Embedded, Ubiquitous, Mission-critical, Real-time, Process-aware, Concurrent, Mechatronic,
0
TABLE 10. COMPARATIVE ANALYSIS BASED ON “RUNTIME ISSUES” Mobile, Robotic, Cloud computing, Smart-*, Autonomic computing, Grid computing
Case study [1,4-9,12,14,15,18-20,22,24,29,35,37,38,40,43,46,47,51, 53,55,60] 27 29 30 41
Sub-Criteria Attributes Approaches P% N Evaluation Mathematical proof [1-3,5,17,19,21,22,29,47,54] 11 36 36 28
87
Fractal [30] 1 Method Example application [3,11,13,14,21,25,27-31,33,34,36,37,39,41,42,44,45,48-50,52,54,59] 26 31 42 27
OSGi [12] 1 Experience report 0
CCM [11] 1
Environment of
Run-Time Evolution
PKUAS
SOA middleware
[10]
[7]
10
1
1
(i) Motivation. The main focus of research is on
SIENA [5]
KOALA, MS COM, SOFA 2.0, EJB, JavaBeans,
1 formalisms with attention to enabling evolution in software
0
OpenCOM, KobrA, .NET, Reo, Linda, MANIFOLD
Reflection [10] 1
systems and enabling automated mechanisms to support
Mechanism of
Causal connection
Runtime binding
[5,9,11]
[7] 8
3
1
evolution. (ii) Application domain. The dominant
Run-Time Evolution
Introspection, Constraint injection, State transfer,
Intercession, Reification, Safe stopping
0 application domain is component software, accounting for
There is a lack of approaches for runtime environments more than 50% of applications. This portion is increasing.
and evolution. We suppose this is because implementations Services have also gained attention recently. These two
of evolution mechanisms on top of these technologies are domains require dynamic reconfiguration. In particular,
not feasible in research domains and researchers prefer to dynamic components and service reconfiguration are active
rather provide evidence or theoretically prove their areas of research, with particular attention for rigorous
contributions. Recent efforts identified an extensive list of approaches to support challenges in self-adaptive systems
run-time concerns – mainly by autonomic computing and dynamic selection and composition of elements.
communities [29]. Disciplined engineering approaches to Furthermore, dynamic domains such as robotics and smart
support decentralization and making contextual system cities have gained little attention. (iii) Evaluation method.
models accessible and machine process-able at runtime Although most approaches included some formal proofs as
[14] are still relevant concerns [29]. part of the evaluation, evidence is often obtained from
applying the approach to small case studies and examples.
E. Tool Support Only a few empirical studies have been undertaken; no
This category compares the included studies in terms of industrial evidence is reported. Lack of rigorous validation
infrastructure which support automation of ACSE. This and attention to application domains were major concerns
theme provides a narrow view of automation for ACSE. that we found in the included studies. These concerns and
Based on this, the studies are classified into three sub- domains offer areas for future research on ACSE.
categories (Table 11).
VII.FUTURE RESEARCH DIRECTIONS AND LIMITATIONS
TABLE 11. COMPARATIVE ANALYSIS BASED ON “TOOL SUPPORT” Based on the taxonomical classification and holistic
Distribution %
Sub-Criteria Attributes Approaches P% N 95-
05
06-
08
09-
11
comparison, we summarize research implications and
Creating
Documenting
[15,20,22,27,29,30,34,38,45]
[1,3,8,9,11,35,36,43,52,55,57]
9
11 18 55 27
future directions. We also discuss possible limitations as
Need for
Tool Support
Analyzing [1-3,5,6,8,12-14,17,23,33,38,49]
[3-7,9-11,18,21,24,25,28,31-33,36,37,39-42,44,
95
14 14 29 57 well as threats to validity of this SLR.
Evolving 35 40 43 17
46-48,50,51,53-56,58-60]
Business case
Simulation [7,31]
0
2
A. Research Implications
Dependence analysis [5,11] 2
Analysis
Usage of
Model checking [2,3,5,6,7,14,31,33,43,46]
30
10 40 30 30 This classification and comparison framework has
Conformance [4,10,12,15,40,47] 6
Tool Support
Inspection [20] 1 implications for researchers and practitioners as
Interface consistency 0
Full [4,21,30,41,54] 5 summarized with the ACSE dimensions in Table 12. For
Level of Partial [1-3,5-12,14,17,18,20,22,24,26-29,31,33,39,40,
Automation 43,44,48,49,50,51,53,57,58]
100 34 29 41 30 researchers, the classification framework provides a
Manual [13,15,16,19,23,25,32,34-38,42,45-47,52,55,56,59,60] 21 33 29 38
comprehensive view of different aspects to be considered
(i) Need for tool support. Although most of the studies when addressing an ACSE problem. The comparison of
use tools for architectural evolution, the trend shows that existing work highlighted several areas where extensive
employing tools for analysis purposes is gaining significant support is provided or gaps are pointed out, see Table 13.
attention. (ii) Analysis usage of tool support. The studies Implications for future research. Based on the
that use tools employ them mostly for model checking. systematic consolidation, we identified a number of areas
(iii)Level of automation. Most of the studies are either that lack a more rigorous research as emerging trends in
fully automated for both architecture evolution and analysis ACSE. The list (Table 13) is not exhaustive, but reflects
key challenges identified in the consolidation and large amount of information. For instance, for the 60
corresponds to the classification in Section VI. papers, it makes a collection with 160*60=9600 data points
(i) Evolution: In the context of more classical which are quite significant. As a result, the user can, for
Lehman’s law [34], we observe ever growing needs for example, query the database to find case studies on
adaptive evolution with primary focus on run-time industrial applications of petri-net based approaches for
architectural adaptation. An interesting observation is the runtime evolution. Moreover, this database can grow and be
emergence of ‘adaptation pattern’ [31] to support reuse in used as a knowledge-base for ACSE researchers, such as
evolution for dynamic adaptive software architecture. [28] for practitioners.
Mining software repositories to empirically analyze
changes and fostering them for reuse is also getting more TABLE 13. RESEARCH DIMENSIONS AND FUTURE IMPLICATIONS FOR ACSE
Research
attention. Researchers look at long running, large Needs Evolution Formalism
Architectural
Reasoning
Run-Time
Issues
Evaluation
applications such as Eclipse SDK [15] in order to further Dimensions
Self-adaptive Emerging Quantitative Decentralized Rigorous
Research Trend
this domain by learning from past recurring evolutions. systems properties verification evolution validation
Change patterns, Explicit constraints, Model@run-
(iii) Architectural reasoning: A considerable number Active Topics Repository
Formal
theory
Probabilistic time, Reflective
Empirical
studies
mining modelling environment
of approaches address probabilistic modeling of systems Self-reaction to Constraint Efficiency and
Semantic Real-world
and their environments which they operate and also explicit Targets violations of
constraints
models
preservation,
property verification
Reliability in
evolution
evidences
specification of architectural constraints. Analytic Focus Run-time Design-time Run-time Run-time Both

techniques verify specific properties or preserve the initial Limitations. To validate our model, we analyzed 60
constraints. However, these techniques that are normally peer-reviewed studies. The model was useful for
conceived as design-time activities must extend their scope characterizing the approaches since the average number of
to runtime and comply with the time constraints of runtime dimensions characterizing approaches is 15 according to
environment. We observe that the need for runtime column A, Table 5. However, we believe that neither the
verification and validation to detect violations and plan model nor the comparison is unchangeable. We restricted
self-reactions demands for efficient analytical and ourselves to a limited number of high quality approaches
syntactical “@runtime” techniques [2, 24]. with highest rank as it mentioned in Section IV, indicating
(iv) Run-Time evolution: In a considerable number of that our comparison is not comprehensive enough, though
approaches, formal specifications are applied during it is reliable to cover a variety of studies. We are currently
development activities for verification and validation, working on a more comprehensive comparison, including
rather than the system itself at runtime. More importantly, studies with lower ranks among the 154 ones ranked.
those approaches that utilize formal specification at runtime
use it for mainly modeling and analysis rather than other B. Threats to Validity
ACSE activities. We observe an improvement of adaptive In general, the external validity and construct validity
middleware based on reflection mechanisms. However, are strong for systematic reviews [22, 26, 3]. The main
decentralized evolution and @runtime notions are still threats to validity of this review are biased by our selection
relevant concerns for further research in ACSE. of studies to be included, data extraction and data synthesis.
(vi) Evaluation evidence: An important issue that we (i) To be able to identify relevant studies and ensure that
observed and is asserted by work by Barnes at NASA JPL the process of selection was unbiased, a research protocol
regarding space systems architectural evolution is the lack was developed. However, systematic reviews are per
of rigorous validation [21]. Previous work tended to make definition limited by search period, databases and
heavy use of trial examples and unrealistic assumptions and terminologies. ACSE research is related to different
has not been well supported by real-world scenarios. We communities including software architecture, software
came across this when excluding a large number of studies evolution and self-adaptive software which use different
that had been retrieved based on our search, but violated an terminologies. Therefore, to cover all and avoid bias, we
inclusion criterion (I2 in Table 3). To stimulate ACSE searched for common terms and combined them in our
community to produce empirical real-world case studies, search string. While this approach decreases the bias, it also
such efforts need recognition at established conferences [7]. significantly increases the search work. (ii) To ensure
Implications for practitioners. The data [25] collected correctness in data extraction, we defined a comprehensive
may help practitioners in searching for relevant approaches data extraction Excel sheet [25] to obtain consistent and
before adopting and tailoring them by examining the also relevant data. In addition, we performed quality
context of their own software development, and comparing assessment on the studies to ensure that the identified
with characteristics of relevant approaches. However, [28] findings and implications came from credible sources. (iii)
provides an overview of various approaches evaluated Another challenge of systematic reviews is addressing the
based on real-world industrial scenarios concerning reliability threats. The reliability is mitigated as far as
evolution of sustainable industrial systems. We believe that possible by involving several researchers, and having a
this document can suitably assist practitioners because it is unified scheme, and several steps where the scheme and
more general based on a growing number of experience process were piloted and externally evaluated. If the study
reports which might not necessarily be peer-reviewed. is replicated by another set of researchers, it is highly likely
Practical application of the collected database. Since that some included papers are changed. However, it is
this classification framework and its accompanying highly unlikely that random differences based on personal
templates contain more than 160 attributes, it provides a judgments would change findings. It may only change the
actual count numbers to some extent. Therefore, in general [10] Williams, B. J., Carver, J. C. 2010. Characterizing software
we believe that the validity of the study is high, given the architecture changes: A systematic review. IST 52(1): 31-51.
use of a systematic procedure and the involvement and [11] Cruzes, D.S, and Dyba, T. 2011. Recommended Steps for Thematic
discussion among four researchers and also external Synthesis in Software Engineering, ESEM.
evaluations. The openness of our review by exposing our [12] Medvidovic, N., Dashofy, E. M., and Taylor, R. N. 2007. Moving
data in [25] allows other researchers to judge the architectural description from under the technology lamppost. IST, 49.
trustworthiness of the results more objectively. This [13] Pahl, C., Giesecke, S., and Hasselbring, W. 2007. An ontology-based
initiative is suggested by the evidence-based software approach for modelling architectural styles, ECSA.
engineering community http://www.dur.ac.uk/ebse/. It is [14] Bencomo, N. 2009. On the Use of Software Models during Software
followed by some researchers as [7]. Execution. ICSE Workshop on Modeling in Software Engineering. IEEE.
[15] Wermelinger, M., Yu, Y., Lozano, A., Capiluppi, A. 2011. Assessing
VIII. CONCLUSIONS architectural evolution: a case study. Empirical Software Engineering,
The objective of this study was to consolidate existing 16(5), pp. 623–666.
research on architecture-centric software evolution [16] Zhang, H. 2010. A multi-dimensional architecture description
regarding the claimed benefits and the provided evidences language for forward and reverse evolution of component-based software,
PhD Dissertation.
of evolution. The main contribution of this study is a
classification framework for ACSE and a comparison of [17] Medvidovic, N., Taylor, R. N. 2000. A Classification and
Comparison Framework for Software Architecture Description
systematically selected studies through the framework to Languages. IEEE Trans. Software Eng. 26(1): 70-93.
point out existing research gaps. We identified unexplored
areas by synthesizing collected data, reflecting on areas of [18] Buckley, J., Mens, T. Zenger, M., Rashid, A., Kniesel, G. 2005.
Towards a taxonomy of software change. JSS 17(5): 309-332.
future research. The results of classification and
comparison are presented as structure tables – as a means to [19] Chapin, N., Hale, J. E., Kham, K. M., Ramil, J. F. and Tan, W.G.
2001. Types of software evolution and software maintenance. JSM 13(1).
transfer knowledge among ACSE researchers and
practitioners about a collective impact of existing research. [20] Zhang, P., Muccini, H., Li, B. 2010. A classification and comparison
of model checking software architecture techniques. JSS 83(5), 723–744.
We observed an inclination towards runtime adaptation
to serve self-adaptive applications. We also observed vast [21] Jeffrey M. Barnes. 2012. NASA’s Advanced Multi-mission
interests towards probabilistic modeling and quantitative Operations System: A case study in software architecture evolution. ACM
SIGSOFT Conference on the Quality of Software Architectures.
verification of system properties. Reflective environments
and Models@run-time are identified as the key drivers to [22] Petticrew M, Roberts H. 2006. Systematic reviews in the social
sciences: a practical guide. Oxford: Blackwell.
enable adaptations. However, evidences of rigorous
validation were lacking in existing evidences in ACSE. [23] ISO/IEC/IEEE 42010:2011, Systems and software engineering-
Architecture description, http://www.iso-architecture.org/ieee-1471/.
ACKNOWLEDGMENTS [24] Calinescu, R. Ghezzi, C., Kwiatkowska, M., Mirandola R. 2012. Self-
Adaptive Software Needs Quantitative Verification at Runtime.
The authors would like to thank the following persons
for their feedback and thoughtful suggestions regarding the [25] Jamshidi, P., Ghafari, M., Aakash, A., Pahl, C. A Framework for
Classifying and Comparing Architecture-Centric Software Evolution
methodology, data and the final report: Jim Buckely, Research [Online: Auxiliary Review Material],
Jeffrey M. Barnes, Fereidoon Shams, Mehdi Fahmideh and http://www.computing.dcu.ie/~pjamshidi/SLR-ACSE.html.
Frank Fowley. This work was supported, in part, by [26] Brereton, P., Kitchenham, B., Budgen, D., Turner, M., Khalil M.
Science Foundation Ireland grant 10/CE/I1855 to Lero - the 2007. Lessons from applying the systematic literature review process
Irish Software Engineering Research Centre (www.lero.ie). within the software engineering domain, JSS 80.
[27] Baresi, L., Di Nitto, E., Ghezzi, C. 2006. Toward open-world
REFRENCES software: Issue and challenges. Comput. 39(10), 36–43.
[1] Mens, T., Magee, J., and Rumpe, B. 2010. Evolving software [28] Stammel, J., Durdik, Z., Krogmann, K., Weiss, R. and Koziolek, H.
architecture descriptions of critical systems. Computer, 43(5): 42–48. 2011. Software Evolution for Industrial Automation Systems: Literature
[2] Tamura, G., at al. 2012. Towards practical runtime verification and Overview, Karlsruhe Reports in Informatics.
validation of self-adaptive software systems. Springer. [29] Weyns, D., Andersson, J., Malek, S., Schmerl, B. 2012. Introduction
[3] Zhang, H., Babar, M.A. 2012. Systematic Reviews in Software to the special issue on state of the art in engineering self-adaptive systems,
Engineering: An Empirical Investigation, IST. JSS.
[4] Taylor, R. N., Medvidovic, N., and Dashofy, E. M. 2009. Software [30] Baresi, L., Ghezzi, C. 2010. The disappearing boundary between
architecture: Foundations, theory, and practice. John Wiley & Sons. development-time and run-time. FSE/SDP workshop, 17-22.
[5] Silva, L., and Balasubramaniam, D. 2012. Controlling software [31] Aakash, A., Jamshidi, P., Pahl, C. A Classification and Comparison
architecture erosion: A survey, JSS, vol. 85, no. 1, pp. 132–151. of Software Architecture Evolution Reuse-Knowledge [Online: Auxiliary
Material], http://www.computing.dcu.ie/~pjamshidi/SLR/SLR-ERK.html.
[6] Breivold, H. P., Crnkovic, I., Larsson, M. 2011. A systematic review
of software architecture evolution research, IST, 54(1), 16-40. [32] Jamshidi, P., Ghafari, M., Aakash, A., Pahl, C. 2012. A protocol for
systematic literature review on Architecture-Centric Software Evolution
[7] Weyns, D., Iftikhar, M., Iglesia, D., Ahmad, T. 2012. A Survey on
Research, Technical Report.
Formal Methods in Self-Adaptive Systems. FMSAS.
[33] Ardagna, D., Ghezzi, C., Mirandola, R. 2008. Rethinking the use of
[8] Vestal, S. 1993. A Cursory Overview and Comparison of Four
models in software architecture. Quality of Software Architectures, 1-27.
Architecture Description Languages. Honeywell Technology Centre.
[34] Lehman, M. 1996. Laws of software evolution revisited. Software
[9] Clements, P. 1996. A survey of architecture description languages,
process technology, 108-124.
Eighth International Workshop Software Specification and Design.