See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/268486309

Software Quality Requirements: A Systematic Mapping Study

Conference Paper · December 2013

DOI: 10.1109/APSEC.2013.40

CITATIONS READS
19 910

4 authors, including:

Ali Idri José Luis Fernández-Alemán

Mohammed V University in Rabat/Mohammed VI Polytechnic University University of Murcia
386 PUBLICATIONS 7,871 CITATIONS 179 PUBLICATIONS 5,765 CITATIONS

SEE PROFILE SEE PROFILE

Ambrosio Toval
University of Murcia
195 PUBLICATIONS 4,860 CITATIONS

SEE PROFILE

All content following this page was uploaded by Sofia Ouhbi on 11 June 2015.

The user has requested enhancement of the downloaded file.

 2013 20th Asia-Pacific Software Engineering Conference

Software Quality Requirements: a systematic mapping study

Sofia Ouhbi, Ali Idri Jose Luis Fernández-Alemán, Ambrosio Toval
Software Project Management research team Dept. Informatica y Sistemas
ENSIAS, University Mohammed V Souissi University of Murcia
Rabat, Morocco Murcia, Spain
ouhbisofi[email protected], [email protected] [email protected], [email protected]

Abstract—Software quality requirements (SQR) play a central statements of services the system should provide, how the
role in software quality (SQ) success. The importance of mas- system should react to particular inputs and how the system
tering SQR can be seen as obvious; however, when it comes to should behave in particular situations [5]. Non-functional
customer satisfaction, end-users are often dissatisfied with SQ.
In this paper, a systematic mapping study aims to summarize requirements delineate requirements focusing on how good
SQR research by answering nine mapping questions. In total, 51 software does something unlike the functional requirements,
articles were selected and classified according to multiple criteria: which focus on what the software does [6]. Requirements en-
publication source, publication year, research type, research gineering (RE) aims to obtain clear understanding of the prob-
approach, contribution type of SQR literature, requirements lem domain and customer’s need, and produce complete and
engineering activity, well-known SQ model, software artifact
and SQR type. The results show an increased interest in SQR correct SQ [7]. Standardization organizations have published
research in recent years and reveal that conferences are the standards for SQR such as the IEEE Std 1465-98 [8] which
main SQR publication target. Most SQR research has used case is a guideline for SQR in software packages, the ISO/IEC
studies. The dominant contribution type of SQR research is 25000:2005 standard [9] and the ISO/IEC 25001:2007 stan-
method while specification is the main requirements engineering dard [10]. ISO/IEC 25030:2007 [11] is a guideline for SQR
activity identified. SQ models need to be more used for SQR
identification. Design module and requirements documentation and evaluation. It applies the quality model defined in ISO/IEC
are the principal artifacts reported in SQR literature. External 9126-1 [4] and it complies with the requirement processes
and internal SQR were the main SQR types addressed in defined in ISO/IEC 15288 [12]. According to the ISO/IEC
literature. Identifying empirical solutions to address SQR is a 25030:2007 standard, a target value of the SQ measure repre-
promising research direction for researchers. sents a SQR.
Keywords-Software Quality Requirements; Requirement Engi- In order to obtain a detailed view on SQR research, a
neering; Systematic mapping study. systematic mapping study is conducted to identify and classify
SQR into: publication source, publication year, research type,
I. I NTRODUCTION research approach, contribution type, RE activity, well-known
"Although quality is pursued in order to satisfy users, it SQ model, software artifact and SQR type. To the best of our
is important to note that the quality desired by users is knowledge, no systematic mapping study of the SQR research
not universal but rather subject to constant change." [1]. exists. The results were analyzed, tabulated and synthesized
Developing a software product that responds to the customers’ to provide both an updated and summarized view and a set of
requirements, and offers high value to both the development recommendations for researchers and practitioners. The results
company and the customer, increases the likelihood of market of this study may be a substantial starting point for systematic
success. Therefore, software quality requirements (SQR) play literature reviews (SLRs) in the area of empirical solution to
a central role and can be seen as a key competitive advantage address SQR.
[2]. The international research standardization project SQuaRE The structure of this paper is as follows: Section II presents
(Software Quality Requirements and Evaluation) developed the research methodology. Section III reports the systematic
the ISO/IEC standard series 250nn for software product mapping results. Section IV discusses the finding of this
quality, evaluated from the users and stakeholder’s point of study and implications for researchers and practitioners. The
view. The ISO/IEC 25010 standard [3], which has replaced conclusions and future work are presented in Section V.
the ISO/IEC 9126-1 [4], defines internal, external quality
characteristics and quality in use characteristics. It should be II. R ESEARCH METHODOLOGY
noted that "user requirements and expectations are growing
increasingly diverse, and the importance given to each quality A mapping study differs from an SLR [13]. The SLR
characteristic varies with individual users (and stakeholders). aims to establish the state of evidence, focuses on identifying
In some cases, what is desirable to one user is not so to best practices and shows where particular evidence is missing
another" [1]. or is insufficiently reported in existing studies. While the
Software quality (SQ) could be defined as "the totality of main focus of a systematic map is mainly on classification,
characteristics of an entity that bear on it’s ability to satisfy conducting a thematic analysis and identifying publication fora
stated and implied needs" [4]. Functional requirements are [14].

1530-1362/13 $31.00 © 2013 IEEE 231

DOI 10.1109/APSEC.2013.40
 Table I: Systematic mapping questions
ID Question Main motivation
MQ1 Which publication sources and channels are the main targets for To identify where SQR research can be found as well as good targets for
SQR research? publication of future studies.
MQ2 How has the frequency of SQR research changed over time? To identify the SQR research publication trend over time and to compare it
to the SQR empirical research trend.
MQ3 What are the research types of SQR studies? To explore the different research types reported in SQR literature.
MQ4 What are the research approaches of SQR studies? To discover the empirical approaches used in the validation of SQR research.
MQ5 What are the contribution types of SQR research? To discover the contribution types of SQR research.
MQ6 What are the RE activities that were addressed by SQR research? To discover the main RE activity addressed in SQR research.
MQ7 What are the well-known SQ models that were used for specify- To identify if the selected studies use well-known SQ models as sources to
ing QR? identify and specify SQR.
MQ8 What are the artifacts that were reported in SQR literature? To identify the list of artifacts that were reported in SQR research.
MQ9 What are the most reported SQR type in literature? To identify which one of external, internal or quality in use is the most SQR
type reported in literature.

A. Research questions
The systematic map was performed to obtain the current
research on SQR. This study answers nine mapping questions
(MQs). These questions are presented in Table I with their
corresponding motivations.
Figure 1: Selection process
B. Search strategy
The articles were identified by consulting the following SQR.
sources: IEEE Digital Library, ACM Digital Library, Science EC5 Papers that focus only on functional requirements.
Direct and Google scholar. The PICO (population, interven-
Fig. 1 shows the search process result. 51 studies were
tion, comparison, and outcome) [15] criteria were taken into
selected out of 657 candidate studies.
account when we formulated our search term. However, as the
objective of this mapping study is not to find evidence about D. Data extraction strategy
SQR research, the "Comparison" part of the PICO criteria
The data extraction strategy was based on providing the set
was excluded. The population considered is that of SQR
of possible answers to MQs.
research; our intervention was solutions to address SQR; and
MQ1. To answer this question, publication source and channel
all existing outcomes regarding SQR research are of interest
for each paper should be identified.
in this mapping study. The following search string was used in
MQ2. To draw the publication trend, articles should be clas-
order to perform the automatic search in the digital libraries se-
sified per publication year.
lected: ("Software" OR "Application*") AND "Quality" AND
MQ3. A research type can be classified into the following
("Requirement*" OR "Attribute*" OR "No* functional" OR
categories [13]:
"Need*") AND ("Engineer*" OR "Elicitat*" OR "Analys*"
OR "Specif*" OR "Validat*" OR "Process*"). • Evaluation research: existing SQR approaches are im-
plemented in practice and an evaluation of them is
C. Study selection conducted.
• Solution proposal: a solution for SQR is proposed. This
solution may be a new SQR approach or a significant
The aim of the selection process is to identify the most rel- extension of an existing approach.
evant studies for the mapping study. Each paper was retrieved • Systematic investigation: an analysis which takes into
and evaluated by one author who decided whether it should account known aspects of a SQR by studying sources
be included, by considering its title, abstract and keywords. to establish facts and reach new conclusions.
The final selection result was reviewed and approved by the • Other, e.g. opinion paper, experience paper.
remaining authors. The first step after the articles had been MQ4. The research approach can be classified into [16], [17]:
identified was to eliminate duplicate titles, and titles which are • Case study: an empirical inquiry that investigates an SQR
clearly not related to the review. The inclusion criteria were approach within its real-life context.
limited to the studies that have SQR as topic and the studies • Survey: a method for collecting quantitative SQR infor-
that met at least one of the following exclusion criteria (EC) mation, e.g. a questionnaire.
were excluded: • Experiment: an empirical method applied under con-
EC1 Papers that were published before 1990 and after 2012. trolled conditions to observe its effects on SQR.
EC2 Papers presenting a general focus on SQ. • Theory: non-empirical research approaches or theoretical
EC3 Papers presenting a general focus on RE. evaluation of an SQR approach.
EC4 Papers that focus only on system QR and do not discuss • Review: an analysis of SQR existing literature.

232
 MQ5. A contribution can be classified into [14], [18]: this study, which is usually conducted in SLRs. In a mapping
• Method: a manner of procedure and a series of steps taken study the articles are not evaluated regarding their quality as
to acquire knowledge in SQR. the main goal is not to establish the state of evidence [14].
• Tool: a means to accomplish a task or purpose in SQR.
• Model: a representation of a system that allows for SQR III. M APPING RESULTS
properties to be investigated. This section describes the systematic mapping results pre-
• Guideline: an indication of policy by which a course of sented in Table II. Some studies were chosen to illustrate
action in SQR can be determined. examples of the MQs’ results. They represent some of the
• Framework: a real or conceptual structure intended to
empirical solution proposals identified in this study, which we
serve as a guide for the building of SQR. consider as relevant.
• Technique: a procedure with which an SQR task is
accomplished, e.g. data mining technique. A. MQ1. Publication source
• Other, e.g. process, metric.
Table II presents the publication source for SQR research.
MQ6. RE activity can be categorized into [19]: 43% of SQR selected studies were published in conferences,
• Requirements process. If the paper discusses more than 29% in journals, 20% in workshops and 4% in symposium. A
one RE activity in detail. book and a PhD thesis were also selected. The main journal
• Requirements elicitation. If the paper reports how and
target for SQR research is Requirements Engineering Journal,
where to collect RE. followed by IEEE Software Journal. Computer Software and
• Requirements analysis. If the paper discusses require-
Applications Conference Workshops was the main workshop
ments classification, conceptual modeling, architectural
target for SQR literature while IEEE International Require-
design and requirements allocation.
• Requirements specification. If the paper shows the pro-
ments Engineering Conference is the main conference target
cedure of the production of a document, which can be for SQR. Notice that there is no publication source where more
systematically reviewed, evaluated, and approved. than three SQR studies are published.
• Requirements validation. If the paper presents the process
B. MQ2. Publication trend
used to examine the requirements documents to ensure
that the the right system is being defined. Fig. 2 shows the publication trend of the SQR selected
papers from 1990 to 2012. This figure shows also the trend of
MQ7. A well-known SQ model [20] can be classified into:
empirical studies identified in this study. SQR publications
McCall model [21], Boehm model [22], Dromey model [23],
trend was categorized by a discontinuity in the nineties.
ISO/IEC 9126 standard [4], the ISO/IEC 25010 standard [3],
However, in the beginning of the last decade the trend was
or other.
stable with a poor number of publications. SQR research has
MQ8. An artifact can be classified into [19], [24]: software
gained interest since 2007. Notice that the empirical trend is
requirements specification (SRS), documentation, design mod-
almost parallel to SQR trend in many time slots between 1990
ule, source code or other.
and 2012, which indicates that the majority of SQR research
MQ9. According to the ISO/IEC 25030 standard [11], there
is empirically validated.
are three types of SQR:
• SQ in use requirements: they are normally used for
software validation to check if the software is fit for
its intended purpose. They reflect the user’s view of the
quality of the software product when it is used in a
specific environment and a specific context of use.
• External SQR: they are used for software validation and
verification to check if the software is built according to
specifications. They reflect the quality when the software
is executed.
• Internal SQR: they are normally used for quality monitor- Figure 2: Publication trend of the selected studies
ing, control during development, and to specify properties
of interim products.
C. MQ3 and MQ4. Research types and approaches
E. Synthesis method Table III presents the research types and research ap-
The synthesis method is based on counting the primary proaches of the selected studies. 53% of the SQR selected
studies that are classified in each of the MQs responses. A studies were solution proposals, 29% evaluation research and
variety of evaluation approaches (such as bubble chart and 18% systematic investigation. 18% were theoretical studies
frequency table) have been used in the analysis of the results and 18% were reviews. Around 65% of the selected studies
and a narrative summary with which to recount the principal were empirical validations. They are highlighted in gray in
findings of this systematic map and describe them is presented Table III. Around half of the selected studies were validated
in the discussion. No quality assessment was carried out in by case studies, 12% by experiments and only one survey

233
 Table II: List of primary papers used in this study and their mapping results
Paper P. Year P. Source P. Channel Research type Research approach Contribution type RE activity Artifact Model SQR type
[25] 1990 IST Journal Systematic investigation Review Method Specification Documentation No Internal
[26] 1993 METRICS Symposium Systematic investigation Review Model Specification SRS Boehm External
[27] 1994 EMC Conference Systematic investigation Review Method Analysis Design module No External
[28] 1994 STRQA Conference Evaluation research Case study Method Specification SRS No External
[29] 1994 ICSQ Conference Solution proposal Theory Framework Specification Design module No Internal
[30] 1996 RE Conference Solution proposal Theory Method Analysis Design module Other Internal
[31] 1998 JSS Journal Solution proposal Theory Technique Process Design module No SQ in use
[32] 1999 SQP Journal Systematic investigation Review Method Specification SRS No SQ in use
[33] 2000 QWE Conference Evaluation research Case study Model Specification Design module Other SQ in use
[34] 2001 HICSS Conference Evaluation research survey Method Process Design module ISO/IEC 9126 External
[35] 2002 SEKE Conference Solution proposal Case study Model Process Design module No External
[36] 2003 SwSTE Conference Solution proposal Case study Model Specification SRS No External
[37] 2003 REFSQ Workshop Solution proposal Case study Method Elicitation Design module No Internal
[38] 2003 ICSSEA Conference Evaluation research Case study Tool Process Documentation No Internal
[39] 2003 JOT Journal Systematic investigation Review Method Specification Documentation ISO/IEC 9126 Internal
[6] 2004 REFSQ Workshop Systematic investigation Review Method Process Documentation ISO/IEC 9126 All
[40] 2005 CRIWG Conference Solution proposal Experiment Tool Validation Design module ISO/IEC 9126 External
[41] 2006 IEEE LAT Journal Solution proposal Theory Method Specification Documentation ISO/IEC 9126 Internal
[42] 2006 ECSA Conference Solution proposal Experiment Method Analysis Design module No Internal
[43] 2006 AE Journal Systematic investigation Review Framework Process Documentation ISO/IEC 9126 All
[44] 2007 SSD Symposium Solution proposal Theory Framework Specification Documentation No Internal
[45] 2007 SOQUA Workshop Solution proposal Experiment Method Specification SRS No External
[46] 2008 RE Conference Solution proposal Case study Method Specification SRS No External
[47] 2008 WoSQ Workshop Solution proposal Case study Model Elicitation Documentation ISO/IEC 9126 Internal
[48] 2008 REJ Journal Solution proposal Case study Method Process Design module ISO/IEC 9126 Internal
[49] 2008 ICSE Conference Evaluation research Case study Method Elicitation Design module No Internal
[50] 2008 IEEE Soft. Journal Systematic investigation Review Method Specification Design module No External
[51] 2009 QSIC Conference Solution proposal Experiment Method Elicitation SRS No External
[52] 2009 WoSQ Workshop Solution proposal Case study Method Specification SRS No External
[53] 2009 Book Book Systematic investigation Review Method Process Documentation ISO/IEC 9126 All
[54] 2009 IACC Conference Evaluation research Case study Technique Analysis Design module No Internal
[55] 2010 SEDM Conference Solution proposal Theory Technique Specification SRS No External
[56] 2010 IWSPM Workshop Evaluation research Case study Model Specification Design module No External
[57] 2010 ESWA Journal Solution proposal Case study Technique Analysis Documentation ISO/IEC 9126 Internal
[58] 2010 JOT Journal Solution proposal Case study Model Specification Design module ISO/IEC 9126 Internal
[59] 2010 COMPSAC Conference Solution proposal Theory Method Analysis Design module No Internal
[60] 2010 IEICE TIS Journal Solution proposal Experiment Method Specification SRS ISO/IEC 9126 External
[61] 2011 SEW Workshop Evaluation research Case study Tool Specification SRS No External
[62] 2011 RE Conference Evaluation research Case study Method Process Documentation No SQ in use
[63] 2011 TSE Journal Evaluation research Case study Method Process Documentation ISO/IEC 9126, McCall External
[64] 2011 ICIII Conference Evaluation research Case study Model Analysis Design module No SQ in use
[65] 2011 COMPSACW Workshop Solution proposal Experiment Method Specification Documentation ISO/IEC 9126 Internal
[66] 2011 IEEE Soft. Journal Solution proposal Theory Guidelines Elicitation Design module ISO/IEC 9126 Internal
[67] 2012 COMPSACW Workshop Solution proposal Case study Technique Analysis Design module No SQ in use
[68] 2012 DBKDA Conference Solution proposal Theory Model Process Design module No External
[69] 2012 REJ Journal Evaluation research Case study Method Elicitation Design module No Internal
[70] 2012 COMPSACW Workshop Evaluation research Case study Method Specification SRS ISO/IEC 9126 External
[71] 2012 SEAA Conference Evaluation research Case study Method Elicitation Documentation ISO/IEC 9126 All
[72] 2012 REJ Journal Solution proposal Case study Model Specification Design module No Internal
[73] 2012 PhD Thesis Other Solution proposal Case study Method Process Documentation ISO/IEC 9126 External
[74] 2012 ICCGI Conference Evaluation research Case study Tool Specification SRS No External

Table III: MQ3 and MQ4 results

were empirically validated. All the identified techniques are
Evaluation Solution Systematic data mining techniques. Fig. 3 also shows that requirements
Total %
research proposal investigation
[28], [33], specification is the main RE activity addressed in SQR re-
[35]–[37],
[38], [49], search. Requirements process, analysis, elicitation are also ad-
[46]–[48],
[54], [56],
Case study [52], [57], - 26 51.0% dressed by SQR research. The majority of papers that discuss
[61]–[64],
[58], [67],
[69]–[71], requirements process, elicitation and analysis conduct case
[72], [73]
[74]
[40], [42], studies to validate their contributions. Case studies and experi-
Experiment - [45], [51], - 6 11.8% ments were mainly conducted in the selected papers to validate
[60], [65]
Survey [34] - - 1 2.0% methods and requirements specification, which are targeted
[29]–[31], principally by solution proposals. An SQ function deployment
[41], [44],
Theory -
[55], [59],
9 17.6% negotiation tool for requirements validation named MEG was
[66], [68] proposed by Ramires et al. [40]. MEG was evaluated in two
[6], [25]–[27],
Review - - [32], [39], 9 17.6% pilot experiments involving two stakeholders each. Castillo
[43], [50], [53] et al. [58] propose the REASQ (REquirements, Aspects and
Total 15 27 9 51
% 29% 53% 18% - -
Software Quality) conceptual model, expressed in UML. Sen
and Barach [57] present a fuzzy quality function deployment
approach for determining which of the non-functional require-
was used in an evaluation research. No experiment has been ments reported by earlier studies are important to a company’s
carried out in the evaluation research. 93% of SQR evaluation software selection decision based on and integrated with its
research have used case studies. No survey was conducted for functional requirements. Kaiya et al. [60] propose a method
SQR solution proposals validation. called "spectrum analysis for QR" which enabled analysts to
sort a requirements specification to measure and track QR in
D. MQ5 and MQ6. Contribution types and RE activities the specification using Term Characteristic Map. They discuss
Fig. 3 shows that 55% of the SQR selected studies con- what is written in an SRS and use Quality factor in ISO/IEC
tribution are methods. 20% are models, 10% are techniques, 9126 for their catalog. Through several experiments, they
8% are tools and 6% are frameworks. Only one paper which evaluated their method.
proposes guidelines was identified. All the identified tools

234
 Figure 3: MQ3, MQ4, MQ5 and MQ6 results.

E. MQ7. SQ well-known models by data modeling [35], [48], [58], [68]. 29% of the selected
Fig. 4 shows that only 38% of the solution proposals used studies discuss documentation, 73% of them are requirements
a well-known SQ model to specify SQR. The majority of documentations. Technical documentations in [41], [57], web-
the selected studies have used the ISO/IEC 9126 standard based documentation in [38] and a quality report in [47] were
to specify SQR. McCall model has been used by only one identified. The artifact source code was not addressed by the
paper, the same for Boehm model. Dromney model and the selected studies. SRS were addressed in 26% of studies. 41%
ISO/IEC 25010 standard have not been used by the selected of the selected studies discuss external SQR, 39% of these
studies to identify SQR. The standards IEEE Std 1061-1992 studies discuss internal SQR and 12% address SQ in use
and ISO/IEC 14598-1 have been identified respectively in requirements. The word "All" in Fig. 5 refers to the three
[30] and [33]. Herrmann and Paech [48] present the MO- types. 8% of the selected studies discuss all SQR types. SRS
QARE (Misuse-Oriented QuAlity Requirements Engineering) was identified mainly in the studies that discuss external SQR,
method, which aims to support intuitive and systematic iden- while design module and documentation were reported mainly
tification of QR. This method provides a general conceptual in studies that discuss internal SQR. 67% of the papers that
model of QR, and a checklist-based process for deriving address SQ in use focus on design. Wei et al. [67] present an
them in a top-down fashion using the ISO/IEC 9126 standard. automated reasoning technique to select design alternatives.
MOQARE has shown its merits in a case study applied to the They have applied this technique to analyze the security re-
"Uveitis Database" used at the Interdisciplinary Uveitis Center quirements of an enterprise financial service system supporting
Heidelberg. Kaiya et al. [65] propose an improved method by equity trading in USA, Europe and Australia. Svensson et al.
using analyses records of systems similar to the one to be [72] present the QUality PERformance (QUPER) model that
analyzed. The inputs of the analyzer are a requirements list estimates quality targets in relation to market expectations as
and a catalog of QR derived from ISO 9126. a basis for the architecting design of QR. The QUPER model
was evaluated with a case study. The results show that the
F. MQ8 and MQ9. Software artifacts and SQR types model is useful for supporting early requirements specification.
Fig. 5 presents the software artifacts and the SQR types Salger et al. [52] set up an inspection method to assess SRS,
identified in the selected studies. Design module artifact is re- called "specification quality gate" (QG-Spec) at Capgemini
ported by 45% of SQR research, 17% of them were concerned sd&m. The QG-Spec has been applied to a series of large scale

Figure 4: SQ models in the selected studies Figure 5: MQ7 and MQ8 results

235
commercial projects. The results show that inspections have to designers should be able to analyze and evaluate the effects
be carefully balanced with techniques for constructive quality of various design decisions on the achievement of system
assurance in order to economically arrive at high quality SRS. requirements at an early state [76]. SQRs play a critical role
in driving architectural design and are an important issue in
IV. D ISCUSSION software development. Therefore, they need to be considered,
This section summarizes the principle findings of this specified, and accurately quantified earlier during system
systematic mapping study and presents their implications for analysis [72]. Requirements documentations and SRS were
SQR researchers and practitioners. also artifacts identified in the selected studies. The success
of a software project widely depends on the quality of SRS
A. Principle findings documentation, which serves as an input to the design, coding
The main findings of this study are the following: and testing phases [45]. Source code artifact has not been
The main targeted publication channels were proceedings identified in the selected studies, which could be explained
and 45 publication sources for SQR research were identified, by the fact that the end-user is more interested on both the
which could be explained by the fact that SQR covers dif- functionalities offered by the software product and the non-
ferent SQ areas. SQR has gained more interest since 2007. functional requirements (such as cost, time and quality) than
Maybe, due to the publication of the ISO/IEC 25030 [11] the code details. External SQR are mainly used in the selected
standard in 2007 which has impacted the SQR research interest studies to check if the software is built according to SRS.
since that year. More than the half of the selected papers Internal SQR were identified in the papers that deal with
propose solutions to address SQR. However 33% of them were SQR during the development phase. Although, SQ in use
theoretical suggestions. Our study found that most solution requirements were discussed in a few papers, they are very
proposals related to SQR are based on case studies, which critical to the software product success as they are related to
means that researchers tend to discover the problems faced the application of the software in its operational environment
in practice rather than propose theory-based solution. Almost [4]. According to the quality requirement life cycle presented
all evaluation research were validated through case studies. in the ISO/IEC 25030 standard [11], SQ in use may imply
Case study aims usually to track a specific attribute or identify external quality which again may imply internal quality.
relationships between different attributes [75] which is the case
of the majority of the SQR identified studies. B. Implications for researchers and practitioners
Methods were the most identified approach which address This study aims to give an overview of SQR research
SQR selected studies. Models, frameworks, guidelines, a tool in order to identify existing gaps and to establish future
and data mining techniques were also identified. Data mining research directions. SQR is an important topic and needs to
techniques have been used in the three last years [54], [55], be more explored by researchers and practitioners. Theoretical
[57], [67] and are a promising technique to automate and to solutions need to be applied in practice and validated empir-
analyze accurately the quality of huge amount of requirements. ically. There is a need for more studies focusing on the non-
Requirements specification is the main RE activity addressed functional requirements validation. There is also a need for
by SQR research. Requirements specification is considered more studies which identify and specify SQR according to a
the most important stage in the software lifecycle because chosen SQ model. Researchers should get familiarized with
constructed requirements affect all other stages of the lifecycle, ISO/IEC 25010 standard, which has replaced the ISO/IEC
and thus affect SQ [44]. There is a lack of interest in 9126 standard. Moreover, SQ in use requirements need to
SQR validation, which could be explained by the fact that be more explored by SQR researchers. The results presented
non-functional requirements may be difficult to analyze and in this study provide an overview to practitioners of the
validate. Non-functional requirements must be analyzed to the existing solutions which address SQR in literature. Moreover,
point where they can be expressed quantitatively [19]. Half empirical approaches were identified and can give an overview
of the selected studies have not used an SQ model in the of the maturity of the proposed SQR solutions. Furthermore,
definition of SQR. The quality model serves as a framework practitioners can choose the studies which propose solution to
to ensure that all aspects of quality are considered from RE activities for a specific SQR type and a specific software
the internal, external and quality in use point of view [11]. artifact.
ISO/IEC 9126 [4] standard was the most well-known SQ
model identified in SQR selected studies. The SQ measures de- V. C ONCLUSIONS AND FUTURE WORK
fined in ISO/IEC 9126 [4] standard are useful for formalizing This paper presents the results of a systematic mapping
the stakeholder SQR [11]. These measures may be used as they study on SQR. A total of 657 candidate studies were identified,
are or adopted to specific organization needs. The ISO/IEC of which 51 were selected as primary studies for the systematic
25010 standard has not been identified in the selected papers; mapping study. Data were extracted from these papers and
this could be due to the fact that this standard was published then synthesized against the defined MQs. Our results showed
in 2011. that an increasing amount of attention has been paid to SQR
Design was the most reported artifact in the selected studies. since 2007 and that the main targeted publication channels
In order to build a high quality software system, the software were conferences and journals. Solution proposal is the most

236
research type reported in SQR literature and the main empir- [14] K. Petersen, R. Feldt, S. Mujtaba, and M. Mattsson, “Systematic map-
ping studies in software engineering,” in 12th International Conference
ical approach used in the evaluation of SQR contributions is on Evaluation and Assessment in Software Engineering. Bari, Italy:
case study. Method is the most reported contribution type of Blekinge Institute of Technology, 2008, pp. 71–80.
SQR research and requirement specification was the main RE [15] B. A. Kitchenham and S. Charters, “Guidelines for performing
systematic literature reviews in software engineering,” Keele University
activity addressed in SQR literature. ISO/IEC 9126 standard and University of Durham, Tech. Rep., 2007.
is the most well-known SQ model used for the identification [16] N. Condori-Fernandez, M. Daneva, K. Sikkel, R. Wieringa, O. Dieste,
of SQR in the third of the selected studies. Design module, and O. Pastor, “A systematic mapping study on empirical evaluation of
software requirements specifications techniques,” in 3rd International
requirements documentation and SRS are the principal arti- Symposium on Empirical Software Engineering and Measurement, ser.
facts reported in SQR research. External and internal SQR are ESEM ’09. Washington, DC, USA: IEEE Computer Society, 2009, pp.
the main SQR types addressed in the selected studies. The 502–505.
results of this study may support practitioners in selecting [17] P. Tonella, M. Torchiano, B. Du Bois, and T. Systä, “Empirical studies in
reverse engineering: state of the art and future trends,” Empirical Softw.
worthwhile contributions and present to researchers future Engg., vol. 12, no. 5, pp. 551–571, Oct. 2007.
research directions. For future work, we will conduct an SLR [18] Z. A. Barmi, A. H. Ebrahimi, and R. Feldt, “Alignment of requirements
about SQR empirical research based on the findings of this specification and testing: A systematic mapping study,” in 2011 IEEE
Fourth International Conference on Software Testing, Verification and
study. Validation Workshops, ser. ICSTW ’11. Washington, DC, USA: IEEE
Computer Society, 2011, pp. 476–485.
ACKNOWLEDGMENT [19] A. Abran and J. W. a. Moore, Guide to the software engineering
body of knowledge (SWEBOK), C. Los Alamitos, Ed. IEEE Computer
This research is part of the project PEGASO-PANGEA Society, 2004. [Online]. Available: http://www.swebok.org/
(TIN2009-13718-C02-02) financed by the Spanish Ministry [20] M. Ortega, M. Pérez, and T. Rojas, “Construction of a systemic quality
model for evaluating a software product,” Software Quality Control,
of Science and Innovation (Spain), and part of the GEODAS- vol. 11, no. 3, pp. 219–242, Jul. 2003.
REQ project (TIN2012-37493-C03-02) financed by the Span- [21] J. A. McCall, Quality Factors. John Wiley & Sons, Inc., 2002.
ish Ministry of Economy and competitiveness and also part [22] B. W. Boehm, J. R. Brown, H. Kaspar, and M. Lipow, Characteristics of
software quality, ser. TRW Softw. Technol. Amsterdam: North-Holland,
of the project Software Project Management using DataMin- 1978.
ing Techniques (AP2010-2013), financed by Mohammed V [23] R. G. Dromey, “Cornering the Chimera,” IEEE Softw., vol. 13, no. 1,
Souissi University (Morocco). The mobility grant of Sofia pp. 33–43, 1996.
[24] S. Rastkar, G. C. Murphy, and G. Murray, “Summarizing software
Ouhbi is financed by the Mediterranean Office for Youth. artifacts: a case study of bug reports,” in Proceedings of the 32nd
ACM/IEEE International Conference on Software Engineering - Volume
R EFERENCES 1, ser. ICSE ’10. New York, NY, USA: ACM, 2010, pp. 505–514.
[25] B. Farbey, “Software quality metrics: considerations about requirements
[1] “Guide to the Software Quality Body of Knowledge (SQuBOK),” JUSE: and requirement specifications,” Information and Software Technology,
The Union of Japanese Scientists and Engineers, Tech. Rep., 2007. vol. 32, no. 1, pp. 60–64, 1990.
[2] S. Barney, A. Aurum, and C. Wohlin, “A product management chal- [26] A. Davis, S. Overmyer, K. Jordan, J. Caruso, F. Dandashi, A. Dinh,
lenge: Creating software product value through requirements selection,” G. Kincaid, G. Ledeboer, P. Reynolds, P. Sitaram et al., “Identifying and
Journal of Systems Architecture, vol. 54, no. 6, pp. 576–593, 2008. measuring quality in a software requirements specification,” in Software
[3] “ISO/IEC 25010 standard. Systems and software engineering – Systems Metrics Symposium, 1993. Proceedings., First International. IEEE,
and software Quality Requirements and Evaluation (SQuaRE) – System 1993, pp. 141–152.
and software quality models.” 2011. [27] B. J. Moore, “Achieving software quality through requirements analysis,”
[4] “ISO/IEC – 9126-1 Software engineering – Product quality – Part 1: in Proceedings of the 1994 IEEE International Engineering Manage-
Quality models,” 2001. ment Conference. ’Management in Transition: Engineering a Changing
[5] B. Nuseibeh and S. Easterbrook, “Requirements engineering: a World’. IEEE, 1994, pp. 78–83.
roadmap,” in Proceedings of the Conference on The Future of Software [28] T. Moores and R. Champion, “Software quality through the traceability
Engineering, ser. ICSE ’00. New York, NY, USA: ACM, 2000, pp. of requirements specifications,” in First International Conference on
35–46. Software Testing, Reliability and Quality Assurance. IEEE, 1994, pp.
[6] B. Paech and D. Kerkow, “Non-functional requirements engineering- 100–104.
quality is essential,” in 10th International Workshop on Requirments [29] L. Chung, B. A. Nixon, and E. Yu, “Using quality requirements to
Engineering Foundation for Software Quality, 2004. systematically develop quality software,” in Proceedings of the 4th
[7] J. Neill Colin and A. Laplante Phillip, “Requirements engineering: The International Conference on Software Quality, 1994.
state of the practice,” IEEE Software, vol. 20, pp. 40–45, 2003. [30] J. Vanwelkenhuysen, “Quality requirements analysis in customer-
[8] “IEEE Std 1465-1998//ISO/IEC 12119:1994, IEEE Standard Adop- centered software development,” in Proceedings of the Second Inter-
tion of International Standard ISO/IEC 12119:1994(E), Information national Conference on Requirements Engineering. IEEE, 1996, pp.
Technology-Software Packages-Quality Requirements and Testing,” 117–124.
1998. [31] X. F. Liu, “A quantitative approach for assessing the priorities of
[9] “ISO/IEC 25000:2005 Software engineering – Software product Quality software quality requirements,” Journal of Systems and Software, vol. 42,
Requirements and Evaluation (SQuaRE) – guide to SQuaRE,” 2005. no. 2, pp. 105–113, 1998.
[10] “ISO/IEC 25001:2007 Software engineering – Software product Quality [32] A. M. Davis, “Achieving quality in software requirements,” Software
Requirements and Evaluation (SQuaRE) – planning and management,” Quality Professional, vol. 1, no. 3, pp. 37–44, 1999.
2007. [33] N. Bevan and I. Bogomolni, “Incorporating user quality requirements in
[11] “ISO/IEC 25030:2007 Software engineering – Software product Quality the software development process,” in Proceedings of 4th international
Requirements and Evaluation (SQuaRE) – Quality requirements,” 2007. software quality week Europe, Brussels. Citeseer, 2000, pp. 1192–1204.
[12] “ISO/IEC 15288 Systems and software engineering – System life cycle [34] E. Johansson, A. Wesslén, L. Bratthall, and M. Host, “The importance
processes,” 2008. of quality requirements in software platform development - a survey,”
[13] P. Brereton, B. A. Kitchenham, D. Budgen, M. Turner, and M. Khalil, in Proceedings of the 34th Annual Hawaii International Conference on
“Lessons from applying the systematic literature review process within System Sciences. IEEE, 2001, pp. 10–pp.
the software engineering domain,” J. Syst. Softw., vol. 80, no. 4, pp. [35] A. Moreira, J. Araújo, and I. Brito, “Crosscutting quality attributes
571–583, Apr. 2007. for requirements engineering,” in Proceedings of the 14th international

237
 conference on Software engineering and knowledge engineering. ACM, [57] C. G. Şen and H. Baraçlı, “Fuzzy quality function deployment based
2002, pp. 167–174. methodology for acquiring enterprise software selection requirements,”
[36] C. Denger, D. M. Berry, and E. Kamsties, “Higher quality requirements Expert Systems with Applications, vol. 37, no. 4, pp. 3415–3426, 2010.
specifications through natural language patterns,” in Proceedings. IEEE [58] F. L. Castillo, F. Losavio, A. Matteo, and J. Boegh, “Requirements,
International Conference on Software: Science, Technology and Engi- aspects and software quality: the REASQ model,” Journal of Object
neering. SwSTE’03. IEEE, 2003, pp. 80–90. Technology, vol. 9, no. 4, pp. 69–91, 2010.
[37] L. Karlsson, B. Regnell, J. Karlsson, and S. Olsson, “Post-release [59] X. Liu, P. Inuganti, K. Noguchi, and Y. Kyoya, “Target setting for
analysis of requirements selection quality - an industrial case study,” technical requirements in time-stamped software quality function,” in
in International Workshop on Requirements Engineering: Foundation IEEE 34th Annual Computer Software and Applications Conference
for Software Quality, 2003. (COMPSAC). IEEE, 2010, pp. 198–207.
[38] E. Delor, R. Darimont, and A. Rifaut, “Software quality starts with [60] H. Kaiya, M. Tanigawa, S. Suzuki, A. Osada, and K. Kaijiri, “Improving
the modelling of goal-oriented requirements,” in 16th International reliability of spectrum analysis for software quality requirements using
Conference Software & Systems Engineering and their Applications, TCM,” IEICE TRANSACTIONS on Information and Systems, vol. 93,
2003, pp. 1–6. no. 4, pp. 702–712, 2010.
[39] D. Firesmith, “Using quality models to engineer quality requirements,” [61] J. R. McCoy, “NASA software tools for high-quality requirements
Journal of Object Technology, vol. 2, no. 5, pp. 67–75, 2003. engineering,” in Proceedings. 26th Annual NASA Goddard Software
[40] J. Ramires, P. Antunes, and A. Respício, “Software requirements nego- Engineering Workshop. IEEE, 2001, p. 69.
tiation using the software quality function deployment,” in Groupware: [62] R. B. Svensson, T. Gorschek, B. Regnell, R. Torkar, A. Shahrokni,
Design, Implementation, and Use. Springer, 2005, pp. 308–324. R. Feldt, and A. Aurum, “Prioritization of quality requirements: State of
[41] A. Dávila, K. Melendez, and L. Flores, “Establishing software product practice in eleven companies,” in 19th IEEE International Requirements
quality requirements according to international standards,” Latin Amer- Engineering Conference (RE). IEEE, 2011, pp. 69–78.
ica Transactions, IEEE (Revista IEEE America Latina), vol. 4, no. 2, [63] R. Berntsson Svensson, T. Gorschek, B. Regnell, R. Torkar,
pp. 100–106, 2006. A. Shahrokni, and R. Feldt, “Quality requirements in industrial practice–
[42] H. Schmidt and I. Wentzlaff, “Preserving software quality character- an extended interview study at eleven companies,” IEEE Transactions
istics from requirements analysis to architectural design,” in Software on Software Engineering, vol. 38, no. 4, pp. 923–935, 2012.
Architecture. Springer, 2006, pp. 189–203. [64] Z. Li-li, H. Lian-feng, and S. Qin-ying, “Research on requirement
[43] J. Vaníček, “Software quality requirements,” Agriculture Economics, for high-quality model of extreme programming,” in International
vol. 52, p. 4, 2006. Conference on Information Management, Innovation Management and
[44] J. Eckroth and G.-A. Amoussou, “Improving software quality from the Industrial Engineering (ICIII), vol. 1. IEEE, 2011, pp. 518–522.
requirements specification,” in Proceedings of the 2007 Symposium on [65] H. Kaiya, S. Suzuki, T. Ogawa, M. Tanigawa, M. Umemura, and K. Kai-
Science of Design. ACM, 2007, pp. 38–39. jiri, “Spectrum analysis for software quality requirements using analyses
[45] O. Ormandjieva, I. Hussain, and L. Kosseim, “Toward a text classifica- records,” in IEEE 35th Annual Computer Software and Applications
tion system for the quality assessment of software requirements written Conference Workshops (COMPSACW). IEEE, 2011, pp. 500–503.
in natural language,” in Fourth international workshop on Software [66] M. Hneif and S. P. Lee, “Using guidelines to improve quality in software
quality assurance: in conjunction with the 6th ESEC/FSE joint meeting. nonfunctional attributes,” IEEE Software, vol. 28, no. 6, pp. 72–77, 2011.
ACM, 2007, pp. 39–45. [67] B. Wei, Z. Jin, D. Zowghi, and B. Yin, “Automated reasoning with
[46] E. Knauss and C. El Boustani, “Assessing the quality of software goal tree models for software quality requirements,” in IEEE 36th
requirements specifications,” in 16th IEEE International Requirements Annual Computer Software and Applications Conference Workshops
Engineering, RE’08. IEEE, 2008, pp. 341–342. (COMPSACW). IEEE, 2012, pp. 373–378.
[47] S. Wagner, F. Deissenboeck, and S. Winter, “Managing quality require- [68] H. Thi-Thuy-Hang and A. Pirotte, “Distinguishing soft-goals and quality
ments using activity-based quality models,” in Proceedings of the 6th requirements in software requirements modeling,” in DBKDA 2012, The
international workshop on Software quality. ACM, 2008, pp. 29–34. Fourth International Conference on Advances in Databases, Knowledge,
[48] A. Herrmann and B. Paech, “MOQARE: misuse-oriented quality re- and Data Applications, 2012, pp. 9–14.
quirements engineering,” Requirements Engineering, vol. 13, no. 1, pp. [69] H. Suleiman and D. Svetinovic, “Evaluating the effectiveness of the
73–86, 2008. security quality requirements engineering (SQUARE) method: a case
[49] B. Paech and T. Wetter, “Rational quality requirements for medical study using smart grid advanced metering infrastructure,” Requirements
software,” in Proceedings of the 30th international conference on Engineering, pp. 1–29, 2012.
Software engineering. ACM, 2008, pp. 633–638. [70] H. Kaiya and A. Ohnishi, “Improving software quality requirements
[50] J. D. Blaine and J. Cleland-Huang, “Software quality requirements: How specifications using spectrum analysis,” in IEEE 36th Annual Com-
to balance competing priorities,” IEEE Software, vol. 25, no. 2, pp. 22– puter Software and Applications Conference Workshops (COMPSACW).
24, 2008. IEEE, 2012, pp. 379–384.
[51] D. V. Dzung and A. Ohnishi, “Improvement of quality of software re- [71] L. B. Phillips, A. Aurum, and R. B. Svensson, “Managing software
quirements with requirements ontology,” in 9th International Conference quality requirements,” in 38th EUROMICRO Conference on Software
on Quality Software QSIC’09. IEEE, 2009, pp. 284–289. Engineering and Advanced Applications (SEAA). IEEE, 2012, pp. 349–
[52] F. Salger, G. Engels, and A. Hofmann, “Inspection effectiveness for 356.
different quality attributes of software requirement specifications: An [72] R. B. Svensson, Y. Sprockel, B. Regnell, and S. Brinkkemper, “Setting
industrial case study,” in ICSE Workshop on Software Quality. WOSQ’09. quality targets for coming releases with quper: an industrial case study,”
IEEE, 2009, pp. 15–21. Requirements Engineering, vol. 17, no. 4, pp. 283–298, 2012.
[53] L. Chung and J. C. S. do Prado Leite, “On non-functional requirements [73] R. Djouab, “Software product quality requirements engineering method:
in software engineering,” in Conceptual modeling: Foundations and SOQUAREM,” Ph.D. dissertation, École de technologie supérieure,
applications. Springer, 2009, pp. 363–379. 2012.
[54] K. Seetharam and C. Pujari, “Ranking of tools use, software logical [74] J. Terzakis, “The impact of a requirements specification on software de-
complexity, requirement volatility, quality requirements, efficiency re- fects and quality indicators,” in ICCGI 2012, The Seventh International
quirements in software development,” in IEEE International Advance Multi-Conference on Computing in the Global Information Technology,
Computing Conference. IACC. IEEE, 2009, pp. 1608–1616. 2012, pp. 140–141.
[55] H. M. Jani, “Applying case-based reasoning to software requirements [75] C. Wohlin, P. Runeson, M. Höst, M. C. Ohlsson, B. Regnell, and
specifications quality analysis system,” in 2nd International Conference A. Wesslén, Experimentation in software engineering: an introduction.
on Software Engineering and Data Mining (SEDM). IEEE, 2010, pp. Norwell, MA, USA: Kluwer Academic Publishers, 2000.
140–144. [76] L. Chung, B. A. Nixon, and E. Yu, “Using non-functional requirements
[56] R. B. Svensson, Y. Sprockel, B. Regnell, and S. Brinkkemper, “Cost to systematically select among alternatives in architectural design,” in
and benefit analysis of quality requirements in competitive software Proc. 1st Int. Workshop on Architectures for Software Systems, Seattle,
product management: A case study on the QUPER model,” in Fourth Washington, 1995, pp. 31–43.
International Workshop on Software Product Management (IWSPM).
IEEE, 2010, pp. 40–48.

238

View publication stats