2011 5th Malaysian Conference in Software Engineering (MySEC)

Software Architecture Reliability Prediction Models:

An Overview

Furkh Zeshan Radziah Mohamad

Department of Software Engineering, Faculty of Computer Department of Software Engineering, Faculty of Computer
Science and Information Systems, Universiti Teknologi Science and Information Systems, Universiti Teknologi
Malaysia (UTM), 81310 Skudai, Johor, Malaysia Malaysia (UTM), 81310 Skudai, Johor, Malaysia
[email protected] [email protected]

Abstract - The ability to predict reliability of the software during The overall reliability of the application depends on the
its architectural design not only helps in saving cost but also reliabilities of each component, relationship between them [4]
helps to improve its reliability. With the growing size and and the architecture of the application [6, 8, 11, 15, 27, 29, 28,
complexity of software applications researchers have focused that 41, 45]. Many reliability models for architecture based
how to get software reliability through its architecture.
Architectural design phase is the stage where one can evaluate
software systems has been proposed in [10, 27]. Based on
either the developed software will fulfill the requirements or not; their research and our criteria (defined in section II),
that’s why a highly reliable method is required to analyze and architecture based reliability models can be categorizes into
predict the software’s architectural reliability. Reliability two categories, path-based [3, 6, 14, 16, 19, 32] and state-
prediction at architecture level is a challenging task because the based [8, 12, 15, 17, 18].
architecture reliability depends on the reliability of the individual To ensure the correctness and reliability at architectural
component, their size, complexity, implemented technology and level in component based software development is important
the interaction among the components. In this paper we have issue due to the following reasons.
compared the existing reliability prediction models based on our
criteria. The purpose is to discover that which one is the best and x Architect can find inconsistencies in the composition
what is the shortcoming of these models. We also have suggested early in the development phase.
the research activities needed to overcome these shortcomings.
x Critical software systems often require high levels of
Keywords - Software reliability; software architecture; assurance with respect to the reliability of their
design and implementation [44].
I. INTRODUCTION Software architecture determines how the components
Software systems are working in almost every field of life will interact with each other and how the system will behave
and depend upon the reliability very much. The size and and perform. Reliability prediction at architecture level is
complexity of software systems is grown rapidly from the mid rather challenging task because architectural reliability
1990, since then reliability analysis of the software depends on the reliability of the individual components and
architecture has attracted attention of the architects [6, 7, 8, how frequently the faulty part of the software is executed. In
13, 15, 24], and the trend will continue in future [1]. this paper we have compared different architecture based
reliability analysis models on our criteria to find out the best
Software architecture presents a view of a software one that can be used to predict the reliability for today’s
system as the combination of components and connectors. complex software systems. We have identified their strengths
Components encapsulate some similar set of functionality and shortcomings along with suggested research activities to
whereas the connectors realize the runtime communication overcome these discovered shortcomings. We have used an
between components. Well designed software architecture example application to explain these approaches, rest of the
not only affects the development activities but also help in the paper is categorized as; in Section II model selection criteria is
maintenance phase. In order to reduce maintenance costs an presented, in section III overview of architecture base
architect should investment time in various design reliability model is given, in section IV model evaluation
considerations, including low coupling, high cohesion, criteria and in section V discussion is presented, while section
separation of modules, proper interfaces etc[2]. To calculate VI concludes the whole paper.
the reliability from architecture of software systems
particularly large scale systems is a highly desire of software II. MODEL SELECTION CRITERIA
managers for long time [36, 46], because, it is very difficult
and expensive to fix the detected design fault at testing or In this section we have presented the general requirements
implementation phase as compared to the architectural level of architecture based software reliability models which are
[4]. given below in table I.

978-1-4577-1531-0/11/$26.00 ©2011 IEEE

119
 TABLE I: MODEL SELECTION CRITERIA there are loops in the paths then that path can be executed
Model Selection Criteria infinite number of times [10]. In path based models the
Goal Predict reliability. proposed solution of loops is to limit the number of paths. In
Target Group Software architecture path based approaches multiple occurrences are consumed on
Language UML one path as k occurrences [6]. K defines Degree of
Application Domain General Independence and k > 0. For complete dependence, [27] sets
Platform Platform independent the value of Degree of Independence to 1 and for completes
Assumption Component reliabilities are available independence the value is . Now it is not clear from their
Centricity Stakeholder focused approach work that how to calculate a suitable value of Degree of
Approach Component based Independence [27]. In [33] Component Dependency Graph is
Application behavior Failure behavior of modules.
used to calculate the reliability of the application. The average
Based on these requirements and the approaches proposed execution time of a scenario is used to terminate the depth
in [10, 27], models can be categorizes into two categories, expansion of a path, which guaranties that deadlock will not
path based and state based; detail of these models is given in occur due to the loop between components.
the section below. Path based approaches provides an approximate estimation
of application reliability and considered to be good for large
III. ARCHITECTURE BASED RELIABILITY MODELS scale systems [3], because these states do not face the state
Architecture of the application describes the system space explosion problem, this is a big advantage over the state
consisting of components their relationships and the external based approaches.
visible properties [37]. In this section, the concept how each
model works is presented with the help of example. B. State Based Models
State based models estimate software reliability
analytically and to represent the architecture of application
A. Path based Models
control flow graph is used [40, 42]. State based models can be
In path based models reliability is calculated along with the classified further into Composite Models and Hierarchical
execution paths [38]. Figure 1 show different execution paths Model [15]. Detail of each model is given below.
of the application. Every execution path starts from
component 1 and ends at component 9. The detail of each path
can be obtained either through algorithmically [3],
experimentally [6], or by simulation (if application code is
unavailable) [8, 9]. Reliability calculation is a two step
process, during initial step, calculate the reliability of each
component along with the path and then multiply the
reliabilities of components to get the total reliability of each
path. The reliability of path (Figure 1) 1, 1, 3, 5, 6, 8, 9 is
R1R3R5R6R8R9, while in second step take the average of
these reliabilities [35].

Figure 2. Composite model

a) Composite Models
Composite model can be formed by combining failure
behavior and the software architecture which is used for
calculating the reliability of the application [38]. The
composite model can be seen in Figure 2. To represent the
architecture of the software application, program flow graph is
used by these models. These models assume that the transfer
of control between components has a Markov property, and
the application architecture can be represent with the help of
Figure 1. Simple model discrete-time Markov chain (DTMC) [5,22,23], a continuous
time Markov chain (CTMC) [21], or a semi-Markov process
The major drawback of path based approach is that it [18], where as the behavior of components can be represented
provides an approximate reliability of the application and if with the failure models named, probability of reliability

120
[5,22,23], failure rate [21], and failure intensity [24]. The State based approaches have advantages over path based
reliability model is used to calculated reliability of approaches, because, state based approaches can calculate the
components. The reliability calculated in this way is least reliability of the application even in the presence of loops,
accurate because the components are treated as a black box. while the path based approaches cannot. State based
Moreover model assumes that the transfer of control between approaches can consider the impact of loops while path based
components has a Markov property which means that the approaches cannot, which is the major advantage of state
future behavior of the system is conditionally independent of based approaches over the path based approaches.
the past behavior [39], whereas in some applications,
execution history of the software determines the execution of IV. MODEL EVALUATION
the next component [18, 20].
In-order to in-time market the component based software
Constant failure behavior is used to represent the and to save cost; it is necessary to decide at early phases that
component failure behavior such that, how much time spent which component will be developed in house and which one
on that component. If higher the time is spent on the will be outsourced. A mechanism should be developed to
component, means that the probability of component failure is check the individual component reliability as well as how the
high, whereas time dependent failure model is used to test the individual component will affect the overall reliability of the
characteristics of components either through the test coverage software.
[31] or through the observation of failure data. This model
seems to be most accurate model because it also calculates the To answer the above mentioned situation we have
developed the following six questions; used as the evaluation
reliability of the components in loop [30]. Different
combinations of architectural and failure models are possible parameters to analyze the strengths and the weaknesses of the
which can be seen in [26]. The exact estimate of application architecture based reliability models.
reliability can be obtained with the help of composite model Q1. Is the model calculates reliability exactly?
and the DTMC analysis [25].
Q2. Is the model faces state space explosion problem?
In case of large software systems, state space explosion is
the main drawback of composite models. Researchers [47] are Q3. Can the model perform sensitivity analysis?
trying to develop a model to address this problem but their
research is still at its infancy. Such systems have hundreds of Q4. Can we use the model for predictive analysis?
states and it is almost impossible to do analysis for each state
[38]. Q5: Can we apply the model in SDLC, to calculate reliability?

Q6: Is the component failure behavior captured by the model?

b) Hierarchical based Models
To overcome the problem of composite models, We have used these parameters in Table II, which shows
hierarchical models calculate the reliability of the application the evaluation results of different reliability prediction models
in two steps [5, 20, 34]. During the first step mean and the at architecture level.
variance of each component is obtained by solving the
architectural model into architectural statistics. To solve the
architectural model Stochastic Petri Net’s [5] can be used. In V. DISCUSSIONS
the second step, failure parameters and the architectural State based approaches assume that the transfer of control
statistics are combined together to get the analytical reliability between modules has a first order Markov chain property, and
function. Failure parameter can be obtained with the help of the software architecture and failure behavior can be modeled
Software Reliability Growth Model [24], unit testing of as a Markov chain or semi Markov process. This means that
component, fault seeding [31] and the combination of fault the future behavior of the system is independent to the past
estimation techniques [21,22,23]. Authors in [5] use behavior [39]. While in some software applications, execution
hierarchical analysis to get analytical reliability function: history of components determine the execution of the next
component [18, 20]. Specification and analysis of model
In the above equation vi and represent the mean and
becomes difficult due to the large number of components, it
variance of component i. To get vi and , DTMC representation
also makes it difficult to measure and parameterization of
of the architecture has been solved. The reliability obtained
components [39]. For making such decisions that which
through hierarchical method relies on the assumption of intra-
component should be developed in-house and which one
component independence; this means that successive
should be outsourced, sensitivity information can be used.
execution of a component is independent from one another.
Sensitivity analysis can also be used to analyze the influence
Hierarchical model gives approximate reliability of the
of the change in input parameters on the performance and
application, which is their major drawback. Sensitivity and
reliability metrics [38]. It is very expensive and time
predictive analysis can be performed with the help of
consuming to conduct the sensitivity analysis with the help of
analytical reliability function obtained by hierarchical method
composite model, because composite model needs to be
where as in composite model this is not easily possible.
reconstructed and resolved for each state of the system, if the

121
 TABLE II: MODEL EVALUATION hierarchical model meet most of the requirements. So, we can
say, that, the use of hierarchical model for reliability
Model Evaluation Criteria
prediction may be the most beneficial for the industry.
Model Q1 Q2 Q3 Q4 Q5 Q6 However, the researchers may focus their attention on
calculating the exact reliability with the help of hierarchical
Path Based × × × × × ×
models.
Composite   × × × 
Hierarchical × ×    
VI. CONCLUSION
Legends
 Requirement fully satisfied This paper provides an overview of the architecture base
× Requirement unsatisfied reliability analysis models. We have defined criteria for
comparing the reliability prediction models at architectural
system has a considerable amount of states. While it is
level and after comparison; we came to know that none of
relatively easy to conduct sensitivity analysis with the help of
them can provide full support for the prediction of software
analytical reliability functions. The analytical reliability
reliability at architecture level. In our study we also find that
function can be used to put the uncertainty in the architecture
hierarchical model among other is much better which,
models and the component failure parameters to estimate the
however, still have a serious shortcoming. These
reliability of the application.
shortcomings are that hierarchical models could not calculate
In the composite model, the impact of an individual the reliability exactly.
component should be analyzed on the overall behavior of the
software. For such analysis, the combined model should be
ACKNOWLEDGEMENT
reconstructed and resolved again and again to obtain the
revised predictions. If software has hundreds of different states We would like to thank Malaysia Ministry of Higher
and the construction and solution of composite model for these Education for sponsoring the research through the grant with
states become very costly. Moreover some of the vote number 78600 and Universiti Teknologi Malaysia for
combinations of the composite model cannot be solved. For providing the facilities and support for the research.
example, if components failure behavior is represented with
the help of time dependent failure model and the architecture REFERENCES
of the application with DTMC then composite model cannot
[1] R. Taylor and A. van der Hoek, “Software Design and Architecture: The
be solved. Behavioral transition probabilities can be obtained Once and Future Focus of Software Engineering,” Future of Software
from the data obtained through functionally similar Engineering 2007, L. Briand and A. Wolf (eds.), IEEE-CS Press, 2007.
components or from the components architectural models. [2] Michael R. Lyu, “Software Reliability Engineering: A Roadmap,”
Morkov based approaches may consider one-to-one Future of Software Engineering (FOSE'07)
correspondence between the events and the transitions in the [3] S. Yacoub, B. Cukic, H. H. Ammar, “A scenario-based reliability
model, while such relation may not exist especially in analysis approach for component-based software”, IEEE Transactions
on Reliability, Vol. 53, No. 4, pp. 465-480, December 2004.
functionally similar components [46]. In both models state
[4] Yuanjie Si, Xiaohu Yang, Xinyu Wang, Chao Huang, Aleksander J.
based models or the path based models, reliabilities of Kavs, “An Architecture-Based Reliability Estimation Framework
individual components and transition probability between Through Component Composition Mechanisms”, 2nd International
components is needed. Both models calculate system Conference on Computer Engineering and Technology, 2010 IEEE
reliability on the assumption that the reliabilities of individual [5] Swapna S. Gokhale and Kishor S. Trivedi “Reliability Prediction and
Sensitivity Analysis Based on Software Architecture,” Proceedings of
components and transition probability between components the 13th International Symposium on Software Reliability Engineering
are known [43]. To calculate the reliabilities of individual (ISSRE’02), Nov. 2002 IEEE
components and transition probability before the [6] S. Krishnamurthy and A. P. Mathur. “On the estimation of reliability of
implementation of application needs to be studied. a software system using reliabilities of its components”. In Proceedings
of Eighth International Symposium on Software Reliability Engineering,
Both approaches like path based and hierarchical calculate pages 146– 155, Albuquerque, New Mexico, November 1997.
the approximate reliability of the software architecture, while [7] K. Goseva-Popstojanova, A.P. Mathur, and K.S. Trivedi, “Comparison
state based model calculates exact reliability but if the of Architecture-Based Software Reliability Models,” In Proceedings of
International Symposium Software Reliability Engineering., pp. 22-31,
software has higher number of states then state space 2001.
explosion is a biggest problem. In order to overcome this [8] S. Gokhale, M.R. Lyu, and K.S. Trivedi, “Reliability Simulation of
problem, researchers should focus to develop an appropriate Component-Based Software Systems,” In Proceedings of Ninth
model for decomposition strategies. Such model may divide International Symposium of Software Reliability Engineering. (ISSRE
components into groups (based on their characteristics) and ’98), pp. 192-201, Nov. 1998.
combining reliability function of each group into the [9] R. Huang, M.R. Lyu, and K. Kanoun, “Simulation Techniques for
Component-Based Software Reliability Modeling with Project
application reliability function. Application,” In Proceedings of Int’l Symp. Information Systems and
Engineering. (ISE ’01), pp. 283-289, 2001.
Table II, shows that none of approach fully satisfies the
[10] Swapna S. Gokhale, “Architecture-Based Software Reliability Analysis:
requirements for predicting reliability at architectural level. Overview and Limitations,” IEEE transactions on dependable and secure
Composite method is slightly close to the requirements while computing, vol. 4, no. 1, january-march 2007

122
[11] W.Wang, Y.Wu, M.H. Chen, “An architecture-based software reliability Based Embedded Software,” International Conference on Frontier of
model,” in Proceedings of the Pacific Rim Dependability Symposium, Computer Science and Technology, 2009 IEEE
Hong Kong, 1999. [33] S. Yacoub, B. Cukic and H. Ammar, “Scenario-based reliability analysis
[12] B. Littlewood. “A semi-Markov model for software reliability with of component-based software,” In Proceedings of 10th International
failure costs”. In Proceedings of Symp.Comput. Software Engineering, Symposium on Software Reliability Engineering(ISSRE’99), 1999,
pages 281–300, Polytechnic Institute of New York, April 1976. pp.22-31.
[13] K. Goseva-Popstojanova and S. Kamavaram, “Software Reliability [34] Littlewood, B. “Software reliability model for modular program
Estimation under Uncertainty: Generalization of the Method of structure.” IEEE Trans. Reliability 28(3), 241–246 (1979)
Moments,” In Proceedings of Eighth IEEE International Symp. High [35] Runeson, P., Regnell, B. “Derivation of an integrated operational profile
Assurance Systems Engineering., pp. 209-218, 2004. and use case model.”In Proceedings. The Ninth International
[14] Vittorio Cortellessa and PasqualinaPotena “Path-Based Error Symposium on Software Reliability Engineering (1998)
Propagation Analysis in Composition of Software Services”, M. Lumpe [36] Reussner, R.H., Schmidt, H.W., Poernomo, I.H. “Reliability prediction
and W. Vanderperren, SC 2007, LNCS 4829, pp. 97–112, 2007. for component-based software architectures.” J. Systems Softw. 66(3),
Springer-Verlag Berlin Heidelberg 2007 241–252 (2003)
[15] S. Gokhale and K.S. Trivedi, “Structure-Based Software Reliability [37] Bass, L., Clements, P., Kazman, R. “Software Architecture in Practice.”
Prediction,” In Proceedings of Fifth International Conf. Advanced Addison-Wesley, Reading, 452 p (1998)
Computing (ADCOMP ’97), pp. 447-452, Dec. 1997.
[38] Anne Immonen, Eila Niemelä, “Survey of reliability and availability
[16] Fan Zhang, Xingshe Zhou, Junwen Chen, Yunwei Dong “A Novel prediction methods from the viewpoint of software architecture,”
Model for Component-based Software Reliability Analysis”, 2008 11th SoftwSyst Model (2008) 7:49–65, Springer-Verlag 2007
IEEE High Assurance Systems Engineering Symposium
[39] K. Goseva-Popstojanova, K. S. Trivedi, “Architecture-Based
[17] B. Littlewood. “A reliability model for Markov structured software”. In Approaches to Software Reliability Prediction,” Computers and
Proceedings of 1975 International Conf. Reliable Software, pages 204– Mathematics with Applications 46 (2003) 1023-1036, 2003 Elsevier.
207, Los Angeles, CA, April 1975.
[40] Wen-Li Wang, Dai Pan, Mei-Hwa Chen. “Architecture based software
[18] J. Muppala, M. Malhotra, and K.S. Trivedi, “Stiffness-Tolerant Methods reliability modeling.” 2006.
for Transient Analysis of Stiff Markov Chains,” Microelectronics and
[41] W.W. Everett, “Software component reliability analysis,” in Proceedings
Reliability, vol. 34, no. 11, pp. 1825-1841, 1994.
of the Application Specific Software Engineering and Technology,
[19] S. Yacoub, BojanCukic, and H. Ammar.“Scenario based analysis of Dallas, TX, 1999.
component-based software”. In Proceedings of Tenth International
Symposium on Software Reliability Engineering, Boca Raton, FL, [42] J.Ledoux, “Availability modeling of modular software,” IEEE Trans,
November 1999. Reliability 48(2)(1999)159-168
[43] Dong WANG, Ning HUANG, “Reliability Analysis of Component-
[20] M. Xie and C. Wohlin.“An additive reliability model for the analysis of
Based Software Based on Rewrite Logic,” 12th IEEE International
modular software failure data”. In Proceedings of Sixth International
Workshop on Future Trends of Distributed Computing Systems, 2008
Symposium on Software Reliability Engineering, pages 188–193,
IEEE
Tolouse, France, October 1995.
[44] Jing Dong, Paulo S.C. Alencar, Donald D. Cowan, ShEngineering Yang,
[21] J.C. Laprie and K. Kanoun, “X-Ware Reliability and Availability
“Composing pattern-based components and verifying correctness,” The
Modeling,” IEEE Trans. Software Engineering., vol. 15, pp. 130-147,
Journal of Systems and Software 80 (2007) 1755–1769, Elsevier
1992.
[22] K. Seigrist, “Reliability of Systems with Markov Transfer of Control,” [45] D. Hamlet, D. Woit, D. Mason, “Theory of software reliability based on
IEEE Trans. Software Engineering, vol. 14, no. 7, pp. 1049-1053, July components,” in Proceedings of the International Conference on
Software Engineering, Toronto, Canada, 2001, pp. 361–370.
1988.
[46] L. Cheung, R. Roshandel, N. Medvidovic, L. Golubchik, “Early
[23] K. Seigrist, “Reliability of Systems with Markov Transfer of Control,
prediction of software component reliability”, In Proceedings of
II,” IEEE Trans. Software Engineering., vol. 14, no. 10, pp. 1478-1480,
ICSE’08, May10-18, 2008, Leipzig, Germany.
Oct. 1988.
[47] Franz Brosch, Heiko Koziolek, Member, Barbora Buhnova, Ralf
[24] S. Gokhale, W.E. Wong, K.S. Trivedi, and J.R. Horgan, “An Analytic
Reussner, “Architecture-based Reliability Prediction with the Palladio
Approach to Architecture-Based Software Performance and Reliability
Component Model”, IEEE TRANSACTIONS ON SOFTWARE
Prediction,” Performance Evaluation, vol. 58, no. 4, pp. 391-412, Dec.
ENGINEERING, (accepted for publication) SEPTEMBER 2011
2004.
[25] K.S. Trivedi, “Probability and Statistics with Reliability, Queuing and
Computer Science Applications”. John Wiley & Sons, 2001.
[26] S. Gokhale and K.S. Trivedi, “Analytical Models for Architecture-Based
Software Reliability Prediction: A Unification Framework,” IEEE Trans.
Reliability.
[27] K. Goseva-Popstojanova and K. S. Trivedi.“Architecture-based
approach to reliability assessment of software systems”. Performance
Evaluation, 45(2-3), June 2001.
[28] J. Ledoux, G. Rubino, “Simple formulas for counting processes in
reliability models,” Adv. Appl. Probab. 34 (1997) 847–858.
[29] J. Ledoux, G. Rubino, “A counting model for software reliability
analysis,” Int. J. Modell. Simul. 17 (1997) 289–297.
[30] S. Gokhale and K.S. Trivedi, “Dependency Characterization in Path-
Based Approaches to Architecture-Based Software Reliability
Prediction,” In Proceedings of First Application Specific Software
Engineering. Technology (ASSET ’98), pp. 86-89, Mar. 1998.
[31] S. Gokhale and K.S. Trivedi, “A Time/Structure Based Software
Reliability Model,” Annals of Software Engineering., vol. 8, pp. 85-121,
1999.
[32] Changde Li; Xingshe Zhou; Yuying Wang; Yunwei Dong;, “An
Improved Path-based Reliability Prediction Model for Component-

123