Volume 2, Issue 3, March 2012

ISSN: 2277 128X

International Journal of Advanced Research in Computer Science and Software Engineering

Research Paper Available online at: www.ijarcsse.com

A Straight Map of Software Reliability on Component Based Engineering

Upasna Jaglan*
M. Tech (C.S.E.) Amity University Noida, India [email protected]

Deepesh Shrivastava
M.Tech (C.S.E.) Assistant Professer Amity University Noida, India

Abstract: - Software reliability is the ability of a system or component to perform its required functions under stated conditions for a specified period of time. Reliability Engineering is a sub-discipline within Systems Engineering. Reliability is often measured as probability of failure, frequency of failures, or in terms of availability, a probability derived from reliability and maintainability. Further to estimate as well as to predict the reliability of software systems, failure data need to be properly measured by various means during software development and operational phases. Moreover, credible software reliability models are required to track underlying software failure processes for accurate reliability analysis and forecasting. Although software reliability has remained an active research subject over the past 34 years, challenges and open questions still exist. In particular, vital future goals include the development of new software reliability engineering paradigms that take software architectures, testing techniques, and software failure manifestation mechanisms into consideration. In this given paper, we review the history of software reliability engineering, the current trends and existing problems, and specific difficulties. Keywords: Software reliability engineering, reusability, metrics, testing. high visibility programs and have led to loss of business [28]. 1. INTRODUCTION The ubiquitous software is also invisible, and its Software permeates used in our daily life. There is invisible nature makes it both beneficial and probably no other human-made material which is more omnipresent than software in our modern society. It has become a crucial part of many aspects of society: some appliances, harmful. From the positive side, systems around us telecommunications, automobiles, airplanes, work seamlessly thanks to the smooth and swift shopping, auditing, web teaching, personal execution of software. From the negative side, we entertainment, and so on. In particular, science and often do not know when, where and how software technology demand high-quality software for ever has failed, or will fail. Consequently, while making improvements and break throughs. The size reliability engineering for hardware and physical and complexity of software systems have grown systems continuously improves, reliability dramatically during the past few decades, and the engineering for software does not really live up to trend will certainly continue in the future. our expectation over the years. The data from industry show that the size of the software for various systems and applications has been growing exponentially for the past 40 years [20]. The trend of such growth in the telecommunication, business, defence, and transportation industries shows a compound growth rate of ten times every five years. Because of this ever-increasing dependency, software failures can lead to serious, even fatal, consequences in safetycritical systems as well as in normal business. Previous software failures have impaired several This situation is frustrating as well as encouraging. It is frustrating because the software crisis identified as early as the 1960s still stubbornly stays with us, and software engineering has not fully evolved into a real engineering discipline. The demands on, techniques of, and enhancements to software are continually increasing, and so is the need to understand its reliability. The unsettled software crisis poses tremendous opportunities for software engineering researchers as well as practitioners. The ability to manage quality

Volume 2, Issue 3, March 2012 software production is not only a necessity, but also a key distinguishing factor in maintaining a competitive advantage for modern businesses. Software reliability engineering is cantered on a key attribute, software reliability, which is defined as the probability of failure-free software operation for a specified period of time in a specified environment [2]. Among other attributes of software quality such as functionality, usability, capability, and maintainability, etc. Software reliability is generally accepted as the major factor in software quality since it quantifies software failures, which can make a powerful system inoperative. Software reliability engineering (SRE) is therefore defined as the quantitative study of the operational behavior of software-based systems with respect to user requirements concerning reliability. As a proven technique, SRE has been adopted either as standard or as best current practice by more than 50 organizations in their software projects and reports [30], including AT&T, Lucent, IBM, NASA, Microsoft, and many others in Europe, Asia, and North America. However, this number is still relatively small compared to the large amount of software producers in the world. Existing SRE techniques suffer from a number o weaknesses. First of all, current SRE technique collects the failure data during integration testing or system testing phases. Failure data collected during the late testing phase may be too late for fundamental design changes. Secondly, the failure data collected in the in-house testing may be limited, and they may not represent failures that would be uncovered under actual operational environment. This is especially true for high-quality software systems which require extensive and wide-ranging testing. The reliability estimation and prediction using the restricted testing data may cause accuracy problems. Thirdly, current SRE techniques or modelling methods are based on some unrealistic assumptions that make the reliability estimation too optimistic relative to real situations. Of course, the existing software reliability models have had their successes; but every model can find successful cases to justify its existence. Without cross industry validation, the modelling exercise may become merely of intellectual interest and would not be widely adopted in industry. Thus, although SRE has been around for a while, credible software reliability techniques are still urgently needed, particularly for modern software systems [24]. In the following sections we will discuss the past, the present, and the future of software reliability engineering. We first survey what techniques have been proposed and applied in the past, and then describe what the current trend is and what 2012, IJARCSSE All Rights Reserved

www.ijarcsse.com problems and concerns remain. Finally, we propose the possible future directions in software reliability engineering. II. SOFTWARE RELIABILITY ENGINEERING TECHNIQUES In the literature a number of techniques have been proposed to attack the software reliability engineering problems based on software fault lifecycle. We discuss these techniques, and focus on two of them. A. Fault lifecycle techniques Achieving highly reliable software from the customers perspective is a demanding job for all software engineers and reliability engineers. [28] Summarizes the following four technical areas which are applicable to achieving reliable software systems, and they can also be regarded as four fault lifecycle techniques: 1) Fault prevention: to avoid, by construction, fault occurrences. 2) Fault removal: to detect, by verification and validation, the existence of faults and eliminate them. 3) Fault tolerance: to provide, by redundancy, service complying with the specification in spite of faults having occurred or occurring. 4) Fault/failure forecasting: to estimate, by evaluation, the presence of faults and the occurrences and consequences of failures. This has been the main focus of software reliability modelling. Fault prevention is the initial defensive mechanism against unreliability. A fault which is never created costs nothing to fix. Fault prevention is therefore the inherent objective of every software engineering methodology. General approaches include formal methods in requirement specifications and program verifications, early user interaction and refinement of the requirements, disciplined and tool-assisted software design methods, enforced programming principles and environments, and systematic techniques for software reuse. Formalization of software engineering processes with mathematically specified languages and tools is an aggressive approach to rigorous engineering of software systems. When applied successfully, it can completely prevent faults. Unfortunately, its application scope has been limited. Software reuse, on the other hand, finds a wider range of applications in industry, and there is empirical evidence for its effectiveness in fault prevention. However, software reuse without proper certification could lead to disaster. The explosion of the Ariane 5 rocket, among others, is a classic example where seemly harmless software reuse failed miserably, in which critical software faults slipped through all the testing and verification procedures, and where a system went

Page | 346

Volume 2, Issue 3, March 2012 terribly wrong only during complicated real-life operations. Fault prevention mechanisms cannot guarantee avoidance of all software faults. When faults are injected into the software, fault removal is the next protective means. Two practical approaches for fault removal are software testing and software inspection, both of which have become standard industry practices in quality assurance. Directions in software testing techniques are addressed in [4] in detail. When inherent faults remain undetected through the testing and inspection processes, they will stay with the software when it is released into the field. Fault tolerance is the last defending line in preventing faults from manifesting themselves as system failures. Fault tolerance is the survival attribute of software systems in terms of their ability to deliver continuous service to the customers. Software fault tolerance techniques enable software systems: (1) Prevent dormant software faults from becoming active, such as defensive programming to check for input and output conditions and forbid illegal operations. (2) Contain the manifested software errors within a confined boundary without further propagation, such as exception handling routines to treat unsuccessful operations; (3) Recover software operations from erroneous conditions, such as check pointing and rollback mechanisms; and (4) Tolerate system-level faults methodically, such as employing design diversity in the software development. Finally if software failures are destined to occur, it is critical to estimate and predict them. Fault/failure forecasting involves formulation of the fault/failure relationship, an understanding of the operational environment, the establishment of software reliability models, developing procedures and mechanisms for software reliability measurement, and analyzing and evaluating the measurement results. The ability to determine software reliability not only gives us guidance about software quality and when to stop testing, but also provides information for software maintenance needs. It can facilitate the validity of software warranty when reliability of software has been properly certified. The concept of scheduled maintenance with software rejuvenation techniques [46] can also be solidified. The subjects of fault prevention and fault removal have been discussed thoroughly by other articles in this issue. We focus our discussion on issues related to techniques on fault tolerance and fault/failure forecasting. B. Software reliability measurement and models As a major task of fault/failure forecasting, software reliability modelling has attracted much research attention in estimation (measuring the 2012, IJARCSSE All Rights Reserved

www.ijarcsse.com current state) as well as prediction (assessing the future state) of the reliability of a software system. A software reliability model specifies the form of a random process that describes the behavior of software failures with respect to time. A historical review as well as an application perspective of software reliability models can be found in [7, 28]. There are three main reliability modelling approaches: the error seeding and tagging approach, the data domain approach, and the time domain approach, which is considered to be the most popular one. The basic principle of time domain software reliability modelling is to perform curve fitting of observed time-based failure data by a pre-specified model formula, such that the model can be parameterized with statistical techniques (such as the Least Square or Maximum Likelihood methods). The model can then provide estimation of existing reliability or prediction of future reliability by extrapolation techniques. Software reliability models usually make a number of common assumptions, as follows. (1) The operation environment where the reliability is to be measured is the same as the testing environment in which the reliability model has been parameterized. (2) Once a failure occurs, the fault which causes the failure is immediately removed. (3) The fault removal process will not introduce new faults. (4) The number of faults inherent in the software and the way these faults manifest themselves to cause failures follow, at least in a statistical sense, certain mathematical formulae. Since the number of faults (as well as the failure rate) of the software system reduces when the testing progresses, resulting in growth of reliability, these models are often called software reliability growth models (SRGMs). Since Jelinsky and Moranda proposed the first SRGM [23] in 1972, numerous SRGMs have been proposed in the past 35 years, such as exponential failure time class models, Weibull and Gamma failure time class models, infinite failure category models, Bayesian models, and so on [28]. Unified modelling approaches have also been attempted [19]. As mentioned before, the major challenges of these models do not lie in their technical soundness, but their validity and applicability in real world projects. Figure 1 shows an SRE framework in current practice [28]. First, a reliability objective is determined quantitatively from the customer's viewpoint to maximize customer satisfaction, and customer usage is defined by developing an operational profile. The software is then tested according to the operational profile, failure data collected, and reliability tracked during testing to determine the product release time.

Page | 347

Volume 2, Issue 3, March 2012 This activity may be repeated until a certain reliability level has been achieved. Reliability is also validated in the field to evaluate the reliability engineering efforts and to achieve future product and process improvements. It can be seen from Figure 1 that there are four major components in this SRE process, namely: (1) Reliability objective (2) Operational profile (3) Reliability modelling and measurement (4) Reliability validation.

www.ijarcsse.com Figure: Software Reliability Engineering Process A reliability objective is the specification of the reliability goal of a product from the customer viewpoint. If a reliability objective has been specified by the customer, that reliability objective should be used. Otherwise, we can select the reliability measure which is the most intuitive and easily understood, and then determine the customer's "tolerance threshold" for system failures in terms of this reliability measure. The operational profile is a set of disjoint alternatives of system operational scenarios and their associated probabilities of occurrence. The construction of an operational profile encourages testers to select test cases according to the system's likely operational usage, which contributes to more accurate estimation of software reliability in the field. Reliability modelling is an essential element of the reliability estimation process. It determines whether a product meets its reliability objective and is ready for release. One or more reliability models are employed to calculate, from failure data collected during system testing, various estimates of a product's reliability as a function of test time. Several interdependent estimates can be obtained to make equivalent statements about a product's reliability. These reliability estimates can provide the following information, which is useful for product quality management: (1) The reliability of the product at the end of system testing. (2) The amount of (additional) test time required to reach the product's reliability objective. (3) The reliability growth as a result of testing (e.g., the ratio of the value of the failure intensity at the start of testing to the value at the end of testing). (4) The predicted reliability beyond the system testing, such as the product's reliability in the field. Despite the existence of a large number of models, the problem of model selection and application is manageable, as there are guidelines and statistical methods for selecting an appropriate model for each application. Furthermore, experience has shown that it is sufficient to consider only a dozen models, particularly when they are already implemented in software tools [28]. Using these statistical methods, "best" estimates of reliability are obtained during testing. These estimates are then used to project the reliability during field operation in order to determine whether the reliability objective has been met. This procedure is an iterative process, since more testing will be needed if the objective is not met. When the

Determine Reliability Objective Perform Software Testing

Develop Operational Profile

Collect Failure Data

Apply Software Reliability Tools

Continue Testing

Select Appropriate Software Reliability Models

Use Software Reliability Models to Calculate Current Reliability Start to Deploy

Reliability Objective met?

Validate Reliability in the Field

Feedback to Next Release

2012, IJARCSSE All Rights Reserved

Page | 348

Volume 2, Issue 3, March 2012 operational profile is not fully developed, the application of a test compression factor can assist in estimating field reliability. A test compression factor is defined as the ratio of execution time required in the operational phase to execution time required in the test phase to cover the input space of the program. Since testers during testing are quickly searching through the input space for both normal and difficult execution conditions, while users during operation only execute the software with a regular pace, this factor represents the reduction of failure rate (or increase in reliability) during operation with respect to that observed during testing. Finally, the projected field reliability has to be validated by comparing it with the observed field reliability. This validation not only establishes benchmarks and confidence levels of the reliability estimates, but also provides feedback to the SRE process for continuous improvement and better parameter tuning. When feedback is provided, SRE process enhancement comes naturally: the model validity is established, the growth of reliability is determined, and the test compression factor is refined. C. Software fault models and tolerance techniques: Fault tolerance, when applicable, is one of the major approaches to achieve highly reliable software. There are two different groups of fault tolerance techniques: single version and multiversion software techniques [29]. The former includes program modularity, system closure, atomicity of actions, error detection, exception handling, checkpoint and restart, process pairs, and data diversity; while the latter, so-called design diversity, is employed where multiple software versions are developed independently by different program teams using different design methods, yet they provide equivalent services according to the same requirement specifications. The main techniques of this multiple version software approach are recovery blocks, N-version programming, N self-checking programming, and other variants based on these three fundamental techniques. Reliability models attempt to estimate the probability of coincident failures in multiple versions. Eckhardt and Lee (1985) [15] proposed the first reliability model of fault correlation in design diversity to observe positive correlations between version failures on the assumption of variation of difficulty on demand space. III. CURRENT TRENDS AND PROBLEMS The challenges in software reliability not only stem from the size, complexity, difficulty, and novelty of software applications in various domains, but also 2012, IJARCSSE All Rights Reserved

www.ijarcsse.com relate to the knowledge, training, experience and character of the software engineers involved. We address the current trends and problems from a number of software reliability engineering aspects.

A. Software reliability and system reliability

Although the nature of software faults is different from that of hardware faults, the theoretical foundation of software reliability comes from hardware reliability techniques. Previous work has been focused on extending the classical reliability theories from hardware to software, so that by employing familiar mathematical modelling schemes, we can establish software reliability framework consistently from the same viewpoints as hardware. The advantages of such modelling approaches are: (1) The physical meaning of the failure mechanism can be properly interpreted, so that the effect of failures on reliability, as measured in the form of failure rates, can be directly applied to the reliability models. (2) The combination of hardware reliability and software reliability to form system reliability models and measures can be provided in a unified theory. Even though the actual mechanisms of the various causes of hardware faults and software faults may be different, a single formulation can be employed from the reliability modelling and statistical estimation viewpoints. (3) System reliability models inherently engage system structure and modular design in block diagrams. The resulting reliability modelling process is not only intuitive (how components contribute to the overall reliability can be visualized), but also informative (reliability-critical components can be quickly identified). The major drawbacks, however, are also obvious. First of all, while hardware failures may occur independently (or approximately so), software failures do not happen independently. The interdependency of software failures is also very hard to describe in detail or to model precisely. Furthermore, similar hardware systems are developed from similar specifications, and hardware failures, usually caused by hardware defects, are repeatable and predictable. On the other hand, software systems are typically one-of-akind. Even similar software systems or different versions of the same software can be based on quite different specifications. Consequently, software failures, usually caused by human design faults, seldom repeat in exactly the same way or in any predictable pattern. Therefore, while failure mode and effect analysis (FMEA) and failure mode and effect criticality analysis (FMECA) have long been established for hardware systems, they are not very well understood for software systems.

Page | 349

Volume 2, Issue 3, March 2012

www.ijarcsse.com

B. Software reliability modelling

Among all software reliability models, SRGM is probably one of the most successful techniques in the literature, with more than 100 models existing in one form or another, through hundreds of publications. In practice, however, SRGMs encounter major challenges. First of all, software testers seldom follow the operational profile to test the software, so what is observed during software testing may not be directly extensible for operational use. Secondly, when the number of failures collected in a project is limited, it is hard to make statistically meaningful reliability predictions. Thirdly, some of the assumptions of SRGM are not realistic, e.g., the assumptions that the faults are independent of each other; that each fault has the same chance to be detected in one class; and that correction of a fault never introduces new faults. Nevertheless, the above setbacks can be overcome with suitable means. Given proper data collection processes to avoid drastic invalidation of the model assumptions, it is generally possible to obtain accurate estimates of reliability and to know that these estimates are accurate. Although some historical SRGMs have been widely adopted to predict software reliability, researchers believe they can further improve the prediction accuracy of these models by adding other important factors which affect the final software quality [12, 31, 48]. Among others, code coverage is a metric commonly engaged by software testers, as it indicates how completely a test set executes a software system under test, therefore influencing the resulting reliability measure. To incorporate the effect of code coverage on reliability in the traditional software reliability models, [12] proposes a technique using both time and code coverage measurement for reliability prediction. It reduces the execution time by a parameterized factor when the test case neither increases code coverage nor causes a failure. These models, known as adjusted Non-Homogeneous Poisson Process (NHPP) models, have been shown empirically to achieve more accurate predictions than the original ones. In the literature, several models have been proposed to determine the relationship between the number of failures/faults and the test coverage achieved, with various distributions, suggests that this relation is a variant of the Rayleigh distribution, while [30] shows that it can be expressed as a logarithmic-exponential formula, based on the assumption that both fault coverage and test coverage follow the logarithmic NHPP growth model with respect to the execution time. More metrics can be incorporated to further explore this new modelling avenue. 2012, IJARCSSE All Rights Reserved Although there are a number of successful SRE models, they are typically measurement-based models which are employed in isolation at the later stage of the software development process. Early software reliability prediction models are often too insubstantial, seldom executable, insufficiently formal to be analyzable, and typically not linked to the target system. Their impact on the resulting reliability is therefore modest. There is currently a need for a creditable end-to-end software reliability model that can be directly linked to reliability prediction from the very beginning, so as to establish a systematic SRE procedure that can be certified, generalized and refined.

C. Metrics and measurements

Metrics and measurements have been an important part of the software development process, not only for software project budget planning but also for software quality assurance purposes. As software complexity and software quality are highly related to software reliability, the measurements of software complexity and quality attributes have been explored for early prediction of software reliability [39]. Static as well as dynamic program complexity measurements have been collected, such as lines of code, number of operators, relative program complexity, functional complexity, operational complexity, and so on. The complexity metrics can be further included in software reliability models for early reliability prediction, for example, to predict the initial software fault density and failure rate. In SRGM, the two measurements related to reliability are: 1) the number of failures in a time period; and time between failures. An important advancement of SRGM is the notation of time during which failure data are recorded. It is demonstrated that CPU time is more suitable and more accurate than calendar time for recording failures, in which the actual execution time of software can be faithfully represented. More recently, other forms of metrics for testing efforts have been incorporated into software reliability modelling to improve the prediction accuracy [8, 18]. One key problem about software metrics and measurements is that they are not consistently defined and interpreted, again due to the lack of physical attributes of software. The achieved reliability measures may differ for different applications, yielding inconclusive results. A unified ontology to identify, describe, incorporate and understand reliability-related software metrics is therefore urgently needed. D. Data collection and analysis The software engineering process is described sardonically as a garbage-in/garbage-out process.

Page | 350

Volume 2, Issue 3, March 2012 That is to say, the accuracy of its output is bounded by the precision of its input. Data collection, consequently, plays a crucial role for the success of software reliability measurement. There is an apparent trade-off between the data collection and the analysis effort. The more accuracy is required for analysis, the more effort is required for data collection. Fault-based data are usually easier to collect due to their static nature. Configuration management tools for source code maintenance can help to collect these data as developers are required to check in and check out new updated versions of code for fault removal. Failure-based data, on the other hand, are much harder to collect and usually require additional effort, for the following reasons. First, the dynamic operating condition where the failures occur may be hard to identify or describe. Moreover, the time when the failures occur must be recorded manually, after the failures are manifested. Calendar time data can be coarsely recorded, but they lack accuracy for modelling purposes. CPU time data, on the other hand, are very difficult to collect, particularly for distributed systems and networking environment where multiple CPUs are executing software in parallel. Certain forms of approximation are required to avoid the great pain in data collection, but then the accuracy of the data is consequently reduced. It is noted that while manual data collection can be very labor intensive, automatic data collection, although unavoidable, may be too intrusive . The amounts and types of data to be collected for reliability analysis purposes vary between organizations. Consequently, the experiences and lessons so gained may only be shared within the same company culture or at a high level of abstraction between organizations. To overcome this disadvantage, systematic failure data analysis for SRE purposes should be conducted. Given field failure data collected from a real system, the analysis consists of five steps: 1) Pre-processing of data, 2) Analysis of data, 3) Model structure identification and parameter estimation, 4) Model solution, if necessary 5) Analysis of models. In Step 1, the necessary information is extracted from the field data. The processing in this step requires detailed understanding of the target software and operational conditions. The actual processing required depends on the type of data. For example, the information in human-generated reports is usually not completely formatted. Therefore, this step involves understanding the situations described in the reports and organizing the relevant 2012, IJARCSSE All Rights Reserved

www.ijarcsse.com information into a problem database. In contrast, the information in automatically generated event logs is already formatted. Data processing of event logs consists of extracting error events and coalescing related error events. In Step 2, the data are interpreted. Typically, this step begins with a list of measures to evaluate. However, new issues that have a major impact on software reliability can also be identified during this step. The results from Step 2 are reliability characteristics of operational software in actual environments and issues that must be addressed to improve software reliability. These include fault and error classification, error propagation, error and failure distribution, software failure dependency, hardware-related software errors, evaluation of software fault tolerance, error recurrence, and diagnosis of recurrences. In Step 3, appropriate models (such as Markov models) are identified based on the findings from Step 2. We identify model structures and realistic ranges of parameters. The identified models are abstractions of the software reliability behavior in real environments. Statistical analysis packages and measurement-based reliability analysis tools are useful at this stage. Step 4 involves either using known techniques or developing new ones to solve the model. Model solution allows us to obtain measures, such as reliability, availability, and performability. The results obtained from the model must be validated against real data. Reliability and performance modelling evaluation tools such as SHARPE can be used in this step. In Step 5, what if questions are addressed, using the identified models. Model factors are varied and the resulting effects on software reliability are evaluated. E. Methods and tools In addition to software reliability growth modelling, many other methods are available for SRE. We provide a few examples of these methods and tools. Fault trees provide a graphical and logical framework for a systematic analysis of system failure modes. Software reliability engineers can use them to assess the overall impact of software failures on a system, or to prove that certain failure modes will not occur. If they may occur, the occurrence probability can also be assessed. Fault tree models therefore provide an informative modelling framework that can be engaged to compare different design alternatives or system architectures with respect to reliability. In particular, they have been applied to both fault tolerant and fault intolerant (i.e., non-redundant) systems. Since this technique originates from hardware systems and has been extended to software

Page | 351

Volume 2, Issue 3, March 2012 systems, it can be employed to provide a unified modelling scheme for hardware/software codesign. Reliability modelling for hardwaresoftware interactions is currently an area of intensive research. In addition, simulation techniques can be provided for SRE purposes. They can produce observables of interest in reliability engineering, including discrete integer-valued quantities that occur as time progresses. One simulation approach produces artifacts in an actual software environment according to factors and influences believed to typify these entities within a given context [28]. The artifacts and environment are allowed to interact naturally, whereupon the flow of occurrences of activities and events is observed. This artifact-based simulation allows experiments to be set up to examine the nature of the relationships between software failures and other software metrics, such as program structure, programming error characteristics, and test strategies. It is suggested that the extent to which reliability depends merely on these factors can be measured by generating random programs having the given characteristics, and then observing their failure statistics. Another reliability simulation approach [28] produces timeline imitations of reliability-related activities and events. Reliability measures of interest to the software process are modelled parametrically over time. The key to this approach is a rate-based architecture, in which phenomena occur naturally over time as controlled by their frequencies of occurrence, which depend on driving software metrics such as number of faults so far exposed or yet remaining, failure criticality, workforce level, test intensity, and software execution time. Ratebased event simulation is an example of a form of modelling called system dynamics, whose distinctive feature is that the observables are discrete events randomly occurring in time. Since many software reliability growth models are also based on rate (in terms of software hazard), the underlying processes assumed by these models are fundamentally the same as the rate-based reliability simulation. In general, simulations enable investigations of questions too difficult to be answered analytically, and are therefore more flexible and more powerful. Various SRE measurement tools have been developed for data collection, reliability analysis, parameter estimation, model application and reliability simulation. Any major improvement on SRE is likely to focus on such tools. We need to provide tools and environments which can assist software developers to build reliable software for different applications. The partition of tools, environments, and techniques that will be engaged 2012, IJARCSSE All Rights Reserved

www.ijarcsse.com should reflect proper employment of the best current SRE practices. F. Testing effectiveness and code coverage As a typical mechanism for fault removal in software reliability engineering, software testing has been widely practiced in industry for quality assurance and reliability improvement. Effective testing is defined as uncovering of most if not all detectable faults. As the total number of inherent faults is not known, testing effectiveness is usually represented by a measurable testing index. Code coverage, as an indicator to show how thoroughly software has been stressed, has been proposed and is widely employed to represent fault coverage. Despite the observations of a correlation between code coverage and fault coverage, a question is raised: Can this phenomenon of concurrent growth be attributed to a causal dependency between code coverage and fault detection, or is it just coincidental due to the cumulative nature of both measures? In one investigation of this question, an experiment involving Monte Carlo simulation was conducted on the assumption that there is no causal dependency between code coverage and fault detection [6]. The testing result for published data did not support a causal dependency between code coverage and defect coverage. Nevertheless, many researchers consider coverage as a faithful indicator of the effectiveness of software testing results. A comparison among various studies on the impact of code coverage on software reliability is shown in Table 1. G. Operational profiles and testing The operational profile is a quantitative characterization of how a system will be used in the field by customers. It helps to schedule test activities, generate test cases, and select test runs. By allocating development and test resources to functions on the basis of how they are used, software reliability engineering can thus be planned with productivity and economics considerations in mind. Using an operational profile to guide system testing ensures that if testing is terminated and the software is shipped because of imperative schedule constraints, the most-used operations will have received the most testing, and the reliability level will be the maximum that is practically achievable for the given test time. Also, in guiding regression testing, the profile tends to find, among the faults introduced by changes, the ones that have the most effect on reliability. Examples of the benefits of applying operational profiles can be found in a number of industrial projects [30]. Although significant improvement can be achieved

Page | 352

Volume 2, Issue 3, March 2012 by employing operational profiles in regression or system testing, challenges still exist for this technique. First of all, the operational profiles for some applications are hard to develop, especially for some distributed software systems, e.g., Web services. Operational profiles of software cannot be duplicated in order to speed the testing, because the failure behavior of software depends greatly on its input sequence and internal status. While in unit testing, different software units can be tested at the same time, this approach is therefore not applicable in system testing or regression testing. As a result, learning to deal with improper operational profiles and the dependences within the operational profile are the two major problems in operational profile techniques. IV. POSSIBLE FUTURE DIRECTIONS SRE activities span the whole software lifecycle. We discuss possible future directions with respect to five areas: software architecture, design, testing, metrics and emerging applications. A. Reliability for software architectures Due to the ever-increasing complexity of software systems, modern software is seldom built from scratch. Instead, reusable components have been developed and employed, formally or informally. On the one hand, revolutionary and evolutionary object-oriented design and programming paradigms have vigorously pushed software reuse. On the other hand, reusable software libraries have been a deciding factor regarding whether a software development environment or methodology would be popular or not. In the light of this shift, reliability engineering for software development is focusing on two major aspects: software architecture, and component-based software engineering. The software architecture of a system consists of software components, their external properties, and their relationships with one another. As software architecture is the foundation of the final software product, the design and management of software architecture is becoming the dominant factor in software reliability engineering research. Wel ldesigned software architecture not only provides a strong, reliable basis for the subsequent software development and maintenance phases, but also offers various options for fault avoidance and fault tolerance in achieving high reliability. Due to the cardinal importance of, and complexity involved in, software architecture design and modelling, being a good software architect is a rare talent that is highly demanded. A good software architect sees widely and thinks deeply, as the components should eventually fit together in the overall framework, and the anticipation of change has to be considered in the architecture design. A clean, carefully laid 2012, IJARCSSE All Rights Reserved

www.ijarcsse.com out architecture requires up-front investments in various design considerations, including high cohesion, low coupling, separation of modules, proper system closure, concise interfaces, avoidance of complexity, etc. These investments, however, are worthwhile since they eventually help to increase software reliability and reduce operation and maintenance costs. One central research issue for software architecture concerning reliability is the design of failure-resilient architecture. This requires an effective software architecture design which can guarantee separation of components when software executes. When component failures occur in the system, they can then be quickly identified and properly contained. Various techniques can be explored in such a design. For example, memory protection prevents interference and failure propagation between different application processes. Guaranteed separation between applications has been a major requirement for the integration of multiple software services in complicated modern systems. It should be noted that the separation methods can support one another, and usually they are combined for achieve better reliability returns. Exploiting this synergy for reliability assessment is a possibility for further exploration. These methods favour reliability engineering in multiple ways. First of all, they directly increase reliability by reducing the frequency and severity of failures. Run-time protections may also detect faults before they cause serious failures. After failures, they make fault diagnosis easier, and thus accelerate reliability improvements. For reliability assessment, these failure prevention methods reduce the uncertainties of application interdependencies or unexpected environments. So, for instance, having sufficient separation between running applications ensures that when we port an application to a new platform, we can trust its failure rate to equal that experienced in a similar use on a previous platform plus that of the new platform, rather than being also affected by the specific combination of other applications present on the new platform. Structure based reliability models can then be employed with this system aspect in place. With this modelling framework assisted by well-engineered software architecture, the range of applicability of structure based models can further be increased. Examples of new applications could be to specify and investigate failure dependence between components, to cope with wide variations of reliability depending on the usage environment, and to assess the impact of system risk when components are checked-in or checked-out of the system. B. Testing for reliability assessment

Page | 353

Volume 2, Issue 3, March 2012 Software testing and software reliability have traditionally belonged to two separate communities. Software testers test software without referring to how software will operate in the field, as often the environment cannot be fully represented in the laboratory. Consequently they design test cases for exceptional and boundary conditions, and they spend more time trying to break the software than conducting normal operations. Software reliability measurers, on the other hand, insist that software should be tested according to its operational profile in order to allow accurate reliability estimation and prediction. In the future, it will be important to bring the two groups together, so that on the one hand, software testing can be effectively conducted, while on the other hand, software reliability can be accurately measured. One approach is to measure the test compression factor, which is defined as the ratio between the mean time between failures during operation and during testing. This factor can be empirically determined so that software reliability in the field can be predicted from that estimated during testing. Another approach is to ascertain how other testing related factors can be incorporated into software reliability modelling, so that accurate measures can be obtained based on the effectiveness of testing efforts. Recent studies have investigated the effect of code coverage on fault detection under different testing profiles, using different coverage metrics, and have studied its application in reducing test set size [30]. Experimental data are required to evaluate code coverage and determine whether it is a trustworthy indicator for the effectiveness of a test set with respect to fault detection capability. Also, the effect of code coverage on fault detection may vary under different testing profiles. The correlation between code coverage and fault coverage should be examined across different testing schemes, including function testing, random testing, normal testing, and exception testing. In other words, white box testing and black box testing should be cross checked for their effectiveness in exploring faults, and thus yielding reliability increase. Furthermore, evidence for variation between different coverage metrics can also established. Some metrics may be independent and some correlated. The quantitative relationship between different code coverage metrics and fault detection capability should be assessed, so that redundant metrics can be removed, and orthogonal ones can be combined. New findings about the effect of code coverage and other metrics on fault detection can be used to guide the selection and evaluation of test cases under various testing profiles, and a systematic testing scheme with predictable reliability achievement can therefore be derived. 2012, IJARCSSE All Rights Reserved

www.ijarcsse.com Reducing test set size is a key goal in software testing. Different testing metrics should be evaluated regarding whether they are good filters in reducing the test set size, while maintaining the same effectiveness in achieving reliability. This assessment should be conducted under various testing scenarios [8]. If such a filtering capability can be established, then the effectiveness of test cases can be quantitatively determined when they are designed. This would allow the prediction of reliability growth with the creation a test set before it is executed on the software, thus facilitating early reliability prediction and possible feedback control for better test set design schemes. Other than linking software testing and reliability with code coverage, statistical learning techniques may offer another promising avenue to explore. In particular, statistical debugging approaches [26], whose original purpose was to identify software faults with probabilistic modelling of program predicates, can provide a fine quantitative assessment of program codes with respect to software faults. They can therefore help to establish accurate software reliability prediction models based on program structures under testing. C. Metrics for reliability prediction Today it is almost a mandate for companies to collect software metrics as an indication of a maturing software development process. While it is not hard to collect metrics data, it is not easy to collect clean and consistent data. It is even more difficult to derive meaningful results from the collected metrics data. Collecting metrics data for software reliability prediction purposes across various projects and applications is a major challenge. Moreover, industrial software engineering data, particularly those related to system failures, are historically hard to obtain across a range of organizations. It will be important for a variety of sources (such as NASA, Microsoft, IBM, Cisco, etc.) across industry and academia to make available realfailure data for joint investigation to establish credible reliability analysis procedures. Such a joint effort should define (1) what data to collect by considering domain sensitivities, accessibility, privacy, and utility; (2) how to collect data in terms of tools and techniques; and (3) how to interpret and analyze the data using existing techniques. In addition to industrial data collection efforts, novel methods to improve reliability prediction are actively being researched. For example, by extracting rich information from metrics data using a sound statistical and probability foundation, Bayesian Belief Networks (BBNs) offer a promising direction for investigation in software

Page | 354

Volume 2, Issue 3, March 2012 engineering [7]. BBNs provide an attractive formalism for different software cases. The technique allows software engineers to describe prior knowledge about software development quality and software verification and validation (SV&V) quality, with manageable visual descriptions and automated inferences. The software reliability process can then be modified with inference from observed failures, and future reliability can be predicted. With proper engagement of software metrics, this is likely to be a powerful tool for reliability assessment of software based systems, finding applications in predicting software defects, forecasting software reliability, and determining runaway projects [1]. Furthermore, traditional reliability models can be enhanced to incorporate some testing completeness or effectiveness metrics, such as code coverage, as well as their traditional testing-time based metrics. The key idea is that failure detection is not only related to the time that the software is under testing, but also what fraction of the code has been executed by the testing. The effect of testing time on reliability can be estimated using distributions from traditional SRGMs. However, new models are needed to describe the effect of coverage on reliability. These two dimensions, testing time and coverage, are not orthogonal. The degree of dependency between them is thus an open problem for investigation. Formulation of new reliability models which integrate time and coverage measurements for reliability prediction would be a promising direction. One drawback of the current metrics and data collection process is that it is a one-way, open-loop avenue: while metrics of the development process can indicate or predict the outcome quality, such as the reliability, of the resulting product, they often cannot provide feedback to the process regarding how to make improvement. Metrics would present tremendous benefits to reliability engineering if they could achieve not just prediction, but also refinement. Traditional software reliability models take metrics (such as defect density or times between failures) as input and produce reliability quantity as the output. In the future, a reverse function is urgently called for: given a reliability goal, what should the reliability process (and the resulting metrics) look like? By providing such feedback, it is expected that a closed loop software reliability engineering process can be informative as well as beneficial in achieving predictably reliable software. D. Reliability for emerging software Applications Software engineering targeted for general systems may be too ambitious. It may find more successful 2012, IJARCSSE All Rights Reserved

www.ijarcsse.com applications if it is domain-specific. In this Future of Software Engineering volume, future software engineering techniques for a number of emerging application domains have been thoroughly discussed. Emerging software applications also create abundant opportunities for domain-specific reliability engineering. One key industry in which software will have a tremendous presence is the service industry. Service oriented design has been employed since the 1990s in the telecommunications industry, and it reached software engineering community as a powerful paradigm for Web service development, in which standardized interfaces and protocols graduall enabled the use of third-party functionality over the Internet, creating seamless vertical integration and enterprise process management for cross-platform, cross-provider, and cross-domain applications. Basedon the future trends for Web application development as laid out in [22], software reliability engineering for this emerging technique poses enormous challenges and opportunities. The design of reliable Web services and the assessment of Web service reliability are novel and open research questions. On the one hand, having abundant service providers in a Web service makes the design diversity approach suddenly appealing, as the diversified service design is perceived not as cost, but as an available resource. On the other hand, this unplanned diversity may not be equipped with the necessary quality, and the compatibility among various service providers can pose major problems. Seamless Web service composition in this emerging application domain is therefore a central issue for reliability engineering. Extensive experiments are required in the area of measurement of Web service reliability. Some investigations have been initiated with limited success [27], but more efforts are needed. Researchers have proposed the publish/subscribe paradigm as a basis for middleware platforms that support software applications composed of highly evolvable and dynamic federations of components. In this approach, components do not interact with each other directly; instead an additional middleware mediates their communications. Publish/subscribe middleware decouples the communication among components and supports implicit bindings among components. The sender does not know the identity of the receivers of its messages, but the middleware identifies them dynamically. Consequently new components can dynamically join the federation, become immediately active, and cooperate with the other components without requiring any reconfiguration of the architecture. Interested readers can refer to

Page | 355

Volume 2, Issue 3, March 2012 [21] for future trends in middleware-based software engineering technologies. The open system approach is another trend in software applications. Closed-world assumptions do not hold in an increasing number of cases, especially in ubiquitous and pervasive computing settings, where the world is intrinsically open. Applications cover a wide range of areas, from dynamic supply-chain management, dynamic enterprise federations, and virtual endeavors, on the enterprise level, to automotive applications and home automation on the embedded-systems level. In an open world, the environment changes continuously. Software must adapt and react dynamically to changes, even if they are unanticipated. Moreover, the world is open to new components that context changes could make dynamically available, for example, due to mobility. Systems can discover and bind such components dynamically to the application while it is executing. The software must therefore exhibit a self-organization capability. In other words, the traditional solution that software designer adopted carefully elicit change requests, prioritize them, specify them, design changes, implement and test, then redeploy the software is no longer viable. More flexible and dynamically adjustable reliability engineering paradigms for rapid responses to software evolution are required. V. CONCLUSION Todays as the cost of software application failures grows and as these failures increasingly impact business performance, software reliability will become progressively more important. Employing effective software reliability engineering techniques to increase product and process reliability would be the industrys best interests as well as major challenges. In this research paper, we have reviewed the history of software reliability engineering, the current trends and existing problems, and specific difficulties. Possible future directions and promising research problems in software reliability engineering have also been addressed. We have laid out the current and possible future trends for software reliability engineering in terms of meeting industry and customer needs. In this paper, we have identified new software reliability engineering paradigms by taking software architectures, testing techniques, and software failure manifestation mechanisms into consideration. REFERENCES
[1] A. Bertolino, Software Testing Research: Achievements, Challenges, Dreams, Future of Software Engineering 2007, L. Briand and A. Wolf (eds.), IEEECS Press, 2007. [2] S. Amasaki, O. Mizuno, T. Kikuno, and Y. Takagi, A Bayesian Belief Network for Predicting Residual

www.ijarcsse.com
Faults in Software Products, Proceedings of 14th International Symposium on Software Reliability Engineering (ISSRE2003), November 2003, pp. 215-226, [3] ANSI/IEEE, Standard Glossary of Software Engineering Terminology, STD-729-1991, ANSI/IEEE, 1991. [4] A. Bertolino, Software Testing Research: Achievements, Challenges, Dreams, Future of Software Engineering 2007, L. Briand and A. Wolf (eds.), IEEECS Press, 2007. [5] J. Bishop and N. Horspool, Cross-Platform Development: Software That Lasts, IEEE Computer, October 2006, pp. 26-35. [6] L. Briand and D. Pfahl, Using Simulation for Assessing the Real Impact of Test Coverage on Defect Coverage, IEEE Transactions on Reliability, vol. 49, no. 1, March 2000, pp. 60-70. [7] J. Cheng, D.A. Bell, and W. Liu, Learning Belief Networks from Data: An Information Theory Based Approach, Proceedings of the Sixth International Conference on Information and Knowledge Management, Las Vegas, 1997, pp. 325-331. [8] X. Cai and M.R. Lyu, The Effect of Code Coverage on Fault Detection Under Different Testing Profiles, ICSE 2005 Workshop on Advances in Model-Based Software Testing (A-MOST), St. Louis, Missouri, May 2005. [9] X. Cai, M.R. Lyu, and K.F. Wong, A Generic Environment for COTS Testing and Quality Prediction, Testing Commercial-off-the-shelf Components and Systems, S. Beydeda and V. Gruhn (eds.), SpringerVerlag, Berlin, 2005, pp. 315-347. [10] X. Cai, M.R. Lyu, and M.A. Vouk, An Experimental Evaluation on Reliability Features of NVersion Programming, in Proceedings 16th International Symposium on Software Reliability Engineering (ISSRE2005), Chicago, Illinois, Nov. 8-11, 2005. [11] X. Cai and M.R. Lyu, An Empirical Study on Reliability and Fault Correlation Models for Diverse Software Systems, in Proceedings 15th International Symposium on Software Reliability Engineering (ISSRE2004), Saint-Malo, France, Nov. 2004, pp.125136. [12] J.B. Dugan and M.R. Lyu, Dependability Modelling for Fault-Tolerant Software and Systems, in Software Fault Tolerance, M. R. Lyu (ed.), New York: Wiley, 1995, pp. 109138. [13] D.E. Eckhardt and L.D. Lee, A Theoretical Basis for the Analysis of Multiversion Software Subject to Coincident Errors, IEEE Transactions on Software Engineering, vol. 11, no. 12, December 1985, pp. 1511 1517. [14] P.G. Frankl and E.J. Weyuker, An Applicable Family of Data Flow Testing Criteria, IEEE Transactions on Software Engineering, vol. 14, no. 10, October 1988, pp. 1483-1498. [15] V. Issarny, M. Caporuscio, and N. Georgantas: A Perspective on the Future of Middleware-Based Software Engineering, Future of Software Engineering 2007, L. Briand and A. Wolf (eds.), IEEE-CS Press, 2007. [16] M. Jazayeri, Web Application Development: The Coming Trends, Future of Software Engineering 2007, L. Briand and A. Wolf (eds.), IEEE-CS Press, 2007. [17] Z. Jelinski and P.B. Moranda, Software Reliability

2012, IJARCSSE All Rights Reserved

Page | 356

Volume 2, Issue 3, March 2012

Research, in Proceedings of the Statistical Methods for the Evaluation of Computer System Performance, Academic Press, 1972, pp. 465-484. [18] C. Liu, L. Fei, X. Yan, J. Han, and S. Midkiff, Statistical Debugging: A Hypothesis Testing-based Approach, IEEE Transaction on Software Engineering, vol. 32, no. 10, October, 2006, pp. 831-848. [19] M.R. Lyu (ed.), Handbook of Software Reliability Engineering, IEEE Computer Society Press and McGraw- Hill, 1996. [20] T. Margaria and B. Steffen, Service Engineering: Linking Business and IT, IEEE Computer, October 2006, pp. 45-55. [21] J.D. Musa, Software Reliability Engineering: More Reliable Software Faster and Cheaper (2nd Edition), Author House, 2004. [22] J.D. Musa, Operational Profiles in Software Reliability Engineering, IEEE Software, Volume 10, Issue 2, March 1993, pp. 14-32. [23] J.D. Musa, A. Iannino, and K. Okumoto, Software Reliability: Measurement, Prediction, Application, McGraw- Hill, Inc., New York, NY, 1987. [24] H. Pham, Software Reliability, Springer, Singapore, 2000. [25] P.T. Popov, L. Strigini, J. May, and S. Kuball, Estimating Bounds on the Reliability of Diverse Systems, IEEE Transactions on Software Engineering, vol. 29, no. 4, April 2003, pp. 345359. [26] Rome Laboratory (RL), Methodology for Software Reliability Prediction and Assessment, Technical Report RLTR- 92-52, volumes 1 and 2, 1992. [27] W. Torres-Pomales, Software Fault Tolerance: A Tutorial, NASA Langley Research Center, Hampton, Virginia, TM-2000-210616, Oct. 2000. [45] K.S. Trivedi, SHARPE 2002: Symbolic Hierarchical Automated Reliability and Performance Evaluator, in Proceedings International Conference on Dependable Systems and Networks, 2002.

www.ijarcsse.com

2012, IJARCSSE All Rights Reserved

Page | 357