Module 7

Software Testing & Maintenance

7.1 Introduction to Software Testing The development of software systems involves a series of production activities where opportunities for injection of human fallibilities are enormous. Errors may begin to occur at the very inception of the process where the objectives may be erroneously or imperfectly specified, as well as in! later design and development stages. "ecause of human inability to perform and communicate with perfection, software development is accompanied by a #uality assurance activity. Software testing is a critical element of software #uality assurance and represents the ultimate review of specification, design, and code generation.

The increasing visibility of software as a system element and the attendant $costs$ associated with a software failure are motivating forces for well%planned, thorough testing. &t is not unusual for a software development organi'ation to e(pend between )* and +* percent of total project effort on testing. &n the e(treme, testing of human%rated software ,e.g., flight control, nuclear reactor monitoring- can cost three to five times as much as all other software engineering steps combined. What is it? /nce source code has been generated, software must be tested to uncover ,and correct- as many errors as possible before delivery to your customer. Software Testing techni#ues provide systematic guidance for designing tests that ,0- e(ercise the internal logic of software components, and ,1- e(ercise the input and output domains of the program to uncover errors in program function, behavior and performance. Who does it? 2uring early stages of testing, a software engineer performs all tests. 3owever, as the testing process progresses, testing specialists may become involved. Why is it important? 4eviews and other S56 activities can and do uncover errors, but they are not sufficient. Every time the program is e(ecuted, the customer tests it. Therefore, we have to e(ecute the program before it gets to the customer with the specific intent of finding and removing all errors. &n order to find the highest possible number of errors, tests must be conducted systematically and test cases must be designed using disciplined techni#ues. What are the steps? Software is tested from two different perspectives7 ,0- internal program logic is e(ercised using 8white bo(9 test case design techni#ues. Software re#uirements are e(ercised using 8blac: bo(9 test case design techni#ues. &n both

cases, the intent is to find the ma(imum number of errors with the minimum amount of effort and time. What is the work product? 6 set of test cases designed to e(ercise both internal logic and e(ternal re#uirements is designed and documented, e(pected results are defined, and actual results are recorded. 7.1.1 Software is tested before implementation for the following reasons: 0- ;eave no error for correction in maintenance. 1- To unearth potential problems in design and architecture in the early stages of development. )- To ensure that the right software product is developed, which is valid and verifiable. +- To achieve user satisfaction and customer acceptance of the product. 7. Testing !rocess The testing process usually commences with a test plan, which is the basic document guiding the entire testing of the software. &t specifies the levels of testing and the units that need to be tested. <or each of the different units, first the test cases are specified and then they are reviewed. 2uring the test case e(ecution phase, the test cases are e(ecuted, and various reports are produced for evaluating testing. The main output of the e(ecution phase is the test log, the test summary report, and the error report.

7." #pproaches to Testing There are two approaches to testing7 <unctional and structural.

7.".1 $unctional Testing <unctional testing focuses on the functionality of software system. The tests should confirm that software meets all functional re#uirements such as business processing, features and facilities that are in S4S and succeeds in meeting the needs of the users. <unction or functionality is a result of wor:ing together of modules and the applications within them. &n functional testing, the internal logic of the system under testing is not considered and the test cases are decided from the Specifications or the re#uirements. &t is often called 8"lac: "o( Testing9. 7.". Structural Testing Structural testing is concerned with testing the implementation of the program. The intent of structural testing is not to e(ercise all the different input or output conditions but to e(ercise the different programming structures and data structures used in the program. To test the structure of a program, structural testing aims to achieve test cases that will force the desired coverage of different structures. There are three different approaches to structural testing7 Control flow- based testing, data flow-testing, and mutation- testing. Control-based testing7 % here the control flow graph of a program is considered and coverage of various aspects of the graph is specified as criteria. Data flow-testing7 % here information about where the variables are defined and where the definitions are used is also used to specify the test cases. The basic idea behind data flow testing is to ma:e

sure that during testing, the definitions of variables and their subse#uent use is tested. Mutation testing:- &t is another structural testing techni#ue that differs from the other approaches. &n control flow based and data flow based testing, the focus was on which paths to e(ecute during testing. Mutation testing does not ta:e path%based approach. &nstead, it ta:es the program and creates many mutants of it by ma:ing simple changes to the program. &t re#uires the set of test cases to be such that they can distinguish between the original program and its mutants. 7.% &e'els of Testing 2ifferent levels of testing are used in the testing process, each level of testing aims to test different aspects of the system. =lient >eeds 6cceptance testing

4e#uirements

System Testing

2esign

&ntegration Testing

=ode $ig: &e'els of testing

?nit testing

The basic levels are unit testing, integration testing, and system and acceptance testing. These different levels of testing attempt to detect different types of faults.

The first level of testing is called unit testing. &n this, different modules are tested against the specifications produced during design for the modules. ?nit testing is essentially for verification of the code produced during the coding phase, and hence goal is to test the internal logic of the modules. The ne(t level of testing is called integration testing. &n this, many unit tested modules are combined into subsystems, which are then tested. The goal here is to see if the modules can be integrated properly. 3ence, the emphasis is on testing interfaces between modules. The ne(t levels are system and acceptance testing. 3ere the entire software system is tested. The reference document for this process is the re#uirements document, and the goal is to see if the software meets its re#uirements. This is essentially a validation e(ercise, and in many situations it is the only validation activity. 6cceptance testing is sometimes performed with realistic data of the client to demonstrate that the software is wor:ing satisfactorily. 7.( )lack )o* and white bo* testing 6 rich variety of test case design methods have evolved for software. These methods provide the developer with a systematic approach to testing. More important, methods provide a mechanism that can help to ensure the completeness of tests and provide the highest li:elihood for uncovering errors in software. 6ny engineered product ,and most other things- can be tested in one of two ways7

,0- @nowing the specified function that a product has been designed to perform, tests can be conducted that demonstrate each function is fully operational while at the same time searching for errors in each functionA ,1- @nowing the internal wor:ings of a product, tests can be conducted to ensure that $all gears mesh,$ that is, internal operations are performed according to specifications and all internal components have been ade#uately e(ercised. The first test approach is called blac:%bo( testing and the second, white%bo( testing. Bhen computer software is considered, black-box testing alludes to tests that are conducted at the software interface. 6lthough they are designed to uncover errors, blac:%bo( tests are used to demonstrate that software functions are operational, that input is properly accepted and output is correctly produced, and that the integrity of e(ternal information ,e.g., a database- is maintained. 6 blac:%bo( test e(amines some fundamental aspect of a system with little regard for the internal logical structure of the software.

White-box testing of software is predicated on close e(amination of procedural detail. ;ogical paths through the software are tested by providing test cases that e(ercise specific sets of conditions andCor loops. The $status of the program$ may be e(amined at various points to determine if the e(pected or asserted status corresponds to the actual status.
7.5.1 BLACK-BOX TESTING

lack-box testing, also called beha!ioral testing, focuses on the functional re#uirements of the software.

That is, blac:%bo( testing enables the software engineer to derive sets of input conditions that will fully e(ercise all functional re#uirements for a program. "lac:%bo( testing is not an alternative to white%bo( techni#ues. 4ather, it is a complementary approach that is li:ely to uncover a different class of errors than white%bo( methods. "lac:%bo( testing attempts to find errors in the following categories7 ,0- incorrect or missing functions, ,1- interface errors, ,)- errors in data structures or e(ternal data base access, ,+behavior or performance errors, and ,D- initiali'ation and termination errors. "lac:bo( testing tends to be applied during later stages of testing. "ecause blac:%bo( testing purposely disregards control structure, attention is focused on the information domain. Tests are designed to answer the following #uestions7 E 3ow is functional validity testedF E 3ow is system behavior and performance testedF E Bhat classes of input will ma:e good test casesF E &s the system particularly sensitive to certain input valuesF E 3ow are the boundaries of a data class isolatedF E Bhat data rates and data volume can the system tolerateF E Bhat effect will specific combinations of data have on system operationF "y applying blac:%bo( techni#ues, we derive a set of test cases that satisfy the following criteria ,0- test cases that reduce, by a count that is greater than one, the number of additional test cases that must be designed to achieve reasonable testing and ,1- test cases that tell us something about the presence or absence of classes of errors, rather than an error associated only with the specific test at hand. 7.5.2 WHITE-BOX TESTING

Bhite%bo( testing, sometimes called glass-box testing, is a test case design method that uses the control structure of the procedural design to derive test cases. ?sing white%bo( testing methods, the software engineer can derive test cases that ,0- guarantee that all independent paths within a module have been e(ercised at least once, ,1- e(ercise all logical decisions on their true and false sides, ,)- e(ecute all loops at their boundaries and within their operational bounds, and ,+- e(ercise internal data structures to ensure their validity. 7.+ The art of ,ebugging Software testing is a process that can be systematically planned and specified. Test case design can be conducted, a strategy can be defined, and results can be evaluated against prescribed e(pectations. Debugging occurs as a conse#uence of successful testing. That is, when a test case uncovers an error, debugging is the process that results in the removal of the error. 6lthough debugging can and should be an orderly process, it is still very much an art. 6 software engineer, evaluating the results of a test, is often confronted with a $symptomatic$ indication of a software problem. That is, the e(ternal manifestation of the error and the internal cause of the error may have no obvious relationship to one another. The poorly understood mental process that connects a symptom to a cause is debugging. 7.+.1 The ,ebugging !rocess 2ebugging is not testing but always occurs as a conse#uence of testing. the debugging process begins with the e(ecution of a test case. 4esults are assessed and a lac: of correspondence between

e(pected and actual performance is encountered. &n many cases, the non corresponding data are a symptom of an underlying cause as yet hidden. The debugging process attempts to match symptom with cause, thereby leading to error correction. The debugging process will always have one of two outcomes7 ,0- the cause will be found and corrected, or ,1- the cause will not be found. &n the latter case, the person performing debugging may suspect a cause, design a test case to help validate that suspicion, and wor: toward error correction in an iterative fashion. 7.+. Why is debugging so difficultF &n all li:elihood, human psychology has more to do with an answer than software technology. 3owever, a few characteristics of bugs provide some clues7 1. The symptom and the cause may be geographically remote. That is, the symptom may appear in one part of a program, while the cause may actually be located at a site that is far removed. 3ighly coupled program structures e(acerbate this situation. . The symptom may disappear ,temporarily- when another error is corrected. ". The symptom may actually be caused by nonerrors ,e.g., round% off inaccuracies-. %. The symptom may be caused by human error that is not easily traced. (. The symptom may be a result of timing problems, rather than processing problems. +. &t may be difficult to accurately reproduce input conditions ,e.g., a real%time application in which input ordering is indeterminate-.

7. The symptom may be intermittent. This is particularly common in embedded systems that couple hardware and software ine(tricably. -. The symptom may be due to causes that are distributed across a number of tas:s running on different processors 2uring debugging, we encounter errors that range from mildly annoying ,e.g., an incorrect output format- to catastrophic ,e.g. the system fails, causing serious economic or physical damage-. 6s the conse#uences of an error increase, the amount of pressure to find the cause also increases. /ften, pressure sometimes forces a software developer to fi( one error and at the same time introduce two more.

7.7 .ecessity of software maintenance Software maintenance is becoming an important activity of a large number of software organi'ations. This is no surprise, given the rate of hardware obsolescence, the immortality of a software product per se, and the demand of the user community to see the e(isting software products run on newer platforms, run in newer environments, andCor with enhanced features. Bhen the hardware platform is changed, and a software product performs some low% level functions, maintenance is necessary. 6lso, whenever the support environment of a software product changes, the software product re#uires rewor: to cope up with the newer interface. <or instance, a software product may need to be maintained when the operating system changes. Thus, every software product continues to evolve after its development through maintenance efforts. Therefore it can be stated that software maintenance is needed to correct errors, enhance features, port the software to new platforms, etc.

7.7.1 Types of software maintenance There are basically three types of software maintenance. These are7 /orrecti'e: =orrective maintenance of a software product is necessary to rectify the bugs observed while the system is in use. #dapti'e: 6 software product might need maintenance when the customers need the product to run on new platforms, on new operating systems, or when they need the product to interface with new hardware or software. !erfecti'e: Software product needs maintenance to support the new features that users want it to support, to change different functionalities of the system according to customer demands, or to enhance the performance of the system 7.- !roblems associated with software maintenance Software maintenance wor: typically is much more e(pensive than what it should be and ta:es more time than re#uired. &n software organi'ations, maintenance wor: is mostly carried out using ad hoc techni#ues. The primary reason being that software maintenance is one of the most neglected areas of software engineering. Even though software maintenance is fast becoming an important area of wor: for many companies as the software products of yester years age, still software maintenance is mostly being carried out as fire%fighting operations, rather than through systematic and planned activities. Software maintenance has a very poor image in industry. Therefore, an organi'ation often cannot employ bright engineers to carry out maintenance wor:. Even though maintenance suffers from a poor image, the wor: involved is often more challenging than development wor:.

2uring maintenance it is necessary to thoroughly understand someone elseGs wor: and then carry out the re#uired modifications and e(tensions. 6nother problem associated with maintenance wor: is that the majority of software products needing maintenance are legacy products. 7.0 Software re'erse engineering Software reverse engineering is the process of recovering the design and the re#uirements specification of a product from an analysis of its code. The purpose of reverse engineering is to facilitate maintenance wor: by improving the understandability of a system and to produce the necessary documents for a legacy system. 4everse engineering is becoming important, since legacy software products lac: proper documentation, and are highly unstructured. Even well%designed products become legacy software as their structure degrades through a series of maintenance efforts. The first stage of reverse engineering usually focuses on carrying out cosmetic changes to the code to improve its readability, structure, and understandability, without changing of its functionalities. 6 process model for reverse engineering has been shown in fig. below.

6fter the cosmetic changes have been carried out on a legacy software, the process of e(tracting the code, design, and the re#uirements specification can begin. These activities are schematically shown in fig. 0+.1. &n order to e(tract the design, a full understanding of the code is needed. Some automatic tools can be used to derive the data flow and control flow diagram from the code. The structure chart ,module invocation se#uence and data interchange among modules- should also be e(tracted. The S4S document can be written once the full code has been thoroughly understood and the design e(tracted.

$ig: 1%. 7.11 $actors on which software maintenance acti'ities depend The activities involved in a software maintenance project are not uni#ue and depend on several factors such as7 0- E(tent of modification to the product re#uired 1- 4esources available to the maintenance team )- =onditions of the e(isting product ,e.g., how structured it is, how well documented it is, etc.-

+- E(pected project ris:s, etc. Bhen the changes needed to a software product are minor and straightforward, the code can be directly modified and the changes appropriately reflected in all the documents. "ut more elaborate activities are re#uired when the re#uired changes are not so trivial. ?sually, for comple( maintenance projects for legacy systems, the software process can be represented by a reverse engineering cycle followed by a forward engineering cycle with an emphasis on as much reuse as possible from the e(isting code and other documents. 7.11 Software maintenance process models Two broad categories of process models for software maintenance can be proposed. The first model is preferred for projects involving small rewor:s where the code is changed directly and the changes are reflected in the relevant documents later. &n this approach, the project starts by gathering the re#uirements for changes. The re#uirements are ne(t analy'ed to formulate the strategies to be adopted for code change. 6t this stage, the association of at least a few members of the original development team goes a long way in reducing the cycle team, especially for projects involving unstructured and inade#uately documented code. The availability of a wor:ing old system to the maintenance engineers at the maintenance site greatly facilitates the tas: of the maintenance team as they get a good insight into the wor:ing of the old system and also can compare the wor:ing of their modified system with the old system. 6lso, debugging of the reengineered system becomes easier as the program traces of both the systems can be compared to locali'e the bugs.

Fig. 14.3: Maintenance process model 1

The second process model for software maintenance is preferred for projects where the amount of rewor: re#uired is significant. This approach can be represented by a reverse engineering cycle followed by a forward engineering cycle. Such an approach is also :nown as software reengineering. This process model is depicted in fig. 0+.+. The reverse engineering cycle is re#uired for legacy products. 2uring the reverse engineering, the old code is analy'ed ,abstracted- to e(tract the module specifications. The module specifications are then analy'ed to produce the design. The design is analy'ed ,abstracted- to produce the original re#uirements specification. The change re#uests are then applied to

this re#uirements specification to arrive at the new re#uirements specification. 6t the design, module specification, and coding a substantial reuse is made from the reverse engineered products. 6n important advantage of this approach is that it produces a more structured design compared to what the original product had, produces good documentation, and very often results in increased efficiency. The efficiency improvements are brought about by a more efficient design. 3owever, this approach is more costly than the first approach. 6n empirical study indicates that process 0 is preferable when the amount of rewor: is no more than 0DH. "esides the amount of rewor:, several other factors might affect the decision regarding using process model 0 over process model 17 E 4eengineering might be preferable for products which E e(hibit a high failure rate. .

Fig. 14.4: Maintenance process model 2

7.1 Software /onfiguration 2anagement "oftware configuration management ,S=M- is an umbrella activity that is applied throughout the software process. "ecause change can occur at any time, S=M activities are developed to ,0- identify change, ,1- control change, ,)- ensure that change is being properly implemented, and ,+report changes to others who may have an interest. &t is important to ma:e a clear distinction between software support and software configuration management.

Support is a set of software engineering activities that occur after software has been delivered to the customer and put into operation.

Software configuration management is a set of trac:ing and control activities that begin when a software engineering project begins and terminate only when the software is ta:en out of operation. 6 primary goal of software engineering is to improve the ease with which changes can be accommodated and reduce the amount of effort e(pended when changes must be made. &n this chapter, we discuss the specific activities that enable us to manage change What is it? Bhen you build computer software, change happens. 6nd because it happens, you need to control it effectively. Software configuration management ,S=M- is a set of activities designed to control change by identifying the wor: products that are li:ely to change, establishing relationships among them, defining mechanisms for managing different versions of these wor: products, controlling the changes imposed, and auditing and reporting on the changes made. Who does it? Everyone involved in the software engineering process is involved with S=M to some e(tent, but speciali'ed support positions are sometimes created to manage the S=M process. What are the steps? "ecause many wor: products are produced when software is built, each must be uni#uely identified. /nce this is accomplished, mechanisms for version and change control can be established. To ensure that #uality is maintained as changes are made, the process is auditedA and to ensure that those with a need to :now are informed about changes, reporting is conducted. The output of the software process is information that may be divided into three broad categories7 ,0- computer programs ,both source level and e(ecutable forms-A ,1- documents that describe the computer programs ,targeted at both technical practitioners and users-, and ,)- data ,contained within the program or e(ternal to it-. The items that comprise

all information produced as part of the software process are collectively called a software configuration. 6s the software process progresses, the number of software configuration items ,S=&s- grows rapidly. 6 "ystem "pecification spawns a "oftware #ro$ect #lan and "oftware %e&uirements "pecification ,as well as hardware related documents-. These in turn spawn other documents to create a hierarchy of information. &f each S=& simply spawned other S=&s, little confusion would result. ?nfortunately, another variable enters the processIchange. =hange may occur at any time, for any reason. There are four fundamental sources of change7 E >ew business or mar:et conditions dictate changes in product re#uirements or business rules. E >ew customer needs demand modification of data produced by information systems, functionality delivered by products, or services delivered by a computer%based system. E 4eorgani'ation or business growthCdownsi'ing causes changes in project priorities or software engineering team structure. E "udgetary or scheduling constraints cause a redefinition of the system or product. Software configuration management is a set of activities that have been developed to manage change throughout the life cycle of computer software. S=M can be viewed as a software #uality assurance activity that is applied throughout the software process.. 7.1" Software /onfiguration ,ocumentation )aseline

&n the conte(t of software engineering, a baseline is a milestone in the development of software that is mar:ed by the delivery of one or more software configuration items and the approval of these S=&s that is obtained through a formal technical review. <or e(ample, the elements of a Design "pecification have been documented and reviewed. Errors are found and corrected. /nce all parts of the specification have been reviewed, corrected and then approved, the Design "pecification becomes a baseline. <urther changes to the program architecture ,documented in the Design "pecification- can be made only after each has been evaluated and approved. 6lthough baselines can be defined at any level of detail, the most common software baselines are shown in <igure J.0

7.1% Software /onfiguration Items

S=&s are organi'ed to form configuration ob$ects that may be cataloged in the project database with a single name. 6 configuration object has a name, attributes, and is $connected$ to other objects by relationships. 4eferring to <igure J.1, the configuration objects, ,esign Specification3 data model3 component .3 source code and Test Specification are each defined separately. 3owever, each of the objects is related to the others as shown by the arrows. 6 curved arrow indicates a compositional relation. That is, data model and component . are part of the object ,esign Specification. 6 double%headed straight arrow indicates an interrelationship. &f a change were made to the source code object, the interrelationships enable a software engineer to determine what other objects ,and S=&s- might be affected.