Software Testing Techniques and Strategies 11.1 Introduction Software has infiltrated almost all areas in the industry and has over the years become more and more Wide spread as a crucial component of many systems. System failure in any industry can be very costly and in the case of critical systems (fight control, nuclear reactor monitoring, medical applications, etc.) it can mean lost human lives. These "cost" factors call for some kind of system failure prevention. One way to ensure system's reliability is to extensively test the system. Since software is a system component, it requires a testing process also. Software testing is the process of executing a program with the intention of finding errors in the code. It is the process of exercising or evaluating a system or system component by manual or automatic means to verify that it satisfies specified requirements or to identify differences between expected and actual results (Shooman 1983; Conte 1986; Sommerville 1998; Pressman 1997). Y aah jecti ing i i d testing is considered to succeed The objective of testing is to show incorrectness an ‘ c when an Sroe is detected (Myers 1979). An error is @ conceptual mistake mule by Sits : i i between a computed value and a _ th designer or a discrepancy bets ee A fault is a specific manifestation of an error. An error may be failure is the inability of @ system or component to perform ‘its required function within the specified limits. A failure may be produced when a fault is executed or exercised. Testing should not be 2 distinct phase in system development but should be applicable See intenance phases. throughout the design, development and maintenance pl : reas ; ; i in the direction of generating some good automat Researchers are also making efforts » Software testing tools and have also been successful to some extent.esi. tins 101 r, al 11.2 Software tet ian of stare etn BOWENE, all OF these, = ‘There ae THEY the same thing? jown to es ccuting software in @ controlled manner ie seth process of executing ogy, fied?” se it HP ee ble a8 See to answer the a Software cation withthe toms verification and yi in ing is often used in 858 1998; Pressman 1997) aad val op eocess of checking that what has been specified is What typ + lidatin isthe actually wanted. Vitidation: Are we doing the right job? Verification: Are we doing the job right? The term bug is often used to refer to a problem or fe software bugs and hard in @ computer. Thee se bugs. The term originated in the United States, tof valves, when a series of previously inexplicable fais traced to moths flying about inside the computer. Sing oa De TT Debuazng isthe process of analysing and loc 45 expected, Although the ident were eventual -_ jin camot rele ttn Fes NBR is therefore an activity that support bugs. ‘wever, no amount of testing can be guaranteed to Other actives that dynamic analy — with software testing are static problems and gather looks at the behaviour rection traces, timing prof igttes the source code of software, | ae ee eae an 113 Testing and the Software Lifecycle * should be thought of as an integral part of th eye carted out throughout the life eycle. SAE process and an activity that ach ml hase in the software lifeeyele has a distin ns specification (SRS) documentation secre outst SUH A the sfc ree Program unit design an romiggch end product can be checked for confomanes wih e ene oes tie Sena ce sr a Validation and Verification should occur thoughout the stateless «Verification is the process of evaluating each phase the end product of the previous phase. + Validation is the process of testing software, matches uset requirements, Software testing is that part of vali mai and analysing program code. It is one of the two most expensive ie other being maintenance. Software of the program units and continues unt fend product to ensure consistency oF a speci {to ensure that it Changing a Requirements document during the first re more when requirements change after the code has been wri Bug fixes are much cheaper when programmers find before releasing a program is much cheaper than sending new disks, or 'o each customer's site to fix Cost Requirements | Coded leased Fig. 11.1: Cost of Finding and Fixing Software Errorsbelow ; ‘A : itn the clients rapid Prototyping gy wirements reviewed with jrements. tel tee nines pdt cane regimens i eat ree checked for feasibility, tr, sfiton jcument ae trace eee specication ee ‘of contradictions and ambiguit compensa ugh of inspections) ate especialy effecig «Speen Design jews, but more technical, terface faults, | a face alts, lack of exci «The design mi i, and non-conformanice * Code modules are informally tested by the programmer while they are being ented (desk checking). ; formal testing of modules is done methodically by a testing team, testing can include non-execution-based methods (code inspect walitoughs) and exeutin-based methods (black-box testing, white-box ing. Integration + Inegration testing i performed to ensure that the modules combine together coneciy to achieve a product that meets its specifications. Particular care must be given the imerfaces between modules, * Ths apport ork of combinstion must be determined as top-down, botmig, 4 & combination there, "i ce ening Tociguts and Stateios Sr eet isee iia ei I product must be tested to ensure that changes have 1 inst previous te hanges have been introduced. ‘This lat ing. st cases to ensure that no ter consider d pre Process Management se rhe software process management plan must undergo portant that cost and duration estimates be cheskes is especially Ny left unchecked, errors can propagate through the development lifecycle and ampli fama fo be more costly as the system develops. An error found dusing the operation phase wate most costly (0 114 Objectives of Software Testing ly performed for the following objectives: All the above software testing objectives are discussed in the following sections. MA, Software Quality Improvement the outcome of a bug can be an cause huge losses. Bugs 1s have: irplane crashes, ns t0 go awry, * Halted trading ‘market, and Worse Bugs can kill and cause disasters. The so-called year 2000 (¥2K) bug could have sulted into more screeching halt on the first day of the century. ees nied ihe quality "ead ty of software is a matter of life and deat ins thare quality means the conformance to the specified sofware design request Dang correct, the minimum requirement of quality, means performing as req ‘Pecified circumstances, Debugging, a narrow view of software testing, is performed kot °S by the programmer, ‘The imperfection of human nature mal to find out design (foc almost impossiblesis, of negative tes ry tests 5 npex poaren of software mo debug i” P survive a significant level of diny icon’ eat exception handling design is a design that can be easily vl A se a rigorous effort and tequites significa also an important design rule for software den rogrammning raely te popes ted, f nt *d and maintained, © and cost, design for pment, verification and vali V), isn the VEV process. Tester caret er either the product works me .3 Software Reliability Estimation important relations with man the amount of testing it has been subj ii jected to, © discover the residual design erors before dalivery to the the testing process are taken down in order to estimne , ing process may function with regular feedback from the is to the testers and designers that is shown in Fig. 11.2. not be sofware uaity hee sets oF Facts: Test inputs ‘TABLE 1.4: Typleal Software Quality Factors Test Engineering ‘Adaptabil ee eae | ety (—) Corecess Efiiney Felabity Testability ¥ Usabity Doce a See J , i ‘Good testing provides measures for all relevant factors, The importance of any particu from application to appli must place extreme emphasis on usability and maintain Fig, 11.2: Role of Reliability in Software Testing as a statis Based on an operational profile, testing can serve Sh failure data for software el eptimation. situations. On the contrary, software does not workagi ‘aman error that is NOt casly traceg SY timing problems, rather the ip) when another error is ort’ POM eed by nonverrOrs (C8. Tound-o are distributed across a mumpg fay rly common in embea, nbedded Yen, ‘hardware and software, af enors (or bugs) in software (OF Programm). Some j + These are generally caught by compiler. Logic/Algorithn: Errors ‘+ These errors occur due 10: = Branching t00 soon tranching errors are often associated wi = Data type mismatch ~ Incorrect formula or compt * These occur due to mismatch between documentation and code. * These errors lead to difficulties especially during maintenance. * These errors ar S "Tor ate due to system performance degradation at capacity. Timing/eoordination Errors {These emors are mainly found in real ti * These errors deal esting Techniques and Strategie vat 2 vaation and Precision Errore Comm ese errors are caused duc to ro * jumbers and conversion, unding/trunca "SSeS, while dealing with rat reslverioad Errors Sr These eHTOTS ate caused When usersdey 1 examples: butfer sizes, ete, roaghpuilperformance Brrors «+ These errors come across due to throughput response time degradation, et a or Performance deradaon For example, ice capacities exceed, Recovery Errors + These are error-handling faults, swandards/Procedures + These don't cause errors in and of themselves but rater erate where errors are created/introduced as the syst is tested environment 116 Reasons For Software Bugs The following are certain reasons why software bugs exis 1, Miscommunication or no communication specifies of what an application should or shouldn't do (the application's requirements), 2 Software Complexity * The complexity of current software applications can be difficult to comprehend for muted apy ial databases, and sheer size of appl Project unless it is well-enginecered, 3: Programming Errors * Programmers, ce anyone else, can make mistakes. 4 Changing Requirements ae * The customer may not understand the effects of changes, or may undirie Fequest them anyway ~ redesign, rescheduling of engineers, effets on oles PrN. Work already completed that may have to be redone or thrown out Tequirements that may be affected, etc.fected. In some Fast-chang ements may be a fact of Pg, i Tisks, and QA and teg¢ ty if 10 Keep the ingyy, ite, jets is cificul at best, offen requiring a tox of BU veasiztons management provides no incentive for programmes jy document tht cade or write clear, understandable code. i usally te opposite: they get points mostly for quickly turning out cit 4b security if nobody else can understand it. * Sofwatedeeopnent tos - visual tools, clas libraries, compilers, scripting ik ten introduce tet ovn bugs or are poorly documented, resulting in a3 gs. ‘ 17 Principles of Software Testing Sofware testing i of sata quiy en COMPTEN of the software engineering process prog Such @ manner a8 to tacos 8 & described as a process of runing. 4 P cis tiresome and un ia 2 STO. This process, while seen by some as necessary, The pres of sages idle in software developmen before these can be deny Mitt itolves creating test cases to “break the ined, *TE" incites have to be observed: pO esting Techniques and Strategies are testing is an extremely creative ang iq ‘re some important pinipes that shouldbe fant Eooman 1983; Somme 8; Pressman 199; sting should be based on user reguiremenes Thi defects that might cause the program or system resting time and resources are |i ly challenging task. The ind while carrying software ossible to test everything, Exhaustive to er nar ie hy the number of paths a program ‘ow might aie «Use effective resources to test. This procedures and individuals to conduct the tests, The test team shen use toc they are confident and familar with, Testing procedures should be cat arn ‘Testing personnel may be tchrical group of people independent othe dee + Test planning should be done early le This is because test planning independently of coding and as soon as the client requirements ee ey + Test for invalid and unexpected input conditions as well as program should generate correct messages when an is should generate correct results when the test is vai + The probability of the existence of more errors in @ module or group of modules irectly proportional to the number of errors already found. * Testing should begin at the module. The focus of testing should be concentrated on the smallest programming units first and then expand 10 other parts of the system * Testing must be done by an independent party. Testing should not be performed by the person or team that developed the software since they tend to defend the correctness of the program. Tepresents use of the mos ‘can begin conditions. The ‘encountered and * Assign best personnel to the task. Because testing requires high creativity and ‘esponsibility only the best personnel must be assigned to design, implement, and analyze test cases, test data and test results. * Testing should not be planned under the implicit assumption that no errors will be found, Testing is the process of executing software with the intent of finding errors. Keep software static during test. The program must not be modified during the ‘implementation of the set of designed test cases. Document test cases and test results. Provide expected test results if possible. A necessary part of se cos ion is the specification of expected results, even if providing such results isTest 11.9 Software ‘Testing programs, systems shotl Process cout of sub init can easily be te, sted stics of testability. | Ps ‘. The testing process should therefore pr. rementally in conjunction with system impl. ee Testing (acre Errors in program com process. The process is there stages to eatlier parts of the p s defects ae discovered at any one stage, “arts rem and this may require other stages in the testing process ponents, say may come t fore an iterative one wit roves, they require program i Testing tem The sul med Wi tem, nal ag Keptance This ig £ The sys This phase involves Iabsystems, Sub-sys Techniques and Strategies Integration Testing Fig. 11.8: lterative Software Testing Process sting is code-oriented testing, Individual components ate teed to enute that sted independently, withom other system A module is a collection of dependent components such as an object class, an abstract type or some looser collection of procedures and functions, A module encapsulates ied components so it can be tested without other system modules, Testing ing collections of modules, which have been integrated into oriented testing and is also known as integration ‘resting. tems may be independently designed and implemented. The most common It is a desi lems, which arise in large software systems, are sub-systems interface mismatches, The 'ystem test process should therefore concentrate on the detec ly exercising these interfaces. en Testing of interface errors by - ing. process is beysems are integrated to make vp the ete system The sig pes i th finding errors that result from unanticipated ee the system meets its components. It is also concerned with validating that ind non-functional requirements. Testing js accepted for operational accepted for ope the final stage in the testing process before the system 15 rather than simulated test tem is tested with data supplied by the system client289 program cone else” ‘Hat & programmer perceives exity is difficult to evaluate objectives ons in the 8 requirem omission ght be viewed as complex by som: ve eas an te ETE YS From hg, © “ T requirement problems ‘HETE the syeten, 6 ea a al oF the stem Performance (ny compl sa A 93 structural Analysis ay also analysis i oot of structural analysis iS (0 Uncover tact any zed (fu ies of a test object od Data-flow Analysis ww analysis is intended to help discover paflow formation about whether a data object. fee object is used after an assignment. : ta-flow anomalies, Data-l . Dataflow analysis ws a value before its use and whether ing concern syntactic, structural and gp af ee le ‘ te ge as early as possible, emorpna ~ pacflow analysis applies to both the body of atest object and the ineraces between Ta objects fc program analysis are: ‘i ant actives of static program analys uit Dynamic Analysis for dmamic testing the test objects are executed or simulated. Dynamie analysis is what is eelly considered as testing i.e, it involves running the system, Dynamic testing is an imperative process in the software life cycle. Every procedure, every module and class, every subsystem and the overall system must be tested dynamically, i ications have been carried out. for dynamic testing include: 11.10.1 Code Inspection pe Code inspection is a useful technique for localizing design and implement Sy eno prove eas) fo id if the author Would only read the program carefully CCode inspections have already been discussed in detail in the earlier Chapter, ea behind code inspection is to have the author of a program discuss Software engineers. The moderator notes every detected error inves, The task ofthe inspec of the inspection do the + Preparation of the test object for error localization * Availability of a test environment * Selection of appropriate test cases and data * Test execution and evaluation 112 Software Test Design is subject to the same basic engineering principles as the design of a number of stages that progressively elaborate the ligh-level strategy to detailed test procedures (Pressman 1997; 1998). These stages are: Test Strategy Test Planning Test Case design Test Procedures The results of compl product (within certain system, Test Results Documentationresting Techniques and Strategies gue sre negration Test Plan sofa seibing the plan for integration of his may form part of the Design Spee ification of the software, Ay ee ee soars faith ne ‘verify that items of software iy ian and Detailed Design Spe ign process should be subjec ‘ed software components, ication, at rest Plan(s) nM peseribing the plans for testing of individuat 1 ese may form part of the Detaled Design Units of software Specifications. eis highly dependent on the des, pease ign yy as a Key requirement for any qy0! He ‘he objestive of eath test plan Isto provide a plan for veo Soft ication, by testi e software produced fulfils the functional or design satomcacey © eoeie jon, Im the case of acceptance testing and sytem testing, as ee the ion. sty, th rtegy. A test SIEEY iS a statement of vels of testing are to be applied st strategy should ideally be orga software development. meets the needs of an org: success of software development in the organi: SEATS Shewe doelpmest projet should be det 23 Test Case Design cases should be designed in such a way as to uncover ‘They should “exercise” the program by uss cuits that are both correct and incorrect. Once the test plan for a level of testing has been writen, isto specify a Quickly and easily as many ing and producing inputs and tion is ert ion of at led in the project's softwue 11.122 Test Plans Te next sage of test design is the development of a test plan. A test plan states: What the items to be tested are? «At what level they will be tested? uence they are to be tested in? ‘ex arate will be applied to the testing of each item and describes The objective of test case design is to test all modules and then the whole system as ‘ompletely as possible using a reasonably wide range of conditions. Variables should be tested using all pos values (for small ranges) or typical (rofbound values (for larger ranges). They should also be tested using valid and i ions. Arithmetical and logical comparisons should be examined as wi correct and incorrect parameters. St cases may be documented with the test plan, as a section of a softwar ' oF in a separate document called a test specification or test description. ‘Acceptance Test Specification * Specifying the test cases for acceptance testing of the software. oe * Thi mi blishec ld usually be published as @ separate document, but might be pu?” This would usually be Published as a separate document, but might be pt 2 : With the acceptance test pl Test Specification * Specifying the test eases for system integration and testing. eras * This Would also usually be published as a separate document, but mi With the system test plan. A test plan may be project ee Wide, oF may in fact be a hierarchy of plans rel of specification and testing: Acceptance Test Plan + Desi Ti Stem lan for ist for system integration and testing. 0 usualy ; with he scepane ty re ‘shed a8 a separate document, but might Dsing Techniques and Strategies yo Te! Software Gin ey sypes of Software Testing any ways $0 conduc sofware testing, but the most common methods a jos ion of tested softwar ntegeation Tet ees stage of integration of tested software OPO fore TERI a cas sgn Specification. ‘ + en seaions of he Desi SP we modules ae informally tested by the programmer is referred to as desk checking. After the progtnmer i ‘ese may fo 2 rs 10 function correctly, methodical testing of the modu ‘e are two basic types of methodical testing: Ter Nomexecution-based testing: the module is reviewed by a team. recution-based testing: the module is run agtinst west cases ih above testing {yes are discussed below. in ; hes te ‘software has even been Tt is not uncommon ts tal tests than when executing tests. Te prowess of wil on ey vote bugs when designing jist Non-execution Based Testing non-exccution based testing relies on fault-detection strategy. The fault-detecting power ne based techniques leads to rapid, thorough, and early fault det s-more than repaid by the increased prod 11424 Test Procedures ee of test design isto implement a set of test cases as a test procedye xc process fo be followed to conduct each of the test cases. This hich can be equated to designing units of code from hig is at the integration phase. In general, non-execution-based code testing is less expensive than execution-based sing because: + Execution-based testing (running test cases) can be extremely time-consuming, and of faults earlier also known as static testing (ot static program analysis), his chapter. strighforvard proces functional descriptions. For each item to be tested, at each level of testing, a test procedure will specity is + Reviews lead to det Nonexecution-based testing wich is already discussed abov items, because they contain a great ded of is inleant 0 sofware specifications. 11192 Execution Based Testing 1125 Test Results Documentation fe ele i ee ee jen tess ate execute, the outputs of each test execu ting test data to test a module: Rests Fie, These resus are then assessed against detemine the overal outcome of » ts in the test speci Each : cs ea Ferd Pea also be noted in a test log, The test log will contain so ince key aberatons ensue the outcome of each test execution, and maf He. inte Tevels of testing (ar oetS #68 execution. Often a test log is not maintained ing (unt test and software integration test). Bie 21 Various points during the often forms a contaan "8 tnd document any anal actual document within which ee of feware testing technique y the tester. It i alsoresting Tochnlaues nd Stratepiog go does not need knowledge of + The tester ; any specific programm re wat is done from ihe Pol of view ofthe user non te Laneuces. 1 Jon eases com be designed as 5008 as the pea signer ions are complete 42 pisadvantages of Black-box Testing yvanages of this (pe of testing include, 1 The rex can be redundant the sofware designer has already nna rex a The test cases are difficult to design se, Oe cand tern oh 1 Butemal database access jonality of each esting every possible input stream is unre i because it would take an inordi ; therefore, many program paths i take/an inordinate 80 untested, jodule is tested with regards ‘Only the correct inpuv/output rel noant of ze 1115 Black-box Testing Techniques ‘pe fllowing are main techniques to black-box testing: + Equivalence Class Partitioning + Boundary Value Analysis (BVA) + Cause-Effect Graphs + Comparison Testing lence testing, combined with boundary value analysis, is a black-box technique cases in such a way that new. cases are chosen to detect previously detected faults, Ceuse-effect graph overcomes the weakness of the above two methods of not consi ns. the above techniques are discussed below in more details. For example, in a back box test on soft 1 Equivalence Class Partitioning and what the expected outcomes sho * method divides the input of a program defining an equivalent id and inval to classes of data. Test case design is based nce class represents a set seam ettivalence class is a set of test eases such that any one member of the cassis TPrsematve of any other member of the cls. ; 6 teat the specifications fora database product state that the product must te ale eon, © "'Y number of records from 1 through 16,383. If the pro wh, ttt 14.870 record, then the chances ae good th 'gn a small, manageable set of while minimizing the redundancy 4m0% TIA Advantages of Black. " The advantages of ck-box Testing ‘ype of * Thetis eabiogg 8 include: laced, ote the dsoner andthe tester are independenting Techniques and Strategies es ies to the input specifi valence classes: a i ons; the same techni hnique should be applied vce 1 thre ae record tale ais pote shen one at Fe tce C8 1 18 TT 399 reco mo eran ley aa Py va Price cst 2 FO 6383 records (eta Sil 5 of et dat wa ata sha a et : most faults ith a high proba eo req — A owing ae SOME Important guides for pero Petforming boundary value analysis: ut te database product ect. ; valence Partioning sing to the following. guid JoH8 «an input range bounded by value oi it i ce a a ie ae valence cs values x and y. nape scion species ange, ne valid and {0 inal gy um and Minimum Numbers pt see an input condition specifies a number of alls, fet cases should be developed that exercise the minimum and maximum numbers, + Vales above and below the minimum and maximum are als tse, pt Conditions + Apply the above guidelines to output conditions, eral Data Structures internal data structures, be certain to design test cases 10 exercise the data boundary. irons ETRE VUE, One vai ag Iatence les are defined srr an ip cnn spies 8 member Of 2.36, on ay nce class are def + Rather thin complementary to equivalence parti ase designe | partition the equivalence class, th ness of the above two methods is that they do not consider potent output conditions. Cause-effect graphs connect input classes (e clsts (effects) yielding a directed graph, effect graphing is a test case design approach that offers a concise depiction of ‘nal conditions and associated actions, case designers to look a id design test cases for the extreme conditions in output, case is one that detects a previously undetected fault, In order 0 maximize the chances of finding a new fault, a high-payoff technique is boundary-al analy Ags Experience has shown that when a im fuivalence clas is selected, the probability of detecting a fa ‘Siting the database produc, the following cases should be selected: Test case 1 [0 records Member of equivalence class 1 and adjacent to boundary ial [Test case 2 | 1 record Boundary value. = Tent ease 3 |B records raw fa ‘Acjacont to boundary value e380 4 | 783 rm nat = records Member of Squivalence class 2 ees es ‘cots |" Aaecent to boundany value 1868 records | Bounda uy value est case 7 | é) See wees [nbs of egvainse daar G and adjacent to bowndan 1 raph symbology is shown below. The left hand igure gives the various logical associations among causes and effects. The the right-hand columns indicates potentials constraining associations thet , either causes or effects, Ample Symbols Serle symbols used for drawing cause-effect graphs Folowing are some important guidelines for causeffet gra | Causes and effects are listed for modules and an identifier is assigned to each. s, qieetet 8raph is developed (special symbols are required). "ph is converted to a decision table. "table rules are converted to test cases case on or just to onk lustrated in Fig, 11.5.ox testing, tet C2565 ae seecteg prite-box testing, ted on the yan une specifications. White-box testing ig que Of examination of the code ed in Fig, 11,6, ae or re ® Fig. 115: Causo-Etect Graph Symbols se ¥ tanslate equivalence partitions into decision table form via be generated from the decision table form, reducing the number required. Some au state that partion testing is more effective than randomly gene However, random testing is more cost effective in terms of time and manpower 18 approach that examine the program structure sic. Structural testing is sometimes ref testing since white boxes are considered opaque and do not really perm in the code. ite-box testing requires the intimate knowledge of program internals, tox testing is based solely on the i y ile black 11.15.4 Comparison Testing is obvious in software engineerin ‘methodology has been devot e, a number of independent ve In that case, test cases using bl lo a certain number of useful black box testing techniques were developed. [is important to understand that these methods are used during the test design phase, ‘d their influence is hard to see in the tests once they're implemented. If cuppa from each version isthe same, then it is assumed that correct. Ifthe output is diferent, each version is examined to see differing outpu iplementat responsible ‘ot foolproof. If the specification applied to all vesi reflect the error, and there may ‘be the same OH 97). 1 Advantages of White-box Testing Tre major advantages of white-box testing include the following: * Forces test developer to reason carefully about implementation. * Approximates the partitioning done by execution equivalence. * Reveals errors in hidden code. * Beneficent side effects. * Opts 11.16 White-box Testing White-box test design allows one to internal knowledge of the software oe White-box testing ‘esting, Clear Box testing ide the box, and it focuses specifically ie selection of test data. meniow by other names such as Glass-Box testing, Si ‘en-bor testing, Logic-driven testing, and Path-orientedaS " ‘Testing ‘White-box ? es of box testing? 11.162 Daan efsadvanages of WI xe ‘ie followin PEt ad i 1 ss cases om iques - Testing Technigu' x ta of white-box test 1g. The important of th ese sh, 1 Siractural Testing 1 Loie-based Testing 1 FaulBased Testing All the above whitebox testing teciques 356 discussed below. Basis Path Testing ; vg isa whv-box technique. 1 allows the design and definition of g he basis set allow the Mecaton plks. Te test cass created from U aa sf ca ny ao examine each possible palh through the program by erga each statement at least once (Pressman 1997). ‘o be able to determine the diferent program paths, the ei of the logical flow of control. The cot trol structure can be ow graph ean be used to represent any procedural design. IFaoRb THEN proc y ELSE proc x ENDIF Fenn 117: Fw Gah of an tihenele! Statement ym condition IF-ELSE-ENDIF ang 301 strated in Fig. 11.7. nd its corresponding Control flow graph Number, ob Coen it determine the numberof ind ee imber of independent cone 4 provides the number of This insures coverage of all program statemeng used in more det . atements (Pressman, 1 sman, 1997), 712 Structural Testing ing examines source code and anal uted during analysis. This kage, file management and rogran control flow. oF interpretation, tin taion time. Test cases are derived from analysis of the p ‘5 Control Flow Graph is a represe }ow of control bet trol between program gas such a8 a group of statements bounded by single enty and ext nega ee Suuctural testing cannot expose errors of code omission but can e can estimate the test sioyaey in terms of code coverage, that is, execution of components by the test oa fault-finding, ability. ‘The following are Some important types of structural testing: + Statement Coverage Testing + Branch Coverage Testing * Condition Coverage Testing * Loop Coverage Testing Path Coverage Testing Domain and Boundary Testing Daia-flow Testing All the above structural testing techniques are discussed below. 1721 Statement Coverage Testing is isthe simplest form of pat, Statement is executed least o1 idence in a software product's behavior. Often, a CASE tool keeps track of how many times each statement has been executed Weakness of this approach ig that there is no guarantee that all outcomes of branches are ted Example: whereby a series of test cases are run such | Tis achievement is insufficient to provide A Droperly if (s>1 && t==0) x=% Test case: 8 = 258 = 05echniques and Strategies ng of reducing the number path j method of reducing the mumber paths is alder anotheeach oceurtence of & variable is labelled ate cope variable. 6 Domain and Boundary Testing ! f path coverage, Path domi ing is a form of ath domains are a subset i =a ps We on wats oe rl oundar th coverage. In of the variable the compound data, however, allow the stateng + prance Covrege Tet 122 ‘an improvement rerat all branches a SE rage ate Ci yer sateen Over fat least once. Tt is yi a ve ies ck 1 see y eases for each program ty a : ach overeat ooeurs at Feast once. Tt ror example, in @ program analysing the height and weigh of « the input at go ha each Psi cee products, but decision covers height and weight, where the inputs ate real numbers gears ac iu ‘be executed fof coverage for mos software PF BC alone a igh i. The statement Bree than zero and a wd ‘two from the true and false evaluation of the predicate "ment SI being executed and 2 is executed when the ei ion Coverage Testing i ao io ey et cases foc each condition iN & program dees differs from branch coverage only when ie eval gediee evaluates 10 false, ‘est inputs for branch, statement and domain testing could be: cases to exercise al Test I: weight = 48.0 height = 18 Test 2: weight = 50.0 height = 18 A boundary test would incorporate test inputs on and slightly off the boundaries of the us. To determine data off the boundary an amount, €, must be added or subtracted ‘athe value which lies on the boundary. ‘When the boundary is determined by an integer, ¢ is 1. That is, the value 1 must be ied or subtracted to the value in a predicate to form an input value that will be close to th: domain boundary. When working with real numbers the procedure is more complex. t value © must be the smallest number distinguishable by the base system of the °gram under test. For example, if the reals are single precision { could be of the order 6.000001 To test the boundary of ‘weight = 50.0" the following three input cases would be valid: combinations of conditions in a program decision. 724 Loop Coverage Testing rogram loops to be executed for zero, on, ypical running and termination 5 Path Coverage Testing Path coverage isthe most powerful form of white-box testing; all paths are t 4 product with loops can be very large, however, and met tron it fede th numberof pats to be examined. This technique more faults then by branch testing (Woodward 1980). This criteria requires sufficient test ent test cases f fat eit ofa dened progam sent, fo be ckeerten Test 1: weight = 50.0, height = 1.8 I le. The amouy 5 est 2: weight = 50.0, = 16 amount of path coverage is normally established based on th Test 3; cae a ret i a = 50.000001, height = in predicates because of with cl numbers i prediesBeans °F ton problems of reals. Boundary tes ote . st s is onl mats Of path selection, However, domain and boundary analysis i Pots with'a small number of input variables and with simple linear predicates. ion for Fe tS Paths is to restrict test cases L, from which cont tis, one fst ee Z contin en i these of poiariables receive values and yy, 1998). This kind of which 8 of faults in code, 0 ‘YPe checking, ete The igus echch is discussed below. cases for each feasible data flow to quences of a at teas ee aloag proeram che 11 te interactions between a variable. definition cs ust vere a variable definition and a use without an he program no path Variables can be created, killed and/or yseq, mown as defsiton is tet program with one component, sucha a @ syntactically correct Programming faul i late p ses, in which the variable is referen use, efers to all other references analysis simulates simple programming ; alent mutants are detected by generating incorrect ona Datsow eng uses the flow graph fo explore unreasonable things tht can hap et oa sng is ‘ed to ensure that each and every variable has bee the above statemei ie above statement has to be trav test input defined variables have been used or referenced fg le for incorrect uses of variables and constants a code coverage strategies, data flow t constants 11 and 9, boundary include cases of index = 9, 10 and 11 to detect ‘index’ or replacing use of it by another integer lc in scope, data flow anomalies can be detected, ines for the development of the test intensive requiring a large number of mutants to be created tected on the test suite. Research umber of code statements and variables squared. test of a large program, such as would be found in an indus well as misspelled identifiers, AS wi detect ising statements, L173 Logie Based Testing 18 processing are amembl ion. The following steps are applied to develop a decisioe 1 Uist all actions that can be associated with a specific algorithm : 7 a condions (or decisions made) during execution of the algorithm sola speci a ae ager of condi Be tons with specific actions eliminating impossible combina 4 Dale des by inaicaring hay Alternatively % ‘eal any inosine? 7 lp every possible permutation of conditions. Tht "SP inthe user specification and can thus be 62! ‘ing applicable to unit testing in a reasonable time scale and without 1 Tesources such as time and manpower. ‘acton(s) occurs for a set of conditions 18 Software Testing Strategies y afovare ‘esting strategy provides a road map for the software developer, qt ani “ation, and the customer. assurancego ¢ main advantage of joner and a Set of milestones pra tegration aust provide fo ems must surface 98 soon gypsy ‘The pe seen and tested early "Sh the base skeleton ofthe program pegs st Be MESH ing satis include the follow) al vantage isthe use of program stubs meen coer and works outward toward the integray we up-flow of information and therefore denen estes are writen, co pein the mle level Typ-level modules. loes not provide for a pood + rin beatin ves are appropriate ot diferent times + et eis ae deeper an fo Tage PEIS LY indepen 1 eng conic on «Tang 2 en ig stacey Inert cant 8 rope acon bal be performed inthe + Uni Tsing tegration Tsing onal Testing 1 itm and Acceptance Testing However, some system or hardware can happen concurrently with software test | modules built and tested first on le insures each module is fully tested bet but debugging mist be acconmajy, debugging are dierent act tical modules early. Main disadvantage is the fact that most or many modules must be sing program can be presented, wegration testing procedure can be performed in three ways (Humphrey 1989; Humphrey 1995): Top-down Strategy and thorough set of tests, the types of testing me ‘order in which they are described (Bas + Sandwiched Strategy I the above integration testing strategies are discussed below, 182.1 Top-Down Strategy Top down integration is basically an approach where modules are de the top level of the programming hierarchy and continuing is an incremental approach because we proceed one level ther “depth” or “breadth” manner. * Depth means we proceed from the top level all the way down to the lowest level * Breadth, on the other hand, means that we start atthe top of the hierarchy and then 80 to the next level. We develop and test all modules at this level before continuing With another level. is testing procedure allows us to establish a comy 1118.1 Unit Testing Unit testing procedure utilizes the white-box method and concentrates on testi programming units. These units are sometimes referred to as modules or ator and they represent the ing entity. Unit testing is essen paths though the mod ‘rogram are solid and without erors and will not cause abnormal termi or other undesirable results, 11.18.2 Integration Testing ing focuses on testing multiple modules working together. Two basic FE ion are usually used: * Top-down Integration or * Bottom up integrat ped and tested th the lower levels. ime. It can be done plete skeleton of the Above types of int Integration are discussed. below. Bsa The following are the major benefits of benefits of top-down integration testing: * Having the skeleton, we can test major functions early in the soon pene "Ar the same time, we can also test any interfaces that we have “%) errors in that area very early on. AS the term # branches. This canbe dane, EP of the rogram hierarchy and travels Sf) OF breadthist (across the hinges” SePtfitst (Shortest path down to the deePe ierarchy, before proceeding to the next level)-_and Strategies Tochniques. rest advantage of this approach is ing down on the developmen we have a partially working procedure is that management. This of course buitay® nus € en the) oe . and the 107 matean but also in the modet fnsey en, as with anything that is quickly slapped together, thi develop Hoes than the other (WO, Since these erors have eae tS POEs usually yids ve fe si etied uly conan yah ie 1 we Drovbocks svbacks 10 this procedure 25 ane de 55 also very demanding on ‘There ae 5 the necessary ion that is required tain the stubs 10 feed back 10 the calli resources: ; "hit can not be real tested properly and oe Petr cnback is that thee is Fay ting denen ee lules, the calling modules shout, built and integrated. Subs are replaced ‘actual modules, the calling tould be se been 1324 Sandwiched Strategy sandwiched strat pecoming, tHe ate of ‘his is also known as mixed integration strategy for eer oss. 2 Bottom-Up Strategy Haas 2 roach, a the name suggests, is the opposite of the Top-down metho ‘of them into what is sometimes called a cluster or bui order t0 test them propery Then 0 test these builds, a test driver has to be writen and put in place, {83 Funetional Testing ve form of black-box testing isto base the test data onthe functionality of the fe led functional testing. In functional testing, each function implemented mmole is idemified. From this, test data are devised to test each funtion enon Functional testing verifies that an application does what itis supposed to do and deesit do what it shouldn't. do, For example, if you are functior checks you would perform mini ‘ad printing documents. Function testing usually includes testing of all the interfaces ‘and should tt fo le the in the process. Because every aspect of the software system is being Benefits The advantages of bottom-up integration are: * There is no need for program snubs as we start developing and testing wit ‘actual modules * Starting at the bottom of the hierarchy also means that the critical mod iualy bul first and therefore any errors in these modules are discovered earh the process, Drawbacks testing a word processing application, a partial list ludes creating, saving, editing, spell checking As with Topdoun integration, there are some drawbacks to is procedure: * (oer to est the modules we have to build the rest drivers that are more complet than stabs. And in addi is fe ‘Ard in addition to that they themselves have to be tested. So more efor * No working 5 Ths ate a mt Presented or tested until many modules have been the process. e015 in any of the interfaces are discovered very li usually conducted as an alpha use the system. They take notes ing reasons: Functional testing can be difficult, however, for the foll vhich * The functions within a module may consist of lower-level functions, each of whic Im be tested fran, « Lewer-tevel functions may not be independent : - indep 7 plur the Limctonality may not coincide with module boundaries rs “sinction berveen module testing and integration testing. Tis 11823 Bie-Bang Strategy Big-Bang a ir builds we cena, ee its philosophy where basically ll the mod in they are all put together at pays endey of each other and when they. are fi .esting Techniques and Strategiog oc os case TE ‘pe spin Sr pte from the reted sam se and the complex re ‘onal Test i aeildla of the © System Te Pa Pn e's fens ith valid Input and yen, fala th the execution of test eases to a in entails exerising coneeequirements. A SyStem test checks for ‘unexpected a are correct oes and also evaluates the system for compliance eon less ive test for the prin Example ssing example, ie + Cotiuing With he Wot re aning both text and graphics to a print to pint @ docu er which the correct drivers are installed, led with paper tional Testing the process of executing the t est cases agreed with the customer we in i fe Functional ests. ‘These terms make reference to the tests bein af he code. They are concentrated on analysing fo the test suite ig unconcerned with the intemal structure 11.18.32 Negative Fut ‘the performance of the code with respect ing involves inputs, unexpected 0p functionality using @ combination 1d other “out-of-bounds” scenarios OF inva ‘At-any stage in the software life-cycle errors nges in design and/or code update resulting in a system or acceptance The process of re-testing a unit during its development is called a Revi n_test that occurs during maintenance when a is the selective ig of system or unit to verify that side-effects and that the system or unit still may be discovered. This may lead to re-application of it, integration, hres plication of any of unit, integration, iC the word ing i for the printing fun rocessing example, a negative test a * Contig comect the piste from the computer while a document is ig the word processing software simply hangs up « crashes beceuse the “abnormal” loss of communications with the printer isn’ This type of test involves exami jon of the whole computer system: all the software propery. components, all the hardware components and any interfaces. 1184 Regression Testing The whole computer based system is checked not only for vay bu also for meeting objectives. It should clude the following types tes Recovery testing Security testing Performance testing iability testing ~ Robustness: Testing ~ Stress Testing ~ Load Testing Thread Testing Back-to-back Testing . Seeding Technique ing is the process of running a subset of previously executed i eas : ure that program changes have not degraded the system. Or 998) The regressive phase concerns the effect of newly. into the previously integrated code, Problems arise when errors mi new fanctions affect the previously tested functions, which are comm em, Regression wig may be conducted manually or using automated y rcch is to incorporate selected test cases into a regression 0 fiseri ical reso BRres, di 1 how easly and complete en cet, dak conn era im capable of Recovery Testing uurces that need to be consi cases des 854 Reliability Testing A sotware el refers to the probat tp many aspects of software, includit Guided by the ed (0 obtain failure ing) can be used {0 analyze the data to ‘can decide whether to release the id use the software, Risk of using software can also be assessed based on information. Hamlet ig should be to measure the 8.5.3 Performance Testing very software system has Performance requirements, ‘The software shoul mt take it ic. The software system should be fee fa ways. The following two are the variances of reli imple criterion: * Robustness Testing * Stress Testing * load Testing “Performance bugs” sometimes are used t0 refer to those design problems in ‘that cause the system performance to degrade. The goal of performance 1 ‘ing can be performance bottleneck ide! Perfomance comparison and evaluation, ete ~8S4.1 Robustness Testing et of a software component isthe degee 1 wich de ag sence of exceptional inputs or stressful environmental c * Resource usage, + Throughput,recnss testing i” TRE SENSE thar ye th mt only watches for robustness termination. Therefore ro omeciness testing, 118.52 Stress Testing encompasses eft fed for perform je system rather than the softy, ith or beyond the specifieg sires testing is of soch tests the software oF ces (eg RAM, Disk, Fares ym the bcaking point in ode to find bugs may not be repai lure mode, consequences, etc. The load (incor sca) in tress testing is often deliberately distorted so as to force the system in depletion ILI8S43 Load Testing vmer, but unfortunate misuse of terminology is treating “load tei as synonymous, in without running excessive delay, MABSS Thread Testing Thread is test i ss & Popular testing technique ble for testing real-time systems. 11 17 sucral ‘threads’ its way through the system proe avied out at cach stage (Sommerville 1998). for | resus ing is used wh, Versions are em problems, wn Several versions of a system ex tered with same set of tests and then the if exist ing ae some steps that May help carving oy pa, eral-purpose Set Of test cases is prepared, these test cases, run the different allow +A ge ing: ‘sem versions and store the res the automatic comparison ofthe results stored iy : ele rence Report. 'm different files and generate the if exists among the ditterent errors. Real Seeded errors are realistic, ly if the kind of seeded errors in the software, ing technique works sat ie kind of defects that actu The result are presented in this form: Found F Total F real Found 6 seeded Tolal S seeded errors Tore ‘we can estimate the number of errors in the software. HR = S/S “S: We can find out the tofal real defects in this way, THs the Remaining defects = R— r = r * (Sk - 1) Based on these proportions, R= (Sh) *r 9 Defect Testing in® #8 aimed at discovering the latent defects before the system is sat remonstrates the presence of program faults not the absence. A dete S that causes the system to perform incorrectly and thus exposes a defect. "8 Of a system has two object ‘" is intended to show that the system meets its spt ae ‘s intended to exercise the system in such a way 1sting Techniques and Strategies namely component and module 4 ces pe discovery of defects in the fore a tet tat discovers a problem in the sy. ee could be exhaustive. In practi for defects should be ex teat amt fps ph co rs intode mily Back-bOX and White-box testing, chapter. ize many of Just some aspects oft fest the OO products (Pre ‘object commu fon the analysis and desi ssman = ee emphasis imperative to rTectness” and co le down to design and develo 1221 00 Test Case Design Conetonal test ease designs are based on the process they ate to test and caus. 00 test cases need to concentrate on the slates of ¢ o sas, the cases have 10 follow the appropriate sequence of ope ss an encapsulation of attributes and procedures that can be i sain target of OO testing (Pressman 1997), interfaces of objects omy reading the spe terface errors as the errors are a result of the interain components) rather than the modules alone themselves, ich a5 parameter interface errors, ice errors, procedural interfé This ype of testing allows for desig code of b design oF code that may lead to these faults a test case is developed to “flush” the errors out. These tests “So fore each tine of code to be executed (Marick 1994; Pressman 1997) jis testing method does not find all xypes of errors, however. Incomect spec ‘ad wiePace errors can be missed. You may remember that these overed by function 1121 Alpha and Beta Testing Acceptans involves delivering i & Sistem fal cust 1: that ayer (0 number of potenti BYout to real use eg ee on in 00 ned above, a class (and its operations) is the module most concentrated Ti sets of clases. Just ike is expo ments. From here it should expand to other classes an ef al Tal we and deat en EMS 10 the system developers. This € a nel model ane teed tena te vl ft and conn builders Of that may not have been anteipated by tH ra Pa sed ys Aer this feedback = ™ b he sien is pon = : 7 ce OH maid an eter elena for fre Random Testing developing a random test is tl er of he cs “uence Ne Of Methods used to exercise a class. * tht tries the minimum number of operarr < Software rechniques and Strategies re resin a 318 ting ws of 0 clas i test them se, he inputs and OPS Oe co be designed (Pressman ogy, wo te, parttoning C88 be Broken gay, 5 to test < sepoizes class operations based On how they hangs oh $f Ponsible for + Srate-based partitioning i 2 state of a cl i ss operations based on att mer-oriented speci + Category-based part fanetion the operations perform Ts Ts thd htt has been developed. These tests are based on ihe i nthe formal process can be eared out in paalel with software deueepncte 11.224 Scenario-based Testing « tes on what the Ives capturing ihe Ths ef ie ome ot ing he to find interaction type of errors (Marick 1994; Pressman 1997), 1124 Real-Time Systems Testing specific characteristic of real-time systems makes them a the time-dependent nature of real-time applications adds a ne: sie only does the developer have to look at black and white box ‘the data and the parallelism of the tasks, mn test data for real-time system may produce erors when the system to in others. Comprehensive test cases design methods for real-time ave not evolved yet 11.23 Cleanroom Software Development Cleanroom software development is an approach 10 static techniques for program verification and st iy cenifiction, The ‘cleanroom process is named by analogy with semiconductor fabrication | Units, where defects are avoided by manufacturing in an ultra-clean atmosphere. Cleanroom software development has been successful in producing system: 4 high level of reliability. The cleanroom approach to software development notion thet defects in software should be avoided rather than detected and repaired. Instead of unit and module testing, software cor 1125 Automated Software Testing Tools fort expanded on the software development process tools wolved is useful. As a response to this various ting tools. Miler described various categories for test tools: Portedly no more expensive than conventional development “We Analyzers cleanroom approach is re I 'g but it results in software with very few errors, am-analysis. support “proving” of static allegations-weak statements about ture and format. 11.231 Key Characteristics Following are the key characteristics of cleanroom development (Sommerville 1998): * Incremental Dev ‘lopment: The is elopment: The sofware is developed separately using the cleanroom Dreceas + Formal Specification vmal Specification: The software to be developed is formally specified. * Static Verification: The devel he based correctness argumens "* is statically verified using mathemati into increments t ned The speck The 8 ie it the program are aly pee tems tell whether the programmer-suplied assertions aboualways occu HS aS a sors Shooman 1986), The npettSM°® Of testing (Pre ats eras ass te wet ih ee DURBIN is also treated a, MENG sess Test assist ted an an zins These processors in Fig. 11. 7 stone ef oupaHs FFM 8 PORTE With poy, 1 the appropriate test datg, man Cradley 15 wih 198). The debugeing ves 0 on cugsing POSS EMPL 0 math sympiom wih 1 There are two outcomes of the debugging: M8: thereby leading to rhe cause will be found, corrected, and removed ‘me cause will not be found, too al “ the difference among fed additional categories of ff automated tools inclu Bxecution Systems = ae, Ti rns program testing using algebraic input, instead . tool performs program testing using fm Environmental Simulators 127 Debugging Techniques Jowing are some important debugging techniques + ante Force Debugging 1 packircking Debugging 1 pehugging by Induction + Debugging by Deduction + Cause Elimination Debugging + Debugging by Testing + Debugging by Program Slicing 1 me computer-based system that ulate oper ata Flow Ai ‘sa neste Nw of ta ough the sytem and tries 0 Identity data rela eng ., Compuware, Ex 6. ARE Some of inl, Mercy Interv, Se ani companis tt prove stole software testing tol 11.26 Debugging Sofware testngis a process that can be systematically planed and specified Test dep can be conducted, a stategy canbe defined, and results can be evaluated against preset expectations. ‘hese approaches are discussed below: 11271 Brute Force Debugging nce of successfi results in the removal of the error, Debugging occurs as a co an error, debugging is the process g- When a testcase umes 2X; lace debugging methods are applied when all else fails. Brute force debugging is ing with a storage dump, te force debugging ‘fcent one. In ‘ntmediate values Satment with error, the most common debi} is technique, the program is loaded th the hope that some of the printed values _,| Becction ui el ee Me ‘Adional ‘Suspected te |} Se People who use thinking rather than a set of helps, exhibit superior performance. This Sh becomes more systematic with the use of a symbolic debugger because values of “variables can be easily examined, & ? Backtracking Debugging i ves backtracking the eet isa fay common debugging ec les backing te ‘eed eget through the logic of the program. In this technique, the “ctvards until the error is discovered. causes Fla. 118: Debugaing Process x of source lines to be traced back sschnique becomes unfit when the numbe: ie nd the number of potential backward paths becomes unmanageabeduction apter 16. st cases. There are 2 types of EXERCISES ftware testing? “ware testing? Explain each stage in detail. {nd Validation. sng ft in f ‘tin Sotware Quality Asstrance (SOA)? "8 sofware development? Explain, and devises a hypo xd for what, whei que. Determine the px a debugging technique sine the remaining cause into a hypothe formation in debugging to locate an error, cover many conditions per test case. Butt ingle condition for each test case. at a particular stateme luence the value of that ‘ate lesing satin sofware ie cycle? When can planning ft spender tester? Lusty youy Is the psychology software test design? Sta je-box testing is complementary to Bla prove this statement iite-box is @ Sort of ‘coverage’ test. What dé What do you mean by basis-path testing? Ex 3t do you understand by comparison 1 slaps in basis-path testing method, ing? Discuss is relevance, black or white box testing? Jusiy your answer cuss the stops involved in software test design, o you mean by ‘during code and design reviews rather than leaving them to b strate, need special type of testing st Det rave ments and demeris of dient “to between postive and negate funcional tsingrategios? lustate their importance, rato through a suitable example, {rat are important debuoging techniques? Explain iat aro the diferent Kinds of system testing tha Elan nus testing uerni that he progam is 10% cone? Jy {o. Wiy top-down oF bottom-up integration is not 5 that could result from add possible solutions to these problems” “¥s debu agree to the statement “System testing can be considered as a pure me 10 iored as a pure back box testing, Differentiate between random and partition testing, What do you understand by interface testing? State its ae ea going wtige? DSC 8 racy, sting? Why is stress testing applicable t0 only conay 10 devolopnment? understand by acceptance testing? lustre. between alpha and beta testing, joms and cause may be at distance. Explain. 1g cant discover interface errors. ts SRS document, how can you design black-box test suites for tion by binary partitioning’ technique for debugging. ack-box and structural testing techniques and suggest how they canbe us together for defect testing, a, y? Musa, ‘% ©iplain the Scenario-based and Random-iesting at class level What causes interface errors’ plain why regression testing is necessary and how aulomated testing tools can esi wth his ‘ype o testing, y testing? Discuss diferent types of reliabiliy testing. based testing? iustrate with a suitable example. 10 define thresholds for software modules? Justify your answor, ing? List important structural testing tech ng tchriques is stronger and why? tustrate with suitable © eFt of cleanroom software development? ersland by Object Oriented Testing? iustrate. Testing actualy stats at reviewing object fs documents: 49% fed analysi detecting errors in ” Justify your af