Unit 4: Chapter 6 Software Testing ‘Structure of the Unit ‘Unit Outcomes ‘Testing Objectives Veaification versus validation Softwaue Testing Process Design of Test Cases Levels of testing. Debugging. Functional and Stctral Testing ‘White Box and Black Box Testing 10. Object-oriented and Web-based Testing ‘11. Self-Assessment Questions and Activities 12. Multiple-Choice Questions 13, Keys to Multiple Choice Questions 14. Summeay of the Unit ern eee 6.1Unit Outcomes After the success completion ofthis uit, the stuclent will be able to: Explain testing objectives. Construct test cases and Explain white box and black box testing methods, Perform structural and functiorel testing on developed software. Analyze object-oriented and Web-based Testing. pees 6.2 Testing Objectives Testing is the technique wsed for the detection of defects in software. It plays an important mle in developing quality software. There are two goals for perforing the software testing process. 1. Toillustate thatthe developed software ull the cliea's requirements, 2. To seach the scenarios and data where the software my not work as per specification or may result inan eor or comay deta, ‘Testing is performed using relevant data and the software is executed to check the expected output.‘Testing is performed on the software under test (SUT). The sample data used for checking the behavior ‘of the SUT is known asa testcase ard test suite. Some ofthe commonly used tenes in testing ae: 1 2 3 4 5 Defect: ‘The defect inthe softwares infonally knownas a bug, Whenthe systemis not meeting the requirerrents, it is said to have a defect. Defects can be major, critical, or minor. ‘Bug: The bug may be due to an algorithm, resources, or logic. ‘Error: When there is a mistake in a code then it is called an error. Eors may be the syntax, semantics, harivaue, testing errs, et. Fault: The state of inconect working of a system is known as a fault There may be a fault in the interface, fiort end, security, software performance, etc. Failure: The systemeads to failure if there are multiple defects. If the defect is causing incomect functioning of the system, then the system is in failure condition, ‘Testing objectives can be further elaborated as given below, 1 2 Testing is wed to show to the software developer and client that the developed software meets the specified requirerments. Ifthe software is a cusorrized proche, then foreach requirerrent there should beat least asirgle test case. This is a validation testing process that refers to finding errors by executing the rogram ina real environment. ‘Testing is sed to fine those tuations where the software behavior is erroneous, not desirable, and not meeting its specification If defects in the developed software may lead to accidents such as security violations, system. ‘crashes, wrong computations, ar loss of deta. Ths testing is Inown as defect testing. This is ‘also knowns verification which is a process of checking thatthe software meets specifications. 6.3 Verification versus Validation Invetification the software requirements specification design and code are checked. It may not inchure ‘Program execution. Code inspections, reviews, or code walkthroughs are performed in this stage. The ‘etification is performed by the system anwlysts ard designers. High-level design and database design ‘are tested in verification in the early stages of software develoyarent. Validationis a generic process that is used to ensure thatthe software micets the client's requirements ‘tinwolves the execution of the code. The target used in the validation is always the code. Validation is performed after the verification ofthe code. The people involved in validation inclure the software ‘development team Bany Bochm, a well-known scientist in software engineering, has shown this difference in validation and verification as given in Figure &.1. “Verification: Are we building the product, right?” [ “Validation: Are we building the right product Figure 6.1: Verification versus Validation‘The objective of verification and validation is to develop confidence in the softwear system tht itis fit for its pupose. The level of confidence in the system depends on the purpose of the system. Consider developing a safety alann systern and a word processing typesetting system The level of confidence is more importart in the safety alarm system than in the typesettirg project. The expectation from the users is yet another issue in building the system's confidenee. Along with software testing, the verification andi validation process may inwolve software inspections and reviews. The requirement specifications, design models, code, and proposed tests are checked! in the inspection, This is known as V&V techrique, 6. Software Testing Process “The software testing process corsists of the stages for preparing test cases, preparing input dat, running the software, and verifying the ourput data. A software testing process is depicted in Figure 8.2. eb fs} Hel Ca) Ca) SRS) ED Figure 8.2 Stages of Software Testing Process “The test cases specify the input and output in the system with docurnertation about it. Test data is the input ftom the test cases. Test data can be generated automatically from test cases, but the test cases canwot be generated autometically. Test cases ate manuzlly prepared by a team of people who are aware of the requirements, design, and coding of software. Test results and reports can be generated by ‘automatic method and manually. The tests are encoded in the program to be tested. The looks of the ‘utp, mer options, graphical interface, etc. must be tested using the ranma tho. Manual esting ‘also inwolves the preparation of test cases arditest data. Checking of side effects cane carefully observed! {in amu methods. The commercial software testing process goes through the followire stages Development > Release testing > User testing 1. Development testirg includes Systern designers and software developers. Development testing is wed to check for exors and defects during system development. This type includes unit, componert, an integration testing methods. 2, Release testing is also known as furctional testing, Here testing is camied out by the developanent team and it is aimed to convince the system supplier that itis good enough to use. Tn releasetesting, only the functionality is checked and not the software implementation The differert forms of release testing are @) Requirerrent-based testing D) Scenatiorbased testing ©) Performance:based testing @ Regresiontesting 3. User testing inclues acceptance testing where the enc-users test the system and decide whether the system is ready for deployment. User testing is performed after the performance and release testing, This testing is clasified baser on whois perfomning the testing. Inthis testing, one category is alpha and beta testing performed by experts and the second category is acceptance testing dane Dy the wer, 6.5 Design of Test Cases Atest cases a set of test inputs an execution conditions ofthe software It also mentions the expected outcome from the software under test Test [Test Case|Test Data Expected |Actual —_| Pass/Fail Case# |Description Results [Results 1 (Check Email: Successful |Login was |Pass response | [email protected] |Login _| successful after valid’ in fogin and | sword: PASSWOFd | seseeee are entorod Figure 83 Sample Test Case A singe test case to login ino an eral is shown in Figure 8.3. The testcase inclues a description test ‘data, results thet a expected, and actual rests showninthe test case. A group of related test cases forms A test suite. Test suites result in some specific behavior of the software. Test cases and suites are manual ‘or automatic. A test site for purchasing ecpipmert is shown in Figure 84.Checkout ‘iw he tstmedels [a] en] Ps Figure 8.4 A Sample Test Suite 6.6. Levels of Testing ‘These testing techniques can be summarized using levels of testing, The differert levels of testing are shown inFigue 85. TESTING ner INTEGRATION Sxrem REcREssion| sesnhe. TESTING. eons earn tT Ceoremce] | | Ciara Cems) | | Com Coe | | Geer) Figure 8.5 Levels of Testing 1. Unit testing: Unit testing is done in the coding and developmert phuse. Inthis method, the smallest element of the software such as the module is tested. Each module is tested sepatately before combining. The differert procedures with diferent inpus/outpuss are tested. The global data structures with module access are verified. Unit testing helps to ‘comet the enrs at the early stage. The simplest unit testing is executing every staterrent of a program. Examples: checking loogs, functions working ar not ina program, checking Initialization values, verifying incomect precedence of operators, etc. Coreider the student recort as shown in Figure & 6, The Data structures are RollNo, Narve, and Fees pid ard methods are arking a record, deleting, and printing a record must be verified with the detabase,Figure 88 Integration Testing Integration testing is classified as follows: a) Top-down: In top-down integration testing, the main moxtile or higher mochles, are tested frst, Lower modules ae tested later: If they are not ready, then program, stubs are useeias chummy components. Program stubs are stvll prograns that work. asa substitute forlong progras. }) Bottom-up: In bottorn-up testing, lower modes are tested first. Lower modes, are then integrated and tested as a whole. Lower mexhiles are submodhles and higher modules are main modules. If higher modules are not ready, then drivers are wsed as chummy modules. © Big-Bang: The simplest form of integration testing is a big-barg technique, where all the moctles are simply put together, ane the system is tested. The hig barg is \sefid in small systems. Error handling is very difficult in the Big-bang method. Highetisk modules are not given more prionty. Due to many modules getting tested, it takes more time, Mix Integration testing con be a mixture of top-down arel bottom-up approaches and itis known as sandwich’ mixed testing. It perfomrs the testing as ancl when the rmodkies are available. It is used in lange systems with several modules. Tt is expersive because ofthe combination of top downand bottom-up Itis not stitabe if there is high coupling. 3° System testing: System testing is also known as user testing. It is done by the users by centering, ingat are suggestions on system testing, Inthis method, the wser erwironmert can help test the usability and reliability of the developed system. For example, in a hospital ‘management system, a developer can’t prepare clinical environments such as patients, ruses, doctors, emergencies, tc In such situations where live or reel data is necessary, system testing is done. System testing can be perfomed informally or from a formal extemal supplier. Systemn testing also inclu testing technigues besed on non-functional cherctetistics such as volume stress, compatibility, configuration, etc. eg. A website getting tested for the number of users or several download, etc. This testing is further asified as alpha and beta testing. @) Alpha Testing: Alpha testing isa form of acceptance testing where the defects anel {issues are idertified before the release of the final product. It is cone in the ealystage of software developmert. Example: Google releases the newly developed software componert to ther high-level developers. 1) Beta Testing: A Beta version of the software is given to linted users ofthe system and it helps inimproving the quality of the system. Users can experiment ard raise problems tht they discover with the system developers. to test the software. Inbeta testing, users can expetimert with the software ane! produce issues (if ary) forthe developer. The developed system is tested with real data and environment, Example: Techro-Certer is a company thet develops an infonetion management system for colleges andi universities to manage all academic activities. The activities include details of a studere related to acivission, exam, library, hostels, laboratory, attendance, etc. The company frst releases the software to a few sirall colleges ancl gets feedback Based on the feedback, some minor defects are corrected and then released in the customer matt. ©) Acceptance Testing: In acceptance testing, the end-users test the system and decide whether the system is ready for deploymert. Inthis method, customers give the decision to the system developers. After acceptance testing, the payment can be done tothe developers. Is the last phase of software testing, 4. Regression testing: Regression testing refers to testing when an existing, systems charged corupgracked This esting method checks if there are any new bugs after the action of ew feccures or ae if there any performance issues. In this testing method, it may rot be necessary to nthe testing for the entire code, int ony those test cases thet can aifect the functions with charge can be 1un Testi is pesfonned in the maintenance stage. For exaunple, consider a bookshop management system, where there are features such as add (), delete (), and print () operations for book records up to 1000. Suppose this systemis enhanced with several records up to 5000 and operations append 0, and updite 0) is upgraded. Then the test cases required only for testing these two features are designed and ‘regression testing is performed 6.7 Debugging Delagging is a process of comecting erors. Its the ait to remove erors. Debugging isnot testing but it canbe called a consequence of testing. The debugging process as shown in Figure 8.9. Do the correction Gheck Ine ener Figure 89 Debugging Debogging sttegies are given below.1. Bnte Force Approacte It is the most used method in debugging. It is of the type tial andl enor method. Inthis method, the programis tested multiple times to check the cause of the enor 2. Backtacking Method In this approach, when an error is found the source program is traced beckwerd until the cause is found, 3. Catse Elimination Technique: The data causing the enor is intocuced and the software is, debugged to build its support. 4. Autoreted Tools Debugging compilers, dynamic program tracers, and automatic test-case generators are used. 6.8 White Box and Black Box Testing These two testing techniques are used for testing the functionality and structure ofthe software. The pictorial represertation of these two tectniques is shown in Figure &.6. Figure 8.6 White Box and Black Box Testing White Bax Testing: White box testing is also known a5 clear box/glass box/code-bised testing. The structure, codirg, security holes, and functionality of the developed software are tested. The tester understands the code, andl inner workings of the application and creates test cases, finds the code coverage. Poorly structured loops, da flow based on input and outpat, execution of objects, functions, testing of each statemert, and the expected output are tested in white box testing, It also checks for untested parts of code. White box testing can be simple or complex, dependirg on the application. This testing is performed by the software developer and not the test engineer. The structure, design, and coding of the developed software are tested. Different techniques in white hax testing are shown in Figure 8 7.v 2 ‘Mutation no Statement Coverage Testing: The testing process aims at desiring test cases s0 that every Saterert ina program is executed atleast once. ‘The main idea isto check whether there is a possibility of mistakes in computation, unused statements, Trissing statements, unused branches, ‘wrong arithmetic calculations, etc. If some statemert is not executed, then there is a bug in that statement. But the drawback is statement coverage may work property for one set and not for others. Branch Coverage Testing: ‘Branch testing is also known as edge testing where each conditional ‘branching is tested for true and false values. Tt guarantees that all the test cases ave fulfilled, and it is stronger than the statemert coverage. The branch may be due to for, while loop, or if-else statements. Designing test cases is easy in smell programs but in large programs, coverage tools ‘generate test cases, Consider an exaarple given in Figure 88. Figue 8.7 Methods in White Box Testing Figue 88 Check () Functionin € Programming Coverage measurement = Number of executed statemerts | Total no of statesneres ‘Test case a= 2 Number of statements executed = 5, ‘Total numnber of statements. Statement coverage = 5/7 * 100 = 71% Test case a= 3 [Number of statements executed = 6 ‘Total number of stetemerts = 7Statement coverage = 5/7 * 100 = 85% 3) Condition Testing: When there is more than one condition or composite condition, with different branch statements, then a condition testing method is wed. For example, consider a statemert ((X OR B) NOT C). If there are n expressions in a composite contitional statemert, then, 2 test cases are required. The number of test cases increases exponentially in condition testing. 4) Path Testing: Path testing is performed to check the control paths of the program. The control of the program depends on the if, else, loops, etc. This testing requires corrplete knowledge of the program The valves of differere variables and the status of the program can be traced sing path testing, 5) Data-flow Testing: The defirtions and uses of different variables in a program can be wwed as, test paths. Data flowbased testing methods make wse of such paths. The issues such as undeclared variables, deleting the memory of a variable before its wse, or multiple declarations of variables are pointed out in data flow testing. 6) Mutation Testing: In this testirg, some statements of the source program are charged. Itis also nownas mutated to verify whether the test cases canlocate ents. The charges inthe program. ‘axe very stall and random. The charged program is called a natant. The objective is to build the test cases robust so that they fail the mutated source program. If the output of the muart is lifferent from the origine, then itis KilleeL The different faults to be introduced may be the charge of data type, alteing the constant value, charging the order of the operators in an ‘expression, etc. The details of mutation testing are in Figure & 9, Figure 8.9 Mutation Testirg Black Box Testing: Hack box testing irwolves testing functionalities ofthe software using requirement specifications. Ths testing is perfouned without the knowledge of design or code. Black box testing is also known as behavioral testing. Fg. testing the working of websites such as grail. com, rmakerrytrip.com inet. inete. The diferent steps in black box testing ave: 1) Examine the requirerrent specifications of the system. 2). Select aset of proper inputs and deterrine the possible output 3) Prepare test cases for the inputs ancl execute the test cases.4) Compare the results with expected outcomes. 5) Ifthere are any undesired resuits ar errors, then comect the errors ane repeat the testing Black box testing is classified irto different categories as given below. ‘Equivalence Pautitioning: In this testing type, inputs are patitioned into smaller pats, and every ‘ati tested forthe results. 2. ‘Boundlary Value Testing: Testing of the softwere is done within specific boundaries. 3. Decision Table Testirg: Conditions are mertioned under certain conditions in the form of decision tates. 4. State Trarsition Testing: Transitions of the states from one fou to another are tested inthis method. 6.9 Functional and Structural Testing 1) Punetionel Testing refers to testing whether the system meets the predetermined requirerrents. The processing of information indifferent functions is verified using tet cases in functional testing. ‘Along with the functional requirement, the functional testing methods addhess quelity atibutes suchas riability, maintainability, security, etc. An example of functional testing is shown below. A software developer builds an app for the calculator, To test this software the developer must check whether the addition of two numbers gives an accurate sum. Similarly, the other arithmetic and scientific operations must be checked for the functionality of the software. Online shopping websites announce discounts and reward points after a cestain number of purchases sich as one T-shint being free after the puuchase of 2 T-shints or a 20% discount if the bill is above Rs. 2000, The developer mst check whether these promises are happening during the online parchse 2) Structural testing refers to the testing ofthe code anc design by software developers Specifications of the code, input/output constraints, Data structures, hardware, front end, menus, low-level ‘components, etc, are tested in the structural testing. The logical ane syntax errors are identified in. structural testing, The most used structural testing for softwear is control flow testing. An example of structural testing is control flow testing. Control flow testing tests the order of the [program execution when conditions ancl loop statements are vsed. In the example given in Figure 8.8, there isa MAX vaniable subjected to the condition. Based on the results of the if-else condition, either Fal or Pass is prirted.wax = 50 Figure 88 Cortrol Flow testing forthe C program 6.10. Object-oriented and Web-based Testing Object oriented programming (OOP) includes abstraction, encapsulation, overloading, message passing, ef. These methods mest be tested separately in the testing process. The testing methods in OOP cannot be conventional. The OOP testing includes dependencies fiom one class to another, There is a need to test the dependencies of variables and classes or methods. The dependencies of the method to messages inOOP mast be verified. The tectniques used in OOP testirg are given below. Fault based Testing: Test cases are bat to fel the faults and elirinate the ems. Class Testing: Members of the class andl methods are tester. Random Testing: Random testing for uncovering errs Partition Testing: Test cases designed to partition the program hased on categories. ‘Testing based on scenarios: Testing actions based on input/outpat and conditions. Testing applications onthe web or websites is performed in web testing, The ene users must use bxg-fee ‘web applications or websites. For web testing, sophisticated tools are used. Web applications rust be checked ondifferert operating systems and browsers In web-based testing different testirg methods used. areas follows. © Static Website Testing ‘+ Dynamic Website Testing + Mobile based Testing ‘+ Testing for E-commerce Testing 6.11. Self-assessment Questions (Q1. Explain a software testing process andits necessity. [ 5 Marks] (Q2. Differenticte between verification and validation with a suitable example [5 Marks} Q3. Explain integration testing and its different types, [6 Marks](Q4. Explain the need of performing regression testing. [5 Marks} Q5. Explain clferert levels of testing with a simple example [10 Marks} (Q6. What the difference between acceptance andl system testing? Justify your answer with a suitable example [8 Matis] Q7. What ave alpha and beta testing? Whereis it sed? [10 Marks] (Q8. Differentiate between top-dawn ane bottor+up testing method. Discuss the test cases where these two methods are used. [10 Marks} Q9. Compare unit testing and fimctionel testing, [8 Marks] Q10. Discuss the advantages and drawbacks of program inspection over testirg. [6 Marks} Q11. Discuss Object-oriented testing methods. [5 Marks} (Q12. What is the need for web-based testing methods? [5 Marks] 6.12. Multiple-chaice Questions (MCQs) (QL. The functional requirement of a systemcan be given by: A. Availability B, Efficiency C. Pontability 1D. None ofthe above (Q2. The ircomect testing method out of the following is: A. Combination Testing B, Systemtesting C. Usertesting D. Functional Testing Q8. Alpha testing is performed at_ A. Clicnt’s end B. Developer's end C. Botha andb D. None of the above (QU Integration testing is used for__ A. Detecting errors in modules B. Detecting synter enurs C. Detecting imerface enors D. None of the above 5. Systemtesting deals with only functional testing (True or False?) A. The