Unit: 3 Unit Testing

Dr. Jigna Solanky

Agenda

 3.1 Introduction, Importance

 3.2 Coverage Analysis: Levels, Process
 3.3 Mutant: Types, Determination, Test Case Execution, Mutation Score
Derivation, Result Inference
 3.4 Walkthroughs: Basic Test, and Data-Driven Unit Test Assertions
 3.5 Test Plan Design: Test Environment, Configurations, Execution,
Analysis, Revisions
Unit Testing
 Unit Testing is a type of software testing where individual units or components
of a software are tested.
 The purpose is to validate that each unit of the software code performs as
expected.
 Unit Testing is done during the development (coding phase) of an application by
the developers.
 Unit testing is a White Box testing technique that is usually performed by the
developer.
 Objective of Unit Testing:
1. To isolate a section of code.
2. To verify the correctness of the code.
3. To test every function and procedure.
4. To fix bugs early in the development cycle and to save costs.
5. To help the developers to understand the code base and enable them to make
changes quickly.
6. To help with code reuse.
Advantages of Unit testing
 Reduces Defects in the Newly developed features or reduces bugs when
changing the existing functionality.
 Reduces Cost of Testing as defects are captured in very early phase.
 Improves design and allows better refactoring of code.
Disadvantages of Unit testing
 Unit Testing will not cover all the errors in the module because there is a chance
of having errors in the modules while doing integration testing.
 Unit Testing is not efficient for checking the errors in the UI(User Interface)
part of the module.
 It requires more time for maintenance when the source code is changed
frequently.
 It cannot cover the non-functional testing parameters such as scalability, the
performance of the system, etc.
Types of Unit testing
 There are two types of testing methods
1. Manual unit testing
2. Automated unit testing

3. Manual Testing:
 Manual testing is one of the most important ways to test software because it can
find both obvious and hidden bugs.
 Types of Manual Testing
 Manual testing can be done in many ways.
 White Box Testing
 Black Box Testing
 Gray Box Testing
Cont..

 White Box Testing

 Testing the internal structure or workings of an application
 Requires code knowledge
 Black Box Testing
 Testing the functionality of an application without knowing the internal
workings.
 Does not require code knowledge
 Gray Box Testing
 A combination of White Box and Black Box testing, where testers have partial
knowledge of the internal workings.
White Box Testing

 White Box Testing is also known as clear box testing, open box
testing or glass box testing.

 Testing the internal structure or working of an application.

 The tester needs to have knowledge of the code, architecture, and logic
behind the application.
Techniques used in White Box Testing:

 Unit Testing:
 Testing individual units or components of the software.
 Integration Testing
 Testing the interaction between integrated units or components.
 Code Coverage Analysis
 Ensuring that all code paths are tested.
 a technique used to measure how much of your code is executed when your tests
run.
 Statement Coverage: Ensures that each statement in the code is executed at
least once
 Branch Coverage: Ensures that each branch (true/false) in control structures (like
if-else) is executed at least once.
 Function Coverage: Ensures that each function is called at least once.
 Condition Coverage: Ensures that each Boolean sub-expression in the code is
evaluated to be both true and false.
 Name Stmts Miss Cover Missing
 ---------------------------------------------
 factorial.py 12 0 100%
 test_factorial.py 10 0 100%
 ---------------------------------------------
 TOTAL 22 0 100%
Cont..

 Control Flow Testing

 Testing the logical flow of the program.
 Example: a simple function that classifies a number as positive, negative, or
zero.
 def classify_number(n):
if n > 0:
return "Positive“
elif n < 0:
return "Negative"
else:
return "Zero"
Cont.. Control flow graph
Cont..

 Data Flow Testing

 Testing the flow of data within the program.
 ensure that variables are properly defined, used, and destroyed.
 Branch Testing
 Ensuring that every branch (true/false) of the control structures is tested.
Cont..

 Advantages:
 Identifies hidden errors and vulnerabilities in the code.
 Optimizes code by discovering unnecessary code paths.
 Enhances code quality and security.

 Disadvantages:
• Requires knowledge of programming and code structure.
• Can be time-consuming and complex.
Black Box Testing

 Black box testing focuses on testing the functionality of the application

without any knowledge of the internal workings.

 Testers only know what the software is supposed to do, not how it does
it.
Techniques Used in Black Box Testing:
 Functional Testing:
 Verifying that the software functions as expected.
 Non-Functional Testing
 Testing performance, usability, reliability, etc.
 Regression Testing
 Ensuring that new code changes do not adversely affect existing functionality.
 Boundary Value Testing
 Testing the boundaries between partitions.
 It involves testing at the edges of input ranges to uncover errors at the
boundaries of input space.
 Example
 If a given age qualifies for a senior citizen discount, where the qualifying range is
between 60 to 100 inclusive.
 The boundaries for age input are:
 Lower Boundary – 60
 Upper Boundary - 100
Cont..

 Equivalence Partitioning
 Dividing input data into valid and invalid partitions and testing each partition.
 A set of valid or invalid input values that are treated similarly by the system.
 Example
 consider a function that validates the age of a user for an online service, where the
valid age range is between 18 and 65 inclusive.
 Valid age range: 18 to 65
 Invalid age ranges:
 Below 18
 Above 65
Cont..

 Decision Table Testing

 Using decision tables to determine test scenarios.
 Example: an online store that offers discounts based on the following
conditions:
 Customer Type: Regular or Premium
 Purchase Amount: Above or Below $100
 The rules are as follows:
 If the customer is Premium and the purchase amount is above $100, a 20% discount is
applied.
 If the customer is Premium and the purchase amount is $100 or below, a 15% discount is
applied.
 If the customer is Regular and the purchase amount is above $100, a 10% discount is
applied.
 If the customer is Regular and the purchase amount is $100 or below, no discount is applied.
 Cont..
Conditions:
•Customer Type (Regular, Premium)
•Purchase Amount (Above $100, $100 or Below

•Actions:
•Apply 20% discount
•Apply 15% discount
•Apply 10% discount
•Apply No Discount

Rule No Customer Type Purchase Amount Discount

1 Premium Above $100 20% Discount
2 Premium $100 or Below 15% Discount
3 Regular Above $100 10% Discount
4 Regular $100 or Below No Discount
Cont..

 State Transition Testing

 Testing state changes based on events.
 This technique is particularly useful for systems where the output depends
not only on the current input but also on the history of previous inputs.
 Example: user login system with the following states and transitions:
 States:
• Logged Out
• Logged In
• Locked
Cont..

 Transitions:
• From Logged Out to Logged In on valid login
• From Logged Out to Locked on three consecutive invalid login attempts
• From Locked to Logged Out on reset
• From Logged In to Logged Out on logout
Cont..

 Advantages:
• Tests are done from the user’s perspective.
• No need for programming knowledge.
• Effective for large-scale software systems.
 Disadvantages:
• Limited coverage of the software’s internal structure.
• May miss certain types of errors that white box testing would catch.
Gray Box Testing

 Gray box testing is a combination of both white box and black box
testing.
 Testers have partial knowledge of the internal workings of the
application, which allows them to design better test cases.
Techniques used in Gray Box Testing:
 Matrix Testing
 Using matrices to determine the impact of different input combinations on the
system.
 It is particularly useful for identifying dependencies and interactions among
different parts of the system.
 Example: Testing a Login System
 Input Variables:
1. Username (valid, invalid, empty)
2. Password (valid, invalid, empty)
Matrix

Username Password Expected Result

Valid Valid Login successful
Valid Invalid Login failed
Valid Empty Prompt for password
Invalid Valid Login failed
Invalid Invalid Login failed
Invalid Empty Prompt for password
Empty Valid Prompt for username
Empty Invalid Prompt for username
Prompt for username
Empty Empty
and password
Cont..

 Regression Testing
 Ensuring that changes do not affect existing functionality.
 Scenario: E-commerce Website

 Initial Features:
1. User Registration
2. User Login
3. Product Search
4. Add to Cart
5. Checkout
Cont..

 Recent Changes:
1. Added a new payment gateway
2. Updated the user profile section

 Regression Testing Approach:

 Test Cases for Initial Features:
 User Registration:
1. Register with valid details.
2. Register with an already existing email.
3. Register with invalid email format.
4. Register with empty mandatory fields.
Cont..

User Login:
1. Login with valid credentials.
2. Login with invalid credentials.
3. Login with an unregistered email.
4. Login with empty field
Product Search:
5. Search for an existing product.
6. Search for a non-existing product.
7. Search with an empty query.
Cont..

Add to Cart:
1. Add an available product to the cart.
2. Add an out-of-stock product to the cart.
3. Update the quantity of a product in the cart.
4. Remove a product from the cart

Checkout:
5. Checkout with all mandatory fields filled.
6. Checkout with missing shipping information.
7. Checkout using different payment methods.
Cont..

2. Regression Test Cases After Recent Changes:

 User Registration (Re-tested due to recent changes):
Ensure user registration still works correctly with all previous test
cases.
 User Login (Re-tested due to recent changes):
Ensure user login still works correctly with all previous test cases.
 Product Search (Re-tested due to recent changes):
Ensure product search functionality is not impacted.
 Add to Cart (Re-tested due to recent changes):
Ensure adding to cart functionality is not impacted.
 Cont..
 Checkout (Re-tested due to recent changes):
Ensure the checkout process works correctly, especially with the new
payment
gateway.

 New Payment Gateway:

1.Checkout using the new payment gateway with valid payment details.
2.Checkout using the new payment gateway with invalid payment details.
3.Ensure error messages are shown correctly for failed transactions.

 User Profile Update:

1.Update profile with valid details.
2.Update profile with invalid email format.
3.Update profile with missing mandatory fields.
4.Ensure the changes are reflected correctly in the user profile.
Automated Regression Testing
Example:
public class LoginTest {
WebDriver driver;
@BeforeClass
public void setup() {
System.setProperty("webdriver.chrome.driver","path/to/
chromedriver");
driver = new ChromeDriver();
driver.get("https://example.com/login");
}
Cont..

@Test
public void testValidLogin() {
WebElement emailField = driver.findElement(By.id("email"));
WebElement passwordField = driver.findElement(By.id("password"));
WebElement loginButton = driver.findElement(By.id("loginButton"));
emailField.sendKeys("[email protected]");
passwordField.sendKeys("validPassword");
loginButton.click();
WebElement homepage = driver.findElement(By.id("homepage"));
Assert.assertTrue(homepage.isDisplayed(), "Login failed with valid credentials");
}
Cont..

@Test
public void testInvalidLogin() {
WebElement emailField = driver.findElement(By.id("email"));
WebElement passwordField = driver.findElement(By.id("password"));
WebElement loginButton = driver.findElement(By.id("loginButton"));
emailField.sendKeys("[email protected]");
passwordField.sendKeys("invalidPassword");
loginButton.click();
WebElement errorMessage = driver.findElement(By.id("errorMessage"));
Assert.assertTrue(errorMessage.isDisplayed(), "Error message not displayed
for invalid login");
}
Cont..

 Pattern Testing
 Using patterns to test common faults.
 This approach helps identify issues related to common design patterns, data
structures, or functional requirements.
 Scenario: Input Validation for User Registration Form
 Fields to Test:
1. Username
2. Password
3. Email
4. Phone Number
Cont..

 Patterns to Test:
1. Username: Alphanumeric, 3-20 characters
2. Password: At least 8 characters, includes a mix of uppercase, lowercase,
digits, and special characters
3. Email: Standard email format
4. Phone Number: 10 digits
Cont..

 Pattern Testing Approach:

1. Username Patterns:
1. Valid: "user123", "User_456", "username_789"
2. Invalid:
1. Less than 3 characters: "us"

2. More than 20 characters: "thisisaverylongusername"

3. Contains special characters: "user@name"

Cont..

 Pattern Testing Approach:

2. Password Patterns:
• Valid: "Password123!", "Pass@1234", "Secure#Password1"
• Invalid:
• Less than 8 characters: "Pass12!"
• No uppercase letters: "password123!"
• No digits: "Password!"
• No special characters: "Password123"
Cont..

 Pattern Testing Approach:

3. Email Patterns:
• Valid: "[email protected]", "[email protected]",
"[email protected]"
• Invalid:
• Missing @: "userexample.com"
• Missing domain: "[email protected]"
• Invalid characters: "user@domain,com"
Cont..

 Pattern Testing Approach:

4. Phone Number Patterns:
• Valid: "1234567890", "0987654321"
• Invalid:
• Less than 10 digits: "123456789"
• More than 10 digits: "12345678901"
• Contains letters: "12345abcde"
• Contains special characters: "12345-6789"
Cont..

 Orthogonal Array Testing

 Using orthogonal arrays to create test cases that cover combinations of
inputs.
 Test Scenarios Based on Code Structure
 Designing test cases based on an understanding of the code structure and
logic.
Cont..

 Advantages:
• Balances the benefits of both white box and black box testing.
• Provides a more comprehensive testing approach.
• More effective in finding defects that are related to system behavior and code
structure.
 Disadvantages:
• Requires knowledge of both the internal structure and external behavior.
• Can be complex and time-consuming.
2. Automation Testing:
 Automated unit testing is another approach to software testing, and it involves
using specialized tools to test scripts automatically.
 Methodologies for Automated Testing
 The following are three strategies and approaches within automation testing that
will aid the test engineer in improving the quality of the software product being
tested.
1. Code-Driven
2. GUI Testing
3. Test Automation Framework
Code-Driven Testing

 Code-driven testing involves writing scripts or code to test other code.

 This method is commonly used in unit testing, where individual

functions or methods are tested in isolation.
Cont..

 Key Aspects:

 Unit Testing: Automated tests are written to test the smallest parts of
an application, typically functions or methods.

 Automated Test Scripts: Scripts are created to run the tests, compare
actual results with expected results, and report discrepancies.

 Continuous Integration: Tests are often integrated into the build

process to run automatically whenever code changes are made.
Cont..

 Advantages:
• Ensures that individual components work as expected.
• Helps in identifying bugs early in the development process.
• Can be run frequently without manual intervention.
 Disadvantages:
• Requires knowledge of programming and test scripting.
• Maintenance can be challenging if the codebase changes frequently.
GUI Testing

 GUI (Graphical User Interface) testing involves testing the software’s

user interface to ensure that it behaves as expected.

 This can include verifying the visual elements, user interactions, and
overall user experience.
Cont..

 Key Aspects:

 Automated GUI Test Tools: Tools like Selenium, QTP, or TestComplete can
be used to automate GUI tests.

 User Interaction Simulation: The tests simulate user interactions with the
application, such as clicking buttons, entering text, and navigating through
menus.

 Visual Verification: Ensures that the visual elements of the application

(buttons, icons, menus, etc.) are displayed correctly and function as
intended.
Cont..

 Advantages:
• Validates the user interface and user experience.
• Can catch visual bugs and issues with user interactions.
• Useful for regression testing to ensure new changes don’t break existing
functionality.
 Disadvantages:
• Can be brittle; minor changes in the UI may require test script updates.
• Typically slower than other forms of automated testing.
• Requires tools and frameworks specific to GUI testing.
Test Automation Framework

 A test automation framework provides a structured approach to

automate tests, making it easier to create, execute, and manage
automated test scripts.

 It includes guidelines, tools, and practices designed to achieve efficient

and effective test automation.
Types of Test Automation
Frameworks:
 Modular Testing Framework: Tests are divided into small, independent
modules, each representing a specific part of the application.

 Data-Driven Framework: Uses external data sources (e.g., CSV, Excel)

to drive test cases, allowing tests to run with multiple sets of data.

 Keyword-Driven Framework: Uses keywords to represent actions,

making it possible for testers to create tests without scripting.

 Hybrid Framework: Combines elements of modular, data-driven, and

keyword-driven frameworks to leverage the benefits of each.
Cont..

 Advantages:
• Provides a structured approach, reducing maintenance effort.
• Enhances reusability of test scripts and components.
• Improves test coverage and efficiency.
 Disadvantages:
• Initial setup can be complex and time-consuming.
• Requires knowledge of both the application under test and the framework
being used.
• May require ongoing maintenance as the application and framework evolve.
Coverage Analysis
 This technique is very popular due to its simplicity and effectiveness. We identify
paths of the program and write test cases to execute those paths.
 Each path covers a portion of the program.
 We define ‘coverage’ as a ‘percentage of source code that has been tested with
respect to the total source code available for testing’.
 We may like to achieve a reasonable level of coverage using control flow testing.
 Following testing techniques are used to analyse the coverage:
1) Statement Coverage
2) Branch Coverage
3) Condition Coverage
4) Path Coverage
1) Statement Coverage:
In a programming language, a statement is nothing but the line of code or instruction
for the computer to understand and act accordingly.
 It is the method of validating whether each and every line of the code is executed
at least once.
 It is used to calculate the total number of executed statements in the source code
out of total statements present in the source code.
 The goal of statement coverage technique is to cover all the possible executing
statements and path lines in the code.
Statement coverage= * 100
Cont..

#include<stdio.h>
#include<conio.h>

void main()
{
int a,b,c,x=0,y=0;
clrscr();
printf("Enter three numbers:");
scanf("%d %d %d",&a,&b,&c);
if((a>b)&&(a>c)){
x=a*a+b*b;
}
if(b>c){
y=a*a-b*b;
}
printf("x= %d y= %d",x,y);
getch();
}
2) Branch Coverage:
 Branch coverage or Decision coverage technique is used to cover all branches of
the control flow graph.
 It covers all the possible outcomes (true and false) of each condition of decision
point at least once.
 branch coverage follows decision point and branch coverage edges.
 Many different metrics can be used to find branch coverage and decision
coverage, but some of the most basic metrics are:
finding the percentage of program
paths of execution during the execution of the program.

Branch coverage= * 100

Example:
READ Username
READ Password
IF Count (Username) < 8
PRINT “Enter a Valid Username”
ENDIF
IF Count (Password) < 5
PRINT “Enter a Valid Password”
ENDIF
IF Count(Username & Password) < 1
PRINT “Please Fill the Username & Password Fields”
ENDIF
ELSE
PRINT “Login Successfully”
Cont..
3) Condition Coverage:
 Condition Coverage or expression coverage is a testing method used to test and
evaluate the variables or sub-expressions in the conditional statement.
 The goal of condition coverage is to check individual outcomes for each logical
condition.
 Condition coverage is better than branch coverage because we want to test every
condition at least once. However, branch coverage can be achieved without testing
every condition.
 Condition coverage does not give a guarantee about full decision coverage.
if ((a > b) && (a > c) )
{
<< Few Statements >>
}
else
{
<< Few Statements >>
}

There are four possibilities:

1) Both are true
2) First is true, second is false
3) First is false, second is true
4) Both are false
 Cont..
4) Path Coverage:
 Path Coverage testing is a structured testing technique for designing test cases
with intention to examine all possible paths of execution at least once.
 Path testing guarantee to 100% statement coverage, 100% branch coverage and
100% condition coverage.
 In this technique, control flow graphs are made from code or flowchart and then
Cyclomatic complexity is calculated which defines the number of independent
paths so that the minimal number of test cases can be designed for each
independent path.

Path coverage= * 100

Read A
Read B
IF A+B > 50 THEN
Print "Large"
ENDIF
If A+B<50 THEN
Print "Small"
ENDIF

 All possible paths

are-
1-3-5-7
1-3-5-6-8
1-2-4-5-7
 Read A and
Cyclomatic Complexity
B

V(G)= E-N+2P
A+B>50

= 8-7+2
Large
Number
V(G) = 3
End if

A+B<50

Small
Number
End if
Mutation Testing

 It is a popular technique to assess the effectiveness of a test suite.

 We may have a large number of test cases for any program.
 We can’t execute all of them so, we may select a few test cases using
any testing technique and prepare a test suite.
 But the test suite created is effective or not needs to be assessed.

 Mutation testing helps assess the effectiveness and quality of a test suit
and enhances the test suite, if it is not adequate for a program.
Mutation and Mutants

 The process of changing a program is known as mutation.

 This change is limited to one, two, or very few changes in the program.
 We prepare a copy of the program under test and make a change in a
statement of the program.

 The changed version of a program is called a mutant of the original

program.
 The behavior of the mutant may be different from the original program
due to the introduction of a change.
Cont..
Cont..
Cont..

 The mutants generated by making only one change are known as first
order mutants.
 We may obtain second order mutants by making two simple changes in
the program and third order mutants by making three simple changes,
and so on.
 The second order mutants and above are called higher order
mutants.
 Generally, in practice, we prefer to use only first order mutants in order
to simplify the process of mutation.
Cont..
Mutation Operators

 Mutants are produced by applying mutant operators.

 An operator is essentially a grammatical rule that changes a single
expression to another expression.
 The changed expression should be grammatically correct as per the
used language.
 If one or more mutant operators are applied to all expressions of a
program, we may be able to generate a large set of mutants.
 To kill a mutant, we should be able to execute the changed statement of
the program.
 If we are not able to do so, the fault will not be detected.
Cont..

 Some of the mutant operators for object-oriented languages like Java,

C++ are given as:
 Changing the access modifier, like public to private.
 Static modifier change
 Argument order change
 Super Keyword change
 Operator change
 Any operand change by a numeric value
Mutation Score

 When we execute a mutant using a test suite, we may have any of the
following outcomes:
 The results of the program are affected by the change and any test case of
the test suite detects it. If this happens, then the mutant is called a killed
mutant (the test fails).
 The results of the program are not affected by the change and any test case
of the test suite does not detect the mutation. The mutant is called a live
mutant (the test passes despite the change).
 The mutation score associated with a test suite and its mutants is
calculated as:
Cont..
 The total number of mutants is equal to the number of killed mutants plus
the number of live mutants.
 The mutation score measures how sensitive the program is to the changes
and how accurate the test suite is.
 A mutation score is always between 0 and 1.
 A higher value of mutation score indicates the effectiveness of the test suite
although effectiveness also depends on the types of faults that the mutation
operators are designed to represent.
 The test cases that identify the changed behaviour should be preserved and
transferred to the original test suite in order to enhance the capability of the
test suite.
 Hence, the purpose of mutation testing is not only to assess the capability of
a test suite but also to enhance the test suite.
 Mutation testing tools are also available in the market like Insure++, Jester
for Java (open source) and Nester for C++ (open source).
Example - Consider the program to find the largest of three numbers

 The test suite selected by a testing technique is given as:

A B C Expected
Outcome
6 10 2 10
10 6 2 10
6 2 10 10
6 10 20 20

 Generate five mutants (M1 to M5) and calculate the mutation score of this test suite
Solution

Muted Statement
Mutant No. Line No Original Line Modified Line
M1 11 If (A > B) If (A < B)
M2 11 If (A > B) If (A > (B + C))
M3 12 If (A > C) If ( A < C)
M4 20 If ( C > B) If ( C = B)
M5 16 Printf(“The Printf(“The
Largest Number Largest Number
is :%f\n”,C) is :%f\n”,B)
The actual output obtained by
executing the mutants M1-M5
 Actual Output of Mutant M1

Test A B C Expecte Actual

Case d Output
Outcome
1. 6 10 2 10 6
2. 10 6 2 10 6
3. 6 2 10 10 10
4. 6 10 20 20 20
Cont..

 Actual Output of Mutant M2

Test A B C Expecte Actual

Case d Output
Outcome
1. 6 10 2 10 10
2. 10 6 2 10 10
3. 6 2 10 10 10
4. 6 10 20 20 20
Cont..

 Actual Output of Mutant M3

Test A B C Expecte Actual

Case d Output
Outcome
1. 6 10 2 10 10
2. 10 6 2 10 2
3. 6 2 10 10 6
4. 6 10 20 20 20
Cont..

 Actual Output of Mutant M4

Test A B C Expecte Actual

Case d Output
Outcome
1. 6 10 2 10 10
2. 10 6 2 10 10
3. 6 2 10 10 10
4. 6 10 20 20 10
Cont..

 Actual Output of Mutant M5

Test A B C Expecte Actual

Case d Output
Outcome
1. 6 10 2 10 10
2. 10 6 2 10 10
3. 6 2 10 10 2
4. 6 10 20 20 10
Cont..

 Mutation Score = Number of mutants killed/ Total number of mutants

 Mutation Score = 4 /5 = 0.8
 Higher the mutant score, better is the effectiveness of the test suite.
 The mutant M2 is live in the example. We may have to write a specific
test case to kill this mutant.
Summary

 Mutant M1:
• Test Case 1: Expected 10, Actual 6 (Killed)
• Test Case 2: Expected 10, Actual 6 (Killed)
 Mutant M1 is killed.

 Mutant M2:
• All test cases produce the expected output.
 Mutant M2 is not killed.
Summary

 Mutant M3:
• Test Case 2: Expected 10, Actual 2 (Killed)
• Test Case 3: Expected 10, Actual 6 (Killed)
 Mutant M3 is killed.

 Mutant M4:
• Test Case 4: Expected 20, Actual 10 (Killed)
 Mutant M4 is killed.
Summary
 Mutant M5:
• Test Case 3: Expected 10, Actual 2 (Killed)
 Mutant M5 is killed.

• M1: Killed
• M2: Not killed
• M3: Killed
• M4: Killed
• M5: Killed
 Number of mutants killed = 4
 Total number of mutants = 5

 Mutation testing helps identify weak spots in the test suite, prompting the creation
of more effective tests to catch these bugs.
 Mutation Testing
Types of Mutation Testing:
1) Value Mutations:
In this type of testing the values are changed to detect errors in the program. Basically a small value
is changed to a larger value or a larger value is changed to a smaller value. In this testing basically
constants are changed.
Initial Code:
int mod = 1000000007;
int a = 12345678;
int b = 98765432;
int c = (a + b) % mod;
Changed Code:
int mod = 1007;
int a = 12345678;
int b = 98765432;
int c = (a + b) % mod;
2. Decision Mutations:
In decision mutations logical or arithmetic operators are changed to detect errors in the
program.

Initial Code:
if(a < b)
c = 10;
else
c = 20;

Changed Code:
if(a > b)
c = 10;
else
c = 20;
3. Statement Mutations:
In statement mutations a statement is deleted or it is replaces by some other statement.

Initial Code:
if(a < b)
c = 10;
else
c = 20;

Changed Code:
if(a < b)
d = 10;
else
d = 20;
Walkthrough
 Walkthroughs are informal meetings for evaluating and discussing the contents of
a software artifact (Ex. Requirements, design documents, code , test plans).
 The walkthrough is a review meeting process but it is different from the
Inspection, as it does not involve any formal process.
 They aim to identify defects, gather feedback, and improve the
quality of the product.
 The walkthrough is started by the Author of the code.
 In the walkthrough, the code or document is read by the author, and others who
are present in the meeting can note down the important points or can write notes
on the defects and can give suggestions about them.
Cont..

 Key Points:
• Objective: Find defects early, understand the document/code, and
improve it.
• Participants: Author, peers, technical experts, stakeholders.
• Process: The author presents the artifact, and participants provide
feedback.
• Outcome: List of identified issues and suggestions for improvement.
Unit Test Assertions

 Assertions are used in unit testing to check if a particular condition is

true.
 If the assertion fails, the test fails.
 Assertions help verify that the code behaves as expected.
 Common assertions include checking for equality, truth, null values, and
exceptions.
 Common Assertions:
 assertEqual(a, b): Checks if a equals b.
 assertTrue(x): Checks if x is true.
 assertFalse(x): Checks if x is false.
 assertIsNone(x): Checks if x is None.
 assertIsNotNone(x): Checks if x is not None.
 assertRaises(exception, callable, *args, **kwargs): Checks if an
exception is raised.
Types of Unit Test

1) Basic Unit Test:

 Basic unit tests are simple tests that focus on a single piece of
functionality, usually a function or a method, in isolation.
 Purpose: Verify that the individual units of code perform as expected.
Example

 public class CalculatorTest {

@Test
void testAdd() {
Calculator calculator = new Calculator();
assertEquals(5, calculator.add(2, 3));
assertEquals(0, calculator.add(-1, 1));
assertEquals(0, calculator.add(0, 0));
}
}
class Calculator {
public int add(int a, int b) {
return a + b;
}}
Types of Unit Test

1) Test-Driven Unit Test:

 Test-Driven Development (TDD) is a software development process
where tests are written before the actual code.
 The process follows a cycle of writing a test, running the test (which
initially fails), writing the code to pass the test, and then refactoring the
code.
 Purpose: Ensure that the code meets its requirements and is testable
from the start.
Example
 Step 1: Write a failing test:

public class CalculatorTest {

@Test
void testAdd() {
Calculator calculator = new Calculator();
assertEquals(5, calculator.add(2, 3));
}
}

class Calculator {
public int add(int a, int b) {
// Initially, this method does not exist or is empty
return 0;
}
}
Cont..
 Step 2: Write the code to pass the test:
class Calculator {
public int add(int a, int b) {
return a + b;
}
}
Cont..
 Step 3: Refactor (if needed) and ensure the test passes:

public class CalculatorTest {

@Test
void testAdd() {
Calculator calculator = new Calculator();
assertEquals(5, calculator.add(2, 3));
assertEquals(0, calculator.add(-1, 1));
assertEquals(0, calculator.add(0, 0));
}
}

class Calculator {
public int add(int a, int b) {
return a + b;
}
}
Data Driven Unit Test Assertion
 Data Driven Testing is a software testing method in which test data is stored in
table or spreadsheet format.
 Data driven testing allows testers to input a single test script that can execute
tests for all test data from a table and expect the test output in the same table.
 It is also called table-driven testing or parameterized testing.
 A development team can save time and money by using Data driven testing.
 We can reuse the test case as often as We like in other situations by altering its
parameters.
 Example

public class CalculatorTest {

@ParameterizedTest
@CsvSource({
"2, 3, 5",
"-1, 1, 0",
"0, 0, 0",
"10, -5, 5"
})
void testAdd(int a, int b, int expected) {
Calculator calculator = new Calculator();
assertEquals(expected, calculator.add(a, b));
}
}

class Calculator {
public int add(int a, int b) {
return a + b;
}
}
Why Data Driven Testing?
 Data Driven Testing is important because testers frequently have multiple data
sets for a single test and creating individual tests for each data set can be time-
consuming.
 Data driven testing helps keeping data separate from test scripts and the same test
scripts can be executed for different combinations of input test data and test
results can be generated efficiently.
Data driven testing framework

Test Data Application

Data File Driver Script
under Test

Expected Output Test Result Actual Output

(Compare)

Stored/written back in the same data file

Test Plan

 Key components involved in designing a test plan.

1. Test Environment
2. Test Configurations
3. Test Execution
4. Test Analysis
5. Test Revision
Test Environment
 A testing environment is a setup of software and hardware for the
testing teams to execute test cases.
 In other words, it supports test execution with hardware, software and
network configured.
 Test environment is configured as per the need of the Application Under
Test.
 There are some people involved in test environment setup
 System Admins,
 Developers
 Testers
 Sometimes users
Components

 Hardware Requirements: Details of the hardware needed for the

testing

 Software Requirements: Operating systems, databases. Browsers,

and other software components required

 Network Configuration: Details of network settings and configurations

 Test Data: Information on the data needed for testing, including how it
will be created and managed

 Tools: Any testing tools or utilities required for the testing process
Steps to set up

 Identify and document the hardware and software requirements.

 Set up the environment as per the documented requirements.
 Verify the environment to ensure it matches the requirements.
 Maintain and update the environment as needed during the testing
cycle
Test Configurations

 Configurations refer to the settings and setups specific to the test

environment to ensure tests can be conducted accurately.
 Components:
 Configuration Management Plan: Processes and tools used to
manage configurations.
 Version Control: Ensuring the right software and hardware versions
are used.
 Environment Variables: Any specific settings needed for the test
environment.
 Compatibility Settings: Ensuring compatibility with different
configurations (e.g., different OS versions, browsers).
Steps to Manage

 Define configuration items and manage them using a configuration

management tool.
 Document and control changes to configurations.
 Ensure configurations are applied consistently across different
environments.
Test Execution

 Test execution is the process of running the test cases and reporting the
results.
 Components:
 Test Scripts: Automated or manual scripts to be executed.
 Test Schedule: Timing and sequencing of test case execution.
 Execution Logs: Keeping logs of test execution for reference and
analysis.
 Defect Reporting: Process for reporting and tracking defects found
during testing.
Steps to Execute

 Prepare test scripts and review them for accuracy.

 Execute tests as per the test schedule.
 Log the execution details and report any defects.
 Monitor the test execution progress and manage any issues that arise.
Test Analysis

 Test analysis involves examining the results of test execution to

determine if the software meets the requirements.
 Components:
 Test Results: Data collected from test execution.
 Defect Analysis: Reviewing and categorizing defects.
 Coverage Analysis: Ensuring all requirements and functionalities have
been tested.
 Risk Analysis: Identifying any remaining risks after testing.
Steps to Analyze

 Collect and review test results.

 Perform defect analysis to categorize and prioritize defects.
 Conduct coverage analysis to ensure all areas have been tested.
 Assess any remaining risks and document findings.
Test Revisions

 Revisions are changes made to the test plan or the testing process
based on the analysis of test results and feedback.
 Components:
 Feedback Loop: Incorporating feedback from the testing team and
stakeholders.
 Continuous Improvement: Regularly updating and improving test
plans and processes.
 Documentation Updates: Ensuring all documents are updated to
reflect changes.
 Retesting: Executing tests again after changes to verify fixes.
Steps to Revise

 Collect feedback and review analysis results.

 Update test plans and processes based on feedback and analysis.
 Document all changes and communicate them to the team.
 Perform retesting to validate the changes.
Key areas to set up in Test Environment
 For the test environment, a key area to set up includes
 System and applications
 Test data
 Database server
 Front-end running environment
 Client operating system
 Browser
 Hardware includes Server Operating system
 Network
 Documentation required like reference documents/configuration
guides/installation guides/ user manuals
Process of Software Test environment setup

1) Setup of Test Server:

 Every test may not be executed on a local machine. It may need
establishing a test server, which can support applications.
 For example, Fedora set up for PHP, Java-based applications with or
without mail servers, cron set up, Java-based applications, etc.
2) Network
 Network set up as per the test requirement. It includes,
 Internet setup
 LAN Wifi setup
 Private network setup
 It ensures that the congestion that occurs during testing doesn’t affect
other members. (Developers, designers, content writers, etc.)
3) Test PC setup
 For web testing, you may need to set up different browsers for
different testers. For desktop applications, you need various types of
OS for different testers PCs.
 For example, windows phone app testing may require
 Visual Studio installation
 Windows phone emulator
 Alternatively, assigning a windows phone to the tester.
4) Bug Reporting
 A bug reporting tools for bug reporting, documentation and analysis.
5) Creating Test Data for the Test Environment
 Many companies use a separate test environment to test the software
product.
 The common approach used is to copy production data to test.
 This helps the tester, to detect the same issues as a live production
server, without corrupting the production data.
 Testers or developers can copy this to their individual test environment.
They can modify it as per their requirement.
Test Execution
 Test execution is the process of running tests on the software and
recording the results.
 Test execution is the process of executing the code and
comparing the expected and actual results.
 The test execution phase begins after the completion the test
planning stage and is carried out with the assistance of test cases,
which are prepared during the test planning phase, based on the pre-
decided business requirements.
 Two way to test execution:
1. Manual testing
2. Automation testing
Prerequisites of Test Execution:
 The process of designing and defining the tests should be completed.
 This requires basic computer skills and can often be assigned to interns
or employees with little technical background.
 Team members should have a thorough understanding of the test
data that needs to be tracked or monitored.
 The tools used for test execution must be ready before the test
execution, especially the test management tools.
 Process for tracking the test results, including the metrics should be
working accurately.
Test Automation or Automation testing
 Test Automation is the best way to increase the effectiveness, test
coverage, and execution speed in software testing.
Automated software testing is important due to the following reasons:
 It is difficult to test for multilingual sites manually.
 Test Automation in software testing does not require Human
intervention.
 Test Automation increases the speed of test execution
 Automation helps increase Test Coverage
Which Test Cases to Automate?

Test cases to be automated can be selected using the following

criterion
 High Risk – Business Critical test cases
 Test cases that are repeatedly executed
 Test Cases that are very tedious or difficult to perform manually
 Test Cases which are time-consuming
The following category of test cases are not suitable for
automation:
 Test Cases that are newly designed and not executed manually at
least once
 Test Cases for which the requirements are frequently changing
Automated Testing Process:

 Following steps are followed in an Automation Process

Step 1) Test Tool Selection
Step 2) Define scope of Automation
Step 3) Planning, Design and Development
Step 4) Test Execution
Step 5) Maintenance

Step 1) Test Tool Selection

 Test Tool selection largely depends on the technology the Application
Under Test is built on.
 Each type of tool or framework may serve different demands,
therefore having a thorough understanding of multiple tool types is
also a important factor in choosing your best tool.
Step 2) Define scope of Automation
 The scope of automation means the area of your Application Under
Test that will be automated.
 Following points help determine scope:
 The features that are important for the business
 Scenarios which have a large amount of data
 Common functionalities across applications
 Technical feasibility
 The extent to which business components are reused
 The complexity of test cases
 Ability to use the same test cases for cross-browser testing
Step 3) Planning, Design and Development
 In this step, we create an Automation strategy & plan, which contains
the following details-
 Automation tools selected
 Framework design and its features
 In-Scope and Out-of-scope items of automation
 Schedule and Timeline of scripting and execution
 Deliverables of Automation Testing
Step 4) Test Execution
 Automation Scripts are executed during this phase. The scripts need
input test data before there are set to run. Once executed they provide
detailed test reports.
 Execution can be performed using the automation tool directly or
through the Test Management tool which will invoke the automation tool.
Step 5) Maintenance
 Test Automation Maintenance Approach is an automation testing
phase carried out to test whether the new functionalities added to the
software are working fine or not.
 Maintenance in automation testing is executed when new automation
scripts are added and need to be reviewed and maintained in order to
improve the effectiveness of automation scripts.
Test Report
 Test Report is a document which contains a summary of all test activities and final
test results of a testing project.
 Test report is an assessment of how well the Testing is performed.
 Based on the test report, stakeholders can evaluate the quality of the tested
product and make a decision on the software release.
 if the test report informs that there are many defects remaining in the product,
stakeholders can delay the release until all the defects are fixed.
 What does a test report contain?
1) Project Information
Project name
Description
2) Test Objective
Test type
Purpose
3) Test Summary
The number of test cases executed
The numbers of test cases pass
The numbers of test cases fail
Pass percentage
Fail percentage
Comments
4) Defect
Total number of bugs
Status of bugs (open, closed, responding)
Number of bugs open, resolved, closed
Test Closure Activity
 A test closure report is a report that describes the testing activities performed by a
QA team.
 It is created once the testing phase is completed successfully or until the product
being tested meets the exit criteria.
 Test closure reports are created by lead QA engineers and are reviewed by all
other stakeholders.
 Test closure reports provide a detailed analysis of the types of testing carried out,
processes followed, the number of test cycles performed, the status of bugs found,
and open defects details.
 Every QA team prepares test closure reports to ensure that the product release is
stable.
 Key points to cover in a test closure report:
1) Project Summary
2) Test Environment Details/Build Information
3) Features Tested (In Scope)
4) Out of Scope
5) Test Execution Details
6) Defect Matrix
7) Open Bugs Details: Open defect details should be mentioned like test case Id,
summary, and priority.
8) QA Notes/Learnings: QA engineers can write additional remarks here as new
findings and learnings from the project.
9) Conclusion
 Sem 1

Dropped

Pass %

Pass

Appeared

Registered

0 20 40 60 80 100 120