Module 4

Validation
Validation
• It
is a set of activities that ensures the software under consideration has
been built right and is traceable to customer to customer requirements.
• Validation is performed after the coding is over
• Last chance to discover bugs
• It enhances the quality of software

2
Validation Activities
1. Validation Test plan
2. Validation Test Execution

Validation Test plan

Validation starts once the SRS is prepared. The tester performs two parallel
activities:
• Verification at the phase
• Validation test plan

3
Validation Activities
Validation Test plan’s during the SDLC phase:
1. Acceptance test plan
2. System test plan
3. Function test plan
4. Integration test plan
5. Unit test plan

4
Validation Activities
Validation Test Execution
1. Unit validation testing
2. Integration testing
3. Function testing
4. System testing
5. Acceptance testing
6. Installation testing

5
1. Unit validation testing
It ensures whether the software meets atleast a baseline level of functionality prior
to integration and system testing

Bugs are detected and removed

Software is divided into modules and each module is not an isolated entity

To test a module, we need either Drivers or Stubs

6
1. Unit Testing

Fig. Design hierarchy of an example system

7
Stubs and Drivers

8
Drivers
• The Stubs and Drivers are those components that are used as a short-term alternative
for a module.

• Stubs and drivers both are dummy modules and are only created for test purposes.

• Driver module is a simulated module which provides the required input to the
testing module

• The Drivers establish the test environments and takes care of the communication,
estimates results, and also sends the reports.

• The drivers are mainly developed in the Bottom-up approach of incremental

integration testing.
9
Stubs
A stub can be defined as a piece of software that works similar to a unit which is
referenced by a unit being tested.

It is a dummy module and works as a stand-in for the subordinate unit and provides
minimum required behavior for that unit

Stubs are used in top down testing approach

It does not perform any action of its own

A constant or null must be returned by the stub to the calling module

It may simulate exceptions or abnormal conditions, if required.

10
Stubs and Drivers
Consider the following program:

main ()
{
int a, b, sum;
scanf(“%d”, &a, &b);
c = sum(a,b);
printf(“%d”,c);
}

Design a stub for the sum module?

11
2. Integration Testing
• It exposes inconsistencies between the modules
• Data can be lost across the interfaces
• One module combined with other will not give a desired result
• Datatypes and valid ranges may mismatch

Three approaches
1. Decomposition based integration
2. Call graph-based integration
3. Path-based integration
Fig. Decomposition Tree
12
Decomposition based integration
Two approaches
1. Non-incremental Integration testing
2. Incremental Integration testing

Non-incremental Integration testing

All untested modules are combined together and are tested
This approach is called Big-Bang method
It cannot be adopted practically

13
Decomposition based integration
Non-incremental Integration testing

How many drivers and stubs are

required for the given decomposition
tree?

14
Decomposition based integration
Incremental Integration testing
• It
tests one module at a time and integrate another one module, test it and
the process continues until all the modules are integrated and tested.
• Incremental testing uses several stubs and drivers to test the modules one
by one.
• This helps us discover many errors and defects in the modules.

15
Decomposition based integration
Types of Incremental Integration testing
1.Top-down Incremental testing
Depth-first integration
Breadth-first integration

Here, the higher level modules are tested first then the lower level modules are tested.
This approach uses Stubs which are mainly used to simulate the submodule, if the
invoked submodule is not developed this Stub works as a momentary replacement.

16
Decomposition based integration
Breadth-first integration

17
Decomposition based integration
Breadth-first integration

18
Decomposition based integration
Breadth-first integration

19
Decomposition based integration
Drawbacks of Top-down Integration Procedure
• Stubs and drivers must be prepared for testing one module
• Stubs are more complicated than they first appear
• Representations of test cases in stubs can be difficult

20
Decomposition based integration
2. Bottom-up Integration Procedure
In Bottom Up Integration Testing, testing takes place from bottom to up.
Lowest level modules are tested first and then high-level modules and finally
integrating the high-level modules to a low level to ensure the system is working as
intended.
Drivers are used as a temporary module for integration testing.

It is useful for integrating OO systems, real-time systems and systems with strict
performance requirements.

21
Decomposition based integration
2. Bottom-up Integration Procedure

22
Decomposition based integration
2. Bottom-up Integration Procedure

23
Decomposition based integration
It is better suited for monitoring the progress of integration testing along the
decomposition tree
• Failures can be easily tracked
• It requires more effort in the form of stub and drivers

The integration testing effort is computed as the number of test sessions. A test
session is one set of test-cases for a specific configuration.

The total no. of test sessions in decomposition based integration

= nodes – leaves + edges

24
Call Graph-Based integration
It is used to detect some behavioural bugs/defects.
Refine the functional decomposition tree into module call graph
A call graph is a directed graph, wherein the nodes are either modules or units, and a
directed edge from one node to another means, one module has called another
module.
A call graph can be captured in a matrix form called Adjacency Matrix
This avoids the use of stubs and drivers.

Two Strategies: Pair-wise integration testing and Neighbourhood testing.

25
Call Graph-Based integration: Example

26
Call Graph-Based integration

Figure: Example Call graph

27
Call Graph-Based integration

28 Figure: Adjacency Matrix

1. Pair-wise integration testing
Pairwise Testing (All-Pairs) is a test design technique that delivers hundred percent test
coverage.
Pairwise Testing is a black-box test design technique in which test cases are designed to execute
all possible discrete combinations of each pair of input parameters.
It is impractical to test all possible combinations of values for all those factors. So instead a
subset of combinations is generated to satisfy all factors.
All-Pairs technique is very helpful for designing tests for applications involving multiple
parameters.
Tests are designed such that for each pair of input parameters to a system, there are all possible
discrete combinations of those parameters.
The pairwise testing technique can dramatically reduce the number of combinations to be
covered but remains very effective in terms of fault detection.
29
1. Pair-wise integration testing
Example:
Car Ordering Application:
• The car ordering application allows for Buying and Selling cars. It should support trading in
Delhi and Mumbai.
• The application should have registration numbers, may be valid or invalid. It should allow the
trade of following cars: BMW, Audi, and Mercedes.
• Two types of booking can be done: E-booking and In Store.
• Orders can be placed only during trading hours.

30
1. Pair-wise integration testing
Answer
Step #1: Let’s list down the variables involved. 5) Order type
• 1) Order category a. E-Booking
a. Buy b. In-store
b. Sell
• 2) Location 6) Order time
a. Delhi a. Working hours
b. Mumbai b. Non-working hours
• 3) Car brand
a. BMW If we want to test all possible valid
b. Audi combinations:
c. Mercedes = 2 X 2 X 3 X 5000 X 2 X 2
• 4) Registration numbers = 240000 Valid test cases combinations
a. Valid (5000)
b. Invalid Invalid combinations will be infinite
31
1. Pair-wise integration testing
Answer
Step #2: Let’s simplify
– Use a smart representative sample.
– Use groups and boundaries, even when data is non-discrete.
– Reduce Registration Number to Two
• Valid registration number
• Invalid registration number

Now let’s calculate the number of possible combinations

=2X2X3X2X2X2
= 96

32
1. Pair-wise integration testing
Answer
Step #3: Arranging variables and values involved.
• When we arrange variables and values involved, it looks something like this.

Now order the variables so that the one with the most number of values is first and the least is last.

33
1. Pair-wise integration testing
Answer
Step #4: Arrange variables to create a test suite
• Let’s start filling in the table column by column. Initially, the table should look something
like this. The three values of Product (variable having the highest number of values)
should be written two times each (two is the number of values of next highest variable
i.e. Order category).

34
1. Pair-wise integration testing
Answer
Step #4: Arrange variables to create a test suite
• The Order Category column has two values. That’s how many times we need to insert the
values of the first column, Product.

35
1. Pair-wise integration testing
Answer
Step #4: Arrange variables to create a test suite
• For each set of values in column 1, we put both values of column 2. Repeat the same for
column 3.

36
1. Pair-wise integration testing
Answer
Step #4: Arrange variables to create a test suite
• We have a Buy and Delhi, but wait – there’s no Buy and Mumbai. We have a Sell and
Mumbai, but there’s no Sell and Delhi. Let’s swap around the values in the second set in the
third column.

37
1. Pair-wise integration testing
Answer
Step #4: Repeat the same steps for column 3 and 4.

When columns 3 and 4 are compared, each value in column 3 has both the values of column 4.
But when you compare the 2nd and 4th column, we have Buy and Valid & Sell and Invalid .i.e. Buy
does not have ‘Invalid’ and Sell does not have ‘Valid’.
Hence we need to interchange the last set of values in the 4th column.
38
1. Pair-wise integration testing
Answer
Step #4: Adding column 5 and 6.

39
1. Pair-wise integration testing
Answer
Step #4:
Column 6 (Order time) is problematic. We are missing Buy/Non-working hours and
Sell/Working hours. We can’t fit our missing pairs by swapping around values as we already
swapped all the rows if we swap now we may miss other possible pairs which are already
sorted. So, we add two more test cases that contain these pairs. Hence, the blank rows!

40
1. Pair-wise integration testing
Answer
Step #4:
Now we will fill in the empty cells as we desire because the other variable values are purely
arbitrary (or Don’t Cares ~).

All pairs in 8 cases, instead of all combinations in 96!

41
1. Pair-wise integration testing
Another Example

42
1. Pair-wise integration testing
Limitations
• It fails when the values selected for testing are incorrect.
• It fails when highly probable combinations get too little attention.
• It fails when interactions between the variables are not understood well.

43
2. Neighbourhood integration testing
A form of integration testing where all of the nodes that connect to a given node are the
basis for the integration testing.
The neighborhood for a node is the immediate predecessor as well as the immediate
successor nodes.
They can be defined as the set of nodes that are one edge away from the given node.

Neighborhood Integration Testing

• Reduces the number of test sessions
• Fault isolation is more difficult

44
2. Neighbourhood integration testing
The neighborhood of each node in the call graph is shown in the figure:

45
2. Neighbourhood integration testing
Another Example:

46
2. Neighbourhood integration testing – Pros and
Cons

47
3. Path Based Integration

• It combines structural and behavioral type of testing for

integration testing as we did for unit testing.

• Focus on interactions among system units rather than merely to

test interfaces among separately developed and tested units

• Every module has instructions to call another module or to return

to called module. This must be tested.
3. Path Based Integration
Some of the definitions are:

Source node: A program statement fragment at which program execution begins or

resumes.
Sink node: A statement fragment at which program execution terminates
Module Execution Path:
A sequence of statements within a module that
• Begins with a source node
• Ends with a sink node
• With no intervening sink nodes
3. Path Based Integration
Some of the definitions are:

Message:
• A programming language mechanism by which one unit transfers control to another
unit
• Usually interpreted as subroutine / function invocations
• The unit which receives the message always returns control to the message source
3. Path Based Integration
Module Execution Path
3. Path Based Integration
Some of the definitions are:

MM-Path:
A module to module path is an interleaved sequence of module execution paths and
messages
Used to describes sequences of module execution paths that include transfers of control
among separate units
MM-paths always represent feasible execution paths, and these paths cross unit
boundaries
3. Path Based Integration
MM-Path Examples:
3. Path Based Integration
Some of the definitions are:

MM-Path Graph:
Given a set of units their MM-path graph is the directed graph in which
• Nodes are module execution paths
• Edges correspond to messages and returns from one unit to another

• It directly supports composition of units and composition-based integration testing

3. Path Based Integration
MM-Path Details:
3. Path Based Integration
MM-Path Graph:
3. Path Based Integration
Pros:
Benefits of hybrid of functional and structural testing
• Functional – represent actions with input and output
• Structural – how they are identified
Avoids pitfall of structural testing (Unimplemented behaviours cannot be tested)
Works well with OO testing
No need for stub and driver development

Cons:
There is a significant effort involved in identifying MMpaths
Comparison of Integration-based Testing Techniques
Function Testing
It is the process of attempting to detect discrepancies between the functional specification
of a software and its actual behavior.
Its objective is to measure the quality of the functional components of the system.

It must determine, if each component

1. Performs in accordance to the specifications
2. Responds correctly in all situations
3. Moves data correctly from one event to another

To keep a record of the function testing, a function coverage matrix is used.

Function Testing
The primary processes for requirements based function test are:

1. Test Planning
2. Functional decomposition
3. Requirement definition Functions/Features Priority Test Cases
4. Test case design
F1 3 T2, T4, T6
5. Traceability matrix formation
6. Test case execution F2 1 T1, T3, T5
System Testing
It is the process of attempting to demonstrate that a program or system does
not meet its original requirements and objectives as stated, in the requirement
specification.

It tests with respect to the performance, security, maximum load, etc.

It is difficult to perform system testing as there is no specified methodology

It is a black-box testing. This testing evaluates the working of the system

from a user point of view, with the help of a specification document.
System Testing - Types
System Testing
Functionality Testing - Also known as functional completeness testing. To ensure that the
functionality of the product working as per the requirements defined, within the capabilities of
the system.
Recoverability Testing - To ensure how well the system recovers from various input errors and
other failure situations. Recovery testing done to show a software solution is reliable,
trustworthy and successfully recover from possible crashes.
Performance Testing - To make sure the system’s performance under the various condition,
regarding performance characteristics.
Interoperability Testing: To make sure whether the system can operate well with third-party
products or not.
Scalability Testing: To make sure the system’s scaling abilities in various terms like user
scaling, geographic scaling, and resource scaling.
Reliability Testing: To make sure the system can be operated for a longer duration without
developing failures.
System Testing
Regression Testing: To make sure the system’s stability as it passes through an integration of
different subsystems and maintenance tasks.
Documentation Testing: To make sure that the system’s user guide and other help topics
documents are correct and usable.
Security Testing: To make sure that the system does not allow unauthorized access to data and
resources.
Compatibility testing is done to ensure that the developed product is compatible with different
browsers, Hardware Platforms, Operating System and databases as per the requirement
document.
Stress Testing is done by increasing the number of users (at the same time) on an application to
an extent that the application breaks down. This is done to verify the point at which the
application will break down.
Installation Testing is to verify if the software gets installed without any issues.
System Testing – Steps involved
Acceptance Testing
It is the formal testing conducted to determine whether a software system satisfies its acceptance
criteria and to enable buyers to determine whether to accept the system or not.
It takes place at the end of the development process.
It checks functionality, performance, quality and interface criteria acceptable to user.
System testing is done by the development team whereas the user acceptance testing is done by
the end-users. It is a black-box testing.

The Acceptance test is designed,

• To determine whether the software is fit for the user
• Making users confident about the product
• To determine whether the software system satisfies its acceptance criteria
• Enable the buyer to determine to accept the system or not
User Acceptance Testing Process
Acceptance Testing - Types
1. Alpha Testing
The complete product is ready and tests are conducted at the development site by the
end-users.
It gives confidence to the users about the software and also user finds some bugs.
Testers and users are part of the testing

2. Beta Testing
Tests are conducted at the customer site by the end-users.
Development team does not have any control over the test environment.
References
1. Ian Sommerville, Software Engineering, 9th Edition, Addision-Wesley, 2016
2. Naresh Chauhan, Software Testing Principles and Practices, OXFORD University
Press, 2013.