Verification & validation & levels

of software testing
 Static Testing Dynamic Testing
Testing done without executing the program Testing done by executing the program

This testing does verification process Dynamic testing does validation process

Static testing is about prevention of defects Dynamic testing is about finding and fixing
the defects
Static testing gives assessment of code and Dynamic testing gives bugs/bottlenecks in
documentation the software system.
Static testing involves checklist and process Dynamic testing involves test cases for
to be followed execution
This testing can be performed before Dynamic testing is performed after
compilation compilation
Static testing covers the structural and Dynamic testing covers the executable file
statement coverage testing of the code
Cost of finding defects and fixing is less Cost of finding and fixing defects is high

Return on investment will be high as this Return on investment will be low as this
process involved at early stage process involves after the development
phase
More reviews comments are highly More defects are highly recommended for
recommended for good quality good quality.
Requires loads of meetings Comparatively requires lesser meetings
Types of software testing (verification & validation)

Verification tests are also known as static tests as no execution of

artifact is required to run it during these tests. Verification tests
include requirement specification reviews, design reviews, source
code reviews etc.

Validation tests are also known as dynamic tests as execution of

artifact under test is required to run it during these tests. Validation
tests are further divided into white box and black box tests.

Verification and Validation are two measures used to check that the
software product meets the requirements specifications. Together
they help improve software quality.
 Types of software testing
(verification & validation)
White box tests are named so as the source code is used during
software testing. White box tests are also known as structural tests
as they test small parts (structures) of the software product. Unit
tests and integration tests fall under these tests.

Black box tests are named so as the source code is not used during
software testing. Instead the executable binary machine code is
used. Black box tests are further divided into functional and non
functional tests. Non functional tests are further divided into
performance, security, usability etc. tests.

Black box tests are done both at system level as well as at

deployment (user acceptance) level.

Regression tests are done for both white box as well as black box
tests.
Verification & validation and associated software testing
types
 Verification & Validation

Verification is that part of software testing

where the testing is done on any software
project artifact which has no executable part
available. Software requirement specifications,
software designs, source code (testing to view
only the source code and not running it) are
project artifacts which are tested using
verification techniques. Requirement
specifications are tested using requirement
specification reviews, software designs are
tested using design reviews and source code
(construction) is tested using source code
reviews.
 Verification & Validation
Validation is that part of software testing where
the testing is done on software product by
executing its source code. Business logic at class
(component) level is tested using unit testing,
integration of component is tested using
integration testing, the software product is tested
using system testing and finally the deployment of
the software product is tested by end users using
user acceptance testing.
Levels of software testing (validation)
 Chapter 9
Software Testing (Verification & Validation)
Software testing (validation) is done at many levels. These levels
include unit, integration, system and deployment. All of these
levels come under validation.

Software components are tested for business logic correctness at unit

tests level. These components are then tested for integration with
source code build using integration testing.

The completely built software product after integration of all

components is again tested at system level. At this level functional
tests are done to test if software product features are working as per
software requirement specifications.

When the software product is ready then it is deployed at client site.

The end users do user acceptance testing to ensure that the
software product is working as per requirement specifications.
 Chapter 9
Software Testing (Verification & Validation)
Barry Boehm’s V Model for software testing
 Software Testing

Barry Boehm’s V Model is all about linking

artifacts developed during software development
lifecycle with levels of testing. Software
requirement specifications are used to perform
user acceptance testing. Software design
specifications are used to perform system
testing. Software unit construction (software
component) is used to perform unit testing.
Software module or sub system is used to
perform integration testing.
 Static testing: Manual or automated reviews of the documents .
Also known as Non-execution technique. This testing is
done during initial phase of testing to catch Defects early in
SDLC. It examines work documents and provides review
comments
Work document can be of following:
•Requirement specifications
•Design document
•Source Code
•Test Plans
•Test Cases
•Test Scripts
•Help or User document
•Web Page content
 Software verification
Verification tests include requirement specification reviews, software design
reviews, source code reviews etc.

Requirement specification reviews are done after these artifacts get completed.
Software testers or business analysts review the requirement specifications for
completeness, ambiguity and errors. If any errors or incompleteness is found
then it is rectified.

Software design reviews are done to check if the software design is correct as per
requirement specifications. Software design reviews also include reviewing
software component structure. If large classes were with many methods are
designed then it will lead to difficult to understand source code. Similar structural
problems can be found during design reviews. The identified design problems
will need to be rectified
.
Source code reviews are done to check for dead code, wrong implementation of
business logic etc. If any of these defects are found then they are rectified.

Code walkthroughs are also used for source code checking. Here the developer
presents his/her source code in front of other project team members. Team
members review the source code and find any defects are there in the code.
Software developer later rectifies those defects.
 Unit testing
Unit testing is done after a class or component is
constructed. Each class implements some business
logic. The source code in the class is checked to verify if
the business logic is implemented properly.

Unit testing is also done for extended testing when

database access is also involved. In such cases
business logic as well as database access aspect also
need to be considered. Testing database connection and
testing if database operation actually happens are also
done on these tests.
 Unit Testing Example
class addition {
public boolean adding (integer one, integer two){
if one >= 1 and one <= 100 and two >= 1 and two <= 100 {
integer x = one + two;
print (x);
return true;
}
else
return false;
}
}

The above class implements a business logic which needs to be tested. The test code can be
as following:
class testAddition {
public method void testAdding () {
boolean x = addition.adding(-3, 5);
}
}
The above test case is a negative test case as the expected result should fail. Using
positive and negative tests, we can find out if the business logic works fine.
For positive and negative testing, this test class can be provided with various values and
these values will be passed to the class under test through parameters.
 Unit Test: Example

public class interest_compute {

public integer interest_compute (float interest) {
integer principal = 293;
integer accrued_amount;
accrued_amount = principal + principal x interest/100;
return accrued_amount;
}
}
In some cases, the business logic seems to be fine but its
implementation is incorrect. The above example will
compute the result correctly but the final result is
incorrect as due to faulty variable declaration, the final
computation is truncated.
 Chapter 9
Software Testing (Verification & Validation)
class data_access {
public delete_row () {
string y = ‘Ammar’;
integer x;
search in the database table and find the record number where customer name is Ammar
and assign it to x;
if x = record number of Ammar then
delete row from customer table where customer name = y;
}
}
When a class involves database access then apart from unit testing, database
connection needs to be checked as well as to find out if database operation also took
place.

In the above example, you will also need to find out if the mentioned record has been
deleted from the database.
Database operations work on the principle of first searching
a particular record and then doing database operations.
If a particular record is changed or deleted after a test then
the same test script will not work next time.
Since the particular database record was changed after
execution of test script the first time, next time you may
need to change test script.

If database connection and database operations are not

needed to be tested then the database environment can
be simulated using tools.
 Integration testing

Integration testing is used to find out if a newly created

class or component or module can be integrated
correctly with the rest of the software product. Integration
testing thus is a test of the interfaces of a class or
component or module. If the interface of the class or
component or module is correct then the class or
component or module will integrate with the rest of the
software product.
Integration testing in continuous integration of source code
build environment
 Continuous integration
Continuous integration of source code build is used on agile
projects.
Here when a unit of source code (class or component) is
written by developers is integrated with the source code
build.
Integration testing is performed before a piece of source code
is integrated with the source code build.
If integration testing fails then the unit piece of source code is
checked if it has any interface problems due to which it is
not able to integrate.
Corrected piece of source code is again tested for integration.
This continues till there are no integration failures. Once
integration test passes then this piece of source code is
integrated with the source code build.
Frequency of integration of source code
 Explanation
On Waterfall projects, separate modules of software
product are developed first. They are later integrated
with each other.
On agile projects however, source code is continuously
integrated.
These 2 strategies imply that frequency of integration of
source code vary considerably.
Whereas in case of Waterfall projects, frequency of
integration of source code is infrequent; in case of agile
projects, the frequency is very high.
In fact on agile projects, frequency of integration of source
code with the source code build is 100%.
Every time a new class is created, it is immediately
integrated with the source code build.
 Explanation
This discussion implies that integration testing on agile
projects is done frequently.
This leads to a better software product with less number of
integration software defects.
When modules of software product are tried to be integrated
with each other then a large number of integration problems
will arise.
In this scenario, it is also difficult to pin point the cause of
integration problems.
Thus fixing integration problems is also difficult and it takes
time in fixing them. This is the reason, software projects are
increasingly using continuous integration of source code.
 System, regression and user acceptance
testing
Many types of testing are done on software projects during system and
user acceptance testing.
Functional testing is done to test if software product features are
working as per software design and requirement specifications.
Non functional testing is done to test if software product is working
under acceptable limits even when environmental conditions are
deteriorated. If a website receives tremendous web traffic but even
with so much traffic, the website is able to respond user page
requests within acceptable limit then performance of this website is
considered good. Similarly if a website receives malware attacks but
is still able to overcome these attacks well and keeps running then
security of this website is considered good.
Non functional tests include performance, security, usability,
localization etc. types of tests.
Regression tests are also performed during system testing for software
products which are built incrementally. Regression tests check if
existing functionality of a software product features still work after
integration of increment of software product features.
Software testing done in software development,
deployment and maintenance
Software testing is not only done during software development
but also during deployment and maintenance.

During software development all verification and validation tests

are performed (requirement specification reviews, design
review, code reviews, unit, integration, system (functional and
non functional tests). Regression tests are also performed.

But during deployment, only user acceptance testing is

performed. During deployment, all black box testing
(functional and non functional) tests are performed. The goal
here is to validate that the software product works as per the
requirement specifications.

After deployment a software product is used by software users.

This is the production phase of the software product. During
production, software product is tested for performance and
sanity tests. When the software product is taken for
maintenance then regression tests are also done.
Alpha-Beta testing cycle
 Other important software tests

There are some other tests which are also done on software products.

If alpha and beta release of a software product are done then alpha
and beta testing is done for the software product.
Smoke testing is done when a piece of source code is integrated with
the source code build.
Planned testing involves writing test cases and then executing them to
find software defects.
Planned testing has a limitation that only limited testing can be done
within the allowed time limits for testing.
Exploratory testing removes this barrier. Instead of spending too much
time in writing test cases, exploratory testing does not use test
cases at all. A software tester will just run various commands to
execute business processes and find out if they are working
correctly. If the tester finds any incorrect behavior then he/she will
file a test defect with the details. Using this technique more testing
of the software product can be done within a short span of time.
 coverage based testing
• A coverage based testing strategy is used when it is
known in advance as to which part of the software
product needs to be tested. This strategy is adopted
when it is known that it is not possible to test the entire
software product in the allotted time for software testing
on a software project. A good strategy in such situation
is to prioritize test cases and do testing only for test
cases which are high on priority.
Penetration testing is a usually a form of
black box security testing. In a pen test you are
trying to break into the server as many times as
possible and report back on how they where
able to break in. This is often done multiple
times to make sure that the patches hold water.
This is an important type of security testing
because its as real world as it gets. (Common
tools for web application penetration testing are
Acunetix)
 Chapter 9
Software Testing (Verification & Validation)
Test case description
A test case consists of test case description, entry criteria, exit criteria,
expected result, actual result and test execution result (pass/fail). Test case
description contains steps which are needed to be done during execution of
a test case. Entry criteria is the conditions which should be valid before a
test case can be executed. Exit criteria is the conditions which should be
valid after a test case is executed. Expected result is the expected outcome
of computation after execution of a test case. Actual result is the actual
outcome of computation after execution of a test case. If actual result is
same as expected result then the test case passes and it is assumed that
the tested business process has worked as per expected. Thus there is no
software defect. In contrast if the actual result is not the same as expected
result then it is assumed that the tested business process has not worked
as per expected. Thus there is a software defect and it needs to be fixed.
 Chapter 9
Software Testing (Verification & Validation)
Test case design
Designing good test cases is important because all testing needs to be
carried out within time limits. If test designs are not good then it may
lead to ineffective testing. Even with limited number of good test
cases it is possible to test a software product effectively.
Using boundary value analysis it is possible to test a business process
with just 3 test cases. In one test case, a test value which is above
the allowed range of valid input can be used for a negative result
and assess if the business logic works correctly. Similarly a value
below the allowed range of valid input can be used for a negative
result and assess if the business logic works correctly. Finally a
value within the allowed range of valid input can be used for a
positive result and assess if the business logic works correctly.
Using decision trees it is possible to test complex business logic where
a matrix of values are needed to test a business process.
 Chapter 9
Software Testing (Verification & Validation)
Test environment
A good test environment is important for finding software
defects. If a good test environment is not provided then it
can lead to difficulty in finding defects.
Once a software release needs to be tested then a stable
and isolated test environment is created on a computer.
Exact replica of the software product is installed on this
environment. Test data is also created.
Apart from the test environment, supporting facilities like
defect tracking system, configuration management
system etc. also need to be prepared for a testing cycle.
 Chapter 9
Software Testing (Verification & Validation)
Test cycle
 Chapter 9
Software Testing (Verification & Validation)
Test cycle is the process which is followed from test planning to test
case writing to test execution to defect logging.
Test planning involves finding out which software features need to be
tested, how testing will be done and for how long testing activities
will be carried out.
Test case design involves finding out what testing techniques
(boundary value analysis, decision trees etc.) will be used and how
much test coverage will be done during test cycle.
Test cases are written by testers after test design is decided. How
many test cases needs to be written will depend on test coverage,
testing techniques used etc.
Test case execution starts when test preparation is done and test
cases have been written. All the failed test cases will result in
pointing to software defects. These software defects are logged so
that they can be fixed.
 Chapter 9
Software Testing (Verification & Validation)
Software defect cycle
 Chapter 9
Software Testing (Verification & Validation)
When test case execution starts then some of the test
cases will pass while others will fail. The failed test case
point to software defects. These software defects are
logged in a defect tracking system. The software
developer who owns the software product feature which
contains the software defect is notified. The software
developer fixes the software defect and notifies the
software tester who had logged the software defect. The
tester then verifies if the software defect indeed has
been fixed. If the defect is found to be fixed then the
tester will close the software defect in the defect tracking
system.
This is the complete lifecycle of software defect and its
fixing.