Levels of Testing
Levels of Testing
9
LEVELS OF TESTING
OBJECTIVES
This chapter details verification and validation activities associated with various stages of software
development life cycle starting from proposal. It also covers various ways of integration
testing
9.2 PROPOSALTESTING
A proposal is made to the customer on the basis of Request for Proposal (RFP) or
(RFI) or Request for Quotation (RFQ) by the customer. Request for Information
Before making
understand the purpose of such request, and devise the proposed solution any proposal, the supplier must
customer problem and the possible solution. Any proposal accordingly. One must understand
prepared in response to such request is
by an organisation before sending it to the customer. It 1s reviewed reviewed
by ditferent panels or
organisation such as technical group and cpmmercial group. The intention is to ensure that the groups in the
must be able to stand by its words, the proposal getS converted into
if a contract. The
organisation
the impact of proposal on the organisation as well as on the organisation must assess
prospect/customer.
TechnicalReview Technical review mainly involves
technical feasibility
h e availabilíty and requirement of skill sets, the
of the kind of system or application
cal is also reviewed on the basis of time hardware/software and other
required
for requirements of
developing such
required to make the proposal suceeest, The technical proposal is a
description of overailsystem, and efforts
approach, and not
sOFTWARE TESTING: Levels of Testing 223
Principles, Techniques and Tools
Table9.1 Different agencies
involved at different may be specific, generic, present, future,
legal, operational and system requirements. Similarly, requirements these
levels of be unaware of many of requirements. The business analyst
Testing testing expressed, implied, etc. The customer may
and solution that the customer is
and system analyst must study the customer's line of business, problem
or intended requirement is a gray area
Proposal review
Customer, Business Agencies involved looking for before arriving at these requirements. Assumed, implied makes sure that requirements
and may be a reason for many defects in the final product. Requirement testing
CUstomer, Business Analyst, System Analyst Project Manager
Proposal testing criteria.
defined in requirement specifications meet the following
Requirement review Analyst, System Analyst,
CUstomer, Project
Leader Business Analyst, System Analyst, Project Manager
is from them. Requirements
Clarity Allrequirements must be very clear in their meaning and what expectedthe
Requirements testing Manager, Project Leader, Test clearly what the expected result of each transaction is, and who are actors taking part in the
must state very
Customer,
Leader BusinessAnalyst, System Analyst, Project Manager,
must be removed or clarified.
transactions. Anything which hampers the clarity of requirements
Design review Customer, Project Leader, Test
Design testing
Leader, TestBusiness Analyst, System Analyst,
Leader, Developer Architect, Project Manager, Project
CUstomer, Business
Leader, Test Leader, Analyst, System Analyst, Architect, Project
llustration 9.1
Test artifact review Team, Customer fair Such requirements cannot be implemented as nobody knows the exact requirements
One may have to raise queries to understandthe meaning of user friendlyness or fair perfomance. If possible such
Project Leader, Project
Unit testing Team,
Project Leader, Project Team, CUstomer, Tester, Test Leader
statements must have some numeical figures to illustrate them.Tester will be able to wnte exact test case on the basis
of these numbers.
Module testing Project Leader, Project Custormer
Integration testing Project Leader, Project
Team, Tester, Customer
System testing Project Manager, Test
Team, Tester, Customer Complete Requirement statement must be complete and must talk about all business aspects of the
Acceptance testing Leader, Tester Customer application under development. It must consider all permutations and combinations possible when application
Project Manager, Tester,
User/Customer is used in production. Any assumption must be documented and approved by the customer.
the final
accepted approach solution/method and
with customer. undergo many iterations of
Requirement and estimations at themay
indicative. It works on stage of proposal are not very changes but per discussions
as
lustration 9.2
rough-cut methods, and estimations are specific rather, they are
Commercial Review A ball-park figures that prospect may expect.
proposal undergoes financial feasibility Sometimes requirement statement does not speciiy the transaction. li requirement statement says Enter valid Login and
With respect to the Password, then it is not a complete
business. Commercial review and other types of feasibilities involved to Password. It could be
requirement
as (it
not does speciy how one should enter valid Login and now to go
in terms of may stress on the gross margins of the by Tab' by clicking the mouse or it will automaticaly go to Password)
or
money going out and coming in the project and fund flow fany part of transaction requires an assumption, it indicates incomplete requirements. One has to go back to cus
into installations
depending upon development organisation. Generally, total tomer or business analyst/system analyst to get this clanfied
completion
of phases of
some payment is split
completed, money is realised at those
instances. development
activity. As different phases are
Several iterations of
tor
Quotation (RFQ) and
proposals and scope changes may happen before all the
proposal agree on some terms and conditions for parties involved in Request Measurable
project.
a doing Requirements must be measurable in numeric terms as far as possible. Such definition helps
Validation of Proposal to
understand whether the requirement has been met or not while testing the application. Qualitative tems like
to Aproposal sometimes involves development of
explain the proposed
customer problem in the
approach to the prototypes or
proof of concept
problem of the customer. One must define the approach
good, 'suficient', and 'high' must be avoided as different people may attach diferent
meanings to these words.
model or prototype. of handling
n case of product
decide about the new organisation,
the product development
group along with marketing and sales
release of a functions
product. llustration 9.3
9.3 REQUIREMENT TESTING
Sometimes, requirenment statement may states that-enter valid credential and click Login to
very clear how mucth time it will take to reach next go to next page. tisine
Requirement creation involves These can not be vernied under page
and validate gathering customer requirements and arranging
them in a form to
usability as performance testin9
them. The
requirements may be categorised into different types such as technical, verify
economical,
ECnnques analools
Testable The Levels of Testing 225
requirement must help in creating use
application. Any requirement
be achieved which cases/scenario
by implementing these cannot be tested must be drilled downwhich can be used for testing the
requirements. further, to understand what is to 9.5 CODE REVIEW
Not
Conflicting The
between quality factors requirements must not conflict with each procedures, and
which needs to be Code reviews include reviewing code files, database schema, classes, object definitions,
problem in requirement collection agreed by the customer. other. There is possibility of trade-off
a
methods. Code review is applied to ensure that the design is implemented correctly by the developers, and
process. Conflicting requirements indicate possible followed Code must
correctly.have following
ldentifiable The guidelines and standards available for the purpose of coding are
way which can help in requirements must be distinctively identifiable. characteristics.
creating requirement They must have numbers some other
Validation of traceability matrix. Indexing or
requirement is very important.
Clarity Code must be written correctly as per coding standards and syntax, requirements torthe grven
statement. No Requirements
assumption is to be made Requirement testing involves writing use
platform. It must follow the standards and guidelines defined by the organisation/project/customer,as thecase
may be. It must declare variables, functions, and loops very clearly with proper comments. Code commenting
while using requirement
cases
cases can be
developed using the requirement writing use cases from requirement statement. must include which design part has been implemented, author, date, revision number, etc so that it can be
measurable, testable and not statement, requirements can be considered as If complete use traced to requirements and design.
statement. One has to ask the confilicting with each other. Each assumption indicates clear, complete,
customer whether the lacunae in Complete Code, class, procedure, and method must be complete in allrespect.It must suffice the purpose
the customer,
requirement statements must be updated. requirement
assumptions are correct or not. As per feedback for
given by for which it is created. One class doing multiple things, or multiple objects samecreated
a problem in design. Code must be readable and compilable. Proper indenting, and variable nitialisation
purpose ndicates
9.4 DESIGNTESTING must be followed as defined by coding standards.
Once the
requirement statement satisfies all characteristics defined Traceable Code must be traceable with design components. must It declare clearly about the
be considered
no traceability to design
etc. Code files as
system high-level architectural designing. The above, the system architect starts with requirements, design, author, date, having can
requirements. The system designer starts building designs made by system architects must
the low-level or detail
be traceable to redundant code which will never get executed even if design is correct.
design. Low-level design must be traceable to architectural design with the help of architectural Maintainable Code must be maintainable. Any developer with basic knowledge and training about
The design must also possess the characteristics design which in turn is traceable to
given
below. requirements. coding must be able to handle the code in future while maintaining or bug fixing it
Clarity A design must define all
functions, components, tables, stored procedures, and reusable
components very clearly. It must define any 9.6 UNIT TESTING
interdependence between different components and inputs/
outputs from each module. It must cover the information to and from the
application to other systems. Unit is the smallest part of a software system which is testable. It may include code files, classes and methods
Complete A design must be
complete in all respect. It must define the parameters to be which can be tested individually for correctness. Unit testing is a validation technique using black box
formats of data handled, etc. Once the
design is finalised, programmers must do their work of passed/received, methodology. Black box testing mainly concentrates on requirements of the system. In unit testing.units are
designs mechanically and system must be working implementing tested for the design in addition to requirements as low-level design defines the unit.
properly.
Traceable design must be traceable to requirements. The second column of
A
are tested to ensure that they work correctly as an individual as defined
matrix is design column. The project manager must check if there is any requirement
a requirement traceability Individual components and units
which does not have in design
corresponding design or vice versa. It indicates a defect if traceability cannot be established requires throwaway drivers and stubs as individual files may not be testable or executable
testing
completely. Unit
Implementable A design must be made in such a way that it can be implemented easily with selected without them
the
technology and system. It must guide the developers in coding and compiling the code. The design must include Unit testing may be performed in debugger mode how
to findvariable
the are
execution. But, it may not be termed 'black box testing' in such case as code is seen in
values changed during
interface requiremènts where different components are
communicating with each other and with other systems. Gray box testing is also considered as unit testing technique' sometimes as it cxamines the code m
debugging
Testable Testers make structural test cases on the basis of design. Thus, good design code and
in creating structural test cases.
a must help testers detail along with its functioning. Gray box testing may need some tools which can check the
functionality at the same time.
Validation of Design Design testing includes creation of data flow diagrams, activity diagrams, .Unit test cases must be derived from use cases/design component used at lowest levels of designs.
information fow diagram, and state transition diagram to show that information can flow through the system
Where the fow gets interrupted, it indicates that design is not complete. Flow of data and 9.6.1 DIFFERENCE BETWEEN DEBUGGING ANDTESTING
completely.
information in the system must complete the loop. It must consider the data definitions, attributes and input/ Many developers consider unit testing and debugging as the same thing. In reality there is no connection
output formats. Another way of testing design is creating prototypes from design. between the two. Difference between them can be illustrated as shown in Table 9.2
SOFTWARETESTING: Principles,
Table 9.2 Techniques and Tools Levels ofTesting 227
Difference between
debugging and unit testing how the system is integrated. Different
Debugging There are various approaches of integration testing depending upon
It involves code checking have different benefits and limitations
to approaches
locate the causes Unit testing
Code may be updated during of defect
t checks the
The test cases are
debugging defect and not the causes of the
9.8.1 BOTTOM-UP TESTING
units and modules, and then goes
not defined for
debugging
t does not involve
any correction of code defect Bottom-up testing approach
focuses on testing the bottom part/individual The
Generally covers positive cases modules for system testing and intersystem testing.
to see Test cases dehned based on
are
upward by integrating tested and working units and
works whether unit bottom-up integration and testing.
Covers positive as well negativerequirements
correctly or not and design as mentioned below,. Figure 9.2 shows a
as process goes
cases
are designed for a special pur-
stubs/drivers. Stubs and drivers
9.7 MODULE TESTING Units at the lowest level are tested using
must be tested before using them
for unit testing.
pose. They combined to form modules.
Modules may
Once the units are tested and
found to be working, they are
Many units come tested and found to be working may
act as
its execution, if thetogether
and form a units which are already
module. The module need only drivers, as the low-level
module cannot be may work of its own or
needs stubs and compiled into an executable. If the module can
is tested by tester. If it may need stubs/drivers for stubs. WIite the stubs
one goes upward. Developers may
are designed for testing as
testing mainly concentrates on the drivers, developers must create the work lfrequired, drivers and stubs
structure of the same and independently, it the input/output parameters must be
known while designing.
system. perform testing. Module and drivers as
termed 'classical approach' as it may
indicate a normal wayof doing things.
Module testing is done Bottom-up approach is also to imnplement.
on related unit-tested but practically, it is very difficult
work together as a components to find whether Theoretically, it is an excellent approach next level is taken. This gives very
module or not. individually the hierarchy is tested first and then
Module test cases tested units can Each component which is lowest in
must be traceable to software. But, it is a time consuming approach.
low-level design. requirements/design. Generally, module test cases are derived from
defect-free
robust and
System objects
where designed and tested individually before using them
are in a system.
Object-oriented design are already tested
calls the same object, we know that they are working correctly (as they
When the system
and found to be working in unit testing). bottom
routines which are used across the application,
Low-level components are general-purpose utility
libraries.
This approach is used extensively in defining
Module 1 Module 2 Module3 up
recommended
testing is the approach.
I f an organisation has optimised processes, this approach can be used as a validation of development
process and not a product validation. If all levels of testing are completed and all defects are found and
System closed, then system testing may act as certification testing to see if there are any major issues still left in
the system before delivering it to the customer.
.No stub/driver is required to be designed and coded in this approach. The cost involved is very less as it
tested before system is made. Tnis is can be very difficult to locate the problematic
for
correcting
areas them.
The systems tested by done during unit modules, and finally, the modules are Interface faults may not be distinguishable from other defects.
modified approach are testing followed
used for critical better in terms of residual by integration testing. Testers conduct testing based on few test cases by heuristic approach and certify whether the system
components, and also better for defects as bottom-up
approach is
vis used in
general.
It also saves time as all
customer-requirement elicitation as top-down
approach
works/does not work.
and the module correct(if the system has a user interface). User interface testing may involve colors, navigations, spellings,
Both top-down and
bottom-up
itself is as
large as a system, thensubprojects. When development fol- and fonts. Sometimes, there can be a thin line between functional testing ánd user intertace testing,
and one
Summary
This chapter details verification and validation activities
related to various stages of software
development phases. It gives advantages and disadvantages of the various
testing. approaches of integration