As per Revised Syllabus of

MSBTE · I SCHEME

·\ .

..
Software Testing .
T. Y. Diploma (Semester - V) ·
Computer Engineering Group (CO/CM/CW) .

Mrs. Anuradha A. Puntambekar

M.E'. (Computer)
Formerly Assistant Professor in
P.E.S. Modern College of Engineering,
Pune

Vaishali Rane
B.E., M.E. (CMPN)
HOD (Computer Engineering),
. Thakur Polytechnic
Thakur Complex, Kondivoli E, Mumboi-401101

Nilesh J. Vispute
M.Tech. (Computer Science)
Senior Lecturer,
Provin Rohidos Pati!"Polytechnic
Miro Bhoyondar, Mumbai 401105 ·-

~ ®TECHNICAl,. . ·
. -~ · PUBLICATIONS Website: www.technicalpublications.org
An Up-Thrust for Knowledge IJ_https://www.facebook.com/technicalpublications

(i)
.-S YLLABUS
· Software Testing (22518)
THchlng Scheme. Credit · Examination Scheme
(L + T + P) Practical
Theory

L T p Paper l--..:E:::S=E-L__:_P::,A_.J___:_Tota:=l..:..-~-E'"TSE_--t__P
Hrs. 7A_-t__To1ta_1---I
Max Min Max Min Max Min
Max Min Max Min Max Min

3 2 5 3 70 28 30* 00 100 40 25@ 10 25 10 50 20

Unit Unit Outcomes (UOs)

-Topics and Sub • topics
(in cognitive domain)

Unit· I la. Identify errors and bugs in the 1.1 Software Testing, Objectives of Testing.
given program. 1.2 Failure, Error, Fault, Defect, Bug
Basics of Terminology.
Software lb. · Prepare test case for the given
Testing and application. 1.3 Test Case, When to Start and St9p
Testing le. Describe the Entry and Exit Testing of Software (Entry and Exit
Methods Criteria for the given test Criteria).
application. 1.4 Verification and Validation (V Model),
ld. Validate the given application Quality Assurance, Quality Control.
using V model in relation with 1.5 Methods of Testing: Static and dynamic
quality assurance. Testing
le. Describe features of the . given 1.6 The box approach : White Box Testing:
.testing method. Inspections, Walkthroughs, Technical
Reviews, Functional Testing, Code
Coverage Testing, Code Complexity
Testing.
1.7 Black Box Testing: Requirement Based
Testing, Boundary Value Analysis,
Equivalence Partitioning,

Unit- II 2a Apply specified testing level 2.1 Levels of testing

for the given web based Unit Testing : Driver, Stub
.Types and application.
Levell of 2.2 Integration Testing Top-Down
2b Apply Acceptance testing for Integration, Bottom-Up Integration
Testing given web based application Bi-Directional Integration.
2c Apply the given performance 2.3 Testing on Web Application
testing for the specified Peformance Testing Load Testing,
. application. Stress Testing, Security Testing,
2d Generate test cases for the Client-Server Testing
given application using · 2.4 Acceptance Testing : Alpha Testing and
. regression and GUI testing. Beta Testing, Special Tests : Regression
Testing, ·GUI testing.

(n,\
 Unit-Ill 3a. Prepare test plan for the given 31 ·Test Plannning : Preparing a Test Plan,
application. · Deciding Test Approach. Setting Up
Test
Management 3b Identify the resow-ce Criteria for testing, identifying
requirement of the given Responsibilities, Staffing, Resow-ce
application. Requirements, Test Deliverables, Testing
Tasks. . ·
3c. Prepare test cases for the given
application. 3.2 Test Managment : Test Infrastructure
Managem~t, Test People Management.
3d. Prepare test report of executed
test cases for given application. · 3.3 Test Process : Base Lining a Test Plan,
Test Case specification.
3.4 Test Reporting .: Executing Test Cases,
Preparing Test S y Report.
Unit. rv 4a. Classify defects on the basis
estimated impact.
Defect 4.1 Defect Classification, Defect
Management 4b. Prepare defect template on the Management Process.
given application.
4.2 Defect Life Cycle, Defect Template
4c. Apply defect m~gement
4.3 Estimate Expected Impact of a Defect; · · · ---
process on the given
application. Techniques for finding Defects~ '--•------
Reporting a Defect __ _ . -- ·-· _,_ ____ ____
4d. Write procedure to . find defect .
using the given technique.

,__----+--------"-----+---------------:-..:" "-""""'
:' -- ·1- t ...-....
Unit· V Sa. Improve testing efficiency 5.1 Manual Testing and Need · · for-- .... ·
using automated tool for given Automated Testing Tools ----
Testing Tools application.
and . 5.2 Advantages and Disadvantages of Using_. . ____... .
Measurements Sb. Identify different testing tools Tools
to test the given application. 5.3 Selecting a Testing Tool.
Sc. Describe Metrics and 5.4 When to Use Automated Test Tools,
Measurement for the given Testing Using Automated Tools.
application
5.5 Metrics and Measurement : Types of
Sd. Explain Objct oriented metrics Metrics, Product Metrics and Process
used in the given testing Metrics, Object oriented metrics in
application testing.
 T~·L E OF CONTENTS

Unit - I 1.6. 1.1 Inspections . . . . . . . . . . . . . . . . .

.. · 1• 10
1.6.1.2 Wallabrough ...... , ... ·..... 1 .
. . · · lt
Chapter • 1 Basics of Software testing and 1.6.1.3 Technical Reviews ........ . . . ... 1•
12
Testing Methods (1 -1) to (1 • 20) 1.6.1.4 Unit/Code Functional Testing · .. :. 1. 12

1.1 Software Testing ........................ 1 _1 1.6.1:s Code Coverage T ~ · · · · · · •... l -1 3

1.6.1.6 Code Complexity Testing.····• ... 1- ts
1.1.1 Definition : . . . . . . . . . . . . . . . . . . . . . . . . . 1 _ 1

1.1.2 Objectives ?fSoftware Testing .......... 1. 1 1.7 Black Box Testing ... • • • • · · · · · · · · · · · · · • • 1-17
1.7.1 Requirement Based Testing • • • · · · • • ... 1-17
1.1.3 ·Principles of Testing .................. 1. 1
1.7.2 Bo1D1dary Value Analysis • • • • · · · · • • • .. 1 • 18 .
l.1.4 Skills of Software Tester ......... ; ..... 1 - l
1. 7.3 Equivalence.Partitioning . • • • • • • • • • • ... 1 -·19
1.2 Failure, Error, Fault, Defect, Bug Tenninology 1 • 2
1. 7.4 Difference between Black Box and White Box:
13 TestCase ......... '.····················1-2
. Testing ........... •• .·••••·····•••• .1-20

l.3.1 What is Test Case? .......... ; ........ 1-2 Unit - II

1.3.2 Features of Test Cases ................. 1 - 3
Chapter • 2 Types and Levels of Testing
1.3 .3 When to Start and Stop Testing of Software ? (2 • 1) to (2 • 1~)
................................... l - 3
2.1 Levels of Testing ..... , .................. 2 : 1 .
'
1.4 ·Verification and Validation, Quality Assurance,
., Quality Control .............. .: .......... l - 3 2.2 Unit Testing ........................ .- ... 2 - 2

. · t.4.1 Verification and Validation (V Model) .... l - 3 2.2.1 · Errors Identified during Unit Testing ..... 2 - 3
1.4.1.i Verification ..................... 1-3 2.2.2 Unit Testing and Debugging . .......... : 2 - 3
1.4.1.2 Validation ...................... 1 -4 2.2.3 Driver and Stub ...................... 2 - 3 .
1.4.2 V Model ........................... l - 5
2.3 Integration Testing .......... 1 • • • • • • • • • • • 2 - 4·
1.4.3 Quality Assurance .... . ............... I - 6
2J.l Top Down Integration ................. 2-4
1.4.4 Quality Control ...................... I - 6
2.3.2 Bottom Up Integration . ; .............. 2-5
1.4.5 Difference between Quality Assurance and
Quality Control ...................... l - 7 2.3.3 Bi-Directional Integration .............. 2- 6

l.S Methods of Testing ...................... 1- 7 2.4 Testing on Web Application ............... 2- 7

I

1.5.1 Static Testing ........................ 1 - 7 2.4.1 Performance Testing .................. 2 • 8 j

1.5.2 Dynamic Testing·....... _.............. 1 - 8 2.4.2 Process for Perfonnance Testing ........ 2 ~ 8 ,
1
, 1.6 The Box Approach ......... .............. 1 - 8 2,4.3 Load Testing ........................ 2 • 9

1.6.1 White Box Tes~g ................... 1 • 8 . 2.4.4 Stress Testing ....................... 2 - 9 1

l

2.4.5 Security Testing ............ .. ....... 2. 10 ,

 -----
...··········--··-··········..·····-
· ................................................................ _ _...;_(vil
_..:.,
) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __

2.4.6 Client Server Testing ................ 2 _ 11

Unit - IV
2.$ Acceptance Testing ..................... 2 _ 12
2.5.1 Alpha Testing ................... ; .. 2 -. l2 Chapter - 4 Defect Management
(4 - 1) to (4 - 8)
2.5.2 Beta Testing ....................... 2 _ 13
4.1 Defect Classification and Management .. : .... 4 ·_ 1
2.6 Special Tests ........ .. ................ 2 _
14
4.1.1 Defect Classification .................. 4 - 1
2.6.1 Regression Testing .......... . ....... 2 _ 14
4.1.2 Defect Management .... . ............. 4 - 2
2.6.2 GUI Testing........................ 2 _ 14 .
4.2 Defect Life Cycle and Template ............ 4 - 4
Unit - III
4.2.1 Defect Life Cycle ........ .. .......... 4 - 4

Chapter • 3 Test Management 4.2.2 Defect Template ....... .. ............ 4 - 4

(3 - 1) to (3 - 12) 4.3 Impact, Technique and Reporting ... ... ..... 4 - 5

3.1 Test Planning ............................ 3 - 1 4.3.1 Estimate Expected Impact of a Defect .. . . 4 - 5

3.1.1 Preparing aTest Plan .................. 3 - 1 4.3.2 Techniques for Finding Defects ......... 4 - 6

3 .1.2 Deciding Test Approach ............... 3 - 2 4.3.3 Reporting a Defect .......... : ........ 4 - ·6

3.1.3 Setting up Criteria for Testing . ......... 3 - 2 Unit - V

3.1.4 Identifying Responsibilities, Staffing and
Chapter-5 Testing Tools and Measurements
.Training Needs ...................... 3 - 2
(5-1) to (5-22)
3.1.S Identifying Resource Requirements ...... 3 - 3
S.1 Manual Testing and Need for Automated
3 .1.6 Identifying Test Deliverables ........... 3 - 3
Testing Tools ........... .. .. . ........... 5 - 1
3.1.7 Testing Tasks ................ ; . : · · · · 3 - 3 5.1.1 Manual Testing ......... . ............ S - 1
. 3 4 5.1.2 Advantages of Manual Testing .... .' ..... 5 - 1
3.2 Test Management ......... • • • • • • • • • · · · · · -
5.1.3 Disadvantages of Manual Testing/ Limitation
3.2. 1 T est Infrastructure Management .. • • • • · · · 3 - 4 ofManual Testing .................... S -2
3.2.2 Test People Managem.ent · · · · · · · · · · · · · · 3-6
. . 5.1.4 Comparison between ~anual Testing and
Automation Testing................... S - 3
3.3 Test Process .... •••···········.······
.... 3-6
5.1.S Need of Automated Testing Tools . ...... S - 3
6
3.3.1 . Base Lining a Test Plan . • • · · · · · · · · · · · · 3 - 5.1.6 Why Automated Testing??? ............ 5 - 4
3 6
3.3.2 Test Case Speeification · · · · · · · · · · · · · · · · -
5.2 Advantages and Disadvantages ofusing..ToolL.i.:.~ L_., _
3.3.3 Executing Test C ases ........... . ..... 3-6
5.2.1 Advantages of using Tools ............. S - 4
. 3 - 11
3.4 · Test Reporting · · · · · · · · · · · · · : · · · · · · · · · · · · 5.2.2 Disadvantages of using Testing Tools .... 5 - S
. · rt .... 3-11 5.2.3 Features of Test Tool : Guidelines for Static and
3 .4.1 Test Incident Repo · · · · · · · · · · · · · ·
Dynamic Testing Tools ................ S - 1
rt ......... 3-11
3 .4.2 Test Cycle R epo · · · · · · · · · · ·
D rt 3-11 . 5.3 Selecting a, Testing Tool .................. S - 9
3 _4 _3 Preparing Test Summary L ' ~ • • • • • • • •

TECHNICAL PUBLICATIONS'" - An up thrust for /cnowledge

 ......................
.<viii).....................·.........................-....................................................... .
---------
5.4 When to use Automated Test Tools, Testing
us'ing Automated Tool. ........ . ....... : . S - 10
5.4.1 When Does Test Automation Make S~e ?
•.•••.• • ••.••••..••••.• . •• . ••••.•• S - 10
5.4.2 Testing using Automated Tools
(Test Automation) ................. .. S - 11

5.5 Metrics and Measurement : Types of Metrics,

Product Metrics and Process Metrics, Object
Oriented Metrics in Testing ............... S - 11
5.5.1 Metrics ... . ... . .... . ............ . . 5 - 12
~.5.2 What is Software Test Measurement? ... 5 - 13
5.5.3 Types of Metrics .............. .. ... . 5 - 14
5.5.3.1 Types of Manual Test Metrics .. . .. 5 -14
5.5.3.2 Examples of Software Testing Metrics
....•.... .. ... .. ..... :-.. . ........ 5 - 14
5.5.4 Product Metrics and Process Metrics, Object.
Ori~ted Metrics in Testing ........... 5 - 15
5.5 .4.1 Product metrics and Process metrics 5 - 15
5.5.4.2 Object Oriented Metrics .. .. . . .. . . ~ -20

Solved Sample Papen ......... ~ .... (S - 1) to .(S - 4)

-
TECHNICAL PUBLICATIONS~· An up thrust for knowledge
 -------------
UNIT· I

1 Basics ·of Software Testing

and Testi~g Methods

(II] Software Testing 3. Testing is . to be carried out throughout the ·

software life cycle and not simply at the end of the
OJJ] Definition sof~are development process. ,
• Testing is defined as execution of work product 4. First, test the test cases adopted for testing.
with an intent to find the defect. .
5. Testing encompasses defect prevention.
• Testing is a process of uncoverin$ as many errors as 6. Testing should help for both defect prevention and
possible. defect detection.
• Testing is used to confirm that a program performs 7. Testing is a process that need to be carried out
its intended functions correctly. constantly.
~ Objectives of Software Testing _ 8. Exhaustive testing is not possible. During the
testing of the program, only the presence of defects
Following are objectives of software testing -
can be shown and not their absence.
1) Testing should find the defects before customer
9. The testing activity must be done with intelligent
finds them out.
and systematic automation tool.
2) Testing should be applied all through the software_
10. Testing requires talented, committed staff who
life cycle.
possess the ability to work in a team.
3) The testing must be conducted with some goal or
reason. [I]]] Skills of Software Tester
4) Testing must prevent the defects. Following skills are required by software tester -

5) Tests the tests firsts. 1) Analytical Skills :

6) During testing, find .the scenario where the product • Analytical skills are those skills that help to break
does not perform as per the requirements and up a complex software system into smaller units to _
perform testing there. gain better understapding of the system.
7) During testing, _·find the scenario where product • This kind of skill also helps in creating appropriate
does things that it is not supposed to do. test cases.
8) · Testing must be a fine balance of defect prevention
2) Comm.unication Skills :
and defect detection.
• A software tester must have verbal as well as
(iJ]] Principles of Testing written communication skills.
The fundamental principles of testing are - • He/she should be a good listener.
1. The goal of testing is to find the defects before · • He should be able to convince the need for testing
·customer finds them out. the module to developer as well as customer._·.
2. Always understand the reason behind the testing.
(1-1)
 Software Testing_ 1-2 Basics of Software Testing and TestinE_ Methods
r-
3) Presentation Skllls : Error: It is an issue identified internally or during unit
• A good tester must also possess good presentation testing. Normally error occurs when human actions
skills to provide the exa·ct status of the test project produce undesirable results
and application under test. Defect: It is an issue identified by customer.
• The tester is supposed to present the test results to Bug : Bug is an initiation of error or problem because
developer team, customer and management team of which the fault may occur in the system.
and convince them for further improvements. Fault : It is a condition that causes the software to fail

4) Technical Sklll : to perform its required function.

Failure : It is the inability of a system or component to
• A good tester must have the knowledge of database,
perform required function according to ·its
programming, and commands.
specification.
• He/She should have knowledge and hands on
experience of test management tools, and Board Question
automation tools. 1. Explain the terms mistake, error, defect, bug, fault and
• He/She should be enthusiastic to learn: new failure in relation with software testing. - - -
MSBTE : Summer- I 6, Marks 6 , Winter-I7 . Marks 4
techniques and skills required for testing.

5) Management and Organization Skllls :

.[III Test Case
• Testing at times could be a demanding job I1.3~ 11 What is Test Case ?
especially during the release of code. A Test case is a set of conditions and expected results
• A software tester must efficiently manage under which a tester will determine whether a system
. workload, have high productivity, exhibit optimal under test satisfies requirements or works correctly.
time management, and organization skills. Test cases are normally designed for particular test
scenario in order to verify compHance against a
Board Questions
specific requirement.
1. List all objectives of testing.
MSBTE: Winter-15 , Marks 4 Various parameters using which the test case is
2. What is software testing ? State objectives of software prepared are -
testing. 1) Test case ID: It is an ID for the test case
MSBTE : Summer- I 6. I 8, Winter- I 7, 18, Marks 4
2) Test case scenario: The description of the scenario
3. State any four testing principles.
MSBTE: Summer-16, Marks 4
for which the test case is to be prepared.

4. Define software · testiM. List all skills 3) Test case description: The purpose of the test case
tester. is described under this section
5. List and describe any four skills of software tester. 4) Prerequisites : Any . precondition that must be
MSBTE: Winter-16. 18, Marks 4
fulfilled before execution of the test case must be
6. Define software testing and role of testing. mentioned here .
MSBTE : Summer- I 9, Marks 4
5) Test procedure : The step by step procedure that
[j]] Failure, Error, Fault, Defect, Bug Tennlnology demonstrates the testing procedure.
Following are some important terminologies used in 6) Test data : The data to be used while conducting
relation with software testing - the test.
Mistake : It is an issue or a problem identified during
7) Expected result : The expected result of the test
peer reviewing. It is of low cost · and can be fixed
quickly.

T~ Technical Publications -An up throst for knowledge

 1-3 Basics o So re Testin and Testin Methods
8) Actual result : Actual result of the test which is to • The exit criteria can identify the intermediate
be filled after executing the test deliverables.
9) Status : The status can be PASS or FAIL or being a • The following exit criteria should be considered for
test passed or failed, completion of a testing phase:
I1.3.2 IFeatures of Test Cases · o Ensuring all critical Test Cases are passed
1. Test cases must be simple and transparent. o Achieving complete Functional Coverage
2. Do not assume functionalities and some extra o Identifying and fixing all the high-priority
features for test case design. defects
to
. 3. A void repetition of test cases. • The output achieved through exit criteria are -
ts
4. Test cases must be identifiable. o Test summary report .
5. C_reate test case by keeping end user in mind. o Test logs

~ When to Start and Stop Testing of Software? Board Questions

1. What is a 'test case' ? State its s~cification
Entry criteria parameter. HMM♦ jjjj,hiiiill3MHI
• Software testing should start early in the Software 2. What is entry and exit criteria of software testing ?
MSBTE : Sumnwr-15. 16. !\fork., 4
Development Life Cycle. This helps to capture and
3. Explain when to start and stop testing.
eliminate defects in the early stages of SDLC i.e
< I MSBTE : Summer- I 9. Mark., 4
requirement gathering and design phases.
• An early start to testing helps to reduce the number fil] Verification and Validation, Quality Assurance,
Quality Control ·
of defects and ultimately the rework cost in the end.
• Definition : Entry criteria for testing can be· defined I1.4.1 !Verification and Validation r,J Model)
as "Specific conditions or on-going activities that • The software verification and validation is a
must be present before a process can begin." The checking and analysis process of developing
Software Testing Life Cycle (STLC) specifies the software.
entry criteria required during each testing phase. • In V and V requirements review, design review,
code inspection and testing are the various activities
• It also defines the time interval or the expected
that are conducted.
amount of lead-time to make the entry criteria item
• Boehm has described the verification and validation
available to the process.

l
as:
• Following are t!te inputs necessary for the entry
o Verification means asking "Are we building the
criteria.:_
right product ?"
o The requirements documents o Validation means asking "Are we building the
I o Complete understanding of the application flow product right ?"
o Test-plan !1.4.1.1 !Verification
Exit criteria • Verification is a technique of evaluating whether
• Definition : It can be defineq.. as "The specific software product fulfills the requirements or
conditio~ or on-going activities that should be conditions imposed on them by standards and
fulfilled before completing the software testing life processes.
cycle." • It is a static technique as there is no execution of
code · or product. During verification, the work

Technical Publications -An up thrust for knowledge

 1-4
Basics of_ Sojiware Testing and Testing Methods
i
Software Testing
product is carefully read and analyzed for detecting Disadvantages
defects with respect to standard processes. 1. .The validation is the most time consuming process
' - because its aim is to execute · the
. • Verification cri_teria : The verification is to ensure
whether the program running on a particular application/software or code and hence as a result
pl~tform satisfies the requirements and more test cases are needed to execute.
development processes. 2. The cost of the product may get increased due to
validation as validation process is during the
Advantages execution of the work product. The bugs fixed in
1. Each work product is analyzed for finding out the later stage of software development can result in
Ii defects_. Hence it reduces the cost of finding and
fixing the defects from the system as a whole.
increase of cost.
v

/' ,~ n e e between Verification and Valldatlon

2. Verification confirms that during the development
of work product the software development
processes are correctly followed.
3. People can be trained for · verification .proces·s as
;,\t~d~cit~§ . ~f~s .;to
there is no execution of code or product.
4; The defects can be located £asters as the individual
~ ofacfi~!t!J ~
correctly implements \isorrware ma~ i

~,,11➔
work product is getting analyzed.
~e•p-fundioa
Disadvantages

1'1
1. The verification process simply confirms that the
development process is completely followed or not,
. it does not show whether the developed software
product is correct or not.
2. The defects during the execution of work product
After valid ·and
a
complete specification
the verification starts.
Verification is for
prevention of errors.
~
~~iI~iU!iflJlti
,~ia,
.I

~· cannot be detected by verification. Verification is

1'
!1.4.1.2 !Validation conducted
reviews,
using

walkthroughs, · ;,,,
• Validation is a technique, to evaluate whether the , an'a
inspections
final built software .product fulfils the customer audits..

{j
'I
requirements.
• It is also called as dynamic testing as the application
Verificaµofr;t t-~ fL;~
termed . .is \vhi~ "fi'ox ,¥:ti,
If
I

-: , . · , ~:i· : _ 4'·',,,:;, .
I' is executed during validation in order to find out testing or/static,,testing . :&t.l
;; - . . ,. --~-~- .-~ -·:- ,.,;t ,<'; : ~:Mt
/1 the defects. i as w,ork product'goes . f"
,,
I
thr~ghreviews,,_, .•
• •• ,,,, •. ·J,:
• Normally the validation must be done by
independent users and functional experts.
Verification: >finds
abbut 50 t<J'60%'of th~
.. defectsi '. C/
Advantages
1. Validation is the only to get ensured about the VJrifi&tion ·ii'.' baseci'
functioning of the software product. Validation
help to test the system during the execution.
2. The defects that can not be 1dentified during
verification can be identified during validation.

T~ TM
Technical Publications -An up thrust for knowledge
Ocfs

"
~Ss
lle
.ilt

to
le

n
n ----- - ·
\ JQ!) VModel
• V model means the Verification and Validation
model.
• This model is V-shaped hence is the name. It is just
like waterfall model in which each phase must be
completed before the next phase begins.
• This model contains various stages of software
development along with various types of testing
that can be con~ucted at each stage of development.
The test plans serve as a link between the Fig. 1.4.1 The V-model

. development stages and various tests. Requirements analysis

When wlll be V model applicable ? As per the waterfall model of software development
Following are the situations ·in which the V .model is process model the V model begins with requirements
applicable gathering and analysis. The Software Requirements
Specification(SRS) is created during this phase. The
1. The requirements , are well defined and are not
acceptance and system testing is conducted in which
ambiguous.
the focus is on meeting of functionalities specified in
2. The acceptance criteria is clear.
the requirements gathering.
3. Project is short to medium size.
4. Technology and .tools are not changing. High level design

During this phase the system architecture is designed.

Model
It provides the overview of platform used, system,
• As shown by this model at each stage specific
product, and service or process. An integration test
testing can be carried out. Refer Fig. 1.4.1.
plan is prepared and testing is conducted to test the
pieces of the software systems to ensure about the
ability to work together.

Low level design

During this phase actual software components ·are .

designed. The relationship with other components are
defined at this level. Hence component testing is
conducted for this phase.

T. Technical Publications -An up thrust for knowledge

 Testing and Testing_ Methods
Software Test· · 13a9ics of sof!!!!!_re ~
,. 1-6 ----
Implementation [1.4.3] Quality As9 uranc• be defined as "the
All codin t k · quality
can · d functional and
. g a es place at this phase. The unit testing is • Software licitly state
cames out to test e onformance to exp .citly documented
ti very · source code module. The c · uirements, exp11 ct .sti~
crea on and review of various test cases are quried performance req d ds and implicit chara en
out in this phase. development stan ar £ • onally developed
. ed of all pro1ess1
Advantages that are expect
software"• · . It is planned and
1. 'The V model is simple and easy to manage as each al'ty
1 assurance. .
• Definition of qu . . . ecessary to provide
phase has well defined objectives and goals. · . of activities n
systematic pattern . the quality of a
2. Development and progress of this model is very . de ee of confidence m
systematic. a high gr lity assessment of the
od ct It provides qua
3. It works well for small to medium sized projects. pr u . . . .es and determines the validity
quality control activiti . . al. ty
4. Testing starts from beginning of the development of the data or procedures for deternurung qu I ~
- stage and .errors can be identified and corrected . . ance consists of set of reporting
• The quahty assur ,
from the beginning of the software development. and auditing functions.
Disadvantages • These functions are useful . for assessing and
controlling the effectiveness and completeness of .
1. This model is not suitable for large and complex
projects. quality control activities.
• The goal of quality assurance is to ensure the
2. For the projects in which the requirements are not
consistent, this model is not suitable. . management of data which is important for product
quality.
3. The working software can not be produced during
the intermediate stage. It is available only at the !1.4.41 Quality Control
end of the development cycle. • Definition : Quality control is a process in · which
4. There is no provision for risk analysis. Hence there activities are conducted in order to maintain the
is always an uncertainty about the risks. quality of product. These activities are series of
inspections, reviews and tests used throughout the
Board Questions
software process. These activities ensure whether
1. Describe V-model with labelled diagram. State its any
two advantages and disadvantages. Also write where it each work product is satisfying the requirements
is applicable. imposed on it.
MSBTE : Wint er 15, 17. 18. Marks 6 • While applying the quality control there should be a
2; Differentiate between verification and validation. feedback loop to the process which generates the
MSBTE: Summer-16. Marks 4 , Summer 18. Marks 6
work product. With the help of such feedback we
3. Explain V model with diagram.
can tune the process if it does not satisfy the
l\fSBTE : Winter-16. Marks 6
requirements. The feedback loop helps in
4. Describe V-model with labelled diagram.
MSBTE : Summer-17. M<1rks 6 minimizing the defects in_the software product.
5. Explain verification and validation with neat diagram. • The quality control activities can be fully automated
MSBTE: Summer-19. Marks 8 or it can be completely manual or it can be a
combination of automated tools and manual
procedures.

T~ Technical Publications -An up thrust for knowledge

Software Testing 1-7
Basics 0! Software Testing and Testing Methods
( 1.4.5J Difference between Quality Assurance and
Quality Control
documents are tested manually in order to find
errors, it is called static.
• This kind of testing is also called as verification
testing.
Static testing techniques :
• Informal reviews : .
o In this technique, the team of reviewers just
checks the documents and give comments.
o The purpose is to maintain the quality from the
initial stage. It is non-documented in nature
• Formal reviews :
o It is well structured and documented and follow
six main steps: Planning, kick off, preparation,
review meeting, rework follow-up
o Technical reviews : The team of technical
• experts will review the software for technical
specifications. The purpose _is to pin out the
difference between the required specification
and product designed and then correct the flaws.
It focuses on technical documents such as test
strategy, test plan, and requirement specification ·
documents.
• Walk-through : The author explains about the
software to the team and teammates can raise
questions if they have any. It is headed by the
author and review comments are noted down.
• Inspection process : The meeting is headed by a
trained moderator·. A formal revi·ew is done, a
record is maintained for all the errors and the
· authors are informed to make rectification on the
Board Questions given feedbacks.
1. Give difference between quality assurance and quality • Static code review : Code is reviewed without
control. (Any Four) execution, it is checked for syntax, coding standard,
MSBTE : Summer- I 5. 18. Winter- I 6, 18. Marks 4 and code optimization. It is also referred as white
2. Describe quality assurance and quality control. box testing.
MSBTE: Summer-19. Marks 4
Advantages :
[!) Methods of Testing 1) It is fast and easy technique used to fix errors
\ [ill static Testing 2) It helps in identifying flaws in code
• Definition : Static testing is a testing technique in 3) With the help of automated tools it becomes very
which software is tested without executing the code. easy and convenient to scan ·and review the
As the code, requirement documents and design software.
Technical Publications - An up thrust for knowledge
 Software Testing . O;,IS01'
Bastes ,ltware Testing and Testing Methods
1-8

4) With static testing it is possible to find errors at tatlc and le testing

'/-.:., ... ::..:
.e arly stage of development life cycle.

Disadvantages :

1) It takes lot of time to conduct the tes~g procedure

if done manually. ·
2) Automated tools work for restricted set of
programming languages.
3) The automated tools simply scan the code and can
not test the code deeply.

I1.5.2 IDynamic Testing

Definition : Dynamic testing is a process by which
code is executed to check how software will perform in
a runtime environment. As this type of testing is
conducted during code execution it is called dynamic.
It is also called as validation testing.
/

Dynamic testing techniques

Unit testing ; As the name suggests individual units or
modules are tested. Th~ source code is tested by the Board Questions

developers. 1. What is static testing ? State advantages and

disadvantages of static testing two each.
Integration testing : Individual modules are clubbed
MSBTE : Winter- t 5. Marks 4
and tested by the developers. It is performed in order
2. Explain the static testing and dynamic testing.
. to ensure that modules are working in a right manner MSB IE : Summer- I 6. Marks 4
and will continue to perform flawlessly even after 3. Describe inspection under static testing.
integration MSBTE : Summer- 15. 17. Marks 4

System testing: It is performed on a complete system 4. Describe structural walk through under static testing.
MSBTE : Winter- I 6 . Marks 4
to ensure that the application is designed according to
5. Describe technical review under static testing.
the requirement specification document.
MSBTE : Summer-] 8. !\-forks 4

Advantages
[!]] The Box Approach
1) It identifies weak areas in a runtime.-
2) It helps in performing detail analysis of code.
[ill] White Box Testing
3) It can be applied with any application. • In white box testing the procedural details are
closely examined.
Disadvantages
• In this testing the internals of software are tested to
1) It is not easy to find trained software tester for
make sure that they operate according to
performing dynamic_ testing specifications and designs.
· 2) It becomes costly to' fix errors in dynamic testing.
• ~us major focus of white box testing is on internal
structures, logic paths, control flows, data flows,
internal data structures, conditions, loops, etc.

T" Technical Publications - An up thrust _for knowledge

 Software Testing 1-9 Basics of Software Testing and Testing Met~
• The white box testing is also called as clear box, 4. Maintaining the white box testing is very difficult
glass box or open box testing. because it Illay use specialized tools like code
analyzer and debugging tools are required.
Advantages :
5. The missing functionality can not be identified.
1. Each procedure can be tested_ thoroughly. The
internal structures, data flows, logical . paths, Classlflcatlon of white box testing

conditions and loops can be tested in detail. Following _Fig. 1.6.1 represents the classification of
white box testing-
2. It helps in optimizing the code.
3. White box testing can be easily automated. Static testing :
• Static testing is carried out only on source code. It
4. Due to knowledge of internal coding struchtre it is
does not require any executable rode.
easy to find out which type of input data can help
• Following are the tasks performed · during static
in testing the application ; fficiehtJy.
testing-
Disadvantages : 1) To check whether code works according to
1. ·The knowledge of internal structure and coding is functional requirements or not.
desired for the tested. Thus the skilled tester is 2) To check whether the code is written according to
required for white box testing. the design of system.

2. This type of testing is costly. 3) To check if any functionality of the code is missed

3. Sometimes it is difficult to test each and every path

of the software and hence many paths may go
untested.
out or not,
4) To check whether the code handles errors properly
or not.
II
'

II
I
· Desk ct.eking

Code
walldhrough·
. . Code Statement coverage Cyclomatic
inspection complexity
Path coverage

Condition coverage

Fuhction coverage

Flg.1.6.1

Technical Publlcstlons - An up thrust for knowledge

 I
.,. 1'
l l LC.U C
Testing and Testing Methods '
Software Testing Basics of Soft!!!!!.re · -
- a ._ _ _ _.._c.-_ _ _ _ _ _ _ _ _ _ _ _ _ _.!.,l,:_;-1~0~----- ch for programming
. we can sear
• . Various methods of static testing are - 3. Using inspections nuning style or use of
1) Desk.checking defects, poor progra
2) Code wal~through inappropriate algorithm• be detected. by
oA of errors can ·
3) ·Code review 4. More than 90 ° . If the defects are
· al inspections•
4) Code inspection conducting fonn ---""ers can avoid
. . pections, prograuuu
I1.6. 1.1 llnspectlons found dunng ins 1 t phases of software
- · the some mistakes in the a er
The main goal of inspection is to find out the defects. ·development.
Inspection is a kind of revie~ by the group of peers by Disadvantages : pt that the
following clearly defined processes. The inspection is 1 Software engineers are reluctant to acce .
very formal process of verification. . effective for defect
inspections can be more
Following are some guidelines given for the inspection detection and testing. .
process- . tha . ctions may reqwre
2. Project managers feel t mspe d . the
1. The inspection must be conducted by the technical additional cost if they are conducted unng .
people for technical people. software design and development.
2. It is a structured process in which every participant .
3. Since inspections take time t o arrange and it slows
have definite role. down the development process, conducting
3. The focus of code inspection is to identify th~ inspection is avoided.
0
problems and not to solve them.
Roles and responslbllltles p
4. The review data is recorded and monitored for The program inspection is a formal process. It is 1
further improvemen_!. conducted with the help of four people and roles of
The inspection is b~sically carried out by the team of them are: Author, reader, tester and moderator.
reviewers. The author of the team is moderator.
Author or owner is a person who has created the
The moderator has overall responsibility to ensure that program or design. This person is responsible for ,
the review is done in proper manner and all the steps fixing the defects during the inspection process.
of review process are foUowed . .
Moderator is the leader' of the inspection process who
The inspection process can be carried out in· different plans and co-ordinates the inspection. He reports the
phases such as planning, preparation and overview, process results to the chief moderator.
group review meeting, rework and followup.
Reader is the person who reads the _code aloud to the
Advantages : inspection team.
1. Single inspection can discover multiple errors; on Tester or inspector is the person who inspects the code
the other hand one error may hide another error. from testing perspective. He finds the errors,. omissions
From output of the system we cannot predict and inconsistencies in the software.
whether the wrong output is due to existing error Scribe_records the results of inspection meeting.
·or because of some new error. Chief moderator is responsible for inspection process
2. Incomplete yersion of the system can be inspected improvements, preparing or updating checklists.
but it is. difficult to test such version.
Inspection _process ·

The inspection process can be illustrated by following

Fig. 1.6.2.
M
Technical Publications -An up thrust for knowledge
 ~ r e Testing 1-11 Basics of Software Testing and Testing Methods

Inspection checks
During the inspection a checklist is prepared for
coinrnon programming errors. It can be prepared by
I by
the person(s) having adequate experience in
· are
application domain. This checklist is very useful to the
void
programmer. The checklist varies for different
Nare
programming languages because differ~t
programming languages have different prograrnmmg
constructs. For example :
t the
Fig. 1.6.2 Inspection process Checklist for C Program
lefect
0 Is the loop execute• for correct number of
Planning : The inspection process is planned by the
times?
ruire moderator. Various tasks that can be conducted during
planning are : o Do the parameter and argument . types in the
the
o Selecting inspection team
function call and definitions match ?

o Organising meeting room 0 Are the global variables definitions consistent

ows
throughout the program ?
ting o Ensuring that the required documents and
resources for the inspection are available. !1.6.1.2 !Walkthrough
Overview : The program_or the code to be inspected is
• Walkthrough is more formal than peer review.
presented to the inspection team during the overview.
Many times it is called as semi-formal review.
is The author of the program/code explains its purpose.
: of • During walkthrough author lead the review process
Individual preparation: Each inspection team.member
and other team members ask the questions and
then studies the code and tries to find the possible
identify the all possible errors.
the defects.
• The main purpose of walkthrough is to enable to
for Inspection meeting : During this meeting, the focus
understand the contents of the documents under
should be on pinpointing the defects from each part of
review and to find the defects.
·h o _the program, non-compliance .to the standards and
• Walkthrough helps the teams to develop the
:he · poor-quality programming. But the inspection team
communication during review.
m~t not suggest how to correct the defects.
Rework : After the inspection meeting, author should Advantages :
understand the mistakes and should correct the 1. Jt is useful for the communication about the issues
de ' identified defects. He should try to remove as many under review.

ns anomalies as possible. 2. Team members can understand the things that are
1 Follow-up : During this stage the reworked program is expected during the review.
' presented to the moderator. The moderator then has to 3. The problems are recorded and suggestions can be
1 decide whether the re-inspection is needed or not. If all . obtained from all the team members. Hence all the
!SS the identified defects are removed 'successfully and the . team members can participate in improvement of
1
inspection team gets sa_tisfied with the work product, the work product.
moderator approves the program ~or the release.
,g

Technica/ Publications - An up thrust for knowledge

 Software TestinE_
~-~~-----~9L~==- B~ of Sofh!!!!.'e Testing and Testing Methods
'l'

Disadvantages :
1 -12
product is ready for r,'7ieW- The reviewer re,,;;- .:
the work product as per his or her convenience an' '
1. It is difficult to handle the team of large size during sends the review rep<>rl to the author. There is Q
walkthrough.
direct interaction of author and reviewer. no
2. It can be time consuming activity.
Difference between walkthrough and Inspection Advantages :
1. Review is a very useful tool in defect finding.
2. Review can be conducted at convenient timings of
both the author and reviewer.

Disadvantages : . If review may not

1 · volved m se
1. Sometimes peop em . ality due to time ··
·
conduct the review m re
constraints. .
. er rson doing the reVIew t
2 In peer review, if the oth pe 1d
· . . the lack of know e ge
is not expert or possessing .
about the system under review then his/her
suggestions may not be valid.
I
j 1.6.1.4 Unit/Code Functional Testing

• This is structural testing method in which some '

quick checks are made before the code coverage
testing or code complexity testing.
!
!1.6.1.3 Technical Reviews • Following are three ~pproaches used during
unit/code functional testing -
• This . is an informal way of verification and
o Initially the developer performs some typical
valid.ation. There are two types of review - self
tests or obvious tests with known input sets
review and peer review.
and corresponding expected outputs. These tests
• The self review is normally done by the tester
himself who has written the test cases. He can verify are repeated for multiple inputs and any obvious
whether all the requirements are covered or not by mistakes can be removed from the code. This
looking into software requirements specification. increases the level of confidence. These quick
Whereas peer review is done by another tester who tests are generally performed before the formal
hasn't written those test cases but is familiar with reviews of static testing.
the system under test. This type of review is also
known as maker and checker review. o For compkx logic testing, developer prepares

• There are two kinds of peer reviews - 1. Online peer the debug version of the code. The debug
review and offline peer review. version is a kind of version of your source code
• The online peer review in which the author and the in which the print statements are inserted at
reviewer sit together and review the work product intermediate pl~ces in the code. This can be done
jointly. In this meeting any explanation can be to test if the program is executing through right
immediately provided by the author to the loops and iterations for right number of times.
reviewer.
. After fixing the errors, these print statements can
• The offline peer review is a kind of review in which be removed.
the author informs the reviewer that the work
T. Technicai Publications -An up tht'U$1 for lcnowl«Jge
, I
 -----,r. :.;:!Jll.!!E_;a,;;;..;;a- -- ~

So. l -13 Basics of Software Testing and Testing Methods

·o In this approach the code is run under a standard d_ebugger or Integrated development
Environment(IDE). These standard tools debug each and every instruction of your program. The
developer can watch the values of variables or other functional parameters at each step in the
iteration. ·
!1.6.1.5 !Code Coverage Testing
• This is a testing process in which the test cases are designed and executed for different parts of code .
. Thus some amount of code is covered by testing. The amount of code covered by testing is found out
by a technique called instrumentation of code. The instrumentation of code can be performed used ·
some standard tools.
• Code coverage testing can be performed using following techniques -
o Statement coverage .
o Path coverage
o Condition <;:overage
o Function coverage
Let us discuss these techniques -

1) Statement coverage :
o 1?ere are various types of programming statements such as sequential control flow, if-else
statements, switch-case statements, loops such as for, while,do-while
o Sequential control flow statements: For these kind of statements, the test can be designed to run ·
through from top to bottom. A test case starts from top and runs covering full section of the code
upto bottom.
o If-else statement : The test cases are written to cover all parts of code. That means the set of test
cases.are executed for if part and another set of test cases are executed for else part.
0 Switch-case statement: There should be multiple test cases executed covering each case statement
in the switch-case control structure.
0 Loop construct(for, while, do while) : This is the most complex part for statement coverage. Many
defects occur in the program if the loops do not execute correctly. Hence set of t~st cases must be
executed for -
1) Exercising the loop at least once and maximum number of times to ensure all the normal
flow of execution of loop.

2) Skipping the loop completely, so that termination condition of the loop can be tested before
starting the loop.
3) Testing the boundary of the loop. For example if the statement is .
wbile(i < =n)
{

}
Then the test case must be written when i has exact value of n or n-1
;...

T. Technical Publlcations -An up thrust for knowtedge

 BasicS of sopware Testing and Testin~
Software Testing
1 -14

Calculatlon of 1tatement coverage

St te . . lly executed in a set of tests.
a ment coverage is an indication of the percentage of statements actua
The statement coverage can be calculated using the formula -
Total statements exercised >< 100
Statement coverage = Total number of executable statements

2) Path coverage :
. . . f distinct paths and test
0 This is a kind of testing in which the program is divided into number__0:........-- - - - - - : - - - - .
cases can be written for each path of the program.
o F~r example - Consider following code for simple subtraction
1. Input x and y
2. if x > y then
3. z=x-y
4. else z = y - x
5. endif
6. outputz
Refer.Fig. 1.6.3 for flow graph.
Here we need to test two paths
Path 1: 1,2,3,5,6
_Path 2: 1,2,4,5,6 Flg.1.6.3

o Thus regardless of the number of statements in each of these paths, if we·can execute these paths,
then we would have covered most of the typical scenarios.
o Hence path coverage c_an be calculated using formula-
Total paths exercised
Path coverage = Total number of paths in program x lOO

3) Condition coverage
o Condition coverage exercises all kinds of situations that the path
coverage may not perforin
o For example - Consider following code for simple subtraction
1. Input x and y
2. if x > y then
3. z=x-y
4. else z = y - x
_5. endif
6. outputz

Flg.1.6.4

Technicsl Publications - ~n up thrust for knowledge

 Software TestinK_ 1-15 Basics of Software Testing_ and T_esting Methods

Here by path coverage criteria two paths can be tested (1-2-3-5-6 and 1-2-4-5-6) for the conditions x > y
and x < y. But what if the value of x = y, as there is no such path for this condition. Hence it is preferred to
perform testing based on various conditions.

o Hence condition coverage can be calculated using formula -

Total decisions exercised

Condition coverage = Total number of deas1ons
. . . program x 100
m
4) Function coverage :

0
In this technique the tests cases are executed for various functions present in the program.
0 While providing ~ction coverage, test cases can be written so as to exercise each of ·the different
functions in the code.

Advantages of functlonal coverage :

• Functions are easier to identify in the program and hence test case be written easily for
them.

• . The complete code coverage for the function is possible during testing.

• Functions are logical mapping of requirements hence direct testing of direct requirements
of is possible.

• It is easy to prioritize functions for testing.

• It provides natural transition to black box testing.

o Hence function coverage can be calculated using formula -

Total functions exercised
Function
. covera ge -- Total number of fun ctions
. . program
m >< 100

I1.6.1.6 ICode Complexity Testing

• The code complexity is measured by cydomatic complexity measure.

• The cyclomatic complexity can be defined as a measure used to determine the complexity of a piece
of code or functionality.

• This metric measures independent paths through program source code. Independent path is defined
as a path that has at least one edge which has not been traversed before in any other paths.

• The cyclomatic complexity can be calculated for function, module, class, methods present in the
program.

• For computing the cyclomatic ~omplexity the flow graph need to be designed.

• Flow Graph notation for a program is defines. several nodes connected through the edges. Below are
Flow diagrams for statements like if-else, While, until and normal sequence of flow.

T. Techniclll Publications - An up thrust for knowledge

 1 · 16
-
BasKS of ~re T,sting and T,sUng Md""' '

Sequence WtdJe

-------
lf~lhen-else Untl

~
Flg.1.6.5

Ex. 1.6.1 : Draw the flow graph for finding maximum

• Cydomatic complexity can be computed in three of three numbers and derivt! the testcases using
ways- cyclomatic complexity.
o Cydomatic complexity = E + V - 2 Sol. : The flow graph for given program is - (From
o Where E is number of edges, V is number of Fig. 1.6.6)
nodes.
o Cyclomatic complexity = P + 1
Where P is number of predicate nodes. The
predicate node is a node that contains some
condition.
. • This method .is also called as basis path testing
· method as number of independent paths are
obtained in this method.
• How to perform Code Complexity_testing ?
Following steps are followed for code complexity
- _Predicate
node

testing .
· Step 1 - Construction of graph with nodes and edges
from the code Fig. 1.6.6 Flow graph for finding maximum of t~ree numberl

Step 2 - Identification of independent paths • Cydomatic complexity =E - N + 2 = 7 - 6 + 2 =3

Step 3 - Cyclomatic complexity calculation Cydomatic complexity =P + 1 =2 + 1 = 3
. Step 4 - Design of test cases. The number of test cases Cyclomatic complexity = Regions encountered = 3
is equal to cycomatic complexity mea~ure. He~ce cyclomatic compl~ty of given program is 3.
. Once the basic set is formed, test cases should be
.written to execute all the paths.

T. Technical Publications • An up thrust for knowledge

 -----,.---~l~'LWAE . b:tll ! 'ta l l l F ! ! ~ ~ t! :!.~ 9111f;ut;Wl.:ait£ :~-:.-:;:as;i." W l ~::u~ . . - - -

~ ·,,
"-.... t

Software Testing

Board Questions
1 -17
[Ir] Black Box Testing
Basics of Software Testing and Testing Methods
I. . .

Explain what the static white box and black box

1. • The black box testing is used to demonstrate that
testing is required. I\JSBTI . S1111111wr- I<, _ M,nk., t1 the software functions are operational.
2 Describe inspection process performed under white box • As the name suggests in black box testing it is tested
testing.
whether the input is accepted properly and output
3. Give any two advantages and any two limitations of
function/test matrix. is correctly produced. I
4.
MSBT[ : Summer- I 8. J\fark-. t1
State purpose of code coverage. How it is used in
analysing coding of software ?
Advantages :
1. The black box testing focuses on fundamental
I
MSBTE : Winter- 1 S . Mark., 4 aspect of system without being concerned for
5. What is white box testing ? Classify static white box internal logical structure of the software.
testing. State any one situation where white box 2. The advantage of conducting black box testing is to
testing used. MSBTE : Winter-IS . Mark-, 4 uncover following types of errors.
6. Draw classification of white box testing. Explain any
i. Incorrect or missing functions
one type of white box testing in detail.
MSIHT : Summer- I 5 . M.irk-, tJ ii. Interface errors
7. With the suitable example, explain how 'Basis Path iii. Errors in external data structures
Testing' is used to derive the code
testing. HMHf 5
comzesl,rw:,
-iiihiiU ,.. .
the iv. Performance errors
v. Initialization or termination errors
8. What is code coverage testing ? Explain the following
types of coverage : Disadvantages :
i) Path coverage ii) Condition coverage. 1. All the independent paths within a module cannot
MSBTE : Winter- I 6. Marks H
be tested.
9. What is the use of code complexity testing ? Also
compute code complexitt1 with the he/µ of suitable 2. Logical decisions along with their true and false
example. sides cannot lbe tested.
10. What is white box testing ? Explain any one technique 3. All the loops and the boundaries of these loops
of static white box testing. cannot be exercised with black box testing.
I\ISBTE : WintN-17. Mark, '1
4. Internal data structure cannot be validated.
11. With the suitable example, e.\plain how 'Basi/ Path
Testing' is used to derive the code coml¢r• for the Types of black box testing
testing. HMihM{#d -3 N!j Some commonly used black box testing techniques
12. Explain any one type of structural white box testin~ in are-
detail.
1. Requirement based testing
13. Give any four differences · between walkthrou{. and
inspection. HM;iihi&Nii:MCI 2. Boundary value analysis
14. Draw classification of white box testing. Explain the 3. Equivalence partitioning
purpose of code coverage testing
I\ISBTE : Winter- I 8. Mark., 4 mI) Requirement Based Testing
15. Explain inspection and walkthrough. • This is a testing approach in which test cases are
;\lSBTE : Summer- I 9. J\ldrk., ti
designed using complete and consisten( set of
16. Explain code functi.onal testing and code coverage requirements.
testing with example.
:\lSBTF . Sumnu'r-19 . M.uk., S • This is testing technique which revolves around the
requirements.

T. Technical Publications • An up thnltlt for~

-~ ~
]Mi""l fl! .
. . .-, I
.
• •- • •
Ie
Software Testin.
Basics of Soft_ware Testing and Testing Methods
~
,0
' I/
1 -18
Parameters to be included In requirement Traceablllt~
• The strategy of requirement based ·testing is to
integrate testing · throughout the life cycle of the matrix G

software development process, to assure quality of (1) Requirement ID \ I/.~

the requirement specification. (2) Requirement type and description \ \·/
How to perform requirements based testing ? (3) Test cases 1 I
(4) Test case status

'
Step 1 : Define test Completion criteria :
Testing should be defined in quantifiable terms. The
~
goal is considered to be achieved only when test ! ~;

coverage is 100 %.
i~
Step 2 : Design test cases :
Test cases must be in accordance with requirements
specification.

Step 3 : Build test cases :

i
Join the logical parts together to form/build test cases .
I·
Step 4 : Execute test cases :
Execute the test cases to evaluate the results.

Step 5 : Verify test results :

Check whether actual results deviate from the
expected. ones.
Advantages ·of requirements traceability matrix
Step 6: Verify test coverage :
· Check for fµnctional test coverage. 1. It confirms 100 % test coverage. \
2. It highlights any requirements missing or
Step 7: Manage test library :
document inconsistencies.
Test manager i~ responsible for monitoring the test
3. It shows the overall defects or execution status with
case executions, that is, the tests passed or failed, or to
ascertain whether all tests have been successfully
a focus on busines_s require~ent~. .
4. It helps in analyzing or estimating the rmpact on ,
l
performed.
the QA team's work with respect to revisiting or re-
Concept of requirement traceablllty matrix working on the test cases.
• Requirements are tracked by Requirements
Traceability Matrix (RTM).
l1.7.2 IBoundary Value Analysis
• Boundary value analysis is done to check boundary
• RTM traces all the requirements from design,
conditions.
development, and testing.
• A boundary value analysis is a testing technique in
• The traceability matrix is typically a worksheet that
contains the requirements with its all possible test
which the _elements at the edge of the domain are
selected and tested.
scenarios and cases and their current state, i.e. if
they have been passed or failed. This would help • Using boundary value analysis, instead of focusing
the testing team to understand the level of testing on input conditions only, the test cases from output
· domain are also derived.

,·
activities done for the specific product.
.Technical PublcstiollS -An up thrust for knowledge
 Ce11,,., _
~ Software Testing 1 - 19 Basics of Software Testing and Testing Methods
I
'bti"1
• · Boundary .value analysis is a test case design I
11 .7.3 Equivalence Partitioning
technique that complements equivalence • It is a black box technique that divides the input
partitioning technique. domain into classes of data. From this data test cases
• Guidelines for boundary value analysis technique
are-
1. If the input condition specified the range
can be derived.
• An ideal test case uncovers a class of errors that
might require many arbitrary test cases to be
II
bounded by values x and y, then test cases executed before a general error.is observed.
should be designed with values x and y. Also • In equivalence partitioning the equivalence classes
test cases should be with the values above and are evaluated for given input condition
, below x and y. Equivalence class represents 1:\ set of valid ot invalid
2. If input condition specifies the number of values states for input conditions.
then the test cases should be designed with • Equivalence class guidelines can be as given below:
minimum and maximum values as well as with Input set
the values that are just above and below the
maximum: and minimum should be tested.
}: 3. If the output condition specified the range
ff
w . bounded by values x and y, then test cases
i.:
should .be designed with values x and y., Also·
test cases should be with the values above and
below x and y.
4. If output condition specifies the number of
values then the test cases should be designed
with minimum and maximum values as well as
with the values that are just above and below the
maximum and minimum should be tested.
5. If the internal program data structures specify
such boundatjes then the test c;ases must be
designed such that the values at the boundaries
of data structure can be tested.
Output generated
For example :
Flg.1.7.1
Integer D with input condition _[-2, 10],
· • If input condition specifies a range, one valid and
Test values : - 2, 10, 11, - 1, 0
two invalid equivalence classes are defined.
, If input condition specifies a number values, test cases
• . If an input condition requires a specific value, one
should developed to exercise the minimum and
valid and two invalid equivalence classes are ·
maximum numbers. Values just aqove and below this defined.
min and max should be tested.
• If an input condition specifies a member of a set,
Enumerate data E wifu input condition : {2, 7, 100, 102}
one valid and one invalid equivalence class is
Test values : 2, 102, - 1, 200, 7 defined.
• If an input condition is Bool~ one valid and one
invalid equivalence class is defined.
T. Technical Publications
111
-An up thrust for kno'lllledge
 SoffWare Testing · - and TestinK Met

For example :

Area code : Input condition, Boolean - The

area
code may or may not be present.
Input condition, range - Value defined between
200 and 700.

Password : Input condition, Boolean - A password

may or may not be present. · .
Input condition, value - Seven character string.
Command : Input condition, set - Containing
commands noted before.

I1.7.41 Difference befyieen Black Box and White Box

Testing Board Questions
1. Distinguish between white box testing and black box

~ testing. (any six)

i&~i m
aa::m:::::::ss; i't'iifffNE-l&MiBN
fl iii Mi ii4 § ,:•Wttttil
2. With the help of example explain boundan; value , .
g;pqm;p

analysis.
MSBTE : Summer- I 5, · 17. Winter-I 7. Marks 8
3. Explain the boundary value analysis technique used in·
black box testing with example.
MSBTE : Summer- I 6. Mt1rks 4

4. Why boundary value analris is ? Give rlred

■§?,: •MiJ@ · ri§Mff •
-itlllitl lJ~i~
example.
5. What is boundary value analysis ? List any three
guidelines for boundary value analysis.
.,. ,.··,<""';,,,,,_ ' 'i·'·"·· ,,. ·,_-ation MSBTE : Summer- I 8, Marks tJ
6. Explain concept of application of equivalence
i~ . partitioning. How it i ~ ' ·
/out ·
diploma?
g.
7. Illustrate process of equivalence partitioning with
example.
MSBTE : Summer- I 5, 17, Marks 4, Winter- I 6, Marks 6

8.Explain equivalence ,artitionini with

equivalence classes. •~ =,:Ji
to "'it
3 i!Jii,iiif!IJ . Hi
9. Explain requirement based testing with its stages and
requirement testing process.
MSBTE: Winter-18, Marks tJ
· 10. Explain the impact of equivalence partitioning in
coding and testing.
:\1SBTE : Summer- I 9. J\'fork., rJ

□□□

,• r,,.,.1v,v-o1 o. ~i;,.o&,,.,.., • 4n 11n ,,,,.,,,,, fnr knowfedae

 Bffi'''•"'"'
·, il fl' ''i'"'"'&Vlfaiilll!.""'
11
;" · '- '•· •,;_
.!!. - - ;J:r: . . . . . - ,~
._ . ._,_,, ,_ .,,"• 'llu"•'i••·•11".
u .;,1 " ,,.,_ ._. _ -

UNIT· II

2 Types and Levels of Testing

fjIJ Levels of Testing

We begin by 'testing-in-the-small' and move toward 'testing-in-the-large'.
Various testing strategies for conventional software are
1. Unit testing
'.t 2. Integration testing
3. Validation testing
I ·4, System testing
e
1. Unit testing - In this type of testing techniques are applied to detect the errors from each software
I component individually.
z'

Fig. 2.1.1 Testing strategy

2. Integration testing - It focuses on issues associated with verification and program ·construction as
components begin interacting with one another. •

-T

· .· :(2-1)
 "&~'1-f
ryr,es and Levels of Tes~ , ~e JI' , t
,,1·

Softwm-e Testing 2-2 'teria (established during .

• 1·.f~ eO l
3. Validation testing - It provides assurance that the software validatton en . ements.
. . nee requ1r
off/
#' ei' A

reqwrements analysis) meets all functional, behavioural and perfonna · d whole. ~ /

. stem is teste as a cO
4. System testing - In system testing all system elements fonning the sy . _ # 1l
I2.2 IUnit Testing . ~tf ,tt
.
In urut . . . . . l t ensure their quality. o~I~ ·t 11
• testing the ind1V1dual components are tested independent Y O ~, A
,tf0 ,heV''
'~11· Jt1'1
•
•
The focus is to uncover the errors in design and implementation.
The various tests that are conducted during the unit test are described as below·
.
1· Module interfaces . d t of the program.
are tested for proper information flow in an ou
~:~rf ..le o
1v· ,110"
· ~v·
2. Local data are examined to ensure that integrity is maintained. .
#
i _:Aioi'
jl\aC'

3.
.
Boundary conditions are tested to ensure that the module operates pr
operly at boundanes
i ~ pi,sD!Io
established to limit or restrict processing. . 4,C~ , fest
4. · (independent)
All the basis · · th at a11 st atements in the module
paths are tested for ensuring ~u~"
have been executed only once. develof
5. All error handling paths should be tested. ~.:
Things to be tested ce II"
~. asfoJlo·
theJll IS '

. Generating

Fig. 2.2.1 Unit testing

6. Drivers and stub software need to be developed to test incomplete software. The driver" is II

a program that accepts the test data and prints the relevant results. And the stub" is a II

subprogram that uses the module interfaces and performs the minimal data manipulation if
required. This is illustrated by following Fig. 2.2.2.

l Fig. 2.2.2 Unit _testing environment

TU -
Technical Publications_ - An up thrust for knowledge
 2-3 Types and Levels of Testing
7. The unit testing is simplified when a component • The driver and stubs are generally dummy codes or
· with high cohesion (with one function) is fake codes ·that help to simulate the behavior of
designed. In such a design the nu~ber of t~st existing code.
cases are less and one can easily ·predict or • The stubs and drives are specifically developed to
uncover errors. meet the necessary requirements of the unavailable
modules and are useful in getting expected results.
[2.2.1] Errors Identified during Unit Testing
• Generally driver and stubs are used in unit testing
Various data structure errors that can be identified or during integration testing.
during the unit testing are -
1. Incorrect arithmetic precedence.
2. Mixed mode operations.
3. Precision inaccuracy. ·
4. Comparison of different data types.

I2.2.2 IUnit Testing and Debugging

Many developers think that unit testing and
Fig. 2.2.3 Driver and stub
debugging are one and the same thing. But there is
difference between the two. The comparison between • Stub
them is as follows - o These are dummy code.
o These· are used in testing when lower level of
code are not developed.
o To test the upper level of code the stubs are
used.
o It just accepts the value from calling module and
returns null value.
• Driver
o These are dummy code.
o These are used in testing when upper level of
code is not developed.
o To test the lower level code the drivers are used.
o Basically we call the driver as main function
which calls other modules to form complete
applications.

Board Questions
1. Describe how drivers and stubs can be used in unit
testing with neat diagrams.
MSHTF : Winti>r-16. i\1.uk., I

I2.2.3 IDriver and Stub 2. Explain unit testing. State its additional re'Luirements.
• Stubs and drivers are two such elements used in hi~i&f S lhhhh411¥1HNI
3. With the help of neat diagram, describe unit testing.
software testing process, which act as a temporary i\1SKrl-. Sumnwr- 18 . Winter-IS. i\1,,rk., 1
replacement for a module. 4. . Explain the need of stubs and drivers with diagram
and its importance in software testing.

T&clinical Publications - An up thrust for knowl9dg&

 Software Testing
2-4 This process continues
I2.3 IIntegration Testing corrected new 0
nes appear. .

• A group of dependent components are tested infinitely. . proach is simple.

. . This ap
together to ensure their quality of their integration Advantage of big-bang ·
unit. . Disadvantages :
• The objective is to take unit tested components and 1. It is hard to debug. .
' hil testiJlg.
build a program structure that has been dictated by . eas to isolate errors w e
software design. · 2. It 1s not Y J"d t test results
to va 1 a e ·
Th t f" \ · . 3 In this approach it is not easy
• e ocus o integration testing is to uncover errors . . . it is impossible to form an
in: 4. After performing teStiI1g,

o Design and construction of software integrated system. · eludes

stru ction strategy 10
architecture. • An incremental con

o Integrated functions or operations at subsystem 1. Top down integration

level.
2. Bottom up integration
o Interfaces and interactions between them.
3. Bi-directional integration
o Resource integration and/or environment
integration. I2.3.1 ITop Down Integration
• The integration testing can be carried out using two • Top down .testing is an incremental ap_proach in
approaches. 1
which modules are integrated by movmg down . i
1. The non-incremental integration through the control structure.

2. Incremental integration • Modules subordinate to the main control module

• The non-incremental integration . is given by the are incorporated into th·e system in either a depth
"big bang'' · approach. All components are first or breadth first manner.
combined in advance. The entire program is tested • Integration process can be performed using
as a whole. And chaos µsually results. A set of following steps.
·errors is tested as a whole. Correction is difficult
because isolation of causes is complicated by the
size of the .entire program. Once these errors are

T~ dawn int~ratiQq:te.tRt'-1$
8ottom upJ~r~on tes.tinq
. Bi~dktctjqntil ih1t;1gt~JioJlt.,ting
Fig. 2.3.1 Integration testing approach

hf
Technical Publications -An up thrust for knowfedge
 2-5 Types and Levels of Testing

1. The main control module is used as a test driver Disadvantages of Top Down Integration Testing
and the stubs are · substituted for all modules 1. The top level modules can not be tested perfectly
directly subordinate to _the main control module. as every time _stubs are replaced _w ith _real
2. Subordinate stubs are replaced one at a time modules.
with actual modules using either depth first or 2. It is difficult to test the units and modules alone
breadth first method. before integration. Hence the problems might
3. Tests are conducted as each module is remain in the individual unit
integrated. 3. Stubs need to be written and tested before they
4. On completion of each set of tests, another stub can be used for integration testing. These tested
is replaced with the real module. · stubs are then finally getting replaced by the
5. ~egression testing is conducted to prevent the actual module. Hence creating and testing large
introduction of new errors. number of stubs arid making them defect free is
actually time consuming.
For example :
In top down integration if the depth ·first approach is i2.3.21 Bottom Up Integration
adopted then we will start integration from module In bottom up integration the modules at the lowest
Ml then wewill integrate M2 then M3, M4, MS, M6 levels are integrated at first, tl).en integration is done by
and thenM7. moving upward through the control structure.
in If breadth first _approach is adopted then we will The bottom up integration process can be carried out
/1\ integrate module Ml first then M2, M6. Then we will using following steps.
integrate module M3, M4, MS and finally M7. 1. Low-level modules are combined into clusters
that perform a specific software subfunction.
le 2. . A driver program 1s written to co-ordinate test
1 case input and output.
3. The whole cluster is tested.
4. Drivers are removed and clusters are combined
moving upward in the program structure.

For example ;

Fig. 2.3.2 Program structure

Advantages of Top Down Integration Testing .

1. The major problems can be detected at the early
stage of the module as the top most layer is
created first.
2. We can create partially working framework
.(prototype) to the customers as well as .to the top
management and the flaws can be · detected
immediately by taking input from the customer.
3. As the framework is created at the topmost layer
...
c,tiie,z
the feasibility .of the program can be detected at
Fig. 2.3.3 Bottom up Integration testing
the early stage.
tu
Technical Publications - An up thrust for knowledge

:...•...,...,
 Software Testing

First components are coJJected together to form cluster

1 and cluster 2. Then· each cluster is tested using a
driver program. The dusters subordinate the driver
module. After testing the driver is removed and
cl1,1sters are directly interfaced to the modules.

Advantages of Bottom up Integration Testing

1. Starting at the bottom of hierarchy means critical

modules are created and tested first. If these
modules are working perfectly then only they
. are integrated and move upward in the
hierarchy. • Integration
12.3.3] Bi-Dlrect1ona 1 . . ombination of the top
2. As _
stubs are created first, test conditions can be . I . tegration is a c .
• Bi-directiona in .. · tion testing approaches
easily identified or created . . d bottom up mtegra
down an . the integration process.
3'. The test results are . easy to observer and used together to denve .
therefore the robust system can be created. • h integration.
• It is also called as san d wic .
Disadvantages of Bottom up Integration Testing . of·b1' directional testing is as follows -
• Process - .
1. The program as an entity does not exist until the . f t ting, the testing begms from the
o In this type O es .
last module is added. middle layer.
2. Stubs and drivers need to be written before o When the bottom up approach is used then
using them into integration testing. It can be testing is done from middle layer to the top
waste of time as they are not going to be the final layer.
worki{lg module of the program. 0 When the top down approach is used then
3. As the testing begins from the bottom, there is testing is done from middle layer to bottom
no partially working module that can be layer.
represented to the customer. • For example - Consider the program structure as ~
4. Any form of errors in the interface can be caught given in Fig. 2.3.4.
at the later stage of the testing as we move
upwards towards the top most layer

Difference between Top Down and Bottom UP Testing

Fig. 2.3.4 Program structure

The modules Ml, M2, M3, M4, M5 and M6 will be

tested separately. The bidirectional integration is
performed initially with the help of stubs and drivers.
 --
sol!!!!!!! Testing
Using stub the downward modules ate connected and
using drivers the upward modules are connecte~.
2-7

4.
Types and Levels of Testing

State process of bi-directional integration testing with

two advantages and two disadvantages.
MSBTL : Su,nrner-1 :i. M,nk" t1
;.fter testing the functionality of these integrated
coJil_ponents, the drivers and stubs are discarded. s. Explain the integration testing and its two types. Inif
wi,en the modules M7, MB and M9 becomes available,
each type, explain with example the stys
0

the actual components are tested and integrated. The

integration. •3MHf 5jj .. @§4-id!'Bi
6. Describe bi-directional!sandwitch integration testing
integration can be carried out in following steps.
with neat diagram.
Bottom up integration approach : MSBTE : Winter - I 6 . 18 . M,nb t1

(?J7-M2-M3-M4), (MB-MS), (M9-M6) 7. 1i1ustrate process of bi-directional integration testing.

Top down integration approach : State its two adva~tages and disadvantages.
MSBTl: : Summer- 17. M.uk" t1
(Ml-M7-M2-M3-M4), (Ml-MB-MS), (Ml-M9-M6) 8. Explain following concepts reiated to integration .
Advantages of B1-cllrectlonal Integration testing with neat and labelled diagram :
1. Top down and bottom up layers can be handled (i) Top-down testing. (ii) Bottpm-up testing.
MSBTl:: Summer-18 . M.uk" 8
in parallel.
9. Explain bi-directional integration.
2. This type of integration requires less number of MSBTE : Smnmer-19. M,1rks 4
stubs and drivers.
3. As this integration has better coverage control,
!2.4 !Testing on Web Application
this integration technique can be used for • Testing strategy suggests _· to use following bask
han.d ling the large projects. principles of software testing -
4. Construction of tests cases is easy. 1. The content model must be reviewed in order to

S. 'Integration can be done as soon as the uncover the errors.

component is implemented. 2. The interfaces model is, reviewed to ensure all
the use cases.
, Disadvantages of B1-dlrectlonal Integration 3. The design model is reviewed to uncover
1. This approach still needs to throw away the navigation errors.
stubs and drivers as the actual component is 4. User interface must be tested to remove the
available. navigation errors.
2; The different skill sets of tester are required as it 5. For selected function components unit testing is
is a combined approach. done.
3. This type of integration is not suitable for 6. Navigation must be tested for the complete web
.smaller project~. architecture .
4. The cost involved in this testing is more than top 7. The web application is tested in different
down or bottom up-approach. environmental configuration.
8. Security tests must be conducted to exploit
Board Questions .
1. State and explain top-down a;oach ofintegratwn vulnerabilities of the web application.
· wit. h d',agram. •~L,-,•••P
testing
•--·-:-
; _ •vmmp
ASlnllci
-· 9. Performance ·of the web application must be
2. Describe top-down inteS!ation testinf WJiJbelled tested using performance testing.
diagram. i3~-fi•f6i::::::::11'-;:!f~f\lfiil~I 10. Finally controlled and monitored group of users
. 7 L'15 t types of integration
3. What is integration testing · . . :. . ·must test the web application.
testing and explain any four types in brief

- Technical Publications - An up thrust for knowledge

 TY1!_e9 and Levels of Tes~

SoA.·~re Testino s --- ----

2 -~
.::::!.:1=:.•"""':.:..:.::::.::~o~---------------=~ - b evaluated repeatedly
ed to e
rmance testing ne t"Ving tiJne interval.
• The web application evolves continuously · and perfO es at vaA; - .
for d.1fferent resourc; the process of performance testing. 1

hence it is an on-going activity. .

•· There are four approaches of web application Fig. 2.4.1 represents a e) .
(See Fig. 2.4.l on next p g . and analysis : The
testing and those are - gathertng tin
1. Performance testing · 2. ·Load testing 1. Requirements for the performance tes Mg_ ~e
3. Stress testing 4. Security testing requirements t1 for the performance. a1onty
hanaing frequen Y . ework or changes in ,
I2.4.1 IPerfomiance Teating c o- - .
of performance issu .
es reqt11re r
Hence it is important to
Performance is the basic requirement of any product . or design. t
the architecture .· lly important o test
. ts It is equa
and it is a major ·concern for testing as well. collect reqwremen . . t various performance
.Performance testing is done to check whether system all those requirements agalllS
meets its performance requirements under the normal
critera. Ian . The test plan
load. rf Orroance test P ·
2. Create pe d more focus during
Definition of performance· testing : Performance th issues that nee l
documents e . the design of test cases. The l
testing is a kind of testing performed to evaluate the the testing or dunng .
respo~e time, throughput, and utilization of the test plan includes -
All the required 1
system to execute the required functionality in a. Resource requirements
comparison with the competitive products. resources must be listed.
· nment · ,
b. Required . test set up or envtro .
Why to perform the performance testing ?
Performance testing requires large number of
Following are few reasons for performance testing - test resources with special configuration which
1. Perfe>rmance testing is carried ~ut to check if the is to be documented in the test plan.
product is available or running on different load
c. Responsibilities : Changes in the architecture or
conditions.
design, or the requirements changes need to be
2. It is done to check that the product responds fast documented promptly and must be conveyed to
enough on different load conditions. all the team members.
· 3. It is carried out to ensure that the required d. List of traces or audits : Schedule for regular
number of transactions can be processed by the
traces and audits to evaluate the work must be
product at any given interval. documented in the test plan.
4. The performance testing can be carried out to
e. Entry and exit criteria Execution of
decide the kind of resources required by the
performance test case occurs only after m~ting
I '
system for different load conditions. .
the performance criteria. This set of criteria must
5. It is done to compare the product with other be planned well in advance. Similarly the exit
versions or other competitive products in the criteria is needed to conclude the result of test
market. case execution. The set of exit criteria must be
present in the test plan.
I
12.4.2 Process for Performance Testing
· are more . 3. Design and automate test Case : The test case must
Efforts required for the performance testing
be designed based on the performance
\lso there is a need for multiple resources d unng
· the
.
tin 1s · requirements. The perfonnance testing is always ,
,erformance testing. Hence performance tes g . .
carried out with the help 0{ some suitable
ostly. It is also repetitive testing as the test cases m automation.tool.

-----~--~-----~=:= ·.:i--;P;:ub~"~ca;tion;s-;ifii~-~A;n;up~th;,;ru~st~for~kno~wl~ed~ge;---------------
Techmcal • 1
 -
\
5oftu"!_re Testing
2-9 Types and Levels of Testing

Fig. 2.4.1 Process for performence testing

d -4. Evaluate entry criteria : During performance • It also helps in identifying the bottlenecks in the .
testing when · certain performance criteria get systems under heavy workload conditions.
satisfied then the performance test begins its • Load testing helps in identifying -
ll execution. This is called entry criteria. The entry Response time for each transaction
O
h criteria must be evaluated at regular intervals
o Performance of system components under load.
during the performance testing. There is separate
o Network delay between client and server.
)!
· set of criteria for each of performance test case.
o Server configuration issues such as web server,
5. Evaluate test case : Execute and evaluate each test
application server, database server
to case.
6. Evaluate exit criteria : After exe_cution of the test Advantages of load testing :

ar case product is evaluated to see whether it has met 1. Performance bottlenecks identification before
,e all the required performance criteria after test case production
execution. If the criteria are not met then the test 2. Improves the scalability of the system
case is again executed. This process is repeated 3. Minimize risk related to system down time
of
until the performance criteria are met. The results 4. Reduced costs of failure
are collected after evaluation of exit criteria.
1st 5. Increase customer satisfaction
<lt Thus performance testing is conducted. This is one of
Disadvantages of load testing : -
~I the crucial testing process.
1. Need programming knowledge to ·use load testing
I
be b.4.3 Load Testing tools.
• l.oad testing is a testing process used to check how 2. Tools can be expensive as pricing depends on the
systems behave under normal or peak load number of virtual users supported.
,ce conditions.
3ys • Load testing · gives the confidence in system about I2.4.4 IStress Testing
i,te its reliability and performance. • Determines breakpoint of a system to establish
maximum service level.
Technical Publications - An up thrust for knowledge
 ~ jl
'ltlll!tiill!lllllllf.\j ?iliiffl'!§tlhliffii" ~ 1" lll! ..,iI' ~;..~

------

rxres and Levels ofTestj~ l'1.~

So!ware Testing_ 2 -10
maintains functionality as . ~
• In stress testing the system is executed in a manner system protects data and
that demands resources in abnormal quantity, intended.
frequency or volume. • Security testing is done to protect the application , 1

against unforttJnllle incidences. The unfortuna1tll

• A variation of stress testing is a ·technique called
incidences could be loss _of privacy, loss of data,
sensitivity testing.
modification in system data and so on. .
• The sensitive testing is a testing in which it is tried
• The unfortunate incidences could be internal or
to uncover data from a large class of valid data that
may cause instability or improper processing. external.
• During security testing system is basically finding
Difference between-load testing and stress testing
out all possible loopholes and weakness which
might result in vulnerability to outside attacks or f''
unauthorized entry in the system. ff
Objectives of Security Testing
• The goal of security testing is to identify the threats
in the system and measure its potential
vulnerabilities.

i,;~~,!~
1'J
,- Th;~j,i,lic:hJ~!~,~ g,
• Security testing helps ii1. improving the current
system and also help in ensuring that the system
will work for longer time,.
• Security testing helps in finding out loopholes that
can cause loss of important information.

Security Testing Technique

The main security testing techniques are

Advantages
1. Vulnerability scanning : The vulnerabilities are ,. ,
scanned using automated testing.
1) It finds out if data can be corrupted by
overstressing the system. 2. Security scanning : In this scanning, the network ·
and system weaknesses are identified and solutions
2) It helps to determine what kind of failures are most
are obtained. This scanning is done using both
valuable to plan for. 1
automated and manual testing.
3) Offers assistance in determining the stability of the
3. Penetration testing : In this testing, tester tries to
software system beyond its normal operational
peep into the system via the loophole that !
capacity.
application has kept unknowingly. It is one kind of
1

4) Makes the software crash-proof in scenarios where

simulation of attack from malicious hacker.
the computational resources are scarce.
4. . Ethical hacking : In this type of testing, the number
5) It helps to find deadlocks in software product.
of penetration te~ts are conducted over the wide
I2.4.5 ISecurity Testing network on the system under test with the intention ~
Definition of · security testing : Security testing is a to expose security flaws in the system.
testing ~echnique to de~ermine if an information

tM
Technical Publications - An up thrust for knowledge ____________ .
;.~~ ·
'@
\-~ - . . ..........,1;1~ 111c ~ ~!%if QIL 11/'flFl"Wil 1:~~ jiffl ;,.11.. - -

re resting 2 - 11
Types and Levels of Testing
~essment : In this testing, the analysis of
~s,curity risks is done With the help of interviews,
discussion and analysis.
7. How to perform security testing ? State elements of
security testing.
MSHII : ~lllllllll'l-17 M,irk .. t'I

security auditing : It is a technique in which

~ jntemal inspection of operating system . and
I2.4.6 lClient Server Testing
• This is a type of testing which is applied for 2 tier
application is done by line by line inspection of
code. applications. The two tier includes front end and
back end .
7, Password cracking : There are some special Request
programs _that can be used to identify the weak h': criehf ,' ;;1
.·. (Front end)/; Response
pa_sswords.
Fig. 2.4.2 Two tier architecture
common Areas of Security Testing
• In client server applications - the front ends are
There are common area for which the security testing
is must. These are as discussed below _ developed in VB, VC++, Java and so on. The
backend are developed as the database systems in
1. For E-commerce systems high protection for the
M.S ACCESS, MySQL, Oracfe and so on.
user is required in order to protect their privacy
• Testing of client server architecture is very difficult
and information involving financial transactiqns.
because of following reasons -
2. Communication systems require the protection
1. Distributed nature of client server
from loss of data during the transmission from one
system to another. 2. Performance issues of different hardware
platforms
3. The database systems, or business logic systems
3. Complexities of communication network
are prone to threats as penetrators can attack and
modify the data present in such systems. Hence 4. Need for servicing multiple clients
system testing must be carried out for such 5. Database access by multiple clients and their
systems. transaction processing.

Features of Client Server Application

Board Questions
1. How performance testing is performed ? List steps o This application runs in two or more machines
involved in it ? o Application is menu driven
MSBTE : Winter-15 , Marks 4
o:.- There are limited number of users for this
2. Explain the performance testing and its criteria. application
MSBTE: Summer-16 , winter-18, Marks 4

3. Describe any four testing approaches of web o It has less number of network issues when
compared to web app.
application.
MSBTE : Winter-16, Marks 4
o The client and server work in a connected mode.
4. What is load testing and stress testing ? Describe with
Types of Tests Conducted for Client Server Architecture
respect to system testing.
MSBTE : Winter- I 6, Marks 4
Various approached of testing the client server
· 5. Describe how to perform security and performance architecture are as follows -
testing. (1) Application function test : The standalone
MSBTE : Summer- I 5. Marks 4
application function is tested in this type.
6. Describe following testing with example. (1) Security
testing (2) Performance testing
MSBTE : Summer- I 5 , Marks 4
r,;;
Technical Publications : An up thrust for knowledge
T.
 ·- - - - --m,;::-,.~....__......... wt~ • .. l!&

TYT'.es and Levels of Testi"l__ Ifli

Software Testing_ 2- Zi
3. It also gives the critical success factor to detemune ,
(2) Server test : Co-ordination and data management whether purchasing such products will be useful Or !
activities are tested. The server performance is also
not. I
- tested
(3) Database test and transaction tests : The integrity
4. Acceptance criteria determines the confidence level I
m the final deliverable of the product. I
of data store and performance of transaction
processing are tested.
5. It defines clearly whether the system meets !
1
predefined attrlbute~ or not. :
(4) Network communication test : The communication
among various nodes, performance and network Types of Acceptance Criteria .
security are tested. 1. Product acceptance : Ea~ requirement is normally
associated with the acceptance criteria. Whenever
Board Questions
there is change in requirements the acceptance
1. With the help of diagram describe client-server testing.
J\1SH 11. S111111111•r- I 5 . \Vrntn- 17 . I H. 1\1.irk-, '1 criteria must be modified accordingly.
2. Explain the client-server application testing. 2. Procedure acceptance : Acceptance criteria can be
MSH 11 · S11111nwr- I 6 . I H. 1\1,,rk-, '1
defined based on procedures followed for delivery
3. List any four features of client-server application.
of the product. For example - 1. User manual,
Explain any four testing approaches of client - server
testing.
troubleshooting guides should be the part of the
MSHl I. · \Virrf Pr - 16. 1\1,,rk-, H release. 2. The training should be provided to the
4. Describe use of load testing and stress testing to test users of the system before the onsite deployment.
online result display f~cility of MSBTE website. 3. Service level agreements : The service level
MSHII : \\'intn - 17 . 1\1.irk-, f,
agreement can also become the part of acceptance
[!jJ Acceptance Testing criteria. It is sort of contract signed between
• Acceptance testing is a testing phase which is customer and owner of the product. For example -
conducted normally after system testing by the If the defects occur within first 2 months of
customer or representative of customer. deployment then those need to be fixed free of cost.
• The goal of acceptance testing is to establish the This is a service level agreement.
confidence in the system. !2.5.1 !Alpha Testing
• The acceptance testing defines criteria for Alp~a testing is a kind of acceptance testing in which .
acceptance or rejection of the software.
version of complete software system is tested by the·
• The acceptance test cases are black box type test customer under the supel"Vls1on
•. •
of developer. This
cases. They make use of one or more acceptance type of testing is per£ ed
. orm at developer's site. The
criteria for accepting the product. software lS used in natural setti.n .
developer This test . gs m presence of
. · IS actually cond ucted m
. controlled
Importance of Acceptance Criteria
enVIIOnment
1. Acceptance criteria are the ~nditions that a
software product must satisfy to be accepted by a Advantages
user, customer, or in the case of system level 1. The users of the system get systematic training to
functionality. use the new system.
' The acceptance criteria is very useful in acceptance 2. The acceptance of the system can be immediately
testing because it represents the customers' known to the developers.
expectations from the system.

111
Tec/nCIII put,Jicaljons -An up thtu$t for knowledge
r·
 ~
-!I
-- - ~~:-~,f.....
~:lif- 1:"i'- ~ - -
!ffllr~-
I': I

' t
~lz~ .
!J-..

;~ j
~ 2 -13 Types and Levels of Testing ]\
l
3 'fhe pnmary problems in the identified and fixed in 5. Beta testing allows the organization to test post-
~d · iIJUI'lediately before delivery of the product to the
launched infrastructure.
customer. \
, l~\>eJ 6. Due to customer feedback product failure risk can
4_ AnY mis-understanding about the scope of the be reduced.
project or in requirements given by the customer
t\~ts 7. Usability of the application can be identified and ,,J1
can be cleared during this type of testing.
verified by the customer. 1i
pisadvantages
I
Disadvantages : 11t
I 1. In depth functionality cannot be tested as software
laJJy 1. Finding right beta user and maintaining his

e"er ,
is still under development stage. Sometimes
developers and testers are dissatisfied with the
participation can be real challenge. I
I 2. Tests can not be management because beta testing
lllce / results of alpha testing.
is conducted in actual working environment and
2. Alpha testing is conducted in lab environment or
n,ot in the organization.
I be ' testing environment which may not be similar to
ery ·real-life environment. Difference between Alpha and Beta Testing
.la}, ' 3. Data used during this testing may not be similar to
the : the business data or .actual data. Hence false image
the I
of working module can · be created and actual
problems remain hidden.
1el
I2.5.2 IBeta Testing
1ce I
The beta testing is a kind of testing in which the
en / version of complete software is tested by the customer
I
>-
without the developer being present. This testing is
of
done at Customer's site. As there is no presence of
;t, I
developer during testing, it is not controlled by
developer. The end user records the problems and
report them to developer. The developer then .makes
h appropriate modifications.
!
e Advantages :
1. .Beta testing improves the product quality via
customer feedback. ·
2. Any training ~quire~entfor use of the product can
be identified during beta teS ting.
work environment
3. The testing is done in actual Board Questions
developer. Hence
without the interference of
changes in the 1. List any four advantages of acceptance test before
addition in requirements, launching any software.
requirements can be identified. l\lSHT[ : \\i intn-15 11i . !\I.irk., l

4. The security loopholes can be identified by the 2. Explain the alpha testing. State its limitations. -
l\lSBTL : S1111111wr - I 6 i\l.nk., ,}
customer itself.

Tl1
Technical Publications - An up th~ for knowledge

T·.
... . J
 _._._..= --~~- "~~-~,,,_:::-·-

~ l~
So ' .
di"-e
"',_, nces). . .
--
=-__;________
I
2 -14 --- 1Yi"' and ,.,,,," oJr,.,.
Following are some ways to test the GUI - r

1. Use of standard tests: J\,lanY modem GUis ha'-'e

:\1SH1 I. . \\ 111tn - l 6 . 17. I k . Summc>r-1 H. Mrtrk--. '1 same look .and {eel hence a series of standard tests
4; Describe alpha testing with its entry and exit criteria.
can be derived.
l\1SHTI · Sumnwr- 17. M .. rk-. '1 2. Use of finite state 111odeling graph : The finite state
5. Differentiate between alpha testing and beta testing. modeling graphs can be created for testing speci,fit
.
J\lSBTI · Summn- 19. Mrtrk-. '1
data and program objects of GUI.
I2.6 ISpecial Tests 3. Use of auto111ated tools : Due to large number of
I2.6.1 IRegression Testing permutations in GUI applications the automated

• Regression testing is used to check for defects tools are used for testing.
propagated to otl:ter modules by changes made to
Checklist for GUI Testing
existing program. Thus regression testing is used to 1. Check all the GUI elements for size, position, '.
reduce the side effects of the changes. 'dth l gth and acceptance of characters or :
M , en ~
• There are three different classes of test cases
numbers.
involved in regression testing -
2. Check you can execute the intended functionality of .,
• Representative sample of existing test cases is used
the application using the GUI
to exercise all software functions.
3. Check error messages are displayed correctly
• . Additional test cases focusing software functions
4. Check fo r dear representation of di ffe rent sections
likely to be affected by the change.
on screen
• Tests cases that focus on the changed software
5. Check font used in application is readable
components. .
6. Check the alignment of the text is proper
• After product had been deployed, regression testing
would be necessary because after a ch~nge has been 7. Check the color of the font and warning messages is
made to the product an error that can be discovered aesthetically pleasing
and it should be corrected. Similarly for deployed 8. Check that the images have good clarity
product addition of new feature may be requested 9. Check that the images are properly aligned
and implemented. For that reason regression testing
10. Check the positioning of GUl elements for different
is essential. screen resolution.
Board Question
Advantages of GUI Testing
1. Explain the regression testing. State when the
regression testing sh.all be done. 1. Good GUI help the ~r to give the better
,1sB 11 : Summ,•r-16. \\'intn-17 . '.\l.uks -1 experience of handling the app lication.

I2.6.2 IGUI Testing 2. The user can readily accept the application if the
GUI is well tested and w orking in appropriate
Testing the graphical user interface is very challenging manner.
because GUI consists of many reusable components.
3. The availability of help feature allows even the
This actually increases the complexity of GUI and
leads to more difficulty in design and execution of test novice user to handle the application in the
required manner.
. cases.
4. Consistency in the page layout and arrangement
Approaches of GUI Testing improves the usability of the application.

'I'll
T. TechnicaJ Publications - An up thrust for knowledge
........
. Types and Levels of Testing
~5oftwt!_re Testing
. 2 - 15

s. User can understand the behavior of the system

with the help of good GUI.

01sadvantages of GUI Testing

1. Special users such as blind, or underage users can

not handle the application using GUI.
2. If the spelling or arrangement mistakes occur in
GUI then it affects the overall aesthetics of the
application.
3. Tester give least importance to the GUI testing and
more attention is paid to functionality of the system
component. If the GUI is not well tested then it may
lead to confusion in handling the application.
4. If the GUI testing is not done properly, then
consistency in layout and . arrangement of
documents is lost.

Board Questions
1. Describe Graphical User Interface (GUI) testing and
its important traits.
MSBTE : Winter- I 5 . Marks '1
2. Explain·GUI testing with suitable example.
MSBTE : Sunmwr-15. Marks 6, Winter-17. Marks 4
3. Describe any two special tests in testing process.
MSBTE: Summer-17 . Mark-. 4

4_ Explain GUI testing with example.

MSBTE : Sununer-19 . Marks 4

□□□

n,
Technical PublicatkJns -An up thrust for knowledge
 ,1,----
• IJF~t; ~ -...- --·-

UNIT - Ill
-,,,,-

·3 Test Management

[i] Test Plannlng 4. Test plan ·ensures all functional and design
requirements as specified in the documentation.
Testing is an important phase which can be planned,
executed, tracked and periodically reported on. Standard Template for Test Plan

Definition of test plan : Test pian is a detailed_ 1. Introduction

document that outlines the test strategy, testing 1.1 Scope
objectives,- resources such as manpower, software, 2. References
hardware required for testing, test schedule, test 3. Test methodology and Strategy/Approach
estimation and test deliverables. 4. Test criteria
The test plan serves as a blueprint to conduct software 4.1 Entry criteria
testing activities. 4.2 Exit criteria
I
13,1.1 Preparing a Test Plan 4.3 Suspension criteria
4.4 Resumption criteria
Test plan is an imp~rtant document for execution,
tracking and reporting the entire testing. 5· Assumptions, dependencies and-risks
5.1 Assumption
Following things are required to prepare the test plan.
5.2 Dependencies
Step 1 : Write the Scope of testing including the dear
indication of what will be tested and what will not be 5.3 Risk and risk manag~ent plans
tested. 6. Estimations

Step 2 : Testing is performed by breaking it down into 6.1 Size estimate

small ·and manageable tasks and identifying strategies 6.2 Effort estimate
to be used for carrying out the tasks·. · 6.3 Schedule estimate
Step 3 : Find out the resources needed for testing. 7. Test deliverables and milestones
Step 4 : Design timeline by which testing activities will 8. Responsibilities
be performed. 9. Resource requirement
Step s : List of Risks faced in all of the above steps. 9.1 Hardware resources
Advantages of Test Plan 9.2 Software resources
1. Serves as a guide to testing throughout the 9.3 People resources
development. 9.4 Other resources
2. Test plan defines test points during the testing 10. Training requirements
phase. 11. Defect logging and tracking process
3. :3erves as a valuable record of what testing was 12. Metrics plan
done.
13. Product release criteria
(a~f:1)···

~ -
 Software TestinE_ Test Maruig~t
3-2
Board Questions • Testing can be suspended at various points of tune
1. What is .test plan ? List test planning activities. and then can be resumed.
U~id-RihiihhliMMMW■ • Some of the suspension criteria include -
2. How to prepare a test plan ? Explain in detail.
o Encountering more thart a certain number
Nkid-&iihliihllMiMHiiW■ defects, causing frequent stoppage of testing
3. State the contents of standard template of a test plan.
emm+ ;;;;;;;;;11,■MtN•
4. . 'What is test plan ? Write its two advantage.
activity.
o Hitting · show stoppers that prevent further
;\ISHTr. · S111111111•r- I 7 . l\1<1rh 1
progress of testing.
5. Give any four advantages of test planning.
o Developers releasing a new version.
MSHTE : Surnnwr-I8. i\l,1rk., 1J
6. Explain how a test plan is prepared. • When such conditions are addressed, the tests
I . •
can
J\ISH Ir : Summn-I9 , Mdrk .. 1J resume.
I3.1.2 IDeciding Test Approach · Board Question
• Test approach is a part of test plan. It identifies right 1. ¼'hy is it essential to setup criteria for testing ? List
type of testing required for the system. any three criteria in different situations.
l\lSRTE : \Vinh>r- 15. M.uk., 1J
• Test approach is a kind of approach in which we
have to decide what needs to be tested in order to I3.1.4 Ildentlfying Responsibilities, Staffing and Training
Needs
estimate size, effort and schedule.' This includes
identifying followi!lg factors - • Testing process requires different people to play
different roles. The most common roles are - test
o What type of testing is to be used for testing
engineers, test leads and test managers.
functionality?
• Each role must accomplish certain responsibilities.
o What scenarios are required for testing the
features? • How to identify responsibilities ?
FoUowing guideline need to be considered for
o What kind of integration testing is required ?
identifying the responsibilities -
o What localization validations would be needed ?
1) Each person should know clearly what he or she
o What kind of non functional tests(tests related to
should has to do.
reliability, performance, security and so on) are
required? 2) List out the responsibilities of each function in
the testing process.
Board Question
3) Everyone should know how his work is
1. Describe the factors considered to decide test strateil
important in performing testing activity.
or test approach. Q@GA@hihiiJiiMi+ 4) Complement and co-operate each other.
I3.1.3 ISetting up Criteria for Testing 5) No task should be left unassigned.
• There are various criteria used for testing such as • Staffing
entry criteria, exit criteria, suspension and
o Staffing is done based on estimation of effort and
resumption criteria.
time available for the project for completion.
• Entry criteria for test specify threshold criteria for
o The tasks in testing are prioritized on the basis of
each phase or type of test. There must be entry
effort, time and importance.
criteria for entire testing '!ctivity to start.
o The people are assigned to tasks based on the
• The exit criteria specify when test cycle can be
requirements of job, skills and experience of
completed. people.

T. T«lncal Publicstiona · An 1.q1 thrust for knowledge

 'W"'

If

~
....... Testiri_g_ _________ 3 - 3
Test MJ,,uigement
-~ 0 people are not haVing the skills for specific
requirements of the job then appropriate ttaining • are the indicates about the completion
Milestones
programs are needed. of key project tasks.
'l"
- - - ;Question .
• Some common test deliverables are -
g 1) Test plan itself
,.. Describe how to identify responsibilities in testing.
2) Test case design specifications
1. DMH-R ;.. ... ,:maca,_
r
am
~

Identifying Resource Requirements

--■;;:
HPH"t711
3) Test cases
4) Test fogs produced by running the tests
, While planning for testing the project, th~ project 5) Test summary report.
manager should get the estimates of hardware and
Board Questions
\software requirements.
1. Explain test deliverables in details.
, Fo11owing factors shall be considered while H~i;i·- iiiiiiiiiYEN\iffiWiMMfHI
selecting resource requirements - 2. What is test deliverables and milestones ? Explain ~ny
0 four test deliverables.
RAM, processor, disk required for the system i\lSBTE : S111111rn•r - l 6 . \Vi111t •1- IS . i\1,rrk, (,
under test.
0 Test automation tool - if required .
3. Explain the need of test deliverables for test planning.
MSHII:: Sumuwr - 19 . M,,rk., 1I
~1 r
1

.
0
Supporting tools such as compiler, test I3.1.7 lTesting Tasks
configuration management tool and so on.
0 Appropriate number of licenses of all the
• Various tasks of tester are - ih,
o Identify bugs and errors, describe and send for
software.
revision.
o Operating systems required for the execution.
o Check the implemented improvements.
0 Some machine intensive tests such a_s load tests 0 Test the product until the desired result is
and performance tests.
achieved.
Additional Resource Requirements ...
o Test the software solution on all necessary
o Office space devices and screen resolutions. ·
0 Some supporting staff such as HR. o Test the product compliance with the
requirements.
Board Questions
1. What factors shall be considered while selecting • Estimation is one of the ~porlant testing task.
resource requirements ? There are three phases of estimation -
l\lSRTE : \Vinter- 16. M,trks '1 o Size estimation
2. How to identify resource requirement of test plan ?
o Effort estimation
i\lSB rE : Summer- I 7. l\lark-, ·I

3. Explain the need of staff training and resource o Schedule estimation

requirements in test planning in software testing. 1) Size estimation
MSRTE : Summer- I 9 . Mark-,. 8
• It can be defined as actual amount of testing that
l!Ifil Identifying Test Deliverables need to be done.
• Definition of test deliverables : Test deliverables • Size estimated is expressed in terms of
are the artifacts which means "things" that are o Number of test cases.
Produced by people involved in the process and are o Number of test scenarios.
given to the stakeholders.
o Number of configurations to be tested.
--- ii,
,• Technical Publc:ation$ -An up thrust for knowledge

.....
 Test Maruig~t
--.;.;.::_
Software Testing 3-4

· 2) Effort estimations
• Effort estimation is derived from size estimation. -~_..rest case Oatabase(TCOB)
For example - if some test case is reused in the
Oefeci RepositOry
project from existing standard library of test cases
then effort involved in developing the test cases is Lo _ configuration
zero. Managment
Respository and
• Effo;t estimate can be given in person days, person Tools
months c:>r person years. , · Fig. 3.2.1

3) Schedule estimation 1) Test Case Oatabase(TCDB):

• Effort estimation is translated into schedule Test case database is a collection of -the information
estimate. about test cases in an organization. The contents of test
• Schedule estimation can be defined as the case database are as given below -
·translation of efforts into particular time frame.
:li)?s;: ~: !]~~~~'1\1;'.~~j:~i:[1!k
• Following are the steps required for performing
schedule estimation -
1. Identify external and internal dependencies
among the activities.
2. Sequence the activities based on expected
duration.
3. Identify time required by each activity. --
, ct-·
Provides _
between
4. Monitor the progress in terms of time and !6.E and correspondi[)g ·
efforts. :erence.
V 1. ♦
feature of the
product.
5. Rebalance the schedule as well as resources if
requirecf. ,.: ·.t est.iefclse Provi~les the · hi~tO.f½
of when .test was
I3.2 fTest Management ;;~ ;·;i,>,
-., .
running and .,W,hat
was the result ; ')+/J:
• Test planning [s a process of identifying the test
activities and resource requirements . for
implementation of testing strategy.
• Test management is a process of managing the test
process . by using series of activities such as
planning, execution, monitoring and controlling.

I3.2.1 ITest Infrastructure Management 2) Defect repository

• The robust test infrastructure or skeleton must be
Defect repository captures all the relevant details of
planned in advance.
defects reported for a product.
, There are three elements of test infrastructure
The information stored in defect repository is given as
follows -

•• . TechnicaJ Publcstions -An up thrust for knowledge

 ,.----.11:u,..._,;m'W,.,.__,,.· - - -

~I 3-5
Test Miinas.ement

I o F_
or each change made in the file appropriate
version of the file must be created.
I [:-.:
'r-.i
·~
o Everyone must get an access of most updated I
version of the file.
• Version control ensures that the test scripts I.! .!I.
associated with a given release of a product are i \
baselined along with the product files. I, l
I)
Working of Test Infrastructure Components
The TCDB, SCM and DR work together by cooperating ·
each other as shown in Fig. 3.2.2.
r\
\

it For example Defect Repository(DR) links defects, fixes i I.

and tests. These files are present in SCM. The metadata I\

about the modified files is present in TCDB and find
the corresponding test case files and source files from
SCM.

Fig. 3.2.2 Components of test Infrastructure

3) Configuration management and tools :
• Software Configuration Management(SCM) Board Questions
repository is also known as Configuration
1. What is test planning and test management ?
Management(CM) repository. MSBTE : Summer-16, Winter-17, Marks 4
• It keeps track of change control and version control 2. Explain the 'Test Infrastructure' components
of.all the files present. in the. software product.
. /
diagram.
• Change control ensures that - ·3. Write the entity, purpose and attributes of following
.o Changes made during testing of a file must be in elements of test infrastructure management :
· controlled manner. (i) A Test Case Database (TCDB )
o Changes made by one tester must not be lost or (ii) A defect repository. MSBTE : Winter- I 6 . Mark~ 8

,. overwritten by another tester.

Technical Publications
4. Explain test infrastructure manarment with its
components.
-An up thrust for knowledge
H@~ii+ %· ,i44@13Mi51

..i..'.:
" ",
p ~
·--~1
.;tt!'. ,l - •• .
_ ._..__
~rw •-~

Test Managemen
~

·~
/
~ ,J
Software Testing 3-6 -!_
I
J3.2,2 Test People Management • If any significant cl,aJ18" in made
ct d is in the
the test planteSling then
and aga·
r· :
fi
fl
• People
. management is an important aspect of any that should be re e Ian e must be m . confi gurationlI\
proJect.
• Good people help in .
this changed test P
management repository·
·
·1
I
(1) Achievin~ th~ project target. 13_3_2] Test case specification . .
(2) Commurucaling more effectively. Using test plan as basis, the testing team desi~ lest
(3) Co-operate effectively· case spea'fi ca· tt'on which. then becomes the baSis for
t
• For people management following are general steps preparing individual teS cases. .
to be taken up - . · b d fined as series of steps executed on
' Test case e e
1. Have effective communication with the people. product using predefined set of input dat~, expe~g
2. Build the healthy relationships. to produce predefined set of outputs m ·a given
3. Influence people. environment.
4. Motivate people. Following things need to be identified -
5. Handle ethical issues properly. (1) The purpose of test : The intention for which the
• The team-building exercises should be ongoing and . test is performed is specified.
sustained. The effects of these exercises tend to wear (2) Items to be tested.
out 1:11'der the pressure of deadlines of delivery and (3) Software and hardware environment setup.
quality. · (4) Input data to be used for test case.
(5) Steps to be followed to execute test.
Board Question
. (6) Expected results that are considered to be correct
1. Explain people management in test planning.
MSHT[ : Sumnwr-19 . MMk., '1
result.
(7) Actual result produced after performing tests.
[!!] Test Process (8) Relationship of current test with other tests.
I3.3.1 IBase Lining a Test Plan !3.3.3 !Executing Test Cases
• Test plan is an important document that acts as a • The prepared test Cases has to be executed at
anchor point for entire testing project. appropriate times during the project. For example -
• Using some checklist the test plan is prepared. Each system testing must be executed during system test.
testing project puts together a test plan based on • During the test case executiqn, the Defect
template. If some changes are required in the test Repository(DR) must be updated with - I
r,;:
plan then those changes are made only after careful
o Defects that are fixed currently.
deliberations.
.'
• · The test plan is then reviewed by expert people in
o New defects that are found. l
the organization. • Test team communicates with the development l
team with the help of defect repository.
• It then is approved by a competent authority, who
is independent of the project manager directly • Tests has to be suspended during its run and it l
responsible for testing. should _be r~s~ed only after satisfying the t
resumption mtena. · ,
• After this the test plan is baselined into
configuration management repository. From this • Tests should be run when entry .criteria for the test . j
test plan becomes the basis for testing process of the gets satisfied and should be accomplished only
project. when the exit criteria satisfied.
• On successful execution of test cases-the traceability
matrix must be updated.
'!'11
T. Technical Publications -An up thrust for knowledge

\ '.
 -~ -;.....-~,- -

~ , re Testing 3-7
h~ . ~ .

~~ ~ 1 : Prepare six test cases for admission form for college admission.
io11 501-:

est
Jt /fm. ~~~pt .
. nain~ 'at1d p~ompt
for , f<>r ) f ., +. ne)Cfr
sttbsaj_u~t
....
'·'
field.
... ·,,
,.·;. :-,,

Ol) ·,rr will · accept

ng ~ddr.ess , and
prompt for ·. rtext
~
subsequent field.

le

It will · - accept · rf ~J~~~pi~'.$~~~~, ·

number number, ·\} :. and;.
rr~~ llft' .
phone
and _prompt for
next subsequent
:t I field.
-- ·. It'·
It will prompt to
click submit
button and on
clicking submit
button admission
procedure page
t I
will be displayed.
It will display
'Please enter
marks'.

Ex. 3.3.2: Write important six test cases for the 'Login Form' of the facebook website.
MSBTE: Summer-15, 16, Marks 4
Sol.:

~~~~.,- 1
~,t~~;• •,.
fii
Technical Publications -An up thrust for knowledge
 3-8

.;~i#pfay ...
71'lame/pa.ssworp'•

· ._ ; · \~?
· ·w.'·.

fIIIIL-;,f;~m;~,u, ·
..·.. /~i.~pt.aY
acc6unt'.s

w . • . first class s,;--

·n as - distin.c 1ton, , ~
Ex. 3.3.3 : The student passing the diploma shall be awarded class of pasSi g I , s' in all O sub ·ects. Write
class, pass class. Tlte class will be applicable only if tlte student has ,h ts
· /her resu t L'.I as .. .. . , 1:1 • • ,
1
the test cases (minimum-4) for the class awarding algorithm (module).
Sol.:

It wi~ display
'Pass Cla~s'.

It will
'Fail' .

Ex. 3.3.4 : Write four test cases to test sign-in form of g,nail account. MSBTE: Winter-16, Marks 4
Sol.:

~r..·/ Technical Publications

TM
- An up thrust for knowledge

__,,,,,...
 ~I

i'
~
l·
~
!li!
f~
11
I
;,-;.3.5: Prepare six test cases for home page of marketing site.. MSBTE : Summer-17 . Mark!-> 6
Sol.:

.,,,.
,. Technical Publicatiqns
'M
- An up thrust for knowledge
 S0,~
1 ,ware Testing 3. 10
~I ,
- ,
:~::::::~~===========================================~=~::::::_::;;;;;;;iiiill~ielstiMa~nage,,,_
I n: : Illr, MSH
--
Surnnwr-18 . M,irk,
Ex. 3.3.6 : Prepare and write six test cases for simple calculator application.
Sol.: .

;i ,. . ·:,
.·· · l,resuw •·. 0
', 56 # 579:,

~l'\?tfot~~T~f'. It wm 'dispi~y
.· : :ii>\ t~in ;}W the resiiit of .,
123-:111 ,as 12.
~;f~l::Ilt0>'
. .· . ·.-,:,;.c.: ·:~, .: ,:" ·.

Ex. 3.3.7: Prepare and write four test cases for library management system of college ..
MSBTE : Summer- I 8. Mark~ '1
Sol.:

[
 Board Questions
1. How test case speci~ t· · MSBTE : Wint..r-15 . M,irk-. H
'J .ca tons useful in designing test cases ?
2. l-Vhat are the thing that · MSBTE : Winl.-r-16. M.irk" '1
. s test case specification shall identify ?
· 3. Describe test case sp ;~ .
eci,.cation of test process.
4. l-Vhat is test case 7 Wh · h · .
· ic parameters are to be considered while documenting test cases ?
MSHTI~ : Wint..r - 15 . 17 . I H. i\l.irk-. '1

[3.{j ~est Reporting

During testing constant communication between test team and development team takes place with the
help of a document called test report. ·
Various types of test reports are ~
(1) Test incident report (2) Test cycle.report (3) Test summary report
I3.4.1 ITest Incident Report
• Test incident report is nothing but an entry made in Defect Repository(DR).
• Each defect has unique ID and it is used to identify the incidents.
• A test incident report is a communication that happens through the testing cycle when the defects are
encountered.
• The high impact test incidents or defects are highlighted in the test summary report.

I3.4.2 ITest Cycle Report

Testing takes place in units of test cycles.

I
A test cycle report at the end of each cycle gives following things -
1) Description of activities carried out during test cycle.
2) Defects that are detected during the test cycle along with their severity and impact.
3) Progress in fixing the defect.from one cycle to next cycle.
4) Outstanding defects that are yet to be fixed.
5) Any variation in schedule.

ijA.3j .Preparing Test Summary Report

A report that summarizes the.results of test cycles is called test summary report.
There are two types of test summary reports -
1)_ Phase wise test s1.1Ill;111ary report which is produced at the end of every cycle.
2) Final test summary report: It contains following things -
A) Summary of the activities carried out d~g the test cycle
Technical Publcations - An up thrust for knowledge
 Test Mina~
Software Testing .3 . ·12

B) Variation in actual activities and planned

activities. Tius includes -
• Tests that are planned to run but could not
be run.
• Modified tests. \
• Additional tests that a.re required. 3.
• Difference in effort and time between ,I t art ? Write contents o
4. What are types O; tes r
planned and actual.
summary report.
t . in detail. How to yrrptare a
• Any other deviation from plan.
5. Describe test repor
summary report ? I t¥n
ml llllllll•ll
w: :::::r r lti ■lll'i
iliQ
C) Summary of results should include
• Tests that are failed with root cause. 6. Explain
'
. how summa I ,1
■ •···

M",H t f
ort is ,,.ared
,.,,.,
",,111111wr -
•
m test
1111~111••
planning.
• Severity of impact of uncovered defects.
D) Comprehensive assessment and
recommendation for release.

Technical Publications • An up thrust for knowledge

 ~~ ~'lra'ila;·
-- ar:1 .-.,wa~'/iar~._..~,,,.
• wlillllll;---._a~-&,
_
-:..~ _..
..__
~

~
q
[UNIT-IV
I =--- I

-
4 Defect Management

~
~ Defect Classlflcatlon and Managamant'
_,·, • · Defect is something b hi . . • th ··1
. . Y w ch the customer requirements do not get satisfied. A defect 1s basically e
difference betwee th
n e expected result and the actual result. . · ·1

Defect is a specific concem about the quality

. of an application under test.

l Defects are expensive

· because fin . ding
• and correcting defects· is supposed to be a comp1ex acti"V1·ty m
· .
software development. Defect is supposed to be realization of risks. .
Causes of defects

I Following are some important causes of defects -

1. The requirements defined by the customer are not clear. Sometimes development team makes some
assumptions.
2. The software designs are incomplete and does not accommodate requirements of the customer.
3. People working on product design, development and testing are not skilled.
4. The process that are intended for product design, development and testing are not capable of
producing the desired results.

Effects of defects

Due to the defects present in the system, customer has total dissatisfaction about the system. Moreover
there are some serious effects of defects as given below -
1. · Performance of the system will not be at the acceptable level.

2. Security of the system can be problematic and there are chances·of external attacks on the system.

3. Required functionality might be absent from the system· which may result in rejection of the system
by the customer.

I
&.1.1 Defect Classification

Various types of defects are

{4;;Jf > ...

..
 '~tl
YI
~So?,t,ftwa~~re:_2Ti~es~ti~'ngL_ _ _ _ _ _ _ _ _ _ _ _ _ _ ___: 4~_~2~-------========D::'.:eJl:ec:t=Ma=na~gernen::.:.:-!._

Fig. 4.1.1 Types of defects

Let us discuss them in detail The defects by which the variables do not get
1. Requirement defects : When the developers can initialized properly is a coding defect. pi

not understand the customer requirements 4. Testing defects ·: If the testing is not conducted
' i)
properly, the requirements defect occur. properly then testing defects occur.
The requirement defects can be further classified as - Various types of testing defects are -
u:
i) · Functional defect : If the customer expected 1. Test design defects: If the test plan, test cases, test ,
functionality is not present in the system then it scenarios and test data are not properly defined
is called functional defect. then test design defects occur.
ii) Interface defect: If the defect remain the module 2. Test tool defect : If there is defect in the test tool
when one module is interfaced with the other then it is difficult to identify and resolve that defect.
module then it is called interface defects. One has to conduct manual testing instead of using
2. Design defects : If the software design is not automated testing tools.
correct or created without understanding the Board Questions
requirements thoroughly, the design defects occur. 1. Which are different causes of software defects ?
The design defects occur normally due to the MSH II . : Winln-1 ;", , M,irk-. ,1

algorithm chosen for the design and the design 2. Explain defect classification.
MStHf . : Winier - I 5 . Mdrk-. '1 . Winln-1 Ii . M.irk-. (,
interfaces.
3. ~at ~ deject management ? Give defect classification
Algorithm defects : If the design of algorithm is
unable to translate the requirements correctly to the
in detail. ■JM;if F h@uiiid§~H•
4. Give the defect classification and its meaning.
coding then algorithmic defects occur. MSBTE : Summer- I 6 . Winter-I7. MMk-. ,1
Interface defects : Due to lack of communication 5. Describe the requir ·n
the interface defect occurs. When parameter from details.
one module does not get passed to other module 6. List all defect class
defect in brief.
correctly then interface defect occurs. Similarly due
to look and feel of the application and due to 7. Explain requirement defects and design defects.
MSHTE : Sumnwr-I S M.irk-. ti
navigation problem the user interface defects get
introduced in the system. I4.1.2 IDefect Management
3. Coding defects : ff the coding standards and design Defect management process can be carried out in
standards are not followed properly the coding v~ous phases as - defect prevention, baseline
delivery, defect discovery, defect resolution and
defects occur._For example
process improvement.
Technical PublicatioM · An up thrust for knowledge
 ~Testing_
---------
Defect Mana~t
4-3

Fig. 4·1-2 Defect management process ·

1. Defect prevention . D £
highest priority . .' . e ect prevention is the 3. Notify it to all the concern parties when defect
. activity m the d eiect
.c
management
process. This stage gets resolved.
. . t
suggeS s to prevent the 5. Process improvement : This is the most neglected
introduction of defects . th
F ll · m e software product step in defect management process. This . step
o owmg are the steps taken for defe~ suggests that participants should go back to the
prevention -
process that originated the defect to understand
i) Identify the cause of the d eiec
.c t
and reduce the what caused the defect. Then they should go back
occurrence of defect. to the validation process that should have caught
ii) There are some co~only occurring defects - the defect earlier in the process. Thus · treat the
such as coding defects or interface defects. Focus defects as an opportunity to improve the ~oftware
on these common causes of defects. development activities.
iii) Identify critical risks, assess it and minimize the 6. Management reporting : It is important that the
expected impact of the risks. defect information must be analyzed and
2. Baseline delivery : The baseline means the work communicated to both the project management
product which is in deliverable stage. The and senior management The purpose of collecting
deliverable is baselined when it reaches a such information is -
predefined milestone in its development. If newly 1. To know the status of individual defect.
created product satisfies the acceptance criteria, 2. To provide insight into the processes that need
then it is baselined. And only baselined product the improvement.
moves to the next stage. 3. To provide the strategic information to senior
3. Defect discovery : Defect discovery means defects management to make important decisions.
are brought to attention to the developers. The
Board Questions
defect must be identified before it gets a serious
1. lllustrate defect prevention process of defect fixing
problem. As soon as defect gets identified, it must
process with diagram.
be reported to the authority team in order to MSBTE : Summer-15 . l\ldrk, tl
resolve it. 2. Draw labelled diagram of defect management process.
4. Defect resolution : Once the developers have List any two characteristics- e c t maiiement
acknowledged a valid defect, the resolution process process. ■§@GiMk_~ M@•§Jij
begins. The defect resolution must be done in 3. Explain defect management .cess with
diagram. · •®MGi _ ,@$f.j§ ~-
i&er$
following steps - ·
4. Describe steps in defect management process.
1. Determine the importance of the defect. MSBTE : Summer-15 . M.irk, 6
2. Schedule and fix the defect as per the order of 5. Explain defect management process.
its importance. MSBTE : Summer- I 6 . Wintn-17 . M,nk-. tl

Technical Publications -An up thrust for knowledge

 Defect Management

6• Descn'be de•f.1 ec t manag_emen!...J!!ocess with neat and

4 4
-------------=~=:::~::::-
~:S~o--:ijt~wa~~r--:eii~e_s!ti~n-::-g~-=--;-~-:_-:_-:_-:_-:_-:_-:_-:_-:_-:_-:.:-:.:-=:-=:-=:-=:-=:-=:-=:-=:-=:-=:-=:-=:-=:-=:~~_~ amely Duplicate, Deferred
the beIow fOur states-based
n upon the specific '
labelled diagram.
7. Explain defect
diagram.
■41Hi-jjjjjjjjji4#•ddii
""'narrnenurocess
with suitable
•~HilL@ilijj@@§MR•
Rejected or Not a Bug
4) Fixed : When the developer finish:s tas: of:e
fixing a defect by making the require 'c . ai:1g,es t en
. · f the defect as Fixed .
!4.2 IDefect Life Cycle and Template he can mark the status
· .
0
. t the tester starts the task of
! 4.2.1 !Defect Life Cycle t
S) ReteS : At this pom '.
working on the retesting o
f the defect to verify if
Definition : Defect life cycle is .a cycle of defect in the defect is fixed accurately by the developer as
. which defect goes through different stages during its per the requirements or not.
lifetime. · If . still persists in the defect then
6) Reopen : any issue
The defect .life cycle starts as a soon as new defect is it will be assigned to the developer again for testing
found by a tester and comes to an end when tester and the status of the defect gets changed to
closes that defect assuring that it won't get reproduced
again: 'Reopen.'.
7) Rejected : If the defect is not considered as a
Defect States genuine defect by the developer then it is marked
as 'Rejected' by the developer.
8) Duplicate : If the developer finds the defect as same
as any other defect or if the 'concept of the defect
matches with any other defect then the status of the
defect is changed to 'Duplicate' by the developer.
9) Deferred : If the developer feels that the defect is
not of very important priority and it can get fixed in
the next releases or so in such a case, he can change
the status of the defect as 'Deferred'.
lO)Closed : When the defect does not exist any longer
then the tester changes the status of the defect to
'Closed'.

Board Questions
Fig. 4.2.1 Defect life cycle
1. Explain defect life cycle to identity status of defect with
1) New: This is the first state of a defect in the defect praper labelled diagram.
life cycle. When any new defect is found, it falls in a MSTE : Winter- I 5, Winter-17. Marks 4
'New' state and validations and testing are
performed on this defect in the later stages of the
2. Draw_ t~ diagram of Defect I Bu?
explam zts process. ■§M!•
lit
91..cle and
3 j!U',jj!~•§Mff j
defect life cycle. 3. E~lain the defect tracking with defect life cycle
2) Assigned : In this stage, a· newly created defect is diagram and the different defect states.
assigned to the development team for working on MSBTE : Summer-] 6, Marks 8
the defect. This is assigned by the proj~ct lead or 4. Describe defect life cycle with neat diagram.
the manager of the testing team to a developer. MSBTE: Summer-17, f8, 19, Winter-18 , M,irks 4

3) Open : Here, the developer starts the process of !4.2.2 !Defect Template
analyzing the defect and works on fixing it, if
required. If the developer feels that the defect is not Defect template is a document used by every
appropriate then it may get transferred to any of organization for capturing defect data.

Technic8/ Publications - An up thrust for knowledge

 L ■

.».i?fiJ.:I
ht//:., '• · f th · ·. •.
~[(i},:,rxt,,,,ti:l'~zcur m,ar1y ? ,· ,;;~ :'
!%ttPl1ases.'- , . The · defect '
Assigned-to :

fl]tefrrpJat~ gives info;m~tidnl)

{ :agou't ~hen defec.t has been '·
t ifittodu~d. If the tester is Board Questions

,not aware of the de_fect · 1. Explain the defect template with its attributes.
introducti on phase, the MSBTE : Winter- I 6. M,uk ... ti
> p~rson doing an analysis 2. Enlist any six attn'butes .§( de~. Describe them with
.·• inay add this defect · suitable ex~mple. ■§ ;Jf jjj.,hiiJIG§bdd
introduction in defect> 3. Explain defect template.
Ai{t$mplate. This also helps in MSRTE : Sumnwr- 18. Mcirk.., tJ
;};Ji\}nti,fying defect leakage _ 4. Which parameters are considered while writing good
f~x' ftQIO orie phase to another.
~- ,,'.;_-',)'
defect report ? Also write contents of defect template.
' ' 1e project when MSRTE : Winter- I 5 . 17 . Mcirk ... tJ
' tind is added
5. Describe defect template with its attriputes.
,:::-.:
MSRTE : Summer- I 9 . M,irk ... tJ

G] Impact, Technique and Reporting

• Actual impact of the defect can be realized when the
risk becomes a reality. But it is possible to estimate
the probable impact with some level of certainty.
• Some organize classify the risk as high,. medium
and low based on some model.

!4.3.1 IEstimate Expected Impact of a Defect

Following are some ways to handle the risks -
1) Accept the risk as it is : If there occurs some
natural disaster then there would not be any
solution for this. The actions or solutions to handle
Technical Publications "' -An up thrust for knowledge
 ~
~~~~~L______________
Software Testing _!~----------==~::i:D:ef.:ec~t
4-6
th ways w hi
Ma~na~g:eme,i:-!.
ch are already planned
such risk are very costly. In this case, customer or · • These are e . in mind that the preventive and
user is given information about probable failure actions by keeping . .
. might fail.
and effect of such failure. corrective actions
2) Bypassing the risk : The risk can be by passed I4.3.2 ITechnique•.for Flues
n
ding Defects
.
for findmg defects -
when user cannot accept the risks, or no action can There are three techniq . . .
be taken to reduce probability of risk. . . This is a technique m which the
1. Static technique ·
"th t executing the program. The
How to minimise risk Impact or probablllty ? testing is done Wl ou .
. . an example of static technique.
Minimization of risk impact can be done in following code review 1s .
. t chnique . This is testing technique in
manner 2. 0ynanuc e · t d. d
mponents are execu e m or er
which the system co · .
1) Ellmlnate risk: . .fy th. d fects Execution of test cases 1s an
to 1denti e e ·
• Risk probability can be reduced by reducing the example of dynamic testing technique. .
cause of the risk. 3. Operational technique : The system containing
• Although it is not possible to eliminate risk defects are delivered to the users . and then defects
completely, the probability of the risk . can be .are identified by the users, customers or auditors.
reduced to certain level. Thus ·defects are actually detected during the
• There are some preventive controls . that help in operation of the system.
reducing the probability of risks. These preventive
controls are usually decided by organizational I4.3.3 !Reporting a Defect
management. • Finding and reporting defects is an important step
·• If probability of risks ~re very high, then there is no in software development life cycle.
use of any preventive control but if the probability • It is necessary to find root cause of defect in all the
of risk is low then preventive controls become software development life cycle and prepare the
useful. document about it.
• Following are some to be noted in reporting
2) Mitigation of risk :
defects-
• Actions taken by organization to minimize possible ·
damage due to realization of risks are called o Give complete Record of inconsistency :
mitigation actions. ■ Complete description of defect help the testet
• The mitigation actions are planned by organization. and user to take preventive, and corrective
. ' actions about the defect.
• Corrective controls help in mitigation of risks.
■ Complete record description also helps in
3) Detection ability Improvement : process improvement.
• If people are prepared for the risk .then they can be o Defect- report forms a base of quality
handled effectively. measurement :
• Detective controls can be used to visualize the
■ Number of defects serve as a measure of
risks.
software quality. It must define severity,
·4) Cont~ngency planning : priority and·category of defects.

• Contingency planning means the actions initiated ■ More defects mean quality of software is
poor.
by organization when preventive or corrective
actions fail and risk actually occurs.

Technical Publications ~ An up thrust for knowledge

~Testing . . . 4-7

• Thus defect report 1s helpful in deciding the

quality of software.

rd Questions

~ Describe techniques for finding defects.

. ii~iii■MSii@W;s,.g;;;;;;;it§;94w§M&
2. Give any two root causes o de ects. Also ive an two
effects of defects.
3. What are the points. considered while estimating
impact of a defect ? Also explain techniques to find
defect in short.

4.
i~iii■M@dMtiMWISiWWNGiriiMN
State how to minimize risk impact while estimatinf
defect. •®MMR liiiilihiid3MW .
5. What do you mean by 'Defect Impact' ? Explain haw
to estimate the defect impact.
MSBTE : Summer- I 6, Marks 6

6. What are the different PfJ"U'J/7U,O"';f'.UJli'.Mj

defects ? · P91r1;Jlft't'WWPffPl_ P.
. 'l
□□□ ~\

l
J'
!

Technical Publications - An up thrust for knowledge

 '""""' *.:- _. -.. ·~··~i~!
.re iiiiE;••I ..c~ C
,,
L••
--..'1"""'·:,l'
- ••
;,.,_., ~,:,

~
[llNIT-V ]
--

5 .Testing Tools and Measurements

[iJ Manual Testing and Need for Automated Definition : Manual testing is the process of using the
Testing Tools functions and features of an application as an end-user
would in order to verify the software is working as
introduction
required. With manual testing, a tester manually
• Now a days, rigorous application testing is a critical conducts tests on the software by following a set of
part of all software development life cycles. The pre-defined test cases.
need of the same is greatly increasing as the
• In manual testing, testers manually execute test
organizations are developing mission - critical
cases without using any automation tools.
systems to support their business activities in day to
• It requires a tester to play the role of an end user.
day life.
• Any new application must be manually tested
Also it becomes necessary to ensure that these
before its testing can be automated.
systems are reliable, built according to specification
• Manual testing requires more effort, but is
and have the ability to support business processes.
necessary to check automation feasibility.
May internal and external factors are forcing the
organizations to ensure a high level of software • Manual testing does not require knowledge of any
quality and reliability. testing tool.
• One of the software testing fundamental is "100%
[1]J Manual Testing automation is not possible". This makes manual
l\lSBJ'I \Vi111<•1 - I:. . lh . 17. IX . S1111111wr - 19
testing imperative.
• To ensure the completeness of software , the testers
and the developers often follow a written test plan. !s.1.2 IAdvantages of Manual Teating
This leads them through a set of important test We have seen what is manual testing with its
cases. definition. Now, let's see what are the advantages of
• These test cases are physically exerted upon the using this technique.to test the software.
software to test many it for many factors. This is Some of the advantages are as listed below :
what is the manual testing is!!! 1. It is preferable for products with short life cycles.
• Manual testing is the process of manually testing 2. It is preferable for products that have GUis that
software for defects. It requires a tester to play the constantly change.
role of an end user whereby they use most of the 3. It requires less time and expense to begin
application's features to ensure correct behaviour. productive manual testing.
i • To guarantee completeness of testing, the tester 4. Automation cannot replace human intuition,
often follows a written test plan that lead~ them inference, and inductive reasoning.
through a set of important test cases.

(5-1)

.
 Testing Tools and Measurements
Software Testing 5-2
7. Manual tests don't scale well. As the complexity
5. Automation cannot change course in the middle of
a test run to examine something that had not been of the software increases the complexity of the
previously considered. testing problem grows exponentially. This leads-
6. Automation tests are more easily fooled than to an increase in total time devoted to testing as
human testers. well as total cost of testing.
7. It is preferable for products with s_h ort life cycle~. 8. Manual testing is not consistent or repeatable.
8. It is preferable for products that have GUis that Variations in how the tests are .performed as
constantly change. inevitable, for various reasons. One tester may
9. It · requires less · time and expense to begin approach and perform a certain test differently
productive manual testing. from another, resulting in different"results on the
10. Automation cannot replace human intuition, same . test, because the tests are · not being
inference, and inductive reasoning. performed identically.
11. Automation cannot change course in the middle of 9. Lack of training is the common problem,
a test run to examine something that had not been although not unique to manual software testing.
previously considered.
10. GUI objects size difference and color
12. Automation tests are more easily fooled than
combinations are not easy to find in manual.
human testers.
· te·sting.
13. Batch Testing is not possible, for each and every
11. Not suitable for large scale projects and time
test . execution Human user interaction is
mandatory. bound projects.
12. Batch testing is not possible, for each and every
I5.1.3 I Disadvantages of Manual Testing / Limitation of test execution Human user interaction is
Manual Testing MSBTE: Winter-15. 16. 17. 18

• In spite of having so many advantages of testing the mandatory.

software manually, this . technique is now being 13. Comparing large amount o f data is impractical.
obsolete in the market due to certain reasons. 14. Pro_cessing change requests during software
People need the work to .be finished faster. More maintenance takes more time.
work has to completed in less time with greater
Board Questions
efficiency.
-1. State limitations of manual testing. Write any four.
• · The limitations/ disadvantages of using the manual
MSBTE : Winter-15 . Marks 4
testing techniques are as follows :
2. Describe any four limitations of manual testing. ~

1. Manual Test Case scope is very limited. MSBTE : Winter-16 . Marks 4

2. Comparing large amount of data is impractical. 3. Write down any four limitations of manual testing.
~

MSBTE : Winter-17 , Marks 1

3. Checking ·relevance of search of operation is
4. State any eight limitations of manual testing.
difficult. u
MSBTE: Winter-18 . Marks 4
4. l'rocessing change requests during software 5. State various advantages and disadvantages of using
maintenance takes more time. manual testing tools.
5. Manual testing is.slow and costly. MSBTE: Summer-19. Marks 4

6. It is very labour intensive, it takes a long time to

c·oinplete tests.

m
-Technical Publications - An up thrust for knowledge
i ~"
~
 ------~
~ ;m;~~ ·----- -
~re-Testing_
~ comparison between 5-3 Testing__ Tools and Measurements
~ Automation Testing Manual Testing and
IL1M-m::::::sw:
AllhhliiiMl:t
Needs of automated testing tools can ·be listed as
follows:
1. Speed : Think about how long it would take you to
manually try a few thousand test cases for the
windows calculator. You might average a test case ,
every five seconds or so. Aut~mation might be able

j9f
q~ej{tiy~t:.; >:case·
· 'sF
?/;;-~! ·~~:'::·:· ~::~- ~ : '.:... to run 10, 100 even 1000 times that fast.
2. Efficiency : While you are busy _running test cases,
you can't be doing anything else. If you have. a test

--~--
i;~,li!~'
.;,•• . .• .• . ·... '.
tool that reduces the time it takes for you to run
your tests, you have more time for test planning
and thinking up new tests.
3. Accuracy and Precision : After trying a few
hundred cases, your attention may reduce and you ·
will start to make mistakes. A test tool will perform .
the same test and check the result perfectly, each
and every time.
4. Resource Reduction : Sometimes it can be
physically impossible to perform a certain test case.
The number of people or the amount of equipment
required to create the test condition could be
prohibitive. A test tool can used to simulate the real
world and greatly reduce the physical resources
Board Questions necessary to perf?rm the testing.
1. DifJ_erentiate between manual testi\$ and automation 5. Simulation and Emulation: Test tools are used to
testing. \ii" HMmS @111365• replace hardware or software that would normally
2. Give any four ·diff!rence between manual and interface to your product. This "face" device or
automated testing. MSBTE : Summer- I 8, Marks 4 application can then be used to drive or respond to
I
f 5.1.5 Need of Automated Testing Tools your software in ways that you choose-and ways
that might otherwise be difficult to achieve.
MSBTE : Winter-15 , Summer-16. 19

i. An automated · testing tool is able to playback pre- 6. Relentlessness : Test tool and automation never
recorded and predefined actions, compare the tire or give up. It will continuously test the
software.
results to the expected behavior and report the
success or failure of these manual tests to a test Board Questions

engineer. 1. Which different benefits help to recommend automated

testing ? Write advanta£es of switchin£ to automated
ii. Once automated tests are created they can easily be
testing.
repeated and they can be extended to perform tasks
2. What is software test
. impossible with manual testing. automation tools.
iii. Because of this, savvy managers have found that 3. Which types of test
automated software testing is an , essential automation? Why?
component of successful development projects. 4. Explain the need of automation testing.

,.
' .
MSBTE: Summer-19. Mark!> 4

Technical Publications - An up thrust for knowledge

• . . ..... ,

50
/
Testing Tools and Measurements
Software Testing 5-4 - ,l ' l
. . t Ex~cutionttn·: Automated testing
I5.1.6 IWhy Automated Testing ??? 6. Distributed TeS
'th distributed tes g
feature. One can
• Difficult to test for multi lingual sites manually · comes WI •pts on more than one
. ·1 xecute the test sen
• Does not require human intervention easi .Y e . hared network or server
. computer on a s 1 •
• Increases speed of test executi~n . So more than one too 1s not
simultaneous1Y· ' . 1 •
• Increase test coverage d d b t nly one automated testing too will be
· neee, uo ·
• Manual testing can become boring and hence error d d
nee e .
prone. R b t Reporting : Automation testing
7. Easy and o us . ch
r;--:i
L=!J Advantages and Disadvantages of using· Tools tools have this amazing · benefit of tracking ea and .
r;-:;-;, ·
·every • test scnp. t · Each and every test scnpt·
~ Advantages of using Tools . visual logs. These reports
J\ISH 11 .. S11111111<>1 -16 . 17 . I~ . Winll'r-16 I7 executed can be seen m
can dearly show the no. of test scripts already
There are many benefits of using the testing tools for
executed, scheduled, their reported bugs and how
supporting the software testing process. They are :
they had.been fixed.
1. It Saves Time : Most of the testers find problems
with the time required to write long test scripts to 8. Testing .· Capabilities : When it ~omes to
perform .testing especially when it is regression capabilities, automated testing tools can test the
testing. This takes a lot of time of the testers and the web applications on the various browsers available
delivery of the bug-free application is delayed. in the market via browser testing automation.
A· delayed product delivery is not good for any Also, wheri it comes to mobile application testing,
business. o~e can test them on various devices. This is next to
2. Checks Quality: Once manual testing procedures impossible to achieve with manual testing.
are finished, if one applies automated testing 9. Improves Test Coverage : Many a time, there are
processes, it helps to cross-check the test results. test cases with more than thousand lines of code ·
Thus, improves the quality .of the · manual test and to write it and test it would be very difficult
scripts. with manual testing. This can be easily done with
3. Early Bug Detection : While developing a software, automated testing. Also, these tools will make sure
bugs are easily found when software testing is about the in and out of the applicat;ion like the
perfonned via automated software testing tools. databases, UI, web services, etc. works according to
This can save a lot of time and efforts on the SDLC.
the requirements. Thus, improving the overall test
4. Perlorming Tests 24/7 : The tests are exerted coverage.
without stoppage day and night. The machines and
10.M~power Utilization : This is relative to the
tools never get tired. Whatever may be the
above benefits. · Implementing test automation in
quantity if testing, can be performed without any
kind of human intervention and continuously. The testing processes wili require les~ manual efforts.
test results are also produced automatically after Thus, it will decrease the number of people on just
the tests are performed. one particular project and can utilize them in the
5. Reusability : This goes perfectly very easy fo various testing projects.
understand with automated testing. When the have 11.Improves Team Motivation: This is a big challenge
test scripts are already prepared using test in the testing industry. When the manpower is
automation tools, they are saved for the · future distributed among various testing projects, the
requirements. So, they can be _ utilized as many
team can learn about new challenges, new bugs,
times as the testers want especially for automating
new skill sets, etc. ·
. regression testing.

T-- Technical Publications -An up thrust for knowledge

 •··
.
' - ~ -...;,-~ ll:Wj\. .~
,;~1~~:~ ~~:-- - D J:; .,,_.. _
~
-sp/!!!!!!.re Testing

Testing Tools and Measurements

lZ Testing Flexibility : The automated testing tools s -s condition, it will wor WI . • for
. . k .th the same eff1C1ency
are developed by the teams who have been into the long time.
manual testing for years; So, these tools are going to
be flexible to match the future testing specifications. 19.Quality -Software Automation can allow us to
These testing tools can be utilized for longer spend more time to spend on adding the new
periods of time - say for years. features and improving the quality of the software.
l3, Improves Accuracy : Here, no teste~ is being Instead of wasting the time on testing manually,
blamed. A tester with more than 10 years of one can think of increasing the grade of the
·experience can also make mistakes when they have software.
to prepare the same old boring manual test scripts 20. Cost Factor : Automation in testing can replace the

a·g ain and again. When this done with automated manual repeated tasks. Ultimately it will reduce the
testing, not only the results are accu~ate, but also costing of the project.
·saves time. 21. Correct Estimation After some iteration of
, 14. Overcomes Failures of Manual T ti Th _automation testing, person would be aware of the
_ . _ es ng : ere are
times h · exact time required for the execution. So ftte correct
w en accurate . test results are .next to
impo_ssible with manual testing·, 11.ke m
. the stress estimate with more accuracy can be given .
testing, test automation has proved to be a benefit Board Questions
here.
1. Elaborate the advanta,es (an"'ur) o' usi; the test
15. Improves ROI : The most important benefit is the automation tools. $hHQ\hiil]i# $65•
return on investment. Obviously,·when planning to 2. Explain three types of product metrics.
MSBTE : Sumnwr- 17 . M.nk-. tl
invest in automated testing tools, first thing
3. Give any two advantages and any two disadvantages
needed is to figure out how it will be benefited of using testing tools
with those tools in terms of ROI.
MSHTE : Summ er- I 8 . 1\1,irk-. 11
The cost of manual testing includes th~ time, cost of
manual hours and the efforts of the testers, QA
!
·, s.2.2 Disadvantages of using Testing Tools

managers, etc. And, if the · automated testing tools Though the automation testing has many advantages,
are available, testing will be faster, easily, it has its own disadvantages too. Some of the
disadvantages are :
efficiently, accurately and would be delivering bug-
free application within the delivery time period. 1. Proficiency is required to write the automation test
scripts.
_16. Repeatable : Execution of the same set of
automation test cases is possible. That's too without 2. Debugging the test script is major issue. If any error
any human intervention and faster. is present in the test script, sometimes it may lead
to deadly consequences.
17. Reusable : Re-we of the same set of automation
test cases on a different version of an application is 3. Test maintenance is costly in case of playback
possible. Also the same version of an application methods. Even though a minor change occurs in the
on a different browser can be tested wi_th the same GUI, the test script has to be rerecorded or replaced
by a new test script.
test cases.
18. Reliable : Automation is no~g but the program. 4. Maintenance of test data fil~s is difficult, if the test
script tests more .screens.
·If it is written as per expectation, then there are no
chance automation program will do anything 5. Test automation requires lot of efforts at initial
which is not written. If any program is in working stage.

T* Technical Publications -An up thrust for knowledge

.... .. ··- --- -·:11---· •- ;:• :,,.,-,~, lltiia'M\\\\.,.,M

-a a:-,c 1 ~; iiiiilll
f)f,-~ ·; ..,...~,.t'_____
!I _
,,
50±
:;::-
Software
· Testing

In software testing two important tasks, one is test

5-6
-·
Testing Tools and Measure'"-!

Selenium supports web application test automation

··~~s I:

only, doesn't support desktop / windows based

, design and another is test execution, For test design
user (Tester) interaction is mandatory, Testers only applications.
For manual testing no environment limitations, we
create test scripts using Test Tool features and ·
can test computer software or mobile software on
Programming features, It takes more time than
manual test case design. any operating system and any browser.)
12. Not suitable for dynamically changing UI designs. C
6. 100% Test automation is impractical._
Generally it is tried to automate maximum test Most of the test tools support test automation based
cases, not all Test cases. For some test human user on · front-end objects, if User' Interface design 1
observation is required and is must. Due to some changes dynamically then it is difficult to automate.
environment limitations we can't automate all 13. Wrong expectations : Team l<eeps a lot of
testable requirements. expectation from the automation tool. We need to
7. All types of Testing not possible (Ex: Usability). understand it is just a program which will do what
Functionality tests, performance Tests can be it is asked to do ..
automatep., but it is not possible to automate tests 14. Wrong Costing : In initial days of framework
that verify user friendliness of the system(AUT). development and converting manual test case in
8. Debugging issues : Programming syntax/logic is automation test cases takes time. And as this is the
used to write tests,. some times locating errors in first time testing team implementing the
test script is difficult. It is difficult to write such test automation, this can lead the inaccurate estimation.
cases. 15. Results false sense of quality : Automation program
9. Tools may have their own defects. will do only things which have bee!\ developed. As
i,
Test tool is also a software, it may have i~s own automation does not have its own brain to sense the
defects in it, so that we may not achieve desired defect and test accordingly.
benefits. 16. Not a replacement of actual testing : It is not the )
10. Programming Knowledge is required. replacement of actual testing, Manual testing can be ·~

Every test tool uses any one of the programming more interactive and innovative while testing.
languages (Example UFT supP,orts VBScript, Innovation from the automation while execution
Selenium supports Java, Perl, PHP, C#, PHP and cannot be expected.
Ruby) to write test scripts. So in order to create and 17. Maintenance time : If an application changes, it is
• edit · test scripts programming knowledge is needed to spend separate time to update the
mandatory. · automation scripts. ·
In manual testing, no programming knowledge is 18. Less number of bugs identified : Automation will
required. execute same test cases again and again, so over the
11. Test tools have environment limitations. period of time Automation will become less
Test tools have some compatibility issues with effective.
operating systems and browsers etc. Some of the above disadvantages often cause
Example: damage to the benefit gained from the automated
QTP supports windows operating environment scripts. Though the automation testing has pros
only, doesn't support other operating environments and corns, it is adapted widely all over the world.
like UNIX, Macintosh etc ...

¥~ Technical Publications - An up thrust for knowledge

~

..
 - 'l>!l !lRllf- -i. _~·-·•··-------
· ,~Testing ements·
5-7 Testing Tools and Measur
saard questions · · an also be
iv. Other than software code, static ana1ysis c
1. What ared the advantages and d zsa
. d · '
vantages ol using carried out on things. · like, static analysis of
automate tools for testing? List " · of websites (for
• ana1ys1s
· any two each. requirements or static
. IMii■MWMWIMGMiiii example, to assess for proper use of accessibility
2. What is automated testing 7 W ·t d
• · rz e own advantages
of using automated testing tools . ,., • . tags or the following of HTML standards).
m so, tware testing
IUfo ■ MtiMMS,; v. Static analysis tools for code can help the
-------- ■11nms111

,
ai1J Features of Test Tool . Guid r
Dynamic Testing Tool~
·
e ines for Static and
developers to understand the structure of the code,
and can also be used to enforce coding standards.
MSRTE : S11nm1Pr- I 5 . 16 . 19 Features or characteristics of static analysis tools are :
Test tools : Software testing tools .. Too1s from a • To calculate metrics such as cyclomatic complexity
software testing context can be d efi ned as a. product or nesting levels (which can help to identify where
that s_upports o~e or more test activities right from more testing may be needed due to increased risk).
pJanru~g, requirements, creating a build, test • To enforce coding standards.
execution, defect logging and test analysis.
• To analyze structures and depenp.encies.
Classification of tools : Tools can be classified based
• Help in code understanding.
on several parameters. They include :
• To identify anomalies or defects in the code.
, The purpose of the tool
2. Dynamic Testing Tool MSRTE: Summer-15. WintPr-18
• The activities that are supported within the tool
i. Dynamic analysis tools are 'dynamic' because they
. • TI:te type/level of testing it suppo:i:ts require the code to be in a running state. They are
• The kind of licensing (open source, freeware, 'analysis' rather than 'testing' tools because they
commercial) analyze what is happening 'behind the scenes' that
• The technology used. is in the code while the software is running
(whether being executed with test cases or being
This section deals with the major classification of
used in operation).
testing tools and how they are used.
ii. Let us take an example of a car to understand it in a
1. Static Testing Tool better way. If you go to a showroom of a car to buy
i. Static analysis tools are generally used by it, you might sit in the car to see if is comfortable
developers as part of the development and and see what sound the doors make - this would be
static analysis because the car is not being driven. If
component testing process. The key aspect is that
you take a test_drive, then you would check that .
the code (or other artefact) is not executed or run
how the car performs when it is in the running
but the tool itself is executed, and the source code
mode e.g. the car turns right when you turn the
we are interested in is the input data to the tool.
steering wheel clockwise or when you press the
iL- These tools are mostly used by developers. break then how the car will respond and can also
iii. Static analysis tools are an extension of compiler check the oil pressure or the brake fluid, this would
tecfulology - in fact some compilers do offer static be dynamic analysis, it can only be done while the
analysis features. It is worth checking what is engine is running.
available from existing compilers or development Features or characteristics of dynamic analysis tools
environments before looking at purchasing a more are as follows :
sophisticated static analysis tool. i. To detect memory leaks;
ii. To identify pointer arithmetic errors sucll as null
pointers;
Technical Publications -An up thrust forknowledge

hr
 Testing Tools and Measur~
Software Testing 5-8
zers : It ensures_that all logic
b. Coverage anal Y
Hi. To identify time dependencies.
aths are tested. .
• Eventually when your computer's response time P al r . It examines the effects of
I terface an yze · ·
gets slower and slower, but it get improved after c. n . bl and data between modules.
passing vana es .
re-booting, .this may be because of the 'memory Th find unused code and code
leak', where the programs do not correctly d. Path tests : ey
with contradictions,
release blocks of memory back to the operating . . easurement tools can be used
system. Sooner or later the system will run out of • Code complexity m .
complexity of a given code.
memory completely and stop. Hence, rebooting to measure the
filing tools can be used to
restores all of the memory that was lost, so the Similarly, data-pro
• b . Code-profiling tools can be
performance of the system is now restored to its optimize a data ase. .
. . de Test-generators are used
normal state .. used to optiIIUZe co ·
- I fonn code . Syntax-
for generating a test p an
• The~ tools would typically be used by
developers in component testing and component checking .to~ls are used to verify correctness of
integration testing, e.g. when testing code.
middleware, when testing security or when 2. t Dynamic test tools
looking for robustness defects. Dynamic testing tools are used at different levels of
• Another fonri of dynamic analysis for websites is testing starting from unit testing and which may go
to check whether each link does actually link to up to system testing and performance testing.
something else (this type of tool may be called a • These tools are generally used by tester.
'web spider'). The tool does not know if you
• These tools test the software system with _live
have linked to the correct page, but at least it can
data. e.g.
find dead links, which may be helpful.
a. Test driver : It inputs data into a module -
Testing tools can be classified into following two
categories : under-test (MUT)

1. Static test tools 2. Dynamic test tools b. Test beds: It simultaneously displays source .
code along with . the program under
1. Static test tools
execution.
.
Static
.
testing tools.are used during static analysis of
'
a system. c. Emulators : The response facilities are used to
·They do not involve actual input and output. Static emulate parts of . the system - not yet
testing tools : are used throughout a software developed.
development life cycle, · e.g. tools used for d. Mutation analysers The errors are
ve~fication purposes. deliberately fed into the code in order to test
• there are many varieties of static testing tools fault tolerance of the system.
used by different people as per the type of • There are many different tools used for dynamic
system being developed .. .testing.
• These tools do not involve actual input and • Some of the areas covered by testing fools are :
output. Rather, they take a symbolic approach to 1. Regression testing using auto~ated tools.
, testing, i.e. they do not test the actual execution
2. Defect tracking and commurucation
. . systems
of the software. e.g.
used by tracking and communication.
a~ Flo_w analyzers : Ensure tl}e consistency in
Performance, Load, stress-testing tools.
data flow from input and output.
hi
Technical Publications - An up thrust for knowledge

L
 Testing MeiJSUremetJfS
Testing Tools and
5-9
QuestJona _ f available in
Explain· needs of automatiQn testmg
. Also, does the tool have all eatures .

R~iiiF► i the trial version ? d

5) Is the current tool version stable ? Is the ven or
. •th d customer support
company established WI goo al ?
as well as online help resources and user manu . .
6) How is the tool learning curve 7 Is the learning
time acceptable for your goals ?
7) Do you want automation tool for only your
project needs or you are looking for a common
. . y ? It would be
tool for all prOJects m your compan ·
[rj] Selecting a Testing Tool a good choice if you select a tool that supports
l\lSHI!: : Su11111H•r - 15 . 19 . \Vi11h•1 - 15 . IS most of the coding languages on your projects.
~ Automation testing success largely depends on the 8) Which testing types does it support ? A tool which .
selection of right testing tools. It takes a lot of time supports maximum testing types (Unit, functional,
. · to evaluate relevant automation tools available in regression etc.) is always a better choice. Warning
·, : the market. But this is a must one-time exercise that - Don't go for a tool just because it is supporting
-will benefit your project in long run. all testing types. It's also important that the tool
should be powerful enough to automate your .
~ .There were few situations where I got the chance to
complex requirements.
' . review and select automation tool for my proj~cts.
9) Does the tool support easy interface to create and
The task was difficult as we had to manage our.
maintain test scripts ? Record and playback tool
testing needs and cost restrictions but it was a worth
with abilities to edit recorded scripts could be a
experience.
good solution.
··.Here .are the criteria you need to consider before
10) Does it provide simple interface yet powerful
-selecting any testing tool :
features to accomplish complex tasks?
, Test Automation Tool Evaluation Criteria
11) How easy is it to provide input test data for
Do you have the necessary skilled resource to complex or load tests ? A tool supporting test data
allocate for automation tasks ? input from various data files such as Excel, XML,
2) What is your budget ? text file etc. .would be a big relief for the_
3) . Does the tool .satisfy your testing needs ? Is it automation the testers.
suitable for the project environment and 12) Does it provide the powerful reporting with
technology you are using ? Does it support all graphical interface ? dear and concise reports will
tools and objects used in the cod e : S metime you always help you to conclude the · test results.
may get stuck for small tests cil;" "O ;:1abi lities of quickly.
. the tool to identify the obic'c::- :1sed 11 the 13) Does it integrate well with your other testing tools
~pplication. like project planning and test management tools ?
Above three factors are considered as most You may also want to consider other criteria like :
important for selecting any tool. The factors 14) Tool vendor refunq policy
include: 15) Existing customer reviews for the tool
·4) Does the tool provide you the free trial version so 16) Is the vendor providing initial traiI)ing ?
that you ~ evaluate it before making a decision ?

Technical Publlcstions - An up thrust for knowledge

.;;
- - - -- -- - - - -- - - - - - -- -- -· -- ------- - -
 Software Testing 5-10
Testing Tools and Mmsurtmtnta
-
i. While introducing t:he tool in the organization Board Question•
1.
.
·
Enlist factors considered for selecting tesY!
.
toolt
.

-■..SMA::!ii~M•
it must match a need within the organization,
and solve ~t need in a way that is both test automatwn. ,--·-
. Mnt es for selecting static test tools ? Also
effective and efficient. 2 Which are r- ur
. list any two available test tools (static) .
.ii. The tool should help in l;>uilding the strengths
of the organization and should also address its E 1. t factors considered ~ selecti'!S__a testiM tool for
weaknesses.
3· n rs . ·
test automation.
phi&&&j;ilMM
-------
• · The organization needs to be ready for the changes 4. Explain criteria to select testing tool.
l\lSHII S111111111•1 - l'J '.\l,11I,, I
that will come along with the new tool.
i. If the current. testing practices are not good I5.4 IWhen to use Automated Test Tools, Testing
·enough and the organization is not mature, then using Automated Tool
it is always recommended to. improve testing I5.4.1 IWhen· Does Test Automation Make Sense ?
practices first rather than to try to find tools to
Automated tests are a very useful and impressive tool
support poor practices. . Certainly, we can
used to make testing more efficient. However,
sometimes improve our own processes in
automated tests are not suited to all projects - this may
parallel with introducing a tool to support those
be due to lack of time available, or 4ue to technical
.practices and we can always pick up some good
limitations.
ideas for improvement from the ways that the
tools work. Automated tests take time.to create. Depending on the
testers, it can take 3-10 times the amount of time to
ii. Do not depend on the tool for everything, but it
create an automated test as it takes to run the same test
should provide support to your organization as
manually. Therefore, automated tests will only start to
expected.
be valuable when they are run more than 3-10 times.
The following factors · are important during to.ol
Automated tests are suitable for the following
selection:
purposes:
i. Assessment of the organization's maturity (e.g.
readiness for change); - Regression testing for a stable system that will be
ii. Identification of the areas within the organization run on a regular basis.
where tool support will help to improve testing - Fast data creation in test systems where the
processes; database must be wiped on a regular basis.
iii. Evaluation of tools against clear requirements and i. When there.are many repetitive tests.
objective criteria; ii. When there are frequent regression ~sting
iv. Proof-of-concept to see whether the product works iterations.
as desired and meets the ~uirements and
iii. When you need to simulate large number of
objectives defined for it;
users who are using the application resources.
v. Evaluation of the vendor (training, support and
iv. When AUT is having comparatively stable UI.
other commercial aspects) or open-source network
of support; v. When you have large set of BVT cases.

vi. Identifying and planning internal implementation vi. .When you can't rely solely on manual test
(including coaching and mentoring for those new execution for critical functionality. .
to the use of the tool).
 ~Testing _,1 MetlB"mr,ents
Testing Tools arw
,: 5-11
• Automated tests are NOT su'tabl
1 . fo .
e r the following Type• of Framework• : . orks
purposes: tin framew
Typ1'cally, there are four test automa o · · the
- Testing new functionality thi h
- s s ould be done . that are. adop te d popular while automating
manually before automated te ts
s are created. applications : .
- Regression testing systems that
. are expected to . • Data Driven Automation Framework
have significant user interface .changes. Large
• Keyword Driven Automation Framework
~ges to the user interface require a lot of
• Modular Automation Framework
mamtenance for automated tests.
• Hybrid Automation Framework.
. ·.
When automating tests' 1·t is
· ·
wise to only automate as
rnany tests as your team can easily maintain. If some Tool• that are uaed for functlonal automation :
tests are b_eco~g difficult to maintain, it may be
worth cons1denng retiring those tests.
· Above all, reme~ber that automated tests will never
detect as many bugs as a human tester executing the
same steps. This is because the human tester's eyes can
pick up many things, whereas the test will only notice
, , what it is programmed to notice.
Popular tools that are used for non functlonal automation :
15.4.2 I Testing using Automated Tools
(Test Automation)

What Is Test Automation?

• Software test automation makes. use of specialized
tools to control the ~xecution of tests and compares
the actual results against the expected result.
Usually, regression tests, which are repetitive IS.S IMetrics and Measurement : Types of Metrics,
actions, are automated. Product Metrics and Process Metrics,
Object Oriented Metrics In Testing
• · Testing tools not only helps us to perform l\lSH 11 S111111111'r 1 S 17 \\ 11111 •1 17
regression -tests but also helps us to automate data • In ,recent years software testing technologies have
set . up generation, product installation, GUI emerged as a dominant software _engineering
interaction, defect logging, etc. Automation tools are practice which helps in effective cost control, quality
used for both functional and non-functional testing. improvements, time and risk reduction etc.
Criteria lot Tool Selection : • The growth of testing practices has required
For automating any application, the following software testers to find new ways for estimating
parameters should be considered : their projects. A key researdt area in this field has
• Data driven capabilities been 'measurement of and metrics for' the software
• Debug~g and logging capabilities testing.
· • Platform independence · • Measurement since plays a critical role in effective
• Extensibility and customizability and efficient softw~ development, making
· • E-mail notifications measurements of the software development and test
• Version control friendly process is very compl~x as shown in the Fig. 5.5.1.
, • • Support unattended test runs.
Techtllcal Publlcetiona - An up thrust for _knowledge

.-
------· ----r
 Testing Tools and Measuremen!!_

11. Tracking ta•t metric• .

. the metrics the next step 1s to track
• A fter ·d efinin g .
the metrics.
· a constant activity so it's always
• Since· tracl<lng 1s . ·
nice to automate the tracking Part ·
. reduces time required to track the
• Automation
metrics, analyze them and measure th~-
111. Reporting
• Reporting of the metrics is the most important step,
you should report test metrics to stakeholders so
that they have clear picture of project progress.
Fig. 5.5.1 : Metrics
• A metric is a quantitative measure of the degree to
11

1s.s.11 Metrics which a system, system component, or process

Software Metrics : Metric is a standard unit of possesses a given attribute. Metrics can be defined
measurement that quantifies results. Metric used for as "STANDARDS OF MEASUREMENT". Software
evaluating the software processes, products and metrics are used· to measure the quality of the
services is termed as software metrics. project. Simply, metric is a unit used for describing
Definition of Software Metrics : Software metrics is a an attribute. Metric is a scale for measurement. "
measurement based technique which is applied to • "How many issues are found in thousand lines of
processes, products and services to supply engineering
code?", here No. of issues is one measurement and
. and management information . and working on the
No. of lines of code is another measurement
information supplied to improve processes, products
Metric is defined from these two measurements.
and services, if required.
Importance of Metrics Metrics Llfecycle

• Metrics is used to improve the quality and The process involved in setting up the metrics· :
productivity of products and services thus
achieving customer satisfaction.
• Easy for management to digest one number and
drill down, if required.
• ,Different Metric(s) trend act as monitor when the
process is going out-of-control.
• Metrics provides improvement for current process.
• In software testing there are three main areas which
needs to be considered while thinking about metrics
and measurement.

I. Defining the metrics

• .Small and quality set of metrics should be chosen,
large set of metrics should be avoided as it is very
confusing to understand large set of metrics.
• Metrics should also be uniform and everybody in
· team should agree with it. Fig. 5.5.2: Metrics llfecycle
TechniciJJ Publications -An up thrust for knowledge

"· ... ... ,.... ... ··-- .. . . ...........

 lf!!:!!!!_;Testing . Testing Tools and Measurements
5-13
~ What Is Software Test M
.~ eaaurement? For Example : A Test Analyst has to,
•
J·
Measurement is the quantitati · • .
extent, amount d · · . ve indication of i. Design the test cases for five requirements
, unens1on capa .
some attribute of a prod ct ' cty, or size of ii. Execute ·the designed test cases
~ or process.
iii. Log the defects and need to fail the related test
ii; Test measurement exa 1
· defects. mp e : Total number of
cases
iii, Please refer below Fig 5 _5 3 f . iv. After the defect is resolved, need to re-test the
of the difference b · · or clear understanding defect and re-execute the corresponding failed test
metrics. etween measurement and
case.
• In_above scenario, "if m~trics are not followed, then
the work c~mpleted by the test analyst will be
subjective i.e. the test report will not have the
proper information to know the status of his
work/project.
• If metrics are involved in the project, then the exact
status of his/her work with proper numbers/data
can be published.
For example, in the Test report, we can publish:
1. How many test cases have been designed per
requirement ?
· 2. How many test cases are yet to design ?
.Fig. 5.5.3 Example of measure and metric
3. How many test cases are executed?
Why Test Metrics ?
4. How many test cases are passed/failed/blocked ?
Generation of software test metrics is the most
important responsibility of the software test 5. How many test cases are not yet executed ? ·
lead/manager. 6. How many defects are identified and what is the
. Test Metrics are used to, severity of those defects?
i. Take the decision for next phase of activities such 7. How many test cases are failed due to one
as, estimate the cost and schedule of future projects.
particular defect ? etc.
ii. Understand the kind of improvement required to
• Based on the project needs we can have more
success the project.
metrics than above mentioned list, to know the
iii. Take decision on process or technology to be status of the project in detail.
modified etc.
Based on the above metrics, test lead/manager will
Importance of Software Testing 1Metrics :
get th e understanding of the below mentioned key
i. As explained above, test metrics are the most
points:
important to measure the quality of the software.
a) Percentage of work completed
ii. Now, how ~an we . mea~ure the quality of the
software by using Metrics ? b) Percentage of work yet to be completed
iii. Suppose, if a project· does not have any metrics, c) Time to complete the remaining work
then how the quality of the work done by Test a d) Whe~er the project is going as per the schedule or
·analyst will be measured ? lagging ? etc. . ·
· Technical Publications • An up thrust for knowledge

.:Z:&&.::::wwws ...:a::e
~.:".l'M
 Software Testing

• Based on the metrics, if the project is not going to

complete as per the schedule, then the manag~r will
raise the alarm to the client and other stake holders
by providing the reasons for lagging to avoid the
last minute surprises.
I5.5.31 Types of Metrics
MSB fl. : S1111111wr- I 6 . 18 . WiutPr-18

!5.5.3.1 !Types of Manual Test Metrics.

Testing Metrics are mainly divid_ed into two categories.
1. Base Metrics : Base Metrics are the Metrics which
are derived from the data gathered by the test
analyst during the test case development and
execution.
This data will be tracked throughout the test life Importance of metrics and measurement In SDLC
cycle. i.e. collecting the data like, total no. of test i. During all the software development life cycle it is
cases developed for a project (or) no. of test cases very important to apply metrics and measurement
need to be executed (or) no. of test cases because metrics and measurement set expectations.
passed/failed/blocked etc. If there are well established metrics and
2. Calculated Metrics : Calculated Metrics are derived me.asurements in project then the test analyst can
from the data gathered in base metrics. These easily track and report quality results to the
metrics are generally tracked by the test management.
lead/manager for test reporting purpose. ii. If the metrics and measurements are not established
f5.5.3.2 / Examples of Software Testing Metrics properly then the as_sessment of software quality is
purely subjective which arises disputes at the end
Let's take an example to calculate various test metrics of development life cycle. You can consider some
used in software test reports :
the following areas where you can apply metrics
Below is the table format for the data retrieved from and measurement. This list is not exhaustive, you
the test analyst who is actually involved in testing : can have metrics for lot more things :
1) Schedule of project
2) Coverage
3) Planned and actual cost
4) Workload and resource usage
5) Product risk and project risk
6) Defects
iii. While doing test plannil}g we set the expectations
for the stakeholders or we set the baselines for
them. If you have established the baselines the test
reporting is consistent to the management and you
can avoid subjective assessment of testing.

W1f1" Technical Publications - An up thrust for knowledge

:it
 5-15 Testing Tools and Measurements
,: r,ltlons and Formulas for C
· 8 1curating Metrics •
Software quahty . metncs· are a subset of software
11 percentage test cases execut d . . ·
µ e -Thismetri · metrics that f~cus on the quality aspects of the product,
to obtain the execution t tu c 1s used
s a s of th process, and project. These are more closely associa~ed
terms of %ge. · e test cases in
with process and product metrics than with proJect ,
%ge Test cases Executed _ .
executed I Total no. of test - (No.' of Test cases metrics.
. cases wntten) ,., 100 Software quality metrics can be fu rther d'~vi'd ed into ·
So, from the above data, ·
three categories -
%ge Test cases Executed = (6S /
~. . _ 100)"'100=65% • Product quality metrics
.Percentage test cases not executed . Thi . . . • In-process quality metrics
used to obtain the • · s metric is
. pend mg execution status of the • Maintenance quality metrics
test cases m terms of %ge.
'- %ge test cases not executed _ (N f 1. Product Quallty Metrics
- o. o test cases not
executed I Tot~l no. of test cases written) ,., 100. This metrics include the following -
So, from the above data • Mean Time to Failure
I

• Defect Density
%ge Test cases blocked = (35 / 100) * 100 = 35 %
• Customer Problems
Board Question • Customer Satisfaction.
· 1. Define metrics and . measurements. Explain need of Mean Time to Failure : It is the time between failures.
software measurement. This metric is mostly used with safety critical systems
MSBTE : Summer- I 5. Winter-17. Marks 4
such as the airline traffic control systems, avionics, and
I5.5.4 IProduc;t Metrics and Process Metrics, Object weapons.
· Oriented Metrics in Testing Defect Density: It measures the defects relative.to the
l5.5.4.1 lProduct metr~cs and Process metrics software size expressed as lines of code or function
point, etc. i.e., it measures code quality per unit. This
MSBTE : Winter-16
metric is used in many commercial software systems.
· -Software metrics can be classified into three categories Customer Problems : It measures the problems that
Product metrics - Describes the characteristics of customers encounter when using the product. It
the product such as size, complexity, design contains the customer's perspective towards the
features, .performance, and quality level. problem space of the software, which includes the non-
defect oriented problems together with the defect
• Process metrics - .These characteristics can be used
problems.
to improve the development and maintenance
The problems metric is usually expressed in terms of
activities of the software. Problems per User-Month (PUM).
• Project metrics - This metrics describe the project PUM = Total Problems that customers reported (true
characteristics and execution. Exampl~s ~elude the defect and non-defect oriented problems) for a time :
number of software developers, the staffing pattern period + Total number of license months of the
over the life cycle ~f the software, ~ost, schedule, software during the period
and productivity. Where,
Some metrics _b elong to multiple categories. For Number of license-month of the software = Number
example, the in-process quality metrics of a project are of install license of the software x Number of month.s
··: both process metrics and project metrics. in the calculation period

Technical Publications -An up thrust for knowledge

··-- - --- - ----

 Testing Tools and Measuremen!!.,
Softmm, Testing 5-16
0
is still being tested. It is especially useful to monitor
PUM is usually calculated for each month after the 11
subsequent releases of a product in the same ,,
software is released to the market, and also for
development organization.
monthly averages by year.
l
Customer Satisfaction : Customer satisfaction is often Defect arrlvel pattern during machine testing
measured by customer survey data through the five- The overall defect density during teStmg will provide
point scale - only the summary of the defects. The p~ttern of defect
• Very satisfied arrivals gives more information about ~1~erent quality
• Satisfied levels in the field. It includes the following -
• Neutral • The defect arrivals or defects reported during the
• Dissatisfied testing phasf? by time interval (e.g., week). Here all
• ·very dissatisfied of which will not be valid defects.
• The pattern of valid defect arrivals when problem
Satisfaction with the overall quality of the product and
determination is done on thE: reported problems.
its specific dimensions is usually obtained through
This is the true defect pattern.
various methods of customer surveys. Based on the
five-point-scale data, several metrics with slight • The pattern of defect backlog overtime. This metric
is needed ·because development organizations
variations can be constructed and used, depending on
cannot investigate and fix all the reported problems
the purpose of analysis. For example -
immediately. This is a workload statement as well
• Percent of completely satisfied customers
as a quality statement. If the defect backlog is large
• Percent of satisfied customers at the end of the development cycle and a lot of
• Percent of dis-satisfied customers fixes have yet to be integrated into the system, the
• Per~t of non-satisfied customers stability of the system (hence its quality) will be.
Usually, this percent satisfaction is used. affected. Retesting (regression test) is needed to
ensure that targeted product quality levels are
· In-process quality metrics : In-process quality metrics
reached.
deals with the tracking of defect arrival during formal
machine testing for some organizations. This metric Phase-based defect removal pattern : This is an
_extension of the defect density metric during testing. In
includes-
addition to testing, it tracks the defects at all phases of
• Defect density during machine testing
the development cycle, including the design reviews,
• Defect arrival pattern during machine testing code inspections, and formal verifications before
• Phase-based defect removal pattern · testing.
• Defect removal effectiveness. Because a large percentage of progr~g defects is
Defect density during machine testing i Defect rate related to design problems, conducting formal reviews,
·during formal machine testing (testing after code is or functional verifications to enhance the defect
integrated into the system library) is correlated with removal capability of the process at the front-end
. the ~efect rate in the field. Higher defect rates found reduces error in the software. The pattern of phase-
based defect . removal reflects the overall defect
during testing is an indicator that the software has
removal ability of the development process.
. experienced .higher error injection during its
development process, unless the higher testing defect
Wifu regard to ~e metrics for · the design and coding ·
phase~, in addition to defect rates, many development
rate is due to an extraordinary testing effort.
orgaruzations use metrics such as inspection coverage
This simple metric of defects per KLOC or function
and inspection · effort for in-process quality
point is a good indicator_of quality, while the software management.
Techni~I Publications -An up thrust for knowledge
 5-17
--iii·------ Testing Tools and
MeiJSU,ements
removal effectlveneaa
. ss . The fix
;Jfcan be defined as follows _ Fix response time and fi x respons1vene · an
. . 11 tculated as the xne
response time metric 1s usua Y ca fi
'!1)RE-=(Defect_removed durin d . lose Short x
- g_a_ evelopment time of all problems from open to c . .
..,Jiase/Defects_latent in the prod t) -
.C ' . - - - UC >< 100 % . response time leads to customer. satisfaction.
This metric can . be caicu1 t d . f . . sponsiveness are
.. ae or the entire The important elements of fi x re d th
evelopment process, for the front-end bef d fix time an e
ti df ore co e customer expectations, the agreed-to ' ·
ilt~a on an or each phase. It is called early defect . .
ability to meet one's conumtmen t to the customer.
removal when used for the . fr t d
. . on -en and phase
!effectiveness
/ . for specific phases
· • e higher the value
Th Percent delinquent fixes
of the metric' the m ore effective· the development It is calculated as follows -
process and the fewer the defects passed to the next . (N. b r of fixes that
Percent Deliquent fixes = um e
·phase or to the field. This metric is a key concept of the · exceeded the response time . . . b severity level /
cntena Y
defect removal model for software development. · Number of fixes delivered in a specified time )
Maintenance Quality Metrics : Although much cannot x100%
be done to alter the quality of the product during this Fix Quality : Fix quality or the number of defective
phase, following are the fixes that can be carried out to · · for the
fixes is another important quality metric .
.eliminate the defects as soon as possible with excellent
maintenance phase. A fix is defective if it _d id not fix •
fix quality.
the reported problem, or if it fixeq. the otjginal problem
• Fix backlog and backlog management index
but injected a new defect. For mission-critical software,
• Fix response time and fix responsiveness defective fixes are detrimental to customer satisfaction.
• Percent delinquent fixes The metric of percent defective fixes is the percentage
• Fix quality. of all fixes in a time interval that is defective.
· Fix backlog and backlog management index : Fix A defective fix can be recorded in two ways: Record it
~backlog is related to the rate of defect arrivals and the in the month it was discovered or record it in, the
' 'r ate at which fixes for reported problems become month the fix was delivered. The fir.s t is a customer
,available. It is a simple count of reported problems that measure; the second is a process measure. The
remain at the end of each month or each week. Using it difference between the two dates is the latent period of
' in the format of a trend chart, this metric can provide the defective fix.
/ meaningful information for managing the maintenance Usually the longer the latency, the more will be the
. process. customers that get affected. If the number of defects is·
. Backlog Management Index (BMI) is used to manage large, then the small value of the percentage metric
the backlog of open and unresolved problems. · will show an optimistic picture. The quality goal for
·BMI=(Number_of_problems_closed_during_the_ the maintenance process, of course, is zero defective
monthiNumber_of_problems_arrived_during_the_ fixes without delinquency.
_month) x 100 %
Board question
If BMI is larger than 100, it means the backlog is 1. . Explain three types of product metrics.
reduced. If BMI is less 'than 100, then the backlog t\tSIHL . \\'111ti•r-l6 . J\1.nk, •l
· increased.

Technical Publications -An up thrust for knowledge

 Testing Tools and Measuremen~
Software Testing 5-18
.. d th think and find out why those
deviations, an en .
2. Progress Metrics . . . hich could point to problems.
deVIations eXIst w
• The software test metrics can be used for different
tanned versus actual deviates on
purposes. One of them was tracking progress. If we find out tha t P
. d f the week or that deviations occur only
Usually when we track progres~, it is related to certain ays o ' .
.. t and those testers are working on
time, or other unit that indicates a schedule. Most from certain tes ers,
often we use progress metrics to track planned specific parts of the software, this is useful
versus actual over time. information. .
• What we track depends on our role. If we are . t d/planned : This just keeps us on
Test cases execu e
bare minimum done in
financial people, then we track money spent. But for track to make sure we ge t the
software quality assurance, we want to track terms of getting our test cases executed.
progress of such things as defects, test cases, man'." If we find that it takes too long, on a repeated basis,
hours, etc. then something needs to be changed. Or if we go faster
• Basically anything that is related to results or effort than normal on a regular basis, then this may point to a
spent to get those results. For instance, here are a problem with the test cases (especially if they find no
few: defects).
Man-hours/test case executed : The natural tendency Test cases executed/defects found : This is a metric
_ in driving costs down is to force this as low as possible, that indicates how good our test cases are at finding
but remember the faster they ·go, and the more tests defects. Test cases run with no defects found or with a
that are executed, does not translate into higher quality low ratio does not mean there are no _defects in the
software.
software.
Planned hours/actual hours : We want to track the
Here is a generic chart in Fig. 5.5.4 which shows
· effort we plan versus what we spend not only just to
planned versus actual in terms of test cases executed.
· see if we are planning our resources well, but to_see

Text Execution Progress

120r--------------------.;_,

• Initial plan

- Actual

0
Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7 Day a Day 9 Day 10

Fig. 5.5.4 Executed versus planned test cases

Technical Publications " • An up thrust for knowledge

 Measurements
~re Testing Testing Tools and
5 -19

• Defect rate
-----Fix rate

0 2 3 4 5 6 7
Time

Fig·. 5.5.5 Defect fix rate

The chart for the defect fix rate is as shown in Function Points Per Measuring the
Week
Fig. 5.5.5. productivity of a development team by how many
Productivity Matrix: Productivity metrics are ways to function points they impl~ment in a month or week.
measure how much is produced for an input such as Story Points : Development productivity can also be
an hour of work. They are commonly used to manage measured by story points per week. ·
and improve performance. Units Per Hour : The number of units completed in an
Productivity was defined as how many "simple tasks" hour by a production line.
can be delivered by day. It can be computed at various Average Handle Time : The average time to handle a
levels within a project: individual, profile, task phase customer service request such as a refund. Used alone
or project. this can be a dangerous metric that encourages poor
The_definition "simple task" will always raise heated service. Typically, tl:tls is a secondary measurement
conversations; _we finally define it as the amount of after quality metrics such as customer satisfaction.
time required by a resource to deliver something and · Revenue Per Employee : A broad metric to measure •
settle it to five hours (of course some simple tasks take the productivity of an organization. Calculated by
less or more, but we settled for'that number). dividing ~evenue by total employees.
Given our definition "simple tasks that can be Revenue Per Salesperson : A common sales
delivered by day-_8 hours-" the formula was defined productivity metric is simply to look at how much
as: revenue was generated per salesperson.
Productivity= ((Planned Effort/ 5) / Actual Effort) *8 Labor Productivity : Productivity can be measured for
The following are common examples of productivity a nation, region or industry by calculating GDP or _
metrics. revenue per hour _worked. Generally · speaking,
-Lines of Code Per Day : The amount of source code productivity increases over time due to technological
produced per software developer per day. This and process improvements.
measure isn't pa_rticularly accurate ·as much code is
3. Project metrics :
autogenerated or cut and pasted.
• The first application of project metrics occurs during
estimation-Metrics from past projects are used as a
basis for estimating time and effort.
. M
Technical Publications - An up thrust for knowledge
 .Testing Tools and Measurements
Softwiire Testing 5-20
ctional point metrics can be used
• As a project proceeds, the amount of time and effort Lines of code and fun .
. b' ct-oriented software proJects.
expended are compared to original estimates . . for estimating O Je . .
. are not appropnate m the case
However, these metncs
• As technical work commences, other project metrics . . al ftware development as they do not
of increment so
become important-production rates are measured tails for effort and schedule
provide adequate d e . .
(represented in terms of models created, review bj ect-oriented pro1ects, different
estimation. Thus, orf O •
hours, function points, and delivered source lines of . h been proposed. These are listed
sets of metncs ave · •
code)-Error uncovered during each generic
framework activity (i.e, ~ommunication, planning, below.
• scripts : Scenario scripts are a
· modeling, construction, deployment) are measured. • Number of scenano
which depict the interaction
sequence of steps, .
Use of Project Metrics the application. A number of
between the user and
• Project metrics are used to-Minimize the scenarios 'is directly related to application size and
development schedule by making the adjustments
number of test cases that are developed to test the
ne~ssary to avoid delays and mitigate potential
software, once it is developed. Note that scenario
problems and risks-Assess product quality on an
. ongoing basis and, when necessary, to modify the scripts are analogous to use-cases .
technical approach to improve.quality. • Number of key classes : Key classes are
• In summary-As quality improves, defects are independent componen~, · which are defined in
minimized-As defects go down, the amount. of object-oriented analysis. As key classes form the
rework required during the project is.also reduced- core of the problem dom·ain, they indicate the effort
As rework goes down, the overall project cost is required to develop software and the amount of
reduced 'reuse' feature to be applied during the ~evelopment
• Project metrics can be consolidated to create process process.
metrics for an organization
• Number of support classes : Classes, which ·are
l
/ 5.5.4.2 Object Oriented Metrics required to implement the system but are indirectly
It is widely accepted that object oriented development related to the problem domain, are known as
requires a different way of thinking than traditional support_classes. For example, user interface classes
structured developmentl and software projects are and computation class are support classes. It is
•
shifting to object oriented design. possible to develop a support class for each key
· The main advantage of o~ject oriented design is its class. Like key classes, support classes indicate the
modularity and reusability. Object oriented metrics are effort required to develop software and the amount
· used to measure properties of object oriented designs.
of 'reuse' feature to be applied during the
Compared to structural development, object oriented
development process.
design is a comparatively new technology. The
metrics, which were· useful for evaluating structural • Avera·g e number of support qasses per key class :
development, may perhaps not affect the design using Key classes are defined early h\ the software project
00 language. while support classes are defined throughout the
As for example, the "Lines of Code" metric is used in project. The estimation process is simplified if the
structural development whereas it is not so much used average number of support classes per key class is
in object oriented design. Very few existing metrics (so already known.
called traditional metrics) can measure object oriented
design properly.
T&chnical Publications -An up thrust for knowledge
«
I ,

5 - 21
Software Testing

• Number of subsystems : A collection of classes that

supports a function visible to the user is known as a
subsystem. Identifying. subsystems makes it easier
to prepare .,t reasonable schedule in which work on
subsystems is divided among project members.
The afore-mentioned metrics are collected along with
other project metrics like effort used, errors and defects
detected, and so on. After an organization completes a
number of projects, a database is developed, which
shows the relationship between ·object-oriented
measure and project measure. This relationship
provides metrics that help in project estimation.
□□□

Technical Publications . • An up thrust for knowledge

Ill

SOLVED SAMPLE TEST.PAPER. - I

Software Testing CM/CW)
T.Y. Diploma (Sem. V) Computer Engg. Group (CO/
(Total Marks : 20
Time : 1 Hour)
Instructions :
1) All questions are compulsory.
l) fflustrate your answers with neat sketches wherever necessary.
3) Figures to the right indicate full marks.
4) Assume suitable data if necessary.
S) Preferably, write the answers in sequential order. (8)
Q.1 Attempt any FOUR.
a) · (Refer section
Explain the terms mistake, defect,fault and failure in relation with software testing. . 1.2)

b) Enlistfeatures oftest cases (Refer section 1;3.2)

c) What is GUI testing? (Refer section 2.6.2)
d) What is bi-directional integration testing ? (Refer section 2.3.3)
e) What kind oferrors can be identified during unit testing? (Refer section 2.2.1)
f) What is size estimation? (Refer section 3.1.7(1))
Q.2 Attempt any THREE. (12)
a) Explain driver and stub in detail. (Refer section 2.2.3)
b) What/actors shall be considered while selecting resource requirements? (Refer section 3.1.5)
c) Difference between top down integration '!-nd bottom up integration. (Refer section 2.2.2)
d) Explain requirement based testing. (Refer section 1.7.1)
e) How to prepare a test plan ? Explain in detail. (Refer section 3.1.1)
f) Differentiate between verification and validation (Any four points). (Refer section 1.4)

□□□

(S -1) .
 SOLVED SAMPLE TEST PAPER
. - II
.

Software Testing
T.Y. Diploma (Sem. V) Computer Engg. Group (CO/CM/CW)
· [Total Marks : 20
Time : 1 Hour]
,..
Instructions :
1) All questions are compulsory.
2) filustrate your answers with neat sketches wherever necessary.
3) Figures to the right indicate full marks.
4) Assume suitable data if necessary.
S) Preferably, write the answers in sequential order.
[8]
Q.1 Attempt any F9UR.
a) Enlist th'! states in defect life cycle. (Refer section 4.2.1)
b) Give any. two advantage~ ofmanual testing. (Refer section 5.1.2)
c) Enlist different types ofdefects. (Refer section 4.1.1)
d) Explain the need for testing tool (Any two points). (Refer section 5.1.5)
e) Enlist the techniques for finding defects. (Refer section 4.3.2)
f) Specify the any two criteria that should be considered before selecting any testing tool.
(Refer section 5.3)
Q.2 Attempt any THREE. [12)
a) Differentiate between manual testing and automation testing (Any four points). (Refer section S.1.4)
· b) What do you mean by 'Defect Impact'? Explain how to estimate the def~ct impact. (Refer section 4.3)
c) Explain the importance ofsoftware metrics. (Refer section S.5.1)
d) Explain defect management
.- .
process·in detail. (Refer section 4.1.2)
e) What are types of manual testing metrics. (Refer section S.S.3)
f) Explain the defect template with its attributes. (Refer section 4.2.2)

□□□
 1
SOLVED SAMPLE QUESTION PAPER
Software ·Testing
T.Y. Diploma (Sem. V) Computer Engg. Group (CO/CM/CW)
I •
[Total Marks : 70
Time : 3 Hours)
Instructions :

1) All questions are compulsory.

2) filustrate your answers with neat sketches wherever necessary.
3) Figures to the right indicate full marks.
4) Assume suitable data if necessary.
S) Preferably, write the answers in sequential order.
(10]
Q.1 Att~mpt any FIVE of the following.
a) What is software testing? (Refer section 1.1.1)
b) Enlist various levels of testing. (Refer section 2.1)
c) What is the purpose of Test plan ? (Refer section 3.1)
d) Give any two possibie causes of def~cts. (Refer section 4.1)
e) What is man!'al testing? (Refer section 5.1)
- f) What is performance testing? (Refer section 2.4.1)
g) Define - Static and dynamic testing, (Refer section 1.5)
Q.2 Attempt any THREE of the following. (12] ·
a) Give difference between quality assurance and quality control. (Any four) (Refer section 1.4.5)
b) Describe top-down integration testing with labelled diagram. (Refer section 2.3.1)
c) Why is it essential to setup criteria for testing? List any three criteria in different situations.
(Refer section 3.1.3)
d) Explain requirement defects and design defects. (Refer section 4.1.1)
Q.3 Attempt any THREE of the following. (12]
a) State any eight limitation; of manual testing. (Refer section 5.1.3)
b) Explain the 'Test Infrastructure' components with diagram. (Refer section 3.2.1)
c) Explain static and dynamic testing tools in details. (Refer section 5.2.3)
d) With the help of neat diagram, describe unit testing. (Refer section 2.2)
Q.4 Attempt any THREE of the following. c121
a) Describe v::.model with labelled diagram. (Refer section 1.4_.2)
b) Explain deject life cycle to identity status of defect with proper labelled diagram. (Refer section 4.2.l)
c) Explain people management in test planning. (Refer section 3.2.2)
· d) What is load testing and stress testing ? Describe with respect to system testing. (Refer section 2.4)
e) Enlist factors considered for selecting a testing tool for test automation. (Refer section 5.3)

· · <s~ar>·
. . ·• -
 Solmi Sample PIIJ'ff's
SofttNre Testing s. 4 (12)
Q.S Attempt any TWO of the following.
6 12
a) W?rar is walkthrollgh ? Give advantages and disadvantages of ii. (Refer 9ectlon 1. - · >
b) Write imporranr sil: resr cases/or the 'Login Form· of the Facebook website. (Refer eumple J.J.l)
c) Describe defect template with its attributes. (Refer sttdon 4.2.2)
Q.6 Attempt any TWO of the following. (12)
a) . Draw the.flow gmphfm·.finding maxim,,m of three numbers and deri ve the test cases using cyclomatic
· comple.¥.ity. (Refer example J.6.1)
b) Differentiate between alpha testing and beta testing (Any six point). (Refer section l.5.l)
c) ~fine metrics and meas11T'ements. Explai,i need of software measurement. (Refer section 5.5)

□□□

Technical Publications - An up thrust for knowledge

 Price : t 85/-
1SBN 978 -93-89180-04-6

ll 11 lll \II
-------ii.ii~ 9 789389 18004