Unit-1

Basics of Software Testing and Testing Methods

Prof. Bipin B. Shinde

Department of Computer Technology
Amrutvahini Polytechnic, Sangamner
Contents
Types of Testing
White Box- Inspection,Walkthrough,Technical Review
Black Box- Required Based Testing, Boundary Value
Analysis,Equivalance Partitioning ,Positive –negative
testing,
Decision table, user documentation testing, state-
transition Testing
Contents ……
 Types of testing(White Box and Black box)
 What is White Box testing?
 Static White Box testing(Inspection, Structured
Walkthrough, Technical Review)
 Dynamic testing or Structural testing(Code Functional
testing, Code Coverage testing, Code Complexity
testing.)
 What is Black Box testing?
 Techniques for Black Box testing(Required based
testing, Positive Negative testing, Boundary Value
Analysis, Decision Tables,Equivalance Partitioning, User
Documentation testing, Graph based testing)
 Sample Examples on White box and Black Box Testing.
 TYPES OF TESTING

WHITE BLACK BOX

BOX
White Box Testing
White box testing is also known as Clear Box testing,
Open Box testing, Structural testing, Transparent Box
testing, Code-Based testing, and Glass Box testing.
The term "white box" was used because of the see-
through box concept. The clear box or white box name
symbolizes the ability to see through the software's
outer shell (or "box") into its inner workings.
 White Box Testing:

In white box testing, test cases are created by looking

at the code to detect any Potential failure scenarios.
 The suitable input data for testing various APIs and
the special code paths that Need to be tested by
analyzing the source code for the application block.
Therefore, the test plans need to be updated before
starting white box testing and only after a stable build
of the code is available.
One of the basic goals of white box testing is to verify
a working flow for an application. It involves testing a
series of predefined inputs against expected or desired
outputs so that when a specific input does not result in
the expected output, you have encountered a bug.
 White box testing assumes that the tester can take a
look at the code for the application block and create
test cases that look for any potential failure scenarios.
 During white box testing, analyze the code of the
application block and prepare test cases for testing the
functionality to ensure that the class is behaving in
accordance with the specifications and testing for
robustness.
 A failure of a white box test may result in a change
that requires all black box testing to be repeated and
white box testing paths to be reviewed and possibly
changed.
 What do you verify in White Box Testing?
White box testing involves the testing of the software code
for the following:
Internal security holes
Broken or poorly structured paths in the coding processes
The flow of specific inputs through the code
Expected output
The functionality of conditional loops
Testing of each statement, object and function on an
individual basis
Advantages of White Box Testing
 Code optimization by finding hidden errors.
 White box tests cases can be easily automated.
 Testing is more thorough as all code paths are usually covered.
 Testing can start early in SDLC even if GUI is not available.
Disadvantages of White Box Testing
 White box testing can be quite complex and expensive.
 Test Script maintenance can be a burden if the implementation
changes too frequently.
 White box testing requires professional resources, with a detailed
understanding of programming and implementation.
 White-box testing is time-consuming, bigger programming
applications take the time to test fully.
 Classification of White Box Testing
 White Box testing

Static Structural(Dynamic)

Inspection Structured Technical Code Code Code

Walkthrough Review Functional Coverage Complexity

Static Testing
 Static Testing is a technique by which we can check the defects in software
without actually executing it.
 Static testing is done to avoid errors at an early stage of development as it
is easier to find sources of failures then failures themselves.
 Static testing helps find errors that may not be found by Dynamic Testing.
Why Static Testing
Static testing is performed due to following reasons
 Early defect detection and correction
 Reduced development timescales
 Reduced testing cost and time
 For improvement of development productivity
 To get fewer defect at later stage of testing
What is Tested in Static Testing
In Static Testing, following things are tested
 Unit Test Cases
 Business Requirements Document (BRD)
 Use Cases
 System/Functional Requirements
 Prototype
 Prototype Specification Document
 DB Fields Dictionary Spreadsheet
 Test Data
 Traceability Matrix Document
 User Manual/Training Guides/Documentation
 Test Plan Strategy Document/Test Cases
 Automation/Performance Test Scripts
How Static Testing is Performed
To perform Static Testing, it is done in following ways,
Carry out the inspection process to completely inspect
the design of the application
Use a checklist for each document under review to
ensure all reviews are covered completely
Static Testing Techniques
 Inspections
 Structured Walkthroughs
 Technical Review
1.Inspection(Formal Inspection or Fagan Inspection)
 Inspections are the most formal type of reviews.
 They are highly structured and require training for each
participant.
 Inspections are different from peer reviews and
walkthroughs, in that the person who presents the code, the
presenter or reader, isn’t the original programmer.
 These forces someone else to learn and understand the
material being presented, potentially giving a different slant
and interpretation at the inspection meeting.
The other participants are called inspectors.
 Each is tasked with reviewing the code from a
different perspective, such as a User, a tester, or a
product support person.
 This helps bring different views of the product under
review and very often Identifies different bugs.
One inspector is even tasked with reviewing the code
backward—that is, from the end to the beginning—to
make sure that the material is covered evenly And
completely.
In inspection there are four roles involves.
1. Author-the author of document under review.
2. Moderator -who is responsible to formally run the
inspection.
3. Inspectors- these are the people who actually
provides ,review comments.
4. Scribe- it is person who takes detailed notes
during the inspection meeting.
2. Structured Walkthrough:

 Walkthroughs are the next step up in formality from peer

reviews.
 In a walkthrough, the programmer who wrote the code
formally presents (Walks through) it to a small group of five
or so other programmers and testers.
 The reviewers should receive copies of the software in
advance of the review so they can examine it and write
comments and questions that they want to ask at the review.
 Having at least one senior programmer as a reviewer is very
important.
 In this method author is leader and other are participants.
3. Technical Review:
 i. Formal Review:
 A formal review is the process under which static white-
box testing is performed.
 A formal review can range from a simple meeting between
two programmers to a
 detailed, rigorous inspection of the code.
 There are four essential elements to a formal review
 1. Identify Problems:
 2. Follow Rules:
 3. Prepare: -
 4. Write a Report: -
.
 Peer Review
 The easiest way to get team members together and doing
their first formal reviews of the software is through peer
reviews, the least formal method.
 Sometimes called buddy reviews, this method is really
more of a discussion. Peer reviews are often held with just
the programmer who wrote the code and one or two other
programmers or testers acting as reviewers.
 Small group simply reviews the code together and looks for
problems and oversights.
 To assure that the review is highly effective all the
participants need to make sure that the four key elements of
a formal review are in place: Look for problems, follow
rules, prepare for the review, and write a report.
 As peer reviews are informal, these elements are often
scaled back. Still, just getting together to discuss the code
can find bugs
Static Analysis Tools
It perform analysis of the code to find out errors such as—
1. Whether there are unreachable codes
2. Variables declared but not use
3. Mismatch in definition and assignment of values to variables.
4. Memory allocated but not having corresponding statements.
5. Calculation of cylomatic complexity.
 Tools used for Static Testing
 Various tools used for Static Testing are as follow,
 Checkstyle
 IntelliJ IDEA
 FindBugs
 Jarchitect
 Soot
 Squale
 ThreadSafe
 SourceMeter
Structural Testing
Structural testing is the testing of structure of the
system or component.
It is also referred as “glass box” or “clear box” because
the internal structure(design/implementation) known to
us.
Here tester should have the knowledge of internal
implementation of the code.(i.e. how it is implemented
or how it is works).
Structural testing is done by executing software
product.
Structural testing types
Code Functional Testing.
Code Coverage Testing.
Code Complexity Testing.
1 ) Code Functional Testing (Data Flow)
i. Data flow coverage involves tracking a piece of data
completely through the software.
ii. At the unit test level this would just be through an
individual module or function.
iii. The same tracking could be done through several
integrated modules or even through the entire software
product—although it would be more time consuming
to do so.
 iv. During data flow, the check is made for the proper
declaration of variables declared and the loops used are
declared and used properly.
 For example
 1. #include<stdio.h>
 2. void main()
 3. {
 4. int i , fact= 1, n;
 5. printf(“enter the number “);
 6. scanf(“%d”,&n);
 7. for(i =1 ;i <=n;i++)
 8. fact = fact * i;
 9. printf (“the factorial of a number is ”%d”, fact);
 10. }
Code Coverage Testing
Code coverage testing involves designing and
executing test cases and finding out the percentage of
code that is covered by testing.
The % of code covered by a test is found by adopting
a technique called instrumentation of code.
For this purpose specialized tools are used, which
rebuilds the product with set of libraries provided by
tool vendors.
The instrumented code can monitor and keep an audit
of what portion of code are covered.
Code Coverage Testing Types
Statement coverage
Path Coverage
Condition Coverage
Function Coverage.
Statement Coverage
Statement coverage refers to writing test cases that
execute
Each of program statement.
Based on conventional programming lang. the program
constructs can be classified as
1. Sequential control flow
2. Two-way decision statement like if then else
3. Multi-way decision statement switch
4. Loops like while do, repeat until and for
Path testing
 Attempting to cover all the paths in the software is called path
testing.
 The simplest form of path testing is called branch coverage
testing.
 To check all the possibilities of the boundary and the sub
boundary conditions and it’s branching on those values.
 Test coverage criteria requires enough test cases such that each
condition in a decision takes on all possible outcomes at least
once, and each point of entry to a program or subroutine is
invoked at least once.
 Every branch (decision) taken each way, true and false.
 It helps in validating all the branches in the code making sure that
no branch leads to abnormal behavior of the application.
Condition testing
i. Just when you thought you had it all figured out,
there’s yet another Complication to path testing.
ii. Condition coverage testing takes the extra
conditions on the branch statements into account.
Path testing covers all the possible paths, but it does
not mean that the program is fully tested.
Hence for every path all condition need to be test.
Condition coverage is indication of percentage of
conditions covered by a set of test cases.
Code Complexity Testing
Cyclomatic Complexity
To understand Cyclomatic Complexity, lets first
understand - What is Software Metric?
Measurement is nothing but quantitative indication of
size / dimension / capacity of an attribute of a product /
process.
Software metric is defined as a quantitative measure of
an attribute a software system possesses with respect to
Cost, Quality, Size and Schedule.
What is Cyclomatic Complexity?
 Cyclomatic complexity is a software metric used to measure
the complexity of a program. These metric, measures
independent paths through program source code.
 Independent path is defined as a path that has at least one edge
which has not been traversed before in any other paths.
 Cyclomatic complexity can be calculated with respect to
functions, modules, methods or classes within a program.
 This metric was developed by Thomas J. McCabe in 1976 and
it is based on a control flow representation of the program.
Control flow depicts a program as a graph which consists of
Nodes and Edges.
 In the graph, Nodes represent processing tasks while edges
represent control flow between the nodes.
Flow graph notation for a program:
Flow Graph notation for a program is defines.
several nodes connected through the edges.
Below are Flow diagrams for statements like if-
else, While, until and normal sequence of flow.
Mathematically, it is set of independent paths through
the graph diagram. The complexity of the program can
be defined as -
V(G) = E - N + 2
Where,
E - Number of edges
N - Number of Nodes
V (G) = P + 1
Where P = Number of predicate nodes (node that
contains condition)
Example
i = 0;
n=4; //N-Number of nodes present in the graph
while (i<n-1) do
j = i + 1; Flow graph
while (j<n) do
if A[i]<A[j] then
swap(A[i], A[j]);
end do;
i=i+1;
End do;
Computing mathematically,
V(G) = 9 - 7 + 2 = 4
V(G) = 3 + 1 = 4 (Condition nodes are 1,2 and 3
nodes)
Basis Set - A set of possible execution path of a
program
1, 7
1, 2, 6, 1, 7
1, 2, 3, 4, 5, 2, 6, 1, 7
1, 2, 3, 5, 2, 6, 1, 7
Uses of Cyclomatic Complexity:
Cyclomatic Complexity can prove to be very helpful in
Helps developers and testers to determine independent path
executions
Developers can assure that all the paths have been tested at
least once
Helps us to focus more on the uncovered paths
Improve code coverage
Evaluate the risk associated with the application or program
Using these metrics early in the cycle reduces more risk of
the program
Black box testing
 Black box testing is a software testing techniques in which functionality of
the software under test (SUT) is tested without looking at the internal code
structure, implementation details and knowledge of internal paths of the
software.
 It is also called as behavioral testing, because you are testing how the
software behaves when its used.
 This type of testing is based entirely on the software requirements and
specifications.
 In Black Box Testing we just focus on inputs and output of the software
system without bothering about internal knowledge of the software
program.
 It is done from user perspective.
 It helps in overall verification of system under test.
 Here test cases are prepared by looking at software requirements
specification
Techniques of black box testing
Required based testing
Positive Negative testing
 Boundary Value Analysis
 Decision Tables
Equivalence Partitioning
User Documentation testing
Graph based testing
Required based testing
 What is Requirement Based Testing?
 Requirement based testing is delivering what you promised or finalized.
Also, it is a method to verify that software meets software application’s
requirement.
 The software is built and test based on requirements.
 Requirement based testing uses a functional specification document or
Requirement specification documents, user story (in an agile environment),
use cases or flow diagrams, etc.
 Testers get the requirements specification documents from the business
team. Post that, there the first job is to go through all requirements and
understand them. If there are any unclear requirements, they should discuss
with business team to clarify.
 Next, testers prioritize them as critical, high, medium and low based on
risk criteria. This will ensure that the most important and most critical
requirement tests are included in the testing efforts.
What is RTM (Requirement Traceability Matrix)?
Requirement Traceability Matrix or RTM captures all
requirements proposed by the client or software
development team and their traceability in a single
document delivered at the conclusion of the life-cycle.
In other words, it is a document that maps and traces
user requirement with test cases. The main purpose of
Requirement Traceability Matrix is to see that all test
cases are covered so that no functionality should miss
while doing Software testing.
Requirement Traceability Matrix – Parameters
include
Requirement ID
Risks
Requirement Type and Description
Trace to design specification
Unit test cases
Integration test cases
System test cases
User acceptance test cases
Trace to test script
Types of Traceability Test Matrix
 Forward traceability: This matrix is used to check whether the project
progresses in the desired direction and for the right product. It makes sure that
each requirement is applied to the product and that each requirement is tested
thoroughly. It maps requirements to test cases.

Backward or reverse traceability: It is used to ensure whether the current
product remains on the right track. The purpose behind this type of
traceability is to verify that we are not expanding the scope of the project by
adding code, design elements, test or other work that is not specified in the
requirements. It maps test cases to requirements.

Bi-directional traceability ( Forward + Backward): This traceability
metrics ensures that all requirements are covered by test cases. It analyzes the
impact of a change in requirements affected by the Defect in a work product
and vice versa.
 Let's understand the concept of Requirement
How to create Requirement Traceability Matrix

Traceability Matrix through banking project.

On the basis of Business Requirement Document
(BRD) and Technical Requirement Document (TRD),
testers start writing test cases.
Here the scenario is that the customer should be able to
login to banking website with the correct password and
userId while manager should be able to login to the
website through customer login page.
 Technical Requirement Document
(TRD)or Functional Requirement
Document(FRD)
Step 1: Our Test Case is
Step 2: Identify the Technical Requirement that this
test case is verifying.
For our test case, the technical requirement is T94 is
being verified.
Step 3: Note this Technical Requirement (T94) in the
Test Case.
Step 4: Identify the Business Requirement for which
this TR (Technical Requirement-T94) is defined.

Step 5: Note the BR (Business Requirement) in Test Case

 Step 6: Do above for all Test Cases. Later Extract the First 3
Columns from your Test Suite. RTM in testing is Ready!

 Advantage of Requirement Traceability Matrix

 It confirms 100% test coverage
 It highlights any requirements missing or document
inconsistencies
 It shows the overall defects or execution status with a focus on
business requirements
 It helps in analyzing or estimating the impact on the QA team's
work with respect to revisiting or re-working on the test cases
Positive Negative testing
 What is Positive Testing?
Positive testing is the type of testing that can be performed on the
system by providing the valid data as input.
 It checks whether an application behaves as expected with positive
inputs. This test is done to check the application that does what it is
supposed to do.

 There is a text box in an application which can accept only numbers.

 Entering values up to 99999 will be acceptable by the system and any
other values apart from this should not be acceptable.
 To do positive testing, set the valid input values from 0 to 99999 and
check whether the system is accepting the values.

What is Negative Testing?

 Negative Testing is a variant of testing that can be

performed on the system by providing invalid data as input.
 It checks whether an application behaves as expected with
the negative inputs. This is to test the application does not do
anything that it is not supposed to do so.

 Negative testing can be performed by entering characters A

to Z or from a to z. Either software system should not accept
the values or else it should throw an error message for these
invalid data inputs.
Boundary Value Analysis
What is BVA?
In software testing, the Boundary Value Analysis
(BVA) is a black box test design technique based on
test cases. This technique is applied to see if there are
any bugs at the boundary of the input domain. Thus,
with this method, there is no need of looking for these
errors at the center of this input.
BVA helps in testing the value of boundary between
both valid and invalid boundary partitions. With this
technique, the boundary values are tested by the
creation of test cases for a particular input field.
 The extreme ends or boundary partitions might depict the
values of lower-upper, start-end, maximum-minimum, inside-
outside etc.
This technique is an easy, quick and brilliant way to catch any input errors that
might occur to interrupt the functionality of a program.
So, to save their time and to cut the testing procedure short, the experts
delivering software testing and quality management services rely on the
Boundary Value Analysis method.
For testing of data related to boundaries and ranges, the method is considered
as a very suitable one.
Equivalence Partitioning
 This is a software testing technique which divides the input data
into many partitions.
 Values from each partition must be tested at least once. Partitions
with valid values are used for Positive Testing. While, partitions
with invalid values are used for negative testing.
 Equivalence Partitioning Technique is also known
as Equivalence Class Partitioning or ECP.
 The idea behind this technique is to divide/partition a set of test
conditions into groups or sets that can be considered the same or
equivalence .
 This is used to reduce the total number of test cases to a finite set
of testable test cases, still covering maximum requirements.
Decision Tables

A Decision Table Testing is a good way to deal with

different combination of inputs which produce
different results. It is also called Cause-Effect Table.
 It provides a systematic way of stating complex
business rules, which is useful for developers as well
as for testers.
Decision tables can be used in test design as they help
testers to explore the effects of combinations of
different inputs.
Decision tables can be used in test design whether or
not they are used in specifications, as they help testers
explore the effects of combinations of different inputs
and other software states that must correctly
implement business rules.
It helps the developers to do a better job can also lead
to better relationships with them.
 Testing combinations can be a challenge, as the
number of combinations can often be huge.
Testing all combinations may be impractical if not
impossible.
We have to be satisfied with testing just a small subset
of combinations but making the choice of which
combinations to test and which to leave out is also
important.
Decision tables are precise and compact way to model
complicated logic.
 It helps the developers to do a better job and can also
lead to better relationships with them.
It may be not be possible to test all combinations as
the number of combinations can be huge.
It is better to deal with large numbers of conditions by
dividing them into subsets and dealing with the subsets
one at a time.
 The developer has to be satisfied with the choice of
testing just a small subset of important combinations.
 It helps reduce test effort in verifying each and every
combinations of test data and ensures complete
coverage
A customer requests a cash withdrawal. One of the
business rules for the ATM is that the ATM machine
pays out the amount if the customer has sufficient
funds in their account or if the customer has the credit
granted. Already, this simple example of a business
rule is quite complicated to describe in text. A decision
table makes the same requirements clearer to
understand:
Importance of Decision Table:
Essentially it is a structured exercise to formulate
requirements when dealing with complex business
rules. Decision tables are used to model complicated
logic.
 They can make it easy to see that all possible
combinations of conditions have been considered and
when conditions are missed, it is easy to see.
User Documentation testing
 State any four objectives of user documentation testing. How
these are useful in planning user documentation test?
 User Documentation covers all the manuals, user guides,
installation guides, setup guides, read me files, software release
notes, and online help that are provided along with the software
to help the end user to understand the software system.
 Documentation Testing should have two objectives:-
 To check if what is stated in the document is available in the
software
 To check if what is there in the product is explained correctly in
the document
 This testing is plays a vital role as the users will refer this
document when they start using the software at their location.
 a badly written document can put off a user and bias them
against the product even the product offers rich functionality.
 Few essential software testing documents that need to be
used and maintained on a day to day basis:
 Test design document
 Test case specification
 Test Strategy
 Test summary reports
 Document of Weekly Status Report
 User Documents
 Document of User Acceptance Report
 Report of Risk Assessment
 Test Log document
 Test plan document
 Bug reports document
 Test data document
 Test analysis
Benefits of user Documentation Testing
Highlight the problem overlooked during reviewed.
Result in less difficult support calls.
New programmers and testers who join a project group
can use the documentation to learn the external
functionality of product.
The high-quality user documentation can result in the
reduction in overall cost for user organization.
Graph based testing methods are applicable to generate
Graph Based Testing or State Transition Testing

test case for state machines such as language

transaction, workflows, transaction flows and data
flows.
Let's consider an ATM system function where if the
user enters the invalid password three times the
account will be locked.
In this system, if the user enters a valid password in
any of the first three attempts the user will be logged in
successfully. If the user enters the invalid password in
the first or second, try the user will be asked to re-enter
the password. And finally, if the user enters incorrect
password 3rd time, the account will be blocked.
 State Transition testing, a black box testing technique, in
which outputs are triggered by changes to the input
conditions or changes to 'state' of the system. In other words,
tests are designed to execute valid and invalid state
transitions.
When to use?
 When we have sequence of events that occur and associated
conditions that apply to those events
 When the proper handling of a particular event depends on
the events and conditions that have occurred in the past
 It is used for real time systems with various states and
transitions involved
State Transaction Table