Unit 4

Testing
What is Testing?
• “Testing is the process of executing a program with the intent of finding errors.”
• Testing a program consists of providing the program with a set of test inputs (or test cases) and
observing if the program behaves as expected.
• If the program fails to behave as expected, then the conditions under which failure occurs are
noted for later debugging and correction.
• Some commonly used terms associated with testing are:
i. Failure: This is a manifestation of an error (or defect or bug). But, the mere presence of an
error may not necessarily lead to a failure.
ii. Test case: This is the triplet [I,S,O], where I is the data input to the system, S is the state of the
system at which the data is input, and O is the expected output of the system.
iii. Test suite: This is the set of all test cases with which a given software product is to be tested.
Software Testing
• Software testing can be divided into two steps:
1. Verification: it refers to the set of tasks that ensure that software
correctly implements a specific function.
2. Validation: it refers to a different set of tasks that ensure that the
software that has been built is traceable to customer requirements.
Verification: “Are we building the product right?”
Validation: “Are we building the right product?”
Static VS Dynamic Testing

• Static testing largely maps to verification and dynamic testing to

validation. Static testing is performed without executing the code.

• Dynamic testing is performed with the execution of code.

• Dynamic testing techniques, namely black-box and white-box

techniques, are very popular.
Methods of Software Testing

white-box black-box
methods methods

Methods

Strategies
Black-box Testing
• Black-box technique is one of the major techniques in dynamic testing
for designing effective test cases.
• This technique considers only the functional requirements of the
software or module. In other words, the structure or logic of the
software is not considered. Therefore, this is also known as functional
testing.
• The software system is considered as a black box, taking no notice of its
internal structure, so it is also called as black-box testing technique.
Input test data is given to the system, which is a black box to the tester,
and results are checked against expected outputs after executing the
software, as shown in Fig. 4.1.
Black-box testing attempts to find errors in the following categories: „1
1.To test the modules independently .„
2.To test the functional validity of the software so that incorrect or missing
functions can be recognized.
3 .To look for interface errors „
4.To test the system behavior and check its performance „.
5. To test the maximum load or stress on the system „ .
6.To test the software such that the user/customer accepts the system within
defined acceptable limits
 1.BOUNDARY VALUE ANALYSIS (BVA)

• It has been observed that test cases designed with boundary input values have a
high chance to find errors. It means that most of the failures crop up due to
boundary values.

• BVA is considered a technique that uncovers the bugs at the boundary of input
values. Here, boundary means the maximum or minimum value taken by the
input domain. For example, if A is an integer between 10 and 255, then boundary
checking can be on 10(9,10,11) and on 255(256,255,254). Similarly, B is another
integer variable between 10 and 100, then boundary checking can be on
10(9,10,11) and 100(99,100,101), as shown in Fig. 4.2
BVA offers several methods to design test cases as discussed in the following sections:
1. BOUNDARY VALUE CHECKING (BVC)
2. ROBUSTNESS TESTING METHOD.
3. WORST-CASE TESTING METHOD
 1.1 BOUNDARY VALUE CHECKING (BVC)
• In this method, the test cases are designed by holding one variable at its extreme
value and other variables at their nominal values in the input domain.
• The variable at its extreme value can be selected at:
(a) Minimum value (Min)
(b) Value just above the minimum value (Min+ )
(c) Maximum value (Max)
(d) Value just below the maximum value (Max−)

A and B lies between[1,50]

It can be generalized that for n variables in a module, 4n + 1 test cases can be designed with boundary value
checking method
1.2 ROBUSTNESS TESTING METHOD
• A program reads three numbers, A, B, and C, within the range [1, 50] and prints
the largest number. Design test cases for this program using BVC, robust testing,
and worst-case testing method,.
2.EQUIVALENCE CLASS TESTING
IDENTIFYING THE TEST CASES
Example
3.CAUSE-EFFECT GRAPHING BASED TESTING

• Like decision tables, cause-effect graphing is another technique for

combinations of input conditions. But cause-effect graphing takes the
help of decision table to design a test case. Therefore, cause-effect
graphing is the technique to represent the situations of combinations
of input conditions and then we convert the cause effect graph into
decision table for the test cases.
Example: A program has been designed to determine the nature of roots of a
quadratic equation. The quadratic equation takes three input values from the range
[0, 100]. Design the test cases using cause-effect graphing technique.
White-Box Testing
• Also known as Structural Testing, Clear-Box Testing, Open-
Box Testing, Logic-Driven Testing.
• Using White-Box testing methods, the software engineer
can derive test cases that.
Guarantee that all independent paths within a module have
been exercised at least once.
Exercise all logical decisions on their true and false sides.
Execute all loops at their boundaries and within their
operational bounds.
Exercise internal data structures to ensure their validity.

• white-box testing: the internal structure of the software is

taken into account to derive the test cases
GRAPH MATRICES
Difference Between Black Box Testing and
White Box Testing
BLACK BOX TESTING WHITE BOX TESTING

Internal workings of an application are not

Knowledge of the internal workings is must.
required.

Also known as closed box/data driven testing. Also knwon as clear box/structural testing.

End users, testers and developers. Normally done by testers and developers.

THis can only be done by trial and error method. Data domains and internal boundaries can be
better tested.
Test process in software development
V-model:
acceptance
requirements test

system
specification test

detailed integration
design test

implementation unit
code test
 Unit Testing
• Involves testing a single isolated module

• Note that unit testing allows us to isolate the errors to a single module
• we know that if we find an error during unit testing it is in the module
we are testing

• Modules in a program are not isolated, they interact with each other.
Possible interactions:
• calling procedures in other modules
• receiving procedure calls from other modules
• sharing variables

• For unit testing we need to isolate the module we want to test, we do this
using two things
• drivers and stubs
Stubs and Drivers :
• The Stubs and Drivers are considered as elements which are equivalent to to-do
modules that could be replaced if modules are in their developing stage, missing
or not developed yet, so that necessity of such modules could be met. Drivers
and stubs simulate features and functionalities, and have ability to serve features
that a module can provide. This reduces useless delay in testing and makes the
testing process faster.
• Stubs are mainly used in Top-Down integration testing while the Drivers are
used in Bottom-up integration testing, thus increasing the efficiency of testing
process.
• 1.Stubs :
Stubs are developed by software developers to use them in place of modules, if
the respective modules aren’t developed, missing in developing stage, or are
unavailable currently while Top-down testing of modules. A Stub simulates
module which has all the capabilities of the unavailable module. Stubs are used
when the lower-level modules are needed but are unavailable currently.
• Drivers :
Drivers serve the same purpose as stubs, but drivers are used in Bottom-up integration testing and are
also more complex than stubs. Drivers are also used when some modules are missing and unavailable at
time of testing of a specific module because of some unavoidable reasons, to act in absence of required
module. Drivers are used when high-level modules are missing and can also be used when lower-level
modules are missing.
• Ex : Suppose, you are told to test a website whose corresponding primary modules are, where each of
them is interdependent on each other, as follows:
• Module-A : Login page website,
• Module-B : Home page of the website
• Module-C : Profile setting
• Module-D : Sign-out page
• It’s always considered good practice to begin development of all modules parallelly because as soon as
each gets developed they can be integrated and could be tested further as per their corresponding
interdependencies order with a module. But in some cases, if any one of them is in developing stage or
not available in the testing process of a specific module, stubs or drivers could be used instead.
• Assume Module-A is developed. As soon as it’s developed, it undergoes testing, but it requires Module-
B, which isn’t developed yet. So in this case, we can use the Stubs or Drivers that simulate all features
and functionality that might be shown by actual Module-B. So, we can conclude that Stubs and drivers
are used to fulfill the necessity of unavailable modules. Similarly, we may also use Stubs or Drivers in
place of Module-C and Module-D if they are too not available.
 Integration Testing
• The modules that may work properly and independently may not
work when they are integrated.
• Integration Testing will test whether the modules work well
together.
• This will check whether the design is correct.
• Integration testing: Integrated collection of modules tested as a
group or partial system
• Integration plan specifies the order in which to combine modules
into partial systems
• Different approaches to integration testing
• Bottom-up
• Top-down
• Big-bang
• Sandwich
 Module Structure
A B
• We assume that
the uses hierarchy is
a directed acyclic graph.
• If there are cycles merge
D them to a single module

level 1 C H

level 0 E F G I

• A uses C and D; B uses D; C uses E and F; D uses F, G, H and I; H uses I

• Modules A and B are at level 3; Module D is at level 2
Modules C and H are at level 1; Modules E, F, G, I are at level 0
• level 0 components do not use any other components
• level i components use at least one component on level i-1 and no
component at a level higher than i-1
 Bottom-Up Integration
• Modules at lower levels are tested using the previously tested higher level modules

• Non-terminal modules are not tested in isolation

• Requires a module driver for each module to feed the test case input to the interface
of the module being tested
• However, stubs are not needed since we are starting with the terminal modules
and use already tested modules when testing modules in the lower levels
Bottom-up Integration

A B

C H

E F G I
 Top-down Integration
• Only modules tested in isolation are the modules which are at
the highest level

• After a module is tested, the modules directly called by that

module are merged with the already tested module and the
combination is tested

• Requires stub modules to simulate the functions of the missing

modules that may be called
• However, drivers are not needed since we are starting with
the modules which is not used by any other module and use
already tested modules when testing modules in the higher
levels
Top-down Integration

A B

C
H

E F G I
 Other Approaches to Integration
• Sandwich Integration
• Compromise between bottom-up and top-down testing
• Simultaneously begin bottom-up and top-down testing and meet at a
predetermined point in the middle
• Instead of completely going for top down or bottom up, a layer is
identified in between.
• Above this layer we go for top down and below this layer bottom up.

• Big Bang Integration

• Every module is unit tested in isolation
• After all of the modules are tested they are all integrated together at once
and tested
• No driver or stub is needed
• However, in this approach, it may be hard to isolate the bugs!
System Testing
• At the system testing level, the system is tested as a whole, and primarily functional testing
techniques are used to test the system.
• System testing ensures that each system function works as expected and it also tests for non-
functional requirements like performance, security, reliability, stress, load, etc.
• This is the only phase of testing which tests both functional and non-functional requirements of
the system. A team of the testing persons does the system testing under the supervision of a test
team leader. We also review all associated documents and manuals of the software.
• This verification activity is equally important and may improve the quality of the final product.
• A proper impact analysis should be done before fixing the defect. Sometimes, if the system
permits, instead of fixing the defects, they are just documented and mentioned as the known
limitations.
• This may happen in a situation when fixing is very time consuming or technically it is not possible
in the present design, etc.
• Progress of system testing also builds confidence in the development team as this is the first phase
in which the complete product is tested with a specific focus on the customer’s expectations. After
the completion of this phase, customers are invited to test the software.
System Testing Types
• Regression Testing
• Load Testing
• Smoke Testing
• Usability Testing
• Security Testing
• Performance Testing
• Stress Testing
System Testing: Regression Testing
• REGRESSION TESTING is a type of software testing that intends to ensure that changes
(enhancements or defect fixes) to the software have not adversely affected it. Rerunning of tests
can be on both functional and non-functional tests.
• Reasons for Regression Testing
• The need for Regression Testing could arise due to any of the changes below:
i. Defect fix
ii. New feature
iii. Change in an existing feature
iv. Code refactoring
v. Change in technical design / architecture
vi. Change in configuration / environment (hardware, software, network)
Smoke Testing
• Smoke testing covers most of the major functions of the software but none of them in depth.
• The result of this test is used to decide whether to proceed with further testing.
• If the smoke test passes, go ahead with further testing. If it fails, halt further tests and ask for a
new build with the required fixes.
• If an application is badly broken, detailed testing might be a waste of time and effort.
• Smoke test helps in exposing integration and major problems early in the cycle.
• It can be conducted on both newly created software and enhanced software.
• Smoke test is performed manually or with the help of automation tools/scripts. If builds are
prepared frequently, it is best to automate smoke testing.
Stress Testing
• It is a type of non-functional testing
• It involves testing beyond normal operational capacity, often to a breaking point, in order to
observe the results
• It is a form of software testing that is used to determine the stability of a given system
• It put greater emphasis on robustness, availability and error handling under a heavy load, rather
than on what would be considered correct behavior under normal circumstances
• The goals of such tests may be to ensure the software does not crash in conditions of insufficient
computational resources (such as memory, disk space, network request etc)
• Stress testing is also called fatigue testing
• The main purpose of stress testing: Make sure that the system fails and recovers easily, this
quality is also known as recoverability.
Acceptance Testing

• Acceptance Testing is done by the customer to test whether the

system is meeting the requirements
• Acceptance Testing is of two types
Alpha Testing
Beta Testing
Acceptance Testing
Alpha Testing –
The Alpha test is conducted in the developer’s environment by the
end-users. The environment might be simulated, with the developer
and the typical end-user present for the testing. The end-user uses
the software and records the errors and problems. Alpha test is
conducted in a controlled environment.
Acceptance Testing
Beta Testing –
The Beta test is conducted in the end-user’s environment. The
developer is not present for the beta testing. The beta testing is
always in the real-world environment which is not controlled by the
developer. The end-user records the problems and reports it back to
the developer at intervals. Based on the results of the beta testing the
software is made ready for the final release to the intended customer
base.
Static testing
• Static testing is the form of software testing where you
do not execute the code being examined.
• This technique could be called non-execution technique.
• It is primarily syntax checking of the code or manually
reviewing the code, requirements documents, design
documents etc. to find errors.
Static testing (Contd.)
• From the black box testing point of view, static testing
involves reviewing requirements and specifications.
• This is done with an eye toward completeness or
appropriateness for the task at hand.
• This is the verification portion of Verification and
Validation.
• The fundamental objective of static testing technique is to
improve the quality of the software products by finding
errors in early stages of software development life cycle.
Static testing (Contd.)
• Following are the main Static Testing techniques used:
• Formal Technical Reviews
• Walkthrough
• Code Inspection
 Formal Technical Reviews
• Formal Technical Reviews are conducted by software engineers.
• The primary objective is to find errors during the process so that they do not
become defects after release of software as they uncover errors in function, logic
design, or implementation.
• The idea is to have early discovery of errors so they do not propagate to the next
step in the process.
• They also ensure that the software has been represented according to predefined
standards and it is developed in a uniform manner.
• They make projects more manageable and help groom new resources as well as
provide backup and continuity.
 Formal Technical Reviews (Contd.)
• FTRs are usually conducted in a meeting that is successful only if it
is properly planned, controlled.
• The producer informs the Project Manager that the Work Product
is ready and the review is needed.
• The review meeting consists of 3-5 people and advanced
preparation is required.
• It is important that this preparation should not require more than 2
hours of work per person.
• It should focus on specific (and small) part of the overall software.
• For example, instead of the entire design, walkthroughs are
conducted for each component, or small group of components. By
narrowing focus, FTR has a high probability of uncovering errors.
Formal Technical Reviews (Contd.)
• It is important to remember that the focus is on a work
product (WP) for which the producer of the WP asks the
project leader for review.
• Project leader informs the review leader.
• The review leader evaluates the WP for readiness and if
satisfied generates copies of review material and
distributes to reviewers for advanced preparation.
Formal Technical Reviews Review Meetings
(Contd.)
• Review meeting (RM) is attended by the review leader, all reviewers, and the
producer.
• One of the reviewer takes the roles of recorder.
• Producer walks through the product, explaining the material while other
reviewers raise issues based upon their advanced preparation.
• When valid problems or errors are recorded, the recorder notes each one of
them.
• At the end of the RM, all attendees of the meeting must decide whether to:
Accept the product without further modification
Formal Technical Reviews Review Meetings
(Contd.)
• Reject the product due to severe errors
– Major errors identified
– Must review again after fixing
• Accept the product provisionally
– Minor errors to be fixed
– No further review
Formal Technical Reviews
Review Reporting and Record keeping(Contd.)

• During the FTR the recorder notes all the issues.

• They are summarized at the end and a review issue list is prepared.
• A summary report is produced that includes:
• What is reviewed
• Who reviewed it
• What were the findings and conclusions
• It then becomes part of project historical record.
 Formal Technical Reviews
(Contd.)
The review issue list
• It is sometimes very useful to have a proper review issue list. It has two objectives.
• Identify problem areas within the WP Action item checklist
• It is important to establish a follow-up procedure to ensure that items on the issue list
have been properly addressed.
Review Guidelines
• It is essential to note that an uncontrolled review can be worse than no review.
• The basis principle is that the review should focus on the product and not the
producer so that it does not become personal.
• Errors should be pointed out gently and the tone should be loose and constructive.
Walkthrough

• A walkthrough is a term describing the consideration of a process at an abstract

level.
• The term is often employed in the software industry (see software walkthrough)
to describe the process of inspecting algorithms and source code by following
paths through the algorithms or code as determined by input conditions and
choices made along the way.
• The purpose of such code walkthroughs is generally to provide assurance of the
fitness for purpose of the algorithm or code; and occasionally to assess the
competence or output of an individual or team.
Guidelines of Walkthrough
• The team performing code walk through should not be either too big or too
small.
• Ideally, it should consist of between three to seven members.
• Discussion should focus on discovery of errors and not on how to fix the
discovered errors.
• In order to foster cooperation and to avoid the feeling among engineers that they
are being evaluated in the code walk through meeting, managers should not
attend the walk through meetings.
Code Inspection
• The aim of code inspection is to discover some common types of errors caused
due to oversight and improper programming.
• During code inspection the code is examined for the presence of certain kinds of
errors, in contrast to the hand simulation of code execution done in code walk
throughs.
• For instance, consider the classical error of writing a procedure that modifies a
formal parameter while the calling routine calls that procedure with a constant
actual parameter Errors.
Code Inspection (Contd.)
• It is more likely that such an error will be discovered by looking for these kinds of
mistakes in the code, rather than by simply hand simulating execution of the
procedure.
• In addition to the commonly made errors, adherence to coding standards is also
checked during code inspection.
• Good software development companies collect statistics regarding different types
of errors commonly committed by their engineers and identify the type of errors
most frequently committed.
• Such a list of commonly committed errors can be used during code inspection to
look out for possible
Code Inspection (Contd.)
• Following is a list of some classical programming errors which can be checked
during code inspection:
• Use of un-initialized variables.
• Jumps into loops.
• Non-terminating loops.
• Incompatible assignments.
• Array indices out of bounds.
• Improper storage allocation and de-allocation.
• Mismatches between actual and formal parameter in procedure calls.
• Use of incorrect logical operators or incorrect precedence among operators.
• Improper modification of loop variables.
• Comparison of equally of floating point variables, etc.
Coding
• The goal of coding phase is to translate the design of the system into code in a
given programming language, which can be executed by a computer and that
performs the computation specified by the design.
• A coding standard lists several rules to be followed during coding, such as the way
variables are to be named, the way the code is to be laid out, error return
conventions, etc.
• A coding standard gives a uniform appearance to the codes written by different
engineers.
• It enhances code understanding.
• It encourages good programming practices.
Importance of Good Programming Style

To simplify software maintenance

To avoid problems
To simplify the testing process.
To achieve readability.
To improve modifiability
To improve transportability
To make a robust product
Coding standards and guidelines
• Rules for limiting the use of global: These rules list what types of data can be
declared global and what cannot.
• Contents of the headers preceding codes for different modules: The following
are some standard header data:
• Name of the module.
• Date on which the module was created.
• Author’s name.
• Modification history.
• Synopsis of the module.
• Different functions supported, along with their input/output parameters.
• Global variables accessed/modified by the module.
Coding standards and guidelines
• Naming conventions for global variables, local variables, and constant
identifiers: A possible naming convention can be that global variable names
always start with a capital letter, local variable names are made of small letters,
and constant names are always capital letters.
• Error return conventions and exception handling mechanisms: The way error
conditions are reported by different functions in a program are handled should be
standard within an organization.
Coding standards and guidelines: Recommended by SW Development
Organization

• Do not use a coding style that is too clever or too difficult to understand
• Do not use an identifier for multiple purposes
• The code should be well-documented
• The length of any function should not exceed 10 source lines
• Do not use goto statements
• Do not do hard coding