SE COMPUTER SEM-II

SOFTWARE ENGINEERING

UNIT –VI SOFTWARE TESTING

(Source: Pressman, R. Software Engineering: A Practitioner’s Approach. McGraw-Hill, 2005)

• What is Software Testing
• Software testing is a process of identifying the correctness of
software by considering its all attributes (Reliability, Scalability,
Portability, Re-usability, Usability) and evaluating the execution
of software components to find the software bugs or errors or
defects.
• It involves testing of all components under the required services to confirm that whether
it is satisfying the specified requirements or not.
• The process is also providing the client with information about the quality of the
software.
• Testing is mandatory because it will be a dangerous situation if the software fails any of
time due to lack of testing.
• So, without testing software cannot be deployed to the end user.
What is Testing
Testing is a group of techniques to determine the correctness of the application under the
predefined script but, testing cannot find all the defect of application. 
The main intent of testing is to detect failures of the application so that failures can be
discovered and corrected.
Testing includes an examination of code and also the execution of code in various
environments, conditions as well as all the examining aspects of the code.
In the current scenario of software development, a testing team may be separate from the
development team so that Information derived from testing can be used to correct the
process of software development.
• The success of software depends upon acceptance of its targeted audience, easy
graphical user interface, strong functionality load test, etc.
• For example, the audience of banking is totally different from the audience of a
video game.
• Therefore, when an organization develops a software product, it can assess whether
the software product will be beneficial to its purchasers and other audience.
• Type of Software testing
• We have various types of testing available in the market, which are used to test
the application or the software.
• Manual testing
• The process of checking the functionality of an application as per the customer
needs without taking any help of automation tools is known as manual testing.
• While performing the manual testing on any application, we do not need any
specific knowledge of any testing tool, rather than have a proper understanding
of the product so we can easily prepare the test document.

• Manual testing can be further divided into three types of testing, which are as
follows:
• White box testing
• Black box testing
• Gray box testing
• White Box Testing
• In white-box testing, the developer will inspect every line of code before handing it
over to the testing team or the concerned test engineers.

• Subsequently, the code is noticeable for developers throughout testing; that's why
this process is known as WBT (White Box Testing).
• In other words, we can say that the developer will execute the complete white-box
testing for the particular software and send the specific application to the testing
team.
• The purpose of implementing the white box testing is to emphasize the flow of
inputs and outputs over the software and enhance the security of an application.
• White box testing is also known as open box testing, glass box testing, structural
testing, clear box testing, and transparent box testing.
• Black Box Testing
• Another type of manual testing is black-box testing.
• In this testing, the test engineer will analyze the software against requirements,
identify the defects or bug, and sends it back to the development team.
• Then, the developers will fix those defects, do one round of White box testing, and
send it to the testing team.
• Here, fixing the bugs means the defect is resolved, and the particular feature is
working according to the given requirement.
• In other words, we can say that black box testing is a process of checking the
functionality of an application as per the customer requirement.
• The source code is not visible in this testing; that's why it is known as black-box
testing.
Types of Black Box Testing
Black box testing further categorizes into two parts, which are as discussed below:
•Functional Testing
•Non-function Testing

Functional Testing

The test engineer will check all the components systematically against requirement
specifications is known as functional testing.
Functional testing is also known as Component testing.
In functional testing, all the components are tested by giving the value, defining the
output, and validating the actual output with the expected value.
Types of Functional Testing
Just like another type of testing is divided into several parts, functional testing is also
classified into various categories.
The diverse types of Functional Testing contain the following:
•Unit Testing
•Integration Testing
•System Testing
• 1. Unit Testing
• Unit testing is the first level of functional testing in order to test any software.
• In this, the test engineer will test the module of an application independently or
test all the module functionality is called unit testing.
• The primary objective of executing the unit testing is to confirm the unit
components with their performance.
• Here, a unit is defined as a single testable function of a software or an application.
• And it is verified throughout the specified application development phase.
• 2. Integration Testing
• Once we are successfully implementing the unit testing, we will go 
integration testing.
• It is the second level of functional testing, where we test the data flow between
dependent modules or interface between two features is called integration testing.
• The purpose of executing the integration testing is to test the statement's accuracy
between each module.
• 3. System Testing
• Whenever we are done with the unit and integration testing, we can proceed with
the system testing.
• In system testing, the test environment is parallel to the production environment.
• It is also known as end-to-end testing.
• In this type of testing, we will undergo each attribute of the software and test if the
end feature works according to the business requirement.
• And analysis the software product as a complete system.
• Non-function Testing
• The next part of black-box testing is non-functional testing. It provides detailed
information on software product performance and used technologies.
• Non-functional testing will help us minimize the risk of production and related
costs of the software.
• Non-functional testing is a combination of performance, load, stress, usability
and, compatibility testing.
•Automation testing
• Automation testing is a process of converting any manual test cases into the test scripts with the
help of automation tools, or any programming language is known as automation testing.
• With the help of automation testing, we can enhance the speed of our test execution because
here, we do not require any human efforts.
• We need to write a test script and execute those scripts.

• Automation Testing Tools

• Automation testing tools can describe in two categories, which are as follows:
• Functional Testing Tools
• QTP[Quick Test Professional]
• Rational Functional Tester [RFT]
• TestComplete

• Non-Functional Testing Tools

• LoadRunner
• Silk Performer
Software Testing Strategies

A strategic approach to testing

 Test strategies for conventional software
 Test strategies for object-oriented software
 Validation testing
 System testing
 The art of debugging
A strategy for software testing integrates the design of software test cases
into a well-planned series of steps that result in successful development of
the software
The strategy provides a road map that describes the steps to be taken,
when, and how much effort, time, and resources will be required
The strategy incorporates test planning, test case design, test execution,
and test result collection and evaluation
The strategy provides guidance for the practitioner and a set of milestones
for the manager
Because of time pressures, progress must be measurable and problems
must surface as early as possible
A Strategic Approach to Testing
General Characteristics of Strategic Testing

To perform effective testing, a software team should conduct effective

formal technical reviews
Testing begins at the component level and work outward toward the
integration of the entire computer-based system
Different testing techniques are appropriate at different points in time
Testing is conducted by the developer of the software and (for large
projects) by an independent test group
Testing and debugging are different activities, but debugging must be
accommodated in any testing strategy
Verification and Validation

Software testing is part of a broader group of activities called verification

and validation that are involved in software quality assurance
Verification (Are the algorithms coded correctly?)
 The set of activities that ensure that software correctly implements a specific
function or algorithm
Validation (Does it meet user requirements?)
 The set of activities that ensure that the software that has been built is
traceable to customer requirements
Organizing for Software Testing

Testing should aim at "breaking" the software

Common misconceptions
 The developer of software should do no testing at all
 The software should be given to a secret team of testers who will test it
unmercifully
 The testers get involved with the project only when the testing steps are about
to begin
Reality: Independent test group
 Removes the inherent problems associated with letting the builder test the
software that has been built
 Removes the conflict of interest that may otherwise be present
 Works closely with the software developer during analysis and design to ensure
that thorough testing occurs
A Strategy for Testing Conventional Software

System Testing

Validation Testing
Integration Testing

Unit Testing

Code
Design

Requirements
System Engineering
Levels of Testing for Conventional Software

Unit testing
 Concentrates on each component/function of the software as implemented in
the source code
Integration testing
 Focuses on the design and construction of the software architecture
Validation testing
 Requirements are validated against the constructed software
System testing
 The software and other system elements are tested as a whole
Testing Strategy applied to Conventional Software

Unit testing
 Exercises specific paths in a component's control structure to ensure complete
coverage and maximum error detection
 Components are then assembled and integrated
Integration testing
 Focuses on inputs and outputs, and how well the components fit together and
work together
Validation testing
 Provides final assurance that the software meets all functional, behavioral, and
performance requirements
System testing
 Verifies that all system elements (software, hardware, people, databases) mesh
properly and that overall system function and performance is achieved
Testing Strategy applied to Object-Oriented Software

Must broaden testing to include detections of errors in analysis and design

models
Unit testing loses some of its meaning and integration testing changes
significantly
Use the same philosophy but different approach as in conventional software
testing
Test "in the small" and then work out to testing "in the large"
 Testing in the small involves class attributes and operations; the main focus is
on communication and collaboration within the class
 Testing in the large involves a series of regression tests to uncover errors due to
communication and collaboration among classes
Finally, the system as a whole is tested to detect errors in fulfilling
requirements
When is Testing Complete?

There is no definitive answer to this question

Every time a user executes the software, the program is being tested
Sadly, testing usually stops when a project is running out of time, money, or
both
One approach is to divide the test results into various severity levels
 Then consider testing to be complete when certain levels of errors no longer
occur or have been repaired or eliminated
Ensuring a Successful Software Test Strategy

Specify product requirements in a quantifiable manner long before testing

commences
State testing objectives explicitly in measurable terms
Understand the user of the software (through use cases) and develop a
profile for each user category
Develop a testing plan that emphasizes rapid cycle testing to get quick
feedback to control quality levels and adjust the test strategy
Build robust software that is designed to test itself and can diagnose certain
kinds of errors
Use effective formal technical reviews as a filter prior to testing to reduce
the amount of testing required
Conduct formal technical reviews to assess the test strategy and test cases
themselves
Develop a continuous improvement approach for the testing process
through the gathering of metrics
Test Strategies for Conventional Software

A strategy for software testing integrates the design of software test cases
into a well-planned series of steps that result in successful development of
the software
The strategy provides a road map that describes the steps to be taken,
when, and how much effort, time, and resources will be required
The strategy incorporates test planning, test case design, test execution,
and test result collection and evaluation
The strategy provides guidance for the practitioner and a set of milestones
for the manager
Because of time pressures, progress must be measurable and problems
must surface as early as possible
 Test Strategies for
Conventional Software
Unit Testing

Focuses testing on the function or software module

Concentrates on the internal processing logic and data structures
Is simplified when a module is designed with high cohesion
 Reduces the number of test cases
 Allows errors to be more easily predicted and uncovered
Concentrates on critical modules and those with high cyclomatic complexity
when testing resources are limited
Targets for Unit Test Cases

Module interface
 Ensure that information flows properly into and out of the module
Local data structures
 Ensure that data stored temporarily maintains its integrity during all steps in an
algorithm execution
Boundary conditions
 Ensure that the module operates properly at boundary values established to
limit or restrict processing
Independent paths (basis paths)
 Paths are exercised to ensure that all statements in a module have been
executed at least once
Error handling paths
 Ensure that the algorithms respond correctly to specific error conditions
Common Computational Errors in Execution Paths

Misunderstood or incorrect arithmetic precedence

Mixed mode operations (e.g., int, float, char)
Incorrect initialization of values
Precision inaccuracy and round-off errors
Incorrect symbolic representation of an expression (int vs. float)
Other Errors to Uncover

Comparison of different data types

Incorrect logical operators or precedence
Expectation of equality when precision error makes equality unlikely (using
== with float types)
Incorrect comparison of variables
Improper or nonexistent loop termination
Failure to exit when divergent iteration is encountered
Improperly modified loop variables
Boundary value violations
Drivers and Stubs for Unit Testing

Driver
 A simple main program that accepts test case data, passes such data to the
component being tested, and prints the returned results
Stubs
 Serve to replace modules that are subordinate to (called by) the component to
be tested
 It uses the module’s exact interface, may do minimal data manipulation,
provides verification of entry, and returns control to the module undergoing
testing
Drivers and stubs both represent overhead
 Both must be written but don’t constitute part of the installed software product
Unit Test Procedure
Integration Testing

Defined as a systematic technique for constructing the software

architecture
At the same time integration is occurring, conduct tests to uncover errors
associated with interfaces
Objective is to take unit tested modules and build a program structure
based on the prescribed design
Two Approaches
 Non-incremental Integration Testing
 Incremental Integration Testing
Non-incremental Integration Testing

Commonly called the “Big Bang” approach

All components are combined in advance
The entire program is tested as a whole
Chaos results
Many seemingly-unrelated errors are encountered
Correction is difficult because isolation of causes is complicated
Once a set of errors are corrected, more errors occur, and testing appears to
enter an endless loop
Incremental Integration Testing

Three kinds
 Top-down integration
 Bottom-up integration
 Sandwich integration
The program is constructed and tested in small increments
Errors are easier to isolate and correct
Interfaces are more likely to be tested completely
A systematic test approach is applied
Top-down Integration

Modules are integrated by moving downward through the control hierarchy,

beginning with the main module
Subordinate modules are incorporated in either a depth-first or breadth-
first fashion
 DF: All modules on a major control path are integrated
 BF: All modules directly subordinate at each level are integrated
Advantages
 This approach verifies major control or decision points early in the test process
Disadvantages
 Stubs need to be created to substitute for modules that have not been built or
tested yet; this code is later discarded
 Because stubs are used to replace lower level modules, no significant data flow
can occur until much later in the integration/testing process
Bottom-up Integration

Integration and testing starts with the most atomic modules in the control
hierarchy
Advantages
 This approach verifies low-level data processing early in the testing process
 Need for stubs is eliminated
Disadvantages
 Driver modules need to be built to test the lower-level modules; this code is
later discarded or expanded into a full-featured version
 Drivers inherently do not contain the complete algorithms that will eventually
use the services of the lower-level modules; consequently, testing may be
incomplete or more testing may be needed later when the upper level modules
are available
Sandwich Integration

Consists of a combination of both top-down and bottom-up integration

Occurs both at the highest level modules and also at the lowest level
modules
Proceeds using functional groups of modules, with each group completed
before the next
 High and low-level modules are grouped based on the control and data
processing they provide for a specific program feature
 Integration within the group progresses in alternating steps between the high
and low level modules of the group
 When integration for a certain functional group is complete, integration and
testing moves onto the next group
Reaps the advantages of both types of integration while minimizing the
need for drivers and stubs
Requires a disciplined approach so that integration doesn’t tend towards
the “big bang” scenario
Regression Testing

Each new addition or change to baselined software may cause problems

with functions that previously worked flawlessly
Regression testing re-executes a small subset of tests that have already
been conducted
 Ensures that changes have not propagated unintended side effects
 Helps to ensure that changes do not introduce unintended behavior or
additional errors
 May be done manually or through the use of automated capture/playback tools
Regression test suite contains three different classes of test cases
 A representative sample of tests that will exercise all software functions
 Additional tests that focus on software functions that are likely to be affected
by the change
 Tests that focus on the actual software components that have been changed
Smoke Testing

Taken from the world of hardware

 Power is applied and a technician checks for sparks, smoke, or other dramatic
signs of fundamental failure
Designed as a pacing mechanism for time-critical projects
 Allows the software team to assess its project on a frequent basis
Includes the following activities
 The software is compiled and linked into a build
 A series of breadth tests is designed to expose errors that will keep the build
from properly performing its function
 The goal is to uncover “show stopper” errors that have the highest likelihood of
throwing the software project behind schedule
 The build is integrated with other builds and the entire product is smoke tested
daily
 Daily testing gives managers and practitioners a realistic assessment of the
progress of the integration testing
 After a smoke test is completed, detailed test scripts are executed

Benefits of Smoke Testing

Integration risk is minimized

 Daily testing uncovers incompatibilities and show-stoppers early in the testing
process, thereby reducing schedule impact
The quality of the end-product is improved
 Smoke testing is likely to uncover both functional errors and architectural and
component-level design errors
Error diagnosis and correction are simplified
 Smoke testing will probably uncover errors in the newest components that
were integrated
Progress is easier to assess
 As integration testing progresses, more software has been integrated and more
has been demonstrated to work
 Managers get a good indication that progress is being made
 Test Strategies for
Object-Oriented Software
Test Strategies for Object-Oriented Software

With object-oriented software, you can no longer test a single operation in

isolation (conventional thinking)
Traditional top-down or bottom-up integration testing has little meaning
Class testing for object-oriented software is the equivalent of unit testing
for conventional software
 Focuses on operations encapsulated by the class and the state behavior of the
class
Drivers can be used
 To test operations at the lowest level and for testing whole groups of classes
 To replace the user interface so that tests of system functionality can be
conducted prior to implementation of the actual interface
Stubs can be used
 In situations in which collaboration between classes is required but one or more
of the collaborating classes has not yet been fully implemented
Two different object-oriented testing strategies
Thread-based testing
 Integrates the set of classes required to respond to one input or event for the
system
 Each thread is integrated and tested individually
 Regression testing is applied to ensure that no side effects occur
Use-based testing
 First tests the independent classes that use very few, if any, server classes
 Then the next layer of classes, called dependent classes, are integrated
 This sequence of testing layer of dependent classes continues until the entire
system is constructed
Validation testing

Validation testing follows integration testing

The distinction between conventional and object-oriented software
disappears
Focuses on user-visible actions and user-recognizable output from the
system
Demonstrates conformity with requirements
Designed to ensure that
 All functional requirements are satisfied
 All behavioral characteristics are achieved
 All performance requirements are attained
 Documentation is correct
 Usability and other requirements are met (e.g., transportability, compatibility,
error recovery, maintainability)
After each validation test
 The function or performance characteristic conforms to specification and is
accepted
 A deviation from specification is uncovered and a deficiency list is created
A configuration review or audit ensures that all elements of the software
configuration have been properly developed, cataloged, and have the
necessary detail for entering the support phase of the software life cycle
Alpha and Beta Testing

Alpha testing
 Conducted at the developer’s site by end users
 Software is used in a natural setting with developers watching intently
 Testing is conducted in a controlled environment
Beta testing
 Conducted at end-user sites
 Developer is generally not present
 It serves as a live application of the software in an environment that cannot be
controlled by the developer
 The end-user records all problems that are encountered and reports these to
the developers at regular intervals
After beta testing is complete, software engineers make software
modifications and prepare for release of the software product to the entire
customer base
System Testing

Recovery testing
 Tests for recovery from system faults
 Forces the software to fail in a variety of ways and verifies that recovery is
properly performed
 Tests reinitialization, checkpointing mechanisms, data recovery, and restart for
correctness
Security testing
 Verifies that protection mechanisms built into a system will, in fact, protect it
from improper access
Stress testing
 Executes a system in a manner that demands resources in abnormal quantity,
frequency, or volume
Performance testing
 Tests the run-time performance of software within the context of an integrated
system
 Often coupled with stress testing and usually requires both hardware and
software instrumentation
 Can uncover situations that lead to degradation and possible system failure