7/23/2024

Unit 1

INTRODUCTION

• Software testing is a process, or a series of

process, designed to ensure that the
computer code does what it is designed for
and does not do something that is not
intended. Software must be predictable and
consistent, offering no surprises to its users.

1
 7/23/2024

PURPOSE OF SOFTWARE TESTING

• While testing a program, it is required to add value to it. This
would mean increasing the quality or reliability of the
program i.e. finding errors and removing them.
• Thus, one must not test a program to show that it works but
with an aim to reveal errors and that too as much as possible.
• Testing is the process of executing a program with the intent
of finding errors. Thus, the objectives of testing can be listed
as below:
 Executing a program in order to find errors.
 A good test case is the one that has a high probability of
finding an undiscovered error.
 A successful test is the one that reports an as–yet
undiscovered error.
• The objective while designing tests is to come up with the test
cases that systematically uncover different classes of errors in
minimum possible time and effort.

Cont....
Productivity and Quality in Software
• In production of consumer goods and other products, every
manufacturing stage is subjected to quality control and testing from
component to final stage.
• If flaws are discovered at any stage, the product is either discarded
or cycled back for rework and correction.
• Productivity is measured by the sum of the costs of the material,
the rework, and the discarded components, and the cost of quality
assurance and testing.
• There is a tradeoff between quality assurance costs and
manufacturing costs: If sufficient time is not spent in quality
assurance, the reject rate will be high and so will be the net cost. If
inspection is good and all errors are caught as they occur, inspection
costs will dominate, and again the net cost will suffer.

2
 7/23/2024

Cont...
Testing goals
• There will be bugs because we are human. To
the extent that testing fails to reach its
primary goal, bug prevention, it must reach its
secondary goal, bug discovery. Bugs are not
always obvious. A bug is manifested in
deviations from expected behavior.

Cont...
Phases in a Tester's Mental Life
• PHASE 0-There's no difference between testing and
debugging.Other than in support of debugging, testing has no
purpose.
• PHASE 1-The purpose of testing is to show that the software
works.
• PHASE 2-The purpose of testing is to show that the software
doesn't work.
• PHASE 3-The purpose of testing is not to prove anything, but
to reduce the perceived risk of not working to an acceptable
value.
• PHASE 4-Testing is not an act. It is a mental discipline that
results in low-risk software without much testing effort.

3
 7/23/2024

Cont
Test Design
• programmers, testers, and programming managers know that
code must be designed and tested, many appear to be
unaware that tests themselves must be designed and tested-
designed by a process no less rigorous and no less controlled
than that used for code.
• The test-design phase of programming should be explicitly
identified.

Cont…
Testing Isn't Everything
• Inspection Methods -walkthroughs, desk checking, formal
inspections
• Design Style - adopting stylistic objectives such as test- ability,
openness, and clarity can do much to prevent bugs.
• Static Analysis Methods-These methods include anything that
can be done by formal analysis of source code during or in
conjunction with compilation.
• Languages-The source language can help reduce certain kinds
of bugs.
• Design Methodologies and Development Environment-The
design methodology (that is, the development process used
and the environ- ment in which that methodology is
embedded), can prevent many kinds of bugs.

4
 7/23/2024

Cont…
The Pesticide Paradox and the Complexity
Barrier
• First Law: The Pesticide Paradox-Every method you use to
prevent or find bugs leaves a residue of subtler bugs against
which those methods are ineffectual.
• Second Law: The Complexity Barrier-Software complexity (and
therefore that of bugs) grows to the limits of our ability to
manage that complexity.

Cont...
• Testing and Quality assurance costs for 'manufactured' items can
be as low as 2% in consumer products or as high as 80% in
products such as space-ships, nuclear reactors, and aircrafts,
where failures threaten life. Whereas the manufacturing cost of
software is trivial.
• The biggest part of software cost is the cost of bugs: the cost of
detecting them, the cost of correcting them, the cost of
designing tests that discover them, and the cost of running
those tests.
• For software, quality and productivity are indistinguishable
because the cost of a software copy is trivial.
• Testing and Test Design are parts of quality assurance should
also focus on bug prevention. A prevented bug is better than a
detected and corrected bug

5
 7/23/2024

A MODEL FOR TESTING

6
 7/23/2024

1.4.1 The Environment

• A program’s environment comprises of the hardware and the

software required for making it run. It also includes all the
programs interacting with it and used to create this program
under test e.g. operating system, loader, linker, compiler, etc.
• A testing environment comprises of a test plan which details
the tests that will be carried out and the test cases which
describe how we will test each of the components. Creation
of a test environment reduces the risk of errors occurring
when the software is put to actual use and thereby reducing
the downtime of the software.

1.4.2 The Program

• A lot of programs are too complicated to understand. Thus,

we need to simplify our concept i.e. ignore a lot of details, of
the program in order to test it. We might ignore the called
subroutines and concentrate on the program unless we
suspect the former. Similarly, we can ignore the process
details so that we can focus on the control structure of the
program or vice versa. If the simple model of an environment
does not explain the unexpected behavior we might have to
include more facts and details.

7
 7/23/2024

1.4.3 Bugs
Bugs are more dangerous than we expect them to be. A lot of
developers and testers have some preconceived notions
about the bugs. These are just myths and need to be removed
from the mind in order to test the system effectively. These
are listed below:
• Benign bug hypothesis: The belief that bugs are nice, tame
and logical. Only week bugs are logical and are open to
exposure based on logical means. Subtle bugs have no
definable patterns.
• Bug locality hypothesis: A bug occurring in a particular
module affects only that module locally and not the other
modules. However, this is not the case with the subtle bugs.
Their consequences are far removed from the cause in time
and space from the component in which they exist.

Cont...
• Control bug dominance: The belief that errors in the control structure are
more dominant. While bugs related to control-flow, data-flow and data
structures flow can be traced easily, subtle bugs that violate the data
structures boundaries and data/code separation can’t be found by merely
looking at the control structures.
• Code/Data separation: The belief that bugs respect the separation of data
and code. The distinction between data and code is hard to make in a real
system and this permits the existence of such bugs.
• Correction abide: The belief that a corrected bug remains corrected. We
might have changed one of the interacting components in the event of a
bug believing it as the cause, however, the bug might re-occur as it was
caused by some other component which we did not change.
• Silver bullets: The mistaken belief that A (language, design,
representation, environment, etc.) makes the program immune of bugs. It
might reduce the severity of bugs but not the complete occurrence of
bugs.
• Sadism effect: The belief that intuition and cunningness are sufficient to
detect bugs. This is true for easy bugs but the tough bugs need proper
methodology and techniques for detection.

8
 7/23/2024

1.4.4 Tests
Tests are formal procedures and are prone to bugs. Since, inputs are
prepared, outputs predicted, commands executed and results observed,
these steps are prone to errors. An unexpected test result could have
been caused due to either test bug or real bug. Bugs can creep in
documentation, inputs and the commands and over-shadow our
observation of results. Thus, it might be required to change the mentality
of the tester rather than the tests themselves.
1.4.5 Testing Levels
We can do different kinds of testing on a software system: unit/component
testing, integration testing and system testing. Each of these methods
differs in their objective and can be used in combination with one
another.
• Unit Testing: A unit is the smallest testable piece of software. It is the
work of a programmer and contains fewer lines of code. Unit testing is the
testing done to show that the unit does not satisfy its functional
specification and/or its implemented structure does not match the
intended design structure. The faults resulting from such testing are called
unit bugs.

Cont...
• Component Testing: A component is an integration of one or more units. A
unit is also a component and a component with subroutines it calls is a
component, etc. By this recursive definition, a component can be anything
ranging from a unit to a complete system. Component testing is the
testing done to show that the component does not satisfy its functional
specification and/or its implemented structure does not match the
intended design structure. The faults resulting from such testing are called
component bugs.
• Integration Testing: Integration is the process of aggregating the
components to create larger components. Integration testing is done to
show that even though the individual components were satisfactory
individually, as per the components’ tests, the combination of
components is incorrect or inconsistent. Integration testing is different
from testing the integrated components, which is just a higher level
component testing. Integration testing is aimed at exposing the problems
that arise from the combination of components.

9
 7/23/2024

Cont...
• System Testing: A system is a big component. System testing is done to
reveal the bugs that cannot be attributed to the components, their
inconsistencies or their interactions. System testing concerns issues and
behavior that can be revealed only by testing the entire integrated system
or a major chunk of it. It includes performance testing, security testing,
accountability testing, configuration sensitivity testing, start-up and
recovery testing.
1.4.6 The Role of Models
Testing is a process in which we create mental models of the environment,
program, human nature and the tests themselves. Each model is used
either till the system is accepted or until the model is no longer sufficient
for the purpose. Unexpected results force the revision of the model which
can be more detailed, complicated, abstract or simpler. Thus, the art of
testing comprises of creating, selecting, exploring and revising models.

Bugs
• A bug can be important in the terms of correction cost,
frequency, installation cost, and consequences. The bugs can
affect us in many ways depending upon their severity. The
severity must be measured in humanistic terms rather than
machine terms. The importance of a bug can be calculated
with the help of its frequency of occurrence, correction cost,
installation and consequential cost.
• A bug might be creep in any part of the software development
life cycle. It could be related to requirements specification,
requirements validation, requirement reviews, structural
bugs, etc. We will discuss all these in detail in this lesson.

10
 7/23/2024

Types of Bugs
1. Requirements, Features and Functionality
Bugs Requirements and Specifications
• Requirements and the specifications obtained
from them can be ambiguous or incomplete.
These bugs enter the system at the earliest
and leave it the last.
• Specifications problems lead to respective
feature problems. A feature can be wrong,
missing or superfluous. A missing feature is
the easiest to detect. A wrong feature could
have major design impacts.

Cont...
• The testing of functional bugs can be done
using the functional test techniques i.e. those
techniques which are based on a behavioral
description of software like transaction flow
testing, syntax testing, domain testing, logic
testing and state testing.

11
 7/23/2024

Cont...
2. Structural Bugs
Control and Sequence Bugs
• Control and sequence bugs result from left-out paths,
unreachable code, improper nesting of loops, and missing
steps in a process, rampaging GOTO’s, ill-conceived switches,
etc.
Logic Bugs
• Bugs in the logic e.g. behavior of logical statements in
combination with the logic operators, include non-existent
cases, improper layout of cases, improper simplification and
combination of cases, overlap of exclusive cases, etc.

Cont...
Processing Bugs
• Processing bugs include arithmetic, algebraic, mathematical
function evaluation, algorithm selection, incorrect conversion
from one data type to another and general processing. Other
problems include overflow ignoring, ignoring the difference
between positive and negative zero, improper use of greater-
than, greater-than-orequal, less-than, less-than-or-equal, etc.
Initialization Bugs
• Initialization bugs are common and both experienced
programmers and testers should test them. Improper and
superfluous initialization occurs. Although superfluous
initialization appears to be less harmful but it can affect the
performance. Typical bugs of this category are: forgetting to
initialize working space, registers or data areas; a bug in the
first value of a loop, wrong initialization to an improper
format or data type, etc.

12
 7/23/2024

Cont...
Data-flow Bugs and Anomalies
• Most of the initialization bugs are special cases of the data
flow anomalies. A data flow anomaly occurs when there is a
path along which we expect to do something unreasonable
with data e.g. using an uninitialized variable, attempting to
use a variable before it exists, modifying data and not storing
or using the result, etc.
3. Data Bugs
• Data bugs are those which arise from the specification of data
objects, their formats, the number of such objects and their
initial value. They are as common as the code bugs but are
often ignored.

Cont...
• Dynamic data is momentary irrespective of their purpose.
Their lifetime is normally limited to the processing time of a
transaction. They are stored in the storage objects. In case a
shared object is not initialized properly, it can lead to data-
dependent bugs caused by the residues from a previous use
of that object by another transaction. .
• Static data is fixed in form and content. They appear in the
source code or database directly or indirectly irrespective of
their purpose. any code executed at compile time or assembly
time or before, at load time, at installation time, or some
other time can lead to faulty static data

13
 7/23/2024

Cont...
4. Coding Bugs
• Coding bugs of or more kind can lead to bugs
of the other kinds. Syntax errors are normally
taken care of by themselves if the language
translator has adequate syntax checking. A
good translator can catch bugs related to
undeclared data, undeclared routines,
dangling code and other initialization
problems.

Cont...
5. Interface, Integration and System Bugs
• External interfaces are the means to communicate with the
outside world. These could be devices, actuators, sensors,
input terminals, printers and communication lines. In
principle, internal interfaces are not different from external
interfaces but they do differ in practice as the internal
environment is more controlled. Software bugs related to
hardware crop up from the misunderstanding of how does the
hardware works. Program bugs related to the operating
system are a combination of hardware architecture and
interface bugs, mostly caused by the misunderstanding of
what the operating system does.

14
 7/23/2024

Cont...
• Integration bugs are those that come into being with the
integration of and with the interfaces between, presumably
working and tested components. Most of these bugs result
from inconsistencies or incompatibilities between
components.
• System bug comprises of all those bugs that cannot be
ascribed to components or to their simple interactions, but
result from the totality of interactions between many
components like programs, data, hardware and the OS.

Cont...
6. Memory related Bugs
• Memory related bugs are caused due to improper handling of
the memory objects. These bugs can be used to cause
security breaches in software.
7.Concurrent Bugs
• Concurrent bugs are those that can occur only in multi-
threading or multi-process environment. They are caused
because of the bad synchronization of the operations of
multiple threads. These bugs are un-deterministic in nature
which makes them difficult to reproduce.

15
 7/23/2024

Testing and Design Styles

• Bad designs lead to bugs and are difficult to test; therefore,

the bugs remain. Good designs inhibit bugs before they occur
and are easy to test. The two factors are multiplicative, which
explains the productivity difference. The best test techniques
are useless when applied to abominable code; it is sometimes
easier to redesign a bad routine than to attempt to create
tests for it. The labor required producing new code and design
for the new code is much lesser than the labor required to
design thorough tests for an undisciplined, unstructured
monstrosity. Good testing works best on good code and good
designs.

16