0% found this document useful (0 votes)
7 views82 pages

07

Uploaded by

ctamojit1
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
7 views82 pages

07

Uploaded by

ctamojit1
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 82

Testing and

Debugging

1
Organization of this
lecture
Important concepts in program testing
Black-box testing:
 equivalence partitioning
 boundary value analysis
White-box testing
Debugging
Unit, Integration, and System testing
Summary

2
How do you test a
program?
Input test data to the
program.
Observe the output:
 Check if the program
behaved as expected.

3
How do you test a
system?

4
How do you test a
system?
If the program does not
behave as expected:
 note the conditions under
which it failed.
 later debug and correct.

5
Error, Faults, and
Failures
A failure is a
manifestation of an error
(aka defect or bug).
 mere presence of an error
may not lead to a failure.

6
Error, Faults, and
Failures
A fault is an incorrect state
entered during program
execution:
 a variable value is different
from what it should be.
 A fault may or may not not
lead to a failure.

7
Test cases and Test
suites
Test a software using a set
of carefully designed test
cases:
 the set of all test cases is
called the test suite

8
Test cases and Test
suites
A test case is a triplet [I,S,O]
 I is the data to be input to the
system,
 S is the state of the system at
which the data will be input,
 O is the expected output of
the system.

9
Verification versus Validation

Verification - Process of evaluating the


intermediary work products of a software
development lifecycle to check if we are in
the right track of creating the final
product.
Validation - Process of evaluating the
final product to check whether the
software meets the business needs.

10
Verification versus
Validation
Verification is concerned
with phase containment of
errors,
 whereas the aim of
validation is that the final
product be error free.
11
Design of Test
Cases

Exhaustive testing of any non-
trivial system is impractical:
 input data domain is extremely
large.
Design an optimal test suite:
 of reasonable size and
 uncovers as many errors as
possible.

12
Design of Test
Cases
If test cases are selected randomly:
 many test cases would not contribute to
the significance of the test suite,
 would not detect errors not already being
detected by other test cases in the suite.
Number of test cases in a randomly
selected test suite:
 not an indication of effectiveness of
testing.

13
Design of Test
Cases
Testing a system using a large
number of randomly selected test
cases:
 does not mean that many errors in
the system will be uncovered.
Consider an example for finding
the maximum of two integers x
and y.
14
Design of Test
Cases
The code has a simple programming
error:
 If (x>y) max = x;
else max = x;
test suite {(x=3,y=2);(x=2,y=3)} can
detect the error,
a larger test suite {(x=3,y=2);(x=4,y=3);
(x=5,y=1)} does not detect the error.

15
Design of Test
Cases
Systematic approaches are
required to design an
optimal test suite:
 each test case in the suite
should detect different
errors.
16
Design of Test
Cases
There are essentially two
main approaches to design
test cases:
 Black-box approach
 White-box (or glass-box)
approach
17
Black-box Testing
Test cases are designed using
only functional specification of
the software:
 without any knowledge of the
internal structure of the software.
For this reason, black-box
testing is also known as
functional testing.
18
White-box Testing
Designing white-box test
cases:
 requires knowledge about the
internal structure of software.
 white-box testing is also
called structural testing.
 In this unit we will not study
white-box testing.
19
Black-box Testing
There are essentially two
main approaches to design
black box test cases:
 Equivalence class
partitioning
 Boundary value analysis
20
Equivalence Class
Partitioning
Input values to a program are
partitioned into equivalence classes.
Partitioning is done such that:
 program behaves in similar ways
to every input value belonging to
an equivalence class.

21
Why define equivalence
classes?
Test the code with just one
representative value from
each equivalence class:
 as good as testing using any
other values from the
equivalence classes.

22
Equivalence Class
Partitioning
How do you determine the
equivalence classes?
 examine the input data.
 few general guidelines for
determining the equivalence
classes can be given

23
Equivalence Class
Partitioning
If the input data to the program
is specified by a range of
values:
 e.g. numbers between 1 to 5000.
 one valid and two invalid
equivalence classes are defined.
1 5000

24
Equivalence Class
Partitioning
If input is an enumerated set of
values:
 e.g. {a,b,c}
 one equivalence class for valid
input values
 another equivalence class for
invalid input values should be
defined.

25
Example
A program reads an input
value in the range of 1 and
5000:
 computes the square root of
the input number
SQR
T

26
Example (cont.)
There are three equivalence
classes:
 the set of negative integers,
 set of integers in the range of 1
and 5000,
 integers larger than 5000.
1 5000

27
Example (cont.)
The test suite must include:
 representatives from each of
the three equivalence classes:
 a possible test suite can be:
{-5,500,6000}.
1 5000

28
Boundary Value
Analysis

Some typical programming errors
occur:
 at boundaries of equivalence
classes
 might be purely due to
psychological factors.
Programmers often fail to see:
 special processing required at the
boundaries of equivalence classes.
29
Boundary Value
Analysis
Programmers may improperly
use < instead of <=
Boundary value analysis:
 select test cases at the
boundaries of different
equivalence classes.

30
Example
For a function that
computes the square root of
an integer in the range of 1
and 5000:
 test cases must include the
values:
1 {0,1,5000,5001}.
5000

31
Debugging
Once errors are identified:
 it is necessary identify the precise
location of the errors and to fix them.
Each debugging approach has its
own advantages and
disadvantages:
 each is useful in appropriate
circumstances.

32
Brute-force method
This is the most common
method of debugging:
 least efficient method.
 program is loaded with print
statements
 print the intermediate values
 hope that some of printed values
will help identify the error.

33
Symbolic Debugger
Brute force approach becomes
more systematic:
 with the use of a symbolic
debugger,
 symbolic debuggers get their name
for historical reasons
 early debuggers let you only see
values from a program dump:
determine which variable it corresponds
to.

34
Symbolic Debugger
Using a symbolic debugger:
 values of different variables can be
easily checked and modified
 single stepping to execute one
instruction at a time
 break points and watch points can
be set to test the values of
variables.

35
Backtracking
This is a fairly common
approach.
Beginning at the statement
where an error symptom has
been observed:
 source code is traced
backwards until the error is
discovered.
36
Backtracking
Unfortunately, as the number
of source lines to be traced
back increases,
 the number of potential backward
paths increases
 becomes unmanageably large for
complex programs.

37
Cause-elimination
method
Determine a list of causes:
 which could possibly have
contributed to the error symptom.
 tests are conducted to eliminate
each.
A related technique of
identifying error by examining
error symptoms:
 software fault tree analysis.
38
Program Slicing
This technique is similar to back
tracking.
However, the search space is
reduced by defining slices.
A slice is defined for a particular
variable at a particular statement:
 set of source lines preceding this
statement which can influence the
value of the variable.
39
Example
int main(){
int i,s;
i=1; s=1;
while(i<=10){
s=s+i;
i++;}
printf(“%d”,s);
printf(“%d”,i);
}

40
Debugging
Guidelines
Debugging usually requires a thorough
understanding of the program design.
Debugging may sometimes require full
redesign of the system.
A common mistake novice
programmers often make:
 not fixing the error but the error
symptoms.

41
Debugging
Guidelines
Be aware of the possibility:
 an error correction may
introduce new errors.
After every round of error-
fixing:
 regression testing must be
carried out.
42
Program Analysis
Tools
An automated tool:
 takes program source code as input
 produces reports regarding several
important characteristics of the
program,
 such as size, complexity, adequacy
of commenting, adherence to
programming standards, etc.

43
Program Analysis
Tools
Some program analysis tools:
 produce reports regarding the
adequacy of the test cases.
There are essentially two categories
of program analysis tools:
 Static analysis tools
 Dynamic analysis tools

44
Static Analysis
Tools
Static analysis tools:
 assess properties of a
program without executing it.
 Analyze the source code
provide analytical conclusions.

45
Static Analysis
Tools
Whether coding standards have been
adhered to?
 Commenting is adequate?
Programming errors such as:
 uninitialized variables
 mismatch between actual and formal
parameters.
 Variables declared but never used, etc.

46
Static Analysis
Tools
Code walk through and
inspection can also be
considered as static analysis
methods:
 however, the term static program
analysis is generally used for
automated analysis tools.
47
Example
IBM Rational AppScan Source Edition
HP Fortify Software Static Code
Analyzer
Veracode
Eclipse (software)
IntelliJ IDEA
Google's Closure Compiler

48
Dynamic Analysis
Tools
Dynamic program
analysis tools require the
program to be executed:
 its behavior recorded.
 Produce reports such as
adequacy of test cases.

49
Example
Intel Thread Checker
IBM Rational AppScan
HP Security Suite
ClearSQL
Dmalloc

50
Testing
The aim of testing is to
identify all defects in a
software product.
However, in practice even after
thorough testing:
 one cannot guarantee that the
software is error-free.
51
Testing
The input data domain of
most software products is
very large:
 it is not practical to test
the software exhaustively
with each input data value.

52
Testing
Testing does however expose
many errors:
 testing provides a practical way
of reducing defects in a system
 increases the users' confidence
in a developed system.

53
Testing
Testing is an important
development phase:
 requires the maximum effort among
all development phases.
In a typical development
organization:
 maximum number of software engineers
can be found to be engaged in testing
activities.

54
Testing
Many engineers have the
wrong impression:
 testing is a secondary activity
 it is intellectually not as
stimulating as the other
development activities, etc.

55
Testing
Testing a software product is in
fact:
 as much challenging as initial
development activities such
as specification, design, and
coding.
Also, testing involves a lot
of creative thinking.
56
Testing
Software products are
tested at three levels:
 Unit testing
 Integration testing
 System testing

57
Unit testing
During unit testing, modules
are tested in isolation:
 If all modules were to be tested
together:
it may not be easy to
determine which module has
the error.

58
Unit testing
Unit testing reduces
debugging effort several
folds.
 Programmers carry out
unit testing immediately
after they complete the
coding of a module.
59
Integration testing
After different modules of a
system have been coded and
unit tested:
 modules are integrated in steps
according to an integration plan
 partially integrated system is
tested at each integration step.

60
System Testing
System testing involves:
 validating a fully
developed system against
its requirements.

61
Integration Testing
Develop the integration plan
by examining the structure
chart :
 big bang approach
 top-down approach
 bottom-up approach
 mixed approach

62
Big bang Integration
Testing
Big bang approach is the
simplest integration testing
approach:
 all the modules are simply put
together and tested.
 this technique is used only for
very small systems.

63
Big bang Integration
Testing
Main problems with this approach:
 if an error is found:
it is very difficult to localize the error
the error may potentially belong to any
of the modules being integrated.
 debugging errors found during big
bang integration testing are very
expensive to fix.

64
Bottom-up Integration
Testing
Integrate and test the bottom
level modules first.
A disadvantage of bottom-up
testing:
 when the system is made up of a
large number of small subsystems.
 This extreme case corresponds to
the big bang approach.

65
Top-down integration
testing
Top-down integration testing starts
with the main routine:
 and one or two subordinate routines in the
system.
After the top-level 'skeleton’ has been
tested:
 immediate subordinate modules of the
'skeleton’ are combined with it and tested.

66
Mixed integration testing

Mixed (or sandwiched)


integration testing:
 uses both top-down and
bottom-up testing
approaches.
 Most common approach
67
Integration Testing
In top-down approach:
 testing waits till all top-level
modules are coded and unit
tested.
In bottom-up approach:
 testing can start only after
bottom level modules are ready.

68
System Testing
There are three main
kinds of system testing:
 Alpha Testing
 Beta Testing
 Acceptance Testing

69
Alpha Testing
System testing is carried
out by the test team within
the developing
organization.

70
Beta Testing
System testing performed
by a select group of
friendly customers.

71
Acceptance Testing
System testing performed
by the customer himself:
 to determine whether the
system should be accepted
or rejected.

72
Stress Testing
Stress testing (aka endurance
testing):
 impose abnormal input to stress
the capabilities of the software.
 Input data volume, input data rate,
processing time, utilization of
memory, etc. are tested beyond
the designed capacity.
73
How many errors are still
remaining?
Seed the code with some
known errors:
 artificial errors are introduced
into the program.
 Check how many of the seeded
errors are detected during
testing.
74
Error Seeding
Let:
 N be the total number of errors in
the system
 n of these errors be found by
testing.
 S be the total number of seeded
errors,
 s of the seeded errors be found
during testing.
75
Error Seeding
n/N = s/S
N = S n/s
remaining defects:
N - n = n ((S - s)/ s)

76
Example
100 errors were introduced.
90 of these errors were found
during testing
50 other errors were also found.
Remaining errors=
50 (100-90)/90 = 6

77
Error Seeding
The kind of seeded errors should
match closely with existing errors:
 However, it is difficult to predict the
types of errors that exist.
Categories of remaining errors:
 can be estimated by analyzing
historical data from similar projects.

78
Summary
Exhaustive testing of almost
any non-trivial system is
impractical.
 we need to design an optimal
test suite that would expose
as many errors as possible.

79
Summary
If we select test cases randomly:
 many of the test cases may not add
to the significance of the test suite.
There are two approaches to testing:
 black-box testing
 white-box testing.

80
Summary
Black box testing is also known as
functional testing.
Designing black box test cases:
 requires understanding only SRS document
 does not require any knowledge about
design and code.
Designing white box testing requires
knowledge about design and code.
81
Summary
We discussed black-box test
case design strategies:
 equivalence partitioning
 boundary value analysis
We discussed some important
issues in integration and
system testing.

82

You might also like