Software Testing

What ?
Why ?
Verification and Validation
Defects, Bug, Failure, Error
1
How many errors are still
remaining?
 Seed the code with some known
errors:
 artificial errors are introduced into
the program.
 Check how many of the seeded
errors are detected during testing.

2
Error Seeding
 Let:
 N be the total number of errors in the
system
 n of these errors be found by testing.
 S be the total number of seeded
errors,
 s of the seeded errors be found during
testing.
3
 Error Seeding
 n/N = s/S
 N = S n/s
 remaining defects:
N - n = n ((S - s)/ s)

4
Example
 100 errors were introduced.
 90 of these errors were found
during testing
 50 other errors were also found.
 Remaining errors=
 N - n = n ((S - s)/ s)
=50 (100-90)/90 = 6
5
Error Seeding
 The kind of seeded errors should
match closely with existing errors:
 However, it is difficult to predict the
types of errors that exist.
 Categories of remaining errors:
 can be estimated by analyzing historical
data from similar projects.

6
Question

Which of the following is not

part of the Test type ?
A.Functional Testing
B.Statement Testing
C.Unit Testing
D.System Testing
7
Black-box Testing
 Test cases are designed using only
functional specification of the
software:
 without any knowledge of the internal
structure of the software.
 For this reason, black-box testing is
also known as functional testing.

8
White-box Testing
 Designing white-box test
cases:
 requires knowledge about the
internal structure of software.
 white-box testing is also
called structural testing.

9
Black-box Testing
 Two main approaches to
design black box test cases:
 Equivalence class partitioning
 Boundary value analysis

10
Q) True or false? Generally, in
practice, developers
exhaustively test software.
a. True
b. False

11
White-Box Testing
 There exist several popular white-box
testing methodologies:
 Statement coverage
 branch coverage
 path coverage
 condition coverage
 mutation testing
 data flow-based testing
12
Statement Coverage
 Statement coverage
methodology:
 design test cases so that
 every statement in a
program is executed at least
once.

13
Statement Coverage
 The principal idea:
 unless a statement is executed,
 we have no way of knowing if
an error exists in that
statement.

14
Example
 int f1(int x, int y){
 1 while (x != y){
 2 if (x>y) then
 3 x=x-y;
 4 else y=y-x;
 5}
 6 return x; }

Test suite is: (3,3), (4,3) and (3,4) which

will do 100% statement coverage 15
Example
1. print (int a, int b) {
2. int sum = a+b;
3. if (sum>0)
4. print (“Output")
5. else
6. print (“Output")
7. }

Test Cases ..?

16
Branch Coverage
 Test cases are designed
such that:
 different branch conditions
 given true and false values in
turn.

17
Branch Coverage
 Branch testing guarantees
statement coverage:
 a stronger testing compared
to the statement coverage-
based testing.

18
Example
 int f1(int x,int y){
 1 while (x != y){
 2 if (x>y) then
 3 x=x-y;
 4 else y=y-x;
 5}
 6 return x; }

19
Example
 Test cases for branch
coverage can be:
 {(x=3,y=3),(x=3,y=2),
(x=4,y=3), (x=3,y=4)}

20
Question

Which are the following statements are

true
a.Black box Testing is based on the external specification
of the system
b.White box Testing is based on the internal structure and
logic of the system
c.White box Testing can do without knowledge of how the
system is constructed
d.Both a and b

21
Condition Coverage
 Test cases are designed such
that:
 each component of a composite
conditional expression
 given both true and false values.

22
Example
 Consider the conditional
expression
 ((c1.and.c2).or.c3):
 Each of c1, c2, and c3 are
exercised at least once,
 i.e. given true and false values.

23
Condition coverage

 Consider a boolean
expression having n
components:
 for condition coverage we
n
require 2 test cases. (n is no.
of condtions)
24
Condition Coverage

 2 Condition  3 Condition
TTT
TT TTF
TF TFF
FFF
FF FFT
FTT
FT TFT
FTF 25
Path Coverage
 Design test cases such that:
 all linearly independent
paths in the program are
executed at least once.

26
Linearly independent
paths
 Defined in terms of
 control flow graph (CFG) of a
program.

27
Path coverage-based
testing
 To understand the path
coverage-based testing:
 we need to learn how to draw
control flow graph of a
program.

28
Control flow graph
(CFG)
 A control flow graph (CFG)
describes:
 the sequence in which different
instructions of a program get
executed.
 the way control flows through
the program.
29
How to draw Control
flow graph?
 Number all the statements of
a program.
 Numbered statements:
 represent nodes of the control
flow graph.

30
How to draw Control
flow graph?

 Sequence: 1
 1 a=5; b=2;
2
 2 b=a*b-1;

31
How to draw Control
flow graph?

 Selection:
1
1 if(a>b) then 2 3
2 c=3; 4
3 else c=5;
4 c=c*c;

32
How to draw Control
flow graph?

 Iteration: 1

1 while(a>b){ 2
2 b=b*a; 3
3 b=b-1;} 4
4 c=b+d;

33
Example
 int f1(int x,int y){
 1 while (x != y){
 2 if (x>y) then
 3 x=x-y;
 4 else y=y-x;
 5}
 6 return x; }

34
Path
 A path through a program:
 a node and edge sequence from
the starting node to a terminal
node of the control flow graph.
 There may be several terminal
nodes for program.
35
Independent path
 Any path through the
program:
 introducing at least one new
node:
 that is not included in any other
independent paths.
36
Linear Independent Path
1
2
3 4
5
6
1,2,3,5,1,6
1,2,4,5,1,6
1,6
37
Independent path
 It is straight forward:
 to identify linearly independent
paths of simple programs.
 For complicated programs:
 it is not so easy to determine the
number of independent paths.

38
With 100 percent branch
coverage, every branch of every
statement may or may not be
executed.
A) True
B) False
39
McCabe's cyclomatic
metric
 An upper bound:
 for the number of linearly
independent paths of a program
 Provides a practical way of
determining:
 the maximum number of linearly
independent paths in a program.
40
McCabe's cyclomatic
metric
 Given a control flow graph G,
cyclomatic complexity V(G):
 V(G)= E-N+2
 N is the number of nodes in G
 E is the number of edges in G

41
Example Control Flow
Graph
1
2
3 4
5
6

42
Example
 Cyclomatic complexity =
7-6+2 = 3.

43
Cyclomatic complexity
 Another way of computing cyclomatic
complexity:
 inspect control flow graph
 determine number of bounded areas in
the graph
 V(G) = Total number of bounded
areas + 1
44
Bounded area
 Any region enclosed by a
nodes and edge sequence.

45
Example Control Flow
Graph
1
2
3 4
5
6

46
Example
 From a visual examination of
the CFG:
 the number of bounded areas is
2.
 cyclomatic complexity =
2+1=3.
47
Create CFG Graph

if A = 10 then
if B > C
A=B
else A = C
endif
endif
print A, B, C

48
Cyclomatic complexity
 McCabe's metric provides:
 a quantitative measure of testing
difficulty and the ultimate reliability
 Intuitively,
 number of bounded areas increases
with the number of decision nodes
and loops.

49
Cyclomatic complexity

 The cyclomatic complexity of a

program provides:
 a lower bound on the number of
test cases to be designed
 to guarantee coverage of all
linearly independent paths.

50
Cyclomatic complexity

 Knowing the number of test cases

required:
 does not make it any easier to
derive the test cases,
 only gives an indication of the
minimum number of test cases
required.
51
An input field takes the number from 10 to 50. The
boundary values for testing this field are: Select one:
a.10,30,50
b. 9,30,51
c. 10,15,50,55
d. 9,10,50,51

52
Path testing
 The tester proposes:
 an initial set of test data using
his experience and judgement.

53
Path testing
 A dynamic program analyzer is
used:
 to indicate which parts of the
program have been tested
 the output of the dynamic analysis
 used to guide the tester in selecting
additional test cases.
54
Derivation of Test
Cases
 Let us discuss the steps:
 to derive path coverage-
based test cases of a
program.

55
Derivation of Test
Cases
 Draw control flow graph.
 Determine V(G).
 Determine the set of linearly
independent paths.
 Prepare test cases:
 to force execution along each path.
56
Example
 int f1(int x,int y){
 1 while (x != y){
 2 if (x>y) then
 3 x=x-y;
 4 else y=y-x;
 5}
 6 return x; }

57
Example Control Flow
Diagram
1
2
3 4
5
6

58
Derivation of Test Cases

 Number of independent paths:

3
 1,6 test case (x=1, y=1)
 1,2,3,5,1,6 test case(x=1, y=2)
 1,2,4,5,1,6 test case(x=2, y=1)

59
The test levels are performed in which of
the
following order?
a. Unit, Integration, System, Acceptance
b. It is based on the nature of the project.
c. Unit, Integration, Acceptance, System
d. Unit, System, Integration, Acceptance

60
Software Reliability
 It is truth-worthiness or dependability.
 Defined as probability of the product working
correctly over a given period of time
 It also depends not only number of errors but
also on the exact location of errors.
 It also depends on execution profile.
 It keep changing as errors are detected an fixed

61
 Selenium
 Selenium is a leading test automation framework for web
applicationsFormal Technical Review
 Use multiple programming languages like Java, C#, Python
etc to create Selenium Test ScriptsCode Inspection
 Selenium Software is not just a single tool but a suite of
software

o Selenium Integrated Development Environment

(IDE)
o Selenium Remote Control (RC)
o WebDriver
o Selenium Grid
Selenium
 Selenium
 Selenium was created by Jason Huggins in
2004
 He was working on a web application that
required frequent testing.
 He created a JavaScript program that would
automatically control the browser’s actions.
He named this program as the

“JavaScriptTestRunner“
 He made JavaScriptRunner open-source
which was later re-named as Selenium
Core.
 Selenium

Issue with the Selenium core

Same Origin policy prohibits JavaScript code from accessing
elements from a domain that is different from where it was
launched.
 Selenium

Selenium Remote Control

Testers using Selenium Core had to install the whole
application under test and the web server on their own
local computers because of the restrictions imposed by the
same origin policy.
Paul Hammant, decided to create a server that will act as
an HTTP proxy to “trick” the browser into believing that
Selenium Core and the web application being tested come
from the same domain.
his system became known as the Selenium Remote
Control or Selenium 1
 Selenium

Selenium Grid
Selenium Grid was developed by Patrick
Lightbod
To address the need of minimizing test
execution times as much as possible.
He initially called the system “Hosted QA.”
It captures the browser screenshots
during significant stages,
Summary
 Designing test cases for black box
testing:
 does not require any knowledge of
how the functions have been
designed and implemented.
 Test cases can be designed by
examining only SRS document.
68
Summary
 White box testing:
 requires knowledge about
internals of the software.
 Design and code is required.

69
Summary
 We have discussed a few white-
box test strategies.
 Statement coverage
 branch coverage
 condition coverage
 path coverage
70
Summary
 A stronger testing strategy:
 provides more number of
significant test cases than a
weaker one.
 Condition coverage is strongest
among strategies we discussed.
71
Summary
 We discussed McCabe’s
Cyclomatic complexity metric:
 provides an upper bound for
linearly independent paths
 correlates with understanding,
testing, and debugging difficulty of
a program.
72