Ch2 Theory

Software Testing and Quality Assurance
Theory and Practice

Chapter 2
Theory of Program Testing
© Naik & Tripathy 1

2
• Basic Concepts in Testing Theory

© Naik & Tripathy
• Theory of Goodenough and Gerhart

• Theory of Weyuker and Ostrand
• Theory of Gourlay
• Adequacy of Testing
• Limitations of Testing
• Summary
Outline of the Chapter
3
• Testing theory puts emphasis on

– Detecting defects through program execution
© Naik & Tripathy
– Designing test cases from different sources:

requirement specification, source code, and input
and output domains of programs
– Selecting a subset of tests cases from the entire
input domain
– Effectiveness of test selection strategies
– Test oracles used during testing
– Prioritizing the execution of test cases
Basic Concepts in Testing Theory
– Adequacy analysis of test cases

4
• Fundamental Concepts
– Let P be a program, and D be its input domain.
© Naik & Tripathy
Let T D. P(d) is the result of executing P with


input d.
Theory of Goodenough and Gerhart
Figure 2.1: Executing a program with a subset of the

input domain.
– OK(d): Represents the acceptability of P(d).
OK(d) = true iff P(d) is acceptable.
– SUCCESSFUL(T): T is a successful test iff t

T, OK(t).

– Ideal Test: T is an ideal test if OK(t), t T


=> OK(d), d D. 


5
• Fundamental Concepts (Contd.)

– Reliable Criterion: A test selection criterion C is
© Naik & Tripathy
reliable iff either every test selected by C is

successful, or no test selected is successful.
– Valid Criterion: A test selection criterion C is
valid iff whenever P is incorrect, C selects at least
one test set T which is not successful for P.
6
• Program faults occur due to our

– inadequate understanding of all conditions that a
© Naik & Tripathy
program must deal with.

– failure to realize that certain combinations of
conditions require special care.
• Kinds of program faults
– Logic fault
• Requirement fault
• Design fault
• Construction fault
– Performance fault
– Missing control-flow paths
– Inappropriate path selection
– Inappropriate or missing action
• Test predicate: It is a description of conditions
and combinations of conditions relevant to correct
operation of the program.
7
• Conditions for Reliability of a set of test

predicates C
© Naik & Tripathy
– Every branching condition must be represented by

a condition in C.
– Every potential termination condition must be
represented in C.
– Every condition relevant to the correct operation
of the program must be represented in C.
• Drawbacks of the Theory

– Difficulty in assessing the reliability and validity
of a criterion.
– The concepts of reliability and validity are
defined w.r.t. to a program. The goodness of a test
should be independent of individual programs.
– Neither reliability nor validity is preserved
throughout the debugging process.
8
• Reality: New test cases, in addition to the

planned test cases, are designed while
© Naik & Tripathy
performing testing. Let the test set be T.

• If a test set T does not reveal any more
faults, we face a dilemma:
– P is fault-free. OR
– T is not good enough to reveal (more) faults.
Need for evaluating the adequacy (i.e.
goodness) of T.

Adequacy of Testing
• Some ad hoc stopping criteria

– Allocated time for testing is over.
– It is time to release the product.
– Test cases no more reveal faults.
Adequacy of Testing
Figure 2.4: Context of applying test adequacy.
© Naik & Tripathy 9

10
• Two practical methods for evaluating test

adequacy
© Naik & Tripathy
– Fault seeding
– Program mutation
• Fault seeding
– Implant a certain number (say, X) of known faults
in P, and test P with T.
– If k% of the X faults are revealed, T has revealed
k% of the unknown faults.
– (More in Chapter 13)
• Program mutation
– A mutation of P is obtained by making a small
Adequacy of Testing
change to P.
– Some mutations are faulty, whereas the others are
equivalent to P.
– T is said to be adequate if it causes every faulty
mutations to produce unexpected results.
– (More in Chapter 3)
11
• Dijkstra’s famous observation

– Testing can reveal the presence of faults, but not
© Naik & Tripathy
their absence.
• Faults are detected by running P with a small
test set T, where |T| << |D|, where |.| denotes
the “size-of” function and “<<“ denoted
“much smaller.”
– Testing with a small test set raises the concern of
testing efficacy.
– Testing with a small test set is less expensive.
• The result of each test must be verified with
a test oracle.
Limitations of Testing
– Verifying a program output is not a trivial task.

– There are non-testable programs. A program is
non-testable if
• There is no test oracle for the program.
• It is too difficult to determine the correct
output.
12
• Theory of Goodenough and Gerhart

• Ideal test, Test selection criteria, Program
© Naik & Tripathy
faults, Test predicates

• Theory of Weyuker and Ostrand
• Uniformly ideal test selection
• Revealing subdomain
• Theory of Gourlay
• Testing system
• Power of test methods (“at least as good as”
relation)
• Adequacy of Testing
• Need for evaluating adequacy
• Methods for evaluating adequacy: fault
seeding and program mutation
Summary
• Limitations of Testing
• Testing is performed with a test set T, s.t. |T|
<< |D|.
• Dijkstra’s observation
• Test oracle problem

Ch2 Theory

Uploaded by

Copyright:

Available Formats

Ch2 Theory

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Ch2 Theory

Uploaded by

Copyright:

Available Formats

Software Testing and Quality Assurance

Theory and Practice

© Naik & Tripathy 1

• Basic Concepts in Testing Theory

• Theory of Goodenough and Gerhart

• Testing theory puts emphasis on

– Designing test cases from different sources:

– Adequacy analysis of test cases

Let T D. P(d) is the result of executing P with

Figure 2.1: Executing a program with a subset of the

– Ideal Test: T is an ideal test if OK(t), t T

• Fundamental Concepts (Contd.)

reliable iff either every test selected by C is

• Program faults occur due to our

program must deal with.

• Conditions for Reliability of a set of test

– Every branching condition must be represented by

• Drawbacks of the Theory

• Reality: New test cases, in addition to the

performing testing. Let the test set be T.

• Some ad hoc stopping criteria

Figure 2.4: Context of applying test adequacy.

© Naik & Tripathy 9

• Two practical methods for evaluating test

• Dijkstra’s famous observation

– Verifying a program output is not a trivial task.

• Theory of Goodenough and Gerhart

faults, Test predicates

You might also like