CA 7503

CA 7503
Software Testing
Lecture 2 :
The Realities of Software Testing
What is Software
Testing?
• Software testing is a process of executing
a program or application with the intent of
finding the software bugs.
• It can also be stated as the process of

validating and verifying that a
software program or application or
product: Meets the business and
technical requirements that guided it's3
What is Software
Testing?
• Testing software typically involves:
• Executing software with inputs representative
of actual operation conditions (or operational
profiles )
• Comparing produced/expected outputs
• Comparing resulting/expected states
• Measuring execution characteristics (e.g.,

memory used, time consumed, etc.)
4
Terminology
• Fault: an imperfection that may lead to a
failure
• E.g., missing/incorrect code that may result in a
failure
• Bug: another name for a fault in code
• Error: where the system state is incorrect

but may not have been observed
• Failure: some failure to deliver the5

A few more definitions
• Test Case: set of inputs, execution conditions, and expected
results developed for a particular objective
• Test Suite: collection of test cases, typically related by a testing

goal or an implementation dependency
• Test Driver: class or utility program that applies test cases
• Test harness: system of test drivers and other tools that

support test execution.
• Test Strategy: algorithm or heuristic to create test cases from a

representation, implementation, or a test model.
• Oracle: means to check the output from a program is correct for

the given input.
• Stub: partial temporary implementation of a component (usually6

A Software Testing
Process
7
Software process models are an ideal …
not reality
• No software development effort follows a

process model perfectly. Why?
• The specification never corresponds to the
customers needs perfectly.
• There is never enough time to perform all of the
testing.
• Nevertheless, an ideal model is helpful to make

progress.
• Trade offs and concessions are inevitable.
8
Software testing axioms
1.It is impossible to test a program completely.
2.Software testing is a risk-based exercise.
3.Testing cannot show the absence of bugs.
4.The more bugs you find, the more bugs there are.
5.Not all bugs found will be fixed.
6.It is difficult to say when a bug is indeed a bug.
7.Specifications are never final.
8.Software testers are not the most popular members of
project.
9.Software testing is a disciplined and technical profession.
9
What is an axiom
anyway?
• An axiom is a sentence or proposition that
is not proved or demonstrated and is
considered as obvious or as an initial
necessary consensus for a theory building
or acceptation.
• Therefore, it is taken for granted as true,

and serves as a starting point for deducing
and inferring other (theory dependent)
10
Peano’s axioms for the structure of
natural numbers
1. 1 is a natural number.
2. Every natural number is equal to itself (equality is
reflexive).
3. For all natural numbers a and b, a = b if and only if b = a
(equality is symmetric).
4. For all natural numbers a, b, and c, if a = b and b = c then a
= c (equality is transitive).
5. If a = b and b is a natural number then a is a natural
number.
6. If a is a natural number then successor(a) is a natural
number.
7. If a and b are natural numbers then a = b if and only if
successor(a) = successor(b).
8. If a is a natural number then successor(a) is not equal to 1.
9. For every set K, if 1 is in K, and successor(x) is in K for
every natural number x in K, then every natural number is
in K. 11
Discussion …
• Are software testing axioms really axioms?
• If not, what would you call them?
12
Axiom 1
• It is impossible to test a program completely
• How many test cases do you need to exhaustively test:

• Powerpoint
• A calculator
• MS Word
• Any interesting software!
• The only way to be absolutely sure software works is to

run it against all possible inputs and observe all of its
outputs …
• Oh, and the specification must be correct and complete.

13
Axiom 1
• It is impossible to test a program
completely
• There are the reasons below:

• The number of possible inputs is very large.
• The number of possible outputs is very large.
• The number of paths through the software is

very large.
• The software specification is subjective.
14
Axiom 2
• Software testing is a risk-based exercise
• If you do not test the software for all inputs (a

wise choice) you take a risk.
• Hopefully you will skip a lot of inputs that work

correctly.
• What if you skip inputs that cause a fault?

• Risk: financial loss, security, loss of money, loss of life!
• That is a lot of pressure for a tester!
15
Axiom 2
• If you try to test too
much, the development
cost becomes
prohibitive.
• If you test too little, the

probability of software
failure increases and as
we discussed …
software failures can
cost us big time!
16
Axiom 3
• Testing cannot show the absence of bugs
• “Program testing can be used to show the

presence of bugs, but never to show their
absence!”
- Edsger Wybe Dijkstra
• Are there bugs in our software?
• Are there bugs in other software?
• Can you guarantee that there are no bugs to find?

17
Axiom 4
• The More Bugs You
Find, the More Bugs
There Are
• Bugs appear in groups,
where you see one you will
likely find more … Why?
• Programmers can have
bad days
• Programmers tend to
make the same mistakes
• Some bugs are just the tip
of the iceberg.
18
Axiom 4
• Boris Beizer coined the term pesticide paradox to describe the
phenomenon that the more you test software the more immune it
becomes to your test cases.
• Remedy: continually write new and different tests to exercise different
parts of the software.
19
Axiom 5
• Not all bugs found will be fixed.
• There are several reasons why you might
choose not to fix a bug:
• There’s not enough time
• Some deadlines cannot be extended (e.g., Y2K)
• It’s not really a bug
• Specifications can be wrong
• It’s too risky to fix
• “I’m not touching Murphy’s code!”
• It’s just not worth it
• Bugs in fringe features may have to wait
• Why not charge the customer for bug fixes in the next
release (sound familiar?) 
The decision-making process usually involves the 20
software testers, the project managers, and the
Axiom 6
• It is difficult to say when a bug is indeed a bug.
• if there is a problem in the software but no one ever

discovers it … is it a bug?
• Parody of “if a tree falls in the forest … does it really make
a noise?”
• What is your opinion? Does a bug have to be

observable in order for it to me a bug?
• Bugs that are undiscovered are called latent bugs.
21
Axiom 7
• Specifications are never final
• Building a product based on a “moving

target” specification is fairly unique to
software development.
22
Axiom 8
• Software testers are not the most popular
members of a project
• Goal of a software tester:
• Find bugs
• Find bugs early
• Make sure bugs get fixed
• Tips to avoid becoming unpopular:

• Find bugs early
• Temper your enthusiasm … act in a professional manner
• Don’t report just the bad news
23
Axiom 9
• Software Testing Is a Disciplined Technical
Profession
• When software was simpler and more manageable,

software testers were often untrained and testing was not
done methodically
• It is now too costly to build buggy software. As a result

testing has matured as a discipline.
• Sophisticated techniques
• Tool support
• Rewarding careers
24
Some Terminology
• Verification
• “Are we building the product right?”
• Does the software meet the specification?
• Validation
• “Are we building the right product?”
• Does the software meet the user requirements?
• Software Quality Assurance

• Not the same as software testing …
• Create and enforce standards and methods to improve the

development process and to prevent bugs from occurring. 25

CA 7503 - Lecture 2

Uploaded by

Copyright:

Available Formats

CA 7503 - Lecture 2

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

CA 7503 - Lecture 2

Uploaded by

Copyright:

Available Formats

• It can also be stated as the process of

• Comparing resulting/expected states

• Measuring execution characteristics (e.g.,

• Error: where the system state is incorrect

• Failure: some failure to deliver the5

• Test Suite: collection of test cases, typically related by a testing

• Test Driver: class or utility program that applies test cases

• Test harness: system of test drivers and other tools that

• Test Strategy: algorithm or heuristic to create test cases from a

• Oracle: means to check the output from a program is correct for

• Stub: partial temporary implementation of a component (usually6

• No software development effort follows a

• Nevertheless, an ideal model is helpful to make

• Therefore, it is taken for granted as true,

• If not, what would you call them?

• How many test cases do you need to exhaustively test:

• Any interesting software!

• The only way to be absolutely sure software works is to

• Oh, and the specification must be correct and complete.

• There are the reasons below:

• The number of possible outputs is very large.

• The number of paths through the software is

• If you do not test the software for all inputs (a

• Hopefully you will skip a lot of inputs that work

• What if you skip inputs that cause a fault?

• That is a lot of pressure for a tester!

• If you test too little, the

• “Program testing can be used to show the

• Are there bugs in our software?

• Are there bugs in other software?

• Can you guarantee that there are no bugs to find?

• if there is a problem in the software but no one ever

• What is your opinion? Does a bug have to be

• Bugs that are undiscovered are called latent bugs.

• Building a product based on a “moving

• Tips to avoid becoming unpopular:

• When software was simpler and more manageable,

• It is now too costly to build buggy software. As a result

• Does the software meet the specification?

• Does the software meet the user requirements?

• Software Quality Assurance

• Create and enforce standards and methods to improve the

You might also like