Software Testing
Software Testing
Overview
Definition of Software Testing
Problems with Testing
Benefits of Testing
Effective Methods for Testing
Definition of Software Testing
Software testing is the process of
executing a software system to determine
whether it matches its specification and
executes in its intended environment.
Program testing can be a very effective
way to show the presence of bugs, but it
is hopelessly inadequate for showing
their absence [Dijkstra, 1972]
Why Test?
Q: If all software is released to
customers with faults, why should we
spend so much time, effort, and money
on testing?
Cost of Delaying the Release
of a Software Product
Timing is another important factor to
consider.
New products: The first to the market
often sells better than superior products
that are released later.
Beta Testing
Customers test for free!
Seems to give you test cases
representative of customer use.
Helps to determine what is most
important to the customers.
Can do more configuration
(environment) testing than in your
testing lab.
Problems with Beta Testing
Most beta testers are techies who
have a higher tolerance of bugs. They
do not represent the average customer.
Beta testers usually wont report:
usability problems, bugs they dont
understand and bugs that seem
obvious.
Takes much more time and effort to
handle a user reported bug.
Cutting Testing Costs can
Increase other Costs
Customer support can be very
expensive. Less bugs = less calls.
Customers will look for more reliable
solutions.
Software organizations must perform
cost benefit analysis to determine how
much to spend on testing.
Problems with Testing
Since it is impossible to find every fault
in a software system, bugs will be found
by customers after the product is
released.
Another Problem
In many software companies, testers are ill-
equipped to test software. For example: My
last co-op firm (which will remain unnamed).
Requirements Specification
Design Phase
Phase Phase
Implementation
Phase
Black- White-box
Specification Program
box tests tests
Test Automation
If a manual test costs $X to run the first
time, it will cost $X to run every time
thereafter.
An automated test can cost 3 to 30
times $X the first time, but will cost
about $0 after that.
Any Questions?
Too Many Possible Paths
There are 5 paths from A to X without
passing through the loop.
There are 520 paths from A to X after
passing through the loop 20 times.
There are 5 + 52 + 53 ++ 520 = 100
trillion possible paths in this program.
If you could test a path per second it
would take more than 3 million years!
Black Box Testing
Checks that the product conforms to
specifications
Cannot determine how much code has
been tested
Omissions detected
by black-box tests
Program
White Box Testing
Allows tester to be sure every
statement has been tested.
Difficult to discover missing
functionality.
Commissions detected
Program by White-box tests