SOFTWARE ENGINEERING TECHNIQUES

LESSON 23:

Topics Covered
Importance of testing, Hurdles of testing and the testing Design
fundamentals, Test case design
27%
Objectives
Upon completion of this Lesson, you should be able to: Rqmts.
Design
• Know what is basically the importance of testing Code
Code Other
• Know what are the various hurdles in testing Rqmts. 7%
• Know what are the testing fundamentals 56%
Now let us see the importance of testing. Other
10%
Importance of Testing
Testing activity cannot be eliminated in the life cycle as the end
product must be bug free and reliable one. Testing is important Figure: Software defect Distribution
because: Testing Fundamentals
• Testing is a critical element of software Quality Assurance Before understanding the process of testing software, it is
necessary to learn the basic principles of testing.
• Post-release removal of defects is the most expensive
• Significant portion of life cycle effort expended on testing Testing Objectives
In a typical service oriented project, about 20-40% of project • Testing is a process of executing a program with the intent
effort spent on testing. It is much more in the case of “human- of finding an error.
rated” software. • A good test is one that has a high probability of finding an
For example, at Microsoft, tester to developer ratio is 1:1 as yet undiscovered error.
whereas at NASA shuttle development center (SEI Level 5), the • A successful test is one that uncovers an as yet
ratio is 7:1. This shows that how testing is an integral part of undiscovered error.
Quality assurance. The objective is to design tests that systematically uncover
OK, now we have understood that testing is important. But as different classes of errors and do so with a minimum amount
it is known that important tasks are not easy to be carried on, of time and effort.
same with testing. There are many hurdles in testing. We’ll see Secondary Benefits Include
what it is one by one.
• Demonstrate that software functions appear to be working
Hurdles in Testing according to specification.
As in many other development projects, testing is not free
• Those performance requirements appear to have been met.
from hurdles. Some of the hurdles normally encounters are:
• Data collected during testing provides a good indication of
• Usually late activity in the project life cycle
software reliability and some indication of software quality.
• No “concrete” output and therefore difficult to measure
Testing cannot show the absence of defects, it can only show
the value addition
that software defects are present.
• Lack of historical data
Test Information Flow
• Recognition of importance is relatively less A typical test information flow is shown in the following figure.
• Politically damaging as you are challenging the developer
• Delivery commitments
• Too much optimistic that the software always works
correctly

Defect Distribution
In a typical project life cycle, testing is the late activity. When the
product is tested, the defects may be due to many reasons. It
may either programming error or may be defects in design or
defects at any stages in the life cycle. The overall defect distribu- Figure: Test information flow in a typical software test life
tion is shown in the following figure. cycle

© Copy Right: Rai University

3E.253 83
 In the above fig, most cost-effective defect reduction process is the classic
SOFTWARE ENGINEERING TECHNIQUES

• Software Configuration includes a Software Requirements structural test; the code inspection or walk-through. Code
Specification, a Design Specification, and source code. inspection is like proof reading and developers will be benefited
in identifying the typographical errors, logic errors and devia-
• A test configuration includes a Test Plan and Procedures,
tions in styles and standards normally followed.
test cases, and testing tools.
Dynamic testing is an execution based testing technique.
• It is difficult to predict the time to debug the code, hence it
Program must be executed to find the possible errors. Here, the
is difficult to schedule.
program, module or the entire system is executed (run) and the
Test Case Design output is verified against the expected result. Dynamic execution
Some of the points to be noted during the test case design are: of tests is based on specifications of the program, code and
• Can be as difficult as the initial design. methodology.
• Can test if a component conforms to specification - Black To test computer software, we spiral out along streamlines that
Box Testing. broaden the scope of testing with each turn.
• Can test if a component conforms to design - White box Considering the process from a procedural point of view
testing. testing within the context of software engineering is a series of
• Testing cannot prove correctness as not all execution paths four steps that are implemented sequentially.
can be tested. The steps are shown In Figure given below initially tests focus
Consider the following example shown in fig on each module individually, assuring that it functions as a unit
hence the name unit testing. Unit testing makes heavy use of
white-box testing techniques, exercising specific paths in a
module’s control structure to ensure complete coverage and
maximum error detection. Next, modules must be assembled
or integrated to form the complete software package. Integration
testing addresses the issues associated with the dual problems
of verification and program construction. Black-box test case
design techniques are most prevalent during integration,
although a limited amount of white -box testing may be used
to ensure coverage of major control paths. After the software
has been integrated (constructed), sets of high-order test are
conducted. Validation criteria (established during requirements
analysis) must be tested. Validation testing provides final
assurance that software needs all functional, behavioral and
performance requirements. Black-box testing techniques are
used exclusively during validation.
The last high-order testing step falls outside the boundary of
software engineering and into the broader context of computer
system engineering. Software once validated must be combined
with other system elements (e.g., hardware, people, and
databases). System testing verifies the tall elements mesh properly
and that overall system function/performance is achieved.
A program with a structure as illustrated above (with less than
100 lines of Pascal code) has about 100,000,000,000,000
possible paths. If attempted to test these at rate of 1000 tests
High order tests
per second, would take 3170 years to test all paths. This shows
Requirement
that exhaustive testing of software is not possible.
Test Case Design
Integration test
Introduction Design

Software can be tested either by running the programs and

verifying each step of its execution against expected results or by
statically examining the code or the document against its stated Code

requirement or objective. In general, software testing can be Unit test

divided into two categories, viz. Static and dynamic testing.

Static testing is a non-execution-based testing and carried
Test Direction
through by mostly human effort. In static testing, we test,
design, code or any document through inspection, Figure 5.3: Software Testing Steps
walkthroughs and reviews. Many studies show that the single

© Copy Right: Rai University

84 3E.253
Summary A single misplaced command - one that tells a machine to look

SOFTWARE ENGINEERING TECHNIQUES

The primary objective for test case design is to derive a set of in the wrong data base when a particular piece of information is
tests that have the highest likelihood for uncovering errors in needed, for instance - can throw a computer into confusion.
the software. The purpose of testing is to show that the Usually such bugs are detected during laboratory testing, but
software works. Here most of the time, the software is many survive this process.
demonstrated in a normal sequence/flow. All the branches may DSC’s explanation underlines fears that as computer software
not be tested. This approach is mainly to satisfy the customer grows more complex and takes over
and no value added to the program. Testing is an activity in
greater functions in companies and government, society will be
which a system or component is executed under specified
increasingly vulnerable to massive failures.
conditions; the results are observed and recorded and an
evaluation is made of some aspect of the system or compo- Software engineers are developing new ways to test software
nent. Many approaches have been defined in literatures. The and mathematically prove that it is error-free. Other theories say
importance of any approaches depends on the type of the software should be designed with the assumption that bugs
system in which you are testing. Defects can exist in the will occur but that any damage they cause will be containable.
software, as it is developed by human who can make mistakes Software developers continually come up with refinements in
during the development of software. existing programs. Perpiglia said DSC had made changes at the
request of one customer to basic software that runs the signal
Exercises
transfer points. Between December last year and April, the
1. Explain what are the various hurdles in testing? change was distributed to five of the seven regional telephone
2. Briefly explain what are the various testing fundamentals? companies.
Case Study “DSC did not go through the normal process” of four months
A tiny software “bug” - three faulty computer instructions of testing before sending the change out, Perpiglia told
lurking among several million that modern telephone switches members of the House subcommittee on telecommunications
use to route calls -appears to have sparked the sudden, massive and finance yesterday. “Because the change was small ... we felt
failures of local telephone systems in recent weeks, the manufac- that the change itself did not require such testing.”
turer of the equipment said yesterday. The flaws were hidden in Analysis of the computer instructions led DSC to pinpoint the
minor software changes that the Plano, Tex.-based company, problem in a specific section of the change, Perpiglia said. The
DSC Communications Corp., provided to its telephone flaws, he said, consisted of only three “binary digits” of
company customers in an effort to improve the equipment’s information - three flawed digits among millions of correct
performance. The software was sent out without major testing ones.
because DSC judged that the changes were too small to require
DSC computers are designed to route calls around minor
it, DSC Vice President Frank Perpiglia said yesterday.
failures that routinely occur in a modern telephone network,
It was the first detailed explanation of what caused the such as the failure of a single circuit board. But due to the errors
unprecedented string of failures. In rapid sequence in June and in the program, the computers responded by pumping out
earlier this month, local phone service in the Washington floods of erroneous internal messages that crowded out
region, Pittsburgh, San Francisco and Los Angeles mysteriously routine messages that serve to route phone calls, causing the
crashed, for as long as eight hours. systems to shut down.
Perpiglia said the software flaw appeared to be the “root cause” Phone company executives said they had no way to explain why
of failures of DSC-built computers called signal transfer points the failures on both coasts had occurred so close in time to one
that have been at the heart of the investigation. But he said that another. In a closed session yesterday, the Federal Communica-
in the telephone network as a whole, in which equipment made tions Commission decided to step up its own research into
by many different companies is linked together, other causes network reliability, to establish a special staff for that job and to
might be found as well. establish formal reporting requirements for network outages.
Despite DSC’s statement, major telephone companies and “The commission continues to have full faith in the fundamen-
equipment manufacturers yesterday continued a high-energy tal strength of our public telephone network,” FCC Chairman
investigation of the incidents, which rank among the most Alfred Sikes said. “And we are persuaded that the recent events
disruptive software-related failures in U.S. history. In the are - in all probability - inadvertent side effects of continuing
meantime, telephone companies have installed in their switches progress, rather than evidence of any fundamental flaws.”
special software “patches” that DSC provided, and they said
that the fixes should prevent any recurrence of the failures. Questions
Select two of the questions below, write the answers in the
Software consists of electronic instructions that in complex space provided, and designate a member of the group to read
programs can number in the millions. Computers move the answer(s) on behalf of the group.
through the instructions sequentially to perform tasks as diverse
as routing telephone calls, running a word processor, guiding 1. Describe the incident in your own words. What went
missiles and keeping track of company payrolls. wrong and what was the impact on the human lives?
2. State the root cause of the problem. Why did this problem
shut down the phone system?

© Copy Right: Rai University

3E.253 85
 Further Readings and Information
SOFTWARE ENGINEERING TECHNIQUES

Sources
Software engineering presents both technical and mechanical
challenges. Books by Black(Managing the testing process,
Microsoft Press, 1999); Dustin, Rashka, and Paul (Test Process
Improvement: Step-by Step Guide to Structured Testing,
Addison-Wesley, 1999); Perry (Surviving the Top Ten Chal-
lenges of Software Testing: A People oriented Approach,
Dorset House, 1997).
A wide variety of information sources on software testing and
related subjects is available on the internet. An up-to date list of
World wide web references that are relevant to testing concepts,
methods and testing strategies can be found at the SEPA Web
site:http://www.mhhe.com/engcs/compsci/sepa/resources/
test-techniques.mhtml
Notes

© Copy Right: Rai University

86 3E.253