Software testing MCA

MODULE 2

A PERSPECTIVE ON SOFTWARE TESTING

BASIC DEFINITIONS

Error
 People make errors. A good synonym is “mistake”. When people make
mistakes while coding, we call these mistakes “bugs”.
 Errors tend to propagate; a requirements error may be magnified during
design, and amplified still more during coding
Fault
 A fault is the result of an error. It is more precise to say that a fault is the
representation of an error
 It is the representation of an error, where representation is the mode of
expression, such as narrative text, dataflow diagrams, hierarchy charts,
source code, and so on.
 “Defect” is a good synonym for fault; so is “bug”.
 Faults can be elusive.
 There are 2 types of faults: faults of commission and faults of omission.
 A fault of commission occurs when we enter something into a
representation that is incorrect.
 Faults of omission occur when we fail to enter correct information. Faults
of omission are more difficult to detect and resolve.
Failure

 A failure occurs when a fault executes.

 Two subtleties arise here: and is usually taken to be source code, or loaded
object code. The second subtlety is that this definition relates failures only
to faults of commission.

Incident
 An incident is the symptom(s) associated with a failure that alerts the user
to the occurrence of a failure.
Test

Department of MCA, SDIT Page 1

Software testing MCA

 A test is the act of exercising software with test cases.

 There are two distinct goals of a test: either to find failures, or
todemonstrate correct execution.
Test Case
 A test case has an identity, and is associated with a program behavior.
 A test case also has a set ofinputs, a list of expected outputs.

There are three opportunities for errors to be made, resulting in faults that
propagate through the remainder of the development.
One prominent tester summarizes this life cycle as follows: the first three
phases are “Putting Bugs IN”, the testing phase is Finding Bugs, and the last three
phases are “Getting Bugs OUT”.
The Fault Resolution step is another opportunity for errors (and new faults).
When a “fix” causes formerly correct software to misbehave, the fix is deficient.
It is clear from the diagram that the test cases occupy a central position in
testing. The process of testing can be subdivided into separate steps: test
planning, test case development, running test cases, and evaluating test results.
INSIGHTS FROM A VENN DIAGRAM
Testing is fundamentally concerned with behavior; and behavior is
orthogonal to the structural view common to software (and system) developers.
A quick differentiation is that the structural view focuses on “what it is” and
the behavioral view considers “what it does”.

Department of MCA, SDIT Page 2

Software testing MCA

One of the continuing sources of difficulty for testers is that the base
documents are usually written by and for developers, and therefore the emphasis
is on structural, rather than behavioral, information.
A simple Venn diagram can be developed which clarifies several nagging
questions about testing.

Fig: Specified and Implemented Program Behaviors

Consider a Universe of program behaviors. Given a program and its
specification, consider the set S of specified behaviors, and the set P of
programmed behaviors.
Figure shows the relationship between our universe of discourse and the
specified and programmed behaviors. Of all the possible program behaviors, the
specified ones are in the circle labeled S; and all those behaviors actually
programmed (note the slight difference between P and U, the Universe) are in P.
The intersection of S and P (the football shaped region) is the “correct”
portion, behaviors that are both specified and implemented. A very good view of
testing is that it is the determination of the extent of program behavior that is
both specified and implemented.

Figure :Specified, Implemented, and Tested Behaviors

The new circle in Fig. is for Test Cases. Consider the relationships among
the sets S, P, and T. There may be specified behaviors that are not tested (regions
Department of MCA, SDIT Page 3
Software testing MCA

2 and 5), specified behaviors that are tested (regions 1 and 4), and test cases that
correspond to unspecified behaviors (regions 3 and 7). Similarly, there may be
programmed behaviors that are not tested (regions 2 and 6), programmed
behaviors that are tested (regions 1 and 3), and test cases that correspond to
unprogrammed behaviors (regions 4 and 7). Each of these regions is important. If
there are specified behaviors for which there are no test cases, the testing is
necessarily incomplete

IDENTIFYING TEST CASES

There are two fundamental approaches to identifying test cases; these are
known as functional and structural testing. Each of these approaches has several
distinct test case identification methods, commonly called testing methods.
1) Functional Testing
Functional testing is based on the view that any program can be considered
to be a function that maps values from its input domain to values in its output
range. This type is considered to be “black boxes”. This leads to the term Black
Box Testing, in which the content (implementation) of a black box is not known,
and the function of the black box is understoodcompletely in terms of its inputs
and outputs.

INPUT OUTPUT

Figure:An Engineer’s Black Box

In this approach, the only information that is used is the specification of the
software. There are two distinct advantages to functional test cases:
 They are independent of how the software is implemented, so if the
implementation changes, the test cases are still useful.
 Test case development can occur in parallel with the implementation,
therebyreducing overall project development interval.
On the negative side, functional test cases frequently suffer from two problems:
Department of MCA, SDIT Page 4
Software testing MCA

 There can be significant redundancies among test cases

 There can be possibility of gaps of untested software.

Specification Program Specification Program

Test Cases( Method A) Test Cases( Method B)

Comparing functional test case identification method

Above Figure shows the results of testcases identified by two functional

methods.

- Method A identifies a larger set of test cases than doesMethod B.

- Notice that, for both methods, the set of test cases is completely
contained within the setof specified behavior. Since functional methods are based
on the specified behavior, it is hard toimagine these methods identifying
behaviors that are not specified

2) Structural Testing
Structural testing is the other fundamental approach to test case
identification.
It is sometimes called White Box (or Clear Box) Testing.
The essential difference is that the implementation(of the Black Box) is
known and used to identify test cases. Being able to “see inside” the black
boxallows the tester to identify test cases based on how the function is actually
implemented.
With linear graph theory concepts, the testercan rigorously describe exactly
what is being tested. Because of its strong theoretical basis,structural testing
lends itself to the definition and use of test coverage metrics.

Department of MCA, SDIT Page 5

Software testing MCA

Test coveragemetrics provide a way to explicitly state the extent to which a

software item has been tested, andthis in turn, makes testing management more
meaningful.

Specification Program
Specification Program

Test Cases( Method A)

Test Cases( Method B)

Above Figure shows the results of test cases identified by two structural methods.
- Method A identifies a larger set of test cases than does Method B.
ERROR AND FAULT TAXONOMIES
Faults can be classified in several ways: the development phase where
thecorresponding error occurred, the consequences of corresponding failures,
difficulty to resolve, riskof no resolution, and so on.

Department of MCA, SDIT Page 6

Software testing MCA

Department of MCA, SDIT Page 7

Software testing MCA

LEVELS OF TESTING
Levels oftesting echo the levels of abstraction found in the Waterfall Model
of the software
development life cycle. It is useful for testing as a means of identifying distinct
levels of testing,and for clarifying the objectives that pertain to each level.

Department of MCA, SDIT Page 8

Software testing MCA

There is a practical relationship between levels of testing and functional

and structural testing.
The Structural testing is most appropriate at the unit level, while functional
testing is most appropriate at the system level.
The constructs defined for structural testing make the most sense at the
unit level; and similar constructs are only now becoming available for the
integration and system levels of testing.
TESTING AND VERIFICATION
Program verification aims at proving the correctness of progress by showing that
is contains no errors.
 This is very different from testing that aims at uncovering errors in a
program.
 While verification aims at showing that a given program works for all
possible inputs that satisfy a set of conditions, testing aims to show that
the given program is reliable to that, no errors of any significance were
found.
 Program verification and testing are best considered as complimentary
techniques.
 In the developments of critical applications, such as smart cards or control
of nuclear plants, one often makes use of verification techniques to prove
the correctness of some artifact created during the development cycle, not
necessarily the complete program.

Department of MCA, SDIT Page 9

Software testing MCA

 Regardless of such proofs, testing is used invariably to obtain confidence in

the correctness of the application.
 Testing is not a perfect process in that a program might contain errors
despite the success of a set of tests; verification might appear to be a
perfect process as it promises to verify that a program is free from errors.
 Verification reveals that it has its own weakness.
 The person who verified a program might have made mistakes in the
verification process’ there might be an incorrect assumption on the input
conditions; incorrect assumptions might be made regarding the
components that interface with the program.
 Thus, neither verification nor testing is a perfect technique for proving the
correctness of program.
STATIC TESTING

 Static testing is carried out without executing the application under test.
 This is in contrast to dynamic testing that requires one or more executions
of the application under test.
 It is useful in that it may lead to the discovery of faults in the application,
ambiguities and errors in the requirements and other application-related
document, at a relatively low cost,
 This is especially so when dynamic testing expensive.
 Static testing is complementary to dynamic testing.
 This is carried out by an individual who did not write the code or by a team
of individuals.
 The test team responsible for static testing has access to requirements
document, application, and all associated documents such as design
document and user manual.
 Team also has access to one or more static testing tools.
 A static testing tool takes the application code as input and generates a
variety of data useful in the test process.

Department of MCA, SDIT Page 10

Software testing MCA

Walkthroughs
 Walkthroughs and inspections are an integral part of static testing.
 Walkthrough are an integral part of static testing.
 Walkthrough is an informal process to review any application-related
document.

eg: requirements are reviewed---->requirements walkthrough

code is reviewed---->code walkthrough
(or)
peer code review
Walkthrough begins with a review plan agreed upon by all members of the team.
Advantages:
 improves understanding of the application.
 both functional and non functional requirements are reviewed.
A detailed report is generated that lists items of concern regarding the
requirements.
Inspections
 Inspection is a more formally defined process than a walkthrough.
This term is usually associated with code.
 Several organizations consider formal code inspections as a tool to
improve code quality at a lower cost than incurred when dynamic
testing is used.

Inspection plan:
i. statement of purpose
ii. work product to be inspected this includes code and associated documents
needed for inspection.
iii. team formation, roles, and tasks to be performed.

Department of MCA, SDIT Page 11

Software testing MCA

iv. rate at which the inspection task is to be completed

v. Data collection forms where the team will record its findings such as defects
discovered, coding standard violations and time spent in each task.
Members of inspection team
a) Moderator: in charge of the process and leads the review.
b) Leader: actual code is read by the reader, perhaps with help of a code browser
and with monitors for all in the team to view the code.
c) Recorder: records any errors discovered or issues to be looked into.
d) Author: actual developer of the code.
It is important that the inspection process be friendly and non confrontational.
Use of static code analysis tools in static testing
 Static code analysis tools can provide control flow and data flow
information.
 Control flow information presented in terms of a CFG, is helpful to the
inspection team in that it allows the determination of the flow of control
under different conditions.
 A CFG can be annotated with data flow information to make a data flow
graph.
 This information is valuable to the inspection team in understanding the
code as well as pointing out possible defect.
 Commercially available static code analysis tools are:
 Purify -> IBM Rationale
 Klockwork ->Klockwork
 LAPSE (Light weight analysis for program security in eclipse) ->
open source tool

Department of MCA, SDIT Page 12

Software testing MCA

(a) CFG clearly shows that the definition of x at block 1 is used at block-3
but not at block 5.In fact the definition of x at block 1 is considered killed
due to its redefinition at block 4.
(b) CFG indicates the use of variable y in the block 3.If y is not defined
along the path from start to block 3,then there is a data-flow error as a
variable is used before it is defined. Several such errors can be detected by
static analysis tools. ->compute complexity metrics, used as a parameter in
deciding which modules to inspect first.

GENERALIZED PSEUDOCODE
 Pseudocodeproides a “language neutral” way to express program
source code.
 The given version is based on Visual basic and has constructs at 2
level
 Unit component: Units can be interpreted either as
traditional components( procedures and functions) or as
object oriented components( classes and objects)
Items in angle brackets indicate language elements that can be used at the
identified position

Department of MCA, SDIT Page 13

Software testing MCA

EXAMPLES

1) TRIANGLE PROBLEM

Department of MCA, SDIT Page 14

Software testing MCA

Problem Statement
The Triangle Program accepts three integers as input; these are taken to be sides
of a triangle. The
output of the program is the type of triangle determined by the three sides:
Equilateral, Isosceles,
Scalene, or NotATriangle.
Improved version:
Triangle accepts 3 integers a, b, and c as input. It must satisfy the following
conditions:
c1.1 ≤a ≤ 200 c4. a < b + c
c2.1 ≤ b ≤ 200 c5. b < a + c
c3.1 ≤ c ≤ 200 c6. c < a +b
If an input value fails any of conditions c1, c2, or c3, the program notes this with
anoutput message, for example, “Value of b is not in the range of permitted
values.” If values of a, b, and c satisfy conditions c1, c2, and c3, one of four
mutually exclusive outputs is given:
1. If all three sides are equal, the program output is Equilateral.
2. If exactly one pair of sides is equal, the program output is Isosceles.
3. If no pair of sides is equal, the program output is Scalene.
4. If any of conditions c4, c5, and c6 is not met, the program output is
NotATriangle.

i. Traditional Implementation

Department of MCA, SDIT Page 15

Software testing MCA

The flowchart for this implementation appears in Figure 2.1. The flowchart
box numbes correspond to comment numbers in the (FORTRAN-like) TurboPascal
program given next.

1st Implementation (Traditional Implementation)

Department of MCA, SDIT Page 16

Software testing MCA

Department of MCA, SDIT Page 17

Software testing MCA

ii. Structured Implementation

2nd Implementation (Structured Implemenation)

Department of MCA, SDIT Page 18

Software testing MCA

iii. 3rdImplemenation (for Imroved version problem statement)

Department of MCA, SDIT Page 19
Software testing MCA

2) THE NEXTDATE FUNCTION

Next date is a function of 3 variables: month ,day and year. It returns the
date of the day after the input date. The month, day and year variables
have integervaues subject to these conditions:
c1: 1 ≤ month ≤ 12

Department of MCA, SDIT Page 20

Software testing MCA

c2:1 ≤ day ≤ 31
c3: 1 ≤ year ≤ 2012

If any of conditions c1, c2, or c3 fails, NextDate produces an

outputindicating the corresponding variable has an out-of-range value —
forexample, “Value of month not in the range 1..12”. Because numerous invalid
day–month–year combinations exist, NextDate collapses these into one message:
“Invalid Input Date.”
Discussion
There are two sources of complexity in the NextDate function:
i) Complexity of the input domain
ii) the rule that distinguishes common years from leap years.

Since a year is365.2422 days long, leap years are used for the “extra day”
problem.

If we declared a leap year every fourth year, there would be a slight error. The
Gregorian Calendar resolves this by adjusting leap years on century years.

Thus a year is a leap year ifit is divisible by 4, unless it is a century year.

Century years are leap years only if they are multiplesof 400, so 1992,
1996, and 2000 are leap years, while the year 1900 is acommon year.

i. Implementation (Simple Version)

Department of MCA, SDIT Page 21

Software testing MCA

Department of MCA, SDIT Page 22

Software testing MCA

ii. Implementation (Improved Version)

Department of MCA, SDIT Page 23

Software testing MCA

Department of MCA, SDIT Page 24

Software testing MCA

3) THE COMMISSION PROBLEM

Problem Statement
A rifle salesperson in the former Arizona Territory sold rifle locks,stocks, and
barrels made by a gunsmith in Missouri. Locks cost $45,stocks cost $30, and
barrels cost $25. The salesperson had to sell atleast one complete rifle per
month, and production limits were suchthat the most the salesperson could
sell in a month was 70 locks, 80stocks, and 90 barrels. After each town visit,
the salesperson sent atelegram to the Missouri gunsmith with the number of
locks, stocks, and barrels sold in that town. At the end of a month, the
salespersonsent a very short telegram showing –1 locks sold. The gunsmith
thenknew the sales for the month were complete and computed
thesalesperson’s commission as follows: 10% on sales up to (andincluding)
$1000, 15% on the next $800, and 20% on any sales inexcess of $1800. The
commission program produced a monthly salesreport that gave the total

Department of MCA, SDIT Page 25

Software testing MCA

number of locks, stocks, and barrels sold, thesalesperson’s total dollar sales,
and, finally, the commission

Implementation

Department of MCA, SDIT Page 26

Software testing MCA

Department of MCA, SDIT Page 27

Software testing MCA

4) THE SATM SYSTEM

Problem Statement

The SATM system communicates with bank customers via the fifteen
screens shown in Figure 2.4. Using a terminal with features as shown in figure
below, SATM customers can select any of threetransaction types: deposits,
withdrawals, and balance inquiries, and these can be done on two types of
accounts, checking and savings.
When a bank customer arrives at an SATM station, screen 1 is displayed.
The bank customer accesses the SATM system with a plastic card encoded with a
Personal Account Number (PAN), which is a key to an internal customer account
file, containing, among other things, the customer’s name and account
information. If the customer’s PAN matches the information in the customer
account file, the system presents screen 2
to the customer. If the customer’s PAN is
not found, screen 4 is displayed, and the
card is kept. At screen 2, the customer is
prompted to enter his/her Personal
Identification Number (PIN). If the PIN is
correct (i.e., matches the information in
the customer account file), the system
displays screen 5; otherwise, screen 3 is displayed. The customer has three
chances to get the PIN correct; after three failures, screen 4 is displayed, and the
card is kept.

On entry to screen 5, the system adds two pieces of information to the

customer’s account file: thecurrent date, and an increment to the number of ATM
sessions. The customer selects the desired transaction from the options shown on
screen 5; then the system immediately displays screen 6, where the customer
chooses the account to which the selected transaction will be applied.

Department of MCA, SDIT Page 28

Software testing MCA

If balance is requested, the system checks the local ATM file for any
unposted transactions, and reconciles these with the beginning balance for that
day from the customer account file. Screen 14 is then displayed.
If deposit is requested, the status of the Deposit Envelope slot is
determined from a field in the Terminal Control File. If no problem is known, the
system displays screen 7 to get thetransaction amount. If there is a problem with
the deposit envelope slot, the system displays screen 12. Once the deposit
amount has been entered, the system displays screen 13, accepts the deposit
envelope, and processes the deposit. The deposit amount is entered as an
unposted amount in the local ATM file, and the count of deposits per month is
incremented. Both of these (and otherinformation) are processed by the Master
ATM (centralized) system once per day. The system then displays screen 14.
If withdrawal is requested, the system checks the status (jammed or free)
of the withdrawal chute in the Terminal Control File. If jammed, screen 10 is
displayed, otherwise, screen7 is displayed so the customer can enter the
withdrawal amount. Once the withdrawal amount isentered, the system checks
the Terminal Status File to see if it has enough money to dispense. If it does not,
screen 9 is displayed; otherwise the withdrawal is processed. The system checks
the customer balance (as described in the Balance request transaction), and if
there are insufficient funds, screen 8 is displayed. If the account balance is
sufficient, screen 11 is displayed, and the money is dispensed. The withdrawal
amount is written to the unposted local ATM file, and the count of withdrawals
per month in incremented. The balance is printed on the transaction receipt as it
is for a balance request transaction. After the cash has been removed, the system
displays screen 14.
When the No button is pressed in screens 10, 12, or 14, the system
presents screen 15 and returns the customer’s ATM card. Once the card is
removed from the card slot, screen 1 is displayed. When the Yes button is pressed
in screens 10, 12, or 14, the system presents screen 5 so the customer can select
additional transactions.

Department of MCA, SDIT Page 29

Software testing MCA

5) THE CURRENCY CONVERTER

The currency conversion program is an event driven program that
defines code associated with a Graphical User Interface(GUI). A Sample GUI
is shown below.

Department of MCA, SDIT Page 30

Software testing MCA

The application converts U.S dollars to any of 4 currencies. Brazilian

Real’s, Canadian Dollars, European Euros, and Japanese Yen. Currency
selection is governed by the radio button, which are mutually exclusive.
When a country is selected, The system responds by completing the label
“Equivalent in….” becomes Equivalent in Canadian Dollars” if the Canada
button is clicked. Also a small Canadian flag appears next to the output
position for the equivalent currency amount. The user can click on
Compute, Clear or the Quit button. Compute button is used to find the
result, Clear button resets the currency selection and quit button ends
application.
6) SATURN WINDSHIELD WIPER CONTROLLER
The windshield wiper on the Saturn automobile (at least on the 1992
models) is controlled by alever with a dial. The lever has four positions, OFF,
INT (for intermittent), LOW, and HIGH, andthe dial has three positions,
numbered simply 1, 2, and 3. The dial positions indicate threeintermittent
speeds, and the dial position is relevant only when the lever is at the INT
position. Thedecision table below shows the windshield wiper speeds (in wipes
per minute) for the lever and dialpositions.

Department of MCA, SDIT Page 31

Software testing MCA

Department of MCA, SDIT Page 32