SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING

BASICS OF SOFTWARE TESTING

Why do we need process of testing?

 To make judgement about quality or acceptability
 To discover problems

BASIC DEFINITIONS

Error: Synonym of error is mistake. Mistakes made while coding are called as bugs. Errors
usually tend to propagate.
Ex: Requirement error may be magnified during design and amplified more during coding.

Fault: Fault is the result of an error. It is also considered as a representation of an error.

Synonym of fault is defect. Faults can be elusive (difficult to find).
There are two types of faults
1. Faults of commission: Occurs when we enter something into a representation that is
incorrect.
2. Faults of omission: Occurs when we fail to enter correct information.

Failure: Failure occurs when a fault executes. Failures only occur in an executable
representation which is usually taken to be the source code.

Incident: An incident is the symptom associated with the failure that alerts the user to the
occurrence of a failure.

Test: Testing is concerned with errors, failures, faults and incidents. A test has two distinct
goals - to find failures or to demonstrate correct execution.

Test case: A test case has an identity and is associated with a program behaviour. It also has a
set of inputs and expected outputs.

Testing life cycle:

In development phases, three opportunities arise for errors to be made, resulting in faults that
propagate through the reminder of the development process. Testing life cycle can be
summarised as: the first three phases are putting bugs in; the testing phase is finding the bugs;
and the last three phases are getting the bugs out.

 The fault resolution step is another opportunity for errors.

 Test cases occupy a central position in testing.

The process of testing can be

subdivided into separate steps:
 Test planning
 Test case development
 Running test cases
 Evaluating test results

Dept of ISE,SCE Page 1

 SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING
SOFTWARE QUALITY:
The below attributes define quality of a software.

Quality attributes: There exist several measures of software quality. These can be divided into
static and dynamic quality attributes.

Static quality attributes: Refers to actual code and related documentation. It includes structured,
maintainable, testable code as well as the availability of correct and complete documentation.

Dynamic quality attributes: Relate to the behaviour of the application while in use. It includes
software reliability, correctness, completeness, consistency, usability and performance.

 Reliability: Refers to the probability of failure free operation.

 Correctness: Refers to the correct operation of an application and is always with reference
to some artefact.
 Completeness: Refers to the availability of all the features listed in the requirements or in
the user manual. An incomplete software is one that does not fully implement all
features required.
 Consistency: Refers to adherence to a common set of conventions and assumptions.
 Ex: all buttons in the user interface might follow a common colour coding convention.
 Usability: Refers to the ease with which an application can be used.
 Performance: Refers to the time the application takes to perform the requested task.

Reliability:

Definition 1: ANSI/IEEE STD 729-1983:

Software reliability is the probability of failure free operation of software over a given time
interval and under given conditions.

The probability referred to in this definition depends on the distribution of the inputs to the
program. Such input distribution is often referred to as the operational profile. According to this
definition, software reliability could vary from one operational profile to another.

Implication is that one user might say “this program is lousy” while another might sing praises
for the same program.

Definition 2:
Software reliability is the probability of failure free operation of a software in its intended environment.

This definition is independent of “who uses what features of the software and how often”. Instead
it depends exclusively on the correctness of its features.

As there is no notion of operational profile, the entire input domain is considered as uniformly
distributed.

The term environment refers to the software and hardware elements needed to execute the
application. These elements include the OS, hardware requirements and any other applications
needed for communication.

Dept of ISE,SCE Page 2

 SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING
Requirements, behaviour and correctness
Products, software in particular, design in response to requirements. Requirements specify the
functions that a product is expected to perform. During the development of the product, the
requirements might have changed from what was stated originally. Once the product is ready,
it is the requirements that determine the expected behaviour.

Ex:
Two requirements are given below, each of which leads to a different program.
Requirement 1: It is required to write a program that inputs two integers and outputs maximum
of these.

Requirement 2: It is required to write a program that inputs a sequence of integers and outputs the
sorted version of this sequence.

Requirement 1 is incomplete:
Suppose that program Max is developed to satisfy a requirement 1 above. The expected output
of Max when the input integers are 13 and 19 can be easily determined to be 19.

Suppose now the tester wants to know if the two integers are to be input to the program in one
line followed by a carriage return or on two separate lines with a carriage return typed in after
each number. The requirement as stated above fails to provide an answer to this question.

Requirement 2 is ambiguous:
It is not clear from the requirement whether the input sequence is to be sorted in ascending or
descending order. The behaviour of sort program, written to satisfy this requirement, will depend
on the decision taken by the programmer while writing sort.

Regardless of the nature of the requirements, testing requires the determination of the expected
behaviour of the program under test. The observed behaviour of the program is compared with
the expected behaviour to determine if the program functions as desired.

Input domain and program correctness:

A program is considered correct if it behaves as desired on all possible test inputs. Usually, the
set of all possible inputs is too large for the program to be executed on each input.

For integer value, -32,768 to 32767 are the possible values. This requires 2 32 executions.
Testing a program on all possible inputs is known as “exhaustive searching”.

Ex:
 Using requirement 1 above, we find the input domain of Max to be the set of all pairs of
integers where each element in the pair is in the range -32,768 till 32767.
 Using requirement 2 it is not possible to find the input domain for the sort program.
Therefore, let us modify the requirement 2.

Modified requirement 2:

It is required to write a program that inputs a sequence of integers and outputs the integers in
the sorted manner in either ascending or descending order. The order of the output sequence is
determined by an input request character which should be “A” when an ascending sequence is
desired and “D” for descending.
While providing input to the program, the request character is entered first followed by the
sequence of integers to be sorted; the sequence is terminated with a period.

Dept of ISE,SCE Page 3

 SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING
Based on the above modified requirement, the first element is a character. The second element
is a sequence of zero or more integers ending with a period.
Ex: < A -3 15 12 55. >
< D 23 18. >

Valid and invalid inputs:

The modified requirement for Sort mentions that the request characters can be “A” or “D”, but
fails to answer the question “what if the user types a different character?”

When using sort it is certainly possible for the user to type a character other than “A” or “D”.
Any character other than these is considered as invalid input to Sort. The requirement for sort
does not specify what actions it should take when an invalid input is encountered. Testing a
program against invalid inputs might reveal errors in the program.

Ex: Suppose we are testing the short program. We execute it against the following input : <E 7
19. > . Suppose that upon execution of the above input, the sort program interest to an infinite
loop and neither asks the user for any input nor response to anything typed by the user. This
observed behaviour points to a possible error in the sort.

Instead of typing an integer the user types in a character such as “?”, program should inform
the user that the input is invalid.

Input domain is partitioned into two subdomains: valid and invalid inputs.
Ex:
< D 7 9F 19. > Invalid
< A 7 19. > Valid

Correctness versus reliability:

Correctness

A program is considered correct if it behaves as expected on each element of its input domain.

Though correctness of a program is desirable, it is almost never the objective of testing. To

establish correctness via testing would imply testing a program on all elements in the input
domain. In most cases that are encountered in practice, this is impossible to accomplish. Thus,
correctness is established via mathematical proofs of programs.

While correctness attempts to establish that the program is error free, testing attempts to find
if there are any errors in it. Thus, completeness of testing does not necessarily demonstrate that
a program is error free. Testing, debugging and the error removal processes together increase
our confidence in the correct functioning of the program under test.

Ex: This example illustrates why the probability of program failure might not change upon error
removal. Consider the following program that inputs two integers X and y and print the value
of f(x,y) or g(x,y) depending on the condition x<y .

Dept of ISE,SCE Page 4

 SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING
Integer x, y
Input x, y
If(x < y)  this condition should be x<=y
{
Print f(x, y)
}
Else
{
Print g(x, y)
}

The above program uses two functions f and g, not defined here. Let us assume that function f
produces incorrect result whenever it is invoked with x = y.

In its present form, the program fails when tested with equal input values because function g(x,
y) is invoked instead of function f. When the error is removed by changing the condition x<y to
x<=y, the program fails again when the input values are same. This is due to the error in function
f. In this program when the error in f is also removed, the program will be correct assuming that
all other code is correct.

Reliability:
Reliability of a program p is the probability of successful execution on a randomly selected
element from its input domain.

While correctness is a binary metric, reliability is a continuous metric over a scale from 0 to
1. A program can either be correct or incorrect. Its reliability can be anywhere between 0 and 1.

Ex: How to compute program reliability in easy way:

Consider a program p which takes a pair of integers as input. The input domain of this program
is the set of all pairs of integers. Suppose now that in actual use there are only three pairs that
will be input to p. These are as follows: {< (0, 0) (1,-1) (-1, 1) >}

If it is known that p failed on exactly one of the three possible input pairs then the frequency
with which p will function correctly is 2/3. This number is an estimate of the probability of the
successful operation of p and hence it is the reliability of p.

Program use and the operational profile:

Operational profile:
An operational profile is a numerical description of how a program is used. A program might
have several operational profiles depending on its users.

Ex: Consider a sort program which, on any given execution, allows any one of the two types of
input sequences. One sequence consists of numbers only and the other consists of
alphanumeric strings.
The different operational profiles for sort is specified as follows:

Operation profile 1 used mostly for sorting sequence of numbers.

Operation profile 2 is used for sorting alphanumeric strings.

Dept of ISE,SCE Page 5

 SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING
Testing and debugging:
Testing is the process of determining if a program behaves as expected. In the process, one may
discover errors in the program under test.
When testing reveals error, the process used to determine the cause of this error and to remove
it is known as debugging.
As illustrated in the figure, testing and debugging are often used as to related activities in a
cyclic manner.

Steps are:
1. Preparing a test plan
2. Constructing test data
3. Executing the program
4. Specifying program behaviour
5. Assessing the correctness of program behaviour
6. Construction of Oracle.

1. Preparing a test plan:

A test cycle is often guided by a test plan.
When relatively very small programs are
being tested, a test plan is usually informal
and it is in the tester's mind or there may be
no plan at all.

Ex: Test plan for sort.

The sort program is to be tested to meet the
requirements given earlier. Specifically, the
following needs to be done:

1. Execute sort on at least two input

sequences, one with “A” and the other with
“D” as request characters.
2. Execute a program on an empty input
sequence.
3. Test the program for robustness against
erroneous inputs such as “R” typed in as the
request character.
4. All failures of the test program should be
recorded in a suitable file using the company
failure report form.

2. Constructing Test Data:

A test case is a pair consisting of test data to
be input to the program and the expected
output.
The test data is a set of values, one for each
input variable. A test set is a collection of
zero or more cases.

Program requirements and the test plan help

in the construction of test data. Execution of the program on test data might begin after all or
a few test cases have been constructed. Based on the results obtained, the testers decide
whether to continue the construction of additional test cases or to enter the debugging phase.

Dept of ISE,SCE Page 6

 SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING
The following test cases are generated for the sort program using the test plan in the previous
figure.

3. Executing the program:

Execution of a program under test is the next significant step in the testing. Testers might be
able to construct a test harness to aid is program execution.(fig 1.4) The harness initializes any
global variables, inputs a test case, and executes the program. The output generated by the
program may be saved in a file for subsequent examination by a tester.

In preparing this test harness assume

that:
(a) Sort is coded as a procedure
(b) The get_input procedure reads the
request character & the sequence to be
sorted into variables request_char,
num_items and in_number, test_setup
procedure-invoked first to set up the test
includes identifying and opening the file
containing tests.
Check_output: procedure serves as the
oracle that checks if the program under
test behaves correctly.
Report_failure: output from sort is
incorrect. May be reported via a message (or) saved in a file.
Print_sequence: prints the sequence generated by the sort program. This also can be saved in
file for subsequent examination.

4. Specifying program behaviour:

Program behaviour can be specified in several ways: plain natural language, a state diagram,
formal mathematical specification, etc.
State transmission diagram or state diagram is used to specify the program behaviour.
The state of a program is the set of current values of all its current variables and an indication
of which statement in the program is to be executed next. One way to encode the state is by
collecting the program variables into a vector known as state vector.

Dept of ISE,SCE Page 7

 SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING

An indication of where the control of

execution is at any instant of time can be
given by using an identifier associated with
the next program statement.

Ex: Consider a menu driven application

named myapp.

5. Assessing the correctness of

program behaviour
An important step in testing a program is the
one wherein the tester determines if the
observed behaviour of the program under
test is correct or not.
This step is further divided into 2 smaller
steps:

1. Observation of the behaviour

2. Analyzation of the observed behaviour.
The entity that performs the task of checking the correctness of the observed behaviour is known
as an oracle.
A tester often assumes the role of an oracle and thus serves as a human oracle.

Oracle can also be programs designed to check the behaviour of other programs.

Checking program behaviour by humans has several disadvantages:

 It is error prone as the human oracle might make error in analysis.
 It may be slower than the speed with which the program computed the results.
 It might result in the checking of only I/O behaviour.

Using programs as oracles has advantage of speed, accuracy and ease with which complex
computations can be checked.

6. Construction of oracles:

Construction of automated oracles, such as the one to check a matrix multiplication program
or a sort program requires determination of I/O relationship.

Example: (fig. 1.7) Consider a program named

Hvideo that allows one to keep track of home videos.
In the data entry mode, it displays a screen in which
the user types in information about a DVD. In
search mode, the program displays a screen into
which a user can type some attribute of the video
being searched for and set up a search criterion.

To test Hvideo we need to create an oracle that

checks whether the program functions correctly in
data entry and search modes. The input generator
generates a data entry request. The input generator
now requests the oracle to test if Hvideo performed
its task correctly on the input given for data entry.

Dept of ISE,SCE Page 8

 SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING
The oracle uses the input to check if the information to be entered into the database has been
entered correctly or not. The oracle returns a pass or no pass to the input generator.

TEST CASES
The essence of software testing is to determine a set of test cases for the item to be tested.

A complete test case will contain a test case identifier, a brief statement of purpose (e.g., a
business rule), a description of preconditions, the actual test case inputs, the expected outputs,
a description of expected post conditions, and an execution history.

The execution history is primarily for test management use—it may contain the date when the
test was run, the person who ran it, the version on which it was run, and the pass/fail result.

Test case execution entails establishing the necessary preconditions, providing the test case
inputs, observing the outputs, comparing these with the expected outputs, and then ensuring
that the expected post conditions exist to determine whether the test passed.

Test cases need to be developed, reviewed, used, managed, and saved.

Fig: Typical test case information

Test case ID
Purpose
Preconditions
Inputs
Expected outputs
Post conditions
Execution history
Date result version run by

INSIGHTS FROM A VENN DIAGRAM

Testing is fundamentally concerned with behaviour, and behaviour is orthogonal to the code-
based view (structural view) common to software developers.

A quick distinction is that the code-based view focuses on what it is and the behavioural view
considers what it does.

A simple Venn diagram clarifies many questions related to testing.

Consider a universe of program behaviours. Given a program and its specification, consider the
set S of specified behaviours and the set P of programmed behaviours.
Figure shows the relationship between the specified and programmed behaviours. Of all the
possible program behaviours, the specified ones are in the circle labelled S and all those
behaviours actually programmed are in P.

With this diagram, we can see more clearly the problems that
confront a tester. What if certain specified behaviours have not been
programmed? These are faults of omission. Similarly, what if certain
programmed (implemented) behaviours have not been specified?
These correspond to faults of commission and to errors that occurred
after the specification was complete. The intersection of S and P is
the “correct” portion, that is, behaviours that are both specified and
implemented.

Dept of ISE,SCE Page 9

 SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING

The new circle in Figure below is for test cases.

Now, consider the relationships among sets S, P, and T.

There may be specified behaviours that are not tested
(regions 2 and 5), specified behaviours that are tested
(regions 1 and 4), and test cases that correspond to
unspecified behaviours (regions 3 and 7).

Similarly, there maybe programmed behaviours that are not

tested (regions 2 and 6), programmed behaviours that are
tested (regions 1 and 3), and test cases that correspond to
behaviours that were not implemented (regions 4 and 7).

IDENTIFYING TEST CASES

Two fundamental approaches are used to identify test cases called functional testing
(specification based testing) and structural testing (code based testing).

Specification-Based Testing / Functional testing

It is based on the view that any program can be considered to be a function that maps values
from its input domain to values in its output range.

The system is considered as a black box and hence the name black box
testing. In black box testing, the content (implementation) of the black
box is not known, and the function of the black box is understood
completely in terms of its inputs and outputs.
With the specification-based approach to test case identification, the only information used is
the specification of the software.
Advantages:
(1) They are independent of how the software is implemented, so if the implementation changes,
the test cases are still useful;
(2) Test case development can occur in parallel with the implementation, thereby reducing the
overall project development interval.

Disadvantages:
Significant redundancies may exist among test cases, compounded by the possibility of gaps of
untested software.

Figure shows the results of test cases identified by two specification-based methods. Method
A identifies a larger set of test cases than does method B. For both methods, the set of test cases
is completely contained within the set of specified behaviour.

Fig: Comparing specification-based test case identification methods.

Dept of ISE,SCE Page 10

 SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING
Because specification based methods are based on the specified behaviour, it is hard to imagine
these methods identifying behaviours that are not specified.

Code-Based Testing / structural testing

Also called as white box testing the implementation (of the black box) is known and used to
identify test cases. The ability to “see inside” the black box allows the tester to identify test cases
on the basis of how the function is actually implemented.

The tester can rigorously describe exactly what is tested.

Test coverage metrics provide a way to explicitly state the extent to which a software item has
been tested, and this in turn makes testing management more meaningful.

Figure shows the results of test cases identified by two code-based methods.

For both methods, the set of test cases is completely contained within the set of programmed
behaviour. Because code-based methods are based on the program, it is hard to imagine these
methods identifying behaviours that are not programmed.

Specification-Based versus Code-Based Debate / (Functional vs Structural testing)

Both approaches are used to identify the test cases.

Specification-based testing uses only the specification to
identify test cases, while code-based testing uses the program
source code (implementation) as the basis of test case
identification.

Consider program behaviours: if all specified behaviours have

not been implemented, code-based test cases will never be able
to recognize this. Conversely, if the program implements
behaviours that have not been specified, this will never be
revealed by specification-based test cases. Therefore, both the
methods are needed.

Specification-based testing often suffers from twin problems of redundancies and gaps. When
specification-based test cases are executed in combination with code-based test coverage
metrics, both of these problems can be recognized and resolved.
Test generation strategies

One of the key tasks in any software test activity is the generation of test cases. The program
under test is executed against the test cases to determine whether it conforms to the
requirements.

Any form of test generation uses a source document. In the most informal of test methods, the
source document resides in the mind of the tester who generates tests based on knowledge of

Dept of ISE,SCE Page 11

 SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING
the requirements. In some organisations, tests are generated using of formal and informal
methods directly from the requirements document serving as the source.

The following fig. summarises several strategies for test generation.

Requirements Test generation algorithms

Finite state Test generation algorithms

State charts Test generation algorithms

Petri nets Test generation algorithms

Timed I/O automata Test generation algorithms

Algebraic and Test generation algorithms

logic

Code Test selection algorithm

Run Time Data

Test cases
Fig: Requirements, Models and Test generation

The top row in the figure captures techniques that are applied directly to the requirements.
These May be informal techniques that assign values to input variables without the use of any
rigorous for formal methods.
These could also be techniques that identify input variables, captures the relationship among
these variables and use formal techniques for test generation and cause – effect graphing.

Another set of strategies falls under the category of model based test generation. These strategies
require that a subset of the requirements be modelled using a formal notation. Such a model is
also known as a specification of the subset of requirements.
FSMs, state charts, petri nets and times I/O automata are some of the well-known and used
formal notations for modelling various subsets of requirements.

Languages based on predicate logic as well as algebraic languages are also used to express
subsets of requirements in a formal manner. There also exists techniques to generate tests
directly from the code. Such techniques fall under code based test generation. These techniques
are useful when enhancing existing tests based on test adequacy criteria.

TEST METRICS:
Metrics refer to a standard of measurement.

Dept of ISE,SCE Page 12

 SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING
 Organisational metrics:
Metricsat the level ofanorganisation are useful in overallprojectplanningandmanagement.
Computing this metric at regular intervals and over all products released over a given duration
shows the quality trend across the organisation.
Organisation level metrics allow senior management to monitor the overall strength of the
organisation and points to areas of weakness. Thus, these metrics help senior management in
setting new goals and plan for resources needed to realise these goals.
Given the time frame from now until the time to bid, the management needs to do a feasibility
analysis to determine whether this goal can be met. If a preliminary analysis shows that it can
be met, then a detailed plan needs to be worked out and put into action.

 Project metrics:

Project metrics relate to specific project. These are useful in monitoring and control of a specific
project. The ratio of actual to planned system test effort is one project metric.
Another project metric is the ratio of the number of successful tests to the total number of tests in the
system test phase.

 Process metrics:
The goal of a process metric is to assess the goodness of the process.
When a Test process consists of several phases, for example unit test, integration test and
system test, one can measure how many defects were found in each phase. It is well known that
the later a defect is found, the costlier it is to fix. Hence, a metric that classifies defects according
to the phase in which they are found assists in evaluating

 Product metrics: generic

Product metrics is related to a specific product. These are useful in making decision related to the
product, for example “should this product be released for use by the customer?”
2 metrics are introduced

 The cyclomatic complexity.

 Halstead metrics
The cyclomatic complexity proposed by Thomas Mc Cabe in 1976 is based on the control flow
of a program. Given the CFG G of program P containing N nodes, E edges and p connected
procedures, the cyclomatic complexity V (G) is computed as follows:
V (G) = E – N + 2p
The term p in V (G) counts only procedures that are reachable from the main function. V (G) is
the complexity of CFG G that corresponds to a procedure reachable from the main procedure.
Also, V(G) is not the complexity of the entire program, instead it is the complexity of a procedure
in p that corresponds to G. Larger values of V(G) tend to imply higher program complexity and
hence a program more difficult to understand and test than the one with a smaller values.
Halstead complexity measures using program size (S) and effort (E), the number of errors (B)
found during a software development effort:

B = 7.6 E0.667 S0.333

Dept of ISE,SCE Page 13

 SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING

 Product metrics: OO software

Product reliability is a quality metric and refers to the probability of product failure for a given
operational profile. Product reliability of software truly measures the probability of generating a
failure-causing test input. If for a given operational profile and in a given environment this
probability is 0, then the program is perfectly reliable despite the possible presence of errors.

 Progress monitoring and trends :

Metrics are often used for monitoring progress. This requires making measurements on a
regular basis over time such measurements offer trends.
For ex: suppose that a browser has been coded, unit
tested and its components integrated. It is now in
the system testing phase.
One could measure the cumulative number of
defects found and plot these over time. Such a plot

Dept of ISE,SCE Page 14

 SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING
will rise over time. Eventually, it is likely to show a saturation indicating that product is reaching
stability.

 Static and dynamic metrics:

Static metrics: Computed without having to execute the product. Number of testable entities in
an application is an example of a static product metric.
Dynamic metrics: Require code execution. For example, the number of testable entities actually
covered by a test suite is a dynamic product metric.

Ex: The average number of testers working on a project is a static project metric. The number
of defects remaining to be fixed could be treated as a dynamic metric as it can be computed
accurately only after a code change has been made and the product retested.

 Testability:
According to IEEE, testability is the “degree to which a system or component facilities the
establishment of test criteria and the performance of tests to determine whether those criteria have
been met”.
High testability is a desirable goal. This is best done by knowing well what needs to be tested
and how well in advance. Thus it is recommended that features to be tested and how they are
to be tested must be identified during the requirements stage. This information is then updated
during the design phase and carried over to the coding phase.

In more complex situations, it might require special hardware and probes in addition to special
features aimed solely at meeting a testability requirement.

Error and fault taxonomies

Process: Refers to how we do something.

Product: End result of a process.

SQA (Software Quality Assurance) tries to improve the product by improving the process.

SQA is more concerned with reducing errors in the development process, while testing is more
concerned with discovering faults in a product.

Faults can be classified in several ways: the development phase in which the corresponding
error occurred, the consequences of corresponding failures, difficulty to resolve, risk of know
resolution and so on.
The following table contains a fault taxonomy that distinguishes faults by the severity of their
consequences.

Fault taxonomy Meaning

Mild Misspelled word
Moderate Misleading or redundant info.
Annoying Truncated names, bills
Disturbing Some transactions not processed.
Serious Lose a transaction
Very serious Incorrect transaction execution
Extreme Frequent “very serious” errors.
Intolerable Database corruption.
Catastrophic System shutdown.
Infections Shutdown that spreads to others.

Dept of ISE,SCE Page 15

 SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING
A software anomaly is defined in that document as “a departure from the expected”.
The IEEE standard defines a detailed anomaly resolution process built around four phases:
recognition, investigation, action, and disposition. Some of the more useful anomalies are given
in Tables below:

Dept of ISE,SCE Page 16

 SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING
Levels of Testing

Levels of testing echo the levels of

abstraction found in the waterfall model of
the software development life cycle.
A diagrammatic variation of the waterfall
model, known as the V-Model is shown
in the fig.
In terms of specification-based testing,
the three levels of definition
(specification, preliminary design, and
detailed design) correspond directly to
three levels of testing— system, integration, and unit testing. A practical relationship exists
between levels of testing versus specification-based and code based testing. The code-based
testing is most appropriate at the unit level, whereas specification-based testing is most
appropriate at the system level.

Testing and Verification

Testing aims at uncovering errors in a program.

Verification aims at showing that a given program works for all possible inputs that satisfy a set of
conditions.
Program verification and testing are best considered as complimentary techniques.
Testing is not a perfect process in that a program might contain errors despite the success of a
set of tests. However, our confidence in the correctness of an application increases when an
application passes a set of thoroughly designed and executed tests.

Verification might appear to be a perfect process as it promises to verify that a program is free
from errors. However, a close look at verification reveals that it has its own weakness. The
person who verified a program might have made mistakes in the verification process; there might
be an incorrect assumption on the input conditions; incorrect assumptions might be made
regarding the components that interface with the program; and so on. Thus, neither verification
nortestingis aperfecttechnique forprovingthecorrectness of programs.

Static testing
Static testing is carried out without executing the application under test. This is in contrast to
dynamic testing that requires one or more execution of the application under test. Static testing
is useful in that lead to the discovery of faults in the application as well as ambiguities and
errors in the requirements and other application related documents that a relatively low cost.
This is especially so when dynamic testing is expensive.

Static testing is best carried out by an individual who did not write the code or by a team of
individuals. Simple process of static testing is illustrated
in the figure

The test team responsible for static testing has access to

the requirement document application and all associated
documents such as design document and user manuals.
The team also has access to one or static testing tools.
Static testing tool takes the application code as input and
generates a variety of that are useful in the test process.

Dept of ISE,SCE Page 17

 SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING
Walkthroughs
Walkthrough is an informal process to review any application related document. For example
requirements are reviewed using a process termed requirements walkthrough. Code is reviewed
using code walkthrough also known as peer code review.

Walkthrough begins with a review plan agreed upon by all members of the team. Each item of
the document for example resource module is reviewed with clearly stated objectives in view.
Detail report generated that list item of concern regarding the document reviewed.

Inspections
Inspection is a more formally define process than a walkthrough. This is usually associated with
code. Several organisations consider formal code instructions as a tool to improve code quality
at the lower cost.

Code inspection is carried out by a team. The team works according to an inspection plan that
consists of the following elements:
1. Statement of purpose
2. Work product to be inspected this includes code and associated documents needed for
inspection
3. Team formation, roles and tasks to be performed;
4. Rate at which the inspection task is to be completed.
5. Data collection forms where the team will record its findings such as defects discovered,
coding standard violations, and time spent in each task.

Members of the inspection team are assigned roles of moderator, reader, recorder andauthor. The
moderator is in charge of the process and leads the review. Actual code is read by the reader,
perhaps with the help of code browser and with large monitors for all in the team to view the
code. The recorder records any errors discovered or issues to be looked into. The author is the
actual developer whose primary task is to help others understand the code.

Use of static code analysis tools in static testing

Static code analysis tools can provide control flow information, presented in terms of a CFG, is
helpful to the inspection team in that it allows the determination of the flow of control under
different conditions. A CFG can be annotated with data flow information to make a data flow
graph.
Several such commercial add well as open source tools are available. Purify from IBM rational
and Klockwork from Klockwork, Inc. are two of the many commercially available tools for static
analysis of C and java programs. Lightweight analysis for program security in Eclipse (LAPSE)
is an open source tool for the analysis for Java programs.

Software complexity and static testing

Often a team must decide which of the several modules should be inspected first. Several
parameters enter this decision making process. One of these being module complexity. A more
complex module is likely to have more errors and must be accorded higher priority during
inspection than a lower priority module.
Static analysis tools often compute complexity metrics using one or more complexity metrics.

Dept of ISE,SCE Page 18

 SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING

Generalized Pseudocode
Pseudocode provides a language-neutral way to express program source code.

Dept of ISE,SCE Page 19

 SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING

Dept of ISE,SCE Page 20

 SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING

The Triangle Problem

The triangle problem is the most widely used example in software testing literature.

Problem Statement
Simple version: The triangle program accepts three integers, a, b, and c, as input. These are
taken to be sides of a triangle. The output of the program is the type of triangle determined by
the three sides: Equilateral, Isosceles, Scalene, or NotATriangle.

Improved version: The triangle program accepts three integers, a, b, and c, as input. These are
taken to be sides of a triangle. The integers a, b, and c must satisfy the following conditions:
C1. 1 ≤ a ≤ 200 c4. a < b + c
C2. 1 ≤ b ≤ 200 c5. b < a + c
c3. 1 ≤ c ≤ 200 c6. c < a + b

The output of the program is the type of triangle determined by the three sides: Equilateral,
Isosceles, Scalene, or NotATriangle. If an input value fails any of conditions c1, C2, or c3, the
program notes this with an output message, for example, “Value of b is not in the range of
permitted values.”

If values of a, b, and c satisfy conditions c4, c5, and c6, one of four mutually exclusive outputs
is given:
1. If all three sides are equal, the program output is Equilateral.
2. If exactly one pair of sides is equal, the program output is Isosceles.
3. If no pair of sides is equal, the program output is Scalene.
4. If any of conditions c4, c5, and c6 is not met, the program output is NotATriangle.

Dept of ISE,SCE Page 21

 SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING
Discussion: The first specification presumes the developers know some details about triangles,
particularly the triangle inequality: the sum of any pair of sides must be strictly greater than
the third side. The upper limit of 200 is both arbitrary and convenient;

Dept of ISE,SCE Page 22

 SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING

Dept of ISE,SCE Page 23

 SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING
Pseudocode

Dept of ISE,SCE Page 24

 SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING
Structured Implementations
Program triangle2 ‘Structured programming version of simpler specification

Dept of ISE,SCE Page 25

 SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING

Dept of ISE,SCE Page 26

 SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING

The NextDate Function

To illustrate logical relationships among the input variables.

Problem Statement: NextDate is a function of three variables: month, date, and year. It returns the
date of the day after the input date. The month, date, and year variables have integer values
subject to these conditions (the year range ending in 2012 is arbitrary
C1. 1 ≤ month ≤ 12
C2. 1 ≤ day ≤ 31
c3. 1812 ≤ year ≤ 2012
We can make our problem statement more specific. This entails defining responses for invalid
values of the input values for the day, month, and year. We can also define responses for invalid
combinations of inputs, such as June 31 of any year. If any of conditions c1, C2, or c3 fails,

Dept of ISE,SCE Page 27

 SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING
NextDate produces an output indicating the corresponding variable has an out-of-range value—
for example, “Value of month not in the range 1...12.” Because numerous invalid day–month–
year combinations exist, NextDate collapses these into one message: “Invalid Input Date.”

Dept of ISE,SCE Page 28

 SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING

Dept of ISE,SCE Page 29

 SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING

Dept of ISE,SCE Page 30

 SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING
The Commission Problem Problem
Statement
A rifle salesperson in the former Arizona Territory sold rifle locks, stocks, and barrels made by
a gunsmith in Missouri. Locks cost $45, stocks cost $30, and barrels cost $25. The salesperson
had to sell at least one lock, one stock, and one barrel (but not necessarily one complete rifle)
per month, and production limits were such that the most the salesperson could sell in a month
was 70 locks, 80 stocks, and 90 barrels. After each town visit, the salesperson sent a telegram
to the Missouri gunsmith with the number of locks, stocks, and barrels sold in that town. At
the end of a month, the salesperson sent a very short telegram showing –1 lock sold. The
gunsmith then knew the sales for the month were complete and computed the salesperson’s
commission as follows: 10% on sales up to (and including) $1000, 15% on the next $800, and
20% on any sales in excess of $1800.

Dept of ISE,SCE Page 31

 SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING
The SATM System
The ATM described here is minimal, yet it contains an interesting variety of functionality and
interactions that typify the client side of client–server systems.

When a bank customer arrives at an SATM station, screen 1 is displayed. The bank customer
accesses the SATM system with a plastic card encoded with a personal account number (PAN),
which is a key to an internal customer account file, containing, among other things, the
customer’s name and account information. If the customer’s PAN matches the information in
the customer account file, the system presents screen 2 to the customer. If the customer’s PAN
is not found, screen 4 is displayed, and the card is kept.

At screen 2, the customer is prompted to enter his or her personal identification number (PIN).
If the PIN is correct (i.e., matches the information in the customer account file), the system
displays screen 5; otherwise, screen 3 is displayed. The customer has three chances to get the
PIN correct; after three failures, screen 4 is displayed, and the card is kept.

On entry to screen 5, the customer selects the desired transaction from the options shown on
screen. If balance is requested, screen 14 is then displayed. If a deposit is requested, the status
of the deposit envelope slot is determined from a field in the terminal control file. If no problem
is known, the system displays screen 7 to get the transaction amount. If a problem occurs with
the deposit envelope slot, the system displays screen 12.

Once the deposit amount has been entered, the system displays screen 13, accepts the deposit
envelope, and processes the deposit. The system then displays screen 14.

If a withdrawal is requested, the system checks the status (jammed or free) of the withdrawal
chute in the terminal control file. If jammed, screen 10 is displayed; otherwise, screen 7 is
displayed so the customer can enter the withdrawal amount.

Once the withdrawal amount is entered, the system checks the terminal status file to see if it
has enough currency to dispense. If it does not, screen 9 is displayed; otherwise, the withdrawal
is processed.

Dept of ISE,SCE Page 32

 SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING
The system checks the customer balance (as described in the balance request transaction); if
the funds in the account are insufficient, screen 8 is displayed. If the account balance is
sufficient, screen 11 is displayed and the money dispensed. The balance is printed on the
transaction receipt as it is for a balance request transaction.

After the cash has been removed, the system displays screen 14. When the “No” button is
pressed in screens 10, 12, or 14, the system presents screen 15 and returns the customer’s
ATM card. Once the card is removed from the card slot, screen 1 is displayed.

When the “Yes” button is pressed in screens 10, 12, or 14, the system presents screen 5 so the
customer can select additional transactions.

The Currency Converter

The currency conversion program is another event-driven program that emphasizes code
associated with a GUI. A sample GUI is shown in Figure.
The application converts US dollars to any of four currencies: Brazilian reals, Canadian dollars,
European Union euros, and Japanese yen. Currency selection is governed by the radio buttons
(option buttons), which are mutually exclusive.
When a country is selected, the system responds by completing the label; for example, “
Equivalent in …” becomes “Equivalent in Canadian dollars” if the Canada button is clicked. Also,
a small Canadian flag appears next to the output position for the equivalent currency amount.

Either before or after currency selection, the user inputs an

amount in US dollars. Once both tasks are accomplished, the
user can click on the Compute button, the Clear button, or the
Quit button. Clicking on the Compute button results in the
conversion of the US dollar amount to the equivalent amount
in the selected currency.

Clicking on the Clear button resets the currency selection, the

US dollar amount, and the equivalent currency amount andthe
associated label. Clicking on the Quit button ends th
application.

Dept of ISE,SCE Page 33

 SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING
Saturn Windshield Wiper Controller
The windshield wiper on some Saturn automobiles is controlled by a lever with a dial. The lever
has four positions: OFF, INT (for intermittent), LOW, and HIGH; and the dial has three positions,
numbered simply 1, 2, and 3. The dial positions indicate three intermittent speeds, and the dial
position is relevant only when the lever is at the INT position. The decision table below shows
the windshield wiper speeds (in wipes per minute) for the lever and dial positions.

QP QUESTIONS:
1. Why do we test software? Discuss what a typical test case information should include.
2. Differentiate between black box and white box testing
3. What is software testing? Why it is so important in software development life cycle?
4. Define the following a) error b)fault c)failure d)incident e)test f)testcases
5. Explain with a neat diagram the currency converter and Saturn wind shield wiper control
6. Write pseudo-code for commission problem
7. Explain basic definitions in perspective of testing
8. Define two fundamental approaches that are used to identify the test cases
9. Explain with diagram different steps involved in testing life cycle
10. Explain in detail fundamental approaches used to identify the test cases
11. State the triangle problem with all conditions also explain its traditional and its structural
implementations
12. Explain testing and debugging lifecycle with a neat diagram
13. List and explain different software quality attributes in detail
14. Explain structural and functional testing
15.Explain different types of software test metrics in detail

IMPORTANT QUESTIONS:
1. Define the following
a. Error b) fault c) failure d) incident e) test case f) exhaustive searching
g) Reliability h) correctness i) operational profile
2. Explain testing life cycle with a neat diagram
3. Explain types of software quality attributes
4. Explain testing and debugging life cycle with the help of a diagram
5. What is a test case? What should a typical test case information consist of?
6. Explain in detail two fundamental approaches that are used to identify test cases.
Or
Differentiate between specification based testing (functional testing) and code based
testing (structural testing)
7. With the help of a neat diagram, explain test generation strategies
8. Explain different types of test metrics
9. With the help of a neat diagram, explain levels of testing
Or

Notes by: Chaitanya V, Asst Prof, ISE,SCE Page 34

 SOFTWARE TESTING MODULE 1 BASICS OF SOFTWARE TESTING

Explain the levels of abstraction and testing in waterfall model

10. With the help of pseudocode, flowchart and program, explain the following:
a) Triangle problem
b) NextDate function
c) Commission problem
11. Explain the following:
a) SATM system
b) Currency converter
c) Saturn windshield wiper controller

Notes by: Chaitanya V, Asst Prof, ISE,SCE Page 35