Chapter - 1

Introduction
 SOME SOFTWARE FAILURES

• A major problem of the software industry is its inability to

develop error-free software.
• Had software developers ever been asked to certify that the
software developed is error-free, no software would have
ever been released.
 The Explosion of the Ariane 5 Rocket

• The Ariane 5 rocket was designed by European Space Agency

and it was launched on June 4, 1996.
• It was an unmanned rocket and unfortunately exploded only
after 40 seconds of its take off from Kourou, French Guiana.
• The design and development took ten long years with a cost
of $7 billion.
• The failure of the Ariane 5 was caused by the complete loss of
guidance and altitude information.
• This loss of information was due to specification and design
errors in the software of the system.
• A software fault in the system was identified as a reason for
the explosion by the enquiry committee.
 The Y2K Problem

• The Y2K problem was the most critical problem of the last
century.
• The whole world was expecting something drastic on January
1, 2000.
• Significant sums of money were spent by software companies
to get rid of this problem.
• What was the problem? It was simply the case of using two
digits for the year instead of four digits.
• For instance, 1965 was considered as 65.
• The developers could not imagine the problem of year 2000.
 The Y2K Problem

• What would happen on January 1, 2000? The last two digits

i.e. 00 may belong to any century like 1800, 1900, 2000, 2100,
etc.
• The simple ignorance or a faulty design decision to use only
the last two digits for the year resulted into the serious Y2K
problem.
• Most of the software was re-tested and modified or
discarded, depending on the situation.
 The USA Star-Wars Program

• ‘Patriot missile’ was the result of the USA ‘Star Wars’

program.
• This missile was used for the first time in the Gulf war against
the Scud missile of Iraq.
• Surprisingly, ‘Patriot missiles’ failed many times to hit the
targeted Scud missile.
• One of the failures killed 28 American soldiers in Dhahran,
Saudi Arabia.
• An investigation team was constituted to identify the cause of
failure.
 The USA Star-Wars Program

• The team re-looked at every dimension of the product and

found the reason for the failure.
• The cause of the failure was a software fault.
• There was a slight timing error in the system’s clock after 14
hours of its operation.
• Hence, the tracking system was not accurate after 14 hours of
operations and at the time of the Dhahran attack, the system
had already operated for more than 100 hours.
 Failure of London Ambulance System

• The software controlling the ambulance dispatch system of

London collapsed on October26-27, 1992 and also on
November 4, 1992 due to software failures.
• The system was introduced on October 26, 1992.
• The London Ambulance Service was a challenging task that
used to cover an area of 600 square miles and handled 1500
emergency calls per day.
• Due to such a failure, there was a partial or no ambulance
cover for many hours.
• The position of the vehicles was incorrectly recorded and
multiple vehicles were sent to the same location.
 Failure of London Ambulance System

• Everywhere people were searching for an ambulance and

nobody knew the reason for non-arrival of ambulances at the
desired sites.
• The repair cost was estimated to be £9m, but it is believed
that twenty lives could have been saved if this failure had not
occurred.
• The enquiry committee clearly pointed out the administrative
negligence and over-reliance on ‘cosy assurances’ of the
software company.
 Accounting Software Failures

• Financial software is an essential part of any company’s IT

infrastructure.
• However, many companies have suffered failures in the
accounting system due to errors in the financial software.
• The failures range from producing the wrong information to
the complete system failure.
• There is widespread dissatisfaction over the quality of
financial software.
• If a system gives information in the incorrect format, it may
have an adverse impact on customer satisfaction.
 Testing Process

• TLO-Explain software testing process

 Testing Process

• Testing is an important aspect of the software development

life cycle.
• It is basically the process of testing the newly developed
software, prior to its actual use.
• The program is executed with desired input(s) and the
output(s).
• The observed output(s) is/are compared with expected
output(s).
• If both are same, then the program is said to be correct as per
specifications, otherwise there is something wrong
somewhere in the program.
 What is Software Testing?

• Good testing entails more than just executing a program with

desired input(s).
• There are many definitions of testing. A few of them are given
below:
(i) The purpose of testing is to show that a program performs
its intended functions correctly.
(ii) Testing is the process of establishing confidence that a
program does what it is supposed to do.
• Thus, a more appropriate definition is
• “Testing is the process of executing a program with the
intent of finding faults.”
 Why Should We Test?

• Software testing is a very expensive and critical activity; but

releasing the software without testing is definitely more
expensive and dangerous.
• No one would like to do it. It is like running a car without
brakes.
• The basic issue of this discussion is that we cannot release a
software system without adequate testing.
• The study results may not be universally applicable but, at
least, they give us some idea about the depth and seriousness
of the problem.
• When to release the software is a very important decision.
 Why Should We Test?

• The cost to fix errors increases drastically from the

specification phase to the test phase and finally to the
maintenance phase as shown in Figure 1.4.
 Why Should We Test?

• If an error is found and fixed in the specification and analysis

phase, it hardly costs anything.
• We may term this as ‘1 unit of cost’ for fixing an error during
specifications and analysis phase.
• The same error, if propagated to design, may cost 10 units
and if, further propagated to coding, may cost 100 units.
• If it is detected and fixed during the testing phase, it may lead
to 1000 units of cost.
• If it could not be detected even during testing and is found by
the customer after release, the cost becomes very high.
• We may not be able to predict the cost of failure for a life
critical system’s software.
• The world has seen many failures and these failures have
been costly to the software companies.
 Who Should Do the Testing?

• Testing a software system may not be the responsibility of a

single person.
• Actually, it is a team work and the size of the team is
dependent on the complexity, criticality and functionality of
the software under test.
• The software developers should have a reduced role in
testing, if possible.
• The concern here is that the developers are intimately
involved with the development of the software and thus it is
very difficult for them to point out errors from their own
creations.
 Who Should Do the Testing?
• The testing persons must be cautious, curious, critical but
non-judgmental and good communicators.
• One part of their job is to ask questions that the developers
might not be able to ask themselves or are awkward,
irritating, insulting or even threatening to the developers.
• Some of the questions are :
(i) How is the software?
(ii) How good is it?
(iii) How do you know that it works? What evidence do you
have?
(iv) What are the critical areas?
(v) What are the weak areas and why?
(vi) What are serious design issues?
(vii) What do you feel about the complexity of the source code?
Roles of the persons involved during development and testing are
given in Table 1.6.
 What Should We Test?

• Is it possible to test the program for all possible valid and

invalid inputs? The answer is always negative due to a large
number of inputs.
• We consider a simple example where a program has two 8 bit
integers as inputs.
• Total combinations of inputs are 28*28. If only one second is
required (possible only with automated testing) to execute
one set of inputs, it may take 18 hours to test all possible
combinations of inputs.
SOME TERMINOLOGIES
 Program and Software
• The software is the superset of the program(s).
• It consists of one or many program(s), documentation
manuals and operating procedure manuals.
 Program and Software
• The program is a combination of source code and object
code.
• Every phase of the software development life cycle requires
preparation of a few documentation manuals which are
shown in Figure 1.7.
• These are very helpful for development and maintenance
activities.
 Program and Software
• Operating procedure manuals consist of instructions to set
up, install, use and to maintain the software.
• The list of operating procedure manuals / documents is given
in Figure 1.8.
 Verification and Validation

• Verification:
• “It is the process of evaluating the system or component to
determine whether the products of a given development
phase satisfy the conditions imposed at the start of that
phase.”
• It is the process of reviewing the requirement document,
design document, source code and other related documents
of the project.
 Verification and Validation

• Validation:
• “It is the process of evaluating a system or component during
or at the end of development process to determine whether
it satisfies the specified requirements.”
• It requires the actual execution of the program.
• It is dynamic testing and requires a computer for execution of
the program.
• Here, we experience failures and identify the causes of such
failures.
• Hence, testing includes both verification and validation.
• Thus, Testing = Verification + Validation
 Fault, Error, Bug and Failure

• Error, mistake and defect are synonyms in software testing

terminology.
• When we make an error during coding, we call this a ‘bug’.
• Hence, error /mistake / defect in coding is called a bug.
• A fault is the representation of an error.
• where representation is the mode of expression such as data
flow diagrams, ER diagrams, source code, use cases, etc.
• If fault is in the source code, we call it a bug.
 Fault, Error, Bug and Failure

• A failure is the result of execution of a fault and is dynamic in

nature.
• When the expected output does not match with the observed
output, we experience a failure.
• The program has to execute for a failure to occur.
• A fault may lead to many failures.
• A particular fault may cause different failures depending on
the inputs to the program.
 Test, Test Case and Test Suite
• Test and test case terms are synonyms and may be used
interchangeably.
• A test case consists of inputs given to the program and its
expected outputs.
• Inputs may also contain precondition(s) (circumstances that
hold prior to test case execution), if any, and actual inputs
identified by some testing methods.
• Expected output may contain post-condition(s)
(circumstances after the execution of a test case), if any, and
outputs which may come as a result.
 Test, Test Case and Test Suite

• The set of test cases is called a test suite.

• We may have a test suite of all test cases, test suite of all
successful test cases and test suite of all unsuccessful test
cases.
• Any combination of test cases will generate a test suite.
 Deliverables and Milestones

• Different deliverables are generated during various phases of

the software development.
• The examples are source code, Software Requirements and
Specification document (SRS),Software Design Document
(SDD), Installation guide, user reference manual, etc.
• The milestones are the events that are used to determine the
status of the project.
• For instance, finalization of SRS is a milestone; completion of
SDD is another milestone.
• The milestones are essential for monitoring and planning the
progress of the software development.
 Alpha, Beta and Acceptance Testing

• Customers may use the software in different and strange

ways.
• Their involvement in testing may help to understand their
minds and may force developers to make necessary changes
in the software.
• These three terms are related to the customer’s involvement
in testing with different meanings.
 Acceptance Testing

• This term is used when the software is developed for a

specific customer.
• The customer is involved during acceptance testing.
• He/she may design adhoc test cases or well-planned test
cases and execute them to see the correctness of the
software.
• This type of testing is called acceptance testing and may be
carried out for a few weeks or months.
• The discovered errors are fixed and modified and then the
software is delivered to the customer.
 Alpha and Beta Testing
• These terms are used when the software is developed as a
product for unknown customers.
• Therefore, acceptance testing is not possible.
• Some potential customers are identified to test the product.
• The alpha tests are conducted at the developer’s site by the
customer.
• These tests are conducted in a controlled environment and
may start when the formal testing process is near completion.
• The beta tests are conducted by potential customers at their
sites.
• Unlike alpha testing, the developer is not present here.
 Alpha and Beta Testing
• Customers are expected to report failures, if any, to the
company.
• These failure reports are studied by the developers and
appropriate changes are made in the software.
• The company gets the feedback of many potential customers
without making any payment.
 Quality and Reliability

• Software reliability is one of the important factors of software

quality.
• Other factors are understandability, completeness,
portability, consistency, maintainability, usability, efficiency,
etc.
• These quality factors are known as non-functional
requirements for a software system.
• Software reliability is defined as “the probability of failure
free operation for a specified time in a specified
environment” .
• Although software reliability is defined as a probabilistic
function and comes with the notion of time, it is not a direct
function of time.
 Quality and Reliability
• There is no aging concept in software and it will change only
when we intentionally change or upgrade the software.
• Software quality determines how well the software is
designed (quality of design), and how well the software
conforms to that design (quality of conformance).
• Some software practitioners also feel that quality and
reliability is the same thing.
• If we are testing a program till it is stable, reliable and
dependable, we are assuring a high quality product.
• Unfortunately, that is not necessarily true. Reliability is just
one part of quality.
• To produce a good quality product, a software tester must
verify and validate throughout the software development
process.
 Testing, Quality Assurance and Quality Control

• Most of us feel that these terms are similar and may be used
interchangeably.
• This creates confusion about the purpose of the testing team
and Quality Assurance (QA) team.
• As we have seen in the previous , the purpose of testing is to
find faults and find them in the early phases of software
development.
• We remove faults and ensure the correctness of removal and
also minimize the effect of change on other parts of the
software.
 Testing, Quality Assurance and Quality Control

• The purpose of QA activity is to enforce standards and

techniques to improve the development process and prevent
the previous faults from ever occurring.
• A good QA activity enforces good software engineering
practices which help to produce good quality software.
• The QA group monitors and guides throughout the software
development life cycle.
• This is a defect prevention technique and concentrates on the
process of the software development.
• Examples are reviews, audits, etc.
 Testing, Quality Assurance and Quality Control

• Quality control attempts to build a software system and test it

thoroughly.
• If failures are experienced, it removes the cause of failures
and ensures the correctness of removal.
• It concentrates on specific products rather than processes as
in the case of QA.
• This is a defect detection and correction activity which is
usually done after the completion of the software
development.
• An example is software testing at various levels.
 Static and Dynamic Testing

• Static testing refers to testing activities without executing the

source code.
• All verification activities like inspections, walkthroughs,
reviews, etc. come under this category of testing.
• This, if started in the early phases of the software
development, gives good results at a very reasonable cost.
• Dynamic testing refers to executing the source code and
seeing how it performs with specific inputs.
• All validation activities come in this category where execution
of the program is essential.
 Testing and Debugging

• The purpose of testing is to find faults and find them as early

as possible.
• When we find any such fault, the process used to determine
the cause of this fault and to remove it is known as
debugging.
• These are related activities and are carried out sequentially.
 LIMITATIONS OF TESTING
• We want to test everything before giving the software to the
customers.
• This ‘everything’ is very illusive and has many meanings.
• What do we understand when we say ‘everything’? We may
expect one, two or all of the following when we refer to
everything’:
(i) Execute every statement of the program
(ii) Execute every true and false condition
(iii) Execute every condition of a decision node
(iv) Execute every possible path
(v) Execute the program with all valid inputs
(vi) Execute the program with all invalid inputs
 LIMITATIONS OF TESTING

• These six objectives are impossible to achieve due to time and

resource constraints .
• We may achieve a few of them.
• If we do any compromise, we may miss a bug.
• Input domain is too large to test and there are too many
paths in any program.
• Hence ‘Everything’ is impossible and we have to settle for
‘less than everything’ in real life situations.
 THE V SHAPED SOFTWARE LIFE CYCLE MODEL

• The V shaped model is the modified form of the waterfall

model with a special focus on testing activities.
• The waterfall model allows us to start testing activities after
the completion of the implementation phase.
• This was popular when testing was primarily validation
oriented.
• Now, there is a shift in testing activities from validation to
verification where we want to review / inspect every activity
of the software development life cycle.
 THE V SHAPED SOFTWARE LIFE CYCLE MODEL
• We want to involve the testing persons from the requirement
analysis and specification phase itself.
• They will review the SRS document and identify the weak
areas, critical areas, ambiguous areas and misrepresented
areas.
• This will improve the quality of the SRS document and may
further minimize the errors.
• These verification activities are treated as error preventive
exercises and are applied at requirements analysis and
specification phase, high level design phase, detailed design
phase and implementation phase.
 THE V SHAPED SOFTWARE LIFE CYCLE MODEL

• We not only want to improve the quality of the end products

at all phases by reviews, inspections and walkthroughs, but
also want to design test cases and test plans during these
phases.
• The designing of test cases after requirement analysis and
specification phase, high level design phase, detailed design
phase and implementation phase may help us to improve the
quality of the final product and also reduce the cost and
development time.
 Graphical Representation

• The shape of the model is like the English letter ‘V’ and it
emphasizes testing activities in every phase.
• There are two parts of the software development life cycle in
this model i.e. development and testing and are shown in
Figure.
• We want to carry out development and testing activities in
parallel and this model helps us to do the same in order to
reduce time and cost.
Graphical Representation
Relationship of Development and Testing Parts
• The development part consists of the first four phases (i.e.
requirements analysis and specification, high level design,
detailed design and implementation) whereas the testing part
has three phases (i.e. unit and integration testing, system
testing and acceptance testing).
• The model establishes the relationship between the
development and testing parts.
• The acceptance test case design and planning activities should
be conducted along with the software requirements and
specifications phase.
• Similarly the system test case design and planning activities
should be carried out along with high level design phase.
Relationship of Development and Testing Parts

• Unit and integration test case design and planning activities

should be carried out along with the detailed design phase.
• The development work is to be done by the development
team and testing is to be done by the testing team
simultaneously.
• After the completion of implementation, we will have the
required test cases for every phase of testing.
• The only remaining work is to execute these test cases and
observe the responses of the outcome of the execution.
• This model brings the quality into the development of our
products.
Relationship of Development and Testing Parts

• The encouragement of writing test cases and test plans in the

earlier phases of the software development life cycle is the
real strength of this model.
• We require more resources to implement this model as
compared to the waterfall model.
• This model also suffers from many disadvantages of the
waterfall model like non availability of a working version of
the product, difficulty in accommodating any change, etc.
• This model has also limited applications in today’s interactive
software processes.