TSV 3561/TSE3251 Software Verification & Validation

Course evaluation:

Mid term test 20%

Tutorials/Quiz 10%

Assignment 20%

Final Exams 50%

Reference Books:

1. IEEE Standard for Software Verification and Validation (IEEE Std 1012-2004)
2. IEEE Standard for Software Test Documentation (IEEE Std 829-1998).
3. Ron Patton, Software Testing (2nd Edition), Sams, 2006.
4. Steven Rakitin, Software Verification and Validation for Practitioners and Managers, (Second Edition),
Artech House Publishers, 2001.
5. Aristides Dasso and Ana Funes, Verification, Validation And Testing in Software Engineering, Idea Group
Publishing , 2006.
6. Cem Kaner, Jack Falk, Hung Q. Nguyen, Testing Computer Software, (2nd Edition), Wiley, 1999.
Software Testing Background

Highlights of this chapter include

•How software bugs impact our lives

•What bugs are and why they occur
•Who are software testers and what they do
 Software Testing and Background

• In 1947, computers were big, room-sized machines operating on

mechanical relays and glowing vacuum tubes.
• The state of the art at the time was the Mark II a behemoth being built
at Harvard University.
• Technicians were running the new computer through its paces when it
suddenly stopped working.

• They scrambled to figure out why and discovered, stuck between a set
of relay contacts deep in the bowels of the computer, was a moth.

• It had apparently flown into the system, attracted by the light and heat,
and was zapped by the high voltage when it landed on the relay.

The computer “BUG” was born !

 Infamous Software Error Case Studies

1. Disney's Lion King, 1994-1995

• In the fall of 1994, the Disney company released its first
multimedia CD-ROM game for children, The Lion King
Animated Storybook. On December 26th, the day after
Christmas, Disney's customer support received a huge
amount of complaints from the parents. The CD-ROM
was not working on the Personal Computer.
• Disney failed to test the software on a broad
representation of the many different PC Models available
on the market. The software worked on a few systems -
likely the ones that the Disney programmers used to
create the game - but not on the most common systems
that the general public had.
 Infamous Software Error Case Studies

2. Intel Pentium Floating-Point Division Bug, 1994

- Enter the following equation into your PC's calculator: (4195835/
3145727) * 3145727 – 4195835.
- If the answer is zero, your computer is just fine. If you get anything
else, you have an old Intel Pentium CPU with floating-point
division bug - a software bug burned into a computer chip and
reproduced over and over in the manufacturing process.
- Intel management aware of the bugs before they released the chip
set. However, the management ignored the bug because they
believed that it is a small matter. A replacement of the faulty
chips are offered if a user could prove that he was affected by the
bug.
- Intel apologized for the way it handled the bug and took a charge of
more than 400 millions to cover the costs of replacing bad chips.
Intel now reports known problems on its website and carefully
monitors customer feedback on Internet newsgroup.
 Infamous Software Error Case Studies

3. NASA Mars Polar Lander, 1999

- On December 3, 1999, NASA’s Mars Polar
Lander disappeared during its landing attempt
on the Martian surface.
- A Failure Review Board investigated the failure
and determined that the most likely reason for
the Malfunction was the unexpected setting of
a single data bit.
- Most alarming was why the problem wasn’t
caught by internal tests.
 Infamous Software Error Case Studies

4. Patriot Missile Defense System, 1991.

- The U.S. Patriot Missile defense system is a scaled-
back version of the Strategic Defense Initiative (“Star
Wars”) program proposed by President Ronald
Reagan.
- It was first put to use in the Gulf War as a defense for
Iraqi Scud missiles.
- The software bug caused the system fail to defend
against several missiles, including one that killed 28
U.S. soldiers in Dhahran, Saudi Arabia.
- In the Dhahran attack, the system had been operating
for more than 100 hours.
 Infamous Software Error Case Studies

5. The Y2K (Year 2000) Bug, Circa 1974.

- The payroll program relied heavily on date
processing.
- Due to a 2-digit format for a date, the
computations on year 2000 and 2001 will
display as 00 and 01.
- Several hundred billion dollars were spent,
worldwide, to replace or update computer
programs to fix the Y2K failures.
 Infamous Software Error Case Studies

6. Dangerous Viewing Ahead, 2004.

- On April 1, 1994, a message was posted to
several Internet user groups and then quickly
circulated as an email that a virus was
discovered embedded in several JPEG format
pictures available on the Internet.
- A proof-of-concept virus was created, proving
that a JPEG picture could be loaded with a virus
that would infect the system used to view it.
 What is Bug?

• It can cost a cent to fix a bug but, millions

of dollars to distribute a solution.
• Bug is when a software doesn’t work
properly.
• The software didn’t operate as intended.
Terms for Software Failures

• Defect
• Fault
• Problem
• Error
• Incident
• Anomaly
• Variance
• Failure
• Inconsistency
• Feature
• Bug
 Software Bug

1. The software doesn’t do something that the product

specification says it should do.
2. The software does something that the product
specification says it shouldn’t do.
3. The software does something that the product
specification doesn’t mention.
4. The software doesn’t do something that the product
specification doesn’t mention but should.
5. The software is difficult to understand, hard to use,
slow, or – in the software tester’s eyes – will be viewed
by the end user as just plain not right.
 Source of Bugs

• The largest source of bugs is specification.

Design comes second, Code comes later
and others.
Others
Code

Specification

Design
 Reasons for the existence of bugs …

• Specification
- It keeps changing.
- There is no effective communication to the
entire development team.
- Software planning is not done accordingly.
• Design
- The programmers lay out their plan for the software.
- Bugs occur here for the same reason they occur in

the specification.
- It’s rushed, changed, or not well communicated.
 Reasons why bugs exist …

• Coding errors may be more familiar to you if you’re a

programmer. Typically, they can be traced to the
software’s complexity, poor documentation
(especially in code that’s being updated and revised),
schedule pressure, or just plain dumb mistakes. It’s
important to note that many bugs that appear on the
surface to be programming errors can really be traced
to specification and design errors.
• The other category is the catch-all for what’s left. Some
bugs can be blamed on false positives, conditions that
were thought to be bugs but really weren’t. They
duplicate bugs, multiple ones that resulted from the
same root cause.
• Some bugs can also be traced to testing errors.
 The Cost of Bugs

$1000

$100

$10

$1

Specification Design Code Test Release

Figure 1: The cost to fix bugs can increase dramatically over time
 Software Tester

• What software testers do?

The goal of a software tester is to find
bugs, find them as early as possible, and
make sure they get fixed.
 The Criteria for a Good Software Tester

• They are explorers.

• They are troubleshooters.
• They are relentless.
• They are creative.
• They are (mellowed) perfectionists.
• They exercise good judgment.
• They are tactful and diplomatic.
• They are persuasive.
 IEEE Standards

IEEE Standard for Software Test Documentation

IEEE Standards for Verification & Validation

 Definitions

Verification
The process of evaluating a system or component to determine whether the products of a given
development phase satisfy the conditions imposed at the start of that phase.

Validation
The process of evaluating a system or component during or at the end of the development
process to determine whether it satisfies specified requirements.

Software
Computer programs, procedures, and possibly associated documentation and data pertaining to
the operation of a computer system.

Testing
An activity in which a system or component is executed under specified conditions, the results
are observed or recorded, and an evaluation is made of some aspect of the system or
component.

Software Requirement Specification

Documentation of the essential requirements (functions, performance, design constraints, and
attributes) of the software and its external interfaces.

Test Plan
A document describing the scope, approach, resources, and schedule of intended test
activities. It identifies test items, the features to be tested, the testing tasks, who will do each
task, and any risks requiring contingency planning.