Chapter 1

Introduction to Software Testing

1
Lesson outline
• Software Testing
• Levels of software testing
• Principles of software testing
• Software Bugs, Defect, Error, Fault and Failure
• Sources of problems
• Adverse effects of Faulty Software
• Types of SDLC Models: Traditional vs Agile
• Software Testing Life Cycle
What is Software testing

“Software testing is the act of examining the artifacts

and the behavior of the software under test by validation
and verification.”

• Verification: Are we building the product right?

via Static testing --checks correctness of spec. with code
• Validation: Are we building the right product?
via Dynamic testing --refers back to user needs
Source: Software testing - Wikipedia
What is Software Validation, Verification?

According to the Capability Maturity Model (CMMI-SW v1.1),

Software Validation: The process of evaluating software during

or at the end of the development process to determine whether
it satisfies specified requirements.

Software Verification: The process of evaluating software to

determine whether the products of a given development phase
satisfy the conditions imposed at the start of that phase.

Source: Software testing - Wikipedia

Levels of software testing
Four levels of software testing includes:

• Unit testing tests that verify the functionality of a specific

section of code, usually at the function level.
• Integration testing tests to verify the interfaces between
components against a software design.
• System testing tests a completely integrated system to
verify that the system meets its requirements.
• Acceptance testing tests that a system or component must
satisfy and accepted by a user, customer, or other authorized
entity.
Principles of software testing

Defect clustering: approximately 80% of the problems are found in 20% of the modules.
Pestcide paradox: the same set of repetitive tests will not discover new defects.
Testing shows a presence of defects: if no defects are found, it is not a proof of correctness.
Absence of error: Finding and fixing defects does not help if the system build is unusable and does
not fulfill the user’s needs & requirements
Test Strategy

Myth: “Principles are just for reference.

I will not use them in practice .”

This is so very untrue.

Test Principles will help you create an

effective Test Strategy and draft error
catching test cases.
 What is a computer bug?

 Is a problem causing a program to crash or produce invalid

output.
 The problem is caused by insufficient or erroneous logic.

8
 Bugs a.k.a. …
 Defect  Failure
 Fault  Inconsistency
 Problem  Product Anomaly
 Error  Product Incidence
 Incident  Feature :-)
 Anomaly
 Variance

9
 Defective Software
 We develop programs that contain defects
 How many? What kind?
 Hard to predict the future, however…
it is highly likely, that the software we (including
you!) will develop in the future will not be
significantly better.

10
 Sources of Problems
 Requirements Definition: Erroneous, incomplete, inconsistent
requirements.

 Design: Fundamental design flaws in the software.

 Implementation: Mistakes in chip fabrication, wiring, programming

faults, malicious code.

 Support Systems: Poor programming languages, faulty compilers and

debuggers, misleading development tools.

11
 Sources of Problems (Cont’d)
 Inadequate Testing of Software: Incomplete testing,
poor verification, mistakes in debugging.

 Evolution: Sloppy redevelopment or maintenance,

introduction of new flaws in attempts to fix old flaws,
incremental escalation to inordinate complexity.

12
 Adverse Effects of Faulty Software

 Communications: Loss or corruption of

communication media, non delivery of data.

 Space Applications: Lost lives, launch delays.

 Defense and Warfare: Misidentification of friend or

foe.

13
 Adverse Effects of Faulty Software (Cont’d)

 Transportation: Deaths, delays, sudden

acceleration, inability to brake.

 Safety-critical Applications: Death, injuries.

 Electric Power: Death, injuries, power outages,

long-term health hazards (radiation).

14
 Adverse Effects of Faulty Software (Cont’d)

 Money Management: Fraud, violation of privacy, shutdown of

stock exchanges and banks, negative interest rates.

 Control of Elections: Wrong results (intentional or non-

intentional).

 Control of Jails: Technology-aided escape attempts and

successes, accidental release of inmates, failures in software
controlled locks.

 Law Enforcement: False arrests and imprisonments.

15
 Military Aviation Problems
 An F-18 crashed because of a missing exception
condition: if ... then ...
without the else clause that was thought could
not possibly arise.
 In simulation, an F-16 program bug caused the
virtual plane to flip over whenever it crossed the
equator, as a result of a missing minus sign to
indicate south latitude.
16
 Bank Generosity
 A software flaw caused a UK bank to duplicate every
transfer payment request for half an hour. The bank
lost 2 billion British pounds!

 The bank eventually recovered the funds but lost half

a million pounds in potential interest.

17
 Specification
“if you can’t say it, you can’t do it”
 You have to know what your product is before you can say if
it has a bug.
 A specification defines the product being created and
includes:
 Functional requirements that describes the features the product will
support. E.g., on a word processor
 Save, print, check spelling, change font, …
 Non-functional requirements are constraints on the product. E.g,
 Security, reliability, user friendliness, platform, …

18
 A software bug occurs when at least one of
these rules is true
 The software does not do something that the specification
says it should do.
 The software does something that the specification says it
should not do.
 The software does something that the specification does not
mention.
 The software does not do something that the product
specification does not mention but should.
 The software is difficult to understand, hard to use, slow …

19
Most bugs are not because of mistakes in the
code …

 Specification (~= 55%)

 Design (~= 25%)
 Code (~= 15%)
 Other (~= 5%)

20
 Relative cost of bugs
“bugs found later cost more to fix”
 Cost to fix a bug increases exponentially (10x)
 i.e., it increases tenfold as time increases
 E.g., a bug found during specification costs $1 to fix.
 … if found in design cost is $10
 … if found in code cost is $100
 … if found in released software cost is $1000

21
 Bug Free Software
 Software is in the news for the wrong reason
 Security breach, Mars Lander lost, hackers getting credit card
information, etc.
 Why can’t software engineers develop software that just
works?
 As software gets more features and supports more platforms it
becomes increasingly difficult to make it create bug-free.

22
 Discussion …
 Do you think bug free software is unattainable?
 Are their technical barriers that make this impossible?
 Is it just a question of time before we can do this?
 Are we missing technology or processes?

23
 Goal of a software tester
 … to find bugs
 … as early in the software development processes as
possible
 … and make sure they get fixed.

 Advice: Be careful not to get caught in the dangerous

spiral of unattainable perfection.

24
 SDLC Model
A framework that describes the activities performed
at each stage of a software development project.
26
27
Waterfall Model
 Requirements – defines needed
information, function, behavior,
performance and interfaces.
 Design – data structures, software
architecture, interface representations,
algorithmic details.
 Implementation – source code,
database, user documentation,
testing.
V-Shaped SDLC Model
 A variant of the Waterfall that
emphasizes the verification and
validation of the product.
 Testing of the product is planned
in parallel with a corresponding
phase of development
Incremental SDLC Model
 Construct a partial implementation of a
total system
 Then slowly add increased functionality
 The incremental model prioritizes
requirements of the system and then
implements them in groups.
 Each subsequent release of the system
adds function to the previous release, until
all designed functionality has been
implemented.
Spiral SDLC Model
 Adds risk analysis, and 4gl
RAD prototyping to the
waterfall model
 Each cycle involves the same
sequence of steps as the
waterfall process model
Agile Model
 Developed by the
collaboration between
various cross-functional
teams. is also known as an
iterative and incremental
model.
 Focus on process flexibility
and customer satisfaction by
rapid delivery of working
software product and by
breaking the product into
little incremental builds
Software Testing Life Cycle (STLC)

33
Current Challenges of Software Testing
When I ask my students about the challenges they face in testing they typically
reply:
• Not enough time to test properly
• Too many combinations of inputs to test
• Not enough time to test well
• Difficulty in determining the expected results of each test
• Nonexistent or rapidly changing requirements
• Not enough time to test thoroughly
• No training in testing processes
• No tool support
• Management that either doesn't understand testing or (apparently) doesn't care about
quality
• Not enough time
 Activity1 [page limit: 1, due date: April 3, 2023]

1 In software testing, differentiate terms: error, defect, fault, bug, failure

2 What are types of errors, what are types of faults? Give example of each.
3 Describe the STLC
4 Discuss the testing process in V-model in Software Testing.
5 Discuss the difference between strategy and software plan

35