UNIT I

Introduction to Software Testing

1
 For ensuring the quality & to discover problems!!

 Let’s discuss these questions !!

 What is Software ?
 What is Software Engineering ?
 What is Software Development Life Cycle ?
 What is Software Testing ?
 What are Software Errors, Faults & Failures?
 What is Software Quality?
 What is Software Quality Assurance ?
 Software Quality Factors

2
 What is Software ?
Software is:

– The complete set of computer programs,

procedures, and possibly associated
documentation and data pertaining to the
operation of a computer system.

3
Major Types of Software

Application software
System Software
System software
Operating Systems
Schedules computer events
Hardware
Allocates computer resources
Monitor events
Language translators
Interpreters
Users
Compilers
Application Software Utility programs
Programming languages Routine operations
Assembly language (e.g. sort, list, print)
FORTRAN, BASIC, PL/1 Manage data
PASCAL, C (e.g. create files, merge files
5

5
 What is Software Development
Process ?
Software Development Process is:

- The process by which user needs are translated into a

software product. The process involves translating user
needs into software requirements, transforming the
software requirements into design, implementing the
design in code, testing the code, and sometimes,
installing and checking out the software for operational
use.

6
Software Development Life Cycle

7
 Software Lifecycle Activities
...and their models

Requirements System Object Implemen-

Analysis Testing
Elicitation Design Design tation

Implemented
By
Expressed in Structured By Realized By Verified
Terms Of
By

class...
class...
class... ?
class....?
Use Case Application Solution
Domain Subsystems Source Test
Model Domain
Objects Code Cases
Objects

8
 What is Software Testing??
• Testing is the process of executing a program
with the intent to find the errors.”
• Testing v/s debugging

9
 What is Software Testing ?
One of the practical methods commonly used to detect
the presence of errors (failures) in a computer
program is to test it for a set of inputs.
The output
is correct?
I1, I2, I3, Our program
…, In, …
Expected results
=?
Obtained results

“Inputs”

10
 Let’s define Software Testing
―Testing is the process of demonstrating that errors
are not present.
OR
―The purpose of testing is to show that a program
performs its intended functions correctly.
OR
―Testing is the process of establishing confidence that
a program does what it is supposed to do.

 “Testing is the process of executing a program with the

intent of finding errors”

11
 Software Testing: Definition
―Software testing is a process used to identify the
correctness, completeness and quality of developed
computer software.

Actually, testing can never establish the correctness of

computer software, as this can only be done by formal
verification (and only when there is no mistake in the
formal verification process). It can only find defects,
not prove that there are none.‖

12
 What are Software Errors ,
Faults , Failures ?
 Error (Bug): An incorrect step, process, or data definition
A human action that produces an incorrect result

 Faults: Result of an error OR Representation of an error

 Failure: The inability of a system or component to

perform its required functions within specified
performance requirements.
OR
A software failure occurs if the behavior of the
software is different from expected/specified.
13
 Software Errors, Faults and Failures
 Bug / Defect / Fault consequence of a human error.
 Results in non-conformance to requirements.
 Manifests as failure in running software.

Software development process

software error
software fault
software failure

14
 Testing Terminology

Error

Fault

Failure

Incident

Incident: It’s a symptom associated with failure

that alerts the user to the occurrence of a failure

15
 Objectives of Testing
1.Software Quality Improvement

2.Verification & Validation

3.Software Reliability Estimation

16
 Testing Principles
• Test should be based on customer requirement

• Test cases should be planned before testing

• Should start in small and end towards big

• Testing should be done by independent third party

• Assign best persons for testing

• Document test cases and test result

• Provide expected results if possible.

17
 Why Testing is important?
• Testing is a process that requires more efforts
than any other software engineering activity.
• Testing is a set of activities that can be planned
in advance and conducted systematically.
• If it is conducted haphazardly , then only time
will be wasted and more even worse errors
may get introduced.
• This may lead to have many undetected errors
in the system being developed. Hence
performing testing by adapting systematic
strategies is very much essential in during
development of software. 18
 6.1 strategic approach to software
testing
• A testing strategy provides a process that describes for
the developer, quality analysts and the customer ,the
steps conducted as part of testing. Testing strategy
include-

1.Test planning
2.Test case design
3.Test execution
4.Data collection
5.Effectiveness evaluation
19
• The strategic approach for software testing can
be -
1.Just before starting the testing process the formal
technical reviews must be conducted. this will
eliminate many errors before the actual testing process.
2.At the beginning , various components of the system
are tested , then gradually interface is tested and thus
the entire computer based system is tested.
3. Different testing techniques can be applied at different
point of time.
4.The developer of the software conducts testing. For the
large projects the independent test group (ITG) also
assist the developer.
20
 6.3 Organizing for software testing
• Testing need to be conducted from the beginning of the
software development process.
• Software developer should test the individual unit of
software system for its proper functioning. This is called
as unit testing . That means there must be an
involvement of s/w developers for unit testing
activities.
• In many cases s/w developer performs integration
testing. Thus the complete s/w architecture is been
tested by the s/w developers during development
phases.
• Then the independent test group (ITG) gets involved in
the s/w testing activity.
21
 6.4 Criteria for completion of Testing
• Entry criteria:
 s/w testing should start early in SDLC. This helps to
capture and eliminate defects in the early stages of
SDLC i.e requirement gathering and design phase.
 An early start to testing helps to reduce the number of
defects and ultimately the n/w cost in the end.
 Entry criteria for testing is defined as “specific
conditions or on going activities that must be present
before a process can begin.
 The software testing life cylcel (STLC) specifies the entry
criteria required during each testing phase.

22
 It also defines the time interval or the expected
amount of lead time to make the entry criteria item
available to the process.
 Following are the inputs necessary for the entry criteria
1.The requirements documents
2.Complete understanding of application flow
3.Test plan
• Exit criteria:
 It can be defined as “the specific condition or on going
activities that should be fulfilled before completing the
s/w testing life cycle”.
 The exit criteria can identify the intermediate
deliverables.
23
 Following exit criteria should be considered for
completion of a testing phase
o Ensuring all critical test cases are passed.
o Achieving complete functional coverage.
o Identifying and fixing all the high priority defects.
 The output achieved through exit criteria are-
1.Test summary report.
2.Test logs.

24
 6.5 Strategic issues
• Before the testing starts , specify product requirements
appropriately.
• Specify testing objectives clearly.
• Identify categories of users for the s/w and their role
with reference to s/w system.
• Develop a test plan that emphasizes rapid cycle testing
• Build robust s/w which can be useful for testing
• Use effective formal reviews before actual testing
process begins.
• Conduct formal technical review to access the test
strategy and test cases
25
 Verification and Validation
 Verification:
 Are we building the product right? Requirements verification
 Performed at the end of a phase to
ensure that requirements
established during previous phase Architecture verification

have been met.

Detailed Design verification
 Validation:
 Are we building the right product?
 Performed at the end of the Coding verification

development process to ensure

compliance with product validation
requirements.

26
6.2 Verification and validation

27
28
 Few Causes of Software Errors !
 Faulty requirements definition.
 Client-developer communication failures.
 Deliberate deviations from software requirements.
 Logical design errors.
 Coding errors.
 Non-compliance with documentation and coding
instructions.
 Shortcomings of the testing process.
 User interface and procedure errors.
 Documentation errors.

29
 When are defects introduced?
The majority of defects are introduced in earlier phases.
– Requirements are the top factor in a project’s
success or failure.

Phase % of defects % effort to fix

Requirements 56 82
Design 27 13
Code 7 1
Other 10 4

30
 Cost of fixing defects
 Relative cost of fixing defects
Benchmark: cost at requirements phase = 1

Phase found Cost ratio

Requirements 1
Design 3– 5
Coding 10
Unit / integration testing 15 - 40
System / acceptance testing 30 – 70
Production 40 - 1000

31
 Why Testing is Necessary ?
Testing decreases cost
Cost of finding and
correcting fault
100 x

10 x

1x

0.1 x
Requirements Design Coding Testing Maintenance

32
 What is Software Quality ?
 Conformance to requirements.

 Narrowest sense of software quality.

 Lack of bugs.
 Low defect rate (# of defects/size unit)
 High reliability (number of failures per n hours of
operation).
Mean Time To Failure (MTTF): probability of
failure-free operation in a specified time.

33
 What is Software Quality ?

Software quality is:

 The degree to which a system, component, or
process meets specified requirements.

 The degree to which a system, component, or

process meets customer or user needs or
expectations.

34
What is Software Quality ?

Software quality is:

 Conformance to explicitly stated functional and

performance requirements, explicitly documented
development standards, and implicit characteristics
that are expected of all professionally developed
software.

35
 Importance of Software Quality
 Software is a major component of computer systems
(about 80% of the cost) – used for
 Communication (e.g. phone system, email system).
 Health monitoring.
 Transportation (e.g. automobile, aeronautics).
 Economic exchanges (e.g. e-commerce, banks).
 Entertainment.
 etc.

 Software defects are extremely costly in term of

 money
 reputation
 loss of life
36
Quality Attributes

37
 Quality Attributes
 Need to deliver the functionality required by the user
But also be:

 Maintainable (can evolve to meet changing needs)

 Dependable (error free and reliable)

 Efficient (don’t waste system resources)

 Usable (must be easily mastered by the target users)

 Software engineering is concerned with producing
quality software
38
 Software Quality Factors
Product Operations
 Correctness: Extent to which a program satisfies its
specification and fulfills the user’s functional
requirements
 Does it do what I want?

 Reliability: Extent to which a program can be expected

to perform its intended functions with required precision
 Does it behave properly all the time?

 Efficiency: The amount of computing resources

required by a program to perform its functions
 Will it run fast?

39
 Software Quality Factors
Product Revision

 Maintainability: Effort required to locate and fix an error in

an operational program
 Can I fix it?

 Testability: Effort required to test a program to ensure it

performs its intended functions
 Can I test it?

 Flexibility: Effort required to modify an operational

program
40
 Software Quality Factors
Product Transition

 Portability: Effort required to transfer a program from

one hardware configuration or software system
environment to another
 Can it run on another machine?

 Reusability: Effort to which a program can be used in

other applications
 Can I reuse part of the software?

 Interoperability: Effort required to couple one system

with another
41
What is Software Quality Assurance(SQA) ?

SQA is the process of assuring people that every

effort has been made to ensure that software
products have the desired level of quality
attributes in addition to functional requirements.

42
 Software Quality Assurance (SQA)
Software quality assurance is an umbrella
activity that is applied throughout the software
process.
SQA encompasses:
 A quality management approach
 Effective software engineering technology
 Formal technical reviews
 A multi-tiered testing strategy
 Document change control
 Software development standard and its control
procedure
 Measurement and reporting mechanism.
43
 Error
Testing Life Cycle Error

Requirement Fault
Specification Resolution
Fault Error

Design Fault
Isolation
Error
Fault Incident
Coding Fault
Classification
Fault

Testing

44
 A Testing Cycle
Although testing varies between organizations, there is a cycle to
testing:

 Requirements Analysis: Testing should begin in the

requirements phase of the software life cycle (SDLC).
 Design Analysis: During the design phase, testers work with
developers in determining what aspects of a design are testable
and under what parameter those testers work.
 Test Planning: Test Strategy, Test Plan(s), Test Bed creation.
 Test Development: Test Procedures, Test Scenarios, Test
Cases, Test Scripts to use in testing software.
 Test Execution: Testers execute the software based on the
plans and tests and report any errors found to the development
team.
 Test Reporting: Once testing is completed, testers generate
metrics and make final reports on their test effort and whether or
not the software tested is ready for release.
 Retesting the Defects

45
 Test Cases
Test Case ID

Purpose

Preconditions

Inputs

Expected Outputs

Post Conditions
Execution History
Date Result Version Run By

Typical Test Case Information

46
 Contents of a Test Case
 "Boilerplate": author, date, purpose, test case ID
 Pre-conditions (including environment)
 Inputs
 Expected Outputs
 Observed Outputs
 Pass/Fail
 Common usage: Pass when expected and observed
outputs agree
 BUT Myers [The Art of Software Testing] defines a
successful test as one that reveals a fault, i.e.,
expected and observed outputs disagree.

47
 Levels of Testing
Requirement System
Specification Testing

Preliminary Integration
Design Testing

Detailed Unit Testing

Design

Coding

48
49
 6.6 Testing strategies for conventional software
• Various testing strategies for conventional s/w
are-
1.Unit testing
2.Integration testing
3.Validation testing
4.System testing

50
Thank You

51