Software Testing
Prepared by:Raghav
Target Audience
Beginners
Intermediate
What do we cover?
Basics of testing
Levels of Testing
Testing techniques
Software testing
Relative cost
per defect
125
100
75
50
25
Req Design Code Test Production
Software testing
Points to Ponder:
80% of all defects occur in 20% of the work
Test strategies impact test costs
Conversion of testing from craft to process can
give extensive benefits
Software Testing
What hinders effective testing?
Aspects that hinder Effective
testing
Optimism
Belief that the system works
Negative attitude towards effective testing
Conflict between testers and developers
Testing is expensive
Delivery commitments
The V Model
What is V model?
The V Model
Customer Acceptance
Requirements
System
validation
Software
Requirements testing
Architecture (HLD) Integration
testing
Detailed Design Unit testing
(LLD)
Code
Testing v/s Life Cycle
Stage Test Activity
Requirements Acceptance Test ,System Test
Design (HL) Integration Test
Design (LL) Unit Test
Code Unit test / Debug
Integration Integration test / debug
System test / debug
Acceptance Acceptance test / debug
Why Test?
● Developers not fallible
● Bugs in compilers, languages, DBs, OS
● Certain bugs easier to find in testing
● Don’t want customers to find bugs
● Post release debugging is expensive
Test team
Why separate testing team?
Is Testing everything?
There are other approaches possible to
create good software
Effective methods are
•Inspection
•Design style
•Static analysis
•Language checks
•Development environment
Can Testing be replaced?
NO!!
Even if we use other methods, cannot do away
with testing
Review, inspect, read, walkthrough, better
methodologies
And then TEST
TESTING
Definition: Testing is the process of exercising or
evaluating a system or system component by manual
or automated means to verify that it satisfies specified
requirements. (IEEE 83a)
Testing objective
Process of executing a program with an intent of
finding an error
A good test case is one that has a high probability
of finding an as yet undiscovered error
A successful test is one that uncovers an as yet
undiscovered error
Testing Involves
•Testplan
•Generate test conditions, cases
•Create required test environment
•Execute tests by
Initiating
application under test
Applying inputs that are specified in the already
generated test case
Compare actual results with expected results
Results: Test passed or failed!
A ‘FAILED’ Test
● The term ‘Test Failed’ is used when the program
has failed to give the expected result
● Actually, the test has succeeded in discovering a
fault!
Definitions
•Defect
A deviation from specification or standard
Anything that causes customer dissatisfaction
Verification
All QC activities throughout the life cycle that ensure that
interim deliverables meet their input specification
Validation
The ‘test phase’ of the life cycle which assures that the
end product meets the user’s needs
Debugging
Debugging is the act of attempting to
determine the cause of the symptoms
of malfunctions detected by testing or
by frenzied user complaints. (Beizer)
Testing: static and dynamic
• Static Testing: Verification performed without
executing the system’s code
•Code Inspection
•Reviews
• Dynamic Testing: Verification or validation
performed by executing the system’s code
Testing Techniques
• Black Box
• White Box
• Basis path testing
Black box testing
What is black box testing?
Black box testing
Also called Behavioural testing, focuses on
functional requirement of the software
Black box testing
Detects following categories of errors
Incorrect or missing functions
Interface errors
Errors in data structures and external database
access
Behaviour or performance errors
Initialisation and termination errors
Types of Black box testing
Boundary value analysis
Equivalence partitioning
Special value
BVA(Boundary Value Analysis)
What is BVA?
Boundary value analysis (two variables)
X2
d a<=x1 <=b
c<=x2 <=d
X1
a b
BVA
Great number of errors tends to occur at the
boundaries of the input domain rather than
in the ‘center’
BVA leads to a selection of test cases that
exercise bounding values
Equivalence partitioning
What is equivalence partitioning?
Equivalence partitioning testing
•EC partitions the data to partition
of a set
• Partition refers to collection of
mutually disjoint subsets whose
union is the entire set.
•Choose one data element from
each partitioned set.
• EC based testing allows
- to have a sense of complete
testing.
- helps avoid redundancy
• The KEY is the choice of
equivalence relation.
Special value
What is special value testing?
Special value testing
• Select test data on the basis of features of a function
to be computed
• Tester uses her/his domain knowledge, experience
with similar programs
• Ad-hoc/seat of pants/skirt testing
• No guidelines, use best engineering judgment
• Special test cases/Error guessing
• Is useful – don't discount effectiveness !
Levels of testing
What does each level of testing uncover?
• Unit testing • Regression testing
– implementation deficiencies – fix has not broken existing
functionality
– Code / LLD
- product / code
• Integration testing
– interface deficiencies • Alpha / Beta testing
– architecture / HLD • Acceptance testing
• System testing – customer expectations
– requirement deficiencies – notion of stability (time)
– SRS • Certification testing
– external compliance
– third party
Unit Testing
What is Unit testing?
Unit Testing
● Unit- Smallest testable piece of software
● A unit can be compiled / assembled / linked /
loaded; and put under a test harness
● Unit testing is done to show that the unit does not
satisfy the functional specification and/or its
implemented structure does not match the
intended design structure
Integration Testing
What is integration testing?
Integration Testing
● Integration is the process of aggregating
components to create larger components.
● Integration testing done to show that even though
components were individually satisfactory, the
combination is incorrect or inconsistent
Integration problems
What are the general integration problems?
Types of Integration problems
● Configuration / version control
● I/O format, protocol mismatches
● Conflicting data views/ usage
● Data integrity violated
● Wrong parameters
Integration test approach
• Top down
• Bottom up
• Sandwich M0
M11 M12 M13
M21 M22 M23
M31
Top down integration
• Modules are integrated
by moving downward
• Depth-first or breadth-
first manner M0
• Verifies major control
or decision points early
in the test process M11 M12 M13
• Stubs replace low-level
modules M21 M22 M23
M31
Bottom – up integration
• Construction and
testing with atomic
modules
• Stubs are NOT
needed
• Drivers are needed
Integration testing – some thoughts
• How to choose integration
order – incremental
What to check for ….
• May be based on
criticality of • Interface integrity
interfaces/module • Functional validity
• Risk reduction principle • Information content
may be used
• Performance
• Critically based on ..
- Module used by lot of
other modules
- Module has high
cyclomatic complexity
- Has special non-
functional (e.g,
performance)
requirements
System testing
What is system testing?
System Testing
● A system is the big component
● System testing is aimed at revealing bugs that
cannot be attributed to a component as such, to
inconsistencies between components or planned
interactions between components
● Concern: issues, behaviours that can only be
exposed by testing the entire integrated system
(e.g., performance, security, recovery)
System testing is…
• Functionality Testing
• Compatibility testing
• Installation testing
• Stress testing
• Reliability testing
• Usability testing
• Recovery testing
• Security testing
• Supportability testing
• Performance testing
• Compliance testing
• Storage testing
• …
• Internationalization
testing
Acceptance Testing
● Aims at uncovering ‘implied requirements’
● Aims at evaluating ‘fitness for use’
● Should not find bugs which should have been
found in earlier testing phases
Regression Testing
Testing after changes been made to ensure that
no unwanted changes were made
Test Strategy
● This defines the scope and general directions for
testing for a project
● Should be developed for each project
● Should be reviewed and approved with the
project plan
Test Plan
Test plans specify the test conditions, features,
functions that will be tested for a specific level
of testing. Plans also contain other information
about resources, schedules, etc
Test plans should be prepared as soon as the
corresponding document in the development
life cycle is produced
Test specification
What is test specification?
Test Case
● A sequence of steps, sub steps, and other
actions, performed serially, in parallel, or in some
combination of consecution, that creates the
desired test conditions that the test case is
designed to evaluate
Is complete testing possible?
● To prove that a program is free of bugs is
•Practically impossible
•Theoretically a mammoth exercise
Stop Criteria
When to stop testing?
STOP Criteria
● Time runs out – Poor criteria
● A certain number of errors found
● Require a certain test coverage
● Stop when testing becomes unproductive
Software testing
Exercise
Prepare a checklist for testing
IV & V
What is Independent Verification & Validation?