Software Testing

Lecture - 3

1
Outline

• Testing Levels

• Software Testing Terminology

• STLC

2
Testing Levels

3
Testing Levels Based on Software Activity

Tests can be derived from requirements and specifications, design

artifacts, or the source code. A different level of testing accompanies
each distinct software development activity.

◆ Acceptance Testing – assess software with respect to requirements.

◆ System Testing – assess software with respect to architectural
design.
◆ Integration Testing – assess software with respect to subsystem
design.
◆ Module Testing – assess software with respect to detailed design.
◆ Unit Testing – assess software with respect to implementation

4
Testing Levels Based on Software Activity

5
Beizer’s Testing Levels Based on Test Process
Maturity

Level 0: There’s no difference between testing and

debugging.
Level 1: The purpose of testing is to show that the software
works.
Level 2: The purpose of testing is to show that the software
doesn’t work.
Level 3: The purpose of testing is not to prove anything
specific, but to reduce the risk of using the software.
Level 4: Testing is a mental discipline that helps all IT
professionals develop higher quality software

6
Level 0 Thinking

Testing is the same as debugging

Does not distinguish between incorrect behavior and

mistakes in the program

Does not help develop software that is reliable or

safe

7
Level 0 Thinking

8
Level 1 Thinking

❑ Purpose is to show correctness

❑ Correctness is impossible to achieve
❑ What do we know if no failures?
▪ Good software or bad tests?
❑ Test engineers have no:
▪ Strict goal
▪ Real stopping rule
▪ Formal test technique
▪ Test managers are powerless

9
Level 2 Thinking
❑ Purpose is to show failures

❑ Looking for failures is a negative activity

❑ Puts testers and developers into an adversarial

relationship

❑ What if there are no failures?

10
Level 3 Thinking
❑ Testing can only show the presence of failures, not
their absence.

❑ Whenever we use software, we incur some risk

❑ Risk may be small and consequences unimportant

❑ Risk may be great and consequences catastrophic

❑ Testers and developers cooperate to reduce risk

11
Level 4 Thinking
✧ Testing is only one way to increase quality

✧ Test engineers can become technical leaders of the

project

✧ Primary responsibility to measure and improve

software quality

✧ Their expertise should help the developers

12
Software Testing Terminology

13
Software Testing Terminology

Validation: The process of evaluating software at the end of

software development to ensure compliance with intended
usage.

Verification: The process of determining whether the products

of a given phase of the software development process fulfill
the requirements established during the previous phase.

14
Software Faults, Errors & Failures

✧ Software Fault : A static defect in the software

✧ Software Error : An incorrect internal state that is the

manifestation of some fault

✧ Software Failure : External, incorrect behavior with respect

to the requirements or other description of the expected
behavior

15
Software Faults, Errors & Failures

❑ A patient gives a doctor a list of symptoms

▪ Failures
❑ The doctor tries to diagnose the root cause, the
ailment
▪ Fault
❑ The doctor may look for anomalous internal
conditions (high blood pressure, irregular heartbeat,
bacteria in the bloodstream)
▪ Errors

16
 A Concrete Example

public static int numZero (int [ ] arr) Test 1

{ // Effects: If arr is null throw NullPointerException [ 2, 7, 0 ]
// else return the number of occurrences of 0 in arr
int count = 0;
for (int i = 1; i < arr.length; i++)
{ Test 2
if (arr [ i ] == 0) [ 0, 2, 7 ]
{
count++;
}
}
return count;
}
 A Concrete Example
Fault: Should start
searching at 0, not 1

public static int numZero (int [ ] arr) Test 1

{ // Effects: If arr is null throw NullPointerException [ 2, 7, 0 ]
// else return the number of occurrences of 0 in arr Expected: 1
int count = 0; Actual: 1
for (int i = 1; i < arr.length; i++)
{ Test 2
Error: i is 1, not 0,
if (arr [ i ] == 0) [ 0, 2, 7 ]
on the first iteration
{ Expected: 1
Failure: none
count++; Actual: 0
}
} Error: i is 1, not 0
return count; Error propagates to the variable count
} Failure: count is 0 at the return statement
STLC

19
Software Testing Life Cycle (STLC)

❖ Sequence of specific activities conducted during the testing

process to ensure software quality goals are met.

❖ STLC Phases:

20
STLC

○ Entry Criteria: Entry Criteria gives the prerequisite

items that must be completed before testing can begin.
○ Exit Criteria: Exit Criteria defines the items that must be
completed before testing can be concluded

21
1. Requirement Phase Testing

● Requirement Phase Testing or Requirement Analysis in

which test team studies the requirements from a testing point
of view. Moreover, the resources, test environment, test
limitations and the testing schedule are also determined.

● Activities in Requirement Phase Testing

○ Identify types of tests to be performed.
○ Gather details about testing priorities
○ Prepare Requirement Traceability Matrix (RTM).
○ Identify test environment details where testing is supposed to
be carried out.
○ Automation feasibility analysis (if required)

22
Requirement Phase Testing

● Entry Criteria
— documented requirements, acceptance criteria, and intended
product architecture.
● Exit Criteria
— approved requirement traceability matrix (RTM) and automation
feasibility report.

23
Requirements Traceability Matrix (RTM)

● Requirement Traceability Matrix (RTM) is a document that

maps and traces user requirement with test cases.

● The main purpose of Requirement Traceability Matrix is to

validate that all requirements are checked via test cases such
that no functionality is unchecked during Software testing.

24
Requirements Traceability Matrix (RTM)

● Why RTM is Important?

○ To achieve the above goal, every QA should understand the

requirement thoroughly and create positive and negative test
cases.
○ How to make sure that the requirement is tested considering all

possible scenarios/cases?
○ How to ensure that any requirement is not left out of the testing

cycle?
○ A simple way is to trace the requirement with its corresponding
test scenarios and test cases.

25
Requirements Traceability Matrix (RTM)

● Helps the testing team to understand the level of testing activities

done for the specific product.

26
Requirements Traceability Matrix (RTM)

● Types of Traceability Test Matrix

○ Forward traceability
○ Backward or reverse traceability
○ Bi-directional traceability ( Forward+Backward)

● How to create Requirement Traceability Matrix:

● Step 1: Select a Test Case from the test suite
● Step 2: Identify the Technical Requirement that this test case is verifying.
● Step 3: Note this Technical Requirement (TR) in the Test Case.
● Step 4: Identify the Business Requirement for which the TR is defined
● Step 5: Note the BR (Business Requirement) in Test Case

27
Requirements Traceability Matrix (RTM)

● Example:

28
Requirements Traceability Matrix (RTM)

TC# Test Case Test Data Test Steps Expected Result

1 Verify Id:test 1. Go to Login page Login Successful

Login pass: 1234 2. Enter UserId
3. Enter Password
4. Click Login

TC# BR# TR# Test Test Data Test Steps Expected

Case Result

1 B1 T94 Verify Id:test 1. Go to Login page Login

Login pass: 1234 2. Enter UserId Successful
3. Enter Password
4. Click Login

29
2. Test Planning

● A Senior QA manager determines the test plan strategy along with

efforts and cost estimates for the project.

● Activities Test Planning

○ Preparation of test plan/strategy document for various types of testing

○ Test tool selection

○ Test effort estimation

○ Resource planning and determining roles and responsibilities.

○ Training requirement

30
Test Planning

● Entry Criteria
— requirement analysis, RTM, and automation feasibility
report.
● Exit Criteria
— approved test plan including timelines and risk/cost
analysis.

31
Test Plan
● A Test Plan is a detailed document that describes the test strategy,
objectives, schedule, estimation, deliverables, and resources
required to perform testing for a software product.
● It serves as a blueprint to conduct software testing activities, which
is minutely monitored and controlled by the test manager.
● How to write a Test Plan
Analyze the product
Design the Test Strategy
Define the Test Objectives
Define Test Criteria
Resource Planning
Plan Test Environment
Schedule & Estimation
Determine Test Deliverables
32
Software Test Estimation

● Test Estimation is a management activity which approximates how

long a Task would take to complete.
● Estimating effort for the test is one of the major and important tasks
in Test Management.
● What to Estimate?

33
Software Test Estimation Techniques

● Work Breakdown Structure

● 3-Point Software Testing Estimation Technique
● Wideband Delphi technique
● Function Point/Testing Point Analysis
● Use – Case Point Method
● Percentage distribution
● Ad-hoc method

34
Software Test Estimation

● Work Breakdown Structure

Step 1) Divide the whole project task into subtasks

Step 2) Allocate each task to team member
Step 3) Effort Estimation For Tasks
Step 4) Validate the estimation

35
3. Test Case Development Phase

● Involves the creation, verification and rework of test cases &

test scripts after the test plan is ready.
● Initially, the Test data is identified and created based on the
preconditions. Then the QA team starts the development
process of test cases for individual units.

● Activities of Test Case Development Phase:

○ Create test cases, automation scripts (if applicable)
○ Review and baseline test cases and scripts
○ Create test data

36
3. Test Case Development Phase

● Entry Criteria
— approved test plan including timelines and risk/cost
analysis.
● Exit Criteria
— approved test cases and automation scripts.

37
4. Test Environment Setup

● Decides the software and hardware conditions under which a

work product is tested.
● Test team may not be involved in this activity if the
development team provides the test environment. The test
team is required to do a readiness check (smoke testing) of
the given environment..

● Activities:
○ Understand the required architecture, environment set-up and
prepare hardware and software requirement list for the Test
Environment.
○ Setup test Environment and test data
○ Perform smoke test on the build 38
4. Test Environment Setup

● Entry Criteria
— system design and project architecture definitions.
● Exit Criteria
— a fully functional test environment and approved test
cases.

39
5. Test Execution Phase
● This phase is carried out by the testers in which testing of the
software build is done based on test plans and test cases
prepared.
● The process consists of test script execution, test script
maintenance and bug reporting.

● Activities:
○ Execute tests as per plan
○ Document test results, and log defects for failed cases
○ Map defects to test cases in RTM
○ Retest the Defect fixes
○ Track the defects to closure

40
5. Test Execution Phase

● Entry Criteria
— all exit criteria from previous steps.
● Exit Criteria
— all tests are performed and results are documented.

41
 6. Test Cycle Closure
● Test Cycle Closure phase is completion of test execution which involves
several activities like test completion reporting, collection of test
completion matrices and test results.

● Activities:
○ Evaluate cycle completion criteria based on Time, Test coverage,
Cost,Software, Critical Business Objectives, Quality
○ Prepare test metrics based on the above parameters.
○ Document the learning out of the project
○ Prepare Test closure report
○ Qualitative and quantitative reporting of quality of the work product to the
customer.
○ Test result analysis to find out the defect distribution by type and severity.

42
 6. Test Cycle Closure
● Entry Criteria
— test results and logging from all previous phases.
● Exit Criteria
— delivered and approved test closure report.

43