Final Year of Computer Engineering (2019

Course)
410245 (D): Software Testing and Quality
Assurance

Unit-2 Test Planning and Quality

Management

Ms. S.H. Lokhande

Assistant Professor
Computer Engineering Department

Sinhgad Institute of Technology & Science ,Narhe,Pune

 Test Planning

• A Test Plan refers to a detailed

document that catalogs the test
strategy, objectives, schedule,
estimations, deadlines, and the
resources required for completing that
particular project.
 Why are Test Plans important?

• They help individuals outside the QA teams

(developers, business managers, customer-
facing teams) understand exactly how the
website or app will be tested.
• They offer a clear guide for QA engineers to
conduct their testing activities.
• They detail aspects such as test scope, test
estimation, strategy, and so on. Collating all
this information into a single document
makes it easier to review by management
personnel or to re-use for other projects.
 Components of a Test Plan

• Scope: Details the objectives of the particular

project.
• Schedule: Details start dates and deadlines
for testers to deliver results.
• Resource Allocation: Details which tester will
work on which test.
• Environment: Details the nature,
configuration, and availability of the test
environment.
 Components of a Test Plan
• Tools: Details what tools are to be used for
testing, bug reporting, and other relevant
activities.
• Defect Management: Details how bugs will
be reported, to whom and what each bug
report needs to be accompanied by. For
example, should bugs be reported with
screenshots, text logs, or videos of their
occurrence in the code?
• Risk Management: Details what risks may
occur during software testing.
 Components of a Test Plan
• Exit Parameters: Details when testing
activities must stop. This part describes the
results that are expected from the QA
operations.
 How to create a Test Plan?
1. Product Analysis
2. Designing Test Strategy
3. Defining Objectives
4. Establish Test Criteria
5. Planning Resource Allocation
6. Planning Setup of Test Environment
7. Determine test schedule and estimation
8. Establish Test Deliverables
 How to create a Test Plan?
1. Product Analysis
Start with learning more about the product
being tested, the client, and the end-users of
similar products. Ideally, this phase should focus
on answering the following questions:
• Who will use the product?
• What is the main purpose of this product?
• How does the product work?
• What are the software and hardware
specifications?
 How to create a Test Plan?
2. Designing Test Strategy
• Test strategy is a set of guidelines that
explains test design and determines how
testing needs to be done.

• The Test Strategy document is developed by

the test manager and defines the following:
1. Project objectives and how to achieve them.
2. The amount of effort and cost required for
testing.
 How to create a Test Plan?
3. Defining Objectives
This phase defines the goals and expected
results of test execution.
 How to create a Test Plan?
4. Establish Test Criteria
• Test Criteria refers to standards or rules
governing all activities in a testing project.
The two main test criteria are:

• Suspension Criteria: Defines the benchmarks

for suspending all tests. For example, if QA
team members find that 50% of all test cases
have failed, then all testing is suspended until
the developers resolve all of the bugs that
have been identified so far.
 How to create a Test Plan?
• Exit Criteria: Defines the benchmarks that
signify the successful completion of a test
phase or project. The exit criteria are the
expected results of tests and must be met
before moving on to the next stage of
development.
• For example, 80% of all test cases must be
marked successful before a particular feature
or portion of the software can be considered
suitable for public use.
 How to create a Test Plan?
5. Planning Resource Allocation

• This phase creates a detailed breakdown of

all resources required for project completion.
• Resources include human effort, equipment,
and all infrastructure required for accurate
and comprehensive testing.
 How to create a Test Plan?
6. Planning Setup of Test Environment

• The test environment refers to software and

hardware setup on which QAs run their tests.
 How to create a Test Plan?
7. Determining Test Schedule and Estimation

• For test estimation, break down the project

into smaller tasks and allocate time and effort
required for each.
• Then, create a schedule to complete these
tasks in the designated time with the specific
amount of effort.
 How to create a Test Plan?
7. Determining Test Schedule and Estimation
Creating the schedule, however, does require
input from multiple perspectives:
• Employee availability, number of working
days, project deadlines, daily resource
availability.
• Risks associated with the project which has
been evaluated in an earlier stage.
 How to create a Test Plan?
8. Establish Test Deliverables
• Test Deliverables refer to a list of documents,
tools, and other equipment that must be
created, provided, and maintained to support
testing activities in a project.
• A different set of deliverables is required
before, during, and after testing.
Deliverables required before testing
• Documentation on
• Test Plan
• Test Design
 How to create a Test Plan?
8. Establish Test Deliverables
Deliverables required during testing
Documentation on
• Test Scripts
• Simulators or Emulators (in early stages)
• Test Data
• Error and execution logs
 How to create a Test Plan?
8. Establish Test Deliverables
Deliverables required after testing
Documentation on
• Test Results
• Defect Reports
• Release Notes
 Approach
• This attribute is used to describe the flow of
the application while performing testing and
for the future reference.
 Test Plan Test Strategy
•A test plan for software project
can be defined as a document •Test strategy is a set of
that defines the scope, guidelines that explains test
objective, approach and design and determines how
emphasis on a software testing testing needs to be done.
effort.

•Components of Test plan

•Components of Test strategy
include- Test plan id, features to
includes- objectives and scope,
be tested, test techniques,
documentation formats, test
testing tasks, features pass or
processes, team reporting
fail criteria, test deliverables,
structure, client communication
responsibilities, and schedule,
strategy, etc.
etc.
 Test Plan Test Strategy

•Test plan is carried out by a

•A test strategy is carried out by
testing manager or lead that
the project manager. It says
describes how to test, when to
what type of technique to
test, who will test and what to
follow and which module to test
test

•Test plan narrates about the •Test strategy narrates about

specification the general approaches

•Test strategy cannot be

•Test plan can change
changed
 Test Plan Test Strategy

•Test planning is done to •It is a long-term plan of action.

determine possible issues and You can abstract information
dependencies in order to that is not project specific and
identify the risks. put it into test approach

•In smaller project, test strategy

•A test plan exists individually is often found as a section of a
test plan

•It is set at organization level

•It is defined at project level and can be used by multiple
projects
 What is a Test Script?

• Test Scripts are a line-by-line description

containing the information about the
system transactions that should be
performed to validate the application or
system under test.
• Test script should list out each step that
should be taken with the expected
results.
 Test Case Test Script
Test case is a step by step The test script is a set of
procedure that is used to test instructions to test an
an application. application automatically.

Test Script is used in the

Test Cases are used for
automation testing
manual testing environment.
environment.

It is done according to the

It is done manually.
scripting format.

The test case template

In the Test Script, we can use
includes Test ID, test data,
different commands to
test procedure, actual and
develop a script.
expected results, etc.
 How to write test cases for software:

Use a Strong Title

• A good test case starts with a strong title. As a best
practice, it’s good to name the test case along the
same lines as the module that you are testing.
• For example, if you’re testing the login page, include
“Login Page” in the title of the test case.
 How to write test cases for software:

Include a Strong Description

• The description should tell the tester what
they’re going to test.
• Sometimes this section might also include
other pertinent information such as the test
environment, test data, and
preconditions/assumptions.
• A description should be easy to read and
immediately communicate the high-level goal
of the test.
 How to write test cases for software:
Include Assumptions and Preconditions
• You should include any assumptions that
apply to the test and any preconditions that
must be met prior to the test being executed.
• This information can include which page the
user should start the test on, dependencies on
the test environment, and any special setup
requirements that must be done before
running the test.
• This information also helps keep the test
steps short and concise.
 How to write test cases for software:
Keep the Test Steps Clear and Concise
• Test cases should be simple. Keep in mind, the
person who wrote the test case might not be
the same person who executes the test itself.
• The test steps should include the necessary
data and information on how to execute the
test.
• This is perhaps the most important part of a
test case. Keep this section clear and concise,
but don’t leave out any necessary details.
Write the test case so that anyone can go in
and perform the test.
 How to write test cases for software:
Include the Expected result
• The expected result tells the tester what they
should experience as a result of the test steps.
This is how the tester determines if the test
case is a “pass” or “fail”.
 How to write test cases for software:
Make it Reusable
• A good test case is reusable and provides long-
term value to the software testing team.
• When writing a test case, keep this in mind.
You can save time down the road by re-using
the test case instead of re-writing it.
 Sample of a Test Case

• Title: Login Page – Authenticate Successfully

on gmail.com
• Description: A registered user should be able
to successfully log in at gmail.com.
• Precondition: the user must already be
registered with an email address and
password.
• Assumption: a supported browser is being
used.
 Sample of a Test Case

• Test Steps:
1. Navigate to gmail.com
2. In the ’email’ field, enter the email address of
the registered user.
3. Click the ‘Next’ button.
4. Enter the password of the registered user
5. Click ‘Sign In’

Expected Result: A page displaying the Gmail

user’s inbox should load, showing any new
message at the top of the page.
 Test Execution

• Test execution is the process of executing the

code and comparing the expected and actual
results.
 Entry and Exit Criteria in STLC

• Entry Criteria: Entry Criteria gives the

prerequisite items that must be completed
before testing can begin.
• Exit Criteria: Exit Criteria defines the items
that must be completed before testing can be
concluded
 Software Test Metrics
• Software test metrics is to monitor and
control process and product. It helps to drive
the project towards our planned goals without
deviation.
• Software test metrics are classified into two
types
1. Process metrics
2. Product metrics
 Software Test Metrics
• #1. Process Metrics
• Software Test Metrics used in the process of
test preparation and test execution phase
of STLC.
• The following are generated during the Test
Preparation phase of STLC:
 Software Test Metrics
• #1.1. Test Case Preparation Productivity
• It is used to calculate the number of Test Cases
prepared and the effort spent for the preparation
of Test Cases.
• Formula:
• Test Case Preparation Productivity = (No of Test
Case)/ (Effort spent for Test Case Preparation)
• E.g.:
• No. of Test cases = 240
• Effort spent for Test case preparation (in hours) = 10
• Test Case preparation productivity = 240/10 = 24 test
cases/hour
 Software Test Metrics
• #1.2. Test Design Coverage
• It helps to measure the percentage of test case
coverage against the number of requirements
• Formula:
• Test Design Coverage = ((Total number of
requirements mapped to test cases) / (Total number
of requirements)*100
• E.g.:
• Total number of requirements: 100
• Total number of requirements mapped to test cases:
98
• Test Design Coverage = (98/100)*100 = 98%
 Software Test Metrics
The following are generated during the Test Execution
phase of STLC:
• #1.3. Test Execution Productivity
• It determines the number of Test Cases that can be
executed per hour
• Formula:
(No of Test cases executed)/ (Effort spent for execution
of test cases)
• E.g.:
• No of Test cases executed = 180
• Effort spent for execution of test cases = 10
• Test Execution Productivity = 180/10 = 18 test
cases/hour
 Software Test Metrics
• #1.4. Test Execution Coverage
• It is to measure the number of test cases executed
against the number of test cases planed.
Formula:
• Test Execution Coverage = (Total no. of test cases
executed / Total no. of test cases planned to
execute)*100
• E.g.:
• Total no. of test cases planned to execute = 240
• Total no. of test cases executed = 160
• Test Execution Coverage = (180/240)*100 = 75%
 Software Test Metrics

• #1.5. Test Cases Passed

• It is to measure the percentage no. of test
cases passed
Formula:
• Test Cases Pass = (Total no. of test cases
passed) / (Total no. of test cases executed) *
100

• E.g.:
• Test Cases Pass = (80/90)*100 = 88.8 = 89%
 Software Test Metrics

• #1.6. Test Cases Failed

• It is to measure the percentage no. of test
cases failed
Formula:

Test Cases Failed = (Total no. of test cases failed)

/ (Total no. of test cases executed) * 100

• E.g.:
• Test Cases Failed= (10/90)*100 = 11.1 = 11%
 Software Test Metrics

• #1.7. Test Cases Blocked

• It is to measure the percentage no. of test
cases blocked
Formula:
Test Cases Blocked = (Total no. of test cases
blocked) / (Total no. of test cases executed) *
100

• E.g.:
• Test Cases Blocked = (5/90)*100 = 5.5 = 6%
 Software Test Metrics
#2. Product Metric
• Software Test Metrics used in the process of defect
analysis phase of STLC.
#2.1. Error Discovery Rate
• It is to determine the effectiveness of the test cases.
• Formula:
Error Discovery Rate = (Total number of defects found
/Total no. of test cases executed)*100
E.g.:

• Total no. of test cases executed = 240

• Total number of defects found = 60
• Error Discovery Rate = (60/240)*100 = 25%
 Software Test Metrics
2.2. Defect Fix Rate
• It helps to know the quality of a build in terms of
defect fixing.
Formula:
• Defect Fix Rate = (Total no of Defects reported as
fixed - Total no. of defects reopened) / (Total no of
Defects reported as fixed + Total no. of new Bugs
due to fix)*100
• E.g.:
• Total no of defects reported as fixed = 10
• Total no. of defects reopened = 2
• Total no. of new Bugs due to fix = 1
• Defect Fix Rate = ((10 – 2)/(10 + 1))*100 = (8/11)100 =
72.7 = 73%
 Software Test Metrics
• #2.3. Defect Density
• It is defined as the ratio of defects to requirements.
• Defect density determines the stability of the
application.
Formula:
• Defect Density = Total no. of defects identified /
Actual Size (requirements)

E.g.:
• Total no. of defects identified = 80
• Actual Size= 10
• Defect Density = 80/10 = 8
 Software Test Metrics
• #2.5. Defect Removal Efficiency
• It allows us to compare the overall (defects found pre
and post-delivery) defect removal efficiency
Formula:
Defect Removal Efficiency = ((Total no. of defects found
pre-delivery) /( (Total no. of defects found pre-delivery )+
(Total no. of defects found post-delivery)))* 100
• E.g.:
• Total no. of defects found pre-delivery = 80
• Total no. of defects found post-delivery = 10
• Defect Removal Efficiency = ((80) / ((80) + (10)))*100 =
(80/90)*100 = 88.8 = 89%
Software Test Metrics
Thank You
Tha
!!