Chapter Five

Testing Process
Test plan
• A test plan is a document that outlines the strategy, objectives,
resources, and schedule of a software testing process.
⁻ The test plan will typically include details such as the type and number of
tests that need to be conducted, the purpose of each test, the required
tools, and how test results will be analyzed and reported.

• It is regularly updated throughout the testing process to reflect any

discoveries or changes in strategy.
Why are Test Plans important?
• A test plan is essential for several reasons.
• Firstly, it is a communication tool between stakeholders and testing team
members.

• This ensures that everyone understands what, why, and how to test.

• It will also outline how to report test findings, what to consider as a pass or
fail, and any other criteria that may be applicable.

• It will outline the expected outcomes and ensure that testing happens
according to plan.
How to Create a Test Plan?
• Step 1: Analyze the product
• Before writing an effective test plan, you first need to analyze the product.

• This means understanding what the product does, how it works, and what its
potential users will want to use it for.

• This analysis will help you identify the areas that need to be tested and the types of
tests that will be most effective.

• Once you understand the product well, you can start to uncover areas where
problems may occur.

• By identifying potential problem areas, you can develop a plan for testing the
product.
• Step 2: Define Test Objective
• The test objective will depend on the software’s purpose, such as
verifying whether the software meets user requirements,
identifying software defects, or assessing the software’s
completeness.
• The test objective will determine the type of tests that need to be
conducted and how the test results will be analyzed and reported.
• Step 3: Develop a Test Strategy
• The next step in creating a test plan is to develop a test strategy.

• A number of different factors need to be considered when developing a test

strategy, including the type of testing, resources required, and schedule for
testing.

• This strategy should be tailored to the specific project and consider the
overall objectives, scope, and risks.
Test Scope
Identify Testing Type

• There are various types of testing that can be performed on

software, and each has its own benefits and drawbacks.

• One common type of testing is functional testing,

• Tests the software for compliance with its specified requirements.
• This type of testing is performed manually or using automation
tools.
• Another type of testing is performance testing,
• Assesses the speed and stability of the software.

• Other types include security testing, compatibility testing, and

usability testing.

• In general, however, all software should undergo at least some

basic functional testing and performance testing before being
released.
Document Risk & Issues

• A test plan also outlines the risks and issues associated with a
software project.

• The plan communicates the risks and issues to the project team and
management.

• These could be technical risks, such as problems with the code or

architecture, or non-technical risks, such as changes in the business
environment.
Example

Risk Mitigation

Incomplete or confusing Check with the project’s people to ensure everyone

instructions understands what’s needed.

Software or hardware that Make sure the tech staff works with the software and
doesn’t work together hardware that will be used.
Create Test Logistics

• Creating test logistics is an important part of writing a test

plan.

• This includes deciding where and when testing will take place,
as well as what resources will be needed.

• This includes the timeline for the testing process, the resources
needed for the testing, and any other relevant information.
• Step 4: Define Test Criteria
• It assess whether a test has passed or failed and are
closely linked with test objectives.
• The test criteria will depend on the type of test, and they
should be clearly defined to avoid confusion.
• For example, a test for checking data accuracy may use a
certain percentage of accuracy as the test criteria.
• Suspension Criteria
• Suspension criteria are a set of conditions that, if met, will cause
the testing process to be suspended.
• This could be due to a critical bug being found, a change in scope,
or anything else that could significantly impact the testing process.
• Suspension criteria should be agreed upon by all parties involved
in the testing process and should be clearly defined from the start.
Exit Criteria
• The exit criteria for a test plan are the conditions that must
be met before the testing can be considered complete.

• This includes both functional and non-functional criteria.

• Functional criteria are those that relate to the functionality

of the system under test, while non-functional criteria are
those that relate to aspects such as performance, scalability,
security, etc.
• Step 5: Resource Planning
• The next step when creating a test plan is to outline the resources
required for testing.
• This includes determining whether internal or external resources
will be required and what their skill sets and qualifications are.
• For example:
• If you require specific software programs for testing which are unavailable,
this may delay the testing process.

• A test plan should also outline safety requirements to avoid any last-minute
changes that may disrupt the testing process.

• Human resource
• The human resources required for testing include testers, developers, and other staff
involved in the testing process.

• System Resource
• When creating a test plan for a system, it is important to consider the system resources
required for testing, such as hardware, software, and so on.
• Step 6: Plan Test Environment
• This will depend on the type of testing, such as functional,
usability, or load testing.
• Besides, it is important to consider any variations that may occur
due to changes in the environment.
• For example, if you are conducting a functional test, you will need
a testing environment that resembles the live environment as
closely as possible. This may include real data, network
connections, and so on.
• Step 7: Schedule & Estimation

• When creating a test plan, you need to outline the schedule and estimate for

the testing process.

• This includes determining whether the testing phase is the same length as the

development phase, or if you need to extend the testing phase.

• It is important to consider factors such as the complexity of the software, the

testing environment, and other applicable factors.

• Step 8: Test Deliverables
• Must outline the test deliverables when creating a test
plan.
• Test deliverables are the final outputs of the testing
process, such as test reports, defects, and enhancements.
• They may also include a list of unsupported features, an
explanation of what is missing, and why they were not
tested.
Test Report

• A test report is an organized summary of testing objectives, activities,

and results.

• It is created and used to help stakeholders (product manager,

analysts, testing team, and developers) understand product quality
and decide whether a product, feature, or a defect resolution is on
track for release.
Importance

• It helps in appraising how well testing was performed and identifying

the reasons behind a failed/negative test report. The data derived for
the report is crucial for the business.

• With the help of test reporting analysis, the stakeholders like testers,
developers, analysts, product managers understand the quality of
overall testing and test automation activities.
• It helps stakeholders figure out the origin of the issue or at what stage
it arose.

• It helps analyze whether the problem occurred because of defective

automation scripts, mismanaged backend, unstable infrastructure, or
weak implementation.
• Common test reports can be identified as:

• Test incident report:

• A test incident report intimates about any defect that has occurred
during the testing cycle.
• Each defect has a unique ID in the defect depository; a test
incident report registers all the defects encountered during the
process.
• High impact test incidents are highlighted in the test summary
report.
• Test cycle report:
• It includes a set of test cases required for achieving
specific testing goals test of a test cycle.
• Each cycle uses a different product build.
• So, information on the product progress through various
stages is provided through the test cycle report.
• Test summary report:

• The final stage in a test cycle is the stage of product release.

• So, the team should have enough information at the end of the cycle to
understand the readiness of the product for the release.

• Test summary report summarizes the final results of the test cycle.

• There can be two types of test summary reports:

• The first one is a phase-wise summary produced after the completion of each
phase

• The second is the final test summary report to provide the final test results.
• Apart from the above, there are other forms of test reports like
• test case report,

• Test execution report,

• Bug status report,

• severity/priority report,

• Failure, etc. that depend on the aspects you want to cover.

Components of a Test Report
• Test basic report should essentially consist of the following
information:
• Project Overview:
• It is a detailed description of the project.
• It should mention information like the project
name, project type, project duration, product
name, product version, and description.
•Test Objectives:
• This mentions the objectives of each stage of a
Test cycle.
• E.g., functional testing, UI testing, regression testing,
security testing, performance testing, etc.
• Test Summary:
• It mentions the summary of overall testing activities performed.
• It includes information regarding the number of test cases executed, details
about failed and passed tests, pass/fail percentage, and comments.
• This information is more useful if presented visually using color indications,
graphs, charts, tables, etc.
• Defect Report:
• It is a crucial component of the test reporting.
• The metrics shown under this component are critical for product
improvement and effective decision-making.
• The report generally provides information about the total number
of bugs encountered while testing, and the status of the bugs.
• For example, whether the bugs are open, closed, or responding to several
resolved, open bugs, the density of defects, and criticality/priority.
• In larger organizations, the above information is not
sufficient.

• Their test reports should entail additional insights on the

• logs, network traffic, screenshots, video recording, and other
vital data to support data-driven decision-making.
Test Reporting Analysis and Challenges

• Speedy Software Release Demand

• The entry of Agile and DevOps concepts, the faster releases has
become a norm; testing happens so quickly and so often, that the
timelines to achieve quality have changed from months to weeks,
days, and even hours. If these conditions aren’t adhered to, the
releases either stopped or delivered
• High Data Volume
• Testing creates a large amount of data resulting from the
exhaustive testing process. The data is either produced by
Test automation that involves more and more testing or
by the increase in the number of devices, versions, and
mobile browsers.
• Improper Data Sorting Mechanism
• In large organizations, there are many sources of testing
data. The data is captured from different testing,
development, and business teams. The large volume of
data becomes unmanageable if there is no predetermined
way of capturing and sorting it, making it impossible to
achieve good test reporting.