Introduction To Software Testing: A Practical Guide To Testing, Design, Automation, and Execution
Introduction To Software Testing: A Practical Guide To Testing, Design, Automation, and Execution
Introduction To Software Testing: A Practical Guide To Testing, Design, Automation, and Execution
Software Testing
A Practical Guide to Testing, Design,
Automation, and Execution
—
Panagiotis Leloudas
Introduction to
Software Testing
A Practical Guide to Testing,
Design, Automation,
and Execution
Panagiotis Leloudas
Introduction to Software Testing: A Practical Guide to Testing, Design,
Automation, and Execution
Panagiotis Leloudas
Ilioupoli, Greece
Acknowledgments������������������������������������������������������������������������������xv
Introduction��������������������������������������������������������������������������������������xvii
v
Table of Contents
Maintainability Testing�����������������������������������������������������������������������������������26
Compliance Testing���������������������������������������������������������������������������������������27
Static Testing������������������������������������������������������������������������������������������������������29
Code Reviews������������������������������������������������������������������������������������������������29
Requirement Reviews�����������������������������������������������������������������������������������30
Design Reviews���������������������������������������������������������������������������������������������31
Walk-Throughs����������������������������������������������������������������������������������������������32
Inspections����������������������������������������������������������������������������������������������������33
Summary������������������������������������������������������������������������������������������������������������34
vi
Table of Contents
vii
Table of Contents
Loop Testing������������������������������������������������������������������������������������������������130
Data Flow Testing����������������������������������������������������������������������������������������132
Static Testing����������������������������������������������������������������������������������������������������133
Summary����������������������������������������������������������������������������������������������������������135
viii
Table of Contents
ix
Table of Contents
Afterword������������������������������������������������������������������������������������������205
Index�������������������������������������������������������������������������������������������������207
x
About the Author
Panagiotis Leloudas is a software quality
assurance engineer with more than 10 years of
working experience in the industry. He holds
several ISTQB certifications and is an expert
in testing principles, methodologies, and
techniques.
He decided to write this quick guide to
software testing because he needed a go-to
document for all the decisions he had to make
every day on the job and there wasn’t one. He
tried to write down everything that he would
have liked to know when he started his career.
xi
About the Technical Reviewer
Sourabh Mishra is an entrepreneur, developer,
speaker, author, corporate trainer, and
animator. He is passionate about Microsoft
technologies and a true .NET warrior. Sourabh
has loved computers from childhood and
started his career when he was just 15 years
old. His programming experience includes
C/C++, ASP.NET, C#, VB .NET, WCF, SQL
Server, Entity Framework, MVC, Web API, Azure, jQuery, Highcharts, and
Angular. Sourabh has been awarded Most Valuable Professional (MVP)
status. He has the zeal to learn new technologies, sharing his knowledge on
several online community forums.
He is a founder of IECE Digital and Sourabh Mishra Notes, an online
knowledge-sharing platform where people can learn new technologies.
xiii
Acknowledgments
Writing a book is a significant undertaking, and I couldn’t have done it
without the support and assistance of many people along the way. I want
to express my gratitude to everyone who contributed to the creation of this
book and helped me throughout the process.
First and foremost, I want to thank my family for their unwavering
support, encouragement, and patience during the many months of writing
and editing. Your love and support mean the world to me, and I couldn’t
have done this without you.
I also want to thank my editors, Divya Modi and James Markham, for
their insightful feedback, guidance, and support throughout the writing
process. Your expertise and attention to detail were invaluable in shaping
this book into its final form.
I am grateful to Apress Media LLC for believing in this project and
providing the resources necessary to bring it to fruition.
I am indebted to the many experts, researchers, and consultants who
generously shared their knowledge, insights, and expertise to help me
with my research. Your contributions have added depth and richness to
this book.
Finally, I want to thank my readers for their interest in this book and
for taking the time to read it. Your support and feedback mean the world to
me, and I hope this book can make a positive difference in your life.
Thank you all for your support, encouragement, and contributions to
this book. I am forever grateful.
xv
Introduction
A collection of notes, thoughts, and experiences written down to be shared
with the world: this is how I see the creation of this book. It is not meant
to be the ultimate truth about software testing; there is no such a thing
after all. I have taken several courses about software testing from different
organizations, in numerous countries, and I have spent weeks studying
testing material, but most of my knowledge comes from trial and error in
the industry. It is true that in the end you learn only what you practice.
This book takes you on a journey around the software testing world,
covering the basic principles and techniques and showing examples
of how to apply them. Software testing is a safeguard of the quality
of a software product, and a tester is responsible for reporting on the
quality status. Keep in mind that the quality of the product is not the sole
responsibility of a tester; rather, it’s a collective effort from every individual
in the organization. A chain is only as strong as its weakest link.
Different types of testing are applicable to every phase of the software
development life cycle; of course, the priorities and the risks are not the
same in all the products. Testing is all about identifying those risks and
setting up a mitigation plan by executing the necessary tests and analyzing
the results.
In this book, I will demonstrate some of the most important software
testing types and techniques that exist and how to apply them. You will
find out what testing activities take place during every phase of software
development, how to plan the testing activities, how to design the test
cases, how to execute them, and how to report defects and the status of
your activities. It is meant to be easy to understand for everyone, even
with only the slightest technical background or involvement in a product.
I hope you will enjoy the process and pick up a thing or two!
xvii
CHAPTER 1
The Importance
of Software Testing
Software testing is a crucial part of the software development process. It is
the process of evaluating a software system or application to find defects,
errors, or bugs, and verifying that it meets its intended requirements and
functions correctly. Software testing is essential because it ensures that
the software performs as expected, meets user needs, and is reliable and
efficient.
Testing is not a one-time event, but rather a continuous process that
begins in the early stages of development and continues through the
software’s life cycle. The process involves planning, designing, executing,
and evaluating tests to identify and fix issues and to improve the quality of
the software.
There are various types of software testing, each with a unique focus
and objective. Some of the common types include unit testing, integration
testing, system testing, acceptance testing, regression testing, performance
testing, and security testing. Each type of testing has a specific purpose and
is conducted at different stages of the software development life cycle.
One common misconception about software testing is that it is just
about finding defects or errors in software. While detecting defects is an
essential part of software testing, it is not the only goal. Testing is also
about verifying that the software meets user requirements, is easy to use,
and performs as expected. Testing also involves ensuring that the software
is scalable, secure, and efficient.
2
Chapter 1 The Importance of Software Testing
3
Chapter 1 The Importance of Software Testing
Summary
To summarize, software quality is a collective effort, and the
responsibilities are distributed among the various members of a company.
There are plenty of software testing types and techniques for the core
testing team of a project, and the next chapter will dive into the details of
each and every one.
4
CHAPTER 2
Software Testing
Types and Techniques
Software testing is a complex process that involves various types and
techniques to ensure that the software is working correctly and meets its
intended requirements. In this chapter, we will discuss the different types
of software testing and the techniques used in each type, starting with
functional testing and the four levels it involves. We will continue with
nonfunctional testing, which can include performance testing, compliance
testing, and all the different types of -ility testing (usability, compatibility,
availability, etc.). After that, we will see the benefits of static testing, where
we can detect potential defects just by carefully examining the code
without even executing it.
Functional Testing
This type of testing is performed to validate that the software is working as
expected and meets the user’s requirements. The primary focus is on the
software’s functionality and includes various levels of testing such as unit
testing, integration testing, system testing, and acceptance testing. These
levels are widely known as the testing pyramid, a concept introduced by
Mike Cohn.
The lower we go on the testing pyramid (Figure 2-1), the faster and
more isolated the tests are. On the contrary, as we go higher on the testing
pyramid, the tests become slower, and more components are integrated.
6
Chapter 2 Software Testing Types and Techniques
Unit Testing
Unit testing is a software testing technique that focuses on testing
individual units or components of a software system. A unit is the smallest
testable part of any software application, usually a function, method,
or class. The purposes of unit testing are to verify that each unit of the
software performs as expected and to catch errors as early as possible in
the development process.
During unit testing, the individual units of the software are tested in
isolation from the rest of the system. This is achieved by using stubs or
mock objects to simulate the behavior of other parts of the system that the
unit depends on. Unit tests are typically automated, which enables them
to be run frequently and easily, ensuring that changes to the code do not
break existing functionality.
Unit tests are typically written by developers as part of the
development process. Here are some key characteristics of unit tests:
7
Chapter 2 Software Testing Types and Techniques
8
Chapter 2 Software Testing Types and Techniques
Integration Testing
Integration testing is a type of testing that focuses on testing the interaction
between different software components, subsystems, or modules. The
purpose of integration testing is to verify that the integrated software
system works as intended and that the individual components are able to
work together without errors or issues.
The main goal of integration testing is to identify any defects or issues
that may arise when multiple software components are combined. The
objective is to detect problems early in the software development cycle,
before the product is released to the customers. This helps in reducing
the cost and effort required to fix issues in later stages of the software
development process.
There are different approaches to integration testing such as top-down,
bottom-up, and hybrid. In the top-down approach, testing starts from the
highest level and works its way down to the lowest level of the software
hierarchy. In the bottom-up approach, testing starts from the lowest level
and works its way up to the highest level of the software hierarchy. The
hybrid approach combines the top-down and bottom-up approaches.
In integration testing, individual units of code are combined and tested
as a group. Integration tests can be performed at various levels, such as the
following:
10
Chapter 2 Software Testing Types and Techniques
System Testing
System testing is a type of software testing that is performed on a complete,
integrated system to evaluate the system’s compliance with its specified
requirements. It is generally a black-box testing approach, which means
that testers are not concerned with the internal workings of the system.
Instead, they focus on testing the system as a whole to ensure that it meets
the functional and nonfunctional requirements of the stakeholders.
System testing is typically conducted after integration testing, which
tests the integration of different components or subsystems of the system.
The objective of system testing is to verify that the integrated system satisfies
the requirements and behaves as expected in the target environment.
11
Chapter 2 Software Testing Types and Techniques
12
Chapter 2 Software Testing Types and Techniques
Nonfunctional Testing
This type of testing focuses on the software’s nonfunctional aspects, such
as performance, security, and usability. The objective is to ensure that the
software meets its nonfunctional requirements and provides a positive
user experience. Here are some examples of nonfunctional testing:
Performance testing involves testing the software’s performance
under various conditions, such as load, stress, and scalability.
Security testing involves testing the software’s security features to
ensure that it is secure from various types of attacks and vulnerabilities.
Usability testing involves testing the software’s user interface and user
experience to ensure that it is user-friendly and easy to use.
Compatibility testing is testing whether the system can work with
different software, hardware, and operating systems.
Scalability testing is testing whether the system can handle increasing
amounts of data and users.
Reliability testing is evaluating how well the system performs under
different conditions and over time.
Availability testing is testing whether the system is available to users
and can be accessed at all times.
Installability testing is testing how easy it is to install and configure
the software and whether it meets installation requirements and
specifications.
Maintainability testing is testing the software’s ability to be updated,
maintained, and supported over time, and whether it meets maintenance
requirements and specifications.
Compliance testing is testing whether the software meets specific
regulatory or industry standards, such as HIPAA, GDPR, or PCI DSS.
13
Chapter 2 Software Testing Types and Techniques
Performance Testing
Performance testing measures how well a system or application performs
under a given workload. It helps identify bottlenecks and problems in the
application’s performance, and it can also help determine whether the
application can handle expected volumes of data or users. In this section,
we’ll discuss the different types of performance testing, how to plan for and
execute performance tests, and tools and techniques used in performance
testing.
There are different types of performance testing that can be used to
evaluate the performance of an application.
14
Chapter 2 Software Testing Types and Techniques
15
Chapter 2 Software Testing Types and Techniques
16
Chapter 2 Software Testing Types and Techniques
Security Testing
Security testing is a type of software testing that focuses on identifying
vulnerabilities and potential security risks within an application or system.
Security testing is an essential process for ensuring that an application or
system is secure and safe from potential attacks or malicious activity.
There are various types of security testing techniques and tools
available that can be used to identify and address potential security issues.
The following are some of the common techniques:
17
Chapter 2 Software Testing Types and Techniques
18
Chapter 2 Software Testing Types and Techniques
19
Chapter 2 Software Testing Types and Techniques
Usability Testing
Usability testing is a type of testing performed to evaluate how user-
friendly a product is. The objective of usability testing is to determine how
easy it is for users to learn and use the product, how efficient they are at
completing tasks, and how satisfied they are with the product.
20
Chapter 2 Software Testing Types and Techniques
Compatibility Testing
Compatibility testing is a type of software testing that is performed to
ensure that an application is compatible with various hardware, operating
systems, browsers, databases, and other third-party software components.
The purpose of compatibility testing is to identify compatibility issues
between the software application and the systems on which it will be
deployed. The goal is to ensure that the application functions properly
across a variety of environments and configurations.
To perform compatibility testing, the testing team needs to identify
the hardware, software, and other components that the application will be
required to work with. Based on this information, the testing team creates
a test plan that outlines the specific tests that need to be conducted to
ensure compatibility.
21
Chapter 2 Software Testing Types and Techniques
• Browser compatibility
• Database compatibility
Scalability Testing
Scalability testing is a type of nonfunctional testing that evaluates the
ability of a system or application to scale up or scale down in response to
changing workload and data volume. The purpose of scalability testing is
to identify the maximum capacity of a system or application and to ensure
that it can handle increasing levels of load without compromising its
performance or stability.
Scalability testing is particularly important for systems that are
expected to handle a large number of users, transactions, or data volume.
It is typically performed in a controlled environment, using a combination
of manual and automated testing techniques to simulate different levels
of load and measure the system’s response time, throughput, and resource
utilization.
22
Chapter 2 Software Testing Types and Techniques
Reliability Testing
Reliability testing is a type of software testing that aims to determine the
reliability of a software application by measuring its performance in various
conditions over a certain period of time. The goal of this testing is to identify
any defects or issues that could affect the overall reliability of the system.
Reliability testing typically involves subjecting the software to a series
of tests that simulate real-world scenarios and stress conditions. These
tests may include load testing, performance testing, and endurance
testing. The software is also tested for its ability to recover from errors or
failures and its ability to maintain its performance levels under varying
conditions.
The following are some common techniques used in reliability testing:
23
Chapter 2 Software Testing Types and Techniques
Availability Testing
Availability testing is a type of nonfunctional testing that is performed
to measure the ability of a system or application to remain available
and accessible for use by its users over a specified period of time. The
primary objective of availability testing is to identify and mitigate any
factors that could lead to service disruptions or downtime, such as
hardware or software failures, network connectivity issues, or system
configuration errors.
During availability testing, testers simulate various failure scenarios
and measure the system’s ability to recover from them in a timely manner.
Availability metrics are used to measure the system’s performance.
These metrics may include the percentage of uptime, the mean time
between failures (MTBF), the mean time to repair (MTTR), and the
recovery time objective (RTO).
Installability Testing
Installability testing is a type of software testing that evaluates how easy
it is to install, set up, and configure the software. This type of testing is
essential to ensure that the installation process is straightforward, reliable,
and consistent, and that the software meets installation requirements and
specifications.
Installability testing involves testing the software installation process
on different hardware, operating systems, and configurations, and
verifying that it meets the following criteria:
24
Chapter 2 Software Testing Types and Techniques
25
Chapter 2 Software Testing Types and Techniques
Maintainability Testing
Maintainability testing is a type of software testing that evaluates the software’s
ability to be updated, maintained, and supported over time. This type of
testing is essential to ensure that the software can be modified or enhanced
easily and that it remains stable and reliable after updates and changes.
Maintainability testing involves testing the software’s maintainability
by evaluating the following criteria:
26
Chapter 2 Software Testing Types and Techniques
Compliance Testing
Compliance testing is a type of software testing that ensures that software
applications meet certain regulatory, legal, and industry-specific
standards. This type of testing is essential to ensure that the software meets
legal and regulatory requirements and is safe and secure for users.
Compliance testing involves testing the software for compliance with
the following criteria:
27
Chapter 2 Software Testing Types and Techniques
28
Chapter 2 Software Testing Types and Techniques
Static Testing
Static testing is a software testing technique that involves the examination
of code or software documentation without actually executing the
program. The goal of static testing is to identify defects in the code,
requirements, or design before the software is actually executed. This type
of testing is also known as nonexecution testing or verification testing.
Code reviews involve a team of developers and/or testers reviewing
each other’s code to identify any defects or potential issues.
Requirements reviews are about reviewing the requirements
documentation to ensure that they are complete, clear, and accurate.
Design reviews where the team is reviewing the design documentation
to ensure that it meets the requirements and is technically feasible.
Walk-throughs usually are a team of developers looking through the
code or documentation to ensure that it meets the required specifications
and standards.
Inspections are a formal process of reviewing the code or
documentation to identify and document defects.
Code Reviews
Code review is a form of static testing that involves reviewing the
source code of a software application to identify defects, bugs, or other
issues that may impact performance or functionality. Code reviews are
typically performed by one or more individuals who have expertise in the
programming language, design patterns, and coding standards used in the
software application.
The goals of a code review are to identify any defects or issues that may
impact the quality or functionality of the software and to provide feedback
to the developer so that they can make improvements. Code reviews can
be performed manually, using a document or spreadsheet to track issues,
or via specialized software tools that automate the review process.
29
Chapter 2 Software Testing Types and Techniques
Requirement Reviews
Requirement reviews are a form of static testing that involves reviewing the
software requirements to identify defects, errors, or inconsistencies. This
type of review is typically performed by stakeholders, including business
analysts, project managers, developers, testers, and end users.
The goal of a requirement review is to ensure that the software
requirements accurately reflect the needs and expectations of the
stakeholders and that they are complete, unambiguous, and consistent.
During the review process, stakeholders may ask questions, clarify
requirements, or suggest changes or additions to the requirements.
30
Chapter 2 Software Testing Types and Techniques
Design Reviews
Design reviews are a form of static testing that involves reviewing the
design documents of a software application to identify defects, errors,
or inconsistencies. The design documents may include architectural
diagrams, flowcharts, data models, and other design artifacts that describe
how the software will be implemented.
The goal of a design review is to ensure that the software design meets
the requirements of the stakeholders, and that it is efficient, maintainable,
and scalable. During the review process, stakeholders may ask questions,
suggest improvements, or identify potential issues or risks in the design.
31
Chapter 2 Software Testing Types and Techniques
Walk-Throughs
Walk-throughs are a form of static testing that involves a group of
stakeholders reviewing a software artifact, such as a requirements
document, design document, or code, in a step-by-step manner.
During the walk-through, the participants may ask questions, suggest
improvements, or identify potential issues or risks in the artifact.
The goal of a walk-through is to ensure that the software artifact meets
the requirements of the stakeholders, and that it is complete, accurate,
and well-documented. The walk-through process can help to identify
defects or issues early in the development process, before the software is
implemented or executed.
32
Chapter 2 Software Testing Types and Techniques
Inspections
Inspections are a form of static testing that involves a structured and
formal process of reviewing a software artifact, such as a requirements
document, design document, or code, with the goal of identifying defects
or issues. Unlike walk-throughs, inspections are typically conducted by a
small group of trained individuals who follow a set of defined procedures
and checklists.
The inspection process typically involves several stages, including
planning, preparation, inspection, and follow-up. During the inspection
stage, the participants review the software artifact line by line, looking for
defects such as syntax errors, logical inconsistencies, and violations of
coding standards.
33
Chapter 2 Software Testing Types and Techniques
Summary
The different software testing types have different purposes. Functional
testing is responsible for verifying the correct functionality of our
product and finding edge case scenarios that might have been missed.
Nonfunctional testing is about all the things that might affect the product
while its functionality stays intact. Static testing takes care of everything
around the software, without executing the software itself. That can be
coding standards, documentation, or designs. All these software testing
types have their own place in the software development life cycle, and in
the next chapter we are going to understand where we can apply them.
34
CHAPTER 3
Software Development
Life Cycle
The software development life cycle (SDLC) is a process that software
development teams follow to design, develop, test, and deploy high-quality
software applications. The SDLC provides a framework that ensures
the software meets the user’s requirements, is delivered on time and
within budget, and is of high quality. In this chapter, we will discuss the
different phases of the SDLC (Figure 3-1) and their importance in software
development.
Planning Phase
The planning phase of the SDLC is the initial phase where the project’s
objectives and requirements are established. This phase is crucial to
ensure the success of the project, and it involves several key activities, such
as gathering and analyzing requirements, defining the scope of the project,
identifying the stakeholders, and creating a project plan.
During the planning phase, the project team works with stakeholders to
determine the project’s objectives and to identify the business and technical
requirements that the project must meet. The team must analyze the
requirements carefully to ensure that they are accurate, complete, and consistent
with the business goals. This analysis helps the team to determine the resources
required to complete the project, including the team size, budget, and schedule.
36
Chapter 3 Software Development Life Cycle
Once the requirements have been analyzed, the team defines the project
scope, which outlines the deliverables, the constraints, and the assumptions
of the project. The scope helps to provide a clear understanding of what the
project will deliver and what it will not. The scope also helps to identify the
project’s risks, assumptions, and dependencies, which are important factors
that must be considered during the project planning process.
The next step in the planning phase is to identify the project
stakeholders, which include both the internal and external stakeholders.
The team must identify the stakeholders’ needs, expectations, and
requirements, and develop a communication plan to ensure that the
stakeholders are informed and engaged throughout the project.
Finally, the project team creates a project plan that includes a detailed
timeline, milestones, and deliverables. The project plan outlines the tasks,
resources, and timelines required to complete the project, and it helps to
ensure that the project is completed on time, within budget, and with the
expected quality. The project plan also helps to track progress, identify
issues, and manage changes throughout the project’s life cycle.
Testing is not a typical activity in the planning phase of the SDLC. This
phase is primarily focused on defining the project’s scope, goals, objectives,
and requirements, as well as identifying the resources and technology
required to complete the project. Testing activities are not usually performed
in this phase since there is no code or application to test.
However, there are a few test-related activities that can be performed
during the planning phase. For instance, the test manager or team can start
developing a test strategy that outlines the testing approach, test plan, testing
scope, and test schedule for the project. The test manager can also establish
testing goals and objectives that align with the project’s goals and objectives.
The test team can also start identifying the resources and tools
required for testing, such as test management tools, test automation tools,
and test data management tools. This can help ensure that the necessary
testing resources are available when testing activities begin in the later
phases of the SDLC.
37
Chapter 3 Software Development Life Cycle
Here are some ways that QA can contribute to the planning phase:
38
Chapter 3 Software Development Life Cycle
The planning phase of the SDLC typically ends with the creation of a
detailed project plan or a software project management plan (SPMP).
The project plan outlines the scope, objectives, timelines, resource
requirements, and budget for the project. It also includes a risk
management plan and a quality assurance plan that identifies the testing
methodologies, tools, and techniques to be used during the testing phase.
Once the project plan is finalized, the next step is to get the necessary
approvals and funding for the project, after which the actual development
work begins.
It is important to note that the planning phase is not a one-time event,
but an ongoing process that requires continuous review and updates
throughout the SDLC to ensure that the project stays on track and meets
the objectives and goals of the stakeholders.
When everything is in place, we are ready to proceed to the next phase.
39
Chapter 3 Software Development Life Cycle
The following are the key activities that take place during the
requirements gathering phase:
40
Chapter 3 Software Development Life Cycle
41
Chapter 3 Software Development Life Cycle
Design Phase
During the design phase of the SDLC, the requirements gathered in the
previous phase are used to create a detailed design for the software system.
The design phase typically involves the following steps:
42
Chapter 3 Software Development Life Cycle
43
Chapter 3 Software Development Life Cycle
By asking these questions, the QA team can ensure that the design is
intuitive, accessible, and consistent with the platform’s standards.
When the designs are finalized and everyone involved has agreed, the
actual development can start.
44
Chapter 3 Software Development Life Cycle
Development Phase
During the development phase of the SDLC, the software is designed and
developed based on the requirements gathered and design specifications
created during the previous phases.
The development phase involves creating the software code, building
and testing the software, and fixing any issues that arise during testing. The
development team typically consists of software developers, testers, and
technical writers who create user manuals, guides, and other technical
documentation for the software.
The software code is usually written in a high-level programming
language and is translated into machine code by a compiler or interpreter.
The code is then tested using a variety of testing techniques, including unit
testing, integration testing, and system testing, to ensure that it meets the
functional and nonfunctional requirements specified in the earlier phases.
During the development phase, it is important to follow coding
standards and best practices to ensure that the software code is
maintainable, reusable, and easily extendable in the future. It is also
important to ensure that the software code is well-documented and that
all functions and procedures are clearly defined and documented to aid in
future maintenance and support.
During the development phase, the focus of testing shifts from validating
the system design and requirements to identifying and resolving defects in
the software code. The objective of this phase is to ensure that the software is
functioning as expected and that all the requirements are met.
Testing in the development phase involves the following activities:
45
Chapter 3 Software Development Life Cycle
46
Chapter 3 Software Development Life Cycle
By performing these tests, the QA team can identify any issues or bugs
in the feature and work with the development team to fix them.
47
Chapter 3 Software Development Life Cycle
The development phase stops when all the items on the definition
of done list are checked. This means that all the found defects are either
solved or accepted as a risk, the documentation is in place, and the
product owner has accepted the implementation. Then we can deploy the
changes.
Deployment Phase
The deployment phase in the SDLC is the phase in which the developed
software is delivered to the customer for their use. It involves installing the
software in the customer’s environment, configuring it, and making sure
that it works as expected.
The deployment phase involves the following activities:
48
Chapter 3 Software Development Life Cycle
The deployment phase is critical because it is the final stage before the
software is used in a production environment. A successful deployment
ensures that the software meets the customer’s requirements and is ready
for use.
Testing in the deployment phase is important to ensure that the
software is correctly installed and configured in the target environment.
This phase involves tasks such as the following:
49
Chapter 3 Software Development Life Cycle
50
Chapter 3 Software Development Life Cycle
By performing these tests, the QA team can ensure that the deployment
is successful, that the application works as expected in the production
environment, and that it meets the platform’s quality standards. Then we
can proceed to the maintenance phase.
Maintenance Phase
The maintenance phase in the SDLC is the phase in which the software is
released to the market, and it is made available to the end users. During
this phase, the software development team is involved in fixing any bugs,
errors, or defects that are found in the software.
The maintenance phase is an important phase because it is the time
when the software is in use, and it is the time when any issues that arise
must be addressed. The maintenance phase is also the time when updates,
patches, and new features are added to the software.
51
Chapter 3 Software Development Life Cycle
52
Chapter 3 Software Development Life Cycle
53
Chapter 3 Software Development Life Cycle
54
Chapter 3 Software Development Life Cycle
Summary
The software development life cycle comprises several stages, each with
its set of activities, deliverables, and milestones and the responsibilities
of a tester vary and overlap in every phase. It is important for the tester
to be involved as early as possible and to have a deep understanding of
the system being tested. Since there are plenty of activities for the tester
during the SDLC, it is crucial to plan ahead, estimate, and prioritize all the
activities.
55
CHAPTER 4
Test Planning
Test planning is an essential activity in software testing that involves creating
a comprehensive plan for testing the software application. The test plan
outlines the testing objectives, scope, approach, resources, and schedule
for the testing phase of the software development life cycle (SDLC). Test
planning ensures that testing is done effectively and efficiently and helps to
minimize the risk of defects in the software application.
In this chapter, we will go through the most important aspects of test
planning. First we will examine how to define the testing objectives and
then how to determine the scope of the system under test. It is important
to select the right testing approach for the product, because different
products have different priorities and risks. Then, based on the testing
approach, we are going to identify the resources and develop a test
schedule. We’ll continue with the definition of the test cases, and we will
find the applicable test data for the defined test cases.
We will see what a defect management process should contain and
when we can stop testing. After everything is in place, we will create a test
plan document, and we will see which tools and techniques are useful.
58
Chapter 4 Test Planning
59
Chapter 4 Test Planning
There are several testing approaches, each with its own set of
advantages and disadvantages. The following are some of the most
common testing approaches:
61
Chapter 4 Test Planning
62
Chapter 4 Test Planning
63
Chapter 4 Test Planning
6. Review and refine the test case: Review the test case
to ensure that it is complete, accurate, and effective
in verifying the software functionality. Make any
necessary revisions to improve the test case.
64
Chapter 4 Test Planning
65
Chapter 4 Test Planning
66
Chapter 4 Test Planning
67
Chapter 4 Test Planning
68
Chapter 4 Test Planning
70
Chapter 4 Test Planning
71
Chapter 4 Test Planning
72
Chapter 4 Test Planning
Summary
When a new feature is under test, we need to make sure that we plan it
accurately. This will give a nice overview of what we test and how long it
takes. We started simply, by defining the testing objectives and the scope,
and then we move on with the testing approach, the resources we need,
and the expected schedule. We saw the items that need to be present in the
defect management process and when we can decide to stop testing. At
the end we created a test plan document, using some common tools and
techniques. In my view the most important step is the creation of the test
cases, and thankfully there are several test design techniques we can follow
to generate more test cases and to test the right things.
73
CHAPTER 5
Test Design
Techniques
Test design techniques are methods used to create test cases that cover all
possible scenarios and use cases of the software application being tested.
Several test design techniques are available, each with its own strengths
and weaknesses. In this chapter, we will discuss some of the popular
test design techniques. We will see some common techniques, such as
equivalence partitioning and boundary value analysis, and some more
obscure but quite useful in complex situations, such as pairwise testing
and modified condition decision coverage.
Black-Box Testing
Black-box testing is a software testing technique in which the internal
workings of the system being tested are not known to the tester. The
tester focuses on the system’s inputs and outputs and tests the system’s
functionality based on predefined requirements and specifications.
In black-box testing, the tester treats the system as a black box and tests
it by providing inputs and observing the outputs. The objective of black-
box testing is to detect any errors, bugs, or defects that are present in the
system’s functionality, without knowing how the system works internally.
76
Chapter 5 Test Design Techniques
77
Chapter 5 Test Design Techniques
Equivalence Partitioning
Equivalence partitioning is a technique used in software testing that
involves dividing a set of test conditions into groups or partitions that are
equivalent or similar to each other. The idea behind this technique is to
minimize the number of test cases that need to be executed while still
ensuring that all possible scenarios are covered.
In equivalence partitioning, test cases are created based on the
equivalence classes. Equivalence classes are a set of input conditions that
are expected to behave in the same way. For example, if an application
has a field that accepts numbers, the input range can be divided into two
equivalence classes: valid numbers and invalid numbers. This way, instead
of testing each input number, we can test a representative value from each
equivalence class.
The objective of equivalence partitioning is to minimize the number
of test cases needed to test a system while still ensuring that all possible
scenarios are covered. This technique can be used in both functional and
nonfunctional testing.
These steps can be followed while applying equivalence partitioning:
78
Chapter 5 Test Design Techniques
79
Chapter 5 Test Design Techniques
Then they are split into valid and invalid partitions (Table 5-1).
Then they are split into valid and invalid partitions (Table 5-2).
80
Chapter 5 Test Design Techniques
Then they are split into valid and invalid partitions (Table 5-3).
Then they are split into valid and invalid partitions (Table 5-4).
Exercise
Suppose you are testing a registration form for a mobile app. The form
requires users to enter their name, email address, and phone number. The
name field must be between 3 to 20 characters, the email address must be
in a valid email format, and the phone number must be a 10-digit number.
Use equivalence partitioning to identify the valid and invalid input values
for each field.
81
Chapter 5 Test Design Techniques
82
Chapter 5 Test Design Techniques
Age Validation
Suppose you are testing a form that requires users to enter their age. The
system allows users who are 18 years or older to register. The input field for
age has specific requirements. The user’s age must be between 18 to 60. In
boundary value analysis, the focus is on values that are at the boundaries
of this range, which are 18 and 60. Test cases can be designed for these
boundary values to check if the system handles them correctly.
83
Chapter 5 Test Design Techniques
Exercise
Suppose you are testing a calculator application that allows users to
perform addition, subtraction, multiplication, and division operations.
The system allows the input of only two numbers at a time. The input
fields for numbers have specific requirements. The numbers should be
between -99999 and 99999. Use boundary value analysis to identify the
valid and invalid input values for each field.
84
Chapter 5 Test Design Techniques
E-shop Discounts
An e-commerce website sells various products, and it has different
discount offers for each product. The discounts depend on the product
type, product price, and customer’s age.
For electronics, when the product price is less than $500, then the
discount is 5 percent for customers below 18 and 10 percent for customers
who are 18 and over. When the product price is $500 or more, then the
discount is 10 percent for customers who are below 18 and 20 percent for
customers who are 18 and over.
85
Chapter 5 Test Design Techniques
For clothing, when the product price is less than $100, then the
discount is 5 percent for customers below 18 and 10 percent for customers
who are 18 and over. When the product price is $100 or more, then the
discount is 15 percent for customers who are below 18 and 20 percent for
customers who are 18 and over.
In this case, a decision table can be created to identify the
combinations of inputs that result in a specific discount offer, as shown in
Table 5-5.
In this example, decision table testing can be used to test the system’s
ability to correctly apply the discount offers based on the product type,
product price, and customer age.
Transportation Price
A transportation company provides transportation services for customers,
and it charges based on the distance and the number of passengers. The
pricing depends on the type of transportation, which can be either a taxi
or a bus.
86
Chapter 5 Test Design Techniques
A taxi with one or two passengers charges $5, $10, and $20 for a
distance of 0–5, 5–10, and 10+ km, respectively.
A taxi with three or four passengers charges $10, $20, and $40 for a
distance of 0–5, 5–10, and 10+ km, respectively.
A bus with 1–30 passengers charges $50, $100, and $200 for a distance
of 0–10, 10–50, and 50+ km, respectively.
In this case, a decision table (Table 5-6) can be created to identify the
combinations of inputs that result in a specific price.
In this example, decision table testing can be used to test the system’s
ability to correctly apply the pricing based on the type of transportation,
the number of passengers, and the distance.
Exercise
Suppose you are testing a system that provides insurance quotes for
customers based on the type of vehicle, driver’s age, and driving history.
87
Chapter 5 Test Design Techniques
The system applies different rules for different vehicle types, driver
ages, and driving histories.
For vehicles of less than 10 years, if the driver is between 25 and 50
and has no prior accidents, the quote is $50. With prior accidents, the
quote is $75.
If the driver is 18–25 or more than 50 and has no prior accidents, the
quote is $60. With prior accidents, the quote is $85.
For vehicles of 10 years or more, if the driver is between 25 and 50 and
has no prior accidents, the quote is $65. With prior accidents, the quote
is $100.
If the driver is 18–25 or more than 50 and has no prior accidents, the
quote is $75. With prior accidents, the quote is $110.
Use decision table testing to identify the different combinations of
inputs that result in a specific insurance quote.
88
Chapter 5 Test Design Techniques
89
Chapter 5 Test Design Techniques
90
Chapter 5 Test Design Techniques
Shopping Cart
Suppose you are testing an online shopping cart with the option to add or
remove items from your cart, check out, complete your order, or cancel
everything.
In this example, the shopping cart application has four states: Empty
Cart, One Item, Two Items, and Three Items. There is also a special state
called Checkout that represents the process of checking out and paying for
items in the cart.
The state transition table lists all of the possible events that can occur
in each state, along with the resulting next state. For example, if the current
state is Empty Cart and the Add Item event occurs, the next state will be
One Item. If the current state is One Item and the Proceed to Checkout
event occurs, the next state will be Checkout.
By testing all of the possible state transitions and ensuring that
the application behaves correctly in each state, we can ensure that the
shopping cart application works as expected.
91
Chapter 5 Test Design Techniques
92
Chapter 5 Test Design Techniques
Exercise
Suppose you are tasked with testing a vending machine that dispenses
beverages. The machine can be in three states: idle, beverage selected, and
dispensing. The user can perform the following actions:
Create a state transition diagram for this traffic light system, and test
each possible transition to ensure that the system functions as expected.
What happens if you try an invalid transition?
Use-Case Testing
Use-case testing is a software testing technique that evaluates the
software’s ability to meet the end users’ requirements. A use case is a
specific scenario that describes how a user interacts with the software
system. Use-case testing involves creating test cases based on these
scenarios to verify that the system behaves as expected.
To perform use-case testing, the tester first identifies the system’s
different use cases. Then, for each use case, they create test cases that
include inputs, expected outputs, and steps to be taken. The test cases
are designed to evaluate whether the system meets the specific use case
requirements.
The following are the benefits of use-case testing:
• Improved software quality: Use-case testing helps
to ensure that the software meets the end users’
requirements, resulting in higher-quality software.
93
Chapter 5 Test Design Techniques
94
Chapter 5 Test Design Techniques
Exercise
Suppose you are tasked with testing a library management system. The
use cases for the system include searching for books, checking out books,
returning books, and managing user accounts.
Create a list of use cases for this library management system, and test
each one, including both valid and invalid scenarios. You also need to
ensure that the system can handle all possible combinations of use cases.
Pairwise Testing
Pairwise testing, also known as all-pairs testing, is a technique used
in software testing to ensure that all possible combinations of input
parameters are tested at least once. It is a combinatorial testing method
that reduces the number of test cases required to cover all possible
combinations of inputs, while still providing a high level of test coverage.
The technique is based on the principle that faults often occur as a
result of interactions between input parameters, rather than just individual
parameters. By using pairwise testing, it is possible to uncover such
interactions and identify defects that might not be detected by other
testing methods.
Pairwise testing works by creating test cases that cover all possible
pairs of values for each input parameter. For example, if a system has three
input parameters, each with four possible values, there would be a total
of 64 possible combinations (4×4×4). With pairwise testing, however, it is
possible to reduce the number of test cases required to cover all possible
pairs of values to just 24, significantly reducing the time and effort required
for testing.
Pairwise testing can be performed manually, but there are also many
automated tools available that can generate test cases automatically. These
tools use algorithms to identify the most efficient set of test cases that will
cover all possible pairs of input parameters.
95
Chapter 5 Test Design Techniques
To apply pairwise testing, we first create a table that lists all the
possible values for each input parameter, as shown in Table 5-9.
96
Chapter 5 Test Design Techniques
Next, we group the parameters into pairs and generate all possible
combinations, like so:
97
Chapter 5 Test Design Techniques
For each pair, we select test cases that cover all possible combinations
of values. For example, for the pair of First Name and Last Name, we can
select the following test cases:
• Enter valid values for both fields (e.g., Alex and Smith).
• Enter an empty value for one field and a valid value for
the other (e.g., leave First Name blank and enter Smith
for Last Name).
• Enter an invalid value for one field and a valid value for
the other (e.g., enter 123 for First Name and Smith for
Last Name).
We repeat this process for each pair of input parameters, until we have
selected test cases that cover all possible combinations of values.
Pairwise testing can significantly reduce the number of test cases
required to achieve thorough coverage of the input domain, while still
providing good test coverage.
Exercise
Suppose you are testing a reservation system that has four input fields:
date, time, location, and number of guests. Using pairwise testing, select a
subset of test cases that covers all possible combinations of input values.
Error Guessing
Error guessing is a software testing technique that is based on the tester’s
experience, intuition, and creativity to identify potential defects in the
98
Chapter 5 Test Design Techniques
software. It is a black-box testing technique that does not follow any formal
process or test case design. Instead, the tester applies a range of ad hoc
tests based on their intuition and experience with the application and its
potential weaknesses.
The process of error guessing involves identifying different scenarios
and situations in which the software may behave unexpectedly and
then attempting to replicate those scenarios and test the software’s
response. Testers use their experience and knowledge of the software to
anticipate the types of errors that may occur and then design tests to reveal
those errors.
Error guessing can be particularly effective when combined with other
testing techniques such as equivalence partitioning and boundary value
analysis. By using a combination of techniques, testers can identify a wide
range of potential errors and increase the likelihood of uncovering defects
in the software.
An example of error guessing might be a tester who has extensive
experience with a particular software application and knows that it is
particularly prone to crashing when certain input values are entered. The
tester may design a test case that intentionally enters these input values to
see if the software crashes as expected. If the software does not crash, the
tester may try other variations on the input values until they uncover the
defect. This process of guessing at possible errors and testing them is error
guessing.
Error guessing is useful when the tester has limited time and resources
to design test cases and execute them. It is also useful when the system
being tested has no documentation or formal specifications and the tester
needs to rely on their expertise to identify potential errors.
However, it is important to note that error guessing should not be
the only technique used for testing. It should be used in conjunction
with other techniques such as boundary value analysis, equivalence
partitioning, and decision table testing to ensure comprehensive testing of
99
Chapter 5 Test Design Techniques
Registration Form
Suppose you are testing a registration form that has several input fields:
name, email address, password, and confirmation password. Based on
your experience, you might guess that users will enter invalid characters
or leave required fields blank. You can design test cases that try to enter
unexpected data such as special characters in the name field, an invalid
email address, or a password that is too short.
E-commerce Website
Suppose you are testing an e-commerce website that has a checkout
process with several pages. Based on your experience, you might guess
that users will have trouble entering shipping information, payment
information, or completing the checkout process. You can design test cases
that intentionally try to trigger errors, such as trying to submit an order
with an invalid credit card number.
Exercise
Suppose you are testing a messaging app that allows users to send and
receive messages. Based on your experience, what are some potential
errors or defects that users may encounter? Design test cases that
intentionally try to trigger those errors.
Exploratory Testing
Exploratory testing is a technique of software testing that emphasizes
the freedom and creativity of testers in the software testing process. It is
an approach to testing where the tester actively and dynamically learns
100
Chapter 5 Test Design Techniques
about the software while testing it. In other words, the tester designs and
executes tests while exploring the software application in an unstructured
and informal way.
Exploratory testing is often used in Agile and DevOps environments
where the requirements are rapidly changing, and the software needs to
be tested quickly and efficiently. It allows testers to adapt quickly to the
changes and continuously refine their testing approach based on their
understanding of the system.
The process of exploratory testing involves the following steps:
101
Chapter 5 Test Design Techniques
Mobile App
Suppose you are testing a mobile app that allows users to take photos and
share them with their friends. You might start by exploring the app’s user
interface and testing the different features, such as taking a photo, adding
a filter, and sharing it with friends. As you explore the app, you might try
different scenarios, such as taking a photo in low light conditions, using a
different filter, or sharing the photo with a friend who has a slow Internet
connection.
Online Marketplace
Suppose you are testing an online marketplace that allows users to buy
and sell products. You might start by exploring the website’s search
functionality and testing how well it works for different types of products.
As you explore the website, you might try different scenarios, such as
searching for a product that has multiple variations, testing the checkout
process, or using the website from a mobile device.
102
Chapter 5 Test Design Techniques
Exercise
Suppose you are testing a social media app that allows users to post and
share content with their friends. Design and execute an exploratory testing
session, where you explore the app and test different scenarios. Try to
identify potential defects or issues and document them.
Random Testing
In software testing, there are various techniques and methodologies to
design and execute test cases. One such technique is random testing. As
the name suggests, it involves selecting random input values from the
input domain of the system under test (SUT) and feeding them as input
to the system. This approach can uncover faults that may not be detected
by other testing techniques. In this chapter, we will discuss the basics of
random testing, its advantages, its limitations, and the process involved.
Random testing is a black-box testing technique, which means that the
tester does not have knowledge about the internal workings of the software
application. The testing technique involves selecting random input values
from the input domain of the software application. The input values are
selected randomly using a random number generator. The input values
can be any combination of characters, numbers, special symbols, or any
other data type that the software application accepts as input.
These are some advantages of random testing:
103
Chapter 5 Test Design Techniques
104
Chapter 5 Test Design Techniques
Web Application
Suppose you are testing a web application that allows users to register and
log in. You can use random testing to create random strings of characters
to use as input for the username and password fields. By doing so, you
can test the application’s ability to handle different types of input, such as
special characters and long strings of text.
Game Testing
Suppose you are testing a new video game that has just been developed.
You can use random testing to test the game’s ability to handle unexpected
input from the player. For example, you can use random movements of
the controller or keyboard to simulate different scenarios that may occur
during gameplay, such as sudden changes in direction or button mashing.
Exercise
Suppose you are testing a calculator application. How would you use
random testing to test the application’s functionality?
105
Chapter 5 Test Design Techniques
106
Chapter 5 Test Design Techniques
Exercise
Consider a social media application that allows users to post and view
photos. Perform ad hoc testing on the application to identify defects that
might not be found by scripted testing. Document the defects found during
testing and report them to the development team. Review the ad hoc testing
with other testers to ensure that all areas of the system have been covered.
White-Box Testing
White-box testing is a software testing technique that is used to test the
internal workings of an application or system. In white-box testing, the
tester has knowledge of the internal structure and implementation of the
software being tested. Here are some white-box testing techniques:
107
Chapter 5 Test Design Techniques
108
Chapter 5 Test Design Techniques
Statement Coverage
Statement coverage is a type of white-box testing technique that is used to
determine how many statements in the source code of a software system
have been executed during the testing process. The goal of statement
coverage is to ensure that each statement in the code has been executed at
least once during the testing process.
The statement coverage technique involves executing the test cases
that have been designed for a software system and tracking the number
of statements that are executed during the testing process. A statement
is considered to be executed if it is encountered by the program during
the execution of a test case. Once all test cases have been executed, the
percentage of statements that have been executed is calculated. The goal
of statement coverage is to achieve 100 percent statement coverage, which
means that every statement in the code has been executed at least once.
Statement coverage is a useful technique for finding defects in software
systems. By ensuring that every statement in the code has been executed at
least once, statement coverage can help to uncover defects that might not
have been found through other testing techniques. Additionally, statement
coverage can help to ensure that the software system is functioning as
intended and that all of the code is working as expected.
However, statement coverage has some limitations. For example,
achieving 100 percent statement coverage does not guarantee that the
software system is completely free of defects. It is possible to have a
software system with 100 percent statement coverage that still contains
defects. Additionally, statement coverage can be a time-consuming
process, especially for large software systems.
109
Chapter 5 Test Design Techniques
To test the statement coverage of this code, we need to write test cases
that execute each statement at least once.
Table 5-10 shows the minimum set of test cases to achieve 100 percent
statement coverage.
1 3 4 7
2 0 2 0
110
Chapter 5 Test Design Techniques
1 empty None
2 1,2,3 2
111
Chapter 5 Test Design Techniques
E xercise
Consider the following Python function that takes a list of integers as input
and returns a new list with all the even numbers in the input list:
Write two test cases that achieve statement coverage for this function.
Make sure to include at least one test case where the input list contains no
even numbers.
B
ranch Coverage
Branch coverage is a white-box testing technique that evaluates whether
all the branches in the program source code are executed at least once. In
other words, it ensures that all the possible paths in the source code are
executed at least once. This technique is used to measure the effectiveness
of the testing and quality of the code. It helps to ensure that the program’s
control flow is working as expected.
The primary objective of branch coverage testing is to determine
whether each decision point in the code has been tested. The decision
point is a code statement that determines which of two paths to take.
Branch coverage is measured by the percentage of the total number of
decision points executed during the testing process.
112
Chapter 5 Test Design Techniques
The branch coverage testing process involves following the code paths
to determine whether each possible branch is executed at least once.
This technique requires a thorough understanding of the code structure
to identify all the decision points. Once the decision points have been
identified, the tester creates a test case that exercises each path.
For example, consider a code block with an if-else statement. The
if-else statement has two branches: one that is executed if the condition
is true and the other if the condition is false. To test branch coverage, we
must create test cases that execute both the true and false branches at
least once.
Here are some benefits of branch coverage testing:
• It helps to identify the potential faults in the code by
testing all possible paths.
• It ensures that the program’s control flow is working as
expected, reducing the risk of runtime errors.
• It provides metrics to measure the testing effectiveness
and code quality.
• It provides confidence in the code quality, helping to
reduce the cost of fixing defects.
113
Chapter 5 Test Design Techniques
To test the branch coverage of this code, we need to write test cases
that execute each branch at least once.
Table 5-12 shows the minimum set of test cases to achieve 100 percent
branch coverage.
1 3 4 7
2 0 2 0
1 3 4 4
2 3 2 3
E xercise
Consider the following Python function that takes an integer as input and
returns a string indicating whether the input number is positive, negative,
or zero:
Write the test cases that achieve branch coverage for this function.
P
ath Coverage
Path coverage is a white-box testing technique that involves testing all
possible paths of a program. A path is a sequence of statements that begins
with the entry point and ends with the exit point of the program. Path
coverage aims to test each unique path at least once to ensure that the
program is functioning as expected.
To achieve path coverage, a test case must execute every statement in
the program at least once. It also requires that all possible branches are
executed. A branch is a point in the program where a decision is made,
such as an if statement or a loop condition. Each branch has at least
two possible outcomes, and all possible outcomes must be executed at
least once.
115
Chapter 5 Test Design Techniques
To achieve 100 percent path coverage of this code, we need to test all
possible paths through the code. Table 5-14 shows the minimum set of test
cases to achieve 100 percent path coverage.
1 3 4 7
2 0 2 0
In this example, we have tested every possible path through the code,
ensuring that all possible combinations of control flow paths are executed
at least once. However, even achieving path coverage may not be sufficient
116
Chapter 5 Test Design Techniques
117
Chapter 5 Test Design Techniques
1 1 2 3 5
2 2 1 3 5
3 3 2 1 5
E xercise
Consider the following Python function that takes two lists of integers as
input and returns a new list containing the elements that are common to
both input lists:
Write the test cases that achieve path coverage for this function.
C
ondition Coverage
Condition coverage is a type of white-box testing technique that aims to
ensure that all possible combinations of Boolean conditions in a program
have been executed at least once. The technique focuses on evaluating
the different ways in which a program’s conditions can be evaluated as
true or false. Condition coverage can be applied to both procedural and
118
Chapter 5 Test Design Techniques
119
Chapter 5 Test Design Techniques
120
Chapter 5 Test Design Techniques
Exercise
How would you improve the set of tests in the previous example?
Decision Coverage
Decision coverage is a white-box testing technique that is used to measure
the effectiveness of testing by checking if all decisions made in the source
code have been exercised. A decision is a condition that can lead to either
true or false outcome. Decision coverage measures the percentage of
decisions that have been executed during testing.
The goal of decision coverage is to ensure that each decision in the
code has been executed at least once and that all possible outcomes (true
and false) have been tested. This helps to ensure that the code is free of
errors and will function as expected under all possible scenarios.
To achieve decision coverage, testers must create test cases that
execute every decision point in the code. This means that every possible
outcome of each decision point must be tested. For example, if a decision
point is a simple if-else statement, then there are two possible outcomes
that must be tested.
Decision coverage is important because it helps to ensure that all
possible outcomes of the code have been tested. This is particularly
important in safety-critical systems where a failure in the code could have
catastrophic consequences.
121
Chapter 5 Test Design Techniques
3. If a == b and b == c is true
4. If a == b and b == c is false
122
Chapter 5 Test Design Techniques
5. If a == b or b == c or c == a is true
6. If a == b or b == c or c == a is false
Exercise
How would you improve the set of tests in the previous example?
123
Chapter 5 Test Design Techniques
T T T Positive
T T F Mixed
T F T Mixed
T F F Mixed
F T T Mixed
F T F Mixed
F F T Mixed
F F F Negative
124
Chapter 5 Test Design Techniques
125
Chapter 5 Test Design Techniques
• weight <= 10
• size <= 20
• size <= 50
5 10 “USA” 10
5 30 “USA” 20
15 70 “Canada” 40
-5 10 “Mexico” “Invalid weight”
5 -10 “USA” “Invalid size”
5 10 “Japan” “Invalid destination”
Exercise
Consider the following function that determines the letter grade for a
student based on two exam scores:
126
Chapter 5 Test Design Techniques
Write the test cases required to achieve 100 percent MCC coverage.
127
Chapter 5 Test Design Techniques
128
Chapter 5 Test Design Techniques
1 T T T true
2 T T F true
3 T F T true
4 T F F false
5 F T T true
6 F T F false
7 F F T true
8 F F F false
Now we need to find the test cases where changing a condition’s input
affects the whole decision. We can see that for the condition a > 0 to be the
decisive condition, we need to use test cases 2 and 6. For the condition b
> 0 to be the decisive condition, we need to use test cases 2 and 4. For the
condition c > 0 to be the decisive condition, we need to use test cases 3 and
4 (5 and 6 would also be correct). This leads to the set of test cases shown
in Table 5-19.
1 T T F true
2 T F T true
3 T F F false
4 F T F false
129
Chapter 5 Test Design Techniques
Loop Testing
Loop testing is a technique used in software testing to ensure that loops
within a program are functioning correctly. This type of testing is used
to identify defects or errors that can occur when a program is executed
multiple times, as is often the case with loops.
The goal of loop testing is to ensure that the loop is executed the
correct number of times, that the loop is exited when it is supposed to be
exited, and that the loop is executed with the correct inputs and outputs.
This is accomplished by testing the loop under a variety of conditions,
including boundary conditions and error conditions.
There are several different types of loop testing techniques that can be
used, including the following:
130
Chapter 5 Test Design Techniques
This function takes an array of numbers and returns the sum of all the
numbers in the array. The for loop iterates over each element in the array
and adds it to the sum variable.
131
Chapter 5 Test Design Techniques
To test this function using loop testing, we would want to design our
test cases to exercise the loop as thoroughly as possible. The following are
some examples of test cases we might consider:
132
Chapter 5 Test Design Techniques
Static Testing
Static testing is a software testing technique that involves examining the
code, design, and associated documentation of an application without
actually executing it. The purpose of static testing is to identify defects
early in the development life cycle to reduce the cost of fixing them
later on.
133
Chapter 5 Test Design Techniques
134
Chapter 5 Test Design Techniques
Summary
In this chapter, we went through several examples of test design
techniques. We saw how to create test cases by using these powerful
techniques with real examples. Some techniques are more elaborate than
other; for example, decision table testing makes sure that all the possible
combinations are performed, and some techniques are using several tricks
to eliminate test cases when the time or the budget are not enough for
testing everything. For example, pairwise testing will significantly reduce
the number of test cases, while covering the most important combinations.
There are structured techniques, such as boundary value analysis, and free
format techniques, such as exploratory testing. They are all useful, and
the best practice is to use them in combination. When all the test cases are
defined, it is time to execute them!
135
CHAPTER 6
Test Execution
Test execution is the process of running the tests that have been designed
and planned in the previous stages of the software testing life cycle. This
chapter will cover the importance of test execution, the test execution
process, and the various techniques and tools used in test execution. We
will go through the test execution process, the needs of the environment
setup, and the defect reporting and retesting. We will identify the needs
for regression, how to report the status of the execution, and when to
complete the activities. We will see some techniques and tools that we can
assist us, and we will choose some quality metrics for our product.
G
etting Started
The first step in preparing for test execution is to ensure that the testing
environment is set up correctly. This includes setting up test beds,
configuring hardware and software, and installing the necessary tools and
utilities. It is also important to ensure that the test environment is as close
to the production environment as possible to ensure that the testing results
are accurate.
Another important aspect of test execution preparation is the creation
of test cases. Test cases should be created based on the requirements
and design documents and should cover all possible scenarios that the
software could encounter. The test cases should be reviewed and approved
by the stakeholders before they are executed.
138
Chapter 6 Test Execution
139
Chapter 6 Test Execution
140
Chapter 6 Test Execution
141
Chapter 6 Test Execution
142
Chapter 6 Test Execution
143
Chapter 6 Test Execution
Regression Testing
Regression testing is a type of software testing that is performed to ensure
that changes made to an application or system have not introduced new
defects or caused existing features to fail. It is an important part of the
testing process, as it helps to maintain the quality of the software and
prevent issues from arising in production.
The primary goal of regression testing is to verify that the software still
works as intended after changes have been made. This involves re-testing
all or a subset of the existing test cases that have been previously executed.
The test cases may cover functional, performance, security, or other
aspects of the software.
Regression testing can be executed at different stages of the software
development life cycle, depending on the type of change made. For
example, if a minor bug is fixed, then only a limited number of test cases
may need to be re-executed. However, if major changes are made to
the software, then a more extensive regression test suite may need to be
developed and executed.
144
Chapter 6 Test Execution
145
Chapter 6 Test Execution
146
Chapter 6 Test Execution
Overall, test case status reporting should be clear, concise, and easy to
understand so that stakeholders can make informed decisions about the
software being tested. It should be tailored to the specific needs of each
project and should be updated regularly to reflect the most up-to-date
information.
147
Chapter 6 Test Execution
148
Chapter 6 Test Execution
By following these steps, testers can ensure that all identified test cases
have been executed and analyzed and the testing process is complete.
This information can be used to make decisions on the readiness of the
software for release or further testing.
149
Chapter 6 Test Execution
Quality Metrics
Quality metrics are used to measure and assess the quality of software
products and the software development process. There are various quality
metrics that can be used either to monitor the quality of the product or to
improve the development and testing processes.
Defect Density
Defect density is a quality metric that measures the number of defects found
in a software product per unit of code or function point. It is calculated by
dividing the total number of defects by the size of the software product,
expressed in lines of code, function points, or other relevant measures.
150
Chapter 6 Test Execution
Test Coverage
Test coverage is a quality metric that measures the percentage of code
or functionality that has been tested by the software testing process. It
is calculated by dividing the number of lines of code or functions that
have been tested by the total number of lines of code or functions in the
software product.
For example, if a software product has 10,000 lines of code and the
testing process has covered 8,000 lines of code, the test coverage would be
80 percent.
Test coverage is an important metric because it helps to assess the
effectiveness of the testing process and identify areas of the software
product that require additional testing. A high test coverage suggests
that most or all of the code or functionality has been tested, reducing the
risk of defects going undetected. A low test coverage, on the other hand,
indicates that there may be significant areas of the software product that
have not been adequately tested, increasing the risk of defects going
undetected.
151
Chapter 6 Test Execution
Code Complexity
Code complexity is a quality metric that measures the complexity of
software code. It is often assessed using a software tool that calculates
a complexity score based on factors such as the number of control
structures, conditional statements, and loops in the code.
One commonly used metric for code complexity is cyclomatic
complexity, which measures the number of independent paths through
the code. The higher the cyclomatic complexity score, the more complex
the code is likely to be.
Code complexity is an important metric because complex code can be
difficult to maintain, modify, and test. High levels of code complexity can
also increase the risk of defects, as it can be more challenging to identify
and fix issues in complex code.
By identifying areas of code that are overly complex, developers and
testers can focus on reducing complexity to improve code quality and
reduce the risk of defects. Code complexity can also be used as a factor
in determining code review and testing priorities, with higher complexity
code given greater scrutiny.
Overall, code complexity is an important factor in software quality and
should be measured and monitored as part of a comprehensive quality
assurance strategy.
Code Maintainability
Code maintainability is a quality metric that measures the ease with
which code can be modified, updated, and maintained over time. It is
often assessed using software tools that analyze code for factors such as
readability, clarity, and modularity.
Code maintainability is an important metric because software code is
rarely static and often requires updates and modifications over time. Code
that is difficult to maintain can result in higher development costs, longer
development cycles, and increased risk of defects.
152
Chapter 6 Test Execution
153
Chapter 6 Test Execution
Coding Standards
Coding standards violations are quality metrics that measure the
adherence of software code to a set of predefined coding standards or
best practices. These coding standards can cover a wide range of factors,
including code formatting, variable naming conventions, commenting
style, and more.
Coding standards violations can be detected using automated tools
that scan code for violations of the predefined standards. These tools can
be integrated into the software development process, allowing developers
to identify and correct coding standards violations in real time.
154
Chapter 6 Test Execution
Code Duplication
Code duplication is a quality metric that measures the amount of duplicate
or redundant code in a software project. Code duplication occurs when
similar or identical code is repeated in multiple places within the codebase.
Code duplication can be measured using automated tools that
analyze the codebase and identify sections of code that are repeated or
very similar. These tools can also calculate the percentage of code that is
duplicated, as well as the locations of the duplicated code.
Code duplication is an important metric because it can lead to a
number of negative outcomes, such as the following:
155
Chapter 6 Test Execution
Dead Code
Dead code is a quality metric that measures the amount of code in a
software project that is no longer used or executed. Dead code can occur
due to changes in requirements, due to changes in design, or simply as a
result of refactoring or other code changes.
Dead code can be measured using automated tools that analyze the
codebase and identify sections of code that are not being executed. These
tools can also calculate the percentage of code that is dead, as well as the
locations of the dead code.
Dead code is an important metric because it can lead to a number of
negative outcomes, such as the following:
156
Chapter 6 Test Execution
Lines of Code
The lines of code (LOC) metric is a software quality metric that measures
the size or complexity of a software program by counting the number of
lines of code in the source code files. It is a simple metric that is easy to
measure and understand, but it has limitations and is often criticized as a
measure of software quality.
The LOC metric is calculated by counting the number of lines of code
in a source code file, including blank lines and comments. The resulting
number represents the size or complexity of the software program.
While the LOC metric can provide an indication of the size and
complexity of a software program, it has several limitations.
157
Chapter 6 Test Execution
Fan-Out
Fan-out is a software metric used to measure the number of dependencies
that a module or function has on other modules or functions. Specifically,
it measures the number of other modules or functions that are called or
invoked by a given module or function. The more dependencies a module
or function has, the higher its fan-out metric.
Fan-out is important because modules or functions with a high fan-out
metric can be more difficult to understand, test, and maintain. They can
also increase the risk of defects and other quality issues.
To calculate the fan-out metric, one can count the number of external
modules or functions called or invoked by a given module or function.
Alternatively, tools such as static code analysis tools can automatically
calculate fan-out as part of a comprehensive assessment of software
quality.
By measuring fan-out, developers and testers can identify modules
or functions that may require refactoring or optimization to reduce their
dependencies on other modules or functions. This can help to improve the
maintainability and quality of the software product over time.
Overall, fan-out is an important metric to consider when assessing
software quality, as it can provide valuable insights into the complexity and
maintainability of the codebase.
158
Chapter 6 Test Execution
Compiler Warnings
Compiler warnings are a type of software metric that measures the number
of warnings generated by a compiler during the compilation process of a
software program. Compiler warnings are generated when the compiler
detects potential issues in the code, such as unused variables, uninitialized
variables, or other potential bugs or defects.
The compiler warnings metric is important because it can provide an
indication of the quality and maintainability of the code. A high number
of compiler warnings can indicate that the codebase has a high potential
for defects or other quality issues and may require further attention from
developers and testers.
To calculate the compiler warnings metric, one can simply count the
number of warnings generated by the compiler during the compilation
process. Alternatively, tools such as static code analysis tools can
automatically identify and report compiler warnings as part of a
comprehensive assessment of software quality.
By measuring compiler warnings, developers and testers can identify
areas of the codebase that may require further attention or refactoring
to improve software quality. Over time, this can help to reduce the risk
of defects and improve the overall reliability and maintainability of the
software product.
Summary
In this chapter, we went through the main steps of the test execution
process. We saw the needs for the environment setup and how to
report and retest the defects. We saw the report we can generate at any
moment during the execution but mainly at the completion stage. Several
techniques and tools were also used for the execution, and we explored a
variety of quality metrics for our product. Regression testing is a big part of
test execution, so the best approach for it is to automate it.
159
CHAPTER 7
Test Automation
Test automation is the use of specialized software tools to control the
execution of tests and compare the actual results with the expected results.
The use of automation has become increasingly popular as it can help
reduce the time and cost associated with testing while also improving the
accuracy and reliability of the test results.
In this chapter, I’ll cover the benefits of test automation, along with
what are generally considered to be the best test automation tools and
frameworks.
However, test automation is not without its challenges. The initial cost of
automation can be high, and maintenance can be time-consuming. Therefore,
before implementing automation, it is important to consider the following:
162
Chapter 7 Test Automation
The basic process of using a record and playback tool involves starting
the recording, performing a series of user actions on the application
or system being tested, and then stopping the recording. The tool then
generates a script based on the recorded actions, which can be modified
and enhanced as needed.
Record and playback tools are often used for regression testing, which
involves retesting software after changes have been made to ensure that
existing functionality has not been affected. They can also be used for
smoke testing, which involves running a basic set of tests to quickly verify
that the application is working correctly after a new build or release.
However, record and playback tools have limitations. They can
generate scripts based only on the actions that were recorded, so they may
not be able to handle more complex scenarios or variations in user input.
In addition, any changes to the application or system under test can cause
the recorded scripts to fail, requiring updates to the scripts.
Scripting Tools
Scripting tools are a type of test automation tool that enable testers to
write scripts or code to automate testing tasks. These tools provide a
programming interface or scripting language that allows testers to create
custom test cases and automate complex scenarios that may not be
possible with record and playback tools.
Scripting tools typically require more technical expertise and
programming knowledge than record and playback tools. Testers need
to write the test scripts themselves, which can involve writing code in
languages such as Python, Java, or Ruby, depending on the tool.
The main advantage of scripting tools is their flexibility and
extensibility. Testers can create custom scripts to automate specific test
scenarios and perform more complex testing tasks, such as database
163
Chapter 7 Test Automation
Hybrid Tools
Hybrid tools are a type of test automation tool that combine the benefits of
both record and playback and scripting tools. They allow testers to create
automated test scripts using a combination of record and playback and
script-based techniques.
Hybrid tools typically offer a visual interface for recording user
interactions, which can then be edited and enhanced using a scripting
language or programming interface. This allows testers to easily create
basic test scripts using record and playback functionality, while also
providing the flexibility to add custom code or scripts as needed.
The main advantage of hybrid tools is their versatility and ease of use.
They provide a user-friendly interface for recording and editing test scripts,
while also offering more advanced features for customizing and enhancing
scripts as needed. This makes them suitable for a wide range of testing
scenarios, from simple regression testing to more complex performance
testing or load testing.
164
Chapter 7 Test Automation
Frameworks
Frameworks are a collection of guidelines, standards, and tools that
provide a structured approach to designing, developing, and executing
automated tests. Test automation frameworks aim to standardize the
testing process and ensure consistency in the way tests are created and
executed.
A test automation framework typically consists of a set of components,
such as libraries, APIs, and utilities, that provide the building blocks for
creating automated tests. These components may include functions
for interacting with the application or system under test, test data
management, reporting and logging, and error handling.
There are several types of test automation frameworks, including the
following:
165
Chapter 7 Test Automation
166
Chapter 7 Test Automation
167
Chapter 7 Test Automation
168
Chapter 7 Test Automation
These are just a few examples of the many automated testing tools
available. When selecting an automated testing tool, it’s important to
consider the requirements of the project, the technology used, and the
expertise of the testing team. Selecting the right automation tool can
significantly improve the efficiency and effectiveness of the testing process.
It is essential to choose a tool that meets the specific needs of the project
and the organization.
169
Chapter 7 Test Automation
170
Chapter 7 Test Automation
Summary
This chapter was an overview of the tools we can use for test automation,
the benefits of having automation, and the drawbacks of it. As I often enjoy
saying, test automation will only report defects you already anticipate.
It should be used together with manual testing, always having the needs
and the priorities of the product in mind. Test automation is a continuous
process that requires a lot of effort into the setup of the framework and the
creation of the tests. This effort is distributed among the sprints when we
are working in an Agile environment.
171
CHAPTER 8
Testing in Agile
Environment
Agile development is an iterative and incremental approach to software
development that emphasizes flexibility, collaboration, and continuous
improvement. Testing is an integral part of the Agile development process,
and testers work closely with developers and other stakeholders to ensure
that the software meets the needs of the business and end users.
This chapter covers the following topics related to testing in Agile
development:
• Agile testing principles
174
Chapter 8 Testing in Agile Environment
175
Chapter 8 Testing in Agile Environment
Business Facing
Automated &
Manual
Manual
Critique Product
Unit Tests Performance Tests
Integration Tests Security Tests
System Tests
Automated Tools
Technology Facing
176
Chapter 8 Testing in Agile Environment
Agile teams use the quadrants to plan their testing activities and
ensure that they have a balanced approach to testing. By covering all
four quadrants, the team can ensure that they are testing the system from
different perspectives and reducing the risk of defects slipping through
the cracks.
Test-Driven Development
Test-driven development (TDD) is a software development approach
where test cases are written before the code is developed. The idea is to
create automated test cases for a small piece of functionality and then
write just enough code to make those tests pass. Once the code passes the
test cases, it is refactored to ensure it is clean and maintainable.
The main objective of TDD is to produce clean code that works as
intended and to help ensure that code changes don’t introduce new bugs
or break existing functionality. TDD also encourages developers to write
code in smaller increments, which can help to catch problems earlier in
the development process.
177
Chapter 8 Testing in Agile Environment
2. Run the test case: Run the test case and ensure that it
fails, since the functionality is not yet implemented.
4. Run the test case: Run the test case again and ensure
that it passes.
178
Chapter 8 Testing in Agile Environment
However, TDD also has some challenges, such as the need for a
dedicated testing framework and the potential for tests to become overly
complex. Additionally, it can be difficult to apply TDD to legacy code or to
projects with tight deadlines.
Behavior-Driven Development
Behavior-driven development is an Agile software development technique
that promotes collaboration between developers, testers, and business
stakeholders. BDD is based on the principles of test-driven development
and builds on it by incorporating aspects of domain-driven design (DDD)
and object-oriented analysis and design (OOAD).
BDD emphasizes the behavior of a system or feature, rather than its
implementation. It encourages the use of natural language in writing tests
and specifications, which makes it easier for nontechnical stakeholders to
understand the purpose of the feature being developed.
In BDD, the requirements are expressed in the form of scenarios,
which describe the behavior of the system from the perspective of a user
or stakeholder. These scenarios are written in a specific format called
the given-when-then (GWT) format, which makes them easy to read and
understand.
BDD involves the use of automated tests, such as acceptance tests and
integration tests, to ensure that the system or feature behaves as expected.
BDD also includes the use of automated tools, such as Cucumber, JBehave,
179
Chapter 8 Testing in Agile Environment
180
Chapter 8 Testing in Agile Environment
181
Chapter 8 Testing in Agile Environment
182
Chapter 8 Testing in Agile Environment
183
Chapter 8 Testing in Agile Environment
However, CI/CD also has some challenges, such as the need for a
robust testing infrastructure, the need for good version control practices,
and the potential for complex deployments. Additionally, it can be difficult
to apply CI/CD to legacy systems or to projects with tight deadlines.
184
Chapter 8 Testing in Agile Environment
185
Chapter 8 Testing in Agile Environment
186
Chapter 8 Testing in Agile Environment
Summary
The current trend in the industry is to work in an Agile environment,
typically Scrum or Kanban, which makes the delivery of the products
faster, and then we continuously build on the product. This of course has
led to changes in the testing side, because testing has to start sooner and
different testing principles are applied to the different testing quadrants.
BDD and TDD are increasingly popular, CI/CD is a must have, and testing
automation is required in almost every product. Although software
testing has been already established for several decades, there are some
challenges that we must face on every new or existing project.
187
CHAPTER 9
Challenges and
Solutions in
Software Testing
Software testing is an essential part of software development that helps
to ensure the quality and reliability of software products. However, the
software testing process is often accompanied by various challenges
that can affect the effectiveness and efficiency of the testing process. In
this chapter, we will discuss some of the common challenges in software
testing and provide some solutions to overcome these challenges.
190
Chapter 9 Challenges and Solutions in Software Testing
Time Constraints
A time constraint is a common challenge in software testing, as testing
often has to be completed within a limited time frame. This can be due
to project deadlines, budget constraints, or other factors. In this section,
we will discuss the impact of time constraints on software testing and
strategies for managing time constraints in testing.
192
Chapter 9 Challenges and Solutions in Software Testing
193
Chapter 9 Challenges and Solutions in Software Testing
194
Chapter 9 Challenges and Solutions in Software Testing
Automation Challenges
Automation can bring many benefits to organizations, including
increased efficiency, accuracy, and cost savings. However, there are also
several challenges that come with automation. In this section, we will
discuss some of the challenges you might face in automation and how to
address them.
195
Chapter 9 Challenges and Solutions in Software Testing
196
Chapter 9 Challenges and Solutions in Software Testing
Change Management
Change management is an important aspect of software testing, as
changes to software can impact its functionality, performance, and
security. However, there are several challenges that testers may face when
managing changes in a software testing environment. In this section, we
will discuss some of the common change management challenges in
testing and how to overcome them.
197
Chapter 9 Challenges and Solutions in Software Testing
198
Chapter 9 Challenges and Solutions in Software Testing
199
Chapter 9 Challenges and Solutions in Software Testing
200
Chapter 9 Challenges and Solutions in Software Testing
201
Chapter 9 Challenges and Solutions in Software Testing
Risk-Based Testing
Risk-based testing is an approach to software testing that prioritizes testing
efforts based on the level of risk associated with the software application
or system being tested. This approach involves identifying potential risks
to the system and prioritizing testing efforts on the areas that pose the
greatest risk to the project, rather than testing everything equally.
It is common that during the development of a project there is not
enough time or resources to mitigate all the potential risks. By applying
risk-based testing, the tester can ensure that the most critical foreseeable
risks are mitigated and the product can go live with minimum risk.
202
Chapter 9 Challenges and Solutions in Software Testing
The risks with a high probability and high impact must be tested
first, and they must be mitigated either by eliminating the impact or by
minimizing their probability.
Next, the risks with a low probability and high impact should be tested
to make sure that the probability is indeed low, and then the team should
have a mitigation plan about the impact.
After that, if there is still time, the risks with a high probability and a
low impact could be tested to verify that the impact is indeed low, and then
the team could reduce the probability of the risk.
Finally, the risks with low probability and low impact, always
depending on the time left and on the business, can be ignored, since they
are not likely to happen, and even if they happen, the impact might be
minimal.
203
Chapter 9 Challenges and Solutions in Software Testing
Summary
There are several challenges in the software world, and testing is no
exception. Although every challenge is manageable, we should be
proactive and mitigate all the critical risks in time; otherwise, it is certain
that we will have to pay a higher price later. A structured way to do this is to
create a risk assessment session at the start of every project and revisit risk
every now and then to keep track of all the risks and their impacts.
204
A
fterword
In conclusion, software testing is an essential process that ensures software
products are of high quality and meet the requirements of the end users.
It involves different activities such as test planning, test design, test
execution, and test reporting, among others.
There are different types of software testing, including functional
testing, performance testing, security testing, and user acceptance testing,
among others. Each type of testing is critical and helps to identify defects
or issues that could impact the quality of the software product.
In recent years, agile development has become a popular approach
to software development, and testing is an essential component of this
approach. Agile development allows teams to work collaboratively,
respond to changes quickly, and deliver high-quality software products.
Continuous testing is also becoming increasingly important, especially
with the rise of DevOps and the need to deliver software products
faster. Continuous testing involves integrating testing into the software
development process, which helps to identify issues earlier and reduce the
overall cost of fixing defects.
In the future, software testing is expected to become more automated,
with a focus on artificial intelligence and machine learning. This will help
to improve the accuracy and efficiency of testing, reduce the time taken to
perform testing, and improve the overall quality of software products.
Automated testing is an emerging trend in software testing that can
help to speed up the testing process and improve the overall efficiency
of the testing process. However, automation alone cannot guarantee a
successful software release, and it is necessary to combine automated
testing with manual testing to achieve the best results.
The main message I want you to take from this book is that these are
guidelines and best practices and in no way are they absolute truths about
software testing. There are multiple ways to perform software testing, and
the decisions you will make during a project will be different every time.
The tools and the guidelines are there to help you and assist you in this
journey, but at the end of the day, the human factor, your intuition, will be
the most important asset you bring to the table. Quality assurance starts
and ends with every person in an organization.
There is always room for improvement and learning. Staying up-to-
date with the latest technologies, tools, and best practices can help you
become a better software tester and contribute to the development of
high-quality software. Remember that testing is a collaborative effort
that involves communication, teamwork, and a commitment to quality.
So, keep learning and exploring the exciting and ever-changing world of
software testing!
206
Index
A BVA, 82
definition, 75
Acceptance test-driven
equivalence partitioning, 78
development (ATDD), 181
error guessing, 98–100
Acceptance testing, 46, 185
exploratory, 100–103
Ad hoc testing, 105, 107
limitations, 76
Agile development
pairwise, 95–98
ATDD, 181, 182
random, 103–105
BDD, 179, 180, 187
techniques, 76, 77
CI/CD, 183
use-case, 93–95
quadrants, 175, 176
Blockchain technology, 201
TDD, 177–179, 187
Bottom-up integration testing, 10
test automation, 184, 185
Boundary value analysis (BVA)
testing principles, 174, 175
age validation, 83
All-pairs testing, 95
definition, 82
Authentication testing, 18
exercise, 84
Authorization testing, 18
file size validation, 84
Automated teller machines
values, 82, 83
(ATMs), 89
Branch coverage, 112–115
Automated testing, 60
Bug report, 142
Availability testing, 13, 24
Business analysts, 4
B
Behavior-driven development
C
(BDD), 169, 179 Change management, testing,
Black-box testing, 2, 60 197, 198
ad hoc, 105–107 Code review, 18
advantages, 76 Compatibility testing, 13, 21
208
INDEX
M P
Maintainability testing, 13 Pairwise testing, 75, 95
Maintenance testing, 3 Path coverage, 115
Manual testing, 60 Penetration testing/pen-testing, 17
Mean time between failures Performance testing, 13–17, 46
(MTBF), 24 Political, Economic, Social,
Mean time to Technological,
repair (MTTR), 24 Environmental, and Legal
Modified condition decision (PESTEL), 72
coverage (MC/DC), 127
Multiple condition coverage
(MCC), 123
Q
Quality metrics
code complexity, 152
N code duplication, 155, 156
Nested loop testing, 131 code maintainability, 152
Nonexecution testing/verification coding standards, 154
testing, 29 compiler warnings, 159
Nonfunctional testing dead code, 156
availability, 24 defect density, 150
compatibility, 21 fan-out, 158
compliance, 27, 28 FST, 153
installation, 24, 25 LOC, 157
maintainability, 26 software development
performance, 14–17 process, 150
reliability, 23 test coverage, 151
scalability, 22
security, 17, 19, 20
R
usability, 20
Random testing, 103–105
Recovery time objective (RTO), 24
O Regression testing, 46, 60, 144,
Object-oriented analysis and 159, 185
design (OOAD), 179 Reliability testing, 13, 23
209
INDEX
210
INDEX
211