Unit 4 Software Testing

• Introduction of Software Testing

• S/W Testing Fundamentals, Unit testing, Integration testing, System testing, Black box and White
box testing and Incremental testing
• Formal proof of correctness, Software matrix
• Automated Testing, Software testing with automated tools

Software Testing: Brief Overview

It’s important to understand the testing process to be able to create a better plan for testing
activities, but it’s also important to start slow. Before we dive into the types of software testing and
the specifics of the software QA testing process, let’s get a quick overview of what it is and why
it’s crucial to modern IT development.

What Is Software Testing?

Software testing refers to the process of evaluating a product to ensure it meets the requirements
and to identify errors, missing specifications, and gaps between current and stipulated conditions.
Software testing is performed at virtually every software development stage: teams can test the
software once it’s completed, or use a continuous testing procedure at all stages of development to
verify the quality of the software as it’s being developed.
Software testing often involves executing software manually or with automated tools to make sure
it functions properly and provides a good user experience. Testing can be done without any
specialized software testing tools, but most teams prefer to use at least one testing tool or more to
ensure truly effective testing.

The Principles of Software Testing at a Glance

Key phases: Requirement analysis, existing process review, testing process design, testing
implementation, review of the results, maintenance.
Process setup time: 2 to 6 months
Place in the SDLC: From requirement gathering to maintenance.
Team composition: QA consultant, QA manager, QA engineer, AQA engineer.

Quality Assurance (QA) vs. Quality Control (QC)

Quality assurance and quality control are two essential aspects of the software development life
cycle that ensure the product satisfies the requirements. However, they differ in their approach to
verifying the quality.
QA is process-oriented and aims to confirm that the development processes are efficient. QA
procedures often help prevent defects and come in handy when a project is in trouble and needs
reworking.
On the other hand, QC focuses more on ensuring that the software product complies with the
requirements at a given moment. Unlike QA, QC is generally reactive and detects issues that
already exist in the product.
In this article, we’ll focus more on the QA aspect of software testing.

So, what are the benefits of quality assurance? There are many, but let’s single out just a few of the
key points:
Cost savings. QA makes the development process more cost-effective. By testing each component of
the software before deploying it, testing reduces the expenses of correcting the bugs that are
discovered later, where they have a more drastic impact on the software performance.
Improved software quality. Testing ensures the ability to deliver quality software to your clients
consistently. High-quality software plays a key role in building a positive brand image and attracting
more customers.
Enhanced productivity. Test-driven development often results in better understanding and familiarity
with product requirements, leading to enhanced productivity with the development teams.
With strong arguments in mind, let’s discuss the different types of tests involved in the software
testing process. They will help you understand how these tests are completed, which aspects of the
solution different stages of the QA process in software testing are responsible for, and which part
of your project they support.

What Happens Without a Solid Software Testing Process in Place?

We firmly believe that developing software and testing it are two sides of the same coin, or two
indispensable parts of the same process. In our opinion, you simply cannot develop a product and
then skip the testing stage or treat it with as little concern as possible — the risks of doing that are
too big to ignore. Of course, we are not trying to scare you into implementing a new software
testing workflow in your workplace right away. Still, here are the key risks of not doing it in the
first place:
Critical quality issues. The biggest risk of a faulty software QA process is that the more disjointed it
is, the higher the chance of the team missing crucial defects. This can trigger a whole set of negative
consequences, including increased development costs and a product that falls behind schedule.
Delays in production. Without a comprehensive QA workflow implemented, when developers
constantly have to take care of the bugs discovered at later stages, where they take more time to
resolve, the development and deployment of new features and improvements will inevitably get
delayed.
Trust and reputation damages. With new software products, bugs and minor flaws are nearly
inevitable. However, the worst-case scenario is when those bugs are discovered by the end users. This
leads to the loss of trust that users put into a new product, as well as subsequent reputation damages,
as news of the underperforming or security-compromised application makes it to online communities.
Compatibility problems. With the growing variability of devices, operating systems, and platforms,
developers and testers must consider compatibility to be one of the main requirements for a good
software product. Preemptively testing the software item to detect compatibility issues saves a lot of
trouble for the team, while the lack of firm testing practices creates compatibility gaps that are
difficult to resolve.

What Are the Different Testing Types?

Quality assurance and testing can be categorized in many different ways according to several
criteria, such as the approach, method used, or level of tests performed.
When software testing is categorized by approach, the focus is on whether it is carried out with the
knowledge of a system’s inner workings. When categorized by method, the focus is on how the
testing is actually performed.
In this section, we’ll take a closer look at the different types of software testing by category,
methodology, and approach. First, let’s outline the main categories of QA testing types.

Main Categories of Tests

There are two main categories of tests depending on who or what performs the actual tests.

Manual Testing
Companies manually test software to ensure that the existing features meet requirements by
working through test cases manually, without the aid of automated tools.
While manual testing can become expensive in the long run, it’s usually cheaper at the outset since
you can avoid the costs of automation tools. It’s often used for examining the user interface and
other tests that require human intervention.
Automation Testing
Automation testing involves the use of tools or scripts to verify the quality of the software being
tested. It is ideal for regression testing, which involves repeatedly checking the software after
modification to identify newly introduced defects. It’s also better suited for load and performance
testing.
Automation is often used in conjunction with manual testing, which means that automation QAs
also need manual testing skills in addition to the knowledge of automation techniques. While the
initial investment is higher with automated tools, it will be less expensive and more efficient than
manual in the long run.

Types of Software Testing by Method

Software testing may be broken down into two categories according to the method they use. The
major difference between them is whether or not the software is running during the test.

Static Testing
Static testing, or verification, is a method that involves checking files and documents to verify
requirements are being met without executing the software. It involves activities such as code
inspections, technical reviews, and walk-throughs.
This type is often used in the early stages of the development process to identify and rectify bugs
quickly. It can be carried out manually or with the aid of automation.

Dynamic Testing
Dynamic testing, or validation, is a technique that involves executing a product to evaluate its
behavior with dynamic variables. It ensures the software works consistently without any major
defects.
Oftentimes, dynamic testing helps discover defects that cannot be found through static testing. It’s
also essential for detecting and fixing critical issues, such as security loopholes.

Types of Software Testing by Approach

By approach, there are three types of QA testing based on a QA expert’s involvement in the code
and knowledge of a system’s internals.

White Box Testing

White box testing, also known as glass box, clear box, and code-based testing, is an approach in
the QA testing process that involves examining a product based on the knowledge of a system’s
inner workings.
White box testing involves verifying the flow of specific inputs through the program to ensure the
software works properly. It requires an understanding of the source code and programming skills.
Though white box testing can be complex and time-consuming, it often results in highly optimized
code with minimal errors. Unlike most other types of QA testing, the white box can also be
performed during early development before user interfaces are developed.

Black Box Testing

Black box testing, also known as behavioral or specification-based testing, involves checking
software without knowing the internal design and structure of the code.
Black box testing is based on specifications and requirements and verifies that the software
produces the expected outputs from different cases. Unlike white box testing, this approach does
not require the QA specialist to have programming skills.
Black box testing can be subdivided into:
Functional – focuses on ensuring that the software’s functionality matches the specified requirements.
Non-functional – focuses on requirements such as usability, scalability, and performance of the
software.

Grey Box Testing

Grey box testing is a combination of the white box and black box approaches. In this approach, QA
specialists are partially aware of the inner workings of the software and have access to
documentation of the algorithms and data structures implemented.
Grey box testing is often used for complete testing since it improves the overall quality of the
software product by leveraging the input of both developers and QAs and allowing the latter to
create more intelligent test cases.

Types of Software Testing by Level

By level of product development, we can pinpoint four QA testing types:

Unit Testing
Unit testing is the lowest level and involves verifying individual units of software. A unit is the
smallest component of software that can be tested independently; this verifies that each individual
unit is functioning properly.
Unit testing is usually performed by the developers using automated tools. Writing unit tests makes
it easier to find and fix fundamental issues quickly, thus improving the overall quality of the
codebase.

Integration Testing
Integration testing is the next level. It involves checking the flow of data between the different
modules or groups of components in the application. It helps ensure that the interactions between
the integrated units are error-free.
Integrated testing can be performed by developers or QA specialists and often involves manual and
automated tests. They help secure interface defects, so they don’t escape to higher testing levels.

System Testing
System testing involves evaluating the fully integrated system to verify that it satisfies the set
requirements. It is usually performed by a separate team of testers who test the product from end to
end before it is pushed into production.
Some of the different types of tests performed on this level include usability, load, regression,
performance, and security testing. System testing also detects any bugs missed on earlier testing
levels.

Acceptance Testing
Acceptance testing is the last level of software testing and one of the most crucial types of QA
testing for a positive audience reception. It involves assessing whether your product is ready for
delivery to the end user. This level of testing is often performed in collaboration with end users to
ensure the product not only complies with the business criteria but also meets consumers’ needs.
Acceptance testing enables the development team to get actual feedback from potential users to
iron out any usability concerns. It also further reduces the risk of major bugs slipping through into
production.

As you can see, any part or aspect of your product can be verified by various tests, but how exactly
do QA testing processes happen? Before we look at the different stages in the software testing
process, let’s take a closer look at two preliminary steps: identifying deliverables and choosing a
testing process model.

Deliverables Involved in Testing

The first step in the software testing process is identifying what deliverables must be provided.
Some deliverables have to be provided before testing commences, while others will be provided at
different stages during the process or after its completion. Some examples include:
Test Strategy
A test strategy is an essential management document that provides an overview of the testing
approach. This deliverable is prepared by the test manager and is derived from the business
requirement specification (BRS). The document often contains the test objectives, approach,
existing and required conditions, scope, types, levels, and entry and exit criteria.
The test strategy describes the organization’s overall testing methodology, independent of any
specific project — the “how” of testing for the organization. It includes how testing is used to
manage product and project risks, the division of testing into levels, and the high-level activities
associated with testing. The test strategy, as well as the processes and actions described in it, must
be consistent with the test policy. It should provide general test entry and exit criteria for an
organization or for one or more programs.

“There are several types of test strategies that can be combined. The specific
strategies chosen must be relevant to the organization’s needs and
capabilities, and organizations can adapt strategies to suit specific operations
and projects.”

Test Plan and Estimation

The test plan is a document that contains a detailed description of each step of testing. The
estimation deliverable provides an estimation of how long each step will take, how much effort
will be required, and the overall cost of the process.
The test plan is derived from the software requirements specification (SRS), product description, or
use case documents. It serves as a road map for the entire project and is normally prepared by the
test manager or test lead.

Test Scenario
The test scenario document contains a high-level overview of all the functionality that needs to be
tested. Test scenarios are derived from use cases and help the tester determine what to focus on
during testing.

Test Cases
Test cases are the steps a QA specialist must go through to verify the software meets the
requirements. They often include clearly defined steps, pre-conditions, test data, expected results,
and post-conditions.
Test Data
Test data refers to the information used to run test cases. It needs to be comprehensive and accurate
to ensure that defects are properly detected.
Requirement Traceability Matrix
The requirement traceability matrix is a document that helps the testing team determine if all the
requirements are covered by test cases. It contains a map of requirements to appropriate test cases.
It also provides clients with an easy way to check on the QA processes and track the test execution
status.
Test Summary Report
The test summary report document contains the key points of all the test activities carried out and
the results obtained. This document often includes the objectives, scope, approach, results, and the
members involved in the testing process.

Test Closure Report

The test closure report is prepared by the test manager when the testing process is about to end, and
defects have been resolved. It contains a detailed analysis of the testing process as well as whatever
bugs were found and fixed.

Incident Report
The test incident report document is prepared when defects are encountered during the test
execution process. It contains information such as the status, severity, cause, and description of the
incident.

Choosing the Testing Process Models

The next step is to choose what model will be used for the software QA testing process. The four
major types of testing process models are waterfall, agile, V, and spiral.

Traditional Waterfall Development Model

Waterfall is one of the oldest models of software development and testing. Here, the development
progresses sequentially through the different stages of the software development life cycle.
All the project requirements are defined during the requirements phase, and the testing team only
proceeds once each stage is completed. The testing is carried out by an independent group of
testers.
While the waterfall development model works well for simple projects where requirements are
clearly defined, it’s not ideal for complex projects where requirements change frequently.

Agile Model
The agile model is a more modern method of software development that focuses on breaking down
the development process into small, incremental cycles. Under this model, the software QA testing
process swiftly responds to changing requirements and prioritizes regular interaction between the
client and the development team over long-term planning.
In the agile model, developers continuously write and run unit tests and fix defects as they are
discovered. The agile model is better suited for complex projects where requirements change
frequently.

V Model
The V model, or the verification and validation model, is a rigid model in which development and
testing occur in parallel. This model is an extension of the waterfall model. For every phase in the
development life cycle, there’s a corresponding phase of the testing process flow.
The V model usually divides the testing process into five phases: unit, integration, regression,
system, and testing.
Like the waterfall model, the V is easy to apply and manage but performs poorly when used for
complex dynamic projects.

Spiral Model
The spiral model is similar to agile but focuses more on risk evaluation and analysis. It has four
major phases: planning, risk analysis, engineering, and evaluation.
While it’s ideal for large projects that need continuous risk evaluation, it can be costly and not very
advantageous for smaller projects.

What Are the QA Process Stages?

The testing process is conducted according to a specific procedure that involves testing activities of
different kinds and helps make sure the defects are easy to uncover. Ideally, quality assurance
should begin early in the development life cycle. The software testing process often comprises the
following stages:

1. Requirement Analysis
During this phase of the QA process, testers review the requirements to ensure that they’re
complete, clear, consistent, verifiable, and testable. This helps to facilitate the planning and design
phases and prevents defects caused by incomplete or unclear requirements.

2. Test Planning
Once the requirements have been reviewed, testers can begin developing a strategy and plan
deliverables. At this stage, they can define the scope of testing, what software testing types and
levels will be required, and what tools and approaches will be used during the process.

3. Test Design
With the planning phase complete, QA professionals can begin designing test cases, setting up
automation scenarios, and preparing the environment for test execution.

4. UI/UX Testing
User interface (UI) testing involves confirming that the various UI components, such as buttons
and labels, work as specified in the requirements. User experience (UX) testing aims to detect and
fix usability issues and provide the end user with the best possible experience when using the
software product.

5. Exploratory Testing
Exploratory testing is an approach to software testing that emphasizes the personal freedom and
responsibility of individual testers to continually optimize the quality of their work by learning
from the results of their testing. It is often contrasted with scripted testing, where testers follow
predefined steps.

“For better results of the testing, teams can use tours. This is a concept often
used within exploratory testing to help structure and guide the testing
process while still allowing for the flexibility and creativity that characterize
exploratory testing. They are metaphorical journeys through the application,
focusing on different aspects or themes.”

6. Regression Testing
Regression testing involves retesting already executed test cases after defects are resolved to
ensure no new bugs are introduced into the system and that existing functionality still works as
expected.
7. Test Execution and Defect Reporting
Test execution begins at the unit testing level as developers write and execute unit tests. It later
moves to the testing team, who perform integration and system testing. The team executes the
cases and reports any discovered defects in a bug tracking system. Test automation tools are also
used to execute tests and generate reports automatically.

8. Release Testing
Once a new version of the software product is ready for release, the testing team carries out smoke
testing to ensure the release candidate is stable and bug-free, executes test suites that cover the new
functionality, and issues a test result report.

9. Verification and Validation

Verification and validation are two important parts of the QA test process. Verification refers to
the process of ensuring the software product meets the specified requirements. This part of the QA
process in software testing often involves inspecting and reviewing files, code, and designs to
verify software quality.
Validation, on the other hand, involves ensuring the software functions as intended. Validation
involves executing the program and working through test cases. It consists of activities such as unit
testing, integration testing, system testing, and acceptance testing.
While verification is performed by developers, validation is carried out by the testing team.
Verification is also performed early in the testing process, while validation comes afterward and
can detect bugs that the verification process misses.

10. Post-Run Testing Activities

After the principal part of the testing project is over, the team’s next task is to analyze the
intermediate and final results and determine whether the results align with the initial project goals.
The team may use various analytics tools, including AI-based ones, to see how successful their
activities were and what can be improved in the next test run. Moreover, the team may be scaled
down to one or two engineers who will provide ongoing maintenance of the test suite and work
with the development and support team to test further improvements to the product.

When to Start or Stop Testing in Software Development?

As we’ve already mentioned, software testing should begin early in the software development life
cycle. For example, the testing process flow can start from the requirements phase and continue
until the software is deployed.
However, when testing actually starts will depend on the development model in use. For example,
if the waterfall model is used, formal testing will commence in the testing phase.
Starting the software QA process early reduces the cost of finding and fixing defects. Research
shows that the cost of fixing bugs during testing can be up to 15 times more than the cost of
identifying and fixing them during the requirements stage.
But when should the testing process stop? It can be difficult to determine when to stop testing as
the software development process is often ongoing, and new features are continually being
developed in most projects.
As a result, there are several different factors that must be considered before determining whether
to stop. These include project deadlines, testing deadlines, test coverage, how much of the testing
budget is left, how many test cases have been completed, and whether all test deliverables have
been delivered.

Best Practices and Approaches to Software Testing

At the end of the day, the goal of testing is to ensure the spotless operation of a software product.
However, there are different ways to achieve that goal, and some testing approaches are more
effective than others. Here are the tried and tested best practices for performing software testing:

Know Your Audience

For an experienced testing team, it can be easy to see the process of quality assurance as a purely
technical task. However, there is more to helping create a popular, well-received product than just
technical finesse. All teams involved in the project, including the testing team, should know their
audience and what their users are looking for in a product. This helps create value-driven solutions
that quickly make a name for themselves despite the grueling market competition and start
generating profit for the product owners.

Set Correct Priorities

You can find countless tips and tricks on how to plan your software QA process flow and what is
required to make things work. However, it’s also important to adjust those tips and rules to the
specifics of your project. For example, while testing documentation is vital to a successful QA
project, when resources are limited, the team may want to prioritize polishing and stabilizing the
software product over dedicating hours to maintaining the documentation.
Don’t Underestimate Exploratory Testing
Exploratory testing requires a significant investment of resources and cannot be automated, but its
value for a software testing project can hardly be overrated. Allocating enough time for exploratory
testing allows the team to get a better understanding of the inner logic of the product, encourage
the knowledge sharing process among the team members, and even make up for the limitations of
automated testing, helping the testers fill in the quality gaps not covered by automation.

Mix Manual and Automated Testing

There are instances where manual testing is absolutely indispensable. For example, nothing beats
the precision of human eyes when it comes to UX or accessibility testing. However, it’s also a
good idea to automate tests whenever possible. And some types of testing are practically begging
to be automated. This includes repetitive, time-consuming testing activities such as regression
testing or performance testing. Investing a little more time and effort into setting up the automation
process in the beginning, the team can then enjoy a more streamlined, effective, and resource-
saving testing process.

Move Testing Earlier in the Development Cycle

Starting testing early is part of the Agile development model, and it’s also the basic principle of the
“shift-left” approach to testing. Moving testing to the beginning of the development cycle helps
tackle many of the usual problems development and testing teams encounter in their work. Most
importantly, it helps catch bugs back when they are easier and less expensive to fix and haven’t
done any significant damage to the product overall or to its reputation.

Formal proof of correctness, Software matrix

What is Formal Verification-Proof of Correctness

A proof of correctness is a mathematical proof that a computer program or a part thereof will, when
executed, yield correct results i.e., results fulfilling specific requirements. Before proving a program
correct, the theorem to be proved must, of course, be formulated.

Hypothesis: The hypothesis of such a correctness theorem is typically a condition that the relevant program
variables must satisfy immediately “before” the program is executed. This condition is called the
“precondition”.

Thesis: The thesis of the correctness theorem is typically a condition that the relevant program variables
must satisfy immediately “after” execution of the program. This latter condition is called the ‘post-
condition’.

Thus the correctness theorem can be stated as:

 “If the condition, “V”, is true before execution of the program, “S”, then the condition, “P”, will be true
after execution of “S” “.

Where “V” is pre-condition and “P” is post-condition.

Notation:Such a correctness theorem is usually written as {V} S {P}, where V, S and P have been
explained above.

By “program variable” we broadly include input and output data, e.g., data entered via a keyboard,
displayed on a screen or printed on paper. Any externally observable aspect of the program’s execution
may be covered by the precondition and post-condition.

Automated Testing, Software testing with automated tools

Automation Testing - Software Testing

Automated Testing means using special software for tasks that people usually do when
checking and testing a software product. Nowadays, many software projects use automation
testing from start to end, especially in agile and DevOps methods. This means
the engineering team runs tests automatically with the help of software tools . It will help to
keep the testing team to make the process faster. Continuous delivery (CD) and quickly sends
the new code to users.
Automated testing is important for this because it converts the manual steps into automation.
Continuous integration (CI) checks the new code changes to prevent issues. CD gets after CI
done everything well. Automated testing, CI & CD will together prove that the new code is
error-free and ready for deployment quickly for the project purpose.

What is Automation Testing?

On the other side, it's a technique where the Tester writes scripts independently and uses
suitable Software or Automation Tools to test the software. It is an Automation Process of a
Manual Process. It allows for executing repetitive tasks without the intervention of a Manual
Tester.
 It is used to automate the testing tasks that are difficult to perform manually.
 Automation tests can be run at any time of the day as they use scripted sequences to
examine the software.
 Automation tests can also enter test data compare the expected result with the actual result
and generate detailed test reports.
 The goal of automation tests is to reduce the number of test cases to be executed manually
but not to eliminate manual testing.
 It is possible to record the test suit and replay it when required.

Automated Testing uses specialized software to replace manual testing tasks, speeding up the
process and integrating seamlessly with CI/CD pipelines. It allows for continuous code
verification and quicker deployment.

Table of Content
 Why Transform From Manual to Automated Testing?
 What kinds of software tests should be automated first?
 What kinds of software tests should be done manually?
 Difference between Manual Testing vs Automated Testing
 Automation Testing Types
 Test Automation Frameworks
 Which Tests to Automate?
 Automation Testing Process
 Criteria to Select Automation Tool
 Best Practices for Test Automation
 Popular Automation Tools
 Advantages of Automation Testing
 Disadvantages of Automation Testing

Why Transform From Manual to Automated Testing?

In 1994, an aircraft completing its routine flight crashed just before landing. This was due to
a bug or defect in their software. The testers didn’t even care about the final testing, so this
accident happened. In such high-priority cases, accurate automation testing becomes critical
to replace a few manual tests. Automation testing tools like BrowserStack can help teams
thoroughly test and speed up the execution time of test suites. Below are some of the reasons
for using automation testing:
 Quality Assurance: Manual testing is a tedious task that can be boring and at the same
time error-prone. Thus, using automation testing improves the quality of the software
under test as more test coverage can be achieved.
 Error or Bug-free Software: Automation testing is more efficient for detecting bugs in
comparison to manual testing.
 No Human Intervention: Manual testing requires huge manpower in comparison to
automation testing which requires no human intervention and the test cases can be
executed unattended.
 Increased test coverage: Automation testing ensures more test coverage in comparison to
manual testing where it is not possible to achieve 100% test coverage.
 Testing can be done frequently: Automation testing means that the testing can be done
frequently thus improving the overall quality of the software under test.

What Kinds of Software Tests Should Be Automated First?

1. End-to-End tests
End-to-end testing is a type of software testing used to test whether the flow of software from
the initial stage to the final stage is behaving as expected. The purpose of end-to-end testing
is to identify system dependencies and to make sure that the data integrity is maintained
between various system components and systems. End-to-end testing: End-to-end testing,
also known as end-to-end functional testing, is a type of testing that validates the flow of a
system from start to finish.
2. Unit tests
Unit testing is automated and is run each time the code is changed to ensure that new code
does not break existing functionality. Unit tests are designed to validate the smallest possible
unit of code, such as a function or a method, and test it in isolation from the rest of the
system.
3. Integration tests
Integration testing is the process of testing the interface between two software units or
modules. It focuses on determining the correctness of the interface. The purpose of
integration testing is to expose faults in the interaction between integrated units. Once all the
modules have been unit-tested, integration testing is performed.
4. Performance tests
Performance Testing is a type of software testing that ensures software applications perform
properly under their expected workload. It is a testing technique carried out to determine
system performance in terms of sensitivity, reactivity, and stability under a particular
workload.
What Kinds of Software Tests Should Be Done Manually?
Automating tests will save the time and speed up the productivity, but not every test needs
automation. sometime the investment in creating automated tests the benefits of manual
testing.
1. Exploratory testing
Exploratory Testing is a type of software testing in which the tester is free to select any
possible methodology to test the software. It is an unscripted approach to software testing. In
exploratory testing, software developers use their learning, knowledge, skills, and abilities to
test the software developed by themselves. Exploratory testing checks the functionality and
operations of the software as well as identify the functional and technical faults in it.
2. Visual Regression Testing
Visual regression testing is a software development technique for identifying unintentional
visual modifications made to the user interface (UI) of a web application. To find any
differences, screenshots of the user interface (UI) of the application are compared before and
after code changes. Developers can guarantee that changes to the coding don’t bring about
unexpected visual changes that could degrade the user experience by automating this process.
3. Building a test automation framework for your DevOps team
Automated testing do not have one is not fit for all. When setting up automated testing for
your team, consider a few important things.
4. Frequency of release
For software that releases regular, like in every month or week, manual testing is might be
okay. But for faster purpose, the automated testing is a more helpful because it is need for
the continuous integration and delivery.
5. Available tools and ecosystem
Every programming language it has been its own set of tools and utilities that work well with
it. with the same, each type of automated testing has its own tools for testing, which may not
be available for every programming language. for successfully implement automated testing,
you need tools which match the both the programming language and the testing pattern you
are using.
6. Product market fit and code base maturity
Product-market fit refers to the alignment between a product or service and the market it
serves. It signifies the degree to which a product satisfies strong market demand and
effectively addresses the needs or problems of a specific market segment. Product market
fit means being in a good market with a product that can satisfy that market.
Difference Between Manual Testing vs Automated Testing
Below are some of the differences between manual testing and automated testing:
 Parameters Manual Testing Automated Testing

Manual testing is not Since it is performed by

accurate at all times due to third-party tools and/or
human error, thus it is less scripts, therefore it is more
Reliability reliable. reliable.

Heavy investment in Investment in tools rather

Investment human resources. than human resources.

Manual testing is time- Automation testing is time-

consuming due to human saving as due to the use of
intervention where test the tools the execution is
cases are generated faster in comparison to
Time efficiency manually. manual testing.

There is no need to have It is important to have

programming knowledge to programming knowledge to
Programming knowledge write the test cases. write test cases.

There is a possibility that

the test cases executed the When there are changes in
first time will not be able to the code, regression testing
catch the regression bugs is done to catch the bugs due
due to the frequently to changes in the code.
Regression testing changing requirements.

Automation Testing Types

Below are the different types of automation testing:
 Unit testing: Unit testing is a phase in software testing to test the smallest piece of code
known as a unit that can be logically isolated from the code. It is carried out during the
development of the application.
 Integration testing: Integration testing is a phase in software testing in which individual
software components are combined and tested as a group. It is carried out to check the
compatibility of the component with the specified functional requirements.
 Smoke testing: Smoke testing is a type of software testing that determines whether the
built software is stable or not. It is the preliminary check of the software before its release
in the market.
 Performance testing: Performance testing is a type of software testing that is carried out
to determine how the system performs in terms of stability and responsiveness under a
particular load.
 Regression testing: Regression testing is a type of software testing that confirms that
previously developed software still works fine after the change and that the change has not
adversely affected existing features.
 Security testing: Security testing is a type of software testing that uncovers the risks, and
vulnerabilities in the security mechanism of the software application. It helps an
organization to identify the loopholes in the security mechanism and take corrective
measures to rectify the security gaps.
 Acceptance testing: Acceptance testing is the last phase of software testing that is
performed after the system testing. It helps to determine to what degree the application
meets end users' approval.
 API testing: API testing is a type of software testing that validates the Application
Programming Interface(API) and checks the functionality, security, and reliability of the
programming interface.
 UI Testing: UI testing is a type of software testing that helps testers ensure that all the
fields, buttons, and other items on the screen function as desired.
Test Automation Frameworks
Some of the most common types of automation frameworks are:
 Linear framework: This is the most basic form of framework and is also known as the
record and playback framework. In this testers create and execute the test scripts for each
test case. It is mostly suitable for small teams that don't have a lot of test automation
experience.
 Modular-Based Framework: This framework organizes each test case into small
individual units known as modules each module is independent of the other, having
different scenarios but all modules are handled by a single master script. This approach
requires a lot of pre-planning and is best suited for testers who have experience with test
automation.
 Library Architecture Framework: This framework is the expansion of a modular-based
framework with few differences. Here, the task is grouped within the test script into
functions according to a common objective. These functions are stored in the library so
that they can be accessed quickly when needed. This framework allows for greater
flexibility and reusability but creating scripts takes a lot of time so testers with experience
in automation testing can benefit from this framework.
Which Tests to Automate?
Below are some of the parameters to decide which tests to automate:
 Monotonous test: Repeatable and monotonous tests can be automated for further use in
the future.
 A test requiring multiple data sets: Extensive tests that require multiple data sets can be
automated.
 Business critical tests: High-risk business critical test cases can be automated and can be
scheduled to run regularly.
 Determinant test: Determinant test cases where it is easy for the computer to decide
whether the test has failed or not can be automated.
 Tedious test: Test cases that involve repeatedly doing the same action can be automated
so that the computer can do the repetitive task as humans are very poor at performing the
repetitive task with efficiency, which increases the chances of error.
Automation Testing Process
1. Test Tool Selection: There will be some criteria for the Selection of the tool. The
majority of the criteria include: Do we have skilled resources to allocate for automation
tasks, Budget constraints, and Do the tool satisfies our needs?
2. Define Scope of Automation: This includes a few basic points such as the Framework
should support Automation Scripts, Less Maintenance must be there, High Return on
Investment, Not many complex Test Cases
3. Planning, Design, and Development: For this, we need to Install particular frameworks
or libraries, and start designing and developing the test cases such
as NUnit , JUnit , QUnit , or required Software Automation Tools.
4. Test Execution: Final Execution of test cases will take place in this phase and it depends
on Language to Language for .NET, we'll be using NUnit, for Java, we'll be using JUnit,
for JavaScript, we'll be using QUnit or Jasmine, etc.
5. Maintenance: Creation of Reports generated after Tests and that should be documented to
refer to that in the future for the next iterations.
Criteria to Select Automation Tool
Following are some of the criteria for selecting the automation tool:
 Ease of use: Some tools have a steep learning curve, they may require users to learn a
completely new scripting language to create test cases and some may require users to
maintain a costly and large test infrastructure to run the test cases.
 Support for multiple browsers: Cross-browser testing is vital for acceptance testing.
Users must check how easy it is to run the tests on different browsers that the application
supports.
 Flexibility: No single tool framework can support all types of testing, so it is advisable to
carefully observe what all tool offers and then decide.
 Ease of analysis: Not all tools provide the same sort of analysis. Some tools have a nice
dashboard feature that shows all the statistics of the test like which test failed and which
test passed. On the other hand, there can be some tools that will first request users to
generate and download the test analysis report thus, not very user-friendly. It depends
entirely on the tester, project requirement, and budget to decide which tool to use.
 Cost of tool: Some tools are free and some are commercial tools but many other factors
need to be considered before deciding whether to use free or paid tools. If a tool takes a lot
of time to develop test cases and it is a business-critical process that is at stake then it is
better to use a paid tool that can generate test cases easily and at a faster rate.
 Availability of support: Free tools mostly provide community support on the other hand
commercial tools provide customer support, and training material like tutorials, videos,
etc. Thus, it is very important to keep in mind the complexity of the tests before selecting
the appropriate tool.
Best Practices for Test Automation
Below are some of the best practices for test automation that can be followed:
 Plan self-contained test cases: It is important to ensure that the test is clearly defined and
well-written. The test cases should be self-contained and easy to understand.
 Plan the order to execute tests: Planning the test in the manner that the one test creates
the state for the second test can be beneficial as it can help to run test cases in order one
after another.
 Use tools with automatic scheduling: If possible use tools that can schedule testing
automatically according to a schedule.
 Set up an alarm for test failure: If possible select a tool that can raise an alarm when a
test failure occurs. Then a decision needs to be made whether to continue with the test or
abort it.
 Reassess test plans as the app develops and changes: It is important to continuously
reassess the test plan as there is no point in wasting resources in testing the legacy features
in the application under test.
Popular Automation Tools
 Selenium: Selenium is an automated testing tool that is used for Regression testing and
provides a playback and recording facility. It can be used with frameworks
like JUnit and Test-NG . It provides a single interface and lets users write test cases in
languages like Ruby, Java, Python, etc.
 QTP: Quick Test Professional (QTP) is an automated functional testing tool to test both
web and desktop applications. It is based on the VB scripting language and it provides
functional and regression test automation for software applications.
 Sikuli: It is a GUI-based test automation tool that is used for interacting with elements of
web pages. It is used to search and automate graphical user interfaces using screenshots.
 Appium: Apium is an open-source test automation framework that allows QAs to conduct
automated app testing on different platforms like iOS, Android, and Windows SDK.
 Jmeter: Apache JMeter is an open-source Java application that is used to load test the
functional behavior of the application and measure the performance.

Popular Tools:

Type Tool Description

Unit Testing JUnit, PyTest, NUnit Language-specific frameworks
GUI Testing Selenium, Cypress, Playwright Automated browser testing
Load Testing JMeter, Locust Performance and stress testing
CI/CD Integration Jenkins, GitHub Actions, CircleCI Automation pipelines
Test Management TestRail, Zephyr Planning and reporting

Advantages of Automation Testing

 Simplifies Test Case Execution: Automation testing can be left virtually unattended and
thus it allows monitoring of the results at the end of the process. Thus, simplifying the
overall test execution and increasing the efficiency of the application.
 Improves Reliability of Tests: Automation testing ensures that there is equal focus on all
the areas of the testing, thus ensuring the best quality end product.
 Increases amount of test coverage: Using automation testing, more test cases can be
created and executed for the application under test. Thus, resulting in higher test coverage
and the detection of more bugs. This allows for the testing of more complex applications
and more features can be tested.
 Minimizing Human Interaction: In automation testing, everything is automated from
test case creation to execution thus there are no changes for human error due to neglect.
This reduces the necessity for fixing glitches in the post-release phase.
 Saves Time and Money: The initial investment for automation testing is on the higher
side but it is cost-efficient and time-efficient in the long run. This is due to the reduction in
the amount of time required for test case creation and execution which contributes to the
high quality of work.
 Earlier detection of defects: Automation testing documents the defects, thus making it
easier for the development team to fix the defect and give a faster output. The earlier the
defect is identified, the more easier and cost-efficient it is to fix the defects.
Disadvantages of Automation Testing
 High initial cost: Automation testing in the initial phases requires a lot of time and money
investment. It requires a lot of effort for selecting the tool and designing customized
software.
 100% test automation is not possible: Generally, the effort is to automate all the test
cases but in practical real situations not all test cases can be automated some test cases
require human intervention for careful observation. There is always a human factor, i.e., it
can't test everything like humans(design, usability, etc.).
 Not possible to automate all testing types: It is not possible to automate tests that verify
the user-friendliness of the system. Similarly, if we talk about the graphics or sound files,
even their testing cannot be automated as automated tests typically use textual descriptions
to verify the output.
 Programming knowledge is required: Every automation testing tool uses any one of the
programming languages to write test scripts. Thus, it is mandatory to have programming
knowledge for automation testing.
 False positives and negatives: Automation tests may sometimes fail and reflect that there
is some issue in the system but there is no issue present and in some cases, it may generate
false negatives if tests are designed to verify that some functionality exists and not to
verify that it works as expected.