https://searchsoftwarequality.techtarget.

com/definition/user-acceptance-testing-UAT

https://www.altexsoft.com/blog/engineering/how-to-conduct-user-acceptance-testing-
process-stages-deliverables-and-end-user-testing-place-in-quality-assurance/
check

What is User Acceptance Testing (UAT)? with Examples

What is UAT?
User Acceptance is defined as a type of testing performed by the Client to certify
the system with respect to the requirements that was agreed upon. This testing
happens in the final phase of testing before moving the software application to the
Market or Production environment.

The main purpose of this testing is to validate the end to end business flow. It
does NOT focus on cosmetic errors, Spelling mistakes or System testing. This
testing is carried out in a separate testing environment with production like data
setup. It is a kind of black box testing where two or more end users will be
involved.

Who Performs UAT?

Client
End users

When is it Performed?

This is typically the last step before the product goes live or before the delivery
of the product is accepted. This is performed after the product itself is
thoroughly tested (i.e after system testing).

What is User Acceptance Testing (UAT)?

Need of User Acceptance Testing:

Once software has undergone Unit, Integration, and System testing the need of
Acceptance Testing may seem redundant. But Acceptance Testing is required because

What is User Acceptance Testing (UAT)?

Developers code software based on requirements document which is their "own"

understanding of the requirements and may not actually be what the client needs
from the software.
Requirements changes during the course of the project may not be communicated
effectively to the developers.

Acceptance Testing and V-Model

In VModel, User acceptance testing corresponds to the requirement phase of the
Software Development life cycle(SDLC).

What is User Acceptance Testing (UAT)?

Prerequisites of User Acceptance Testing:

Following are the entry criteria for User Acceptance Testing:

Business Requirements must be available.

Application Code should be fully developed
Unit Testing, Integration Testing & System Testing should be completed
No Show stoppers, High, Medium defects in System Integration Test Phase -
Only Cosmetic error is acceptable before UAT
Regression Testing should be completed with no major defects
All the reported defects should be fixed and tested before UAT
Traceability matrix for all testing should be completed
UAT Environment must be ready
Sign off mail or communication from System Testing Team that the system is ready
for UAT execution

How to do UAT Testing

UAT is done by the intended users of the system or software. This type of Software
Testing usually happens at the client location which is known as Beta Testing. Once
Entry criteria for UAT are satisfied, following are the tasks need to be performed
by the testers:

What is User Acceptance Testing (UAT)?

UAT Process
Analysis of Business Requirements
Creation of UAT test plan
Identify Test Scenarios
Create UAT Test Cases
Preparation of Test Data(Production like Data)
Run the Test cases
Record the Results
Confirm business objectives

Step 1) Analysis of Business Requirements

One of the most important activities in the UAT is to identify and develop test
scenarios. These test scenarios are derived from the following documents:

Project Charter
Business Use Cases
Process Flow Diagrams
Business Requirements Document(BRD)
System Requirements Specification(SRS)

Step 2) Creation of UAT Plan:

The UAT test plan outlines the strategy that will be used to verify and ensure an
application meets its business requirements. It documents entry and exit criteria
for UAT, Test scenarios and test cases approach and timelines of testing.

Step 3) Identify Test Scenarios and Test Cases:

Identify the test scenarios with respect to high-level business process and create
test cases with clear test steps. Test Cases should sufficiently cover most of the
UAT scenarios. Business Use cases are input for creating the test cases.

Step 4) Preparation of Test Data:

It is best advised to use live data for UAT. Data should be scrambled for privacy
and security reasons. Tester should be familiar with the database flow.

Step 5) Run and record the results:

Execute test cases and report bugs if any. Re-test bugs once fixed. Test Management
tools can be used for execution.

Step 6) Confirm Business Objectives met:

Business Analysts or UAT Testers needs to send a sign off mail after the UAT
testing. After sign-off, the product is good to go for production. Deliverables for
UAT testing are Test Plan, UAT Scenarios and Test Cases, Test Results and Defect
Log

Exit criteria for UAT:

Before moving into production, following needs to be considered:

No critical defects open

Business process works satisfactorily
UAT Sign off meeting with all stakeholders
Qualities of UAT Testers:
What is User Acceptance Testing (UAT)?

UAT Tester should possess good knowledge of the business. He should be independent
and think as an unknown user to the system. Tester should be Analytical and Lateral
thinker and combine all sort of data to make the UAT successful.

Tester or Business Analyst or Subject Matter Experts who understand the business
requirements or flows can prepare test and data which are realistic to the
business.

Best Practices:
Following points needs to be considered to make UAT Success:

Prepare UAT plan early in the project life cycle

Prepare Checklist before the UAT starts
Conduct Pre-UAT session during System Testing phase itself
Set the expectation and define the scope of UAT clearly
Test End to End business flow and avoid system tests
Test the system or application with real-world scenarios and data
Think as an Unknown user to the system
Perform Usability Testing
Conduct Feedback session and meeting before moving to production

UAT Tools
There are several tools in the market used for User acceptance testing and some are
listed for reference:

Fitness tool: It is a java tool used as a testing engine. It is easy to create

tests and record results in a table. Users of the tool enter the formatted input
and tests are created automatically. The tests are then executed and the output is
returned back to the user.

Watir : It is toolkit used to automate browser-based tests during User acceptance

testing. Ruby is the programming language used for inter-process communication
between ruby and Internet Explorer.

Some Example Guidelines of UAT

Most of the times in regular software developing scenarios, UAT is carried out in
the QA environment. If there is no staging or UAT environment
UAT is classified into Beta and Alpha testing but it is not so important when
software is developed for a service based industry
UAT makes more sense when the customer is involved to a greater extent
Conclusion:
In Software Engineering, UAT is one of the many flavors of testing that has emerged
over last twenty-five years. With UAT, the client can be sure "What to expect" from
the product rather than assuming. The benefit of UAT is that there will be no
surprises when the product is released to the market.

user acceptance testing (UAT)

In software development, user acceptance testing (UAT)—also called application

testing, and end user testing—is a phase of software development in which the
software is tested in the "real world" by the intended audience. UAT is often the
last phase of the software testing process, completed before the tested software is
released to its intended market. The goal of UAT is to ensure the software can both
handle real-world tasks and perform up to development specifications.

In UAT, users are given a chance to interact with the software before the official
release and see if any features have been overlooked or contain bugs. UAT can be
done by in-house testing in which volunteers or paid test subjects use the software
or by making the test version available for downloading a free trial over the Web.
The experiences of the early users are forwarded back to the developers who make
final changes before releasing the software commercially.

UAT is an effective method for ensuring quality in terms of time and cost to
perform while increasing transparency with the software users. UAT also allows
developers to work with real cases and data, and if successful, validates fulfilled
business requirements.

Types of UAT

Multiple types of software tests qualify as user acceptance testing. These tests
include:

Beta testing- Where the software is given to groups of end users, who will use the
software in its intended purpose and will provide feedback to developers for
changes to make improvements.

Black box testing- Where an end user will test specific software functions without
seeing the internal code.

Operational Acceptance Testing- Which puts a focus on proper workflow for the
software in use.

Contract Acceptance Testing- Where software is tested based on specific criteria.

Such specifications are defined in a contract.

Regulation Acceptance Testing- Where a focus on legal regulations are checked

against the software.
Types of User Acceptance Testing
Now that we’ve clearly separated functional testing from User Acceptance Testing,
we can look at the various types of User Acceptance Testing. The following User
Acceptance Testing Types exist:

1.Alpha & Beta Testing

2.Contract Acceptance Testing
3.Regulation Acceptance Testing
4.Operational Acceptance Testing
5.Black Box Testing

1.Alpha & Beta Testing

Alpha Testing normally takes place in the development environment and is usually
done by internal staff. Long before the product is even released to external
testers or customers. Also potential user groups might conduct Alpha Tests, but the
important thing here is that it takes place in the development environment.

Based on the feedback – collected from the alpha testers – development teams then
fix certain issues and improve the usability of the product.

Beta Testing, also known as “field testing”, takes place in the customer’s
environment and involves some extensive testing by a group of customers who use the
system in their environment. These beta testers then provide feedback, which in
turn leads to improvements of the product.

Alpha and Beta Testing are done before the software is released to all customers.

Is there a tool for that?

Did you email me the spreadsheet with the beta test results?

Asking testers via email to provide their test results is still a popular way to
conduct and run alpha/beta tests. And yet you’re probably wondering, “but isn’t
there a better solution for that?” Luckily, there is.

Usersnap – Your testers will love it

Usersnap is a great solution for asking alpha and beta testers for feedback.
It’s an easy-to-use UAT solution that helps QA teams verify if a certain solution
works for the user. By having a simple feedback widget, alpha and beta testers can
provide comprehensive feedback on a software prototype.

With Usersnap, UAT teams can easily gather and analyse qualitative feedback from
testers. And for testers it’s super-easy to work through a first alpha or beta
test, as they can simply draw on their screen to provide feedback.

2.Contract Acceptance Testing

Contract Acceptance Testing means that a developed software is tested against
certain criteria and specifications which are predefined and agreed upon in a
contract. The project team defines the relevant criteria and specifications for
acceptance at the same time when the team agrees on the contract itself.

3.Regulation Acceptance Testing

Regulation Acceptance Testing, also known as Compliance Acceptance Testing,
examines whether the software complies with the regulations. This includes
governmental and legal regulations.

4.Operational acceptance testing

Also known as Operational Readiness Testing or Production Acceptance Testing, these
test cases ensure there are workflows in place to allow the software or system to
be used.
This should include workflows for backup plans, user training, and various
maintenance processes and security checks.

5.Black Box Testing

Black Box Testing is often categorized as functional testing, but can, to some
extent, be seen as a type of User Acceptance Testing.

It’s a method of software testing which analyzes certain functionalities without

letting testers see the internal code structure. Black Box Testing is part of User
Acceptance Testing, because Black Box Tests share the same principles as UAT.

During Black Box Tests the user isn’t aware of any code base, but only about the
requirements which the software should meet.

Blackbox Testing as one of 5 types of User Acceptance Testing

Testers do not require any specific knowledge about the application or any of its
features. The tester conducting Black Box Tests is only aware of what the software
is supposed to do. They don’t know how it should be done.

Many QA and development teams use Black Box Testing for their UAT efforts pretty
frequently.

How to perform UAT testing

The number of steps involved in performing a user acceptance test may range
depending on how granular each team wants to define each step in the process. For
the most part, however, these steps commonly include

1.A planning phase where the business requirements and the strategy for the UAT are
outlined

2.The identification and creation of test scenarios. These test scenarios should
cover as many functional cases as possible that end users would face.
3.Testing teams are chosen. Developers can decide to have a select few end users
test the software or open up the test to many more by offering a free trial over
the Web.

4.A testing and documentation phase where end users begin testing the software and
any potential bugs and other issues are logged.

5.A fix phase where the development team makes adjustments to the code, resolving
any bugs or making suggested changes.

After this, the software should be ready for release into its production
environment.

What Is User Acceptance Testing?

We know what testing is, acceptance means approval or agreement. The user in the
context of a software product is either the consumer of the software or the person
who requested it to be built for him/her (client).

So, following my rule – the definition will be:

User Acceptance Testing (UAT), also known as beta or end-user testing, is defined
as testing the software by the user or client to determine whether it can be
accepted or not. This is the final testing performed once the functional, system
and regression testing are completed.

The main purpose of this testing is to validate the software against the business
requirements. This validation is carried out by the end users who are familiar with
the business requirements.

UAT, alpha and beta testing are different types of acceptance testing.

As user acceptance test is the last testing that is carried out before the software
goes live, obviously this is the last chance for the customer to test the software
and measure if it is fit for the purpose.

When is it Performed?
This is typically the last step before the product goes live or before the delivery
of the product is accepted. This is performed after the product itself is
thoroughly tested (i.e after system testing).

when is UAT performed

Who Performs UAT?

Users or client – This could be either someone who is buying a product (in the case
of commercial software) or someone who has had a software custom built through a
software service provider or the end user if the software is made available to them
ahead of the time and when their feedback is sought out.

The team can be comprised of beta testers or the customer should select UAT members
internally from every group of the organization so that each and every user role
can be tested accordingly.

Need for UAT

Developers and functional testers are technical people who validate the software
against the functional specifications. They interpret the requirements according to
their knowledge and develop/test the software (here is the importance of domain
knowledge).

This software is complete according to the functional specifications but there are
some business requirements and processes that are known only to the end users are
either missed to communicate or misinterpreted.

This testing plays an important role in validating if all the business requirements
are fulfilled or not before releasing the software for market use. Use of live data
and real use cases make this testing an important part of the release cycle.

Many businesses who suffered big losses due to post-release issues know the
importance of a successful User Acceptance Test. The cost of fixing the defects
after release is many times greater than fixing it before.

Is UAT really necessary?

After performing loads of system, integration and regression testing one would
wonder about the necessity of this testing. Actually speaking, this is the most
important phase of the project as this is the time at which the users who are
actually going to use the system would validate the system for its fit to purpose.

UAT is a test phase that largely depends on the perspective of the end users and
the domain knowledge of a department that represents the end users.

As a matter of fact, it would really be helpful to the business teams, if they were
involved in the project quite early, so that they can provide their views and
contributions that would help in effective usage of the system in the real world.

User Acceptance Testing Process

The easiest way to understand this process is to think of this as an autonomous
testing project – which means, it will have the plan, design and the execution
phases.

The following are the pre-requisites before the planning phase begins:

#1) Gather the key Acceptance Criteria

In simple terms, Acceptance criteria is a list of things that are going to get
evaluated before accepting the product.

These could be of 2 types:

(i) Application Functionality or Business Related

Ideally, all the key business functionality should get validated, but due to
various reasons, including time, it is not practical to do it all. Therefore, a
meeting or two with the client or the users who are going to be involved in this
testing can give us an idea on how much testing is going to be involved and what
aspects are going to be tested.

(ii) Contractual – We are not going to go into this and the involvement of the QA
team in all this is almost nothing. The initial contract that gets drawn up even
before the SDLC begins is reviewed and an agreement is reached upon whether all the
aspects of the contract have been delivered or not.

We are going to focus only on the application functionality.

#2) Define the scope of QA involvement.

QA team’s role is one of the following:

(i) No Involvement – This is very rare.

(ii) Assist in this testing – Most common. In this case, our involvement could be
training the UAT users on how to use the application and be on standby during this
testing to make sure that we can help the users in case of any difficulty. Or in
some cases, in addition to being on standby and assisting, we might share their
responses and record the results or log bugs etc., while the users perform the
actual testing.

(iii) Perform UAT and present Results – If this is the case, the users will point
the areas of the AUT that they want to evaluate and the evaluation itself is
performed by the QA team. Once done, the results are presented to the clients/users
and they will make a decision on whether the results that they have in hand are
sufficient or not and in accordance with their expectations in order to accept the
AUT. The decision is never that of the QA team.

Depending on the case on hand, we decide which approach is best.

The primary Objectives and expectations:

Objectives and Expectation of UAT

Usually, UAT is undertaken by a Subject Matter Expert (SME) and /or a business
user, who might be the owner or the customer of a system under test. Similar to the
System testing phase, the UAT phase also encompasses religious phases before it is
brought to closure.

Key activities of each UAT phase is defined below:

Key Activities of each UAT Phase

UAT Governance
Similar to system testing, effective governance is enforced for UAT to ensure that
strong quality gates along with the defined Entry and Exit criteria (provided below
**).

** Please note that it is just a guidance. This could be modified based on the
project needs and requirements.

UAT Governance

UAT Test Planning

The process is almost the same as with the regular test plan in the system phase.

The most common approach followed in most of the projects is to plan for both
system and UAT testing phases together. For more information on UAT test plan along
with a sample, please check out the attached test plan document’s UAT sections.

User Acceptance Test Plan

(This is the same that you would find on our site for the QA training series as
well).

Click on the below image and scroll down to find the test plan document sample in
various formats. In that template check the UAT section.

UAT Test Plan

The dates, environment, actors(who), communication protocols, roles and

responsibilities, templates, results and their analysis process, entry-exit
criteria – all of this and anything else that is relevant will be found in the UAT
test plan.

Whether the QA team is participating, partially participating or not participating

at all in this test, it is our job to plan this phase and make sure that everything
is taken into consideration.

=> Here is a User Acceptance Test Plan Sample Document

UAT Design
The gathered acceptance criteria from the users are used in this step. Samples
could look as shown below.

(These are excerpts from CSTE CBOK. This is one of the best references available
about this testing.)

User Acceptance Testing Template:

UAT Template

Based on the criteria, we (QA team) give them the users a list of UAT test cases.
These test cases are not different from our regular system test cases. They are
just a subset as we test all of the application as opposed, just to the key
functional areas.

In addition to these, the data, templates to record test results, administrative

procedures, defect logging mechanism, etc., has to be in place before we move to
the next phase.
Test Execution
Usually, when possible, this testing happens in a conference or a war room sort of
a set up where the users, PM, QA team representatives all sit together for a day or
two and work through all the acceptance test cases.

Or in case of the QA team performing the tests, we run the test cases on the AUT.

Once all the tests are run and the results are in hand, the Acceptance Decision is
made. This is also called the Go/No-Go decision. If the users are satisfied it’s a
Go, or else it’s a No-go.

Reaching the acceptance decision is typically the end of this phase.

Tools & Methodologies

Typically, the type of software tools that are used during this testing phase is
similar to the tools used while performing functional testing.

Tools:

As this phase involves validating the complete end to end flows of the application,
it might be difficult to have one tool to automate this validation completely.
However, to some extent, we would be able to leverage the automated scripts
developed during system testing.

Similar to system testing, users would also use test management and defect
management tool like QC, JIRA etc. These tools can be configured to cumulate data
for the User Acceptance phase.

Methodologies:

Though conventional methodologies such as specific business users performing UAT of

the product is still relevant, in a truly global world like today, User Acceptance
Testing sometimes has to involve varied customers across countries based on the
product.

For Example, an e-commerce website would be used by customers across the globe. In
scenarios like this, crowd testing would be the best viable option.

Crowd testing is a methodology where people from all over the world can participate
and validate the usage of the product and give suggestions and recommendations.

Crowd testing platforms are built and are being used by many organizations now. A
website or a product which needs to be crowd tested is hosted in the platform and
the customers can nominate themselves to do the validation. The feedbacks provided
are then analyzed and prioritized.

Crowd Testing methodology is proving to be more effective as the pulse of the

customer across the globe can be easily understood.

UAT in Agile Environment

The agile environment is more dynamic in nature. In an agile world, business users
will be involved throughout the project sprints and the project would be enhanced
based on the feedback loops from them.

At the beginning of the project, business users would be the key stakeholders to
provide requirement thereby updating the product backlog. During the end of each
sprint, business users would participate in the sprint demo and would be available
for providing any feedbacks.
Moreover, a UAT phase would be planned before the sprint completion where the
business users would do their validations.

The feedbacks which are received during sprint demo and sprint UAT, are collated
and added back to the product backlog which is constantly reviewed and prioritized.
Thus in an agile world, the business users are more close to the project and they
evaluate the same for its use more frequently unlike the traditional waterfall
projects.

UAT Team – Roles & Responsibilities

A Typical UAT organization would have the following Roles and responsibilities. The
UAT team would be supported by the project manager, development and testing teams
based on their needs.

Roles Responsibilities Deliverables

Business Programme Manager • Create and maintain Programme Delivery plan

• Review and Approve UAT Test Strategy and Plan
• Ensure the successful completion of the programme on schedule and budget
• Liaise with IT programme Manager and monitor the progress of the programme
• Work closely with business operations team and equip them for Day 1 operation
• Sign-off Business Requirement Document
• Review the e-learning course content
• Programme progress report
• Weekly status report

UAT Test Manager • Crete UAT Strategy

• Ensure effective collaboration between IT and Business BA and PMO
• Participate in requirements walkthrough meetings
• Review Effort Estimation, Test Plan
• Ensure Requirement Traceability
• Drive metrics collection to quantify the benefits derived out of the updated
testing methodology, tools and environment usage
• Master Test Strategy
• Review & approve Test Scenarios
• Review & approve Test Cases
• Review & Approve Requirement Traceability Matrix
• Weekly Status report

UAT Test Lead & Team • Verify & Validate Business Requirement against Business
process
• Estimation for UAT
• Create & Execute UAT test Plan
• Participate in requirement JAD session
• Prepare test scenarios, test cases and test data based on Business Process
• Maintain Traceability
• Execute test cases and prepare test logs
• Report defects in test management tool and manage them throughout their lifecycle
• Produce UAT End of test report
• Provide Business Readiness Support and Live proving
• Test Log
• Weekly Status Report
• Defect Report
• Test Execution Metrics
• Test Summary Report
• Archived Reusable Test artifacts

7 Challenges Of UAT And Mitigation Plan

UAT testing

It doesn't matter if you are a part of a billion-dollar release or a startup team,

you should overcome all these challenges for delivering successful software for the
end user.

#1) Environment setup and deployment process:

Carrying out this test in the same environment used by the functional test team
will certainly end up overlooking the real-world use cases. Also, crucial testing
activities like performance testing can't be carried out on a test environment with
incomplete test data.

Separate production like environment should be set up for this test.

Once the UAT environment is separated from the test environment, you need to
control the release cycle effectively. Uncontrolled release cycle may lead to
different software versions on test and UAT environment. Valuable acceptance test
time is wasted when the software is not tested on the latest version.

Meanwhile, the time required for issue tracking on incorrect software version is
high.

#2) Test Planning:

This testing should be planned with a clear acceptance test plan in the requirement
analysis and design phase.

In strategy planning, the set of real-world use cases should be identified for
execution. It is very important to define the test objectives for this testing as a
complete test execution is not possible for large application in this testing
phase. Testing should be carried out by prioritizing critical business objectives
first.

This testing is carried out at the end of the testing cycle. Obviously, it is the
most critical period for the software release. Delay in any of the previous stages
of development and testing will eat up the UAT time.

Improper test planning, in worst cases, leads to an overlap between the system
testing and UAT. Due to less time and pressure to meet deadlines, the software is
deployed to this environment even if functional testing is not completed. The core
goals of this testing can’t be achieved in such situations.

The UAT test plan should be prepared and communicated to the team well before
beginning this test. This will help them for test planning, writing test cases &
test scripts and creating a UAT environment.
#3) Handling new business requirements as incidents/defects:

Ambiguities in requirements get caught in the UAT phase. UAT testers find issues
arising due to ambiguous requirements (by looking at the complete UI which wasn't
available during requirement gathering phase) and log it as a defect.

The customer expects these to be fixed in the current release without considering
the time for the change requests. If a timely decision is not taken by the project
management on these last-minute changes, then this could lead to the release
failure.

#4) Unskilled testers or testers without business knowledge:

When there is no permanent team, the company selects UAT staff from various
internal departments.

Even if the staff is well familiar with the business needs, or if they are not
trained for the new requirements that are being developed, they can’t perform
effective UAT. Also, a non-technical business team might face many technical
difficulties in executing the test cases.

Meanwhile, assigning testers at the end of the UAT cycle do not add any value to
the project. Little time to train the UAT staff can significantly increase the
chances of UAT success.

#5) Improper Communication Channel:

Communication between remote development, testing, and UAT team is more difficult.
Email communication is often very difficult when you have an offshore tech team. A
small ambiguity in incident reports can delay its fix for a day.

Proper planning and effective communication are critical to effective team

collaboration. Project teams should use a web-based tool to log defects and
questions. This will help to distribute the workload evenly and avoid reporting
duplicate issues.

#6) Asking Functional test team to perform this testing:

There is no worse situation than asking the functional test team to perform UAT.

Customers offload their responsibility to the test team due to lack of resources.
The whole purpose of this testing gets compromised in such cases. Once the software
goes live, the end users will quickly spot the issues which are not considered as
real-world scenarios by the functional testers.

Solution to this is to assign this testing to the dedicated and skilled testers
having business knowledge.

#7) The Blame Game

Sometimes business users just try to find reasons to reject the software. It might
be their selfdom to show how superior they are or blame the development and testing
team to get respect in the business team. This is very rare but happens in teams
with internal politics.

It’s very difficult to deal with such situations. However, building a positive
relationship with the business team would definitely help to avoid the blame game.

I hope these guidelines will certainly help you to execute a successful user
acceptance plan by overcoming various challenges. Proper planning, communication,
execution, and motivated team are the keys to successful user acceptance testing.

System Testing Vs User Acceptance Testing

Involvement of the testing team starts quite early in the project right from the
requirement analysis phase.

All through the project life cycle, some kind of validation is performed for the
project, i.e. Static testing, Unit testing, System testing, integration testing, an
end to end testing or regression testing. This leaves us to understand better about
the testing performed in the UAT phase and how different it is from the other
testing performed earlier.

Though we see the differences in SIT and UAT, it is important that we leverage
synergies but still maintain the independence between both the phases which would
enable faster time to market.

Concluding Points
#1) UAT is not about the pages, fields or buttons. The underlying assumption even
before this test begins is that all that basic stuff is tested and is working fine.
God forbid, the users find a bug as basic as that – it is a very bad news for the
QA team. :(

#2) This testing is about the entity that is the primary element in the business.

Let me give you an Example: If the AUT is a ticketing system, the UAT is not going
to be about, searching the menu that opens a page etc. It is about the tickets and
their reservation, the states that it can take, its journey through the system,
etc.

Another Example, if the site is a car dealership site, then the focus is on the
“car and its sales” and not really the site. Hence, the core business is what is
verified and validated and who is better to do it than the business owners. That’s
why this testing makes the most sense when the customer is involved to a major
extent.

#3) UAT is also a form of testing at its core which means that there is a good
chance of identifying some bugs at this phase too. It sometimes happens. Aside from
the fact that it is a major escalation on the QA team, the UAT bugs usually mean a
meeting to sit and discuss how to handle them as following this testing there is
usually no time to fix and retest.

The decision would be either to:

Push the go-live date, fix the issue first and then move on.
Leave the bug as it is.
Consider it as a part of the change request for future releases.
#4) UAT is classified as Alpha and Beta testing, but that classification is not
that important in the context of typical software development projects in a
service-based industry.

Alpha testing is when UAT is carried out in the software builder’s environment and
is more significant in the context of commercial off the shelf software.
Beta testing is when the UAT is carried out in the production environment or the
client’s environment. This is more common for customer-facing applications. The
users here are the actual customers like you and me in this context.
#5) Most of the times in a regular software development project, UAT is carried out
in the QA environment if there is no staging or UAT environment.

In short, the best way to find out if your product is acceptable and fit for
purpose is to actually put it in front of the users.

Organizations are getting into the Agile way of delivering, business users are
getting more involved and the projects are being enhanced and delivered via
feedback loops. All being done, User Acceptance phase is considered as the gate for
getting into implementation and production.

But of all forms of testing, user acceptance testing is often the most essential to
get right. Why? Because when implemented correctly, it’s the most effective in
reducing both time and cost, whilst increasing customer satisfaction.

User Acceptance Testing, also known as UAT (or UAT testing), in a nutshell, is:

A process of verifying that a solution works for the user.

And the key word here, is user. This is crucial because they’re the people who will
use the software on a daily basis. There are many aspects to consider with respect
to software functionality.

There’s unit testing, functional testing, integration testing, and system testing,
amongst many others. As part of these processes, we regularly ask questions such
as:

Does the application crash?

Do all the functions accept the correct inputs and give the correct outputs?
Does the application consume only the minimum amount of resources?
What’s the load time of the application?
Whilst these are valid and essential, ultimately they’re quite meaningless – if the
application doesn’t perform as the end user expects. As software artisans, we need
to avoid situations such as:

Can the user use the software?

Is it really what they asked for?
Do they have trouble using it?
Does it behave exactly as anticipated?
What Is User Acceptance Testing?
I’ll keep it simple; according to Techopedia, UAT (some people call it UAT testing
as well) is:
User acceptance testing (UAT) is the last phase of the software testing process.
During UAT, actual software users test the software to make sure it can handle
required tasks in real-world scenarios, according to specifications. UAT is one of
the final and critical software project procedures that must occur before newly
developed software is rolled out to the market.

User acceptance testing (UAT), otherwise known as Beta, Application, or End-User

Testing, is often considered the last phase in the web development process, the one
before final release or installation of the website or software for the client, or
final distribution of it.

UAT is the usage of the software by people from the intended audience and recording
and correcting of any defects which are discovered. It’s the closest thing to a
“_real world_” test available. It gives users the chance to interact with the
software and find out if everything works as it should if features have been
overlooked, miscommunicated, not communicated, and so on.

DevelopMentor puts it most succinctly when they describe user acceptance testing
(UAT) as:

The goal of User Acceptance Testing is to assess if the system can support day-to-
day business and user scenarios and ensure the system is sufficient and correct for
business usage.

When Should You Start User Acceptance Testing

Whilst UAT – User Acceptance Testing – is essential, typically, it’s not able to
be undertaken until the application is largely feature-complete. Guru99 lists 10
prerequisites, which must be met before UAT can begin. These are:

1.Business Requirements must be available

2.Application Code should be fully developed
3.Unit Testing, Integration Testing & System Testing should be completed
4.No Show stoppers, or High or Medium defects in the System Integration Test Phase
5.Only Cosmetic errors are acceptable before UAT
6.Regression Testing should be completed with no major defects
7.All the reported defects should be fixed and tested
8.Traceability matrix for all testing should be completed
9.UAT Environment must be ready
10.Sign off mail or communication from System Testing Team that the system is ready
for UAT execution
Recommended Reading:

Who should be involved in User Acceptance testing?

The most important peer group to include in UAT testing are “real” end users of
your software. Every role and stakeholder group should be included which means that
people from each group should be selected to join the UAT team.

How To Get Started With User Acceptance Testing

Now that we’ve laid the groundwork for what UAT is and why it’s essential, let’s
finish up by seeing how to get started. Normally, UAT consists of four steps. But
it can vary, based on whether the application is being delivered to a single
customer, or whether it’s intended to be off-the-shelf software, available for
purchase by anyone.

Firstly, the criteria which the software is considered to be “working” needs to be

assembled. These are likely to be collated from the system requirements, and user
stories. Next, a set of UAT test cases must be created. Centric defines a UAT test
case as:

A set of test steps, execution conditions and expected results developed for a
particular objective, such as to exercise a particular program path or to verify
compliance with a specific requirement

Each case covers a specific usage scenario of the software. It is normally a set of
actions which the user can carry out and be able to verify if the software’s worked
as intended.

With these in place, the tests then have to be run and the results recorded. Were
the tests successful, or did defects result? Any bugs then need to be corrected and
re-tested.

Finally, assuming that everything is working as expected, an orderly sign-off needs

to be completed. This is done with your client or the team they have assembled for
the project, where they state that what they’ve received works as expected and
meets their criteria.

User Acceptance Testing framework

User Acceptance Tests might not be called User Acceptance Tests in your
organizations. There are various buzzwords – such as alpha or beta testing – out
there. And sometimes we also get ask about the differences between UAT and
functional testing.

Therefore we decided to collect all our thoughts and knowledge on the different
types of UAT in this article.

Conclusion
And that’s what, why, and how of integrating User Acceptance Testing as a standard
part of your web development projects. It reduces the likelihood of issues being
raised, which in turn reduces the amount of work required in development and
maintenance.

Sure, it’s another process which you have to manage, but the reduction in overall
cost and a higher level of user satisfaction more than offsets the associated
costs.

6 Tips For a Successful User Acceptance Testing Plan

A while ago, we published an article on “What User Acceptance Testing is all about”
and a follow up article on “5 UAT Testing Types”. Since then, we got a lot of
feedback from users and people asking for further advice on the topic of UAT.

Therefore we decided to sum up all those inquiries and answer the following
question: ”What’s the key to a successful User Acceptance Testing?”

We put our heads together and collected the following six tips for you. Enjoy
reading & have fun executing your next User Acceptance Test Plan.

A quick summary on User Acceptance Testing

User Acceptance Testing is one of the last steps in a software development process.

It’s the big test before going public. Sometimes User Acceptance testing is
referred to as “beta testing”. As we’ve shown you in this article on the 5 types of
UAT, you probably know that there’s more than beta testing to UAT.

All in all, UAT is about the user and if a certain product or service works for the
user.

A process of verifying that a solution works for the user.

1. Plan your User Acceptance Testing efforts

As with every project, planning is everything. A well-structured UAT plan is key.
Therefore we collected a few questions you need to answer before getting started:

Which of your employees and project members are involved in the UAT project phase?
Who is responsible for which tasks? Who’s doing the strategic planning? Who’s
managing the logistics?
Who is responsible for the operative UAT test execution?
Work on your test criteria before starting to execute your plan. And work on a
detailed User Acceptance Testing plan.
We’ve collected all those test criteria and created this UAT checklist for getting
started with your test plan.

2. User Acceptance Tests are nerve-racking

As mentioned before, UAT tests are the last project phase in every software
development. At least, they should be. In many real projects those UAT tests are
overlooked. Mostly because of time restrictions and a lack of knowledge.

Sometimes those User Acceptance tests are well executed. But then, those results
show some major defects which can’t be fixed without delaying the project
deadlines.

Depending on your project size, it definitely makes sense to include your potential
users right from the beginning. In order to avoid developed features which users
won’t get, you can include them in an agile software development process.

3. Your user is your central hub

As the headline suggests, you should always put your users first. Especially when
it comes to User Acceptance Testing.

When analyzing testing results, you should always consider the characteristics of
your users too. Don’t simply perform UAT tests with friends, colleagues, and
family.

Make sure to get real users. Potential users should take a key role in User
Acceptance Tests.

So how does one select users for a User Acceptance Test group? Make sure you get a
diverse set of people which have not only different backgrounds and skills but also
vary in their use cases and problems they have.

4. Documentation is key.
Conducting User Acceptance Tests is a great thing to do. Yep – it is.
However, you won’t believe how many organizations perform UAT, though forget to
document testing results.
User Acceptance Testing Plan

A test only makes sense to be conducted, if you document the outcome in order to
analyze and improve what you have tested.

There are various ways to document your UAT results. The easiest form of
documenting the errors and failures your users encounter is by making use of a
simple bug tracking and error reporting tool.

Everything must be documented. Even if it looks just like a tiny bug. It could be
useful for a later test analysis.

5. One-on-one sessions
User Acceptance Test outcomes should always include qualitative data about the
user. And the easiest way to collect this data is by one-on-one sessions. If you
can’t be physically in the same room as your testers, make sure to set up a video
call for the test performance.

Emotional expressions, reaction time and the overall mood is essential when
analysing the UAT test results.

And most importantly, record those video calls and tests.

6. Make communication easier.

A UAT team is embedded in a larger project or product team. And since those UAT
teams are working remotely in many cases, you need to find an efficient form of
communication.

User Acceptance Testing Team

Emails are highly inefficient for UAT communication. Communicating with users,
testers, developers and other key players via email will drive you nuts.

We eat our own dog food, which means that we use our own software testing tool for
UAT purposes. It enables us to execute UAT tests and communicate with UAT testers
throughout the project phase via this feedback tool.

Wrapping it up.
User Acceptance Testing plays a crucial role in every software development
department. The benefits of well-executed UAT tests are obvious. You can ensure
that your product or service actually works for your users.

Therefore launching your product isn’t a leap into the unknown. When conducting
User Acceptance Tests you have already gathered a lot of information on how your
potential users and customers will use your product.

And most importantly, you will avoid such stressful situations when going live:
What is User Acceptance Testing, Why is it required, and How to do it?

What Is User Acceptance Testing?

User acceptance testing or UAT is a type of validation which ensures that the
product or the solution works for the user and meets all his/her requirements.

It is usually the last step in the Software testing process. And the real Software
users carry out this activity to certify whether the product has all intended
functionality or not.

The standard definition of UAT also states the same.

User acceptance testing decides the fate of the Solution and hence becomes the most
critical step in the product development/testing.

The word “user” in the UAT represents the client or a member of his team or a group
of professionals authorized for performing the testing.

The UAT is primarily to assert that the final solution delivers to the expectations
of users. Also, it confirms the application is providing an excellent end-to-end
user experience.

However, it is imperative that UAT might reveal some issues or new requirements
which need to be fixed or implemented. In such cases, the product goes back to
development based on the UAT feedback.

Whether the product is final or not would depend on the approval from the
designated stakeholders at the customer end.

User Acceptance Testing

Why Is User Acceptance Testing Required?

While a product is going through the development phase, it also has to pass through
the different levels of Software testing. Both the developers and the testers
perform validation activities. Out of these, user acceptance plays a vital role in
determining the approval of a solution before delivering it to the customer. Here
are a few important reasons to do it.

Reason-1:
It is to confirm that the new features are working correctly or bug fixes are
getting fixed. But sometimes, they could use workarounds to ignore an issue which
could hide another real problem to get discovered later.

Reason-2:
After spending so many efforts on testing the product, there are still chances the
team might miss a few areas due to the use of workarounds or the shortcuts for
speeding up the whole process. Also, the developers and testers are professionals
for whom a few execution steps could not matter but not the same case for the end
users.

Reason-3:
Most of the end users are not proficient in using complicated software but knows a
part of it quite well which they handle. Also, they may concern how an application
or a new feature would behave. So, they can validate the new features or a product
with a fresh mindset. They can go on testing the product with a non-evasive
approach keeping focus on the quality and user friendly-ness. Hence, you can think
of user acceptance testing as a tool to determine the product behavior in standard
conditions.

Reason-4:
Moreover, there could be a situation where the development team missed to add some
of the requirements or implemented incorrectly. Such a case may arise if the PM
(product manager) is inefficient, doesn’t interact with the team on a regular
basis, or doesn’t participate in user stories demo. A good PM will always make sync
with the team on what the real requirements are and how they are getting
implemented. Anyways, user acceptance testing is an ideal approach to identify and
spot such differences. The end users are the first to catch and report these
discrepancies if there are any.

What Are The Differences Between User Acceptance Testing Vs. Functional Testing?
Since both, the above validation methods test a Software against a set of
specifications, so it is customary to ask the difference between the two.

The user acceptance testing targets to confirm whether the product works as per the
specific customer requirements or not. The UAT test plan should be ready while
setting up the development agreement with the customer. Once the test cases for UAT
are available, the work can start.

On the contrary, the functional testing targets the feature-level requirements

while taking care of various other aspects such as support for multiple browsers
and platforms, backward compatibility, etc. The objective of a functional test plan
is to confirm that the Software shall comply with the specifications. It may
overlook the user element from testing.

How To Perform User Acceptance Testing Efficiently?

In the previous sections, we’d explained about the UAT and the primary reasons to
use it. The next part is to understand the right approach to conduct user
acceptance testing. We’ll discuss it in detail and will also guide you to prepare a
UAT template to do it efficiently.

I) Define Users And Their Roles

It is essential to define the users and roles before you start user acceptance
testing. There may be different sets of functions for distinct products and
solutions. The users should also get the permissions based on their roles. Also,
you must have UAT test cases ready specific for each user.

II) Story Based Test Case Creation

PO (product owner) creates user stories for customer requirements. If they have
enough details inside the US, then it’s easy for QA to define test cases. PO should
also mention the acceptance criteria for each user story. It would help to make
sure the test coverage for the user stories.

III) Focus On Functionality Rather Than Technicalities

The users who perform user acceptance testing are no expert like the real testers.
They can’t understand if the test cases are too complicated. Hence, it is necessary
to use more business specific languages to make UAT efficient.

IV) Product Training

It is essential to train end users before they jump on to UAT. You should have
enough sessions planned to give them practical experience of using the solution. It
would encourage and generate confidence amongst the users.

V) Use Of An Optimized UAT Template

You need to provide a user acceptance testing template to end users. It’ll help
them become a little organized while executing tests.

VI) The Right Time To Start UAT

If you know the right time to start UAT, then it can be more effective and produce
desired results. Since the objective of user acceptance testing is to confirm the
requirements as per user’s acceptance, so it is recommended to begin once all of
them get implemented. Another approach is the iterative method which requires you
to decide the features to be part of an iteration. It is a more practical approach
and increases the chances of getting an early approval.

How To Create A UAT Template?

A UAT template can play a significant role in improving the output from the UAT
team. Here, we are outlining the steps to create a generic document. Please do
discuss with your groups to make any enhancement or changes.

I) Unique Identifier
Users should be easily able to identify test cases. Hence, you must assign a test
case ID which a user can easily distinguish. You may try to adopt the following
pattern.

Test case Id: <Module Name>_<Sub Module>_<Sequence No>

e.g. PAYMENT_COD_1101, ELEC_MOBILE_1001

II) Function-Based Groups

Split the UAT test cases based on the functionality or the module. For example, if
a product is an e-commerce website, then it could have a payment module with many
sub-modules like cash on delivery, credit card, net banking, etc.

Distributing tests in such a manner makes it a lot easier for the users to test
more accurately. However, follow this approach only if the product is big with many
features. Otherwise, it may lead to extra efforts without yielding any real
benefits.

III) Use Of Customer Request Identifier

Use this field to mention the customer requirement or the no. of a business
request. Also, it can point to a reference such as a link or a document containing
functional details. The users can utilize this information to focus on the main
aspects and clear any doubts on the testing requirements.

IV) Segregation By Role

If the users execute tests for user stories respective to their responsibilities,
then they are likely to perform better. Segeration of modules will help them focus
on specific functionality, and they are likely to reveal any deviation in the
application from the expected behavior.

V) Define Execution Steps

The UAT template must include a dedicated field giving details of the execution
sequence of the test cases. The step definition should convey the expected behavior
rather than the technicalities.

VI) Expected Output

It should represent the results expected from test execution. The user must
validate his result with the one mentioned under this field. It will decide whether
the test will either pass or fail.

VII) Actual Output

The user shall report the outcome of the tests after executing all the steps
belonging to the cases. If the results are matching the expected output, then they
can write either “Expected” or “No deviation.”

Otherwise, they should make it more descriptive by adding the details of the
failure.

VIII) Test Case Status

This field indicates whether the test has passed or failed. In case of failure, you
must file a bug report and assign to the developer or the scrum master for a fix.

IX) Test Case/Requirement Severity

Not all features have the same gravity as some may block the functionality and some
may not. Hence, it is better to define at first hand. After that, the users may
prioritize execution accordingly.

X) Remarks
The UAT template must have a provision to add comments or any relevant details
related to the requirements.

What Are The Actions Planned For UAT Feedback?

The users submit their final reports after completing the UAT. The feedback may
result in any of the following actions.

Release the solution

Bug/defect fixing
Implement change requests
New requirements
I) Release The Solution
The feedback is positive, and the product behaved as expected. Hence, the users
shall recommend to release it to the customers.

II) Bug/Defect Fixing

There were bugs found during the user acceptance testing. The scrum master or the
development manager should plan their fix.

III) Implement Change Requests

The solution didn’t behave as per the specification. Apart from the human error, it
could also be due to a new platform introduced where the product didn’t work as
expected.

IV) New Requirements

In some cases, where the user felt the feature is working as expected but lacking
in usability, he may prompt to ask for a new feature addition. Also, it differs
from the change request as the following result in re-implementation instead of a
new one.

The Science of Running Effective User Acceptance Testing Cycles

User Acceptance Test (UAT) programs have traditionally been areas of contention
between IT and the Business. IT teams get critical systems readied through
development and testing, while Business teams verify that these systems meet their
requirements. The division of responsibilities might seem clear cut, but realities
on the ground are far different. The planning, management, execution and reporting
phases of UAT cycles have always been problematic since responsibilities fall in
grey areas.

This whitepaper will outline a well-defined 5-Phase UAT process that assists with
Planning, Coverage, Execution & Tracking, Reporting, and Reuse.

Planning
An important truth for any process where quality must be determined is the fact
that the earlier an issue is found, the less expensive it is. This includes UAT as
well. Traditionally, in waterfall methodologies, UAT doesn’t occur until later in
the cycle closer to the delivery date. Even today, this practice exists by default
within organizations. The risk with this approach is simple: wait until the end
game to discover that the requested functionality was misunderstood by development
teams and the costs for fixing before release becomes much higher, risk is
accentuated, and quality is often sacrificed.

UAT should occur early and often. Regardless of process, the planning phase for UAT
needs to be included in the master project plan and scheduled across the life of
the project. This needn’t be complex or difficult to achieve - nor does the actual
testing process need to take a long time. For example, XYZ Company practices Agile
and schedules UAT every two sprints. It lasts for 1/2 a day or less, and provides
feedback soon enough for the process to flex with changes. Company ABC takes a
different approach and has a business representative participate in the agile team
who verifies user acceptance each sprint within the agile process. UAT is scheduled
every three sprints and is conducted by a team of analysts as well as two external
customer representatives in order to assure the project remains on track.

Environments are another key factor in UAT testing: Where is this test going to
occur? Steer clear of utilizing development and QA environments – these change
frequently and any disruption in the development process could stress the schedule.
Ideally, a separate environment for UAT would work best although a shared 'demo'
environment can be used as well as an unused staging environment. Regardless of
where the testing will take place it is crucial to begin setting this up well in
advance of the UAT to be sure everything is working. Finally, this goes without
saying: Always, always, always have QA do a full smoke test on the UAT environment
at least a day prior to the customer's access. This will give time to correct any
inconsistencies that may be found.

Planning Checklist:
Determine the point person for directing UAT testing. This can be someone from the
business, a technical project manager, a QA lead, etc

Form a UAT team. A mix that works well is a combination of all three from the above
(or more).

Collect the requirements that will be verified. If using an Agile process, the
Acceptance Criteria usually provide the necessary information.

Assure UAT testing falls at an earlier point on the project schedule. This does not
negate a final UAT which is often required by the customer.

Identify, build, and verify the environment well before the testing date.

Planning Checklist

HOW ZEPHYR CAN HELP WITH PLANNING:

Zephyr’s Test Repository and Scheduling applications make it very easy to identify
areas of test and assign and schedule those specific tests in UAT cycles. The
Business may or may not be involved in the direct creation of test cases in this
phase. If UAT checklists or tests already exist, they can be easily imported into
Zephyr and tracked. The planning and scheduling process is real-time and is visible
to both teams. User acceptance tests are very easily authored in the Test Case
Creation application that has an Excel-like feel to it, thereby allowing non-IT
resources to easily interact with it – be it for actually creating tests or merely
reviewing and annotating them.

Planning

Coverage
User Acceptance Testing is often confused with a ‘regression by client.' This often
occurs because expectations haven't been clearly understood or communicated
throughout those involved in the process (including the customer). UAT is defined
as the process whereby the customer verifies requirements that have been requested
exist and provide the functionality as outlined in the user story or requirements
document. Hence the term 'Acceptance.'

UAT should not be considered to be a functional regression of the software or a

time to change requirements and log the changes as bugs. However, always remember:
Bugs will likely be uncovered, especially if the UAT falls earlier on the schedule,
and the customer often has a better idea of what they really want once they see the
functionality. For any project it is important to mitigate both of these
challenges. It is best to have a QA lead involved in the UAT process as a point
person to receive any issues that are found in the software. The lead can then
review the list for known issues and add any new ones as bugs. Managing requirement
changes is a function of the project team – they determine what the process is for
those changes. If using Agile, a common strategy is to introduce the stories into a
sprint where appropriate and negotiate schedule overruns with the customer.

Given the above, it is important to clearly define the requirements that will be
reviewed by the customer. If using Agile, either the stories or the acceptance
criteria will provide the necessary information. In other processes, any document
or system which identifies and tracks requirements against development will provide
the required coverage specifics.

It is also helpful to have a set of test cases to track against during testing.
These don't need to be full blown, step-by-step instructions in most cases but they
are useful for encapsulating the requirements and directing folks through the
application. Also, using a test management system to track the execution and
progression of testing can be a valuable exercise both during the testing and as a
historical document to refer to, should questions arise in the future around what
exactly was looked at.

Coverage Checklist:
Set clear expectations around desired outcome for UAT testing.

Involve QA to assist with capturing issues.

Identify the requirements to be reviewed. Stick to these only!

Capture requirement changes and fit into project as warranted while keeping an eye
on project slippage and delivery schedules.

Use a system (such as Jira, GreenHopper, or a robust test management system) to

track requirements under test.

Write test cases which encapsulate the requirements that need to be reviewed. These
need not be complicated or indepth for most UAT testing however, be sure to provide
enough information so the testers, whether they be internal or external customers,
can navigate and understand what is being asked.

Coverage Checklist

HOW ZEPHYR CAN HELP WITH COVERAGE:

Test Coverage of the various features and functionalities that have been added to
the new systems is very important. The Business needs to get a good feel - through
both quantitative and qualitative means – that the system indeed does meet their
requirements. Zephyr’s Requirements Management application in conjunction with the
ability to directly map UAT testcase(s) to those requirements allows for real-time
coverage reports and tracking metrics. Very simplified click-n-select processes
allow for such mapping. End-to-End Requirements traceability is also available.

Coverage

Execution & Tracking

There are several ways to perform User Acceptance Testing: Hands on in-house (lab,
training room etc.) and remote. In some cases, if there is no way for the customer
to interact with the application, an in-depth walk-through often suffices. The
execution phase often holds the most anxiety of the UAT process. What if the
customer doesn’t like it? What if there is a major issue? And so it goes – that
nagging fear of software development. This is the phase within which to relax. If
you’re not prepared for the UAT and have questions as to whether things will go
wrong then cancel it! The execution phase is an opportunity to get closer to your
client – whether external or internal. Full disclosure is important in UAT. If
there is a major bug, don’t hide it – discuss it and work around it. Issues during
the software development process are normal and expected and helping the client
understand this will go a long way in building a trusting, positive relationship.

The team for UAT testing should include at minimum a business representative
(Product Owner etc), Technical Project Manager, and a Quality Assurance
representative (preferably the lead QA engineer for the project). It is also
recommended to have a developer and network admin on-call for unintended
occurrences (and anyone else you identify as important). You will know the best
make-up for your team given your process, product, etc.

As mentioned earlier, it is helpful to have basic test cases written and available
in a test management system that tracks execution. They should be organized under
UAT and kept separate from on-going project work. The customer can either interact
with the test management system or the execution can be tracked by a member of the
UAT team. Regardless, a historical record of test case executions should be
recorded and saved. This does not negate the importance of allowing customers to
interact with the system in an organic natural way (black box testing) but rather,
provides a framework within which to establish the guidelines for the UAT testing
to assure all requirements have been reviewed. Always remember: Clear expectations
are key for a successful UAT.

UAT can be performed from anywhere. Customers can visit and use local machines or
their laptops, the application can be made available over the internet, VPN
privileges can be given, etc. The best place for UAT is determined by multiple
factors and must be addressed by the project team. Regardless of test location, all
of the recommendations in this paper can be utilized. Finally, assure you have the
full support of your networking team prior to any UAT execution – especially if you
are performing it remotely.

Execution and Tracking Checklist:

Determine the location for testing: in house, internet, VPN etc. and assure network
support is available prior to testing starting.

Assure that you are prepared for the customer to view the application. Bugs are OK
but be sure the client is aware.

Use UAT to build relationship and trust with the client.

Define a dedicated SQA role and either have them present at the event or
participating remotely. They will be invaluable for clearing up questions regarding
issues, functionality, etc.

Assemble a technology team who can be on-call to support the event. A development
and network engineer are a good start.

Track test case executions or the areas being looked at and record all results.

Coverage Checklist

HOW ZEPHYR CAN HELP WITH EXECUTION AND TRACKING:

UAT execution in Zephyr is highly simplified to allow non-technical domain experts
and business users to easily run them. Akin to performing simple everyday tasks
such as using a Web Browser or Excel spreadsheets, the Test Case Execution
application in Zephyr is accessed via a web browser. The user logs in with a
username/password provided to them, clicks on the Test Case Execution application
and clicks on the UAT Cycle. A list of user acceptance tests is presented to them.
On selecting one of them, the detailed steps to perform are shown. The user can
then (with a simple drop-down menu) “pass” or “fail” test steps and/or entire tests
and type in their comments. They then move on to the next test. That’s it. They
don’t have to fill out any spreadsheets, or keep track of where they left off the
execution or write reports.

Execution and Tracking

Reporting
Two underlying themes throughout this paper have been communication and setting
expectations. A successful UAT does not ignore the Reporting phase if it seeks to
assure that the customer is happy and everyone has agreed with the results of the
testing. The Reporting phase encapsulates and communicates the degree to which
expectations were met. It also provides a historical document from which to
mitigate future questions or issues that may arise. The report will need
contributions from the business as well as SQA and any other project participants
deemed necessary.

The UAT report can include as little or as much information that it needs in order
to fulfill expectations. A report for an external client may consist of more
information than that of one used internally because internal systems hold much of
the historical information. However, regardless of client orientation, it is
advisable to always document the results.

Recommended content for the UAT Report:

Overview

Requirements/stories under test

Test cases

Test Metrics - # tests executed, #passed, #failed, pass/fail percentage, who

executed tests

Testing results

Free-form testing results (black box testing)

Observations

Requested changes

Requested additions

HOW ZEPHYR CAN HELP WITH REPORTING:

The Zephyr systems monitors all test activity in real-time and presents the rest of
the project team and upper management with this information in live Dashboards.
Live Dashboards show details about the UAT execution schedule, project news and
detailed metrics about how well the UAT cycle is progressing in realtime. They show
progress, pass and failure rates, areas of risk, defects and provide overall
quality insight. All of this without having to build reports, run queries, hound
the non-IT users for updates or do a ton of paperwork.

Reporting

Reuse
Most projects will require more than one UAT, especially if the ‘test early and
often’ approach is used. Remember, the earlier an issue is found in any project,
the less expensive it will be to fix. This holds especially true when UAT issues
are discovered late in the process close to the delivery date. When subsequent UAT
tests are performed, it is best to test previously reviewed functionality as well.
To maintain consistency, it is advised to utilize the same test cases that were
executed in previous UAT iterations. Not only will this reverify customer
acceptance, but it also builds trust by showing that a consistent quality system
and process is in place. It is advised that a system which allows a new iteration
to be created and run is used so results are stored separately from previous
sessions and customers see unexecuted test cases.
Reuse Checklist:
Use a system which will allow new test iterations to be created with previously
executed test cases

Re-test previously tested functionality as well as new

Build confidence and trust with customers by presenting an organized test process
and system

HOW ZEPHYR CAN HELP WITH REUSE:

Once the UAT cycle is completed successfully, none of this information is thrown
away but is recycled for the next UAT cycle. Tests, schedules, reports, metrics are
all reused and modified appropriately for a new cycle. This tremendously speeds up
successive UAT cycle planning and execution. All historical information about past
UAT cycles is stored and available immediately for instant comparisons and reuse.

Reuse

User acceptance testing is a valuable process for any project. Whether it is an

internal or external facing application, it is important to verify that the
application meets the expectations of the end-user. Contention between the business
and development organizations need not exist and can be mitigated by advanced
planning, setting expectations, and open, clear, concise communication. Forming a
small team of individuals from various departments will create ownership across the
organization and, if executed well, spread the burden of the UAT event among
multiple individuals. UAT need not be a negative, time consuming process.

Using systems and a simple process will add to the ease of UAT execution. Storing
and executing predetermined test cases from a test management system not only lends
a core stability to the process, it also demonstrates a high degree of organization
and consistency around the quality process; something the customer will notice and
appreciate. After all, UAT is all about the customer. Reframe UAT from a ‘must do’
process to an opportunity to further engage the customer in a positive, trust-
building relationship. As outlined in the previous pages, UAT is not complicated
and doesn’t need to take a lot of time to prepare for. Start early, form the
correct team, outline and schedule tasks, set clear expectations around coverage,
utilize test management and process technology to provide a solid core for the
project, track results, and provide a full, detailed report. All that remains is
engaging the customer in high integrity communication throughout the process and
using the experience to build a stronger, more trusting relationship.