Unit-6 Acceptance Testing

Dr. Jigna Solanky

Acceptance Testing

 Extension of system testing

 When QA team is ready with the product, they invite customers for
demonstration

 The testing done to accept a product is known as acceptance testing.

 Testing carried out by the customers or persons authorized by the

customers.
 Testing Venue: Developer’s site or customer’s site (depends on mutual
agreement)
 Generally venue is customer’s site.
Cont..

 It is carried out only when the software is developed for a particular

customer.
 If we develop ‘standardised’ software for anonymous users at large (like
operating system, compilers, case tools etc.), then acceptance testing is
not feasible.

 Acceptance testing, a testing technique performed to determine

whether or not the software system has met the requirement
specifications.
 Acceptance testing is done by the customer for their satisfaction, and
check whether the application is working according to given business
scenarios, real-time scenarios.
Why we use acceptance testing?
 To gain confidence in the product that is getting released to the market.
 To ensure that the product is working in the way it has to.
 To ensure that the product matches current market standards and is
competitive enough with the other similar products in the market.
Types of acceptance testing
1) User Acceptance Testing
2) Business Acceptance Testing
3) Contract Acceptance Testing
4) Alpha Testing
5) Beta Testing
6) Regulations Acceptance Testing
7) Operational Acceptance Testing
Cont..

 User Acceptance Testing

 This is the final testing performed by the end-users to ensure the system
meets their requirements.
 It's typically done in a controlled environment that mimics production.
 Business Acceptance Testing
 Focuses on testing the system from the perspective of fulfilling the business
goals and ensuring the software aligns with business processes.
 Contract Acceptance Testing
 Verifies that the product meets the contractual obligations set by the client or
regulatory bodies.
 This ensures the project delivers everything agreed upon in the contract.
Cont..
 Alpha Testing:
 Done by some potential customers at the developer’s site under the direction
and supervision of developers, Testers.
 Key Characteristics:
 Internal Testing: Performed within the organization, often in a controlled
environment.
 Focus on Functionality: Aims to test the software's functionality and usability to
ensure it meets the requirements.
 Feedback Loop: Testers provide feedback for immediate fixes, allowing for rapid
iterations.
 Limited User Involvement: This usually involves a small group of users, such as
employees or stakeholders, rather than a large audience.
 Purpose:
The main goal of alpha testing is to catch bugs and issues early in the
development process, ensuring that the product is stable and ready for
external testing.
Cont..
 Beta Testing:
 Done by many potential customers at their sites without any involvement of
developers / testers.
 It helps identify bugs that were not discovered during alpha testing and provides
real-world feedback on usability.
 Key Characteristics:
 External Testing: Conducted by actual users outside the organization, often in real-world
environments.
 User Experience Focus: Aims to evaluate the software's performance, usability, and
functionality based on real user interactions.
 Feedback Collection: Testers provide feedback on bugs, usability issues, and feature
requests, which can lead to further improvements.
 Broader User Base: Involves a larger group of users, often including both experienced
and novice users.
 Purpose:
The main goal of beta testing is to validate the software in a real-world setting,
identify any remaining issues, and ensure it meets user expectations before the final
release.
Cont..

 Regulations Acceptance Testing (RAT):

 RAT is used to determine whether the product violates the rules and
regulations that are defined by the government of the country where it is
being released.
 This is important in industries like healthcare, or finance
 RAT, as different countries or regions have different rules and regulations
defined by its governing bodies.
 If any rules and regulations are violated for any country then that country or
the specific region then the product will not be released in that country or
region.
 If the product is released even though there is a violation then only the
vendors of the product will be directly responsible.
7) Operational Acceptance Testing (OAT):
 Also called Production Acceptance Testing, this is the testing done to
verify the system's operational readiness.
 It focuses on things like backup/restore, disaster recovery,
compatibility, reliability and maintenance tasks.
 OAT assures the stability of the product before it is released to
production.
 Example: An e-commerce company performs OAT on its new website
to ensure it has robust failover mechanisms, performs scheduled
backups, and can recover data in case of server crashes or disasters.
User Acceptance Criteria

 User Acceptance Criteria are a set of predefined requirements or

conditions that the software product must meet before it is accepted by
the user or customer.
 They define success for a particular feature or the entire system from
the end user's perspective.
Example
 E-commerce website that allows users to purchase items online.
 Feature: Add to Cart Functionality
 User Acceptance Criteria:
 When a user clicks the "Add to Cart" button on a product page, the
product must be added to the shopping cart.
 The cart should update in real-time, showing the number of items and
the total price.
 The user should be able to view the cart at any time by clicking the cart
icon.
 The cart should persist even after a user navigates away from the page
or refreshes the browser.
 If the user is not logged in, the system should prompt the user to log in
or create an account before proceeding to checkout.
Why UAC is Important?

 Aligns with business goals: UAC helps ensure that the development
team knows exactly what the customer expects from a feature.

 Measurable success: It provides clear, measurable conditions that the

software must meet for it to be accepted.

 Prevents misunderstandings: UAC helps avoid ambiguity, ensuring

the product delivers what the end user wants, and the development
team knows what to build.
Test Criteria Identification
Entry Criteria
 Business requirements should be clear and available.
 The system and Regression testing phase should be completed.
 All the Critical, Major & Normal bugs should be fixed and closed
 Known issues list should be prepared and shared with the
stakeholders.
 Acceptance Test Bed should be set up and a high-level check should
be performed for no environmental issues.
 The system Testing phase should be signed-off letting the product
move to the AT phase (Usually done through Email communication).
Exit Criteria
 Acceptance tests should be executed and all the tests should pass.
 No Critical/Major defects left Open. All the defects should be fixed
and verified immediately.
 AT should be Signed-off-by all the included stakeholders with Go/No-
Go Decision on the product.
Go decision will take the product ahead to be released to
the market.
No-Go decision marks the product as a Failure.
Acceptance Test Checklist Preparation
 Checklist preparation involves creating a list of tasks, requirements, or validation
points that must be completed or verified during software testing to ensure the
product is functional, reliable, and ready for release.
 A checklist provides a clear, step-by-step approach to testing features,
processes, or systems.
 It is important to ensure that the following stages and their test activities are
covered as part of the User Acceptance Testing to ensure optimum results from
UAT.
1) Initiating the User Acceptance Testing project
2) Planning the User Acceptance Testing
3) User Acceptance Testing Design
4) User Acceptance Testing Execution
5) Release Decisions
6) Post User Acceptance Testing Actions
1) Initiating the User Acceptance Testing project
This stage involves defining the overall scope and objectives of the UAT.
It is the starting point where stakeholders, end-users, and testers
understand what is expected from UAT.

 Identify key stakeholders and end users who will be involved in UAT.
 Define the business requirements and ensure all acceptance criteria are
documented.
 Ensure that the environment where UAT will be conducted is ready.
 Set clear goals and timelines for UAT completion.
 Prepare documents such as user manuals and training materials if
needed.
2) Planning the User Acceptance Testing
 This step involves creating a detailed plan for the UAT process,
including resources, timelines, and tools that will be used.

 Checklist Preparation:
 Ensure all test cases are derived from business requirements and user stories.
 List out the resources needed (e.g., testers, test environments, and data).
 Prepare a schedule and milestones for UAT.
 Plan for communication with stakeholders throughout the testing process.
 Ensure the tools for reporting bugs and progress are in place.
3) User Acceptance Testing Design

 This step focuses on creating test scenarios and test cases based on
the user acceptance criteria.
 It involves designing what will be tested and how.

 Checklist Preparation:
 Prepare test cases that align with the acceptance criteria defined in
earlier stages.
 Prioritize test cases based on critical functionality and business impact.
 Ensure test cases cover both positive and negative scenarios.
 Prepare test data that accurately reflects real-world usage scenarios.
 Identify any automation scripts or tools that will assist in the testing
process (if applicable).
4) User Acceptance Testing Execution

 This is where the actual testing happens.

 Testers execute the UAT test cases and verify if the system meets the
user acceptance criteria.

 Checklist Preparation:
 Ensure that the environment is stable, and test data is available before
execution begins.
 Monitor and document all test results clearly, including pass/fail status
and any issues encountered.
 Report bugs or defects immediately to the development team.
 Verify if all acceptance criteria are met during the execution phase.
 Ensure there is a process for re-testing after bug fixes.
5) User Acceptance Testing Release Decisions

 After testing is complete, a decision is made on whether the software

can be released.
 This step involves validating if the software meets all business and user
requirements.

 Checklist Preparation:
 Review all test results and confirm that all critical test cases passed.
 Ensure that all identified bugs and defects are resolved or documented
for post-release fixes.
 Gain formal approval from stakeholders to release the software.
 Confirm that the product is ready for deployment, and all necessary
documentation is in place (e.g., release notes, user guides).
6) Post User Acceptance Testing Actions

 After UAT is completed and the release decision is made, this stage
involves documenting lessons learned and preparing for post-release
monitoring.

 Checklist Preparation:
 Collect feedback from testers and stakeholders about the UAT process.
 Document lessons learned for future projects (e.g., what went well, what
needs improvement).
 Ensure that any minor defects not resolved during UAT are documented
for future patches.
 Monitor the software post-release to ensure no critical issues are
encountered by the users.
 Prepare for the next iteration of testing or support as needed.
System User Stories and Business Scenario
User Stories:
 Focus on what the system needs to do technically. It’s more about the system's internal
workings.
 A user story is an informal, general explanation of a software feature written from the
perspective of the end user.
 The user story describes the type of user, what they want and why. A user story helps to
create a simplified description of a requirement.
 The purpose of a user story is to write down how a project will deliver value back to the end
user.
 Characteristics of a user story
 A user story template often follows the same format. The three components of a user story
are:
 Who. This is typically a job role, customer or type of user.
 What. This is the goal that the user wants the product to accomplish or implement.
 Why. This is the reason the user needs the feature or functionality.
Example:
 As a user, I want to upload photos so that I can share photos with others.
 As an administrator, I want to approve photos before they are posted so that I
can make sure they are appropriate.
 As a social media manager, I want to tag the photos under specific categories
so that I can filter and search the photos for future use.
Business Scenario:
 Focus on the business outcomes and customer interactions. It’s more about
meeting the business’s goals and user expectations.
 A Business Scenario is a business process, application, or set of applications, that
the architecture, the business, and technology environment can enable.

 A business scenario is developed over a number of iterative phases of Gathering,
Analysing, and Reviewing the information in the Business Scenario.
A Business Scenario describes:
 A business process, application, or set of applications, that can be enabled by the
architecture
 The business and technology environment
 The people and computing components (called "actors") who execute the
scenario
 The desired outcome of proper execution
Validation Process
 Process validation is defined as the collection and evaluation of data, from the
process design stage throughout production, which establishes scientific evidence
that a process is capable of consistently delivering quality products.
 The process validation activities can be described in three stages.
 Stage 1 – Process Design: Define how the system or process should behave.
 Stage 2 – Process Qualification: Verify that the system operates as expected and
meets requirements.
 Stage 3 – Continued Process Verification: Ensure that the system or process
remains functional and controlled during actual use or production.
Cont..

 Stage 1: Process Design

 In this stage, the process is designed based on knowledge gained through
development and scale-up activities.
 Software Validation Perspective:
 During the design phase of software, the business requirements are understood,
and the system architecture and user stories are developed.
 Developers create a detailed design and define the software's behavior, setting the
foundation for future validation.
 This stage focuses on how the system will meet the business objectives,
establishing testing strategies, and identifying validation requirements.
Cont..

 Stage 2: Process Qualification

 This stage confirms that the process is capable of consistent and
reproducible results.
 In manufacturing, this is where the design is tested to ensure it meets
commercial requirements. For software, this is where testing and
validation occur.
 Software Validation Perspective:
 The software undergoes various levels of testing (unit testing, integration
testing, system testing) to ensure the design works as intended.
 This phase also includes User Acceptance Testing (UAT) to confirm that the
software meets the end user's requirements.
 If the software passes the tests, it is deemed qualified for release
Cont..

 Stage 3: Continued Process Verification

 After the process is deployed (in a manufacturing or production environment),
continued monitoring ensures that the process remains under control.
 In software, this corresponds to post-deployment monitoring and
maintenance.
 Software Validation Perspective:
 Once the software is live, ongoing validation ensures the software continues to
operate correctly in real-world conditions.
 Continuous testing in production, monitoring for bugs, performance testing, and
addressing any new issues that arise are part of this stage.
 Regular updates and patches are deployed to ensure the software remains compliant
with user requirements and any regulatory changes.
Test Case Matrix

 A Test Case Matrix is a tool used in software testing to track and

manage the coverage of test cases with respect to the requirements or
features of the application under test.
 It helps ensure that all functionalities are tested, and nothing is
overlooked.
Cont..

 The matrix typically lists:

• Requirements or Features: The functionalities or user stories that
need to be tested.
• Test Cases: The specific tests designed to verify the corresponding
feature or requirement.
• Test Status: Indicates whether the test case passed, failed, or is yet to
be executed.
• Other Information: Such as the priority, test type (functional, non-
functional), test environment, and testers.
Structure of a Test Case Matrix

Requirem Test Case

Test Case
ent/ Descripti Priority Status Remarks
ID
Feature on

Login
Verify
Functionali TC01 High Passed -
valid login
ty

Login Verify
Defect
Functionali TC02 invalid High Failed
#123
ty login

Profile Update
TC03 Medium Not Tested -
Update profile info
Test case Metrics:
Cont..

 The Test Case Matrix ensures that:

• All critical functionalities are covered.
• Defects are tracked and documented.
• Testing progress is visible to stakeholders.
Post Deployment Testing

 Post-Deployment Testing may reveal those problems that went

undetected before the deployment of the web application.
 Despite all the planning and testing carried out before deployment,
obtaining user opinion is important for improvement of a website.
 It ensures that the website adapts the needs of the user.
 User feedback may come in various forms like reporting of faults or
suggestion for improvement of website.
 Effective way to obtain a user’s opinion is to get a questionnaire or
survey filled by the user.
 The questionnaire/survey can be used to detect trends and may provide
valuable information for improvement of the website.
Cont..

 Once the user’s opinion is obtained, it is important to identify useful fault

reporting, suggestions and recommendations.
 The following criteria can be used to decide which suggestion needs
attention:
 Frequency of suggestion:
 How many users have given the same suggestion or recommendation?
 If a small number of users are making the same request, then we must think
twice before implementing the suggestion.
 Source of feedback:
 Who is providing the suggestion?
 Need to make sure that suggestions come from regular users and not from
accidental users.
Cont..

 Cost of Implementing the suggestion:

 Is the suggested idea worth implementing?
 The correctness of the proposed change and its impact on the cost and
schedule must be analysed carefully.
 The benefits of implementing the suggested ideas to the business must be
determined.
 Impact of implementing the suggestion:
 Will implementing the suggestion increase the complexity of the website?
 Will the changes be compatible with the other functionalities of the website?
 It is important to obtain the answers to these questions as the results of
implementing a change are sometimes unpredictable.
Questionnaire
Typical post deployment testing
procedure of a web application