11 Agile Testing 2
11 Agile Testing 2
• They may note down ideas about what to test before test execution.
• Exploratory Testing is widely used in Agile models and is all about discovery,
investigation, and learning.
• Such test execution activity can be automated does not require many cognitive
skills.
Exploratory Testing
Exploratory Testing
• Under scripted testing, you design test cases first and later proceed with test
execution.
• Such test execution activity can be automated does not require many cognitive
skills.
Exploratory Testing
• From Boeing to Instagram, there are plenty of examples where the rush to deliver
on deadline and poor-quality testing led to reputational and financial
damage. You can explore such case studies for reference.
Exploratory Testing
• Why use exploratory testing
• Most software quality testing uses a structured approach.
• Test cases are defined based on already defined user stories and the test data is
structured based on the test cases defined.
• Test coverage is measured using software engineering metrics and, in most cases,
the coverage is adequate technically.
Exploratory Testing
• Why use exploratory testing
• What it often misses are edge cases, which are discovered through
• User Acceptance Testing (UAT) and are tested based on user personas.
• Testers can annotate defects, add assertions and voice memos, and create
documentation on the fly.
• This is how a user story is converted into a test case. This information can also be
used for QA.
Exploratory Testing
• Why use exploratory testing
• Effectively, test execution is implemented without formally authoring test steps.
• The exploratory testing tool then becomes a precursor to automation. It helps
formalize the findings and document it automatically.
• With the help of visual feedback and collaborative testing tools, everyone can
participate in exploratory testing.
• This enables teams to react and adapt to changes quickly – facilitating an agile
workflow.
Exploratory Testing
• Why use exploratory testing
• Furthermore, the tester can convert exploratory testing sequences into functional
test scripts using tools for automated test case documentation.
• By integrating with tools such as Jira and test management products, teams can
directly export the recorded documentation to test cases.
Exploratory Testing
• When should you use exploratory testing?
• Exploratory testing is suited for specific testing scenarios, such as when someone
needs to learn about a product or application quickly and provide rapid feedback.
• Use exploratory testing to aid unit test process, document the steps and use that
information to test extensively during later sprints.
• It is especially useful to find new test scenarios to enhance the test coverage.
Exploratory Testing
• How to do Exploratory Testing
• Following is a step by step process on How to do Exploratory Testing
• Create a Bug Taxonomy (classification)
• Categorize common types of faults found in the past projects
• Analyze the root cause analysis of the problems or faults
• Find the risks and develop ideas to test the application.
• Test Charter
• Test Charter should suggest
• what to test
• how it can be tested
• What needs to be looked
• Test ideas are the starting point of exploration testing
• Test charter helps determine how the end user could use the system
Exploratory Testing
• How to do Exploratory Testing
• Following is a step by step process on How to do Exploratory Testing
• Time Box
• This method includes a pair of testers working together not less than 90 minutes
• There should not be any interrupted time in those 90 minutes session
• Timebox can be extended or reduced by 45 minutes
• This session encourages testers to react on the response from the system and prepare for the correct outcome
• Review Results:
• Evaluation of the defects
• Learning from the testing
• Analysis of coverage areas
• Debriefing:
• Compilation of the output results
• Compare the results with the charter
• Check whether any additional testing is needed
Exploratory Testing
• For Example, during exploratory execution, the following needs to be done:
• Test Coverage - Whether we have taken notes on the coverage of test cases and
improve the quality of the software
• Risks - Which risks need to be covered and which are all important ones?
• Test Execution Log - Recordings on the test execution
• Issues / Queries - Take notes on the question and issues on the system
Exploratory Testing
• https://www.youtube.com/watch?v=F-Nzlq2-ivk
Risk based Testing in Agile
• What is risk based testing?
• The most common complaint that comes from software testing using the Agile
method is the lack of time.
• Since the term is itself a synonym for speed, its emphasis on getting things done
is self-explanatory.
• However, this brings up a burning question for every Agile tester in the world.
• The goal of RBT is to identify potential problems that could arise in a software
application and to focus testing efforts on those areas that are most likely to be
problematic.
Risk based Testing in Agile
• What is risk based testing?
• The risk assessment process involves analyzing the application and identifying
the areas that are most critical to the application's functionality, as well as any
potential areas of weakness or vulnerabilities.
• The testing team will focus on testing the highest risk areas first and then
gradually move on to lower risk areas.
• This approach ensures that the testing effort is optimized and that critical issues
are identified and addressed early in the testing process.
Risk based Testing in Agile
• What is risk based testing?
• RBT is an efficient testing approach that helps organizations optimize their
testing effort and reduce the time and resources required for testing.
• It also helps to identify and address critical issues early in the development
process, which can reduce the cost and time required for rework and bug fixes
later in the development cycle.
Risk based Testing in Agile
• What is risk based testing?
• Most Agile sprints last a couple of weeks.
• That timeframe is undoubtedly not enough to test every or even most features of
modern websites and apps.
• As development progresses, software becomes more complex and requires more
tests to verify its functionality.
• Running thousands of tests is completely unfeasible, and testers must prioritize
what needs to be tested within increasingly shorter timelines.
• The answer lies in risk-based testing.
Risk based Testing in Agile
• What is risk based testing?
• Risk Based Testing (RBT) is a software testing type which is based on the
probability of risk.
Risk based testing prioritizes testing of features and functions of the software
application which are more impactful and likely to have defects.
Risk based Testing in Agile
• What is risk based testing?
• If a QA team struggles with deciding how to allocate time and effort in each
sprint, their best bet is using risk-based testing.
• This refers to a testing strategy that uses ‘defined risk’ to determine testing goals.
• In other words, the risk-based testing approach organizes testing efforts in ways
that lower the residual level of product risk when the software goes into
production.
• This strategy is useful for test analysis, planning, estimation, design, execution,
and results reporting.
Risk based Testing in Agile
• What is risk based testing?
• RBT (Risk-Based Testing) is a testing strategy that follows the principles of Risk
Management but in a testing context.
• In software development projects, there is a set of common objectives, including:
• “quality”
• cost control
• time-to-market (i.e. target release/delivery dates)
Risk based Testing in Agile
• What is risk based testing?
• These goals can be affected by risks coming from different sources (both internal
or external) due to many different factors. In case of negative risks, the risk level
will depend on things such as the complexity of the change, the number of
impacted users, frequency of usage, likelihood of failure, etc.
• Risks can affect the overall quality, including the customer perceived quality.
Risk based Testing in Agile
• What is risk based testing?
• Quality is a broad word embracing “fit” of the product to customer needs and the
many quality attributes which risks may affect.
• An intrinsic part of quality is ensuring that a product has few bugs (or at least a
reduced probability of bugs on important/risky areas).
• If testing is performed as a means to find/avoid bugs, RBT can be used as a
“mitigation,” or even as an “avoidance” strategy.
• On the other hand, testing is also about understanding, discovering and
providing valuable feedback that can help build “better” features and better
products. Therefore, RBT assists in making testing related decisions based on the
assessment of risks.
Risk based Testing in Agile
• Purpose of Risk-Based Testing
• The main purpose of RBT is to use risk management principles for adequate
testing.
• First of all, RBT provides a framework for clear communication and discussion
about risks between testers, developers, clients and stakeholders.
• RBT will help you define terms and agree on a common language, and through
this it will make risk visible and actionable for you.
• RBT takes into consideration the big picture, as it covers customer needs and also
the needs of the development team. It specifically uses risks as the input to
support testing activities.
Risk based Testing in Agile
• Purpose of Risk-Based Testing
• Typically, customers are most concerned about business features, timing, visible
quality and costs.
• On the other hand, the development team has similar concerns; however, it sees
quality in a broader scope as it has to maintain and possibly evolve with the
product being developed.
• Timing and costs need to be managed effectively to be on a budget and avoid
delays; however quality must not be hurt and if decisions need to be taken to
meet timing/cost criteria, RBT can provide the means to ensure that focus will be
given to the features/issues that matter most to customers.
Risk based Testing in Agile
• Purpose of Risk-Based Testing
• Both customers and the development team want to avoid important defects, so,
RBT focuses the testing efforts on what matters most – where we can get the most
value from.
Risk based Testing in Agile 1.CUSTOMER MARO BHAGVAN
2.RISK MNE NA GAME.AAVO NA JOIE
• RBT’s benefits can be summed up as: ANE AAYO TO OCHI ASAR
• RBT can help find more important defects first, by focusing testing on higher
risks
• by focusing testing on higher risks (or risky areas/features), RBT ensures that
important items are exhaustively tested and important defects are found
sooner; thus, product and it’s most important items (e.g. features) can be
released with a high degree of confidence, while ensuring they meet expected
quality goal
Risk based Testing in Agile
Optimize QA Effort/cost
• RBT’s benefits can be summed up as:
• Optimized QA efforts and cost
• RBT can answer questions such as…
• What should we test?
• Where should we start?
• When can we stop testing?
• Can we reduce the testing effort somehow? How and at what “cost”?
• If we have more time for testing, what is the best way to take advantage of it?
• RBT provides the means to define the testing scope, focusing on what is relevant, by
identifying what tests to execute and their execution priority, given time and resource
constraints; the overall amount of test cases may be reduced as not everything needs to be
tested or be tested in the same depth
• RBT provides a way (based on risks) to choose tests for regression testing
• RBT provides some clues for selecting candidates for automation
Risk based Testing in Agile
• RBT’s benefits can be summed up as:
• Optimized QA efforts and cost
• RBT can answer questions such as…
• What should we test?
• Where should we start?
• When can we stop testing?
• Can we reduce the testing effort somehow? How and at what “cost”?
• If we have more time for testing, what is the best way to take advantage of it?
• RBT provides the means to define the testing scope, focusing on what is relevant, by
identifying what tests to execute and their execution priority, given time and resource
constraints; the overall amount of test cases may be reduced as not everything needs to be
tested or be tested in the same depth
• RBT provides a way (based on risks) to choose tests for regression testing
• RBT provides some clues for selecting candidates for automation
Risk based Testing in Agile
• RBT’s benefits can be summed up as:
increase positive risks
• Increased risk level of positive risks
• Risk Analysis
• risks are discussed together in the team (may involve customer)
• risks impact and probability are calculated, and thus the related risk level
Risk based Testing in Agile
• The overall RBT process
• How will we handle it in the most effective way (i.e. what can return the
most value)?Risk Evaluation and Treatment
• Test Planning
• using the input from Risk Analysis, the test manager can…
• define test strategy (e.g. level of testing to perform, techniques to use, environments to choose)
• estimate testing effort
• define/estimate schedule (target dates, number of testing iterations)
• Test Design
• specification of the Tests taking the identified risks as input; tests are designed to mitigate the risk
(i.e. diminish their probability)
• make use of more extensive data with data-driven testing and automated testing/checking, if needed
Risk based Testing in Agile
• The overall RBT process
•Test Execution
• perform testing by descending order of risk level (i.e. execute Tests related to higher risks first);
experienced testers, with a high degree of domain knowledge, may provide more valuable
feedback
• look at items where you assessed the risk and evaluate if you need to take additional
measures/treatments (“Are those items having failed tests? Were bugs found? Are those bugs
relevant? Did we find any new risk while testing?”)
Risk based Testing in Agile
• Risk Management Process
• Let's now understand the steps involved in Risk Management Process
• Risk Identification
• Risk identification can be done through risk workshops, checklists, brainstorming,
interviewing, Delphi technique, cause and effect diagrams, lessons learnt from previous
projects, root cause analysis, contacting domain experts and subject matter experts.
• Risk Register is a spreadsheet which has a list of identified risks, potential responses,
and root causes. It is used to monitor and track the risks (both threats and
opportunities) throughout the life of the project. Risk response strategies can be used to
manage positive and negative risks.
• Risk breakdown structure plays an important role in risk planning. The Risk Breakdown
structure would help in identifying the risk prone areas and helps in effective evaluation
and risk monitoring over the course of the project. It helps in providing sufficient time
and resources for risk management activities. It also helps in categorizing many sources
from which the project risks may arise.
Risk based Testing in Agile
• Risk Breakdown structure sample
Risk based Testing in Agile
• Risk Analysis (Includes Quantitative and Qualitative Analysis)
• Once the list of potential risks has been identified, the next step is to analyze them and
to filter the risk based on the significance. One of the qualitative risk analysis technique
is using Risk Matrix (covered in the next section). This technique is used to determine
the probability and impact of the risk.
• Risk Response planning
• Based on the analysis, we can decide if the risks require a response. For example, some
risks will require a response in the project plan while some require a response in the
project monitoring, and some will not require any response at all.
• The risk owner is responsible for identifying options to reduce the probability and
impact of the assigned risks.
• Risk mitigation is a risk response method used to lessen the adverse impacts of
possible threats. This can be done by eliminating the risks or reducing them to an
acceptable level.
Risk based Testing in Agile
• Risk Contingency
• Contingency can be described as a possibility of an uncertain event, but the
impact is unknown or unpredictable. A contingency plan is also known as the
action plan/back up plans for the worst case scenarios. In other words, it
determines what steps could be taken when an unpredictable event materializes.
• Risk Monitoring and Control
• Risk control and monitor process are used to track the identified risks, monitor
residual risks, identify new risks, update the risk register, analyze the reasons for
the change, execute risk response plan and monitor risk triggers, etc. Evaluate
their effectiveness in reducing risks.
• This can be achieved by risk reassessments, risk audits, variance and trend
analysis, technical performance measurement, status update meetings and
retrospective meetings.
Risk based Testing in Agile
The table below gives information about the
• Then, you might focus on testing the data privacy controls to ensure that
sensitive information is not exposed. Next, you would test the financial
calculations to ensure that they are accurate and reliable.
• By prioritizing your testing efforts based on the identified risks, you can ensure
that you are focusing on the areas of the application that are most important to
the business and its users, and that you are identifying and addressing the most
critical issues first.
Regression Testing in Agile
Regression Testing in Agile
Regression testing in agile helps development teams concentrate on new
functionality, while maintaining stability with every new product increment.
• Teams use regression testing to make sure that tested software continues to
perform after every modification.
• Regression testing is the “stepchild” of agile testing, loved by few, but is essential
to enable the high velocity that agile teams strive to achieve.
Regression Testing in Agile
• In agile, testing needs to develop with each sprint and testers must make sure
that new changes do not affect existing functionality of the application. This is
known as regression testing.
nano moto badha change
• For successful regression testing in agile, a testing team should build the
regression suite from the onset of product development.