Risk Based Testing - Approach, Matrix, Process & Examples
Risk Based Testing - Approach, Matrix, Process & Examples
Positive risks are referred to as opportunities and help in business sustainability. For example investing in a New
project, Changing business processes, Developing new products.
Negative Risks are referred to as threats and recommendations to minimize or eliminate them must be implemented for
project success.
https://www.guru99.com/risk-based-testing.html 1/32
2/3/23, 5:10 PM Risk Based Testing: Approach, Matrix, Process & Examples
Risk Identification
https://www.guru99.com/risk-based-testing.html 2/32
2/3/23, 5:10 PM Risk Based Testing: Approach, Matrix, Process & Examples
Risk identification can be done through risk workshops, checklists, brainstorming, interviewing, Delphi technique, cause and
effect diagrams, lessons learnt from previous projects, root cause analysis, contacting domain experts and subject matter
experts.
Risk Register is a spreadsheet which has a list of identified risks, potential responses, and root causes. It is used to monitor
and track the risks (both threats and opportunities) throughout the life of the project. Risk response strategies can be used to
manage positive and negative risks.
Risk breakdown structure plays an important role in risk planning. The Risk Breakdown structure would help in identifying
the risk prone areas and helps in effective evaluation and risk monitoring over the course of the project. It helps in providing
sufficient time and resources for risk management activities. It also helps in categorizing many sources from which the
project risks may arise.
https://www.guru99.com/risk-based-testing.html 3/32
2/3/23, 5:10 PM Risk Based Testing: Approach, Matrix, Process & Examples
https://www.guru99.com/risk-based-testing.html 4/32
2/3/23, 5:10 PM Risk Based Testing: Approach, Matrix, Process & Examples
The risk owner is responsible for identifying options to reduce the probability and impact of the assigned risks.
Risk mitigation is a risk response method used to lessen the adverse impacts of possible threats. This can be done by
eliminating the risks or reducing them to an acceptable level.
https://www.guru99.com/risk-based-testing.html 5/32
2/3/23, 5:10 PM Risk Based Testing: Approach, Matrix, Process & Examples
Risk Contingency
Contingency can be described as a possibility of an uncertain event, but the impact is unknown or unpredictable. A
contingency plan is also known as the action plan/back up plans for the worst case scenarios. In other words, it determines
what steps could be taken when an unpredictable event materializes.
https://www.guru99.com/risk-based-testing.html 6/32
2/3/23, 5:10 PM Risk Based Testing: Approach, Matrix, Process & Examples
This can be achieved by risk reassessments, risk audits, variance and trend analysis, technical performance measurement,
status update meetings and retrospective meetings.
We need to remember that risk increases with changes in technology, the size of the project, length of the project (Longer
project timeframe), the number of sponsoring agencies, project estimates, efforts, and a shortage of appropriate skills.
https://www.guru99.com/risk-based-testing.html 7/32
2/3/23, 5:10 PM Risk Based Testing: Approach, Matrix, Process & Examples
subsequent approvals.
4. Assess the risks by calculating the likelihood and impact each requirement could have on the project taking the defined
criteria’s like cost, schedule, resources, scope, technical performance safety, reliability, complexity, etc. into
consideration.
5. Identify the probability of failure and high-risk areas. This can be done using risk assessment matrix.
6. Use a risk register to list the set of identified risks. Update, monitor and track the risks periodically at regular intervals.
7. Risk profiling needs to be done at this stage to understand the risk capacity and risk tolerance levels.
8. Prioritize the requirements based on the rating.
9. Risk-based test process is defined
10. Highly critical and medium risks can be considered for mitigation planning, implementation, progress monitoring. Low
risks can be considered on a watch list.
11. Risk data quality assessment is done to analyze the quality of the data.
12. Plan and define test according to the rating
13. Apply appropriate testing approach and test design techniques to design the test cases in a way that the highest risks
items are tested first. High-risk items can be tested by the resource with good domain knowledge experience.
14. Different test design techniques can be used for e.g. using the decision table technique on high-risk test items and using
‘only’ equivalence partitioning for low-risk test items.
15. Test cases are also designed to cover multiple functionalities and end to end business scenarios.
16. Prepare test data and test conditions and test bed.
17. Review the Test plans, Test Strategy, Test cases, Test reports or any other document created by the testing team.
18. Peer review is an important step in defect identification and risk reduction.
19. Perform dry runs and quality checks on the results
20. Test cases are executed according to the priority of the risk item.
21. Maintain traceability between risk items, tests that cover them, results of those tests, and defects found during testing.
All testing strategies executed properly will reduce quality risks.
22. Risk-based testing can be used at every level of testing, e.g. component, integration, system, and acceptance testing
https://www.guru99.com/risk-based-testing.html 8/32
2/3/23, 5:10 PM Risk Based Testing: Approach, Matrix, Process & Examples
23. At the system level, we need to focus on what is most important in the application. This can be determined by looking at
the visibility of functions, at frequency of use and at the possible cost of failure.
24. Evaluation of exit criteria. All high-risk areas fully tested, with only minor residual risks left outstanding.
25. Risk-based Test Results reporting and metrics analysis.
26. Reassess existing risk events and new risk events based on Key Risk Indicators.
27. Risk register updation.
28. Contingency plans- This works as a fallback plan/emergency plans for the high exposure risks.
29. Defect analysis and defect prevention to eliminate the defects.
30. Retesting and Regression testing to validate the defect fixes based on pre-calculated risk analysis and
high-risk areas should be most intensively covered.
1. Technical System Test –This is referred to as environment test and integration test. Environment test includes testing in
development, testing, and the production environment.
2. Functional System Test– Testing of all functionalities, features, programs, modules. The purpose of this test is to
evaluate if the system meets its specified requirements.
3. Non-functional System Test-Testing the non-functional requirements performance, load tests, stress-test, configuration
tests, Security tests, backup and recovery procedures and documentation (system, operation and installation
documentation).
https://www.guru99.com/risk-based-testing.html 9/32
2/3/23, 5:10 PM Risk Based Testing: Approach, Matrix, Process & Examples
Functional testing ensures that the product/application meets customer and business requirements. On the other hand,
non-functional testing is done to verify if the product stands up to customer’s expectations in terms of quality, reliability
usability, performance, compatibility, etc.
https://www.guru99.com/risk-based-testing.html 10/32
2/3/23, 5:10 PM Risk Based Testing: Approach, Matrix, Process & Examples
1. Risk Identification
2. Risk Analysis
3. Risk Response
4. Test Scoping
5. Test Process definition
1. In this process, the risks are identified and categorized, a draft register of risks are prepared, risk sorting is done to
identify the significant risks.
https://www.guru99.com/risk-based-testing.html 11/32
2/3/23, 5:10 PM Risk Based Testing: Approach, Matrix, Process & Examples
2. Risk response involves formulating the test objectives from the risks and selecting appropriate techniques to
demonstrate the test activity /test technique to meet the test objectives.
3. Document dependencies, requirements, cost, time required for Software testing, etc. are considered to calculate the
test effectiveness score.
4. Test scoping is a review activity that requires the participation of all stakeholders and technical staff. It is important to
adhere to the agreed scope of risks. These risks need to be addressed by testing, and all members agree with the
responsibilities assigned to them and budget allocated for these activities.
5. After the scope of testing has been finalized the test objectives, assumptions, dependencies for each test stages has to
be compiled in the standard format.
https://www.guru99.com/risk-based-testing.html 12/32
2/3/23, 5:10 PM Risk Based Testing: Approach, Matrix, Process & Examples
Lets, consider the functional requirements F1, F2 ,F3 and Non-functional requirements N1 & N2
https://www.guru99.com/risk-based-testing.html 13/32
2/3/23, 5:10 PM Risk Based Testing: Approach, Matrix, Process & Examples
Test Objective 1- Demonstrate using a Test that the expected features and functionalities of the system work fine, and
the risk R1 can be addressed by functional testing
Test-Browser Page testing is done to execute important user tasks and verify that the R1 ( Risk associated with F1) could
be addressed in a range of scenarios.
Test Objective 2- Demonstrate using a Test that the expected features and functionalities of the system works fine, and
the risk R2 can be addressed by functional testing
Test-Browser Page testing is done to execute important user tasks and verify that the R2 could be addressed in a range
of scenarios
Test Objective 3- Demonstrate using a Test that the expected features and functionalities of the system works fine, and
the risk R3 can be addressed by functional testing
Test-Browser Page testing is done to execute important user tasks and verify that R3 could be addressed in a range of
scenarios
Test Objective N1-Demonstrate using a Test that the operational characteristics of the system works fine and the risk
NR1 can be addressed by non-functional testing
Test-Usability testing is a technique used to assess how easy user interfaces are to use and verify that the NR1 could be
addressed by usability testing
https://www.guru99.com/risk-based-testing.html 14/32
2/3/23, 5:10 PM Risk Based Testing: Approach, Matrix, Process & Examples
Test Objective N.2- Demonstrate using a Test that the operational characteristics of the system works fine, and the risk
NR2 can be addressed by non-functional testing
Test-Security testing is a technique used to check whether the application secured or is it vulnerable to attacks, whether
there is any information leakage and verifies that NR2 could be addressed by security testing.
Specific Test objectives: The risks and test objectives listed are specific to the test types.
https://www.guru99.com/risk-based-testing.html 15/32
2/3/23, 5:10 PM Risk Based Testing: Approach, Matrix, Process & Examples
Prepare a risk register.This records the risks derived from generic risk list, existing checklist, brainstorming session.
Include the risks associated with the system functional and non-functional requirements (Usability,
security,performance)
Each risk is allocated a unique identifier
Col
Column Heading Description
No.
How confident are the testers that they can address this
6 Test effectiveness
risk?
8 Test objective(s) what test objective will be used to address this risk
https://www.guru99.com/risk-based-testing.html 16/32
2/3/23, 5:10 PM Risk Based Testing: Approach, Matrix, Process & Examples
Col
Column Heading Description
No.
The Probability(1 Low -5 High ) and consequences(1 Low -5 High ) of each risk are assessed
https://www.guru99.com/risk-based-testing.html 17/32
2/3/23, 5:10 PM Risk Based Testing: Approach, Matrix, Process & Examples
These testing activities can be classified into stages (Component Testing/Unit testing, Integration Testing, System
Testing, Acceptance Testing)
At times, a risk might be addressed by one or more than one test stage
Identify the dependencies and assumptions (Availability of skills, tools, test environments, resources)
Test effectiveness is computed. Test effectiveness relates to the confidence level of the tester that the risk will be
definitively addressed through testing. Test effectiveness score is a number between one and five.( 5-High Confidence ,
1-Low Confidence)
Estimate of the effort, the time required, cost to prepare and execute these tests.
https://www.guru99.com/risk-based-testing.html 19/32
2/3/23, 5:10 PM Risk Based Testing: Approach, Matrix, Process & Examples
https://www.guru99.com/risk-based-testing.html 20/32
2/3/23, 5:10 PM Risk Based Testing: Approach, Matrix, Process & Examples
Test priority number is calculated. It is the product of probability, consequences, and test effectiveness scores.
Based on the test priority number the test importance can be classified as High(Red ), Medium (Yellow) &Low (Green).
Highest risk items are tested first.
Allocation the test activities to the test stages.Designate the group that will perform testing for each objective in the
different test stages(Unit testing, Integration Testing, System Testing, Acceptance Testing)
What is in scope and out of scope for testing is decided in the test scoping phase
For each stage test objectives, component under test, responsibility,environment,entry criteria,exit
criteria,tools,techniques,deliverables are defined.
https://www.guru99.com/risk-based-testing.html 21/32
2/3/23, 5:10 PM Risk Based Testing: Approach, Matrix, Process & Examples
Generic Test Objectives- These generic objectives are applicable to multiple projects and applications
Component meets the requirement and is ready for use in larger subsystems
The risks associated with the specific test types are addressed, and the test objectives are accomplished.
Integrated components are correctly assembled. Ensure interface compatibility among the components.
The system meets the specified functional and nonfunctional requirements.
Product components satisfy end user needs in their intended operating environment
Risk management strategy is used to identifying, analyzing, and mitigating risks.
The System meets industry regulation requirements
The System meets contractual obligations
https://www.guru99.com/risk-based-testing.html 22/32
2/3/23, 5:10 PM Risk Based Testing: Approach, Matrix, Process & Examples
Institutionalization and the achievement of other specific objectives established such as cost, schedule, and quality
objectives.
System, processes and people meet business requirements
Generic Test objectives can be defined for the different test stages
Component Testing
Integration Testing
System Testing
https://www.guru99.com/risk-based-testing.html 23/32
2/3/23, 5:10 PM Risk Based Testing: Approach, Matrix, Process & Examples
Acceptance Testing
1. G4 & G5 demonstrates’s the system meets the functional (F1,F2,F3) and non-functional requirements(N1,N2) .
2. Demonstrate using tests that the expected features and functionalities of the system work fine and the risk associated
with F1, F2, F3 can be addressed by functional testing
3. Demonstrate using tests that the operational characteristics of the system work fine and the risk associated with N1, N2
can be addressed by non-functional testing
4. Based on the test priority number the test importance can be classified as High(Red ), Medium (Yellow) &Low (Green).
Probability is the measure of the chance for an uncertain event will occur. Exposure in terms of time, proximity and
repetition. It is expressed in terms of percentage.
https://www.guru99.com/risk-based-testing.html 24/32
2/3/23, 5:10 PM Risk Based Testing: Approach, Matrix, Process & Examples
Severity is the degree of impact of damage or loss caused due to the uncertain event. Scored 1 to 4 and can be classified as
Catastrophic=1, Critical=2, Marginal=3, Negligible=4
Catastrophic – Harsh Consequences that make the project completely unproductive and could even lead to project
shutdown. This must be a top priority during risk management.
Critical– Large consequences which can lead to a great amount of loss. Project is severely threatened.
Marginal – Short term damage still reversible through restoration activities.
Negligible– Little or minimal damage or loss. This can be monitored and managed by routine procedures.
The priority is classified into four categories, which is mapped against the severity and probability of the risk as shown in
below image.
Serious
High
Medium
Low
https://www.guru99.com/risk-based-testing.html 25/32
2/3/23, 5:10 PM Risk Based Testing: Approach, Matrix, Process & Examples
Serious: The risks that fall in this category are marked in Amber color. The activity must be stopped, and immediate action
must be taken to isolate the risk. Effective controls must be identified and implemented. Further, the activity must not
proceed unless the risk is reduced to a low or medium level.
High: The risks that fall in this category are marked in Red color ate action or risk management strategies. Immediate action
must be taken to isolate, eliminate, substitute the risk and to implement effective risk controls. If these issues cannot be
resolved immediately, strict timelines must be defined to resolve these issues.
Medium: The risks that fall in this category are marked in Yellow color. Reasonable and practical steps must be taken to
minimize the risks.
https://www.guru99.com/risk-based-testing.html 26/32
2/3/23, 5:10 PM Risk Based Testing: Approach, Matrix, Process & Examples
Low: The risks that fall in this category are marked in green color) marked can be ignored as they usually do not pose any
significant problem. Periodical review is a must to ensure the controls remain effective
https://www.guru99.com/risk-based-testing.html 27/32
2/3/23, 5:10 PM Risk Based Testing: Approach, Matrix, Process & Examples
2. Metrics Preparation
Metrics is a combination of two or more measures used to compare software processes, projects, and products.
Defect Leakage %
Defect detection efficiency
Requirement Stability Index
Cost of Quality
3. Analyze the risks in nonfunctional categories (performance, reliability, and usability) based on defect status and a
number of test pass/fail status, based on their relationship to risks.
4. Analyze the risks in functional categories metrics of testing, defect status and test pass/fail status, based on their
relationship to risks.
5. Identify key lead and lag indicators and create early warning indicators
6. Monitor and report on lead and lag risk indicators (Key Risk Indicators) by analyzing the data patterns, trends, and
interdependencies.
Inherent Risk: The risks that were identified/already present in the system before the controls and responses were
implemented. Inherent risks are also known as Gross risks
Residual Risk: The risks that are left over after the controls and responses have been implemented. Residual risks are
known as the net risks
Secondary Risk: The new risk caused by the implementation of risk response plan
Recurrent risks: Likelihood that the initial risks will occur.
Test result measurement based on risk helps the organization to know the residual level of quality risk during test execution,
and to make smart release decisions.
Risk profiling is a process for finding the optimal level of investment risk for the client considering the risk required, risk
capacity and risk tolerance.
1. Risk Required is the level of risk the client needs to take in order to obtain a satisfactory return
2. Risk capacity is the level of financial risk the client can afford to take
3. Risk tolerance is the level of risk which the client would prefer to take
Customer Feedback
Gather customer feedback and reviews to improve the business, product, service and experience.
https://www.guru99.com/risk-based-testing.html 30/32
2/3/23, 5:10 PM Risk Based Testing: Approach, Matrix, Process & Examples
Continuous risk monitoring and assessment throughout the project’s entire lifecycle helps in identification and
resolution of risks and address the issues that could endanger the achievement of overall project goals and objectives.
Summary:
In Software Engineering, Risk based testing is the most efficient way to guide the project based on risks.
The testing efforts are effectively organized, and level of priority of each risk item is rated. Each risk is then associated with
the appropriate test activities, where a single test having more than one risk item, then the test is assigned as the highest
risk.
Tests are executed according to the risk priority order. Risk monitoring process helps in keeping track of the identified risks,
and reducing the impacts of residual risks.
https://www.guru99.com/risk-based-testing.html 31/32
2/3/23, 5:10 PM Risk Based Testing: Approach, Matrix, Process & Examples
About
About Us
Advertise with Us
Write For Us
Contact Us
Career Suggestion
SAP Career Suggestion Tool
Software Testing as a Career
Interesting
eBook
Blog
Quiz
SAP eBook
Execute online
Execute Java Online
Execute Javascript
Execute HTML
Execute Python
https://www.guru99.com/risk-based-testing.html 32/32