Solution Analysis and

Management
Module 11: Solution Limitations

Note: this module is largely based on the Solution Evaluation chapter

of the BABOK v3.
Next Week – Case Reflection (10%)

• Due Date: Friday, July 30 at 5PM

• Preparation for the next class:
• Read the case
• Answer the question set before coming to the class
• Grading rubric will be explained in more details
 SAM Process
Project Plan

Requirements Solution Solution Solution Implement

Outputs Options Evaluation Choice Solution

Pilot / Project Assess Corrective Continuous

prototype? Completed Limitations Actions Improvement
Introduction

• Short vs long term performance.

• Solution vs organizational limitations.
• Techniques that support finding and addressing limitations:
• Root cause analysis
• Lessons learned
Long Term Performance
• The last module dealt with solution evaluation in the short term – evaluating
whether components of a solution or the entire solution are meeting the
requirements.
• The solution evaluation that occurs prior to choosing a given solution normally
details longer term performance – for example, cash flow improvements over a
period of years.
• A solution may not be performing optimally in the short term due to various factors
– the solution may have a phased improvement approach, there can be a learning
curve for employees (as they apply their training and gain experience), and any
process changes will take time to be fully ‘tuned in’.
• Therefore we need to evaluate whether a solution is performing as expected over
a longer time frame, and continually adjust to take advantage of opportunities.
• This is a key benefit of business analysis as a discipline, as often once a project is
‘completed’ the project team members disband and begin to work on other
projects. It is assumed not only that the solution will continue to ‘work’, but that it
will continue to work at its full potential.
Long Term Performance

• We can evaluate the ‘short-term’ benefits on a periodic schedule, as well as

evaluating the overall performance and impact of a solution.

Examples:
• Is a change to a production process maintaining the efficiency improvements?
• Is a new business intelligence solution actually driving improved decision making
– and how can we quantify any anecdotal examples?
• What are the results from improved targeted marketing software?

Normally we will use the KPIs developed during the solution choice and planning
phase.
Addressing Limitations
• Depending on the gap between
estimated and actual
performance, this could involve a
few small changes managed
informally, or in extreme cases
could involve a full project or
initiative designed to address the
limitations.
• This could include further cost-
benefit analysis, process mapping,
elicitation (job shadowing,
interviews, questionnaires, focus
groups, etc).
Analyzing Results
• If there is a disparity between the expected and actual long term performance
metrics, we need to analyze the results to try and find out why the problem is
occurring, and plan how to address the issue.

For example, in the last module we discussed how to measure the actual vs
expected results from a projected decline in the amount of time an insurance
adjuster spends on processing manual claims. In the first quarter after launch the
time required had not declined as much as expected due to a high number of
exceptions. However, after a year, the times are still similar to the first quarter.
There could be many reasons for this:
• Longer term weather anomalies.
• The original estimates of solution performance were unrealistic.
• Adjusters are not fully using the new system due to lack of training or trust.
• The workload for adjusters has increased (i.e. they have a higher load of claims).
Analyzing Results

• The time period used is important in separating long term from short term results.
• If the business is seasonal, it may be necessary to collect a year of ‘clean’ data.
Normally we will compare year over year results.
• Also, we need to be aware of any other critical changes in the organization. For
example, if the adjusters’ workload had increased, this could be due to increased
business or it could be due to reducing the number of adjusters per claim.
• It could also be possible that the data itself is not accurate and consistent. Make
sure that there haven’t been any changes to the underlying data or measurements
as a result of the solution implementation.
• Once we are sure that the data and measurements are correct, the time period is
reasonable, and have accounted for other changes that may have impacted our
estimates, we will look at the variances (the difference between expected and
actual performance) in order to provide a basis for recommending improvements to
increase the solution value.
Assess Solution Limitations

• The purpose of this step

is to determine the
factors internal to the
solution that restrict the
full realization of value.

• What are the root

causes for under-
performing and
ineffective solutions and
solution components?
• Start by identifying
which factors are
limiting value.
Assess Solution Limitations

• Assessing solution limitations can occur at any point in the project lifecycle –
during development, just prior to final implementation, or an existing solution.
• It can be that specific factors are acting on their own to limit the solution
effectiveness, or it could be based on dependencies between factors.
• In the case of dependencies, the solution effectiveness could be based on the
least effective component. Therefore addressing this one factor could have a
multiplicative effect on improving the solution effectiveness.

For example, suppose your company implemented a solution to improve order

shipping efficiency. It could be that all types of orders are being prepared more
quickly except for one type of item. But that one item would slow down the overall
order efficiency, since all items must be present to ship the order.
Impact Assessment

• Once the solution factors, dependencies, and problems have been identified,
they can be assessed for their relative impact on the solution performance.
• Each problem will have a given level of severity, the probability of the re-
occurrence of the problem, the impact on the business operations, and the
capacity of the business to absorb the impact.
• Therefore some problems may not need to be ‘solved’, and there will be a
prioritization in place for those problems that do need attention.
• Problems can be sorted into three groups:
• Must be resolved.
• Can be mitigated through other activities or approaches.
• Can be accepted.
• Other activities or approaches may include additional quality control measures,
new or adjusted business processes, or additional support for exceptions to the
desired outcome.
 Assess Enterprise Limitations

• The purpose of this step is to

determine how factors
external to the solution are
restricting value realization.

• Solutions themselves may

operate across the
enterprise and may have
interactions and
interdependencies.
• They may also depend on
environmental factors
external to the enterprise.
Assess Enterprise Limitations

• Enterprise limitations can include such things as:

• Culture
• Operations
• Technical components
• Stakeholder interests
• Reporting structures
• As with solution limitations, this can occur at any point in the solution life cycle.
Culture Assessment

Enterprise culture is defined as the deeply rooted beliefs, values, and

norms shared by the members of an enterprise.

• These beliefs are not directly visible, but are expressed through organizational
documentation (such as a vision statement or team charters). They drive the
actions taken by the enterprise.
• Business analysts perform cultural assessments to:
• Identify whether or not stakeholders understand the reasons why a
solution exists.
• Ascertain whether or not the stakeholders view the solution as something
beneficial and are supportive of the change.
• Determine if and what cultural changes are required to better realize value
from a solution.
Culture Assessment

• The enterprise culture assessment evaluates the extent to which the culture can
accept a solution.
• If cultural adjustments are needed to support the solution, the assessment is
used to judge the enterprise’s ability and willingness to adapt to these cultural
changes.
• Business analysts also evaluate internal and external stakeholders to:
• Gauge understanding and acceptance of the solution.
• Assess perception of value and benefit from the solution.
• Determine what communication activities are needed to ensure awareness
and understanding of the solution.
Stakeholder Impact Analysis
• A stakeholder impact analysis provides insight into how the solution affects a
particular stakeholder group.

Functions
The processes in which the stakeholder uses the solution – including inputs the
stakeholder provides, how they use the solution to execute a process and what
outputs the stakeholder receives from the process.

Locations
The geographic locations of the stakeholders interacting with the solution.

Concerns
The issues, risks, and overall concerns stakeholders have with the solution – both real
and perceived.
Organizational Structure Changes

• Sometimes the organizational structure is impacted by a solution.

• The use of a solution and ability to adopt a change can be enabled or blocked by
formal and informal relationships among stakeholders.
• The reporting structure may be too complex, or too simple.
• A key activity is to assess whether the organizational hierarchy supports the
solution.
• Informal relationships – such as alliances, friendships or matrix reporting can
impact the ability of a solution to deliver potential value.
Operational Assessment

• This is performed to determine if an enterprise is able to adapt to or effectively

use a solution.
• It identifies which processes and tools within the enterprise are adequately
equipped to benefit from the solution, and if sufficient and appropriate assets
are in place to support it.
• When conducting an operational assessment, business analysts consider:
• Policies and procedures.
• Capabilities and processes that enable other capabilities.
• Skill and training needs.
• Human resources practices.
• Risk tolerance and management approaches.
• Tools and technology that support a solution.
 Recommend Actions to Increase Solution Value

• The purpose is to
understand the factors
that create differences
between potential value
and actual value, and to
recommend a course of
action to align them.

• Here we need to
understand the overall
picture of the
assessments,
alternatives, and actions
to improve solution
performance.
Recommendation Options
Do Nothing
When the value of a change is low relative to the effort required, or when the risks
of making the change significantly outweigh the risk of maintaining the status quo.
Or, it could be that the resources required to make the change are not available.

Organizational Change
If there are organizational or cultural issues, then it may be possible to influence
the attitudes and perceptions surrounding the solution implementation. This
requires a change management plan. It may also be possible to physically change
the organizational structure or personnel.

Reduce Complexity of Interfaces

Sometimes how work is transferred between systems or people can complex.
Reducing this complexity can improve understanding.
Recommendation Options

Eliminate Redundancy
Stakeholder groups may have common needs that can be met with a single
solution, which reduces the implementation cost.

Avoid Waste
Completely remove those activities that do not add value and minimize those
activities that do not contribute to the final product directly.

Identify Additional Capabilities

Sometimes solutions can offer capabilities beyond those identified in the
requirements. Often these options will have longer term impacts.
Retiring Solutions

A solution or a solution component may need to be replaced. Reasons include:

• Technology has reached its end-of-life.
• Services will now be insourced or outsourced.
• A solution is no longer fulfilling the goals for which it was created (or the goals
no longer exist).

Factors that can impact the decision to retire a solution:

• The ongoing costs are high compared to an alternative with low maintenance
costs.
• An alternative solution may have higher projected benefits.
• There could be market or regulatory reasons.
• Sunk costs should be recognized as such when considering alternatives.
Root Cause Analysis
Root Cause Analysis

• What is the issue we need to address?

• Gather relevant data about the problem.
• Identify the list of actions that contribute to the problem.
• What corrective actions will prevent or minimize recurrence.

Two options for conducting root cause analysis:

1. The Fishbone Diagram
Used to identify and organize the possible causes of a problem. It focuses more on the
cause rather than the solution.

2. The Five Whys

This approach repeatedly asks questions to find the root cause of the problem. It
focuses on issues that involve human interaction.
Fishbone Diagram
Root Cause Analysis Strengths and Limitations
Lessons Learned
Lessons Learned
Lessons Learned - Strengths
Lessons Learned - Limitations