Technology Portfolio Management

for Project Managers

28 Feb 17
PMI Tysons Corner Chapter
Steven R. Meier, Ph.D., PMP

SRM Consulting LLC © 1

Technology Vernacular

SRM Consulting LLC © 2

Technology Readiness Levels (TRLs)

Key
decision
point

http://www.acqnotes.com/acqnote/tasks/technology-readiness-level

Applies to both hardware and software projects

SRM Consulting LLC © 3
Manufacturing Readiness Levels (MRLs)

Key
decision
point

http://www.dodmrl.com/MRL_Deskbook_V2.4%20August_2015.pdf
4
SRM Consulting LLC ©
Impacts of Poor Technology
Management

SRM Consulting LLC © 5

Standish Group CHAOS Report 2016 (1)

71% of projects in 2015

failed or were challenged

Ref. Standish CHAOS Summary Report 2016 www.standish/reports/reports.php.

Reprinted with the explicit consent of the Standish Group.
SRM Consulting LLC © 6
 Standish Group CHAOS Report 2016 (2)

Medium size
projects had
worst
performance.
57% failed or
were
challenged

Ref. Standish CHAOS Summary Report 2016 www.standish/reports/reports.php

Reprinted with the explicit consent of the Standish Group.
SRM Consulting LLC © 7
Standish Group CHAOS Report 2016 (3)

Ref. Standish CHAOS Summary Report 2016 www.standish/reports/reports.php

Reprinted with the explicit consent of the Standish Group.
SRM Consulting LLC © 8
Cost and Schedule Impacts Due To Requirements
Changes

Average RDT&E cost growth for programs over initial estimates Average delay in providing initial operating capability (GAO-09-
(GAO-09-326SP) 326SP)

Requirement changes can lead to cost growth and schedule delays

SRM Consulting LLC © 9

Immature Technology Impacts

Percentage of programs that achieved critical technology Average program research, development, test, and evaluation
maturity levels at key milestones from GAO-06-391 (2006). cost growth from first full estimate from GAO-06-391 (2006).

Immature technology leads to cost growth

SRM Consulting, LLC © 10

Technology Portfolio
Management Process

SRM Consulting LLC © 11

 Corporate Strategic
Planning
• Technology management starts
with strategic planning by senior
leadership
• This planning guides technology
development and defines critical investment
priorities
• The Strategic Plan must be agile to accommodate
the ever-changing customer needs; the ability to
react to new threats; adjust to fast-paced
technology changes; and changing world events
SRM Consulting LLC © 12
 Segmentation

• From the strategic planning guidance each project is

segmented into its domain thereby delineating its
scope and boundaries
• Segmentation assists in narrowing down each
project according to its respective technology

SRM Consulting LLC © 13

 Generate
Technology
Roadmaps
• A technology
roadmap identifies
technology “roads”
that can be followed to meet a customer’s requirements
• The focus of roadmaps is on finding a set of solutions for a
given set of needs
• Technology roadmaps are critical when the technology
investment decision is not straight forward

SRM Consulting LLC © 14

 Technology
Roadmap
Purposes
• Facilitates the integration
of new technology
• Identifies critical technologies
and technology gaps
• Develops milestones for meeting specific performance targets
• Identifies technology insertion points
• Assists in making informed technology investment decisions
• Serves as a communication tool among upper management,
business development, R&D, and operations
• Supports decision-making, resource allocation, and risk
management SRM Consulting LLC © 15
 Technology Roadmap Considerations
• Technology insertion is used when new technology is
being applied to provide a new functional capability

• Technology refresh applies to new technology being

used to replace an existing function usually because
the previous technology has either become obsolete
or prohibitively expensive to maintain

SRM Consulting LLC © 16

Technology Roadmap Actions
• Assess TRL & MRL
• Time – Faster is better
• Low switching cost
• Technology order of preference:
1. Re-use of existing product, service, or result
2. Commercial off-the-shelf (COTS) solution
3. Government off-the-shelf (GOTS) solution
4. Non-developmental item (NDI) solution
5. New developmental item solution

SRM Consulting LLC © 17

Assess the Value of Each Project
𝑢(𝑡) 1
𝑉𝑎𝑙𝑢𝑒(𝑡) = ×
𝑐(𝑡) 𝑡𝑑

• 𝑢(𝑡) = represents the utility and is the ability to satisfy customer-validated

requirements evaluated against established baselines. MOP or MOE
• 𝑐(𝑡) = is the projected lifecycle dollar outlay necessary to design, develop,
test, certify, deploy, sustain, and operate the capability of interest
• 𝑡𝑑 = refers to the combined calendar time it takes to collect requirements,
evaluate options, perform contract actions, develop, test, certify, and deploy
an increment of capability

Value provides a method to measure projects

within a portfolio

SRM Consulting, LLC © 18

Use Cost Reimbursable Contracts for
New Technology Projects
Federal Acquisition Regulation 16.301 General.
16.301-1 Description.
• Cost-reimbursement types of contracts provide for payment of allowable incurred
costs, to the extent prescribed in the contract. These contracts establish an
estimate of total cost for the purpose of obligating funds and establishing a ceiling
that the contractor may not exceed (except at its own risk) without the approval
of the contracting officer.
16.301-2 Application.
• (a) The contracting officer shall use cost-reimbursement contracts only when—
• (1) Circumstances do not allow the agency to define its requirements sufficiently
to allow for a fixed-price type contract (see 7.105); or
• (2) Uncertainties involved in contract performance do not permit costs to be
estimated with sufficient accuracy to use any type of fixed-price contract.
• (b) The contracting officer shall document the rationale for selecting the contract
type in the written acquisition plan and ensure that the plan is approved and
signed at least one level above the contracting officer (see 7.103(j) and 7.105).
SRM Consulting LLC © 19
Establish Project Selection Criteria
Establish detailed criteria and weightings:
 Does it fall within a focus area (strategic) (15%)
 Benefit to more than one customer or business unit (strategic)
(20%)
 Number of years from initiation to completion (schedule) (15%)
 Percentage of cost sharing/matching funds (cost) (10%)
 Cost reduction compared to current product (cost) (5%)
 Percent capability improvement over the current state
(technical) (20%)
 The number of years it will take generate a top line impact (sales
or orders) (financial) (5%)
 Cost reduction for production, operations, or maintenance of a
product or system (cost) (10%)
SRM Consulting LLC © 20
Develop a Project Portfolio
Project Detailed Price Schedule TRL Value Overall End System
Technical (in TY$) Duration (1-9) Score
Metrics (months) MRL (1-10)
(1-9)

A % $27M 13 5 7 7 Airborne

B ““ ““ ““ ““ ““ ““ IT

C ““ ““ ““ ““ ““ ““ Space

D ““ ““ ““ ““ ““ ““ ““

SRM Consulting, LLC © 21

Portfolio Benefits
– Traceable to and directly support
strategic goals
– Better alignment of human and financial
resources
– Improved productivity
– Efficiency gains
– Greater economic efficiency
– Increased shareholder value

SRM Consulting LLC © 22

Implement a Stage Gate Process
1. A series of discrete phases that translate
system requirements into an integrated project
2. These phases are separated by reviews and
decision points to review progress
3. Implement Go/No Go decisions at reviews and
key milestones
4. Software can utilize spiral or agile approaches
that have reviews to assess progress at each
increment
SRM Consulting, LLC © 23
 Typical Stage Gate Process

Concept Technology
Design Build/Test Operations Disposal
Development Development

Concept SRR SDR PDR ▲PDR CDR TRR CAs

Decision DCR ATP ORR DRR
Decision Decision Decision Decision

= Decision Point = Review Point

DCR = Design Concept Review SRR = System Readiness Review
SDR = System Design Review PDR = Preliminary Design Review
ATP = Authority to Proceed CDR = Critical Design Review
TRR = Test Readiness Review CA = Physical/Functional Configuration Audit
ORR = Operational Readiness Review DRR = Disposal Readiness Review
SRM Consulting LLC © 24
 The Old Adage says…..

If you can’t measure it, you

can’t manage it

SRM Consulting LLC © 25

Collect Metrics to Measure Progress –
Deliverable in Contract
• Dollars expended per month
• # of defects per 1000 lines of code
• Customer satisfaction
• # of bugs per line of code per week
• # of change requests per month
• # of drawing revisions per month
• CPU usage per month
• Total resources utilized per month
• Mean time between failures (MTBF)
• Mean time to repair (MTTR)
• Weight margin as a function of time
SRM Consulting LLC © 26
 Common Metrics Pitfalls
• Metrics where data cannot or will not be collected
• Metrics that are complex, confusing, costly, and
difficult to explain to stakeholders
• Too many metrics – aim for 3-5 key metrics
• Metrics that do not align with the current business
strategy
• Metrics that drive the wrong behavior

SRM Consulting LLC © 27

Keep Stakeholders Informed
• They have political influence
• They can choose to be supportive or resistant
• If your stakeholders hear nothing they will suspect
progress is not being made
• Define the stakeholders, their information needs,
how often to communicate, and what medium
• Weekly emails work wonders
• They can make or break your
project portfolio

SRM Consulting LLC © 28

Feedback Loop
• Review projects monthly in depth
• Feed back into technology roadmaps
and project portfolio
– Update cost, schedule, & performance
– Review project stage gate results This is a continuous
– Reassess Value of each project process that goes
on throughout the
– Review metrics from the project portfolio
entire year!
– Examine industry technology changes
– Incorporate sponsor inputs
– Confirm each project’s position within the portfolio

SRM Consulting LLC © 29

 Summary
• Technology management processes
enable successful outcomes
• Enhance communication throughout the project lifecycle by utilizing
technology roadmaps
• Assess technology maturity using TRLs and MRLs
• Introduce a new metric Value to prioritize projects within portfolio
• Perform rigorous reviews during stage gate process
• Utilize cost-reimbursable contracts for new technology
• Keep sponsors informed of progress
• Implement a feedback loop to assess progress

SRM Consulting LLC © 30

 Contact Info
[email protected]

SRM Consulting LLC © 31