Project Management Institute

Building professionalism in project management.®




CONTENTS

1 OVERVIEW

3 THE ROLE OF METRICS IN BENCHMARKING

4 BENCHMARKING IN PROJECT MANAGEMENT

5 THE METRICS

9 CONCLUSION

10 ABOUT THE PROJECT MANAGEMENT INSTITUTE

(PMI)

10 A SPECIAL THANKS TO OUR CONTRIBUTING

EDITORS



OVERVIEW
Robert C. Camp, in his book, Benchmarking: The Search for Industry Best Practices that Lead to Superior
Performance, states that benchmarking, by way of a working definition, can best be described as “the search
for industry best practices that lead to superior performance.”

Benchmarking encourages an external view to ensure the correctness of setting objectives and developing
the internal actions necessary to achieve those objectives. It involves key process steps that are indigenous to
any industry:

B ENCHMARKING P ROCESS S TEPS

1 IDENTIFY WHAT IS TO BE BENCHMARKED

PLANNING

2 IDENTIFY COMPARATIVE COMPANIES

3 DETERMINE DATA COLLECTION METHOD

AND COLLECT DATA

4 DETERMINE CURRENT PERFORMANCE "GAP"

ANALYSIS
5 PROJECT FUTURE PERFORMANCE LEVELS

6 COMMUNICATE BENCHMARK FINDINGS AND

GAIN ACCEPTANCE

INTEGRATION
7 ESTABLISH FUNCTIONAL GOALS

8 DEVELOP ACTION PLANS

ACTION

9 IMPLEMENT SPECIFIC ACTIONS AND
MONITOR PROGRESS

10 RECALIBRATE BENCHMARKS

LEADERSHIP POSITION ATTAINED

MATURITY
PRACTICES FULLY INTEGRATED INTO PROCESSES

Illustration reprinted with permission from Robert C. Camp, author of Benchmarking: The Search for
Industry Best Practices that Lead to Superior Performance (ASQC Qualtity Press, 1989).

Effective Benchmarking for Project Management 1

 Executives have long recognized that it is not acceptable to stand still in a world where change can lead to
loss of competitive advantage in the blink of an eye. Leaders must monitor current performance and
establish programs of improvement that continually enhance the performance of their organizations.
Benchmarking recognizes and addresses the fact that you cannot effectively manage what you cannot
measure. Accordingly, it is heavily focused on self-awareness gained through measurement and the use of
key metrics.

For an organization dedicated to substantial improvement, the benefits of benchmarking are manifold.
Among the most important are:

The continuous assessment of an operation’s performance against that of its competitors.

The adoption of world-class practices to improve performance and competitive advantage and gain
superiority.

The facilitation of breakthrough thinking by direct observation of what has been possible elsewhere.

The mitigation of risk associated with change, since the change is built on the observations of what has
allowed others to succeed.

Leaders must set the improvement agenda by defining the areas for improvement and setting the priorities.
Secondly, they must champion actions based on data (metrics) and organizational self-awareness. They must
also ensure or be sure that the metrics in use in the organization are truly diagnostic of the processes deliver-
ing value and relevant to the value proposition of the operating units.

As an example, or a case study in the aggregate, Dr.William Ibbs and Justin Reginato in their research to
benchmark the Value of Project Management documented that, on average, companies that are less mature
in their project management processes miss schedule targets by 40 percent. Additionally, they miss their cost
estimates by as much as 20 percent. Their research, contained in their book, Quantifying the Value of Project
Management, further documents that the cost of project management in less mature companies ranges
between 11 and 20 percent as compared to more mature companies, where the range is from six to seven
percent. Who can afford to leave this potential cost savings untapped? Clearly the research by Ibbs and
Reginato provides a compelling argument for an organization to self-assess its project management maturity
and set goals to achieve the maturity levels where the return on investment (ROI) is at a minimum.

As suggested by James D. Young, senior consultant at Indeco Limited, benchmarking is essential for effective
project management. Young states that benchmarking can take many forms and can be undertaken for a
broad range of reasons:

To establish performance in comparison to competitors – determining whether a company’s project

delivery capability is as effective as its competitors’.

To determine whether the performance of the organization is improving – monitoring a metric over a
period of time to determine whether improvements are being achieved.

To establish performance by comparison with companies or organizations that have a similar focus – for
example, measuring the competency of project management staff. This is often done where no industry
standard is available for the metric.

To monitor the effect of an initiative, for example, training on the performance of projects.

Good benchmarking, whether for measuring the success of a specific project or an organization’s overall
maturity and/or performance levels, is dependent on selecting the right set of metrics for useful and
productive measurement. The purpose of this report is to provide the perspective of five noted contributors
in the field of project management benchmarking, and to present a balanced set of metrics to executives who
have a keen interest in project management. Some or all of these metrics may be applicable in helping
leaders compare the effectiveness of their organizations against key competitors within their industries.
These metrics, however, are only a subset of what can be considered relevant. It clearly remains the
responsibility of the executive to ensure the proper selection and use of metrics that are most relevant to
his or her organization and the identified goals for improvement.

2 Effective Benchmarking for Project Management




THE ROLE OF METRICS IN BENCHMARKING
“A company’s portfolio of projects is the vehicle for developing and delivering
new products and services to clients. Whichever it is, the company needs to
know how well its portfolio is performing.”
James D. Young, Senior Consultant, Indeco Limited

Young believes that the measurement and monitoring of specific areas of performance allows management
to determine whether desired performance levels are being achieved or whether improvements are being
made. Crucially, it also allows failures in performance to be identified and management actions to be taken.
These measures are commonly known as “metrics.”

Young further states that metrics can either be empirical, as in “number of concurrent projects,” or subjec-
tive, as in “client satisfaction.” They can also be classified as input or output measures. Input measures, for
example, “highly competitive project managers,” indicate that projects should perform well, while output
measures indicate that projects are performing well, such as “return on investment.”
EMPIRICAL

Risk Profile Return on

of a Project Investment
SUBJECTIVE

Project
Client
Management
Satisfaction
Competency

INPUT OUTPUT

Figure 1: Examples of metrics by classification (Graph provided courtesy of James D. Young)

Metrics that have an empirical basis are generally more reliable and easy to update as the data can generally
be derived from project control, finance and human resources systems. While metrics do not have to have
an empirical basis, efforts must be made to ensure that subjective measures are described in sufficient detail
to allow them to be applied consistently every time.

Adds Young, “When using metrics and considering which are most relevant to your particular organization
and industry, one should be well aware that, even though metrics provide a valuable window into underly-
ing organizational strengths and weaknesses, they should be considered as ‘means’ not ‘ends.’ To have real
meaning, metrics must be compared with a reference point. This process is known as benchmarking, and it
is critical in the management of any organization.”

Effective Benchmarking for Project Management 3




BENCHMARKING IN PROJECT MANAGEMENT
The identification of appropriate benchmark metrics for project management requires an identification of
the objectives of benchmarking, according to Mark E. Mullaly, PMP, president of Interthink Consulting
Incorporated. He believes that benchmarking has an underlying goal of improvement. He further believes
that in looking at the improvement of project management capabilities, there are three key dimensions that
drive organizational performance of project management:

Process Maturity. Process maturity defines the quality, rigor or level of performance of an overall process.
In effect, it is a measure of the quality and capability of a process.

Process Effectiveness. Process effectiveness does not evaluate the quality of a process for its own sake, but
instead looks at how useful and relevant the process is in supporting the specific types of projects being
conducted and the overall culture of an organization. Do the processes make sense? Are they appropriate
for the size and type of projects being conducted?

Project Effectiveness. Project effectiveness explores how successful projects are in delivering on their
objectives. It looks at the traditional performance measures of schedule, cost and resource effort against
original project baselines, and whether the project delivered on its expected scope and outcomes.

“All too often, the benchmarking of project management looks only at project effectiveness – are we
delivering projects on time, on budget and to specification, and how much better are we at this than our
competitors? – while ignoring the effectiveness and maturity of the underlying processes,” says Mullaly.
“The latter points are particularly relevant to the services sector, where the results of a project tend to be less
tangible than other market sectors, and the need to be able to reliably and measurably deliver positive end
results is therefore that much more important.”

“While financial outcomes are certainly important, a balanced set of benchmarking

metrics should also include measures for people and their capabilities, for
organizational processes and practices, and for achievements in the marketplace.”
Dr. Terence Cooke-Davies, Managing Director, Human Systems Limited

Ten common metrics for effective benchmarking in project management are listed below, along with points
to consider when evaluating the usefulness and applicability of the metric. As there is no single set of metrics
that applies to all organizations, consider these metrics generic. It therefore becomes essential that each
organization choose the performance indicators that reflect its own unique strategies and situations. They
are not so much a “top ten” list but more a list of essential measurement categories that can be helpful guides
in an organization’s project management planning.

4 Effective Benchmarking for Project Management




THE METRICS
PROJECT COST

William Ibbs, professor in the department of civil and environmental engineering at the University of
California at Berkeley and also president of The Ibbs Consulting Group, believes that an organization must
know how much is invested in project management to know if gains attributable to project management are
appropriate. This involves tracking a broad range of cost factors including salaries, wages and benefits of
project managers and project support personnel; the information technology costs of project management
tools; and the amortized value of training, consulting, building rent, travel, etc.

“Other costs factor in as well, such as the cost of quality,” says James S. Pennypacker, director of the Center
for Business Practices at PM Solutions. Pennypacker defines “cost of quality” as the amount of money a
business loses because its product or service was not delivered correctly in the first place. It includes total
labor, materials and overhead costs associated with delivering products or services that fail to meet
specifications or customer expectations.

Both Ibbs and Pennypacker believe that measuring cost efficiency, by means of a Cost Performance Index
(CPI), is another useful tool within the project cost metric. According to earned value analysis, which
integrates scope, cost and schedule measures to monitor project performance, the CPI is calculated by
dividing the earned value of the project by the actual cost. Both feel this is an efficiency metric since it
provides a measure of the value a project has generated per dollar spent. The standard deviation of CPI is a
useful measure to gauge an organization’s ability to estimate costs accurately.

PROJECT SCHEDULE PERFORMANCE

Another useful project schedule metric emanating from earned value analysis is the Schedule Performance
Index (SPI). SPI is defined as the earned value divided by the planned value delivered by the project. As in
CPI, the standard deviation of SPI is a useful metric for establishing the organization’s ability to schedule
accurately. Both Ibbs and Pennypacker concur that the ability of an organization to estimate costs and
schedule accurately enables it to make the most efficient use of its resources, both human and capital.

Gaining greater insight into cost and schedule performance was the objective of a detailed 1994
benchmarking survey authorized by B.C. Hydro, headquartered in Vancouver, British Columbia, Canada.
The survey’s purpose was to understand the key drivers that account for differences between average and
industry-leading performers in the electric utilities industry. The study sought to identify those companies
that had found ways to significantly reduce project management costs relative to the other companies
surveyed, while maintaining a higher-than-average service level. The study provided numerous insights
about project management, which are briefly summarized as follows:

Increasing the sophistication of project management is a worthy and wise investment, i.e., project
spending variances were much lower when more sophisticated forms of project management were used.

Project managers in companies with specialized project management organizations handled more
projects at the same time.

Companies were more likely to meet cost targets than schedule targets.

The engineering function is the least cost and schedule conscious.

Leading performer attributes included establishing a well-defined capital planning process; a strong
corporate commitment to the project management concept; allowing a substantial level of project
manager control; preparation of a detailed project plan; and flexible and responsive supporting systems for
monitoring, controlling and adjusting project parameters through the life of the project. Brunner,Walter;
and McLeod, Doug. “Benchmarking Provides Insights on How to Improve Project Management
Performance,” Project Management Institute Annual Seminars & Symposium, October 16-18, 1995,
New Orleans, Louisiana, USA.

Effective Benchmarking for Project Management 5

 RETURN ON INVESTMENT

Any organization involved in project management must at some point determine what the value of project
management is to its operation. Return on investment, defined by Young as “a calculation of the return
(additional revenue or projected revenue) that undertaking a project will achieve over a given period of
time,” is one way of determining this value.

Pennypacker believes that the most appropriate formula for evaluating project investment (and project man-
agement investment) is net benefits divided by cost. “By multiplying this result by 100, you can determine
the percentage return for every dollar you have invested,” says Pennypacker. “The key to the effectiveness of
this metric is in placing a dollar value on each unit of data that can be collected and used to measure net
benefits. This data can include contribution to profit, cost savings, an increase in quality of output converted
to a dollar value, etc. Costs could include project design and development costs, cost of resources, cost of
travel and expenses, overhead etc.”

STAFFING
“Senior management plays a vital role during a project. Organizations where senior
management is involved only at the outset of a project demonstrate a much less
mature overall process than do those organizations where senior management is
involved throughout the duration of the project.”
Mark E. Mullaly, PMP, President, Interthink Consulting Incorporated

Effective project management requires an adequate staffing of project personnel. “People are the most
critical project management resource,” says Ibbs. “Project managers oversee project teams and are aided by
project support personnel. Organizations need to be certain that they have not only the optimum number of
staff but also the appropriate personnel ratios among those responsible for, and involved with, all aspects of
project management.”

Pennypacker believes that employee morale is also critical. He believes every organization should explore
the turnover rate of its project managers. If it is significant, executives should know why. He recommends
measuring morale by using an Employee Satisfaction Index (ESI). An ESI comprises a mix of soft and hard
measures that are each assigned a weight based on their importance as a predictor of employee satisfaction
levels. The ESI should include the following (percentage represents weight): climate survey results (e.g., pay,
growth opportunities, benefits, stress levels, supervisor competence, trust, etc.) (35 percent); focus group
results (10 percent); rate of complaints/grievances (10 percent); stress index (20 percent); voluntary turnover
rate (15 percent); absenteeism rate (five percent); and the rate of transfer requests (five percent).

Equally important in matters of staffing is which area of the organization has responsibility for resource
assignment. “An independent approach to resource management, where project staffing is coordinated
through a human resources function, just as with organizational staffing approaches, results in a significant
increase in overall project performance and strong recognition and respect for project management within
the organization,” says Mullaly.

Mullaly adds that a standardized definition of roles and responsibilities for those involved in project
management should also be established. When roles and responsibilities are clearly defined, there is
measurable improvement in overall process performance.

“Ensuring growth within these roles is important as well,” says Dr. Terence Cooke-Davies, managing
director of Human Systems Limited, “Organizations must regularly measure how project staff develop their
own personal knowledge and competence, as well as how frequently and effectively these individuals are
encouraged in the areas of creativity, learning and innovation.”

6 Effective Benchmarking for Project Management

 PRODUCTIVITY
We all want our money’s worth, which is why the productivity metric is so important. Pennypacker defines
productivity as “output produced per unit of input.” He believes that productivity measures tell you whether
you are getting your money’s worth from your people and other inputs to your organization.

Pennypacker suggests that a straightforward way to measure productivity across the board is to use revenue
per employee as the key metric. Dividing revenue per employee by the average fully burdened salary per
employee yields a ratio, which is the average-per-employee “Productivity Ratio” for the organization as a
whole. The key to selecting the right productivity measurements, he adds, is to ask yourself whether the
output being measured (the top half of the productivity ratio) is of value to your customers or key
stakeholders. Cooke-Davies suggests that gross profit per employee is, likewise, a “bread and butter” metric.

Why is measuring productivity important? According to a study conducted by Fred Blanchard and
Robert Hassold in 1995, a fiber glass company used standard productivity measures to evaluate its ability
to deliver product cheaply and on schedule. Using the original productivity measures as a benchmark, the
company introduced an incentive program linked to worker performance on a project. For every 40 hours
worked, an employee received shares that could be redeemed at the end of the project if schedule, budget
and other critical factors were met. At the end of the test project, the company found that production was
two months ahead of schedule, productivity had increased by 2.5 percent and a portion of the budgeted
workforce of 50 on that project was reassigned, allowing for more effective use of personnel.Workers
redeemed their shares for cash incentives based on the amount of cost savings achieved. Blanchard, Fred;
and Hassold, Robert. “Enhancing Productivity Through Incentives,” Proceedings from the Project
Management Institute Annual Seminars & Symposium, October 4-10, 1996, Boston, Massachusetts, USA.

PROJECT CYCLE TIME

According to Pennypacker, the project life cycle defines the beginning and the end of a project. Cycle-time
measures are based on standard performance, meaning similar projects can be benchmarked to determine a
Standard Project Life-Cycle Time. Measuring cycle times can also mean measuring the length of time to
complete any of the processes that comprise the project life cycle.

“The shorter the cycle times, the faster the investment is returned to the
organization. The shorter the combined cycle time of all projects,
the more projects the organization can complete.”
James S. Pennypacker, Director, Center for Business Practices, PM Solutions

Mullaly adds that there is a strong correlation between process maturity and the means by which project
completion is evaluated. Where project completion criteria are defined, either formally or informally, there
is a noticeable improvement in process performance and maturity as compared to organizations that do not
define completion criteria.

The Idaho National Engineering and Environmental Laboratory developed an expedited environmental
management process to address the fact that environmental clean-ups were costing too much and taking
too long. A team analyzed 20 clean-up projects to identify the factors associated with successful clean-ups.
The new process that evolved included repeatable, measurable processes based on EPA requirements; and
combined systems engineering and project management processes for better control. Blacker, Paul B.; and
Winston, Rebecca. “Integration of Project Management and Systems Engineering: Tools for a Total Cycle
Environmental Management System,” Proceedings from the Project Management Institute Annual
Seminars & Symposium, September 29-October 1, 1997, Chicago, Illinois, USA.

Effective Benchmarking for Project Management 7

 POST-PROJECT REVIEWS
Young believes that post-project reviews represent a particularly important metric. He states that project
practitioners holding formal reviews of their projects at completion greatly facilitate the process of identify-
ing lessons learned while providing valuable feedback that can be very useful for future projects.

MDS Sciex, an organization new to project management, reported, “…We learned that our lack of a
clear understanding of the execution of the post-mortem process prevented us from gaining many of the
anticipated benefits.” The company decided that to fix the problem, it would hold post-mortems after each
project phase, collect information related to that specific project phase and involve only those people who
contributed to that phase of the project. Through its new process, the organization felt that it would reduce
or eliminate frequently occurring issues and that it would see a renewed interest in participation on project
teams. Rasper, Vlad; Stanier, Michel; and Carluccio, Patrick. “A Real-Life Approach to Lessons Learned,”
Proceedings of the Project Management Institute Annual Seminars & Symposium, October 3-10, 2003,
San Antonio, Texas, USA.

RISK MANAGEMENT
As with just about anything in life, identifying and managing risk makes things go smoother. This is equally
true in project management. Risk management is consistently the single greatest indicator of overall project
process maturity.

Mullaly suggests that particular areas of focus within an effective risk management metric should include a
formal approach to risk identification and assessment, active monitoring of project risk factors throughout
the project, and a commitment to conduct periodic risk reviews during the execution of the project.

“Risks can be classified as either medium or high,” says Young. “It is often helpful in project management
planning and implementation to classify ‘tendency’ risks, in other words, the number of new medium/high
risks and the number of risks reducing in severity.”

Essilor of America, Inc., a leader in ophthalmic optical products, operated in a competitive and dynamic
environment that required acceptance of risk, particularly for market leaders. To identify and evaluate
risk in projects, Essilor adopted a Nominal Group Technique (NGT) that relied on the knowledge and
experience of the company’s diverse range of experts within its workforce of 20,000 employees worldwide.
The company developed a software tool to standardize NGT data and capture risk by specific type, whether
cost, quality, schedule, etc. The risk was then ranked and qualified for analysis and decision-making
by company managers. The project team conducted the initial risk assessment, and management made
the final determination regarding the level of risk exposure the organization would accept. Githens,
Gregory D., and Peterson, Richard J. “Using Risk Management in the Front End of Projects,” Proceedings of
the Project Management Institute Annual Seminars & Syposium, November 1-10, 2001, Nashville,
Tennessee, USA.

ALIGNMENT TO STRATEGIC BUSINESS GOALS

“Most project management metrics benchmark the efficiency of project management – doing projects right,”
says Pennypacker. “You also need a metric to determine whether or not you’re working on the right projects,
in other words, are the projects aligned with strategic goals?”

“One way to ensure your projects are strategically aligned with your organization is to conduct an internal
survey among project management practitioners, business unit managers and executives. You can use a
Likert scale from 1-10 to rate the statement: Projects are aligned with the organization’s strategic objectives.”

Adds Cooke-Davies, “For an organization to attain ‘portfolio success’ within its project management func-
tion, its projects must be aligned with organizational strategy. This includes an alignment between project
spending and corporate strategic goals, as well as the overall corporate level of project delivery against plan,
scope and budget.”
8 Effective Benchmarking for Project Management
 CUSTOMER SATISFACTION
“Delivering consistent customer satisfaction enables an organization to command greater loyalty from its
customers than can its competitors. It’s often the difference between simply doing business… and doing
business well,” says Cooke-Davies.

Young interprets customer satisfaction to mean that customer expectations have been met and that clients
are pleased with the performance of projects. He suggests that this metric can often be measured by using
questionnaires asking clients to rank various aspects of the project team’s performance.

Pennypacker suggests that there is another good way to measure the comfort level of your customers - the
Customer Satisfaction Index. Based on a scale of 1-100, this index comprises hard measure of customer
buying/use behavior and soft measures of customer opinions or feelings. The index is weighted based on
how important each value is in determining overall customer satisfaction and buying/use behavior. This
includes repeat and lost customers (30 percent); revenue from existing customers (15 percent); market
share (15 percent); customer satisfaction survey results (20 percent); complaints/returns (10 percent) and
project-specific surveys (10 percent).

Mullaly believes that there is one other important point to consider in your quest for customer satisfaction,
and that is customer involvement. “Ensuring the involvement of your customers throughout a project
not only contributes to project success but also represents a key indicator of process maturity. Today,
48.5 percent of organizations actively demonstrate the involvement of customers throughout the life cycle
of a project from initiation through completion.”

CONCLUSION
“It is tempting to try collecting and analyzing a lot of data. After seven years of
project management benchmarking, we have concluded that it is better to collect a
few strategic pieces of information, where Information = Quantitative data +
Qualitative, anecdotal and honest opinions.”
Dr.William Ibbs, Professor, Department of Civil and Environmental Engineering,
University of California at Berkeley President, The Ibbs Consulting Group

While benchmarking for project management should be a staple in the management practices of any
successful business or organization, it is important to remember that establishing useful and meaningful
metrics is very much an individual decision. The optimum set of metrics depends on the organization’s
strategies, technology levels, as well as the particular industry and environment in which it competes.

In addition, establishing workable benchmarking metrics should not be a short-term exercise. To get
maximum informational benefit, they should be averaged, or indexed, over a large number of similar types
of projects over a period of time, perhaps for a minimum of one year.

Effective Benchmarking for Project Management 9




ABOUT THE PROJECT MANAGEMENT INSTITUTE (PMI)
With more than 130,000 members in approximately 150 countries, Project Management Institute (PMI) is
the global advocacy organization for the project management profession. PMI sets professional standards,
conducts research, promotes professional development, and provides education, certification and networking
opportunities. For more than 35 years, PMI has advanced the careers of practitioners, ultimately contributing
to the success of businesses and organizations across all industries. For more information, please visit
www.pmi.org.

For more information on benchmarking, please visit the PMI® James R. Snyder Center for Project Management
Knowledge & Wisdom, which can be accessed by visiting the PMI Web site at www.pmi.org.

Project Management Institute

Building professionalism in project management.®

A SPECIAL THANKS TO OUR CONTRIBUTING EXPERTS
The information contained in this document has been assembled thanks to the very able and much appreciated
input of five renowned experts in the profession of project management with whom PMI has both the honor
and privilege of working. These individuals are:

Dr. Terence Cooke-Davies

Managing Director, Human Systems Limited

Dr.William Ibbs

Professor, Department of Civil and Environmental Engineering
University of California at Berkeley
President, The Ibbs Consulting Group

Mark E. Mullaly, PMP

President, Interthink Consulting Incorporated

James S. Pennypacker

Director, Center for Business Practices, PM Solutions

James D. Young

Senior Consultant, Indeco Limited

© 2004 Project Management Institute, Inc. All rights reserved.

“PMI” and the PMI logo are registered marks of Project Management Institute, Inc.
Printed in the U.S.A. 030-006-2004 (05-04)