Data Analytics and KPIs for Project Success

TABLE OF CONTENTS
How to use data analytics to improve project outcomes
The Role of KPIs
Measuring Team Strength and Performance
Project-Performance-Based KPIs
Team-Based KPIs
Steps in Determining Appropriate KPIs
About the Author
How to use data analytics to improve
project outcomes

Worldwide revenues for big data and business analytics is expected to grow
to more than $187 billion by 2019 and the project management industry is
projected to hit $5.81 trillion by 2020. Here is how some organizations are
leveraging data analytics to improve their project performance.

Ted Friedman, vice president and analyst at Gartner, predicts the following
four trends will drive fundamental changes in the use of data and analytics:

Instead of just reflecting business performance, data analytics will become

the driver of operations.
Data and analytics will become infused in an organization’s architecture
from end to end, creating a holistic approach -- and this will include
 strategic project management in EPMOs (enterprise program management
offices).
Executives will use data and analytics for business strategy and growth,
creating new additional roles for professionals.
Experts share insights on how data improves project performance

Companies of all sizes have been using data analytics to seek out
opportunities, reduce costs, create efficiencies, make better and faster decisions,
and ultimately increase customer satisfaction; this also translates at the project,
program and portfolio levels since these greatly enable company-wide strategy.

At the Chicago Bulls, Matthew Kobe, director of analytics, says its Business
Strategy and Analytics team uses consumer insights to drive the strategic
direction of the organization. They use data analytics to focus on three key areas
of insight -- fan levels, business transactions, and digital engagement -- to inform
the organization’s strategic choices. He shares more about their focus on the
three areas below:

1. Fan Level Insights -- The Bulls are building a robust CRM and data
warehouse solution that delivers a more holistic view of our fans. “We seek
to understand psychographic elements that help us to understand why a
person is engaging and transacting with the Bulls,” says Matthew. They
also want to “understand satisfaction and areas for improvement by
capturing fan specific feedback on all elements of the fan experience.”
2. Transactional Insights -- The team analyzes all business transactions
including ticketing, concessions, and merchandise, and wherever possible,
Matthew says “We tie these transactional elements back to the fan to build
out a more complete customer view.”
3. Digital Engagement Insights -- “The Bulls have a significant digital
presence illustrated by the second largest social media following for any
 sports team in North America,” says Kobe. Because of this, they work to
understand the types of content fans are engaging with and how those
engagements drive their fans downstream behaviors. They again make
every effort to link engagements back to the fan to help their continued
effort to further expand on their customer view.

“With these three areas under our purview, we are able to more effectively
influence change across the organization. Specifically, we have impacted nearly
every area that influences a fan’s experience with the Bulls: Ticketing,
Sponsorship, Digital Content, Marketing, and Concessions,” he says.

Jason Levin, vice president of Institutional Research at Western Governors

University (WGU), also shared how they use data analytics to create project
wins. “Conceptually, the most important data for project success is having a
measurement plan that includes implementation fidelity and efficacy,” he says.

He suggests answering this question: “How do we know we are doing what

we intended to do?” and “How do we know if what we did worked?” Jason
elaborates further on their methods for measuring implementation fidelity and
efficacy.

For implementation fidelity, WGU has used many methods, ranging from
analyzing log data of student sessions with electronic learning materials to
having faculty use hashtag notations in the student notes.

For efficacy, “our bias is to use randomized control trials, but we also use
quasi-experimental methods. The most important data is to have a clearly
defined outcome variable that can be reliably measured. Western Governors
University (WGU) has a competitive advantage with outcome variables
compared to traditional higher education institutions. At WGU, all our
assessments are centrally developed to rigorous standards. This system of
assessment produces much more reliable data than having faculty individually
assigning letter grades.”

He also describes another unique aspect of data at WGU - its “domain

taxonomy or hierarchy of learning outcomes mapped to learning materials and
assessments. Student learning behavior can be mapped between the electronic
course materials and assessment. Formative assessment data is more predictive
of success on the high stakes assessment than simple page views.”

To make the best decisions, companies need to be able to extract precise and
relevant information from the data available. Absent this, raw data, no matter the
quantity, serves no purpose. Ultimately, companies are seeking the type of
information that tells them what their customers want most and is critical for
guidance on project initiatives, direction, execution, and metrics.

How are companies using data

analytics to improve project
outcomes?
No matter what the industry, from technology to sports or education, data
analytics has become an essential tool for enabling successful project outcomes
and ultimately company-wide strategy.

“We use data analytics to examine almost everything about our platform,
including how many times our users request customer support, says Jonathan
Rodriguez, Founder, and C.E.O. at BitMar Networks. “The first thing that we
realized was the more solutions we offered before our users even requested
them, the less our users requested customer service”.
 He has confidence that by implementing data analytics, BitMar found a
completely new approach to recruiting. Data told BitMar that “your users do not
need your tech support, they prefer to talk to one another, instead. So, provide
that functionality and let them be.” This highlighted the need for the company to
hire community enthusiasts instead of customer service staff.

BitMar embarked on a project to develop a self-help platform for customers.

“Who would have thought that we would have been able to provide a platform in
which the users get to help themselves, at virtually zero cost on our end? ” says
Rodriguez. Data analytics not only helped BitMar zoom in on the types of
projects they should be taking on, it also identified opportunities within projects
to improve customer satisfaction and still reduce internal costs.

Jason Levin (WGU) says that “probably the most successful project to date
has been the Leadership and Communication course designed to educate
students along the affective domain. Using quasi-experimental methods, we
demonstrated significant improvement in retention and credit accumulation.
Based on that research the course was implemented in the undergraduate Health
Professions programs, which now serves about 1,000 students per month.”

When it comes to the Chicago Bulls, Matthew says, using fan level and
transactional insights to do an initial customer segmentation of their ticket
buyers was a top priority over the last year. “We wanted to understand whether
we had any vulnerabilities across segments and any gaps in our product
portfolio.” Specifically, he says they identified opportunities to further develop
fans that fall into the young professionals and families segments and took fan
level insights to further build out personas for these segments to help functions
understand how to engage them.”

Further, the Bulls used these consumer insights to accomplish the following:

Identify opportunities to further develop each of the segments

 As the functions built out strategic plans, the Strategy and Analytics team
was able to partner with them to establish metrics to evaluate success
Develop a new charity event targeted at young professionals and,
Make modifications to ticket products with a greater emphasis on creating
Bulls Snapchat content

What are the limitations of working

with data analytics?
The Chicago Bulls Strategy and Analytics team learned two important
lessons.

They had to begin with “why.” -Why do we want to capture certain data
points and what are the resulting use cases. “We have very limited
opportunities with our most important fans to capture data. We need to
ensure that we are capturing data that will advance our consumer insights
and provide opportunities to more personally engage our fans in the future,”
says Matthew.
Finding the right time to use technology to sustain and accelerate a process.
“We have found that leading with technology results in lower adoption and
force fitting the technology into a less efficient process. By outlining the
process and bootstrapping an analytical solution, we are better equipped to
evaluate technology options and select one that really pushes the
organization forward.”

“There is a quote that has been attributed to Albert Einstein that says ‘not
everything that can be counted counts, and not everything that counts can be
counted,’” says Jason Levens, of Western Governors University. “In education,
this is very true. Understanding what is going on with student and faculty
psychology is critically important but difficult to measure. This is especially true
if you are trying to measure these concepts in real-time and not relying on survey
instruments. It is clear, by the research generated by scales like grit or mindset
how important these data are to educational outcomes.”

Bottom-line
With the data analytics and project management industries growing at an
explosive rate, it only makes sense to use both powerful tools in combination
and interwoven into a company’s fabric to create a more sustainable competitive
advantage. The following section discusses the role of Key Performance
Indicators in measuring project success.
The Role of KPIs

Key performance indicators do a lot more than provide ratios for measuring
progress and success. Key performance indicators play an indispensable role in
project management and aid in measuring success or failure of project activities.

Key performance indicators serve as a key measure of how effectively

project or project activities have performed in relation to identified, required,
and agreed-to strategic objectives. Without KPIs, the success of projects and
project activities is simply a guessing game, with no way to quantify goal
attainment. They offer companies a way to do the following:

Identify and reinforce the link back to strategy. Ultimately key performance
indicators help organizations either affirm or refute project efforts,
 processes, and deliverables in relation to the company-wide goals and
vision.

Provide insight to all stakeholders and team members as to their roles and
expectations. This is a very important factor; it keeps everyone on the same
page throughout project execution and avoids surprises and confusion down
the road. KPIs shouldn’t be a secret; they should be shared with everyone
involved to enable the best possible outcome.

Determine the “lead” or “lag” in performance. KPIs offer a way to monitor

performance along the way and provide visibility. KPIs provide project
teams with a heads-up when things start to lag behind. It indicates where
things are going right and where they’re falling short.

Great thought leaders should always strive toward having advance awareness
and multiple options as well as being as abundantly prepared to adopt project
strategies that fully align with company-wide strategies.
Measuring Team Strength and
Performance

Understanding and selecting the best suited KPIs to measure project-team

performance may not always be easy. As mentioned earlier, the types of KPIs
will vary depending on critical success factors (CSF), like industry, product, or
service and stakeholder business and operational strategies; nonetheless, KPIs
must be clearly identified, achievable, and quantifiable in an appropriate form
and within specified parameters in order to be completely suitable.

As a general example, if one CSF of a new software business is to develop a

web-based portal that will garner a specified number of new customers to sign
up for a monthly subscription service, then possibly one type of KPI may be
quantitative in nature, and it could be new customer subscriber tallies. When
determining KPIs, clearly and accurately define critical success factors first and
then determine KPIs that most accurately directly tie back to the CSFs. Again
both the CSFs and KPIs must be clearly defined and agreed to by stakeholders
and ranked in order of importance to the overall business strategy and goals.

Types of Project Management KPIs

for Team Performance and Strength
Financial gauges: These are typically utilized when the measure is
monetary.

Quantitative gauges: These are typically utilized when the measure is not
financial but is a numerical count.

Qualitative gauges: These are utilized when the measure is not numerical
in nature.

Process gauges: These can be used when measuring efficiency levels.

Team-performance gauges: These can be a form of qualitative gauge for

measuring team performance and strength

KPIs that are likely the most suitable for measuring team performance and
strength would typically be qualitative in nature; depending on the nature of the
project, product, service, or industry, this could also be a quantitative or process
gauge also, since team and individual activities may involve the utilization of
specific processes or tallies in order to accomplish goals. How these KPIs are
identified and utilized will determine how effective they are as a measure.
Project-Performance-Based KPIs

Project-performance-based indicators can measure various factors such as

costs, progress, value, performance, and other success factors. Here are just a
few of the more commonly used key performance indicators in project
management:

Cost Performance Index (CPI)

Cost Schedule Index (CSI)
Actual Cost (AC)
Budgeted Cost of Work Scheduled (BCWS)
Cost Variance (CV)
Earned Value (EV)
Estimate at Completion (EAC)
Planned Value (PV)
Planned Value (PV)
Return on Investment (ROI)
Schedule Performance Index (SPI)
Schedule Variance (SV)
Team-Based KPIs

Team-based KPIs should attempt to capture least some of these (as

applicable):

Project and task dependencies and handoffs among team members

The level of team-member independence and interdependence

Demonstrated leadership abilities
Ways for assisting one another with varying workloads when applicable
Ways for supporting one another through issues and resolving conflict
Absenteeism and impact on team workload and schedules
Actual standards of work quality in relation to the requisite standards
Level of participation and commitment to project work as evidenced by
inputs and outputs
Level of buy-in, attitude, and interaction throughout a project
Levels of clear and effective team member communication in various forms
Ability to meet project deliverables and keep within the project scope
Steps in Determining Appropriate
KPIs

Step One: Consider Outlining the

Team “To Be” List
When trying to determine KPIs for measuring your project team
performance, strength, and overall synergy think about your expectations and
requirements of a successful team in relation to project and business-
requirements objectives. It is important to know as precisely as possible what
you are looking for before you can decide how to accurately measure it. Give
consideration to at least some of the following:

Determine what specific individual team member characteristics, qualities,

and skills you are looking for.

Consider what a successful team should look like in terms of how they
work together, interact, and communicate, as well as the level of
professionalism, commitment, participation, and group dynamics, among
other things.

Factor in how members are anticipated to interact and communicate with

other stakeholders external to the team, such as front-line staff, as well as
management, vendors, and clients.

Identify individual and team-based factors you believe can positively or

negatively impact project outcomes.

Determine exactly how you envision the team working together, as well as
with other stakeholders, to accomplish project and overall business
objectives.

Gain input from company management as well as team members and

stakeholders prior to solidifying KPIs. This can greatly aid in smoothing
buy-in from all parties.

Outlining this “to be” list should help greatly in determining KPIs for
effectively and accurately measuring team performance and strength. As
mentioned above, regardless of their nature, remember that KPIs must be clearly
identified, achievable, and quantifiable in an appropriate form and within
specified parameters in order to be fully achieved.

Step Two: Zoom In on Important

Selection Criteria
To further solidify and expand on your requirements from your team and its
members, layer in the ten PM knowledge areas (scope, time, cost, quality, human
resource, communications, risk, procurement, and stakeholder management) and
try to identify exactly how your team influences each of them and is in turn
influenced by integrating them. This will aid in ensuring that you have assessed
and incorporated all applicable areas into your selection criteria.

Step Three: Determine Which KPIs

Work as an Accurate Measure for the
Required Need(s)
Bernard Marr, a leading global authority and best-selling author on
organizational performance and business, has written a book specifically on
KPIs. In Key Performance Indicators (KPI): The Seventy-Five Measures Every
Manager Needs to Know, he outlines twelve KPIs specific to measuring
employee performance. Although they are employee-based in general, some of
the following may be adapted to measure project team performance:
 Employee Satisfaction Index
Employee Engagement Level
Staff Advocacy Score
Employee Churn Rate
Absenteeism Bradford Factor
360-Degree Feedback Score
Training Return on Investment

Project-team-based KPIs enable businesses to accurately measure project-

team performance to ensure that teams are optimally working together to achieve
project objectives in ways that can be directly tied back to company-wide goals
and strategy. Remember, all KPIs must be appropriately identified, reachable,
and realistically measurable, as well as clearly communicated, documented, and
agreed to.

There need not be a divide or distinction between thought leaders and project
leaders. The ability to effectively execute on business strategy through projects
should always remain a primary focus.

==============================================
All content: Copyright 2017 by CIO.com—IDG Enterprise Inc., 492 Old
Connecticut Path, Framingham, MA. 01701.
About the Author
 Moira Alexander, PMP, I.S.P., ITCP/IP3P

Moira is a media recognized thought leader, the author of “LEAD or LAG:

Linking Strategic Project Management & Thought Leadership” and Founder &
President of Lead-Her-Ship Group. Moira has 20+ years in business (IS&T) and
project management for small to large businesses in the US and Canada. To find
out more about Moira, go to www.leadhershipgroup.com.

She is a certified IT and Project Management Professional, a project

management and IT freelance columnist for publications like CIO and
TechRepublic. Moira also creates ghostwritten thought leadership content for
executives and blog content for businesses worldwide under her byline.

Moira is a co-host and contributor for the Price of Business Talk Radio
“technically speaking” segment, part the BizTalkRadio Network.
Learn More at Atavist.com