Module 3 PerfMetrics
Module 3 PerfMetrics
Module 3 PerfMetrics
Performance Metrics
and Strategic Planning
for Improvement
1
Agenda
Day 1 Day 2
8:00 A.M. Welcome and Module 3.
Introductions •Performance Metrics
•Strategic Planning for
Improvement
Module 1. Module 4.
Quality Product and Service Design
Management and Development
Overview
Lunch Time
Module 4.
Module 2. Product and Service
Key Human Factors Improvement
Next Steps
4: 30 P.M. Adjourn Adjourn
2
Quality Management Course
Roadmap
1. Quality Management Overview
4. Product/Service
Design & Development and
Improvement
Improve
3
Learning Objectives
• Participants will understand the linkage and integration of strategic planning
for improvement and performance metrics within Baldrige.
• Participants will understand key elements of an integrated causal path
measurement system using the Balanced Scorecard (BSC) to illustrate.
• Participants will understand the two types of improvements.
• Participants will use the Kaplan/Norton BSC Hall of Fame questionnaire, to
critique their organizations and determine improvement areas.
• Participants will understand the Voice of the Customer (VOC) and its linkage
within Baldrige and ISO 9000.
• Participants will understand how quality professionals help to affect top line
results using a perceived value to profit model.
• Participants will understand VOC best practices and tools; critique their
organizations listening posts; critique a VOC measurement plan after learning
a survey/questionnaire feedback process.
• Participants will complete their Learning Log and Action Plan based on ideas
for shaping the future of quality in their own work and in the work they
influence and skills they or their people need to succeed in the future.
4
Performance Metrics & Strategic
Planning for Improvement
The module is divided into sections:
1. Overview of metrics, framework for balanced
performance metrics, strategic planning for
improvement process
2. What is the Strategic Voice of the Customer (VOC)?
3. Evolution of Strategic VOC
4. How strategic VOC fits into management models and
standards (Baldrige, Six Sigma, ISO, Lean)
5. Strategic VOC Plan and Tools
6. Strategic VOC best practices
7. Summary Review
5
Baldrige Criteria: Framework for an Integrated
Management System
7 is the Outcome
6
Metrics, Performance and Linkage
• Strategic objectives are the starting point for all performance
measurement.
• Leaders use metrics to set expectations,
drive actions, measures drive behaviors.
• Metrics are the major means of communication.
both internally and externally; means to align everyone’s
efforts, what gets measured gets managed (or manipulated?!)
• Performance measures are key elements to make decisions,
set direction, and select improvements.
• Metrics are an input, an output, and part of each business
process.
• Measures are only helpful when the organization has the
capacity to respond to them.
7
Select the Right Key Performance
Indicators
• Measures of performance used by management for centuries:
– Grew out of accounting and measures of productivity
– Review current operational capabilities
– Assess both departmental and corporate performance, as well as
trend performance achieved against plan
8
Framework for Balanced
Performance Metrics with Causal
Chains
Questions:
How can we build realistic, practical metrics which help drive change and
gauge progress?
How can we articulate company objectives through enterprise-wide
metrics in an integrated measurement system?
Answer:
A coherent framework of balanced and integrated metrics are needed and
established to:
-Help drive strategy and cover all vital topics in a Business Excellence
Model.
-Define and deploy measures that best judge overall performance; use
both “leading” and “lagging” indicators
-Provide indicators to determine and communicate progress to future
goals.
9
Balanced Scorecard
While there are popular approaches that each provide a valuable view point,
the most widely used is the Balanced Scorecard (BSC) (Kaplan and Norton)
To make informed decisions, business leaders need to understand how their
organizations are performing and where change will have the greatest
impact. Applying aligned and strategic analytics to processes can provide that
insight
• BSC deals with four perspectives: Financials, Customer, Internal /
Business Processes, and Learning and Growth
10
Create Waves of Change
Ways to Link Strategy and Process
Strategic planning for improvement focuses on improving
the vital few processes that support strategy
11
Balanced Scorecard
• Is derived from mission, vision, values, and strategy.
12
Balanced Scorecard
Links Measures to Organizational Strategic Priorities
in a Balanced Approach
Financial Customer
Balanced
Scorecard
Internal/Business Learning and
Processes Growth
• Set of hypotheses about cause and effect measurement system;
should make relationships between cause (performance
drivers/leading indicators) and effect (outcome/lagging indicators)
explicit so they can be managed
13
Balanced Scorecard
Balanced Scorecard: Four Perspectives:
1. Financial Perspective
• Measures of economic consequences of actions already
taken indicate if organization’s strategy and business
operations are contributing to the bottom line
• Typical measures: Return on investment (ROI); economic
value-added; profitability; revenue growth and mix; net
present value (NPV); internal rate of return (IRR); return on
assets (ROA), return on net tangible assets (RONTA)
2. Customer Perspective
• Measures specific drivers and outcomes to the customer
• Typical measures: Customer satisfaction; customer retention;
customer loyalty; customer profitability; customer retention;
market share; new customers
14
Balanced Scorecard
Balanced Scorecard: Four Perspectives:
3. Internal / Business Process Perspective
• Key measures of processes that have the greatest impact
on customers and financial outcome
• Typical Measures: Quality; response time; cost;
productivity; new/revised product introductions
15
Balanced Scorecard-Strategy Map
Link Measures to Strategic Priorities in a Balanced Approach
Mix of Outcomes and Performance Drivers and Casual Relationships
Financial Mission/Profit
Customer Customer
Retention
Customer Satisfaction
16
Balanced Scorecard
Sets and reviews: Objectives, measures, targets, and initiatives
Streamline
appointments Military Families with
Access to 82%
meeting web access to
Care TRICARE access appointments
standards
17
Improvement Types
Vision
Mission
Values
Strategic Plan
Business
Operating Plan
“Kaizen” or Incremental
Improvements “Breakthrough”
Within Quality in Strategic
Daily Work Improvements
18
Exercise: Wall Street Journal
Article Activity
Read the article and be prepared to discucss:
• Environment at the time (what was going on; driving forces)
• Strategy (How):
• Short-term (objectives):
• Motivating actions:(tactics)
19
Possible Answer: Wall Street
Journal Article Activity
• Environment at the time (what was going on)
– External engine failures-customer dissatisfaction, internal defects, cost
of poor quality (COPQ)
• Long-term (5 year) direction (goals):
– COPQ is a cost avoidance of 10-15% of a $70 billion/year revenue,
improve external image and customer satisfaction/competitiveness
• Short-term goals (objectives):
– Annual goals to reduce defects and COPQ
• Enabling actions:
– Implement a Six Sigma program, hire full time MBB, BB, identify
Champions from leadership to select and realize project success,
$100k investment in training and a computer system to track projects
• Motivating actions:
-40% bonus tied to Six Sigma successes; “we have the people who can do
it”
20
Strategic Planning Process
Internal Assessment External Assessment
Mission/Vision/Values Strategy
Strategic Objectives
21
Example
Saint Luke’s Hospital – Strategic Plan 2004-2008
Mission: Why we exist
Saint Luke’s Hospital is a not–for–profit tertiary referral center committed
to the highest levels of excellence in providing health services to all
patients in a caring environment. We are dedicated to medical research
and education. As a member of Saint Luke’s Health System, we are
committed to enhancing the physical, mental, and spiritual health of the
communities we serve.
22
Balanced Scorecard Examples
Saint Luke’s Hospital – Strategic Plan 2004-2008
• Financial
• Customer
• Clinical/Administrative Quality
• People
23
Balanced Scorecard Examples
Saint Luke’s Hospital – Strategic Plan 2004-2008
24
Balanced Scorecard Examples
Saint Luke’s Hospital – Strategic Plan 2004-2008
25
26
Balanced Scorecard Relationships
Organizational Scorecard Process Scorecard
Provides Inputs
Linked to Processes
27
Organizational Scorecard Example
28
Balanced Scorecard Hall of Fame
29
Principles of the Strategy-Focused Organization*
*From 2007 Balance Scorecard Hall of Fame Application. Balanced Scorecard Collaborative, a
Palladium company.
30
Exercise: BSC Hall of Fame Partial
Questionnaire
Complete the “Balanced Scorecard Hall of Fame for Executing Strategy™
Application (modified)” Best Practice Survey
31
VOC- Alpha and Omega of Metrics
32
Performance Metrics & Strategic
Planning for Improvement
The module is divided into sections:
1. Overview of metrics, framework for balanced
performance metrics, strategic planning for
improvement process
2. What is the Strategic Voice of the Customer
(VOC)?
3. Evolution of Strategic VOC
4. How strategic VOC fits into management models and
standards (Baldrige, Six Sigma, ISO, Lean)
5. Strategic VOC Plan and Tools
6. Strategic VOC best practices
7. Summary Review
33
What Is the Strategic VOC?
Definition:
• “Voice of the customer” (VOC)- multi-faceted process used to capture
the ever-changing requirements/feedback from customers; The VOC
is your process for capturing customer-related information
• The VOC is both proactive and reactive and involves tools such as,
interviews, surveys, focus groups, observation, mystery shoppers,
complaints, and others
34
What Are Benefits of VOC?
Benefits
• Delighted, loyal retained customers
• Enhanced market and wallet share and productivity
• Motivated employees
• Streamlined operations that focus on customer needs
• Highly motivated and productive employees
35
What Are Benefits of VOC?
High correlation of employee satisfaction, customer
satisfaction, customer loyalty, revenues and profits
Spearman Rank Correlation of 123 business units in banks, engineering,
transportation, insurance, high tech, restaurants, software, telecom
Cust Sat
Revenue
37
Baldrige Criteria: Framework for an Integrated
Management System
Customer and
Market Results
7 is the Outcome
38
ISO, Baldrige, Six Sigma, Lean all…start
and end with the VOC
C R
S C
a
u e
Management t u
q
s u Responsibility I
s
s
t i
r
f t
Resource a
o e
Measure,
management analyze & improve c o
m
m e
t
i
m
e n
t
Product
realization Product o e
r s
n
r
Reference: ISO 9000:2000
39
ISO 9001:2000 and the Customer
Lean
6 Sigma
Lean Six Sigma
40
Show Me the Money!
Income and an Integrated VOC Process
Choosing Value and Satisfaction
Income
Quality Drivers
Profit
Satisfaction
Expenses
Value Streams
Process Management
Income
Quality Drivers
Profit
Satisfaction
Expenses
Value Streams
Process Management
Delivering Value
42
Advanced Quality Management
and the Customer
• In Advanced Quality Management, Quality
Professionals :
43
2007 Interview Insights from Dr.Feigenbaum
CTQs?
• Some companies have an outdated idea of quality and how to
improve it. Managers think of quality as minimizing defects,
especially in production. This aspect has long been an entry-
level requirement in competition but is no longer enough from a
customer perspective. Customers evaluate quality based on the
value of the entire offer
44
Performance Metrics & Strategic
Planning for Improvement
The module is divided into sections:
1. Overview of metrics,framework for balanced
performance metrics, strategic planning for
improvement process
2. What is the Strategic Voice of the Customer
(VOC)?
3. Evolution of Strategic VOC
4. How strategic VOC fits into management models and
standards (Baldrige, Six Sigma, ISO, Lean)
5. Strategic VOC Plan and Tools
6. Strategic VOC best practices
7. Summary Review
45
Evolution of
Strategic Voice of the Customer
Contributions have been made by many leading specialists.
Importance
of attribute Six Sigma, Lean,
Zeithaml Customer
Value Lean Six Sigma
Baldrige/ Brad Gale CTQs: 1980-90s
ISO
Loyalty/
Satisfaction 3 levels of Retention=
Surveys satisfaction Profits
Popularized Noriaki Kano Frederick
Reichheld
46
Exercise: Plot where you are in
strategic VOC evolution…
X
Have aligned strategic
system, good tools,
visibility, accountability &
application
X
Have systematic multiple listening
posts & good integration of data
X
Have good surveys with benchmark data
X
Have superficial surveys
X
Don’t measure
anything systematically; reactive
47
Best practice example: Pareto Chart
X=80% 80%
40%
Profits #1
#2 #3
#4 #5 #6
20%
0%
Complaints
Wallet
Attrition rate
Share
Customer satisfaction index
X=80%
Defective
80%
60%
Invoice
40% Goods Accuracy
#1
Supplier sat
#2 #3
#4 #5 #6
20%
0%
49
VOC Plans and Tools
Action to Traction Best Practice
Integrated Systems Approach
Management must:
-Develop analytical capability to understand
customers, using multiple listening posts
-Integrate customer intelligence into operational
systems
-Enable rapid responses to opportunities and threats
-Conduct regular reviews
-Communicate, coach and celebrate; close loops
Customer and market intelligence turns data into
measurable features and initiatives that serve to
acquire, develop, and retain customers
50
VOC Plans and Tools
Action to Traction Best Practice
Commitment
Plan & goals
Multiple listening posts
Diligent deployment
Action- short & long term
Celebration
Links come from a system…not a single
survey
51
Customer Listening Post Examples
Note: Better organizations survey more than just their
current customers
• Needs assessment: deep dive interviews
• Perceptual surveys
• Customer satisfaction surveys
• Value analysis surveys
• Transaction surveys
• Customer surveys by mail, telephone, web
• Win/loss surveys
• Mystery shopping
• Advisory and user groups
• Focus groups or interviews
• Suggestion boxes
• Post card transactional surveys
• Customer Information Files
• Call Centers Complaints, Warranty claims, returns…
(See VOC Handout in Reference Section- Survey design best practices)
52
VOC Plans and Tools
Action to Traction Best Practice
Best-in-class integrates multiple listening posts
Mail,
Win-loss
Telephone, Web
surveys
Interviews surveys Call center
queries
Strategic & Product
Comparative Needs Assessment Hard data:
studies (sales, returns)
Client project
Complaints CAPAs reports
55
VOC Plan, Steps 3. Select, Test, Deploy & Analyze
1.Tool selection & design
&Tools •Written, phone, web surveys
4. Strategic
•Focus groups
Quality Plan
Target Digging & Action
•Interviews
2.Segment 3.Analyze •Sales input
1.VOC Plan •Lost customer surveys
Market/ Needs •Complaint tracking •Celebrate
Customer •Moments of Truth
•Inventories •Integrate
Find out •Perceptual studies marketing
•Mission & vision Products •Buying needs •Transaction studies messages
•Trends Revenue •Usage needs •Attrition rates
Profit •Product
•Retention needs •Intercept studies
•Objectives for CSS
AUM design/dev/
•Competition •Process measures
changes
•Analysis of business etc. •Other??
• Project plan Key quality
2. Test tools initiatives
3. Tool deployment plan
4. Deploy
•Standards
5. Ad hoc research
•Requirements
6. Analyze & Integrate •Benchmarks
synthesize results •Lean, 6Sigma
56
Four Critical Elements of VOC Plan
1. Target customers & products 2. Tools & deployment
Surveys, Focus groups
Attrition measures
3. Responsibility matrix G an tt chart
4. Gantt C h art
Step Who Input JAN FEB M AR APR M AY JU N JUL AUG SEP OCT NOV DEC
primary from C u s t S a t P la n
L o s t c u st s u rv e y
2. Do Survey & S. Kessler XXX
Complaint Analysis XXX R e tre a t &
d e c is io n s
4. Review D. Owens
milestones
57
VOC Plan, Steps &Tools: Surveys
Objectives
Who
Prioritize How
Check Analyze Do
Ask
58
VOC Plan, Steps &Tools: Surveys
60
VOC Measurement Plan (or lack of)
Exercise-Your Turn
Read your assigned case(s) and be prepared to discuss:
1. What objectives are implied in the following case studies?
2. What are the good points with the following tool selection approaches?
3. What are the problems with the following tool selection approaches?
Case Study 1:
A major multinational company discovered that they were losing market share to a competing chemical
company. The V. P. of Marketing heard that the competing company had done some focus groups and
tailored the new products to the customers. Several customers had talked about how delighted they
were at being part of these focus groups. The Vice President of marketing had been asked by the CEO
to do some focus groups to get caught up with the competition. They had just lost 3% of market share
to the competition in the last year and the Board was on the CEO's back to reverse that trend. The vice
president of marketing calls you and asks you to do three focus groups on product X. You tell him that
three focus groups (30-40 people) is not enough data to generalize to the rest of the customer base. He
says he doesn't care. He heard the focus groups worked for the other company and he wants to get the
job done in a month before the Strategic Quality Planning meeting.
Case Study 2:
You were asked by the VP. of operations to start a written customer satisfaction survey. Your company
is a $20 million/year distribution company and has about 80 key health food stores for whom you
handle their entire line of products. The VP. of operations wants the written survey to be mailed four
times a year to make sure that you are on top of each of the key customers. He has already penciled out
about twenty questions (about two pages) so the survey won't irritate the health food storeowners by its
length. You are supposed to implement this survey and turn the results back to him in two months. He
wants a sense of how satisfied customers are.
Competitive Edge copyright, 2007
61
VOC Measurement Plan
(or lack thereof) Exercise
Case Study 3:
A regional restaurant with 22 restaurants proudly declares it is doing customer satisfaction measurement. They
advertise this point as part of their marketing strategy. When you investigate how it is done, you notice that
each restaurant has a comment card rack in an unobtrusive place in the restaurant. You also see an 800 number
on the rack where the comment cards are held. You ask how the responses are handled and they proudly
declare that each comment card receives a signed form letter from the President. With about 400 customers
per day in each restaurant and 22 restaurants, they receive about 15 cards a month. They want to take the data
from these comment cards and use it to monitor their restaurant managers.
Case Study 4:
A tool company has two lines of tools: one that appeal to "do-it-yourself" homeowners that are reasonably
priced and another line that appeals to professional contractors. The executives realize that they are losing
market share to a mid-level consumer that does extensive repairs and remodeling on his/her own home. Those
professional amateurs have typically gone for a competitive Japanese product that advertises sturdy but slightly
more expensive tools. To solve this problem, the tool company first organizes a scoping exercise. They talk to
two key hardware stores and others to find out what the customers are saying as they purchase various levels of
tools. Then the tool company decides to do a very targeted living survey with 50 heavy home tool users. Tool
company employees spend time with the customers in their garage, and watch how they use the tools. They
accompany these tool users on shopping sprees and ask questions as they go. With those observations, they
devise a questionnaire that is distributed to a larger sample of their potential heavy user customers. Next
comes the design of the new types of tools. A group of engineers, marketers, designers, and production people
are told to pour over the living survey data before they proceed. After four months, the new product is born.
Customers are then used to provide feedback on prototype tools. Changes are made. The tools are tested over
a wider base of beta sites. A few bugs are discovered. The tools are then showcased at the Annual Hardware
show and win the "Retailers Choice" awards. Competitive Edge copyright, 2007
62
Complaint Resolution:
TARP Research on Complaints
• 96% of unhappy customers never complain
• 68% of customers defect because of indifference
• 46% of dissatisfied complainers repurchase
• 70% of satisfied complainers will buy again
• 8-10 others are told about the dissatisfaction
Remember: good word-of-mouth is the best advertising
Complainers
Complaint
resolved
reported higher
70%
Didn't
37%
complain
Recovery and
• Elimination of key problems, prevent
Prevention = recurrences = Prevention
Retention
64
Complaint Management Process
Recovery
Prevention
65
Action to Traction Best Practice
Link to Financial Measures
66
Action to Traction: What’s New
About Customer Value
Choosing Value: Effective Competition for Customers
• Businesses need:
– A systematic and disciplined approach to create
and retain a differential advantage, that is deployed
throughout functional and operational areas
67
Exercise: The End of Satisfaction?
68
Possible Answer:
The End of Satisfaction?
69
Levels of Competitive Planning
Strategic competitive market planning
has different purposes at different
levels of the organization:
• Corporate level, planning for growth; Tools-
Customer Value Analysis
• Business Unit or Division level, identify strategic
opportunities for future investment, identified in
terms of the key product lines and markets served,
Tools-Product/Market Matrix=achieves strategic
focus
• Product/Market level, identify value-based,
marketing plans for a sustainable competitive
advantage. Can require changes to
products/services, operations, pricing, etc.
Reference: Strategic Six Sigma for Champions, R.Eric Reidenbach & Reginald Goeke, ASQ, Quality Press, 2006
70
Product/Market Planning
• Key question, “How does the organization compete
effectively?” Requires planning team to address four
questions:
ibid
71
Customer’s Point of View
72
Customer Value Measurement: Improve
Your Unique Value Proposition
73
Customer Value Model
•Customer value model is
product/market specific
•Based on customer ratings of
supplier performance across a
broad spectrum of performance
criteria
•Performance criteria are acquired
from customers through research,
and are grouped into quality drivers
on the basis of a statistical
analysis
•“Weights” attributed to the quality
and value drivers are also
empirically derived, and reflect the
relative importance of each driver
•Has a predictive component and
a managerial component
•CQI= Customer Quality Index;
Quality ratings are the summated
products of the weighted value
drivers
ibid
74
Example: Print Equipment/Box Makers
Most important quality driver. Comprised of 7
individual performance criteria
questionnaire items (CTQ-Critical to
Quality):
1. Prompt delivery of emergency repair parts
2. Routine maintenance parts readily available
3. Ability to talk to the right person with
complex technical questions
4. Prompt response to technical service
problems
5. Prompt response to requests for product
information
6. Warranty period length
7. History of reputable service
MANAGERIAL part of the model.
Tells which Quality (CQI) components are having the greatest impact
on customer perceptions of value, and what their relative impact is.
ibid
75
Example: Print Equipment/Box Makers
PREDICTIVE part of the value model.
Tells which value driver, Quality (CQI) Image, or Price, best predicts
value, and what the relative predictive power is. This part of the
customer value model describes the trade-offs among Quality, Image,
and Price, and also tells the extent to which value predicts loyalty
Ultimate set of
“worth it”
questions;
Usually 3.
R2 = .824
Tells the % of value that is predicted
or explained by the value
components within the model. More
than 82% of value is explained by
the model components
ibid
76
Quality Drivers: Head to Head Comparisons:
Delivering Value
•CQIs can be disaggregated into its constituent quality drivers for a head-to-
head performance comparison of you Vs competitors
•Comparisons are based on statistical significance, and interpreted in terms of
advantages, disadvantages, or parity positions
All comparisons are from XYZ enjoys a competitive advantage
the perspective of XYZ. over all competitors on the Product
Support
XYZ driver,Competitor
as well as on
1 the Competitor 2 Competitor 3
Product Quality driver.
Mean Mean Mean Mean
Ability to talk to the right person to help you with 8.11 7.44 6.99 8.71
complex technical problems
The manufacturers prompt response to your technical 8.22 7.44 7.26 7.38
service problems
Responding promptly to your requests for product 8.57 8.15 7.20 7.74
information
Length of warranty period 7.61 8.48 7.54 5.98
History of reputable service from the manufacturer 7.76 7.63 7.15 6.80
78
Lean Six Sigma Meets Value
The metrics of customer value bring an external perspective to
Lean Six Sigma initiatives that make them far more strategic,
capable of increasing revenues (top line results) while also
reducing waste.
79
It’s up to you!
• How will you shape
the future of quality
in your own work and in the
work you influence?
80
Learning Log
Change/Improvement Ideas:
Areas you would like to change to be more effective
81
Action Plan
Idea #____________________________________________
Desired Outcome: ________________________________________
Action Step Who is Possible Possible Comments
Responsible Start Complete (Issues, Barriers,
Date Date Resources)
82