National Council for Performance Excellence

2004 Education Baldrige Express Survey Process

In partnership with the Michigan Quality Council 248-370-4552 phone [email protected] 425 B Pawley Hall Rochester, MI 48309-4401

National Council for Performance Excellence

Introduction
The Baldrige Criteria help organizations identify strengths and key areas for improvement. The Baldrige approach has been used for approximately 16 years by tens of thousands of organizations to stay abreast of competition and to increase effectiveness. 55 countries throughout the world have adopted the Baldrige framework to enhance competitiveness.

National Council for Performance Excellence

5 Reasons to Choose Baldrige

The Baldrige Criteria:

Provide a framework for improvement without being prescriptive. Are inclusive. The Criteria describe an integrated management framework that addresses all the factors that define the organization, its operations, and its results. Focus on common requirements, rather than procedures, tools, or techniques. Are adaptable. They can be used by large and small businesses, and organizations with one site or worldwide locations. Are at the leading edge of validated management practices.

National Council for Performance Excellence

Baldrige Award Winners Outperformed the S&P 500

The U.S. Department of Commerce studies the performance of publicly held Baldrige Award winning companies. Every year since 1991 they outperformed the Standard and Poors index (S&P).

Baldrige winners outperformed the S&P 500 by 4.8 to 1.

1,101% increase in stock performance was demonstrated, compared to 228% from the S&P index companies for the same period.

National Council for Performance Excellence

It Makes Good Sense

Organizations that use a Performance Excellence process achieve results*:

59% Higher Stock Price 111% Higher Operating Income 116% Higher Sales 114% Higher Total Asset
*Data from Singhal and Hendricks Study

National Council for Performance Excellence

Baldrige Express eSurvey

The National Council for Performance Excellence administers an organizational assessment survey utilizing the Malcolm Baldrige Criteria for Performance Excellence The organizational assessment survey guides you through the 7 categories, addressing each item of the Criteria

The survey consists of 57 questions and takes approximately 2 hours to complete online
The assessment tool uses a maturity scale to determine the performance level

The Survey tool and Performance Indicator Feedback Report are effective and efficient aides to organizational assessment

National Council for Performance Excellence

Baldrige Criteria for Performance Excellence

Leadership Strategic Planning Student, Stakeholder, and Market Focus Measurement, Analysis, and Knowledge Management Faculty and Staff Focus

Process Management
Organizational Performance Results

National Council for Performance Excellence

Education Criteria for Performance Excellence Systems Framework

National Council for Performance Excellence

Eleven Core Values

Visionary Leadership

Learning-Centered Education
Organizational and Personal Learning Valuing Faculty, Staff, and Partners Agility Focus on the Future Managing for Innovation Management by Fact Social Responsibility Focus on Results and Creating Value Systems Perspective

National Council for Performance Excellence

Choosing Baldrige

Your organization is facing increasing challenges everyday. Assess your readiness to respond to those challenges by using the Baldrige Education Criteria In the Education sector, organizations with world-class results are able to achieve a score about 700 (70%) on the Baldrige scale. Organizations early in their performance improvement journey frequently score around 200 points. Where would your organization score?

National Council for Performance Excellence

Benefits of the eSurvey Approach

Online, web-based, easily accessed from any PC Accessible 24/7 Cost effective compared to alternative approaches Technical support provided Engage staff at all levels Easy to use maturity scale to determine performance level

Receive internal and external feedback

Obtain high performance diagnostics from the feedback 3-day turnaround of Performance Indicator Feedback Report upon completion of survey

Measure performance on a wide range of key organization performance indicators

Evaluate key processes, systems and results Identify organizational strengths, deployment gaps, and pinpoint vital opportunities for improvement

National Council for Performance Excellence

The Performance Indicator Feedback Report

When all survey data has been collected, an automated Performance Indicator Feedback Report will be generated and sent to you in three business days. The Performance Indicator Feedback Report consists of: Demographic Charts Overall Scores Category-Level Scores Item-Level Scores Gap Analysis Charts Individual Question Scores Pareto Charts for each Category Priority Improvement Data Charts Approach and Deployment Comments Priority Improvement Comments

National Council for Performance Excellence

The Presentation

The following slides guide you through the eSurvey process

Included with the following eSurvey snapshots are sample reports to show how the data collected is later represented in the Performance Indicator Feedback Report.

National Council for Performance Excellence

Type in your User ID which you received through email (usually your email address) and unique password. Click Login to the Survey.

National Council for Performance Excellence

Customized Demographic Profile

This is a sample of the table used to customize the Demographic Profile for your organization.
Position Administrator Support Staff Teacher/Instructor Location Elementary Secondary Central Office Function Finance Info Technology Math Science Language Arts Other Years of Service 0 yrs. < 1 yr. 1 yrs. < 3 yrs. 3 yrs. < 5 yrs. 5 yrs. < 10 yrs. 10+ yrs.

The Demographic Profile provides valuable comparative attributes within the organization. Survey participants choose appropriate selections from the Demographic Profile to indicate their position within the organization. Comparative charts and data are presented in the Performance Indicator Feedback Report according to the selections of the participants.

National Council for Performance Excellence

Baldrige Express eSurvey Demographic Selection (Sample Participants View)

Each different demographic attribute will appear on separate pages. Participants select the demographic value that describes their position within the organization.

Vermont Council for Quality

Organization Demographics
The Demographic Section is designed to provide the organization with an overall response rate relative to the total number of respondents and their percentage weight for each demographic attribute they have chosen.
Attribute/Value Position Administrator Support Staff Teacher/Instructor Total 7 10 8 25 28.0 40.0 32.0 100% Number Responded Response as % of Total

This chart reflects sample demographic data collected from the eSurvey and reported in the Performance Indicator Feedback Report.

National Council for Performance Excellence

eSurvey Status Page (Sample Participants View)

Once you have completed the Demographic Profile, click on the Leadership Category to begin the survey.

National Council for Performance Excellence

Category 1, Question 1A of the Survey

Click the radio button that best describes the level of development in your organization. Mouse over the levels to view the descriptors of each level.

National Council for Performance Excellence

Overall Score by Question and Current-State Descriptors

The Overall Score by Question and Current-State Descriptors show the overall percentage of points for a theme within each category. Refer to the Scoring Guidelines in Appendix B to see the percentage scoring. The Current text describes the maturity level of where the organization is now. The Role Model text describes where the organization could strive to be.
%
% 56

Row
1A Current Role Model

Description
Communicating Values and Performance Expectations [Baldrige ref: 1.1a(1)] Mature - Often done well by many. Effectiveness is sometimes checked. Nearly always done well. Effectiveness is regularly checked and improvements are constant. Leadership Commitment to Providing Maximum Value for Students and Other Stakeholders [Baldrige ref: 1.1a(1)] Mature - Often done well by many. Effectiveness is sometimes checked. Nearly always done well. Effectiveness is regularly checked and improvements are constant.g improvements. Empowering Faculty and Staff and Establishing a Climate for Empowerment, Innovation, Learning, Equity for All Students, Organizational Agility, and Ethical Behavior [Baldrige ref: 1.1a(2)] Basically Effective - Occasionally done well by some. Nearly always done well. Effectiveness is regularly checked and improvements are constant.
View Appendix B

% 56

1B Current Role Model

1C

38

Current Role Model

National Council for Performance Excellence

Priority Improvement Selections

(Sample Participants View)

After each category, the eSurvey asks participants to identify two areas they believe needs the most improvement. This data is reported as a Pareto chart and Priority Improvement Data Chart in the Feedback Report.

National Council for Performance Excellence

Pareto Charts

Staff members in your organization will be asked to select two areas in each of the seven categories, which they feel, is in most need of improvement. The concept of a Pareto chart is to show that the areas in need of improvement are usually a vital few, allowing leaders to concentrate their focus on specific improvement efforts. We will supply you with a Pareto chart for each of the seven categories, which reference each of the seven sections of the Organization Assessment Survey. Each chart will display the letters that correspond to the themes for each of the seven categories along the x-axis. Along the y-axis both Count and Percentages are charted. The count of staff votes (Hits) for each area they believe is most in need of improvement is represented as bars on the graph. The chart is arranged with the largest hit count on the left of the graph, indicating the area selected by staff which they feel is most in need of improvement. The line charted above the bars represents the cumulative percentages of each bar, from left to right.

National Council for Performance Excellence

Category 1 - Leadership Areas Most Needing Improvement - Pareto Chart

The letters F, C, and G represent an opportunity to solve 80% of key issues in 30% of the areas (3 of 10) for the category (Leadership). In this example, the letters H, I, and J, did not receive any votes since these areas were not selected by staff as areas most in need of improvement.
50 96 45 40 35 30 25 20 36 15 10 5 0 F C G B E A D H I J 30 20 10 0 58 80 88 98 100 100 100 Count Cumulative 100 100 90 80 70 60 50 40

Letter Key for Category 1 - Leadership - Areas Most Needing Improvement

A B C Communicating Values and Performance Expectations [Baldrige ref: 1.1a(1)] Leadership Commitment to Providing Maximum Value for Students and Other Stakeholders [Baldrige ref: 1.1a(1)] Empowering Faculty and Staff and Establishing a Climate for Empowerment, Innovation, Learning, Equity for All Students, Organizational Agility, and Ethical Behavior [Baldrige ref: 1.1a(2)] Ensuring Management Accountability To Protect Stakeholder Interests through Effective Governance [Baldrige ref: 1.1b] Assessing Organizational Performance and Progress Relative to Goals [Baldrige ref: 1.1c(1)] F G H Setting Priorities and Making Improvements Based on Progress Reviews [Baldrige ref: 1.1c(3)] Improving Leadership Effectiveness Throughout the Organization [Baldrige ref: 1.1c(4)] Following Laws and Regulations and Reducing Public Risks and Concerns [Baldrige ref: 1.2a(1&2)] Ensuring Ethical Behavior [Baldrige ref: 1.2b] Providing Community Support [Baldrige ref: 1.2c]

D E

I J

Hit Percentage

Hit Count

National Council for Performance Excellence

Priority Improvement Counts and Percentages

Category 1 - Leadership - Priority Improvement Counts and Percentages - Position
Count F Administrator Support Staff Teacher/Instructor Total
6 4 8 18

Percentage D
1 0 0 1

C
1 9 1 11

G
2 7 2 11

B
1 0 3 4

E
2 0 2 4

A
1 0 0 1

H
0 0 0 0

I
0 0 0 0

J
0 0 0 0

Total
14 20 16 50

F
43 20 50 36

C
7 45 6 22

G
14 35 13 22

B
7 0 19 8

E
14 0 13 8

A
7 0 0 2

D
7 0 0 2

H
0 0 0 0

I
0 0 0 0

J
0 0 0 0

This chart shows priority improvement selections according to the various Demographic attributes and values. Look at C and G and you will see that Support Staff identified the need to improve these areas by a greater margin than did Administrators and Teachers/Instructors. This tends to indicate a deployment gap and suggests that Administrators and Teachers/Instructors are not perceived as effective as they believe themselves to be.

National Council for Performance Excellence

Priority Improvement Comments

(Sample Participants View)

Participants offer approach/deployment comments and suggestions for improvement on their first selection.

Approach and Deployment comments are requested for each Category priority improvement selection. Participants offer suggestions for improvement on the process.

National Council for Performance Excellence

Priority Improvement Comments

(Sample Participants View)

Participants offer approach/deployment comments and suggestions for improvement on their second selection.

National Council for Performance Excellence

Leadership Row-Level Comments by Position

1A Communicating Values and Performance Expectations [Baldrige ref: 1.1a(1)]
Position/Administrator
Approach & Deployment: We do a great job communicating to all. Through e-mails, town hall meetings, Intranet. Scorecards for the whole organization are in place. Rules of Operation being implemented and communicated very efficiently. Approach & Deployment: We use many techniques to communicate including all hands meetings each quarter, weekly notes from the email, voice mail.

Position/Support Staff
Approach & Deployment: The scorecard is resented and the financial result and forecast are sometimes known. Action Steps to Improve: Get more input from staff for the scorecard. Make the results known. Approach & Deployment: Some goals are linked to the balanced scorecard but many have not been seen as major objectives. Many goals are very ad hoc and changeable, leaving staff unsure on direction. Many times effort is expended only to see that it has been wasted because of a change in direction or emphasis.

Position/Teacher/Instructor
Approach & Deployment: The issue is that most of the top administration has left the organization and is now not fully replaced. There are still some gaps in the organization, which create some lack of communication. The checking of the effectiveness to focus on students does not seem to be done often.

Comments are provided by the respondents and can be prepared and used by examiners and organization leaders for improvement planning. The comments are sorted by each category and theme statement. Comments are reported out by one attribute, in this case, Position. They are then sorted by each value such as Administrator, Support Staff, and Teacher/Instructor.

National Council for Performance Excellence

Completed Survey Status Page

When you have entered answers to every question for each category, you can submit your survey.

National Council for Performance Excellence

Summary Results
Overall Category-Level Totals
Baldrige Criteria Total Points 120 85 85 90 85 85 450 Totals 1000 Points Scored 65 34 34 25 34 30 205 427 % of Total Pts. 54 40 40 28 39 35 45 43%

1. Leadership 2. Strategic Planning 3. Student, Stakeholder, and Market Focus 4. Measurement, Analysis, & Knowledge Mgmt 5. Faculty and Staff Focus 6. Process Management 7. Organizational Performance Results

The Overall Category-Level Totals table explains the amount of points scored for each category, the total possible points to score in a category and the percentage of total points for each category.

National Council for Performance Excellence

Overall Category-Level Percent Scores

100 90

80

70
Percent of Possible Points

The Overall Category-Level Percent Scores show the aggregate percent score of each Category according to the Baldrige Scoring Guidelines. This helps determine the overall strengths and weaknesses of the organization.
54

60

50 45 40 40 40 39 35 30 28

20

10

0 Leadership Strategic Planning Student, Stakeholder, and Market Focus Measurement, Analysis, & Knowledge Mgmt Faculty and Staff Focus Process Management Organizational Performance Results

National Council for Performance Excellence

Overall Item-Level Percent Scores

100 90

80

The Overall Item-Level Percent Scores show the percent of points on a 0 to 100 scale, for each Item. This helps users determine, at a glance, the strengths and weaknesses by item for each category. The Items and Point Values are explained in Appendix A.
60 62

70

Percent of Possible Points

60 53 50 50 45 40 44 36 36 29 36 33 26 45 45 51 53

35 27 27

30

20

View Appendix A

10

0 Item 1.1 Item 1.2 Item 2.1 Item 2.2 Item 3.1 Item 3.2 Item 4.1 Item 4.2 Item 5.1 Item 5.2 Item 5.3 Item 6.1 Item 6.2 Item 7.1 Item 7.2 Item 7.3 Item 7.4 Item 7.5 Item 7.6

National Council for Performance Excellence

Overall Category-Level Point Scores

Position Point Scores by Category
1000 900 800 700 600 500 400 300 200 100 0 Leadership Strategic Planning Student, Stakeholder, and Market Focus Measurement, Analysis, and Knowledge Mgmt Faculty and Staff Focus Process Management Organizational Performance Results

The Category-Level Point Scores show the cumulation of total points for each demographic view within each category. This is a sample of one demographic view.

Total Possible Points

Administrator 84 55 41 37 43 37 237

Support Staff 53 19 30 17 29 26 190

Teacher/Instructor 61 31 32 28 34 28 181

National Council for Performance Excellence

Item-Level Percent Scores by Position

Position Percent Scores by Item
100 90

Notice the gap between Administrators and Support Staff and Teachers/Instructors in Item 1.1. This suggests an area to examine further. However, Item 7.4 shows a high degree of agreement indicating a consistent deployment of results information.

Administrators Support Staff Teachers/Instructors

80

70

Percent of Possible Points

60

50

40

30

20

View Appendix A

10

0
Item 1.1 Item 1.2 Item 2.1 Item 2.2 Item 3.1 Item 3.2 Item 4.1 Item 4.2 Item 5.1 Item 5.2 Item 5.3 Item 6.1 Item 6.2 Item 7.1 Item 7.2 Item 7.3 Item 7.4 Item 7.5 Item 7.6

This chart shows the ratings by demographic attribute, in this case, Position. On the previous graph, Item 1.1, Leadership System, reflected a rating of 50%. However, according to the breakout above, Administrators. believe the processes are much stronger (69 percent) than Support Staff (less than 38 percent) and Teachers/Instructors (45 percent). This typically indicates incomplete systems development or poor deployment of existing systems and processes required by the Item.

National Council for Performance Excellence

Other Services

Additional Performance Indicator Feedback Reports can be extracted from the demographic profile. For example: Sub-Reports can be generated for each organizational site or department identified in the demographic profile. The data that is entered into the survey is available in an ASCII Text File to create additional charts. Comparative data at the Category-level is available from previous assessments from the same organization. Additional copies of the Performance Indicator Feedback Report are available.

National Council for Performance Excellence

Thank you!
Michigan Quality Council
[email protected] 248-370-4552
In cooperation with National Council for Performance Excellence

National Council for Performance Excellence

Appendix A: Assessment Themes and Values

The point values of the scoring system are linked directly to the points and scoring system of the Malcolm Baldrige Criteria for Performance Excellence. Each item and the questions included in the items are also derived from the Baldrige Criteria.

2004 Categories, Items, and Point Values

1.0 Leadership 120 points
1.1 Organizational Leadership 80 points 1A Communicating Values and Performance Expectations [Baldrige ref: 1.1a(1)] 1B Leadership Commitment to Providing Maximum Value for Students and Other Stakeholders [Baldrige ref: 1.1a(1)] 1C Empowering Faculty and Staff and Establishing a Climate for Empowerment, Innovation, Learning, Equity for All Students, Organizational Agility, and Ethical Behavior [Baldrige ref: 1.1a(2)] 1D Ensuring Mgmt Accountability To Protect Stakeholder Interests through Effective Governance [Baldrige ref: 1.1b] 1E Assessing Organizational Performance and Progress Relative to Goals [Baldrige ref: 1.1c(1)] 1F Setting Priorities and Making Improvements Based on Progress Reviews [Baldrige ref: 1.1c(3)] 1G Improving Leadership Effectiveness Throughout the Organization [Baldrige ref: 1.1c(4)] 1.2 Social Responsibility 40 points 1H Following Laws and Regulations and Reducing Public Risks and Concerns [Baldrige ref: 1.2a(1&2)] 1I Ensuring Ethical Behavior [Baldrige ref: 1.2b] 1J Providing Community Support [Baldrige ref: 1.2c]

2.0 Strategic Planning

85 points

Back to Overall Item Scores

2.1 Strategy Development 40 points 2A Developing Strategic Plans [Baldrige ref: 2.1a(1&2)] 2B Developing Clear Strategic Objectives and Timetables for Meeting the Objectives [Baldrige ref: 2.1b(1)] 2C Ensuring that Strategic Objectives Address Key Educational Services Challenges and Balance the Needs of Key Students and Stakeholders [Baldrige ref: 2.1b(2)] 2.2 Strategy Deployment 45 points 2D Developing and Deploying Action Plans Based on Strategic Objectives [Baldrige ref: 2.2a(1)] 2E Developing Human Resource Plans Based on Strategic Objectives and Action Plans [Baldrige ref: 2.2a(3)] 2F Developing Appropriate Goals and Allocating Resources to Achieve Them [Baldrige ref: 2.2a(4) and 2.2b] 2G Aligning Work to Achieve Strategic Goals, Objectives, and Action Plans [Baldrige ref: 2.2a]

Back to Gap Analysis

National Council for Performance Excellence

Appendix B: Scoring Guidelines for Categories 1-6:

SCORE PROCESS No systematic approach is evident; information is anecdotal. (A) Little or no deployment of an approach is evident. (D) No evidence of an improvement orientation; improvement is achieved through reacting to problems. (L) No organizational alignment is evident; individual areas or work units operate independently. (I) The beginning of a systematic approach to the basic requirements of the Item is evident. (A) The approach is in the early stages of deployment in most areas or work units, inhibiting progress in achieving the basic requirements of the Item. (D) Early stages of a transition from reacting to problems to a general improvement orientation are evident. (L) The approach is aligned with other areas or work units largely through joint problem solving. (I) An effective, systematic approach, responsive to the basic requirements of the Item, is evident. (A) The approach is deployed, although some areas or work units are in early stages of deployment. (D) The beginning of a systematic approach to evaluation and improvement of key processes is evident. (L) The approach is in early stages of alignment with your basic organizational needs identified in response to the other Criteria Categories. (I) An effective, systematic approach, responsive to the overall requirements of the Item, is evident. (A) The approach is well deployed, although deployment may vary in some areas or work units. (D) A fact-based, systematic evaluation and improvement process and some organizational learning are in place for improving the efficiency and effectiveness of key processes. (L) The approach is aligned with your organizational needs identified in response to the other Criteria Categories. (I)

0% or 5%

10%, 15%, 20%, or 25%

30%, 35%, 40%, or 45%

50%, 55%, 60%, or 65%

The overall percentage of points for 1A on the previous slide is 56%, indicating the above Process guideline.

Back to Survey Priority Imp