Case Study

10 |
February 2020   LEAN & SIX SIGMA REVIEW
Pitch Perfect
Topic
Learning the ins and outs of Capability analysis

capability analysis by examining Author

a baseball pitcher’s performance Dennis Chakey

Email
[email protected]

O
n a summer’s evening in 2018 at PNC Park in Pittsburgh,
Pirates starting pitcher Jameson Taillon allowed only two
hits—throwing just 65 pitches—during the first six innings
of the game, leading his team as it led 2-0 over the Philadelphia
Phillies. That was before the 6’6” right-hander fell off the wagon in
the seventh—surrendering three runs in the blink of an eye. There
were 28,150 fans watching the battle between the Pirates and
Phillies that night.
Until the seventh, Taillon had been in conformance with the
umpire’s and league’s specification regarding what constitutes a
ball versus a strike, while also eluding the batter’s ability to convert
any pitch into a hit, or even worse, a home run.
Occasionally, the TV announcer would show a scatter diagram
of where previous pitches had crossed home plate. It became
immediately clear that Taillon’s accuracy and precision were fading
and could be evaluated in terms of capability and conformance to
the umpire’s specification.
Similar to how Taillon’s manager and pitching coach were obvi-
ously monitoring his decline in performance and failure to deliver
results, organizations must keep tabs on how well they deliver on
customer expectations and satisfy their customers with product and
service offerings. Organizations, just like the Pirates’ coaching staff,
must remain vigilant to recognize problems when they occur—and
make the appropriate changes when necessary. One way to do that is
through calculating process capability and using capability analysis.
Within an organization’s production and delivery processes, for
example, managers must measure the impact of the product’s
fitness and conformance to customer specifications, and respond
with the appropriate corrective actions that will remain sustainable
to the tests of time or succumb to financial disaster and the inevita-
ble dissolution of the business.

|
LEAN & SIX SIGMA REVIEW   asq.org/pub/sixsigma 11
Case Study > Pitch Perfect

But to understand the true nature of an orga- Because Figure 1 depicts a measles chart of the
nization’s performance and inherent significance pitches, simply counting the number contained
or impact on the customer requires the ability in the strike zone is an obvious initial look at the
to quantify performance and track its behavior pitcher’s performance. Of the last 25 pitches, 12
accurately and precisely over time. In essence, this Organizations, just like were called strikes.
is the act of ensuring that the products and services the Pirates’ coaching Figure 2 offers a much better view with a scatter
remain in conformance with customer specifications staff, must remain diagram created from the Minitab graphical tool.
and expectations.   vigilant to recognize With the addition of a few reference lines, you can
This is accomplished by calculating process problems when they see exactly how the starting pitcher was performing.
capability and using capability analysis tech- occur—and make the If you express the accuracy and precision of
niques and methods. Such methods can be appropriate changes the pitches thrown by the pitcher as the average
demonstrated by reflecting on one of America’s when necessary. distance away from the batter and target, the mean
greatest pastimes to gain an understanding and standard deviation (SD) could be used to eval-
of how capability analysis can best serve any uate the pitcher’s capability in meeting the strike
organization’s performance endeavors. zone specification along the x and y axes.
Now, back to the action. The strike zone is the area bounded by the rect-
angle in Figures 1 and 2. It’s also defined as the area
Behind the plate over home plate from the midpoint between a batter’s
Consider looking at the strike zone from the catcher shoulders and the top of the uniform pants—when
and umpire’s perspectives and the location of the the batter takes his or her stance while preparing to
pitcher’s last 25 pitches of the total 77 pitches he swing at a pitched ball—and a point just below the
threw through seven innings. kneecap. To get a strike call, part of the ball must cross
over part of home plate while in this area. To make this
illustration a bit simpler, focus on the x axis and ignore
the vertical height of the pitcher’s pitches.
Figure 1  CATCHER’S STRIKE ZONE PERSPECTIVE

Strike zone

LSL = USL =
12 in. 29 in.

Last 25 pitches

Assume batter is
standing 12 in. away
from strike zone Target = 20.5 in.

LSL = lower specification limit USL = upper specification limit

12 |
February 2020   LEAN & SIX SIGMA REVIEW
 Figure 2  SCATTERPLOT OF STARTING PITCHER PERFORMANCE Figure 3  STARTING PITCHER
DATA SAMPLE
12 29
46 Ball distance Vertical distance
45
from batter from ground
Vertical distance from ground (in.)

23.8 36.02
30
17.6 33.37

35 12.41 32.35

8.97 36.55
30
13.89 37.33

28.76 34.88
25
6.92 29.6
22
20 23.44 36.52
-20 -10 0 10 20 30 40 50
Horizontal ball distance from batter (in.) 7.01 34.52

22.58 35.96

-3.38 32.48
From the following collected data sample rep- around the mean but are, as noted earlier,
resented in Figure 3 and the use of the graphical widely scattered along the horizontal axis 8.82 32.38
summary option from the Minitab statistics tab, and not located on the home plate target
46.24 31.98
you can begin to evaluate: of 20.5 inches.
++ What do the descriptive statistics convey Given the data are distributed normally 28.06 34.3
about the pitcher’s performance in delivering about the mean, you would choose the nor-
36.62 33.29
strikes? Are the pitches consistently in the mal capability analysis option for evaluating
strike zone? the starting pitcher’s throwing capability. 4.03 35.23
++ Are they accurately on target? Is the pitcher
31.71 33.6
consistent or predictable? This is another way Careful selection
of asking whether the pitches from the pitching Why is this important? If the wrong 1.9 33.6
process are stable over the observed period. capability analysis option is selected—such
20.38 32.57
++ Are they precisely clustered around the target? as the nonnormal option—the capability
++ What type of data distribution do you see to results would be calculated erroneously, 2.52 35.42
guide the selection of the correct capability skewing the capability indexes and
14.5 34.38
analysis method? defect estimates.
From the graphical summary in Figure 4 (p. 14) To answer how predictable or consistent 17.82 35.56
you can see that the data sample of 25 pitches the pitcher is performing (process stability), -15.2 34.23
is normally distributed (p value > 0.05) around a look at the data in chronological sequence,
mean of ~15 and an SD of 13.67. Because the sample and an individual moving range (IMR) chart 17.16 35
reflects a random moment in time observation, you is a handy tool to use. Looking at the pro- -0.46 30.44
might expect from these 25 data points that there cess behavior chart for the starting pitcher
is a 95% probability the true mean will fall between in Figure 5 (p. 14), a baseline of the current
~9.4 and ~20.7, and a 5% risk that it will not. state performance is established.
The standard error of the mean and a z factor The starting pitcher will continue (based on the
of 1.96 are used in the calculation of the 95% confi- data collected) to throw pitches predictably that will
dence interval for the mean. fall anywhere from -32.8 to 62.9 inches from the bat-
Additionally, the range of the data sample also pro- ter 99.73% of the time, if nothing changes with his
vides another measurement of the variability of the delivery process. So, it appears there’s a stable and
data set and is simply the maximum number minus predictably off-target performance of ~5.5 inches.
the minimum number, or a range of 61.4 inches. Because the moving range (MR) section of the IMR
Seeing such inherent variation, you might chart is used to establish the three-sigma control lim-
conclude the data points are not clustered tightly its, this section is always investigated or evaluated first.

|
LEAN & SIX SIGMA REVIEW   asq.org/pub/sixsigma 13
Case Study > Pitch Perfect

Figure 4  STARTING PITCHER GRAPHICAL SUMMARY Should any data points exceed
the specified MR upper control
Summary report for ball distance from batter limit (UCL), the resulting individ-
uals chart also will be, by default,
considered out of control.
Anderson-Darling normality test
A-squared 0.11 This is because the resulting
P-value 0.991
individuals chart limits are partially
Mean 15.044
StDev 13.67 calculated by multiplying 2.66 and
Variance 186.871
Skewness 0.123453 the average moving range (AMR),
Kurtosis 0.278636
N 25 and would be considered null
Minimum -15.197 and void should the AMR prove
1st quartile 5.476
Median 14.497
3rd quartile 23.621
unstable. The rationale behind this
Maximum 46.24 guidance has everything to do
95% confidence interval for mean
9.401 20.687
with the stability and predictabil-
95% confidence interval for median ity of the process outcomes. If the
7.371 22.145
95% confidence interval for StDev process is not stable or predict-
10.674 19.017
able, the calculated limits would
be unreliable, and any inferences
drawn from an unstable process
95% confidence intervals would be questionable at best.
Mean
In the example from Figure 5, the
Median UCL is equal to the average Xbar of
8 12 16 20 24 15 + (2.66 x 18) = 62.9, and the lower
control limit is equal to the average
StDev = standard deviation Xbar of 15 - (2.66 x 18) = -32.8.
The origin of the 2.66 factor
derives from a statistical estimate
of the short-term common cause
Figure 5  STARTING PITCHER IMR CHART
variation, often regarded as the
process SD. Because the MR
Overview of starting pitcher—ball distance from batter
chart is capturing the differences
60 UCL = 62.9 between successive individuals
Ball distance in inches

chart data points, it calculates

40
the three SD widths by multiply-
Home plate target = 20.5
20 ing three times the average MR
X = 15
0 divided by 1.1278, the control chart
constant for a sample size of two.
-20
LCL = -32.8
This can be seen in Figure 6.
1 5 10 15 20 25
Simplifying the equation
Pitching sequence by reducing the coefficient
(3/1.1278) from that of
3 * AMR/1.1278 = 2.66 * AMR
yields the three SD estimate of
60 UCL = 58.81 47.88. Adding this value to the
Xbar or mean of 15 results in the
45
Moving range

earlier UCL of 62.9.

30 What you automatically recog-
nize is that the estimated process
MR = 18
15 SD (47.88/3) is ~15.96. Notice
how this estimate differs from
0 LCL = 0
the actual overall SD of 13.67 or
1 5 10 15 20 25
Pitching sequence ~14 derived from the data sample
and illustrated in the graphical
LCL = lower control limit UCL = upper control limit MR = moving range summary depicted in Figure 4.

14 |
February 2020   LEAN & SIX SIGMA REVIEW
Figure 6  CONTROL CHART EQUATIONS AND CONSTANTS

Control chart Standard deviation Formula

Moving range (MR) Short term

How much drifting can
the process central
tendency incur before
broaching any of
Chart type Upper control limit Lower control limit the limits imposed
on the process
Individual x + 2.66 MR x - 2.66 MR by the customer
MR 3.27 MR 0 specifications? This is
commonly referred to
as the distance to the
nearest specification.

Table of control chart constants

Xbar chart constants     
For sigma  R chart constants   S chart constants
estimate

Sample size = m A2 A3 d2 D3 D4 B3 B4

2 1.88 2.659 1.128 0 3.267 0 3.267

3 1.023 1.954 1.693 0 2.574 0 2.568

4 0.729 1.628 2.059 0 2.282 0 2.266

5 0.577 1.427 2.326 0 2.114 0 2.089

6 0.483 1.287 2.534 0 2.004 0.03 1.97

7 0.419 1.182 2.704 0.076 1.924 0.118 1.882

8 0.373 1.099 2.847 0.136 1.864 0.185 1.815

9 0.337 1.032 2.97 0.184 1.816 0.239 1.761

10 0.308 0.975 3.078 0.223 1.777 0.284 1.716

Having established the degree of stability or predictability regarding What you hope to determine by performing a capability analysis is:
the process outcomes, what are the parameters needed to determine 1. Will the process width fit within the customer width? This is just
and calculate the pitcher’s process capability indexes Cp and Cpk? another way of asking whether you’re capable of producing in
Of the four required parameters, two of them deal specifically with conformance to the customer specifications that constitute a
the customer specifications that establish the tolerance or limits on good product or service. You’ll determine whether the starting
process outcomes. The lower and upper specification limits have pitcher is capable of throwing strikes consistently.
previously been identified as the strike zone specifications or distance 2. Are the process outcomes centered within the limits specified,
of the pitched ball away from the batter in relation to the home plate. and is there adequate room for the process central tendency
This is what you would call the customer width. to drift without producing a defect or bad product or service?
Here again for clarity, the customer width is from the perspective of Simply put, as the baseballs cut the plane of home plate, are
the umpire, who adheres to the operating definition of the strike zone they centered over the target of 20.5 inches?
when calling balls and strikes. The remaining two parameters are the 3. How much drifting can the process central tendency incur
estimated SD and the average (Xbar) that define the process width. before broaching any of the limits imposed on the process by
Because the sample size is only 25, it may present a capability the customer specifications? Usually this is measured by the
analysis warning with respect to the precision level of the calculated number of SD increments between the central tendency of the
results. In most cases, the greater the sample size, the better the pitches thrown and the closest specification limit. This is com-
results will be for inferring future performance. monly referred to as the distance to the nearest specification.

|
LEAN & SIX SIGMA REVIEW   asq.org/pub/sixsigma 15
Case Study > Pitch Perfect

Figure 7  Cp CALCULATION Because the scatter diagram is scaled to the batter, you can see
the pitcher’s performance varies greatly, and well outside of the strike
zone in most cases. The dispersion pattern is quite long and strung
Process capability — Cp calculation out horizontally.
As the voice of the customer (VOC) width is compared to the voice
Allowed variation (spec.) of the process (VOP) width (the ratio VOC width / VOP width), a Cp
Cp =
Normal variation of the process of one would indicate that the process width fits exactly within the
customer width—but without any room to spare should the process
drift one way or another.
(USL − LSL)
or Cp = A Cp of 1.33, in most cases, offers an adequate amount of room to
6s
account for any drifting without the risk of exceeding either specifica-
tion limit.
Using s = 15.956 or ~16
From Online Figure 2, there’s a sizeable amount of variation, but
Cp = (29 − 12) / 6 * 16 = 17 / 96 = 0.177 or ~0.18
how close to the target value are the pitches? Is the pitcher consis-
LSL = lower specification limit tently accurate in hitting the target?
USL = upper specification limit You can calculate the pitcher’s Cpk using the formula in Figure 8.
You can see that the pitcher’s accuracy is not only off target by ~5.5
Figure 8  Cpk CALCULATION inches, but the capability ratio also is a great deal smaller than the 1.33
value. With the strike zone—illustrated in red (Online Figure 2)—the
scattering of data is nowhere close to fitting within the bounded rect-
angle depicting the strike zone (customer) specification. The greater
Process capability — Cpk calculation
the process variation is in comparison to the customer specification,
the smaller the Cp value.
USL − x x − LSL
Cpk = Min. or A verification of these calculations—based on the collected sample of
3s 3s
data and the use of the Minitab Assistant—is shown in Figures 9 and 10.
The Figure 9 report card offers an overall assessment of the pro-cess
S = ~16 as noted earlier stability, data subgrouping, normality of the sample data distribution and
= Min. ((29 − 15 / 3 * 16) or ((15 − 12) / 3 * 16) whether the total sample is sufficient for the intended purpose.
= Min. (14 / 48 or 3 / 48) Note that a precautionary warning is raised regarding the precision
= Min. (0.29 or 0.06) of the capability estimates. Other than that, all other items appear to
Cpk = 0.06 have passed the initial review checks and balances.
Figure 10 not only has the data that have been raised to a visible
level in the form of a histogram, but the customer specifications also
A scatter diagram of the last 25 pitches is illustrated in Online are shown in relation to the distribution of data. By simply review-ing
Figure  1, which can be found on this article’s webpage at asq.org/ the diagram, it’s obvious that the process outcomes (pitches)
pub/sixsigma, with the average distance from the batter of ~15 inches are completely beyond the desired boundaries established by the
(5.5 inches off target) and having a SD of 14 inches. specifications, and the central tendency of the distributed data is off
target. The distribution mean is to the left of the target value and not
Calculating the starting pitcher’s capability centered within the specification boundaries.
At this point, a word of caution is needed regarding which SD value Within the process characterization section, notice that the SDs for
(actual from sample or estimated) is used to accurately calculate the overall and estimated are stated, along with the number of data obser-
capability Cp index. The correct approach is to use the estimated SD vations, subgroup size and mean values.
(s ~ 16 derived from the IMR chart 3 sigma limit calculations) value Looking at the capability section of Figure 10, the reported statistics
because this would comport with how Minitab and other statistical are categorized by “Actual (overall) and potential” groups.
software packages compute this statistic.
In general, the software automatically performs these typical cal-
culations. If calculated by hand for training purposes, however, using
the estimated value would afford the comfort of a verified result when
compared to the software output. EDITOR’S NOTE
The second part of this article, scheduled to appear in the May
So, getting back to the matter at hand: Is the pitcher capable?
2020 edition of Lean & Six Sigma Review, will explore actual
Based on the calculation driven by Figure 7, you would be inclined
(overall) and potential categories to reveal what they really
to say “No” because the ratio of the strike zone specification to the mean. The article will continue using the baseball example
process width of 6s is a great deal smaller than the rule of thumb and shift matters to the coach’s decision on whether to pull
value of 1.33 (generally accepted threshold). the starting pitcher and replace him with a relief pitcher.

16 |
February 2020   LEAN & SIX SIGMA REVIEW
Figure 9  MINITAB ASSISTANT CAPABILITY REPORT CARD

CAPABILITY ANALYSIS FOR STARTING PITCHER REPORT CARD

Check Status Description

Stability  The process mean and variation are stable. No points are out of control.

You have 25 subgroups. For capability analysis, this is usually enough to

Number of
subgroups
i capture the different sources of process variation when collected over a 
long enough period of time.

Normality  Your data passed the normality test. As long as you have enough data,
the capability estimates should be reasonably accurate.

The total number of observations is fewer than 100. You may not have
Amount enough data to obtain reasonably precise capability estimates. The
of data ! precision of the estimates decreases as the number of observations
becomes smaller.

Figure 10  MINITAB ASSISTANT CAPABILITY RESULTS

DENNIS CHAKEY
Capability analysis for starting pitcher
is president and
Process performance report
CEO of Partners
Capability histogram in Technology
Are the data inside the limits and close to the target? Consulting Group
in Murrysville,
LSL Target USL
Process characterization
PA. He is also an
Total N 25
Subgroup size 1
Accenture Federal Services consul-
Mean 15.044
Standard deviation (overall) 13.67 tant and an adjunct professor at the
Standard deviation (within) 15.956
New England College of Business
Capability statistics
Actual (overall) in Boston. He holds an MBA from
Pp 0.21
Ppk 0.07 the University of Pittsburgh Katz
Z-Bench -0.17
% out of spec (observed) 52 Graduate School of Business. Chakey
% out of spec (expected) 56.55
PPM (DPMO) (observed) 520,000 is also a former board examiner
PPM (DPMO) (expected) 565,541
Potential (within) for the Malcolm Baldrige National
Cp 0.18
Cpk 0.06 Quality Award Program and the
Z-Bench -0.29
% out of spec (expected) 61.52 Pennsylvania Keystone Alliance for
PPM (DPMO) (expected) 615,241
Performance Excellence. He is a
senior member of ASQ and an ASQ-
-20 -10 0 10 20 30 40 50
certified Six Sigma Black Belt.

Actual (overall) capability is what the

customer experiences. BIBLIOGRAPHY
Brook, Quentin, Lean Six Sigma and Minitab: The Complete Toolbox Guide
Potential (within) capability is what could
for Business Improvement, fifth edition, OPEX Resources Ltd., 2017.
be achieved if process shifts and drifts Glantz, Stanton A., Primer of Biostatistics, seventh edition, McGraw-Hill Cos.
were eliminated. Inc., 2012.
Minitab, Minitab 18 Statistical Software, Minitab LLC, 2017.
Sahay, Amar, Process Capability Analysis for Quality and Lean Six Sigma,
PPM = parts per million QMS Global LLC, 2017.
Wheeler, Donald J., Making Sense of Data, Statistical Process Controls
DPMO = defects per million opportunities
Press, 2003.
LSL = lower specification limit Wheeler, Donald J., Understanding Variation—The Key to Managing Chaos,
USL = upper specification limit Statistical Process Controls Press, 2000.

|
LEAN & SIX SIGMA REVIEW   asq.org/pub/sixsigma 17
Online Figure 1  RELATIONSHIP OF STARTER’S PITCHES TO STRIKE ZONE

Strike zone
Scatterplot of vertical distance from ground vs. distance from batter
38
from ground (in.)
Vertical distance

35

32

29
- 20 -10 0 10 20 30 40 50
Horizontal pitches from batter (in.) (starting pitcher)

Width = 17 in.
LSL = USL =
12 in. 29 in.

Target = 20.5 in.

LSL = lower specification limit

USL = upper specification limit
X-bar = 15 in.
Actual SD = 14 in.
Online Figure 2  STARTING PITCHER’S VARIABILITY

Strike zone
Scatterplot of vertical distance from ground vs. distance from batter
38
from ground (in.)
Vertical distance

35

32 X-bar = +15.0 in.

Actual SD = 14.0 in.
29
- 20 -10 0 10 20 30 40 50
Horizontal pitches from batter (in.) (starting pitcher)

Width = 17 in.
LSL = USL =
12 in. 29 in.

Target = 20.5 in.

LSL = lower specification limit

USL = upper specification limit
SD = Standard Deviation