Process Capability Analysis

RAUL SOTO, MSC, CQE

V
RS
The contents of this presentation represent the opinion
of the speaker; and not necessarily that of his present
or past employers.

[c] 2020 Raul Soto 2

KENX JULY 2020 1 of 32

 About the Author
• Senior Principal Software Engineer with Johnson & Johnson Vision Care

• 27+ years of experience in the medical devices, pharmaceutical, biotechnology, and consumer electronics industries
– MS Biotechnology and Bioinformatics, emphasis in Biomedical Engineering
– BS Mechanical Engineering
– ASQ Certified Quality Engineer (CQE)

• I have led validation / qualification efforts in multiple scenarios:

– High-speed, high-volume automated manufacturing and packaging equipment;
machine vision systems; SCADA / HMI systems
– Laboratory information systems and instruments
– Enterprise resource planning applications (i.e. SAP)
– Manufacturing Execution Systems (MES)
– IT network infrastructure
– Business intelligence systems (i.e. SAS, Cognos & Business Objects reports)
– Mobile apps
– Product improvements, material changes, vendor changes

• Contact information: Raul Soto [email protected]

[c] 2020 Raul Soto 3

V What this talk is about

RS
• Present and discuss the principles and application of
Process Capability Analysis, and related subjects.

• Understand basic concepts, underlying assumptions,

and limitations.

• Understand why we can’t just plug in numbers into Minitab/

JMP, etc. without knowing the fundamental assumptions
(“the fine print”).

• Can’t teach a semester of Statistics in 45 minutes …

[c] 2020 Raul Soto 4

KENX JULY 2020 2 of 32

 Table of Contents
• What is Process Capability • Tolerance Intervals for process
• Process capability vs Process control
control • Assumption of Normality
• Indices: Cpk/Ppk, Cp/Pp, • Normality Tests
Cpm/Ppm • How to handle non-normal data
• What Cpk really means • Process Capability for non-
• Long-term vs short-term normal processes
capability • Tolerance Intervals for non-
• The 3-validation-lots rule of normal processes
thumb: 2 examples • Extra: Test your validation lots
• Estimate lot and process % for homogeneity
defective from Cpk / Ppk
• Confidence intervals for
Cpk/Ppk
• Sample sizes for Confidence
www.phdcomics.com/comics/archive.php?comicid=1553 Intervals
[c] 2020 Raul Soto 5

V
RS
Minitab 30-day free
trial

• http://www.minitab.com/e
n-us/downloads/

• Download and install

[c] 2020 Raul Soto 6

KENX JULY 2020 3 of 32

 Process Capability
• Capable Process: We can make product Process Capability Report for Line 1
(using 95.0% confidence)
that meets specifications
LSL Target USL
Process Data Overall
LSL 125 Within
Target 130

• Process Capability: Quantifies USL 135

Sample Mean 130.5 Pp
Overall Capability

CI for Pp
2.47
(2.12, 2.81)
Sample N 100
numerically how capable a process is of StDev(Overall) 0.675586
StDev(Within) 0.704465
PPL
PPU
2.71
2.22
Ppk 2.22

meeting its specifications CI for Ppk (1.91, 2.53)

Cpm 0.79
LB for Cpm 0.70
Potential (Within) Capability
Cp 2.37
CI for Cp (2.00, 2.73)

• Based on an assumption of normality of CPL

CPU
2.60
2.13
Cpk 2.13
the data CI for Cpk (1.80, 2.46)

– If your data set does not follow a normal 126.0 127.4 128.8 130.2 131.6 133.0 134.4

distribution, CPK / PPK is meaningless Observed

Performance
Expected Overall Expected Within
PPM < LSL 0.00 0.00 0.00
PPM > USL 0.00 0.00 0.00
PPM Total 0.00 0.00 0.00

(c) 2020 Raul Soto 7

V
Process Capability vs Control
RS
• Controlled Process:
– mean and standard
deviation are consistent

• Capable Process:
– consistently meets
specifications

(c) 2020 Raul Soto 8

KENX JULY 2020 4 of 32

 Process Capability Indices :
( − )
=
3
 Cpk = lowest of CPU or CPL ( − )
=
3

 Cp : ratio of the design spread to the process spread ( − )

=
6

 Cpm : overall capability. −

=
Penalizes when the process is off-center 6 ∗ s +( − )

(c) 2020 Raul Soto 9

V Example
RS
 Take a sample from a validation lot ( ) ( )
= = = 1.41
( . )
 n = 60 units

( ) ( . )
 Spec limits are: = = = 1.68
( . )
 USL = 35, target = 30, LSL = 25
( ) ( . )
= = = 1.14
( . )
 Calculate:
= 1.14
 ( ) Sample mean = 29.050
 (s) Sample std.dev = 1.184 −
=
6 ∗ s +( − )

= = 1.10
∗ . ( . )

(c) 2020 Raul Soto 10

KENX JULY 2020 5 of 32

 CPK calculation in Minitab

[c] 2020 Raul Soto 11

V CPK vs Sigma Levels

RS
• sigma levels : how many standard
deviations can we fit between the process
mean and the closest specification limit

• Cpk = means how many times

• we can fit
3 standard deviations
between
the sample mean and
the closest spec limit
http://www.six-sigma-material.com/Tables.html

(c) 2020 Raul Soto 12

KENX JULY 2020 6 of 32

 Cpk (short-term) vs Ppk (long-term) capability
• The basic difference between Ppk and Cpk is the
standard deviation used for the calculation. The formulas
used are basically the same.
• For Cpk we use the short-term variability (i.e. individual
lots in a campaign, individual raw material lots within a
CPK
production lot).
PPK
• For Ppk we use the overall, long-term variability (i.e. all
lots in a campaign, or all raw material lots in a
production lot).
• Cpk will typically be higher than Ppk.
• As we reduce short-term variability, the process becomes
stable, and Ppk approaches Cpk.
(c) 2020 Raul Soto 13

V
Short-Term vs Long-Term Capability
RS
• Short – term variation: the variation within a subgroup (i.e. one validation lot, one shift,
one operator, or one raw material batch.)

• Overall, long-term variation: the variation of all measurements, which is an estimate of

the overall process variation (i.e. all your validation lots, multiple shifts, multiple raw
material lots.

• Long-term capability of a process allows us to view the bigger picture.

• LT capability is typically lower than the process capability of individual lots.

• Some authors assume a short term / long term process shift of 1.5 sigma as a rule of
thumb. This means that the long term variation will be approximately 1.5 times the short
term variation.
(c) 2020 Raul Soto 14

KENX JULY 2020 7 of 32

 The 3-consecutive-lots Rule of Thumb?
Short-term vs Long-Term capability
• When you base your validation only on the Cpk of individual lots,
you are not taking into account long-term variation

• This method does not provide a realistic view of how the process
will perform in the long term.

• Long term process capability should also be calculated, by using the

data from all the lots.
(c) 2020 Raul Soto 15

V
RS
Process Capability Example 1:
Calculate lot-to-lot (short term) and process (long term) capability
using 4 validation lots

Descriptive Statistics: Lot A, Lot B, Lot C, Lot D

Variable N Mean SE Mean StDev Minimum Q1 Median Q3 Maximum

Lot A 60 29.748 0.129 0.999 27.271 28.965 29.737 30.331 31.944
Lot B 60 30.071 0.131 1.015 27.520 29.351 30.135 30.709 32.050
Lot C 60 30.192 0.132 1.022 27.560 29.548 30.203 30.763 33.302
Lot D 60 29.671 0.143 1.106 27.045 28.835 29.694 30.541 31.859

Minitab : Stat / Quality Tools / Capability Analysis / Normal

(c) 2020 Raul Soto 16

KENX JULY 2020 8 of 32

 35.0

Consistent process

32.5
Data

30.0

27.5

25.0
Lot A Lot B Lot C Lot D

(c) 2020 Raul Soto 17

V Process Capability - Lot A

RS
(using 95.0% confidence)

LSL Target USL

Process Data Overall
LSL 25 Within
Target 30
USL 35 Overall Capability
Sample Mean 30 Pp 1.67
Sample N 60 CI for Pp (1.37, 1.97)
StDev(Overall) 0.999348 PPL 1.67
StDev(Within) 1.00359 PPU 1.67
Ppk 1.67
CI for Ppk (1.37, 1.97)
Cpm 1.62
LB for Cpm 1.37
Potential (Within) Capability
Cp 1.66
CI for Cp (1.36, 1.96)
CPL 1.66
CPU 1.66
Cpk 1.66
CI for Cpk (1.36, 1.96)

25.5 27.0 28.5 30.0 31.5 33.0 34.5

Performance
Observed Expected Overall Expected Within
PPM < LSL 0.00 0.28 0.31
PPM > USL 0.00 0.28 0.31
PPM Total 0.00 0.56 0.63

(c) 2020 Raul Soto 18

KENX JULY 2020 9 of 32

 Process Capability - Lot B
(using 95.0% confidence)

LSL Target USL

Process Data Overall
LSL 25 Within
Target 30
USL 35 Overall Capability
Sample Mean 30 Pp 1.64
Sample N 60 CI for Pp (1.35, 1.94)
StDev(Overall) 1.01513 PPL 1.64
StDev(Within) 1.01944 PPU 1.64
Ppk 1.64
CI for Ppk (1.35, 1.94)
Cpm 1.64
LB for Cpm 1.39
Potential (Within) Capability
Cp 1.63
CI for Cp (1.34, 1.93)
CPL 1.63
CPU 1.63
Cpk 1.63
CI for Cpk (1.34, 1.93)

25.5 27.0 28.5 30.0 31.5 33.0 34.5

Performance
Observed Expected Overall Expected Within
PPM < LSL 0.00 0.42 0.47
PPM > USL 0.00 0.42 0.47
PPM Total 0.00 0.84 0.94

(c) 2020 Raul Soto 19

V Process Capability - Lot C

RS
(using 95.0% confidence)

LSL Target USL

Process Data Overall
LSL 25 Within
Target 30
USL 35 Overall Capability
Sample Mean 30 Pp 1.63
Sample N 60 CI for Pp (1.34, 1.92)
StDev(Overall) 1.02202 PPL 1.63
StDev(Within) 1.02636 PPU 1.63
Ppk 1.63
CI for Ppk (1.34, 1.92)
Cpm 1.60
LB for Cpm 1.36
Potential (Within) Capability
Cp 1.62
CI for Cp (1.33, 1.92)
CPL 1.62
CPU 1.62
Cpk 1.62
CI for Cpk (1.33, 1.92)

25.5 27.0 28.5 30.0 31.5 33.0 34.5

Performance
Observed Expected Overall Expected Within
PPM < LSL 0.00 0.50 0.55
PPM > USL 0.00 0.50 0.55
PPM Total 0.00 1.00 1.11

(c) 2020 Raul Soto 20

KENX JULY 2020 10 of 32

 Process Capability - Lot D
(using 95.0% confidence)

LSL Target USL

Process Data Overall
LSL 25 Within
Target 30
USL 35 Overall Capability
Sample Mean 30 Pp 1.51
Sample N 60 CI for Pp (1.24, 1.78)
StDev(Overall) 1.10599 PPL 1.51
StDev(Within) 1.11068 PPU 1.51
Ppk 1.51
CI for Ppk (1.24, 1.78)
Cpm 1.44
LB for Cpm 1.22
Potential (Within) Capability
Cp 1.50
CI for Cp (1.23, 1.77)
CPL 1.50
CPU 1.50
Cpk 1.50
CI for Cpk (1.23, 1.77)

25.2 27.0 28.8 30.6 32.4 34.2

Performance
Observed Expected Overall Expected Within
PPM < LSL 0.00 3.08 3.37
PPM > USL 0.00 3.08 3.37
PPM Total 0.00 6.16 6.74

(c) 2020 Raul Soto 21

V
 SHORT TERM (lot to lot)
RS
results:
Lot A
Cpk = 1.66
expected defects per million = 0.56  If our validation criteria is based solely on
the Cpk’s of individual lots being equal
Lot B or greater than 1.0, we would pass.
Cpk = 1.63
expected defects per million = 0.84
 Let’s look at Long-Term variation:
Lot C
Cpk = 1.62
expected defects per million = 1.0

Lot D
Cpk = 1.50
expected defects per million = 6.16

(c) 2020 Raul Soto 22

KENX JULY 2020 11 of 32

 Process Capability - Long Term (All lots)
(using 95.0% confidence)

LSL Target USL

LONG TERM
Process Data Overall
LSL
Target
25
30
Within

Overall Capability
RESULTS
USL 35
Sample Mean 30 Pp 1.58
Sample N 240 CI for Pp (1.44, 1.73)
StDev(Overall) 1.05268
StDev(Within) 1.03755
PPL
PPU
1.58
1.58
Ppk = 1.58
Ppk 1.58
CI for Ppk (1.44, 1.73)
Cpm 1.58
LB for Cpm 1.46
Potential (Within) Capability
Cp 1.61
Expected defects
CI for Cp (1.46, 1.75)
CPL 1.61 per million units = 2.04
CPU 1.61
Cpk 1.61
CI for Cpk (1.46, 1.75)

25.5 27.0 28.5 30.0 31.5 33.0 34.5

Performance
Observed Expected Overall Expected Within
PPM < LSL 0.00 1.02 0.72
PPM > USL 0.00 1.02 0.72
PPM Total 0.00 2.04 1.44 This is a 4.5 sigma process
(c) 2020 Raul Soto 23

V
RS
Process Capability Example 2:
Calculate lot-to-lot (short term) and process (long term) capability using 3 validation lots

Descriptive Statistics: Lot 1, Lot 2, Lot 3

Variable N Mean SE Mean StDev Min Q1 Median Q3 Max

Lot 1 60 15.500 0.0754 0.584 14.138 15.020 15.513 15.807 16.992
Lot 2 60 11.914 0.0605 0.469 10.562 11.633 11.902 12.218 13.114
Lot 3 60 17.869 0.0810 0.627 16.299 17.401 17.959 18.232 19.513

Minitab 17: Stat / Quality Tools / Capability Analysis / Normal

(c) 2020 Raul Soto 24

KENX JULY 2020 12 of 32

 Boxplot of Lot 1, Lot 2, Lot 3
20

18

16
*Inconsistent process*
Data

14

12

10
Lot 1 Lot 2 Lot 3

USL = 20, LSL = 10

(c) 2020 Raul Soto 25

V Process Capability for Lot 1

RS
(using 95.0% confidence)

LSL Target USL

Process Data Overall
LSL 10 Within
Target 15
USL 20 Overall Capability
Sample Mean 15 Pp 2.85
Sample N 60 CI for Pp (2.34, 3.37)
StDev(Overall) 0.584095 PPL 2.85
StDev(Within) 0.565121 PPU 2.85
Ppk 2.85
CI for Ppk (2.34, 3.37)
Cpm 2.16
LB for Cpm 1.83
Potential (Within) Capability
Cp 2.95
CI for Cp (2.42, 3.48)
CPL 2.95
CPU 2.95
Cpk 2.95
CI for Cpk (2.42, 3.48)

10.5 12.0 13.5 15.0 16.5 18.0 19.5

Performance
Observed Expected Overall Expected Within
PPM < LSL 0.00 0.00 0.00
PPM > USL 0.00 0.00 0.00
PPM Total 0.00 0.00 0.00
(c) 2020 Raul Soto 26
(c) 2020 Raul Soto

KENX JULY 2020 13 of 32

 Process Capability for Lot 2
(using 95.0% confidence)

LSL Target USL

Process Data Overall
LSL 10 Within
Target 15
USL 20 Overall Capability
Sample Mean 15 Pp 3.56
Sample N 60 CI for Pp (2.92, 4.20)
StDev(Overall) 0.468703 PPL 3.56
StDev(Within) 0.499807 PPU 3.56
Ppk 3.56
CI for Ppk (2.92, 4.20)
Cpm 0.53
LB for Cpm 0.45
Potential (Within) Capability
Cp 3.33
CI for Cp (2.73, 3.93)
CPL 3.33
CPU 3.33
Cpk 3.33
CI for Cpk (2.73, 3.93)

10.5 12.0 13.5 15.0 16.5 18.0 19.5

Performance
Observed Expected Overall Expected Within
PPM < LSL 0.00 0.00 0.00
PPM > USL 0.00 0.00 0.00
PPM Total 0.00 0.00 0.00
(c) 2020 Raul Soto 27
(c) 2020 Raul Soto

V Process Capability for Lot 3

RS
(using 95.0% confidence)

LSL Target USL

Process Data Overall
LSL 10 Within
Target 15
USL 20 Overall Capability
Sample Mean 15 Pp 2.66
Sample N 60 CI for Pp (2.18, 3.13)
StDev(Overall) 0.627216 PPL 2.66
StDev(Within) 0.594381 PPU 2.66
Ppk 2.66
CI for Ppk (2.18, 3.13)
Cpm 0.56
LB for Cpm 0.48
Potential (Within) Capability
Cp 2.80
CI for Cp (2.30, 3.31)
CPL 2.80
CPU 2.80
Cpk 2.80
CI for Cpk (2.30, 3.31)

11.2 12.8 14.4 16.0 17.6 19.2

Performance
Observed Expected Overall Expected Within
PPM < LSL 0.00 0.00 0.00
PPM > USL 0.00 0.00 0.00
PPM Total 0.00 0.00 0.00
(c) 2020 Raul Soto 28
(c) 2020 Raul Soto

KENX JULY 2020 14 of 32

 Individually, based solely in short term process variation, each lot looks good.
Lot 1
Cpk = 2.95
expected defects per million =0

Lot 2
Cpk = 3.33
expected defects per million =0

Lot 3
Cpk = 2.80
expected defects per million =0

 If our validation criteria is based solely on the Cpk’s of individual lots being equal or greater
than 1.0, we would pass.

 But when we look at the long-term variation of the overall process, the picture changes:
(c) 2020 Raul Soto 29

V Process Capability - Long Term

(using 95.0% confidence)
RS
LSL Target USL
Process Data Overall
LSL 10 Within
Target 15
USL 20 Overall Capability
Sample Mean 15 Pp 0.66
Sample N 180 CI for Pp (0.59, 0.73)
StDev(Overall) 2.51829 PPL 0.66
StDev(Within) 0.564786 PPU 0.66
Ppk 0.66
CI for Ppk (0.59, 0.73)
Cpm 0.66
LB for Cpm 0.60
Potential (Within) Capability
Cp 2.95
CI for Cp (2.64, 3.26)
CPL 2.95
CPU 2.95
Cpk 2.95
CI for Cpk (2.64, 3.26)

10 12 14 16 18 20

Performance
Observed Expected Overall Expected Within
PPM < LSL 0.00 23545.98 0.00
PPM > USL 0.00 23545.98 0.00
PPM Total 0.00 47091.95 (c) 0.00
2020 Raul Soto 30
(c) 2020 Raul Soto 30

KENX JULY 2020 15 of 32

 The overall (Long term) capability of the process is significantly lower than the
individual capability of the individual lots.

This is so because the process mean is shifting.

Overall:

Ppk = 0.66
expected defects per million = 47,092 (4.7%) this is a 2 sigma process

Look at your long term process capability

This will allow you to obtain a better picture of how the process will perform in the long
term.
(c) 2020 Raul Soto 31

V
Short-Term vs Long-Term Capability
RS
or “why is my process performing poorly, if my three validation batches were so good?”

• The distribution or the mean of a process may be stable for a single set of raw materials,
operator, machine, etc.
• Long term variation may increase because of process shifts and drifts.
• Possible causes of process shifts : different operators, raw material batches, machines,
changes in the environment.
• Possible causes of process drifts : equipment aging, parts wear off, learning curve, fatigue,
environmental factors.
• Shifting / drifting may be present in your manufacturing and measurement/ inspection
processes.
(c) 2020 Raul Soto 32

KENX JULY 2020 16 of 32

 Process Mean Shifting Over Time Process Mean Shifting Over Time
Normal
15.0
18 Variable
L ot 1
16 L ot 2
L ot 3
14 L ot 4 12.5
L ot 5
12 L ot 6
L ot 7
F re q u e n c y

10 Overall

D ata
10.0
Mean StDev N
8
7.392 0.9618 30
10.29 0.9634 30
6 13.30 0.8041 30
10.07 1.226 30
4 7.5
6.635 0.8716 30
12.71 0.8753 30
2 9.237 0.9759 30
9.947 2.492 210
0
4.5 6.0 7. 5 9.0 10.5 12.0 13.5 15.0 5.0

Data Lot 1 Lot 2 Lot 3 Lot 4 Lot 5 Lot 6 Lot 7

(c) 2020 Raul Soto 33

V
Process Mean Drifting Over Time Process Mean Drifting Over Time
RS
Normal 30

20 Variable
Lot A
Lot B
Lot C 25
Lot D
15 Overall_

Mean StDev N
F re q u e n c y

10.09 0.6285 30
20
14.83 0.8489 30
D a ta

10 20.36 1.138 30
24.82 1.142 30
17.53 5.666 120
15
5

10
0
4 8 12 16 20 24 28
Data Lot A Lot B Lot C Lot D

(c) 2020 Raul Soto 34

KENX JULY 2020 17 of 32

 Process Shifts, Drifts
• Looking only at short term distributions may give the
impression that the process is in control and capable
• But if the process mean or the process variability are
drifting/shifting over time, the long term distribution will
show a different picture
• Usually when this happens, the validation of the process is
questioned.
• Validation should not be seen as a one-shot deal, but as
part of a continuous improvement process.

(c) 2020 Raul Soto 35

V
Estimate % Defective and dpm from CPK
RS
EXAMPLE Using the Normal Distribution Table

Spec limits are: P (Z < 1.6) = 0.9452;

USL = 90, LSL = 50 above USL = 1 – 0.9452 = 0.055
Calculate: Below USL = P (Z < -1.75) = 0.0401
Sample mean = 70.9
Sample std.dev = 11.92 Total % Out of Spec = 0.055 + 0.0401 = 0.095 = 9.5%

 P(LSL < X < USL) : within spec

Defects per million (dpm) = OOS * 106
= < < = 0.095 * 106 = 94,289
. .
= < <
. .

= P (-1.75 < Z < 1.6)

= P (Z < 1.6) – P (Z < -1.75)

(c) 2020 Raul Soto 36

KENX JULY 2020 18 of 32

 Confidence Interval and Sample size for Cpk
• Cpk value calculated from sample mean, stdev is a point estimate.
• Based on your sample size n, you can calculate a confidence interval for your Cpk for
a given confidence level:

Upper / Lower Cpk limits = ± √( + )

For 95% confidence, zα = 1.96

• Larger sample sizes will result in tighter confidence limits for Cpk .
• You can solve this equation for n, and then calculate the minimum n for a specified
CL
Wu, Chin-Chuan and Kuo, Hsin-Lin. Sample Size Determination for
• Minitab calculates 95CIs for CPK, PPK the Estimate of Process Capability Indices. Information and
Management Sciences. Vol 15 No 1 March 2004. p 1-12.

(c) 2020 Raul Soto 37

V
Sample sizes for Cpk confidence limits
RS
• If you use this Excel code, you can enter [1] your CPK, [2] the sample size you used
to calculate it, and [3] the confidence level you want
• It will calculate the confidence interval for you
(c) 2020 Raul Soto 38

KENX JULY 2020 19 of 32

 Sample sizes for Cpk confidence limits
Cpk upper / lower limits
as a function of sample size
4.00
Series1
3.80 Series2
Series3
3.60

3.40

3.20

Cpk
3.00

2.80

2.60

2.40

2.20
• CPK = 2.95 (95CI: 2.64, 3.26)
2.00
0 50 100 150 200 250 300
sample size n

(c) 2020 Raul Soto 39

V
RS
Sample sizes for Cpk confidence limits

• If you use this Excel code, you can enter [1] your target CPK, [2] the width of the CI
you want, and [3] the confidence level you want.
• It will calculate for you the sample size you need.
(c) 2020 Raul Soto 40

KENX JULY 2020 20 of 32

 Sample sizes for Cpk confidence limits

• Solve the CPK confidence

limits equation by sample
size n

• Enter the desired ± CL,

and the desired
confidence level
(i.e. 95%) for a given CPK,
and calculate the
minimum sample size n
required

(c) 2020 Raul Soto 41

V Normality
RS
• Many statistical tools for variables data have an assumption of normality
• If data does not follow a normal distribution, results provided by the tool will not
reflect reality

• Visual methods:
• Histograms, Normal probability plots

• Statistical Normality tests:

• Kolmogorov-Smirnov (KS), Lilliefors-corrected KS, Shapiro-Wilk test, Anderson-Darling,
Cramer-von Mises, D’Agostino skewness test, Anscombe-Glynn kurtosis test, D’Agostino-
Pearson omnibus test, Jarque-Bera test.
• KS is frequently used, highly sensitive to extreme values
• Some researchers consider the Shapiro-Wilk (SW) test as the best choice
• Minitab includes AD, KS, and Ryan-Joiner which is similar to SW.
Ghasemi A, Zahediasl S. Normality Tests for Statistical Analysis:
A Guide for Non-Statisticians. [c] 2020 Raul Soto 42
Int J Endocrinol Metab. 2012;10(2):486-9. DOI: 10.5812/ijem.3505
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3693611/

KENX JULY 2020 21 of 32

 Normal data Non-Normal data
• Most points aligned along the diagonal line • Multiple points NOT aligned along the diagonal line
• p-value > 0.10 • p-value ≤ 0.01
• At least 90% confidence that the data follows a • CANNOT claim with least 90% confidence that the data
normal distribution follows a normal distribution

Probability Plot of C1 Probability Plot of C2

Normal Normal
99.9 99.9
Mean 49.99 Mean 0.9325
StDev 1.875 StDev 0.9503
99 N 100 99 N 100
RJ 0.996 RJ 0.901
95 P-Value >0.100 95 P-Value <0.010
90 90
80 80
70 70
Percent

Percent
60 60
50 50
40 40
30 30
20 20
10 10
5 5

1 1

0.1 0.1
45.0 47.5 50.0 52.5 55.0 57.5 -2 -1 0 1 2 3 4 5 6
C1 [c] 2020 Raul Soto C2 43

V Normality Testing
RS
• Use both graphical methods and p-values, don’t depend exclusively
on either Probability Plot of C2
Normal
• Small p-value => high probability data is NOT normal 99.9
Mean 0.9325
StDev 0.9503
• Normality tests are “finicky”, sensible to sample sizes 99
N 100
RJ 0.901
95 P-Value <0.010
• Small sample sizes (<20) will tend to come out “normal” even if the 90

data is not normal (underpowered) 80

70
P e rc en t

60
50
• Large sample sizes (>60) will tend to come out “not normal” even if 40
30
the data is normal (overpowered) 20
10
• If your plot “looks” normal, the p-values come out small, and your 5

data set is larger than n = 60: take a random sample and run your 1
normality test on that sample.
0.1
-2 -1 0 1 2 3 4 5 6
• “Fat pencil” test…  C2

[c] 2020 Raul Soto 44

KENX JULY 2020 22 of 32

 Tolerance Intervals

• Tolerance intervals allow us to make claims about a % of individual units

– “We are 95% confident that 99% of the units in lot ABC are within this range [ A, B ]”
– Can use this for product release, if the Tolerance Interval is within the specification
– Minitab can compute non-normal tolerance intervals for data sets that do not follow a
normal distribution.

[c] 2020 Raul Soto 45

V Tolerance Intervals
RS
• Where: Where do we get the k-factor?

– : sample mean • Statistical software (i.e. Minitab, JMP,

Stata) can calculate it
– s : sample standard deviation
– k : k-factor, parameter related to: • Book tables:
• ASQ Certified Quality Engineer
• Sample size Handbook, 4th Ed, Appendix C
• Confidence %
• Calculate manually using formulas in:
• Coverage % • https://www.itl.nist.gov/div898/hand
• One-sided or two-sided interval book/prc/section2/prc263.htm

[c] 2020 Raul Soto 46

KENX JULY 2020 23 of 32

 Tolerance Intervals in Minitab

Minitab : Stat / Quality Tools / Tolerance Intervals

[c] 2020 Raul Soto 47

V
RS
Minitab : Stat / Quality Tools / Tolerance Intervals

95/99 TOLERANCE INTERVAL FOR INDIVIDUAL UNITS

Tolerance Interval: Weight (mg)

Method
Confidence level 95%
Percent of population in interval 99%

Statistics
Variable N Mean StDev
Weight 60 488.569 6.826

95% Tolerance Interval

Nonparametric Achieved
Variable Normal Method Method Confidence
Weight (467.628, 509.510) (476.218, 504.279) 12.1%

Achieved confidence level applies only to nonparametric method

[c] 2020 Raul Soto 48

KENX JULY 2020 24 of 32

 Tolerance Interval Plot for Tablet.W(mg)
95% Tolerance Interval
At Least 99% of Population Covered Claims:
Statistics
N 60 • We are at least 95% confident that the mean
Mean 488.569
StDev 6.826 weight of the units in the lot is between
Normal 486.8 mg and 490.3 mg (95% CI)
Lower 467.628
Upper 509.510
472.5 480.0 487.5 495.0 502.5 510.0 Nonparametric • We are at least 95% confident that 99% of
Lower 476.218 the individual tablets in the lot have weight
Upper 504.279
Normal between 467.6 mg and 509.5 mg (95%/99%
Achieved Confidence
12.1%
TI)
Nonparametric
Normality Test
470 480 490 500 510 AD 0.420 • You can use this for accept / reject decisions.
P-Value 0.315
Normal Probability Plot
If the 99/95 TI is within spec, accept a lot.
99.9
99
• Based on the normality test (p-value > 0.05)
90
and normal probability plot, we are at least
P e rce n t

50 95% confident that the data follows a normal

10 distribution; therefore we use the normal
1 tolerance interval.
0.1
470 480 490 500 510

[c] 2020 Raul Soto 49

V
Selecting Confidence and Coverage %
RS
• Select these based on risk EXAMPLE
– Tighter limits for validation than Severity Validation Manufacturing
regular production
Critical 99% confidence 95% confidence
– Tighter limits for parameters 99% coverage 99% coverage
associated with high risk to patient Major 95% confidence 95% confidence
safety
95% coverage 90% coverage
Minor 95% confidence 90% confidence
80% coverage 80% coverage

[c] 2020 Raul Soto 50

KENX JULY 2020 25 of 32

 CPK / PPK vs Tolerance Intervals
• CPK and Tolerance Intervals are just two ways of mathematically representing
the same claim.
• From a practical standpoint these are NOT different acceptance criteria.
• Based on this claim and the sample size, a k-value is determined.
• The tolerance interval is constructed as : ±
• The Ppk cutoff is ≥ k-value/3.
• It is not possible to meet the tolerance interval claim and fail the ppk claim.
• They are both dependent on the same pre-determined k-value and will
pass/fail the acceptance criteria together.

(c) 2020 Raul Soto 51

V
Sample Sizes for Tolerance Intervals
RS
 Power analysis allows you to determine the minimum sample size required the
construct a tolerance interval
 EXAMPLE:

 We have a one-sided spec (USL = 10), and want to calculate the minimum sample size
for a tolerance interval with 95% confidence and 95% population.
 In Minitab:
 Stat > Power and Sample Size > Sample Size for Tolerance Intervals
 You will see the following window:

(c) 2020 Raul Soto 52

KENX JULY 2020 26 of 32

Sample Sizes for Tolerance Intervals
 [1] m%: margin of error for the population; this is the
additional % of the population beyond the desired p%, that
may be included in the interval. Select a number between 1
and 5. The smaller the number, the higher your sample size
will be.
 [2] Confidence level for the tolerance interval. 95% in this
example
 [3] p%: population coverage of the tolerance interval. 95% in
this example.
 [4] probability for the margin of error [1]: probability that the
interval will be wider than p% by m% or more. Use 0.01, 0.05,
or 0.1; the larger the number you put here, the higher the
chance that your interval will be wider than p% by m%
 [5] Upper bound: select here if your spec has both an
upper/lower limit, only a lower limit, or only an upper limit

(c) 2020 Raul Soto 53

V
Sample Sizes for Tolerance Intervals
RS
• If your data follows a NORMAL
distribution, nmin = 70 units

• If your data does not follow a

normal distribution, nmin = 181
units.

(c) 2020 Raul Soto 54

KENX JULY 2020 27 of 32

 Potential Causes of Non-Normality

 Multi-modality

(c) 2020 Raul Soto 55

V
Potential Causes of Non-Normality
RS
(c) 2020 Raul Soto 56

KENX JULY 2020 28 of 32

Potential Causes of Non-Normality

 Potential outliers

(c) 2020 Raul Soto 57

V Non-Normality
RS
• Sometimes the quality characteristic we want to measure does
not follow a normal distribution
• Look at the histogram:
• Cause of non-normality: Bimodal? Fat tails (kurtosis)? Skewed? Outliers?
• Alternatives:
• Non-normal capability analysis (i.e. fit data with another distribution)
• transform the data (for example, take a natural logarithm) and check if
the transformed data is normally distributed
• Non-Parametric methods (less sensitive, require higher sample sizes for
the same statistical power)
• ALWAYS consult with your company’s statisticians (the real
ones!) before you use any of these, or any other, alternatives.
[c] 2020 Raul Soto 58

KENX JULY 2020 29 of 32

 How to Handle non-Normal data
 If your data comes out as non-normal, please contact your company’s statisticians BEFORE you try to do any
kind of data manipulation such as transformations.

 It’s extremely important to check the data and determine what are the possible sources of non-normality, for
example:
 bimodality / multimodality
 data truncation (common in destructive tests)
 skewness
 kurtosis
 outliers
 or maybe the process just does NOT follow a normal distribution

 Depending on what is the underlying source of non-normality, statisticians can provide appropriate alternatives
and the associated risks.

 The fundamental risk of manipulating a non-normal data set is that you will no longer be making decisions
based on your real data, but on data that is at least one step removed from reality.
(c) 2020 Raul Soto 59

V Transformations
RS
 Common transformations used for non-normal data:

 Johnson transformation:
 Minitab: Stat > Quality Tools > Johnson Transformation

 Box-Cox transformation:
 see https://www.itl.nist.gov/div898/handbook/pmc/section5/pmc52.htm

 raises each data point to a given power λ

 if you select λ = 0, takes the natural logarithm of your data point
 if you select λ = 0.5, takes the square root of your data point

(c) 2020 Raul Soto 60

KENX JULY 2020 30 of 32

 Test validation lots for homogeneity
 The best way to do this is the following:
 [1] using Levene’s test for equal variances to test for within-lot homogeneity
 Minitab: Stat > ANOVA > Test for Equal Variances

 [2] if Levene’s test concludes that your within-lot variation is consistent throughout your lots, use a

One-Way ANOVA to test your lot-to-lot variability

 Stat > ANOVA > One-Way
 Options: CHECK “Assume equal variances”
 Use Tukey’s Test

 [3] if Levene’s test concludes that your within-lot variation is NOT consistent throughout your lots, then
use a Welch’s ANOVA
 Stat > ANOVA > One-Way
 Options: DO NOT CHECK “Assume equal variances”

(c) 2020 Raul Soto 61

V
RS
https://xkcd.com/552/

[c] 2020 Raul Soto 62

KENX JULY 2020 31 of 32

 Questions

[c] 2020 Raul Soto 63

V
RS
KENX JULY 2020 32 of 32