UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

National Guard
Black Belt Training
Module 23

Measurement
System Analysis (MSA)
Introduction
UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO
UNCLASSIFIED / FOUO

CPI Roadmap – Measure

8-STEP PROCESS
6. See
1.Validate 2. Identify 3. Set 4. Determine 5. Develop 7. Confirm 8. Standardize
Counter-
the Performance Improvement Root Counter- Results Successful
Measures
Problem Gaps Targets Cause Measures & Process Processes
Through

Define Measure Analyze Improve Control

TOOLS
•Process Mapping
ACTIVITIES
• Map Current Process / Go & See •Process Cycle Efficiency/TOC
• Identify Key Input, Process, Output Metrics •Little’s Law
• Develop Operational Definitions •Operational Definitions
• Develop Data Collection Plan •Data Collection Plan
• Validate Measurement System •Statistical Sampling
• Collect Baseline Data •Measurement System Analysis
• Identify Performance Gaps •TPM
• Estimate Financial/Operational Benefits •Generic Pull
• Determine Process Stability/Capability •Setup Reduction
• Complete Measure Tollgate •Control Charts
•Histograms
•Constraint Identification
•Process Capability
Note: Activities and tools vary by project. Lists provided here are not necessarily all-inclusive. UNCLASSIFIED / FOUO 2
UNCLASSIFIED / FOUO

Learning Objectives
 Understand the importance of good measurements
 Understand the language of measurement
 Understand the types of variation in measurement
systems

UNCLASSIFIED / FOUO 3
UNCLASSIFIED / FOUO

Exercise: The Three Rs

UNCLASSIFIED / FOUO 4
UNCLASSIFIED / FOUO

Examples
 The Hale Koa Hotel manager wants to reduce
customer check-in time
 The VA wants to reduce VA Home Loan Guarantee
Program processing errors
 The Army Community Service organization wants to
improve its customer service performance
 A VA Hospital is interested in finding ways to improve
in-patient and out-patient care

UNCLASSIFIED / FOUO 5
UNCLASSIFIED / FOUO

Why Is MSA Important?

 Our ability to assess the performance of a process we wish

to improve is only as good as our ability to measure it
 The measurement system is our “eyes” for our process
 We need to be able to see the performance of our
process clearly in order to improve it
 Sometimes, improving the ability to measure our
process results in immediate process improvements

Can you trust your measurements to tell you the truth?

Measurement System Analysis UNCLASSIFIED / FOUO 6

UNCLASSIFIED / FOUO

Sources Of Observed Process Variation

Observed
Observed Variation
Variation

Actual
Actual Process
Process Variation
Variation Measurement
Measurement Variation
Variation
- Long-term Process Variation
- Short-term Process Variation
Variance
Variance Variance
Variance
Due
Due to
to Instrument
Instrument Due
Due to
to Operators
Operators
- Repeatability - Reproducibility
- Calibration
- Stability
- Linearity

The variation due to the measurement system must be identified first,

then separated from actual process variation

UNCLASSIFIED / FOUO 7
UNCLASSIFIED / FOUO

Variation Is Additive

Measured values

Actual values

s2 Observed = s2 Measurement + s2 Part + s2 Error

s2 Measurement = s2 Observed – s2 Part – s2 Error
s2 Measurement = s2 Repeatability + s2 Reproducibility + s2 Error

UNCLASSIFIED / FOUO 8
UNCLASSIFIED / FOUO

Why Worry About Measurement Variation?

 Consider the reasons why we measure:

Assist in
Verify process How might measurement continuous
conformity to variation affect these decisions? improvement
specifications
activities
What if the amount of
Process
Process measurement variation
is unknown
Measurement
Measurement

Measurement variation can make our process capabilities appear worse than they are.

UNCLASSIFIED / FOUO 9
UNCLASSIFIED / FOUO

Measurement Variation
 Measurement Variation is broken down into two components: (The
two Rs of Gage R&R)
 Reproducibility (Equipment or Gage or Operator Variability)
 Different individuals get different measurements for the same thing
 Repeatability (Equipment or Gage or Operator Variability)
 A given individual gets different measurements for the same thing when
measured multiple times

 The tool we use to determine the magnitude of these two sources of

measurement system variation is called Gage R&R

UNCLASSIFIED / FOUO 10
UNCLASSIFIED / FOUO

Reproducibility (Operators’ Precision)

 Reproducibility is the variation in the average of the
measurements made by different operators using the
same measuring instrument when measuring the
identical characteristic on the same part

Inspector A

s  s s
2
m
2
g
2
o

Inspector B Inspector C

UNCLASSIFIED / FOUO 11
UNCLASSIFIED / FOUO

Repeatability (Gage Precision)

 Repeatability is the variation between successive
measurements of the same part, same characteristic,
by the same person using the same equipment
(gage). Also known as test /re-test error, used as an
estimate of short-term variation.

Ideal Process Target

s  s s
2
m
2
g
2
o

UNCLASSIFIED / FOUO 12
UNCLASSIFIED / FOUO

Measurement Error
Gage R & R variation is the percentage that Generally recognized criteria for
measurement variation (repeatability and gage acceptability is when
reproducibility) represents of the variation observed Gage R & R variability to process
in the process variability is :
Under 10%: Acceptable gage
10% to 30%: Might be
Observed Measurements acceptable
Over 30%: Gage is
unacceptable and should be
corrected or replaced
True Values Measurement Error

Bias Gage R&R Stability Discrimination Linearity

Repeatability Reproducibility

Operator Operator * Part

UNCLASSIFIED / FOUO 13
UNCLASSIFIED / FOUO

Bias (Instrument Accuracy)

 Bias is the difference between the observed average
value of measurements and the master value. The master
value is determined by precise measurement typically by
calibration tools linked to an accepted, traceable reference
standard.

Master Value (Reference Standard)

Average Value

UNCLASSIFIED / FOUO 14
UNCLASSIFIED / FOUO

Stability
 Stability = If measurements do not change or drift
over time, the instrument is considered to be stable

Time One

Time Two

UNCLASSIFIED / FOUO 15
UNCLASSIFIED / FOUO

Discrimination
 Discrimination is the capability of detecting small changes in the
characteristic being measured
 The instrument may not be appropriate to identify process variation or quantify
individual part characteristic values if the discrimination is unacceptable
 If an instrument does not allow differentiation between common variation in
the process and special cause variation, it is unsatisfactory

.28 .28 Ruler .28 .28

.279 .282 Caliper .282 .279
.2794 .2822 Micrometer .2819 .2791

UNCLASSIFIED / FOUO 16
UNCLASSIFIED / FOUO

Linearity
A measure of the difference in bias (or offset) over the range of
the sample characteristic the instrument is expected to see
determines linearity. If the bias is constant over the range of
measurements, then linearity is good.
 Over what range of values for a given characteristic can the
device be used?
 When the measurement equipment is used to measure a wide range of
values, linearity is a concern.
Measurement
Variation

Low High
End Measurement Scale End

UNCLASSIFIED / FOUO 17
UNCLASSIFIED / FOUO

Name That Problem!

Master Value
Instrument 1

Instrument 2
Average Value

Master Value
(Reference Standard)
Time One

Time Two

.28
1. Discrimination
.279 2. Bias/Accuracy
.2791
3. Repeatability
4. Reproducibility
Inspector A 5. Instrument Bias
Inspector B
Inspector C
6. Stability

UNCLASSIFIED / FOUO 18
UNCLASSIFIED / FOUO

Measurement Systems Analysis Template

The Measurement System used to collect data has been calibrated and is considered to have no potential for significant
errors. The data collection tool is reliable, can be counted on, has good resolution, shows no signs of bias and is stable.

Type of
Measurement Description Considerations to this Project
Error
The ability of the measurement Work hours can be measured to <.25
Discrimination
system to divide measurements into hours. Radar usage measure to +- 2
(resolution)
“data categories” minute.
The difference between an observed No bias - Work hours and radar start-
Bias average measurement result and a stop times consistent through
reference value population.
No bias of work hours and radar
Stability The change in bias over time
usage data.
Not an issue. Labor and radar usage
Repeatability The extent variability is consistent is historical and felt to be accurate
enough for insight and analysis.
- Example - Remarks in usage data deemed not
Different appraisers produce reproducible, therefore were not
Reproducibility
consistent results considered in determining which
radars were used in each op
Variation The difference between parts N/a to this process.
Required Deliverable UNCLASSIFIED / FOUO 19
UNCLASSIFIED / FOUO

Reported by :
Gage
Tolerance:
name:

Measurement Systems Analysis Template Date

Misc:
of study :

Gage R&R (ANOVA) for Response

Gage R&R
Components of Variation Response by Part
%Contribution
Source VarComp (of VarComp) 100 % Contribution
% Study Var
10.00
Total Gage R&R 0.0015896 3.70

Percent
Repeatability 0.0005567 1.29 9.75
Reproducibility 0.0010330 2.40 50
Operator 0.0003418 0.79 9.50
Operator*Part 0.0006912 1.61
0
Part-To-Part 0.0414247 96.30 Gage R&R Repeat Reprod Part-to-Part 1 2 3 4 5 6 7 8 9 10
Total Variation 0.0430143 100.00 Part
R Chart by Operator
Study Var %Study Var Response by Operator
1 2 3
Source StdDev (SD) (6 * SD) (%SV) UCL=0.1073
0.10 10.00
Total Gage R&R 0.039870 0.23922 19.22
Sample Range

Repeatability 0.023594 0.14156 11.38

Reproducibility 0.032140 0.19284 15.50 _ 9.75
0.05
Operator 0.018488 0.11093 8.91 R=0.0417
Operator*Part 0.026290 0.15774 12.68 9.50
Part-To-Part 0.203531 1.22118 98.13 0.00 LCL=0
1 2 3
Total Variation 0.207399 1.24439 100.00
Operator
Xbar Chart by Operator
Number of Distinct Categories = 7 1 2 3 Operator * Part Interaction
10.00 10.00 Operator
Sample Mean

UCL=9.8422
__ 1

The Measurement

Average
2
X=9.7996 9.75
9.75 LCL=9.7569
3

System is acceptable
9.50
with the Total Gage 9.50
1 2 3 4 5 6 7 8 9 10
R&R % Contribution Part

<10% - Example - Optional BB DeliverableUNCLASSIFIED / FOUO 20

UNCLASSIFIED / FOUO

Takeaways
 It is important to be able to rely on the accuracy and precision
of the measurement system to make good decisions
 Understand the various types of measurement system variation
 Eliminate as much of the variation in the measurement system
as possible to focus on and improve the true cause of variation
in process performance

UNCLASSIFIED / FOUO 21
UNCLASSIFIED / FOUO

What other comments or questions

do you have?

UNCLASSIFIED / FOUO 22