What is MSA?

1
 MSA stands for Measurement SystemAnalysis
MSA is a tool for the
evaluation of statistical
properties of process
of Measurement
System
AIAG’s 4th edition effective June, 2010 is the
latest version in use in aautomotive industry
as a reference manual.

2
What is Measurement?

3
 Measurement is defined
as…
ASSIGNMENT OF NUMBERS (VALUE) TO
MATERIAL THINGS TO REPRESENTTHE
RELATIONS AMONG THEM WITH RESPECT TO
PARTICULAR, PROCEDURES, GAUGES AND
PROPERTIES.

C. EISENHART (1963)

4
How far would you drive without
your gauges?

5
6
What is a Measurement System?

7
 Measurement System consists
The collectionof…
of instruments or gauges, standards, operations, methods,
fixtures, software, Personnel, environment and assumptions used to quantifya
unit of measure or fix assessment to the feature characteristics being
measured, the complete process used to obtain measurements.

S Standard
W Work piece (item formeasurement)
I Instrument
P Person/ Procedure
E Environment
8
What is a Measurement
System Analysis?

9
 MEASUREMENT SYSTEM ANALYSIS

STUDYY OF COMBINED EFFECT (ON RESULT OF MEASUREMENT) OF ALL

CONTRIBUTORS (I.E MEASUREMENT SYSTEM USED IN MEASUREMENT
PROCESS), ASSESSING THEIR SUITABILITY AGAINST MEASUREMENT
OBJECTIVE, AND IMPROVEMENT THEREOF.

A TOOL USED TO EVALUATE THE STATISTICAL PROPOETIES OF PROCESS

MEASUREMENT SYSTEM.

10
What are the objectives of
Measurement?

11
Objective of measurement
process
• PRODUCT CONTROL:-
IS THE PART IN A SPECIFIC CATEGORY i.e. MEETS
SPECIFICATION REQUIREMENTS?

• PROCESS CONTROL :-
IS THE PROCESS VARIATION STABLE &
ACCEPTABLE

12
Why MSA?

13
 The Hidden Factory --
Process/Production OK
First
Inspect
Input Operation time
correct
Rework NOT
Hidden Factory OK MSA Helpsin
preventing
Scrap Time, cost, these wastes
people
What comprises the hidden factory in a process/ production
area? Reprocessed and Scrap materials -- First time out of
spec, not reworkable

Over-processed materials -- Run higher than target with higher than

needed utilities or reagents

Over-analyzed materials -- High Capability, but multiple in-process

samples are run, improper SPC leading to overcontrol 14
 The Hidden Factory -- Measurement
Sample
Systems OK
Lab work Inspect Production
Input

Retest NOT
Hidden Factory OK MSA Helpsin
preventing
Waste Time, cost, these wastes
people
What Comprises the Hidden Factory in a Laboratory Setting?

Incapable Measurement Sytems -- purchased, but are unusable due tohigh repeatability variation and poor
discrimination

Repetitive Analysis -- Test that runs with repeats to improve known variation or to unsuccessfully deal with
overwhelming sampling issues

Laboratory “Noise” Issues -- Lab Tech to Lab Tech Variation, Shift to Shift Variation, Machine to Machine Variation,
Lab to Lab Variation

15
What are the purposes of MSA?

16
Purposes of
MSA
What are the purposes of MSA?
MSA serves following purposes:
1. A criterion to accept new measuring equipment.
2. A comparison of one measuring device againstanother.
3. A basis for evaluating a gage suspected of beingdeficient.
4. A comparison for measuring equipment before and after repair.
5. A required component for calculating process variation, and the
acceptability level for a production process.

17
What is the flow of measurement
process?

18
Flow of process for
Measurement
• STANDARD
• WORK PIECE (PART)
• INSTRUMENT
• PERSON MEASUREMENT MEASUREMENT RESULT
• PROCEDURE
• ENVIRONMENT

DECISION (ACTION) ANALYSIS

19
 Q: What is the effect on
measurement if there are errors in
measurement system?

20
If there are errors in measurement system, then following
is the effect on measurement:

21
If there are errors in measurement system, then following
is the effect on measuremen t:
 A wrong decision may be taken
whenever any part of the
measurement distribution
overlaps a specification limit.

 A “Good” part will sometimes

be called as “Bad” (type I error,
producer’s risk or false alarm)

22
If there are errors in measurement system, then following
is the effect on measurement:
 A “Bad” part will sometimes
be called as “Good” (type II
error, consumer’s risk or miss
rate)

 This is most risky as

Consumer will be suffering as
“Bad” part will be delivered
to him as “Good” part.
23
Q: What are concepts of Accuracy
& Precision?

24
Concept of accuracy and
precision:
Accuracy : difference between the observed
measurement and the actual measurement.

Precision : variation that occurs when measuring the

same part with the same instrument.

25
Conditions of accuracy and
precision:

Precise but not Accurate but Not accurate Accurate

accurate not precise or precise and
precise

26
Concept of Precision and
Accuracy
Accuracy refers to how close a
measurement is to its
standard or know value
Precision refers to how close
two or more measurements
to each other, regardless of
whether those measurements
are accurate or not.

27
What are the sources of errors
in Measurement System?

28
Sources of errors in a measurement
system: Sources of Variations

Manufacturing Process Product

Input Material Design
Process (Due to Tolerance errors)
design

Parameter Method Processing Environment Measurement

Machine Toolin
variation variation material
g

29
What are the types of errors
in measurement system?

30
Types of errors in a measurement
system:
- BIAS
- LINEARITY LOCATION
ERRORS
- STABILITY

- REPEATABILITY
WIDTH ERRORS
- REPRODUCIBILITY
Types of errors in a accuracy of measurement system:

1. Accuracy: a measure of the amount of bias in the

system.

2. Linearity : a measure of the bias values through

the expected range of measurements.

3. Stability: the consistency of measurements

over time.
Types of errors in precision of a measurement system:

1. Repeatability (Equipment variation): variation in

measurements under exact conditions.

2. Reproducibility (Appraiser variation): variation in the

average of measurements when different operators
operators measure the same part.

33
Results of errors in measurement
system: LSL
15

USL
10

Freque
Actual process

n cy
variation - 5

No measurement error 0

1
5 LSL USL

Observed process 1
0

variation

Frequen
- With measurement
cy
5

error 0

3 4 5 6 70 8 9 100
3 0 0 0 0
Observ
0 0 110
4 ed
Q: What is traceability relationship?

35
Traceability

36
Q: What is Discrimination?

37
Discrimination
The number of decimal places that can be measured by the system
Increments of measure should be about one-tenth of the width of the product
specification or process variation
Poor
Discrimination

1 2 3 4 5

Good Discrimination

1 2 3 4 5

38
Q: What are features of a good
measurement system?

39
Features of a good measurement
system:
 ADEQUATE SENSITIVITY – 10 TO 1 RULE – Instrument
ACCURACY (and therefore Least Count) should divide the
tolerance ( or process variation) into 10 parts or more –
practical minimum point for gauge selection

 Measurement system ought to be in statistical control –

presence of only common causes of variation and not the
special causes

 Variation of measurement system must be small compared

to specification limits or process variation – more PRECISION
40
Q: What is Bias study and how is
it done?

41
Determining and assessing Bias
 Bias is the difference between the
Study: Observed
observed average of
measurement and the reference Average
 value. Value
The reference value, also known as Bia
the accepted reference value or s
master value, is a value that serves
as an agreed-upon reference for
the measured values. A reference
value can be determined by
Referenc
averaging several measurements
with a higher level of measuring e
equipment. Value
42
Determining and assessing Bias
Objective : To determine the variation between the
Study:
observed
measurement and the actual measurement ofa part.
Process:
1. Choose sample standards.
- Obtain accepted reference value for part
- Use tool room or layout inspection equipment

2. Measure sample standards 15 to 25 times using the same

measuring device, the same operator, and the same
setup. 43
Determining and assessing Bias
Study:
3. Calculate x-bar
4. Calculate bias
Bias = Average – Reference Value
(To find out if the bias is significant or not, t statistics
is applied as below. To be covered in later
classes…)
5. Calculate the upper and lower confidence limit (CL).
6. Find out Absolute t cal

44
Determining and assessing Bias
Study:
Condition : For the bias to be Insignificant
1.) Lower limit ≤ zero ≤ Upper limit
Error (bias) should lies within the acceptable limit (Lower
& Upper)

2.) Absolute t cal < t critical

45
 Accuracy
Exercise
How would we investigate a measurement
system’s accuracy?
Who would typically provide this function?
What is the typical term used to describe the
accuracy assessment of a measurementsystem?

46
 Accuracy
How would we investigate
Exercisea measurement system’saccuracy?
Calibration Study
Who would typically provide this function?
Lab personnel
What is the typical term used to describe the
accuracy assessment of a measurementsystem?
Repeatability

47
Q: What are the possible causes
of bias?

48
Possible causes of
Bias
• Error in master or reference value. Check mastering
procedure
• Worn Instrument
• Instrument made to wrong dimension
• Instrument measuring the wrongcharacteristics
• Instrument not calibrated properly. Review Calibration
Procedure.
• Instrument used improperly by appraiser. Review
measurement instructions.
• Environment – temperature, humidity, vibration,
cleanliness. 49
END OF SESSION ONE

50
Q: What is Linearity study and
how is it done?

51
Linearity Study
Linearity is the difference in the bias values through the expected operating range
of the gauge.
Reference Referenc
Observed Observed
Valu e
e Average Average
Value Value
Value
Smaller Larger
Bias Bias

50mm 200mm
52
Linearity Study
Objective:
To determine the difference between the obtained value and a
reference value using the same instrument over the entire
measurement space.
Process:
1. Choose three to five sample standards that cover the
measurement space.

2. Measure sample standards 10 to 25 times.

53
Linearity Study
3. Calculate the average of the readings.
4. Calculate bias.
5. Plot reference values on x-y graph.

(Detailed calculations to be covered later…)

6. Calculate slope of the linear regression

line.
7. Find out T(slope) & T(intercept)
54
Linearity Study
Condition : For the Linearity to be Insignificant

 For Graphical analysis i.e bias '0' line should not intersect the
confidence band of fitted line.

 T cal value should be more than T slope & T uncertainty

55
Linearity Study: Charting Linearity – graphical
analysis

56
Linearity Study:
A measure of the difference in accuracy or precision
over
the range of instrument
Gage 1: capability. Gage 2:
Linearity is an issue here
Linearity is not an
issue here

Accuracy Accuracy

0
0

Measurement Units Measurement Units

57
Linearity Study: Possible causes of Linearity not ok
• Instrument needs calibration, reduce calibration interval
• Worn instrument, equipment or fixture
• Worn or damaged master , error in master
• Poor quality instrument – design or conformance
• Wrong gauge for the application
• Distortion (gauge or part) with part size
• Application – part size, position, operator skill, fatigue,
observation error (readability, parallax)
• Environment – temperature, humidity, vibration, cleanliness.

58
Q: What is Stability study and how is
it done?

59
Stability Study: Determining and Assessing
Stability
 Stability (or drift) is the total Tim
variation in the measurements e
obtained with a measurement
system
• on the same master or partwhen
measuring a single characteristic
• over an extended

time period

Reference
Value
60
Stability Study: Evaluate
Stability
Objective : To evaluate measurement system to determine if the
system is in statistical control.
Procedure:
1. Choose sample standards.
2. Establish it’s reference value
3. Measure sample standards three to five times.
4. Measuring should be over a period of days/week (not
in hours)
5. Plot data on a x-bar and R chart.

61
Stability Study: Evaluate Stability -
analysis
Analysis:

1.Establish control limits & evaluate for out of control or

unstable conditions using standard chart control analysis.

2. If process is unstable determine and correct thecause.

62
Stability Study: Possible causes of in stability
• Instrument needs calibration, reduce calibration interval
• Worn instrument, equipment or fixture
• Normal Aging or obsolescence
• Worn or damaged master , error inmaster
• Poor quality instrument – design or conformance
• Different measuring method – setup, loading, clamping, technique
• Wrong gauge for the application
• Distortion (gauge or part) with part size
• Application – part size, position, operator skill, fatigue, observation error
(readability, parallax)
• Environment – temperature, humidity, vibration, cleanliness.
63
Q: What is GRR variable study and
how is it done?

64
 Precision: Repeatability  rp t
 The inherent variability of the measurement system
 Variation that occurs when repeated measurements are made of the
same variable under similar conditions
Same operator
Same set-up
Same units
Same
environmental
conditions
Short-term
 Estimated by the pooled (average) standard deviation of the
distribution of repeated measurements
 Repeatability is usually less than the total variation of 65
 Repeatability
The variation between successive measurements of the
same part, same characteristic, by the same person using the
same instrument. Also known as test - retest error; used as
an estimate of short-term variation.
Master
Value
Poor
Good Repeatability
Repeatabilit
y

Mean Mean 66
 Precision: Reproducibility  rpd
The variation that results when different conditions are used to make
the measurements
Different operators
Different set-ups
Different test units
Different environmentalconditions
Long-term
Estimated by the standard deviation of the averages of
measurements from different measurement conditions

67`
 Reproducibility
The difference in the average of the measurements made
by different persons using the same or different instrument
when measuring the identical characteristic.
Master Poor
Value
Good Reproducibility
Reproducibility

Mean Mean

Operator 1 Operator 2 Operator 3 Operator 1 Operator 2 Operator 3

68
 Conducting Gage R&R study
(variable)
 Sample of Ten units are selected from
manufacturing process.
 Three assessors who usually do measurements
are selected to conduct study.
 Each part is measured Two / Three times at random

by three assessors and results are collected &

tabulated.
 Equipment variation & Appraiser variation is calculated

69
 Conducting Gage R&R study

(variable)
Calculations R = 0.3417, n = 10, r = 3, Xdiff = 0.4446, Rp = 3.511
 Repeatability = Equipment variation (EV) = R * K1
 = 0.3417*0.5908 = 0.20188
 Reproducibility = Appraiser variation (AV) = Sq. root {(Xdiff * K2) – (EV*EV /

nr)}
 = 0.22963
 n = parts, r = trials Trials (r) 2 3
K1 0.8862 0.5908
Appraisers 2 3
K2 0.7071 0.5231

70
 Conducting Gage R&R study
 Repeatability &(variable)
Reproducibility (GRR) = Sq. root (EV*EV + AV*AV)
 GRR = 0.30375
 Part variation (PV) = Rp*K3 = 1.10456
 Total variation (TV) = Sq. root (GRR*GRR + PV*PV)
 = 1.14610
 % EV = 100 [EV / TV] = 17.62 %
 % AV = 100 [AV / TV] = 20.04 %
 % GRR = 100 [GRR / TV] = 26.68 %
 % PV = 100 [PV / TV] = 96.38 %
Parts 2 3 4 5 6 7 8 9 10

K3 0.7071 0.5231 0.4467 0.4030 0.3742 0.3534 0.3375 0.3249 0.3146

71
 % Repeatability &
Reproducibility
Observed Process
Variation

%R&R = measurement
20% system
variation
%R&R =
75%

%R&R = 100%

72
 Gage R&R X / R
G a g e R & R (ANOVA) for
Chart G ag e n am e:

•
D a t e o f s tu d
y : R ep or te d
We want to see variability
Measurem b y : T o le r a n

in the X chart outside the

c e : Mis c :

Xbar Chart by
Ope rator
Control Limits
Mar y
8.
2
SampleMea

Fr ed Jo e

8. 3 . 0 S L = 8 .1 0
1

8.
2
X =8 .04
4
• This indicates Part-to-
- 3 .0 S L = 7 . 9 8

Part variability
0
6
n

7.
9

R Chart by • If there was none, you

Operator Jo e Mar y

probably did not get samples

0.
2 Fr ed
SampleRa

3 . 0 S L = 0 .1 4 5

0.
1
9
that cover the normal rangein
production
nge

R=0 .0566
7

0.0 - 3 .0 S L = 0 . 0 0
0

• The Range chart shows the impact of the Discrimination

Levels
• We want at least 3 levels within the controllimits
73
 X-Chart
Indicators
If the averages for each operator is different, the
reproducibility is suspect
We want more averages to fall outside the control limits but
consistently for all operators.This indicates more part-to-part
variability which is what we want.
We want to see the majority of the points on the chart
outside
the control limits
If this is the case and the R-Chart is in control, then we will be able
to determine the percent of the process variability that is consumed
74
by the measurement system
 R-Chart
Indicators
Suspect inadequate Discrimination if:
 the range chart has less than 3 distinct levels within the Control
Limits
Repeatability is questionable if the range chart shows out-of-
control conditions
If the range for an operator is out-of-control and the others
are not, the method is suspect
If all operators have ranges out-of-control, the system is
sensitive to operator technique
75
 Operator-Part InteractionPlot
Operator* Part
8. Interaction O p e ra
2 to r
Fr e d
Joe
8. Ma
Averag
1 ry
e

8.
0

7.
9
Part ID 1 2 3 4 5 6 7 8 9
10

• A line is plotted for each operator by part.

• Significant interactions are indicated by crossing lines between operators.
• We desire parallel lines for all operators, all parts.
• If there exists Operator-Part interactions, we need to understand and resolve them.
76
 Components of
10
0
Variation %Total Var
% Study
90 V ar
80 % T o l er
70
Perc

60
nt

50
e

40
30
20
10
0 GageR&R Repeat Reprod Part-to-Part

• A graphical representation of the data discussed before.

• We want the Gage R&R bars to be as small as possible, driving the Part-
to- Part bars to be larger! 7
7
 Analysis of GRR
studies
If Repeatability is large compared to Reproducibility
 The instrument needs maintenance

 The gage may need to be redesigned to be more rigid

 The clamping or location for gauging needs to be improved

 There is excess within part variation

78
 Analysis of GRR
studies
If Reproducibility is large compared to Repeatability
 The Appraiser needs to be bettertrained in how to use & read

gage instrument
 Calibrations on the gage dial are not clear

 A fixture of some sort may be needed tohelp the Appraiser

use the gage more consistently

79
END OF SESSION Three

80
GRR ATTRIBUTE
STUDY

81
 Attribute Measurement Systems
Study
 Discrete qualitative data
 Go/no-go basis; or limited data categories
 Compares parts to specific criteria for accept/not acceptor to
be placed in category
 Must screen for effectiveness to discern good parts from bad
 At least two appraisers and two trials each
 If available, have Quality Master rate parts first
82
 Attribute MSA
Study
Appraiser A Appraiser B
Attribute MSA
Master 1 Example
2 1 2
1 G G G G G
2 G G G G G
3 G NG G G G
4 NG NG NG NG NG
5 G G G G G G = Good
6 G G G NG G NG = Not Good
7 NG NG NG G NG
8 NG NG NG G G
9 G G G G G
10 G G G G G
11 G G G G G
12 G G G G G
13 G G NG G G
14 G G G G G
15 NG G G G G
16 G G G G G
17 G G G G G
18 G G G G G
19 G G G G G
20 NG G G G G

83
 metrics for msa attribute
Effectiveness (E) –data
Effectiveness is the ability of an appraiser to accurately detect
conforming & non confirming parts. Effectiveness is the
fraction whose value is between 0 and 1. Higher is the
number more effective is the appraiser. E equal to 1 he
correctly identifies the non-confirming parts as non-
confirming parts & confirming parts as confirming parts.

E = No. of parts correctly identified

Total no. of opportunities to be correct
84
 metrics for msa attribute
data
Probability of a Miss – P(Miss)
The probability of a miss is the chance of not rejecting non
conforming part or accepting a non conforming part. This
error is serious as non confirming part is accepted, it goes
to customer.

P(Miss) = No. of misses No. of

opportunities for misses
85
 metrics for msa attribute
data
Probability of a False Alarm – P(FA)
The probability of a false alarm is chance of rejecting a
confirming part. If P(FA) goes large, the wasteful cost
of rework & re-inspection increases.

P(FA) = No. of false alarms

No. of opportunities for false
alarm
86
 metrics for msa attribute
Bias (B) data
Bias is a measure of person tendency to classify an item as
conforming or non conforming, and is a function of P(Miss) &
P(FA). Bias value equal to or greater than 0 , have following
meaning
B = 1 implies no bias, B >1 implies bias towards rejecting parts (Type I error),
B<1 implies bias towards accepting parts (Type II error)
B = B(FA) is a function of P(FA)
B(Miss) is a function of
P(Miss)
87
 msa attribute decision
table

For any marginally acceptable or unacceptable gauges or appraisers, corrective action

is required. After corrective action, re-do the inspection capability study.

88
Table for getting B(FA) &
B(MISS)

89
 msa attribute – kappa
analysis
Kappa analysis allows us to determine the level of agreement with the standard or
reference value and among the appraiser. Kappa value ranges from -1 to +1. Higher
the value of Kappa, stronger is the agreement between the rating and standard
or between two appraisers.
If,
Kappa Value = 1, than perfect agreement exist
Kappa Value = 0, the agreement would be same as expected by chance
Kappa Value = Negative value occurs when agreement is weaker by
chance
rarely occurs
Kappa value above 0.9 is excellent
Kappa value less than 0.7 indicates that measurement system needs
improvement
90
 Application of appropriate
TYPE OF GAUGE method
BIAS / LINEARITY STABILITY GRR
UNCER-

TANITY
VERNIER
0-150mm    
SKILL IN HOLDING
THEJAWS PARALLALY

  
CMM
0-1500mm

 
SLIP GAUGES

SKILL OF RINGING

 
PLUG GAUGE

SKILL ON
BORDERLINE CASES

91
According to David C. Crosby, “If you don’t know the capability of
your measurement system, you don’t know if your
measurements, or your products, are good or
bad.”

Thanks

92
93
According to David C. Crosby, “If you don’t know the capability
of your measurement system, you don’t know if your
measurements,or your products, are good or bad.”

Thanks

Visit www.SubMastery.com for best

knowledge
94