Gage Repetability and

Reproducibility Analysis
Gage R&R

1
 Collect
baseline Validate
data on Develop a your Analyze Determine
defects Sampling measurement patterns process
and Strategy system in data capability
possible using
causes Gage R&R

2
 The Objective

• Understand the language of Measurement

• Show the importance of Measurement

• Walk away knowing how to perform a Gage

R&R, and how to interpret results

• Share some lessons learned (Pre-warnings)

• Minitab Application in Measurement

3
 The Science of Measurement
“I often say that when you measure what you
are speaking about and express it in numbers,
you know something about it”
LORD KELVIN

He Clearly stressed that little progress is possible in any

field of investigation without the ability to measure. The
progress of measurement is, in fact, the progress of
science

THE FOUNDATION OF TEST IS MEASUREMENT

4
 Foundation of the Tools
Data is derived from objects, situations, or phenomenon in the
form of measurements.

Data is used to classify, describe, improve, or control objects,

situations, or phenomena.
Levels of Analysis:
1. We only use experience, not data.
2. We collect data, but just look at the numbers.
3. We group the data so as to form charts and graphs.
4. We use census data with descriptive statistics.
5. We use sample data with descriptive statistics.
6. We use sample data with inferential statistics.
Cost, likelihood of improvement, ability to generalize, depth of understanding, quality of knowledge and
complexity increases as the level of analysis increases. So how far should we go with this stuff? “After all, I
just want to lower my costs, please the customer some more . . . you know, so I can stay in business and all
that stuff.” Why is there such a strong emphasis on data? “I mean, I’ve been successful so far and we don’t
take a lot of measurements!” Why should I start now?
5
 Measurement Issues

• Does the measurement system have adequate

discrimination?

• Is the measurement system statistically stable over time?

• Is the measurement error (variation) small?

6
 Knowledge to be obtained

• How big is the measurement error?

• What are the sources of measurement error?

• Is the gage stable over time?

• Is the gage capable for this study?

• How do we improve the measurement system?

7
 Measurement Analysis
An evaluation of the measurement system MUST be
undertaken to ensure effective analysis of any
subsequent data generated for a given process/product
characteristic
Observed value = True value + Measurement Error

Measurement error is a statistical term meaning the net effect of all sources of
measurement variability that cause an observed value to deviate from the
master

Total Variability = Process Variability + Measurement Variabil

Measurement Variability << Process Variability

8
 Measurement System Analysis
Applications

• Criteria to accept new measuring equipment

• Compare one measuring device against another
• Evaluate gage suspected of being bad
• Comparison of measuring device before and
after repair
• Used to determine true process variation
• Used to determine goodness of gage

9
 Possible Sources of Process
Variation

Observed Process Variation

Actual Process Variation Measurement Variation

Long-term Short-term Variation Variation due Variation due

Process Variation Process Variation w/i sample to gage to operators

Repeatability Calibration Stability Linearity

To address actual process variability, the variation due to the

measurement system must first be identified and separated
from that of the process
10
Possible Sources of Process Variation

Observed Process Variation

Actual Process Variation Measurement Variation

Long-term Short-term Variation Variation due Variation due

Process Variation Process Variation w/i sample to gauge to operators

Repeatability Calibration Stability Linearity

We will look at ―repeatability‖ and ―reproducibility‖ as

these are the primary contributors to measurement error
11
 Measurement System Analysis-
Measurement Process
 Measurement tools:
 Hardware
 Software
 All procedures for using the tools
 Which operators
 Set-up and handling procedures
 Off-line calculations and data entry
 Calibration frequency and technique

12
Sources of Measurement Variation
Why do Measurement Vary?

Mea Tool Work Methods

Mechanical instability Ease of Data Entry
Fixturing Operator Training
Wear Calibration Frequency
Maintenance Standard
Electrical Instability Sufficient WorkTime
Standard Procedures
Algorithm Instabilty
Operator Technique
MEASUREMENT
noise Humidity VARIATION
Cleanliness

Vibration

Line Voltage Variation

Temperature Fluctuation

Environment Methods
Environment
13
 Sources of Variation

Product Variability
(Actual variability)

Measurement
Variability

Total Variability
(Observed variability)

14
Effects of Measurement Error

Measurement
System Bias -
Averages Determined through
“Calibration Study”

 total   product   Measurement  Error

Measurement System

Variability Variability - Determined

through “R&R Study”

 total
2
  product
2
  measurement
2

15
 Terminology
 Accuracy related terms:
 True value
 Bias
 Linearity
 Precision-related terms
 Repeatability
 Reproducibility
 Linearity
 Stability (over time)

16
 Accuracy
• Accuracy - Does the average of the
measurements deviate from the true value?
• True value:
– Theoretically correct value
– NIST standards
• Bias
– Distance between average value of all
measurements and true value
– Amount gage is consistently off target
– Systematic error or offset

17
 BIAS Definition

BIAS: Is the difference between the observed Reference

average of the measurement and the reference Value
value. The reference-value is the value that serves
as an agreed-upon reference. The reference value
can be determined by averaging several
measurements with a higher level (e.g., metrology
lab) of measuring equipment.

Observed
Average
Accuracy is linked to Bias
Value
18
 Bias Example:
What is the Bias of the following situation
One part measured 10
X1=0.75 mm X6=0.8 mm
times by one Appraiser X2=0.75 mm X7=0.75 mm
X3=0.8 mm X8=0.75 mm
X4=0.8 mm X9=0.75 mm
X5=0.65 mm X10=0.7 mm

What else do you need to determine BIAS?

The reference Value determined by the layout inspection
equipment (ensure this equipment went through a Gage R&R) is
0.8mm. The process variation for the part is .70mm.


Bias = 0.75-0.8= -0.05
X
X = 0.75
% Bias=100[0.05/0.70]=7.1%
10 This means 7.1% of the process variation is BIAS
19
 Precision
 Total variation in the measurement system

 Measure of natural variation of repeated

measurements

 Terms: Random Error, Spread, Test/Retest error

 Repeatability and Reproducibility

 2MS   2rpt   2rpd

20
 Repeatability Definition

Repeatability: the variation in

measurements obtained with one measurement
instrument when used several times by one
appraiser while measuring the identical
characteristic on the same part

REPEATABILITY

21
 Reproducibility Definition

Operator-B

Reproducibility: Is the variation

in the average of the measurements
made by different appraisers using
the same measuring instrument
when measuring the identical
characteristic on the same part Operator-C

Operator-A

Reproducibility 22
 Stability (drift) Definition
Stability: Is the total variation in Time-1
the measurement obtained with a
measurement system (test / gage) on
the same master parts when
measuring a single characteristic over
an extended time period

Time-2

Points to the frequency of Mean center Calibration Stability over time

23
 Resolution Definition
X Adequate Resolution
X There is enough resolution in
X X the measurement device so that
X X X the product can have many
X X X X different values.
X X X X X
5.1 5.2 5.3 5.4 5.5

Good if 5 or more distinct

values are observed

24
 The Nature of Process Variation

Precise, but not

1 2 3 4 5 Accurate

1 2 3 4 5

Accurate, but not Precise

Rule of . . . . . .Test equipment MUST be a

thumb: least 10 times more accurate &
precise than what’s being tested
25
 Measurement Resolution
 The capability of the measurement system to detect the
smallest tolerable changes- also know as Discrimination
 # of increments in the measurement system at full range
 Instruments talk bits of resolution(2 to the N-bits)
 Example: 1inch scale 1/16 resolution
» What if your small change is 1/64?
 The discrimination is unacceptable for analysis if it cannot
detect the changes in variation

Check to ensure acceptable resolution,

avoid misapplication of measurement
26
 Capability Index (1)
 Precision to Tolerance Ratio

6 *  MS
P /T 
Usually expressed
in percent
Tolerance

 Addresses what percent of the tolerance is

taken up by measurement error.
 Best case: 10% Acceptable: 30%
 Includes both repeatability and
reproducibility
 Operator x Unit x Trial experiment

27
 Capability Index (2)
 %R&R

 MS
% R& R 
Usually expressed
in percent
 tot

 Addresses what percent of the total variation

is caused by measurement error.
 Best case: 10% Acceptable: 30%
 Includes both repeatability and
reproducibility
 Operator x Unit x Trial experiment

28
 Conducting the Gage R&R

 Parts should be randomly selected across a

FULL RANGE OF THE PROCESS - it is
important that the parts, in as much as is
possible, be representative of the total process
(variation)
 We are dealing with 10 parts (typical)
 Generally two to three operators
 Each unit is measured 2-3 times by each
operator
 10 parts is NOT a significant sample size for
sound judgment on a gage UNLESS

29
 Types of R&R studies

• Variable Gage R&R

– Numbers
– Units of measure
• Attribute Gage R&R
– Subjective (cosmetic
defects)
– Scatter of defects
– feel/visual

30
Procedure for performing an R&R study
1: Calibrate the gage, or assure that it has been calibrated
2: Have the first operator measure all the samples once in random
order
3: Have the second operator measure all the samples once in
random order
4: Continue until all operators have measured the samples once
(this is Trial 1)
5: Repeat steps 2-4 for the required number of trials
6: Use the form provided to determine the statistics of the R&R
study
Repeatability
Reproducibility
%R&R
Standard deviations of each of the above
% Tolerance analysis
7: Analyze results and determine follow-up action, if any
31
 Other Issues
 Number of operators
 If process uses multiple operators, choose 2-4 at random
 If process uses only one operator, or no operators, perform
study without operator effects (reproducibility effects ignored)
 Number of samples
 Select enough samples so that
(number of samples) X (number of operators) > 15
 If not practical or possible, choose number of trials so that:
 if S x O < 4, trials = 6

 if S x O < 5, trials = 5

 if S x O < 8, trials = 4

 if S x O < 15, trials = 3

32
 Gage R&R study report

 Objective
 Gage Description
 Gage study procedures
 Results:
 Calibration
 Accuracy
 Precision
 Conclusions
 Recommendations for improvements

33
 Variable Gage R&R

Guidelines

%R&R or P/T Results

<5% No issues
<=10% Gage is OK
10%-30% Maybe acceptable based upon importance
of application, and cost factor
Over 30% Gage system needs improvement /
corrective action

34
 Real Life Example
• Remember the Bolt Slot Cap from the Assignment?
• A Gage R&R study is peformed by 3 operators on 10 parts and
resultes are recorded in Bolt_slot_cap_gagerr .mtw

After the first attempt,

the operators tried to
impove their
performance and
remeasured the parts as
shown in column
ölçümler1. Were they
succesful?
35
Gage Run Chart

36
Gage R&R study

37
Gage R&R study - improved

38
 How a Gage R&R Study Works
– Select units or items for measuring that represent the full range of
variation typically seen in the process
• Measurement systems are often more accurate in some parts of the
range than in others, so you need to test them over the full range
– Have each operator measure those items repeatedly
• In order to use Minitab to analyze the results, each operator must
measure each unit the same number of times
• It is extremely desirable to randomize the order of the units and not let
the operator know which unit is being measured
– Minitab looks at the total variation in the items or units measured

39
 How a Gage R&R Study Works, cont.
– Minitab then estimates the proportion of the total variation that is due to
1. Part-to-part variation: physical or actual differences in the units being
measured
2. Repeatability: Inconsistency in how a given person takes the measurement
(lots of inconsistency = high variation = low repeatability)
3. Reproducibility: Inconsistency in how different people take the measurement
(lots of inconsistency = high variation = low reproducibility)
4. Operator-part interaction: An interaction that causes people to measure
different items in different ways (e.g., people of a particular height may have
trouble measuring certain parts because of lighting, perspective, etc.)
– If there is excessive variation in repeatability or reproducibility (relative to part-to-
part variation), you must take action to fix or improve the measurement process
• The goal is to develop a measurement system that is adequate
for your needs

40
 Adequate vs. Inadequate Measurement
Systems
Adequate Inadequate
• Most of the variation is accounted for • Variation in how the measurements are
by physical or actual differences taken is high
between the units
– You can’t tell if differences between units
– What Minitab calls part-to-part are due to the way they were measured,
variation will be relatively large or are true differences

– All other sources of variation will – You can’t trust your data and therefore
be small shouldn’t react to perceived patterns,
special causes, etc.—they may be false
– You can have higher confidence that signals
actions you take in response to data are
based on reality

• The measurement system has sufficient • The measurements fall into less than four
precision to distinguish at least four categories
groups or ―categories‖ of measurements

41
 Note on Calibrating Measurement
Equipment
• Measurement instruments should only be recalibrated when
they show special cause evidence of drift. Otherwise,
variation could be increased by as much as 40%. This is
because adjusting for true common cause variation adds
more variation (Deming’s rule 2 of the funnel).

Measurements taken Measurements taken with stable

from stable instrument instrument recalibrated before
each reading

42
Checking a Measurement System
for Discrete Data

43
 The Inspection Exercise
Task: Count the number of times the 6th letter of the alphabet appears in the
following text.
The Necessity of Training Farm Hands for First

Class Farms in the Fatherly Handling of Farm Live

Stock is Foremost in the Eyes of Farm Owners.

Since the Forefathers of the Farm Owners Trained

the Farm Hands for First Class Farms in the

Fatherly Handling of Farm Live Stock, the Farm

Owners Feel they should carry on with the Family

Tradition of Training Farm Hands of First Class

Farmers in the Fatherly Handling of Farm Live

Stock Because they Believe it is the Basis of Good

Fundamental Farm Management.

44
 Gage R & R Attribute
• Assessing the accuracy, repeatability, and reproducibility
• of a discrete measurement system

– For use in situations where all that is available is defect data. This can be in
a manufacturing facility or in a business environment to evaluate defects

– Most often this data is collected by people that inspect for quality, and for
this reason it is unreliable.

– Minitab (version 13.31 and later) has the capability to analyze the
reliability of the human inspection process.

– As with the continuous gage study, it will evaluate the repeatability and
reproducibility of inspectors. It will also evaluate how effective the
inspector is as compared to a known standard or attribute.

45
 Gage R & R Attribute, Cont
• As with the Continuous study, the Attribute study is a set of
trials where:

– Multiple Operators measure multiple units a multiple number of times

– Example: 3 operators each measure 50 units twice (note – the

increased sample size)

– “Blindness” is desirable. Ensure that there is sufficient time for the

inspectors to forget the previous results. It may be necessary to run
the tests on separate days/weeks to accomplish this.

– Randomization will help to reduce any inspector bias

46
 Planning and Executing the study
• Identify the process to be studied and which attributes to include in the study

• Determine the sample size. Because this is attribute date, a larger sample will
be necessary. A minimum would be 30 samples, but more is better. The larger
the sample size the smaller the confidence intervals in the study. A
determination will need to be made to balance sample size and the
confidence intervals

• Collect a set of samples with an equal number of good and bad products or
any attribute we want to verify, such as call type or error type. Collect samples
that are not obvious as to the correct answer. The study will determine how
well the inspector can make the “tough” decision

• Have subject matter experts or Customers help in the determination of which

of the samples are good and which are bad. (This becomes the attribute)

• Perform at least two trials (Minitab will accommodate more trials, but two is
standard) having the inspectors “blindly” inspect the samples

47
 Known Population Y/N
Sample # Attribute Try 1 Try 2 Try 1 Try 2 Try 1 Try 2
1 A A A A A A A Y
2 D D D D D D D Y

Attribute 3
4
5
A
A
A
A
A
A
D
A
D
A
A
A
A
A
A
A
A
A
A
A
A
N
Y
N

GR&R 6
7
8
A
A
D
A
A
D
A
A
D
A
A
D
A
A
D
A
A
D
A
A
D
Y
Y
Y
9 D D D D D D D Y
10 A A A A A A A Y
11 A A A A A A A Y
12 D D D D D D D Y
13 A D D A A A A N
14 A A A A A A A Y
15 A A A A A A A Y
16 A A A A A A A Y
17 D D D D D D D Y
18 A A A A A A A Y
19 A A A A A A A Y
20 A A A A A A A Y
21 D D D D D D D Y
22 D D D D D D D Y
23 D D D D D D D Y
24 A A A A A A A Y
25 A D D D D D D N
26 D D D D D D D Y
27 A A A A A A A Y
28 A A A A A A A Y
Total Score = 80% 29 A D D A A A D N
30 A D A A A A A N
Appraiser Score: 24/30= 80% 29/30= 97% 28/30= 93% 80%

48
 How an Attribute R & R works
• Minitab estimates the following:

– Within Appraiser: the ability of the appraiser to repeat their assessment

across trials

– Each Appraiser vs. Standard – agreement : Identifies how well the

appraiser agrees with the know standard

– Each Appraiser vs. Standard – disagreement: Shows where the appraiser

disagrees – identifies good as bad, (causing rework), identifies bad as good
(passes defects to the customer)

– Between Appraisers: the ability of all appraisers to agree with each other

– All Appraisers to the Standard: This is the overall capability of the gage

49
 Data for the Study
• Data must be balanced for Minitab – each operator must measure
each unit the same number of times

• Operators should randomly and “blindly” test the units; they should
not know which unit they are measuring when they record the data

• Minitab requires that the spreadsheet be formatted a specific way.

Whether putting the data in one column (stacked) or in separate
columns (unstacked) requires each operator have the
measurements for each trial be next to each other in the columns
(unstacked) or after each other in one column (stacked). See
examples on next page.

• If listing the attribute, it will be necessary to format the

spreadsheet in separate columns (unstacked)

50
Minitab Output

51
 Action Guidelines
• There are no absolutes for an Attribute R & R. The following are
guidelines as to what actions to take:

– > 90% adequate gage for most applications – no action required unless
team determines it is necessary

– 70 – 90% gage is questionable – determine if improvement is feasible and

necessary

– < 70% gage is not adequate – gage improvement required

• The above are only guidelines. In some situations, teams have been
able to move forward with projects when the gage has been less
than 70%. It is still recommended that efforts be made to improve
the gage for future uses.

52
 M & M Exercise for
Attribute Gage R & R
– Conduct an attribute gage R & R study with the
materials ( M & M ‘s ) provided by the instructor.
– Determine who will be the subject matter experts –
Subject matter experts will pick 30 M & M’s for the
exercise – half good & half bad
– Determine the three inspectors
– Use the criteria below for a good M & M
• Has a “shiny” surface
• M is correct and complete
– Conduct the experiment having each inspector
perform the inspection two times
– Collect the data and evaluate the results.

• DON’T EAT THE DATA

53
 Summary
• Attribute Gage R & R can be used to evaluate
a measurement system with Discrete data

• It has a large application in Transactional

Business processes where there exists large
amounts of Discrete/Attribute date

• It requires larger sample sizes than a

Continuous Gage R & R study
54