KAN Pd-02.05 (Jan 2019) Guide On Measurement Assurance
KAN Pd-02.05 (Jan 2019) Guide On Measurement Assurance
KAN Pd-02.05 (Jan 2019) Guide On Measurement Assurance
05 Revisi: 0
Table of Content
1. INTRODUCTION ............................................................................................................ 1
2. MEASUREMENT ASSURANCE CONCEPT ................................................................... 1
3. THE USE OF CHECK STANDARD................................................................................. 4
4. DEFINING AND CREATING CONTROL CHART ............................................................ 5
5. USING CONTROL CHART ............................................................................................. 9
6. SOURCE OF MEASUREMENT DATA FOR MEASUREMENT ASSURANCE .............. 19
7. REFERENCES ............................................................................................................. 20
1. INTRODUCTON
Chapter 7.7 of ISO/IEC 17025:2017 “General requirements for the competence of
testing and calibration laboratories” on “Ensuring the validity of results” requires:
7.7.1 The laboratory shall have a procedure for monitoring the validity of results. The
resulting data shall be recorded in such a way that trends are detectable and, where
practicable, statistical techniques shall be applied to review the results. This
monitoring shall be planned and reviewed and shall include, where appropriate, but
not be limited to:
a) use of reference materials or quality control materials;
b) use of alternative instrumentation that has been calibrated to provide
traceable results;
c) functional check(s) of measuring and testing equipment;
d) use of check or working standards with control charts, where applicable;
e) intermediate checks on measuring equipment;
f) replicate tests or calibrations using the same or different methods;
g) retesting or recalibration of retained items;
h) correlation of results for different characteristics of an item;
i) review of reported results;
j) intralaboratory comparisons;
k) testing of blind sample(s).
7.7.2 The laboratory shall monitor its performance by comparison with results of other
laboratories, where available and appropriate. This monitoring shall be planned and
reviewed and shall include, but not be limited to, either or both of the following:
a) participation in proficiency testing;
This document gives the recommended method to implement the above requirement
that is applicable for calibration and testing laboratories that willing to be accredited
by National Accreditation Body of Indonesia or Komite Akreditasi Nasional (KAN)
based on ISO/IEC 17025:2017
The assumptions that relate to measurement process apply statistical control; namely
that the errors of measurement are uncorrelated over time and come from a
population with a single distribution. The tests for control depend on the assumption
that the underlying distribution is normal (Gaussian), but the test procedures are
robust to slight departures from normality. Practically speaking, all that is required is
that the distribution of measurements be bell-shaped and symmetric.
Measurements on a check standard provide the mechanism for controlling the
measurement process. Measurement on a check standard should produce identical
results except for the effect of random errors, and tests for control are basically tests
of whether or not the random errors from the process continue to be drawn from the
same statistical distributions as the historical data on the check standard. Changes
that can be monitored and tested with the check standard database are:
changes in bias and long-term variability; and
changes in instrument precision or short-term variability
A change in the variability of the measurement on the check standard can be due to
one of many causes such as:
loss of environmental controls;
change in handling techniques
severe degradation in instrumentation
The artifacts must be of the same type and geometry as items that are measured in the
workload, such as:
items from the workload
a single check standard chosen for this purpose
a collection of artifacts set aside for this specific purpose
EXAMPLE:
PLAN:
The duplication of the above calibration process for a check standard can be
described as:
S : reference standard
Sc : check standard
Sc : check standard
S : reference standard
We can use one check standard to monitor this process because we can’t
incorporate check standard into the measurement and achieve the goals of the
procedure, we will duplicate the process by substituting a check standard into the
place of unit under test and repeating the process for a check standard.
EXAMPLE:
PLAN:
d. Select and calibrate check standard: For the measurement described before in
the example, we will choose a standard that will monitor bias, and determining
the variability of the process. We have an older set of stainless steel standards
that have demonstrated stability over time, and have an independent calibration
value from the other accredited calibration laboratory. In this case we use a 200
g stainless steel mass standard of F1 class from this set to be a check standard
for the calibration described in this example. The calibration value of this
standard is used as the reference value for this check standard.
Control chart is a graphical tool used to visualize data when monitoring, evaluating
and improving a measurement process:
monitoring and evaluating may include the need for controlling process
through corrective action and process improvement
the measurement process include the evaluation of many contributing factors
including standards, measurement, variability, uncertainties, environmental
condition and staff performance
The type of chart, which is commonly used for monitoring, evaluating and improving
the measurement process, using control shart allow us to:
ensure agreement with reference value, or to detect bias, offset, or change in
the set point of references;
ensure ongoing stability of process variability and reported uncertainty or
detec change in the variability of the process that effects the uncertainty
analysis;
ensure and document ongoing stability of the standards and process for
accuracy and traceability and to predict future measurement values;
make decision regarding corrective action or process improvement
The question is often asked in defining and creating control chart, “how many initial
measurements do we need?“, considering our knowledge about statistic and the
measurement process, we can answer the question as follows:
2 to 3 measurements, we need at least 2 measurement to calculate mean and
standard deviation, however, only this amount of data provides limited values
for our process
7 to 12 measurements, we need at least 7 to 12 data to obtain initial estimates
on sample and population to begin developing control chart. The first set of 7
to 12 measurements will be used to establish initial reference limits. 10 is good
number of data points to use as the baseline for our initial control
measurements. Be sure to gather data over a period of time, with not more
than 1 measurement from the same day
25 to 30 measurements, we need at least 25 to 30 data points on which to
have statistical decision about the data. We will need this many points to
establish whether the data are stable, randomly distributed and symmetric
about the mean. We should also have at least this much data for estimating
and calculating measurement uncertainty
50 to 100 measurements, once we have maintained data for a while, it is good
idea to keep at least 50 to 100 data points or at least 3 years data to allow
monitoring trends. We will need to determine and document how long data will
need in our laboratory and the basis for archiving our old data
There is specific information that should always be on a control chart and some
information should be always obtained with the data and maintained in a database.
The data collection forms and a spreadsheet template or specialty software program
Date of issue : 2 January 2019 5
KAN Pd-02.05 Revisi: 0
should be used to gather complete information consistently. There are some essential
information and good information to have in a control chart:
Essential information:
1. Titles: identify laboratory, standard operating procedure, standard(s) and/or
check standard(s), nominal value, and time of measurement
2. Data: measured or calculated values, number of data point(s), mean,
standard deviation
3. X-axis: identify observations by date or time
4. Y-axis: observations or calculated values, with measurement unit identified
5. Central line: mean, and reference value whenever available, though it may
not be at the center
6. Limit: identification of upper and lower warning and control (action) limits.
Good information:
1. Legend: when more tha
2. n one series present
3. Tolerances: when applicable
4. Uncertainties: for reference value, check standard and process output
5. Equipment information: device readability, configuration setting
6. Standard information: calibration date and interval information
7. Responsible staff: need on chart or in database
8. State of Control
9. Information about previous limits and history of the chart/data: if available
It is critical to understand the concept associated with each following step in creating
control chart no matter what tools are used to create the chart. Various tools may be
used, but the most common are spreadsheet or a specialty software product created
for this application. The following is the explanation to create a control chart, with an
example on the measurement assurance of the mass standard calibration using
Microsoft Excel Spreadsheet:
EXAMPLE:
DO:
This is the simplest approach to creating a chart. We will need to enter labels and
titles next. Be sure to label your values with the appropriate measurement units. To
create control chart in Microsoft Excel, you should use “LINE chart”.
The initial statistics in creating a control chart are central tendency and dispersion.
The mean and standard deviation are used to represent the central tendency and
dispersion. Spreadsheet allows us to calculate these statistics more easily than
Date of issue : 2 January 2019 7
KAN Pd-02.05 Revisi: 0
The accepted value may be the mean values as determined from the data, may be
calibration value determined by another laboratory, or may be a calibration value
determined in-house using a higher level procedure than the one being monitored.
If the reference value is the mean of corrections, we can only monitor variability in
the process and change of the standard, no monitoring of bias is possible.
If the reference value comes from a higher-level independent laboratory, bias from
reference value can be monitored as well as process variability. Having a
reference value with the fewest measurement possible in traceability chain (e.g.
from an NMI) will provide a measure of bias with the highest level of confidence. If
the reference value comes from a higher level measurement inhouse, bias within
the laboratory can be monitored also as well as process variability, but with lower
level of confidence that is possible when using an independence reference value
from an NMI or higher level accredited calibration laboratory.
Measurement assurance should be a real time monitoring process and not one in
which data are saved for entry into control charts just prior to laboratory audit or when
customer calls with measurement discrepancy. Data should be plotted and evaluated
immediately.
After plotting data, look for stability and randomness, be sure the latest point entered
is within the established limits, or we will need to take appropriate action. Plotting real
time data on the control charts can prevent the release of questionable data and
possible recall of certificate for items you have already tested or calibrated.
Normal distribution is probably the most important and most frequently used
distribution, both in the theory and the application of statistics. We expect our
measurement data to come from a normal distribution. We looked at this distribution
and considered confidence intervals and probabilities. In this chapter, think about the
probability that a point will come from this distribution and be within limits on the
control chart unless there is something wrong. We will consider what might be wrong
and what action steps may be needed to be taken to correct the problem.
When all data incorporated on the same control chart, we can see approximately
where the change occurred and predict future value, barring additional damage. The
unique pattern shown on the chart is the result of a change in the value of the check
standard. To conduct further analysis of these data, we could separate the before and
after data and conduct t-test and F-test, or we could compare the histograms of the
before and after data. The differences in the two distributions are quite obvious when
they are viewed graphically.
sold is the standard deviation used to establish existing limit in the control chart, and
snew is the standard deviation of the most recent data. Our measurement system is
considered to be “out of control” if we meet the following conditions
xold is the mean of the old data existing in the control chart; xnew is the mean of most
recent data; sold is the standard deviation used to establish existing limit in the control
chart; snew is the standard deviation of the most recent data; nold the number of data
points existing in the control chart; nnew is the number of data points of most recent
data. Our measurement system is considered to be “out of control” if we meet the
following conditions:
tcritical is the critical tvalue that depends on degrees of freedom calculated using the
following formula:
Our control charts helps to identifying stability or change in the measurement process
and in the standards we need to monitor the process and the standards to ensure the
quality of our measurements. In the following we identify the number of examples of
“things to look for” in the control charts that show instability or changes:
EXAMPLE:
PROCESS DEGRADATION
The most recent data have wider dispersion than the earlier data on which the limits
were based as shown in the following control chart.
PROCESS IMPROVEMENT
The most recent data have a narrower dispersion than the earlier data on which the
limits were based as shown in the following control chart.
In the above cases, the control chart shows that the variability of the most recent data
is out of the action limits and the result of F-test also shows “out of control” condition,
However, the variability is still acceptable compared the tolerance limits, then there is
no need to look for possible assignable causes if we used the tolerance limits as a
action limits for our measurement processes.
If our control chart shows that the variability of most recent data exceeds tolerance
limits as shown in the following control chart:
All of the most recent data are above the mean value of the earlier data as shown in
the following control chart:
The recent values seem to be steadily increasing above the previous mean values as
shown in the following control chart:
Remember that the stability is less important than the predictability. In fact, some
laboratories apply correction to their check standard based on a consistent history of
drift, they have a level of confidence for predicting drift and incorporate an uncertainty
associated with the predicted correction.
Possible assignable causes:
Our working standard worn from use
Our check standard getting dirty
Our standards are not stable enough for this process
Possible action steps:
Evaluate drift rate of standard, which may be used to evaluate calibration
intervals
Incorporate the level of offset at some future points in time
Include the uncertainty of the offset from reference values now and in the
future
In our example, there is difference between the mean value and the reference value
obtained from the independent calibration of check standard as shown in the following
control chart:
OUTLIERS
There are a few of data from a recent measurement fall outside the action limits as
shown in the following control chart:
If we can identify the cause of outliers and take appropriate action, we can make a
note of the action in our control database and flag the points so they are not included
in the statistical calculations. Do not delete these points from control database, if we
find problems that cause outliers every 10th data point, it means the corrective action
steps are not effective. If we arbitrarily delete observations, we will not be able to
track erratic readings.
7. REFERENCES
1. Harris G. L, “NIST Special Publication 1001: Basic Mass Metrology CD ROM”,
National Institutes of Standards and Technologies, Technology Administration, US
Department of Commerce, September 2003
2. NIST/SEMATECH, “Engineering Statistics Handbook”, electronic file web based
handbook at http://www.itl.nist.gov/div898/handbook/, National Institutes of Standards
and Technology, Technology Administration, US Department of Commerce,
November 2003