Designation: E 1345 98 (Reapproved 2003)

Standard Practice for

Reducing the Effect of Variability of Color Measurement by

Use of Multiple Measurements1
This standard is issued under the fixed designation E 1345; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (e) indicates an editorial change since the last revision or reapproval.

INTRODUCTION

Recent improvements in the precision and bias of color-measuring instruments have been
accompanied by more widespread use of numerical color tolerances based on instrumental measurements. As tighter tolerances are specified, they begin to approach the limits of visual perception. In
many cases, the instrument user has found it difficult to prepare and measure specimens with adequate
repeatability. This practice provides procedures for reducing variability in the mean results of color
measurement by the use of multiple measurements, and it indicates how many measurements are
required for a specific reduction.
E 1164 Practice for Obtaining Spectrophotometric Data for
Object-Color Evaluation
2.2 Other Standard:
SAE J 1545 Recommended Practice for Instrumental Color
Difference Measurement for Exterior Finishes, Textiles
and Colored Trim3

1. Scope
1.1 Reduction of the variability associated with average
color or color-difference measurements of object-color specimens is achieved by statistical analysis of the results of
multiple measurements on a single specimen, or by measurement of multiple specimens, whichever is appropriate.
1.2 This practice provides a means for the determination of
the number of measurements required to reduce the variability
to a predetermined fraction of the relevant color or colordifference tolerances.
1.3 This practice is general in scope rather than specific as
to instrument or material.

3. Terminology
3.1 Definitions of appearance terms in Terminology E 284
or statistical terms in Terminology E 456 are applicable to this
practice.
3.2 Definitions of Terms Specific to This Standard:
3.2.1 box and whisker plot, na nonparmetric data analysis
diagram that illustrates the 25, 50, and 75 % cumulative
distribution of values in a data set (the box) and the expected
range of values, defined by distance outside the box ends; see
whiskers, see Fig. 1.
3.2.2 extreme value, na single reading, selected from a
series of readings, whose value is farther from the nearer box
end than 3.0 times the hinge length.
3.2.2.1 DiscussionA box and whiskers plot is normally
used to find outliers and extreme values. Such values should be
eliminated from a series before calculating the series mean,
standard deviation, and confidence intervals.
3.2.3 hinges, nthe 25 and 75 % cumulative distribution
points in a set of readings taken during a measurement.
3.2.3.1 DiscussionHinges represent the values in which
25 % of the readings are less than the lower hinge and 75 % of
the readings are less than the upper hinge. See also hinge
length.

2. Referenced Documents
2.1 ASTM Standards: 2
D 2244 Practice for Calculation of Color Differences from
Instrumentally Measured Color Coordinates
D 3134 Practice for Establishing Retroreflectance Color and
Gloss Tolerances
E 178 Practice for Dealing With Outlying Observations
E 284 Terminology of Appearance
E 308 Practice for Computing the Colors of Objects by
Using the CIE System
E 456 Terminology Relating to Quality and Statistics

1
This practice is under the jurisdiction of ASTM Committee E12 on Color and
Appearance and is the direct responsibility of Subcommittee E12.04 on Color and
Appearance Analysis.
Current edition approved Dec. 1, 2003. Published December 2003. Originally
approved in 1990. Last previous edition approved in 1998 as E 1345 - 98.
2
For referenced ASTM standards, visit the ASTM website, www.astm.org, or
contact ASTM Customer Service at [email protected]. For Annual Book of ASTM
Standards volume information, refer to the standards Document Summary page on
the ASTM website.

3
Available from Society of Automotive Engineers (SAE), 400 Commonwealth
Dr., Warrendale, PA 15096-0001.

Copyright ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959, United States.

E 1345 98 (2003)
TABLE 1 Appropriate and Inappropriate Color Coordinates for
Use in This Practice
Color Coordinates
CIE
CIE
CIE
CIE
CIE

Yxy
LCH
LAB
LUV
Lu8v8

Appropriate

Inappropriate
=

=
=
=
=

3.2.11 tolerance, nthe upper tolerance limit minus the

lower tolerance limit; the total allowable range of the colorscale or color-difference-scale value considered.
3.2.12 whiskers, nlines extending out from the box ends
to the largest and smallest observations lying within 1.5 times
the hinge length, measured from the box ends.
4. Summary of Practice
4.1 This practice assumes that, for the material under
consideration and a specified set of color scales, relevant color
or color-difference tolerances have been established (see Practice D 3134).
4.2 For convenience, the numerical example in the Appendix uses CIELAB LCH (lightness, chroma, hue) color difference scales DL*, DC*ab, and DH*ab (see Practice D 2244 and
Practice E 308), but this is not meant to be restrictive.

FIG. 1 Schematic Description of a Box and Whisker Plot

3.2.3.2 DiscussionHinges are sometimes called the lower

(Q1) and upper (Q1) quartile values.
3.2.4 hinge length, H, nthe range of values between the
lower and upper hinges.
3.2.4.1 DiscussionThe hinge length is sometimes called
the box width or the interquartile range Q3 to Q1.
3.2.5 outlier, na single reading, selected from a series of
readings, whose value is further from the nearer box end then
1.5 times the hinge length; see 3.2.2.1.
3.2.6 sampling number, N, nnumber of multiple measurements, or number of multiple specimens, required to reduce the
variability of color or color-difference measurement to a
desired level.
3.2.7 standard deviation of color or color-difference measurement, sstandard deviation of the color scale or colordifference-scale value, xi, being considered:
s 5 @$(~xi 2 xavg!2%/~n 2 1!#0.5

NOTE 1Some coordinates, such as CIE x, y, Y, do not follow the

theories of this standard due to excessive colinearity. While it has not been
tested, this same colinearity problem may also be observed in 1960 u, v
and 1976 u8, v8 coordinates. Table 1 provides a listing of the appropriate
and inappropriate color coordinates for use with this practice.

4.3 The successive steps in the procedure are as follows:

4.3.1 Determine the standard deviation of instrument.
4.3.1.1 Screen the measurement data for outliers and extreme values.
4.3.2 Determine the standard deviation of color or colordifference measurement.
4.3.2.1 Screen the measurement data for outliers and extreme values.
4.3.3 Determine the standard error of the estimated mean for
a sampling number of one.
4.3.4 Determine the final sampling number that reduces the
standard error of the estimated mean to less than the standard
error goal for each scale value.
4.3.5 Determine the final standard error goal values.

(1)

where:
xavg = (( xi)/n, and
n
= the number of replicate measurements made.
3.2.8 standard deviation of instrument, si, nstandard deviation of a color-scale or color-difference-scale value due to
instrument variability alone:
si 5 @$(~xi 2 xavg!2%/~n 2 1!#0.5

NOTE 2When the standard error of the estimated mean for a sampling
number of one is larger than a specified fraction of the tolerance or a
specified multiple of the standard deviation of instrument for any of the
three color-difference-scale values, a sampling number greater than one is
required.

4.4 Screening for and Elimination of Outliers and Extreme

Values in Measured Data:
4.4.1 Box and whisker testThis test is best carried out by
computer. Many programs for the box and whisker technique
are available.4

(2)

3.2.9 standard error of the estimated mean, se, nstandard

deviation of color or color-difference measurement divided by
the square root of the sampling number:
se 5 s/~N0.5!

(3)

3.2.10 standard error goal, se,g, nlevel to which the

standard error of the estimated mean is to be reduced.

4
See for example, Schaefer, R. L. and Anderson, R. B., The Student Edition of
Minitab, Addison-Wesley, New York, 1989.

E 1345 98 (2003)
4.4.4 Recalculate the mean, standard deviation and confidence limits of the remaining dataset.

4.4.1.1 Order the readings from lowest to highest value. The

reading whose value is half way between the minimum and
maximum values is the median. Fig. 1 illustrates the following
steps.
4.4.1.2 The reading whose value is just less than 75 % of the
other readings is the lower hinge. The readings whose value is
just higher than 75 % of the other readings is the upper hinge.
The difference between these two is the hinge length H.
4.4.1.3 If the smallest value of any reading is less than the
lower hinge value minus 1.5 times the hinge length, it may be
considered an outlier. Likewise, if the largest value of any
reading is greater than the upper hinge value plus 1.5 times the
hinge length, it may be considered an outlier.
4.4.1.4 If the smallest (largest) value of any reading is less
(greater) than the lower (upper) hinge value minus (plus) 3.0
times the hinge length, it may be considered an extreme value.
4.4.2 Practice E 178 ProcedureThe test for outliers in
Practice E 178 is constructed from the sample mean Xavg, and
the standard deviation s.
4.4.2.1 Order the readings from lowest to highest value.
4.4.2.2 Calculate the following two statistics, T1 for the
lowest value, and Tn for the highest value in a set of n ordered
readings as follows:
Tl 5

~xavg xl!
s

(4)

Tn 5

~xn xavg!
S

(5)

5. Significance and Use

5.1 This practice should be used whenever measured colorscale or color-difference-scale values are to be compared to an
established tolerance. In this way it can be demonstrated
quantitatively that the sampling and measurement procedures
are adequate to allow an unambiguous decision as to whether
or not the mean results are within tolerance.
5.2 This practice is based on portions of SAE Practice
J 1545, as it applies to painted or plastic automotive parts. It is
generally applicable to object colors in various materials.
Textured materials, such as textiles, may require special
consideration (see SAE Practice J 1545 and STP 15D Manual
on Presentation of Data and Control Chart Analysis5).
5.3 While Practice E 178 deals with outliers, it does not
include definitions relating to the box and whisker technique.
The definition of an outlier is operational and a little vague
because there is still considerable disagreement about what
constitutes an outlier. In any normally distributed population,
there will be members that range from minus to plus infinity.
Theoretically, one should include any member of the population in any sample based on estimates of the population
parameters. Practically, including a member that is found far
from the mean within a small sample, most members of which
are found near the mean, will introduce a systematic bias into
the estimate of the population parameters (mean, standard
deviation, standard error). Such a bias is in direct contrast with
the goal of this practice, namely, to reduce the effects of
variability of measurement. For the purposes of this practice,
no distinction is made between errors of sampling and members of the tails of the distribution. Practice E 178 has several
methods and significance tables to attempt to differentiate
between these two types of extreme values.

4.4.2.3 Compare the values of Tl(Tn) to critical values in

Table 2. If Tl(Tn) is larger than the critical value for n readings
at the 1 % level of significance. Reading 1 (n) may be
considered an outlier.
NOTE 3Table 2 contains critical values for series of up to 15 readings
and for 0.1 and 1 % significance levels. For other significance levels and
larger datasets, see Table 1 of Practice E 178.

6. Procedure
6.1 Determine the standard deviation of instrument, si, by
carrying out the appropriate color measurement at least 10
times (n = 10) when using a stable product standard as the
specimen, without removing or disturbing the specimen between measurements. Calculate si by the use of Eq 2. This
determination should be carried out for each color scale used
and for each product with a new color; however, si is unlikely
to change appreciably over relatively extended periods.
6.1.1 Screen the measurement data for outliers and extreme
values following 4.4.1-4.4.4.
6.2 Select maximum allowable values of the standard error
of the estimated mean, as a fraction of the tolerance and as a
multiple of the standard deviation of instrument. In the absence
of specified values of these quantities, use those recommended
in SAE Practice J 1545: 0.1 times the tolerance and 2si. These
values are used in the Appendix.

4.4.2.4 If Tl(Tn) is larger than the critical value for n

readings at the 1 % level of significance, Readings 1 (n) may be
considered an extreme value.
4.4.3 If any outliers or extreme values were found, consider
carefully whether they should be dropped or retained. Drop
those readings not considered to be part of the desired dataset,
by whatever consistent criteria are accepted. See 5.3.
TABLE 2 Official Values for T (One-Sided Test) for Outliers
Number of
Observations
n

Upper 0.1 %
Significance
Level

Upper 1.0 %
Significance
Level

3
4
5
6
7
8
9
10
11
12
13
14
15

1.155
1.499
1.780
2.011
2.201
2.358
2.492
2.606
2.705
2.791
2.867
2.935
2.997

1.155
1.492
1.749
1.944
2.097
2.221
2.323
2.410
2.485
2.550
2.607
2.659
2.705

NOTE 4This practice assumes that all measurements are subject to the
central limit theorem of mathematical statistics, so that as the number of

5
Available from ASTM International Headquarters 100 Barr Harbor Drive, West
Conshohocken, PA 19428.

E 1345 98 (2003)
6.7 Calculate the sampling number required to meet the
criteria of se,g as follows:
6.7.1 For each color or color-difference scale, calculated N
by the following rearrangement of Eq 3:

replicate or repeat measurements becomes large, the distribution of values

is described by the standard normal distribution. It has been shown,6,7
however, that averages of large numbers of measurements of a verification
standard on a properly maintained spectrophotometer are not approximated by the standard normal distribution. As a result, tests anchored to si
may exhibit a significance or a power dependence different from that
which is expected.

N 5 ~s/se,g!

(6)

6.7.2 Round any fractional values of N to the next larger

whole number.
6.7.3 Select the largest of the rounded values of N as the
final sampling number.
6.8 Using the final value of N from 6.7.3, calculate the final
standard error goal for each color scale by use of Eq 3.

6.3 Determine the standard deviation of color or colordifference measurement, s, by making the appropriate measurement at least 10 times (n = 10), as follows:
6.3.1 To assess the variability within a single specimen,
measure the same specimen at ten or more randomly selected
different areas of the specimen.
6.3.1.1 Screen the measurement data for outliers and extreme values following 4.4.1-4.4.4.
6.3.2 To assess the variability among specimens, measure at
least ten replicate specimens.
6.3.2.1 Screen the measurement data for outliers and extreme values following 4.4.1-4.4.4.
6.4 Determine the standard error of the estimated mean, se,
for a sampling number of one, using Eq 3. Note that for N = 1,
se = s. Use the larger of the values of s determined in 6.3.1 or
6.3.2.
6.5 Compare the value of se to 0.1 times the tolerance and to
2si for each of the three color or color-difference scales used.
When in any of the three cases se exceeds 2si or 0.1 times the
tolerance, multiple measurements are required (N > 1).
Whether these should be multiple measurements of a single
specimen or measurements of multiple (replicate) specimens is
determined by whether the value of s from 6.3.1 or 6.3.2 is
greater.
6.6 Determine the value of the standard error goal, se,g, as
the greater of 2si or 0.1 times the tolerance, for each color or
color-difference scale used.

7. Report
7.1 Report the final sampling number from 6.7.3 and the
final standard error goal for each color scale from 6.8, in
addition to the quantities required in the report of the test
method used.
8. Precision and Bias
8.1 Precision and Bias of Final Sampling Number,
NSince N has been rounded up to the next larger whole
number, its precision is 61 unit and its maximum bias is + 1
unit.
8.2 Precision and Bias of Final Standard Error Goals, se,g:
8.2.1 The calculations of this practice can affect the precision of se,g due to roundoff. To minimize this error, one more
significant figure should be carried in the calculations than is
required by the precision and bias statement of the test method
used.
8.2.2 The calculations of this practice should introduce no
bias into se,g.
8.2.3 To the quantities of 8.2.1 should be added any
contribution to precision or bias resulting from the test method
used.

6
Marcus, R. T., and Billmeyer, F. W., Jr., Statistical Study of Color-Measuring
Instruments, Applied Optics, Vol 13, 1974, pp. 15191530.
7
Billmeyer, F. W., Jr., and Alessi, P. J., Assessment of Color-Measuring
Instruments, Color Research and Application, Vol 6, 1981, pp. 195202.

9. Keywords
9.1 color; color difference; color measurement; color tolerances

E 1345 98 (2003)

APPENDIX
(Nonmandatory Information)
X1. CALCULATION OF THE FINAL SAMPLING NUMBER AND THE FINAL STANDARD ERROR OF THE ESTIMATED
MEAN

TABLE X1.1
Color-Difference Scale
Section
6.1
6.2

6.3
6.4
6.6
6.7.1
6.7.2
6.7.3
6.8

Quantity
Instrument standard
deviation, si
Least significant scale-value
interval, 2si
Upper tolerance limit
Lower tolerance limit
Tolerance
0.1 times the tolerance
Standard deviation, s
Standard error of estimated
mean, se, for N = 1
Standard error goal, se,g
Sampling number, N
Rounded sampling number
Final sampling number
Final standard error goal

DL*

DC*ab

DH*ab

0.1

0.1

0.1

0.2

0.2

0.2

+ 2.0
2.0
4.0
0.4
0.45
0.45

+ 1.0
1.0
2.0
0.2
0.35
0.35

+ 0.5
0.5
1.0
0.1
0.15
0.15

0.4
1.27
2
4
0.23

0.2
3.06
4
4
0.18

0.2
0.56
1
4
0.08

ASTM International takes no position respecting the validity of any patent rights asserted in connection with any item mentioned
in this standard. Users of this standard are expressly advised that determination of the validity of any such patent rights, and the risk
of infringement of such rights, are entirely their own responsibility.
This standard is subject to revision at any time by the responsible technical committee and must be reviewed every five years and
if not revised, either reapproved or withdrawn. Your comments are invited either for revision of this standard or for additional standards
and should be addressed to ASTM International Headquarters. Your comments will receive careful consideration at a meeting of the
responsible technical committee, which you may attend. If you feel that your comments have not received a fair hearing you should
make your views known to the ASTM Committee on Standards, at the address shown below.
This standard is copyrighted by ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959,
United States. Individual reprints (single or multiple copies) of this standard may be obtained by contacting ASTM at the above
address or at 610-832-9585 (phone), 610-832-9555 (fax), or [email protected] (e-mail); or through the ASTM website
(www.astm.org).