Pergamolv Atmospheric

Enuironmmf
Vol. 29, No. 11, pp. 1185-1193, 1995
Copyright 0 1995 Elwicr Scicncc Ltd
Printed in Great Britain. All rights rcsetved
1352-2310/95 $9.50 + 0.00

1352-2310(94)00210-X

THE STANDARD ERROR OF A WEIGHTED MEAN

CONCENTRATION-I. BOOTSTRAPPING VS OTHER
METHODS

DONALD F. GATZ
Ihinois State Water Survey, 2204 Griffith Drive, Champaign, IL 61820, U.S.A.

and

LUTHER SMITH
ManTech Environmental Technology, P.O. Box 12313,2 Triangle Drive, Research Triangle Park,
NC 27709, U.S.A.

(First received 27 October 1993 and in finalfinn 7 July 1994)

Abstract-Concentrations of chemical constituents of precipitation are frequently expressed in terms of the

precipitation-weighted mean, which has several desirable properties. Unfortunately, the weighted mean has
no analytical analog of the standard error of the arithmetic mean for use in characterizing its statistical
uncertainty. Several approximate expressions have been used previously in the literature, but there is no
consensus as to which is best. This paper compares three methods from the literature with a standard based
on bootstrapping. Comparative calculations were carried out for nine major ions measured at 222 sampling
sites in the National Atmospheric Deposition/National Trends Network (NADP/NTN). The ratio variance
approximation of Cochran (1977) gave results that were not statistically different from those of bootstrap
ping, and is suggested as the method of choice for routine computing of the standard error of the weighted
mean. The bootstrap method has advantages of its own, including the fact that it is nonparametric, but
requires additional effort and computation time.

Key word index: Precipitation chemistry, bootstrap methods, statistics, wet deposition, precipitation-
weighted mean.

INTRODUCTION When Hawley et al. (1988) discussed the standard

error of M,, they pointed out that the method for
Central tendencies of ion concentrations in precipit- estimating its value varies according to the purpose for
ation are often expressed in terms of the weighted which it is to be used. For example, in the complete
mean, M,, with precipitation amount, or the roughly sampling case, all of the possible samples for a specific
equivalent quantity, sample volume, as the weighting location and time interval were collected and ana-
factor. M, is useful because it limits the influence of lyzed. Method for estimating the standard error of the
the very high concentrations that can occur in very weighted mean, SEM,, for this case were dealt with in
light rains, and yields an overall concentration that is detail by Hawley et al. (1988).
proportional to ion mass deposition (also known as The representative sampling case, however, is more
loading). This proportionality makes it possible to often encountered in research, where observations
compute spatial deposition fields using the “indirect” from a finite set of samples are used to estimate some
method (Vong et al., 1989), in which spatial fields of feature (such as M,) of a broader population. This
ion concentration and precipitation amount are estim- paper describes methods of estimating SEM, that
ated separately and then multiplied together at grid pertain to the representative sampling case.
intersections to produce the deposition field. This Although M, is a very useful concept in pre-
“indirect” method has the advantage of allowing us to cipitation chemistry, it has one notable drawback:
estimate the precipitation field from all available there is no readily derivable, generally applicable,
precipitation measuring stations, which typically form analog of the standard error of the mean to express the
a much denser network than do the precipitation uncertainty of the precipitation-weighted mean. A
chemistry sampling stations. theoretical mathematical-statistical development of a
As with any measured or derived quantity, the formula for SEM, would require knowledge of the
statistical uncertainty of M, is an important issue. statistical distributions of ionic concentration and

1185
1186 D. F. GATZ and L. SMITH

precipitation amount, and possibly the relationship of 1988-1990 (NADP/NTN, 1992). A map of the sampling
these two quantities to each other. Thus, a theoretical site locations and descriptions of sampling and analytical
methods have been published by the National Atmospheric
formula that would be valid for the conditions at Deposition Program (1992). Separate values were computed
one location would not necessarily hold at another if covering the entire 1988-1990 data period for each of the
the distribution of precipitation were different. This three formulations from the literature, and compared to
would also be true for another place or another ion those computed by the bootstrap algorithm.
with a different distribution of concentration, or for a This period of record was chosen to insure reasonable
numbers of samples even at sites with limited precipitation; it
different relationship between precipitation and con- was the latest three-year period available when the work was
centration. begun. The maximum number of samples at any sampling
Several expressions for SEM, have been proposed site was 147, and 90% of the sites had 45 samples or more. All
and used in varying degrees in the atmospheric chem- available sites were included, without regard to data com-
pleteness criteria. Samples not collected under the standard
istry literature. Miller (1977) suggested an expression
wet-only protocol were excluded. Normal network sample
that was also used by Liljestrand and Morgan (1979) duration is one week, but samples were not excluded based
and by Top01 et al. (1985) to express the uncertainty of on duration. All calculations were performed using SYSTAT
the M,. Galloway et al. (1984) used a somewhat statistical software, version 5.03 (Wilkinson, 1990).
different equation for SEM,, and Endlich et al. (1988)
computed confidence intervals for M, based on the Methods Proposed in the Literature
formula for the ratio variance given by Cochran Miller (1977) chose the following formula to compute
(1977). Most of these proposed expressions for SEM, SEM,:
were unaccompanied by any discussion (or justifica-
tion) of the assumptions required in their derivation.
There is no general consensus among precipitation
where P, is the precipitation amount for sample i, X, the
chemistry researchers on which expression is prefer-
concentration in sample i, x, the precipitation-weighted
able, or the conditions under which one expression mean concentration over samples i= l-n, _%, the
might be. favored over another. Hawley (1985) com- c (Pi XJ/x P,, and n the number of samples.
pared three methods, including that of Miller (1977) The choice of Galloway et al. (1984), using the same
mentioned earlier, and the jackknife technique de- notation, was:
scribed by Efron (1982). He concluded that each of the “cP:-C-(cp,&)2
(SEM,)2 =n
three methods for computing the variance of M, had cc PJ2 n(n-1) I
shortcomings, especially for sample sets < 50.
No rationale, justification, or discussion of any assumptions
Fortunately, a very satisfactory new method ap- or limitations inherent in these formulations was provided in
pears to be available. Bootstrap methods (Efron and either paper.
Tibshirani, 1986, 1991) were developed as a means of Endlich et al. (1988) expressed SEM, as an approximate
using today’s readily available computational power ratio variance given by Cochran (1977). The formula that
follows is given in an SRI International (1986) report:
to obtain a numerical estimate of the standard error of
estimators other than the mean, such as, e.g. M,. (SEM,)2 = (n_1);CP,)2 CC (pi xi-pxw)2
Efron and Tibshirani (1991) pointed out that the I
bootstrap algorithm can be applied to almost any -2x,c (Pi-q (Pi x,-P BJ
statistical estimation problem. Further, they argued
+x,2 c (Pi-Fy].
that the same logic that makes the usual expression of
the standard error reasonable as an assessment of the
The Bootstrap Method
standard error of the arithmetic mean applies equally
Efron and Tibshirani (1991) recently described the boot-
well to the bootstrap method as an assessment of the strap method, and the following description borrows heavily
standard error of any general statistic. Applied to ion from theirs. The name “bootstrap” refers to the repeated
concentrations in precipitation, the bootstrap pro- random resampling, with replacement, of some original data
vides a statistically valid nonparametric estimate of the set x=(q, X2, . . . , x,,),to generate new data sets of size n. A
bootstrap sample is denoted as x* =(x7, x3, ... , x: ). Each
SEM,.
x: is one of the original x values, randomly selected.
The purpose of this paper is to illustrate how the The bootstrap estimate of SEM, is computed as follows:
bootstrap technique can be used to estimate SEM, for (1) a large number B of independent bootstrap samples, each
ion concentrations in precipitation, and to compare its of size n, is generated using a random number generator, (2)
results with SEM, values estimated from the three M, is computed for each bootstrap sample, and (3) the
empirical standard deviation of the B bootstrapped weighted
literature methods listed earlier. means is computed as the bootstrap estimate of SEM,. In
this work, B was 200. The value of n varied from site to site,
with a maximum of 147 for weekly sampling over three years.
At a few sites with limited precipitation, n was ~50.
METHODS Efron and Tibshirani (1991) also indicated that, while
ideally B would be infinite, randomness in the bootstrap
Values of SEM, were computed for the nine major ions standard error that comes from using a tinite B is usually
in precipitation at each of 222 (i.e. essentiallv all) sites in negligible for B>2OD, and that values of B as small as 25
the NaGonal Atmospheric Deposition Prog&m/‘National can often give satisfactory results. For the special case of
Trends Network (NADP/NTN) database for the three years the arithmetic mean, as B increases without bound, the
 Bootstrapping vs other methods 1187

bootstrap standard error estimate approaches the square The hypothesis that the difference between the
root of (n-1)/n times the usual formula for the standard bootstrap and each alternative method was zero was
error of the mean. If the same accuracy applied to all other
applications of the bootstrap method, the method would be tested for each ion. The paired t-test was used when
accurate to within 2% for n > 25. the differences (computed for each sampling site)
The bootstrap method typically assumes that data are between the SEM., calculated by the bootstrap and
independent and identically distributed (iid), and that has the alternative method from the literature appeared to
been assumed in this work. Presumably, the previously
come from reasonably normally distributed popula-
published methods would make the same assumption, al-
though only Cochran (1977) makes it clear. In the case of M,,, tion, as judged by inspection from a stem and leaf plot;
the relevant distribution should be the joint distribution of it tests the hypothesis that the mean is zero. When the
precipitation amount and ion concentration (Hawley, 1985). stem and leaf plots suggested that the distributions
Because of the well-known inverse relationship (Junge, 1963) were reasonably symmetric, but not normally dis-
between ion concentration in precipitation and precipitation
amount, and also because.ion concentrations frequently have tributed, the Wilcoxon signed ranks test was used;
seasonal variations, the assumption that the single variable when asymmetry appeared in the plots, Fisher’s sign
concentration is identically distributed is generally not valid. test was used. The signed rank and sign tests address
However, the literature does not discuss the matter of the whether the median difference is zero.
joint distribution of ion concentration and precipitation
For each ion, the overall error rate of the testing was
amount.
protected by employing Hornmel’s (1988) modtica-
RESULTS tion of the standard Bonferroni procedure (Miller,
1981) for simultaneous hypothesis testing. Hornmel’s
Comparisons of SEM, values computed by the modification is more powerful than the usual Bonfer-
bootstrap method with those computed by the formu- roni method or other modifications (Hommel, 1989).
lations of Miller (1977), Galloway et al. (1984), and Hommel’s procedure is conservative (i.e. the true
Cochran (1977) are shown for all major ions at overall error rate does not exceed the nominal rate) if
NADP/NTN sites in Figs l-9 and Table 1. The Simes’ (1986) test, on which Hommel based his test, is
bootstrap method is taken as the standard because of conservative. For the situation here, all test statistics
its established statistical validity and because it re- are asymptotically distributed as standard normal
quires no assumptions about distributions. Each point variates, and Simes’ (1986) simulation results indicate
in the three panels of these figures represents one that the testing done here should be conservative.
NADP/NTN network sampling site. If the values The results of the hypothesis tests are presented in
computed by the respective literature formulations Table 1 in the form of adjusted p-values, as described
agreed perfectly with those computed by bootstrap- by Wright (1992). The adjusted p-value for a particular
ping, all the points would fall on the 1: 1 lines shown in hypothesis represents the smallest overall error rate
the three panels. To the extent that the points fall for which that particular hypothesis can be rejected.
predominantly above or below the 1: 1 line, the Thus, the reader is free to choose any overall error rate,
method shows a bias with respect to the bootstrap p, and read directly from Table 1 whether any particu-
method. The degree of scatter in the points represents lar test is statistically significant (i.e. if the value in
the imprecision of the method, relative to the boot- Table 1 is <p, then that test is statistically significant
strap results. at the overall error rate of p).

Table 1. Probabilities that the mean SEM, values (median values for the Wilcoxon and the
Fisher sign tests) computed by three literature formulations are the same as those computed
by the bootstrap method, @ven the observed differences in results, for 222 sampling sites in the
NADP/NTN database

Galloway
Miller et al. Cochran
Ions Tests (1977) (1984) (1977)

cl- Wilcoxon test 0.0000 0.0000 0.3404

so:- t-test 0.0000 0.0000 0.1280
;y; t-test O.oooO 0.0000 0.3636
Fisher sign test O.oooO O.oooO 0.5028
NH: t-test 0.0000 0.0000 0.9268
Na+ Wilcoxon test 0.0118 O.C@OO 0.9571

$+ Wilcoxon test 0.0004

0.5045 0.0000
0.0003 0.2863
0.1824
Mg’+ t-test 0.8502 0.0000 0.8502

Note: The paired t-teat was used for ions where the distributions of the differences between
methods were approximately normal; the distribution-free Wilcoxon signed rank test was
used for non-normal symmetric distributions and the Fisher sign test otherwise.
1188 D. F. GATZ and L. SMITH

Figure 1 shows results of the comparisons for Cochran method. (With a single off-scale outlier elim-
chloride (Cl-). All three methods show a general inated, the Miller RMS value dropped from 0.057 to
overall agreement with the bootstrap results. The top 0.016, while values of the other two methods changed
panel, comparing bootstrap results at the NADP/ very slightly.)
NTN sites to those computed from the Miller (1977) Figures 2 and 3 show similar results for sulfate
formula, shows a predominance of points with boot- (SO:-) and nitrate (NO;). Both ions show signifi-
strap SEM, values greater than 0.1 rnge-’ falling cance at the 5% level, based on the t-test, that the
below the 1: 1 line, while for the Galloway et al. mean differences between SEM, values computed by
method the points in the same range are predomin- the Miller and Galloway et al. methods are not the
antly above the line. Table 1 indicates significant same as those computed by bootstrapping. The biases
differences (all references to significance refer to the are in the same directions as those seen for Cl-. No
5% level) from the bootstrap method for both the bias appears in the Cochran method, which also
Miller and the Galloway et al. methods, based on the exhibits a much smaller relative scatter (Fig. 10) than
nonparametric Wilcoxon method. The Cochran for- the results of the other two methods. (Elimination of
mulation shows excellent agreement with the boot- one outlier from the SO:- data did not materially
strap method in Fig. 1, and correspondingly the affect the probabilities; however, it reduced the Miller
probabilities show no significant bias in Table 1. RMS value (Fig. 10) from 0.062 to 0.025, without
A comparison of the dispersion or degree of scatter substantially affecting those of the other methods.)
of the points from the 1: 1 line, in the comparisons of The results for hydrogen ion (H+) (Fig. 4) are also
Figs l-9 is shown in Figure 10. The degree of scatter is similar to those of the three anions. The Miller and
expressed by the pairwise root-mean square (RMS) Galloway et al. methods show apparent biases, in the
differences between the SEM, values of the respective same directions as seen above, and indeed, both
literature methods and the bootstrap method. For methods yield SEM, values different from the boot-
Cl-, for example, Fig. 10 shows the greatest scatter for strap values at the 5% level based on the t-test (Table
the Miller method, an intermediate value for the 1). The RMS scatter (Fig. 10) is greatest for the
Galloway et al. method, and the smallest value for the Galloway et al. method, intermediate for the Miller

0.6

0.5

0.4

0.3

0.2

0.1

0.0
0.6 , I I I
2 Galloway
et a/. (1964) o,5 Galloway et a/. (1964)
0.4 0 0
: 0
0.4

0.3

0.2

0.1

0.0
0,s
0.5
0.4

0.3

0.1

0.0
0.0 0.1 0.2 0.3 0.4 0.5 0.6
BOOTSIRAPPED SEM, (mg/L) BOOTSTRAPPED SEM, (mgk)

Fig. 1. Comparison of bootstrapped SEM, values for Fig. 2. Comparison of bootstrapped SEM, values for
Cl- at 222 NADP/NTN sites with values computed SOi- at 222 NADP/NTN sites with values computed
by three formulations proposed previously in the by three fornudations proposbd previously in the
literature. One off-scale outlier is not shown. literature. One off-scale outlier is not shown.
 Bootstrapping vs other methods 1189

0.014
0.012

0.010
0.008
0.006
0.004
0.002
3 0.000

0.010
4
2 0.008
0.006

5 0.014 1 I I I I
o o,2 Cochran (1977) 0

0.1 0.2 0.3 0.4 0.000 0.002 0.004 0.006 0.008 0.010 0.012 0.014
BOOTSTRAPPED SEh$, (m@) BOOTSTRAPPED SEM, (j&L)

Fig. 3. Comparison of bootstrapped SEM, values Fig. 4. Comparison of bootstrapped SEM, values for
for NO; at 222 NADP/N’MU sites with values com- H+ at 222 NADP/NTN sites with values computed by
puted by three formulations proposed previously in three formulations proposed previously in the literature.
the literature.

method, and smallest for the Cochran method. This For calcium (Ca”) and magnesium (Mg’+) (Figs 8
same pattern is evident for ammonium (NH:) as well and 9, respectively), the r-test (Table 1) indicated that
(Fig. 5). The biases in the Miller and Galloway et al. only the Galloway et al. method is significantly biased
methods are highly significant by the r-test (Table l), (Table l), although this bias is not obvious from the
while the Cochran method again shows no bias and figures. The RMS scatter (Fig. 10) is greatest for the
the least scatter (Fig. 10). Miller method and least for the Cochran method for
For sodium (Na’) and potassium (K+), the pattern both ions.
is somewhat different, however (Figs 6 and 7, respect- Uncertainties in M, ion concentrations may be
ively). The previous bias toward low SEM, values expressed simply by stating the standard error. How-
from the Miller method is not evident in these figures, ever, it is often more useful to use both M, and SEM,
although Wilcoxon tests (Table 1) still found signifi- to express the uncertainty by the approximate con-
cant differences between the medians of the respective fidence interval M, + SEM, z@),where z(“)is the 100 t(
methods for both ions at the 5% level. The previously percentile point of a standard normal distribution. If
seen bias towards high SEM, values in the Galloway the M, concentrations were normally distributed, this
et al. method is still evident for Naf in Fig. 6, how- interval would have the approximate coverage prob-
ever, and is confirmed by the Wilcoxon test (Table 1). ability of 1-2~~. For a 95% confidence interval
The bias of the Galloway et al. method for K+ is (a =0.025, l -2a =0.95), the z(@is - 1.96. While the
visually much smaller (Fig. 7), but the Wilcoxon test assumption of normally distributed M,‘s permits the
(Table 1) still found a significant difference from the convenient computation of confidence intervals, the
bootstrap results. normal assumption is not always justified. Hawley’s
The RMS scatter of both the Na+ and K+ plots is (1985) analyses of data from several Multistate Atmo-
greatest for the Miller formulation (Fig. lo), although spheric Power Production Pollution Study (MAP3S)
the Na+ result is strongly affected by two off-scale sites indicated that the M, is approximately log-
data points. The Co&ran method again achieved the normal for n< 100 and approximately normal for
best agreement with the bootstrap method for both larger n. Figure 11 shows a histogram of the distribu-
ions, showing no bias and the least scatter. tion of 200 bootstrapped VWM values for K+ at the
 0.20 0.06 I , , I
0
Miller (IQn) Miller (1977j
0.05
0.15
0.04

0.10 0.03

0.10 0.02
0.05
0.05 0.01

3 0.00 s 0.00
.k 0.30 2 0”:;;
Galloway et a/. (i QS4) P
r’ D 0.25 5% 0.05
8 0.15
“4 0.04
2 2 o.20
$ 0.10 z 0.15 0”
2 0.03

e 6I 0.10 :: 0.02
2 0.05 w
I 0.05 I 0.01
2
d 0.00 s 0.00 d 0.00
5 0.20 0.30
Cochran (1977)
0.25

0.20

0.15

0.10

0.05

0.00
0.00 0.05 0.10 0.15 0.20 0.25 0.30
BOOiSTRAPPED SEM, (mQ/L) SOOTSTRAPPED SEM, (m@L) BOOTSTFIAPPED SEM, (ma)

Fig. 5. Comparison of bootstrapped SEM, values for Fig. 6. Comparison of bootstrapped SEM, values for Fig. 7. Comparison of bootstrapped SEM.,, values for
NHf at 222 NADP/NTN sites with values computed Na+ at 222 NADP/NTN sites with values computed K+ at 222 NADP/NTN sites with values computed by
by three formulations proposed previously in the by three formulations proposed previously in the liter- three formulations proposed previously in the literat-
literature One nff-scale nutlier is not shown. ature. Two off-scale outliers are not shown. ure. One off-scale outlier is not shown.
 Bootstrapping vs other methods 1191

0.10

0.08

0.06

0.04

0.02

0.00
0.5 ,1 I I I 0.10
1
Gallowayet a/. (I 984)
0.4 0.08
H
4
2 0.3 0.06

f 0.2 0.04

k 0.02

z 0.0
0.1 0.00
I I 0.10

0.4 0.08

0.3 0.06

0.2 0.04

0.1 0.02

0.0 0.00
0.0 0.1 0.2 0.3 0.4 0.5 0.00 0.02 0.04 0.08 0.08 0.10
BOOTSTRAPPED SEM, (mgll) BOOTSTRAPPED BE\ (III@)

Fig. 8. Comparison of bootstrapped SEM, values for Fig. 9. Comparison of bootstrapped SEM, values for
Caz+ at 222 NADP/NTN sites with values computed Mg’+ at 222 NADP/NTN sites with values computed
by three formulations proposed previously in the by three formulations proposed previously in the
literature. literature.

0.10 Galloway eta/. (1984)

$ m Cochran (1977)
g <
er3 0.08
cn*
#
$0 0.06

If2
3 z 0.04

b
= 0
8 0.02

cc so,” N03’ He NH,* Na+ K’ d+ t&J2+

Fig. 10. Degree of scatter in the method comparisons in Figs l-9, as ex-
pressed by the RMS diiTerences of the SEM, values of the respective literature
methods from the corresponding bootstrap values. The H+ SEM, values have
been multiplied by lo5 to make them large enough for visuai comparison.
EtTects of removing out&s from the data sets are discussed in the text.

Florida Everglades site, along with a smooth curve distribution in this case would lead to an inaccurate
showing the normal distribution corresponding to the assessment of the 95% conlidenceinterval. The follow-
measured VWM and the bootstrapped SEM,. Close ing paper (Gatz and Smith, 1994) explores this issue in
inspection reveals that the assumption of a normal more detail.
1192 D. F. GATZ and L. SMITH

25

0.00 0.04
I
0.06 0.12
PRECIP-WTD MEAN K CONC, mg/L
0.16
I
0.20

Fig. 11. Histogram of 200 bootstrapped precipitation-weighted mean K+ coneentra-

tions at the Florida Everalades site in the NADP/NTN network. The data were
collected over a 3 yr period, 1988-1990. The smooth’curve is the normal distribution
based on the mean and standard deviation of the 200 M, values. Standard 95%
confidence limits based on the assumption of a normal distribution are 0.026 and
O.l26mgG-‘, and those estimated from the bootstrapped data by the percentile
method (Efron and Tibshirani, 1986)are 0.037 and 0.136mg e- I.

DISCUSSION AND CONCLUSIONS computing of SEM,. However, smaller sample sizes

than those utilized here may warrant investment in the
The results presented graphically in Figs l-9 and additional computation of the bootstrap, particularly
the statistical comparisons in Table 1 leave no doubt when confidence limits are desired.
that the Cochran method gives results that are not
statistically different from those produced by the Acknowledgements and disclaimer-We thank Sherman
bootstrap method, at least over the three year data Bauer for carrying out many of the calculations for this
period of this study. The Miller (1977) method pro- paper, and Robin Shealy for statistical advice. Donald Gatx’s
duced mean SEM, values that were significantly work was partially supported by the U.S. Environmental
Protection Agency (EPA) under Contract No. GL995476-Ol-
different from those of the bootstrap method for seven 1. Luther Smith’s work was supported by the U.S. EPA
of the nine ions tested. The SEM, values computed through Contract No. 68-DO-0106 to ManTech Environ-
from the Miller method were typically underestimates, mental Technology. This paper has not been subjected to
compared to those from the bootstrap method. This EPA policy review and should not be construed to represent
the policies of the agency. Mention of any product name does
agrees with results reported earlier by Hawley (1985). not constitute endorsement.
The same comparisons between the bootstrap method
and that of Galloway et al. (1984) produced signifi-
cantly different results for all nine ions. The Galloway REFERENCES
et al.. results were frequently biased high (overestima-
tes) compared to those from the bootstrap method. Cochran W. G. (1977) Sampling Techniques (3rd Edn). Wiley,
In contrast, the agreement between the Cochran New York.
method and the nonparametric bootstrap methods Efron B. (1982) The Jackknife, the Bootstrap, and Other
Resampling Plans. Society for Industrial and Applied
was excellent. These two completely different methods
Mathematics, Philiadelphia.
gave results that were not statistically different for nine Efron B. and Tibshirani R. (1986) Bootstrap methods for
different ions at more than 200 sampling locations standard errors, confidence intervals, and other measures
over a three-year time period. of statistical accuracy. Statist. Sci. l(l), 54-77.
In terms of ease of computation, the Cochran Efron B. and Tibshirani R. (1991) Statistical data analysis in
the computer age. Science 253, 390-395.
method has an edge over the bootstrap method and Endlich R. M., Eynon B. P., Ferek R. J., Valdes A. D. and
would appear to be the method of choice for routine Maxwell C. (1988) Statistical analysis of precipitation
 Bootstrapping vs other methods 1193

chemistry measurements over the eastern United States - Technical Information Service, U.S. Dept. of Commerce,
I. Seasonal and regional patterns and correlations. J. appl. Springfield, VA.
Met. 27, 1326-1333. Miller R. G. (1981) Simultaneous Statistical Inference, pp.
Galloway J. N., Likens G. E. and Hawley M. E. (1984) Acid 67-68. Springer, New York.
precipitation: natural versus anthropogenic components. National Atmospheric Deposition Program (1992) NADP/
Science 22$._829-83 1. NTN Annual Data Summarv: Preciuitation Chemistry in
Gatz D. F. and Smith L. (1995) The standard error of a the United States. 1991. Natural Resources Ecology -La-
weighted mean concentration - II. Estimating confidence boratory, 475 pp. Colorado State University, Fort Collins.
intervals. Atmospheric Environment 29, 1195-1200. Simes R. J. (1986) An improved Bonferroni procedure for
Hawlev M. E. (1985) Comuarison of volume-weiahted mean multiple tests of significance. Biometrika 73, 751-754.
concentrations in precipitation chemistry. Ninth Confer- SRI International (1986) Statistical analysis of precipitation
ence on Probability and Statistics in the Atmospheric Scien- chemistry measurements over the eastern United States.
ces, pp. 55-59. American Meteorological Society, Boston. Final Report, Contract UlOl-3, Utility Acid Precipitation
Hawley M. E., Galloway J. N. and Keene W. C. (1988) Study Program, pp. 3-9, UAPSP-112, Palo Alto, CA.
Standard error calculations for non-sea salt constituents in Top01 L. E., Lev-On M., Flanagan J., Schwa11 R. J., and
marine precipitation. Water Air Soil Pollut. 42, 87-102. Jackson A. E. (1985) Quality assurance manual for pre-
Hommel G. (1988) A stagewise rejective multiple test proced- cipitation measurement systems. Contract No. 68-02-3767.
ure based on a modified Bonferroni test. Biometrika 75, Environmental Monitoring Systems Laboratory, Office of
383-386. Research and Development, US. Environmental Protec-
Hommel G. (1989) A comparison of two modified Bonferroni tion Agency, Research Triangle Park, NC.
procedures. Biometrika 76, 624-625. Vong R.,Clink S., Reems G., Be&ert J., Charles D., Gibson J.,
Junge C. E. (1963) Air Chemistry and Radioactiuity, pp. Husar R.. Olson A.. Simuson J. and Sielkoo S. (1989)
311-315. Academic Press, New York. Regional analysis of wet deposition for effecis research:
Liljestrand H. M. and Morgan J. J. (1979) Error analysis Report EPA/600/3-89/030, Environmental Research
applied to indirect methods for precipitation acidity. Tellus Laboratory, Office of Research and Development, U.S.
31,421-431. Environmental Protection Agency, Corvallis, OR.
Miller J. M. (1977) A statistical evaluation of the U.S. Wilkinson L. (1990) SYSTAT: The System for Statistics.
precipitation chemistry network. In Precipitation Scaueng- SYSTAT, Inc., Evanston, IL.
ing (1974) (edited by Semonin R. G. and Beadle R. W.), Wright S. P. (1992) Adjusted p-values for simultaneous
pp. 639-659. Available as CONF 74100 from National inference. Biometrics 48, 1005-1013.