A Managers Guide to Interpreting Analytical Data Kenneth E.

Osborn East Bay Municipal Utility District Oakland, CA 94623 Important arguments require evidence and quantitative evidence requires statistical evaluation. (Stephen Stigler, Statistics on the Table). Managers use analytical laboratories to provide them with measurement based information. All measurements have measurement error. Sources and types of measurement error influence the generation, interpretation, and regulatory uses of laboratory data. The successful manager intuitively understands analytical data and measurement error. This paper offers examples using visual models to reinforce that intuitive understanding and provides a concept based tool not found in traditional texts on either statistics or data interpretation. Measurements repeatedly made of the same sample1 or of a series of samples from a single process will generate a distribution of analytical results centered around a value that we accept as more representative than any of the other measurements. This scattering of values occurs because all measurements contain error. There is no way that errors in the individual measurements can be eliminated. Thus to properly interpret a collection of measurements or any analytical result, it is necessary to understand the nature of analytical error. Analytical error includes both random and systematic components. Systematic error is constant from one observation to the next. Sources of systematic error include improper instrument calibration, contamination introduced during sample processing, and matrix interference. Because systematic errors are constant they are predictable and thus potentially controllable. An example of a source of systematic error would be a yardstick that was short by one half-inch. Random errors vary from one observation to the next, have an equal likelihood for increasing or decreasing the measurement value, and decrease in occurrence with increasing magnitude of the error. Because random errors are not individually predictable, they are less controllable than systematic errors. An example of random error is depicted in figure 1. A sample of twenty bodyweight readings rounded to the nearest whole weight in pounds, was taken by repeated weighings on a bathroom scale.2 The weights range from 158 pounds to 162 pounds. Considering the variability in readings of the scale, it would be appropriate to ask how much does this person really weigh?
1

The use of sample in this context refers to a collection of data. Thus a sample of size = 7 refers to 7 distinct observations or measurements. 2 Data used in all plots for this article were generated by a computer based simulation program written by the author. It is available as a QBASIC program by request to the author at [email protected].

Any distribution of data has a value around which the values tend to center. Values are scattered around that value so that some are greater than and some are less than the central value. Figure 2 displays the same readings as figure 1, with the values ranked from low to high. This presentation by order of magnitude rather than order of occurrence emphasizes the statistical properties of the distribution such as central tendency and type of symmetry. One measure of central tendency is the mode. It is the most frequent reading and equals 161 pounds in the example. Another measure of the center would be the value in the middle, or the measurement that is exceeded by as many values as it exceeds. This is known as the median and equals 160 pounds in the example. The measure we are most familiar with is the arithmetic average, also known as the mean. The mean is defined as the sum of all values divided by the number of values and equals 160 pounds in the example. The mean is symbolized as [nb: symbols are in arial font] for the population mean, and as x-bar [nb: find symbol for xbar] for the sample. Which measure of central tendency is best? It depends! For a series of symmetrically distributed measurements, such as in this example, the mean and the median will be the same. For a collection of asymmetrically distributed data, the median may provide a more useful center of the distribution, especially if the data contain extreme values. An example of an asymmetrical distribution would be the collection of all family incomes in the United States. An arithmetic mean would tend to indicate a higher income than would be typical for the average family because a small percentage of very wealthy families would have a disproportionate effect on the mean. The median, being the middle value, tends to be insensitive to extreme values. Given that the mean is 160 pounds and the scales have been accurately calibrated, can it be stated that the true weight based on this sample of twenty weights is 160 pounds? Suppose another sample of twenty weights were taken, would the mean of that set also be 160 pounds? If repeated weights gave the same answer over and over, we could conclude that the true weight was the same as the mean. However, another series of weights will not produce the exact same sequence. Because of measurement error, a sample of measurements can provide only an estimate of the population mean of all possible measurements. The true population mean is beyond our reach. So the question is not is the true weight equal to 160 pounds, but rather, how close is the 160 pounds to the true answer. Figure 3 is a plot of 100 scale readings (rounded to the nearest ounce) and ranked in order by magnitude. The range is a low of 158 pounds and a high of 162 pounds and the mean is 160 pounds. This plot can be converted into a probability curve by exchanging the axes (see figure 4). The readings are symmetrically scattered around the mean, the mean and median are equal, and there is a greater density of readings near the mean that at the

extremes. The conditions of symmetry, equal mean and median, and decreasing probability of occurrence with extreme values are the properties of normal distributions. There is nothing abnormal about other types of distributions, but there is an almost universal tendency of physical measurements to be normally distributed. A generalized plot of the normal distribution is depicted in figure 5 and is also referred to as the Gaussian distribution and the Bell Curve. When the mean is zero and the standard deviation is one, the term standard normal distribution is used. So how does this help answer the question, how close is the mean of a sample to the population mean? Distributions have three important properties: shape, a center, and dispersion around the center. If the errors consist of many small and unrelated effects, the errors, and hence the measurements, will be normally distributed. A normal distribution is uniquely represented by two, and only two, statistics: the mean and the standard deviation. The standard deviation is a measure of dispersion in the same fashion that the mean is a measure of central tendency. Knowledge of one statistic in the absence of the other provides incomplete information regarding a collection of measurements. The standard deviation of the distribution for figures 3 and 4 is 1 pound and is calculated as the square root of the average squared distance of data from the mean (Conick and Smith, The Cartoon Guide to Statistics). The standard deviation is symbolized by for the population and s for the sample. Now we have something! A normal distribution of measurements, with a mean of 160, and a standard deviation of 1. Referring to figure 4, it is apparent that roughly 60 to 70 percent of the values lie within 1 standard deviation of the mean and more than 90 percent of the values lie within 2 standard deviations of the mean. Thus we would expect to obtain readings within one pound of the true weight 60 to 70 percent of the time and readings more than one pound from the true weight 30 to 40 percent of the time. We also would expect to obtain a small percentage of readings that exceed the true weight by more than two pounds. But is the mean of a series of weights no more reliable than an individual weighing? Intuitively, it seems that repeated weights will provide a more representative answer since the sum of random errors will approach zero as the number of measurements increase. This brings us to the Central Limit Theorem, which states: if repeated random measurements are made of a population, the collection of averaged values will approximate a normal distribution with a mean equal to the population mean and standard deviation equal to the population standard deviation divided by the number of measurements in the individual samples. As the sample size increases, the approximation will improve.

The Central Limit Theorem is consistent with our intuition: collect more measurements, and the mean of the data converges to a constant. Furthermore, the standard deviation of the sample mean is smaller than the population standard deviation: as the size of the sample increases, the standard deviation of the mean decreases with the square root of the sample size. For the example of twenty weights on the bathroom scale, the standard deviation of the mean is 0.2 pounds ( 1 pound divided by the square root of 20). Is it necessary to make replicate measurements to evaluate an analytical result? Of course not! It is reasonable to apply the statistics of past efforts to future events. For example, if I weigh myself after the Christmas holidays and see a scale reading of 165, based on the previous readings ranging from 158 to 162 pounds, I can conclude that I have gained weight. Of course it is always possible that the scales malfunctioned or my daughter is playing a trick on me and adjusted the dial. But if the scale calibration and the zero reading are acceptable, then I will should eat fewer hamburgers and French fries for awhile. I hope this brief introduction to visualizing analytical data through probability plots has provided the reader with a new tool for interpreting quantitative information. While this presentation has been limited to a discussion of measurement error and the most basic of statistical concepts, probability plots are simple to construct, and have universal applications for data interpretation. Future articles and presentations will explore additional uses.

Definition of terms

Error Difference between the observed measurement and the true value. Systematic error The component of error that is constant. Random error The component of error that is not constant. Sample A collection of measurements from a population. Population The collection of all possible measurements. Mean The arithmetic average of a sample or population. Median The middle value in a distribution of measurements. Mode The most frequently occurring value in a distribution. Standard deviation A measure of variability or spread in a sample or population. Central Limit Theorem A fundamental theorem of probability that states that the mean of a sample from a population is approximated by the normal distribution as the number in the sample becomes large (Websters Ninth New Collegiate Dictionary). Normal distribution The mathematically defined scatter of a set of measurements around a central value for which the errors are random, less probable at extreme values, and the consequence of multiple, small factors. Z-number The relative distance of a measurement from the average expressed as a fraction of the standard deviation. (mu) Population arithmetic mean. True value. x-bar Sample mean calculated from data. An estimate for mu. (sigma) Population standard deviation. s Estimate of sigma calculated from data.