MODULE 2

STATISTICAL TESTING FOR TREATMENT OF DATA I

Introduction

Let me begin my lecture by recalling some essential information you already knew about
this topic. Let me ask you few questions before I start my discussions.

Why is experiment necessary to subjects like chemistry? What activities were carried out
when you performed your experiment? What did you obtain when you performed your
experiment? How did you manipulate the results of your experiment? What did you do then with
these manipulated results?

You are all made aware that being an experimental science, chemistry involves
laboratory activities designed to explain scientific theories. In your general chemistry laboratory,
you did not only measure the value of the scientific property but you also gathered data as part of
the experimentation. Perhaps the bigger challenge that you encountered was how you were going
to manipulate or treat the data systematically in order to find the value of the property that was
being measured.

Interestingly, statistics will teach you how to manipulate or treat experimental data
systematically. As a scientific study, statistics is not only used in treating experimental data as it
is also utilized extensively in almost all fields of disciplines. One common situation that every
one of you might have already encountered is finding the simple mean or the average of the
values. This is a situation that we usually experience not only in the laboratory particularly in
treatment of data but also in every day activity.

This module covers the introductory concepts of statistics. The topics that I included in
this module are pre-requisite to statistical testing and treatment of data. Some of these topics
might have already been discussed in high school mathematics. But because of their extensive
use in the statistical treatment of data in introductory analytic chemistry, it is necessary that we
review these topics.

1 Statistical Testing for Treatment of Data I –Module 2

 The last part of this module tackles essential mathematical tools in measurement. We will
also review the rules and calculations involving significant figures and dimensional analysis.
Although this is not part of the regular analytical chemistry course, its application is still useful
in some topics in analytical chemistry.

2 Statistical Testing for Treatment of Data I –Module 2

After the end of the module, you should be able to:
1. differentiate the three expressions of central value particularly on how these are
computed and used appropriately to describe a set of data;
2. differentiate qualitatively between precision and accuracy;
3. compute absolute error, relative error, and percent error;
4. compute average deviation, standard deviations of the sample and the population, and
the pooled standard deviation of the population;
5. differentiate between exact and inexact numbers;
6. report measured data accurately with the correct number of significant figures based on
the accuracy of the instrument used;
7. report accurately the answers of performing various arithmetic operations with the
correct number of significant figures; and
8. convert a given unit into the desired unit using dimensional analysis.

: 3 hours

Before we proceed, let us first check your prior knowledge and skills on the
topics included in this module. Please take the test below. For multiple-choice
questions, just encircle the letter you think is the correct answer. Otherwise,
provide the correct answer to the question. You have 30 minutes to do this.
1. This is the value that occurs with the 2. This is the value of the middle term in
highest frequency in a data set. a data set that has been ranked in
a. mean b. median increasing order.
c. mode d.standard deviation a. mean b. median
c. mode d. standard deviation

3 Statistical Testing for Treatment of Data I –Module 2

 3. Find the mean, median, and mode for 7. What is the percent relative error of
the following data set:4, 17, 9, 11, 26, the data set in item #6?
11, 13, 7, 22, and 11
a. mean = 15.1, median = 11, mode = 13 8. The following data is obtained for a
b. mean = 14.4,median = 18.5,mode = 11 sample with 9 measurements: 8.0, 2.0,
c. mean = 12, median = 12,mode = 11 2.0, 7.0, 4.0, 6.0, 5.0, 3.0, and
d. mean = 13.1,median = 11, mode = 11 4.0.Calculate the standard deviation for
4. Data were collected on the number of this set.
minutes spent in cooking a meal. The
data are as follows: 8, 10, 15, 25, 30, 9. The average weight of 8 people
40, 12, 20, and 19. What is the range of increases by 2.5 kg when a new person
this data? comes in place of one of them weighing
65 kg. What might be the weight of the
a. 24 b. 22 c. 32 d.48 new person?
5. The more dispersed the data values in
a particular data set are, the smaller the 10. A particular method for the
variance and standard deviation will determination of copper yields results
be. that are low by 0.5 mg. What will be the
a. True b. False percent relative error due to this source
6. Given the following data: 61.45, if the mass of the copper in a sample is
61.53, and 61.32. If the accepted value 25 mg.
for this data set is 61.71, what is the
absolute error for the mean of the data
set?

Check your answer against the KEY found at the end of this module.

How did you perform in the test? If your score is in the range 8-10, you are equipped to succeed
in your study in this module. If you scored 6 or 7, you have the potential of doing far better. If
you scored 4 or 5, you probably need to focus more on the reading materials and the
mathematical procedure as we go along. But if your score is lower than 4, you need to review
your past lessons of the subject. I advise you to retake the test until you get a score of 4 or better.

4 Statistical Testing for Treatment of Data I –Module 2

 Let me start my lecture by reviewing some fundamental concepts of statistics. You might
have tackled this in your high-school mathematics or your previous algebra.

1.1 Central Value

Central value is expressed in three expressions and the use of each depends on how one
wants to describe and interpret the data.

1.1.1 Arithmetic mean / Mean

The most common central value used by chemists is the arithmetic mean,𝑥, which
is obtained by dividing the sum of the individual values by the number of values.
Mathematically,

𝑥1+ 𝑥2+ 𝑥3+ . . . + 𝑥𝑛 𝑥𝑖

𝑥 = 𝑛
=
𝑛

where x1, x2, x3, . . . , xn are the individual values, n is the number of values, and 𝑥𝑖 the
sum of values of x.

1.1.2 Median
Another central value which is less commonly used is the median. It is the middle
numerical value in a set of values.

Example 1-1

Find the median of the five values 20.4, 20.6, 20.1, 20.7, and 20.0.

Rules:

1) Rearrange the values from the lowest to the highest.

2) Identify the value that is physically located in the middle of the set of data.

20.0 20.1 20.4 20.6 20.7

5 Statistical Testing for Treatment of Data I –Module 2

When the set of data contains an even number of values, the median is the average of the
two numerical values.

Example 1-2

Find the median of 20.4, 20.6, 20.1, and 20.7.

Rearranging,

20.1 20.4 20.6 20.7

middle numerical values = 20.4 and 20.6

The median is calculated as follows:

20.4 + 20.6
median = = 20.5
2

1.1.3 Mode

The mode, which is not so common in analytical chemistry, of data is the value
that is most frequently repeated in the data set.

Example 1-3

The mode of the data 20.2, 20.1, 20.0, 20.1, 20.4, 20.0, 20.1, and 20.7 is 20.1
because it appeared in the data the most frequent at three times.

Example 1-4

Calculate the mean and the median for each of the following sets of data:

Set A: 6.37, 6.33, 6.41, 6.80

Set B: 6.37, 6.33, 6.41, 6.93.

Solution

According to the definitions stated above, for data set A,

6.37 + 6.33 + 6.41 + 6.80 25.91

𝑥= = = 6.48
4 4

6 Statistical Testing for Treatment of Data I –Module 2

 6.37 + 6.41
median = = 6.39
2
For data set B,
6.37 + 6.33 + 6.41 + 6.93 26.04
𝑥 = = = 6.51
4 4

6.37 + 6.41
median = = 6.39
2

1.2 Precision and accuracy

The terms precision and accuracy are often used when dealing with the uncertainties of
measured values. Precision is a measure of how closely individual measurements agree with one
another while accuracy refers to how closely individual measurements agree with the correct, or
“true,” value. The dart analogy in Figure 1-1 illustrates the difference between these two
concepts.

Figure 1-1. Precision and accuracy

(Source: Brown, T. L., et. al., 2012)

1.2.1 Expressions of accuracy

As defined, accuracy describes the nearness of an experimental value, xi, or a
mean,𝑥, to the true value, µ. It is expressed as error. The following are various
expressions of error based on how it is computed.

Absolute error is calculated this way,

7 Statistical Testing for Treatment of Data I –Module 2

 error = xi - µ or 𝑥 - µ

Note: error carries the units of xi, and µ.

Relative error is used when comparing errors at different quantities. It is

calculated by dividing the absolute error by the true value as follows:

𝑒𝑟𝑟𝑜𝑟
relative error =
µ

Relative error is often expressed by statisticians as pph or ppt, as defined below:

Parts per hundred (pph) or percent error is the relative error multiplied by 100.
Parts per thousand (ppt) error is the relative error multiplied by 1000, and so on.

Notes from the author:

pph and ppt used as expressions of relative error should not be confused with or
used interchangeably as the percent concentration (mass %, volume %, and mole
%) and the ppt. Although the mathematical thought and the procedure of
calculating them may be the same in some respect, percent concentration and ppt
are some of the expressions of concentration.

Example 1-5

Calculate the absolute error, percent error, and parts per thousand error for the
mean of the following data set.

xi (mg) 8.33 8.29 8.28 8.34 8.36

µ (mg) 8.27

8 Statistical Testing for Treatment of Data I –Module 2

 Solution
To determine the error, you must calculated first the mean of the given data set:

𝑥𝑖 8.33 + 8.29 + 8.28 + 8.34 + 8.36 41.60

𝑥 = = = = 8.32
𝑛 4 4

error = 𝑥 - µ = 8.32 mg - 8.27 mg = 0.05 mg

𝑒𝑟𝑟𝑜𝑟 0.05 𝑚𝑔
% error = x 100 = x 100 = 0.6
µ 8.27 𝑚𝑔

𝑒𝑟𝑟𝑜𝑟 0.05 𝑚𝑔
ppt error = x 1000 = x 1000 = 6
µ 8.27 𝑚𝑔

The average error is calculated like the average value or arithmetic mean except
that the individual errors rather than the individual values are used.

1.2.2 Expressions of Precision

Precision, a term used often mistakenly used in place of accuracy, refers to the
agreement between values in a set of data. The fact that the values of replicate
measurements all agree well does not necessarily mean that they are close to the true
value. There are several common ways to express the precision of data, as shown in the
following:

1.2.2.1Average deviation (or mean deviation)

It is one of the methods of showing dispersion or way of ascertaining the

deviation, taken without regard to sign, of the experimental value from the central value.
The average deviation,𝑑 , is found by summing the individual deviations and dividing by
the number of measurements. Thus the average deviation from the mean is given by

ǀ 𝑥𝑖 − 𝑥 ǀ
𝑑 =
𝑛

9 Statistical Testing for Treatment of Data I –Module 2

 where 𝑥𝑖 = observation

𝑥 = mean of the observations

n = number of observations

ǀ 𝑥𝑖 − 𝑥 ǀ = absolute value of the difference between an observation and the

mean (inside the absolute value sign is always a positive number)

ǀ 𝑥𝑖 − 𝑥 ǀ = sum of the absolute values of the differences between an

observation and the mean

𝑑 = average deviation between the experimental values and the mean

Similar to accuracy, precision measurement such as average deviation can be

expressed as an absolute error or as a relative error (% or pph, ppt, etc.)

1.2.2.2 Standard deviation of the sample

The standard deviation, s, or root-mean-square deviation as it is sometimes

called, is the preferred measure of precision and is calculated from the equation

(𝑥 𝑖 − 𝑥 )2
s =
𝑛− 1

where 𝑥𝑖 = observation

𝑥 = mean of the observations

n = number of observations

n–1 = degree of freedom

(𝑥𝑖 − 𝑥 )2 = square of the differences between an observation

and the mean

(𝑥𝑖 − 𝑥 )2 = sum of the squares of the differences between an

observation and the mean

s = standard deviation or the measure of the spread of

observations

10 Statistical Testing for Treatment of Data I –Module 2

 The most common mistake made by students using this equation is they “square
the sum of the deviations” rather than “sum the squares of the deviations”. Look carefully
at the following example to make sure that you learn to use the formula correctly.

Example 1-6

Quantitative analysis of student obtained the following results for the determination of
isooctane in gasoline using Gas Chromatography.
Determination Percent
Number isooctane
1 3.83
2 3.97
3 3.94
4 3.88
5 3.94
6 3.90

Calculate the standard deviation from the mean.

Solution:

You can calculate the standard deviation of the given data set using the formula,

(𝑥 𝑖 − 𝑥 )2
s = . The formula indicates that the mean is necessary. So
𝑛− 1
you need to calculate first the mean.
3.83 + 3.97 + 3.94 + 3.88 + 3.94 + 3.90 41.60
𝑥 = = = 3.91%
6 4

𝑥𝑖 ǀ 𝑥𝑖 − 𝑥 ǀ (𝑥𝑖 − 𝑥 )2

3.83 0.08 0.0064

3.97 0.06 0.0036
3.94 0.03 0.0009
3.88 0.03 0.0009
3.94 0.03 0.0009
3.90 0.01 0.0001
0.0128 = (𝑥𝑖 − 𝑥 )2
Substituting, you have

0.0128
s = = 0.051%
6− 1

11 Statistical Testing for Treatment of Data I –Module 2

 Both average and standard deviations can also be expressed in relative terms to facilitate
comparison between data sets:

𝑑
relative average deviation =
𝑥
𝑠
relative standard deviation =
𝑥

Relative standard deviation (RSD) is also referred to as the coefficient of

variation. Again, relative values can be expressed fractionally (as above), as parts per
hundred or percent (multiplied by 100), as parts per thousand (multiplied by 1000), and
so forth.

Range

The range, is the absolute difference between the largest and smallest values in
the data set.

1.2.2.3 Pooled Standard Deviation

The equation for computing a pooled standard deviation from several sets of data
takes the form. You use the formula given by Holler, F. and Crouch, S. (2014) to solve
pooled standard deviation.

where N1 is the number of results in set 1, N2 is the number in set 2, and so forth. The
term Nt is the total number of data set pooled.

Now, I am going to demonstrate to you how to solve pooled standard deviation

using the equation that I just mentioned. Consider the following example.

12 Statistical Testing for Treatment of Data I –Module 2

 Example 1-7

Glucose levels are routinely monitored in patients suffering from diabetes. The glucose
concentrations in a patient with mildly elevated glucose were determined at different
months through a spectrophotometric analytical method. The patient was placed on a low
sugar diet to reduce the glucose levels. The frequency of monitoring varies every month
as shown below and the days of the month when monitoring is conducted are chosen
randomly. The following results were obtained during a study to determine the
effectiveness of the low-sugar diet. Calculate a pooled estimate of the standard deviation
for the method.

Time Glucose Concentration Mean Sum of Squares Standard

(mg/L) Glucose (mg/L) of Deviations Deviation
from Mean
Month 1 1108, 1122, 1075, 1099 1100.3 1687.43 16.8
1115, 1083, 1100

Month 2 992, 975, 1022, 1001, 991 996.2 1182.80 17.2

Month 3 788, 805, 779, 822, 800 798.8 1086.80 16.5

Month 4 799, 745, 750, 774, 777, 800 771. 9 2950.86 22.2
758

Total number of measurements = 24 Total sum of squares = 6907.89

Solution

For the first month, the sum of squares in the next to the last was calculated as follows:

Sum of squares = (1108 – 1100.3)2 + (1122 – 1100.3)2 + (1075 – 1100.3)2

+ (1099 – 1100.3)2 + (1115 – 1100.3)2 + (1083 – 1100.3)2

+ (1100 – 1100.3)2

= 1687.43

13 Statistical Testing for Treatment of Data I –Module 2

 The other sums of squares were obtained similarly. The pooled standard deviation is then

6907.89
spooled = = 18.58 or 19 mg/L
24−4

Note that this pooled value is a better estimate of σ than any of the individual s values in
the last column. Note also that one degree of freedom is lost for each of the four sets.

1.3. Reporting computed data

Another essential element of experimentation is how you report data that is consistent
with the rules set in various standards. You look closely at the following topics that I am going to
explain. Understand the rules and procedures that I emphasize.
A numerical result is worthless to users of the data unless they know something about its
quality. Therefore, it is always essential to indicate your best estimate of the reliability of your
data. According to Holler and Crouch (2014), a much less satisfactory but more common
indicator of the quality of the data is significant figure convention.

1.3.1 Significant figures

We encounter significant figures when we deal with numbers obtained from

measurements particularly when we need to report our data resulting from the
experiment. The number of digits that we need to include in reporting the data depends
on the accuracy of the instrument used in measurement. The following paragraphs
illustrate how we can appropriately report a measured quantity.
Suppose you determine the mass of a coin on an analytical balance capable of
measuring to the nearest 0.0001 g. You could report the mass as 4.8405 ± 0.0001 g. The
notation ± (read “plus or minus”) expresses the magnitude of the uncertainty of your
measurement. In some scientific work we drop ± the notation with the understanding that
there is always some uncertainty in the last digit reported for any measured quantity.
Figure 1-2 shows a thermometer with its liquid column between two scale marks.
We can read the certain digits from the scale and estimate the uncertain one. Seeing that

14 Statistical Testing for Treatment of Data I –Module 2

 the liquid is between the 25 °C and 30 °C marks, we estimate the temperature to be 27
°C, being uncertain of the second digit of our measurement.

Figure 1-2. Uncertainties and significant figure in measurement

(Source: Brown, T. L., et. al., 2012)

All digits of a measured quantity, including the uncertain one, are called
significant figures. A measured mass reported as 2.2 g has two significant figures,
whereas one reported as 4.8405 g has five significant figures. The greater the number of
significant figures, the greater the certainty implied for the measurement.

Example 1-8

A sample that has a mass of about 25 g is placed on a balance that has a precision of ±
0.001 g. How many significant figures should be reported for this measurement?

Answer: five, as in the measurement 24.995 g or 25.005 g, the uncertainty being in the
third decimal place

Rules on counting the number of significant figures

1. All nonzero digits are significant. (significant figures in bold)

423.444(6 sf)
2. Zeros between two significant figures are themselves significant.
42,300045 (8 sf) 42,340.0025(9 sf)
3. Zeros at the beginning of a number are never significant.

15 Statistical Testing for Treatment of Data I –Module 2

 00042345.0 (6 sf) 0.00048(2 sf)

4. Zeros at the end of a number are significant if a decimal point is written in the
number.
0.0500 g (3 sf) 5.0 cm (2 sf)
5. Zeros at the end of a number without a decimal point may or may not be significant.
Exponential notation can be used to indicate whether end zeros are significant.
For example, a mass of 20,700 mg can be written to show three, four, or five
significant figures depending on how the measurement is obtained such as the
accuracy of the instrument used:

2.07x 104 mg (three significant figures)

2.070 x 104 mg (four significant figures)
2.0700 x 104mg (five significant figures)

(The exponential term 104 does not add to the number of significant figures.)

Example 1-9

How many significant figures are in each of the following measurements: (a) 3.549 g,
(b) 4.5 x 10-3 m, (c) 0.00146 mL?
Answers: (a) four, (b) two, (c) three

1.4 Dimensional Analysis

The purpose of doing dimensional analysis is to get the desired unit out of a given unit. In
doing so, the given quantity should be multiplied with a conversion factor. Conversion factor is
ratio of two equal quantities expressed in different units. The unit of the quantity in the
numerator is the unit desired and the unit in the denominator is similar to that of the given unit.
Conversion tables found in textbooks and general references are used as conversion factors.

Here are some pointers that you need to follow in carrying out dimensional analysis.

What do you need on top? (This is the desired unit in a single-step conversion.)

16 Statistical Testing for Treatment of Data I –Module 2

 What do you need at the bottom? (This is the given unit in a single-step conversion.)

What do you know? (This refers to the conversion factor to be used that can be obtained
from conversion tables.)

How do you get there? (This is the mathematical manipulation to show how the desired
unit is obtained out of the given unit. Often, the desired unit cannot
be obtained from the given unit by using only one conversion
factor. This calls for the use of more than one conversion factor to get the
desired unit.)

Note: Aside from conversion tables, you may also obtain conversion factor from the
relationships of quantities given or cited in the situation or in the problem.

Example 1-10

An individual with a high cholesterol level has 232 mg cholesterol per 100.0 mL of
his blood. How many grams of cholesterol are in his blood if he has a total blood volume
of 5.2 L?

Solution

𝑚𝑔 𝑐ℎ𝑜𝑙𝑒𝑠𝑡𝑒𝑟𝑜𝑙
What are given: 232 ; 5.2 L of blood ; 1000 mg = 1 g
100.0 𝑚𝐿
What is required: grams (g) of cholesterol
What I know: 1 L = 1000 mL, 1g = 1000mg
What do I need on top: grams (g)
What do I need at the bottom: none

1000 mL 232 mg 1g
5.2 L blood x x x = 12 g
1L 100.0 mL 1000 mg

Note: In this example, only two (2) significant figures are included in the final
answer since only multiplication and divisions are involved, in which the number of
significant figure in the final answer must agree with the quantity with the least number

17 Statistical Testing for Treatment of Data I –Module 2

 of significant figure. In this case, 5.2 L has the lowest number of significant figures. Note
further that exact numbers are not considered in counting number of significant figure
because of their nature being exact. Exact numbers contain indefinite number of
significant figure. Hence, 1 g, 1 L (with only one digit), 1000 mg, and 100 mL should not
be considered because they exact numbers.

1.4.1 Exact and inexact numbers

There are two kinds of numbers that are encountered in scientific work: exact
numbers are those whose values are known exactly while inexact numbers are those
whose values have some uncertainty. Exact numbers have defined values or can result
from counting objects. Inexact numbers are obtained from measurements whose
uncertainties may have caused by the inherent limitations of the equipment and by human
differences.

Exact Inexact

1000 g/ 1 kg ruler measure

2.54 cm/ 1 in temperature, volume or mass reading
12 in 1 dozen etc.
any conversion factor

Example 1-11

Indicate whether the number is exact or inexact: (a) the mass of the 32-oz can of coffee;
(b) the volume of blood in the capillary tube; (c) the number of inches in a mile; (d) the
average height of the students in the class; and (e) the number of pages in you book.

Answers: (a), (b), and (d) are inexact while (c) and (e) are exact.

Example 1-12
The volume of a van container used to deliver frozen fishes is 35.00m3. What is the
volume in liters?

18 Statistical Testing for Treatment of Data I –Module 2

 Solution:

What is given: 35.00 m3

What is required: volume in liters (L)
What you know: 1000 L = 1 m3
What you need on top: liters (L)

What you need at the bottom: none

1000 L
35.00 m3 x = 35.00 x 103 L
1 m3

Example 1-13
The density of a certain substance is 1.945 x103kg/m3. What is its density in g/mL?
Solution
What is given: density equal to 1.945 x 103 kg/m3
What is required: density in g/mL
What you know: 1000 g = 1 kg; 1000 mL = 1L; 1000 L = 1 m3
What you need on top: grams (g)

What you need at the bottom: mL

kg 1000 g 1 m3 1L g
1.945 x 103 m3 x x x = 1.945
1 kg 1000 L 1000 mL mL

1.5 Arithmetic Rules

Often, you will have to carry out arithmetic operations with measured quantities that are
governed by significant figures. In doing so, it is easy to generate meaningless digits, especially
with electronic calculators, and it is important that you learn to recognize and eliminate these
digits.

1.5.1 Rounding Off

There are several different rules commonly used for discarding unwanted digits in
a number. The following rule is the simplest and most common. If the digit to be

19 Statistical Testing for Treatment of Data I –Module 2

 discarded is 5 or greater, increase the retained preceding digit by 1. If it is less than 5, do
not change the preceding digit. For example,

5.175 rounded off to three significant figures becomes 5.18.

7.009216 rounded off to three significant figures becomes 7.01.

1.082 rounded off to two significant figures becomes 1.1.

The rule is applied one time to the first digit only following the last retained digit. Under
no circumstances should the rounding off be done sequentially. For example, 9.1547
should be rounded with three significant figures to 9.15 because 4 is less than 5. You
should not round off the 7, making the number 9.155, and then round off the 5, making it
9.16.

1.5.2 Addition and subtraction

The result has the same number of decimal places as the measurement with the
fewest decimal places. When the result contains more than the correct number of
significant figures, it must be rounded off.
Rules: Express all numbers with the same exponent and align all number with respect to
the decimal point.
Round-off the answer according to the number of decimal places in the number
with the fewest decimal places.

Example 1-14
Perform the following operations. Report the answer with the correct number of
significant figures.
Solutions
(a) 14.6481 (4 decimal places)
+ 17.347 (3 decimal places)
+ 44.31 (2 decimal places, least uncertain)
76.3051 → 76.31

20 Statistical Testing for Treatment of Data I –Module 2

 When the numbers to be added or subtracted have exponents, they must be made
to have the same exponent before the addition or subtraction is carried out.
Consider the following example.

(b) 2.17 x 10-3 → 0.0217 x 10-1

3.163 x 10-1 → 3.163 x 10-1

2.76 x 10-4 → 0.00276 x 10-1

3.18746 X 10-1 → 3.187 X 10-1

1.5.3Multiplication and division

The result contains the same number of significant figures as the measurement
with the fewest significant figures. When the result contains more than the correct
number of significant figures, it must be rounded off.

Example 1-15

Perform the following operations. Report the answer with the correct number of
significant figures.

Solutions

(a) 3.26 x 10-5

x 1.78
5.8028 x 10-5 → 5.80 x 10-5

(b) 4.3179 x 1012

x 3.6 x 10-19
1.554444 x 10-6 → 1.6 x 10-6

(c) 34.60
x 2.46287
85.215302 → 85.22

21 Statistical Testing for Treatment of Data I –Module 2

 Note that the power of 10 has no influence on the number of digits that should be
retained.
Consider the following example to illustrate operations involving mixed
operations (or composite operations)

Example 1-16

Compute the answer to the following expression using the correct number of significant
18.1 x 0.219
figures. + 12.045
2.7

Solution
The result of the multiplication/division should contain two significant figures (same as
the value 2.7). This rounded-off result is then added to 12.045, with the answer rounded
off according to the rules of addition.

18.1 x 0.219
+ 12.045
2.7
1.468111… + 12.045 (Only the underlined digits, up to the tenths digit
are significant of the first addend.)
1.468 . . . + 12.045 = 13.513 → 13.5
Exercise: What is the answer, with the correct number of significant figures, to the
following arithmetic expressions?
20.3 x 0.1533
104.228

You can now assess yourselves if you understand the mathematical and statistical
procedures that you just read. Consider the assignments given in the following page for your
exercise to reinforce what you learned from my discussions.

22 Statistical Testing for Treatment of Data I –Module 2

 ASSIGNMENT
1. An analysis of the city drinking water for total hardness produced the following results (in
ppm CaCO3): 228.3, 226.4, 226.9, 227.1, and 228.6. Calculate the following.
(a) mean (c) range (e) standard deviation
(b) median (d) average deviation
2. The following pH data were collected in the analysis of water from a certain country in Asia
as part of an acid rain study: 4.17, 4.20, 4.19, 4.23, 4.22, 4.14, 4.20, and 4.15. Calculate
the following:
(a) mean (c) percent average deviation (e) relative standard deviation
(b) median (d) standard deviation
3. If the true concentration of CaCO3 in the drinking water described in Problem 1 is 225.9 ppm,
calculate the absolute and parts per thousand error.
4. An alloy from National Bureau of Standards was analyzed for its chromium content and the
following results were obtained: 2.61, 2.66, 2.61, 2.70, and 2.68% Cr. If the NBS
certified value is 2.69% Cr, what is the absolute error? What is the percent error?
5. Analytical Chemistry student Carl Roger analyzed an ore sample for its copper content and
obtained a mean of 4.67% Cu for three measurements. How many additional
measurements must Carl make to obtain a mean that is five times more reliable?
6. Which of the following data sets has the best precision? Justify your answer by showing
mathematical proof.
A B C
2.31 11.74 56.33
2.33 11.82 56.21
2.30 11.79 56.27
2.30 11.80 56.16

23 Statistical Testing for Treatment of Data I –Module 2

7. Analysis of several plant-food preparations for potassium ion yielded the following data:
Sample Percent K+
1 6.02, 6.04, 5.88, 6.06, 5.82
2 7.48, 7.47, 7.29
3 3.90, 3.96, 4.16, 3.96
4 4.48, 4.65, 4.68, 4.42
5 5.29, 5.13, 5.14, 5.28, 5.20

The preparations were randomly drawn from the same population.

(a) Find the mean and standard deviation s for each sample.
(b) Obtain the pooled value spooled.
(c) Why is spooled a better estimate of σ than the standard deviation from anyone sample?

8. Classify the following as exact or inexact number?

(a) the number of microseconds in a week
(b) the number of millilitres in a cubic meter of water
(c) the mass of a postage stamp
(d) the average mass of the students in the class
(e) the temperature of the loaf bread in the oven

9. (a) What is the length of the pencil in the following figures if the scale reads in
centimeters? How many significant figures are in the measurement? (b) An oven
thermometer with a circular scale reading degrees Fahrenheit is shown. What temperature
does the scale indicate? How many significant figures are in the measurement? (c) The
analytical balance shown can read up to 0.1 mg. Which number is uncertain in the
reading? What place value is this?

24 Statistical Testing for Treatment of Data I –Module 2

 For No. 9 (a) and (b) For No. 9 (c)

10. Indicate the number of significant figures in each of the following measured quantities:
(a) 0.0234 cm2 (b) 5.500 mm (c) 5.404 x 102 km (d) 430.98 (e) 204.080
11. Carry out the following operations, and express the answer with the appropriate number of
significant figures:
(a) 340.55 – (3216.6/2.6) (b) (5.03 x 10-4) (3.8765)
(c) (0.0045 x 20.000.0) + (2813 x 12) (d) 863 x [1255 – (3.45 x 108)]
12. Perform the following conversions:
(a) 0.076 L to mL (b) 1.55 kg/m3 to g/L (c) 5.850 lb/ft3 to g/mL
13. (a) The recommended adult dose of Elixophillin®, a drug used to treat asthma, is 6 mg/kg of
body mass. Calculate the dose in milligrams for a 150-lb person if 1 kg = 2.205 lb (b) A
pound of coffee beans yields 50 cups of coffee (4 cups = 1 qt). How many millilitres of
coffee can be obtained from 1g of coffee beans?

25 Statistical Testing for Treatment of Data I –Module 2

Key Answers for the Check-Up Test
1. C 6. – 0.28
2. B 7. – 0.45%
3. D 8. 2.1
4. C 9. 85
5. B 10. -2.0 %

What you learned

After going through this module, you learned how to compute and
used appropriately the three expressions of central value in order to describe a set of data. You
also learned the difference of accuracy and precision and how they are being expressed. After
following the examples closely and solving the given assignments, you were able to compute
average deviation, standard deviation of the sample and of the population or the pooled standard
deviation.

In the later part of the module, you learned about the difference between exact and
inexact numbers. You also learned that exact number is not governed by the rules on counting
and operations involving significant figures because they have indefinite number of significant
figures. You also learned how to report measured data accurately with the correct number of
significant figures based on the accuracy of the instrument. Further, you now gained techniques
in performing dimensional analysis in converting a given unit into a desired unit.

26 Statistical Testing for Treatment of Data I –Module 2

 References

Brown, T. L., et al. (2012). Chemistry the central science. 12th ed. Illinois: Pearson Education,
Inc.

Chang, R. (1994). Chemistry. 5th ed. New York: Brooks Publishing

Hargis, L.G. (1988). Analytical chemistry principles and techniques. New Jersey: Prentice-Hall,
Inc.

Holler, F. J. and Crouch, S. R. (2014). Skoog and West’s fundamentals of analytical chemistry.
9th ed. USA: Brooks/Cole CENGAGE Learning Inc.

Mann, P. S. (2011). Introductory statistics. New Jersey: John Wiley and Sons, Inc.

End of Module 1

27 Statistical Testing for Treatment of Data I –Module 2