Lecture 4 Experimental Design 18.9.18ND
Lecture 4 Experimental Design 18.9.18ND
N1 PHYSICS 1
Semester I, 2018-2019
Experimental Design, Error Analysis and Statistics
Learning Objectives
After this lecture you should be able to
1) Appreciate the specific and focused nature of a research question.
2) Design a simple experiment by
I. Accounting for independent, dependent and control variables.
II. Accounting for sources of error by choosing appropriate instruments.
III. Making appropriate assumptions.
IV. Taking appropriate measurements by minimizing errors in measuring a particular
quantity.
V. Making appropriate graphical analyses including error analyses.
3) Write results in appropriate significant figures.
Introduction
Experiments are designed to answer a specific question. The degree of certainty that we can attach
to the answer of that question depends on experimental error.
1. At the beginning of the experiment we evaluate potential sources of errors to ensure that
they will not adversely affect our results and thus allow us to draw firm conclusions about
the experiment.
2. During the experiment we monitor our measurements to ensure that errors remain within an
acceptable range.
3. After the experiment and during the analyses we evaluate the quality of the measurements
and results.
In this section of the course and in the labs, you will be introduced to sources of errors in taking
measurements and statistical methods to quantify errors.
Research questions
Before we design an experiment, we must design our research question first.
A research question is a clear, focused, concise, complex and arguable question around which you
center your research.
A specific research question helps in directing your research. Most times you will have to rewrite
your research question until it is clear, focused, and sufficiently complex.
Let us examine a general interest question, “What is the effect of global warming on the
environment?”
So the question “What is the effect of global warming on the environment?” makes us ask other
questions,
We can focus the research question by looking at a specific cause (glacial melting), a specific group
that is affected (penguins), and a specific location (Arctic circle). A more focused research question
will therefore be: “How is glacial melting affecting penguins in the Arctic Circle?” This is now a
research question that can be used to write a research paper.
Another example is: What factors affect the period of a simple pendulum?
A more focused question is: “Is the period of a simple pendulum related to the length of the
pendulum’s string?”
Identify dependent, independent and control variables and reduce sources of errors
We can devise a procedure in which we can vary the length of the string and for each length of the
string we can record the period. Since we want to measure the effect of length only on period, we
must keep all other possible variables constant. For instance, the mass of the Reducing the
pendulum must be the same. Also, the environment must be the same. If wind source of
speeds around the pendulum varies, the air resistance changes. Ideally, we will try errors.
to vary the length of the pendulum and measure its period when it is swinging in a
vacuum. However, in a laboratory setting we can reduce variations in the environment by
performing the experiment in a laboratory with environmental controls. In this way we reduce the
source of environmental errors in the experiment.
Figure 1: A metre rule with two scales. One scale is subdivided by centimetres and the other in millimetres.
(Source: https://www.jacksonsart.com/jakar-wooden-metre-ruler-0-100cm-on-one-edge-and-0-1000mm-on-
opposite-edge)
In the experimental design you have to choose an appropriate measuring instrument. In the simple pendulum
experiment you can use a metre rule to measure length. You may have to also choose an appropriate
measuring scale. For example, in Figure 1, we can use the scale with the centimeter grading to measure the
length of the pendulum. The uncertainty (error) in that measurement will be 0.5 cm. If however, we use the
scale with the millimeter grading, then the uncertainty reduces to 0.05 cm. The smaller the uncertainty in
length then the smaller the uncertainty in the period of the oscillation, provided that the period is in fact
dependent on the length of the pendulum. The scale with the millimeter grading is said to be more accurate
than the scale with the centimeter grading.
A measurement of a quantity such as length is said to be accurate if it is close to its true value. Multiple
readings of a quantity is required in determining accuracy. If the mean value is close to the true value then the
readings are said to be accurate.
Precision
In addition to obtaining accurate measurements, they must also be precise. A precise measurement is a
measurement where all of the repeated readings are very close to the mean value of the readings.
In Figure 2(a), the measured values are close to the true value, the center, and as such are accurate. The
measurements are also precise because they are clustered closely together. The measured values in Figure 2(b)
are close to each other (precise) but far from the true value (inaccurate). The individual measurements in
Figure 2(c) are far away from the true value but randomly distributed about the true value; the average value
lies close to the true value (accurate but imprecise). In Figure 2(d) the mean value of the measurements are far
from the true value and the measurements are widely dispersed. Therefore, Figure 2(d) represents inaccurate
and imprecise measurements.
The results of the experiments performed by student a are both accurate and precise.
The results of the experiments performed by student B are precise but not accurate.
Worksheet question
Table E.1 shows three sets of results for the measurement of the diameter of a metal rod of true diameter 8.0
mm. Which sets of results are accurate, which are precise and which are both?
Table E.1 - Independent measurements of the diameter of a metal rod by three students
Student a/mm 7.3 8.6 8.5 7.2 7.9 8.1
Student b/mm 8.1 7.9 8.0 7.9 7.9 8.1
Student c/mm 7.7 7.8 7.6 7.8 7.6 7.7
Student b’s values are all very close to the true value so they are accurate. As the values have a low spread
from each other they are precise.
Student c’s values are all less than the true value so the measurements are inaccurate. However, the
measurements are all close to each other so they are precise.
Systematic errors occur when measurements are shifted from their true value by the same amount or fraction
and in the same direction. Systematic errors affect accuracy but not reliability. Systematic errors may occur
because:
Equipment were not calibrated or were poorly calibrated.
The experimenter did not use the instrument properly.
Two types of systematic error can occur with instruments having a linear response (source:
https://www.physics.umd.edu/courses/Phys276/Hill/Information/Notes/ErrorAnalysis.html):
1. Offset or zero setting error in which the instrument does not read zero when the quantity to be measured is zero.
2. Scale factor error in which the instrument consistently reads changes in the quantity to be measured greater or less than the
actual changes.
3. If the experiment research question is flawed in that it does not measure a fair and representative sample.
(source: https://www.physics.umd.edu/courses/Phys276/Hill/Information/Notes/ErrorAnalysis.html)
(a) (b)
To reduce the systematic error of a data set, you must identify the source of the error and remove it.
Unfortunately, unless you do that, you will never reduce the systematic error by taking more measurements.
In the game of darts, if we consistently aims to the right of the spot we want the arrows to hit, the arrows will
cluster around the wrong place; no matter how many shots we take, we will arrive at the wrong conclusion.
(https://www.nap.edu/jhp/oneuniverse/intro_knowledge_concept_5.html).
A common method to remove systematic error is through calibration of the measurement instrument.
Systematic errors can be either constant, or related (e.g. proportional or a percentage) to the actual value of the
measured quantity, or even to the value of a different quantity (the reading of a ruler can be affected by
environmental temperature). When it is constant, it is simply due to incorrect zeroing of the instrument. When it is
not constant, it can change its sign.
Zero reading
Measurement
1.06cm−(+0.03cm)=1.03cm
A simple example is zero error, where the instrument has not been correctly set to zero before
commencing the measuring procedure. An ammeter for instance may show a reading of 0.2A when no
current is flowing. So, as you use the instrument to measure various currents each of your measurements
will be in error by 0.2A. The ammeter needle should have been reset to zero by using the adjusting screw
before the measurements were taken.
Suppose we time the period of a simple pendulum. If our stopwatch is running consistently
slow, then all of our times will be underestimates of the actual period.
Since equipment used in an experiment can only report a measured value with a certain degree of
accuracy, calculating the extent to which a measurement deviates from the value accepted by the
scientific community is often helpful in gaging the accuracy of equipment. Such a calculation is referred
to as the percent error of a measurement and is represented by the following formula:
Example 2 is a real world example to understand the role of percent error calculations in determining
the accuracy of measuring equipment.
EXAMPLE 2
A toy company that ships its products around the world must calculate fuel costs associated with
transporting the weight of their standard 2 by 3 foot box. To predict shipping costs and create a
reasonable budget, the company must obtain accurate mass measurements of their boxes. The
accepted mass of a standard box is 0.525 kg. The company measures a sample of three dozen boxes
with a sophisticated electronic scale and an analog scale each yielding an average mass of 0.531 kg
and 0.49 kg, respectively. A calculation of percent error for each device yields the following results:
Immediately, one notices that the electronic scale yields a far more accurate measurement with a
percent error almost six times lower than the measurement obtained from the analog scale. Also note
that percent error may take on a negative value as illustrated by the calculation for the analog scale.
This simply indicates that the measured average lies 6.67% below the accepted value. Conversely, a
positive percent error indicates that the measured average is higher than the accepted value.
For instance, if a thermometer is affected by a proportional systematic error equal to 2% of the actual temperature,
and the actual temperature is 200°, 0°, or −100°, the measured temperature will be 204° (systematic error = +4°), 0°
(null systematic error) or −102° (systematic error = −2°), respectively. Thus, the temperature will be overestimated
when it will be above zero, and underestimated when it will be below zero.
Random errors shift each measurement from its true value by a random amount and in a random direction.
Random errors affect the precision of a measurement but not necessarily the accuracy of the measurement.
What about human error? Don’t ever use “human error” as an answer in questions. You must classify errors as
random or systematic and identify the source of the error.
Drawing a line of best fit helps to reduce random error in the final experimental result. First outliers can be
eliminated. Secondly, the line of best fit is drawn to accommodate as much of the data as possible by cutting
in between the set of data points. In this way, the data is averaged, with most weighting given to the most
similar values. This reduces the effects of random error and increases reliability.
You can increase the amount of information you get from your measurements by taking a number of readings
and carrying out some basic statistical calculations. The two most important statistical calculations are the:
Taking the average of a number of readings to get the best estimate of a reading
If there is variation in readings when they are repeated, then taking an average will give you an estimate of the
‘true’ value. Figure 3 below shows a set of repeated values and their mean value.
Figure 3: A ‘blob plot’ showing a set of readings and their mean value.
Suppose we have the following three readings for the length of a page whose true value is 28.95 cm, all taken
under identical conditions: 𝑙1 = 27.90 cm, 𝑙2 = 27.95 cm and 𝑙3 = 28.00 cm. The average of these three readings
𝑙 is found by adding all the values together and then dividing the total obtained by the number of readings:
In this example, all the individual readings 𝑙1 = 27.90 cm, 𝑙2 = 27.95 cm and 𝑙3 = 28.00 cm are close to the
mean value of 𝑙 = 27.95 𝑐𝑚, and as such the individual readings are said to be precise. Since the mean value
𝑙 = 27.95 𝑐𝑚 coincides with (or close to) the true value of 28.95 cm, the readings are said to be accurate.
Example
The temperature of a room was measured five different times and the following measurements were obtained:
Temperature
T (0C)
23.1
22.5
21.9
22.8
22.5
But performing more readings takes extra effort, and yields ‘diminishing returns’. What is a good number? Ten
is a popular choice because it makes the arithmetic easy. Using 20 would only give a slightly better estimate
than 10. Using more than 20 measurements would not improve the estimate significantly better than 20. As a
rule of thumb usually between 4 and 10 readings is sufficient. Often in the lab we will use three repeated
measurements as a compromise on the time to complete the lab.
In mathematics, the sum of multiple values of the same quantity has a symbol called the summation sign, Σ.
We can represent 𝑥1 + 𝑥2 + 𝑥3 + ⋯ + 𝑥𝑁 as ∑𝑁 𝑁
𝑖=1 𝑥𝑖 . So ∑𝑖=1 𝑥𝑖 tells us to sum all 𝑥𝑖 where 𝑖 takes the value from 1
to 𝑁.
∑𝑁
𝑖=1 𝑥𝑖
𝑥=
𝑁
or
𝑁
1
𝑥= ∑ 𝑥𝑖 .
𝑁
𝑖=1
The standard deviation is a measure of the average deviation of our measurements from our best estimate, the
average of the measurements.
Some books call this uncertainty, the error. Note that error is the difference between the true value and the
measured value. However, most times we do not know the true value so we take multiple measurements and
an average to provide a best estimate of the true value. Uncertainty, however, characterizes the range of values
within which the true value may possibly lie with some level of confidence.
The standard error is smaller than the standard deviation by a factor of 1/√𝑁. This reflects the fact that we
expect the uncertainty of the average value to get smaller when we use a larger number of measurements, N.
The standard error is an indication of the reliability of the average of the sample of measurements as
compared to the mean of all possible measurements (population mean).
𝑚𝑒𝑎𝑠𝑢𝑟𝑚𝑒𝑛𝑡 = 𝑥 ± 𝜎𝑋 .
Example
Suppose you measure the oscillation period of a pendulum with a stopwatch five times and recorded your
measurements in the following table:
Trial number, i 1 2 3 4 5
Measured 4.8 4.8 4.5 4.3 4.6
value, 𝑥𝑖
(seconds)
Determine the:
(i) average time for the oscillation period. The average time is also our best estimate of the oscillation period.
(ii) average deviation of the measurements from the best estimate,
(iii) the standard deviation or the width of the distribution of the measurements,
Dr. Xsitaaz Chadee
Department of Physics, UWI, St. Augustine
Astravanam 12
(iv) the standard deviation of the mean, and
(v) the final expression of the average time with its associated “error”.
Solution
(i) The average time for the oscillation period is:
(4.8 + 4.8 + 4.5 + 4.3 + 4.6)
𝑇= 𝑠 = 4.6 𝑠.
5
(ii) Let us add another row to the table to compute the deviation of each measurement from the average
oscillation period:
Trial number, i 1 2 3 4 5
Measured value, 𝑥𝑖 4.8 4.8 4.5 4.3 4.6
(seconds)
Deviation, 0.2 0.2 -0.1 -0.3 0
𝑑𝑖 = 𝑥𝑖 − 𝑥
Note that the average deviation is zero. Clearly, the average of deviations cannot be used as an error estimate.
In fact, the definition of the average ensured that the average deviation is always zero for any set of
measurements. As some measurements are larger than the average value they produce positive deviations. All
other measurements will be smaller than the average value and will produce negative deviations. The sum of
all these positive and negative deviations will always give zero regardless of the number of measurements that
are taken of the same physical quantity.
The standard deviation gets rid of the negative deviations that cancel the positive deviations in calculating the
mean deviation. It does this by squaring all deviations. Remember a square of a real number is always positive.
Let us calculate the standard deviation now.
(iii) In calculating the standard deviation, we will require the average value. This was calculate in part (i). The
next step is to calculate the deviation of each measurement from the average value, which was done in part
(ii). The third step is to square the deviations. Let us add another row to the table above for these squared
deviations.
Trial number, i 1 2 3 4 5
Measured value, 𝑥𝑖 4.8 4.8 4.5 4.3 4.6
(seconds)
Deviation, 0.2 0.2 -0.1 -0.3 0
𝑑𝑖 = 𝑥𝑖 − 𝑥
Squared deviation 0.04 0.04 0.01 0.09 0
𝑑𝑖2 = (𝑥𝑖 − 𝑥)2
Since our measurements of the period of the oscillation is up to one decimal place, we will express the
standard deviation to also one decimal place.
Note: The standard deviation has the same units as the original measurements.
The standard deviation tells us the average spread or dispersion of experimental results.
1
12
𝜎𝑋 = [ ] 𝜎
5
0.212
𝜎𝑋 = 1 = 0.0948 𝑠
52
𝜎𝑋 = 0.1 𝑠 𝑡𝑜 1 𝑑𝑒𝑐𝑖𝑚𝑎𝑙 𝑝𝑙𝑎𝑐𝑒
(v) Now we can express the average oscillation period of the pendulum as 𝑥 ± 𝜎𝑋 :
𝑇 = 4.6 𝑠 ± 0.1 𝑠.
For more information on standard deviation and standard error, look at the following youtube video:
https://www.youtube.com/watch?v=3UPYpOLeRJg by Kevin Piers.
Tutorial questions
(1) Two student perform an experiment to determine the density of water which is known to be 1 g cm -3. The
students repeat the experiment several times and the density is recorded.
Student A/ g cm-3 0.90 0.95 0.94 0.93 0.96 0.92
Student B/ g cm-3 1.05 1.03 0.97 0.95 1.01 0.99
Which sets of results are accurate, which are precise and which are both?
Where 𝑥 is the best estimate and 𝜎𝑋 is the estimate of the error, then:
For example, the best estimate of a length is 2.59cm, but due to uncertainty, the length might be as small as
2.57cm or as large as 2.61cm.
L can be expressed with its uncertainty in two different ways:
1. Absolute Uncertainty Expressed in the units of the measured quantity: 2.59 ± 0.02 cm
2. Fractional Uncertainty. Since 0.02/2.59 x 100 = 0.0077 ≈ 0.01
3. Percentage uncertainty: 1%
4. we would write L = 2.59 ± 0.02 cm and L = 2.59 cm ± 1%
You already know the convention for reporting your result with an absolute error
Suppose you also regularly monitor the mass of your dog. Your last reading for the dog's mass M, with absolute error
included, is
Let's try it on our dog example. For the length we should divide 3 cm by 85 cm. We get 0.04 after rounding to one
significant digit. For the mass we should divide 1 kg by 20 kg and get 0.05. Note that in both cases the physical units
cancel in the ratio. Thus, relative error is just a number; it does not have physical units associated with it. Moreover, it's
not just some number; if you multiply it by 100, it tells you your error as a percent. Our measurement of the dog's length
has a 4% error; whereas our measurement of the dog's mass has a 5% error. Well, now we can make a direct
comparison. We conclude that the length measurement is more precise.
Finally, let us see what the convention is for reporting relative error. For our dog example, we can write down the results
as follows
The first way of writing is the familiar result with absolute error, and the second and third ways are equally acceptable
ways of writing the result with relative error. (Writing the result in the parentheses form might seem a little bit awkward, but
it will turn out to be useful later.) Note that no matter how you write your result, the information in both cases is the same.
Moreover, you should be able to convert one way of writing into another. You know already how to convert absolute error
to relative error. To convert relative error to absolute error, simply multiply the relative error by the measured value. For
example, we recover 1 kg by multiplying 0.05 by 20 kg.
Thus, relative error is useful for comparing the precision of different measurements. It also makes error propagation
calculations much simpler, as you will see in the next chapter.