Chapter 2: Simple

Comparative Experiments

1
Design of Engineering Experiments
Chapter 2 – Some Basic Statistical Concepts
 Describing sample data
 Random samples
 Sample mean, variance, standard deviation
 Populations versus samples
 Population mean, variance, standard deviation
 Estimating parameters
 Simple comparative experiments
 The hypothesis testing framework
 The two-sample t-test
 Checking assumptions, validity
Portland Cement Formulation (page 26)
Graphical View of the Data
Dot Diagram, Fig. 2.1, pp. 26
If you have a large sample, a histogram
may be useful
Box Plots, Fig. 2.3, pp. 28
The Hypothesis Testing Framework

 Statistical hypothesis testing is a useful

framework for many experimental situations
 Origins of the methodology date from the
early 1900s
 We will use a procedure known as the two-
sample t-test
 Let’s review the one sample t-test first
Five Steps of Hypothesis Testing:
 (1)

 (2)

 (3)

 (4)

 (5)
Example
A random sample of n=8 E-glass fiber test specimens of
a certain type yielded a sample mean interfacial shear
yield stress of 30.5 and a sample standard deviation of
3.0

9
The Hypothesis Testing Framework: Two Sample Tests

 Sampling from a normal distribution

 Statistical hypotheses: H 0 : µ1 = µ 2
H1 : µ1 ≠ µ 2
Summary Statistics (pg. 38)

Modified Mortar Unmodified

Mortar
“New recipe”
“Original recipe”
y1 = 16.76
y2 = 17.04
S = 0.100
2

S 22 = 0.061
1

S1 = 0.316
S 2 = 0.248
n1 = 10
n2 = 10
Two-Sample Test (Known Variance):

Use the sample means to draw inferences about the population means
y1 − y2 =16.76 − 17.04 =
−0.28
Difference in sample means
Standard deviation of the difference in sample means

σ2 σ 12 σ 22
σ =
2
y , and σ 2
y1 − y2 = + , y1 and y2 independent
n n1 n2
This suggests a statistic:
y1 − y2
Z0 =
σ 12 σ 22
+
n1 n2
If the variances were known we could use the normal distribution as the basis of a test
Z0 has a N(0,1) distribution if the two population means are equal
 If we knew the two variances how would we use Z0 to test H0?

Suppose that σ1 = σ2 = 0.30. Then we can calculate

y1 − y2 −0.28 −0.28
Z0 = = = = −2.09
σ 12 σ 22 0.32 0.32 0.1342
+ +
n1 n2 10 10

How “unusual” is the value Z0 = -2.09 if the two population means are equal?
It turns out that 95% of the area under the standard normal curve (probability)
falls between the values Z0.025 = 1.96 and - Z0.025 = -1.96.
So the value Z0 = -2.09 is pretty unusual in that it would happen less that 5%
of the time if the population means were equal
Standard Normal Table (see appendix)

Z0.025 = 1.96
So if the variances were known we would conclude that we should reject
the null hypothesis at the 5% level of significance
H 0 : µ1 = µ 2
H1 : µ1 ≠ µ 2
and conclude that the alternative hypothesis is true.
This is called a fixed significance level test, because we compare the value
of the test statistic to a critical value (1.96) that we selected in advance
before running the experiment.
The standard normal distribution is the reference distribution for the test.
Another way to do this that is very popular is to use the P-value approach.
The P-value can be thought of as the observed significance level.
For the Z-test it is easy to find the P-value.
Normal Table

Find the probability above Z0 =

-2.09 from the table.
This is 1 – 0.98169 = 0.01832
The P-value is twice this
probability, or 0.03662.
So we would reject the null
hypothesis at any level of
significance that is less than or
equal to 0.03662.
Typically 0.05 is used as the
cutoff.
The t -Test
 The Z-test just described would work perfectly if we knew
the two population variances.
 Since we usually don’t know the true population variances,
what would happen if we just plugged in the sample
variances?
 The answer is that if the sample sizes were large enough
(say both n > 30 or 40) the Z-test would work just fine,
even though the population variance is unknown. It is a
good large-sample test for the difference in means.
 But many times that isn’t possible (as Gosset wrote in
1908, “…but what if the sample size is small…?).
 It turns out that if the sample size is small we can no
longer use the N(0,1) distribution as the reference
distribution for the test.
Two-Sample t-Test (small sample and
unknown variance):
Use S and S to estimate σ and σ
1
2 2
2
2
1
2
2

y1 − y2
The previous ratio becomes
2 2
S S
1
+ 2
n1 n2
However, we have the case where σ= σ= σ 2
1
2
2
2

Pool the individual sample variances:

( n − 1) S 2
+ ( n − 1) S 2
Sp = 1
2 1 2 2
n1 + n2 − 2
How the Two-Sample t-Test Works:
The test statistic is
y1 − y2
t0 =
1 1
Sp +
n1 n2
 Values of t0 that are near zero are consistent with the
null hypothesis
 Values of t0 that are very different from zero are
consistent with the alternative hypothesis
 t0 is a “distance” measure-how far apart the averages
are expressed in standard deviation units
 Notice the interpretation of t0 as a signal-to-noise
ratio
The Two-Sample (Pooled) t-Test

(n1 − 1) S12 + (n2 − 1) S 22 9(0.100) + 9(0.061)

S 2
= = 0.081
n1 + n2 − 2 10 + 10 − 2
p

S p = 0.284

y1 − y2 16.76 − 17.04
t0 = = = −2.20
1 1 1 1
Sp + 0.284 +
n1 n2 10 10

The two sample means are a little over two standard deviations apart
Is this a "large" difference?
 The Two-Sample (Pooled) t-Test

 We need an objective
t0 = -2.20
basis for deciding how
large the test statistic t0
really is
 In 1908, W. S. Gosset
derived the reference
distribution for t0 …
called the t distribution
 Tables of the t
distribution – see
textbook appendix page
614
 The Two-Sample (Pooled) t-Test
 A value of t0 between
–2.101 and 2.101 is
t0 = -2.20
consistent with
equality of means
 It is possible for the
means to be equal
and t0 to exceed
either 2.101 or –
2.101, but it would be
a “rare event” …
leads to the
conclusion that the
means are different
 Could also use the
P-value approach
The Two-Sample (Pooled) t-Test

t0 = -2.20

 The P-value is the area (probability) in the tails of the t-distribution beyond -2.20 +
the probability beyond +2.20 (it’s a two-sided test)
 The P-value is a measure of how unusual the value of the test statistic is given that
the null hypothesis is true
 The P-value the risk of wrongly rejecting the null hypothesis of equal means (it
measures rareness of the event)
 The exact P-value in our problem is P = 0.042 (found from a computer)
Approximating the P-value
Our t-table only gives probabilities greater than positive values of t. So
take the absolute value of t0 = -2.20 or |t0|= 2.20.
Now with 18 degrees of freedom, find the values of t in the table that
bracket this value.
These are 2.101 < |t0|= 2.20 < 2.552. The right-tail probability for t =
2.101 is 0.025 and for t = 2.552 is 0.01. Double these probabilities
because this is a two-sided test.
Therefore the P-value must lie between these two probabilities, or
0.05 < P-value < 0.02
These are upper and lower bounds on the P-value.
We know that the actual P-value is 0.042.
Computer Two-Sample t-Test Results
Checking Assumptions –
The Normal Probability Plot
Importance of the t-Test

 Provides an objective framework for simple

comparative experiments
 Could be used to test all relevant hypotheses
in a two-level factorial design, because all of
these hypotheses involve the mean response
at one “side” of the cube versus the mean
response at the opposite “side” of the cube
Confidence Intervals (See pg. 43)
 Hypothesis testing gives an objective
statement concerning the difference in
means, but it doesn’t specify “how different”
they are
 General form of a confidence interval
L ≤ θ ≤ U where P ( L ≤ θ ≤ U ) =1 − α
 The 100(1- α)% confidence interval on the
difference in two means:
y1 − y2 − tα / 2,n1 + n2 − 2 S p (1/ n1 ) + (1/ n2 ) ≤ µ1 − µ 2 ≤
y1 − y2 + tα / 2,n1 + n2 − 2 S p (1/ n1 ) + (1/ n2 )
What if the Two Variances are Different?