8/10/2015

P-Value
Observed significance level
• P-value (or p-value or probability value)
• The probability of getting a value of the test
HYPOTHESIS TESTING statistic that is at least as extreme as the one
Lecture--2
Lecture representing the sample data, assuming that
the null hypothesis is true.
Niranjan S. Kanaki • Measure of the strength of evidence the
KBIPER sample data provides against the null
hypothesis
• The null hypothesis is rejected if the P-value is
very small, such as 0.05 or less.

P-Value Conclusions
Observed significance level in Hypothesis Testing
• Measure of the strength of evidence the
sample data provides against the null

always test the null hypothesis
hypothesis
• The null hypothesis is rejected if the P-value is
1. Reject the H0
very small, such as 0.05 or less.
2. Fail to reject the H0
P − value : p = P( Z ≥ zobs )

Accept versus Fail to Reject

Wording
Some texts use “accept the null
hypothesis”…this is BAD
the
Final Conclusion
We are not proving the null hypothesis

Sample evidence is not strong enough

to warrant rejection (such as not
enough evidence to convict a suspect)

1
 8/10/2015

Type I Error (α) Type II Error (β

(β)

A Type I error is the mistake of
A Type II error is the mistake of failing
rejecting the null hypothesis when it is to reject the null hypothesis when it is
true. false.

The symbol α (alpha) is used to
The symbol β (beta) is used to

represent the probability of a type I represent the probability of a type II
error. error.

Controlling Type I and

Type II Errors

For any fixed α, an increase in the sample size n
will cause a decrease in β.

For any fixed sample size n , a decrease in α will

cause an increase in β. Conversely, an increase
in α will cause a decrease in β .

To decrease both α and β, increase the sample

size.

Decision Criterion Decision Criterion

Traditional method P-value method

• Reject H0 if the test statistic falls within • Reject H0 if P-value ≤ α (where α is the

the critical region. significance level, such as 0.05).

• Fail to reject H0 if the test statistic does • Fail to reject H0 if P-value > α.

not fall within the critical region.

2
 8/10/2015

Decision Criterion Decision Criterion

Another option Confidence Intervals

• Instead of using a significance level such • Reject H0 if the sample parameter has a

as 0.05, simply identify the P-value and value that is not included in the

leave the decision to the reader. confidence interval.

Testing a single sample mean

Confidence intervals Example
Calculating confidence interval using a z-distribution • A certain breed of rats shows a mean weight
– when population variance is known gain of 65 gm, during the first 3 months of life.
16 of these rats were fed a new diet from birth
until age of 3 months. The mean weight was
60.75 gm. If the population variance is 10 gm ,
Calculating confidence interval using a t-distribution –
when population variance is not known is there a reason to believe at the 5% level of
significance that the new diet causes a change
in the average amount of weight gained.

• Ho: µ=65
Answer
• HA :µ≠ 65
• α=0.05 Zα =1.96 (critical value)
_
x -µ 60.75-65
Z=---------- = ----------- = -5.38
σ √n √10/ √16
Sine the calculated values falls in the rejection
region , we reject the Ho, and accept the HA

3
 8/10/2015

• Find the confidence intervals using z- • In the above example , if the population
distribution. variance is unknown, and the sample Sd is
3.84

Answer
α =5%
Degrees of freedom (df)= n-1= 15
t 1- α/2 =± 2.13
_
x -µ 60.75-65
t =-------------=------------= - 4.1315
s /√n 3.84/ √16
Sine the calculated values falls in the rejection region , we
reject the Ho, and accept the HA

Problem
• Find the confidence intervals using t- • A random sample of 10 boys had the following
distribution. IQ levels:
70, 120, 110, 101, 88, 83, 95, 98, 107, 100

• Do these data support the assumption of a

population mean IQ of 100? Find a range in
which most of the IQ values of the population
lie? (α=5%)

4
 8/10/2015

Problem Problem
• A drug in the powder form is packed into tins • The following data show the blood pressure
by a machine. A random sample of 10 tins is reduction ( in mm Hg) caused in 10 animals by
drawn and their contents are found to weigh a new antihypertensive compound.
in gms as follows:
20, 18, 15, 12, 8, 16, 18, 17, 14, 21
94, 83, 101, 99, 85, 100, 102, 94, 98, 95
• Test the hypothesis that the blood pressure
reduction for the population is 15 mm Hg.
• Test the hypothesis that average content in
the population is 100 gms. Also find the Find 95% fiducial limits for the population
fiducial limits for the drug content at α=5%. mean.

Problem Problem
• A tablet coating machine is designed to produce The data show the lives of a sample of bulbs
sugar coating with average thickness of 0.025 cm. manufactured by a company. The company
A random sample of 10 tablets was found to have
average thickness of 0.024 cm with standard claims that the average life of its bulbs is 90
deviation of 0.002. days. Do the data support company’s claim, at
• Do the data support the hypothesis that the 5% level of significance?
average thickness in the population is 0.025, at Life of bulb (days) No. of bulbs
5% level of significance? 40-55 10
• Find the confidence interval at 5% level of 55-70 12
significance. 70-85 15
• Find the confidence interval at 1% level of 85-100 13
significance. 100-115 10

Comparisons of Means from Two

Example
Independent Groups
• A clinical study involved comparison of an antihypertensive
When population variances of both the groups are known drug and a placebo, where each treatment was tested on
different patients.
• The results of the study showed an average blood pressure
reduction of 10mmHg for 11 patients receiving drug, and an
average reduction of 1 mmHg for 10 patients receiving
placebo.
X1 and X2 = means of two groups • The variances corresponding to drug and placebo are known
to be 144 and 100, respectively.
σ1 and σ2 = populations standard deviations of the two
• Is there a significant reduction in blood pressure by test drug
groups
as compared to placebo at 5% level of significance.
N1 and N2 = sample size of the two groups

5
 8/10/2015

Comparisons of Means from Two Comparisons of Means from Two

Independent Groups Independent Groups
When population variances of the groups are not known
When population variances of the groups are not known,
pooled variance (Sp) is calculated from the sample
variance of each group Compute the value of t statistic (Unpaired t-test)

S1 and S2 = sample standard deviations of the two groups X1 and X2 = means of two groups
N1 and N2 = sample size of the two groups Sp = pooled standard deviation of the two groups
N1 and N2 = sample size of the two groups

Example - Efficacy Test for New drug

Example • A drug company, which has a new antiobesity drug, wishes to
• Two types of chemicals solutions A and B were tested compare it with current standard treatment.
for their pH. • Federal regulators tell company that they must demonstrate that
new drug is better than current treatment to receive approval.
• Analysis of 6 samples of A showed a mean pH of 7.52 • Firm runs clinical trial where some patients receive new drug (A),
with a standard deviation of 0.024. and others receive standard treatment (B).
• The reduction in weight in kg. after using the drugs for six
• Analysis of 5 samples of B showed a mean pH of 7.49 months was recorded.
with a standard deviation of 0.032. Drug A 11 13 12 14 10
• Using a 5% significance level, determine whether the Drug B 12 9 8 15 14 9 10
two types of solutions have different pH values.
• Will the company get approval for its new drug? (α = 0.05)

Problem Comparisons of Means from Two

Sr. Pigeons Rabbits
related samples
• The blood glucose levels
no.
(mg/dl) of pigeons and Paired t-
t-test
rabbits are given in 1 200 145
following table. 2 186 125 • Observations of one sample are related to those
3 176 100 of the other sample (not independent)
• Is there any significant
4 184 112 • The samples are either matched or paired.
difference in their blood
glucose levels at 5% level 5 170 127 • Matched samples : sample items matched by
of significance? 6 172 139 some criteria, eg. Age
7 170 151 • Paired samples : samples from the same object
8 163 140 or individual. Eg. Pre- and post-treatment
9 176 159 observations
10 173 132

6
 8/10/2015

Problem – Paired sample t-

t-test
Paired sample t-
t-test
• A new formulation of a marketed drug is to be tested for
• Formula for computing test statistic, t : bioavailability, comparing the extent of absorption to the
marketed form on six laboratory animals.
• Each animal received both formulations in random order on two
different occasions.
d • The results, the area under the blood level versus time curve
t= (AUC), are shown in Table.
Sd / N • Is there any difference in the bioavailability of the two
formulations at 5% level of significance?

Animal 1 2 3 4 5 6
d = mean of differences of all pairs of observations
A 136 168 160 94 200 174
Sd = standard deviation of differences of all data pairs B 166 184 193 105 198 197
N = sample size

Problem Problem
• Ten individuals had supine systolic blood pressures taken at the • The following data were observed comparing two assays using
beginning and end of a half-an hour rest period. The readings are 12 batches of material. Do both the assays give similar results?
given below. Is there any change in the blood pressure after (α= 5%)
rest? (α= 5%)

Participant 1 2 3 4 5 6 7 8 9 10 Batch 1 2 3 4 5 6 7 8 9 10 11 12
Before 140 136 160 172 154 150 138 146 152 128 Test A 8.1 9.4 7.2 6.3 6.6 9.3 7.6 8.1 8.6 8.3 7.0 7.7
After 132 134 162 170 146 154 142 138 144 124 Test B 9.0 9.9 8.0 6.0 7.9 9.0 7.9 8.3 8.2 8.9 8.3 8.8

7
 8/10/2015

Problem
F-test for equality of two variances • The nicotine content (in mg) of two samples of tobacco were
found to be as follows:
• To find whether two independent samples can
be regarded as drawn from the normal
population with the same variance. Sample A 24 27 26 21 25
Sample B 27 30 28 31 22 36
• H0 : σ12=σ22

Can it be said that the two samples came from the same
F = S 12 / S 22 normal population?

S1 and S2 are standard deviations of two independent

samples
D.f. is n1-1 and n2-1

Problem
• Two granulations were prepared by different procedures. Seven
random samples of powdered mix of equal weight were
collected from each batch and assayed for active material, with
following results:
Granulation A 20.4 20.6 20.6 20.7 21.0 20.9 19.8
Granulation B 20.2 21.0 20.4 19.0 21.5 18.9 21.8

Test whether the two samples can be regarded to have come

from the populations having equal variances.

Chi (χ2)
Chi--Square Test (χ (χ2)
Chi--Square Test (χ
Chi
• For comparison of proportions from two • The chi-square statistic is calculated as
independent groups follows:
• For chi-square test, data is expressed as a
four-fold table (2 x 2 table)

Group I Group II
No. of successes a b • O= Observed frequency in each cell
No. of failures c d
• E= expected frequency in each cell

8
 8/10/2015

Chi (χ2)
Chi--Square Test (χ (χ2)
Chi--Square Test (χ
Chi
Example • The expected number is the number that would result if
Group I Group II Total
each group had same proportion of successes or failures
No. of a= b= a+b= • The common proportion of success (p) is calculated by
successes 16 18 34 following formula:
No. of failures c= d= c+d=
4 12 16 • Pooled p = total no. of successes in both groups
Total a+c= b+d= a+b+c+d= Total no. of trials
20 30 50

= a+b
a+b+c+d

Chi (χ2)
Chi--Square Test (χ (χ2)
Chi--Square Test (χ
Chi
Expected frequencies
Group I Group II Total
No. of 13.6 20.4
successes 34
No. of failures 6.4 9.6 16
Total 20 30 50

Degrees of freedom = (R-1) (C-1)

Where,
R= Number of rows
C=Number of columns

D.f. = 1

Chi--Square distribution table

Chi (χ2)
Chi--Square Test (χ
Chi
Example
• In a preclinical test, the carcinogenicity potential of a new
compound is determined by administering several doses to
different groups of animals.
• A control group (placebo) is included in the study as a
reference.
• The drug treated group showed an incidence of the carcinoma
in 9 of 60 animals (15%).
• The control group exhibited 6 carcinomas in 65 animals
(9.2%).
• Is there a difference in the proportion of animals with the
carcinoma in the two groups ( α=5%)?

9
 8/10/2015

Chi (χ2)
Chi--Square Test (χ (χ2)
Chi--Square Test (χ
Chi
For Goodness of Fit Example
A die is thrown 150 times and the following results are obtained
as shown in the table below. Test the hypothesis that the die is
To investigate the agreement between theoritical
unbiased at 5% level of significance.
distribution and observed distribution
Number turned 1 2 3 4 5 6
up
Frequency 19 23 28 17 32 31

Chi (χ2)
Chi--Square Test (χ Example 1
Example
In an ecological study, water samples were collected in alternate • Suppose you conducted a drug trial on a group of animals and
you hypothesized that the animals receiving the drug would
months to study the phytoplantons populations. The following
show increased heart rates compared to those that did not
table shows the number of organisms per cubic cm in the six receive the drug. You conduct the study and collect the
samples. Test the hypothesis that the number of organisms following data. Test your hypothesis using appropriate
present in each sample does not depend on the particular statistical method.
sample. Heart rate No heart Total
increased rate
Sample 1 2 3 4 5 6 increase
No. of organisms 79 84 102 64 90 85 Treated 36 14 50

Untreated 30 25 55
Total 66 39 105

Example 2 Example 3
A new analytical method is to be compared to an old method.
• In a Mandelian experiment on hybridization of pea plant
The experiment is performed by a single analyst. She selects four
having round yellow seeds with one having wrinkled green
batches of product at random and obtains the following results.
seeds, the following results were obtained in F1 generation.
Test whether the data fits the Mandelian ratio of 9:3:3:1. Do you think that the two methods give different results on the
average?

Round Wrinkled
Yellow 315 101

Green 108 32

10
 8/10/2015

Example 4 Example 5
• Quality control records show the average tablet • Two batches of tablets were prepared by two
weight to be 502 mg with a standard deviation of different processes.
5.3. There are sufficient data so that these values • The potency determinations made on five
may be considered known parameter values.
tablets from each batch were as follows:
• A new batch shows the following weights from a
• Batch A: 5.1, 4.9, 4.6, 5.3, 5.5;
random sample of six tablets: 500, 499, 504, 493,
497, and 495 mg. • Batch B: 4.8, 4.8, 5.2, 5.0, 4.5.
• Do you believe that the new batch has a different • Test to see if the means of the two batches are
mean from the process average? equal.

Example 6 ANALYSIS OF VARIANCE (ANOVA)

• A placebo treatment results in 8 patients out • ANOVA is a general method of analyzing data
of 100 having elevated blood urea nitrogen from designed experiments, whose objective
(BUN) values. is to compare two or more group means.
• The drug treatment results in 16 of 100 • The t test is a special case of ANOVA in which
patients having elevated values. only two means are compared.
• Is this significantly different from the placebo? • Two types:
– One way ANOVA
– Two way ANOVA

One way ANOVA One way ANOVA

• Typical null hypothesis is H0: μ1= μ2= μ3 = … • First calculate the mean of all the values in all
• Alternate hypothesis, H1: At least two of the the groups.
populations means are not equal. • Calculate total sum of squares (SST)

OR by shortcut formula

11
 8/10/2015

One way ANOVA One way ANOVA

• The between-group sum of squares (between • The within-group sum of squares (WSS) is
SS or BSS) is calculated as follows: equal to the difference between the TSS and
BSS.
• TSS = BSS-WSS
– Total degrees of freedom = N-1
Ti = sum of observations in group i and – Between group d.f. = k-1
Ni = the number of observations in group I – Within group d.f. = N-k
CT= correction term =

One way ANOVA One way ANOVA

• Calculate mean squares = sum of squares divided by
d.f.
• Calculate F ratio.
F = Mean squares between groups/mean squares
within groups
• Find Ftab from F-table for d.f. (k-1) and (N-k) at 5%
level of significance.

One way ANOVA One way ANOVA

Results of assay comparing 3 analytical methods. A random sample is selected from each of the three
brands of ropes and their breaking strengths (in kg) are
Method A Method B Method C measured with the following results:
102 99 103
101 100 100 1 35 36 38 40 42
101 99 99 2 50 55 54 56 57 60 54
100 101 104 30 33 29 42 44 37
3
102 98 102
Test whether the breaking strength of the ropes differ
significantly at 5% level of significance?
Test at 5% level of significance that there is no
significant differences among the three methods.

12
 8/10/2015

One way ANOVA Problem

A pharma company wishes to test whether its three The record of weight loss after 8 weeks of following three
salesmen A, B and C tend to make sale of the same size different weight loss programs. Is there a significant
or not. Following are their weekly sales records: difference in the mean weight loss among the diet groups?

A 20 30 20 40 30 Low Calorie Low Fat Low Carbohydrate Control

8 2 3 2
B 50 20 20 30 40
9 4 5 2
C 60 20 30 50 40
6 3 4 -1
7 5 2 0
Determine whether the average sales of three salesmen
3 1 3 3
differ in size.

Two--way ANOVA
Two Two--way ANOVA
Two
• Also known as two-factor ANOVA Steps
• When there are two factors affecting the • Suppose there are h rows and k columns in
values of the response variable. the data.
Examples- • First calculate the row totals R1, R2, … and
– Comparing different analytical methods on column totals C1, C2, ….
different batches of products
• Calculate the grand total T.
– Comparing the effect of treatments on people of
different age groups or gender • Calculate the correction term, C.T.= T2/hk
– Comparing dissolution profile of formulations

Two--way ANOVA
Two Two--way ANOVA
Two
Steps (contd..) Steps (contd..)
• Calculate the total sum of squares, TSS • Calculate the column sum of squares, CSS
TSS=Σy2 - C.T. CSS=ΣCi2/h - C.T.
d.f.= hk-1 d.f.= k-1
• Calculate the row sum of squares, RSS • Calculate the error sum of squares, ESS
RSS=ΣRi2/k - C.T. ESS=TSS-RSS-CSS
d.f.= h-1 d.f.= (h-1)(k-1)

13
 8/10/2015

Two--way ANOVA
Two Two--way ANOVA
Two
Steps (contd..) Steps (contd..)
• Calculate the row mean sum of squares, MSR To test H01: All row means are equal,
MSR=RSS/h-1 calculate Fr.
• Calculate column mean sum of squares, MSC Fr = MSR/MSE
MSC=CSS/k-1 To test H02: All column means are equal,
• Calculate the error mean sum of squares, MSE calculate Fc.
MSE=ESS/(h-1)(k-1) Fc = MSC/MSE

Two--way ANOVA table

Two Two--way ANOVA
Two
Problem
Source of variation d.f. SS MSS F
Due to rows h-1 SSR MSR Fr
Due to columns k-1 SSC MSC Fc
Error (h-1)(k-1) SSE MSE
Total hk-1 SST

14
 8/10/2015

Power of a Test
Problem
• Power - Probability a test rejects H0 (depends on µ1- µ2)
Crop I Crop II Crop III Crop IV – H0 True: Power = P(Type I error) = α
Fertilizer A 6 4 8 6 – H0 False: Power = 1-P(Type II error) = 1-β

Fertilizer B 7 6 6 9
Fertilizer C 8 5 10 9 · Example:
· H0: µ1- µ2 = 0 HA: µ1- µ2 > 0
• σ12 = σ22 = 25 n1 = n2 = 25
· Decision Rule: Reject H0 (at α=0.05 significance level) if:

y1 − y 2 y1 − y 2
z obs = = ≥ 1 .645 ⇒ y 1 − y 2 ≥ 2 .326
σ 12 σ 22 2
+
n1 n2

Power of a Test Power of a Test

• Now suppose in reality that µ1-µ2 = 3.0 (HA is true)
• Power now refers to the probability we (correctly) 2.326− 3
reject the null hypothesis. Note that the sampling Power= P(Y 1 − Y 2 ≥ 2.326) = P(Z ≥ = −0.48) = .6844
1.41
distribution of the difference in sample means is
approximately normal, with mean 3.0 and standard • All else being equal:
deviation (standard error) 1.414.
• As sample sizes increase, power increases
• Decision Rule (from last slide): Conclude population
means differ if the sample mean for group 1 is at least • As population variances decrease, power increases
2.326 higher than the sample mean for group 2
• As the true mean difference increases, power
• Power for this case can be computed as: increases
P (Y 1 − Y 2 ≥ 2.326) Y 1 − Y 2 ~ N (3, 2.0 = 1.414)

15
 8/10/2015

Power of a Test
Power of a Test
Distribution (H0) Distribution (HA)

Power Curves for group sample sizes of 25,50,75,100 and varying true values µ1-µ2
with σ1=σ2=5.
• For given µ1-µ2 , power increases with sample size
• For given sample size, power increases with µ1-µ2

Sample Size Calculations for Fixed Power

• Goal - Choose sample sizes to have a favorable chance of
detecting a clinically meaning difference
• Step 1 - Define an important difference in means:
– Case 1: σ approximated from prior experience or pilot study - dfference
can be stated in units of the data
– Case 2: σ unknown - difference must be stated in units of standard
deviations of the data

µ1 − µ 2
δ=
σ
• Step 2 - Choose the desired power to detect the the clinically meaningful difference (1-
β , typically at least .80). For 2-sided test:

2(zα / 2 + z β )
2

n1 = n2 = 2
δ

16