Computing 90% Confidence Intervals with Small Samples

(From pg 134-135)

This shows the steps on pages 134-135 for computing the upper and lower bounds
of a 90% confidence interval. The "t-stat" is used for small samples. Note that two
excel formulas are introduced here that give the same answers as the steps in the
book: VAR() and TINV(). Th will slightly simplify the calculation. You can use this
spreadsheet to compute ranges for larger samples, too. Just add however many
rows you need and don't forget to change the references in the cells (i.e. "B5:B9")
to capture all the new rows. See how close the answers would be if you just used
the "Mathless" table on the next page.
Squared Differences
Sample Value (step 1.b in book)
1.4 0
1.4 0
1.5 0.01
1.6 0.0400000000000001
1.1 0.0899999999999999

90% Confidence Interval Calculations (by steps described in book)

Average of Sample Values (step 1.a in book) 1.4
Sample Variance (step 1.c in book) 0.035
Population Variance computed with Excel "Var()" formula
(gives the same answer as step 1.c) 0.035
SD of the estimate of the mean with Excel "=stdev()"
formula 0.0836660026534076
t-stat using Excel's "Tinv()" formula (same answer as step 3
in book) 2.13184678632665
Sample error (step 4 in book) 0.178363098881464
Computing bounds for 90% CI (step 5 in book)
Upper bound 1.57836309888146
Lower bound 1.22163690111854
 The "Mathless" 90% Confidence Interval Table
(Exhibit 9.3)

This is just another copy of exhibit 9.3 from the book so you can
quickly compare it to the answers you get in "JellyBeanTstat" page
in the spreadsheel. This is a much simpler approximation than the
t-stat method. As described in the book, you simply use the table to
determine which values in the sample approximate the 90% CI. For
example, if you have 11 random samples, simply take the 3rd largest
and 3rd smallest values to approximate the upper and lower
bounds.
sample size largest/smallest Actual confidence
5 1st 93.8%
8 2nd 93.0%
11 3rd 93.5%
13 4th 90.8%
16 5th 92.3%
18 6th 90.4%
21 7th 92.2%
23 8th 90.7%
26 9th 92.4%
28 10th 91.3%
30 11th 90.1%
 Chapter 9: Computing 90% Confidence Intervals with Small Samples

Example follows steps on Pages 202-203 of the 3rd edition of How to Measure Anything

This example follows the steps shown on pages 202-203 of the 3rd edition How to Measure
Anything for computing the upper and lower bounds of a 90% confidence interval. The "t-stat"
is used for small samples. Note that two excel formulas are introduced here that give the same
answers as the steps in the book: VAR() and TINV(). This will slightly simplify the calculation.
You can use this spreadsheet to compute ranges for larger samples, too. Just add however
many rows you need and don't forget to change the references in the cells (i.e. "C5:C9") to
capture all the new rows.

90% Confidence Interval Calculations (by steps described in book)

Average of sample values (Step 1.a in book) 8.825
Population Variance 7.53666666666667
SD of the estimate of the mean 0.686324753062766
t-stat using Excel's "Tinv()" formula (same answer as Step 3 in book) 1.75305035569257
Sample error (Step 4 in book) 1.2031618524773
Computing bounds for 90% CI (step 5 in book)
Upper bound 10.0281618524773
Lower bound 7.6218381475227

Data

Data Point Sample Value

1 3.2
2 5.8
3 6.5
4 6.5
5 7.1
6 7.5
7 8.4
8 8.9
9 9
10 9.5
11 10.1
12 10.2
13 10.5
14 11.3
15 12.2
16 14.5
 Distribu-
tion
Sample
Value
Lower
Bound

2 4 6 8 10 12 14 16 18 20

<-- Be sure ranges of these values cover whole sample set

Std Error for

Estimate of cpf for Incremental
Sample Confidence Interval Bars
Mean Sample P
(Normalized)

-4.2 1.662376E-05 7.6218381475 0

-4.1 2.560882E-05 8.985053E-06 6.011069 7.6218381475 0.009
-4 3.90756E-05 1.346678E-05 6.079701 10.028161852 0
-3.9 5.905891E-05 1.998332E-05 6.148333 10.028161852 0.009
-3.8 8.841729E-05 2.935837E-05 6.216966
-3.7 0.00013112 4.270287E-05 6.285598
-3.6 0.000192616 6.149542E-05 6.354231
-3.5 0.000280293 8.76777E-05 6.422863
-3.4 0.000404058 0.0001237645 6.491496
-3.3 0.000577025 0.0001729672 6.560128
-3.2 0.000816352 0.0002393273 6.628761
-3.1 0.001144207 0.0003278545 6.697393
-3 0.00158887 0.0004446628 6.766026
-2.9 0.002185961 0.0005970918 6.834658
-2.8 0.002979763 0.0007938018 6.903291
-2.7 0.004024589 0.0010448253 6.971923
-2.6 0.005386146 0.0013615574 7.040556
-2.5 0.007142811 0.0017566648 7.109188
-2.4 0.009386706 0.0022438948 7.177821
-2.3 0.012224473 0.0028377671 7.246453
-2.2 0.015777607 0.0035531347 7.315086
-2.1 0.020182215 0.004404608 7.383718
-2 0.02558806 0.0054058441 7.45235
-1.9 0.032156775 0.0065687153 7.520983
-1.8 0.040059157 0.0079023821 7.589615
-1.7 0.049471468 0.0094123112 7.658248
-1.6 0.060570758 0.01109929 7.72688
-1.5 0.07352926 0.0129585016 7.795513
-1.4 0.088507991 0.0149787318 7.864145
-1.3 0.105649774 0.0171417822 7.932778
-1.2 0.125071936 0.019422162 8.00141
-1.1 0.146859056 0.0217871207 8.070043
-1 0.171056126 0.0241970699 8.138675
-0.9 0.197662543 0.0266064168 8.207308
-0.8 0.226627352 0.0289648093 8.27594
-0.7 0.257846111 0.0312187584 8.344573
-0.6 0.291159687 0.033313576 8.413205
-0.5 0.32635522 0.0351955335 8.481838
-0.4 0.363169349 0.0368141285 8.55047
-0.3 0.401293674 0.0381243255 8.619103
-0.2 0.440382308 0.0390886333 8.687735
-0.1 0.480061194 0.0396788865 8.756368
0 0.519938806 0.0398776117 8.825
0.1 0.559617692 0.0396788865 8.893632
0.2 0.598706326 0.0390886333 8.962265
0.3 0.636830651 0.0381243255 9.030897
0.4 0.67364478 0.0368141285 9.09953
0.5 0.708840313 0.0351955335 9.168162
0.6 0.742153889 0.033313576 9.236795
0.7 0.773372648 0.0312187584 9.305427
0.8 0.802337457 0.0289648093 9.37406
0.9 0.828943874 0.0266064168 9.442692
1 0.853140944 0.0241970699 9.511325
1.1 0.874928064 0.0217871207 9.579957
1.2 0.894350226 0.019422162 9.64859
1.3 0.911492009 0.0171417822 9.717222
1.4 0.92647074 0.0149787318 9.785855
1.5 0.939429242 0.0129585016 9.854487
1.6 0.950528532 0.01109929 9.92312
1.7 0.959940843 0.0094123112 9.991752
1.8 0.967843225 0.0079023821 10.06038
1.9 0.97441194 0.0065687153 10.12902
2 0.979817785 0.0054058441 10.19765
2.1 0.984222393 0.004404608 10.26628
2.2 0.987775527 0.0035531347 10.33491
2.3 0.990613294 0.0028377671 10.40355
2.4 0.992857189 0.0022438948 10.47218
2.5 0.994613854 0.0017566648 10.54081
2.6 0.995975411 0.0013615574 10.60944
2.7 0.997020237 0.0010448253 10.67808
2.8 0.997814039 0.0007938018 10.74671
2.9 0.99841113 0.0005970918 10.81534
3 0.998855793 0.0004446628 10.88397
3.1 0.999183648 0.0003278545 10.95261
3.2 0.999422975 0.0002393273 11.02124
3.3 0.999595942 0.0001729672 11.08987
3.4 0.999719707 0.0001237645 11.1585
3.5 0.999807384 8.76777E-05 11.22714
3.6 0.99986888 6.149542E-05 11.29577
3.7 0.999911583 4.270287E-05 11.3644
3.8 0.999940941 2.935837E-05 11.43303
3.9 0.999960924 1.998332E-05 11.50167
4 0.999974391 1.346678E-05 11.5703
4.1 0.999983376 8.985053E-06 11.63893
4.2 0.999989311 5.935238E-06 11.70756
 Distribu-
tion
Sample
Value
Lower
Bound

20
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
 Chapter 9 Example: Computing 90% Confidence with
Catch/Recatch
Example starts on Page 214 of the 3rd edition of How to Measure Anything

This shows the calculations for the estimation of the size of a population
when you can't actually count or even see the entire population.

90% Confidence Interval Calculations (by steps described in book)

First sample size (i.e. the fish that were tagged) 1000
The second sample size 1000
Overlap between samples (i.e. the tagged fish that
50
showed up in the second sample)
Percentage overlap 0.05
Sample Variance 0.0475
SD of the estimate of the mean with Excel "=stdev()"
0.00689202437604511
formula
t-stat using Excel's "Tinv()" formula 1.64637881728548
Sample error (Step 4 in book) 0.0113468829409359

Computing bounds for 90% CI for the percentage of tagged fish

Upper bound 6.13%
Lower bound 3.87%

Computing bounds for 90% CI for the total population of fish

Upper bound 25,871
Lower bound 16,301
 Chapter 9 Example: Customer Support Training Experiment
Exhibit 9.10, Page 229- 3rd edition of How to Measure Anything

This controlled experiment example compares each of three groups - the test group, the control
group and the original baseline of both groups before the additional training started. The experiment
showed that the training improved the results in the test group compared to the control group. But
what is the chance this is a random fluke? The three values under "Chance the population order is
different" shows the probability that the test group could really be equal to or worse than the control
group, even though the experiment showed otherwise. We also check the control vs. the baseline
and the test vs. the baseline. This shows us that the control and baseline were virtually identical (it is
almost a coin flip which one is higher) and that the test group is also a significant improvement on
the baseline. It is highly unlikely that the difference is due to chance alone or due to factors other
than the new training program.

Sample Size Mean Variance

Test Group 30 2.433 0.392
Control 85 2.094 0.682
Original baseline of both groups
(before training started) 115 2.087 0.659

Variance of
Chance the
Difference Difference
population order is
Between Means between
different
means
Test vs. Control 0.34 0.145199619328 1.0%
Control vs. Baseline 0.01 0.11725428952 47.6%
Test vs. Baseline 0.35 0.137098166133 0.6%
 Chapter 9 Example: Television Ratings Regression Example
Example starts on Page 236 of the 3rd edition of How to Measure Anything

This is the television ratings regression example shown on pages 236-242 of the 3rd
edition of How to Measure Anything. It shows how simple Excel formulas compare to
Excel's regression wizard (see below). These simple Excel formulas only work for
situations where we are correlating a single variable to another. The regression wizard,
however, can work for multiple variables (as explained in the example in the book).
This shows a correction to the book where the slope and intercept were different
because column references in the Excel function were reversed.

Excel Formulas for Simple (one variable) regression

Correlation of ratings to promotion period 0.702952136634

Slope: Additional ratings points for one extra week of promotion 0.215453402729

Intercept: Estimated ratings points if no time was spent on promotion 0.877331144348

Enter X to estimate Y 3 <= INPUT

Upper bound of y estimate 2.42

Mean of y estimate 1.523691352536

Lower bound of y estimate 0.63

Duration of
Promotion Ratings Points
Period (weeks) Below is the "summary output" table generated by the regression
"Analysis Toolpack" add on in Excel).
2 0.3
SUMMARY
1 1.3
1 1.0 Regression Statistics
1 0.9 Multiple R 0.702952136634
3 1.6 R Square 0.494141706399
2 1.3 Adjusted R Square 0.474685618183
4 1.9 Standard Error 0.543101204661
1 0.3 Observations 28
4 2.2
3 2.1 ANOVA
4 1.2 df SS
5 2.1 Regression 1 7.491305636
3 2.5 Residual 26 7.668931881
5 1.8 Total 27 15.16023752
5 2.3
6 2.5 Coefficients Standard Error
6 2.4 Intercept 0.877331144348 0.226161926
6 1.9 Duration of Promo 0.215453402729 0.042751895
6 1.5
6 2.6
8 2.1
6 2.5
7 3.3
7 1.3
7 1.8
7 2.4
9 3.0
9 2.7
 Ratings Points for the Program

12

10

0
0 2 4 6 8 10 12
Duration of Promotion Period (Weeks)

enerated by the regression wizard in Excel's data analysis tool (available if you set up the

SUMMARY OUTPUT

MS F Significance F
7.491306 25.39779 3.027E-05
0.294959
 t Stat P-value Lower 95% Upper 95% Lower 95.0% Upper 95.0%
3.879217 0.00064 0.41244865 1.34221364 0.4124486471 1.3422136416
5.039622 3.03E-05 0.12757562 0.30333118 0.1275756249 0.3033311806