1.

Probabilities for events

For events A, B, and C P (A ∪ B) = P (A) + P (B) − P (A ∩ B)

S P P P
More generally P( Ai ) = P (Ai ) − P (Ai ∩ Aj ) + P (Ai ∩ Aj ∩ Ak ) − ∙ ∙ ∙

The odds in favour of A P (A) / P (A)

P (A ∩ B)
Conditional probability P (A B) = provided that P (B) > 0
P (B)
Chain rule P (A ∩ B ∩ C) = P (A) P (B A) P (C A ∩ B)
P (A) P (B A)
Bayes’ rule P (A B) =
P (A) P (B A) + P (A) P (B A)
A and B are independent if P (B A) = P (B)

A, B, and C are independent if P (A ∩ B ∩ C) = P (A)P (B)P (C) , and

P (A ∩ B) = P (A)P (B) , P (B ∩ C) = P (B)P (C) , P (C ∩ A) = P (C)P (A)

2. Probability distribution, expectation and variance

The probability distribution for a discrete random variable X is the complete set of
probabilities {px } = {P (X = x)}
X
Expectation E(X) = μ = xpx
x
1 X
Sample mean x = xk estimates μ from random sample x1 , x2 , . . . , xn
n k
X
Variance var (X) = σ 2 = E{(X − μ)2 } = E(X 2 ) − μ2 , where E(X 2 ) = x2 px
x
X X 2 
1 1
Sample variance s2 = x2k − xj estimates σ 2
n−1 k n j
Standard deviation sd (X) = σ
If value y is observed with frequency ny
X X X X X
n= ny , xk = yny , x2k = y 2 ny
y k y k y
X
For function g(x) of x, E{g(X)} = g(x)px
x
   
X −μ 3 1 X xi − x 3
Skewness β1 = E is estimated by
σ n−1 s
   
X −μ 4 1 X xi − x 4
Kurtosis β2 = E −3 is estimated by −3
σ n−1 s
 Sample median xe . If the sample values x1 , . . . , xn are ordered x(1) ≤ x(2) ≤ ∙ ∙ ∙ ≤ x(n)
1
xe = x( n+1 ) if n is odd, and xe = 2
(x( n2 ) + x( n+2 ) ) if n is even.
2 2

α-quantile Q(α) is such that P (X ≤ Q(α)) = α

b
Sample α-quantile Q(α) b
is the sample value for which the proportion of values ≤ Q(α) is
α (using linear interpolation between values on either side)
The sample median xe estimates the population median Q(0.5).

3. Probability distribution for a continuous random variable

Z x
The cumulative distribution function (cdf) F (x) = P (X ≤ x) = f (x0 )dx0
x0 =−∞
dF (x)
The probability density function (pdf) f (x) =
Z ∞ dx
E(X) = μ = x f (x)dx , var (X) = σ 2 = E(X 2 ) − μ2 ,
−∞
Z ∞
2
where E(X ) = x2 f (x)dx
−∞

4. Discrete probability distributions

Discrete Uniform Uniform (n)

1
px = (x = 1, 2, . . . , n) μ = 12 (n + 1) , σ 2 = 1
12
(n2 − 1)
n
Binomial distribution Binomial (n, θ)
!
n
px = θx (1 − θ)n−x (x = 0, 1, 2, . . . , n) μ = nθ , σ 2 = nθ(1 − θ)
x

Poisson distribution Poisson (λ)

λx e−λ
px = (x = 0, 1, 2, . . .) (with λ > 0) μ = λ , σ2 = λ
x!
Geometric distribution Geometric (θ)
1 1−θ
px = (1 − θ)x−1 θ (x = 1, 2, 3, . . .) μ= , σ2 =
θ θ2

5. Continuous probability distributions

Uniform distribution Uniform (α, β)



 1

 (α < x < β), μ = 12 (α + β) , σ 2 = 1
12
(β − α)2
f (x) = β−α



 0 (otherwise).
 Exponential distribution Exponential (λ)



 λe−λx (0 < x < ∞), μ = 1/λ ,
f (x) =


 0 (−∞ < x ≤ 0). σ 2 = 1/λ2 .

Normal distribution N (μ, σ 2 )

  2 
1 1 x−μ
f (x) = √ exp − (−∞ < x < ∞)
2πσ 2 2 σ
E(X) = μ , var (X) = σ 2

Standard normal distribution N (0, 1)

X −μ
If X is N (μ, σ 2 ), then Y = is N (0, 1)
σ

6. Reliability

For a device in continuous operation with failure time random variable T having pdf
f (t) (t > 0)

The reliability function at time t R(t) = P (T > t)

The failure rate or hazard function h(t) = f (t)/R(t)

Z t
The cumulative hazard H(t) = h(t0 ) dt0 = − ln{R(t)}
0

The Weibull(α, β) distribution has H(t) = βtα

7. System reliability

For a system of k devices, which operate independently, let

Ri = P (Di ) = P (“device i operates”)
The system reliability, R, is the probability of a path of operating devices
A system of devices in series operates only if every device operates
R = P (D1 ∩ D2 ∩ ∙ ∙ ∙ ∩ Dk ) = R1 R2 ∙ ∙ ∙ Rk
A system of devices in parallel operates if any device operates
R = P (D1 ∪ D2 ∪ ∙ ∙ ∙ ∪ Dk ) = 1 − (1 − R1 )(1 − R2 ) ∙ ∙ ∙ (1 − Rk )

8. Covariance and correlation

The covariance of X and Y cov (X, Y ) = E(XY ) − {E(X)}{E(Y )}

 X 1 X X
From pairs of observations (x1 , y1 ), . . . , (xn , yn ) Sxy = xk yk − ( xi )( yj )
k n i j
X 1 X X 1 X 2
Sxx = x2k − ( xi )2 , Syy = yk2 − ( yj )
k n i k n j
1
Sample covariance sxy = Sxy estimates cov (X, Y )
n−1
cov (X, Y )
Correlation coefficient ρ = corr (X, Y ) =
sd (X) ∙ sd (Y )
Sxy
Sample correlation coefficient r = q estimates ρ
Sxx Syy

9. Sums of random variables

E(X + Y ) = E(X) + E(Y )

var (X + Y ) = var (X) + var (Y ) + 2 cov (X, Y )
cov (aX + bY, cX + dY ) = (ac) var (X) + (bd) var (Y ) + (ad + bc) cov (X, Y )
If X is N (μ1 , σ12 ), Y is N (μ2 , σ22 ), and cov (X, Y ) = c,
then X + Y is N (μ1 + μ2 , σ12 + σ22 + 2c)

10. Bias, standard error, mean square error

If t estimates θ (with random variable T giving t)

Bias of t bias(t) = E(T ) − θ

Standard error of t se (t) = sd (T )

Mean square error of t = E{(T − θ)2 } = {se (t)}2 + {bias(t)}2

MSE(t)
√ √
If x estimates μ, then bias(x) = 0 , se (x) = σ/ n , MSE(x) = σ 2 /n , sc
e (x) = s/ n
Central limit property if n is fairly large, x is from N (μ, σ 2 /n) approximately

11. Likelihood

The likelihood is the joint probability as a function of the unknown parameter θ.

For a random sample x1 , x2 , . . . , xn
`(θ; x1 , x2 , . . . , xn ) = P (X1 = x1 θ) ∙ ∙ ∙ P (Xn = xn θ) (discrete distribution)

`(θ; x1 , x2 , . . . , xn ) = f (x1 θ)f (x2 θ) ∙ ∙ ∙ f (xn θ) (continuous distribution)

The maximum likelihood estimator (MLE) is θb for which the likelihood is a maximum.
12. Confidence intervals

If x1 , x2 , . . . , xn are a random sample from N (μ, σ 2 ) and σ 2 is known, then

σ σ
the 95% confidence interval for μ is (x − 1.96 √ , x + 1.96 √ )
n n
If σ 2 is estimated, then from the Student t table for tn−1 we find t0 = tn−1,0.05
s s
The 95% confidence interval for μ is (x − t0 √ , x + t0 √ )
n n

13. Standard normal table Values of pdf φ(y) = f (y) and cdf Φ(y) = F (y)

y φ(y) Φ(y) y φ(y) Φ(y) y φ(y) Φ(y) y Φ(y)

0 .399 .5 .9 .266 .816 1.8 .079 .964 2.8 .997
.1 .397 .540 1.0 .242 .841 1.9 .066 .971 3.0 .998
.2 .391 .579 1.1 .218 .864 2.0 .054 .977 0.841 .8
.3 .381 .618 1.2 .194 .885 2.1 .044 .982 1.282 .9
.4 .368 .655 1.3 .171 .903 2.2 .035 .986 1.645 .95
.5 .352 .691 1.4 .150 .919 2.3 .028 .989 1.96 .975
.6 .333 .726 1.5 .130 .933 2.4 .022 .992 2.326 .99
.7 .312 .758 1.6 .111 .945 2.5 .018 .994 2.576 .995
.8 .290 .788 1.7 .094 .955 2.6 .014 .995 3.09 .999

14. Student t table Values tm,p of x for which P (|X| > x) = p , when X is tm

p .10 .05 .02 0.01 p .10 .05 .02 0.01

m 1 6.31 12.71 31.82 63.66 m 9 1.83 2.26 2.82 3.25
2 2.92 4.30 6.96 9.92 10 1.81 2.23 2.76 3.17
3 2.35 3.18 4.54 5.84 12 1.78 2.18 2.68 3.05
4 2.13 2.78 3.75 4.60 15 1.75 2.13 2.60 2.95
5 2.02 2.57 3.36 4.03 20 1.72 2.09 2.53 2.85
6 1.94 2.45 3.14 3.71 25 1.71 2.06 2.48 2.78
7 1.89 2.36 3.00 3.50 40 1.68 2.02 2.42 2.70
8 1.86 2.31 2.90 3.36 ∞ 1.645 1.96 2.326 2.576
 15. Chi-squared table

Values χ2k,p of x for which P (X > x) = p , when X is χ2k and p = .995, .975, etc

k .995 .975 .05 .025 .01 .005 k .995 .975 .05 .025 .01 .005
1 .000 .001 3.84 5.02 6.63 7.88 18 6.26 8.23 28.87 31.53 34.81 37.16
2 .010 .051 5.99 7.38 9.21 10.60 20 7.43 9.59 31.42 34.17 37.57 40.00
3 .072 .216 7.81 9.35 11.34 12.84 22 8.64 10.98 33.92 36.78 40.29 42.80
4 .207 .484 9.49 11.14 13.28 14.86 24 9.89 12.40 36.42 39.36 42.98 45.56
5 .412 .831 11.07 12.83 15.09 16.75 26 11.16 13.84 38.89 41.92 45.64 48.29
6 .676 1.24 12.59 14.45 16.81 18.55 28 12.46 15.31 41.34 44.46 48.28 50.99
7 .990 1.69 14.07 16.01 18.48 20.28 30 13.79 16.79 43.77 46.98 50.89 53.67
8 1.34 2.18 15.51 17.53 20.09 21.95 40 20.71 24.43 55.76 59.34 63.69 66.77
9 1.73 2.70 16.92 19.02 21.67 23.59 50 27.99 32.36 67.50 71.41 76.15 79.49
10 2.16 3.25 13.31 20.48 23.21 25.19 60 35.53 40.48 79.08 83.30 88.38 91.95
12 3.07 4.40 21.03 23.34 26.22 28.30 70 43.28 48.76 90.53 95.02 100.4 104.2
14 4.07 5.63 23.68 26.12 29.14 31.32 80 51.17 57.15 101.9 106.6 112.3 116.3
16 5.14 6.91 26.30 28.85 32.00 34.27 100 67.33 74.22 124.3 129.6 135.8 140.2

16. The chi-squared goodness-of-fit test

The frequencies ny are grouped so that the fitted frequency n b y for every group exceeds
about 5.
X (ny − n b y )2
X2 = is referred to the table of χ2k with significance point p,
y ny
b
where k is the number of terms summed, less one for each constraint, eg matching total
frequency, and matching x with μ.

17. Joint probability distributions

Discrete distribution {pxy }, where pxy = P ({X = x} ∩ {Y = y}) .

Let px• = P (X = x), and p•y = P (Y = y), then
X pxy
px• = pxy , and P (X = x Y = y) =
y p•y
 Continuous distribution
Z x Z y
Joint cdf F (x, y) = P ({X ≤ x} ∩ {Y ≤ y}) = f (x0 , y0 ) dx0 dy0
x0 =−∞ y0 =−∞

d2 F (x, y)
Joint pdf f (x, y) =
dx dy
Z ∞
Marginal pdf of X fX (x) = f (x, y0 ) dy0
−∞
f (x, y)
Conditional pdf of X given Y = y fX|Y (x|y) = (provided fY (y) > 0)
fY (y)

18. Linear regression

To fit the linear regression model y = α + βx by ybx = α b
b + βx from observations
(x1 , y1 ), . . . , (xn , yn ) , the least squares fit is
b = y − xβb ,
α βb = Sxy /Sxx
2
Sxy
The residual sum of squares RSS = Syy −
Sxx
RSS n − 2 c2
σc2 = , σ is from χ2n−2
n−2 σ2

b = α,
E(α) b = β,
E(β)
P 2
x i b σ2 b x 2
var (α)
b = σ2 , var (β) = , cov (α,
b β) = − σ
n Sxx Sxx Sxx
( )
b 1 (x − x)2
ybx = α
b + βx , E(ybx ) = α + βx , var (ybx ) = + σ2
n Sxx
αb−α βb − β ybx − α − β x
, b
, are each from tn−2
sc
e (α)
b sce (β) sc
e (ybx )

19. Design matrix for factorial experiments With 3 factors each at 2 levels
 
1 −1 −1 1 −1 1 1 −1
 

 1 1 −1 −1 −1 −1 1 1
 


1 −1 1 −1 −1 1 −1 1
 


1 1 1 1 −1 −1 −1 −1 

X = 
 1 −1 −1 1 1 −1 −1 1
 

 1 1 −1 −1 1 1 −1 −1 

 
 1 −1 1 −1 1 −1 1 −1 
 
1 1 1 1 1 1 1 1