Statistics and risk modelling using Python

Eric Marsden
<[email protected]>

Statistics is the science of learning from experience,

particularly experience that arrives a little bit at a
time.
— B. Efron, Stanford
Learning objectives

Using Python/SciPy tools:

1 Analyze data using descriptive statistics and graphical tools

2 Fit a probability distribution to data (estimate distribution parameters)

3 Express various risk measures as statistical tests

4 Determine quantile measures of various risk metrics

5 Build flexible models to allow estimation of quantities of interest and

associated uncertainty measures

6 Select appropriate distributions of random variables/vectors for stochastic

phenomena
Where does this fit into risk engineering?

curve
fitting
data probabilistic model consequence model

event probabilities event consequences

risks
Where does this fit into risk engineering?

curve
fitting
data probabilistic model consequence model

event probabilities event consequences

risks costs

criteria

decision-making
Where does this fit into risk engineering?

curve
fitting
data probabilistic model consequence model

event probabilities event consequences

These slides
risks costs

criteria

decision-making
Angle of attack: computational approach to statistics

▷ Emphasize practical results rather than formulæ and proofs

▷ Include new statistical tools which have become practical thanks to

power of modern computers
• “resampling” methods, “Monte Carlo” methods

▷ Our target: “Analyze risk-related data using computers”

▷ If talking to a recruiter, use the term data science or data engineering

• very sought-after skill!
A long history

John Graunt collected and published public

health data in the uk in the Bills of Mortality
(circa 1630), and his statistical analysis identified
the plague as a significant source of premature
deaths.

Image source: British Library, public domain

Python and SciPy

Environment used in this coursework:

▷ Python programming language + SciPy + NumPy + matplotlib libraries

▷ Alternative to Matlab, Scilab, Octave, R

▷ Free software

▷ A real programming language with simple syntax

• much, much more powerful than a spreadsheet!

▷ Rich scientific computing libraries

• statistical measures
• visual presentation of data
• optimization, interpolation and curve fitting
• stochastic simulation
• machine learning, image processing…
How do I run it?

▷ Cloud without local installation

• Google Colaboratory, at colab.research.google.com

▷ Microsoft Windows: install one of

• Anaconda from anaconda.com/download/
• pythonxy from python-xy.github.io

▷ MacOS: install one of

• Anaconda from anaconda.com/download/
• Pyzo, from pyzo.org

▷ Linux: install packages python, numpy, matplotlib, scipy

• your distribution’s packages are probably fine
Google Colaboratory

▷ Runs in the cloud, access via web browser

▷ No local installation needed

▷ Can save to your Google Drive

▷ Notebooks are live computational

documents, great for “experimenting”

→ colab.research.google.com
Jupyter: interact with Python in a web browser

▷ Jupyter python notebook: interact via your

web browser

▷ Type expressions into a cell then press

Ctrl-Enter

▷ Results appear in the web page

▷ Jupyter notebooks are live computational

documents
• allow the embedding of rich media (anything
that a web browser can display)
• try online at jupyter.org/try, or install on
your computer for free

More on Jupyter at jupyter.org

Python as a statistical calculator
In [1]: import numpy

In [2]: 2 + 2
Out[2]: 4

In [3]: numpy.sqrt(2 + 2)
Out[3]: 2.0

In [4]: numpy.pi
Out[4]: 3.141592653589793

In [5]: numpy.sin(numpy.pi)
Out[5]: 1.2246467991473532e-16
as a
Download this content
In [6]: numpy.random.uniform(20, 30) Python notebook at
rg
Out[6]: 28.890905809912784 risk-engineering.o

In [7]: numpy.random.uniform(20, 30)

Out[7]: 20.58728078429875
Python as a statistical calculator
In [3]: obs = numpy.random.uniform(20, 30, 10)

In [4]: obs
Out[4]:
array([ 25.64917726, 21.35270677, 21.71122725, 27.94435625,
25.43993038, 22.72479854, 22.35164765, 20.23228629,
26.05497056, 22.01504739])

In [5]: len(obs)
Out[5]: 10

In [6]: obs + obs

Out[6]:
array([ 51.29835453, 42.70541355, 43.42245451, 55.8887125 ,
50.87986076, 45.44959708, 44.7032953 , 40.46457257,
52.10994112, 44.03009478])

In [7]: obs - 25
Out[7]:
array([ 0.64917726, -3.64729323, -3.28877275, 2.94435625,
0.43993038,
-2.27520146, -2.64835235, -4.76771371, 1.05497056,
-2.98495261])

In [8]: obs.mean()
Out[8]: 23.547614834213316

In [9]: obs.sum()
Out[9]: 235.47614834213317

In [10]: obs.min()
Out[10]: 20.232286285845483
Python as a statistical calculator: plotting
In [2]: import numpy, matplotlib.pyplot as plt

In [3]: x = numpy.linspace(0, 10, 100)

In [4]: obs = numpy.sin(x) + numpy.random.uniform(-0.1, 0.1, 100)

In [5]: plt.plot(x, obs)

Out[5]: [<matplotlib.lines.Line2D at 0x7f47ecc96da0>]

In [7]: plt.plot(x, obs)

Out[7]: [<matplotlib.lines.Line2D at 0x7f47ed42f0f0>]

 Some basic notions in
probability and statistics
Discrete vs continuous variables

Discrete Continuous
A discrete variable takes separate, countable A continuous variable is the result of a
values measurement (a floating point number)

Examples: Examples:
▷ outcomes of a coin toss: {head, tail} ▷ height of a person
▷ number of students in the class ▷ flow rate in a pipeline
▷ questionnaire responses {very unsatisfied, ▷ volume of oil in a drum
unsatisfied, satisfied, very satisfied} ▷ time taken to cycle from home to
university
Random variables

A random variable is a set of possible values from a stochastic experiment

Examples:
▷ sum of the values on two dice throws (a discrete random variable)

▷ height of the water in a river at time 𝑡 (a continuous random variable)

▷ time until the failure of an electronic component

▷ number of cars on a bridge at time 𝑡

▷ number of new influenza cases at a hospital in a given month

▷ number of defective items in a batch produced by a factory

Probability Mass Functions

▷ For all values 𝑥 that a discrete random variable 𝑋 may take, we

define the function

𝑝𝑋 (𝑥) ≝ Pr(𝑋 takes the value 𝑥)

▷ This is called the probability mass function (pmf) of 𝑋

0.7

0.6
▷ Example: 𝑋 = “number of heads when tossing a coin twice” 0.5
• 𝑝𝑋 (0) ≝ Pr(𝑋 = 0) = 1/4 0.4

0.3
• 𝑝𝑋 (1) ≝ Pr(𝑋 = 1) = 2/4
0.2

• 𝑝𝑋 (2) ≝ Pr(𝑋 = 2) = 1/4 0.1

0.0
0 1 2
Probability Mass Functions: two coins
▷ Task: simulate “expected number of heads when tossing a coin twice”

▷ Let’s simulate a coin toss by random choice between 0 and 1

> numpy.random.randint(0, 2)
1
inclusive lower bound exclusive upper bound

as a
Download this content
Python notebook at
rg
risk-engineering.o
Probability Mass Functions: two coins
▷ Task: simulate “expected number of heads when tossing a coin twice”

▷ Let’s simulate a coin toss by random choice between 0 and 1

> numpy.random.randint(0, 2)
1
inclusive lower bound exclusive upper bound

▷ Toss a coin twice:

> numpy.random.randint(0, 2, 2)
array([0, 1]) count
as a
Download this content
Python notebook at
rg
risk-engineering.o
Probability Mass Functions: two coins
▷ Task: simulate “expected number of heads when tossing a coin twice”

▷ Let’s simulate a coin toss by random choice between 0 and 1

> numpy.random.randint(0, 2)
1
inclusive lower bound exclusive upper bound

▷ Toss a coin twice:

> numpy.random.randint(0, 2, 2)
array([0, 1]) count
as a
Download this content
Python notebook at
rg
risk-engineering.o
▷ Number of heads when tossing a coin twice:
> numpy.random.randint(0, 2, 2).sum()
1
Probability Mass Functions: two coins

▷ Task: simulate “expected number of heads when tossing a coin twice”

▷ Do this 1000 times and plot the resulting pmf:

import numpy
import matplotlib.pyplot as plt

N = 1000
heads = numpy.zeros(N, dtype=int)
for i in range(N):
# second argument to randint is exclusive upper bound
heads[i] = numpy.random.randint(0, 2, 2).sum()
plt.stem(numpy.bincount(heads))
heads[i]: element
number
i of the array hea ds
More information on Python programming

For more information on Python syntax, check out the

book Think Python

Purchase, or read online for free at

allendowney.github.io/ThinkPython/
Introductory online course on probability (free)

→ https://oli.cmu.edu/courses/probability-statistics-open-free/
Free self-paced course whose only prerequisite is basic algebra
Probability Mass Functions: properties

▷ A pmf is always non-negative

𝑝𝑋 (𝑥) ≥ 0

▷ Sum over the support equals 1

∑ 𝑝𝑋 (𝑥) = 1
𝑥

▷
Pr(𝑎 ≤ 𝑋 ≤ 𝑏) = ∑ 𝑝𝑋 (𝑥)
𝑥∈[𝑎,𝑏]
Probability Density Functions

▷ For continuous random variables, the probability density

function 𝑓𝑋 (𝑥) is defined by
𝑏
Pr(𝑎 ≤ 𝑋 ≤ 𝑏) = ∫ 𝑓𝑋 (𝑥)𝑑𝑥
𝑎

▷ It is non-negative

𝑓𝑋 (𝑥) ≥ 0

▷ The area under the curve (integral over ℝ) is 1

∞
∫ 𝑓 (𝑥)𝑑𝑥 =1
−∞ 𝑋
Probability Density Function

In reliability engineering, you are often concerned about 1.2

Exponential distribution PDF

the random variable 𝑇 representing the time at which a

λ = 0.5
λ = 1.0
1.0 λ = 10.0

component fails.
0.8

The PDF 𝑓 (𝑡) is the “failure density function”. It tells you 0.6

the probability of failure around age 𝑡. 0.4

0.2
𝑡+Δ𝑡
Pr(𝑡 < 𝑇 < 𝑡 + Δ𝑡) ∫𝑡 𝑓 (𝑡)𝑑𝑡
lim = lim 0.0
0 1 2 3 4
Δ𝑡→0 Δ𝑡 Δ𝑡→0 Δ𝑡
Expectation of a random variable

▷ The expectation (or mean) is defined as a weighted average of all

possible realizations of a random variable

▷ Discrete random variable:

𝑁
𝔼[𝑋] = 𝜇𝑋 ≝ ∑ 𝑥𝑖 × Pr(𝑋 = 𝑥𝑖 )
𝑖=1

▷ Continuous random variable:

∞
𝔼[𝑋] = 𝜇𝑋 ≝ ∫ 𝑢 × 𝑓 (𝑢)𝑑𝑢
−∞

▷ Interpretation:
• the center of gravity of the pmf or pdf
• the average in a large number of independent realizations of your experiment
Important concept: independence

▷ Definition of statistical independence of two events: the outcome of one

has no effect on the outcome of the other

▷ Typical example: successive coin tosses

▷ The probability of two independent events both happening is the

product of their individual probabilities

▷ Often important in safety analysis:

• if you are responsible for a car accident, your car insurance will become more
expensive, because the event indicates you are more likely to have future car
accidents (they are not independent events)
• if an accident of a specific type occurs on an industrial site, it is less likely it
will occur again the following year because people will make extra efforts to
prevent that type of accident
Illustration: expected value with coins

▷ 𝑋 = “number of heads when tossing a coin twice”

▷ pmf: T
T
• 𝑝𝑋 (0) ≝ Pr(𝑋 = 0) = 1/4 H
• 𝑝𝑋 (1) ≝ Pr(𝑋 = 1) = 2/4
T
• 𝑝𝑋 (2) ≝ Pr(𝑋 = 2) = 1/4 H
H
1 2 1
▷ 𝔼[𝑋] ≝ ∑𝑘 × 𝑝𝑋 (𝑘) = 0 × 4
+1× 4
+2× 4
=1
𝑘
Illustration: expected value of a dice roll

6
1
▷ Expected value of a dice roll is ∑ 𝑖 × = 3.5
𝑖=1
6

▷ If we toss a dice a large number of times, the mean value should

converge to 3.5

▷ Let’s check that in Python

> numpy.random.randint(1, 7, 100).mean()

4.2
> numpy.random.randint(1, 7, 1000).mean()
3.478

(These numbers will be different for different executions. The greater the number of
random “dice throws” we simulate, the greater the probability that the mean will be
close to 3.5.)
Illustration: expected value of a dice roll

We can plot the speed of convergence of the mean

towards the expected value as follows. 6
Expectation of a dice roll
5

Expected value
4
N = 1000
roll = numpy.zeros(N) 3
expectation = numpy.zeros(N)
2
for i in range(N):
roll[i] = numpy.random.randint(1, 7) 1

for i in range(1, N):

0
expectation[i] = numpy.mean(roll[0:i]) 0 150 300 450 600 750 900
Number of rolls
plt.plot(expectation[1:N])
Mathematical properties of expectation

If 𝑐 is a constant and 𝑋 and 𝑌 are random variables, then

▷ 𝔼[𝑐] = 𝑐

▷ 𝔼[𝑐𝑋] = 𝑐𝔼[𝑋]

▷ 𝔼[𝑐 + 𝑋] = 𝑐 + 𝔼[𝑋]

▷ 𝔼[𝑋 + 𝑌 ] = 𝔼[𝑋] + 𝔼[𝑌 ]

Note: in general 𝔼[𝑔(𝑋)] ≠ 𝑔(𝔼[𝑋])

Aside: existence of expectation

▷ Not all random variables have an expectation

▷ Consider a random variable 𝑋 defined on some (infinite) sample space Ω

so that for all positive integers 𝑖, 𝑋 takes the value
• 2𝑖 with probability 2−𝑖−1
• −2𝑖 with probability 2−𝑖−1

▷ Both the positive part and the negative part of 𝑋 have infinite expectation
in this case, so 𝔼[𝑋] would have to be ∞ − ∞ (meaningless)
Variance of a random variable

▷ The variance provides a measure of the dispersion around the mean

• intuition: how “spread out” the data is

▷ For a discrete random variable:

𝑁
Var(𝑋) = 𝜎2𝑋 ≝ ∑(𝑋𝑖 − 𝜇𝑋 )2 Pr(𝑋 = 𝑥𝑖 )
𝑖=1

▷ For a continuous random variable:

∞
Var(𝑋) = 𝜎2𝑋 ≝ ∫ (𝑥 − 𝜇𝑥 )2 𝑓 (𝑢)𝑑𝑢
−∞

▷ In Python:
• obs.var() if obs is a NumPy vector
• numpy.var(obs) for any Python sequence (vector or list) In Excel, function VAR
Variance with coins

▷ 𝑋 = “number of heads when tossing a coin twice”

▷ pmf:
• 𝑝𝑋 (0) ≝ Pr(𝑋 = 0) = 1/4
• 𝑝𝑋 (1) ≝ Pr(𝑋 = 1) = 2/4
• 𝑝𝑋 (2) ≝ Pr(𝑋 = 2) = 1/4

𝑁
Var(𝑋) ≝ ∑(𝑥𝑖 − 𝜇𝑋 )2 Pr(𝑋 = 𝑥𝑖 )
𝑖=1
1 2 1 1
= × (0 − 1)2 + × (1 − 1)2 + × (2 − 1)2 =
4 4 4 2
Variance of a dice roll

▷ Analytic calculation of the variance of a dice roll:

𝑁
𝑉 𝑎𝑟(𝑋) ≝ ∑(𝑋𝑖 − 𝜇𝑋 )2 Pr(𝑋 = 𝑥𝑖 )
𝑖=1
1
= × ((1 − 3.5)2 + (2 − 3.5)2 + (3 − 3.5)2 + (4 − 4.5)2 + (5 − 3.5)2 + (6 − 3.5)2 )
6
= 2.916

▷ Let’s reproduce that in Python count

> rolls = numpy.random.randint(1, 7, 1000)

> len(rolls)
1000
> rolls.var()
2.9463190000000004
Source: IPPC report Climate Change 2001: The Scientific Basis, page 155. This is a famous figure that attempted to explain the effets of
global warming. It also helps to understand the difference between mean (or mathematical expectation) and variance.
Properties of variance as a mathematical operator
If 𝑐 is a constant and 𝑋 and 𝑌 are random variables, then
▷ Var(𝑋) ≥ 0 (variance is always non-negative)

▷ Var(𝑐) = 0
Beware:
▷ Var(𝑐 + 𝑋) = Var(𝑋) ▷ 𝔼[𝑋 2 ] ≠ (𝔼[𝑋])2

▷ Var(𝑐𝑋) = 𝑐2 Var(𝑋) ▷ 𝔼[√𝑋] ≠ √𝔼[𝑋]

▷ Var(𝑋 + 𝑌 ) = Var(𝑋) + Var(𝑌 ), if X and Y are independent

▷ Var(𝑋 + 𝑌 ) = Var(𝑋) + Var(𝑌 ) + 2Cov(𝑋, 𝑌 ) if 𝑋 and 𝑌 are dependent

Properties of variance as a mathematical operator
If 𝑐 is a constant and 𝑋 and 𝑌 are random variables, then
▷ Var(𝑋) ≥ 0 (variance is always non-negative)

▷ Var(𝑐) = 0
Beware:
▷ Var(𝑐 + 𝑋) = Var(𝑋) ▷ 𝔼[𝑋 2 ] ≠ (𝔼[𝑋])2

▷ Var(𝑐𝑋) = 𝑐2 Var(𝑋) ▷ 𝔼[√𝑋] ≠ √𝔼[𝑋]

▷ Var(𝑋 + 𝑌 ) = Var(𝑋) + Var(𝑌 ), if X and Y are independent

▷ Var(𝑋 + 𝑌 ) = Var(𝑋) + Var(𝑌 ) + 2Cov(𝑋, 𝑌 ) if 𝑋 and 𝑌 are dependent

Note:
▷ Cov(𝑋, 𝑌 ) ≝ 𝔼 [(𝑋 − 𝔼[𝑋])(𝑌 − 𝔼[𝑌 ])]

▷ Cov(𝑋, 𝑋) = Var(𝑋)
Standard deviation

𝑁
▷ Formula for variance: Var(𝑋) ≝ ∑(𝑋𝑖 − 𝜇𝑋 )2 Pr(𝑋 = 𝑥𝑖 )
𝑖=1

▷ If random variable 𝑋 is expressed in metres, Var(𝑋) is in 𝑚2

▷ To obtain a measure of dispersion of a random variable around its

expected value which has the same units as the random variable itself,
take the square root

▷ Standard deviation 𝜎 ≝ √Var(𝑋)

▷ In Python:
• obs.std() if obs is a NumPy vector
• numpy.std(obs) for any Python sequence (vector or list) EV
In Excel, function STD
Properties of standard deviation

▷ Suppose 𝑌 = 𝑎𝑋 + 𝑏, where
• 𝑎 and 𝑏 are scalar
• 𝑋 and 𝑌 are two random variables

▷ Then 𝜎(𝑌 ) = |𝑎| 𝜎(𝑋)

▷ Let’s check that with NumPy:

> X = numpy.random.uniform(100, 200, 1000)

> a = -32
> b = 16
> Y = a * X + b
> Y.std()
914.94058476118835
> abs(a) * X.std()
914.94058476118835
Properties of expectation & variance: empirical testing with numpy
▷ 𝔼[𝑋 + 𝑌 ] = 𝔼[𝑋] + 𝔼[𝑌 ]

> X = numpy.random.randint(1, 101, 1000)

> Y = numpy.random.randint(-300, -199, 1000)
> X.mean()
50.575000000000003
> Y.mean()
-251.613
> (X + Y).mean()
-201.038
Properties of expectation & variance: empirical testing with numpy
▷ 𝔼[𝑋 + 𝑌 ] = 𝔼[𝑋] + 𝔼[𝑌 ]

> X = numpy.random.randint(1, 101, 1000)

> Y = numpy.random.randint(-300, -199, 1000)
> X.mean()
50.575000000000003
> Y.mean()
-251.613
> (X + Y).mean()
-201.038

▷ Var(𝑐𝑋) = 𝑐2 Var(𝑋)

> numpy.random.randint(1, 101, 1000).var()

836.616716
> numpy.random.randint(5, 505, 1000).var()
20514.814318999997
> 5 * 5 * 836
20900
Properties of expectation & variance: empirical testing with numpy
▷ 𝔼[𝑋 + 𝑌 ] = 𝔼[𝑋] + 𝔼[𝑌 ]

> X = numpy.random.randint(1, 101, 1000)

> Y = numpy.random.randint(-300, -199, 1000)
> X.mean()
50.575000000000003
> Y.mean()
-251.613
> (X + Y).mean()
-201.038

▷ Var(𝑐𝑋) = 𝑐2 Var(𝑋)

> numpy.random.randint(1, 101, 1000).var()

836.616716
> numpy.random.randint(5, 505, 1000).var()
20514.814318999997
> 5 * 5 * 836
20900
Cumulative Distribution Function

▷ The cumulative distribution function (cdf) of random variable 𝑋, denoted

by 𝐹𝑋 (𝑥), indicates the probability that 𝑋 assumes a value ≤ 𝑥, where 𝑥 is
any real number

𝐹𝑋 (𝑥) = Pr(𝑋 ≤ 𝑥) −∞≤𝑥 ≤∞

▷ Properties of a cdf:
• 𝐹𝑋 (𝑥) is a non-decreasing function of 𝑥
• 0 ≤ 𝐹𝑋 (𝑥) ≤ 1
• lim𝑥→∞ 𝐹𝑋 (𝑥) = 1
• lim𝑥→−∞ 𝐹𝑋 (𝑥) = 0
• if 𝑥 ≤ 𝑦 then 𝐹𝑋 (𝑥) ≤ 𝐹𝑋 (𝑦)
• Pr(𝑎 < 𝑋 ≤ 𝑏) = 𝐹𝑋 (𝑏) − 𝐹𝑋 (𝑎) ∀𝑏 > 𝑎
CDF of a discrete distribution

Heads from two coin tosses: PMF

𝐹𝑋 (𝑥) = Pr(𝑋 ≤ 𝑥) −∞≤𝑥 ≤∞ 0.7

0.6

= ∑ Pr(𝑋 = 𝑥𝑖 ) 0.5
𝑥𝑖 ≤𝑥 0.4

0.3
The CDF is built by accumulating probability as 𝑥 0.2

increases. 0.1

0.0
0 1 2

Consider the random variable 𝑋 = “number of heads when

tossing a coin twice”. Heads from two coin tosses: CDF
1.0

0.8

𝑥 0 1 2 0.6

0.4

PMF, 𝑝𝑋 (𝑥) = Pr(𝑋 = 𝑥) /4

1 2
/4 /4
1 0.2

0.0

0.0 0.5 1.0 1.5 2.0

CDF, 𝐹𝑋 (𝑥) = Pr(𝑋 ≤ 𝑥) /4
1 3
/4 1
CDF of a discrete distribution

𝐹𝑋 (𝑥) = Pr(𝑋 ≤ 𝑥) −∞≤𝑥 ≤∞

= ∑ Pr(𝑋 = 𝑥𝑖 )
𝑥𝑖 ≤𝑥

Example: sum of two dice

Sum of two dice: CDF

Sum of two dice: PMF
1.0
0.150
0.8
0.125

0.100 0.6

0.075 0.4

0.050
0.2
0.025
0.0
0.000
0 2 4 6 8 10 12 0 2 4 6 8 10 12
CDF of a continuous distribution

Python: scipy.stats.norm(loc=mu, scale=sigma).pdf(x)

𝐹𝑋 (𝑥) = Pr(𝑋 ≤ 𝑥)
𝑥
=∫ 𝑓 (𝑢)𝑑𝑢
−∞

Python: scipy.stats.norm(loc=mu, scale=sigma).cdf(x)

Interpreting the CDF

1
In reliability engineering, we are often
interested in the random variable 𝑇 P(T ≤ t)
representing time to failure of a component.

Probability F(t)
The cumulative distribution function tells you
the probability that lifetime is ≤ 𝑡.

0
t
𝐹(𝑡) = Pr(𝑇 ≤ 𝑡) Time to failure (T)
Exercise

Problem
Field data tells us that the time to failure of a pump, 𝑋, is normally distributed.
The mean and standard deviation of the time to failure are estimated from
historical data as μ = 3200 hours and σ = 600 hours.
What is the probability that a pump will fail before 2000 hours of operation?
Exercise

Problem
Field data tells us that the time to failure of a pump, 𝑋, is normally distributed.
The mean and standard deviation of the time to failure are estimated from
historical data as μ = 3200 hours and σ = 600 hours.
What is the probability that a pump will fail before 2000 hours of operation?

Solution
We are interested in calculating Pr(𝑋 ≤ 2000) and we know that 𝑋 follows
a norm(3200, 600) distribution. We can use the CDF to calculate Pr(𝑋 ≤
2000).

We want norm(3200, 600).cdf(2000), which is 0.022750 (or 2.28%).

Exercise

Problem
Field data tells us that the time to failure of a pump, 𝑋, is normally distributed.
The mean and standard deviation of the time to failure are estimated from
historical data as μ = 3200 hours and σ = 600 hours.
What is the probability that a pump will fail after it has worked for at least
2000 hours?
Exercise

Problem
Field data tells us that the time to failure of a pump, 𝑋, is normally distributed.
The mean and standard deviation of the time to failure are estimated from
historical data as μ = 3200 hours and σ = 600 hours.
What is the probability that a pump will fail after it has worked for at least
2000 hours?

Solution
We are interested in calculating Pr(𝑋 > 2000) and we know that 𝑋 follows a
norm(3200, 600) distribution. We know that Pr(𝑋 > 2000) = 1 − Pr(𝑋 ≤
2000).

We want 1 - norm(3200, 600).cdf(2000), which is 0.977 (or 97.7%).

The quantile function

The quantile function is the inverse of the cdf.

The median is the point where half the

population is below and half is above (it’s the
0.5 quantile, and the 50th percentile).

Consider a normal distribution centered in 120

with a standard deviation of 20.

> import scipy.stats

> distrib = scipy.stats.norm(120, 20)
> distrib.ppf(0.5)
120.0
> distrib.cdf(120.0)
0.5
Quantile measures

A quantile measure is a cutpoint in the

probability distribution indicating the value
below which a given percentage of the sample
falls.

The 0.05 quantile is the 𝑥 value which has 5% of

the sample smaller than 𝑥.
The 0.05 quantile of the standard normal distribution
It’s also called the 5th percentile. (centered in 0, standard deviation of 1)

import scipy.stats
scipy.stats.norm(0, 1).ppf(0.05)
-1.6448536269514729
Quantile measures

Quantile measures are often used in health.

To the right, illustration of the range of baby

heights and weights as a function of their age.

Image source: US CDC

Quantile measures in risk analysis

Risk analysis and reliability engineering: analysts are

interested in the probability of extreme events
▷ what is the probability of a flood higher than my dike?

▷ how high do I need to build a dike to protect against

hundred-year floods?

▷ what is the probability of a leak given the corrosion

measurements I have made?

Three percentile measures (95% = green, 99% =

Problem: these are rare events so it’s difficult to obtain
blue, 99.99% = red) of the spatial risk of fallback
confidence that a model representing the underlying from a rocket launcher. Dotted lines indicate
mechanism works well for extremes uncertainty range.

Image source: aerospacelab-journal.org/sites/www.aerospacelab-journal.org/files/AL04-13_0.pdf

Quantiles and confidence intervals

A 90% confidence interval is the set of points

between the 0.05 quantile (5% of my
observations are smaller than this value) and
the 0.95 quantile (5% of my observations are
larger than this value).

In the example to the right, the 90% confidence

interval is [87.1, 152.9].
Scipy.stats package

▷ The scipy.stats module implements many

continuous and discrete random variables and their
associated distributions
• binomial, poisson, exponential, normal, uniform,
weibull…

▷ Usage: instantiate a distribution then call a method

• rvs: random variates
• pdf: Probability Density Function
• cdf: Cumulative Distribution Function
• sf: Survival Function (1-cdf)
• ppf: Percent Point Function (inverse of cdf)
• isf: Inverse Survival Function (inverse of sf)
Simulating dice throws
▷ Maximum of 1000 throws
> dice = scipy.stats.randint(1, 7)
> dice.rvs(1000).max()
exclusive upper
6 bound

▷ What is the probability of a die rolling 4?

> dice.pmf(4)
0.16666666666666666

▷ What is the probability of rolling 4 or below?

> dice.cdf(4)
0.66666666666666663

▷ What is the probability of rolling between 2 and 4 (inclusive)?

> dice.cdf(4) - dice.cdf(1)
0.5
Simulating dice

> import numpy

> import matplotlib.pyplot as plt

> toss = numpy.random.choice(range(1, 7))

> toss
2
> N = 10000
> tosses = numpy.random.choice(range(1, 7), N)
> tosses
array([6, 6, 4, ..., 2, 4, 5])
> tosses.mean()
0.175
3.5088
0.150
> numpy.median(tosses)
0.125
4.0
0.100
> len(numpy.where(tosses > 3)[0]) / float(N) 0.075
0.5041 0.050
> x, y = numpy.unique(tosses, return_counts=True) 0.025
> plt.stem(x, y/float(N)) 0.000
1 2 3 4 5 6
Scipy.stats examples

▷ Generate 5 random variates from a > import scipy.stats

continuous uniform distribution between 90

> u = scipy.stats.uniform(90, 10)
and 100 > u.rvs(5)
array([ 94.0970853 , 92.41951494,
▷ Check that the expected value of the 90.25127254, 91.69097729,
96.1811148 ])
distribution is around 95
> u.rvs(1000).mean()
94.892801456986376
▷ Check that around 20% of variates are less > (u.rvs(1000) < 92).sum() / 1000.0
than 92 0.193
 Some
important
probability
distributions
Some important probability distributions

Coin tossing with uneven coin Bernoulli scipy.stats.bernoulli

Rolling a dice uniform scipy.stats.randint

Counting errors/successes Binomial scipy.stats.binom

Trying until success geometric scipy.stats.geom

Countable, rare events whose occurrence is independent Poisson scipy.stats.poisson

Random “noise”, sums of many variables normal scipy.stats.norm

Bernoulli trials

▷ A trial is an experiment which can be repeated many times with

the same probabilities, each realization being independent of the
others

▷ Bernoulli trial: an experiment in which 𝑁 trials are made of an

event, with probability 𝑝 of “success” in any given trial and
probability 1 − 𝑝 of “failure”
• “success” and “failure” are mutually exclusive
Jakob Bernoulli (1654–1705)
• example: sequence of coin tosses
• example: arrival of requests in a web server per time slot
The geometric distribution (trying until success)

▷ We conduct a sequence of Bernoulli trials, each with success

probability 𝑝 Geometric distribution PMF, p=0.3
0.30

0.25
▷ What’s the probability that it takes 𝑘 trials to get a success?
0.20
• Before we can succeed at trial 𝑘, we must have had 𝑘 − 1 failures 0.15

• Each failure occurred with probability 1 − 𝑝, so total probability 0.10

(1 − 𝑝)𝑘−1 0.05

0.00
• Then a single success with probability 𝑝 0.0 2.5 5.0 7.5 10.0 12.5 15.0

▷ Pr(𝑋 = 𝑘) = (1 − 𝑝)𝑘−1 𝑝
The geometric distribution: intuition

▷ Suppose I am at a party and I start asking girls to dance. Let 𝑋 be the number
of girls that I need to ask in order to find a partner.
• If the first girl accepts, then 𝑋 = 1
• If the first girl declines but the next girl accepts, then 𝑋 = 2

▷ 𝑋 = 𝑘 means that I failed on the first 𝑘 − 1 tries and succeeded on the 𝑘th try
• My probability of failing on the first try is (1 − 𝑝)
• My probability of failing on the first two tries is (1 − 𝑝)(1 − 𝑝)
• My probability of failing on the first 𝑛 − 1 tries is (1 − 𝑝)𝑘−1
• Then, my probability of succeeding on the nth try is 𝑝

▷ Properties:
• 𝔼[𝑋] = 1
𝑝
1−𝑝
• Var(𝑋) = 𝑝2
The binomial distribution (counting successes)

▷ Also arises when observing multiple Bernoulli trials

• exactly two mutually exclusive outcomes, “success” and “failure”

▷ 𝐵𝑖𝑛𝑜𝑚𝑖𝑎𝑙(𝑝, 𝑘, 𝑛): probability of observing 𝑘 successes in 𝑛 trials, Binomial distribution PMF, n=20, p=0.3

where probability of success on a single trial is 𝑝

0.15
• example: toss a coin 𝑛 times (𝑝 = 0.5) and see 𝑘 heads
0.10

▷ We have 𝑘 successes, which happens with a probability of 𝑝𝑘

0.05

▷ We have 𝑛 − 𝑘 failures, which happens with probability (1 − 𝑝)𝑛−𝑘 0.00

0 5 10 15 20

▷ We can generate these 𝑘 successes in many different ways from

𝑛 trials, (𝑛𝑘) ways

▷ Pr(𝑋 = 𝑘) = (𝑛𝑘)(1 − 𝑝)𝑛−𝑘 𝑝𝑘 Reminder: binomial

coefficient (𝑛𝑘) is 𝑘!(𝑛−𝑘)!
𝑛!
Binomial distribution: application

▷ Consider a medical test with an error rate of

0.1 applied to 100 patients
> import scipy.stats
▷ What is the probability that we see at most 1
test error? > test = scipy.stats.binom(n=100, p=0.1)
> test.cdf(1)
0.00032168805319411544
▷ What is the probability that we see at most > test.cdf(10)
10 errors? 0.58315551226649232
> test.ppf(0.05)
▷ If the random variable 𝑋 represents the 5.0

number of test errors, what is the smallest 𝑘

such that 𝑃(𝑋 ≤ 𝑘) is at least 0.05?
make sure you pay
When reporting results,
the num ber of significant figures
attention to
this case).
in the input data (2 in
Binomial distribution: application
Q: A company drills 9 oil exploration wells, each with a 10% chance of success. Eight
of the nine wells fail. What is the probability of that happening?
Binomial distribution: application
Q: A company drills 9 oil exploration wells, each with a 10% chance of success. Eight
of the nine wells fail. What is the probability of that happening?

Analytic solution Answer by simulation

Each well is a binomial trial with 𝑝 = 0.1. We want Run 20 000 trials of the model and count the
the probability of exactly one success. number that generate 1 positive result.

> import scipy.stats > import scipy.stats

> wells = scipy.stats.binom(n=9, p=0.1) > N = 20_000
> wells.pmf(1) > wells = scipy.stats.binom(n=9, p=0.1)
0.38742048899999959 > trials = wells.rvs(N)
> (trials == 1).sum() / float(N)
0.38679999999999998
Binomial distribution: application
Q: A company drills 9 oil exploration wells, each with a 10% chance of success. Eight
of the nine wells fail. What is the probability of that happening?

Analytic solution Answer by simulation

Each well is a binomial trial with 𝑝 = 0.1. We want Run 20 000 trials of the model and count the
the probability of exactly one success. number that generate 1 positive result.

> import scipy.stats > import scipy.stats

> wells = scipy.stats.binom(n=9, p=0.1) > N = 20_000
> wells.pmf(1) > wells = scipy.stats.binom(n=9, p=0.1)
0.38742048899999959 > trials = wells.rvs(N)
> (trials == 1).sum() / float(N)
0.38679999999999998

The probability of all 9 wells failing is 0.99 = 0.3874 (and also wells.pmf(0)).

The probability of at least 8 wells failing is wells.cdf(1). It’s also wells.pmf(0) +

wells.pmf(1) (it’s 0.7748).
Binomial distribution: properties

Exercise: check empirically (with SciPy) the following properties

of the binomial distribution:
▷ the mean of the distribution (𝜇𝑥 ) is equal to 𝑛 × 𝑝

▷ the variance (𝜎2𝑋 ) is 𝑛 × 𝑝 × (1 − 𝑝)

▷ the standard deviation (𝜎𝑥 ) is √𝑛 × 𝑝 × (1 − 𝑝)

Gaussian (normal) distribution

▷ The famous “bell shaped” curve, fully described by its mean and
standard deviation

▷ Good representation of distribution of measurement errors and

many population characteristics
• size, mechanical strength, duration, speed

▷ Symmetric around the mean

▷ Mean = median = mode

▷ Python: scipy.stats.norm(μ, σ)

▷ Excel: NORMINV(RAND(), μ, σ)
Scipy.stats examples

▷ Consider a Gaussian distribution centered in

5, standard deviation of 1 > dist = scipy.stats.norm(5, 1)
> dist.cdf(5)
0.5
▷ Check that half the distribution is located to > dist.ppf(0.5)
the left of 5 5.0
> dist.ppf(0.01)
▷ Find the first percentile (value of 𝑥 which has 2.6736521259591592
> dist.isf(0.99)
1% of realizations to the left)
2.6736521259591592
> dist.cdf(2.67)
▷ Check that it is equal to the 99% survival 0.0099030755591642452
quantile
Area under the normal distribution

0.40

0.35

0.30

0.25

In [1]: import numpy 0.20

0.15

In [2]: from scipy.stats import norm 0.10

0.05

In [3]: norm.ppf(0.5)
Out[3]: 0.0
quantile function 0.00
−4 −3 −2 −1 0 1 2 3 4

In [4]: norm.cdf(0)
Out[4]: 0.5

In [5]: norm.cdf(2) - norm.cdf(-2)

0.40

Out[5]: 0.95449973610364158 there is a 95% chance that 0.35

the number drawn falls within

0.30

0.25

In [6]: norm.cdf(3) - norm.cdf(-3)

Out[6]: 0.99730020393673979 2 standard deviations of the mean 0.20

0.15

0.10

0.05

0.00
−4 −3 −2 −1 0 1 2 3 4

ed large tables
istics textbooks contain
In prehistoric times, stat tion . With cheap
the nor mal dist ribu
of quantile values for
longer necessary!
computing power, no
The “68–95–99.7 rule”
▷ The 68–95–99.7 rule (aka the three-sigma rule)
states that if 𝑥 is an observation from a normally
distributed random variable with mean μ and
standard deviation σ, then
• Pr(𝜇 − 𝜎 ≤ 𝑥 ≤ 𝜇 + 𝜎) ≈ 0.6827
• Pr(𝜇 − 2𝜎 ≤ 𝑥 ≤ 𝜇 + 2𝜎) ≈ 0.9545
• Pr(𝜇 − 3𝜎 ≤ 𝑥 ≤ 𝜇 + 3𝜎) ≈ 0.9973

▷ The 6σ quality management method pioneered by

Motorola aims for 99.99966% of production to
meet quality standards
• 3.4 defective parts per million opportunities (dpmo)
• actually, that’s only 4.5 sigma!
• (scipy.stats.norm.cdf(6) -
scipy.stats.norm.cdf(-6)) * 100 →
99.999999802682453

Image source: Wikipedia on 68–95–99.7 rule

Central limit theorem
▷ Theorem states that the mean (also true of the sum) of a set of
random measurements will tend to a normal distribution, no
matter the shape of the original measurement distribution

▷ Part of the reason for the ubiquity of the normal distribution in 1.4

science 1.2

1.0

▷ Python simulation: 0.8

0.6

0.4

N = 10_000 0.2

sim = numpy.zeros(N) 0.0

34.0 34.5 35.0 35.5 36.0
for i in range(N):
sim[i] = numpy.random.uniform(30, 40, 100).mean()
plt.hist(sim, bins=20, alpha=0.5, density=True)

▷ Exercise: try this with other probability distributions and check

that the simulations tend towards a normal distribution
Galton board

▷ The Galton board (or “bean machine”) has two parts:

• top: evenly-spaced pegs in staggered rows
• bottom: evenly-spaced rectangular slots

▷ Balls introduced at the top bounce of the pegs, equal probability

of going right or left at each successive row
• each vertical step is a Bernoulli trial

▷ Balls collect in the slots at the bottom with heights following a

binomial distribution
• and for large number of balls, a normal distribution
Named after English
ncis
psychometrician Sir Fra
▷ Interactive applet emulating a Galton board: Galton (18 22– 1911 )

→ randomservices.org/random/apps/GaltonBoardGame.html
Exponential distribution
Exponential distribution PDF
1.2
λ = 0.5
λ = 1.0
1.0 λ = 10.0

▷ pdf: 𝑓𝑍 (𝑧) = 𝜆𝑒−𝜆𝑧 , 𝑧 ≥ 0 0.8

0.6

▷ cdf: Pr(𝑍 ≤ 𝑧) = 𝐹𝑍 (𝑧) = 1 − 𝑒−𝜆𝑧 0.4

0.2

▷ The hazard function, or failure rate, is constant, equal to λ 0.0

0 1 2 3 4

• 1/λ is the “mean time between failures”, or mtbf

total number of failures
• λ can be calculated by total operating time Exponential distribution CDF
1.0
λ = 0.5
λ = 1.0
λ = 10.0

▷ Often used in reliability engineering to represent failure of 0.8

electronic equipment (no wear) 0.6

0.4

1
▷ Property: expected value of exponential random variable is 𝜆 0.2

0.0
0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0
Exponential distribution

> import scipy.stats

> lda = 25
> dist = scipy.stats.expon(scale=1/float(lda))
> obs = dist.rvs(size=1000)
Let’s check that the expected value of
> obs.mean()
an exponential random variable is 𝜆1 0.041137615318791773
> obs.std()
0.03915081431615041
> 1/float(lda)
0.04
Exponential distribution: memoryless property

▷ An exponentially distributed random variable 𝑇 obeys

Pr(𝑇 > 𝑠 + 𝑡 ∣ 𝑇 > 𝑠) = Pr(𝑇 > 𝑡), ∀𝑠, 𝑡 ≥ 0

where the vertical bar | indicates a conditional probability.

▷ Interpretation: if 𝑇 represents time of failure

• The distribution of the remaining lifetime does not depend on how long the
component has been operating (item is “as good as new”)

• Distribution of remaining lifetime is the same as the original lifetime

• An observed failure is the result of some suddenly appearing failure, not due to
gradual deterioration
Failure of power transistors (1/2)

▷ Suppose we are studying the reliability of a power system, which fails if

any of 3 power transistors fails

▷ Let 𝑋, 𝑌, 𝑍 be random variables modelling failure time of each transistor

(in hours)
• transistors have no physical wear, so model by exponential random variables
• failures are assumed to be independent

▷ 𝑋 ∼ 𝐸𝑥𝑝(1/5000) (mean failure time of 5000 hours)

▷ 𝑌 ∼ 𝐸𝑥𝑝(1/8000) (mean failure time of 8000 hours)

▷ 𝑍 ∼ 𝐸𝑥𝑝(1/4000) (mean failure time of 4000 hours)

Failure of power transistors (2/2)
▷ System fails if any transistor fails, so time to failure 𝑇 is min(𝑋, 𝑌 , 𝑍)

Pr(𝑇 ≤ 𝑡) = 1 − Pr(𝑇 > 𝑡)

= 1 − Pr(min(𝑋, 𝑌 , 𝑍) > 𝑡)
= 1 − Pr(𝑋 > 𝑡, 𝑌 > 𝑡, 𝑍 > 𝑡)
= 1 − Pr(𝑋 > 𝑡) × Pr(𝑌 > 𝑡) × Pr(𝑍 > 𝑡) (independence)
= 1 − (1 − Pr(𝑋 ≤ 𝑡)) (1 − Pr(𝑌 ≤ 𝑡)) (1 − Pr(𝑍 ≤ 𝑡))
= 1 − (1 − (1 − 𝑒−𝑡/5000 )) (1 − (1 − 𝑒−𝑡/8000 )) (1 − (1 − 𝑒−𝑡/4000 )) (exponential CDF)
= 1 − 𝑒−𝑡/5000 𝑒−𝑡/8000 𝑒−𝑡/4000
= 1 − 𝑒−𝑡( / +1/8000+1/4000)
1
5000

= 1 − 𝑒−0.000575𝑡

▷ System failure time is also exponentially distributed, with parameter 0.000575

▷ Expected time to system failure is 1/0.000575 = 1739 hours
 ▷ A Poisson process is any process where independent events occur at
a constant average rate
• time between each pair of consecutive events follows an exponential
distribution with parameter λ (the arrival rate)
• each of these inter-arrival times is assumed to be independent of other
inter-arrival times
• example: babies are born at a hospital at a rate of five per hour
Poisson process:
exponential ▷ The process is memoryless: number of arrivals in any bounded
interval of time after time 𝑡 is independent of the number of arrivals
arrival times before 𝑡

▷ Good model for many types of phenomena:

• number of road crashes in a zone
• number of faulty items in a production batch
• arrival of customers in a queue
• occurrence of earthquakes
The Poisson distribution
Poisson distribution PMF, λ = 3

0.20

▷ The probability distribution of the counting process 0.15

associated with a Poisson process 0.10

0.05
• the number of events of the Poisson process over a time
0.00

interval 0 5 10 15 20

Poisson distribution PMF, λ = 6

0.150
▷ Probability mass function: 0.125

0.100

𝜆𝑘 𝑒−𝜆
0.075

Pr(𝑍 = 𝑘) = , 𝑘 = 0, 1, 2… 0.050

𝑘! 0.025

0.000
0 5 10 15 20

▷ The parameter λ is called the intensity of the Poisson Poisson distribution PMF, λ = 12

distribution 0.10

• increasing λ adds more probability to larger values 0.08

0.06

0.04

▷ Python: scipy.stats.poisson(λ) 0.02

0.00
0 5 10 15 20
Poisson distribution and Prussian horses
▷ Number of fatalities for the Prussian cavalry resulting from
being kicked by a horse was recorded over a period of 20 years
• for 10 army corps, so total number of observations is 200

120
Prussian army deaths from horse kicks
Deaths Occurrences Poisson fit
Observed

Number of deaths from horse kicks (per corps per year)

0 109 100

1 65 80

2 22 60

3 3 40

4 1 20

>4 0 0
0 1 2 3 4

▷ It follows a Poisson distribution

▷ Exercise: reproduce the plot on the right which shows a fit

between a Poisson distribution and the historical data
The Poisson distribution: properties

▷ Expected value of the Poisson distribution is equal to its parameter μ

▷ Variance of the Poisson distribution is equal to its parameter μ

▷ The sum of independent Poisson random variables is also Poisson

▷ Specifically, if 𝑌1 and 𝑌2 are independent with 𝑌𝑖 ∼ 𝑃𝑜𝑖𝑠𝑠𝑜𝑛(𝜇𝑖 ) for 𝑖 = 1, 2

then 𝑌1 + 𝑌2 ∼ 𝑃𝑜𝑖𝑠𝑠𝑜𝑛(𝜇1 + 𝜇2 )

▷ Exercise: test these properties empirically with Python

llows
Notation: ∼ means “fo
in distribu tion ”
Simulating earthquake occurrences (1/2)

▷ Suppose we live in an area where there are

typically 0.03 earthquakes of intensity 5 or
more per year
> from scipy.stats import expon
▷ Assume earthquake arrival is a Poisson
> expon(scale=1/0.03).rvs(size=15)
process array([23.23763551, 28.73209684, 29.7729332,
• interval between earthquakes follows an 46.66320369, 4.03328973, 84.03262547,
42.22440297, 14.14994806, 29.90516283,
exponential distribution 87.07194806, 11.25694683, 15.08286603,
35.72159516, 44.70480237, 44.67294338])
• events are independent
> expon(scale=1/0.03).ppf(0.25)
▷ Simulate the random intervals between the 9.5894024150593644
# answer is “around 10 years”
next earthquakes of intensity 5 or greater

▷ What is the 25th percentile of the interval

between 5+ earthquakes?
Simulating earthquake occurrences (2/2)
Earthquake locations

▷ Worldwide: 144 earthquakes of magnitude 6 or greater

in 2013 (one every 60.8 hours on average)

1
▷ Rate: λ = 60.8
per hour

▷ What’s the probability that an earthquake of

magnitude 6 or greater will occur (worldwide) in the 1.0

next day?
0.8

Probability of an earthquake
• right: plot of the cdf of the corresponding exponential
0.6
distribution
0.4
• scipy.stats.expon(scale=60.8).cdf(24) =
0.326
0.2

0.0
0 50 100 150 200 250
Elapsed time (hours)

Data source: earthquake.usgs.gov/earthquakes/search/

Weibull distribution
0.6
k=0.5, λ=1
Weibull distribution PDF k=1.0, λ=1
k=2.0, λ=1
0.5 k=2.0, λ=2

0.4

0.3

0.2

0.1

0.0
0 1 2 3 4 5

▷ Very flexible distribution, can model left-skewed, right-skewed, and

symmetric data

▷ Widely used for modeling reliability data

▷ Python: scipy.stats.dweibull(k, 𝜇, 𝜆)
 0.6
k=0.5, λ=1
Weibull distribution PDF k=1.0, λ=1
k=2.0, λ=1

Weibull distribution 0.5

0.4
k=2.0, λ=2

0.3

0.2

0.1

𝑘 < 1: the failure rate decreases over time

0.0
▷ 0 1 2 3 4 5

• significant “infant mortality”

• defective items failing early and being weeded out of the population

▷ 𝑘 = 1: the failure rate is constant over time (equivalent to an exponential

distribution)
• suggests random external events are causing failure

▷ 𝑘 > 1: the failure rate increases with time

• “aging” process causes parts to be more likely to fail as time goes on
 Student’s t distribution PDF
0.45
t(k=∞)
t(k=2.0)
0.40
Student’s t distribution 0.35
t(k=1.0)
t(k=0.5)

0.30

0.25

0.20

0.15

0.10

0.05

0.00
−4 −2 0 2 4

▷ Symmetric and bell-shaped like the normal distribution, but

with heavier tails

▷ As the number of degrees of freedom 𝑑𝑓 grows, the t-distribution

approaches the normal distribution with mean 0 and variance 1

▷ Python: scipy.stats.t(df)

▷ First used by W. S. Gosset (aka Mr Student, 1876-1937), for

quality control at Guinness breweries
Distribution choice flowchart

Source of this useful flowchart is uncertain, possibly it is due to Aswath Damodaran, NYU
Image credits

▷ Dice on slide 39, flic.kr/p/9SJ5g, CC BY-NC-ND licence

▷ Galton board on slide 57: Wikimedia Commons, CC BY-SA licence
▷ Transistor on slide 61: flic.kr/p/4d4XSj, CC BY licence)
▷ Photo of Mr Gosset (aka Mr Student) on slide 72 from Wikimedia Commons,
public domain
▷ Microscope on slide 73 adapted from flic.kr/p/aeh1J5, CC BY licence

For more free content on risk engineering,

visit risk-engineering.org
Further reading

▷ SciPy lecture notes: scipy-lectures.org

▷ Book Statistics done wrong, available online at

statisticsdonewrong.com

▷ A gallery of interesting Python notebooks:

github.com/jupyter/jupyter/wiki/A-gallery-of-interesting-Jupyter-
Notebooks

For more free content on risk engineering,

visit risk-engineering.org
Feedback welcome!
This presentation is distributed under the terms of the
Creative Commons Attribution – Share Alike licence

@LearnRiskEng

fb.me/RiskEngineering

Was some of the content unclear? Which parts were most useful to
you? Your comments to [email protected]
(email) or @LearnRiskEng (Twitter) will help us to improve these
materials. Thanks!

For more free content on risk engineering,

visit risk-engineering.org