Stats for Engineers: Lecture 4

Summary from last time

Standard deviation – measure spread of distribution

𝜇
Variance = (standard deviation)2

𝑘
𝜎 𝜎

Discrete Random Variables

Binomial distribution
– number of successes from independent Bernoulli (YES/NO) trials
𝑃 ( 𝑋 =𝑘 ) = ( )
𝑛
𝑘
𝑘
𝑝 (1 −𝑝 )
𝑛− 𝑘
Example: A component has a 20% chance of being a dud. If five are selected from a
large batch, what is the probability that more than one is a dud?

Answer:

Let X = number of duds in selection of 5

Bernoulli trial: dud or not dud,

P(More than one dud) =

= 1 - 0.32768 - 0.4096 0.263.

 Binomial or not?

A mixed box of 10 screws contains 5 that are galvanized

and 5 that are non-galvanized.

Three screws are picked at random without

replacement. I want galvanized screws, so consider
picking a galvanized screw to be a success.

Does the number of successes have a Binomial

distribution?

1. Yes
2. No

0% 0%
20
1 2 Countdown
 Binomial or not?

A mixed box of 10 screws contains 5 that are galvanized

and 5 that are non-galvanized.

Three screws are picked at random without

replacement. I want galvanized screws, so consider
picking a galvanized screw to be a success.

Does the number of successes have a Binomial

distribution?

No, the picks are not independent

Independent events have but
𝑃 ( second galvanized|first galvanized ) ≠ 𝑃 (second galvanized)
- If the first is galvanized, then only of the
remaining screws are galvanized, which is

Note: If the box were much larger then consecutive picks would be nearly independent
- Binomial then a good approximation.
e.g. if box of 1000 screws, with 500 galvanized
499 1
𝑃 ( second galvanized|first galvanized )= =0.4995 ≈
999 2
Mean and variance of a binomial distribution

If , 𝜇= 𝐸 ( 𝑋 )= 𝑛𝑝 = var(X) =

Derivation

Suppose first that we have a single Bernoulli trial. Assign the value 1 to success, and 0 to
failure, the first occurring with probability p and the second having probability 1 − p.

The expected value for one trial is

Since the n trials are independent, the total expected value is just the sum of the
expected values for each trial, hence .

The variance in a single trial is:

Hence the variance for the sum of n independent trials is, by the rule for
summing the variances of independent variables,
 Mean and variance of a
Polling binomial distribution

In the French population about 20% of people

prefer Le Pen to other candidates (inc. Hollande and
Sarkozy).

An opinion poll asks 1000 people if they will vote for

Le Pen (YES) or not (NO). The expected number of
Le Pen voters (YESs) in the poll is therefore

What is the standard deviation (approximately)?

1. 6.3
2. 12.6
3. 25.3
4. 120
5. 160 0% 0% 0% 0% 0% 30
1. 2. 3. 4 5 Countdown
Polling

In the French population about 20% of people prefer Le

Pen
to other candidates (inc. Hollande and Sarkozy).

An opinion poll asks 1000 people if they will vote for

Le Pen (YES) or not (NO). The expected number of Le
Pen voters (YESs) in the poll is therefore

What is the standard deviation (approximately)?

The number of YES votes has a distribution

The variance is therefore

¿ 1000 × 0.2× 0.8=160

2.6

.6 in poll to say Le Pen

i.e. the fractional error of

Note: quoted errors in polls are not usually the standard deviation – see later
Binomial Distribution Summary

Discrete random variable X ~ B(n, p) if is the number of successes in independent

Bernoulli trials each with probability of success

( 𝑘¿) 𝑝
𝑃 ( 𝑋 =𝑘 )=
𝑛 𝑘
( 1 − 𝑝 )𝑛 − 𝑘

Mean and variance

𝜇 = 𝑛𝑝
2
𝜎 =𝑛𝑝(1 −𝑝 )
Poisson distribution

If events happen independently of each other, with average number of events in

some fixed interval , then the distribution of the number of events in that interval is
Poisson.

A random variable has the Poisson distribution with parameter (> 0) if

If you are interested in the derivation, see the notes.

Examples of possible Poisson distributions

1) Number of messages arriving at a telecommunications system in a day

2) Number of flaws in a metre of fibre optic cable

3) Number of radio-active particles detected in a given time

4) Number of photons arriving at a CCD pixel in some exposure time

(e.g. astronomy observations)

Sum of Poisson variables

If is Poisson with average number and is Poisson with average number
Then is Poisson with average number

The probability of events per unit time does not have to be constant for the total
number of events to be Poisson – can split up the total into a sum of the number of
events in smaller intervals.
 Example: On average lightning kills three people each year
in the UK, What is the probability that only one person is
killed this year?
Answer:
Assuming these are independent random events, the
number of people killed in a given year therefore has a
Poisson distribution:
𝑃 ( 𝑋 = 𝑘)

𝑘
Let the random variable be the number of people killed in a year.

Poisson distribution with

 Poisson distribution

Suppose that trucks arrive at a

receiving dock with an average
arrival rate of 3 per hour. What
is the probability exactly 5 trucks
Question from Derek Bruff
will arrive in a two-hour period?

Reminder: 0% 0% 0% 0% 20
1 2 3 4 Countdown
Poisson distribution

Suppose that trucks arrive at a

receiving dock with an average
arrival rate of 3 per hour. What
is the probability exactly 5 trucks
will arrive in a two-hour period?

In two hours mean number is .

−𝜆 𝑘 −6 5
𝑒 𝜆 𝑒 6
𝑃 ( 𝑋 =𝑘=5 )= =
𝑘! 5!
 Mean and variance

If Poisson with mean ,

then
Example: Telecommunications

Messages arrive at a switching centre at random and at an average rate of 1.2 per
second.

(a) Find the probability of 5 messages arriving in a 2-sec interval.

(b) For how long can the operation of the centre be interrupted, if the probability of
losing one or more messages is to be no more than 0.05?

Answer:

Times of arrivals form a Poisson process, rate sec.

(a) Let Y = number of messages arriving in a 2-sec interval.

Then Y ~ Poisson, mean number

−𝜆 𝑘 − 2.4 5
𝑒 𝜆 𝑒 2.4
𝑃 ( 𝑌 =𝑘=5 )= = =0.060
𝑘! 5!
 Question: (b) For how long can the operation of the centre be interrupted, if the
probability of losing one or more messages is to be no more than 0.05?

Answer:
(b) Let the required time = t seconds. Average rate of arrival is 1.2/second.

Let = number of messages in t seconds, so that

Poisson, with

Want P(At least one message) =

− 1.2 𝑡
𝑒 − 𝜆 𝜆𝑘 𝑒 ( 1.2𝑡 )0 −1 .2𝑡
𝑃 ( 𝑘=0 )=
𝑘!
=
0!
=𝑒 −1.2 𝑡 ⇒ 1 −𝑒 −1 .2𝑡 ≤ 0.05
− 1 .2𝑡
⇒− 𝑒 ≤ 0.05 −1
− 1.2 𝑡
⇒𝑒 ≥ 0.95
⇒− 1.2 𝑡 ≥ ln ( 0.95 )=− 0.05 1 29
⇒𝑡 ≤ 0.043 seconds
 Poisson or not?

Which of the following are

likely to be well modelled by
a Poisson distribution?

(can click more than one)

Can you tell what is fish, or will you flounder?

1. Number of duds found when I test

four components
2. The number of heart attacks in
Brighton each year
3. The number of planes landing at
Heathrow between 8 and 9am
4. The number of cars getting
punctures on the M1 each year
5. Number of people in the UK flooded 60
0% 0% 0% 0% 0%
out of their home in July
1 2 3 4 5 Countdown
Are they Poisson? Answers:
Number of duds found when I test four components

- NO: this is Binomial

(it is not the number of independent random events in a continuous interval)
The number of heart attacks in Brighton each year

- YES: large population, no obvious correlations between heart attacks in

different people
The number of planes landing at Heathrow between 8 and 9am
- NO: 8-9am is rush hour, planes land regularly to land as many as possible
(1-2 a minute) – they do not land at random times or they would hit each other!
The number of cars getting punctures on the M1 each year
- YES (roughly): If punctures are due to tires randomly wearing thin, then expect
punctures to happen independently at random
But: may not all be independent, e.g. if there is broken glass in one lane

Number of people in the UK flooded out of their home in July

- NO: floodings of different homes not at all independent; usually a small number
of floods each flood many homes at once,
Approximation to the Binomial distribution

The Poisson distribution is an approximation to B(n, p), when n is large and p is small
(e.g. if , say).

In that case, if then Where

i.e. X is approximately Poisson, with mean .

Example

The probability of a certain part failing within ten years is 10 -6. Five million of the parts have
been sold so far.

What is the probability that three or more will fail within ten years?

Answer:

Let X = number failing in ten years, out of 5,000,000;

Evaluating the Binomial probabilities is rather awkward; better to use the Poisson
approximation.

X has approximately Poisson distribution with

P(Three or more fail) ¿ 1 − 𝑃 ( 𝑋=0 ) − 𝑃 ( 𝑋=1 ) − 𝑃 ( 𝑋 =2)

−5 0 −5 1 −5 2
𝑒 5 𝑒 5 𝑒 5
¿1− − −
0! 1! 2!

¿ 1 −𝑒 −5 ( 1+5+1 2.5 )=0.875

For such small and large the Poisson approximation is very accurate
(exact result is also 0.875 to three significant figures).
Poisson Distribution Summary

Describes discrete random variable that is the number of independent and randomly
occurring events, with mean number . Probability of such events is
−𝜆 𝑘
𝑒 𝜆
𝑃 ( 𝑋 =𝑘 )=
𝑘!

Mean and variance:

The sum of Poisson variables is also Poisson, with average number

Approximation to Binomial for large and small :

if then where
Continuous Random Variables

A continuous random variable is a random variable which can take values measured on a
continuous scale e.g. weights, strengths, times or lengths.

For any pre-determined value , , since if we measured accurately enough, we are

never going to hit the value exactly. However the probability of some region of
values near can be non-zero.

Probability density function (pdf): 𝑃 (− 1.5< 𝑋 <−0.7 )

𝑓 ( 𝑥)
Probability of in the range a to b.
Normalization:

Since has to have some value

And since , for a pdf, for all .

Cumulative distribution function (cdf) :

This is the probability of .

𝑥
𝐹 ( 𝑥 ) ≡ 𝑃 ( 𝑋 < 𝑥 )= ∫ 𝑓 ( 𝑥′ ) 𝑑𝑥 ′
−∞

𝐹 ( 𝑥 )=𝑃 ( 𝑋 <− 1.1)

Mean Expected value (mean) of :

Variance Variance of :
∞
¿ ∫ 𝑥 𝑓 ( 𝑥 ) 𝑑𝑥 −𝜇
2 2

−∞

Note: the mean and variance may not be well defined for distributions with broad tails.
Mean
𝑃 ( 𝑋 < 𝑥 )=0.5
The mode is the value of where is
maximum (which may not be unique). Mode

The median is given by the value of x where

𝑃 ( 𝑋 > 𝑥 )=0.5

Median
 Probability density function

Consider the continuous random variable = the weight in

pounds of a randomly selected new-born baby. Let be
the probability density function for .
It is safe to assume that P(X < 0) = 0 and P(X < 20) = 1.
Question from Derek Bruff
Which of the following is not a justifiable conclusion about
given this information?

1. No portion of the graph of can

lie below the x-axis.
2. is non-zero for in the range
3. The area under the graph of
between x = 0 and x = 20 is 1.
4. The non-zero portion of the
graph of lies entirely between =
0 and = 20.
0% 0% 0% 0% 30
1 2 3 4 Countdown
 Probability density function

Consider the continuous random variable = the weight in

pounds of a randomly selected new-born baby. Let be
the probability density function for .
It is safe to assume that P(X < 0) = 0 and P(X < 20) = 1.

Which of the following is not a justifiable conclusion about

given this information?

1. No portion of the graph of can lie below the

x-axis.
- Correct, for all probabilities to be
2. is non-zero for in the range
- Incorrect, can be zero
e.g babies must weigh more than an embryo, so at least

3. The area under the graph of between x = 0 and x = 20 is 1.

- Correct.
4. The non-zero portion of the graph of lies entirely between = 0 and = 20.
- Correct. and
Uniform distribution

The continuous random variable has the Uniform distribution between and , with if

 1 1  x   2

𝑓 ( 𝑥 ) =¿  2   1 otherwise
 0

, for short.

f(x)

 2 x
1
Occurrence of the Uniform distribution

1) Waiting times from random arrival time until a regular event (see later)

2) Simulation: programming languages often have a standard routine for simulating

the U(0, 1) distribution. This can be used to simulate other probability distributions.

Actually not very common

Example: Disk wait times

In a hard disk drive, the disk rotates at 7200rpm. The wait

time is defined as the time between the read/write head
moving into position and the beginning of the required
information appearing under the head.

(a) Find the distribution of the wait time.

(b) Find the mean and standard deviation of the wait time.

(c) Booting a computer requires that 2000 pieces of

information are read from random positions. What is the
total expected contribution of the wait time to the boot
time, and rms deviation?

Answer: (a) Rotation rate of 7200rpm gives rotation time = = 8.33ms.

Wait time can be anything between 0 and 8.33ms and each time in this range is as
likely as any other time.

Therefore, distribution of the wait time is uniform, U(0, 8.33ms)

Mean and variance: for

Mid-point
Proof:

f(x)
𝑦
Let y be the distance from the mid-
point,

and the width be  2 x

1
. 𝑤=𝜃 2 − 𝜃 1

Then since , and means add

𝑤 𝑤

𝜃 2 +𝜃 1
2 2
𝜃 2 +𝜃 1 1 𝜃 1+ 𝜃 2
¿
2
+ ∫ 𝑦 𝑓 ( 𝑦 ) 𝑑𝑦¿
2
+∫ 𝑦
𝑤 𝑤
𝑑𝑦 ¿
2
+0
𝑤 −
− 2
2
Unsurprisingly the mean is the midpoint!
 ∞ 𝜇
𝜎 = ∫ ( 𝑥 − 𝜇 ) 𝑓 ( 𝑥 ) 𝑑𝑥
2
Variance:
2
f(x)
𝑦
−∞
𝑤
2
1
¿ ∫ 𝑦2
𝑤
𝑑𝑦
−
𝑤
 2 x
2 1

[ ]
3 𝑤
1 𝑦 2
𝑤=𝜃 2 − 𝜃 1
¿ 𝑤
𝑤 3 −
2

¿
1 𝑤3 𝑤 3
3𝑤 8
+
8 (
=
𝑤2
12 )
( 𝜃2 − 𝜃1 )2
¿
12
Example: Disk wait times

In a hard disk drive, the disk rotates at 7200rpm. The wait

time is defined as the time between the read/write head
moving into position and the beginning of the required
information appearing under the head.

(b) Find the mean and standard deviation of the wait time.

Answer: (b) http://en.wikipedia.org/wiki/Hard_disk_drive

Example: Disk wait times

In a hard disk drive, the disk rotates at 7200rpm. The wait

time is defined as the time between the read/write head
moving into position and the beginning of the required
information appearing under the head.

(c) Booting a computer requires that 2000 pieces of

information are read from random positions. What is the
total expected contribution of the wait time to the boot
time, and rms deviation?

Answer: (c)

𝜇=4.2 ms For 2000 reads the mean total time is 2000 4.2ms = 8.3s.

Note: rms = Root Mean Square = standard deviation

So the variance is = 20005.8ms2= 0.012s2

⇒ 𝜎 𝑡𝑜𝑡 =√ 0.012 𝑠 2=0.11s