1/16

1 Probability

1.1 Some terms and definitions used in Probability

1) An Experiment is the situation under investigation that leads to

outcomes.

2) An Outcome is the result of an experiment.

3) A Sample Space of an experiment, denoted by 𝑆, is the set with elements

all possible outcomes of the experiment.

4) An Event is any subset of 𝑆. Note that ∅ is called the impossible event and

the subset 𝑆 is called the certain event.

5) Let 𝐴 and 𝐵 be two events. 𝐴 and 𝐵 are Mutually Exclusive if and only if

𝐴 ∩ 𝐵 = ∅.

6) The probability of event 𝐴 to occur is written as 𝑝(𝐴).

1/16
 2/16

1.2 Axioms & Laws of Probability

The axioms are:

1) 𝑝(𝑆) = 1

2) 0 ≤ 𝑝(𝐴) ≤ 1

3) If 𝐴 and 𝐵 are mutually exclusive events then

𝑝(𝐴 or 𝐵) = 𝑝(𝐴 ∪ 𝐵) = 𝑝(𝐴) + 𝑝(𝐵)

And the laws (basic theorems) are:

4) 𝑝(∅) = 0

5) 𝑝(𝐴̅) = 1 − 𝑝(𝐴)

6) i) 𝑝(𝐴 ∪ 𝐵) = 𝑝(𝐴) + 𝑝(𝐵) − 𝑝(𝐴 ∩ 𝐵)

ii) 𝑝(𝐴 ∩ 𝐵) = 𝑝(𝐴) + 𝑝(𝐵) − 𝑝(𝐴 ∪ 𝐵)

7) 𝑝(𝐴 − 𝐵) = 𝑝(𝐴 ∩ 𝐵̅) = 𝑝(𝐴) − 𝑝(𝐴 ∩ 𝐵)

where 𝑝(𝐴 ∪ 𝐵) = 𝑝(𝐴 or 𝐵),

𝑝(𝐴 ∩ 𝐵) = 𝑝(𝐴 and 𝐵) and

𝑝(𝐴̅) = 𝑝(not 𝐴)

2/16
 3/16

Example 1.1

Mercury/Lead Example:

Seawater is analyzed for 2 heavy metals: Lead (𝐿) and Mercury (𝑀). It has

been found that 𝑝(𝐿 ∪ 𝑀) = 0.38, 𝑝(𝐿) = 0.32 and 𝑝(𝑀) = 0.16.

We can therefore find that

𝑝(𝐿̅) = 1 − 0.32 = 0.68

̅ ) = 1 − 0.16 = 0.84
𝑝(𝑀

𝑝(𝐿 ∩ 𝑀) = 𝑝(𝐿) + 𝑝(𝑀) − 𝑝(𝐿 ∪ 𝑀) = 0.32 + 0.16 − 0.38 = 0.10

𝑝(𝑀 − 𝐿) = 𝑝(𝑀) − 𝑝(𝑀 ∩ 𝐿) = 0.06

̅ ) = 𝑝(𝐿 − 𝑀) = 𝑝(𝐿) − 𝑝(𝐿 ∩ 𝑀) = 0.22

𝑝(𝐿 ∩ 𝑀

̅ ) = 𝑝(𝐿) + 𝑝(𝑀
𝑝(𝐿 ∪ 𝑀 ̅ ) − 𝑝(𝐿 ∩ 𝑀
̅ ) = 0.94

̅̅̅̅̅̅̅̅
𝑝(𝐿 ∩ 𝑀) = 1 − 𝑝(𝐿 ∩ 𝑀) = 1 − 0.10 = 0.9

3/16
 4/16

Example 1.2

Roll a fair dice and record the score. Let 𝐴 = {1,3,5}.

1 1 1 1
We can find that 𝑝(𝐴) = 𝑝(1) + 𝑝(3) + 𝑝(5) = + + =
6 6 6 2

Note that we add the probabilities because the outcomes “1”, “3” and “5” are

mutually exclusive.

Example 1.3

Toss a coin 3 times. Find the probability to get 2 heads.

H (3 heads)
H
T (2 heads)
H
H (2 heads)
T
T (1 heads)

H (2 heads)
H
T (1 heads)
T
H (1 heads)
T
T (0 heads)

1
All outcomes in the end are equally likely, i.e. they have a probability of .
8

3
There are 3 outcomes involving 2 heads. So 𝑝(2 𝐻𝑒𝑎𝑑𝑠) =
8

4/16
 5/16

Example 1.4

The duration of pregnancy was recorded for 1669 women, (Newell, D. J.

(1964) JRSS, A, 127, 1-33 and Hand, D.J. ‘Small data sets’ p97)

weeks frequency proportion

10-15 1 0.0006
15-20 2 0.0012
20-25 8 0.0048
25-30 17 0.0102
30-35 84 0.0503
35-40 683 0.4092
40-45 859 0.5147
45-50 14 0.0084
50-55 0 0
55-60 1 0.0006
i) Estimate the probability a pregnancy lasts between 40 − 45 weeks.

𝑝(pregnancy lasts from 40 − 45 ) = 0.5147 = 51.47%

ii) Estimate the probability that a pregnancy lasts 50 or more weeks.

𝑝(pregnancy lasts 50 or more weeks’) = 0 + 0.0006 = 0.06%

iii) Estimate the probability that a pregnancy lasts less than 25 weeks.

𝑝(pregnancy lasts less than 25 weeks) = 0.0006 + 0.0012 + 0.0048

= 0.0066 = 0.66%

iv) Estimate the probability that a pregnancy lasts between 35-50

weeks.

5/16
 6/16

𝑝(pregnancy lasts between 35 − 50 weeks) = 0.4092 + 0.5147 + 0.0084

= 0.9323 = 93.23%

Example 1.5

The hair and eye colour were recorded for a sample of 22361 children in

Aberdeen, Scotland, giving the following frequencies (Goodman, L.A. (1981)

JASA, 76, 320-334 and Hand, D.J. ‘Small data sets’ p146)

HAIR
Dark (𝐷) Not Dark  D 
2579 399 2978
Blue (𝐵)
(0.115) (0.018) (0.133)
EYES

13947 5436 19383

Not Blue  B 
(0.624) (0.243) (0.867)
16526 5835 22361
(0.739) (0.261) (1.000)

Using the table we can find:

𝑝(𝐵) = 0.133

𝑝(𝐷) = 0.739

𝑝(𝐵 ∩ 𝐷) = 0.115

Find

𝑝(𝐵̅ ∪ 𝐷
̅ ) = 𝑝(𝐵
̅̅̅̅̅̅̅̅
∩ 𝐷 ) = 1 − 𝑝(𝐵 ∩ 𝐷) = 0.885

̅ ) = 𝑝(𝐵 − 𝐷) = 𝑝(𝐵) − 𝑝(𝐵 ∩ 𝐷) = 0.018

𝑝(𝐵 ∩ 𝐷

𝑝(𝐵 ∪ 𝐷) = 𝑝(𝐵) + 𝑝(𝐷) − 𝑝(𝐵 ∩ 𝐷) = 0.757

6/16
 7/16

1.3 Conditional Probability

The probability of 𝐴 to occur given that 𝐵 has occurred is written as

𝑝(𝐴 ∩ 𝐵)
𝑝(𝐴|𝐵) =
𝑝(𝐵)

Important results:

1) 𝑝(𝐴|𝐴) = 1
2) 𝑝(𝐴̅|𝐵) = 1 − 𝑝(𝐴|𝐵)
3) 𝑝(𝐴 ∪ 𝐵|𝐶) = 𝑝(𝐴|𝐶) + 𝑝(𝐵|𝐶) − 𝑝(𝐴 ∩ 𝐵|𝐶)
4) 𝑝(𝐴 − 𝐵|𝐶) = 𝑝(𝐴|𝐶) − 𝑝(𝐴 ∩ 𝐵|𝐶)

Example 1.6

Mercury/ Lead example

What is the probability that lead is present in a sample, given that the

sample contains mercury?

𝑝(𝐿 ∩ 𝑀) 0.10
𝑝(𝐿|𝑀) = = = 0.625
𝑝(𝑀) 0.16

What is the probability that mercury is present in a sample, given that the

sample contains lead?

𝑝(𝐿 ∩ 𝑀) 0.10
𝑝(𝑀|𝐿) = = = 0.3125
𝑝(𝐿) 0.32

This demonstrates that 𝑝(𝐿|𝑀) ≠ 𝑝(𝑀|𝐿).

7/16
 8/16

Find

̅ ) 0.22
𝑝(𝐿 ∩ 𝑀
̅) =
𝑝(𝐿|𝑀 = = 0.263
̅)
𝑝(𝑀 0.84

𝑝(𝐿̅ ∩ 𝑀) 0.06
𝑝(𝐿̅|𝑀) = = = 0.375
𝑝(𝑀) 0.16

What is the probability that mercury is NOT present in a sample, given

that the sample contains lead?

̅ |𝐿) = 1 − 𝑝(𝑀|𝐿) = 1 − 0.3125 = 0.6875

𝑝(𝑀

1.4 More Definitions and Theorems (without proof):

1) Events A and B are independent iff 𝑝(𝐴 ∩ 𝐵) = 𝑝(𝐴) ∙ 𝑝(𝐵).

𝑝(𝐴∩𝐵) 𝑝(𝐴)∙𝑝(𝐵)
Implication: 𝑝(𝐴|𝐵) = = = 𝑝(𝐴)
𝑝(𝐵) 𝑝(𝐵)

2) The Multiplication rule: 𝑝(𝐴 ∩ 𝐵) = 𝑝(𝐴|𝐵)𝑝(𝐵) = 𝑝(𝐵|𝐴)𝑝(𝐴)

3) The Bayes Theorem:

𝑝(𝐵|𝐴)𝑝(𝐴)
𝑝(𝐴|𝐵) =
𝑝(𝐵|𝐴)𝑝(𝐴) + 𝑝(𝐵|𝐴̅)𝑝(𝐴̅)

8/16
 9/16

Example 1.7

Roll a die twice

𝑝(𝑔𝑒𝑡 𝑎 5 𝑜𝑛 1𝑠𝑡 𝑟𝑜𝑙𝑙 and 6 on 2nd roll)

1 1 1
= 𝑝(𝑔𝑒𝑡 𝑎 5 𝑜𝑛 1𝑠𝑡 𝑟𝑜𝑙𝑙) ∙ 𝑝(𝑔𝑒𝑡 𝑎 6 𝑜𝑛 2𝑛𝑑 𝑟𝑜𝑙𝑙) = × =
6 6 36

we multiply the probabilities because the outcomes of the different dice

rolls are independent.

Example 1.8

Draw a card from a deck of cards

𝐴: card drawn is a spade ( )

𝐵: card drawn is less or equal to 6

i) Determine if 𝐴 and 𝐵 are independent

13
𝑝(𝐴) =
52
24
𝑝(𝐵) =
52
6
𝑝(𝐴 ∩ 𝐵) =
52
13 24 6
𝑝(𝐴) × 𝑝(𝐵) = × =
52 52 52

Since 𝑝(𝐴 ∩ 𝐵) = 𝑝(𝐴) ∙ 𝑝(𝐵) it means that 𝐴 and 𝐵 are

independent.

9/16
 10/16

ii) Determine if 𝐴̅ and 𝐵 are independent

39 24 18
𝑝(𝐴̅)𝑝(𝐵) = × =
52 52 52
18
𝑝(𝐴̅ ∩ 𝐵) = 𝑝(𝐵 − 𝐴) = 𝑝(𝐵) − 𝑝(𝐴 ∩ 𝐵) =
52

So, they are independent

Example 1.9

In a school 60% are boys. It is certain that if a student is a boy he wears

trousers. Also if a student is a girl, there is a probability of 30% that she

wears trousers. The remaining wear skirts.

a) A teacher sees a student from a distance wearing trousers. Find the

probability the student is a girl.

b) What is the probability that the above student is a boy?

c) Given that a student wears a skirt what is the probability that the

student is a girl.

Solution:

Let 𝑀 be “student is a boy”, and 𝑇 be “Student wears trousers”. We are

given the following

10/16
 11/16

𝑝(𝑀) = 0.6 𝑝(𝑀̅ ) = 0.4

𝑝(𝑇|𝑀) = 1 } ⇒ {𝑝(𝑇̅|𝑀) = 0
̅ ) = 0.3
𝑝(𝑇|𝑀 𝑝(𝑇̅|𝑀̅ ) = 0.7

a)

𝑝(𝑇|𝑀̅ )𝑝(𝑀
̅) 0.3 × 0.4
̅ |𝑇) =
𝑝(𝑀 = = 0.167
̅ )𝑝(𝑀
𝑝(𝑇|𝑀 ̅ ) + 𝑝(𝑇|𝑀)𝑝(𝑀) 0.3 × 0.4 + 1 × 0.6

b)

̅ |𝑇) = 1 − 0.167 = 0.833

𝑝(𝑀|𝑇) = 1 − 𝑝(𝑀

or

𝑝(𝑇|𝑀)𝑝(𝑀) 1 × 0.6
𝑝(𝑀|𝑇) = = = 0.833
̅ )𝑝(𝑀
𝑝(𝑇|𝑀)𝑝(𝑀) + 𝑝(𝑇|𝑀 ̅ ) 1 × 0.6 + 0.3 × 0.4

c)

𝑝(𝑇̅|𝑀̅ )𝑝(𝑀̅) 0.7 × 0.4

̅ |𝑇̅) =
𝑝(𝑀 = =1
𝑝(𝑇̅|𝑀
̅ )𝑝(𝑀̅ ) + 𝑝(𝑇̅|𝑀)𝑝(𝑀) 0.7 × 0.4 + 0 × 0.6

11/16
 12/16

Example 1.10

Of all highway accidents we know that 30% involve Alcohol (𝐴) by at least

one of the drivers. Another accident factor is excessive speed by one of the

drivers involved (𝐸). It is known that, if alcohol is involved there is a 60%

probability that excessive speed is involved; otherwise, it is 10%.

𝑝(𝐴) = 0.3 𝑝(𝐴̅) = 0.7

𝑝(𝐸|𝐴) = 0.6} ⇒ {𝑝(𝐸̅ |𝐴) = 0.4
𝑝(𝐸|𝐴̅) = 0.1 𝑝(𝐸̅ |𝐴̅) = 0.9

a) An accident involves speeding. What is the probability that alcohol is

involved? If every year there is an average of 320 accidents involving

speeding, how many are expected to involve alcohol? i.e. we need

𝑝(𝐴 ∩ 𝐸)
𝑝(𝐴|𝐸) =
𝑝(𝐸)

which cannot be calculated based on the available info.

So we use the Bayes theorem instead

𝑝(𝐸|𝐴)𝑝(𝐴) 0.6 × 0.3

𝑝(𝐴|𝐸) = = = 0.72
𝑝(𝐸|𝐴)𝑝(𝐴) + 𝑝(𝐸|𝐴̅)𝑝(𝐴̅) 0.6 × 0.3 + 0.1 × 0.7

12/16
 13/16

320 × 0.72 = 230.4 ≈ 230 expected accidents will involve alcohol given

that they already involve speeding.

b) What is the probability that an accident involves alcohol given that

speeding is not involved?

𝑝(𝐴 ∩ 𝐸̅ )
𝑝(𝐴|𝐸̅ ) =
𝑝(𝐸̅ )

𝑝(𝐸̅ |𝐴)𝑝(𝐴) 0.4 × 0.3

= = = 0.16
𝑝(𝐸̅ |𝐴)𝑝(𝐴) + 𝑝(𝐸̅ |𝐴̅)𝑝(𝐴̅) 0.4 × 0.3 + 0.9 × 0.7

c) Find 𝑝(𝐴 ∩ 𝐸) and 𝑝(𝐸). Are events 𝐴 and 𝐸 independent? Are they

mutually exclusive?

𝑝(𝐴|𝐸)𝑝(𝐸)
𝑝(𝐴 ∩ 𝐸) = { 𝑜𝑟
𝑝(𝐸|𝐴)𝑝(𝐴) = 0.6 × 0.3 = 0.18

Since

𝑝(𝐴 ∩ 𝐸) 𝑝(𝐴 ∩ 𝐸) 0.18

𝑝(𝐴|𝐸) = ⇒ 𝑝(𝐸) = = = 0.25
𝑝(𝐸) 𝑝(𝐴|𝐸) 0.72

13/16
 14/16

For independence:

i) 𝑝(𝐴|𝐸) = 0.72 ≠ 0.3 = 𝑝(𝐴) ⇒ 𝐴, 𝐸 are not independent

or

ii) 𝑝(𝐸) ∙ 𝑝(𝐴) = 0.25 × 0.3 = 0.7 ≠ 0.18 = 𝑝(𝐴 ∩ 𝐸) ⇒ 𝐴, 𝐸 are not

independent

For mutually exclusive events:

𝑝(𝐴 ∩ 𝐸) ≠ 0 ⇒ 𝐴, 𝐸 are not mutually exclusive

Example 1.11

Suppose that three silicon wafer plants (silicon wafer is a material

essential for manufacturing semiconductors, which are found in all kinds

of electronic devices) produce blank DVDs with plant A manufacturing

45%, plant B manufacturing 30%, and plant C manufacturing 25%. The

probability of a defective DVD given plant A produced the DVD is 0.01, the

probability of a defective DVD given plant B produced the DVD is 0.02, and

the probability of a defective DVD given plant C produced the DVD is 0.05.

Let 𝐷 be the event that a defective DVD is produced.

14/16
 15/16

𝑝(𝐴) = 0.45 ⇒ 𝑝(𝐴̅) = 𝑝(𝐵 ∪ 𝐶) = 𝑝(𝐵) + 𝑝(𝐶) = 0.55

𝑝(𝐵) = 0.30 ⇒ 𝑝(𝐵̅) = 𝑝(𝐴 ∪ 𝐶) = 𝑝(𝐴) + 𝑝(𝐶) = 0.7
𝑝(𝐶) = 0.25 ⇒ 𝑝(𝐶̅ ) = 𝑝(𝐴 ∪ 𝐵) = 𝑝(𝐴) + 𝑝(𝐵) = 0.75
Also
𝑝(𝐷|𝐴) = 0.01 ̅ |𝐴) = 0.99
𝑝(𝐷
̅ |𝐵) = 0.98
𝑝(𝐷|𝐵) = 0.02} ⇒ {𝑝(𝐷
𝑝(𝐷|𝐶) = 0.05 ̅ |𝐶) = 0.95
𝑝(𝐷

a) If a defective DVD was found, the probability that it was

manufactured by plant A is

𝑝(𝐷|𝐴)𝑝(𝐴)
𝑝(𝐴|𝐷) =
𝑝(𝐷|𝐴)𝑝(𝐴) + 𝑝(𝐷|𝐵)𝑝(𝐵) + 𝑝(𝐷|𝐶)𝑝(𝐶)

. 1 × .45
= = 0.196
. 1 × .45 + .2 × .3 + .05 × .25

b) Given that a defective DVD was found, the probability that it was

manufactured by plant B is

𝑝(𝐷|𝐵)𝑝(𝐵)
𝑝(𝐵|𝐷) =
𝑝(𝐷|𝐴)𝑝(𝐴) + 𝑝(𝐷|𝐵)𝑝(𝐵) + 𝑝(𝐷|𝐶)𝑝(𝐶)

. 2 × .3
= = 0.261
. 1 × .45 + .2 × .3 + .05 × .25

15/16
 16/16

c) Of the defective DVDs found, the probability that it was

manufactured by plant C is

𝑝(𝐶|𝐷) = 1 − (𝑝(𝐴|𝐷) + 𝑝(𝐵|𝐷)) = 1 − (.196 + .261) = 0.543

d) Given a defective DVD, what is the probability it has not been

manufactured by plant A?

𝑝(𝐴̅|𝐷) = 1 − 𝑝(𝐴|𝐷) = 1 − .196 = 0.804

e) Say, in a batch of DVDs manufactured by all plants, we find that

there are 1500 defective DVD. How many are expected to have been

manufactured at plant C?

1500 × 𝑝(𝐶|𝐷) = 1500 × .543 = 814.5

f) Say, in a batch of 2000 DVDs produced at plant A, how many are

expected to be defective?

2000 × 𝑝(𝐷|𝐴) = 2000 × .01 = 20

16/16