2DI90: Probability and Statistics

Ellen Cardinaels & Remco van der Hofstad

Department of Mathematics and Computer Science

Week 1: Contents and Learning Objectives

• Sample spaces & events (MR 2.1) • Multiplication and total probability
▶ Counting techniques (MR 2.2) rules (MR 2.6)
• The concept of probability • Independence (MR 2.7)
▶ Axioms of probability
▶ Algebraic rules for probabilities • Bayes’ Theorem (MR 2.8)
• Addition rules (MR 2.4) ▶ Simpson’s Paradox
• Conditional probability (MR 2.5)

2 2DI90: Probability and Statistics

Axioms of Probability

Axioms of Probability
In any sample space S with numbers P(E ) assigned to all events E , P can be interpreted
as a probability, if it satisfies
(i) P(S) = 1.
(ii) 0 ≤ P(E ) ≤ 1 for any event E .
(iii) For two events E1 , E2 with E1 ∩ E2 = ∅:

P(E1 ∪ E2 ) = P(E1 ) + P(E2 ).

3 2DI90: Probability and Statistics

Conclusions
Lemma (Probability of complement and empty set)
Let S be the sample space and E ⊂ S any event. The following properties hold:
′
(i) P(E ) = 1 − P(E ), where E ′ is the complement of E .
(ii) P(∅) = 0.

Proof:

4 2DI90: Probability and Statistics

Addition Rules (MR 2.4)

Lemma (Probability of the union of two events)

Let A, B ⊂ S be two events. Then

P(A ∪ B) = P(A) + P(B) − P(A ∩ B).

If A and B are mutually exclusive events (i.e., A ∩ B = ∅)

P(A ∪ B) = P(A) + P(B).

5 2DI90: Probability and Statistics

Conditional Probability (MR 2.5)
Example: You investigate the efficacy of a new vaccine.
What is the probability that someone is infected, given that they are vaccinated?

6 2DI90: Probability and Statistics

Conditional Probability (MR 2.5)
Example: You investigate the efficacy of a new vaccine.
What is the probability that someone is infected, given that they are vaccinated?

Definition (Conditional probability)

The probability of event B, given the information that the outcome is in event A, is
denoted as
P(B|A).
P(B|A) is called the conditional probability of B given A.

6 2DI90: Probability and Statistics

Conditional Probability (MR 2.5)
Example: You investigate the efficacy of a new vaccine.
What is the probability that someone is infected, given that they are vaccinated?

Definition (Conditional probability)

The probability of event B, given the information that the outcome is in event A, is
denoted as
P(B|A).
P(B|A) is called the conditional probability of B given A.

What is A in the example? What is B?

6 2DI90: Probability and Statistics

Conditional Probability

Definition (Conditional probability, calculation)

The conditional probability of event B given the occurrence of event A is

P(A ∩ B)
P(B|A) = .
P(A)

−→ Interpretation: P(B|A) is the relative frequency of B among the trials that produce
an outcome in event A.

7 2DI90: Probability and Statistics

Conditional Probability - Example revisited
Back to vaccine example: Assume that there are 900 people, of which 600 were
vaccinated. After a fixed amount if time, we observe which persons have been infected
afterwards:

Infection \ Vaccine yes no sum

yes 60 120 180
no 540 180 720
sum 600 300 900

8 2DI90: Probability and Statistics

Conditional Probability - Example revisited
Suppose we choose one person at random (Laplace experiment).
• Question 1: What is the probability that the person is vaccinated?
• Question 2: You get the information that the person is vaccinated. Based on this
information, what is the probability that the person has been infected?

Infection \ Vaccine yes no sum

yes 60 120 180
no 540 180 720
sum 600 300 900

9 2DI90: Probability and Statistics

Conditional Probability - Example revisited

• Question 3: You get the information that the person had been infected. Based on
only this information, what is the probability that the person is vaccinated?

Infection \ Vaccine yes no sum

yes 60 120 180
no 540 180 720
sum 600 300 900

10 2DI90: Probability and Statistics

Multiplication Rules (MR 2.6)
Let A and B be two events in a sample space S.

Lemma (Multiplication Rule)

P(A ∩ B) = P(B|A)P(A) = P(A|B)P(B)

11 2DI90: Probability and Statistics

Multiplication Rules (MR 2.6)
Let A and B be two events in a sample space S.

Lemma (Multiplication Rule)

P(A ∩ B) = P(B|A)P(A) = P(A|B)P(B)

Lemma (Total Probability Rule)

P(B) = P(B ∩ A) + P(B ∩ A′ ) = P(B|A)P(A) + P(B|A′ )P(A′ )

11 2DI90: Probability and Statistics

Multiplication Rules

12 2DI90: Probability and Statistics

Multiplication Rules

Lemma (Total Probability Rule (Multiple Events))

Assume E1 , E2 ,. . . , Ek are k mutually exclusive and exhaustive sets, that is

S = E1 ∪ E2 ∪ · · · ∪ Ek and Ei ∩ Ej = ∅ ∀i ̸= j.

Then for any event B,

P(B) = P(B ∩ E1 ) + P(B ∩ E2 ) + · · · + P(B ∩ Ek )
= P(B|E1 )P(E1 ) + P(B|E2 )P(E2 ) + · · · + P(B|Ek )P(Ek ).

13 2DI90: Probability and Statistics

Independence (MR 2.7)
Example: A card is drawn from a perfectly shuffled deck of 52 cards. Let

A = {Selected card is a queen} ,

B = {Selected card is of hearts} .

• P(A) =
• P(B) =
• P(A ∩ B) =
• P(A|B) =

14 2DI90: Probability and Statistics

Independence
Definition (Independence)
Two events are independent if any of the following equivalent properties is satisfied:
(i) P(A|B) = P(A)
(ii) P(B|A) = P(B)
(iii) P(A ∩ B) = P(A)P(B)

15 2DI90: Probability and Statistics

Independence
Definition (Independence)
Two events are independent if any of the following equivalent properties is satisfied:
(i) P(A|B) = P(A)
(ii) P(B|A) = P(B)
(iii) P(A ∩ B) = P(A)P(B)

Definition (Independence (multiple events))

Events E1 , E2 , . . . , En are called independent if for any subset {Ei1 , . . . , Eik } it is true that
P(Ei1 ∩ Ei2 ∩ · · · ∩ Eik ) = P(Ei1 )P(Ei2 ) · · · P(Eik ).

15 2DI90: Probability and Statistics

Independence
Lemma
The following statements are equivalent:
(i) A and B are independent.
(ii) P(A ∩ B) = P(A)P(B).
(iii) P(A′ ∩ B) = P(A′ )P(B).
(iv) P(A ∩ B ′ ) = P(A)P(B ′ ).
(v) P(A′ ∩ B ′ ) = P(A′ )P(B ′ ).

Proof: Exercise :-)

16 2DI90: Probability and Statistics

Independence - Example
In MetaForum, there are three elevators.
Elevator 1 is currently broken.
Elevators 2 and 3 fail independently with probability 0.2 and 0.1, respectively.

17 2DI90: Probability and Statistics

Independence - Example
In MetaForum, there are three elevators.
Elevator 1 is currently broken.
Elevators 2 and 3 fail independently with probability 0.2 and 0.1, respectively.

What is the probability that you cannot get to the 7th floor by elevator?

17 2DI90: Probability and Statistics

Independence - Addition Rules

Lemma (Probability of the union of independent events)

Let E1 , E2 ⊂ S be two independent events. Then

P(A ∪ B) = P(A) + P(B) − P(A) · P(B).

In general, for independent events E1 , E2 , . . . , Ek ,

∏
k
P(E1 ∪ E2 ∪ · · · ∪ Ek ) = 1 − (1 − P(Ei )).
i=1

18 2DI90: Probability and Statistics

Bayes Theorem (MR 2.8)
In many situations we can only observe specific conditional probabilities.

19 2DI90: Probability and Statistics

Bayes Theorem (MR 2.8)
In many situations we can only observe specific conditional probabilities.

Lemma (Bayes’ Theorem)

Let A and B be two events and P(B) > 0. Then

P(B|A)P(A)
P(A|B) =
P(B)

19 2DI90: Probability and Statistics

Bayes Theorem (MR 2.8)
In many situations we can only observe specific conditional probabilities.

Lemma (Bayes’ Theorem)

Let A and B be two events and P(B) > 0. Then

P(B|A)P(A)
P(A|B) =
P(B)

Proof: Follows from P(A ∩ B) = P(A|B)P(B) = P(B ∩ A) = P(B|A)P(A).

19 2DI90: Probability and Statistics

Bayes’ Theorem

Lemma (Bayes’ Theorem - Generalization)

Let A be an event such that P(A) > 0 and let E1 , E2 , . . . , Ek be k mutually exclusive
and exhaustive sets, that is

S = E1 ∪ E2 ∪ . . . ∪ Ek and Ei ∩ Ej = ∅ ∀i ̸= j.

Then
P(A|E1 )P(E1 )
P(E1 |A) = .
P(A|E1 )P(E1 ) + P(A|E2 )P(E2 ) + · · · + P(A|Ek )P(Ek )

20 2DI90: Probability and Statistics

Example1

A 65-year old non-smoker exhibits occasional breathlessness. A lung biopsy is conducted

indicating either lung cancer or sarcoidosis, a common but unfatal lung disease.

21 2DI90: Probability and Statistics

1
HARDEO SAHAI: Teaching Bayes’ Theorem using Examples in Medical Diagnosis
Example1

A 65-year old non-smoker exhibits occasional breathlessness. A lung biopsy is conducted

indicating either lung cancer or sarcoidosis, a common but unfatal lung disease.

We formalise the disease states by

E1 = {Normal} ,
E2 = {Lung Cancer} ,
E3 = {Sarcoidosis}

and the event E = {Occasional breathlessness}.

21 2DI90: Probability and Statistics

1
HARDEO SAHAI: Teaching Bayes’ Theorem using Examples in Medical Diagnosis
Example
From clinical experience, we know

• P(E |E1 ) = 0.002,

• P(E |E2 ) = 0.9,
• P(E |E3 ) = 0.95,
• P(E1 ) = 0.991,
• P(E2 ) = 0.001,
• P(E3 ) = 0.008.

Under the presence of symptoms consistent with biopsy, what are the probabilities for the
diagnoses E1 , E2 and E3 ?

22 2DI90: Probability and Statistics

Example: Kidney Stone Treatment

Which treatment is better?

23 2DI90: Probability and Statistics

Treatment A

24 2DI90: Probability and Statistics

Treatment B

25 2DI90: Probability and Statistics

Example: Simpson’s Paradox
Summarized:

Figure: Wikipedia

Treatment B is better overall, although it’s worse in each group!

26 2DI90: Probability and Statistics

Example: Simpson’s Paradox

Figure: J. v. Kügelen, L. Gresele, B. Schölkopf: Simpson’s paradox in Covid - 19 case fatality rates: a mediation analysis of age-related
causal effects. IEEE transactions on artificial intelligence 2.1 (2021): 18-27.

27 2DI90: Probability and Statistics

Example: Simpson’s Paradox

Figure: J. v. Kügelen, L. Gresele, B. Schölkopf: Simpson’s paradox in Covid - 19 case fatality rates: a mediation analysis of age-related
causal effects. IEEE transactions on artificial intelligence 2.1 (2021): 18-27.

28 2DI90: Probability and Statistics

Monty Hall Paradox

Illustration by San Tuon

29 2DI90: Probability and Statistics

What did we learn today?
• Conditional probabilities, Bayes’ Theorem
• Independent events
• Simpson’s paradox
Suppose we need to test a program designed to search for solutions within a specific set
of problems. The algorithm correctly reports a found solution for 60% of the problems
that are actually solvable. However, it also incorrectly reports a solution for 4% of the
problems that are unsolvable. Given that 20% of the problems in this set are solvable,
determine the following probabilities:
(a) The algorithm reports a solution.
(b) A problem is solvable given that the algorithm reports a solution.
(c) A problem is unsolvable given that the algorithm does not report a solution.

30 2DI90: Probability and Statistics