Dr.Tasnim Hasan Lec.

12

Chapter Seven
Introduction to Probability Theory

Terminology and Notations in Elementary Set Theory

Experiment:
A random experiment is an operation whose output cannot be predicted
with certainty. It is the process that generates observations.
Example: Tossing a coin once or several times, selecting a card or cards
from a deck, obtaining blood types from a group of individuals…etc.
Outcome:
Output of an experiment is called outcome. The number of outcomes
depends upon the nature of the experiment.
Sample Space:
The set of all possible outcomes of an experiment is called the sample
space of the experiment, and is denoted by Ω. Each distinct outcome is
called element or point of the sample space.
A sample space is said to be discrete if it consists of a finite number of
sample points or countably infinite sample points.
A set is called countable if its elements can be placed in a one-to-one
correspondence with the positive integers.
Example 1.
In an experiment of tossing a coin, the possible outcomes are
Ω = {H, T}.If we have two coins, then we would get
Ω = {HH, HT, TH, TT}.
Example 2. In the experiment of rolling a die, the sample space is
Ω = {1, 2, 3, 4, 5, 6}. If the experiment consists of tossing two dice, then
the sample space consists of the 36 points.
Ω = {(i, j): i, j = 1, 2, 3, 4, 5, 6}.
Dr.Tasnim Hasan Lec. 12

Example 3. In the experiment of tossing a coin repeatedly and counting

the number of tosses until the first head appears.
Ω = {1, 2, 3 …}
Note that there are an infinite number of outcomes.
Example 4. In the experiment consists of measuring the lifetime t (in
hours) of an electronic equipment Ω = {t: 0 ≤ t < ∞} which is continuous
and infinite.
Event:
An event is any collection (subset) of outcomes contained in the sample
space Ω, i.e. A is an event iff 𝐴 ⊆ Ω.
An event is said to be simple if it consists of exactly one outcome and
compound if it consists of more than one outcome.
Example:
When a single regular die is rolled once
Ω = {1, 2, 3, 4, 5, 6}. Then each subset A = {1}, B = {2, 4, 6},
C = {1, 3, 5}, D = {1, 2, 4, 5}. A is a simple event while B, C and D are
compound events.

Some Relations from Set Theory:

Let now Ω be an abstract universal set and A, B etc. denote sets,

collections of elements in Ω.
1. 𝜔 ∈ 𝐴 means that an element ω belongs to a set A. 𝜔 ∉ 𝐴
means that ω does not belong to a set A.
2. ∅ denotes the empty set, which has no elements.
3. Ac is the complement set of A. It consists of all elements ω that
do not belong to A.
4. A ⊆ B means that A is a subset of B. This means that if 𝜔 ∈ 𝐴,
then 𝜔 ∈ 𝐵. In addition, we have for any set A ⊆ Ω.
Note that A ⊆ B and B ⊆ A if and only if A = B.
We use also on occasion the notation of strict inclusion, A is a
Dr.Tasnim Hasan Lec. 12

proper subset of B denoted by A⊂B, which means that all elements

of A are in B, but not all elements of B are in A ( A ≠ B).
5. If A ⊆ B, then Bc ⊆ Ac.
6. The union of two sets A and B denoted by A  B is the event consisting
of all elements ω such that 𝜔 ∈ 𝐴 or 𝜔 ∈ 𝐵 or in both events.
For a sequence of sets A1, A2, … the union ⋃∞ 𝑖=1 𝐴𝑖 = 𝐴1 ∪ 𝐴2 ∪ …
consists of elements ω such that there is at least one Ai such that 𝜔 ∈ 𝐴𝑖 .
7. The intersection of two sets A and B denoted A∩B (or AB) is the event
consisting of all elements ω such that 𝜔 ∈ 𝐴 and 𝜔 ∈ 𝐵.
For a sequence of sets A1, A2, … the intersection ⋂∞ 𝑖=1 𝐴𝑖 = 𝐴1 ∩ 𝐴2 ∩
… consists of the elements ω such that 𝜔 ∈ 𝐴𝑖 for all i.

Mutually Exclusive Events: (Disjoint events)

Two events A and B are said to be mutually exclusive or disjoint if the
occurrence of A precludes the occurrence of B and vice-versa,
A∩B = Ø i.e., they cannot occur simultaneously.
The sets A1, A2, … , An are pairwise disjoint if all pairs Ai, Aj are disjoint
for i ≠j.

Example:
- In the random experiment of tossing a coin, the two events
A = {H}, B = {T} are mutually exclusive.
- Suppose that Ω = {1, 2, 3, 4}. If:
A = {1, 2}, B = {1, 3, 4} and C = {4}
Then, A and B are subsets of Ω; A ⊆ Ω and B ⊆ Ω, 2  A, 3  A, A  B =
{1, 2, 3, 4}, A∩B = {1}, A∩C = Ø, Ac= {3, 4}, A  Ac = Ω, A and C are
disjoint or mutually exclusive.
Venn diagrams: A Venn diagram is a graphical representation that is
often useful for discussing the concepts of set relationships:
Dr.Tasnim Hasan Lec. 12

A B A∩B Ac A∩B = Ø
Laws of the Set Relations:

1. Commutative Laws
A B = B A A∩B = B∩A
2. Associative Laws
A  (B  C) = (A  B)  C A∩(B∩C) = (A∩B)∩C
3. Distributive laws
A∩ (B  C) = (A∩B)  (A∩C)
A  (B∩C) = (A  B) ∩ (A  C)
4. Identity Laws
A Ø = A A∩Ø = Ø A Ω = Ω A∩Ω= A
5. Complement laws
A  Ac = Ω A∩Ac = Ø (Ac)c = A Ωc=Ø Øc = Ω
6. Demorgan`s Laws
(A  B)c = Ac∩Bc and (A∩B)c = Ac  Bc

(A  B)c (A∩B)c

For a sequence of sets A1, A2, …

𝑐
(⋃∞ 𝑐 ∞ 𝑐
𝑖=1 𝐴𝑖 ) = ⋂𝑖=1 𝐴𝑖 and (⋂ ∞ ∞ 𝑐
𝑖=1 𝐴𝑖 ) = ⋃𝑖=1 𝐴𝑖
 Dr.Tasnim Hasan Lec. 12

Power set of Ω: The family of all subsets of Ω is known as the power set
of Ω.
𝒫(Ω) = {A: A⊆Ω}
In general, if Ω is a finite set and has n elements then 𝒫(Ω) will have 2n
elements.
Example:
Ω = {a, b} then 𝒫 (Ω) = {{a}, {b}, Ω, ϕ}

Algebras of sets:
Event space: An event space is a sigma field obtained from a sample
space. An element of the event space is called an event.
An event in probability corresponds to a set in set theory.
Remark:
. The sample space Ω is called the sure event.
. The empty set ϕ is called the impossible event.
. The operations , ∩ , ⊂, … on any number of events are also events.

Sigma Field: Let Ω denote a universal set. A collection 𝒜 of subsets of

Ω is called a σ- algebra or a σ- field if
1. 𝜙 ≠ 𝒜 ⊆ 𝒫 (Ω)
2. If A ∈ 𝒜, then Ac ∈ 𝒜.
3. If A1, A2, …,An, … ∈ 𝒜, then ⋃∞ 𝑖=1 𝐴𝑖 ∈ 𝒜 .

Examples:
1. 𝒜 = {ϕ, Ω} is the smallest σ – field (smallest event space).
2. 𝒜 = 𝒫 (𝛺) is the largest σ – field on Ω.
3. Let A ⊂ Ω, then 𝒜 = {A, Ac, Ω, ϕ} is the σ – field generated by the
set A.
4. Let Ω = {1, 2, 3}. Then
Dr.Tasnim Hasan Lec. 12

𝒜 = {{1}, {2, 3}, Ω, ϕ} is a σ – field.

But {{1}, {2}, {3},{1, 2, 3}, ϕ} is not a σ – field.

Properties of σ- Fields
Let A be a σ – field over Ω then:
1. 𝛺∈𝒜
Proof: Since 𝒜 is a σ – field, there exist a set A∈ 𝒜.
Hence Ac ∈ 𝒜.
Consider the infinite sequence of elements A, Ac, Ac, Ac, … ∈ 𝒜
𝐴 ∪ 𝐴𝑐 ∪ 𝐴𝑐 ∪ 𝐴𝑐 … ∈ 𝒜
but 𝐴 ∪ 𝐴𝑐 ∪ 𝐴𝑐 ∪ 𝐴𝑐 … = 𝐴 ∪ 𝐴𝑐 = 𝛺.
Hence 𝛺 ∈ 𝒜

2. 𝜙∈𝒜
Proof: Since 𝛺 ∈ 𝒜 then 𝛺𝑐 ∈ 𝒜
But Ωc = ϕ, hence 𝜙 ∈ 𝒜

3. ∀ 𝐴𝑖 (𝑖 = 1, 2, … )𝑠𝑢𝑐ℎ 𝑡ℎ𝑎𝑡 𝐴𝑖 ∈ 𝒜, 𝑡ℎ𝑒𝑛 ⋂∞ 𝑖=1 𝐴𝑖 ∈ 𝒜

Proof: Since 𝐴𝑖 ∈ 𝒜 for all i, then 𝐴𝑐𝑖 ∈ 𝒜 for all i.
And ⋃∞ 𝑐
𝑖=1 𝐴𝑖 ∈ 𝒜
So (⋃∞ 𝑐 𝑐
𝑖=1 𝐴𝑖 ) ∈ 𝒜
And by De Morgan’s law ⋂∞ 𝑖=1 𝐴𝑖 ∈ 𝒜

4. ∀ 𝐴𝑖 (𝑖 = 1, 2, … , 𝑛)𝑠𝑢𝑐ℎ 𝑡ℎ𝑎𝑡 𝐴𝑖 ∈ 𝒜, 𝑡ℎ𝑒𝑛 ⋃𝑛𝑖=1 𝐴𝑖 ∈ 𝒜

Proof: Consider the infinite sequence of elements
A1, A2, A3, …, An, ϕ, ϕ, ϕ, …which belong to 𝒜
Then 𝐴1 ∪ 𝐴2 ∪ 𝐴3 ∪ … ∪ 𝐴𝑛 ∪ ∅ ∪ ∅ ∪ … = ⋃𝑛𝑖=1 𝐴𝑖
Hence ⋃𝑛𝑖=1 𝐴𝑖 ∈ 𝒜
 Dr.Tasnim Hasan Lec. 12

5. ∀ 𝐴𝑖 (𝑖 = 1, 2, … , 𝑛)𝑠𝑢𝑐ℎ 𝑡ℎ𝑎𝑡 𝐴𝑖 ∈ 𝒜, 𝑡ℎ𝑒𝑛 ⋂𝑛𝑖=1 𝐴𝑖 ∈ 𝒜

Proof: Since ⋃𝑛𝑖=1 𝐴𝑖 ∈ 𝒜 then ⋃𝑛𝑖=1 𝐴𝑐𝑖 ∈ 𝒜 and (⋃𝑛𝑖=1 𝐴𝑐𝑖 )𝑐 ∈ 𝒜
And by De Morgan’s law ⋂𝑛𝑖=1 𝐴𝑖 ∈ 𝒜 as required.

Measurable Space:
The pair (Ω,𝒜) of a sample space Ω and an event space 𝒜 is called a
measurable space.

Sigma field generated by Ȼ

Given a sample space Ω, consider an arbitrary class Ȼ of subsets of Ω.
The smallest sigma field containing all of the sets in Ȼ is called the sigma
field generated by Ȼ and is denoted by σ(Ȼ). Here, by smallest we mean
that if a sigma field contains Ȼ, then it also contains σ (Ȼ).
𝒜 is generated by the class Ȼ over Ω means that
1. 𝒜 is a σ – field over Ω.
2. Ȼ ⊆ 𝒜
3. If 𝒜 1 is a σ – field over Ω, such that Ȼ ⊆ 𝒜 1 then 𝒜 ⊆ 𝒜1 .
Example:
Suppose that Ω = {a, b, c}.
Let Ȼ1= {{b}}, Ȼ2 = {{a}, {b}}, Ȼ3 = {{a}, {c}}
Find 𝒫(Ω), σ-(Ȼ1), σ-(Ȼ2) σ-(Ȼ3)
𝒫(Ω)= {{a}, {b}, {c}, {a, b}, {a, c}, {b, c}, Ω, ϕ}
σ-(Ȼ1)={{b}, {a, c}, Ω, ϕ}
σ-(Ȼ2)= {{a}, {b}, {b, c}, {a, c},{a, b}, {c}, Ω, ϕ}
σ-(Ȼ3)= {{a}, {c}, {b, c}, {a, b}, {a, c}, {b}, Ω, ϕ}
H.W.
1. If A = {1, 2, 3} find the power set of A.
2. Let Ω = {a, b, c, d} and let Ȼ = {{a}, {c, d}}, find σ-(Ȼ).
Dr. Tasnim Hasan Lec. 13

Probability measure space:

A probability space is given by the triple (Ω, 𝒜, P) where Ω is a set of
outcomes, 𝒜 is set of subsets of Ω, the set of possible events and P is a
function assigning probabilities to events P: 𝒜 → [0, 1]. 𝒜 is taken to be
a σ- field.

Probability measure:
For the measurable space (Ω,𝒜), a set function is called a probability
measure if it assigns a real number P(A) to a set 𝐴 ∈ 𝒜
P: 𝒜 → [0, 1] under the constraint of the following axioms
1. If 𝐴 ∈ 𝒜 , P(A) ≥ 0
2. P(Ω)=1
3. ∀ 𝐴𝑖 (𝑖 = 1, 2, … )𝑠𝑢𝑐ℎ 𝑡ℎ𝑎𝑡 𝐴𝑖 ∈ 𝒜, and Ai ∩ Aj = ϕ ∀ i ≠ j
then 𝑃(⋃∞ ∞
𝑖=1 𝐴𝑖 ) = ∑𝑖=1 𝑃 (𝐴𝑖 )

Theorem:
Let (Ω, 𝒜, P) be a probability space, then
1. P(ϕ) = 0
Proof:
Consider the infinite sequence of events A1, A2, … such that Ai = ϕ for i =
1, 2, … ; then by axiom 3
𝑃(𝜙) = 𝑃(⋃∞ ∞ ∞
𝑖=1 𝐴𝑖 ) = ∑𝑖=1 𝑃 (𝐴𝑖 ) = ∑𝑖=1 𝑃(∅),
which can hold only if P(ϕ) = 0

2. For any finite sequence of n disjoint events A1, A2, …, An

𝑃(⋃𝑛𝑖=1 𝐴𝑖 ) = ∑𝑛𝑖=1 𝑃(𝐴𝑖 )
Proof:
Consider the infinite sequence of events A1, A2, … in which
A1, A2, …, An are the n given disjoint events and Ai = ϕ for i > n. Then
Dr. Tasnim Hasan Lec. 13

⋃∞ 𝑛
𝑖=1 𝐴𝑖 = ⋃𝑖=1 𝐴𝑖 , and by axiom 3

𝑃(⋃𝑛𝑖=1 𝐴𝑖 ) = 𝑃(⋃∞ ∞
𝑖=1 𝐴𝑖 ) = ∑𝑖=1 𝑃 (𝐴𝑖 )

= ∑𝑛𝑖=1 𝑃(𝐴𝑖 ) + ∑∞
𝑖=𝑛+1 𝑃 (𝐴𝑖 )

= ∑𝑛𝑖=1 𝑃(𝐴𝑖 ) + 0

= ∑∞
𝑖=1 𝑃 (𝐴𝑖 )

3. For every event A, P(Ac) = 1 – P(A)

A  Ac = Ω and A ∩ Ac = ϕ
P(Ω) = P(A  Ac) = P(A) + P(Ac)
Since, P(Ω) = 1
Hence, P(Ac) = 1 – P(A).

4. If A and B are events in 𝒜, and A  B, then

P(A) ≤ P(B)
Proof:
Since A  B, we can write
B = A  (Ac∩B) Ac∩B
P(B) = P(A) + P(Ac∩B) ( A and Ac∩B are mutually exclusive)
Since P(Ac∩B) ≥ 0
Hence, P(B) ≤ P(A)

5. For any event A, 0 ≤ P(A) ≤ 1 for any event A.

Proof:
For any event A in Ω we have
ØA Ω
Dr. Tasnim Hasan Lec. 13

P(Ø) ≤ P(A) ≤ P(Ω)

Since P(Ø) = 0 and P(Ω) = 1
Hence, 0 ≤ P(A) ≤ 1.
6. For any two events A and B, P(A  B) = P(A) + P(B) - P(A∩B).
Proof:
A  B = A  (B∩Ac)
P(A  B) = P(A) + PB∩Ac) (Disjoint)
B = (B∩A)  (B∩Ac)
P(B) = P(B∩A) + P(B∩Ac) (Disjoint) B∩Ac
P(B∩Ac) = P(B) - P(B∩A)
P(A  B) = P(A) + P(B) - P(B∩A)
And if A and B are mutually exclusive, then
P(A  B) = P(A) + P(B), since P(A∩B) = 0

7. If A, B, and C are any three events in 𝒜, then

P(A  B  C) = P(A)+P(B)+P(C)–P(A∩B)–P(A∩C)–P(B∩C)+ P(A∩B∩C)
Proof:
A  B  C = A  (B  C)
P(A  B  C) = P(A) + P(B  C) – P[A∩(B  C)]
= P(A) + [P(B) + P(C) – P(B∩C)] – P[(A∩B)  (A∩C)]
= P(A) + P(B) + P(C) – P(B∩C) –[P(A∩B) + P(A∩C) – P(A∩B∩C)]
= P(A) + P(B) + P(C) – P(A∩B) – P(A∩C) - P(B∩C) + P(A∩B∩C)
Dr. Tasnim Hasan Lec. 13

Continuity from below and continuity from above:

A probability measure has further the following properties:
Let An, n = 1, 2, … be an infinite sequence of events
1. If A1  A2  … , then 𝑃(⋃∞ 𝑛=1 𝐴𝑛 ) = lim 𝑃 (𝐴𝑛 )
𝑛→∞
2. If 𝐴1 ⊃ 𝐴2 ⊃ ⋯ , then 𝑃(⋂∞
𝑛=1 𝐴𝑛 ) = lim 𝑃(𝐴𝑛 )
𝑛→∞

Proof: (1)
𝐴1 ∪ 𝐴2 ∪ … = (𝐴1 | 𝐴∘ ) ∪ (𝐴2 |𝐴1 ) ∪ …
where 𝐴∘ = ϕ

∞
𝑃(⋃∞
𝑛=1 𝐴𝑛 ) = 𝑃 (⋃𝑘=1 𝐴𝑘 |𝐴𝑘−1 )

= ∑∞ 𝑛
𝑘=1 𝑃 (𝐴𝑘 |𝐴𝑘−1 ) = lim ∑𝑘=1 𝑃 (𝐴𝑘 |𝐴𝑘−1 ) … (1)
𝑛→∞

𝐴𝑛 = (𝐴1 |𝐴∘ ) ∪ (𝐴2 | 𝐴1 ) ∪ … ∪ (𝐴𝑛 |𝐴𝑛−1 )

𝑃(𝐴𝑛 ) = 𝑃(⋃𝑛𝑘=1 𝐴𝑘 |𝐴𝑘−1 ) = ∑𝑛𝑘=1 𝑃(𝐴𝑘 |𝐴𝑘−1 ) … (2)

From (1) and (2) we get;

𝑃(⋃∞
𝑛=1 𝐴𝑛 ) = lim 𝑃(𝐴𝑛 )
𝑛→∞

Proof: (2)
𝐴1 ⊃ 𝐴2 ⊃ ⋯ = 𝐴1𝑐 ⊂ 𝐴𝑐2 ⊂ ⋯
𝑃(⋃∞ 𝑐 𝑐
𝑛=1 𝐴𝑛 ) = lim 𝑃(𝐴𝑛 )
𝑛→∞

By De Morgan’s law ⋃∞ 𝑐 ∞
𝑛=1 𝐴𝑛 = (⋂𝑛=1 𝐴𝑛 )
𝑐

𝑃(⋃∞ 𝑐 ∞ 𝑐 ∞
𝑛=1 𝐴𝑛 ) = 𝑃 (⋂𝑛=1 𝐴𝑛 ) = 1 − 𝑃 (⋂𝑛=1 𝐴𝑛 )
Dr. Tasnim Hasan Lec. 13

lim 𝑃(𝐴𝑐𝑛 ) = lim (1 − 𝑃(𝐴𝑛 )) = 1 − lim 𝑃(𝐴𝑛 )

𝑛→∞ 𝑛→∞ 𝑛→∞

Therefore
1 − 𝑃(⋂∞
𝑛=1 𝐴𝑛 ) = 1 − lim 𝑃 (𝐴𝑛 )
𝑛→∞

Hence, 𝑃(⋂∞
𝑛=1 𝐴𝑛 ) = lim 𝑃 (𝐴𝑛 )
𝑛→∞

Equiprobable Space: P is an equiprobable function on (Ω, 𝒫 (𝛺)), if

1. Ω is a non – empty finite set.
#𝐴
2. P: 𝒫 (𝛺) → [0, 1], such that ∀ 𝐴 ∶ 𝐴 ∈ 𝒫 (𝛺) → 𝑃(𝐴) =
#𝛺
Problem: Verify that P is a probability measure function.
P: 𝒫 (𝛺) → [0, 1]
#𝐴
1.∀ 𝐴: 𝐴 ∈ 𝒫 (𝛺), 𝑃(𝐴) = ≥0
#𝛺
#𝛺
2. 𝑃(𝛺) = =1
#𝛺

3. Let 𝐴1, 𝐴2 , … be infinite sequence of disjoint events (Ai ∩ Aj = ϕ ∀ i

≠ j) belong to 𝒫 (𝛺).
Since Ω is finite
∃ 𝑛 𝑠𝑢𝑐ℎ 𝑡ℎ𝑎𝑡 ∀ 𝑘: 𝑘 = 1, 2, … → An+k = ϕ
# ⋃𝑛
𝑖=1 𝐴𝑖
then 𝑃(⋃∞ 𝑛
𝑖=1 𝐴𝑖 ) = 𝑃 (⋃𝑖=1 𝐴𝑖 ) = #𝛺
𝑛
∑𝑛𝑖=1 #𝐴𝑖 #𝐴𝑖
= =∑
#𝛺 #𝛺
𝑖=1
𝑛

= ∑ 𝑃(𝐴𝑖 ) + 0 + 0 + ⋯
𝑖=1
𝑛

= ∑ 𝑃(𝐴𝑖 ) + 𝑃(𝜙) + 𝑃(𝜙) + …

𝑖=1
Dr. Tasnim Hasan Lec. 13
𝑛 ∞

= ∑ 𝑃(𝐴𝑖 ) + 𝑃(𝐴𝑛+1 ) + 𝑃(𝐴𝑛+2 ) + … = ∑ 𝑃(𝐴𝑖 )

𝑖=1 𝑖=1
Thus (Ω, 𝒫 (𝛺), P) is a probability measure space.
Dr. Tasnim Hasan Lec. 14

Relative frequency definition of probability (Statistical probability):

If n represents sufficiently large number of trials made to see whether
an event A occurs or not, and m represents the number of trials in which
A is observed to occur then the probability of occurrence of A is given by
m
P(A) = lim
n → n
.This is called statistical or empirical probability. Clearly
0 ≤ P(A) ≤ 1 where
P(A) = 0 signifies that the event is impossible while P(A) = 1 signifies that
it is certain.
Note: The fact that P(A) = 0 does not imply that A = Ø.

Example: An experiment consists of throwing two ordinary dice, one die

is red and the other is clear.
Ω = {(x, y)|x = 1, 2, … , 6; y = 1, 2, …, 6 }.
a. What is the probability of throwing a double?
E1 = {(1, 1), (2, 2), …, (6, 6)}
# 𝐸1 6 1
p(E1 ) = = =
# 𝛺 36 6
b. What is the probability that the number on the clear die is at least 3
greater than the number on the red die?
E2 = {(1, 4), (1, 5), (1, 6), (2, 5), (2, 6), (3, 6)}
# 𝐸2 6 1
p(E2) = = = =
#𝛺 36 6
c. What is the probability that the sum of both faces is10?
E3 = {(4, 6), (6, 4), (5, 5)}
# 𝐸3 3 1
p(E3) = = = =
# 𝛺 36 12
d. What is the probability that r ≤ 3 or c ≤ 2?
Let A consists of 18 points of Ω for which r ≤ 3.
B consists of 12 points of Ω for which c ≤ 2.
Dr. Tasnim Hasan Lec. 14

A = {(3, 1), (3, 2), … , (3, 6),(2, 1), (2, 2), … , (2, 6), (1, 1), (1, 2), … , (1,
6)}.
B = {(1, 1), (2, 1), … , (6, 1), (1, 2), (2, 2), … , (6, 2)}.
A∩B = {(3, 1), (3, 2), (2, 1), (2, 2), (1, 1), (1, 2)}
p(A or B) = p(A  B) = p(A) + p(B) - p(A∩B)
18 12 6 24 2
= + − = =
36 36 36 36 3

e. What is the probability that the sum r + c is 7 or 10?

Let A consists of the number of points of Ω for which the sum is 7.
B consists of the number of points of Ω for which the sum is 10.
A = {(1, 6), (6, 1), (2, 5), (5, 2), 3, 4), (4, 3)},
B = {(4, 6), (6, 4), (5, 5)}
A∩B = Ø (mutually exclusive)
6 3 9 1
p(A  B) = p(A) + p(B) = + = =
36 36 36 4

Example: Let 2 items be chosen at random from a lot containing 12 items

of which 4 are defective. Let
A = {both items are defective}
B = {both items are non-defective}
C = {At least one item is defective}
E = {At least one item is non-defective
Find: p(A), p(B), p(C) and p(E).

12
Ω can occur in C 2 = 66 ways in which 2 items can be chosen from 12.The
number of ways that 2 defective items be chosen from 4 defective items
4
is C2 = 6
Dr. Tasnim Hasan Lec. 14

The number of ways that 2 non-defective items be chosen from 8 non-

8
defective items is C2 = 28.
C 24 6 1
p(A) = C 12 = 66 = 11
2

C 28 28 14
p(B) = C 12 = =
2 66 33
Since C is the complement of B,
28 38
p(C) = p(Bc) = 1 – p(B) = 1 − =
66 66
Since E is the complement of A,
1 10
p(E) = p(Ac) = 1 – p(A) = 1 − =
11 11

Example: An urn contains 10 black, 15 white, and 5 red balls. What is the
probability of drawing a black, a white or a red ball?

Let B = {the ball drawn is black}, W = {the ball drawn is white},

R = {the ball drawn is red}
10 1 15 1 5 1
p(B) = = p(W) = = p(R) = =
30 3 30 2 30 6

Example: A card is drawn from a deck.

a. What is the probability that the card drawn is a heart?
b. What is the probability that the card drawn is an ace?

Let A = {the card drawn is a heart}, B = {the card drawn is an ace}

13 1 4 1
p(A) = = p(B) = =
52 4 52 13
Dr. Tasnim Hasan Lec. 14

Example: A class contains 10 men and 20 women of which half the men
and half the women have brown eyes. Find the probability that a person
chosen at random is a man or has brown eyes.
10 1
Let A = {person is a man} p(A) = =
30 3
5 + 10 1
B = {person has brown eyes} p(B) = =
30 2
A∩B = {person is a man and has brown eyes}
5 1
p(A∩B) = =
30 6
p(A  B) = p(A) + p(B) - p(A∩B)
1 1 1 4 2
= + − = =
3 2 6 6 3

H.W.
1. A fair die is rolled twice, find the probability that:
a. The sum of both outcomes is 7.
b. The sum is not 7.
c. The sum is more than 7.
d. The sum is less than or equal to 7.
2. A card is drawn from a deck. What is the probability that:
a. The card is red.
b. The card is a diamond.
c. The card is a picture.
d. The card is a picture or diamond.
e. The card is a joker.
Dr. Tasnim Hasan Lec. 14

3. Consider two events A and B such that P(A) = 1/3, and P(B) = 1/2.
Determine P(B∩Ac) for each of the following cases:
a. A and B are disjoint.
b. A is a subset of B.
c. P(AB) = 1/8
4. For any event A, if P(B) = 1 , show that P(A) = P(AB)
Hint: Take A = AB  ABc
5. Consider the experiment in which a fair coin is tossed once and a
balanced die is rolled once.
a. Describe the sample space.
b. What is the probability that a head is appeared on the coin, and an odd
number is appeared on the die?
6. A school contains students in grades 1, 2, 3, 4, 5, and 6. Grades 2, 3, 4,
5, and 6 all contain the same number of students, but there is twice this
number in grade 1. A student is selected at random from a list of all the
students in the school.
a. What is the probability that the selected student is in grade 3?
b. What is the probability that the selected student is in an odd numbered
grade?
Dr. Tasnim Hasan Lec. 15

Conditional Probability:
Let A and B be two events in 𝒜 of the probability space (Ω, 𝒜, P), the
conditional probability of event A given that event B has occurred is
defined by:
P( A  B) P( AB )
P(A|B) = or if P(B) > 0
P( B) P( B)
Problem: For given event B for which P(B) > 0, show that P( ∙ |B) is a
probability measure function.
Solution:
P( AB )
1. P(A|B) = P( B ) ≥ 0 for every A in 𝒜.

P(B) P( B)
2. P(Ω |B) = P( B) = =1
P( B)
3. If A1, A2, … is a sequence of mutually exclusive events in 𝒜 and
⋃∞𝑖=1 𝐴𝑖 𝜖𝒜 , then
∞
𝑃((⋃∞
𝑖=1 𝐴𝑖 ) ∩ 𝐵) 𝑃(⋃∞
𝑖=1(𝐴𝑖 𝐵)) ∑∞
𝑖=1 𝑃(𝐴𝑖 𝐵)
𝑃 (⋃ 𝐴𝑖 |𝐵) = = =
𝑃(𝐵) 𝑃(𝐵) 𝑃(𝐵)
𝑖=1
∞

= ∑ 𝑃(𝐴𝑖 |𝐵)
𝑖=1
Hence, P( ∙ |B) is a probability measure function.
Example 1: Three fair coins are tossed. Find the probability that they are
all heads if:
i. The first coin is head.
Let A = {the first coin is head} = {HHH, HHT, HTH, HTT}
B = {the three coins are heads} = {HHH}, A∩B = {HHH}
1
p ( AB ) # AB 1
= = 8 =
P(B|A) = p ( A) #A 4 4
8
Dr. Tasnim Hasan Lec. 15

ii. At least one of the coins is heads.

Let A= {HTT, THT, TTH, HHH, HHT, HTH, THH}
B = {HHH}, A∩B = {HHH}
1
p ( AB ) 1
P(B|A) = = 8 =
p ( A) 7 7
8
Example 2: A fair die is thrown, if the number appearing is less than 4.
What is the probability that:
i. The number is odd.
ii. The number is 2.
Let B = {the number is less than 4} = {1, 2, 3}
A = {the number is odd} = {1, 3, 5}, A  B = {1, 3}
2
p ( AB ) 6 = 2
P(A|B) = p ( B ) =
3 3
6
Let C = {the number is 2} = {2}, B∩C = {2}
1
p (CB ) 6 1
p(C|B) = p ( B ) = 3 = 3
6
Example 3: Two digits are selected at random from 1, …, 9.If the sum is
even, find the probability that both numbers are odd.
There are C2 = 36 ways to choose 2 numbers from {1, 2, … , 9}.
9

The sum is even if both numbers are even or both are odd.
4
There are 4 even numbers {2, 4, 6, 8}. Hence there are C2 = 6 ways to
choose two even numbers.
5
There are 5 odd numbers {1, 3, 5, 7, 9}. Hence there are C2 = 10 ways
to choose two odd numbers.
Dr. Tasnim Hasan Lec. 15

Thus, there are 6 + 10 = 16 ways to choose two numbers such that the sum
is even.
Let A = {the sum of the two digits is even} 16 elements
16
P(A) =
36
10
B = {the two digits are odd} 10 elements, P(B) =
36
A∩B = B 10 elements
10
P ( AB ) 10 5
= 36 = =
P(B|A) = P ( A) 16 16 8
36
Properties of P( ∙ |B) :
Assume that the probability space (Ω, 𝒜, P) is given, and let B ∈ 𝒜
satisfy P(B) > 0.
1. If A = B , then P(A|B)= 1

2. P(ϕ|B) = 0 and P(Bc|B) = 0

3. If A1, A2, …, An are mutually exclusive events in 𝒜, then

𝑃(𝐴1 ∪ 𝐴2 ∪ … ∪ 𝐴𝑛 |𝐵) = ∑𝑛𝑖=1 P(𝐴𝑖 |𝐵)

4. For any two events A1 and A2 ∈ 𝒜,

𝑃(𝐴1 ∪ 𝐴2 |𝐵) = 𝑃(𝐴1 |𝐵) + 𝑃(𝐴2 |𝐵 ) − 𝑃(𝐴1 𝐴2 |𝐵)

5. P(A|B) + P(Ac|B) = 1

6. If A1 and A2 ∈ 𝒜, and 𝐴1 ⊂ 𝐴2 , then

P(A1|B) ≤ P(A2|B)
Dr. Tasnim Hasan Lec. 15

H. W.
Given the probability space (Ω, 𝒜, P), where 𝒜={A, Ac, Ω, ϕ},
Construct P(∙|A), P(∙|Ac).

The Multiplication Rule for Conditional probabilities

From the definition of conditional probability, it is possible to compute
the probability of occurrence of two events
P(AB) = P(A) . P(B|A)

Or P(AB) = P(B) . P(A|B)

The principle can be extended to any finite number of events, as stated in

the following theorem.

Theorem
For a given probability space (Ω, 𝒜, P), let A1, A2, …, An be events
belonging to 𝒜 for which P(A1 A2 … An-1) >0; then
P(A1 A2 … An )= P(A1)P(A2|A1)P(A3| A1 A2 )… P(An| A1 A2 … An-1)
Proof:
The product of probabilities on the right side of the above equation is
equal to
𝑃(𝐴1 𝐴2 ) 𝑃(𝐴1 𝐴2 𝐴3 ) 𝑃(𝐴1 𝐴2 … 𝐴𝑛 )
𝑃(𝐴1 ) …
𝑃 (𝐴1 ) 𝑃(𝐴1 𝐴2 ) 𝑃(𝐴1 𝐴2 … 𝐴𝑛−1 )

Since P(A1 A2 … An-1) > 0, each of the denominators in this product must
be positive. All of the terms in the product cancel each other except the
final numerator which is the left side of the equation.

Example 1: From 5 motors, of which one is defective, 2 motors are to be

selected at random. Find the probability that the second motor is not
defective, given that the first was not defective.
Dr. Tasnim Hasan Lec. 15

Let A = {the first motor is not defective}

B = {the second motor is not defective}
C14 4 C13 3
p(A) = 5 = p(B|A) = =
C1 5 C14 4
p(AB) = p(A) . p(B|A)
4 3 3
= . =
5 4 5
Example 2: If we randomly pick two television tubes in succession from
a shipment of 240 television tubes of which 15 are defective. What is the
probability that they will both be defective?
p(AB) = p(A) . p(B|A)
15 14 7
.
= 240 239 1912 =

Example3: Find the probability of randomly drawing two aces in

succession from an ordinary deck of 52 playing cards, if we sample:
i. Without replacement.
4 3 1
p(AB) = p(A) . p(B|A) = . =
52 51 221
ii. With replacement.
4 4 1
p(AB) = p(A) . p(B|A) = . =
52 52 169
Example: A box of fuses contains 20 fuses, of which 5 are defective. If 3
of the fuses are selected at random and removed from the box in
succession without replacement, what is the probability that all the three
fuses are defective?
5 4 3
p(A) = , p(B|A) = , p(C|AB) =
20 19 18
p(ABC) = p(A) . p(B|A) . p(C|AB)
5 4 3 1
= . . =
20 19 18 114
Dr. Tasnim Hasan Lec. 15

Theorem of total probabilities

For a given probability space (Ω, 𝒜, P), if B1, B2, …, Bn is a collection
of mutually disjoint events in 𝒜 satisfying Ω = ⋃𝑛𝑗=1 𝐵𝑗 and P(Bj) > 0
for j = 1, 2, …, n, then for every A ∈ 𝒜,
𝑃(𝐴) = ∑𝑛𝑗=1 𝑃(𝐴|𝐵𝑗 )𝑃(𝐵𝑗 )
Proof:

A= Ω ∩A = (B1  B2  …  Bn)∩A

= (AB1)  (AB2)  …  (ABn), where A∩Bk are also mutually exclusive.

Accordingly,
P(A) = 𝑃(⋃𝑛𝑗=1 𝐴𝐵𝑗 ) = ∑𝑛𝑗=1 𝑃(𝐴𝐵𝑗 )
= p(B1).p(A|B1) + p(B2).p(A|B2) + … + p(Bn).p(A|Bn)
𝑛

= ∑ 𝑃(𝐴|𝐵𝑗 ) 𝑃(𝐵𝑗 )
𝑗=1

Bayes’ Theorem
For a given probability space (Ω, 𝒜, P), if B1, B2, …, Bn is a collection
of mutually disjoint events in 𝒜 satisfying Ω = ⋃𝑛𝑗=1 𝐵𝑗 and P(Bj) > 0
for j = 1, 2, …, n, then for every A ∈ 𝒜 for which P(A) > 0

p ( Bk ) p ( A / Bk )
p(Bk|A) = n

 p( B
j =1
j ) p( A / B j )

Proof:
p ( A  Bk )
Since p(A|Bk) = p ( Bk )
Dr. Tasnim Hasan Lec. 15

Or p(Bk∩A) = p(Bk) . p(A|Bk)

p ( Bk  A) p ( Bk ) p ( A / Bk )
=
p(Bk|A) = n
p ( A)
 p( B
j =1
j ) p( A / B j )

p( Bk ) p( A / Bk )
=
p( B1 ) p( A / B1 ) + p( B2 ) p( A / B2 ) + ... + p( Bn ) p( A / Bn )
Frequently, the original probabilities p(Bj)are called prior probabilities
and the conditional probabilities p(Bj|A) are called the posterior
probabilities.

Example1:
In a certain factory, machine I, II, and III are all producing springs of the
same length. Of their production, machine I, II, and III produce 2%, 1%,
and 3% defective springs, respectively. Of the total production of springs
in the factory machine I produce 35%, machine II produces 25% and
machine III produces 40%. If one spring is selected at random:
a. Find the probability that it is defective.
b. Find the probability that it was produced by machine I, II or III.
Solution:

a. p(D) = p(I) . p(D|I) + p(II) . p(D|II) + p(III) . p(D|III)

35 2 25 1 40 3 215
= . + . + . =
100 100 100 100 100 100 10000

2 35
.
p( I ) p( D / I ) 100 100 = 70
b. p(I|D) = =
p ( D) 215 215
10000
Dr. Tasnim Hasan Lec. 15

1 25
.
p( I I ) p( D / I I ) 25
p(II|D) = = 100 100 =
p( D) 215 215
10000
3 40
.
p ( III) p ( D / III) 100 100 = 120
p(III|D) = =
p( D) 215 215
10000
Example2:
Urn 1 contains 8 black and 2 white balls; urn 2 contains 3 black and 7
white balls. A ball is to be chosen at random from one of the urns.
1. What is the probability that the ball is black?
2. What is the probability that the drawn ball selected from urn 1?
3. What is the probability that the drawn ball selected from urn 2?
Solution:
Let B = {the ball drawn is black}
A1 = {the ball drawn selected from urn 1}
A2 = {the ball drawn selected from urn 2}
1. P(B) = P(A1) P(B| A1 ) + P(A2) P(B| A2)

1 8 1 3 11
= +2 = 20
2 10 10

𝑃(𝐴1) 𝑃(𝐵| 𝐴1 ) 8/20 8

2. P(A1|B) = = =
𝑃(𝐵) 11/20 11

𝑃(𝐴2)𝑃(𝐵|𝐴2 ) 3/20 3
3. P(A2|B) = = =
𝑃(𝐵) 11/20 11
 1
Dr. Tasnim Hasan Lec. 16

Independent Events:
For a given probability space (Ω, 𝒜, P), let A and B be two events
in 𝒜. Events A and B are said to be independent if and only if
p(A|B) = p(A) or p(B| A) = p(B)
This is equivalent to stating that
p(A∩B) = p(AB) = p(A). p(B)
Events that are independent are sometimes called statistically
independent or stochastically independent.

Example 1: A coin is tossed three times and the eight outcomes are
assumed to be equally likely. If A is the event that a head occurs on
each of the first two tosses, B is the event that a tail occurs on the
third toss; and C is the event that exactly two tails occur in the three
tosses; Show that:
a. Events A and B are independent. b. Events B and C are dependent.
2 1
Since A = {HHH, HHT} p(A) = =
8 4
4 1
B = {HHT, HTT, THT, TTT} p(B) = =
8 2
3
C = {HTT, THT, TTH} p(C) =
8
1
A∩B = {HHT} p(AB) =
8
2 1
B∩C = {HTT, THT} p(BC) = =
8 4
1 1 1
a- p(A).p(B) = . = =p(AB), events A and B are independent.
4 2 8
1 3 3
b- p(B).p(C) = . = ≠ p(BC), events B and C are not
2 8 16
independent.
 2
Dr. Tasnim Hasan Lec. 16

Example 2:
If p(A) = 0.4, p(C|A) = 0.5, p(A|C) = 0.4
i. Are A and C independent?
Since p(A|C) = p(A) = 0.4, events A and C are independent
ii. Find p(C).
Since A and C are independent then p(C|A) = p(C) = 0.5
iii. Find p(A  C).
Since A and C are independent
p(A∩C) = p(A).p(C) = (0.4)(0.5) = 0.2
p(A  C) = p(A) + p(C) – p(A∩C) = 0.4 + 0.5 – 0.2 = 0.7
iv. Find p(Ac∩Cc).
p(Ac∩Cc)= p(A U C)c = 1 – p(A  C) = 1 – 0.7 = 0.3
v. Find p(AcUCc).
p(AcUCc) = p(A∩C)c = 1 – p(A∩C) = 1 – 0.2 = 0.8

Definition:
Let (Ω, 𝒜, P) be a probability measure space, Ȼ is a class of
stochastic independent events means:
Ȼ ⊂ 𝒜 and the probability of the intersection of any finite number of
different events in Ȼ is equal to the product of their probabilities.

Definition: The class of events Ȼ= {A, B, C} is said to be mutually

independent if and only if:
i. They are pairwise independent, that is:
 3
Dr. Tasnim Hasan Lec. 16

P(AB) = P(A). P(B)

P(AC) = P(A). P(C)
P(BC) = P(B). P(C)
ii. P(ABC) = P(A). P(B). P(C)
Example:
An urn contains four balls numbered 1, 2, 3, and 4. One ball is drawn
at random. Let A= {1, 2}, B= {1, 3}, C= {1, 4}.
Then A∩B = A∩C = B∩C = {1}
P(A) = P(B) = P(C) = ½
P(AB) = ¼ = P(A). P(B)
P(AC) = ¼ = P(A). P(C)
P(BC) = ¼ =P(B). P(C)
Since A∩B∩C = {1}
P(ABC) = ¼ ≠ P(A).P(B).P(C)=1/8
Hence, pairwise independence does not imply mutual independence.

Theorem
Let Ȼ = {A, B}, be a class of stochastically independent events, then
the classes Ȼ1 = {Ac, B}, Ȼ2 = {A, Bc} and Ȼ3 = {Ac, Bc} are also
independent.
Proof:
i. Ȼ1 = {Ac, B}
B = AB  AcB
Since AB∩Ac B = ϕ
P(B) = P(AB) + P(AcB)
P(Ac B) = P(B)- P(AB) = P(B)- P(A).P(B) A, B indep.
 4
Dr. Tasnim Hasan Lec. 16

= P(B)[1 – P(A)] = P(B). P(Ac)

Hence Ȼ1 is a class of S.I. events.
ii. Ȼ2 = {A, Bc}
A = AB  ABc
Since AB∩ABc = ϕ
P(A) = P(AB) + P(ABc)
P(ABc) = P(A)- P(AB) = P(A)- P(A).P(B) A, B indep.
= P(A)[1 – P(B)] = P(A). P(Bc)
Hence Ȼ2 is a class of S.I. events.
iii. Ȼ3 = {Ac, Bc}
AcBc = (A  B)c
P(AcBc) = P(A  B)c =1- P(A  B)
= 1- [P(A)+ P(B)- P(AB)]
= 1- P(A)- P(B)+P(A).P(B) A, B indep.
= P(Ac)-P(B)[1-P(A)]= P(Ac)- P(B)P(Ac)
=P(Ac)[1-P(B)] = P(Ac).P(Bc)
Hence Ȼ3 is a class of S.I. events.

Theorem
Let Ȼ = {A, B, C}, be a class of stochastically independent events,
then the classes Ȼ1 = {Ac, B, C}, Ȼ2 = {A, Bc, C}, Ȼ3 = {A, B, Cc}, Ȼ4 =
{Ac, Bc, C}, Ȼ5 = {Ac, B, Cc},Ȼ6 = {A, Bc, Cc}, and Ȼ7 = {Ac, Bc, Cc}
are also independent.

Proof:
 5
Dr. Tasnim Hasan Lec. 16

i. Ȼ3 = {A, B, Cc}
AB = ABC  ABCc
P(AB) = P(ABC) + P(ABCc)
P(ABCc) = P(AB) – P(ABC)
= P(A). P(B)- P(A).P(B).P(C)
=P(A)P(B)[1- P(C)] = P(A)P(B)P(Cc)
Hence Ȼ3 is a class of S.I. events.
ii. Ȼ4 = {Ac, Bc, C}
C = (A  B)∩C ( A  B)c∩C
P(C) =P(AC)  P(BC) + P(AcBcC)
= P(AC) + P(BC)- P(ABC)+ P(AcBcC)
P(AcBcC) = P(C)-P(AC)-P(BC) + P(ABC)
=P(C)-P(A) P(C)-P(B)P(C)+ P(A) P(B) P(C)
=P(C)[1-P(A)]-P(B)P(C)[1-P(A)]
=P(C)P(Ac)[1- P(B)]
= P(Ac)P(Bc)P(C)
Hence Ȼ4 is a class of S.I. events.
iii. Ȼ7 = {Ac, Bc, Cc}
AcBcC c= (A  B  C)c
P(AcBcC c) = P(A  B  C)c
= 1- P(A  B  C)
= 1- P(A)- P(B)- P(C)+P(AB)+P(AC)+P(BC)-P(ABC)
=P(Ac)- P(B) – P(C)+P(A)P(B)+P(A)P(C)+P(B)P(C)-P(A)P(B)P(C)
= P(Ac)- P(B)[1-P(A)]- P(C)[1-P(A)]+P(B)P(C)[1- P(A)]
 6
Dr. Tasnim Hasan Lec. 16

= P(Ac)[1-P(B)-P(C)+P(B)P(C)]
=P(Ac)[1-P(B)-P(C)[1-P(B)]]
=P(Ac)[1-P(B)][1-P(C)]
= P(Ac)P(Bc)P(Cc)
Hence Ȼ7 is a class of S.I. events.
H. W:
1. Let A and B be events with: p(A) = 1/2, p(B) = 1/3, p(A∩B) = 1/4.
Find: p(A|B), p(B|A), p(A  B), p(Ac|Bc), p(Bc|Ac).

2. Find p(B|A) if:

i. A is a subset of B.
ii. A and B are mutually exclusive.

3. Let P(A) = 0.5, P(A  B)= 0.7, find P(B) if

- A and B are independent.
- A and B are disjoint.
- P(A|B) = 0.4

4. Two cards are drawn at random from a deck with replacement. Let
A be the event that the first card is an ace and B be the event that the
second card is a spade. Are A and B independent?

5. Box A contains 8 items of which 3 are defective, and box B

contains 5 items of which 2 are defective. An item is drawn at random
from each box:
i. What is the probability that both items are not defective?
ii. What is the probability that one item is defective and one is not?
iii. What is the probability that the defective item in (ii) came from
box A?
 7
Dr. Tasnim Hasan Lec. 16

6. In a certain college for which 60% of the students are women, 4%

of the men and 1% of the women are taller than 6 feet. If a student is
selected at random:
a. Find the probability that he (or she) is taller than 6 feet.
b. Find the probability that the student is a woman.

7. Let Ȼ = {A, B}, be a class of stochastically independent events,

Show that the classes Ȼ1 = {A, B, ϕ}, Ȼ2 = {A, B, Ω} are independent.