Lecture 3: ELEMENTS OF PROBABILITY

CCNY EE311
Instructor: Andrii Golovin, Ph.D. in Physics and Mathematics
[email protected]

1
Sample Space 𝑺 is the set of all possible outcomes of an experiment.
• Example: the outcome of an experiment consists in the determination of
the sex of a newborn child, then 𝑺 = {𝒈, 𝒃} , where the outcome 𝒈
means that the child is a girl and 𝒃 that it is a boy.

Event 𝑬 is a subset of the sample space 𝑺. If the outcome of an experiment

is contained in 𝑬, then we say that 𝑬 has happened.
• Example: if 𝑬 = {𝒈}, then 𝑬 is the event that the child is a girl. Similarly,
if 𝑭 = {𝒃}, then 𝑭 is the event that the child is a boy.

Intersection 𝑬𝑭 is an event that consists all outcomes that are in both 𝑬 and
𝑭, that means that the event 𝑬𝑭 will occur only if both 𝑬 and 𝑭 occur. If
𝑬𝑭 = ∅, implying that 𝑬 and 𝑭 cannot both occur, then 𝑬 and 𝑭 are said to
be mutually exclusive.
• Example: if 𝑬 = {𝒉} and 𝑭 = {𝒕}, where 𝒉 and 𝒕 are coin head and tail,
then 𝑬𝑭 = ∅.
2
Complement event 𝑬𝒄 consists of all outcomes in the sample space 𝑺 that are not in
𝑬. That is, 𝑬𝒄 will occur if 𝑬 does not occur.
• Example: if 𝑬 = {𝒃} is the event that the child is a boy, then 𝑬𝒄 = {𝒈} is the
event that it is a girl. Also, since the experiment must result in some outcome, it
follows that 𝑺𝒄 = ∅.

𝑬 ⊂ 𝑭 all of the outcomes in 𝑬 are also contained in 𝑭.

o the occurrence of 𝑬 necessarily implies the occurrence of 𝑭;
o if 𝑬 ⊂ 𝑭 and 𝑭 ⊂ 𝑬, then we say that 𝑬 and 𝑭 are equal (or identical) and we
write 𝑬 = 𝑭 .

Union 𝑬 ∪ 𝑭 is an event consisted all outcomes that are either in event 𝑬 or inevent
𝑭 or in both.
• Example: if 𝑬 = {𝒈 } and 𝑭 = {𝒃}, then 𝑬 ∪ 𝑭 = {𝒈, 𝒃} than 𝑬 ∪ 𝑭 is the whole
sample space 𝑺.
Union of events 𝑬𝟏, 𝑬𝟐, . . . , 𝑬𝒏, denoted either by 𝑬𝟏 ∪ 𝑬𝟐 ∪ · · · ∪ 𝑬𝒏 or ∪𝒏𝟏 𝑬𝒊 , is
defined as the event consisting of all outcomes that are in 𝑬𝒊 for at least one
𝒊 = 𝟏, 𝟐, . . . , 𝒏.
o in contrast to the intersection, where intersection occurs when all of the events
𝑬𝒊 occur, the union of the 𝑬𝒊 occurs when at least one of the events 𝑬𝒊 occurs.
3
 Venn diagram
• the sample space S is represented as consisting of all the points in a large rectangle;
• the events 𝑬, 𝑭, 𝑮, . . . , are represented as consisting of all the points in given circles within
the rectangle;
• events of interest can then be indicated by shading appropriate regions of the diagram.
Union Intersection
Shaded region: 𝑬 ∪ 𝑭
𝑺

𝑬 𝑭

Complement 𝒄 Contained
Shaded region: 𝑬 𝑬⊂𝑭
𝑺 𝑺
𝑭
𝑬 𝑬𝒄 𝑬

4
 • The operations of forming unions, intersections, and complements of events
obey certain rules not dissimilar to the rules of algebra:
Commutative law: 𝑬∪𝑭 = 𝑭∪𝑬 𝑬𝑭 = 𝑭𝑬

Associative law: (𝑬 ∪ 𝑭) ∪ 𝑮 = 𝑬 ∪ (𝑭 ∪ 𝑮) (𝑬𝑭)𝑮 = 𝑬(𝑭𝑮)

Distributive law: (𝑬 ∪ 𝑭)𝑮 = 𝑬𝑮 ∪ 𝑭𝑮 𝑬𝑭 ∪ 𝑮 = (𝑬 ∪ 𝑮)(𝑭 ∪ 𝑮)

• DeMorgan’s laws:
𝑬∪𝑭 𝒄 = 𝑬 𝒄 𝑭𝒄
𝒄
𝑬𝑭 = 𝑬 𝒄 ∪ 𝑭𝒄

5
Proof for the distributive law: (𝑬 ∪ 𝑭 )𝑮 = 𝑬𝑮 ∪ 𝑭𝑮

Shaded region: 𝑬 ∪ 𝑭 Shaded region: (𝑬 ∪ 𝑭 )𝑮

E F

Shaded region:
Shaded region: 𝑬𝑮 Shaded region: 𝑭𝑮 𝑬𝑮 ∪ 𝑭𝑮

6
Proof for the Associative law: (𝑬𝑭)𝑮 = 𝑬(𝑭𝑮)

Shaded region: 𝑬𝑭 Shaded region: (𝑬𝑭)𝑮

Shaded region: 𝑭𝑮 Shaded region: 𝑬(𝑭𝑮)

7
Proof for the DeMorgan’s laws: 𝑬∪𝑭 𝒄 = 𝑬𝒄 𝑭𝒄
𝒄
Shaded region: 𝑬 ∪ 𝑭 Shaded region: 𝑬 ∪ 𝑭

Shaded region: 𝑬𝒄 Shaded region: 𝑭𝒄 Shaded region:𝑬𝒄 𝑭𝒄

8
Proof for the DeMorgan’s laws: 𝑬𝑭 𝒄 = 𝑬 𝒄 ∪ 𝑭𝒄
𝒄
Shaded region: 𝑬𝑭 Shaded region: 𝑬𝑭

Shaded region: 𝑬𝒄 Shaded region: 𝑭𝒄 Shaded region:𝑬𝒄 ⋃𝑭𝒄

9
 AXIOMS OF PROBABILITY
Probability is a number between 𝟎 and 𝟏, where 0 indicates impossibility and 𝟏
indicates certainty. The higher the probability of an event, the more likely it is that
the event will occur.
• Example: the tossing of a coin: The two outcomes ("heads" and "tails") are both
equally probable, then the probability of "heads" equals the probability of
"tails“. Since no other outcomes are possible, the probability of either "heads"
or "tails" is 0.5 (or 50%).
If an experiment is continually repeated under the exact same conditions, then for
any event 𝑬 of sample space 𝑺, the proportion of time that the outcome is contained
in 𝑬, approaches some constant. As the number of repetitions increases, this limiting
constant of relative frequency is described as the probability 𝑷 𝑬 of the event 𝑬.

Axiom I 𝟎 ≤ 𝑷(𝑬) ≤ 𝟏

Axiom II 𝑷(𝑺) = 𝟏
𝒏 𝒏
Axiom III
𝑷 ራ 𝑬𝒊 = ෍ 𝑷 𝑬𝒊 ,
𝒊=𝟏 𝒊=𝟏
where 𝒏 = 𝟏, 𝟐, . . . , ∞ and 𝑬𝒊𝑬𝒋 = 𝟎 𝐢𝐟 𝒊 ≠ 𝒋
(events 𝑬𝒊 and 𝑬𝒋 are mutually exclusive).
10
 Two Propositions of Probabilities
Events 𝑬 and 𝑬𝒄 are always mutually exclusive, and since 𝑬 ∪ 𝑬𝒄 = 𝑺, by using
Axioms II and III one can get PROPOSITION 3.4.1:

𝑷 𝑺 = 𝟏 = 𝑷 𝑬 ∪ 𝑬𝒄 = 𝑷 𝑬 + 𝑷 𝑬𝒄 ⟹ 𝑷 𝑬𝒄 = 𝟏 − 𝑷 𝑬 .
Proposition 3.4.1 states that the probability that an event does not occur is 1 minus
the probability that it does occur.

PROPOSITION 3.4.2 gives the relationship between the probability of the union
𝑷 𝑬 ∪ 𝑭 of events 𝑬 and 𝑭 in terms of the individual probabilities 𝑷(𝑬) and 𝑷(𝑭)
and the probability of the intersection 𝑷(𝑬𝑭):

𝑷(𝑬 ∪ 𝑭) = 𝑷(𝑬) + 𝑷(𝑭) − 𝑷(𝑬𝑭)

Union Intersection
Shaded region: 𝑬 ∪ 𝑭
𝑺

𝑬 𝑭

11
 Proof of PROPOSITION 3.4.2
PROPOSITION 3.4.2 gives the relationship between the probability of the union
𝑷 𝑬 ∪ 𝑭 of events 𝑬 and 𝑭 in terms of the individual probabilities 𝑷(𝑬) and
𝑷(𝑭) and the probability of the intersection 𝑷(𝑬𝑭) ≠ 𝟎:

𝑷(𝑬 ∪ 𝑭) = 𝑷(𝑬) + 𝑷(𝑭) − 𝑷(𝑬𝑭)

𝑬 𝑭
𝐈 𝐈𝐈 𝐈𝐈𝐈

Proof: by using Axiom III, one can show that

𝑷 𝑬 ∪ 𝑭 = 𝑷 𝑰 ∪ 𝑰𝑰 ∪ 𝑰𝑰𝑰 = 𝑷 𝑰 + 𝑷 𝑰𝑰 + 𝑷 𝑰𝑰𝑰 =

where 𝑷 𝑬 = 𝑷 𝑰 ∪ 𝑰𝑰 = 𝑷(𝑰) + 𝑷(𝑰𝑰) and 𝑷 𝑭 = 𝑷 𝑰𝑰 ∪ 𝑰𝑰𝑰 = 𝑷 𝑰𝑰 + 𝑷 𝑰𝑰𝑰 .

Thus, 𝑷 𝑬 + 𝑷 𝑭 = 𝑷 𝑰 + 𝑷 𝑰𝑰 + 𝑷 𝑰𝑰 + 𝑷 𝑰𝑰𝑰 ⟹ 𝑷 𝑰 + 𝑷 𝑰𝑰 + 𝑷 𝑰𝑰𝑰 = 𝑷 𝑬 + 𝑷 𝑭 − 𝑷 𝑰𝑰

where
= 𝑷 𝑬 + 𝑷 𝑭 − 𝑷 𝑰𝑰 = = 𝑷(𝑬) + 𝑷(𝑭) − 𝑷(𝑬𝑭)
𝑰𝑰 = 𝑬𝑭
12
EXAMPLE 3.4a . A total of 28 percent of American males smoke cigarettes, 7 percent
smoke cigars, and 5 percent smoke both cigars and cigarettes. What percentage of
males smoke neither cigars nor cigarettes?

SOLUTION: Let’s assume that 𝑬 is the event that a randomly chosen male is a
cigarette smoker and event 𝑭 is the event that he is a cigar smoker. Then, the
probability this person is a smoker is

𝑷 𝑬 ∪ 𝑭 = 𝑷 𝑬 + 𝑷 𝑭 − 𝑷 𝑬𝑭 = 𝟎. 𝟐𝟖 + 𝟎. 𝟎𝟕 − 𝟎. 𝟎𝟓 = 𝟎. 𝟑

By using Axiom II, one can show that the probability that the person is not a smoker
is
𝟏 − 𝑷 𝑬 ∪ 𝑭 = 𝟏 − 𝟎. 𝟑 = 𝟎. 𝟕 ,
implying that 70 percent of American males smoke neither cigarettes nor cigars.

13
Odds of the event 𝑬 is the ratio of probability 𝑷 𝑬 that the event 𝑬 occurs over of
probability 𝟏 − 𝑷 𝑬 that the event 𝑬 does not occur:

𝑷 𝑬 𝑷 𝑬
= .
𝟏−𝑷 𝑬 𝑷 𝑬𝒄

Thus, the odds of an event 𝑬 tells how much more likely it occurs than that it does
not occur.

𝟑 𝑷 𝑬 𝟑 𝟏
Example: if 𝑷(𝑬) = , then = : = 𝟑,
𝟒 𝟏−𝑷 𝑬 𝟒 𝟒
Thus, the odds of event 𝑬 are 3 meaning that it is 3 times as likely that event 𝑬
occurs as it is that it does not.

14
 Sample Spaces Having Equally Likely Outcomes
• For many experiments, it is natural to assume that each outcome in the data set is
equally likely to occur.
• For many experiments whose sample space 𝑺 is a finite set {𝑺𝟏 , 𝑺𝟐 , … , 𝑺𝑵 }, it is
often natural to assume that

𝑷 𝑺𝟏 = 𝑷 𝑺𝟐 = · · · = 𝑷 𝑺𝑵 = 𝒑 ⟺ 𝑷 𝑺𝒊 = 𝒑
• By using Axiom II and III, one can show that
𝑷 𝑺 =𝟏
𝑷 𝑺 𝟏
⟹ 𝒑 = 𝑷 𝑺𝒊 = =
𝑷 𝑺 = 𝑷({𝑺𝟏 }) + · · · +𝑷({𝑺𝑵 }) = 𝑵𝒑 𝑵 𝑵

• Probability of the event 𝑬 = ⋃𝒏𝟏 𝑺𝒊 is governed by

𝟏 𝒏
𝑷 𝑬 = 𝑷 ⋃𝒏𝟏 𝑺𝒊 = 𝑷 𝑺𝟏 + … + 𝑷 𝑺𝒏 = 𝒑 + ⋯ + 𝒑 = 𝑵 ∙ 𝒏 = 𝑵 .

Conclusion: if each outcome 𝑺𝒊 of 𝑵 measurments is equally likely to occur, then the

𝒏
probability of event 𝑬 contained by 𝒏 data points is equal the proportion .
𝑵

15
Example: someone is tossing a coin (𝒏 = 𝟏 and 𝑵 = 𝟐), then 𝑷 𝑬𝒉𝒆𝒂𝒅 =
𝟏 𝟏
𝑷 𝑬𝒕𝒂𝒊𝒍 = and 𝒑 = .
𝟐 𝟐

Example: Find 𝑷 𝑺𝒊 = 𝒑, where 𝑺𝒊 is the outcome when we roll one fair

die.
Solution: 𝒏 = 𝟏, 𝑵 = 𝟔 ⟹
𝒏 𝟏
𝑷 𝟏 =𝑷 𝟐 =𝑷 𝟑 =𝑷 𝟒 =𝑷 𝟓 =𝑷 𝟔 =𝒑= = .
𝑵 𝟔

16
Factorial of a positive integer 𝒏, denoted by 𝒏!, is the product of all positive integers
less than or equal to 𝒏:
𝒏! = 𝒏 ∙ (𝒏 − 𝟏) ∙ (𝒏 − 𝟐) · ⋯ · 𝟑 · 𝟐 · 𝟏,
also, 𝟎! = 𝟏.

Permutation is an arrangement of elements into a sequence. The number of

permutation counts all possible rearrangements for the sequence of 𝒏 distinct
objects, which can be calculated as 𝒏! .

17
EXAMPLE 3.5b . Mr. Jones has 10 books on 4 subjects that he is going to put on his
bookshelf. Of these, 4 are mathematics books, 3 are chemistry books, 2 are history
books, and 1 is a language book. Jones wants to arrange his books so that all the
books dealing with the same subject are together on the shelf.
How many different arrangements are possible?

Solution: There are 4! arrangements for the mathematics books, 3! for the chemistry
books, 2! for the history books, and then 1! for the language book. Also, there are 4!
possible orderings by the subjects. Thus total number of different arrangement is
the desired answer is

Answer: 𝟒! 𝟑! 𝟐! 𝟏! 𝟒! = 𝟔, 𝟗𝟏𝟐.

18
 r-permutation
Let’s determine the number of different groups of 𝒓 objects that could be formed
from a set of 𝒏 objects. For example, how many different groups of three (𝒓 = 𝟑)
could be selected from the five (𝒏 = 𝟓) items A, B, C, D, E?

Solution: There are 5 ways to select 𝟏𝒔𝒕 item, 𝟒 ways to then select 𝟐𝒏𝒅 item, and 3
ways to then select 𝟑𝒓𝒅 item, thus there are
𝒏!
𝟓 · 𝟒 · 𝟑=𝒏∙ 𝒏−𝟏 ∙ ⋯ ∙ 𝒏−𝒓+𝟏 = = 𝑷𝒏𝒓 ways of selecting the
𝒏−𝒓 !
group of 3, when the order in which the items are selected is relevant. Note that
𝑷𝒏𝒓 is called as r-permutation of n objects.

𝒏! 𝒏=𝟓 𝟓! 𝟐! ∙ 𝟑 ∙ 𝟒 ∙ 𝟓
𝑷𝒏𝒓 = = = = = 𝟔𝟎.
𝒏−𝒓 ! 𝟐! 𝟐!
𝒓=𝟑

Check-up:
𝒏! 𝒏 ∙ 𝒏 − 𝟏 ∙ ⋯ ∙ 𝒏 − 𝒓 + 𝟏 ∙ (𝒏 − 𝒓) ∙ (𝒏 − 𝒓 − 𝟏) ∙ ⋯ ∙ 𝟏
= =
𝒏−𝒓 ! 𝒏−𝒓 !
𝒏∙ 𝒏−𝟏 ∙ ⋯ ∙ 𝒏−𝒓+𝟏 ∙ 𝒏−𝒓 !
= =𝒏∙ 𝒏−𝟏 ∙ ⋯ ∙ 𝒏−𝒓+𝟏
𝒏−𝒓 !
19
𝒓-permutation of 𝒏 objects is number of arrangements of a fixed length r of
elements taken from a given set of 𝒏 distinct objects:
𝒏!
𝑷𝒏𝒓 = 𝒏 𝒏 − 𝟏 𝒏 − 𝟐 ∙∙∙ 𝒏 − 𝒓 + 𝟏 =
𝒏−𝒓 !
It is used to designate those ordered arrangements in which no element occurs more
than once, but without the requirement of using all the elements from a given set.

Example: 𝑨, 𝑩, 𝑪 , 𝒏 = 𝟑 and 𝒓 = 𝟐:
𝟑!
⟹ 𝑨, 𝑩 , 𝑩, 𝑪 , 𝑨, 𝑪 , 𝑩, 𝑨 , 𝑪, 𝑩 , 𝑪, 𝑨 ⟹ 𝑷𝟑𝟐 = = 𝟔.
𝟑−𝟐 !
• Thus, 𝒏-permutation of 𝒏 objects is the permutation of n distinct objects:
𝒏! 𝒏!
𝑷𝒏𝒏 = = = 𝒏!
𝒏−𝒏 ! 𝟎!

20
 Combination
Let’s determine the number of different groups of 𝒓 objects that could be formed
from a set of 𝒏 objects, if the order of selection does not matter. For example, how
many different groups of three (𝒓 = 𝟑) could be selected from the five (𝒏 = 𝟓) items
A, B, C, D, E?

Solution: There are 5 ways to select 𝟏𝒔𝒕 item, 𝟒 ways to then select 𝟐𝒏𝒅 item, and 3
ways to then select 𝟑𝒓𝒅 item, thus there are
𝒏!
𝟓 · 𝟒 · 𝟑=𝒏∙ 𝒏−𝟏 ∙ ⋯ ∙ 𝒏−𝒓+𝟏 = = 𝑷𝒏𝒓 ways of selecting the
𝒏−𝒓 !
group of 3, when the order in which the items are selected is relevant. But, if the
order of selection does not matter
({𝑨𝑩𝑪} = {𝑨𝑪𝑩} = {𝑩𝑨𝑪} = {𝑩𝑪𝑨} = {𝑪𝑨𝑩} = {𝑪𝑩𝑨}),

𝟔 = 𝒓! irrelevant possibilities
then the total number of different groups that can be formed is a combination:

𝒏 𝒏∙ 𝒏−𝟏 ∙ ⋯ ∙ 𝒏−𝒓+𝟏 𝒏! 5!
= = = = 𝟏𝟎.
𝒓 𝒓! 𝒏 − 𝒓 ! 𝒓! 2! ·3!

21
 Generalized Basic Principle of Counting
𝒏!
• in general, as 𝒏 𝒏 − 𝟏 · · · 𝒏 − 𝒓 + 𝟏 = = 𝑷𝒏𝒓 represents the number
𝒏−𝒓 !
of different ways that a group of 𝒓 items could be selected from n items, where
the order of selection is considered relevant;
• if the order of selection within groups does not matter, then each group will be
counted 𝒓! times; thus, the number of different groups of r items that could be
formed from a set of 𝒏 items is
𝒏 𝒏!
= .
𝒓 𝒏 − 𝒓 ! 𝒓!

𝒏
Combination is number of groups with 𝒓 elements taken from the set of 𝒏
𝒓
objects, where 𝒓 ≤ 𝒏 and the order of elements within groups does not matter.

𝒏 𝒏
• since 𝟎! = 𝟏, note that = = 𝟏.
𝟎 𝒏

22
 𝒏!
• 𝒓-permutation of 𝒏 objects 𝑷𝒏𝒓 = ;
𝒏−𝒓 !
Example: 𝑨, 𝑩, 𝑪 , 𝒏 = 𝟑 and 𝒓 = 𝟐:
𝟑!
⟹ 𝑨, 𝑩 , 𝑩, 𝑪 , 𝑨, 𝑪 , 𝑩, 𝑨 , 𝑪, 𝑩 , 𝑪, 𝑨 ⟹ 𝑷𝟑𝟐 = = 𝟔.
𝟑−𝟐 !

𝒏 𝒏!
• Combination = ;
𝒓 𝒏−𝒓 !𝒓!
Example: 𝑨, 𝑩, 𝑪 , 𝒏 = 𝟑 and r= 𝟐:
𝟑 𝟑!
⟹ 𝑨, 𝑩 = 𝑩, 𝑨 , 𝑩, 𝑪 = 𝑪, 𝑩 , 𝑨, 𝑪 = 𝑪, 𝑨 ⟹ = = 𝟑.
𝟐 𝟑−𝟐 !𝟐!

• By comparing the formulas for r-permutation and r-combination of 𝒏

objects, one can get that
𝒏 𝑷𝒏
= 𝒓!𝒓 ;
𝒓

23
Example 3.5d . A committee of size 𝟓 is to be selected from a group of 𝟔 men and 9
women. If the selection is made randomly, what is the probability that the
committee consists of 3 men and 2 women?

Solution:
𝟔 𝟗
There are =20 possible combinations for group of 3 from 6 men and =36 for 2
𝟑 𝟐
from 9 women.
𝟔+𝟗 𝟏𝟓
Total number of the possible random combinations is = =3003
𝟓 𝟓

𝟔 𝟗
∙ 𝟐𝟎∙𝟑𝟔 𝟐𝟒𝟎
𝟑 𝟐
Answer: The wanted probability is 𝑷 𝑬 = 𝟏𝟓 = = = 0.23976 ≈ 𝟎. 𝟐𝟒 .
𝟑𝟎𝟎𝟑 𝟏𝟎𝟎𝟏
𝟓

24
Example 3.5e. From a set of 𝒏 items a random sample of size 𝒓 is to be selected.
What is the probability that a given item will be among the 𝒓 selected?
𝒏−𝟏
Solution: The number of different selections that contain the given item is
𝒏 𝒓−𝟏
and total number of the possible random combinations is . Thus, the probability
𝒓
that a particular item is selected to the group 𝒓 is
𝒏−𝟏 𝒏−𝟏 !
𝒓 − 𝟏 𝒏−𝟏−𝒓+𝟏 ! 𝒓−𝟏 ! 𝒏−𝟏 ! 𝒏 − 𝒓 ! 𝒓!
𝑷 𝑬 = 𝒏 = = ∙ =
𝒏! 𝒏−𝒓 ! 𝒓−𝟏 ! 𝒏!
𝒓 𝒏 − 𝒓 ! 𝒓!
𝒏 − 𝟏 ! 𝒓! 𝒏−𝟏 ! 𝒓! 𝒏−𝟏 ! 𝒓 𝒓−𝟏 ! 𝟏 𝒓 𝒓
= ∙ = ∙ = ∙ = ∙ = .
𝒓 − 𝟏 ! 𝒏! 𝒏! 𝒓−𝟏 ! 𝒏 𝒏−𝟏 ! 𝒓−𝟏 ! 𝒏 𝟏 𝒏

Example:
Find 𝑷 𝑺𝒊 = 𝒑, where 𝑺𝒊 is the outcome when one rolls one fair die (𝒏 = 𝟔, 𝒓 = 𝟏):
𝒏−𝟏
𝒓 − 𝟏 𝒓 𝟏
𝑷 𝟏 = 𝑷 𝟐 = 𝑷 𝟑 = 𝑷 𝟒 = 𝑷 𝟓 = 𝑷 𝟔 = 𝑷 𝑺𝒊 = 𝒑 = 𝒏 = = .
𝒏 𝟔
𝒓 25
 Conditional Probability
Let’s consider a conditional probability by using a pair of dice that someone rolls.
𝑺 = {(𝒊, 𝒋)}, where 𝒊 = 𝟏, 𝟐, 𝟑, … , 𝟔 and 𝒋 = 𝟏, 𝟐, 𝟑, … , 𝟔 .
we say that each outcome is 𝒊, 𝒋 , where the first die lands on a side 𝒊 and the second on
a side 𝒋. The sample space 𝑺 of this experiment contains 36 outcomes. The equal
𝟏
outcome probability is 𝒑 = 𝑷 𝒊𝒋 = , where 𝑷 𝒊𝒋 is the probability of the
𝟑𝟔
intersection.
Let’s introduce a condition to have a success: if 𝒊 = 𝟑, then 𝒋 should be 𝟓 to gain the total
number of points equal to 𝟖. Thus, the outcome (𝟑, 𝟓) is just one from six possible
successful outcomes: (𝟑, 𝟏), (𝟑, 𝟐), (𝟑, 𝟑), (𝟑, 𝟒), (𝟑, 𝟓), (𝟑, 𝟔), then the conditional
𝟏
probability to get the desirable 𝒋 = 𝟓 is 𝑷 𝟑, 𝒋 = 𝒑 = 𝟔 .
In general, the conditional probability is the probability of event 𝑬 to occur, if event
𝑭 is true or chosen:
𝑬=𝒋=𝟓 𝟏
𝑷 𝑬𝑭 𝟏
𝑷 𝑬𝑭 = = = 𝑷 𝟓 𝟑 = 𝟑𝟔 𝟏 = , 𝑬𝒒. (𝟑. 𝟔. 𝟏)
𝑷 𝑭 𝟔
𝑭=𝒊=𝟑 𝟔

• 𝑷(𝑬|𝑭) is defined only when 𝑷(𝑭) > 𝟎.

• usually 𝑷 𝑬 𝑭 ≠ 𝑷 𝑭 𝑬 ;
• if 𝑷(𝑬 |𝑭) = 𝑷(𝑬), then events 𝑬 and 𝑭 are "independent“. For example the pair of
𝒋=𝟓
𝟏
fair dice are independent from each other: 𝑷 𝒋 𝒊 = 𝑷 𝒋 = = 𝟔.
𝒊=𝟑
 Conditional Probability
Example 3.6a. A bin contains 5 defective (that immediately fail), 10 partially defective
(fail after a short time), and 25 reliable transistors. A transistor is chosen at random from
the bin and put into use. If it does not immediately fail, what is the probability that it is
reliable one?
Solution: Total number of the transistors is 5+10+25=40 .
Since the chosen transistor did not immediately fail,
S F
it is not one of 5 defectives.
10+25
The number of those that do not immediately fail is 10+25.
The number of the reliable ones is 25. E
25
5
Thus, the desired probability is

𝟐𝟓
𝑷 𝑬𝑭 𝑷 𝑬 𝟐𝟓 𝟓
𝑷(𝑬reliable │𝑭does not immediately fail ) = = = 𝟒𝟎 = = .
𝑷 𝑭 𝑷 𝑭 𝟑𝟓 𝟑𝟓 𝟕
𝟒𝟎
where 𝑭does not immediately fail = 𝑬reliable ∪ 𝑮partially defective .

Check-up: Since the chosen transistor is not immediately defective, the problem reduces to
computing the probability that transistor is reliable and𝟐𝟓
it is taken
𝟐𝟓
from
𝟓
group of
(reliable)+(partially defective) transistors, thus 𝑷 = 𝟐𝟓+𝟏𝟎 = 𝟑𝟓 = 𝟕 . 27
 Conditional Probability
The conditional probability for the long-run independent experiments in which
outcome 𝑭 occurs 𝒏 times and it follows by 𝒏 times outcome 𝑭 is described similarly:
𝒏𝑷 𝑬𝑭 𝑷 𝑬𝑭
𝑷𝒏 𝑬 𝑭 = = .
𝒏𝑷 𝑭 𝑷 𝑭

28
 Independent Events
• For independent events 𝑬 and 𝑭
𝑷 𝑬 𝑭 =𝑷 𝑬
𝑷 𝑬𝑭
𝑷 𝑬𝑭 ⟹𝑷 𝑬 = ⟹ 𝑷 𝑬𝑭 = 𝑷 𝑬 ∙ 𝑷 𝑭
𝑷 𝑭
𝑷 𝑬𝑭 =
𝑷 𝑭

• Definition: Two events 𝑬 and 𝑭 are said to be independent if equation

𝑷 𝑬𝑭 = 𝑷 𝑬 ∙ 𝑷 𝑭
holds. Two events 𝑬 and 𝑭 that are not independent are said to be dependent.

Example 3.8a . A card is selected at random from an ordinary deck of 52 playing

cards. If 𝑬 is the event that the selected card is an ace and 𝑭 is the event that it
is a heart, which means that 𝑬 and F are independent, then
𝟒 𝟏𝟑 𝟓𝟐 𝟏
𝑷 𝑬𝑭 = 𝑷 𝑬 ∙ 𝑷 𝑭 = ∙ = 𝟐= .
𝟓𝟐 𝟓𝟐 𝟓𝟐 𝟓𝟐

29
 Proposition 3.8.1
Proposition 3.8.1a: If 𝑬 and 𝑭 are independent events, then 𝑬 and 𝑭𝒄 also
independent.
Proof: By using the Distributive law [(𝑬 ∪ 𝑭)𝑮 = 𝑬𝑮 ∪ 𝑭𝑮] applied to independent
events 𝑬 and 𝑭, one can derive that
S

E F
𝒄 𝒄
𝑬 = 𝑬𝑺 = 𝑺 = 𝑭 ∪ 𝑭𝒄 = 𝑬 𝑭∪𝑭 = 𝑬𝑭 ∪ 𝑬𝑭 .

Axiom III 𝐟𝐨𝐫 𝐢𝐧𝐝𝐞𝐩𝐞𝐧𝐝𝐞𝐧𝐭 𝑬 𝐚𝐧𝐝 𝑭

𝑷 𝑬 = 𝑷 𝑬𝑭 ∪ 𝑬𝑭𝒄 = 𝑷 𝑬𝑭 + 𝑷 𝑬𝑭 𝒄
= =
𝑷 𝑬𝑭 = 𝑷 𝑬 ∙ 𝑷 𝑭
= 𝑷 𝑬 ∙ 𝑷 𝑭 + 𝑷 𝑬𝑭𝒄 ⟹

⟹ 𝑷 𝑬 = 𝑷 𝑬 ∙ 𝑷 𝑭 + 𝑷 𝑬𝑭𝒄

𝑷 𝑬𝑭𝒄 = 𝑷 𝑬 − 𝑷 𝑬 ∙ 𝑷 𝑭

𝑷 𝑬𝑭𝒄 = 𝑷 𝑬 ∙ 𝟏 − 𝑷 𝑭

𝑷 𝑬𝑭𝒄 = 𝑷 𝑬 ∙ 𝑷 𝑭𝒄 that proves the proposition.

30
 The Independence of Three Events 𝑬, 𝑭, and 𝑮
Events 𝑬, 𝑭, and 𝑮 are independent if
𝑷(𝑬𝑭) = 𝑷(𝑬)𝑷(𝑭),
𝑷 𝑬𝑮 = 𝑷 𝑬 𝑷 𝑮 ,
𝑷 𝑭𝑮 = 𝑷 𝑭 𝑷 𝑮 ,
and
𝑷(𝑬𝑭𝑮) = 𝑷(𝑬)𝑷(𝑭)𝑷(𝑮).
Proposition 3.8.1b: If events 𝑬, 𝑭, 𝑮 are independent, then 𝑬 is independent of any
event formed from 𝑭 and 𝑮.
Proof:
Distributive law:
𝑷 𝑬 𝑭∪𝑮 = = 𝑷 𝑬𝑭 ∪ 𝑬𝑮 =∗
(𝑬 ∪ 𝑭 )𝑮 = 𝑬𝑮 ∪ 𝑭𝑮
By using PROPOSITION 3.4.2 that 𝑷(𝑬 ∪ 𝑭) = 𝑷(𝑬) + 𝑷(𝑭) − 𝑷(𝑬𝑭) one can get
∗ = 𝑷 𝑬𝑭 + 𝑷 𝑬𝑮 − 𝑷 𝑬𝑭𝑮 =
= 𝑷 𝑬 𝑷 𝑭 + 𝑷 𝑬 𝑷 𝑮 − 𝑷 𝑬 𝑷 𝑭𝑮 =
= 𝑷 𝑬 𝑷 𝑭 + 𝑷 𝑮 − 𝑷 𝑭𝑮 =

= 𝑷(𝑬 )𝑷(𝑭 ∪ 𝑮). 31

 Independence of More Than Three Events
Events 𝑬𝟏, 𝑬𝟐, . . . , 𝑬𝒏 are said to be independent if for every subset 𝑬1, 𝑬2, . . . , 𝑬r,
where 𝒓 ≤ 𝒏, of these events obey rule of independence:
𝑷(𝑬𝟏𝑬𝟐 · · · 𝑬𝒓 ) = 𝑷(𝑬𝟏)𝑷(𝑬𝟐) · · · 𝑷(𝑬𝒓) .

EXAMPLE 3.8d . A system composed of 𝒏 separate components is said to be a

parallel system if it functions when at least one of the components functions (See
Figure). If component 𝒊, independent of other components, functions with
probability 𝒑𝒊 , 𝒊 = 𝟏, . . . , 𝒏, what is the probability the system functions?

Solution: If 𝑷(𝑬) is probability that system functions 1

and 𝑷(𝑭) is probability that system fails, then 2
3
𝑷 𝑬 = 𝟏 − 𝑷 𝑭 = 𝟏 − 𝑷(𝑬𝒄𝟏 𝑬𝒄𝟐 𝑬𝒄𝟑 ∙ ⋯ ∙ 𝑬𝒄𝒏 ) =
n
where 𝑷 𝑬𝒄𝟏 = 𝟏 − 𝒑𝟏 .
𝒏

= 𝟏 − ෑ 𝟏 − 𝒑𝒊 = If 𝒑𝒊 = 𝟎. 𝟗 and 𝒏 = 𝟒 =
𝒊=𝟏
= 𝟏 − 𝟏 − 𝟎. 𝟗 𝟒 = 𝟎. 𝟗𝟗𝟗𝟗 .
32
 Independence of More Than Three Events
Events 𝑬𝟏, 𝑬𝟐, . . . , 𝑬𝒏 are said to be independent if for every subset 𝑬1, 𝑬2, . . . , 𝑬r,
where 𝒓 ≤ 𝒏, of these events obey rule of independence:
𝑷(𝑬𝟏𝑬𝟐 · · · 𝑬𝒓 ) = 𝑷(𝑬𝟏)𝑷(𝑬𝟐) · · · 𝑷(𝑬𝒓) .
EXAMPLE 3.8d . A pedestrian should travel from “point A” to “point B” in a city (see
map below). The route of that pedestrian in that city can be chosen from a variety of
𝒏 routs. The pedestrian successfully gets to point B when at least one path is not
under construction. If path 𝒊 is independent from others and it’s open for pedestrians
with probability 𝒑𝒊 , where 𝒊 = 𝟏, . . . , 𝒏, what is the probability that the pedestrian
gets to the point B?

Solution:

𝑷 𝑬 = 𝟏 − 𝑷 𝑭 = 𝟏 − 𝑷(𝑬𝒄𝟏 𝑬𝒄𝟐 𝑬𝒄𝟑 ∙ ⋯ ∙ 𝑬𝒄𝒏 ) =

where 𝑷 𝑬𝒄𝟏 = 𝟏 − 𝒑𝒊 .
𝒏

= 𝟏 − ෑ 𝟏 − 𝒑𝒊 = If 𝒑𝒊 = 𝟎. 𝟗 and 𝒏 = 𝟒 =
𝒊=𝟏
= 𝟏 − 𝟏 − 𝟎. 𝟗 𝟒 = 𝟎. 𝟗𝟗𝟗𝟗 .
33
 Bayes’ Formula
Let’s assume that 𝑬 and 𝑭 are events that 𝑬𝑭 ≠ ∅.
The points in 𝑬 are described as
𝑬 = 𝑬𝑭 ∪ 𝑬𝑭𝒄 .
By using Axiom III, one can get that

𝑷 𝑬 = 𝑷 𝑬𝑭 ∪ 𝑬𝑭𝒄 = 𝑬𝑭 𝑬𝑭𝒄 = 𝟎 = 𝑷 𝑬𝑭 + 𝑷 𝑬𝑭𝒄 = ∗

𝑷 𝑬𝑭
where 𝑷 𝑬 𝑭 = ⟹ 𝑷 𝑬𝑭 = 𝑷 𝑬 𝑭 ∙ 𝑷 𝑭 𝐚𝐧𝐝 𝑷 𝑬𝑭𝒄 = 𝑷 𝑬 𝑭𝒄 ∙ 𝑷 𝑭𝒄
𝑷 𝑭

∗ = 𝑷 𝑬 𝑭 ∙ 𝑷 𝑭 + 𝑷 𝑬 𝑭𝒄 ∙ 𝑷 𝑭𝒄 = ∗
by using PROPOSITION 3.4.1: 𝑷 𝑭𝒄 = 𝟏 − 𝑷 𝑭 , one can get that

∗ = 𝑷 𝑬 𝑭 ∙ 𝑷 𝑭 +𝑷 𝑬 𝑭𝒄 ∙ 𝟏 − 𝑷 𝑭 Eq.(3.7.1)
Weighted Weighted probability of 𝑬
probability of 𝑬 when 𝑭 does not occur
when 𝑭 occurs

The equation 3.7.1 𝑷 𝑬 = 𝑷 𝑬 𝑭 ∙ 𝑷 𝑭 + 𝑷 𝑬 𝑭𝒄 ∙ 𝟏 − 𝑷 𝑭 is a useful formula because

it enables one to determine the probability 𝑷 𝑬 by “conditioning” that the second event 𝑭
has or has not occurred.
34
 Bayes’ Formula
Let’s to show that an additional “conditioning” also allows for a better prediction of
conditional probability 𝑷 𝑭 𝑬 .
Suppose that 𝑭𝟏, 𝑭𝟐, . . . , 𝑭𝒏 are mutually exclusive
𝒏
events such that
𝑺 = ራ 𝑭𝒊
𝒊=𝟏
From the diagram one can see that 𝒏

𝑬 = ራ 𝑬𝑭𝒊
𝒊=𝟏
𝑨𝒙𝒊𝒐𝒎 𝑰𝑰𝑰
Thus, 𝑷 𝑬 = 𝑷 ⋃𝒏𝒊=𝟏 𝑬𝑭𝒊 = σ𝒏𝒊=𝟏 𝑷 𝑬𝑭𝒊 =∗

From the Commutative law and 𝐄𝐪. (𝟑. 𝟔. 𝟏):

𝑷 𝑬𝑭𝒊
𝑬𝑭𝒊 = 𝑭𝒊 𝑬 ⟹ 𝑷 𝑬𝑭𝒊 = 𝑷 𝑭𝒊 𝑬 = 𝑷 𝑬 𝑭𝒊 = 𝑷 𝑭𝒊
= 𝑷 𝑬 𝑭𝒊 ∙ 𝑷 𝑭𝒊 ⟹

∗= σ𝒏𝒊=𝟏 𝑷 𝑬 𝑭𝒊 ∙ 𝑷 𝑭𝒊 . 𝐄𝐪. 𝟑. 𝟕. 𝟐
35
 Bayes’ Formula
𝑷 𝑬𝑭 = 𝑷 𝑬 𝑭 ∙ 𝑷 𝑭 Eq.(3.6.1)

𝑷 𝑬 = σ𝒏𝒊=𝟏 𝑷 𝑬 𝑭𝒊 ∙ 𝑷 𝑭𝒊 . Eq.(3.7.2)

As event 𝑬 has occurred, one might be interested in determining which one of 𝑭𝒋 also
occurred, where 𝒊 ≠ 𝒋. One can derive from the Commutative law that
(3.6.1)
𝑷 𝑬𝑭𝒋 = 𝑷 𝑭𝒋 𝑬 = 𝑷 𝑭𝒋 𝑬 ∙ 𝑷 𝑬 ⟹

𝑷 𝑬𝑭𝒋 (3.7.2) 𝑷 𝑬𝑭𝒋 (3.6.1) 𝑷 𝑬 𝑭 ∙ 𝑷 𝑭

𝒋 𝒋
𝑷 𝑭𝒋 𝑬 = = = 𝐄𝐪. (𝟑. 𝟕. 𝟑)
𝑷 𝑬 σ𝒏𝒊=𝟏 𝑷 𝑬 𝑭𝒊 ∙ 𝑷 𝑭𝒊 σ𝒏𝒊=𝟏 𝑷 𝑬 𝑭𝒊 ∙ 𝑷 𝑭𝒊

𝑷 𝑬 𝑭𝒋 ∙ 𝑷 𝑭𝒋
𝑷 𝑭𝒋 𝑬 = 𝒏 is Bayes’ formula named after the English philosopher
σ𝒊=𝟏 𝑷 𝑬 𝑭𝒊 ∙ 𝑷 𝑭𝒊

Thomas Bayes. It describes the probability of an event, based on prior knowledge of

conditions that might be related to the event. For example, if the probability to develop
a health problem 𝑷(𝑭𝒋 ) might increase with age 𝑬, so Bayes’ formula allows calculation
of the probability 𝑷 𝑭𝒋 𝑬 to an individual of a known age 𝑬 to be assessed accurately.
36
Example 3.7f . An airplane is missing, and it is presumed that it was equally likely to
have gone down in any of three possible regions. Let 𝜶𝒊 denote the probability that
the airplane will be not found upon a search of the 𝒊𝒕𝒉 region, even if the airplane is,
in fact, in the 𝒊𝒕𝒉 region. The constants 𝜶𝒊 are called overlook probabilities and they
are generally attributable to the environmental conditions of the regions. What is
the conditional probability that the plane is in the 1st region, given that a search of
region 𝟏 is unsuccessful?
Solution: Let’s assume that the event 𝑹𝒊 means that the plane is in the region 𝒊, thus
𝟏
𝑷 𝑹𝒊 = , where 𝒊 = 𝟏, 𝟐, 𝟑. Also, 𝑬 is the event that a search of region 1 is
𝟑
unsuccessful.
𝑷 𝑬 𝑭𝒋 ∙𝑷 𝑭𝒋 𝑷 𝑬 𝑹𝟏 ∙𝑷 𝑹𝟏
From Bayes’ formula 𝑷 𝑭𝒋 𝑬 = σ𝒏 , one can obtain 𝑷 𝑹 𝟏 𝑬 = =
𝒊=𝟏 𝑷 𝑬 𝑭𝒊 ∙ 𝑷 𝑭𝒊 σ𝒊=𝟏 𝑷 𝑬 𝑹𝒊 ∙ 𝑷 𝑹𝒊
𝒏

𝑷 𝑬 𝑹𝟏 = 𝜶𝟏 , 𝐛𝐜 𝜶𝟏 ≠ 𝟎 𝐟𝐨𝐫 𝐭𝐡𝐞 𝟏 𝐫𝐞𝐠𝐢𝐨𝐧

= 𝑷 𝑬 𝑹𝟐 = 𝟏, 𝐛𝐜 𝐩𝐥𝐚𝐧𝐞 𝐢𝐬 𝐢𝐧 𝐭𝐡𝐞 𝟐 𝐫𝐞𝐠𝐢𝐨𝐧 =
𝑷 𝑬 𝑹𝟑 = 𝟏, 𝐛𝐜 𝐭𝐡𝐞 𝐩𝐥𝐚𝐧𝐞 𝐢𝐬 𝐢𝐧 𝐭𝐡𝐞 𝟑 𝐫𝐞𝐠𝐢𝐨𝐧
where 𝑷 𝑬 𝑹𝒊 is a conditional probability: the likelihood of event 𝑬 occurring given
that 𝑹𝒊 is true.
𝟏
𝜶𝟏 ∙ 𝜶𝟏 𝒊𝒇 𝟎. 𝟒 𝟏
𝟑
= 𝟏 𝟏
=
𝟏 𝜶𝟏 + 𝟐
= = = ≈ 𝟎. 𝟏𝟕.
𝜶𝟏 = 𝟎. 𝟒 𝟐. 𝟒 𝟔
𝜶𝟏 ∙ + 𝟏 ∙ + 𝟏 ∙
𝟑 𝟑 𝟑

37