LECTURE 1

Probability

Lecturer: Nguyen Thi Thu Van

Email: [email protected]
 Content
 Counting rules

 Probability concepts & Rules of Probability

 Sample spaces and events, simple probability, and
joint probability.

 Basic probability rules, like general addition rule,

addition rule for mutually exclusive events, rule for
collectively exhaustive events.

 Conditional probability

 Bayes’ theorem
Counting Rules
Get started by A simple example
 Suppose a communication system consists
of n antennas arranged in a line, with 2 of
them being defective:

 The system is considered functional if no

two defective antennas are consecutive.

 Calculate P(functional system).

Answer. 𝑛 = 4: The possible configurations: 0011, 0101, 0110,

1001, 1010, 1100. Therefore, we get 3 among 6, and thus
P(functional system) = ½.
For more complicated problems, we need an effective way to
count, that is Counting Rules.
Basic Principle of Counting
Basic Principle of Counting.
Suppose 𝑟 experiments to
be performed successively.
• For each outcome of
Experiment 1, 𝑛1 possible
outcomes.
• For each outcome of
Experiment 𝑖, we have
𝑛𝑖+1 outcomes for
Experiment 𝑖 + 1.
Counting: all possible outcomes ς𝑟𝑖=1 𝑛𝑖 = 𝑛1 × 𝑛2 × ⋯ × 𝑛𝑟
 Exercises

Exercise 1. A small community with 10 women and

each woman has 3 children. We have to pick one
pair as mother & child of the year. How many
possibilities?
Exercise 2. Find # possible 7 place license plates if
the first 3 places are letters and the final 4 places
are numbers.
Also, what if no repetition among letters or numbers?
Permutations
Permutations.
• A permutation of n
objects is an ordered
sequence of those n
objects.
• Two permutations
only differ according
to the order of the
objects.

Counting: all permutations 𝑃𝑛 = 𝑛! = 𝑛 × 𝑛 − 1 × ⋯ × 2

We also are interested in ordered arrangements of
some of r elements of a set, this is called an r-
permutation.

Counting: For 1 ≤ 𝑟 ≤ 𝑛, all permutations

𝑛!
𝑃𝑛 𝑟 ≡ 𝑃 𝑛, 𝑟 = 𝑛 × 𝑛 − 1 × ⋯ × 𝑛 − 𝑟 + 1 =
𝑛−𝑟 !
Example. How many ways are there to select a first-prize
winner, a second-prize winner, and a third-prize winner from
100 different people who have entered a contest?
 Exercises
Exercise 1. Suppose that a saleswoman has to visit eight
different cities. She must begin her trip in a specified city, but
she can visit the other seven cities in any order she wishes.
How many possible orders can the saleswoman use when
visiting these cities?
Exercise 2. How many permutations of the letters ABCDEFGH
contain the string ABC?
Exercise 3. In how many ways can we select three students
from a group of five students to stand in line for a picture? In
how many ways can we arrange all five of these students in a
line for a picture?
Combinations
Combinations.
• A combination of 𝑟
objects among 𝑛
objects is non ordered
subset of 𝑟 objects.
• Two combinations
only differ according
to nature of their
objects.
𝑛 𝑛!
Counting: all combinations 𝑛 C𝑟 ≡ 𝐶 𝑛, 𝑟 ≡ 𝑟 = 𝑟! 𝑛−𝑟 !
Exercises
Exercise 1. We have a group of 5 women and 7 men. We
wish to form a committee with 2 women and 3 men. Find
the number of possibilities.
Also, what if 2 men refuse to serve together?
Exercise 2. How many poker hands of five cards can be
dealt from a standard deck of 52 cards?
Also, how many ways are there to select 47 cards from a
standard deck of 52 cards?
Exercise 3. How many ways are there to select five
players from a 10-member tennis team to make a trip to a
match at another school?
Binomial Theorem. Let 𝑎, 𝑏 ∈ ℝ and 𝑛 ≥ 1. Then
𝑛
𝑛 𝑛 𝑘 𝑛−𝑘
𝑎+𝑏 =෍ 𝑎 𝑏
𝑘
𝑘=1
Example. The coefficient on 𝑥 6 𝑦 8 in the

14 14 14!
expansion of 𝑥 + 𝑦 is =
6 6! 14−6 !

Application of Binomial theorem. Let 𝐴 be a set

with 𝐴 = 𝑛 and 𝑃𝑛 = the collection of all subsets
of 𝐴. Then 𝑃𝐴 = 2𝑛 .
Multinomial Coefficients
Multinomial coefficients. Divisions of n objects into r
groups with size 𝑛1 , 𝑛2 , … , 𝑛𝑟 . We have
• 𝑛 objects and 𝑟 groups.
• We want 𝑛𝑗 objects in group 𝑗 and σ𝑟𝑗=1 𝑛𝑗 = 𝑛.

𝑛 𝑛! 𝑛!
Counting: # divisions 𝑛 , 𝑛 , … , 𝑛 ≡ = ς𝑟
1 2 𝑟 𝑛 !𝑛
1 2 !⋯𝑛 𝑟 ! 𝑗=1 𝑛𝑗 !

Example. A police department with 10 officers and 5 have to

patrol the streets, 2 are permanently working at the station and
3 are on reserve at the station. How many divisions do we get?
Multinomial Theorem. Let 𝑥1 , 𝑥2 , … , 𝑥𝑟 ∈ ℝ and 𝑛 ≥ 1.
Then
𝑛
𝑥1 + 𝑥2 + ⋯ + 𝑥𝑟
𝑛 𝑛1 𝑛2 𝑛𝑟
= ෍ 𝑛1 , 𝑛2 , … , 𝑛𝑟 𝑥1 × 𝑥2 × ⋯ × 𝑥𝑟
𝑛1 ,𝑛2 ,…,𝑛𝑟 ∈ 𝐴𝑟,𝑛

where
𝐴𝑟,𝑛 = 𝑛1 , 𝑛2 , … , 𝑛𝑟 ∈ 𝑁 𝑟 |𝑛1 + 𝑛2 + ⋯ + 𝑛𝑟 = 𝑛
Example. The coefficient on 𝑥 5 𝑦 2 𝑧 7 in the expansion of

14 14 14!
𝑥+𝑦+𝑧 is = = 72,072
5, 2,7 5! 2! 7!
 Counting rules
Rule 1
• If any one of k different mutually exclusive and
collectively exhaustive events can occur on each of n
trials, the number of possible outcomes is kn
Rule 2
• If there are k1 events on the first trial, k2 events on the
second trial, … and kn events on the nth trial, the number
of possible outcomes is (k1)(k2)…(kn)

Rule 3
• The number of ways that n items can be arranged in
order is n! = (n)(n – 1)…(1)

Rule 4 - Permutations
• An arrangement of a set of objects with regard to the
order of the arrangement 𝑛 𝑃𝑘

Rule 5 - Combinations
Find the minimum number of
students in the class such that five • An arrangement of a set of objects without regard to the
of them are born in the same 𝑛
month?  order of the arrangement 𝑛 𝐶𝑘 ≡
𝑘
Probability Concepts
Probability Concepts
Random experiments

Events

Mutually exclusive events

Collectively exhaustive events

Empirical probability

Logical probability

Subjective probability

Simple probability

Joint probability
 Random Experiments
 A random experiment is an observational process
whose outcomes cannot be known in advance.

 The set of all outcomes is the sample space for the

experiment. A sample space with a countable number of
outcomes is discrete. Otherwise, it is called continuous.

 For instance, flip a coin, the sample space consists of

2 outcomes S = {H, T} : countable.

 The waiting time for a bus to come, the sample space

𝑆 = {𝑡|𝑡 ≥ 0}.
 Events

 An event is any subset of outcomes in the

sample space.
 A simple event or elementary event is a single
outcome, while a compound event is a
combination of two or more simple events (with
two or more outcomes).
Examples
 Roll a die, the sample
space consists of 6
elementary events:
S = {1, 2, 3, 4, 5, 6}
 Roll 2 dice, the sample
space consists of 36
elementary events:
S = {(1,1), (1,2),…
…(6,5), 6,6)}
 Flip a coin:
 The sample space consists
of 2 elementary events: S = {H, T}
 An event of getting one Head: E = {H}
 Flip a coin twice:
 The sample space consists of 4 elementary events:
S = {HH, HT, TH, TT}
 An event of getting one Head is a compound event:
E = {HT, TH}
It is clear that in both examples, the sample space is a
compound event.
 Probability
 The probability of an event is a number that
measures the relative likelihood that the event will
occur, ranged from 0 to 1. It can be also represented
as a number between 0 and 100, percent (%).
 The probability of event A is denoted P(A).
 How to Assess Probability?
Subjective approach is based on informed option or personal
judgement. For instance, we would say there is a 20% chance that a
new stock offered in an initial public offering (IPO) will reach or
exceed its target price on the first day.
Classical approach is based on logic or theory. For instance, tossing
1
a coin, then, by theory, 𝑃 𝑇𝑎𝑖𝑙 =
2

Empirical approach is estimated from observed outcome frequency.

For instance, an industrial components manufacturer interviewed 280
production workers before hiring 70 of them. So let H be event that a
70
randomly chosen interviewee is hired, then 𝑃 𝐻 = = 0.25
280
Rules of Probability
 Probability of the Complement of an Event
 The complement of an event A is denoted by
𝐴′ (or Ā) and consists of everything in the
sample space except event A.

 𝐴 and 𝐴′ together comprise the entire sample

space: 𝑃(𝐴) + 𝑃(𝐴′ ) = 1 or 𝑃(𝐴′ ) = 1 – 𝑃(𝐴)
 Intersection of Two Events

 The intersection of two events A and B

(denoted A  B or “A and B”) is the event consisting of
all common outcomes of the event A and the event B.
 If there is no common outcomes between 2 events, two
events are called disjoint.

 Probability of the intersection of two events is called

the joint probability and denoted 𝑃(𝐴 ∩ 𝐵)
 Union of Two Events

 The union of two events consists of all outcomes in

the sample space that are contained either in event A
or in event B or both (denoted A  B or “A or B”).
 The probability of union of 𝐴 or 𝐵 is denoted 𝑃(𝐴 ∪ 𝐵)
 Complement of a Union of Two Events

𝑃 𝑛𝑒𝑖𝑡ℎ𝑒𝑟 𝐴 𝑛𝑜𝑟 𝐵
= 𝑃 𝑛𝑜𝑡 𝐴 𝑜𝑟 𝐵
= 𝑃 𝑛𝑜𝑡 𝐴 𝑎𝑛𝑑 𝑛𝑜𝑡 𝐵
= 1 – 𝑃(𝐴 𝑜𝑟 𝐵)
 General Law of Addition
 The general law of addition states that the
probability of the union of two events A and B is
𝑃 𝐴 ∪ 𝐵 = 𝑃 𝐴 + 𝑃 𝐵 − 𝑃(𝐴 ∩ 𝐵)
Outcomes here are double-
counted by 𝑃 𝐴 + 𝑃(𝐵)

When A and B are disjoint (i.e., 𝐴 ∩ 𝐵 = ∅),

𝑃 𝐴∪𝐵 =𝑃 𝐴 +𝑃 𝐵
Example
A survey of introductory statistics students showed
that 29.7 percent have AT&T wireless service
(event A), 73.4 percent have a Visa card (event B),
and 20.3 percent have both (event 𝐴 ∩ 𝐵).
The probability that a student uses AT&T or has a
Visa card is
𝑃 𝐴∪𝐵 =𝑃 𝐴 +𝑃 𝐵 −𝑃 𝐴∩𝐵
= 0.297 + 0.734 − 0.203 = 0.828
Example. A survey asked workers which changes in office
design would increase productivity. Respondents were
allowed to answer more than one type of design change.
70% of workers said noise reduction (Event A), 67% said
more storage space (Event B) and 56% said both noise
reduction and storage space would increase the productivity.
What percentage of workers responded that:
• either noise reduction or increased storage space would
improve productivity
• neither noise reduction nor increased storage space
would improve productivity?
 Mutually Exclusive Events
 Mutually exclusive events
are events that cannot
occur simultaneously.

That is, suppose 𝐸𝑖 and 𝐸𝑗 (𝑖 ≠ 𝑗) are 2 events.

If 𝐸𝑖 ∩ 𝐸𝑗 = ∅ then 𝐸𝑖 and 𝐸𝑗 are mutually exclusive.
Mutually exclusive events, for example, right and
left hand turns, even and odd numbers on a die,
winning and losing a game, or running and
walking.

Non-mutually exclusive events, for example,

driving and listening to the radio, even numbers
and prime numbers on a die, losing a game and
scoring, or running and sweating.
 Collectively Exhaustive Events

A set of events is collectively exhaustive if at least

one of the events must occur.
That is, suppose 𝐸1 , 𝐸2 , … , 𝐸𝑛 are events of an
experiment and 𝑆 is the sample space. The events
𝐸1 , 𝐸2 , … , 𝐸𝑛 are collectively exhaustive if
𝑆 = 𝐸1 ∪ 𝐸2 ∪ ⋯ ∪ 𝐸𝑛 .
Collectively exhaustive events, for example, if we roll a
die, it must land on one of the values 1, 2, 3, 4, 5, or 6.
Or, suppose we have a survey that asks individuals to
select their favorite basketball player position. The only
potential responses are: Point Guard Shooting Guard,
Small Forward, Power Forward, or Center.

Non-collectively exhaustive events, for example, A =

the event that a person was born in Canada, B = the
event that a person was born in US.
 Mutually Exclusive & Collectively Exhaustive - MECE

It is worth mentioning that MECE is a principle that says

events do not overlap and all events account for all
possibilities. The MECE principle provides a number of
benefits, including clear thinking, comprehensive analysis,
efficiency, and better decision-making.

𝑛 𝑛

𝑃 𝑆 = ራ 𝐸𝑖 = ෍ 𝑃(𝐸𝑖 ) = 1
𝑖=1 𝑖=1
𝑖𝑓 𝐸𝑖 ∩ 𝐸𝑗 = ∅, ∀𝑖 ≠ 𝑗
Example. Let S be the event that a randomly
chosen female aged 18–24 is a smoker.
Let C be the event that a randomly chosen female
aged 18–24 is a Caucasian.
Given 𝑃 𝑆 = 0.246; P C = 0.830; P S ∩ 𝐶 = 0.232.

𝑃 𝑆 ′ = 1 − 𝑃 𝑆 = 1 − 0.246 = 0.734
𝑃 𝐶 ′ = 1 − 𝑃 𝐶 = 1 − 0.830 = 0.170
𝑃 𝑆 ∩ 𝐶 ′ = 𝑃 𝑆 − 𝑃 𝑆 ∩ 𝐶 = 0.014
Conditional Probability
 Conditional Probability
 The probability of event A
given that event B has
occurred.
Multiplication rule
P(A  B)  P(A | B) P(B)

P(A  B)  P(A) P(B)

Example: Of the population age 16–21 and not in
college, there are
 13.50% are unemployed (event U)
 29.05% are high school dropouts (event D)
 5.32% are unemployed high school dropouts.
What is the conditional probability that a member of
this population is unemployed, given that the
person is a high school dropout? ≡ 𝑃 𝑈 𝐷 = ?
 Examples Dependent vs. Independent events
 Independent events
are not affected by  Dependent events
previous events. depend on what
happened before.

What color marbles could be picked out of the bag next time?
Example
Suppose 50 percent of the customers at Pizza
Palooza order a pizza, 80 percent order a soft
drink, and 40 percent order both a pizza and a
soft drink.
Is ordering a soft drink
independent of ordering
a pizza?
An other Example
Over 1,000 people try to climb Mt. Everest every year.
Of those who try to climb Everest, 31 percent
succeed. The probability that a climber is at least 60
years old is 0.04. The probability that a climber is at
least 60 years old and succeeds in climbing Everest
is 0.005.
 Find the probability of success, given that a climber
is at least 60 years old.
 Is success in climbing Everest independent of age?
 From the Law of independent events to System Reliability
Suppose that a website has two independent file servers A and B.
If each has 99 percent reliability, what is the total reliability?
Let A, B = the event the server A, B will be uptime, respectively.

𝑃 𝐶 = 1 − 𝑃 𝐴’ × 𝑃 𝐵’ = 1 − 0.01 × 0.01 = 99.99%

The principle of redundancy: when individual components have

a low reliability, high reliability can be still achieved with
massive redundancy.
Example

Bob sets two alarm clocks (battery-powered) to be

sure he arises for his Monday 8:00 a.m.
accounting exam. There is a 75 percent chance
that either clock will wake Bob.
 What is the probability that Bob will oversleep?
 If Bob had three clocks, would he have a 99
percent chance of waking up?
Odds of an Event
 What are odds of an event?
Statisticians usually speak of probabilities rather than
odds, but in sports and games of chance, we often hear
odds quoted.
 The odds in favor of an event A is the ratio of the
probability that event A will occur to the probability that
event A will not occur. P( A) P( A)
Odds = 
P( A ') 1  P( A)

 Its reciprocal is the odds against event A.

P ( A) 1  P ( A)
Odds  
P ( A) P ( A)
 Odds of an Event

The odds in favor of event A The odds against event A

Number of Event A Number of Event Not A

Number of Event Not A Number of Event A

The odds in favor of Red The odds against Red

P ( A) 1  P ( A)
P( A) P( A) Odds  
Odds =  P ( A) P ( A)
P( A ') 1  P( A)
Example: Suppose the IRS tax audit rate is 1.41
percent among taxpayers earning between
$100,000 and $199,999.

Let 𝐴 = the event that the taxpayer is audited and

set 𝑃 𝐴 = 0.0141.

The odds against an audit are

𝑃(𝑛𝑜 𝑎𝑢𝑑𝑖𝑡) 𝑃(𝐴′) 1 − 𝑃(𝐴) 1 − 0.0141

= = = = 70
𝑃(𝑎𝑢𝑑𝑖𝑡) 𝑃(𝐴) 𝑃(𝐴) 0.0141
meaning 70 to 1 against being audited.
How to Find Probability?
 Contingency Table
 Collect data of 100 cars:
 Each car either has AC or no AC

 Each car either has GPS or no GPS

GPS No GPS Total

AC 35 55 90
No AC 5 5 10
Total 40 60 100
Of the 100 cars studied,
 Marginal probability: 90% have air conditioning (AC)
and 40% have a GPS.
 Joint probability:
35% of the cars have both.

GPS No GPS Total

AC 0.35 0.55 0.90
No AC 0.05 0.05 0.10
Total 0.40 0.60 1.00

 Conditional probability:
P(GPS  AC) 0.35
P(GPS | AC)    0.3889
P(AC) 0.90
Decision Trees
 Decision Trees
.35
.90 P(AC ∩ GPS) = 0.35
Given AC or
no AC:

.55 P(AC ∩ GPS’) = 0.55

.90
All Conditional
Probabilities
Cars
.05
.10 P(AC’ ∩ GPS) = 0.05

.05 P(AC’ ∩ GPS’) = 0.05

.10
 Decision Trees
.35
.40 P(GPS and AC) = 0.35
Given GPS
or no GPS:

.05 P(GPS and AC’) = 0.05

.40
All Conditional
Probabilities
Cars
.55
.60 P(GPS’ and AC) = 0.55

.05 P(GPS’ and AC’) = 0.05

.60
Example
Of grocery shoppers who have a shopping cart, 70% pay by
credit/debit card (event C1), 20% pay cash (event C2), and
10% pay by check (event C3).
Of shoppers without a grocery cart, 50% pay by credit/debit
card (event C1), 40% pay cash (event C2), and 10% pay by
check (event C3).
On Saturday morning, 80% of the shoppers take a shopping
cart (event S1) and 20% do not (event S2).
 Sketch a tree based on these data.
 Calculate the probability of all joint probabilities.
Bayes’ Theorem
 Bayes’ Theorem
 Bayes’ Theorem is used to revise previously
calculated probabilities based on new
information.

 It is an extension of conditional probability.

 Developed by
Thomas
Bayes in the
18th Century.
Example. Based on historical data, three hospital
trauma centers have 50, 30, and 20 percent of
the cases, respectively. The probability of a case
resulting in a malpractice suit in each of the three
hospitals is 0.001, 0.005, and 0.008, respectively.
If a malpractice suit is filed, what is the probability
that it originated in hospital 1?
Bayes’ theorem says that
Basic procedure of Bayesian methodology
Example. Machines A, B, and C produce two
parts X and Y. Of all the parts produced,
machine A produces 60%, machine B produces
30% and machine C produces10%. In addition,
• 40% of parts made by machine A are part X.
• 50% of parts made by machine B are part X.
• 70% of parts made by machine C are part X.
Part X has been selected. What is the revised
probability that it came from machine A?
Event (E) Prior Conditional Joint Posterior
probability probability probability probability
𝑷(𝑬𝒊 ) 𝑷(𝑿|𝑬𝒊 ) 𝑷(𝑿 ∩ 𝑬𝒊 ) 𝑷(𝑬𝒊 |𝑿)

Machine A 0.6 0.4 0.24 0.24

=
0.46

Machine B 0.3 0.5 0.15 0.15

=
0.46

Machine C 0.1 0.7 0.07 0.07

=
0.46

𝑷 𝑿 = 𝟎. 𝟒𝟔
Exercise

A drug test for athletes has a 5 percent false

positive rate and a 10 percent false negative
rate. Of the athletes tested, 4 percent have
actually been using the prohibited drug.
If an athlete tests positive, what is the probability
that the athlete has actually been using the
prohibited drug?
Exercise
Suppose that
 Half of a set of the parts are manufactured by
machine A and half by machine B.
 Four percent of all the parts are defective.
 Six percent of the parts manufactured on
machine A are defective.
Find the probability that a part was manufactured
on machine A, given that the part is defective.
 Exercise
Suppose that
 An airport gamma ray luggage scanner coupled with
a neural net artificial intelligence program can detect
a weapon in suitcases with a false positive rate of 2
percent and a false negative rate of 2 percent.
 There is a 0.001 probability that a suitcase contains
a weapon.
If a suitcase triggers the alarm, what is the probability
that the suitcase contains a weapon?
-- The End of Topic --
Thank You!