MS3227 Probability and Applications in

Business
Week 1

Eman Leung, PhD., PMP, LSSBB, CPBI, IHI Improvement Advisor

September 5, 2024
Overview of the course
Tentative plan of the course

ˆ Elements of probability. Including definition of probability, some laws

of probability.

ˆ Characterize random variables by distributions, including univariate

and multivariate distributions.

ˆ Limiting theorems: law of large numbers, central limit theorem.

ˆ Stochastic processes, a collection of random variables.

ˆ Modeling business problem by random variables or stochastic

processes.

We will be a little bit rigorous in math, with some application in business

models.
1
Textbook

ˆ The slides will be self contained.

ˆ Sheldon Ross, A First Course in Probability - 10th edition

ˆ Read the book, so you can be well prepared for the assignment and
exams.

ˆ It’s easy to find a solution manual on the web (beware: might

contain mistakes).

ˆ Sheldon Ross, Introduction to Probability Models - 9th edition. We

may cover some stochastic processes from this book if time permits.

2
Course Administration

ˆ You are expected to have taken CB2200 before, have basic

knowledge of calculus.

ˆ TA: TBA

ˆ Canvas posting and submission: TBA

ˆ Four homework assignments, will be due one week after posting date
(2 hours before the class). We do not accept late submission.

ˆ 30% homework, 30% midterm exam and 40% final exam.

ˆ You are allowed to bing one page, double sides, A4 paper

handwritten cheatsheet and submit with your exam.

3
Course Administration

ˆ Homework Assignments can be done in group (1 - 5 students max).

You should let me know by end of week 2. Groups are not allowed
to change after then.

ˆ Strongly recommend interactions during the lecture. Bonus will be

given to active students.

4
What is probability
What is probability?

Probability is a measurement of uncertainty.

ˆ People talk loosely about probability all the time:

”What’s the chance of raining tomorrow?”
”What’s the chance that Brazil will win the world cup?”

ˆ For scientific purposes, we need a formal definition of probability.

Probability theory yields mathematical tools to deal with uncertainty in

other subjects.

6
What is probability?

If I flip a fair coin, the probability of getting heads is 0.5

ˆ What does it mean?

7
What is probability?

If I flip a fair coin, the probability of getting heads is 0.5

ˆ What does it mean?

ˆ If I were to toss the coin 10 times, roughly 5 times I will see a head.

7
What is probability?

If I flip a fair coin, the probability of getting heads is 0.5

ˆ What does it mean?

ˆ If I were to toss the coin 10 times, roughly 5 times I will see a head.

ˆ In general, you do an experiment many times under the same

conditions, and you count how many times a particular event
occurs. The proportion roughly gives you the probability of that
particular event.

7
What is probability?

If I flip a fair coin, the probability of getting heads is 0.5

ˆ What does it mean?

ˆ If I were to toss the coin 10 times, roughly 5 times I will see a head.

ˆ In general, you do an experiment many times under the same

conditions, and you count how many times a particular event
occurs. The proportion roughly gives you the probability of that
particular event.

ˆ A probability of 1 means it is certain, a probability of 0 means it is

impossible.

7
Wait

ˆ Sometimes people say the probability of raining tomorrow is 50%.

How do you interpret it?

8
Wait

ˆ Sometimes people say the probability of raining tomorrow is 50%.

How do you interpret it?

ˆ many times ... same conditions ... ratio?

8
Wait

ˆ Sometimes people say the probability of raining tomorrow is 50%.

How do you interpret it?

ˆ many times ... same conditions ... ratio?

ˆ Tomorrow (6-Sept-2024) is unique, you can never repeat it many

times under the same condition.

8
Wait

ˆ Sometimes people say the probability of raining tomorrow is 50%.

How do you interpret it?

ˆ many times ... same conditions ... ratio?

ˆ Tomorrow (6-Sept-2024) is unique, you can never repeat it many

times under the same condition.

ˆ 50% represents subjective belief of the speaker about an event.

8
Wait

ˆ Sometimes people say the probability of raining tomorrow is 50%.

How do you interpret it?

ˆ many times ... same conditions ... ratio?

ˆ Tomorrow (6-Sept-2024) is unique, you can never repeat it many

times under the same condition.

ˆ 50% represents subjective belief of the speaker about an event.

ˆ Bayesian statistics deals with subjective belief. We won’t mention it

in the class.

8
Experiment and Event

ˆ Experiment: tossing a coin twice.

ˆ Event: You get two heads.
ˆ Event: You get one head and one tail.
ˆ Event: You get two tails.

ˆ Experiment: throw two dice.

ˆ Event: sum of the roll is six.
ˆ Event: you get two odd faces.
ˆ Event: ....

Experiment is an action that gives you uncertain outcomes. Event is one

realisation of an experiment.

9
Sample space

ˆ The sample space Ω is a set of all possible outcomes of an

experiment.

10
Sample space

ˆ The sample space Ω is a set of all possible outcomes of an

experiment.

ˆ You roll a die, what is Ω?

10
Sample space

ˆ The sample space Ω is a set of all possible outcomes of an

experiment.

ˆ You roll a die, what is Ω?

ˆ {1, 2, 3, 4, 5, 6}

10
Sample space

ˆ The sample space Ω is a set of all possible outcomes of an

experiment.

ˆ You roll a die, what is Ω?

ˆ {1, 2, 3, 4, 5, 6}

ˆ You tossed three coins together, what is Ω?

10
Sample space

ˆ The sample space Ω is a set of all possible outcomes of an

experiment.

ˆ You roll a die, what is Ω?

ˆ {1, 2, 3, 4, 5, 6}

ˆ You tossed three coins together, what is Ω?

ˆ {HHH, HHT , HTH, HTT , THH, THT , TTH, TTT }.

10
Sample space

ˆ The sample space Ω is a set of all possible outcomes of an

experiment.

ˆ You roll a die, what is Ω?

ˆ {1, 2, 3, 4, 5, 6}

ˆ You tossed three coins together, what is Ω?

ˆ {HHH, HHT , HTH, HTT , THH, THT , TTH, TTT }.

ˆ The different elements of a sample space must be mutually

exclusive and collectively exhaustive.

10
Sample space

ˆ The sample space Ω is a set of all possible outcomes of an

experiment.

ˆ You roll a die, what is Ω?

ˆ {1, 2, 3, 4, 5, 6}

ˆ You tossed three coins together, what is Ω?

ˆ {HHH, HHT , HTH, HTT , THH, THT , TTH, TTT }.

ˆ The different elements of a sample space must be mutually

exclusive and collectively exhaustive.
ˆ Ω for three coins toss cannot be {at least one head, at least one tail}.

10
Sample space

ˆ The sample space Ω is a set of all possible outcomes of an

experiment.

ˆ You roll a die, what is Ω?

ˆ {1, 2, 3, 4, 5, 6}

ˆ You tossed three coins together, what is Ω?

ˆ {HHH, HHT , HTH, HTT , THH, THT , TTH, TTT }.

ˆ The different elements of a sample space must be mutually

exclusive and collectively exhaustive.
ˆ Ω for three coins toss cannot be {at least one head, at least one tail}.

ˆ An event is a collection of possible outcomes.

10
Simple and compound events

ˆ Simple event:

11
Simple and compound events

ˆ Simple event:
ˆ You toss two coins, get HH.

11
Simple and compound events

ˆ Simple event:
ˆ You toss two coins, get HH.
ˆ You roll a die and get 6.

11
Simple and compound events

ˆ Simple event:
ˆ You toss two coins, get HH.
ˆ You roll a die and get 6.

ˆ Compound event: can be decomposed into simple events:

11
Simple and compound events

ˆ Simple event:
ˆ You toss two coins, get HH.
ˆ You roll a die and get 6.

ˆ Compound event: can be decomposed into simple events:

ˆ You toss two coins and get two different outcomes:

11
Simple and compound events

ˆ Simple event:
ˆ You toss two coins, get HH.
ˆ You roll a die and get 6.

ˆ Compound event: can be decomposed into simple events:

ˆ You toss two coins and get two different outcomes:
ˆ You get HT or TH.

11
Simple and compound events

ˆ Simple event:
ˆ You toss two coins, get HH.
ˆ You roll a die and get 6.

ˆ Compound event: can be decomposed into simple events:

ˆ You toss two coins and get two different outcomes:
ˆ You get HT or TH.
ˆ The sum of rolled dice is 6.

11
Simple and compound events

ˆ Simple event:
ˆ You toss two coins, get HH.
ˆ You roll a die and get 6.

ˆ Compound event: can be decomposed into simple events:

ˆ You toss two coins and get two different outcomes:
ˆ You get HT or TH.
ˆ The sum of rolled dice is 6.
ˆ You get (1,5), (2,4), (3, 3), (4, 2) or (5,1).

11
Simple and compound events

ˆ Simple event:
ˆ You toss two coins, get HH.
ˆ You roll a die and get 6.

ˆ Compound event: can be decomposed into simple events:

ˆ You toss two coins and get two different outcomes:
ˆ You get HT or TH.
ˆ The sum of rolled dice is 6.
ˆ You get (1,5), (2,4), (3, 3), (4, 2) or (5,1).
ˆ You get two odd faces from rolling two dice

11
Simple and compound events

ˆ Simple event:
ˆ You toss two coins, get HH.
ˆ You roll a die and get 6.

ˆ Compound event: can be decomposed into simple events:

ˆ You toss two coins and get two different outcomes:
ˆ You get HT or TH.
ˆ The sum of rolled dice is 6.
ˆ You get (1,5), (2,4), (3, 3), (4, 2) or (5,1).
ˆ You get two odd faces from rolling two dice
ˆ You get (1,1), (1,3), ......

11
Set

We need to introduce some mathematical concepts to define probability

more concretely.

ˆ A set is a collection of objects, each object is called element.

ˆ The natural numbers are a set, each element is a number.
ˆ This class is a set, elements are the professor, the TA and all the
students.

ˆ If an element x belongs to a set S, we write x ∈ S.

ˆ If a set has no element, we call it an empty set, ∅.

ˆ If a set contains all possible elements, we call it universal set, Ω.

12
Set

ˆ If all elements of set A belong to set B as well, we say A is a subset

of B and write A ⊂ B.

13
Set

ˆ If all elements of set A belong to set B as well, we say A is a subset

of B and write A ⊂ B.
ˆ {1, 2} ⊂ {1, 2, 3, 4}

13
Set

ˆ If all elements of set A belong to set B as well, we say A is a subset

of B and write A ⊂ B.
ˆ {1, 2} ⊂ {1, 2, 3, 4}
ˆ {HH} ⊂ {HH, HT , TH, TT }

13
Set

ˆ If all elements of set A belong to set B as well, we say A is a subset

of B and write A ⊂ B.
ˆ {1, 2} ⊂ {1, 2, 3, 4}
ˆ {HH} ⊂ {HH, HT , TH, TT }

ˆ What if both A ⊂ B and B ⊂ A?

13
Set

ˆ If all elements of set A belong to set B as well, we say A is a subset

of B and write A ⊂ B.
ˆ {1, 2} ⊂ {1, 2, 3, 4}
ˆ {HH} ⊂ {HH, HT , TH, TT }

ˆ What if both A ⊂ B and B ⊂ A?

ˆ A = B!

13
Set

ˆ If all elements of set A belong to set B as well, we say A is a subset

of B and write A ⊂ B.
ˆ {1, 2} ⊂ {1, 2, 3, 4}
ˆ {HH} ⊂ {HH, HT , TH, TT }

ˆ What if both A ⊂ B and B ⊂ A?

ˆ A = B!

ˆ We use curly brackets to describe a set in terms of its elements:

13
Set

ˆ If all elements of set A belong to set B as well, we say A is a subset

of B and write A ⊂ B.
ˆ {1, 2} ⊂ {1, 2, 3, 4}
ˆ {HH} ⊂ {HH, HT , TH, TT }

ˆ What if both A ⊂ B and B ⊂ A?

ˆ A = B!

ˆ We use curly brackets to describe a set in terms of its elements:

ˆ Sample space of a die roll: {1, 2, 3, 4, 5, 6}

13
Set

ˆ If all elements of set A belong to set B as well, we say A is a subset

of B and write A ⊂ B.
ˆ {1, 2} ⊂ {1, 2, 3, 4}
ˆ {HH} ⊂ {HH, HT , TH, TT }

ˆ What if both A ⊂ B and B ⊂ A?

ˆ A = B!

ˆ We use curly brackets to describe a set in terms of its elements:

ˆ Sample space of a die roll: {1, 2, 3, 4, 5, 6}
ˆ Arbitrary set when all the elements meet some criterion C:
S = {x | x satisfies C }

13
Set

ˆ If all elements of set A belong to set B as well, we say A is a subset

of B and write A ⊂ B.
ˆ {1, 2} ⊂ {1, 2, 3, 4}
ˆ {HH} ⊂ {HH, HT , TH, TT }

ˆ What if both A ⊂ B and B ⊂ A?

ˆ A = B!

ˆ We use curly brackets to describe a set in terms of its elements:

ˆ Sample space of a die roll: {1, 2, 3, 4, 5, 6}
ˆ Arbitrary set when all the elements meet some criterion C:
S = {x | x satisfies C }
ˆ All third year students: S = {x | x is a third year student}

13
Set

ˆ A set can be finite (e.g. the set of people in this class), or infinite
(e.g. the set of real numbers).

ˆ If we can enumerate the elements of an infinite set, i.e., arrange

elements in a list, we say it is countable
ˆ Set of positive integers {1, 2, 3, ...}

ˆ If we cannot enumerate the elements, it is uncountable

ˆ The real numbers.
ˆ The set of all subsets of natural numbers, aka, the power set.

14
Operation of sets

ˆ Let the universal set Ω be the set of all objects we are interested in.

ˆ The complement S C of a set S with respect to Ω is the set of all

elements that are in Ω but not S.

ˆ ΩC = ∅.

ˆ S ⊂ T if all elements in S are also in T .

15
Operation of sets

ˆ The union, S ∪ T of two sets S and T is the set of elements that are
in either S or T or both. S ∪ T = {x | x ∈ S or x ∈ T }. S or T .

16
Operation of sets

ˆ The union, S ∪ T of two sets S and T is the set of elements that are
in either S or T or both. S ∪ T = {x | x ∈ S or x ∈ T }. S or T .

ˆ The intersection, S ∩ T is the set of elements that are both in S

and T . S ∩ T = {x | x ∈ S and x ∈ T }. S and T .

16
Operation of sets

ˆ The union, S ∪ T of two sets S and T is the set of elements that are
in either S or T or both. S ∪ T = {x | x ∈ S or x ∈ T }. S or T .

ˆ The intersection, S ∩ T is the set of elements that are both in S

and T . S ∩ T = {x | x ∈ S and x ∈ T }. S and T .

ˆ The difference, S\T is the set of elements that are in S but not T . S
\T = {x | x ∈ S and x ∈
/ T }.

16
Operation of sets

ˆ The union, S ∪ T of two sets S and T is the set of elements that are
in either S or T or both. S ∪ T = {x | x ∈ S or x ∈ T }. S or T .

ˆ The intersection, S ∩ T is the set of elements that are both in S

and T . S ∩ T = {x | x ∈ S and x ∈ T }. S and T .

ˆ The difference, S\T is the set of elements that are in S but not T . S
\T = {x | x ∈ S and x ∈
/ T }.

ˆ Two sets are disjoint if their intersection is ∅.

16
Sample Space and Events
Sample Space & Event

ˆ Sample space (Ω) of a random variable is a set of all possible

outcomes of the random phenomenon.

ˆ Event is a subset of the sample space. A set consisting of possible

outcomes of the experiment.
Example: Flip 3 coins and record the outcome.
ˆ Sample space:
Ω = {HHH, HHT , HTH, HTT , THH, THT , TTH, TTT }.

ˆ Event:
ˆ {All heads} = {HHH}

ˆ {one head} = {HTT , THT , TTH}

ˆ {at least two heads} = {HHT , HTH, THH, HHH}

17
Intersections, Unions, and Complements

ˆ We are often interested in events that formed from simple basic

events.
ˆ Rolling a dice, get 2 or 3.

ˆ Patient who receives the experimental therapy is relieved and suffers

from no side effects.

ˆ The event that A does not occur is called the complement of A

and denoted as Ac (not A).

ˆ The event that A and B occur is called the intersection and

denoted as A ∩ B (A and B).

ˆ The event that A or B occur is called the union and denoted as

A ∪ B (A or B).

ˆ A and B are mutually exclusive if A ∩ B = ∅. 18

 VennasDiagrams
shown in Fig. 4.8, with the colored portion representing the event.
Venn Diagrams
Complements,
Relationships intersections,
Among unions
Events of events can be represented
Complement, intersection and union of events can be represented visually
visually using Venn diagrams:
Each
by eventdiagram.
the Venn E has a corresponding event defined by the condition that
occur.” That event is called the complement of E, denoted (not E). E
Complement
consists of all outcomes notIntersection Union
in E, as shown in the Venn diagram in Fig. 4
Ec A\B A[B
E 4.9
not E),
A or B)
E A B A B

(not E ) (A & B ) (A or B )

(a) (b) (c)

Essentially, probability of an event E is defined as
With
P(E ) = areaany two events,
of E/area of Ω say, A and B, we can associate two new even
event is defined by the condition that “both event A and event B occur”
7 a
P(not E), P(A and B), P(A or B)
19
Axioms of Probability

ˆ Consider an experiment with sample space Ω. For each event E , we

define the probability of event E as P(E ) = #E
#S . It satisfies the
following 3 axioms.

ˆ 0 ≤ P(E ) ≤ 1

ˆ P(Ω) = 1

20
General Rules of Probability
Disjoint Events (Mutually Exclusive Events)

Disjoint, or mutually exclusive events cannot be happen together

ˆ Toss a coin once, the events {H} and {T } are disjoint.

ˆ Draw a card from a deck, the events {get an ace} and

{get a queen} are disjoint.

ˆ Draw a card from a deck, the events {get an ace} and

{get a diamond} are not disjoint. Because you may have a
diamond ace.

21
neral Addition
General Rule
Addition Rule

What is the probability of drawing a jack or a red card from a well

What’s the probability of drawing a jack or a red card from a well
shuffled fullfull
shuffled deck?
deck?

22
neral Addition Rule
General Addition Rule
What is the probability of drawing a jack or a red card from a well
What’s the probability of drawing a jack or a red card from a well
shuffled full full
shuffled deck?
deck?

4 26 2 28
P(jack
P(jack or red)or= red) = P(jack) + P(red)
P(jack)+P(red)−P(jack P(jack
and red) = and
+ −red)=
52 52 52 52
4 26 2 28
= + = 23
 VennasDiagrams
shown in Fig. 4.8, with the colored portion representing the event.
General Addition Rule
Complements, intersections,
Relationships Amongunions of events can be represented
Events
visually using Venn diagrams:
Each event E has a corresponding event defined by the condition that “E does
occur.” That event is called the complement of E, denoted (not E). Event (not
Complement
consists of all outcomes notIntersection Union
in E, as shown in the Venn diagram in Fig. 4.9(a).
Ec A\B A[B
FIGURE 4.9
r (a) event (not E),
d (c) event (A or B)
E A B A B

(not E ) (A & B ) ( A or B )

(a) (b) (c)

With any two events, say, A and B, we can associate two new events. One n
event is defined
P(A orbyB)the=condition
P(A) + that “both
P(B) − P(A A and
eventand B)event B occur”
7 and is deno

If A and B are mutually exclusive, or disjoint, P(A and B) = 0 since

they can never happen together.

24
 ometric shapes) inside the rectangle. In the simplest case, only one event is display
VennasDiagrams
Inequality shown in Fig. 4.8, with the colored portion representing the event.

Complements, intersections,
Relationships Amongunions of events can be represented
Events
visually using Venn diagrams:
Each event E has a corresponding event defined by the condition that “E does
occur.” That event is called the complement of E, denoted (not E). Event (not
Complement
consists of all outcomes notIntersection Union
in E, as shown in the Venn diagram in Fig. 4.9(a).
Ec A\B A[B
FIGURE 4.9
r (a) event (not E),
d (c) event (A or B)
E A B A B

(not E ) (A & B ) ( A or B )

(a) (b) (c)

With
ˆ If A ⊂ B, any ≤
P(A) twoP(B)
events, say, A and B, we can associate two new events. One n
event is defined by the condition that “both event A and event B occur”
7 and is deno
ˆ P(A ∪ B) ≤ P(A) + P(B)

25
The Complement Rule

ˆ An event must either occur or not occur,

P(A) + P(Ac ) = 1

ˆ If we know probability of an event happening, the probability of its

complement is
P(Ac ) = 1 − P(A)
P(A) = 1 − P(Ac )

ˆ This simple complement rule is valid for all events.

26
Example - The Complement Rule

Question: What’s the probability of getting at least one head in 3 tosses

of a fair coin?

ˆ Sample space

S = {HHH, HHT , HTH, THH, HTT , THT , TTH, TTT }

EventA = {at least one heads}

= {HHH, HHT , HTH, THH, HTT , THT , TTH}

ˆ Ac = {notatleastonehead} = {alltails} = {TTT }.

ˆ P(Ac ) = 18 .

ˆ P(A) = 1 − P(Ac ) = 1 − 1
8 = 87 .

27
Conditional Probability

Example:
A card is drawn from a well-shuffled deck.

ˆ What’s the probability that the card drawn is a King?

4 1
P(King ) = = .
52 13

ˆ If the card drawn is known to be a face card (J, Q, K ), what’s the

probability that it is a King again?
4 1
P(King | {J, Q, K }) = = .
12 3

28
Conditional Probability

Consider two events A and B. We denote the probability of event A

happens given that event B is known to happen as

P(A | B),

read as the probability of “A given B.”

For the example on the previous slide, let

A = {the card is a King}

B = {the card is a face card (J,Q,K)}.

We have
4 4
P(A | B) = 6= P(A) = .
12 52
The numerators are the same, but the denominators change. The
sample space changes given conditional information.

29
Conditional Probability

area of (A ∩ B)
P(A | B) = P(A and B)/P(B) =
area of B
Recall that
P(A) = area of A/area of Ω

30
More examples

ˆ P(B | Ac ) = P(face card | it is not a King)?

If the card is not a King, there are 4 Kings in the deck. Then the
card drawn must be one of the remaining 48 cards, among which 8
are face cards (4 Jacks and 4 Queens). The answer is 8/48.

ˆ P(B | A) = P(face card | it is a King)?

1, since King is a face card.

ˆ P(A | B c ) = P(King card | the card is not a face card)?

0, since King is a face card. If given the card is not a face card, it
can never be a King.

31
More examples

A deck of cards is well-shuffled, two cards are drawn without

replacement. What’s the probability that the second card is a King

ˆ given the first card is a King?

3
51

ˆ given the first card is NOT a King?

4
51

32
Example: Age and Rank of Faculty

This table lists faculty members in a university by age and rank.

Age Professor Associate Assistant Lecturer Total

< 40 54 173 220 23 470
40 - 49 156 125 61 6 348
50 - 59 145 68 36 4 253
> 60 75 15 3 0 93
Total 430 381 320 33 1164

We call this way to present data as contingency table.

33
Example: Age and Rank of Faculty

Select one faculty at random, what’s the probability that he / she is a full
professor?

Age Professor Associate Assistant Lecturer Total

< 40 54 173 220 23 470
40 - 49 156 125 61 6 348
50 - 59 145 68 36 4 253
> 60 75 15 3 0 93
Total 430 381 320 33 1164

430
P(professor) = 1164

Probabilities that involve only one of the categorical variables in a

contingency table are called marginal probabilities.

34
Example: Age and Rank of Faculty

Select one faculty at random, what’s the probability that he / she is

under 40 years old?

Age Professor Associate Assistant Lecturer Total

< 40 54 173 220 23 470
40 - 49 156 125 61 6 348
50 - 59 145 68 36 4 253
> 60 75 15 3 0 93
Total 430 381 320 33 1164

470
P(< 40) = 1164

Again, this is marginal probability, since only one categorical variable is

involved.

35
Example: Age and Rank of Faculty

Select one faculty at random, what’s the probability that he / she is a full
professor and under 40 years old?

Age Professor Associate Assistant Lecturer Total

< 40 54 173 220 23 470
40 - 49 156 125 61 6 348
50 - 59 145 68 36 4 253
> 60 75 15 3 0 93
Total 430 381 320 33 1164

54
P(professor and < 40) = 1164

Probabilities that involve combination of multiple categories in a

contingency table are called joint probability.

36
Example: Age and Rank of Faculty

Given a faculty is under 40 years old, what’s the probability that he / she
is a full professor?

Age Professor Associate Assistant Lecturer Total

< 40 54 173 220 23 470
40 - 49 156 125 61 6 348
50 - 59 145 68 36 4 253
> 60 75 15 3 0 93
Total 430 381 320 33 1164

54
P(professor |< 40) = 470

Calculate conditional probability for a contingency table: note that the

denominator is row / column sum instead of the total sum.

37
Example: Age and Rank of Faculty

Given a faculty is a full professor, what’s the probability that he / she is

40 - 49 years old?

Age Professor Associate Assistant Lecturer Total

< 40 54 173 220 23 470
40 - 49 156 125 61 6 348
50 - 59 145 68 36 4 253
> 60 75 15 3 0 93
Total 430 381 320 33 1164

156
P(40 − 49 | professor) = 430

Calculate conditional probability for a contingency table: note that the

denominator is row / column sum instead of the total sum.

38
Conditional Probability

The conditional probability of the outcome event A given B is

P(A and B)
P(A | B) =
P(B)

Age Prof. Assoc. Assis. Lect. Total

< 40 54 173 220 23 470
40 - 49 156 125 61 6 348
50 - 59 145 68 36 4 253
> 60 75 15 3 0 93
Total 430 381 320 33 1164

P(40 − 49 and professor) 156/1164 156

P(40 − 49 | professor) = = =
P(professor) 430/1164 430

39
Joint and Marginal Probability

Sometimes we have a table of joint probability rather than counts.

Age Prof. Assoc. Assis. Lect. Total

< 40 0.046 0.149 0.189 0.020
40 - 49 0.134 0.107 0.052 0.005
50 - 59 0.125 0.058 0.031 0.003
> 60 0.064 0.013 0.003 0.000
Total

40
Joint and Marginal Probability

Fill in the marginal probability first, which is row / column sum of joint
probability (integrate out one variable).

Age Prof. Assoc. Assis. Lect. Total

< 40 0.046 0.149 0.189 0.020 0.404
40 - 49 0.134 0.107 0.052 0.005 0.299
50 - 59 0.125 0.058 0.031 0.003 0.217
> 60 0.064 0.013 0.003 0.000 0.080
Total 0.369 0.327 0.275 0.028 1.000

P(40 − 49 and professor) 0.134

P(40 − 49 | professor) = =
P(professor) 0.369

41
General Multiplication Rule

The definition of conditional probability

P(A and B)
P(A | B) =
P(B)

can be used the other way around, which is the general multiplication
rule.
P(A and B) = P(A | B) × P(B)
When you want to calculate P(A and B), but the two terms on the right
side are easier to calculate, use this rule.

42
General Multiplication Rule

Example: General Multiplication Rule A deck of cards is shuffled and the

two top cards are placed face down on a table. What is the probability
that both cards are Kings?
Let

ˆ A = 1st card is a King.

ˆ B = 2nd card is a King.

ˆ P(A) = P(1st card is a King) = 4

52 .
ˆ Given that the first is a King, the conditional probability
3
P(B | A) = 51 . (why 3 and 51?)
ˆ So
4 3
P(A and B) = P(A | B) × P(B) = × .
52 51

43
General Multiplication Rule

A deck of cards is shuffled and the two top cards are placed face down on
a table. What is the probability that neither card is a K?
Let

ˆ A = 1st card is NOT a King.

ˆ B = 2nd card is NOT a King.

ˆ P(A) = P(1st card is a King) = 48

52 .
ˆ Given that the first is a King, the conditional probability
P(B | A) = 47
51 .
ˆ So
48 47
P(A and B) = P(A | B) × P(B) = × = 0.851.
52 51
ˆ P(at least a K) = 1 − P(neither is a K) = 1 − 0.851 = 0.149.

44
General Multiplication Rule for Multiple Events

P(ABC ) = P(A) × P(B | A) × P(C | A, B)

P(ABCD) = P(A) × P(B | A) × P(C | A, B) × P(D | A, B, C )
= P(B) × P(A | B) × P(D | A, B) × P(C | A, B, D)
and so on.
You can pick different sequence of events to take conditions.

45
General Multiplication Rule for Multiple Events

Five cards are dealt from a deck of well-shuffled card. What is the chance
that none of them are hearts ♥?
Let Ai be the event that the i-th card dealt is not a ♥.
ˆ P(A1 ) = P(1st card is not a♥) = 39
52 .

ˆ Given that the 1st card is not a ♥, the conditional probability that
38
the 2nd is not a ♥= P(A2 | A1 ) = 51 .

ˆ Given neither of the first two cards is a ♥, the condition probability

that the 3rd is not a ♥= P(A3 | A1 , A2 ) = 3750 .

ˆ Likewise, P(A4 | A1 , A2 , A3 ) = 36
49 , P(A5 | A1 , A2 , A3 , A4 ) = 35
48 .

ˆ By the General Multiplication Rule,

39 38 37 36 35
P(A1 , · · · , A5 ) = × × × × ≈ 0.222 46
52 51 50 49 48
Continue the previous slide, what is the probability of getting at least one
heart ♥among the five cards?
Since {at least one ♥}c = {no ♥}, by the complement rule,

P(at least one ♥) = 1 − P(no ♥) = 1 − 0.222 = 0.778

Keep in mind

ˆ The complement of {at least one is...} is {none is...}.

ˆ The complement of {all are...} is {at least one is not...}.

47
Independence
Independence

Two random processes (random variables, events) are independent if

knowing the outcome of one provides no useful information about the
outcome of the other.

ˆ Knowing that the coin landed on a head on the first toss does not
provide any useful information for determining what the coin will
land on in the second toss.
Outcomes of two tosses of a coin are independent.
ˆ Knowing that the first card drawn from a deck is an ace does
provide useful information for determining the probability of drawing
an ace in the second draw.
Outcomes of two draws from a deck of cards (w/o replacement) are
dependent.

48
Independence and Conditional Probabilities

In mathematical notation, if P(A | B) = P(A) then the events A and B

are said to be independent.

ˆ Conceptually: Giving B doesn’t tell us anything about A.

Equivalently, one can also check the independency of the events A and B
by check whether P(B | A) = P(B)

49
Practice – Checking for Independence

Between January 9-12, 2013, SurveyUSA interviewed a random sample of

500 NC residents asking them whether they think widespread gun
ownership protects law abiding citizens from crime, or makes society
more dangerous. 58% of all respondents said it protects citizens. 67% of
White respondents, 28% of Black respondents, and 64% of Hispanic
respondents shared this view. Are opinion on gun ownership and race
ethnicity independent?

P(protects citizens) = 0.58

P(protects citizens | White) = 0.67
P(protects citizens | Black) = 0.28
P(protects citizens | Hispanic = 0.64)
Probability of protects citizens varies given different races, therefore the
opinion on gun ownership and race are dependent.

50
Independence of events vs. independence of variables in a con-
tigency table

Recall that for a two-way contingency table, the two variables are
independent if the row proportions do not change from row to row.
This is consistent with the definition of independence of events since

row proportions = P(column variable | row variable) (1)

and if the row proportions do not change from row to row, they will be
equal to the marginal probability of the column variable.
For example of the faculty, P(rank | age) 6= P(rank), the age and rank of
faculty members are dependent.

51
Multiplication rule for independent events

When A and B are independent

P(A and B) = P(A) × P(B)

ˆ This is simply the general multiplication rule

P(A and B) = P(A) × P(B | A)
where P(B | A) = P(B) if A and B are independent.
ˆ More generally,
P(A1 and A2 · · · and Ak ) = P(A1 ) × · · · × P(Ak )
If A1 , · · · , Ak are independent.
Exercies: If you roll a die twice, what’s the probability of getting two 1 in
a row?
1 1 1
P(1 in the first roll) × P(1 in the second roll) = × =
6 6 36
52
Practice

A recent Gallup poll suggests that 25.5% of Texans do not have health
insurance as of June 2012. Assuming that the uninsured rate stayed
constant, what is the probability that two randomly selected Texans are
both uninsured?
0.2552

53
Put everything together

If we were to randomly select 5 Texans, what is the probability that at

least one is uninsured?
By the complement rule

P( at least 1 uninsured) = 1 − P(none uninsured)

= 1 − P(everyone is insured)
= 1 − [(1 − 0.255)]5
= 1 − 0.7455 = 0.77

Keep in mind that

P(at least one) = 1 − P(none)

54
Abuse of the multiplication rule

As estimated in 2012, of the U.S. populartion

ˆ 13.4% were 65 or older

ˆ 52% were male

True or false? 0.134 × 0.52 = 0.07 of the U.S. populartion were males
age 65 or older.
False, age and gender are dependent. In particular, women on average
live longer than men. There are more old women than old men.
Among the age 65 or older, only 44% are male (this is the condition
probability!), not 52%. So only 0.134 × 0.44 = 0.059 were males age 65
or older.

55
Tree diagrams and Bayes’
Theorem
Example – Nervous Job Applicant

Suppose an applicant for a job has been invited for an interview.

The probability that

ˆ he is nervous is P(N) = 0.7

ˆ the interview is successful given that he is nervous is P(S | N) = 0.2
ˆ the interview is successful given that he is not nervous is
P(S | N C ) = 0.9

What’s the probability that the interview is successful?

P(S) = P(S and N) + P(S and N C )

= P(N)P(S | N) + P(N C )P(S | N C )
= 0.7 × 0.2 + 0.3 × 0.9 = 0.41

56
Tree diagram for the nervous job applicant

Another look at the nervous job applicant example

eed P(N and S) = 0.7 × 0.2

Succ
0.2
Nervous Fail
0.7 0.8 P(N and S C ) = 0.7 × 0.8

·
0.3
eed P(N C and S) = 0.3 × 0.9
Succ
Not 0.9
nervous Fail
0.1 P(N C and S C ) = 0.3 × 0.1

57
Nervous job applicant

Conversely, given the interview is successful, what’s the probability that

the job applicant is nervous during the interview?

P(N and S) P(N and S)

P(N | S) = =
P(S) 0.41
P(N)P(S | N) 0.7 × 0.2
= = ≈ 0.34
0.41 0.41
in which P(S) = 0.41 was found in the previous slide.

58
Bayes’ Theorem

The problem in the previous slide uses Bayes’ Theorem. Knowing

P(B | A), P(B | AC ), and P(A), how do you know P(A | B)? Notice
that you switch the position of event B and condition A.

P(A and B) P(A)P(B | A)

P(A | B) = =
P(B) P(B)
P(A)P(B | A)
=
P(B and A) + P(B and AC )
P(A)P(B | A)
=
P(A)P(B | A) + P(AC )P(B | AC )

59
Medical Testing

A common application of Bayes’ Theorem is in diagnostic testing

ˆ Let D denote the event that an individual has the disease.

ˆ Let T + denote the event that the test is positive.
ˆ Let T − denote the event that the test is negative.
ˆ P(T + | D) is called sensitivity of the test.
ˆ P(T − | D C ) is called specificity of the test.
ˆ Ideally, both P(T + | D) and P(T − | D C ) would equal to 1.
However, the test may give false positives or false negatives.

60
Enzyme Immunoassay test for HIV

ˆ P(T + | D) = 0.98, positive for infected.

ˆ P(T − | D C ) = 0.995, negative for non-infected
ˆ P(D) = 1/300

What’s the probabilitythat the tested person is infected if the test was
positive?

P(D)P(T + | D)
P(D | T +) =
P(D)P(T + | D) + P(D C )P(T + | D C )
1/300 × 0.98
=
1/300 × 0.98 + 299/300 × 0.005
= 0.394

The test is not confirmatory. Need to confirm by a second test.

61
Tree diagram for HIV test

Posit
ive P (D and T +) = 3001 × 0.98
0.2
Infected Nega
1
tive
0 0.99
30 P(D and T −) = 1
× 0.02
5 300

· 29
30 9
0
ive P(D C and T +) = 299
× 0.005
Posit 300
Not .005
Ne0ga
infected tive
0.99 P(D C and T −) = 299
× 0.995
5 300

Only the red cases are positive test results, out of them, the bold one is
D, have disease.
bold 1/300 × 0.98
P(D | T +) = =
red1 + red2 1/300 × 0.98 + 299/300 × 0.005

62