TOPIC 6

Probability
 Definition of Probability
• Experiment: Process that generates observation.
– toss a coin twice
• Sample space: possible outcomes of an experiment
What could happen in the experiment
– S = {HH, HT, TH, TT}
• Event: a subset of possible outcomes
What we want in an experiment?
– An event could be that you got at least one head on the
two tosses , A={HH, HT, TH}
 Sample Space

The Sample Space is the collection of all possible

events
e.g. All 6 faces of a die:

e.g. All 52 cards of a bridge deck:

 Examples

Rolling a die – outcomes all possible events

S ={ , , , , , }
={1, 2, 3, 4, 5, 6}

E = the event that an even number is

rolled
= {2, 4, 6}
={ , , }
 Probability
• One way to describe the likelihood of an event occurring is
with probability.
• The probability of a simple event, like a coin landing heads
up when it is tossed, is a ratio of the number of favorable
outcomes for the event to the total number of possible
outcomes of the event.
• The probability of an event A can be expressed as:
number of favorable outcomes
PA 
total number of possible outcomes
• The probability of event A [denoted P(A)], must lie within the
interval from 0 to 1: 0 < P(A) < 1
0 < P(A) < 1

If P(A) = 0, then the If P(A) = 1, then the event

event cannot occur. is certain to occur.
 Probability
Example:

Throughout her teaching career, Professor Lim has awarded

186 A’s out of 1,200 students. What is the probability that a
student in her section this semester will receive an A?

number of outcomes in A
P (A) = total number of outcomes

186
P ( A)  0.155
1200
You can express a probability as a fraction, a decimal, or a percent.
For example: 186 , 0.155, or 15.5%.
1200
 Question:
Find the probability of rolling a number greater than 2 on a die.

There are six possible outcomes. Four of the outcomes are favorable. That is, four of the six outcomes
are numbers greater than two.

4 numbers
greater than 2
Sample space: 1, 2, 3, 4, 5, 6 4
6
6 possible
outcomes
4 2
P(  2)   or 0.6667
6 3
2 2
P(  2) 1  P( 2) 1   or 0.6667
6 3
 Question:
You toss two dice. What is the probability of the
outcomes summing to five?

This is S:

{(1,1), (1,2),
(1,3), ……etc.}

There are 36 possible outcomes in S, all equally likely (given fair

dice). Thus, the probability of any one of them is 1/36.
P(the roll of two dice sums to 5) = P(1,4) + P(2,3) + P(3,2) + P(4,1)
 Question:
You toss two dice. What is the probability of rolling
doubles?

This is S:

{(1,1), (1,2),
(1,3), ……etc.}

P(doubles) = P(1,1) + P(2,2) + P(3,3) + P(4,4) + P(5,5) + P(6,6)

= 6 * 1/36 = 1/6 = 0.1667
 Special Events

The Null Event, The empty event - 

 = { } = the event that contains no outcomes

The Entire Event, The Sample Space - S
S = the event that contains all outcomes
The empty event, , never occurs.
The entire event, S, always occurs.
 Events
Simple event
◦ An outcome from a sample space with one
characteristic
◦ e.g., A red card from a deck of cards
Complement of an event A (denoted A’)
◦ All outcomes that are not part of event A
◦ e.g., All cards that are not diamonds
Joint event
◦ Involves two or more characteristics simultaneously
◦ e.g., An ace that is also red from a deck of cards
 Visualizing Events

• Contingency Tables
Ace Not Ace Total
Black 2 24 26
Red 2 24 26
Total 4 48 52
Sample
• Tree Diagrams
Sample
A ce 2 Space

Space a c k C ard
Full Deck Bl N o t an A c e 24
of 52 Cards A ce
R ed Ca 2
rd
N o t an
Ac e 24
 Mutually Exclusive Events

• Mutually exclusive events

– Events that cannot occur together

example:

A = queen of diamonds (Q ♦) ;
B = queen of clubs (Q ♣)

– Events A and B are mutually exclusive

Collectively Exhaustive Events
• Collectively exhaustive events
– One of the events must occur
– The set of events covers the entire sample
space

Example: A = Aces; B = Black cards;

C = Diamonds ♦; D = Hearts ♥

– Events A, B, C and D are collectively

exhaustive (but not mutually exclusive – an
ace may also be a heart)
– Events B, C and D are collectively exhaustive
and also mutually exclusive
 Venn Diagram

A’ A B A B
A

Shaded region is A’ Shaded region is A  B Shaded region is A  B

A B A A
B B

AB0 Mutually exclusive Mutually exclusive

AB=0 AB
Complements, Intersections,
&Unions

• Suppose A & B are events.

 Complement
The complement of A is everything in the sample
space S that is NOT in A.

•If the rectangular box

S is S, and the white
circle is A, then
A everything in the box
that’s outside the circle
is A’, which is the
complement of A.

Pr (A’) = 1 - Pr (A)
 Example:
• If A is the event that a randomly selected
student is male, and the probability of A is 0.6,
what is A’ and what is its probability?

• A’is the event that a randomly selected student

is female, and its probability is 0.4.

Pr (A’) = 1 - Pr (A)
= 1 - 0.6
= 0.4
 Intersections: “Both/And”
•The intersection of A & B (denoted A∩B) is
everything in the sample space that is in both A & B.

S  may be read as
“and” since both
A AB B events occur.
This is a joint
probability.

•The intersection of A & B is the pink overlapping

area.
 Unions :“Either/Or”
• The union of A & B (denoted A  B or “A or B”)
is everything in the sample space that is in either A
or B or both.

S  may be read as
“or” since one or
A B the other or both
events may occur.

•The union of A & B is the whole white area.

P(A  B) = P(A) + P(B) – P(A  B)
 Mutual Exclusion
If two events A and B cannot happen simultaneously,
i.e.,
A  B = , we say A and B are mutually exclusive
events.
For mutually exclusive events,
P(A  B) = P(A) + P(B)
Eg. Indian 
Malay
S - You can not be
an Indian at the
A
same time Malay
B (when you fill in
an application
form)
 Example

P(Red or Ace) = P(Red) +P(Ace) - P(Red and Ace)

= 26/52 + 4/52 - 2/52 = 28/52
Don’t
Color count the
Type Red Black Total two red
aces
Ace 2 2 4 twice!
Non-Ace 24 24 48
Total 26 26 52
 Question:
• For the card example:

P(Q) = 4/52 (4 queens in a deck)

P(R) = 26/52 (26 red cards in a deck)
P(Q  R) = 2/52 (2 red queens in a deck)
P(Q  R) = ?

P(Q  R) = P(Q) + P(R) – P(Q  R)

Q and R = 2/52 = 4/52 + 26/52 – 2/52

= 28/52
Q R = .5385
4/52 26/52
 Three Kinds of Probabilities
• Simple or marginal probability
 The probability that a single given event will occur.
The typical expression is P(A).
• Joint or compound probability
 The probability that two or more events occur. The
typical expression is P(A and B).
• Conditional probability
 The probability that an event, A, occurs given that
another event, B, has already happened. The typical
expression is P(A|B).
 Computing Marginal and Joint
Probabilities
• Computing a marginal (or simple)
P(A) P(A
probability:and B1 )  P(A and B 2 )    P(A and B k )

• Where B1, B2, …, Bk are k mutually exclusive and

collectively exhaustive events

• The probability of a of
number joint event,satisfying
outcomes A and AB:and B
P( A and B) 
total number of elementary outcomes
Marginal Probability Example

P(Ace) P(Ace and Re d )  P(Ace and Black)

2 2 4
  
52 52 52

Color
Type Red Black Total

Ace 2 2 4
Non-Ace 24 24 48
Total 26 26 52
Joint Probability Example

P(Red and Ace)

number of cards that are red and ace 2
 
total number of cards
52

Color
Type Red Black Total
Ace 2 2 4
Non-Ace 24 24 48
Total 26 26 52
 Joint Probabilities
Using Contingency Table

Event
Event B1 B2 Total

A1 P(A1 and B1) P(A1 and B2) P(A1)

A2 P(A2 and B1) P(A2 and B2) P(A2)

Total P(B1) P(B2) 1

Joint Probabilities Marginal (Simple) Probabilities

 Conditional Probability
• A conditional probability is the probability of one
event, given that another event has occurred:
• The pipe, | , will indicate a conditional probability.
P(A|B) is read, the probability of A given B.
• Which is to say the probability that A is true, given
that we already know B is true.

P(A  B)
A P(A | B) 
B
P(B)
if P  B  0
 Conditional Probability

We define conditional probability of A given

B, as
P(A  B)
P(A | B)  Assuming P(B) > 0.
P(B)

Similarly, the conditional probability of B

given A is define as
P(A  B)
P(B | A)  Assuming P(A) > 0.
P(A)
Conditional Probability Example

 Of the cars on a used car lot, 70% have air

conditioning (AC) and 40% have a CD player (CD).
20% of the cars have both.

What is the probability that a car has a CD

player, given that it has AC ?

i.e., we want to find P(CD | AC)

P(AC) 0.7
P(CD) 0.4
P(CD and AC) 0.2
Conditional Probability Example
(continued)

 Of the cars on a used car lot, 70% have air conditioning (AC)
and 40% have a CD player (CD). 20% of the cars have both.

CD No CD Total
AC .2 .5 .7
No AC .2 .1 .3
Total .4 .6 1.0

P(CD and AC) .2

P(CD | AC)   .2857
P(AC) .7
Conditional Probability Example
(continued)

 Given AC, we only consider the top row (70% of the cars).
Of these, 20% have a CD player.20% of 70% is about
28.57%.

CD No CD Total
AC .2 .5 .7
No AC .2 .1 .3
Total .4 .6 1.0

P(CD and AC) .2

P(CD | AC)   .2857
P(AC) .7
 Using Decision Trees
.2
P(CD | AC)
Given AC or no AC: .7 P(AC and CD) = .2
a s C D
H

)= .7
(A C Does
P n
have ot .5 P(AC and CD’) = .5
A C CD
Ha s P(CD' | AC) .7
All
Cars
Doe .2
hav s not P(CD | AC' )
eA .3 P(AC’ and CD) = .2
C P(A a s C D
C ’)= H
.3
Does
n
have ot .1 P(AC’ and CD’) = .1
CD
P(CD' | AC' ).3
 Using Decision Trees
(continued)

.2
P(AC | CD)
Given CD or no CD:
.4 P(CD and AC) = .2
a s A C
H

)= .4
( CD Does
P n
have ot .2 P(CD and AC’) = .2
C D AC
Ha s P(AC | CD' ) .4
All
Cars
Doe .5
hav s not P(AC' | CD)
e CD .6 P(CD’ and AC) = .5
P(C a s A C
D ’ )= H
.6
Does
n
have ot .1 P(CD’ and AC’) = .1
AC
P(AC' | CD' ).6
 Question:
The academy awards is soon to be shown.
For a specific married couple the probability
that the husband watches the show is 80%,
the probability that his wife watches the show
is 65%, while the probability that they both
watch the show is 60%.
If the husband is watching the show, what is
the probability that his wife is also watching
the show
P[H]= 0.80
P(W | H) ?
P[W]= 0.65
P[W ∩ H]= 0.60
 Solution:
The academy awards is soon to be shown.
Let H = the event that the husband watches the show
P[H]= 0.80
Let W = the event that his wife watches the show
P[W]= 0.65 and P[W ∩ H]= 0.60

P(W  H)
P(W | H) 
P(H)
0.6
P(W | H)  0.75
0.8
 Independent
• Event A and B are independent when the
probability of one event is not affected by the
other event.
• If P(A|B) = P(A) , then we say A is independent of
B.
• Equivalently,
P(A  B) = P(A) P(B), if A and B are independent.
Note
P(A  B)
P(A | B) 
P(B)
P(A  B)
If P(A|B) = P(A) P(A) 
P(B)
P(A  B) P(A)P(B)
 When Events Are Dependent
• Events are dependent when the
occurrence of one event does change
the probability that another event will
occur.
 If A and B are dependent, P(A|B)  P(A)
because the occurrence of event B does
change the probability that A will occur.
 If A and B are dependent, then
P(A and B) = P(A|B) • P(B)
P(A  B)
P(A | B) 
P(B)
P(A  B) P(A | B)P(B)
 Independent and Dependent
Example : Classifying Events as Independent or
Dependent
Determine if the events are dependent or independent.
A.getting head on a coin toss and rolling a 6 on a die

Tossing a coin does not affect rolling a die, so the two

events are independent.

B. getting 2 red gumballs out of a gumball machine

After getting one red gumball out of a gumball machine,
the chances for getting the second red gumball have
changed, so the two events are dependent.
 Some Formulas Revisited
The probability P(A) of any event A is between 0 and 1
0 ≤P(A) ≤ 1
 Probability of all event other than an event A is denoted
by P(A)  P(A) = 1 – P(A)
P(A or B) = P(A) + P(B) – P(A and B)
If A and B are mutually exclusive, then P(A and B) is
zero.
Thus P(A or B) = P(A) + P(B) .
P(A and B) = P(A) P(B|A).
If A and B are independent, then P(B|A) is the same as
P(B).
 Thus P(A and B) = P(A) P(B).
 Question: Independence
 Of the cars on a used car lot, 70% have air conditioning (AC)
and 40% have a CD player (CD). 20% of the cars have both.

CD No CD Total
P(CD | AC)=?
AC .2 .5 .7
No AC .2 .1 .3
Total .4 .6 1.0
• Are events AC and CD statistically independent?
P(AC ∩ CD) = 0.2
P(AC) = 0.7 P(AC) x P(CD) = (0.7)(0.4) = 0.28
P(CD) = 0.4
P(AC ∩ CD) = 0.2 ≠ P(AC)P(CD) = 0.28
So the two events are not statistically independent
Bayes’ Theorem for the
Revision of Probability
• In the 1700s, Thomas Bayes
developed a way to revise the
probability that a first event occurred
from information obtained from a
second event.
• Bayes’
P(A|BTheorem:
) P(A and B)For two events A
and B P(B)
 P(A)P(B|A)
[P(A)P(B|A)]  [P(A')P(B|A')]
 Bayes’ Theorem

P(A | Bi )P(B i )
P(Bi | A) 
P(A | B1 )P(B1 )  P(A | B 2 )P(B 2 )    P(A | Bk )P(B k )

• where:
Bi = ith event of k mutually exclusive and collectively
exhaustive events
A = new event that might impact P(Bi)
 Bayes’ Theorem Example
• An automobile company has two factories to
produce its cars. 60% of the production is done
by factory X and the rest is done by factory Y.

• Of which, 85% of the production at factory X

and 70% of the production at factory Y is up to
standard specifications.

• If the car is up to standard specification, what is

the probability that it was done by factory X?
 Bayes’ Theorem Example
(continued)
Let X = done by factory X
Y = done by factory Y
P(X) = .6 , P(Y) = .4 (prior probabilities)
Define up to standard specification as S
Conditional probabilities:
• P(S|X) = .85 P(S|Y) = .7
Goal is to find P(X|S)
 Bayes’ Theorem Example
(continued)

Apply Bayes’ Theorem:

P(X) P(S|X)
P(X|S) 
[P(X) P(S|X)]  [P(Y) P(S|Y)]
(.6)(.85)

(.6)(.85)  (.4)(.7)
.51
 .6456
.51  .28

If the car is up to standard specification, the probability

that it was done by factory X is 0.6456
 Bayes’ Theorem Example
(continued)

• Given the up to standard specification, the revised

probability of done by factory X has risen to .6456
from the original estimate of .6
P(X) P(S|X)

Prior Conditional Prob. Joint Revised

Event Up to standard
Prob. Prob. Prob.
specification
X .6 .85 .6*.85 = .51 .51/.79 = .6456
Y .4 .7 .4*.7 = .28 .28/.79= .3544

Sum = .79
P( X | S ) P( X and S ) P(S)
P(S )
Permutations vs. Combinations

Many counting problems can be easily solved if

we can identify them as
permutation or combination problems.
 Permutations vs. Combinations
What's the Difference?
In English we use the word "combination" loosely, without
thinking if the order of things is important.

In other words:
"My fruit salad is a combination of apples, grapes and
bananas" We don't care what order the fruits are in, they
could also be "bananas, grapes and apples" or "grapes, apples
and bananas", it is the same fruit salad.

"The combination to the safe is 472". Now we do care

about the order. "724" won't work, nor will "247". It has to be
exactly 4-7-2.
So, in Mathematics we use more precise language:
If the order doesn't matter, it is a Combination.
If the order does matter it is a Permutation. 51
 Permutations vs. Combinations
A permutation of r objects selected from a set of n objects
is an arrangement of r of the n objects in a specific order.
n n!
P(n, r) Pr 
(n  r )!
A combination of r objects selected from a set of n
objects is a selection of r of the n objects with order
disregarded.
n n!
C ( n, r )  Cr  r!(n  r )!

The factorial function (symbol: ! ) means to multiply a series of

descending natural numbers. Eg. 4! = 4 × 3 × 2 × 1 = 24
Where n is the number of things to Choose from, and we choose r of
them.
 Permutations vs. Combinations
• Both are ways to count the possibilities
• The difference between them is whether order matters
or not
• Consider a poker hand:
– A♦, 5♥, 7♣, 10♠, K♠
• Is that the same hand as:
– K♠, 10♠, 7♣, 5♥, A♦
• Does the order the cards are handed out matter?
– If yes, then we are dealing with permutations
– If no, then we are dealing with combinations

53
 Permutations
A permutation is an ordered arrangement of the
elements of some set S
◦ Let S = {a, b, c}
◦ c, b, a is a permutation of S
◦ b, c, a is a different permutation of S

An r-permutation is an ordered arrangement of r

elements of the set
◦ A♦, 5♥, 7♣, 10♠, K♠ is a 5-permutation of the set of cards
The notation for the number of r-permutations:
P(n,r)
◦ The poker hand is one of P(52,5) permutations
Number of poker hands (5 cards):
P(52,5) = 52*51*50*49*48 = 311,875,200
54
 Combinations
• What if order doesn’t matter?
• In poker, the following two hands are equivalent:
– A♦, 5♥, 7♣, 10♠, K♠
– K♠, 10♠, 7♣, 5♥, A♦

How many different poker hands are there (5 cards)?

52! 52! 52 * 51* 50 * 49 * 48 * 47!

C (52,5)    2,598,960
5!(52  5)! 5!47! 5 * 4 * 3 * 2 *1* 47!

55
55
 Two Problems Illustrating
Combinations and Permutations
 Problem 1: Consider the set {p, e, n}. How
many two-letter “words” (including nonsense
words) can be formed from the members of this
set, if two different letters have to be used?
 Solution: We will list all possibilities: pe, pn, en, ep,
3!
np, ne, a total of 6. 3
P2  6
(3  2)!
 Problem 2: Now consider the set consisting of
three males: {Peter, Eric, Nick}. For simplicity,
denote the set by {p, e, n}. How many two-man
crews can be selected from this set?
 Solution: pe (Peter, Eric), pn (Peter, Nick) and en
(Eric, Nick), and that is all! 3 3!
C2  3
1!2!
 Difference Between
Permutations and Combinations
Both problems involved counting the numbers of arrangements
of the same set {p, e, n}, taken 2 elements at a time, without
allowing repetition.
However, in the first problem, the order of the arrangements
mattered since pe and ep are two different “words”. In the
second problem, the order did not matter since pe and ep
represented the same two-man crew. We counted this only
once.
The first example was concerned with counting the number of
permutations of 3 objects taken 2 at a time.
The second example was concerned with the number of
combinations of 3 objects taken 2 at a time.
 Question:
The joint probabilities of achieving grades of A and not achieving
A’s in two quantitative techniques courses.

Grade of A in Does not achieve

QT1 grade of A in
QT1
Grade of A in 0.08 0.15
QT2
Does not achieve 0.45 0.32
grade of A in QT2
(i) What’s the probability that a student achieves a grade of A in QT
1?
(ii) What’s the probability that a student achieves a grade of A in QT
1,given that he/she does not achieve a grade of A in QT2 ?

(iii) Are achieving grades of A in QT 1 and QT 2 independent event ?

 Question:
The joint probabilities of achieving grades of A and not achieving A’s in
two quantitative techniques courses.
Event
Event AQT1 A’QT1 Total

AQT2 0.08 0.15 0.23

A’QT2 0.45 0.32 0.77
Total 0.53 0.47 1
(i) What’s the probability that a student achieves a grade of A in QT 1 ?P ( AQT 1 ) 0.53

(ii) What’s the probability that a student achieves a grade of A in QT 1,given that
he/she does not achieve a grade of A in QT2 ?
P ( AQT 1  A'QT 2 ) 0.45
P( AQT 1 | A'QT 2 )   0.5844
P ( A'QT 2 ) 0.45  0.32
(iii) Are achieving grades of A in QT 1 and QT 2 independent event ?
P(A and B)=0.08. P(A)=0.53. P(B)=0.23 P(A)P(B) ≠ P(A and B)
End of Chapter 6