ELEC2600 Tutorial 2

Counting Method
Conditional Probability
Total Probability Theorem
Bayes’ Rule
 Computing Probability using Counting Methods
2

 If the outcomes of an experiment are equiprobable

number of outcomes in A
P[ A] =
number of outcomes in sample space

 P[A] is a measure of the size of the set A

 How to determine the size of A and the size of the sample space?
e.g. How many ways are there to put 3 coins in 4 pockets?
 Can more than one coins be put into one pockets?
 Will the order of the pockets being considered?
 Computing Probability using Counting Methods
3
 With replacement with ordering
 With replacement With ordering
Select an object Note its ≠

Put it back
identity
1 2 ≠1 2
n objects

 e.g. a child has 3 coins and 4 labeled pockets, more than 1 coins
can be put in the same pockets. In how many ways can he put the
coins in his pockets?

1 2 3 4
 Choose k objects from a set A that has n members with
replacement with ordering number of outcomes = nk
 Computing Probability using Counting Methods
4
 Without replacement with ordering
 Without replacement With ordering
Select an object Note its ≠
identity
1 2 ≠1 2
n objects

 e.g. a child has 3 coins and 4 labeled pockets, only1 coins can be
put in one pocket. In how many ways can he put the coins in his
pockets?

1 2 3 4
 Choose k objects from a set A that has n members without
replacement with ordering number of outcomes= n ( n − 1)( n − 2 )( n − k + 1)
k terms
 Computing Probability using Counting Methods
5
 Without replacement without ordering
 Without replacement Without ordering
Select an object Note its =
identity
1 2 =1 2
n objects

 e.g. a child has 3 coins and 4 pockets, only1 coins can be put in
one pocket and the order of the pockets are not considered. In
how many ways can he put the coins in his pockets?

1 2 3 4
 Computing Probability using Counting Methods
6

 Without replacement without ordering

 Without ordering: does not note the identity of the objects in an
ordered list.
 Choose k object from a set A that contains n objects without
replacement and without ordering
# of ways to choose k object from n
objects without replacement with
ordering
n ( n − 1)( n − 2 ) ( n − k + 1) n!
=
number of outcomes C = n

k ( k − 1)( k − 2 ) (2)(1) ( n − k )!k !

k

k terms in numerator and denominator

# of ways to arrange the selected k objects
 Computing Probability using Counting Methods –
Example 1
7
Suppose a password has 6 characters: 3 letters (chosen from A – Z) followed
by 3 numbers (chosen from 0 – 9). What’s the probability that the password
contains no duplicated letters or numbers?
Solution
1. What’s the size of sample space?
a) How many ways are there to choose 3 letters from 26 letters? 263
b)How many ways are there to choose 3 numbers from 10 numbers? 103
c) Size of sample space is: 263× 103
2. How many ways are there to choose 3 letters from 26 letters without
duplication? 26 ×25×24
3. How many ways are there to choose 3 numbers from 10 numbers without
duplication? 10 ×9×8
26 × 25 × 24 ×10 × 9 × 8
P = 0.64
26 ×10
3 3
 Computing Probability using Counting Methods –
Example 2
8
A store sells a batch of 60 iPods, 10 of which are defective. Suppose
that you purchase 2 iPods from this store for your FYP project. What
is the probability that:
(1) None of the iPods that you buy are defective.
(2) Only one of the iPods that you buy is defective.
(3) At least 1 of the iPods that you buy is defective.
Solution
How many ways are there to choose 2 iPods from 60 iPods (find the
 60 
size of sample space)?  
2 
(1) How many ways are there to choose 2 iPods from 50 non-defective
iPods and 0 iPods from 10 defective iPods?  50  10 
 50  10   2 0 
  
] =
2 0 
=
P[none of iPods are defective 0.6921
 60 
 
2 
 Computing Probability using Counting Methods –
Example 2
9

(2) How many ways are there to choose 1 iPod from 50 non-defective
iPods and 1 iPod from 10 defective iPods?  50  10 
1  1 
 50  10 
  
=
P[only one iPod is defective  1  1 
] = 0.2825
 60 
 
2 
(3) The 2 iPods can be both defective or only 1 is defective
Computing Probability using Counting Methods – Example 3

Played with pack of 52 cards with 4 suits

13 hearts ♥ : A 2 3 4 5 6 7 8 9 10 J Q K
13 diamonds ♦ : A 2 3 4 5 6 7 8 9 10 J Q K
13 clubs: ♣ : A 2 3 4 5 6 7 8 9 10 J Q K
13 spades: ♠ : A 2 3 4 5 6 7 8 9 10 J Q K

What is the
probability of
being dealt each
type of poker
hand?

10
 Poker Probability – Solution
11

 The sample size is the number of ways to choose 5 cards from 52

cards  52  = 2,598,960
5 

1. Royal Flush: five cards in sequence, ace is the highest card, all five
cards are from the same suit (i.e. 10-J-Q-K-A in any of the four suits)
There is only one royal flush in each suit
The number of ways to choose 1 suit from 4
suits is   = 4
4
 4
1   
=
P[ Royal Flush ]  1
= 1.54 ×10−6
 52 
 
5 
 Poker Probability – Solution
12

2. Straight Flush: 5 cards are from the same suit, from A-2-3-4-5 up to
10-J-Q-K-A

From A to 10, there are 10 groups of 5 cards in sequence

The number of ways to choose 5 cards in sequence from 10 groups is
10  10   4 
    
1   1  1 
=
P[ Straight Flush ] = 1.54 ×10−5
 52 
 
5 
 Poker Probability – Solution
13

3. Four of a kind: contains all four cards of one rank and any other
card (e.g. 9♣ 9♠ 9♦ 9♥ J ♦)

The 4 cards can be any one of the thirteen ranks from 4 suits
13  4
The number of ways to choose 4 cards is    
1   4 
The number of ways to choose the last card is12   4 
  
1  1 
13   4  12   4 
    
[ Four of a kind ] =
1   4  1  1 
P= 0.00024
 52 
 
5 
 Poker Probability – Solution
14
4. Full House: 3 matching cards from one rank and 2 matching cards
from another rank (e.g. 8♣ 8 ♥ 8♦ K ♥ K ♠)

The triple can be any one of the thirteen ranks

The number of ways to choose the triple is    
13 4
1   3 
The pair can be anyone of the remaining 12 ranks
12   4 
The number of ways to choose the pair is    
1   2 
13   4  12   4 
    
[ Four of a kind ] =
1   3  1   2 
P= 0.0017
 52 
 
5 
 Poker Probability – Solution
15

5. Flush: any 5 cards from the same suit but not in sequence

The number of ways to choose such 5 cards is the number of ways to

choose 5 cards from 13 ranks – the number of straight flushes
13   4 
    − 40
 5  1 
13   4 
    − 40
=  5  1 
P[ Flush] = 0.00197
 52 
 
5 
 Poker Probability – Solution
16

6. Straight: The straight consists any one of the 10 possible sequences

of five consecutive cards from A-2-3-4-5 up to 10-J-Q-K-A. Each of
these 5 cards can have any one of the 4 suits. Excluding royal flush
and straight flush

10  5
  4 − 40
P[ Straight ] =
1 
= 0.00392
 52 
 
5 
 Poker Probability – Solution
17

7. Three of a kind: 3 cards are from any of the 13 ranks of the 4 suits,
the remaining 2 cards are from the remaining 12 ranks and each can
have any of the 4 suits. 13   4  12  2
    4
of a kind ] =
1 3 2 
=
P[Three 0.0211
 52 
 
5 

8. Two pair: 2 cards of the same rank, two cards of another rank, one
2
card not of either rank.     11  4 
13 4
     
P[Twopairs ] =
2   2  1  1 
= 0.0475
 52 
 
5 
 Poker Probability – Solution
18

9. Pair: The pair can be any one of the 13 ranks, and any 2 of the 4
suits. The remaining 3 cards are of different ranks from the remaining
12 ranks in any suit. 13   4  12  3
    4
=
P[Three  1  23 
of a kind ] = 0.423
 52 
 
5 

10. High card: no pair, 5 cards of the 13 ranks, discounting the 10

straights, each card can have any of the 4 suits, discounting the 4
flushes. 13   5

  − 10   4 − 4 
 
 5  
=
P[ High card ] = 0.501
 52 
 
5 
 Conditional Probability
19

 The conditional probability, P[A | B], of event A given that event B

has occurred, is defined as:

P[ A  B ]
P[ A | B ] =
P[B ]
where we assume that P[B] >0

 P[A] is a priori probability

 P[A|B] is a posteriori probability
 Indicates if the occurrence of B will increase the probability of the
occurrence of A or not
Properties of Conditional Probability
20
 Conditional Probability – Example 1
21
A number x is selected at random in the interval [-2, 2]. Let A = {x < 0},
B = {|x – 0.5| < 0.5}, and C = {x > 0.75}. Find the conditional
probabilities:
(1) P[A | B]
(2) P[B | C]
(3) P[A | Cc]
(4) P[B | Cc]
Solution
 Conditional Probability – Example 2
22
For each statement below, determine whether the statement is always
true.
(1)
(2)
(3)
(4)
Solution:
(1)
(2)
(3)
But this
does not mean .
(4) F, counter example
 Total Probability Theorem
23

A = (1)  (2)  (n)

(1) = A  B1 (1)
 (3) (5) (7)
(2) (10)
( n ) = A  Bn (4) (6) (8)

(1), (2), … , (n) are

pair wise mutually
exclusive
 Bayes’ Rule
24

B1 B3 B5 B9 S
B7
A
B2 B6 B8 B10
B4
 Total Probability Theorem Example
25

A store sells a batch of 60 iPods, 10 of which are defective. Suppose

that you purchase 2 iPods from this store for your FYP project. Suppose
that if any of the iPods that you purchased were defective, then you
return them to the store to swap for new iPods (you can swap both, if
needed). Also suppose that you could only return to the store at most
once. (Assume no iPod is sold or swapped before you return to the store)
What is the probability that you end up with 2 non-defective iPods?
Solution:
Define the following events B1 B2 B3
A: “end up with 2 non-defective iPod”
B1: “both initial iPods are non-defective” A

B2: “only 1 initial iPod is non-defective”

B3: “both initial iPods are defective”
 Total Probability Theorem Example
26

=
1. From the previous slide, we know P [ B1 ] 0.6921,
= P [ B2 ] 0.2825
10   50 
  
P [ B3 ] =
2 0 
= 0.0254
 60 
 
2 
2. If B1 occurs, you will end up 2 non-defective iPods P [ A | B1 ] = 1
3. If B2 occurs, you return to store to select 1 non-defective iPod
50 − 1
P [ A=
| B2 ]
# of non-defective iPods in the store
= 0.8448
60 − 2 Total # of iPods in the store
4. If B3 occurs, you return to store to select 2 non-defective iPods
 8   50 
  
=0 2 
[ 3]
P=A | B 0.7411
 58 
 
2 
 Total Probability Theorem Example
27

P [ A] = P [ A | B1 ] P[ B1 ] + P [ A | B2 ] P[ B2 ] + P [ A | B3 ] P[ B3 ]
= 1× 0.6921 + 0.8488 × 0.2825 + 0.7411× 0.0254
= 0.9496
 Total Probability Theorem & Bayes’ Rule
– Example
28
A man is worried that he might have a disease. He decides to get
tested. Suppose that the testing methods for this disease are known
to be accurate 99 percent of the time, regardless of whether the
results come back positive or negative. His test results come back
positive. If half of the population has it, what is the chance that he
actually has the disease? If, instead, the doctor tells him that the
disease is actually quite rare, occurring randomly in the general
population in only one of every 10,000 people, what is the chance
that he has the disease?
Solution
Define the following events
A: “the man has disease”
B: “the testing result is positive”
 Total Probability Theorem & Bayes’ Rule
– Example
29
P [ AB ]
We need to find out P [ A | B] =
P [ B]
1. P [ AB ] = P [ B | A] P[ A]

2.
A AC

P [ AB ] 0.99 × 0.5
[ A | B]
P= = = 0.99
P [ B ] 0.99 × 0.5 + (1 − 0.99 ) × 0.5
Total Probability Theorem & Bayes’ Rule
– Example
30