CS109: Probability for

Computer Scientists

Piech, CS109, Stanford University

 Problem Set #1 is out newish

Check your answer

Insert LaTeX

Auto
Submission

Piech, CS109, Stanford University

Write an Agent newish

Piech, CS109, Stanford University

Python Review Session

Friday at 5-7pm PT, recorded

Find links, recordings, and setup here

Piech, CS109, Stanford University

Learn LaTex

Handout to help you get

started

Piech, CS109, Stanford University

What Makes for a Good Answer?

Piech, CS109, Stanford University

Honor Code
Always remember: You need to be able to
recreate your ability on an exam. And in the
real world. This is a foundation course.
Cheating in CS109 is cheating yourself.

Talk to your friends about the concepts, not

the solution. Words must be your own.

Practice the art of teaching. Three most

important things to know:
1. Do not give away the answer
2. Always be respectful
3. Know what you don’t know

Piech, CS109, Stanford University

Late Policy
Three types of extensions:
1. Grace period (24 hours)
2. Quite Late extension (2 class days)
3. After the hard deadline (> 2 class days)

You give them to yourself. Need to talk to us if:

A. You need more than 1 long extensions / qtr
B. Extension past the hard deadline

But CS109 is a fast class. If you want a long

extension I want you to be intentional about how
you are going to catch up.

Piech, CS109, Stanford University

If you notice a bug?

It should be robust, but things can happen.

Let me know: send an email to [email protected] or message

me on slack. I need your email and the approximate time you
encountered the bug.

Piech, CS109, Stanford University

Section Signups Live

Piech, CS109, Stanford University

Class Theme Song…

Piech, CS109, Stanford University

Above and Beyond?

Piech, CS109, Stanford University

Review

Piech, CS109, Stanford University

CS109: From Counting to Machine Learning

Piech, CS109, Stanford University

Core Counting
Counting with steps

Counting with “or”

15
Piech, CS109, Stanford University
How Many Bit Stings?
24 start with 01 24 end with 10
Problem: A 6-bit string is sent over a 010000 000010
network. The valid set of strings recognized 010001 000110
by the receiver must either start with "01" 010010 001010
or end with "10". How many such strings 010011 001110
are there? 010100 010010
010101 010110
010110 011010
010111 011110
Answer 011000 100010
011001 100110
011010 101010
011011 101110
011100 110010
011101 110110
011110 111010
011111 111110
Set A Set B
Piech, CS109, Stanford University
End Review

Piech, CS109, Stanford University

Challenge Problem
1. Strings
– How many different orderings of letters
are possible for the string BOBA?

BOBA,ABOB,OBBA…

18
Piech, CS109, Stanford University
Permutations I

19
Orderings of Letters

How many letter orderings are possible for the following strings?

1. CHRIS

This is Jerry’s dog, Doris. She puts her little Doris paw
up to her chin when she’s thinking.

Piech, CS109, Stanford University 20

Orderings of Letters
chirs crish hicrs hsirc irchs rcish rschi shirc
chisr crshi hicsr hsrci ircsh rcshi rscih shrci
chris crsih hircs hsric irhcs rcsih rshci shric
chrsi cshir hirsc ichrs irhsc rhcis rshic sichr
chsir cshri hiscr ichsr irsch rhcsi rsich sicrh
chsri csihr hisrc icrhs irshc rhics rsihc sihcr
cihrs csirh hrcis icrsh ischr rhisc schir sihrc
cihsr csrhi hrcsi icshr iscrh rhsci schri sirch
cirhs csrih hrics icsrh ishcr rhsic scihr sirhc
cirsh hcirs hrisc ihcrs ishrc richs scirh srchi
cishr hcisr hrsci ihcsr isrch ricsh scrhi srcih
cisrh hcris hrsic ihrcs isrhc rihcs scrih srhci
crhis hcrsi hscir ihrsc rchis rihsc shcir srhic
crhsi hcsir hscri ihscr rchsi risch shcri srich
crihs hcsri hsicr ihsrc rcihs rishc shicr srihc

Piech, CS109, Stanford University 21

Orderings of letters

Step 3: Step 5:
Step 1: Chose 5th letter
Chose first letter Chose 3rd letter
Step 2: Step 4:
Chose 2nd letter Chose 4th letter

Piech, CS109, Stanford University 22

Orderings of letters

H C S I R
Step 3: Step 5:
Step 1: Chose 5th letter
Chose first letter Chose 3rd letter
(3 options) (1 option)
(5 options)
Step 2: Step 4:
Chose 2nd letter Chose 4th letter
(4 options) (2 options)
Piech, CS109, Stanford University 23
Permutations

A permutation is an ordered arrangement of objects.

The number of unique orderings (permutations) of 𝑛 distinct objects is

𝑛! = 𝑛 × 𝑛 − 1 × 𝑛 − 2 × ⋯ × 2 × 1.

Piech, CS109, Stanford University 24

Unique 6-digit passcodes with six smudges

How many unique 6-digit passcodes are possible if a

phone password uses each of six distinct numbers?

25
Piech, CS109, Stanford University
Unique 6-digit passcodes with six smudges

How many unique 6-digit passcodes are possible if a

phone password uses each of six distinct numbers?
Total = 6! = 720 passcodes

What if you had no smudges, but you knew it was still 6

digits long?
Total = 106 = 1 million passcodes

26
Piech, CS109, Stanford University
Permutations II

27
 Summary of Combinatorics
Counting tasks on 𝑛 objects

Sort objects Choose 𝑘 objects Put objects in 𝑟

(permutations) (combinations) buckets

Distinct
(distinguishable)

Piech, CS109, Stanford University 28

 Summary of Combinatorics
Counting tasks on 𝑛 objects

Sort objects Choose 𝑘 objects Put objects in 𝑟

(permutations) (combinations) buckets

Distinct Some
(distinguishable) distinct

𝑛!

Piech, CS109, Stanford University 29

Unique Bit Strings

1,0,1,0,0

30
Piech, CS109, Stanford University
Sort 𝑛 distinct objects

31
Piech, CS109, Stanford University
Sort 𝑛 distinct objects

32
Piech, CS109, Stanford University
Sort 𝑛 distinct objects

Steps:
1. Choose 1st can 5 options
2. Choose 2nd can 4 options
…
5. Choose 5th can 1 option

1st 2nd 3rd 4th 5th

Total = 5 × 4 × 3 × 2 × 1
= 120

33
Piech, CS109, Stanford University
How many ways can we sort coke cans!

Piech, CS109, Stanford University 34

Sort 𝑛 distinct objects

# of permutations =
Piech, CS109, Stanford University 35
 Order 𝑛
Sort semi-distinct objects distinct objects
𝑛!

All distinct Some indistinct

Piech, CS109, Stanford University 36

Sort semi-distinct objects
How do we find the number of permutations considering
some objects are indistinct?

By the product rule, permutations of distinct objects is a two-step process:

permutations Permutations
permutations
of distinct objects = considering some
objects are indistinct
× of just the
indistinct objects

Piech, CS109, Stanford University 37

Sort semi-distinct objects
How do we find the number of permutations considering
some objects are indistinct?

By the product rule, permutations of distinct objects is a two-step process:

permutations Permutations

___________ =
permutations
of distinct objects
considering some
objects are indistinct
× of just the
indistinct objects
Permutations
of just the
indistinct objects

Piech, CS109, Stanford University 38

General approach to counting permutations
When there are 𝑛 objects such that
𝑛1 are the same (indistinguishable or indistinct), and
𝑛2 are the same, and
…
𝑛𝑟 are the same,
The number of unique orderings (permutations) is

𝑛!
.
𝑛1 ! 𝑛2 ! ⋯ 𝑛𝑟 !

For each group of indistinct objects,

Divide by the overcounted permutations.
Piech, CS109, Stanford University 39
 Order 𝑛 semi- 𝑛!
Sort semi-distinct objects distinct objects 𝑛1 ! 𝑛2 ! ⋯ 𝑛𝑟 !

How many permutations?

Piech, CS109, Stanford University 40

 Order 𝑛 semi- 𝑛!
Strings distinct objects 𝑛1 ! 𝑛2 ! ⋯ 𝑛𝑟 !

How many letter orderings are possible for the following strings?

1. BOBA

2. MISSISSIPPI
This is Jerry’s dog, Doris. She puts her little Doris paw
up to her chin when she’s thinking.

Piech, CS109, Stanford University 41

 Order 𝑛 semi- 𝑛!
Strings distinct objects 𝑛1 ! 𝑛2 ! ⋯ 𝑛𝑟 !

How many letter orderings are possible for the following strings?

4!
1. BOBA =
2!
= 12

11!
2. MISSISSIPPI =
1!4!4!2!
= 34,650

Piech, CS109, Stanford University 42

Strings
Is this just a regular permutation? 4! = 24 unique permutations!

boba bboa
boab bbao
bboa boba
bbao boab
baob babo
babo baob
obba abob
obab abbo
obba aobb
obab aobb
oabb abbo
oabb abob

Piech, CS109, Stanford University

Strings
Is this just a regular permutation? 4! = 24 unique permutations!

boba bboa
boab bbao
bboa boba
bbao boab
baob babo
babo baob
obba abob
obab abbo
obba aobb
obab aobb
oabb abbo
oabb abob

Piech, CS109, Stanford University

To the Code!
baob import itertools
bbao def main():
obba letters = ['b','o','b','a']
perms = set(itertools.permutations(letters))
oabb for perm in perms:
boab pretty_perm = "".join(perm)
print(pretty_perm)
babo
abbo import math
aobb
def main():
boba n = math.factorial(4)
abob d = math.factorial(2)
print(n / d)
bboa
obab
Piech, CS109, Stanford University 45
 Order 𝑛 semi- 𝑛!
Unique 6-digit passcodes with five smudges distinct objects 𝑛1 ! 𝑛2 ! ⋯ 𝑛𝑟 !

How many unique 6-digit passcodes are possible if a

phone password uses each of five distinct numbers?
Steps:
1. Choose digit to repeat 5 outcomes
2. Create passcode (sort 6 digits:
4 distinct, 2 indistinct)

6!
Total = 5 ×
2!
= 1,800 passcodes
Piech, CS109, Stanford University 46
 Summary of Combinatorics
Counting tasks on 𝑛 objects

Sort objects Choose 𝑘 objects Put objects in 𝑟

(permutations) (combinations) buckets

Distinct Some
(distinguishable) distinct

𝑛!
𝑛!
𝑛1 ! 𝑛2 ! ⋯ 𝑛𝑟 !
Piech, CS109, Stanford University 47
 02b_combinations_i

Combinations I

48
 Summary of Combinatorics
Counting tasks on 𝑛 objects

Sort objects Choose 𝑘 objects Put objects in 𝑟

(permutations) (combinations) buckets

Distinct
Distinct Some
(distinguishable) distinct

𝑛!
𝑛!
𝑛1 ! 𝑛2 ! ⋯ 𝑛𝑟 !
Piech, CS109, Stanford University 49
Combinations with cake
There are 𝑛 = 20 people.
How many ways can we choose 𝑘 = 5 people to get cake?

Consider the following

generative process…
Piech, CS109, Stanford University 50
Combinations with cake
There are 𝑛 = 20 people.
How many ways can we choose 𝑘 = 5 people to get cake?

1 2 3 4 5 6 7 8 9 10

11 12 13 14 15 16 17 18 19 20
1. 𝑛 people
get in line

𝑛! ways
Piech, CS109, Stanford University 51
Combinations with cake
There are 𝑛 = 20 people.
How many ways can we choose 𝑘 = 5 people to get cake?

1 2 3 4 5 6 7 8 9 10

11 12 13 14 15 16 17 18 19 20
1. 𝑛 people 2. Put first 𝑘
get in line in cake room

𝑛! ways 1 way
Piech, CS109, Stanford University 52
Combinations with cake
There are 𝑛 = 20 people.
How many ways can we choose 𝑘 = 5 people to get cake?

6 7 8 9 10 11 12 13

14 15 16 17 18 19 20
1. 𝑛 people 2. Put first 𝑘
get in line in cake room

𝑛! ways 1 way
Piech, CS109, Stanford University 53
Combinations with cake
There are 𝑛 = 20 people.
How many ways can we choose 𝑘 = 5 people to get cake?

6 7 8 9 10 11 12 13

14 15 16 17 18 19 20
1. 𝑛 people 2. Put first 𝑘 3. Allow cake
get in line in cake room group to mingle
𝑘! different
𝑛! ways 1 way permutations lead to
the same mingle
Piech, CS109, Stanford University 54
Combinations with cake
There are 𝑛 = 20 people.
How many ways can we choose 𝑘 = 5 people to get cake?

6 7 8 9 10 11 12 13

14 15 16 17 18 19 20
1. 𝑛 people 2. Put first 𝑘 3. Allow cake 4. Allow non-cake
get in line in cake room group to mingle group to mingle
𝑘! different
𝑛! ways 1 way permutations lead to
the same mingle
Piech, CS109, Stanford University 55
Combinations with cake
There are 𝑛 = 20 people.
How many ways can we choose 𝑘 = 5 people to get cake?

1. 𝑛 people 2. Put first 𝑘 3. Allow cake 4. Allow non-cake

get in line in cake room group to mingle group to mingle
𝑘! different 𝑛 − 𝑘 ! different
𝑛! ways 1 way permutations lead to permutations lead to the
the same mingle same mingle
Piech, CS109, Stanford University 56
Combinations

A combination is an unordered selection of 𝑘 objects

from a set of 𝑛 distinct objects.

The number of ways of making this selection is

𝑛! 1 1 𝑛
= 𝑛! × 1 × × =
𝑘! 𝑛 − 𝑘 ! 𝑘! 𝑛−𝑘 ! 𝑘
4. Overcounted:
any ordering
1. Order 𝑛 2. Take first 𝑘 3. Overcounted: of unchosen
distinct as chosen any ordering of group is
objects chosen group is same choice
same choice
Piech, CS109, Stanford University 57
Combinations

A combination is an unordered selection of 𝑘 objects

from a set of 𝑛 distinct objects.

The number of ways of making this selection is

𝑛! 1 1 𝑛𝑛 Binomial
= 𝑛! × 1 × × =
𝑘! 𝑛 − 𝑘 ! 𝑘! 𝑛−𝑘 ! 𝑘𝑘 coefficient

𝑛 𝑛
Fun Fact:
𝑘
= 𝑛−𝑘

Piech, CS109, Stanford University 58

 Choose 𝑘 of 𝑛
Probability textbooks 𝑛 distinct objects 𝑘

How many ways are there to choose 3 books

from a set of 6 distinct books?
Cain Prob Makes Sense.

6 6!
= = 20 ways
3 3! 3!

Piech, CS109, Stanford University 59

To the code!
How many unique hands of 5 cards are there in a 52 card deck?

def main():
def main():
cards = make_deck()
total = math.comb(52, 5)
all_hands = itertools.combinations(cards, 5)
print(total)
for hand in all_hands:
print(hand)

Piech, CS109, Stanford University 60

Buckets and The
Divider Method

69
 Summary of Combinatorics
Counting tasks on 𝑛 objects

Sort objects Choose 𝑘 objects Put objects in 𝑟

(permutations) (combinations) buckets

Distinct
Distinct Some 𝑛
(distinguishable) distinct Distinct Indistinct
𝑘
𝑛!
𝑛!
𝑛1 ! 𝑛2 ! ⋯ 𝑛𝑟 !
Piech, CS109, Stanford University 70
Balls and urns Hash tables and distinct strings
How many ways are there to hash 𝑛 distinct strings to 𝑟 buckets?
Steps:
1. Bucket 1st string
madagascar japan
2. Bucket 2nd string
italy
south korea …
𝑛. Bucket 𝑛th string

1 2 r 𝑟 𝑛 outcomes

Piech, CS109, Stanford University 71

 Summary of Combinatorics
Counting tasks on 𝑛 objects

Sort objects Choose 𝑘 objects Put objects in 𝑟

(permutations) (combinations) buckets

Distinct
Distinct Some 𝑛
(distinguishable) distinct Distinct Indistinct
𝑘
𝑛!
𝑛! 𝑟𝑛
𝑛1 ! 𝑛2 ! ⋯ 𝑛𝑟 !
Piech, CS109, Stanford University 72
Servers and indistinct requests
How many ways are there to distribute 𝑛 indistinct web requests to 𝑟 servers?

request request
Goal
Server 1 has 𝑥1 requests,
request request
Server 2 has 𝑥2 requests,
…
Server 𝑟 has 𝑥𝑟 requests
…
constraint: σ𝑟𝑖=1 𝑥𝑖 = 𝑛
1 2 r

Piech, CS109, Stanford University 73

Bicycle helmet sales
How many ways can we assign 𝑛 = 5 indistinct children to 𝑟 = 4 distinct
bicycle helmet styles?

Piech, CS109, Stanford University 74

Bicycle helmet sales
1 possible assignment outcome:

Goal Order 𝑛 indistinct objects and 𝑟 − 1 indistinct dividers.

Consider the
following
generative
process…
Piech, CS109, Stanford University 75
The divider method: A generative proof
How many ways can we assign 𝑛 = 5 indistinct children to 𝑟 = 4 distinct
bicycle helmet styles?
Goal Order 𝑛 indistinct objects and 𝑟 − 1 indistinct dividers.
0.
1. Make objects and dividers distinct

Piech, CS109, Stanford University 76

The divider method: A generative proof
How many ways can we assign 𝑛 = 5 indistinct children to 𝑟 = 4 distinct
bicycle helmet styles?
Goal Order 𝑛 indistinct objects and 𝑟 − 1 indistinct dividers.
0.
1. Make objects and dividers distinct

1. Order 𝑛 distinct
objects and 𝑟 − 1
distinct dividers
𝑛+𝑟−1 !
Piech, CS109, Stanford University 77
The divider method: A generative proof
How many ways can we assign 𝑛 = 5 indistinct children to 𝑟 = 4 distinct
bicycle helmet styles?
Goal Order 𝑛 indistinct objects and 𝑟 − 1 indistinct dividers.
0.
1. Make objects and dividers distinct

1. Order 𝑛 distinct 2. Make 𝑛 objects

objects and 𝑟 − 1 indistinct
distinct dividers 1
𝑛+𝑟−1 ! 𝑛!
Piech, CS109, Stanford University 78
The divider method: A generative proof
How many ways can we assign 𝑛 = 5 indistinct children to 𝑟 = 4 distinct
bicycle helmet styles?
Goal Order 𝑛 indistinct objects and 𝑟 − 1 indistinct dividers.
0.
1. Make objects and dividers distinct

1. Order 𝑛 distinct 2. Make 𝑛 objects 3. Make 𝑟 − 1 dividers

objects and 𝑟 − 1 indistinct indistinct
distinct dividers 1 1
𝑛+𝑟−1 ! 𝑛! 𝑟−1 !
Piech, CS109, Stanford University 79
The divider method

The number of ways to distribute 𝑛 indistinct objects into 𝑟 buckets is

equivalent to the number of ways to permute 𝑛 + 𝑟 − 1 objects such that
𝑛 are indistinct objects, and
𝑟 − 1 are indistinct dividers:
1 1
Total = (𝑛 + 𝑟 − 1)! × ×
𝑛! 𝑟−1 !
𝑛 + 𝑟 − 1
= outcomes
𝑟 − 1

Piech, CS109, Stanford University 80

 Divider method 𝑛+𝑟−1
Integer solutions to equations (𝑛 indistinct objects, 𝑟 buckets)
𝑟−1

How many integer solutions (assignments to each x) are there to the

following equation:
𝑥1 + 𝑥2 + ⋯ + 𝑥6 = 20
where for all 𝑖, 𝑥𝑖 is an integer such that 0 ≤ 𝑥𝑖 ≤ 20?

Piech, CS109, Stanford University 81

 Divider method 𝑛+𝑟−1
Integer solutions to equations (𝑛 indistinct objects, 𝑟 buckets)
𝑟−1

How many integer solutions (assignments to each x) are there to the

following equation:
𝑥1 + 𝑥2 + ⋯ + 𝑥6 = 20
where for all 𝑖, 𝑥𝑖 is an integer such that 0 ≤ 𝑥𝑖 ≤ 20?

Example solution: 2 + 3 + 2 + 5 + 5 + 3

Piech, CS109, Stanford University 82

Piech, CS109, Stanford University
Piech, CS109, Stanford University 84
Piech, CS109, Stanford University 85
 Summary of Combinatorics
Counting tasks on 𝑛 objects

Sort objects Choose 𝑘 objects Put objects in 𝑟

(permutations) (combinations) buckets

Distinct
Distinct Some
(distinguishable) distinct Distinct Indistinct

𝑛! 𝑛
𝑛! 𝑟𝑛 𝑛+𝑟−1
𝑛1 ! 𝑛2 ! ⋯ 𝑛𝑟 ! 𝑘
𝑟−1
Piech, CS109, Stanford University 90
 Summary of Combinatorics
Counting tasks on 𝑛 objects

Sort objects Choose 𝑘 objects Put objects in 𝑟

(permutations) (combinations) buckets

Distinct
Distinct Some
(distinguishable) distinct Distinct Indistinct

𝑛! 𝑛
𝑛! 𝑟𝑛 𝑛+𝑟−1
𝑛1 ! 𝑛2 ! ⋯ 𝑛𝑟 ! 𝑘
𝑟−1
Piech, CS109, Stanford University 91
Ready for the first 6 problems (and the rest on Friday)

Piech, CS109, Stanford University 92

Formalizing a Word Problem is Hard

Piech, CS109, Stanford University 93

Want to go deep?

Piech, CS109, Stanford University 94

The Party Problem and The Birth of Probability

https://fr.wikipedia.org/wiki/Probl%C3%A8me_des_partis

Piech, CS109, Stanford University 95