Mathematical Induction

Part One
Everybody – do the wave!
 The Wave
● If done properly, everyone will eventually
end up joining in.
● Why is that?
● Someone (me!) started everyone off.
● Once the person before you did the wave,
you did the wave.
Let P be some property. The principle of mathematical
induction states that if

P(0) is true …and it stays

If it starts
true…
true… and

∀k ∈ ℕ. (P(k) → P(k+1))

then

∀n ∈ ℕ. P(n)

…then it's
always true.
 Induction, Intuitively
● It's true for 0.
● Since it's true for 0, it's true for 1.
● Since it's true for 1, it's true for 2.
● Since it's true for 2, it's true for 3.
● Since it's true for 3, it's true for 4.
● Since it's true for 4, it's true for 5.
● Since it's true for 5, it's true for 6.
● …
 Proof by Induction
● A proof by induction is a way to use
mathematical induction to show that some result
is true for all natural numbers n.
● In a proof by induction, there are three steps:
● Prove that P(0) is true.
– This is called the basis or the base case.
● Prove that if P(k) is true, then P(k+1) is true.
– This is called the inductive step.
– The assumption that P(k) is true is called the inductive
hypothesis.
● Conclude, by induction, that P(n) is true for all n ∈ ℕ.
Some Summations
 20 = 1 = 21 – 1

2 0 + 2 1 = 1 + 2 = 3 = 22 – 1

20 + 2 1 + 2 2 = 1 + 2 + 4 = 7 = 2 3 – 1

20 + 21 + 22 + 23 = 1 + 2 + 4 + 8 = 15 = 24 – 1

20 + 21 + 22 + 23 + 24 = 1 + 2 + 4 + 8 + 16 = 31 = 25 – 1
Theorem: The sum of the first n powers of two is 2n – 1.
Proof: Let P(n) be the statement “the sum of the first n powers
of two is 2n – 1.” We will prove, by induction, that P(n) is
true for all n ∈ ℕ, from which the theorem follows.
For our base case, we need to show P(0) is true, meaning
that the sum of the first zero powers of two is 20 – 1. Since
the sum of the first zero powers of two is zero and 20 – 1
is zero as well, we see that P(0) is true.
At
At the
For the inductive start
thestep,
start of
of the
assume proof,
thethat we
we tell
for some
proof, k ∈the
tell ℕ that
the
P(k) holds, meaning
reader that
reader what property
what property we're
we're going
going to
to show
show isis
true 2 0
+ 2 1
+ … + 2 k-1
= 2 k
– 1. (1)
true for all natural numbers n, then tell
for all natural numbers n, then tell
We need them
to show thatgoing
we're P(k +to1) holds,
prove meaning
it by that the sum
induction.
them we're going to prove it by induction.
of the first k + 1 powers of two is 2k+1 – 1. To see this,
notice that
20 + 21 + … + 2k-1 + 2k = (20 + 21 + … + 2k-1) + 2k
= 2k – 1 + 2 k (via (1))
= 2(2k) – 1
= 2k+1 – 1.
Therefore, P(k + 1) is true, completing the induction. ■
Theorem: The sum of the first n powers of two is 2n – 1.
Proof: Let P(n) be the statement “the sum of the first n powers
of two is 2n – 1.” We will prove, by induction, that P(n) is
true for all n ∈ ℕ, from which the theorem follows.
For our base case, we need to show P(0) is true, meaning
that the sum of the first zero powers of two is 20 – 1. Since
the sum of the first zero powers of two is zero and 20 – 1
is zero as well, we see that P(0) is true.
For theIn aa proof
Ininductive by
by induction,
proof step, we
assume that
induction, we need
for
need to
to prove
some k ∈ ℕthat
prove that
that
P(k) holds, meaning that
2 0
◻◻ P(0) + 2
is
1
+
true… + 2 k-1
= 2 k
– 1. (1)
P(0) is true
We need to◻show If that P(k
P(k) is + 1) holds,
true, then meaning
P(k+1) is that the sum
true.
◻ If P(k) is true, then P(k+1) is true.
of the first k + 1 powers of two is 2k+1 – 1. To see this,
notice that
20 + 21 + … + 2k-1 + 2k = (20 + 21 + … + 2k-1) + 2k
= 2k – 1 + 2 k (via (1))
= 2(2k) – 1
= 2k+1 – 1.
Therefore, P(k + 1) is true, completing the induction. ■
Theorem: The sum of the first n powers of two is 2n – 1.
Proof: Let P(n) be the statement “the sum of the first n powers
of two is 2n – 1.” We will prove, by induction, that P(n) is
true for all n ∈ ℕ, from which the theorem follows.
For our base case, we need to show P(0) is true, meaning
that the sum of the first zero powers of two is 20 – 1. Since
the sum of the first zero powers of two is zero and 20 – 1
is zero as well, we see that P(0) is true.
For the Here,
Here, we
we state
inductive step, what
what P(0)
state assume thatactually
P(0) for some
actually says.
k ∈Now,
says. ℕ that
Now,
P(k) holds,
can meaning that
can go 0prove1 this
go prove this using
using any
any proof
proof techniques
techniques
2 + 2 + …we'd + 2 like!
k-1
= 2 – 1.
k
(1)
we'd like!
We need to show that P(k + 1) holds, meaning that the sum
of the first k + 1 powers of two is 2k+1 – 1. To see this,
notice that
20 + 21 + … + 2k-1 + 2k = (20 + 21 + … + 2k-1) + 2k
= 2k – 1 + 2 k (via (1))
= 2(2k) – 1
= 2k+1 – 1.
Therefore, P(k + 1) is true, completing the induction. ■
Theorem: The sum of the first n powers of two is 2n – 1.
Proof: Let P(n) be the statement “the sum of the first n powers
of two is 2n – 1.” We will prove, by induction, that P(n) is
true for all n ∈ ℕ, from which the theorem follows.
For our base case, we need to show P(0) is true, meaning
that the sum of the first zero powers of two is 20 – 1. Since
the sum of the first zero powers of two is zero and 20 – 1
is zero as well, we see that P(0) is true.
For the inductive step, assume that for some k ∈ ℕ that
P(k) holds, meaning that
20 + 21 + … + 2k-1 = 2k – 1. (1)
In
In aa proof
proof byby induction,
induction, wewe need
need toto prove
prove that that
We need to show that P(k + 1) holds, meaning that the sum
of the first k + 1 powers of two is 2k+1 – 1. To see this,
notice that ✓✓ P(0)
P(0) isis true
true
20 + 2 1 +◻◻…If+ P(k)
If 2P(k)
k-1
+isis
2ktrue,
= (20then
true, + 21 P(k+1)
then + … + is
P(k+1) 2isk-1true.
)true.
+ 2k
= 2k – 1 + 2 k (via (1))
= 2(2k) – 1
= 2k+1 – 1.
Therefore, P(k + 1) is true, completing the induction. ■
Theorem: The sum of the first n powers of two is 2n – 1.
Proof: Let P(n) be the statement “the sum of the first n powers
of two is 2n – 1.” We will prove, by induction, that P(n) is
true for all n ∈ ℕ, from which the theorem follows.
For our base case, we need to show P(0) is true, meaning
that the sum of the first zero powers of two is 20 – 1. Since
the sum of the first zero powers of two is zero and 20 – 1
is zero as well, we see that P(0) is true.
For the inductive step, assume that for some k ∈ ℕ that
P(k) holds, meaning that
20 + 21 + … + 2k-1 = 2k – 1. (1)
We need to show that P(k + 1) holds, meaning that the sum
of the first k + 1 powers of two is 2k+1 – 1. To see this,
notice that The The goal
goal of
of this
this step
step isis to
to prove
prove
20 + 21 +“If
… P(k)
+ 2k-1is+true,
2k =then
(20 +P(k+1)
21 + …is+true.”
2k-1) + 2k
“If P(k) is true, then P(k+1) is true.”
= 2k – 1 + 2 k (via (1))
To
To dodo this,
this, we'll
we'll choose
= 2(2kan
choose )an– arbitrary
1arbitrary k,
k, assume
assume
that is true, = 2
that P(k) is true, then try
P(k) then
k+1
– 1. to prove
try to prove P(k+1).
P(k+1).
Therefore, P(k + 1) is true, completing the induction. ■
Theorem: The sum of the first n powers of two is 2n – 1.
Proof: Let P(n) be the statement “the sum of the first n powers
of two is 2n – 1.” We will prove, by induction, that P(n) is
true for all n ∈ ℕ, from which the theorem follows.
For our base case, we need to show P(0) is true, meaning
that the sum of the first zero powers of two is 20 – 1. Since
the sum of the first zero powers of two is zero and 20 – 1
is zero as well, we see that P(0) is true.
For the inductive step, assume that for some k ∈ ℕ that
P(k) holds, meaning that
20 + 21 + … + 2k-1 = 2k – 1. (1)
We need to show that P(k + 1) holds, meaning that the sum
of the first k + 1 powers of two is 2k+1 – 1. To see this,
notice that
Here,
Here, we
we explicitly
explicitly stating
stating P(k+1),
P(k+1), which
which k
2 + 2 + … + 2 + 2 = (2 + 2 + … + 2 ) + 2
0 1 k-1 k 0 1 k-1
isis what
what we
we want
want to
to prove.
prove. Now,
Now, we can
=2 –1+2
k k
(viacan
we (1))
use
use any
any proof
proof technique
= 2(2k) –we
technique 1 want
we want to
to try
try
to
to prove –it.
prove
= 2 k+1
1.
it.
Therefore, P(k + 1) is true, completing the induction. ■
Theorem: The sum of the first n powers of two is 2n – 1.
Proof: Let P(n) be the statement “the sum of the first n powers
of two is 2n – 1.” We will prove, by induction, that P(n) is
true forHere,
all n ∈we
Here, ℕ,use
we from
use our inductive
which
our hypothesis
the theorem
inductive follows.
hypothesis
(the
For our base assumption
(the case, we need
assumption thatto P(k)
that P(k) isis
show true)
P(0) to
is true,
true) to meaning
that thesimplify
sum of athe first zero
complex powers ofThis
expression. two is
is 2
a
0
– 1. Since
simplify a complex expression. This is a
the sum of the first zero powers of two is zero and 20 – 1
common theme in inductive proofs.
is zero as well, we see that P(0) is true.proofs.
common theme in inductive
For the inductive step, assume that for some k ∈ ℕ that
P(k) holds, meaning that
20 + 21 + … + 2k-1 = 2k – 1. (1)
We need to show that P(k + 1) holds, meaning that the sum
of the first k + 1 powers of two is 2k+1 – 1. To see this,
notice that
20 + 21 + … + 2k-1 + 2k = (20 + 21 + … + 2k-1) + 2k
= 2k – 1 + 2 k (via (1))
= 2(2k) – 1
= 2k+1 – 1.
Therefore, P(k + 1) is true, completing the induction. ■
Theorem: The sum of the first n powers of two is 2n – 1.
Proof: Let P(n) be the statement “the sum of the first n powers
of two is 2n – 1.” We will prove, by induction, that P(n) is
true for all n ∈ ℕ, from which the theorem follows.
For our base case, we need to show P(0) is true, meaning
that the sum of the first zero powers of two is 20 – 1. Since
the sum of the first zero powers of two is zero and 20 – 1
is zero as well, we see that P(0) is true.
For the inductive step, assume that for some k ∈ ℕ that
P(k) holds, meaning that
20 + 21 + … + 2k-1 = 2k – 1. (1)
We need to show that P(k + 1) holds, meaning that the sum
of the first k + 1 powers of two is 2k+1 – 1. To see this,
notice that
20 + 21 + … + 2k-1 + 2k = (20 + 21 + … + 2k-1) + 2k
= 2k – 1 + 2 k (via (1))
= 2(2k) – 1
= 2k+1 – 1.
Therefore, P(k + 1) is true, completing the induction. ■
 A Quick Aside
● This result helps explain the range of
numbers that can be stored in an int.
● If you have an unsigned 32-bit integer,
the largest value you can store is given
by 1 + 2 + 4 + 8 + … + 231 = 232 – 1.
● This formula for sums of powers of two
has many other uses as well. If we have
time, we'll see one later today.
 Structuring a Proof by Induction
● Define some property P that you'll show, by induction, is
true for all natural numbers.
● Prove the base case:
● State that you're going to prove the property holds for 0, then
go prove it.
● Prove the inductive step:
● Say that you're assuming P is true for some natural number k,
then write out exactly what that means.
● Say that you're going to prove P is true for k+1, then write out
exactly what that means.
● Prove that P is true for k+1 using any proof technique you'd
like.
● This is a rather verbose way of writing inductive proofs.
As we get more experience with induction, we'll start
leaving out some details from our proofs.
 Induction, Intuitively
● You can imagine an “machine” that turns
proofs that the property holds for k into
proofs that the property holds for k + 1.
● Starting with a proof that the property holds
for 0, we can run the machine as many times
as we'd like to get proofs for 1, 2, 3, … .
● The principle of mathematical induction says
that this style of reasoning is a rigorous
argument.
Why Induction Works

P(k) → P(k + 1)

P(0)
The Counterfeit Coin Problem
 Problem Statement
● You are given a set of three seemingly
identical coins, two of which are real and
one of which is counterfeit.
● The counterfeit coin weighs more than
the rest of the coins.
● You are given a balance. Using only one
weighing on the balance, find the
counterfeit coin.
Finding the Counterfeit Coin

3
Finding the Counterfeit Coin

1 2

3
Finding the Counterfeit Coin

3
Finding the Counterfeit Coin

3
Finding the Counterfeit Coin

1 2

3
 A Harder Problem
● You are given a set of nine seemingly
identical coins, eight of which are real
and one of which is counterfeit.
● The counterfeit coin weighs more than
the rest of the coins.
● You are given a balance. Using only two
weighings on the balance, find the
counterfeit coin.
Finding the Counterfeit Coin

1 2 3 4 5 6

7 8 9
Finding the Counterfeit Coin

6
5
4
3
2
1
7 8 9

Now
Now wewe have
have one
one weighing
weighing
to
to find
find the
the counterfeit
counterfeit out
out
of
of these
these three
three coins.
coins.
Finding the Counterfeit Coin

1
2
3
4
5
7 8 9 6

Now
Now wewe have
have one
one weighing
weighing
to
to find
find the
the counterfeit
counterfeit out
out
of
of these
these three
three coins.
coins.
Finding the Counterfeit Coin

1 2 3 4 5 6

7 8 9

Now
Now wewe have
have one
one weighing
weighing
to
to find
find the
the counterfeit
counterfeit out
out
of
of these
these three
three coins.
coins.
 If we have n weighings on the scale, what
is the largest number of coins out of which
we can find the counterfeit?
 A Pattern
● Assume out of the coins that are given, exactly
one is counterfeit and weighs more than the
other coins.
● If we have no weighings, how many coins can
we have while still being able to find the
counterfeit?
● One coin, since that coin has to be the counterfeit!
● If we have one weighing, we can find the
counterfeit out of three coins.
● If we have two weighings, we can find the
counterfeit out of nine coins.
 So far, we have

1, 3, 9 = 30, 31, 32

Does this pattern continue?

Theorem: If exactly one coin in a group of 3n coins is heavier than the rest,
that coin can be found using only n weighings on a balance.
Proof: Let P(n) be the following statement:
If exactly one coin in a group of 3n coins is heavier than the rest,
that coin can be found using only n weighings on a balance.
We'll use induction to prove that P(n) holds for every n ∈ ℕ, from which
the theorem follows.
As our base case, we'll prove that P(0) is true, meaning that if we have
a set of 30=1 coins with one coin heavier than the rest, we can find that
coin with zero weighings. This is true because if we have just one coin,
At
At the start
thethan
it's vacuously heavier start of the
the proof,
ofothers,
all the and no we
proof, we tell
tell the
weighings the
are needed.
reader
reader
For the inductive what
what
step, property
property
suppose we're
we're
that P(k) going
going
is true to
to show
for some show isisso we
k ∈ ℕ,
can find the heavier
true of 3kall
for coins in k weighings.
natural numbers We'll
n, prove tell
then P(k+1): that
we can find thetrue forofall
heavier 3k+1natural
coins in numbers n, then tell
k+1 weighings.
them we're going to prove it by induction.
Suppose we them
have 3we're
k+1
coinsgoing to heavier
with one prove than
it bytheinduction.
others. Split the
coins into three groups of 3k coins each. Weigh two of the groups
against one another. If one group is heavier than the other, the coins in
that group must contain the heavier coin. Otherwise, the heavier coin
must be in the group we didn't put on the scale. Therefore, with one
weighing, we can find a group of 3 k coins containing the heavy coin. We
can then use k more weighings to find the heavy coin in that group.
We've given a way to use k+1 weighings and find the heavy coin out of
a group of 3k+1 coins. Thus P(k+1) is true, completing the induction. ■
Theorem: If exactly one coin in a group of 3n coins is heavier than the rest,
that coin can be found using only n weighings on a balance.
Proof: Let P(n) be the following statement:
If exactly one coin in a group of 3n coins is heavier than the rest,
that coin can be found using only n weighings on a balance.
We'll use induction to prove that P(n) holds for every n ∈ ℕ, from which
the theorem follows.
As our base case, we'll prove that P(0) is true, meaning that if we have
a set of 30=1 coins with one coin heavier than the rest, we can find that
coin with zero weighings. This is true because if we have just one coin,
In
In aa heavier
it's vacuously proof by
by induction,
proof than induction, we
we
all the others, need
need
and to
to prove
prove
no weighings that
arethat
needed.
For the inductive step, suppose that P(k) is true for some k ∈ ℕ, so we
can find the heavier
◻◻ P(0)of 3is
k
coins
true in k weighings. We'll prove P(k+1): that
P(0) ofis3k+1
we can find the heavier true
coins in k+1 weighings.
Suppose we have
◻ If P(k) is true, then P(k+1) is true.
If
◻ 3 k+1 P(k) is true, then P(k+1) is true.
coins with one heavier than the others. Split the
coins into three groups of 3k coins each. Weigh two of the groups
against one another. If one group is heavier than the other, the coins in
that group must contain the heavier coin. Otherwise, the heavier coin
must be in the group we didn't put on the scale. Therefore, with one
weighing, we can find a group of 3 k coins containing the heavy coin. We
can then use k more weighings to find the heavy coin in that group.
We've given a way to use k+1 weighings and find the heavy coin out of
a group of 3k+1 coins. Thus P(k+1) is true, completing the induction. ■
Theorem: If exactly one coin in a group of 3n coins is heavier than the rest,
that coin can be found using only n weighings on a balance.
Proof: Let P(n) be the following statement:
If exactly one coin in a group of 3n coins is heavier than the rest,
that coin can be found using only n weighings on a balance.
We'll use induction to prove that P(n) holds for every n ∈ ℕ, from which
the theorem follows.
As our base case, we'll prove that P(0) is true, meaning that if we have
a set of 30=1 coins with one coin heavier than the rest, we can find that
coin with zero weighings. This is true because if we have just one coin,
it's vacuously heavier than all the others, and no weighings are needed.
For the inductive step, suppose that P(k) is true for some k ∈ ℕ, so we
can find the heavier
Here, we ofstate
3k coins in kP(0)
what weighings. We'llsays.
actually prove Now,
P(k+1): that
we can findHere, we state
the heavier of 3k+1what
coins P(0)
in k+1actually
weighings.says. Now,
can go prove this using any proof techniques
can go3k+1
prove
Suppose we have coins this
with using any proof
one heavier than thetechniques
others. Split the
coins into three groups of 3k we'dwe'd like!
coins each.
like! Weigh two of the groups
against one another. If one group is heavier than the other, the coins in
that group must contain the heavier coin. Otherwise, the heavier coin
must be in the group we didn't put on the scale. Therefore, with one
weighing, we can find a group of 3 k coins containing the heavy coin. We
can then use k more weighings to find the heavy coin in that group.
We've given a way to use k+1 weighings and find the heavy coin out of
a group of 3k+1 coins. Thus P(k+1) is true, completing the induction. ■
Theorem: If exactly one coin in a group of 3n coins is heavier than the rest,
that coin can be found using only n weighings on a balance.
Proof: Let P(n) be the following statement:
If exactly one coin in a group of 3n coins is heavier than the rest,
that coin can be found using only n weighings on a balance.
We'll use induction to prove that P(n) holds for every n ∈ ℕ, from which
the theorem follows.
As our base case, we'll prove that P(0) is true, meaning that if we have
a set of 30=1 coins with one coin heavier than the rest, we can find that
coin with zero weighings. This is true because if we have just one coin,
it's vacuously heavier than all the others, and no weighings are needed.
For the inductive step, suppose that P(k) is true for some k ∈ ℕ, so we
can findIn
theaa
In proof
proof
heavier ofby
by induction,
induction,
3k coins we
we need
need
in k weighings. to prove
toprove
We'll prove that
that that
P(k+1):
we can find the heavier of 3k+1 coins in k+1 weighings.
Suppose we have 3k+1 coins with one heavier than the others. Split the
✓✓ P(0)
coins into three P(0) isisoftrue
groups true
3k coins each. Weigh two of the groups
◻◻ If
IfIfP(k)
against one another.
P(k) isis true,
one group then
is heavier
true, P(k+1)
then than isis true.
the other,
P(k+1) the coins in
true.
that group must contain the heavier coin. Otherwise, the heavier coin
must be in the group we didn't put on the scale. Therefore, with one
weighing, we can find a group of 3 k coins containing the heavy coin. We
can then use k more weighings to find the heavy coin in that group.
We've given a way to use k+1 weighings and find the heavy coin out of
a group of 3k+1 coins. Thus P(k+1) is true, completing the induction. ■
Theorem: If exactly one coin in a group of 3n coins is heavier than the rest,
that coin can be found using only n weighings on a balance.
Proof: Let P(n) be the following statement:
If exactly one coin in a group of 3n coins is heavier than the rest,
that coin can be found using only n weighings on a balance.
We'll use induction to prove that P(n) holds for every n ∈ ℕ, from which
the theorem follows.
As our base case, we'll prove that P(0) is true, meaning that if we have
a set of 30=1 coins with one coin heavier than the rest, we can find that
coin with zero weighings. This is true because if we have just one coin,
it's vacuously heavier than all the others, and no weighings are needed.
For the inductive step, suppose that P(k) is true for some k ∈ ℕ, so we
can find the heavier of 3k coins in k weighings. We'll prove P(k+1): that
we can find the heavier of 3k+1 coins in k+1 weighings.
Suppose we have 3k+1 coins with one heavier than the others. Split the
coins into three groups of 3k coins each. Weigh two of the groups
The
TheIfgoal of
of this
goalgroup this step
step isis to prove
totheprove
against one another. one is heavier than other, the coins in
that group must contain the heavier coin. Otherwise, the heavier coin
“If P(k) is true, then P(k+1) is true.”
must be in the “If P(k)
group we is true,
didn't putthen P(k+1)
on the scale.is true.” with one
Therefore,
weighing,
Towe can
do find
this, a group
we'll of 3 k
choose coins
ancontaining
arbitrary the
k,heavy coin. We
assume
To kdo
can then use this,
more we'll choose
weighings to find theanheavy
arbitrary
coin in k,
thatassume
group.
that
We've giventhat P(k)
a wayP(k) isis true,
to use true,
k+1 then
then try
weighingstry
andto
to prove
prove
find P(k+1).
P(k+1).
the heavy coin out of
a group of 3k+1 coins. Thus P(k+1) is true, completing the induction. ■
Theorem: If exactly one coin in a group of 3n coins is heavier than the rest,
that coin can be found using only n weighings on a balance.
Proof: Let P(n) be the following statement:
If exactly one coin in a group of 3n coins is heavier than the rest,
that coin can be found using only n weighings on a balance.
We'll use induction to prove that P(n) holds for every n ∈ ℕ, from which
the theorem follows.
As our base case, we'll prove that P(0) is true, meaning that if we have
a set of 30=1 coins with one coin heavier than the rest, we can find that
coin with zero weighings. This is true because if we have just one coin,
it's vacuously heavier than all the others, and no weighings are needed.
For the inductive step, suppose that P(k) is true for some k ∈ ℕ, so we
can find the heavier of 3k coins in k weighings. We'll prove P(k+1): that
we can find the heavier of 3k+1 coins in k+1 weighings.
Suppose we have 3k+1 coins with one heavier than the others. Split the
coins into three groups of 3k coins each. Weigh two of the groups
against one Here,
another.weIf one group isstate
explicitly heavier than thewhich
P(k+1), other,isthe coins in
Here,
that group must we explicitly
contain the heavier state
coin. P(k+1),
Otherwise,which is
the heavier coin
what
must be in thewhat we
groupwewewant
want to prove.
toput
didn't on the Now,
prove. we
we can
Now, Therefore,
scale. can with one
weighing, weuse
canany
find proof
a group technique
of 3 k coins containing
we want the
to heavy
try coin. We
can then use use anyweighings
k more proof technique we want
to find the heavy tothat
coin in trygroup.
to prove it.
We've given a way to use k+1to prove it.
weighings and find the heavy coin out of
a group of 3k+1 coins. Thus P(k+1) is true, completing the induction. ■
Theorem: If exactly one coin in a group of 3n coins is heavier than the rest,
that coin can be found using only n weighings on a balance.
Proof: Let P(n) be the following statement:
If exactly one coin in a group of 3n coins is heavier than the rest,
Here,
that coinHere, we
can bewe use our
useusing
found inductive
inductive
our only hypothesis
hypothesis
n weighings on a balance.
(the
(the assumption
We'll use induction to prove thatthat
assumption P(n) P(k)
that holds isisevery
P(k)for true)n∈
true) toℕ, from which
to
the theorem follows.
solve
solve this
this simpler
simpler version
version ofof the
the overall
overall
As our base case, we'll prove that P(0) is true, meaning that if we have
problem.
problem.
a set of 30=1 coins with one coin heavier than the rest, we can find that
coin with zero weighings. This is true because if we have just one coin,
it's vacuously heavier than all the others, and no weighings are needed.
For the inductive step, suppose that P(k) is true for some k ∈ ℕ, so we
can find the heavier of 3k coins in k weighings. We'll prove P(k+1): that
we can find the heavier of 3k+1 coins in k+1 weighings.
Suppose we have 3k+1 coins with one heavier than the others. Split the
coins into three groups of 3k coins each. Weigh two of the groups
against one another. If one group is heavier than the other, the coins in
that group must contain the heavier coin. Otherwise, the heavier coin
must be in the group we didn't put on the scale. Therefore, with one
weighing, we can find a group of 3 k coins containing the heavy coin. We
can then use k more weighings to find the heavy coin in that group.
We've given a way to use k+1 weighings and find the heavy coin out of
a group of 3k+1 coins. Thus P(k+1) is true, completing the induction. ■
Theorem: If exactly one coin in a group of 3n coins is heavier than the rest,
that coin can be found using only n weighings on a balance.
Proof: Let P(n) be the following statement:
If exactly one coin in a group of 3n coins is heavier than the rest,
that coin can be found using only n weighings on a balance.
We'll use induction to prove that P(n) holds for every n ∈ ℕ, from which
the theorem follows.
As our base case, we'll prove that P(0) is true, meaning that if we have
a set of 30=1 coins with one coin heavier than the rest, we can find that
coin with zero weighings. This is true because if we have just one coin,
it's vacuously heavier than all the others, and no weighings are needed.
For the inductive step, suppose that P(k) is true for some k ∈ ℕ, so we
can find the heavier of 3k coins in k weighings. We'll prove P(k+1): that
we can find the heavier of 3k+1 coins in k+1 weighings.
Suppose we have 3k+1 coins with one heavier than the others. Split the
coins into three groups of 3k coins each. Weigh two of the groups
against one another. If one group is heavier than the other, the coins in
that group must contain the heavier coin. Otherwise, the heavier coin
must be in the group we didn't put on the scale. Therefore, with one
weighing, we can find a group of 3 k coins containing the heavy coin. We
can then use k more weighings to find the heavy coin in that group.
We've given a way to use k+1 weighings and find the heavy coin out of
a group of 3k+1 coins. Thus P(k+1) is true, completing the induction. ■
 Some Fun Problems
● Suppose that you have a group of coins where
there's either exactly one heavier coin, or all
coins weigh the same amount. If you only get k
weighings, what's the largest number of coins
where you can find the counterfeit or determine
none exists?
● What happens if the counterfeit can be either
heavier or lighter than the other coins? What's
the maximum number of coins where you can
find the counterfeit if you have k weighings?
Time-Out for Announcements!
Apply to Section Lead!
 Problem Set Four
● Problem Set Four checkpoint was due at
3:00PM today. That's the last checkpoint of the
quarter!
● The remaining problems are due on Friday at
the start of class.
● Make sure to start early! You've probably figured
this out by now, but these questions take time to
think over.
● Ask questions when you have them! You can ask
on Piazza or in office hours.
 Midterm Exam
● The first midterm exam is next Monday,
October 26 from 7PM – 10PM, location TBA.
● Covers material from PS1 – PS3. Later
concepts will not be tested (yet).
● You're responsible for Lectures 00 – 09 and for
topics covered on PS1 – PS3.
● Students with OAE accommodations: please
contact us immediately if you need to take
the exam at an alternate time or need
extended time.
 Midterm Exam
● We want you to do well on this exam. We're not
trying to weed out weak students. We're not trying
to enforce a curve where there isn't one. We want
you to show what you've learned up to this point so
that you get a sense for where you stand and where
you can improve.
● The purpose of this midterm is to give you a chance
to show what you've learned in the past month. It is
not designed to assess your “mathematical
potential” or “innate mathematical ability.”
 Midterm Exam
● The exam is
● closed-book,
● closed-computer, and
● limited-note.
● You can have a single, double-sided sheet of 8.5” × 11”
paper with notes with you while you take the exam.
● You may want to start thinking about what you're
going to put on that note sheet.
● Our advice: write your own notes. The act of
creating them will help solidify your understanding.
 Practice Midterm Exam
● To help you prepare for the midterm, we'll be holding a
practice midterm exam on Wednesday, 7PM – 10PM
in Lathrop 282 and Lathrop 292.
● We've written two exams – a practice exam and a real
exam – that have similar structure and style. We'll give
you the practice midterm on Wednesday under realistic
conditions so that you can prepare for the exam.
● The TAs and I will be on hand to answer your questions.
● Can't make it? We'll release the practice exam online
and the solutions in hardcopy.
 Extra Practice Problems
● We'll be releasing three sets of cumulative
review problems this week that you can use to
prepare for the exam.
● We strongly recommend working through
these practice problems. They're a great way
to get additional practice with the material and
to see where you need to study.
● Solutions will be released in class and will be
available for pickup in the Gates filing cabinet.
 Advice from Generations Past
● We've released a handout (Handout 17)
containing advice from previous CS103
students about how to rock the exam.
● Read it over. There's some good advice in
there!
Your Questions
“I have been finding the problem sets to be
difficult and time-consuming. It takes me
some time to think of the solution prior to
even writing it up. I feel worried about the
upcoming midterm exam. How do you
suggest we go about preparing
effectively?”
I calibrate exam questions and problem set questions differently. With problem
I calibrate exam questions and problem set questions differently. With problem
set questions, I expect that you'll have a full week to work on them and that
set questions, I expect that you'll have a full week to work on them and that
you can talk them over in a group. With exam questions, I try to design them
you can talk them over in a group. With exam questions, I try to design them
so that you can solve them in about 45 minutes.
so that you can solve them in about 45 minutes.
Another detail to keep in mind is that the problem set questions appear harder
Another detail to keep in mind is that the problem set questions appear harder
than they are because this is your first time seeing much of this material. When
than they are because this is your first time seeing much of this material. When
you're solving this problems, you have to build up an understanding of the
you're solving this problems, you have to build up an understanding of the
material at the same time that you're solving the problem. Now that you have
material at the same time that you're solving the problem. Now that you have
some experience, I think you'll find the problems a lot easier to approach.
some experience, I think you'll find the problems a lot easier to approach.
“Welcome back! What was your favorite
part of Grace Hopper?”

There were a lot of fun

There were a lot of fun
parts. It's hard to say which
parts. It's hard to say which
was the best.
was the best.

Students who went to GHC:

Students who went to GHC:
want to share your stories?
want to share your stories?
“What do you think about ~1/5 of Stanford
Students Studying CS?”

I think it's a mixed bag. On the one hand, I think that anyone who
I think it's a mixed bag. On the one hand, I think that anyone who
wants to study CS should have a fair shot at doing so, and inasmuch
wants to study CS should have a fair shot at doing so, and inasmuch
as this makes this easier, I'm all for it. It also means that we have a
as this makes this easier, I'm all for it. It also means that we have a
more interesting mix of CS students and a wider set of interests.
more interesting mix of CS students and a wider set of interests.
That makes me feel really optimistic about the future.
That makes me feel really optimistic about the future.

On the other hand, I don't want people to feel forced to study CS

On the other hand, I don't want people to feel forced to study CS
or feel like they have to pick between CS and what they're really
or feel like they have to pick between CS and what they're really
interested in doing. I'm glad we have CS+X to help out with that.
interested in doing. I'm glad we have CS+X to help out with that.
Back to CS103!
How Not To Induct
 Something's Wrong...
Theorem: The sum of the first n powers of two is 2n.
Proof: Let P(n) be the statement “the sum of the first n powers
of two is 2n.” We will prove, by induction, that P(n) is
true for all n ∈ ℕ, from which the theorem follows.
For the inductive step, assume that for some k ∈ ℕ that
P(k) holds, meaning that
20 + 21 + … + 2k-1 = 2k. (1)
We need to show that P(k + 1) holds, meaning that the sum
of the first k + 1 powers of two is 2k+1. To see this, notice
that
20 + 21 + … + 2k-1 + 2k = (20 + 21 + … + 2k-1) + 2k
= 2k + 2k (via (1))
= 2(2k)
= 2k+1.
Therefore, P(k + 1) is true, completing the induction. ■
When writing a proof by induction,

make sure to show the base case!

Otherwise, your argument is invalid!

Why did this work?
Theorem: The sum of the first n powers of two is 2n.
Proof: Let P(n) be the statement “the sum of the first n powers
of two is 2n.” We will prove, by induction, that P(n) is
true for all n ∈ ℕ, from which the theorem follows.
For the inductive step, assume that for some k ∈ ℕ that
P(k) holds, meaning that
20 + 21 + … + 2k-1 = 2k. (1)
We need to show that P(k + 1) holds, meaning that the sum
of the first k + 1 powers of two is 2k+1. To see this, notice
that
20 + 21 + … + 2k-1 + 2k = (20 + 21 + … + 2k-1) + 2k
= 2k + 2k (via (1))
You
You=can
2(2prove
can prove
k
) anything
anything fromfrom aa
faulty assumption.
faulty= 2k+1.
assumption. This
This isis called
called the
the
Therefore, P(k + 1) is true,principle
principle of
completing explosion.
ofthe induction. ■
explosion.
The MU Puzzle
 Gödel, Escher Bach:
An Eternal Golden Braid
● Pulitzer-Prize
winning book
exploring recursion,
computability, and
consciousness.
● Written by Douglas
Hofstadter, cognitive
scientist at Indiana
University.
● A great (but dense!)
read.
 The MU Puzzle
● Begin with the string MI.
● Repeatedly apply one of the following
operations:
● Double the contents of the string after the M: for
example, MIIU becomes MIIUIIU, or MI becomes MII.
● Replace III with U: MIIII becomes MUI or MIU.
● Append U to the string if it ends in I: MI becomes
MIU.
● Remove any UU: MUUU becomes MU.
● Question: How do you transform MI to MU?
 MI
(a)

MII
(a)

MIIII
(a)
(a)Double
Doublethe
thestring
stringafter
afteran
anM.M. (c)
(b)
(b)Replace IIIwith
ReplaceIII withU.U. MIIIIU
(c)
(c)Append
AppendU,U,ififthe
thestring
stringends
endsin
inI.I. (b)
(d)
(d)Delete UUfrom
DeleteUU fromthe thestring.
string. MUIU
(a)

MUIUUIU
(d)

MUIIU
 Try It!

Starting with MI, apply these

operations to make MU:

(a)
(a)Double
Doublethe
thestring
stringafter
afteran
anM.M.
(b)
(b)Replace IIIwith
ReplaceIII withU.U.
(c)
(c)Append
AppendU,U,ififthe
thestring
stringends
endsin
inI.I.
(d)
(d)Delete UUfrom
DeleteUU fromthe thestring.
string.
 Not a single person in this room
was able to solve this puzzle.

Are we even sure that there is a solution?

Counting I's
MI 1

MII 2

MIIII 4

MIIIIU 4

MIIIIUIIIIU 8

MIIIIUUIU 5

MIIIIUUIUIIIIUUIU 10

MUIUUIUIIIIUUIU 7
 The Key Insight
● Initially, the number of I's is not a
multiple of three.
● To make MU, the number of I's must end
up as a multiple of three.
● Can we ever make the number of I's a
multiple of three?
Lemma 1: If n is an integer that is not a multiple of three,
then n – 3 is not a multiple of three.
Proof: By contrapositive; we'll prove that if n – 3 is a multiple
of three, then n is also a multiple of three. Because n – 3 is
a multiple of three, we can write n – 3 = 3k for some
integer k. Then n = 3(k+1), so n is also a multiple of three,
as required. ■
Lemma 2: If n is an integer that is not a multiple of three,
then 2n is not a multiple of three.
Proof: Let n be a number that isn't a multiple of three. If n is
congruent to one modulo three, then n = 3k + 1 for some
integer k. This means 2n = 2(3k+1) = 6k + 2 = 3(3k) + 2,
so 2n is not a multiple of three. Otherwise, n must be
congruent to two modulo three, so n = 3k + 2 for some
integer k. Then 2n = 2(3k+2) = 6k+4 = 3(2k+1) + 1, and
so 2n is not a multiple of three. ■
Lemma: No matter which moves are made, the number of I's in the string
never becomes multiple of three.
Proof: Let P(n) be the statement “After any n moves, the number of I's in
the string will not be multiple of three.” We will prove, by induction, that
P(n) is true for all n ∈ ℕ, from which the theorem follows.
As a base case, we'll prove P(0), that the number of I's after 0 moves is
not a multiple of three. After no moves, the string is MI, which has one I in
it. Since one isn't a multiple of three, P(0) is true.
For our inductive step, suppose that P(k) is true for some k ∈ ℕ. We'll
prove P(k+1) is also true. Consider any sequence of k+1 moves. Let r be
the number of I's in the string after the kth move. By our inductive
hypothesis (that is, P(k)), we know that r is not a multiple of three. Now,
consider the four possible choices for the k+1st move:
Case 1: Double the string after the M. After this, we will have 2r I's
in the string, and from our lemma 2r isn't a multiple of three.
Case 2: Replace III with U. After this, we will have r – 3 I's in the string,
and by our lemma r – 3 is not a multiple of three.
Case 3: Either append U or delete UU. This preserves the number of
I's in the string, so we don't have a multiple of three I's at this point.
Therefore, no sequence of k+1 moves ends with a multiple of three I's.
Thus P(k+1) is true, completing the induction. ■
Theorem: The MU puzzle has no solution.

Proof: Assume for the sake of contradiction that the MU

puzzle has a solution and that we can convert MI to
MU. This would mean that at the very end, the number
of I's in the string must be zero, which is a multiple of
three. However, we've just proven that the number of
I's in the string can never be a multiple of three.

We have reached a contradiction, so our assumption

must have been wrong. Thus the MU puzzle has no
solution. ■
Algorithms and Loop Invariants
● The proof we just made had the form
● “If P is true before we perform an action, it is true
after we perform an action.”
● We could therefore conclude that after any series
of actions of any length, if P was true beforehand,
it is true now.
● In algorithmic analysis, this is called a loop
invariant.
● Proofs on algorithms often use loop invariants to
reason about the behavior of algorithms.
● Take CS161 for more details!
The Limits of Data Compression
 Bitstrings
● A bitstring is a finite sequence of 0s and
1s.
● Examples:
● 11011100
● 010101010101
● 0000
● ε (the empty string)
● There are 2n bitstrings of length n.
 Data Compression
● Inside a computer, all data are represented as
sequences of 0s and 1s (bitstrings)
● To transfer data (across a network, on DVDs, on a flash
drive, etc.), it is useful to reduce the number of 0s and
1s before transferring it.
● Most real-world data can be compressed by exploiting
redundancies.
● Text repeats common patterns (“the”, “and”, etc.)
● Bitmap images use similar colors throughout the image.
●
Idea: Replace each bitstring with a shorter bitstring
that contains all the original information.
● This is called lossless data compression.
101010101010101010101010101010
Compress

1111010

Transmit

1111010
Decompress

101010101010101010101010101010
 Lossless Data Compression
● In order to losslessly compress data, we need two
functions:
● A compression function C, and
● A decompression function D.
● We need to have D(C(x)) = x.
● Otherwise, we can't uniquely encode or decode
some bitstring.
● This means that D must be a left inverse of C, so
(as you proved in PS3!) C must be injective.
 A Perfect Compression Function
● Ideally, the compressed version of a bitstring
would always be shorter than the original
bitstring.
● Question: Can we find a lossless compression
algorithm that always compresses a string
into a shorter string?
● To handle the issue of the empty string (which
can't get any shorter), let's assume we only
care about strings of length at least 10.
 A Counting Argument
● Let �n be the set of bitstrings of length n, and �<n be
the set of bitstrings of length less than n.
● How many bitstrings of length n are there?
● Answer: 2n
● How many bitstrings of length less than n are there?
● Answer: 20 + 21 + … + 2n – 1 = 2n – 1
● By the pigeonhole principle, no function from �n to �<n
can be injective – at least two elements must collide!
● Since a perfect compression function would have to be
an injection from �n to �<n, there is no perfect
compression function!
 Why this Result is Interesting
● Our result says that no matter how hard we try,
it is impossible to compress every string into a
shorter string.
● No matter how clever you are, you cannot write
a lossless compression algorithm that always
makes strings shorter.
● In practice, only highly redundant data can be
compressed.
● The fields of information theory and
Kolmogorov complexity explore the limits of
compression; if you're interested, go explore!
 Next Time
● Variations on Induction
● Starting induction later.
● Taking larger steps.
● Complete induction.