Induction

(chapter 4.2-4.4 of the book and chapter 3.3-3.6 of the notes)

 This Lecture

Last time we have discussed different proof techniques.

This time we will focus on probably the most important one

– mathematical induction.

This lecture’s plan is to go through the following:

• The idea of mathematical induction

• Basic induction proofs (e.g. equality, inequality, property,etc)

• An interesting example

• A paradox
 Odd Powers Are Odd

Fact: If m is odd and n is odd, then nm is odd.

Proposition: for an odd number m, mk is odd for all non-negative integer k.

Let P(i) be the proposition that mi is odd.

Idea of induction. • P(1) is true by definition.

• P(2) is true by P(1) and the fact.
• P(3) is true by P(2) and the fact.
• P(i+1) is true by P(i) and the fact.
• So P(i) is true for all i.
 Divisibility by a Prime
Theorem. Any integer n > 1 is divisible by a prime number.

• Let n be an integer.

• If n is a prime number, then we are done.

• Otherwise, n = ab, both are smaller than n.

• If a or b is a prime number, then we are done.

• Otherwise, a = cd, both are smaller than a.

• If c or d is a prime number, then we are done.

• Otherwise, repeat this argument, since the numbers are

getting smaller and smaller, this will eventually stop and
we have found a prime factor of n.

Idea of induction.
 Idea of Induction

Objective: Prove

This is to prove

The idea of induction is to first prove P(0) unconditionally,

then use P(0) to prove P(1)

then use P(1) to prove P(2)

and repeat this to infinity…

 The Induction Rule

0 and (from n to n +1), Much easier to

prove with P(n) as
proves 0, 1, 2, 3,…. an assumption.
Very easy
to prove

P (0), P (n)P (n+1)

mN. P (m)
For any n>=0

Like domino effect…

 This Lecture

• The idea of mathematical induction

• Basic induction proofs (e.g. equality, inequality, property,etc)

• An interesting example

• A paradox
 Proof by Induction

Let’s prove:

r  1.

Statements in green form a template for inductive proofs.

Proof: (by induction on n)

The induction hypothesis, P(n), is:

r  1.
 Proof by Induction

Induction Step: Assume P(n) for some n  0 and prove P(n + 1):

r ( n+1)1  1
r  1. 1  r  r 2    r n+1 
r 1

Have P (n) by assumption:

So let r be any number  1, then from P (n) we have

n 1
r 1
1 r  r   r 
2 n

r 1
How do we proceed?
 Proof by Induction

adding r n+1 to both sides,

r n 1  1 n 1
1    r n  r n+1  r
r 1
r n 1  1  r n 1 (r  1)

r 1
r ( n+1) 1  1

r 1
But since r  1 was arbitrary, we conclude (by UG), that
r ( n+1)1  1
r  1. 1  r  r 2    r n+1 
r 1
which is P (n+1). This completes the induction proof.
 Proving an Equality

Let P(n) be the induction hypothesis that the statement is true for n.

Base case: P(1) is true

Induction step: assume P(n) is true, prove P(n+1) is true.

by induction
 Proving a Property

Base Case (n = 1):

Induction Step: Assume P(i) for some i  1 and prove P(i + 1):

Assume is divisible by 3, prove Is divisible by 3.

Divisible by 3 Divisible by 3 by induction

 Proving a Property

Base Case (n = 2):

Induction Step: Assume P(i) for some i  2 and prove P(i + 1):

Assume is divisible by 6

Prove is divisible by 6.

Divisible by 6 Divisible by 2
by induction by case analysis
 Proving an Inequality

Base Case (n = 3):

Induction Step: Assume P(i) for some i  3 and prove P(i + 1):

Assume , prove

by induction

since i >= 3
 Proving an Inequality

Base Case (n = 2): is true

Induction Step: Assume P(i) for some i  2 and prove P(i + 1):

by induction
 This Lecture

• The idea of mathematical induction

• Basic induction proofs (e.g. equality, inequality, property,etc)

• An interesting example

• A paradox
 Puzzle

Goal: tile the squares, except one in the middle for Bill.

n
2

n
2
 Puzzle

There are only L-shaped tiles covering three squares:

For example, for 8 x 8 puzzle might tile for Bill this way:
 Puzzle

Theorem: For any 2n x 2n puzzle, there is a tiling with Bill in the middle.

Did you remember that we proved is divisble by 3?

Proof: (by induction on n)

P(n) ::= can tile 2n x 2n with Bill in middle.

Base case: (n=0)

(no tiles needed)

 Puzzle

Induction step: assume can tile 2n x 2n,

prove can handle 2n+1 x 2n+1.

n
2 Now
2 n+1 what??
 Puzzle

The new idea: A stronger property

Prove that we can always find a tiling with Bill anywhere.

Theorem B: For any 2n x 2n puzzle, there is a tiling with Bill anywhere.

Clearly Theorem B implies Theorem.

Theorem: For any 2n x 2n puzzle, there is a tiling with Bill in the middle.
 Puzzle

Theorem B: For any 2n x 2n puzzle, there is a tiling with Bill anywhere.

Proof: (by induction on n)

P(n) ::= can tile 2n x 2n with Bill anywhere.

Base case: (n=0)

(no tiles needed)

 Puzzle

Induction step:
Assume we can get Bill anywhere in 2n x 2n.
Prove we can get Bill anywhere in 2n+1 x 2n+1.
 Puzzle

Induction step:
Assume we can get Bill anywhere in 2n x 2n.
Prove we can get Bill anywhere in 2n+1 x 2n+1.

n
2

n
2
 Puzzle

Method: Now group the squares together,

and fill the center with a tile.

Done!
 Some Remarks

Note 1: It may help to choose a stronger hypothesis

than the desired result (e.g. “Bill in anywhere”).

Note 2: The induction proof of “Bill in corner” implicitly

defines a recursive procedure for finding corner tilings.

Note 3: Induction and recursion are very similar in spirit,

always tries to reduce the problem into a smaller problem.
 Gray Code

Can you find an ordering of all the n-bit strings in such a way that
two consecutive n-bit strings differed by only one bit?

This is called the Gray code and has many applications.

How to construct them? Think inductively! (or recursively!)

2 bit 3 bit

00 000 Can you see the pattern?

01 001
11 011 How to construct 4-bit gray code?
10 010
110
111
101
100
 Gray Code

4 bit
3 bit 3 bit (reversed)
0 000
000 100 0 001
001 101 0 011 differed by 1 bit
011 111 0 010 by induction
010 110 0 110
110 010 0 111
111 011 0 101
101 001 0 100 differed by 1 bit
100 000 1 100 by construction
1 101
1 111
1 110
Every 4-bit string appears exactly once. 1 010 differed by 1 bit
1 011 by induction
1 001
1 000
 Gray Code

n+1 bit
n bit n bit (reversed)
0 000…0
000…0 100…0 0…
… … 0… differed by 1 bit
… … 0… by induction
… … 0…
… … 0…
… … 0…
… … 0 100…0 differed by 1 bit
100…0 000…0 1 100…0 by construction
1…
1…
Every (n+1)-bit string appears exactly once. 1…
1… differed by 1 bit
1… by induction
So, by induction, 1…
Gray code exists for any n. 1 000…0
 Hadamard Matrix (Optional)

Can you construct an nxn matrix with all entries +-1 and
all the rows are orthogonal to each other?

Two rows are orthogonal if their inner product is zero.

That is, let a = (a1, …, an) and b = (b1, …, bn),

their inner product ab = a1b1 + a2b2 + … + anbn

This matrix is famous and has many applications.

To think inductively, first we come up with small examples.

1 1
1 -1
 Hadamard Matrix (Optional)

Then we use the small examples to build larger examples.

Suppose we have an nxn Hadamard matrix Hn.

We can use it to construct an 2nx2n Hadamard matrix as follows.

Hn Hn
Check this!
Hn -Hn

So by induction there is a 2k x 2k Hardmard matrix for any k.

Inductive Construction

This technique is very useful.

We can use it to construct:

- codes
- graphs
- matrices
- circuits
- algorithms
- designs
- proofs
- buildings
-…
 This Lecture

• The idea of mathematical induction

• Basic induction proofs (e.g. equality, inequality, property,etc)

• An interesting example

• A paradox
 Paradox

Theorem: All horses are the same color.

Proof: (by induction on n)

Induction hypothesis:
P(n) ::= any set of n horses have the same color
Base case (n=0):
No horses so obviously true!

…
 Paradox

(Inductive case)
Assume any n horses have the same color.
Prove that any n+1 horses have the same color.

…
n+1
 Paradox

(Inductive case)
Assume any n horses have the same color.
Prove that any n+1 horses have the same color.

Second set of n horses have the same color

First set of n horses have the same color

 Paradox

(Inductive case)
Assume any n horses have the same color.
Prove that any n+1 horses have the same color.

Therefore the set of n+1 have the same color!

 Paradox

What is wrong? n =1

Proof that P(n) → P(n+1)

is false if n = 1, because the two
horse groups do not overlap.

Second set of n=1 horses

First set of n=1 horses

(But proof works for all n ≠ 1)

 Quick Summary

You should understand the principle of mathematical induction well,

and do basic induction proofs like

• proving equality

• proving inequality

• proving property

Mathematical induction has a wide range of applications in computer science.

In the next lecture we will see more applications and more techniques.
 This Lecture

We will continue our discussions on mathematical induction.

The new elements in this lecture are a few variants of induction:

• Strong induction

• Well Ordering Principle

• Invariant Method
 Unstacking Game

• Start: a stack of boxes

• Move: split any stack into two stacks of sizes a,b>0

• Scoring: ab points

• Keep moving: until stuck

• Overall score: sum of move scores

a+b a b
 Unstacking Game

What is the best way to play this game?

Suppose there are n boxes.

What is the score if we just take the box one at a time?

n n-1 1
 Unstacking Game

What is the best way to play this game?

Suppose there are n boxes. 2n n n

What is the score if we cut the stack into half each time?

Say n=8, then the score is 1x4x4 + 2x2x2 + 4x1 = 28

Not better
than the first
first round second third
strategy!

Say n=16, then the score is 8x8 + 2x28 = 120

 Unstacking Game

Claim: Every way of unstacking gives the same score.

n (n - 1)
Claim: Starting with size n stack, final score will be
2
Proof: by Induction with Claim(n) as hypothesis

Base case n = 0:

0(0  1)
score = 0 
2

Claim(0) is okay.
 Unstacking Game

Inductive step. assume for n-stack,

and then prove C(n+1):

(n  1)n
(n+1)-stack score =
2

Case n+1 = 1. verify for 1-stack:

score = 0  1(1  1)
2

C(1) is okay.
 Unstacking Game

Case n+1 > 1. So split into an a-stack and b-stack,

where a + b = n +1.

(a + b)-stack score = ab + a-stack score + b-stack score

by induction:

a-stack score =
a (a - 1)
2
b-stack score = b (b  1)
2
 Unstacking Game

(a + b)-stack score = ab + a-stack score + b-stack score

a (a  1) b (b  1)
ab   
2 2

(a  b )((a  b )  1) (n  1)n

2 2

so C(n+1) is okay. We’re done!

 Induction Hypothesis

Wait: we assumed C(a) and C(b) where 1  a, b  n.

But by induction can only assume C(n)

the fix: revise the induction hypothesis to

In words, it says that we

Q (n ) :: assume the claim is true
m  n . C (m ) for all numbers up to n.

Proof goes through fine using Q(n) instead of C(n).

So it’s OK to assume C(m) for all m  n to prove C(n+1).

 Strong Induction

Strong induction Prove P(0).

Then prove P(n+1) assuming all of

P(0), P(1), …, P(n) (instead of just P(n)).
equivalent

Conclude n.P(n)

Ordinary induction
0  1, 1  2, 2  3, …, n-1  n.

So by the time we got to n+1, already know all of

P(0), P(1), …, P(n)

The point is: assuming P(0), P(1), up to P(n), it is often easier to prove P(n+1).
 Divisibility by a Prime
Theorem. Any integer n > 1 is divisible by a prime number.

• Let n be an integer.
Remember this slide?
• If n is a prime number, then we are done.
Now we can prove it
• Otherwise, n = ab, both are smaller than n.
by strong induction
• If a or b is a prime number, then we are done.
very easily. In fact
• Otherwise, a = cd, both are smaller than a.
we can prove a
• If c or d is a prime number, then we are done.
stronger theorem
• Otherwise, repeat this argument, since the numbers are
very easily. getting smaller and smaller, this will eventually stop and
we have found a prime factor of n.

Idea of induction.
 Prime Products

Claim: Every integer > 1 is a product of primes.

Proof: (by strong induction)

• Base case is easy.
• Suppose the claim is true for all 2 <= i < n.
• Consider an integer n.
• If n is prime, then we are done.
• So n = k·m for integers k, m where n > k,m >1.
• Since k,m smaller than n,
• By the induction hypothesis, both k and m are product of primes

k = p1 p2   p94

m = q1 q2   q214
 Prime Products

Claim: Every integer > 1 is a product of primes.

…So

n = k m = p1 p2   p94 q1 q2   q214

is a prime product.

 This completes the proof of the induction step.

 Postage by Strong Induction

Available stamps:

5¢ 3¢

What amount can you form?

Theorem: Can form any amount  8¢

Prove by strong induction on n.

P(n) ::= can form (n +8)¢.
 Postage by Strong Induction

Base case (n = 0):

(0 +8)¢:

Inductive Step: assume (m +8)¢ for 0 m  n,

then prove ((n +1) + 8)¢

cases:

n +1= 1, 9¢:

n +1= 2, 10¢:
 Postage by Strong Induction

case n +1  3: let m =n  2.

now n  m  0, so by induction hypothesis have:

+ = (n +1)+8

(n 2)+8 We’re done!

In fact, use at most two 5-cent stamps!

 Postage by Strong Induction

Given an unlimited supply of 5 cent and 7 cent stamps,

what postages are possible?

Theorem: For all n >= 24,

it is possible to produce n cents of postage from 5¢ and 7¢ stamps.

 This Lecture

• Strong induction

• Well Ordering Principle

• Invariant Method
 Well Ordering Principle

Every nonempty set ofnonnegative integers

Axiom
has a least element.

This is an axiom equivalent to the principle of mathematical induction.

Note that some similar looking statements are not true:

Every nonempty set of nonnegative rationals

NO!
has a least element.

Every nonempty set of nonnegative integers

NO!
has a least element.
 Well Ordering Principle

Thm: 2 is irrational
m
Proof: suppose 2
n
…can always find such m, n without common factors…

why always?

m
By WOP,  minimum |m| s.t. 2 .
n
m0
so 2 where |m0| is minimum.
n0
 Well Ordering Principle

but if m0, n0 had common factor c > 1, then

m0 / c
2
n0 / c

and m0 / c  m0 contradicting minimality of |m0|

The well ordering principle is usually used in “proof by contradiction”.

• Assume the statement is not true, so there is a counterexample.

• Choose the “smallest” counterexample, and find a even smaller counterexample.

• Conclude that a counterexample does not exist.

 Well Ordering Principle in Proofs

To prove ``n N. P(n)’’ using WOP:

1. Define the set of counterexamples

C ::= {n   | ¬P(n)}
2. Assume C is not empty.

3. By WOP, have minimum element m0  C.

4. Reach a contradiction (somehow) –

usually by finding a member of C that is < m0 .

5. Conclude no counterexamples exist. QED

 Non-Fermat Theorem

It is difficult to prove there is no positive integer solutions for

Fermat’s theorem

But it is easy to prove there is no positive integer solutions for

Non-Fermat’s theorem

Hint: Prove by contradiction using well ordering principle…

 Non-Fermat Theorem

Suppose, by contradiction, there are integer solutions to this equation.

By the well ordering principle, there is a solution with |a| smallest.

In this solution, a,b,c do not have a common factor.

Otherwise, if a=a’k, b=b’k, c=c’k,

then a’,b’,c’ is another solution with |a’| < |a|,

contradicting the choice of a,b,c.

(*) There is a solution in which a,b,c do not have a common factor.

 Non-Fermat Theorem

On the other hand, we prove that every solution must have a,b,c even.

This will contradict (*), and complete the proof.

First, since c3 is even, c must be even. (because odd power is odd).

Let c = 2c’, then

 Non-Fermat Theorem

Since b3 is even, b must be even. (because odd power is odd).

Let b = 2b’, then

Since a3 is even, a must be even. (because odd power is odd).

There a,b,c are all even, contradicting (*)