Fibonacci numbers and Recursion

Announcements

• Reading Quizzes:
– Out of 17
– Grades are going to be updated
– You still have until MidTerm

• Project 2 is out

CSC 172, Fall 2017

 Agenda

• The worst algorithm in the history of

humanity

• An iterative solution

• A better iterative solution

• The repeated squaring trick

CSC 172, Fall 2017

 And the worst algorithm in the history of humanity

FIBONACCI SEQUENCE

CSC 172, Fall 2017

 Fibonacci sequence

• 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, …

• F[0] = 0
• F[1] = 1
• F[2] = F[1] + F[0] = 1
• F[3] = F[2] + F[1] = 2
• F[4] = F[3] + F[2] = 3
• F[n] = F[n-1] + F[n-2]

CSC 172, Fall 2017

• http://www.youtube.com/watch?v=aB_KstBio
u4

CSC 172, Fall 2017

 Recursion – fib1()

/**
*----------------------------------------------------------
* the most straightforward algorithm to compute F[n]
*----------------------------------------------------------
*/
long fib1(int n) {
if (n <= 1) return n;
return fib1(n-1) + fib1(n-2);
}

CSC 172, Fall 2017

Typical Runtime

CSC 172, Fall 2017

 On large numbers

• Looks like the run time is doubled for each n++

• We won’t be able to compute F[140] if the trend

continues

• The age of the universe is 15 billion years < 260

sec

• The function looks … exponential

– Is there a theoretical justification for this?

CSC 172, Fall 2017

 A Note on “Functions”
• Sometimes we mean a Java function

• Sometimes we mean a mathematical function like

F[n]

• A Java function can be used to compute a

mathematical function
– But not always! There are un-computable functions
– Google for “busy Beaver numbers” and the “halting
problem”, for typical examples.

• What we mean should be clear from context

CSC 172, Fall 2017

 Guess and induct strategy

Thinking about the main body

ANALYSIS OF FIB1()

CSC 172, Fall 2017

 Guess and induct
• For n > 1, suppose it takes c mili-sec in
fib1(n) not counting the recursive calls
• For n=0, 1, suppose it takes d mili-sec
• Let T[n] be the time fib1(n) takes
• T[0] = T[1] = d
• T[n] = c + T[n-1] + T[n-2]
when n > 1

• To estimate T[n], we can

– Guess a formula for it
– Prove by induction that it works

CSC 172, Fall 2017

 The guess

• Bottom-up iteration
– T[0] = T[1] = d
– T[2] = c + 2d
– T[3] = 2c + 3d
– T[4] = 4c + 5d
– T[5] = 7c + 8d
– T[6] = 12c + 13d

• Can you guess a formula for T[n]?

– T[n] = (F[n+1] – 1)c + F[n+1]d

CSC 172, Fall 2017

 The Proof

• The base cases: n=0,1

— The hypothesis: suppose
— T[m] = (F[m+1] – 1)*c + F[m+1]*d for all m < n
— The induction step:
— T[n] = c + T[n-1] + T[n-2]
= c + (F[n] – 1)*c + F[n]*d
+ (F[n-1] – 1)*c + F[n-1]*d
= (F[n+1] – 1)*c + F[n]*d

CSC 172, Fall 2017

 How does this help?

n n
( 1/ )
F [n] = p
5
p
1+ 5
= ⇡ 1.6
2

The golden ratio

CSC 172, Fall 2017

So, there are constants C, D such that

n n
C  T [n]  D

This explains the exponential-curve we saw

𝑇 𝑛 = Θ 𝜙&

CSC 172, Fall 2017

 From intuition to formality

• Suppose fib1(140) runs on a computer

with
C = 10-9:
9
10 (1.6)140 3.77 · 1019 > 100 · age of univ.

CSC 172, Fall 2017

 - A Linear time algorithm using ArrayList

- A linear time algorithm using arrays

- A linear time algorithm with constant space

BETTER ALGORITHMS FOR

COMPUTING F[N]

CSC 172, Fall 2017

 An algorithm using ArrayList

long fib2(int n) {
// this is one implementation option
if (n <= 1) return n;
ArrayList<Long> A = new ArrayList<>();
A.add(new Long(0)); A.add(new Long(1));
for (int i=2; i<=n; i++) {
A.add(A.get(i-1)+ A.get(i-2));
}
return A.get(n);
}

Guess how large an n we can handle this time?

CSC 172, Fall 2017
 Data

n 106 107 108 109

# seconds 1 1 9 Eats up all
my
CPU/RAM

CSC 172, Fall 2017

 How about an array?

long fib2(int n) {
if (n <= 1) return n;

long[] A = new long[n+1];

A[0] = 0; A[1] = 1;
for (int i=2; i<=n; i++) {
A[i] = A[i-1]+A[i-2];
}
return A[n];
}

Guess how large an n we can handle this time?

CSC 172, Fall 2017
 Data

n 106 107 108 109

# seconds 1 1 1 Stack Overflow

Data structure matters a great deal!

Some assumptions we made are false if too

much space is involved: computer has to use
hard-drive as memory

CSC 172, Fall 2017

 Dynamic programming!

long fib3(int n) {
if (n <= 1) return n;
long a=0, b=1, temp = 0;
for (int i=2; i<= n; i++) {
temp = a + b; // F[i] = F[i-2] + F[i-1]
a = b; // a = F[i-1]
b = temp; // b = F[i]
}
return temp;
}

Guess how large an n we can handle this time?

CSC 172, Fall 2017
 Data

n 108 109 1010 1011

# seconds 1 3 35 359

The answers are incorrect because F[108] is

greater than the largest integer representable
by int

But that’s ok. We want to know the runtime

CSC 172, Fall 2017

 - The repeated squaring trick

AN EVEN FASTER ALGORITHM

CSC 172, Fall 2017

 Math helps!

• We can re-formulate the problem a little:

  
1 1 1 1
=
1 0 0 1
  
1 1 1 2
=
1 0 1 1
  
1 1 2 3
=
1 0 1 2

CSC 172, Fall 2017

  
1 1 F [n 1] F [n]
=
1 0 F [n 2] F [n 1]

  n 
F [n + 1] 1 1 1
=
F [n] 1 0 0

CSC 172, Fall 2017

 How to we compute An quickly?

• Want
 n
1 1
1 0

• But can we even compute 3n quickly?

CSC 172, Fall 2017

 First algorithm

long power1(int n, int base) {

long ret=1;
for (int i=0; i<n; i++)
ret *= base;
return ret;
}

When n = 1010 it took 44 seconds

CSC 172, Fall 2017

 Second algorithm

long power2(int n) {
long ret;
if (n == 0) return 1;
if (n % 2 == 0) {
ret = power2(n/2);
return ret * ret;
} else {
ret = power2((n-1)/2);
return base * ret * ret;
}
}

When n = 1019 it took < 1 second

CSC 172, Fall 2017

 Runtime analysis

• First algorithm O(n)

• Second algorithm O(log n)

• We can apply the second algorithm to the

Fibonacci problem: fib4() has the following
data
n 108 109 1010 1019
# seconds 1 1 1 1

CSC 172, Fall 2017

 Conclusion

• Recursion is
– powerful!

CSC 172, Fall 2017

Source: https://i.ytimg.com/vi/Y2kenVSIV8U/maxresdefault.jpg
 Recursion
Examples: From Textbook
1. Largest Number / Cummulative Sum
2. Greatest common divisior (GCD)
3. Log
4. Power
5. Many problems on List ADT

All these problems can be solved using either recursive or

iterative algorithms.

CSC 172, Fall 2017

 Famous Quotations

• To err is human, to forgive divine.

• Alexander Pope, An Essay on Criticism,
• English poet and satirist (1688 - 1744)

• To iterate is human, to recurse, divine.

• L. Peter Deutsch, computer scientist, or ....
• Robert Heller, computer scientist, or ....
• unknown ....

CSC 172, Fall 2017

 Objectives

• Thinking recursively

• Tracing execution of a recursive method

• Writing recursive algorithms

• Towers of Hanoi problem with recursion

• Backtracking to solve search problems, as in

mazes
RECURSIVE THINKING
 Recursive Thinking

• Recursion is:
– A problem-solving approach, that can ...
– Generate simple solutions to ...
– Certain kinds of problems that ...
– Would be difficult to solve in other ways

• Recursion splits a problem:

• Into one or more simpler versions of itself
 Recursive Thinking (cont.)

General Recursive Algorithm

if the problem can be solved directly for the current value

of n
Solve it
else
Recursively apply the algorithm to one or more
problems involving smaller values of n

Combine the solutions to the smaller problems to get the

solution to the original problem
 Examples

38
 Recursive Algorithm for Finding the
Length of a String

if the string is empty (has no characters)

The length is 0
else
3. The length is 1 plus the length of the
string that excludes the first character
Recursive Algorithm for Finding the
Length of a String (cont.)

public static int size(String str) {

if ( str == null || str.equals(""))
return 0;
else {
int output = 1 + size(str.substring(1));
return output;
}
}
Tracing a Recursive function
Recursive Algorithm for Printing String
Characters

public static void print_chars(String str) {

if (str == null || str.equals("")) {
return;
} else {
System.out.println(str.charAt(0));
print_chars(str.substring(1));
}
}
Recursive Algorithm for Printing String
Characters in Reverse Order

public static void print_chars_rev(String str) {

if (str == null || str.equals("")) {
return;
} else {
print_chars_rev(str.substring(1));
System.out.println(str.charAt(0));
}
}
 Recursive Design Example: mystery

• What does this do?

int mystery (int n) {

if (n == 0)
return 0;
else
return n * mystery(n-1);
}

44
 Proving a Recursive Method Correct

• Recall Proof by Induction

• Prove the theorem for the base case(s): n=0
• Show that:
– If the theorem is assumed true for n,
• Then it must be true for n+1
• Result: Theorem true for all n ≥ 0

45
 Proving a Recursive Method Correct (2)

• Recursive proof is similar to induction:

Show base case recognized and solved correctly

Show that
• If all smaller problems are solved correctly,
• Then original problem is also solved correctly

Show that each recursive case makes progress towards the

base case è terminates properly

46
 The Stack and Activation Frames

• Java maintains a stack on which it saves new

information in the form of an activation
frame
• The activation frame contains storage for
– function arguments
– local variables (if any)
– the return address of the instruction that called
the function
• Whenever a new function is called
(recursive or otherwise), Java pushes a new
activation frame onto the stack
Run-Time Stack and Activation Frames
(cont.)
Run-Time Stack and Activation Frames
 Section 7.2

RECURSIVE DEFINITIONS OF
MATHEMATICAL FORMULAS
 Recursive Definitions of Mathematical
Formulas

• Mathematicians often use recursive

definitions of formulas that lead
naturally to recursive algorithms
• Examples include:
– factorials
– powers
– greatest common divisors (gcd)
Factorial of n: n!
 Factorial of n: n! (cont.)

The recursive definition

can be expressed by the
following algorithm:
if n equals 0
n! is 1
else
n! = n * (n – 1)!
The last step can be
implemented as:
return n * factorial(n
– 1);
 Factorial of n: n! (cont.)

int factorial(int n) {
if (n == 0)
return 1;
else
return n * factorial(n – 1);
}
 Recursive Algorithm for Calculating xn
(cont.)
double power(double x, int n) {
if (n == 0)
return 1;
else if (n > 0)
return x * power(x, n – 1);
else
return 1.0 / power(x, -n);
}
Recursive Algorithm for Calculating gcd
(cont.)

int gcd(int m, int n) {

if (m < n)
return gcd(n, m); // Transpose arguments
else if (m % n == 0)
return n;
else
return gcd(n, m % n);
}
 Recursion Versus Iteration

Recursion and iteration are similar

Iteration
A loop repetition condition determines whether to repeat the loop
body or exit from the loop
Recursion
the condition usually tests for a base case

You can always write an iterative solution to a problem

that is solvable by recursion
BUT
A recursive algorithm may be simpler than an iterative
algorithm and thus easier to write, code, debug, and read
 Tail Recursion or Last-Line Recursion

• When recursion involves single call that is at the end

...
• It is called tail recursion and it easy to make iterative

int factorial(int n) {
if (n == 0)
return 1;
else
return n * factorial(n – 1);
}
• It is a straightforward process to turn such a function
into an iterative one

58
 Iterative factorial function

int factorial_iter(int n) {
int result = 1;
for (int k = 1; k <= n; k++)
result = result * k;
return result;
}
 Efficiency of Recursion

Recursive method often slower than iterative;

why?
Overhead for loop repetition smaller than
Overhead for call and return
If easier to develop algorithm using
recursion,
Then code it as a recursive method:
Software engineering benefit probably outweighs
...
Reduction in efficiency
Don’t “optimize” prematurely!
 Efficiency of Recursion (cont.)

• Memory usage
– A recursive version can require significantly
more memory that an iterative version
because of the need to save local variables
and parameters on a stack
Fibonacci Numbers (cont.)

Inefficient
 An O(n) Recursive fibonacci function

int fibonacci_start (int n) {

return fibo(1, 0, n);
}
int fibo (int curr, int prev, int n) {
if (n <= 1)
return curr;
else
return fibo(curr+prev, curr, n-1);
}
Efficiency of Recursion: O(n) fibonacci

Efficient
 Examples

• Towers of Hanoi

65
 Towers of Hanoi

• Move the three disks to a different peg,

maintaining their order (largest disk on
bottom, smallest on top, etc.)
– Only the top disk on a peg can be moved to
another peg
– A larger disk cannot be placed on top of a
smaller disk
 Let’s Play

• http://www.mathsisfun.com/games/towerofha
noi.html

67
Problem Inputs and Outputs
Recursive Algorithm for Towers of Hanoi

Recursive Algorithm for n-Disk Problem: Move n

Disks from the Starting Peg to the Destination
Peg
1.if n is 1
2. Move disk 1 (the smallest disk) from the starting peg to
the destination peg
3.else
4. Move the top n – 1 disks from the starting peg to the
temporary peg (neither starting nor destination peg)
5. Move disk n (the disk at the bottom) from the starting
peg to the destination peg
6. Move the top n – 1 disks from the temporary peg to the
destination peg
Recursive Algorithm for Towers of Hanoi
(cont.)
 Section 7.5

BACKTRACKING
72
 Backtracking

• Backtracking:
– Systematic trial and error search for solution to a
problem
– Example:Finding a path through a maze
• In walking through a maze, probably walk a path
as far as you can go
– Eventually, reach destination or dead end
– If dead end, must retrace your steps
– Loops: stop when reach place you’ve been before
• Backtracking systematically tries alternative paths
and eliminates them if they don’t work
 Backtracking(2)

• If you never try exact same path more than once,

and You try all possibilities
– You will eventually find a solution path if one exists
• Problems solved by backtracking: a set of choices
• Recursion implements backtracking
straightforwardly
– Activation frame remembers choice made at that
decision point
• A chess playing program likely involves
backtracking

74
 Finding a Path through a Maze

• Problem
– Use backtracking to find a display the path
through a maze
– From each point in a maze you can move to
the next cell in a horizontal or vertical
direction if the cell is not blocked
Analysis (cont.)