0% found this document useful (0 votes)
51 views

Recursion: EECS 281: Data Structures & Algorithms

Here are the parameters: a = 2 b = 4 c = 1/2 This falls under the third condition where a < bc. Therefore, T(n) is Θ(n1/2).

Uploaded by

Tian Xie
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
51 views

Recursion: EECS 281: Data Structures & Algorithms

Here are the parameters: a = 2 b = 4 c = 1/2 This falls under the third condition where a < bc. Therefore, T(n) is Θ(n1/2).

Uploaded by

Tian Xie
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 34

Lecture 5

Recursion

EECS 281: Data Structures & Algorithms


Recursion Basics
• A recursive function calls itself to
accomplish its task
• At least one base case is required
• At least one recursive case is required
• Each subsequent call has simpler
(smaller) input
• Single recursion can be used on lists
• Multiple recursion can be used on trees
2
Job Interview Question
• Implement this function
// returns x^n
int power(int x, uint32_t n);

• The obvious solution uses n - 1 multiplications


– 28 = 2*2* ... *2
• Less obvious: O(log n) multiplications
– Hint: 28 = ((22)2)2
– How does it work for 27 ?
• Write both solutions iteratively and recursively

3
Ideas
• Obvious approach uses a subproblem "one step smaller"

1 𝑛 == 0
𝑥𝑛 = ቊ
𝑥 ∗ 𝑥 𝑛−1 𝑛>0

• Less obvious approach splits the problem into two halves


1 𝑛 == 0
𝑥 𝑛 = ቐ 𝑥 𝑛Τ2 ∗ 𝑥 𝑛Τ2 𝑛 > 0, even
𝑥 ∗ 𝑥 𝑛 Τ2 ∗ 𝑥 𝑛 Τ2 𝑛 > 0, odd

4
Two Recursive Solutions
Solution #1 Solution #2
1 int power(int x, uint32_t n) { 1 int power(int x, uint32_t n) {
2 if (n == 0) 2 if (n == 0)
3 return 1; 3 return 1;
4 4
5 return x * power(x, n – 1); 5 int result = power(x, n / 2);
6 } // power() 6 result *= result;
7 if (n % 2 != 0) // n is odd
Recurrence: T(n) = T(n-1) + c
8 result *= x;
Complexity: Θ(n)
9
10 return result;
11 } // power()
Recurrence: T(n) = T(n / 2) + c
Complexity: Θ(log n) 5
Tail Recursion
• When a function is called, it gets a stack frame, which
stores the local variables
• A simply recursive function generates a stack frame for
each recursive call
• A function is tail recursive if there is
no pending computation at each
recursive step
– "Reuse" the stack frame rather
than create a new one
• Tail recursion and iteration are
equivalent

http://xkcd.com/1270/ 6
Recursion and the Stack
1 int factorial(int n) {
2 if (n == 0)
3 return 1;
4 return n * factorial(n - 1);
5 } // factorial()
6
7 int main(int, char *[]) {
8 factorial(5);
9 } // main()

7
The Program Stack (1)
• When a function call is made
1a. All local variables are saved in a special storage
called the program stack
2a. Then argument values are
pushed onto the program stack
• When a function call is received
2b. Function arguments are popped off the stack
• When return is issued within a function
3a. The return value is pushed onto the program stack
• When return is received at the call site
3b. The return value is popped off the the program stack
1b. Saved local variables are restored
8
The Program Stack (2)
• Program stack supports nested function calls
– Six nested calls = six sets of local variables

• There is only one program stack (per thread)


– NOT the program heap, (where dynamic memory is allocated)

• Program stack size is limited


– The number of nested function calls is limited

• Example: a bottomless (buggy) recursion function will


exhaust program stack very quickly
9
Recursion vs. Tail Recursion
1 int factorial(int n) {
2 if (n == 0) Recursive
3 return 1; Θ(n) time complexity
Θ(n) space complexity
4 return n * factorial(n - 1);
(uses n stack frames)
5 } // factorial()

6 int factorial(int n, int res = 1) {


Tail recursive*
7 if (n == 0)
Θ(n) time complexity
8 return res; Θ(1) space complexity
9 return factorial(n - 1, res * n); (reuses 1 stack frame)
10 } // factorial()
* The default argument is used to seed the res parameter. Alternatively, the
"helper function" pattern could be used.
10
Logarithmic Tail Recursive power()
1 int power(int x, uint32_t n, int result = 1) {
2 if (n == 0)
3 return result;
4 else if (n % 2 == 0) // even
5 return power(x * x, n / 2, result);
6 else // odd
7 return power(x * x, n / 2, result * x);
8 } // power()

Θ(log n) time complexity


Θ(1) space complexity

11
Practical Considerations
• Program stack is limited in size
– It's actually pretty easy to exhaust this!
e.g. Computing the length of a very long vector using a
"linear recursive" function with Θ(n) space complexity

• For a large data set


– ”Simple" recursion is a bad idea
– Use tail recursion or iterative algorithms instead

• This doesn't mean everything should be tail recursive


– Some problems can't be solved in Θ(1) space!
12
Recurrence Relations
• A recurrence relation describes the way a problem
depends on a subproblem.
– A recurrence can be written for a computation:

𝑛 1 𝑛 == 0
𝑥 =ቊ
𝑥 ∗ 𝑥 𝑛−1 𝑛>0
– A recurrence can be written for the time taken:
𝑐0 𝑛 == 0
𝑇 𝑛 = ቊ𝑇 𝑛 − 1 + 𝑐
1 𝑛>0
– A recurrence can be written for the amount of memory used*:
𝑐0 𝑛 == 0
𝑀 𝑛 = ቊ𝑀 𝑛 − 1 + 𝑐
1 𝑛>0
*Non-tail recursive

13
A Logarithmic Recurrence Relation
𝑐0 𝑛 == 0
𝑇 𝑛 = ቐ𝑇 𝑛 + 𝑐
1
 Θ(log 𝑛)
2 𝑛>0

• Fits the logarithmic recursive implementation of


power()
– The power to be calculated is divided into two halves
and combined with a single multiplication
• Also fits Binary Search
– The search space is cut in half each time, and the
function recurses into only one half

14
Common Recurrences
Recurrence Example Big-O
Solution
T(n) = T(n / 2) + c Binary Search O(log n)
T(n) = T(n - 1) + c Linear Search O(n)
T(n) = 2T(n / 2) + c Tree Traversal O(n)
T(n) = T(n - 1) + c1 * n + c2 Selection/etc. Sorts O(n2)
T(n) = 2T(n / 2) + c1 * n + c2 Merge/Quick Sorts O(n log n)

15
Solving Recurrences
• Substitution method
1. Write out T(n), T(n - 1), T(n - 2)
2. Substitute T(n - 1), T(n - 2) into T(n)
3. Look for a pattern
4. Use a summation formula
• Another way to solve recurrence equations
is the Master Method (AKA Master
Theorem)
• Recursion tree method
16
Recurrence Thought Exercises
• What if a recurrence cuts a problem into two
subproblems, and both subproblems were
recursively processed?
• What if a recurrence cuts a problem into three
subproblems and…
– Processes one piece
– Processes two pieces
– Processes three pieces
• What if there was additional, non-constant work
after the recursion?
17
Master Theorem
Let T(n) be a monotonically increasing function
that satisfies:
𝑛
𝑇 𝑛 = 𝑎𝑇 + 𝑓(𝑛)
𝑏
𝑇 1 = 𝑐0 𝑜𝑟 𝑇 0 = 𝑐0

where a ≥ 1, b > 1. If 𝑓 𝑛 ∈ Θ 𝑛𝑐 , then:

Θ 𝑛log𝑏 𝑎 𝑖𝑓 𝑎 > 𝑏 𝑐
𝑇 𝑛 ∈ ൞Θ 𝑛𝑐 log 𝑛 𝑖𝑓 𝑎 = 𝑏 𝑐
Θ 𝑛𝑐 𝑖𝑓 𝑎 < 𝑏 𝑐
18
Exercise 1
𝑛 3
𝑇 𝑛 = 3𝑇 + 𝑛+1
2 4
What are the parameters?
a=3
b=2
c=1
Which condition?
𝑛
Θ 𝑛log𝑏 𝑎 𝑖𝑓 𝑎 > 𝑏𝑐 𝑇 𝑛 = 𝑎𝑇 + 𝑓(𝑛)
𝑇 𝑛 ∈ ൞Θ 𝑛𝑐 log 𝑛 𝑏
𝑖𝑓 𝑎 = 𝑏 𝑐
𝑓 𝑛 ∈ Θ 𝑛𝑐
Θ 𝑛𝑐 𝑖𝑓 𝑎 < 𝑏 𝑐
19
Exercise 2
𝑛
𝑇 𝑛 = 2𝑇 + 𝑛+7
4
What are the parameters?
a=2
b=4
c = 1/2
Which condition?
𝑛
Θ 𝑛log𝑏 𝑎 𝑖𝑓 𝑎 > 𝑏𝑐 𝑇 𝑛 = 𝑎𝑇 + 𝑓(𝑛)
𝑇 𝑛 ∈ ൞Θ 𝑛𝑐 log 𝑛 𝑏
𝑖𝑓 𝑎 = 𝑏 𝑐
𝑓 𝑛 ∈ Θ 𝑛𝑐
Θ 𝑛𝑐 𝑖𝑓 𝑎 < 𝑏 𝑐
21
Exercise 3
𝑛 1 2
𝑇 𝑛 =𝑇 + 𝑛 +𝑛
2 2
What are the parameters?
a=1
b=2
c=2
Which condition?
𝑛
Θ 𝑛log𝑏 𝑎 𝑖𝑓 𝑎 > 𝑏𝑐 𝑇 𝑛 = 𝑎𝑇 + 𝑓(𝑛)
𝑇 𝑛 ∈ ൞Θ 𝑛𝑐 log 𝑛 𝑏
𝑖𝑓 𝑎 = 𝑏 𝑐
𝑓 𝑛 ∈ Θ 𝑛𝑐
Θ 𝑛𝑐 𝑖𝑓 𝑎 < 𝑏 𝑐
23
When Not to Use
• You cannot use the Master Theorem if:
– T(n) is not monotonic, such as T(n) = sin(n)
– f(n) is not a polynomial, i.e. f(n) = 2n
– b cannot be expressed as a constant. i.e.

𝑇 𝑛 =𝑇 sin 𝑛
• There is also a special fourth condition if
f(n) is not a polynomial; see later in slides
25
When Not to Use
The recursion does not involve division:
𝑇 𝑛 =𝑇 𝑛−1 +𝑛

Master Theorem not applicable


𝑛
𝑇 𝑛 ≠ 𝑎𝑇 + 𝑓(𝑛)
𝑏

26
Fourth Condition
• There is a 4th condition that allows
polylogarithmic functions

If f (n)  (n log b a


log n) for some k  0,
k

k 1
Then T (n)  (n log b a
log n)

• This condition is fairly limited,


• No need to memorize/write down

27
Fourth Condition Example
• Given the following recurrence:
𝑛
𝑇 𝑛 = 2𝑇 + 𝑛 log 𝑛
2
• Clearly a=2, b=2, but f(n) is not polynomial
• However: 𝑓 𝑛 ∈ Θ 𝑛 log 𝑛 and 𝑘 = 1
𝑇 𝑛 = Θ 𝑛 log 2 𝑛

If 𝑓(𝑛) ∈ Θ 𝑛log𝑏 𝑎 log 𝑘 𝑛 for some 𝑘 ≥ 0,


Then 𝑇(𝑛) ∈ Θ 𝑛log𝑏 𝑎 log 𝑘+1 𝑛
28
Bonus: Binary Max?
• A friend of yours claims to have discovered a (revolutionary!) new
algorithm for finding the maximum element in an unsorted array:
1. Split the array into two halves.
2. Recursively find the maximum in each half.
3. Whichever half-max is bigger is the overall max!

• Your friend says this algorithm leverages the power of "binary


partitioning" to achieve better than linear time.
– This sounds too good to be true. Give an intuitive argument why.
– Use the master theorem to formally prove this algorithm is Θ(n).

29
Job Interview Question
Write an efficient algorithm that searches for
a value in an n x m table (two-dim array).
This table is sorted along
the rows and columns — that is,
1 4 7 11 15
table[i][j] ≤ table[i][j + 1],
table[i][j] ≤ table[i + 1][j] 2 5 8 12 19

• Obvious ideas: linear or 3 6 9 16 22

binary search in every row 10 13 14 17 24

– nm or n log m … too slow 18 21 23 26 30

30
Potential Solution #1: Quad Partition

• Split the region into four quadrants, eliminate one.


• Then, recursively process the other 3 quadrants.
• Write a recurrence relation for 1 4 7 11 15
the time complexity in terms of
n, the length of one side: 2 5 8 12 19

3 6 9 16 22
T(n) = 3T(n/2) + c
10 13 14 17 24
Why n/2 and not n/4 if it's a quadrant?
Remember, n is the length of one side! 18 21 23 26 30

n 31
Potential Solution #2: Binary Partition
How can we improve this?

• Split the region into four quadrants.


Use binary search!
• Scan down the middle column, until
you find "where the value should be" 1 4 7 11 15
if it were in that column.1
• This allows you to eliminate two 2 5 8 12 19
quadrants. (Why? Which ones?)
3 6 9 16 22
• Recursively process the other two.
• Write a recurrence relation, again 10 13 14 17 24
in terms of the side length n:
18 21 23 26 30

T(n) = 2T(n/2) + cn
n 32
1 Of course, you might get lucky and find the value here!
Potential Solution #3: Improved Binary Partition

• Split the region into four quadrants.


• Scan down the middle column, until
you find "where the value should be" 1 4 7 11 15
if it were in that column.1
• This allows you to eliminate two 2 5 8 12 19
quadrants. (Why? Which ones?)
3 6 9 16 22
• Recursively process the other two.
• Write a recurrence relation, again 10 13 14 17 24
in terms of the side length n:
18 21 23 26 30

T(n) = 2T(n/2) + c log n


n 33
1 Of course, you might get lucky and find the value here!
Exercise: Use the Master Theorem
• Use the master theorem to find the complexity of each approach:
• Quad Partition:

T(n) = 3T(n/2) + c
T(n) = (nlog23) ≈ (n1.58)
• Binary Partition:

T(n) = 2T(n/2) + cn
T(n) = (n log n)
• Improved Binary Partition:
T(n) = 2T(n/2) + c log n
T(n) = (n)
34
Another Solution!
Stepwise Linear Search
1 bool stepWise(int mat[][n], int m, int target, int &row, int &col) {
2 if (target < mat[0][0] || target > mat[m - 1][n - 1])
3 return false;
4 row = 0; col = n - 1; 1 4 7 11 15
5 while (row <= m - 1 && col >= 0) {
6 if (mat[row][col] < target) 2 5 8 12 19
7 ++row;
8 else if (mat[row][col] > target) m 3 6 9 16 22
9 --col;
10 13 14 17 24
10 else
11 return true;
18 21 23 26 30
12 } // while
13 return false;
14 } // stepWise()
n 35
Runtime Comparisons
• Source code and data (M = N = 100) available at
http://www.leetcode.com/2010/10/searching-2d-sorted-matrix.html
http://www.leetcode.com/2010/10/searching-2d-sorted-matrix-part-ii.html
http://www.leetcode.com/2010/10/searching-2d-sorted-matrix-part-iii.html
• Runtime for 1,000,000 searches
Algorithm Runtime
Binary search 31.62s
Diagonal Binary Search 32.46s
Step-wise Linear Search 10.71s
Quad Partition 17.33s
Binary Partition 10.93s
Improved Binary Partition 6.56s
36
Questions for Self-Study
• Consider a recursive function that only calls itself.
Explain how one can replace recursion
by a loop and an additional stack.
• Go over the Master Theorem in the CLRS textbook
• Which cases of the Master Theorem were exercised for
different solutions in the 2D-sorted-matrix problem?
• Solve the same recurrences by
substitution w/o the Master Theorem
• Write (and test) programs for each
solution idea, time them on your data

37

You might also like