Basics
Basics
• Cooking rice
• In worst case, with each query, we get the larger of the two
parts.
S = 27
• prefix_sum[0] ← A[0];
for (i ← 1 to n-1)
prefix_sum[i] = prefix_sum[i-1]+A[i];
• Any subarray sum from i to j can now be computed in O(1) time as
prefix_sum[j] - prefix_sum[i-1].
• Now, for each choice of starting point, do a binary search for the minimum
end point such that the subarray sum is at least S.
• If sum of a subarray < S, then choose a larger end point, otherwise smaller.
Exercises
• Given two sorted arrays of size n, find the median of
the union of the two arrays.
O(log n) time?
• Land redistribution:
given list of landholdings a1, a2, …, an,
given a floor value f,
find the right ceiling value c
Division algorithm
• Yes.
More applications
• Finding inverse of an increasing function?
• No.
• Yes.
• And see how this number grows as a function of the input size.
This measure is independent of the choice of the machine.
O(·) notation
• For input size n, running time f(n)
• 2n+3 is O(n2)
• True
• 12 +22 +….+(n-1)2 +n2 is O(n2)
• False (it is Θ (n3))
• 1 + 1/2 + 1/3 + … + 1/n is O(log n)
• True
• nn is O(2n)
• False
O(·) notation
• True or False?
• 23n is O(2n)
• False
• (n+1)3 is O(n3)
• True
• (n+ √n)2 is O(n2)
• True
• log(n3) is O(log n)
• True
Principles of algorithm design
First principle: reducing to a
subproblem
• Subproblem: same problem on a smaller input
• Iterative implementation:
Go over the array from 1 to n and maintain a variable min
• Invariant: after seeing i numbers, min will be the
minimum among first i numbers.
• mini = minimum( mini-1, A[i] )
First principle: reducing to a
subproblem
• mini = minimum( mini-1, A[i] )
• Iterative implementation
• Recursive implementation
f(A, i):
min ← A[1];
if i=1 return A[1];
for (i ← 2 to n)
else return minimum( f(A, i-1), A[i]);
min ← minimum(min, A[i]) Compute f(A, n);
Maximum subarray sum
• Given an integer array with positive/negative numbers.
Find the subarray with maximum possible sum.
}
Sum(1 … n),
Sum(2 … n),
⋮
Sum(n … n), )
O(n)
T(n) = T(n-1) + O(n) ⇒ T(n) = O(n2)
Max subarray sum: subproblem
• Improvement ?
• Observation:
Sum(1 … n) = Sum(1 … n-1) +A[n],
Sum(2 … n) = Sum(2 … n-1) +A[n],
⋮
Sum(n-1 … n) = Sum(n-1 … n-1) +A[n],
• Observation:
Sum(1 … n) = Sum(1 … n-1) +A[n],
Sum(2 … n) = Sum(2 … n-1) +A[n],
⋮
Sum(n-1 … n) = Sum(n-1 … n-1) +A[n],
• Observation:
Sum(1 … n) = Sum(1 … n-1) +A[n],
Sum(2 … n) = Sum(2 … n-1) +A[n],
⋮
Sum(n-1 … n) = Sum(n-1 … n-1) +A[n], Push it to the
subproblem
• Maximum( Sum(1 … n), Sum(2 … n), …, Sum(n-1 … n) )
=
Maximum( Sum(1 … n-1), Sum(2 … n-1), …, Sum(n-1 … n-1)) + A[n]
• max_suffix_sumn-1 is defined as
Maximum(Sum(1 … n-1), Sum(2 … n-1), … Sum(n-1 … n-1))
• = Maximum( max_sumn-1 ,
max_suffix_sumn-1 + A[n],
A[n] )
Asking subproblem to do more
• Subproblem: max_sumn-1 and max_suffix_sumn-1
• max_sumn = Maximum( max_sumn-1 ,
max_suffix_sumn-1 + A[n],
A[n] )
• We are asking the subproblem to compute max_suffix_sum
for size n-1
So, we also need to compute max_suffix_sum for size n
• max_suffix_sumn = ?
• max_suffix_sumn = Maximum( max_suffix_sumn-1 + A[n],
A[n] )
• (5, 12), (3, 8), (8, 12), (11, 16), (9, 20), (15, 17), (7, 15), (2, 13)
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Interval containment
• Given a set of intervals
count the number of intervals which are not contained in any other
interval.
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Subproblem
• Suppose we have a solution for first n-1 intervals.
• (5, 12), (3, 8), (8, 12), (11, 16), (9, 20), (15, 17), (7, 15), (2, 13)
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Interval containment
• When we introduce the nth interval
need to check if it is contained in any other interval
or if it contains other intervals. Seems to take O(n) time.
• (5, 12), (3, 8), (8, 12), (11, 16), (9, 20), (15, 17), (7, 15), (2, 13)
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Reordering input
• Can we reorder the intervals so that the work at the last step reduces?
• (2, 13), (3, 8), (5, 12), (7, 15), (8, 12), (9, 20), (11, 16), (15, 17),
• (3, 8), (5, 12), (8, 12), (2, 13), (7, 15), (11, 16), (15, 17), (9, 20),
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Increasing finish time
• Can we reorder the intervals so that the work at the last step reduces?
• (3, 8), (5, 12), (8, 12), (2, 13), (7, 15), (11, 16), (15, 17), (9, 20),
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Increasing finish time
• Can we reorder the intervals so that the work at the last step reduces?
• (3, 8), (5, 12), (8, 12), (2, 13), (7, 15), (11, 16), (15, 17), (9, 20),
• (2, 13), (7, 15), (11, 16), (15, 17), (9, 20),
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Increasing start time
• Consider increasing start time
• (2, 13), (3, 8), (5, 12), (7, 15), (8, 12), (9, 20), (11, 16), (15, 17),
• Same as whether any previous interval finishes after the last one.
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Algorithm
• Maintain the highest finish time among interval seen so far
• number_of_maximal_intervals ++ ;
• largest_finish_time ← fi
• Reducing to a subproblem
h2
h3 h4
• Updated outline: (0, 5), (2, 6), (3, 3), (5, 7), (6, 9), (8, 4), (9, 0)
0 1 2 3 4 5 6 7 8 9
Divide and conquer approach
• Divide the set of rectangles into two parts with n/2
rectangles each.
h2
h3 h4
`
Merge two outlines
• Outline 1: (0, 2), (4, 6), (6, 3), (7, 2), (11, 0)
• Merged: (0, 2), (2, 4), (4, 6), (6, 4), (8, 2), (11, 0)
0 1 2 3 4 5 6 7 8 9 10 11
Merge two outlines
• Outline 1: (x1, y1), (x2, y2), (x3, y3), …, (xm, 0)
• Outline 2: (a1, b1), (a2, b2), (a3, b3), …, (ak, 0)
• Pointers for two queues i and j.
• Maintain the current height in each outline
height_1, height_2
• If xi < aj then
• height_1 ← yi
• Push (xi , max ( height_1, height_2 ) )
• i←i+1
• Else is similar
• Final clean up: remove consecutively repeating heights in the merged
outline
Alternative implementation
• Maintain the current height in each outline
height_1, height_2
• If xi < aj then
• height_1 ← yi
• height_to_push ← max ( height_1, height_2 )
• If (last_pushed_height ≠ height_to_push )
• Push (xi , height_to_push )
• last_pushed_height ← height_to_push
• i←i+1
• Else is similar
Other approaches
• Divide and conquer with respect to heights?
• Approaches without divide and conquer
• Reordering the input
• Increasing order of left end points: O(n log n) time
implementation possible using a data structure like
balanced binary tree.
• Decreasing order of heights: O(n log n) time
implementation possible using a data structure like
balanced binary tree or heap.
• O(n log n) necessary ?
Significant inversion
• Given an array A of integers
• Alternate approach.
Significant inversion
• a pair (i, j) is called a significant inversion
if i < j and A[i] > 2A[j].
• For any j, count how many are > 2A[j] among first j-1
numbers.
5 45
2 19 33 59
4 12 23 41 53 64
8 15 37
Counting in BST
• Counting number of nodes > x seems to take O(n)
time in the standard BST.
5 45 +1+7
8 7
+1+1
2 19 33 59
2 5 3 3
1+1
4 12 23 41 53 64
1 2 1 1
1 3
8 15 37
1 1
1
Counting in BST
• Store for each node x, the size of the subtree rooted at x
• Can we maintain this information on insertion of a new
element?
• Need to update the counts for all nodes on the path from
the root to the new node.
• O(log n) for balanced binary tree.
• Can counts be maintained in the re-balance step?
• Can you do range count: #nodes in range l to r
• Other data structures?
Algorithm: Significant
inversion
• Maintain a balanced BST. Initially empty.
For each node x, we need to store size of the subtree rooted at x.
• no_of_pairs ← 0;
• For j ← 1 to n
• Look for the right position p for 2A[j] in the BST (no insertion).
• Insert A[j] in the BST and increase the size count for every node on the
path from root to A[j] by 1
Exponentiation
• Given a and n, compute an.
• a×a×…×a n times
• n-1 multiplications
• If n is 1
• return a
Another implementation
• Exp(a, n) :
• If n is even
• If n is odd
• If n is 1
• return a
Iterative implementation
• Input: a, n
• Initialize
a_power_n ← 1; // this will be an at the end
a_two_power ← a; // this will be a2^i after i iterations
• while (n > 0)