6.
006- Introduction to
Algorithms
Courtesy of MIT Press. Used with permission.
Lecture 3
1
Menu
Sorting!
Insertion Sort
Merge Sort
Solving Recurrences
2
The problem of sorting
Input: array A[1n] of numbers.
Output: permutation B[1n] of A such
that B[1] B[2] B[n] .
e.g. A = [7, 2, 5, 5, 9.6] B = [2, 5, 5, 7, 9.6]
How can we do it efficiently ?
3
Why Sorting?
Obvious applications
Organize an MP3 library
Maintain a telephone directory
Problems that become easy once items are
in sorted order
Find a median, or find closest pairs
Binary search, identify statistical outliers
Non-obvious applications
Data compression: sorting finds duplicates
Computer graphics: rendering scenes front to back
4
Insertion sort
INSERTION-SORT (A, n) [ A[1 . . n]
for j 2 to n
insert key A[j] into the (already sorted) sub-array A[1 .. j-1].
by pairwise key-swaps down to its right position
Illustration of iteration j
1 i j n
A:
sorted key
new location of key
5
Example of insertion sort
8 2
4 9 3 6
6
Example of insertion sort
8 2 4 9 3 6
7
Example of insertion sort
8 2 4 9 3
6
2 8 4 9 3 6
8
Example of insertion sort
8 2 4 9 3
6
2 8 4 9 3 6
9
Example of insertion sort
8 2
4 9 3 6
2 8
4 9 3 6
2 4 8
9 3 6
10
Example of insertion sort
8 2
4 9 3 6
2 8
4 9 3 6
2 4 8
9 3 6
11
Example of insertion sort
8 2
4 9 3 6
2 8
4 9 3 6
2 4 8
9 3 6
2 4 8 9
3 6
12
Example of insertion sort
8 2
4 9 3 6
2 8
4 9 3 6
2 4 8
9 3 6
2 4 8 9
3 6
13
Example of insertion sort
8 2
4 9 3 6
2 8
4 9 3 6
2 4 8
9 3 6
2 4 8 9
3 6
2 3 4 8 9 6
14
Example of insertion sort
8 2
4 9 3 6
2 8
4 9 3 6
2 4 8
9 3 6
2 4 8 9
3 6
2 3 4 8 9 6
15
Example of insertion sort
8 2
4 9 3 6
2 8
4 9 3 6
2 4 8
9 3 6
2 4 8 9
3 6
2 3 4 8 9 6
2 3 4 6 8 9 done
Running time? (n2) because (n2) compares and (n2) swaps
e.g. when input is A = [n, n 1, n 2, . . . , 2, 1]
16
Binary Insertion sort
BINARY-INSERTION-SORT (A, n) [ A[1 . . n]
for j 2 to n
insert key A[j] into the (already sorted) sub-array A[1 .. j-1].
Use binary search to find the right position
Binary search with take (log n) time.
However, shifting the elements after insertion will
still take (n) time.
Complexity: (n log n) comparisons
(n2) swaps
17
Meet Merge Sort
MERGE-SORT A[1 . . n]
1. If n = 1, done (nothing to sort).
divide and
conquer 2. Otherwise, recursively sort A[ 1 . . n/2 ]
and A[ n/2+1 . . n ] .
3. Merge the two sorted sub-arrays.
Key subroutine: MERGE
18
Merging two sorted arrays
20 12
13 11
7 9
2 1
19
Merging two sorted arrays
20 12
13 11
7 9
2 1
20
Merging two sorted arrays
20 12 20 12
13 11 13 11
7 9 7 9
2 1 2
21
Merging two sorted arrays
20 12 20 12
13 11 13 11
7 9 7 9
2 1 2
1 2
22
Merging two sorted arrays
20 12 20 12 20 12
13 11 13 11 13 11
7 9 7 9 7 9
2 1 2
1 2
23
Merging two sorted arrays
20 12 20 12 20 12
13 11 13 11 13 11
7 9 7 9 7 9
2 1 2
1 2 7
24
Merging two sorted arrays
20 12 20 12 20 12
20 12
13 11 13 11 13 11
13 11
7 9 7 9 7 9 9
2 1 2
1 2 7
25
Merging two sorted arrays
20 12 20 12 20 12 20 12
13 11 13 11 13 11 13 11
7 9 7 9 7 9 9
2 1 2
1 2 7 9
26
Merging two sorted arrays
20 12 20 12 20 12 20 12
20 12
13 11 13 11 13 11 13 11
13 11
7 9 7 9 7 9 9
2 1 2
1 2 7 9
27
Merging two sorted arrays
20 12 20 12 20 12 20 12
20 12
13 11 13 11 13 11 13 11
13 11
7 9 7 9 7 9 9
2 1 2
1 2 7 9 11
28
Merging two sorted arrays
20 12 20 12 20 12 20 12 20 12 20 12
13 11 13 11 13 11 13 11 13 11 13
7 9 7 9 7 9 9
2 1 2
1 2 7 9 11
29
Merging two sorted arrays
20 12 20 12 20 12 20 12 20 12 20 12
13 11 13 11 13 11 13 11 13 11 13
7 9 7 9 7 9 9
2 1 2
1 2 7 9 11 12
30
Merging two sorted arrays
20 12
20 12 20 12 20 12 20 12 20 12
13 11
13 11 13 11 13 11 13 11 13
7 9
7 9 7 9 9
2 1 2
1
2
7 9 11 12
Time = (n) to merge a total
of n elements (linear time).
31
Analyzing merge sort
MERGE-SORT A[1 . . n] T(n)
1. If n = 1, done. (1)
2. Recursively sort A[ 1 . . n/2 ] 2T(n/2)
and A[ n/2+1 . . n ] .
3.Merge the two sorted lists
(n)
(1) if n = 1;
T(n) =
2T(n/2) + (n) if n > 1.
T(n) = ?
32
Recurrence solving
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
33
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
T(n)
34
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn
T(n/2) T(n/2)
35
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn
cn/2 cn/2
T(n/4) T(n/4) T(n/4) T(n/4)
36
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn
cn/2 cn/2
cn/4 cn/4 cn/4 cn/4
(1)
37
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn
h = 1 + lg n cn/2 cn/2
cn/4 cn/4 cn/4 cn/4
(1)
38
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn cn
h = 1 + lg n cn/2 cn/2
cn/4 cn/4 cn/4 cn/4
(1)
39
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn cn
h = 1 + lg n cn/2 cn/2 cn
cn/4 cn/4 cn/4 cn/4
(1)
40
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn cn
h = 1 + lg n cn/2 cn/2 cn
cn/4 cn/4 cn/4 cn/4 cn
(1)
41
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn cn
h = 1 + lg n cn/2 cn/2 cn
cn/4 cn/4 cn/4 cn/4 cn
(1) #leaves = n (n)
42
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn cn
h = 1 + lg n cn/2 cn/2 cn
cn/4 cn/4 cn/4 cn/4 cn
(1) #leaves = n (n)
Total ?
43
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn cn
h = 1 + lg n cn/2 cn/2 cn
cn/4 cn/4 cn/4 cn/4 cn
(1) #leaves = n (n)
Total = (n lg n)
Equal amount of work done at each level
44
Tree for different recurrence
Solve T(n) = 2T(n/2) + c, where c > 0 is constant.
c c
c c 2c
h = 1 + lg n
c c c c 4c
n/2 c
(1) #leaves = n (n)
Note that 1 + + + < 2 Total = (n)
All the work done at the leaves
45
Tree for yet another recurrence
Solve T(n) = 2T(n/2) + cn2, c > 0 is constant.
cn2 cn2
h = 1 + lg n cn2/4 cn2/4 cn2/2
cn2/16 cn2/16 cn2/16 cn2/16 cn2/4
(1) #leaves = n (n)
Note that 1 + + + < 2 Total = (n2)
All the work done at the root
46
MIT OpenCourseWare
http://ocw.mit.edu
6.006 Introduction to Algorithms
Fall 2011
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.