Scribd
Upload
106 views47 pages

Introduction To Algorithms-MIT

The document summarizes the insertion sort and merge sort algorithms. It provides examples of how insertion sort works by incrementally sorting an array. Insertion sort has a running time of Θ(n2) because it does Θ(n2) comparisons and swaps in the worst case. Merge sort works by recursively dividing the array into halves and then merging the sorted halves. Merging two sorted arrays takes linear time. The overall running time of merge sort is Θ(n log n) because each level of recursion requires linear time and there are log n levels of recursion.

Uploaded by

d
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
106 views47 pages

Introduction To Algorithms-MIT

The document summarizes the insertion sort and merge sort algorithms. It provides examples of how insertion sort works by incrementally sorting an array. Insertion sort has a running time of Θ(n2) because it does Θ(n2) comparisons and swaps in the worst case. Merge sort works by recursively dividing the array into halves and then merging the sorted halves. Merging two sorted arrays takes linear time. The overall running time of merge sort is Θ(n log n) because each level of recursion requires linear time and there are log n levels of recursion.

Uploaded by

d
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 47

6.

006- Introduction to

Algorithms

Courtesy of MIT Press. Used with permission.

Lecture 3

1
Menu

Sorting!
Insertion Sort
Merge Sort
Solving Recurrences

2
The problem of sorting

Input: array A[1n] of numbers.

Output: permutation B[1n] of A such


that B[1] B[2] B[n] .
e.g. A = [7, 2, 5, 5, 9.6] B = [2, 5, 5, 7, 9.6]

How can we do it efficiently ?

3
Why Sorting?
Obvious applications
Organize an MP3 library
Maintain a telephone directory
Problems that become easy once items are
in sorted order
Find a median, or find closest pairs
Binary search, identify statistical outliers
Non-obvious applications
Data compression: sorting finds duplicates
Computer graphics: rendering scenes front to back

4
Insertion sort
INSERTION-SORT (A, n) [ A[1 . . n]
for j 2 to n
insert key A[j] into the (already sorted) sub-array A[1 .. j-1].
by pairwise key-swaps down to its right position

Illustration of iteration j
1 i j n
A:
sorted key

new location of key


5
Example of insertion sort

8 2
4 9 3 6

6
Example of insertion sort

8 2 4 9 3 6

7
Example of insertion sort

8 2 4 9 3
6
2 8 4 9 3 6

8
Example of insertion sort

8 2 4 9 3
6
2 8 4 9 3 6

9
Example of insertion sort

8 2
4 9 3 6

2 8
4 9 3 6

2 4 8
9 3 6

10
Example of insertion sort

8 2
4 9 3 6

2 8
4 9 3 6

2 4 8
9 3 6

11
Example of insertion sort

8 2
4 9 3 6

2 8
4 9 3 6

2 4 8
9 3 6

2 4 8 9
3 6

12
Example of insertion sort

8 2
4 9 3 6

2 8
4 9 3 6

2 4 8
9 3 6

2 4 8 9
3 6

13
Example of insertion sort

8 2
4 9 3 6

2 8
4 9 3 6

2 4 8
9 3 6

2 4 8 9
3 6
2 3 4 8 9 6

14
Example of insertion sort

8 2
4 9 3 6

2 8
4 9 3 6

2 4 8
9 3 6

2 4 8 9
3 6
2 3 4 8 9 6

15
Example of insertion sort

8 2
4 9 3 6
2 8
4 9 3 6
2 4 8
9 3 6
2 4 8 9

3 6
2 3 4 8 9 6

2 3 4 6 8 9 done

Running time? (n2) because (n2) compares and (n2) swaps

e.g. when input is A = [n, n 1, n 2, . . . , 2, 1]


16
Binary Insertion sort
BINARY-INSERTION-SORT (A, n) [ A[1 . . n]
for j 2 to n
insert key A[j] into the (already sorted) sub-array A[1 .. j-1].
Use binary search to find the right position

Binary search with take (log n) time.

However, shifting the elements after insertion will

still take (n) time.

Complexity: (n log n) comparisons

(n2) swaps

17
Meet Merge Sort

MERGE-SORT A[1 . . n]
1. If n = 1, done (nothing to sort).
divide and
conquer 2. Otherwise, recursively sort A[ 1 . . n/2 ]
and A[ n/2+1 . . n ] .
3. Merge the two sorted sub-arrays.

Key subroutine: MERGE

18
Merging two sorted arrays
20 12

13 11

7 9

2 1

19
Merging two sorted arrays
20 12

13 11

7 9

2 1

20
Merging two sorted arrays

20 12 20 12
13 11 13 11
7 9 7 9
2 1 2

21
Merging two sorted arrays

20 12 20 12
13 11 13 11
7 9 7 9
2 1 2

1 2

22
Merging two sorted arrays

20 12 20 12 20 12

13 11 13 11 13 11

7 9 7 9 7 9
2 1 2

1 2

23
Merging two sorted arrays

20 12 20 12 20 12

13 11 13 11 13 11

7 9 7 9 7 9
2 1 2

1 2 7

24
Merging two sorted arrays

20 12 20 12 20 12
20 12
13 11 13 11 13 11
13 11
7 9 7 9 7 9 9
2 1 2

1 2 7

25
Merging two sorted arrays

20 12 20 12 20 12 20 12
13 11 13 11 13 11 13 11
7 9 7 9 7 9 9
2 1 2

1 2 7 9

26
Merging two sorted arrays

20 12 20 12 20 12 20 12
20 12
13 11 13 11 13 11 13 11
13 11
7 9 7 9 7 9 9
2 1 2

1 2 7 9

27
Merging two sorted arrays

20 12 20 12 20 12 20 12
20 12
13 11 13 11 13 11 13 11
13 11
7 9 7 9 7 9 9
2 1 2

1 2 7 9 11

28
Merging two sorted arrays

20 12 20 12 20 12 20 12 20 12 20 12
13 11 13 11 13 11 13 11 13 11 13
7 9 7 9 7 9 9
2 1 2

1 2 7 9 11

29
Merging two sorted arrays

20 12 20 12 20 12 20 12 20 12 20 12
13 11 13 11 13 11 13 11 13 11 13
7 9 7 9 7 9 9
2 1 2

1 2 7 9 11 12

30
Merging two sorted arrays

20 12
20 12 20 12 20 12 20 12 20 12
13 11
13 11 13 11 13 11 13 11 13
7 9
7 9 7 9 9
2 1 2

1
2
7 9 11 12

Time = (n) to merge a total

of n elements (linear time).

31
Analyzing merge sort

MERGE-SORT A[1 . . n] T(n)


1. If n = 1, done. (1)
2. Recursively sort A[ 1 . . n/2 ] 2T(n/2)
and A[ n/2+1 . . n ] .
3.Merge the two sorted lists
(n)

(1) if n = 1;
T(n) =
2T(n/2) + (n) if n > 1.
T(n) = ?
32
Recurrence solving
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

33
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

T(n)

34
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

cn

T(n/2) T(n/2)

35
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

cn
cn/2 cn/2

T(n/4) T(n/4) T(n/4) T(n/4)

36
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

cn
cn/2 cn/2

cn/4 cn/4 cn/4 cn/4

(1)

37
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

cn

h = 1 + lg n cn/2 cn/2

cn/4 cn/4 cn/4 cn/4

(1)

38
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

cn cn

h = 1 + lg n cn/2 cn/2

cn/4 cn/4 cn/4 cn/4

(1)

39
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

cn cn

h = 1 + lg n cn/2 cn/2 cn

cn/4 cn/4 cn/4 cn/4

(1)

40
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn cn

h = 1 + lg n cn/2 cn/2 cn

cn/4 cn/4 cn/4 cn/4 cn

(1)

41
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

cn cn

h = 1 + lg n cn/2 cn/2 cn

cn/4 cn/4 cn/4 cn/4 cn


(1) #leaves = n (n)

42
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

cn cn

h = 1 + lg n cn/2 cn/2 cn

cn/4 cn/4 cn/4 cn/4 cn


(1) #leaves = n (n)
Total ?
43
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

cn cn

h = 1 + lg n cn/2 cn/2 cn

cn/4 cn/4 cn/4 cn/4 cn


(1) #leaves = n (n)
Total = (n lg n)
Equal amount of work done at each level
44
Tree for different recurrence

Solve T(n) = 2T(n/2) + c, where c > 0 is constant.

c c
c c 2c
h = 1 + lg n
c c c c 4c


n/2 c
(1) #leaves = n (n)
Note that 1 + + + < 2 Total = (n)

All the work done at the leaves


45
Tree for yet another recurrence

Solve T(n) = 2T(n/2) + cn2, c > 0 is constant.


cn2 cn2

h = 1 + lg n cn2/4 cn2/4 cn2/2

cn2/16 cn2/16 cn2/16 cn2/16 cn2/4


(1) #leaves = n (n)
Note that 1 + + + < 2 Total = (n2)
All the work done at the root

46
MIT OpenCourseWare
http://ocw.mit.edu

6.006 Introduction to Algorithms


Fall 2011

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

You might also like