CT 363 – Design and Analysis of

Algorithms Lecture# 04: Sorting Algorithms

Instructor: Engr. Vanesh Kumar

1

Sorting
• In computer science, arranging in an ordered sequence is
called "sorting".
• Sorting is a common operation in many applications, and
efficient algorithms to perform it have been developed. •
 The most common uses of sorted sequences are:
• making lookup or search efficient;
• making merging of sequences efficient.
• enable processing of data in a defined order.

• In computer science, a sorting algorithm is an algorithm that

puts elements of a list into an order.
2

Types of Sorting Algorithms

• Non-recursive/Incremental comparison sorting
• Selection sort
• Bubble sort
• Insertion sort
• Recursive comparison sorting
• Merge sort
 • Quick sort
• Heap sort
• Non-comparison linear sorting
• Count sort
• Radix sort
3
• Bucket sort

Selection Sort
• Idea:
• Find the smallest element in the array
• Exchange it with the element in the first position • Find the
second smallest element and exchange it with the element
in the second position
• Continue until the array is sorted
 n ← length[A]
846923114692381269438
for j ← 0 to n - 2
1239468 smallest ← j

1234968

1234698 Cost Steps

1234689 c11
4 c2 n-1
1234689 c3 n-1
Selection Sort ∑ −

n
2

Analysis
Alg.: SELECTION-SORT(A)
 for i ← j + 1 to n-1 A[smallest]
=

0 ∑ −

if A[i] < c4 c5 c6 n
2j jt
A[smallest] j j
=
t =0

smallest ← i 0 ∑ −

exchange A[j] ↔ 2j tj
c7 n-1
tj: # of times the inner for loop statement is executed at iteration j
5
= T n c c n c n c t c t c t c nn
+ −−
= 1 + 2 − + 3 − + 4∑
+
∑ ∑
+
n
−
2
n
−
2
2
( ) ( 1) ( 1) ( 1) 7 5
j
 6
j =
j 0
0 j
j
j
=0

Best/Worst/Average Case

Analysis n n T n c c n c n c
( 1)
−
( ) ( 1) ( 1) 1 2 3 4 5 6 +7 −− = + − + − + cnn n

nn
( 1)
−
( 1)
 +
c
+
c
2
( 1)
2
2 ( 1)2
= an +bn+c nn
2 j
n

−
∑
=−

T(n) = Θ(n2) j
2
=
0

Disadvantage:
Θ(n2) running time in all cases
 6
n
−
2
n
−
2
n
−
2

− + 4∑
+
∑ +∑ +−
T n c c n c n c t c t c t c n ( ) ( 1) ( 1) ( 1) 7
=1 +2 −+3 0
j
5 =
0
j j
=
0
j = j 6 j

Bubble Sort
Given a list of N elements, repeat the following steps N-1 times:
• For each pair of adjacent numbers, if number on the left is greater than the
number on the right, swap them.
• “Bubble” the largest value to the end using pair-wise comparisons and swapping.

126 126 22 14228 17228 1722

2214 17

6 12 148
148 148 17
12 22

6 128 128 14 17 22
7
 8 12 17
6 14 17 22
Bubble Sort c11
c2 n-1
Analysis ∑ −

n
2
BubbleSort(A)
1. n = Length[A];
2. for j = 0 to n-2

cost times
3. for i = 0 to A[i] > A[i+1] 6. A[i] =
n-j-2 4. if 5. temp = A[i] A[i+1] 7.
 0 j
A[i+1] = temp
∑
=
− 0
c3 c4 c5 c6 c7
j t
n
2 ∑ −

n
j j t 2
= j
0 = j t
∑
0 j
−

∑
=
−
n
2
8. return A;
n

j t 2
c81
j jt 0
=

tj: # of times the inner for loop statement is executed at iteration j 8

=+−+
1 2 3 T(n) c c (n 1) c t c t c t c t c t c
n
∑ j
+
∑ + ∑+ ∑+

∑+ −
n
−
2
n
−
2
n
−
2
n
−
2
2
4
j
5
j
6
j
7
j
8
j
0
j
=
0
j
=
0
j
=
0
 j
=
=0

Best/Worst/Average Case Analysis

• Since we don’t know at which iteration the list is completely
sorted, the nested loop will nn
j
execute completely. ∑ −

0
− 2 2
n ( 1) = j
=

nn −
Tnccnc+ +
( 1)
−
nn
( 1)
−
nn
 ( 1)
−
nn
( ) ( 1) c c 1 2 3 4 5 6 7 82( 1)

nn
( 1)
=+−++ −
c
+
c
+
2 c
2
2
2
= an +bn+c a quadratic function of n
2

• T(n) = Θ(n2) order of growth in n2

− − 2 − − 9
n 2 n 2 n− n 2 n 2
 =+−+
∑ j
+
∑ ∑ ∑ ∑
+ + + +

T(n) c c (n 1) c t c t c t c t c t c
123
4
j
5
j
6
j
7
j
8
j
0
j
=
0
j
=
0
j
=
0
j
=
0
=

Bubble Sort Modified

BubbleSort(A)
1. n = Length[A];
2. for j = 0 to n-2
3. swap = 0;
4. for i = 0 to n-j-2
5. if A[i] > A[i+1]
6. temp = A[i]
7. A[i] = A[i+1]
8. A[i+1] = temp
swap = swap + 1;
9. if swap = 0
10. break;
10
11.return A;

Best Case Analysis (Modified)

• The array is already sorted
• Keep track of no. of swaps done in an iteration
• Can finish early if no swapping occurs
• tj = 1 🡺 n – 1 comparisons in a single iteration

T(n) = c1+ c2(n -1) + c3(n -1) + c4(n -1) + c8 =

(c2+ c3+ c4)n + (c1– c2– c3– c4+ c8)
= an + b = Θ(n)
− − 2 − − 11
n 2 n 2 n − n 2 n 2

=+−+
∑ j
+
∑ +∑ +∑ +∑ +
T(n) c c (n 1) c t c t c t c t c t c
123
4
j
5
j
6
j
 7
j
8
j
0
j
=
0
j
=
0
j
=
0
j
=
0
=

Worst Case
Analysis (Modified) n
−
( 1) =2

• The array is in reverse sorted nn

j
order • Always A[i] > A[i+1]
∑ −0
2
j =

nn −
Tnccnc+ +
( 1)
−
nn
( 1)
−
nn
( 1)
n n 12345678
( ) ( 1) c c 2( 1)

−
nn
( 1)
=+−++ −
c
+
c
+
2 c
 2
2
2
= an +bn+c
2 a quadratic function of n

T(n) = Θ(n2) order of growth in n2

Disadvantages
Advantages Θ(n2) running time in worst & average case
Nobody “EVER” uses bubble sort
Simple to code
Requires little memory (in-place) (EXTREMELY INEFFICIENT)
− 2 − 12
2 n 2 − n 2
− 2 − n
n n

=+−+
∑ j
+
∑ +∑ +∑ +∑ +
T(n) c c (n 1) c t c t c t c t c t c
123
4
j
5
j
6
j
7
j
8
 j
0
j
=
0
j
=
0
j
=
0
j
=
0
=

Insertion Sort
• Idea: like sorting a hand of playing cards
• Start with empty left hand and cards face down on the table. •
Remove one card at a time from the table, and insert it into the
correct position in the left hand
• compare it with each card already in the hand, from right to left •
The cards held in the left hand are sorted
 • these cards were originally the top cards of the pile on the table

13

Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key

Array index 0 1 2 3 4 5 6 7 8 9Value 2.78 7.42 0.56 1.12 1.17 0.32 6.21 4.42 3.14 7.71 14

Iteration 0: step 0.

Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key

Array index 0 1 2 3 4 5 6 7 8 9Value 2.78 7.42 0.56 1.12 1.17 0.32 6.21 4.42 3.14 7.71 15

Iteration 1: step 0.

Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key
 Array index 0 1 2 3 4 5 6 7 8 9Value 2.78 7.42 0.56 1.12 1.17 0.32 6.21
4.42 3.14 7.71 0.56 7.42

16

Iteration 2: step 0.

Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key
Array index 0 1 2 3 4 5 6 7 8 9Value 2.78 0.56 7.42 1.12 1.17 0.32 6.21 4.42
3.14 7.71 0.56 2.78

17

Iteration 2: step 1.

Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key
Array index 0 1 2 3 4 5 6 7 8 9Value 2.78 0.56 7.42 1.12 1.17 0.32 6.21 4.42
3.14 7.71 0.56 2.78

18

Iteration 2: step 2.

Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key
 Array index 0 1 2 3 4 5 6 7 8 9Value 0.56 2.78 7.42 1.12 1.17 0.32 6.21
4.42 3.14 7.71 1.12 7.42

19

Iteration 3: step 0.

Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key
 Array index 0 1 2 3 4 5 6 7 8 9Value 0.56 2.78 1.12 7.42 1.17 0.32 6.21
4.42 3.14 7.71 1.12 2.78

20

Iteration 3: step 1.

Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key
 Array index 0 1 2 3 4 5 6 7 8 9Value 0.56 2.78 1.12 7.42 1.17 0.32 6.21
4.42 3.14 7.71 1.12 2.78

21

Iteration 3: step 2.

Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key
 Array index 0 1 2 3 4 5 6 7 8 9Value 0.56 1.12 2.78 7.42 1.17 0.32 6.21
4.42 3.14 7.71 1.17 7.42

22

Iteration 4: step 0.

Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key
 Array index 0 1 2 3 4 5 6 7 8 9 Value 0.56 1.12 2.78 1.17 7.42 0.32 6.21
4.42 3.14 7.71
1.17 2.78

23

Iteration 4: step 1.

Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key

Array index 0 1 2 3 4 5 6 7 8 9Value 0.56 1.12 1.17 2.78 7.42 0.32 6.21 4.42 3.14 7.71 24

Iteration 4: step 2.

Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key
 Array index 0 1 2 3 4 5 6 7 8 9Value 0.56 1.12 1.17 2.78 7.42 0.32 6.21
4.42 3.14 7.71 0.32 7.42

25

Iteration 5: step 0.

Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key
 Array index 0 1 2 3 4 5 6 7 8 9Value 0.56 1.12 1.17 2.78 0.32 7.42 6.21
4.42 3.14 7.71 0.32 2.78

26

Iteration 5: step 1.

Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key
 Array index 0 1 2 3 4 5 6 7 8 9Value 0.56 1.12 1.17 0.32 2.78 7.42 6.21
4.42 3.14 7.71 0.32 1.17

27

Iteration 5: step 2.

Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key
 Array index 0 1 2 3 4 5 6 7 8 9Value 0.56 1.12 0.32 1.17 2.78 7.42 6.21
4.42 3.14 7.71 0.32 1.12

28

Iteration 5: step 3.

Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key
Array index 0 1 2 3 4 5 6 7 8 9Value 0.56 0.32 1.12 1.17 2.78 7.42 6.21 4.42
3.14 7.71 0.32 0.56

29

Iteration 5: step 4.

Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key
Array index 0 1 2 3 4 5 6 7 8 9Value 0.56 0.32 1.12 1.17 2.78 7.42 6.21 4.42
3.14 7.71 0.32 0.56

30

Iteration 5: step 5.

Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key
 Array index 0 1 2 3 4 5 6 7 8 9Value 0.32 0.56 1.12 1.17 2.78 7.42 6.21
4.42 3.14 7.71 6.21 7.42

31

Iteration 6: step 0.

Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key
 Array index 0 1 2 3 4 5 6 7 8 9Value 0.32 0.56 1.12 1.17 2.78 7.42 6.21
4.42 3.14 7.71 6.21 7.42

32

Iteration 6: step 1.

Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key
 Array index 0 1 2 3 4 5 6 7 8 9Value 0.32 0.56 1.12 1.17 2.78 6.21 7.42
4.42 3.14 7.71 4.42 7.42

33

Iteration 7: step 0.

Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key
 Array index 0 1 2 3 4 5 6 7 8 9Value 0.32 0.56 1.12 1.17 2.78 6.21 4.42
7.42 3.14 7.71 4.42 6.21

34

Iteration 7: step 1.

Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key
 Array index 0 1 2 3 4 5 6 7 8 9Value 0.32 0.56 1.12 1.17 2.78 6.21 4.42
7.42 3.14 7.71 4.42 6.21

35

Iteration 7: step 2.

Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key
 Array index 0 1 2 3 4 5 6 7 8 9Value 0.32 0.56 1.12 1.17 2.78 4.42 6.21
7.42 3.14 7.71 3.14 7.42

36

Iteration 8: step 0.

Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key
 Array index 0 1 2 3 4 5 6 7 8 9Value 0.32 0.56 1.12 1.17 2.78 4.42 6.21
3.14 7.42 7.71 3.14 6.21

37

Iteration 8: step 1.

Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key
 Array index 0 1 2 3 4 5 6 7 8 9Value 0.32 0.56 1.12 1.17 2.78 4.42 3.14
6.21 7.42 7.71 3.14 4.42

38

Iteration 8: step 2.

Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key
 Array index 0 1 2 3 4 5 6 7 8 9Value 0.32 0.56 1.12 1.17 2.78 4.42 3.14
6.21 7.42 7.71 3.14 4.42

39

Iteration 8: step 3.

Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key
 Array index 0 1 2 3 4 5 6 7 8 9Value 0.32 0.56 1.12 1.17 2.78 3.14 4.42 6.21 7.42 7.71 40

Iteration 9: step 0.

Insertion Sort
InsertionSort(A, n)
1. for i = 1 to n-1
2. key = A[i]
3. j = i – 1
4. while (j >= 0) and (A[j] > key)
5. A[j+1] = A[j]
6. j = j – 1
7. A[j+1] = key
 Array index 0 1 2 3 4 5 6 7 8 9Value 0.32 0.56 1.12 1.17 2.78 3.14 4.42 6.21 7.42 7.71 41

Iteration 10: DONE!.

Insertion Sort j = i - 1;

Analysis
cost times c1 n-1 c2 n-1 c3
InsertionSort(A, n) { n-1
for i = 1 to n-1 {
key = A[i]
while (j>=0) } ∑ − 1
= 1n
and (A[j] > c4 c5 c6 t
i

∑−=11n
i
key){ A[j+1] =
A[j] ∑ − 1
= 1n
it
j = j – 1 i i t
i
 at iteration i
A[j+1] = key c7 n-1
}
}
42
ti: # of times the while statement is executed

+ −−
= T n c n c n c n c t c t c t c n = 1 − + 2 − + 3 − + 4∑
n +
∑ ∑
+

n
−
1
n
−
1
1
( ) ( 1) ( 1) ( 1) ( 1)7 5
i
6
i i
i i
1
=
1
 i
=
1

Best Case
Analysis “while j >= 0 and A[j] > key”

• The array is already sorted in ascending order

• Inner loop will not be executed
• The number of moves: 2*(n-1)
• The number of key comparisons: (n-1)

T(n) = c1(n-1) + c2(n -1) + c3(n -1) + c4(n -1) + c7(n-1)

= (c1 + c2 + c3 + c4 + c7)n - (c1 + c2 + c3 + c4 + c7) = an +
b = Θ(n)
− 1 − 1 − 1 43
n n n
 +−
− + 4∑
+
∑ ∑
+

T n c n c n c n c t c t c t c n ( ) ( 1) ( 1) ( 1) ( 1) 7
6 i
=1 i =
−+2 i 1
i i
−+3 = =
1
5
i 1

Worst Case
Analysis “while j >= 0 and A[j] > key”
• The array is in reverse sorted order
• The number of moves: 2*(n-1)+(1+2+...+n-1)= 2*(n-1)+ n*(n-1)/2
• The number of key comparisons: (1+2+...+n-1)= n*(n-1)/2 •
Always A[j] > key in while loop test
 • Have to compare key with all elements to the left of the j-th position
⇒ compare with i-1 elements ⇒ tj = i −∑=−
n
nn
1 i
( 1) t

i
2
=
1
nn
Tncncncnccc
⎛−
( 1)
⎞
−
( ) ( 1) ( 1) ( 1) ( ) 1 2 3 4 5 6 ⎟ + 7 ⎠
( 1)
=−+−+−+++cn 2

⎜
2 ⎝
= an +bn+c a quadratic function of n
T(n) = Θ(n2) order of growth in n2
 − 1 − 1 − 1 44
n n n

+−
− + 4∑
+
∑ ∑ +

T n c n c n c n c t c t c t c n ( ) ( 1) ( 1) ( 1) ( 1) 7
6 i
=1 i =
−+2 i 1
i i
−+3 = =
1
5
i 1

Insertion Sort Summary

• Running time not only depends on the size of the array but also
the contents of the array.

• Average-case: 🡺 O(n2)
• We have to look at all possible initial data organizations.
• Advantages
• Good running time for “almost sorted” arrays Θ(n)

• Disadvantages
• Θ(n2) running time in worst and average case
• ≈ n2/2 comparisons and exchanges 45

Types of Sorting Algorithms

• Non-recursive/incremental comparison sorting
• Selection sort
• Bubble sort
• Insertion sort
• Recursive comparison sorting
• Merge sort
 • Quick sort
• Heap sort
• Non-comparison linear sorting
• Count sort
• Radix sort
46
• Bucket sort

Recursive Comparison Sorting

There are many ways to design algorithms:

Insertion/Bubble/Selection sort uses an incremental

approach • Having sorted to array A[i..j]
• We insert the single element A[j+1] into its proper place, to get a sorted
array A[i..j+1]
An alternative design approach is “Divide and
Conquer” • Divide the problems into a number of
sub-problems
• Conquer the sub-problems by solving them recursively. If sub-problem
sizes are small enough, just solve them in a straight forward manner.
• Combine the solutions to the sub-problems into the solution for the original
47
problem.

Merge Sort
• Idea: Order a list of values by recursively dividing the list in half until
each sub-list has one element, then recombining

• More specifically:
• Divide the n element sequence to be sorted into two subsequences
of n/2 elements each.
• Conquer: Sort the two subsequences to produce the sorted answer.
 • Combine: Merge the two sorted sub sequences to produce the
sorted answer.

mergeSort(0, n/2-1) mergeSort(n/2, n-1)

48
sort merge(0, n/2, n-1) sort

Merge Sort Pseudocode

 49

Merge Sort
Unsorted
Base Case: When the sequences to be sorted has length 1.
66 108 56 14 89 12 34 Sorted 12 14 34 56 66 89 108

Divide

89 12 34 Merge
Divide Merge 12 34 89 Divide BCas
56 14
Merge Merge 14 56

66 108 56 14 14 Divide BCas

Divide BCas Merge
56 66 108 Merge Divide BCas
Divide Merge Merge
Divide 66 108 89 12
Divide 66 108 12 89
Merge
Divide BCas
Merge
Divide BCas Divide BCas
Merge Merge
34 50

66 66 108 56 14 89 12

Merge Function
Given two sorted arrays, merge operation produces a sorted array
with all the elements of the two arrays

81432
..
Divide the array in half and conquer .

14823

Merge the halves

 1 2 3 4 8 copy
tempArray:
Original Array:
12348
51

Merge Function Pseudocode

Complexity of Merge Function?

O(n)
 52

Merge Sort Analysis

Let T(n) be the time taken by this algorithm to sort an array of n

elements dividing A into sub-arrays A1and A2. Merge operation takes
the linear time. Consequently,
T(n) = T(n/2) + T(n/2) + θ(n)
T(n) = 2T (n/2) + θ(n)
The above recurrence relation is non-homogenous and can be solved
by any of the methods:
53
• Substitution
• Recursion tree
• Master method
O(n)
Merge Sort
n/4 n/4 n/4 n/4
Analysis Merge two n/2 items: O(n)

Level 0 Level 1

Level 2
n/2 n/2 Merge n items:
 Each level requires O(n)
operations

Tree Height : log2n

111 ... 111

n)
Each level O(n) operations & O(log2n) levels 🡪 O(n*log2 54

Merge Sort Analysis

O(n * log2n).
• Worst case: • O(n *
Average case: log2n).

• Performance is independent of the initial order of the array items.

• Advantage:
 • Mergesort is an extremely fast algorithm.

• Disadvantage:
• Mergesort requires a second array as large as the original array. 55

Quick Sort
• Quick Sort: orders a list of values by partitioning the list around
one element called a pivot, then sorting each partition. It is based
on divide and conquer approach.

• Key Points: Given an array S to be sorted

• Pick any element v in array S as the pivot (i.e. partition element)
• Partition the remaining elements in S into two groups
• S1 = {all elements in S-{v} that are smaller than v}
• S2 = {all elements in S-{v} that are larger than v}
 • apply the quick sort algorithm (recursively) to both partitions

• Trick lies in handling the partitioning

56
• i.e. picking a good pivot is essential to performance

Quick Sort Illustrated

pick a
pivot
40 37 35
18 2
10 32
17 6
12

partition
6
 40 37
10 12 217 18 32 35

quicksort quicksort

2 6 10 12 17 18 32 35 37 40
combine
57
2
6 10 12 17 18 32 35 37 40

Quick Sort Pseudocode

 58

Partitioning Algorithm
Original input : S = {8, 1, 4, 9, 0, 3, 5, 2, 7,
6} Pick the first/last element as pivot
8149035276
pivot
Have two ‘iterators’ – i and j
• i starts at first element and moves forward
• j starts at last element and moves backwards

8149035276

i jpivot
While (i < j)
• Move i to the right till we find a number greater than pivot
• Move j to the left till we find a number smaller than pivot
If (i<j) swap(S[i], S[j])
59
• The effect is to push larger elements to the right and smaller elements to the left
Swap the pivot with S[i]

Partitioning Algorithm Illustrated

i j pivot
8149035276
ij
Move swap pivot 8 1 4 9 0 3 5
2 7 6 i j pivot 2 1 4 9
035876
 i j pivot have crossed
move 2 1 4 9 0 3 5
Swap S[i] with pivot
i j pivot
876 swap 2 1 4 5 2 1 4 5 0 3 6 8 7 9
j i pivot
0 3 9 8 7 6 move 2 60
ji
145039876 pivot

i and j

Partitioning Pseudocode
What is the running time of
PARTITION(A, p, r) 1. pivot = partition()?
A[p];
2. leftPointer = p + 1 3. partition() runs in O(n) time
rightPointer = r 4. while (True)
5. while (A[leftPointer] < pivot)
6. leftPointer++;
7. while (A[rightPointer] >= pivot)
8. rightPointer--;
9. if leftPointer >= rightPointer
10. break;
11. else
12. A[leftPointer] 🡨🡪 A[rightPointer]
13. A[leftPointer] 🡨🡪 pivot
61

Partitioning Algorithm Illustrated

• choose pivot: 4 3 6 9 2 4 3 1 2 1 8 9 3 5 6
• search: 4 3 6 9 2 4 3 1 2 1 8 9 3 5 6
• swap: 4 3 3 9 2 4 3 1 2 1 8 9 6 5 6
• search: 4 3 3 9 2 4 3 1 2 1 8 9 6 5 6
 • swap: 4 3 3 1 2 4 3 1 2 9 8 9 6 5 6
• search: 4 3 3 1 2 4 3 1 2 9 8 9 6 5 6
• swap: 4 3 3 1 2 2 3 1 4 9 8 9 6 5 6
• search: 4 3 3 1 2 2 3 1 4 9 8 9 6 5 6 (left > right) • swap with
pivot: 1 3 3 1 2 2 3 4 4 9 8 9 6 5 6

62

Quick Sort: Best Case Analysis

Assuming that keys are random, uniformly distributed. The best case
running time occurs when pivot is the median • Partition splits array in
two sub-arrays of size n/2 at each step Partition Comparisons
size
Nodes contain problem n
n
 n
n/2 n/2
n/4 n/4
n/4 n/4 n

n/8 n/8 n/8 n/8 n/8 n/8 n/8 n/8

n
63
T(n) = 2 T(n/2) + cn
T(n) = cn log n + n = O(n log n)
Quick Sort: Worst Case Analysis
Assuming that keys are random, uniformly distributed. The worst
case running time occurs when pivot is the smallest (or largest)
element all the time
 • One of the two sub-arrays will have zero elements at each step
( 1) n-1
Height of Tree = n-1
nn
n
n
− 1
−
Total ∑=
Cost =
t
T(n) = T(n-1) + cn n-2
i
2
i
= 0 n-2
1

T(n-1) = T(n-2) + c(n-1) T(n-2) = T(n-3) + c(n-2) …

T(2) = T(1) + 2c
N
=+
∑=
2
0
n-3
 .
.
.

01
n-3 1

64

T(N) T(1) c i O(N ) i 2

=

Quick Sort: Average Case Analysis

Assuming that keys are random, uniformly distributed. The
average case running time occurs when pivot can take any
position at each step
• The size of two sub-arrays will vary at each step
 Per level Cost = Height of Tree =
n
logxn (x depends on how the partition is

split) O (n lg n)
65

Picking the Pivot

• How would you pick one?

• Strategy 1: Pick the first element in S

• Works only if input is random

• What if input S is sorted, or even mostly sorted?

 • All the remaining elements would go into either S1 or S2!
• Terrible performance!

• Why worry about sorted input?

• Remember 🡪 Quicksort is recursive, so sub-problems could be sorted •
Plus mostly sorted input is quite frequent

66

Picking the Pivot (contd.)

• Strategy 2: Pick the pivot randomly
• Would usually work well, even for mostly sorted input
• Unless the random number generator is not quite random!
• Plus random number generation is an expensive operation

• Strategy 3: Median-of-three Partitioning

• Ideally, the pivot should be the median of input array S
• Median = element in the middle of the sorted sequence
 • Would divide the input into two almost equal partitions •

Unfortunately, harder to calculate median quickly, without sorting first!

• So find the approximate median

• Pivot = median of the left-most, right-most and center elements of array S
67
• Solves the problem of sorted input

Picking the Pivot (contd.)

• Example: Median-of-three Partitioning

• Let input S = {6, 1, 4, 9, 0, 3, 5, 2, 7, 8}

• Left = 0 and S[left] = 6

• Right = 9 and S[right] = 8

• center = (left + right)/2 = 4 and S[center] = 0

 • Pivot
• = Median of S[left], S[right], and S[center]
• = median of 6, 8, and 0
• = S[left] = 6 68

Quick Sort: Final Comments

• What happens when the array contains many duplicate elements?
• Not stable

• What happens when the size of the array is small?

• For small arrays (N ≤ 50), Insertion sort is faster than quick sort due to
recursive function call overheads of quick sort. Use hybrid algorithm;
quick sort followed by insertion sort when N ≤ 50

• However, Quicksort is usually O(n log2n)

• Constants are so good that it is generally the fastest algorithm known
• Most real-world sorting is done by Quicksort
 • For optimum efficiency, the pivot must be chosen carefully
• “Median of three” is a good technique for choosing the pivot
69
• However, no matter what you do, some bad cases can be
constructed where Quick Sort runs in O(n2) time

Summary: Sorting Algorithms

Insertion Sort: Suitable only for small n ≤ 50 or nearly sorted inputs
Merge Sort: Guaranteed to be fast even in its worst case; stable
Quick Sort: Most useful general-purpose sorting for very little memory
requirement and fastest average time. (Choose the median of
three elements as pivot in practice
Heap Sort: Requiring minimum memory and guaranteed to run fast;
average and maximum time both roughly twice the average
time of quick sort. Useful for time-critical applications

Counting Sort: Very useful when the keys have small range; stable;
 memory space for counters and for 2n records.
Radix Sort: Appropriate for keys either rather short or with an
lexicographic collating sequence.
70

Bucket Sort: Assuming keys to have uniform distribution.