CSE 2215: Data Structures and Algorithms I

Sherajul Arifin
Lecturer, Department of Computer Science and Engineering,
United International University
Counting Sort

2
Sorting in Linear Time
•Counting sort
o No comparisons between elements!
o But…depends on assumption about the numbers
being sorted
o We assume numbers are in the range 1, …, k
o The algorithm:
o Input: A[1..n], where A[i] ∈ {1, 2, 3, …, k}
o Output: B[1..n], sorted (notice: not sorting in place)
o Also: Array C[1..k] for auxiliary storage

3
 Counting Sort

COUNTING-SORT assumes that each of the input elements is an integer in the

range 0 to k, inclusive.
A [1..n] is the input array, B [1..n] is the output array, C [0..k] is a temporary working
array.
4
Counting Sort
O(k)

O(n)

11/21/21 5
Counting Sort
What will be the running time?
•Total time: O(n + k)
●Usually, k = O(n)

●Thus counting sort runs in O(n) time

11/21/21 6
Simulation

7
Simulation

Initial o n
rati
st Ite
1
nd Iterati
on
2
3 rd It
erati
on

Final Result
8
Counting Sort
•Why don’t we always use counting sort?
Because it depends on range k of
elements

•Could we use counting sort to sort 32 bit

integers? Why or why not?
Answer: No, k is too large (232 =
4,294,967,296)

9
 Divide-and-Conquer Technique
• Divide-and-Conquer is a general algorithm design
paradigm:
o Divide the problem into a number of subproblems that
are smaller
o instances of the same problem
o Conquer the subproblems by solving them recursively
o Combine the solutions to the subproblems into the
solution for the
o original problem
• The base case for the recursion are subproblems of
constant size
• Analysis can be done using recurrence equations

10
Merge Sort

11
Divide-and-Conquer

12
Merge Sort and Quicksort
•Two well-known sorting algorithms adopt this divide-and-conquer
strategy

•Merge sort
o Divide step is trivial – just split the list into two equal parts.
o Work is carried out in the conquer step by merging two sorted
lists.

•Quick sort
o Work is carried out in the divide step using a pivot element.
o Conquer step is trivial.

13
Merge Sort

p q r

q+1

14
Merge Sort
A

p q r

q+1

L R

∞ ∞

1 n1+1 n2+1

1
15
1 6 3 7 9 2 8 4

16
 1 6 3 7 9 2 8 4

1 6 3 7 9 2 8 4

17
 1 6 3 7 9 2 8 4

1 6 3 7 9 2 8 4

1 6 3 7 9 2 8 4

18
 1 6 3 7 9 2 8 4

1 6 3 7 9 2 8 4

1 6 3 7 9 2 8 4

1 6 3 7 9 2 8 4

19
1 6 3 7 9 2 8 4

20
1 6 3 7 9 2 8 4

1 6 3 7 2 9 4 8

21
1 6 3 7 9 2 8 4

1 6 3 7 2 9 4 8

1 3 6 7 2 4 8 9

22
1 6 3 7 9 2 8 4

1 6 3 7 2 9 4 8

1 3 6 7 2 4 8 9

1 2 3 4 6 7 8 9

23
Merge Sort

Divide Step

7 2 9 4|
3 8 6
1

24
Merge Sort

Recursive Call, Divide

7 2 9 4|
3 8 6
1

7 2 | 9 4

25
Merge Sort

Recursive Call, Divide

7 2 9 4|
3 8 6
1

7 2 | 9 4

7 | 2

26
Merge Sort

Recursive Call, Base Case

7 2 9 4|
3 8 6
1

7 2 | 9 4

7 | 2

27
Merge Sort

Recursive Call, Base Case

7 2 9 4|
3 8 6
1

7 2 | 9 4

7 | 2

7 -> 7 2 -> 2

28
Merge Sort

Merge

7 2 9
4| 3 8 6 1

7 2 |9 4

7 | 2 -> 2 7

7 -> 7 2 -> 2

29
Merge Sort

Recursive Call, Partition, Base Case, Merge

7 2 9 4 | 3 8 6
1

7 2 | 9 4

7 | 2 -> 2 7 9 | 4 -> 4 9

7 -> 7 2 -> 2 9 -> 9 4 -> 4

30
Merge Sort

Merge

7 2 9 4 |
3 8 6 1

7 2 | 9 4 -> 2 4 7 9

7 | 2 -> 2 7 9 | 4 -> 4 9

7 -> 7 2 -> 2 9 -> 9 4 -> 4

31
Merge Sort

Recursive Call, … , Merge, Merge

7 2 9 4|
3 8 6
1

7 2 | 9 4 -> 2 4 7 9 3 8 | 6 1 -> 1 3 6 8

7 | 2 -> 2 7 9 | 4 -> 4 9 3 | 8 -> 3 8 6 | 1 -> 1 6

7 -> 7 2 -> 2 9 -> 9 4 -> 4 3 -> 3 8 -> 8 6 -> 6 1 -> 1

32
Merge Sort

Merge

7294|3861 -> 1 2 3 4 6 7 8 9

7 2 | 9 4 -> 2 4 7 9 3 8 | 6 1 -> 1 3 6 8

7 | 2 -> 2 7 9 | 4 -> 4 9 3 | 8 -> 3 8 6 | 1 -> 1 6

7 -> 7 2 -> 2 9 -> 9 4 -> 4 3 -> 3 8 -> 8 6 -> 6 1 -> 1

33
Merge Sort – Run Time

Input Size = n = r-p+1

O(n)

O(n) For loops at line 4,6

n1 = n2 = n/2 = O(n)

O(n) For loop at line 12

r – p = n-1 = O(n)

Total run time T(n) =

O(n)
34
Merge Sort

Total Run Time T(n)

T(n)

T(n/2)
T(n/2)
O(n)

35
Time Complexity Analysis

Let T(n) denote its worst-case running time on input instances of size
n. Thus the running time,T(n) satisfies the following recurrence
relation.

T(n) ≤ 2T(n / 2) + O(n)

→T(n) ≤ 2T(n / 2) + cn
Here, c is some arbitrary constant.

Now, the base case for this recurrence?

T(2) ≤ c.
36
Solving Recurrence Relations
Method-1: Unrolling the Recursive Tree
Step -1: Analyze the first few steps:
o At the first level of recursion, we have a single problem of
size n, which takes time at most cn plus the time spent in
all subsequent recursive calls.
o At the next level, we have two problems each of size n/2.
Each of these takes time at most cn/2, for a total of at
most cn, again plus the time in subsequent recursive calls.
o At the third level, we have four problems each of size n/4,
each taking time at most cn/4, for a total of at most cn.
37
Solving Recurrence Relations

Figure: Unrolling the recurrence T(n) ≤ 2T(n/2) + O(n).

38
Solving Recurrence Relations

Step -2: Identifying a pattern:

What’s going on in general?

o At level j of the recursion, the number of subproblems has doubled
j times, so there are now a total of 2j subproblems.
o Each has correspondingly shrunk in size by a factor of two j times,
and so each has size n/2j, and hence each takes time at most
cn/2j .
o Thus level j contributes a total of at most 2j (cn/2j) = cn to the total
running time.

39
Solving Recurrence Relations
Step -3: Summing over all levels of recursion:

We know, the recursion will ‘bottom out’ when the value of input n
reaches n = 1. We assume the recursion will level down x times. Each
time the input size is halved, after x times it will be n/2x

n/2x = 1 Each level contributes = cn

→n = 2x Total run time = # of levels * level

contribution

→x = log2 n = log2 n * cn

= cn log2 n = O(n log n)

40
Solving Recurrences by Master Theorem
Method-2: Use the magic formula

Master Theorem:

Let us consider the following general recurrence

T(n) = aT(n/b) + f(n)

Where, f(n) is a function of the form f(n) = O(nk)

1. If a > bk, T(n) = O(nlogb a)

2. If a = bk, T(n) = O(nlogb a log n)
3. If a < bk, T(n) = O(nk)
41
Solving Recurrences by Master Theorem
For Merge Sort –

T(n) = 2T(n / 2) + O(n)

a = 2, b = 2, k = 1, so, a = bk

Hence, T(n) = O(nlogb a log2 n)

= O(nlog2 2 log2 n)

= O(n1 log n)

= O(n log n)

42
Thank you!