19CSE212: Data Structures & Algorithms

Lecture 4 : Sorting Algorithms

By
Ritwik M

Based on the reference materials by Prof. Goodrich, OCW METU and Dr. Vidhya Balasubramanian
All images from Goodrich, Michael T., Roberto Tamassia, and Michael H. Goldw asser. Data structures and algorithms in Python. John Wiley & Sons, 2013.

1 Amrita Vishwa Vidhyapeetham Ritwik M

Amrita School of Engineering
 A Quick Recap

• Complexity Analysis
– Counting Method
• Primitive operations are identified and counted to
analyze cost
– Asymptotic Analysis
• Big Oh
• Verification of Big Oh
• Examples
• Exercises

2 Amrita Vishwa Vidhyapeetham Ritwik M

Amrita School of Engineering
 Sorting

• Given an unordered sequence of elements and the task of

rearranging them into an increasing or decreasing order.
• Linear Sorting Algorithms
– Insertion Sort
– Bubble Sort
– Selection Sort
• Non-Linear Sorting Algorithms
– Merge Sort
– Heap sort (later)
– Quick Sort
3 Amrita Vishwa Vidhyapeetham Ritwik M
Amrita School of Engineering
 1-Insertion Sort

Algorithm
To sort an array of size n in ascending order:
1: Iterate from arr[1] to arr[n] over the array.
2: Compare the current element (key) to its predecessor.
3: If the key element is smaller than its predecessor, compare it to
the elements before. Move the greater elements one position up
to make space for the swapped element.

4 Amrita Vishwa Vidhyapeetham Ritwik M

Amrita School of Engineering
 Insertion Sort - Example

• Source: Goodrich

5 Amrita Vishwa Vidhyapeetham Ritwik M

Amrita School of Engineering
 Cost Analysis of Insertion Sort

• Time Complexity - O(n2)

• The nested loops of insertion-sort lead to an O(n 2 ) running
time in the worst case. The most work is done if the array is
initially in reverse order.
• On the other hand, if the initial array is nearly sorted or
perfectly sorted, insertion-sort runs in O(n) time because there
are few or no iterations of the inner loop.

6 Amrita Vishwa Vidhyapeetham Ritwik M

Amrita School of Engineering
 2-Bubble Sort

Pseudocode
BubbleSort(inputList)
for all elements of list
if list[i] > list[i+1]
swap(list[i], list[i+1])
end if
end for
return list

7 Amrita Vishwa Vidhyapeetham Ritwik M

Amrita School of Engineering
 2-Bubble Sort Example
• Unsorted List--> 33 14 27 10 35

• Compare first 2 ele 33 14 27 10 35

• Swap them if not in order 14 33 27 10 35

• Compare next pair 14 33 27 10 35

14 27 33 10 35

14 27 10 33 35

• Repeat previous steps for 14 27 10 33 35

n-1 times to complete ...
...

10 14 27 33 35
8 Amrita Vishwa Vidhyapeetham Ritwik M
Amrita School of Engineering
 Cost Analysis of Bubble Sort

• Time Complexity: O(n2)

• bubble sort is not a practical sorting algorithm as almost all
other sorting algorithms outperform this
• The only significant advantage that bubble sort has over most
other algorithms is that the ability to detect that the list is sorted
efficiently is built into the algorithm.
• Bubble sort should be avoided in the case of large collections. It will
not be efficient in the case of a reverse-ordered collection.

9 Amrita Vishwa Vidhyapeetham Ritwik M

Amrita School of Engineering
 3-Selection Sort

Algorithm
To sort an array of size n in ascending order:
1: Set MIN to location 0
2: Search the minimum element in the list
3: Swap with value at location MIN
4: Increment MIN to point to next element
5: Repeat until list is sorted

10 Amrita Vishwa Vidhyapeetham Ritwik M

Amrita School of Engineering
 3-Selection Sort Example
• Unsorted Array--> 14 33 27 10 35 19 42 44

20 33 27 10 35 19 42 44
• Find smallest
• Swap with current 10 33 27 20 35 19 42 44

• Mark current&Move 10 33 27 20 35 19 42 44

• Repeat until sorted 10 33 27 20 35 19 42 44

10 19 27 20 35 33 42 44

10 19 27 20 35 33 42 44
….
• Sorted Array 10 19 20 27 33 35 42 44

11 Amrita Vishwa Vidhyapeetham Ritwik M

Amrita School of Engineering
 Cost Analysis of Selection Sort

• Time Complexity: O(n2)

• Selecting the minimum requires scanning n elements ( taking
n−1 comparisons) and then swapping it into the first position.
Finding the next lowest element requires scanning the
remaining n − 1 elements and so on.
• Comparison with other sorts
– Outperforms bubble sort as average case is better
– Insertion sort is better than selection as it only scans as many elements
as it needs in order to place the k + 1st element, while selection sort
must scan all remaining elements to find the k + 1st element.
– selection sort is preferable to insertion sort in terms of number of writes
(Θ(n) swaps versus Ο(n2) swaps)
12 Amrita Vishwa Vidhyapeetham Ritwik M
Amrita School of Engineering
 4-Merge Sort
• Pseudocode

MergeSort(S) merge(S1, S2, S):

”””Merge two sorted lists S1 and S2 into properly sized list
n= len(S) S.”””
1. i=j=0
If r > l
2. while i + j < len(S):
1. Find the middle point to divide the array into two
halves: if j == len(S2) or (i < len(S1) and S1[i] < S2[j]):

mid = n/2 # copy ith element of S1 as next item of S

S1 = S[0:mid] S[i+j] = S1[i]
S2 = S[mid:n] I += 1
2. Call mergeSort for first half: else:
# copy jth element of S2 as next item of S
Call mergeSort(S1)
S[i+j] = S2[j]
3. Call mergeSort for second half: j += 1
Call mergeSort(S2)
4. Merge the two halves sorted in step 2 and 3:
Call merge(S1,S2,S)

13 Amrita Vishwa Vidhyapeetham Ritwik M

Amrita School of Engineering
 4-Merge Sort example

Source: Goodrich

14 Amrita Vishwa Vidhyapeetham Ritwik M

Amrita School of Engineering
 4-Merge Sort example

Source: Goodrich

15 Amrita Vishwa Vidhyapeetham Ritwik M

Amrita School of Engineering
 4-Merge Sort example

Source: Goodrich

16 Amrita Vishwa Vidhyapeetham Ritwik M

Amrita School of Engineering
 4-Merge Sort example

Source: Goodrich

17 Amrita Vishwa Vidhyapeetham Ritwik M

Amrita School of Engineering
 Cost Analysis of Merge Sort

• Step1: Analyzing the running time of the merge() algorithm.

– Let n 1 and n 2 be the number of elements of S 1 and S 2 , respectively.
– During each iteration of the loop, one element is copied from either S 1
or S 2 into S (and that element is considered no further).
– Therefore, the number of iterations of the loop is n 1 + n 2 . Thus, the
running time of algorithm merge is O(n 1 + n 2 ).
• Step 2 Analyzing the running time of the mergesort() algorithm.
• In the case of our merge sort function, we account for the time to divide the
sequence into two subsequences, and the call to merge to combine the two
sorted sequences, but we exclude the two recursive calls to merge sort

18 Amrita Vishwa Vidhyapeetham Ritwik M

Amrita School of Engineering
 Cost Analysis of Merge Sort

• Step 2 Analyzing the running time of the mergesort() algorithm.

19 Amrita Vishwa Vidhyapeetham Ritwik M

Amrita School of Engineering
 A simple discussion
• Height of a balanced binary tree is log(n)
– Here log => log to the base 2
• the binary logarithm of 1 is 0

• the binary logarithm of 2 is 1

• the binary logarithm of 3 is 1

• the binary logarithm of 4 is 2

• the binary logarithm of 5, 6, 7 is 2

• the binary logarithm of 8-15 is 3

• the binary logarithm of 16-31 is 4

• and so on

• Now look at the tree in the previous slide again

• Therefore Time complexity is O(n log n)
20 Amrita Vishwa Vidhyapeetham Ritwik M
Amrita School of Engineering
 5 – Quick Sort

• Like merge-sort, this algorithm is also based on the divide-and-

conquer paradigm, but it uses this technique in a somewhat
opposite manner, as all the hard work is done before the
recursive calls.
• The quick-sort algorithm sorts a sequence S using a simple
recursive approach.
• The main idea is to apply the divide-and-conquer technique,
whereby we divide S into sub-sequences, recur to sort each
sub-sequence, and then combine the sorted sub-sequences by
a simple concatenation

21 Amrita Vishwa Vidhyapeetham Ritwik M

Amrita School of Engineering
 5 – Quick Sort

• Algorithm Quick Sort()

– 1. Divide: If S has at least two elements (nothing needs to be done if S has zero
or one element), select a specific element x from S, which is called the pivot. As
is common practice, choose the pivot x to be the last element in S. Remove all
the elements from S and put them into three sequences:
• L, storing the elements in S less than x

• E, storing the elements in S equal to x(usually only the pivot if all elements are distinct)

• G, storing the elements in S greater than x

– 2. Conquer: Recursively sort sequences L and G.

– 3. Combine: Put back the elements into S in order by first inserting the elements
of L, then those of E, and finally those of G.

22 Amrita Vishwa Vidhyapeetham Ritwik M

Amrita School of Engineering
 5 – Quick Sort Example

23 Amrita Vishwa Vidhyapeetham Ritwik M

Amrita School of Engineering
 5 – Quick Sort Example

24 Amrita Vishwa Vidhyapeetham Ritwik M

Amrita School of Engineering
 5 – Quick Sort Example

25 Amrita Vishwa Vidhyapeetham Ritwik M

Amrita School of Engineering
 5 – Quick Sort Example

26 Amrita Vishwa Vidhyapeetham Ritwik M

Amrita School of Engineering
 5 – Quick Sort Example

27 Amrita Vishwa Vidhyapeetham Ritwik M

Amrita School of Engineering
 Cost Analysis of Quick Sort

• The divide step and the final concatenation of quick-sort can be

implemented in linear time. Thus, the time spent at a node v of T is
proportional to the input size n
• Since subsequence E has at least one element (the pivot), the sum of the
input sizes of the children of v is at most n − 1
• We can therefore bound the overall running time of an execution of quick-
sort as O(n · h) where h is the overall height of the quick-sort tree T for that
execution.
– But worst case height of the tree is O(n)
– So Time Complexity of Quick sort is O(n2)
– Then why is it Quick Sort?

28 Amrita Vishwa Vidhyapeetham Ritwik M

Amrita School of Engineering
 Randomized Quick Sort
• One common method for analyzing quick-sort is to assume that the pivot will always
divide the sequence in a reasonably balanced manner.
• The way to get close to the best-case running time is for the pivot to divide the input
sequence S almost equally.
– This would result in a running time that is asymptotically the same as the best-
case running time.
• i.e., having pivots close to the “middle” of the set of elements leads to an
O(n log n) running time for quick-sort.
• Instead of picking the pivot as the first or last element of S, we pick an element of S
at random as the pivot, keeping the rest of the algorithm unchanged.
– This variation of quick-sort is called randomized quick-sort.
– The expected running time of randomized quick-sort on a sequence S of size n
is O(n log n)

29 Amrita Vishwa Vidhyapeetham Ritwik M

Amrita School of Engineering
 In-Place Quick Sort
• An algorithm is in-place if it uses only a small
amount of memory in addition to that needed
for the original input.
• In-place quick-sort modifies the input sequence
using element swapping and does not explicitly
create subsequences.
– Instead, a subsequence of the input
sequence is implicitly represented by a
range of positions specified by a leftmost
index a and a rightmost index b.
– The divide step is performed by scanning
the array simultaneously using local
variables left, which advances forward, and
right, which advances backward, swapping
pairs of elements that are in reverse order

30 Amrita Vishwa Vidhyapeetham Ritwik M

Amrita School of Engineering
 Exercises
• Suppose S is a sequence of n values, each equal to 0 or 1. How long will it take to
sort S with
– Bubble sort, Selection sort, Insertion Sort, Merge Sort, Quick Sort
• What is the best algorithm for sorting each of the following: general comparable
objects:
– long character strings, 32-bit integers, double-precision floating-point numbers,
and bytes? Justify your answer.
• Describe and analyze an efficient method for removing all duplicates from a
collection A of n elements
• Consider a version of deterministic quick-sort where we pick as our pivot the median
of the d last elements in the input sequence of n elements, for a fixed, constant odd
number d ≥ 3. What is the asymptotic worst-case running time of quick-sort in this
case?

31 Amrita Vishwa Vidhyapeetham Ritwik M

Amrita School of Engineering
 More Exercises
• Suppose we are given an n-element sequence S such that each element in S
represents a different vote for a class election, where each vote is given as an
integer representing a particular candidate, yet the integers may be arbitrarily large
(even if the number of candidates is not). Design an O(n log n) time algorithm to see
who wins the election S represents, assuming the candidate with the most votes
wins.
• In the previous case suppose that we know the number k < n of candidates running,
even though the integer IDs for those candidates can be arbitrarily large. Describe
an O(n log k) time algorithm for determining who wins the election.
• Suppose we are given two sequences A and B of n elements, possibly containing
duplicates, on which a total order relation is defined. Describe an efficient algorithm
for determining if A and B contain the same set of elements. What is the running
time of this method?
• Given an array A of n integers in the range [0, n 2 − 1], describe a simple method for
sorting A in O(n) time.

32 Amrita Vishwa Vidhyapeetham Ritwik M

Amrita School of Engineering
 Quick Summary

• Sorting Algorithms
– Bubble
– Selection
– Insertion
– Merge
– Quick
• Time Complexity of the sorting Algorithms
• Examples
• Exercises

33 Amrita Vishwa Vidhyapeetham Ritwik M

Amrita School of Engineering
 Up Next

Linear Data Structures

34 Amrita Vishwa Vidhyapeetham Ritwik M

Amrita School of Engineering