Chapter Eight

ADVANCED SORTING AND SEARCHING ALGORITHMS

DATA STRUCTURES AND ALGORITHM BY: KIBRU G.

 Outline
▪ Advanced Seach Algorithm:
▪ Merge Sort
▪ Quick Sort
▪ Heap Sort
▪ Shell Sort

▪ Advanced Seach Algorithm:

▪ Hashing (Reading Assignment …)

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 2
Merge Sort
▪ Merge Sort is one of the most popular sorting algorithms that is based on the principle of Divide and Conquer Algorithm.
▪ Here, a problem is divided into multiple sub-problems. Each sub-problem is solved individually. Finally, sub-problems
are combined to form the final solution.
▪ So, in this algorithm, the array is initially divided into two equal halves and then they are combined in a sorted
manner.
▪ We can think of it as a recursive algorithm that continuously splits the array in half until it cannot be further divided.
▪ This means that if the array becomes empty or has only one element left, the dividing will stop, i.e. it is the base case to stop
the recursion.
▪ Finally, when both the halves are sorted, the merge operation is applied.
▪ Merge operation is the process of taking two smaller sorted arrays and combining them to eventually make a larger
one.

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 3
Merge Sort
▪ The general pseudo-code for the merge sort technique is given below.
▪ Declare an array Arr of length N
▪ If N=1, Arr is already sorted
▪ If N>1,
▪ Left = 0, right = N-1
▪ Find middle = (left + right)/2
▪ Call merge_sort(Arr,left,middle) =>sort first half recursively
▪ Call merge_sort(Arr,middle+1,right) => sort second half recursively
▪ Call merge(Arr, left, middle, right) to merge sorted arrays in above steps.
▪ Exit

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 4
Merge Sort: Example

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 5
Quick Sort
▪ Like Merge Sort, Quick Sort follows a Divide and Conquer algorithm. It picks an element as a pivot and
partitions the given array around the picked pivot.

▪ The pivot is positioned in such a manner that all the elements smaller than the pivot element are to its left and
all the elements greater than the pivot are to its right.

▪ The elements to the left and right of the pivot form two separate arrays.

▪ Now the left and right subarrays are sorted using this same approach.

▪ This process continues until each subarray consists of a single element.

▪ When this situation occurs, the array is now sorted and we can merge the elements to get the
required array.

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 6
Quick Sort
▪ How to select the pivot element?
▪ There are 4 common ways of selecting a pivot. One can use any one of the following methods:

1. Pick the first element

2. Pick the last element

3. Pick a random element

4. Pick the median element

▪ In our discussion, we will be picking the last element as the pivot.

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 7
Quick Sort: Algorithm
▪ The quick algorithm consists of two different algorithms, one for partitioning the array according to the pivot and one for actual sorting. Let us
look at both the algorithms one by one. Let us first discuss the algorithm of partitioning the array.

Step 1: Take two elements low and high

Step 2: Make low store the starting index of the array

Step 3: Make high store the last index of the array

Step 4: Take a variable j and initialize it to low

Step 5: Take pivot as the last element is the array

Step 6: Traverse through the entire array using the variable j and compare each element with pivot

Step 7: If arr[j] < pivot then increment i and swap element as position i and j

Step 8: When the loop terminates, return j – 1

Step 9: Finally, we place pivot at correct position by swapping arr[i+1] and arr[high] (or pivot)

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 8
Quick Sort: Algorithm
▪ The value returned by the partitioning algorithm gives us the correct position or index of the pivot in the
array.
▪ What is meant by correct position is that all elements preceding the pivot are smaller than the pivot
and all the elements after the pivot are greater than it.
▪ Let us now look at the sorting algorithm.
Step 1: Take pivot as the last element in the array
Step 2: Find the correct index of the pivot using the partition algorithm
Step 3: Recursively call Quicksort on the left subarray
Step 4: Recursively call Quicksort on the right subarray

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 9
Quick Sort: Example
▪ Consider: arr[] = {10, 80, 30, 90, 40, 50, 70}
▪ Indexes: 0 1 2 3 4 5 6
▪ low = 0, high = 6, pivot = arr[h] = 70
▪ Initialize index of smaller element, i = -1

▪ Traverse elements from j = low to high-1

▪ j = 0: Since arr[j] <= pivot, do i++ and swap(arr[i], arr[j])
▪ i=0
▪ arr[] = {10, 80, 30, 90, 40, 50, 70} // No change as i and j are same

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 10
Quick Sort: Example
▪ j = 1: Since arr[j] > pivot, do nothing

▪ j = 2 : Since arr[j] <= pivot, do i++ and swap(arr[i], arr[j])

▪ i=1
▪ arr[] = {10, 30, 80, 90, 40, 50, 70} // We swap 80 and 30

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 11
Quick Sort: Example
▪ j = 3 : Since arr[j] > pivot, do nothing // No change in i and arr[]
▪ j = 4 : Since arr[j] <= pivot, do i++ and swap(arr[i], arr[j])
▪ i=2
▪ arr[] = {10, 30, 40, 90, 80, 50, 70} // 80 and 40 Swapped

▪ j = 5 : Since arr[j] <= pivot, do i++ and swap arr[i] with arr[j]
▪ i=3
▪ arr[] = {10, 30, 40, 50, 80, 90, 70} // 90 and 50 Swapped

▪ We come out of loop because j is now equal to high-1.

▪ Finally, we place pivot at correct position by swapping arr[i+1] and arr[high] (or pivot)
▪ arr[] = {10, 30, 40, 50, 70, 90, 80} // 80 and 70 Swapped

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 12
Quick Sort: Example
▪ Now 70 is at its correct place. All elements smaller than 70 are before it and all elements greater than 70
are after it.
▪ Since quick sort is a recursive function, we call the partition function again at left and right partitions

▪ Again, call function at right part and swap 80 and 90

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 13
Heap Sort
▪ Heap Sort is a popular and efficient sorting algorithm in computer programming.

▪ Heap sort works by visualizing the elements of the array as a special kind of complete binary
tree called a heap.

▪ Note: As a prerequisite, you must know about a complete binary tree and heap data
structure.

▪ Heap sort is a comparison-based sorting technique based on Binary Heap data structure.

▪ “It is similar to selection sort where we first find the minimum element and place the minimum
element at the beginning. We repeat the same process for the remaining elements.”

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 14
Heap Sort
▪ Relationship between Array Indexes and Tree Elements:
▪ A complete binary tree has an interesting property that we can use to find the children and parents of any
node.

▪ If the index of any element in the array is i, the element in the index 2i + 1 will become the left child and
element in 2i + 2 index will become the right child.

▪ Also, the parent of any element at index i is given by the lower bound of (i – 1)/2

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 15
Heap Sort
▪ Relationship between Array Indexes and Tree Elements:

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 16
Heap Sort
▪ Relationship between Array Indexes and Tree Elements:
▪ Let us also confirm that the rules hold for finding parent of any node

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 17
Heap Sort
▪ What is Heap Data Structure?
▪ Heap is a special tree-based data structure. A binary tree is said to follow a heap data structure if,
▪ it is a complete binary tree
▪ All nodes in the tree follow the property that they are greater than their children i.e. the largest
element is at the root and both its children and smaller than the root and so on. Such a heap is
called a max-heap.
▪ If instead, all nodes are smaller than their children, it is called a min-heap

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 18
Heap Sort
▪ How to "heapify" a tree
▪ Starting from a complete binary tree, we can modify it to become a Max-Heap by running a
function called heapify on all the non-leaf elements of the heap.

▪ Since heapify uses recursion, it can be difficult to grasp. So, let's first think about how you would
heapify a tree with just three elements.

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 19
Heap Sort
▪ How to "heapify" a tree

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 20
Heap Sort
▪ How to "heapify" a tree
▪ Now let's think of another scenario in which there is more than one level.

▪ The top element isn't a max-heap but all the sub-trees are max-heaps.

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 21
Heap Sort
▪ How to "heapify" a tree
▪ To maintain the max-heap property for the
entire tree, we will have to keep pushing 2
downwards until it reaches its correct
position.
▪ Thus, to maintain the max-heap property in a
tree where both sub-trees are max-heaps,
we need to run heapify on the root element
repeatedly until it is larger than its children or
it becomes a leaf node.

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 22
Heap Sort
▪ How to "heapify" a tree
▪ We can combine both these conditions in
one heapify function as:
▪ This function works for both the base case
and for a tree of any size. We can thus move
the root element to the correct position to
maintain the max-heap status for any tree
size as long as the sub-trees are max-heaps.

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 23
Heap Sort
▪ Build max-heap

▪ To build a max-heap from any tree, we can thus start heapifying each sub-tree from the bottom up
and end up with a max-heap after the function is applied to all the elements including the root
element.

▪ In the case of a complete tree, the first index of a non-leaf node is given by n/2 – 1. All other nodes
after that are leaf-nodes and thus don't need to be heapified.

▪ So, we can build a maximum heap as:

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 24
Heap Sort
▪ Build max-heap

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 25
Heap Sort
▪ Build max-heap

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 26
Heap Sort
▪ Build max-heap

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 27
Heap Sort
▪ Build max-heap

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 28
Heap Sort
▪Working of Heap Sort
1. Since the tree satisfies Max-Heap property, then the largest item is stored at the root
node.
2.Swap: Remove the root element and put at the end of the array (nth position) Put the
last item of the tree (heap) at the vacant place.
3.Remove: Reduce the size of the heap by 1.
4.Heapify: Heapify the root element again so that we have the highest element at root.
5.The process is repeated until all the items of the list are sorted.

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 29
Heap Sort
▪ Working of Heap Sort

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 30
Heap Sort
▪ Working of Heap Sort
Heap Sort
▪ Working of Heap Sort
▪ The code below shows the operation:

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 32
Shell Sort
▪ Shell sort is a generalized version of the insertion sort algorithm. In insertion sort, we move elements only one position ahead.
When an element has to be moved far ahead, many movements are involved.

▪ The idea of Shell Sort is to allow the exchange of far items. It first sorts elements that are far apart from each other and
successively reduces the interval between the elements to be sorted.

▪ The interval between the elements is reduced based on the sequence used. Some of the optimal sequences that can be used in
the shell sort algorithm are:
▪ Shell's original sequence: (N/2, N/4, … 1)
▪ Knuth's increments
▪ Sedgewick's increments
▪ Hibbard's increments
▪ Papernov & Stasevich increment

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 33
Shell Sort
▪ Working of Shell Sort:
1. Suppose, we need to sort the following array.

2. We are using the shell's original sequence (N/2, N/4, … 1) as intervals in our algorithm.
In the first loop, if the array size is N=8 then, the elements lying at the interval of N/2=4 are compared and swapped if they
are not in order.
a) The 0th element is compared with the 4th
b) If the 0th element is greater than the 4th one then, the 4th element is first stored in the temp variable and the 0th element (ie.
greater element) is stored in the 4th position and the element stored in temp is stored in the 0th position.

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 34
Shell Sort
▪ Working of Shell Sort:

▪ This process goes on for all the remaining elements:

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 35
Shell Sort
▪ Working of Shell Sort:

3. In the second loop, an interval of (N/4=8/4=2) is taken and again the elements lying at
these intervals are sorted.

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 36
Shell Sort
▪ Working of Shell Sort:
▪ The elements at 4th and 2nd position are
compared. The elements at 2nd and 0th
position are also compared. All the
elements in the array lying at the current
interval are compared.

4. The same process goes on for remaining

elements.

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 37
Shell Sort
▪ Working of Shell Sort:

5. Finally, when the interval is

(N/8=8/8=1) then the array elements
lying at the interval of 1 are sorted.
The array is now completely sorted.

5/27/2025 Data Structures and Algorithms Chapter Eight : Advanced Sorting and searching Algorithms By: Kibru G. 38
Advanced Search Algorithm

Hashing: Reading Assignment

5/27/2025 Data Structures and Algorithms Chapter Eight: Advanced Sorting and searching Algorithms By: Kibru G. 39
 The End
DATA STRUCTURES AND ALGORITHMS

5/27/2025 Data Structures and Algorithms Chapter Eight: Advanced Sorting and searching Algorithms By: Kibru G. 40