Data Structures and Algorithms

Bule Hora University

 Chapter three: Advanced Sorting
Algorithms
This chapter covers:
Shell Sort
 Heap Sort
Quick Sort
Merge Sort

2
 Shell Sort
The shell sort, also known as the diminishing increment
sort, was developed by Donald L. Shell in 1959.
The idea behind shell sort is that it is faster to sort an
array if parts of it are already sorted.
The original array is first divided into a number of smaller
subarrays, these subarrays are sorted, and then they are
combined into the overall array and this is sorted.

Pseudocode for shell sort:

Divide data into h subarrays
for (i = 1; i < h; i++)
Sort subarray i
Sort array data 3
 How should the original array be divided into
subarrays?
One approach would be to divide the array into a
number of subarrays consisting of contiguous
elements (i.e. elements that are next to each other).
For example, the array [abcdef] could be divided into
the subarrays [abc] and [def]. However, shell sort uses
a different approach: the subarrays are constructed
by taking elements that are regularly spaced from
each other.
For example, a subarray may consist of every second
element in an array, or every third element, etc.
For example, dividing the array [abcdef] into two
subarrays by taking every second element results in
the subarrays [ace] and [bdf]
5
 …
Actually, shell sort uses several iterations of this technique.
First, a large number of subarrays, consisting of widely spaced
elements, are sorted.
Then, these subarrays are combined into the overall array, a
new division is made into a smaller number of subarrays and
these are sorted.
In the next iteration a still smaller number of subarrays is
sorted.
This process continues until eventually only one subarray is
sorted, the original array itself.
Gap1= floor(N/2)
Gap2=floor(gap1/2)
Gap3=floor(gap2/2)…. 6
How shell sort works
We take the below example to have an idea, how
shell sort works? We take the same array we
have used in our previous examples. For our
example and ease of understanding we take the
interval of 4. And make a virtual sublist of all
values located at the interval of 4 positions. Here
these values are {35, 14}, {33, 19}, {42, 27}
and {10, 14}
Then we take interval of 2 and this gap generates two sublists - {14,
27, 35, 42}, {19, 10, 33, 44}

We compare and swap the values, if required, in the original array. After
this step, this array should look like this −
…

14
 In the above example:

In the first iteration, five subarrays are constructed by taking

every fifth element.
These subarrays are sorted (this phase is known as the 5-sort).
In the second iteration, three subarrays are constructed by
taking every third element, and these arrays are sorted (the 3-
sort).
In the final iteration, the overall array is sorted (the 1-sort).
Notice that the data after the 3-sort but before the 1-sort is
almost correctly ordered, so the complexity of the final 1-sort
is reduced.

15
In the above example, we used three iterations: a 5-
sort, a 3-sort and a 1-sort.
This sequence is known as the diminishing increment.
But how do we decide on this sequence of increments?
Unfortunately, there is no definitive answer to this
question. In the original algorithm proposed by
Donald L. Shell, powers of 2 were used for the
increments, e.g. 16, 8, 4, 2, 1.
However, this is not the most efficient technique.
Experimental studies have shown that increments
calculated according to the following conditions lead
to better efficiency:
h1 = 1
hi+1 = 3hi + 1
For example, for a list of length 10,000 the sequence of
increments would be 3280, 1093, 364, 121, 40, 13, 4, 1.
16
Another decision that must be made with shell sort is
what sorting algorithms should be used to sort the
subarrays at each iteration?
A number of different choices have been made: for
example, one technique is to use insertion sort for
every iteration and bubble sort for the last iteration.
Actually, whatever simple sorting algorithms are used
for the different iterations, shell sort performs better
than the simple sorting algorithms on their own.
It may seem that shell sort should be less efficient,
since it performs a number of different sorts, but
remember that most of these sorts are on small arrays,
and in most cases the data is already almost sorted.
An experimental analysis has shown that the
complexity of shell sort is approximately O(n1.25),
which is better than the O(n2) offered by the simple
algorithms. 17
 Quick Sort
Quicksort was developed by C. A. R. Hoare in 1961, and is
probably the most famous and widely used of sorting
algorithms.
The guiding principle behind the algorithm is similar to
that of shell sort: it is more efficient to sort a number of
smaller subarrays than to sort one big array

18
 The Algorithm
In quicksort the original array is first divided into two
subarrays, the first of which contains only elements that
are less than or equal to a chosen element, called the
bound or pivot.
The second subarray contains elements that are greater
than or equal to the bound. If each of these subarrays is
sorted separately they can be combined into a final
sorted array.
To sort each of the subarrays, they are both subdivided
again using two new bounds, making a total of 4
subarrays.
The partitioning is repeated again for each of these
subarrays, and so on until the subarrays consist of a
single element each, and do not need to be 19
sorted
 …
Quicksort is inherently recursive in nature, since it consists
of the same simple operation (the partitioning) applied to
successively smaller subarrays.
Pseudocode for quicksort is given below:

20
 …
The pivot element can be selected in many ways. We
may select the first element as a pivot [see the demo], or
we may select the middle element of the array or may be
a randomly selected pivot [bound] element can be used.
The following example demonstrates a pivot element
selected from the middle of the data. In the average,
quicksort has complexity of nlogn.

21
QuickSort Pivot Algorithm
Based on our understanding of partitioning in quicksort, we should
now try to write an algorithm for it here.
Step 1 − Choose the highest index value has pivot
Step 2 − Take two variables to point left and right of the list
excluding pivot
Step 3 − left points to the low index
Step 4 − right points to the high
Step 5 − while value at left is less than pivot move right
Step 6 − while value at right is greater than pivot move left
Step 7 − if both step 5 and step 6 does not match swap left and right
Step 8 − if left ≥ right, the point where they met is new pivot
Example [ pivot selected from the middle]

23
 In the above Example
To begin with, the middle element 6 is chosen as the
bound, and the array partitioned into two subarrays.
Each of these subarrays is then partitioned using the
bounds 2 and 10 respectively.
In the third phase, two of the four subarrays only have
one element, so they are not partitioned further.
The other two subarrays are partitioned once more,
using the bounds 3 and 7.
Finally we are left with only subarrays consisting of a
single element. At this point, the subarrays and bounds
are recombined successively, resulting in the sorted
array. 24
 Merge Sort
Mergesort also works by successively partitioning the
array into two subarrays, but it guarantees that the
subarrays are of approximately equal size.
This is possible because in mergesort the array is
partitioned without regard to the values of their
elements: it is simply divided down the middle into
two halves.
Each of these halves is recursively sorted using the
same algorithm.
After the two subarrays have been sorted, they are
merged back together again.
The recursion continues until the subarrays consist of
a single element, in which case they are already sorted.
25
Merge sort keeps on dividing the list into equal
halves until it can no more be divided. By
definition, if it is only one element in the list, it is
sorted. Then merge sort combines smaller sorted
lists keeping the new list sorted too.
Pseudocode code for merge sort

27
Example

28
Example of merge
 Conti..

We first compare the element for each list and then combine
them into another list in sorted manner. We see that 14 and
33 are in sorted positions. We compare 27 and 10 and in the
target list of 2 values we put 10 first, followed by 27. We
change the order 19 and 35. 42 and 44 are placed
sequentially.
 ….
Merge sort is a sorting technique based on divide
and conquer technique. With worst-case time
complexity being Ο(n log n), it is one of the most
respected algorithms.
Merge sort first divides the array into equal
halves and then combines them in a sorted
manner.
 Heap Sort
Heap is a binary tree that has two properties:
The value of each node is greater than or equal to the values
stored in each of its children.
The tree is perfectly balanced, and the leaves in the last level are
all in the leftmost positions. (filled from left to right)
A property of the heap data structure is that the largest
element is always at the root of the tree.
A common way of implementing a heap is to use an array

34
 Cont…
Min-Heap − where the value of root node is less than or equal
to either of its children.

For Input → 35 33 42 10 14 19 27 44 26 31
•
 Cont..
•Max-Heap − where the value of root node is
greater than or equal to either of its children.
 Heap Sort…
The heap sort works by first rearranging the input array
so that it is a heap, and then removing the largest elements
from the heap one by one. Pseudocode for the heap sort is
given below:
HeapSort(data):
Transform data into a heap
for (i = n-1; i > 1; i--)

Swap the root with element in position i

Restore the heap property for the tree data[0] … data[i-1]

40
Example

41