Sorting

Data Structures & Algo- Dr Ahmar Rashid 1

Introduction
Sorting is the process of ordering a list of objects,
according to some linear order, such as
a
1
a
2
a
3
a
4
a
5
a
6


Sorting can be divided into two parts i.e.
Internal, and
external.

Internal sorting
When all the data is stored in the main memory
The random access nature of the memory can be utilized
External sorting
When the size of data too large
The main memory cannot accommodate all the data
Virtual memory (swapping the data in/out of the memory) is
thus used
Data Structures & Algo- Dr Ahmar Rashid 2
Introduction
External sorting
The bottleneck is usually the data movement
between the secondary storage and the main
memory.
Data movement is efficient if it is moved in the
form of large blocks.
However, moving large blocks of data is efficient
only if it is physically located in contiguous
locations.
Data Structures & Algo- Dr Ahmar Rashid 3
Introduction
The simplest algorithms usually take O(n
2
)
time to sort n objects, and suited for sorting
short lists.

One of the most popular algorithms is Quick-
Sort takes O(nlogn) time on average.

Quick-Sort works for most common
applications, although in worst case it can
take O(n
2
) time.
Data Structures & Algo- Dr Ahmar Rashid 4
Introduction
There are other sorting techniques, such as
Merge-Sort and Heap-Sort that take time
O(nlogn) in worst case.

Merge-Sort however, is well suited for
external sorting.

There are other algorithms such as bucket
and radix sort when the keys are integers
of known range. They take time O(n).
Data Structures & Algo- Dr Ahmar Rashid 5
Introduction
The sorting problem is to arrange a sequence of records so that the
values of their key fields form a non-decreasing sequence.

Given records r
1
, r
1
, . r
n
with key values k
1
, k
1
, . k
n
, respectively
we must produce the same records in an order ri
1
, ri
2
, . ri
n
such that
the keys are in the corresponding non-decreasing order.
key(ri
1
) key(ri
2
) key(ri
3
) key(ri
4
) key(ri
5
) key(ri
6
)
ki
1
ki
2
ki
3
ki
4
ki
5
ki
6

The records may NOT have distinct values, and can appear in any
order.

Different criteria to evaluate the running time, as follows:
Number of algorithm steps.
Number of comparisons between the keys (for expensive comparisons).
The number of times a record is moved (for large records).

Data Structures & Algo- Dr Ahmar Rashid 6
Bubble Sort
One of the simplest sorting methods.

The basic idea is the weight of the record.

The records are kept in an array held vertically.

light records bubbling up to the top.

We make repeated passes over the array from bottom
to top.

If two adjacent elements are out of order i.e.
lighter one is below, we reverse the order.
Data Structures & Algo- Dr Ahmar Rashid 7
Bubble Sort
The overall effect, is that after the first pass
the lightest record will bubble all the way
to the top.

On the second top pass, the second lowest
rises to the second position, and so on.

On second pass we need not try bubbling to
the top position, because we know that the
lightest record is already there.

Data Structures & Algo- Dr Ahmar Rashid 8
Bubble Sort
int n =N; // N is the size of the array;
for (int i = 0; i < N; i++){
for (int j = 1; j < n; j++) {
if (A[j] < A[j-1]) {
swap(j-1 , j);
}//end if
} //end inner for
} //end inner for
Complexity ?
O(N
2
)

Data Structures & Algo- Dr Ahmar Rashid 9

Algorithm does not exit
until all the data is checked
Bubble Sort
int n =N; // N is the size of the array;
for (int i = 0; i < N; i++){
int swapped = 0;
for (int j = 1; j < n; j++) {
if (A[j] < A[j-1]) {
swap(j-1 , j);
swapped = 1;
}//end if
} //end inner for
// n = n-1;
if (swapped == 0)
break;
} //end inner for
Complexity ?
O(N
2
)
Data Structures & Algo- Dr Ahmar Rashid 10

Algorithm exits if no swap done
in previous (outer loop) step
Bubble Sort
int n =N; // N is the size of the array;
for (int i = 0; i < N; i++){
int swapped = 0;
for (int j = 1; j < n; j++) {
if (A[j] < A[j-1]) {
swap(j-1 , j);
swapped = 1;
}//end if
} //end inner for
n = n-1;
if (swapped == 0)
break;
} //end inner for
Complexity ?
O(N
2
)
Data Structures & Algo- Dr Ahmar Rashid 11

Algorithm exits if no swap done
in previous (outer loop) step
No bubbling to the top position, because
the lightest record is already there.
Bubble Sort Example (First Pass)
62
(0)
58
(1)
55
(2)
10
(3)
45
(4)
44
(5)
6
(6)
90
(7)
i = 0
j = 1
58
(0)
62
(1)
55
(2)
10
(3)
45
(4)
44
(5)
6
(6)
90
(7)
58
(0)
55
(1)
62
(2)
10
(3)
45
(4)
44
(5)
6
(6)
90
(7)
j = 2
58
(0)
55
(1)
10
(2)
62
(3)
45
(4)
44
(5)
6
(6)
90
(7)
j = 3
58
(0)
55
(1)
10
(2)
45
(3)
62
(4)
44
(5)
6
(6)
90
(7)
j = 4
58
(0)
55
(1)
10
(2)
45
(3)
44
(4)
62
(5)
6
(6)
90
(7)
j = 5
58
(0)
55
(1)
10
(2)
45
(3)
44
(4)
6
(5)
62
(6)
90
(7)
j = 6
58
(0)
55
(1)
10
(2)
45
(3)
44
(4)
6
(5)
62
(6)
90
(7)
j = 7
swap(A[0] , A[1])
swap(A[1] , A[2])
swap(A[2] , A[3])
swap(A[5] , A[6])
swap(A[3] , A[4])
swap(A[4] , A[5])
int n =N; // N is the size of the array;
for (int i = 0; i < N; i++){
int swapped = 0;
for (int j = 1; j < n; j++)
if (A[j] < A[j-1]) {
swap(j-1 , j); swapped = 1;
}//end if
n = n-1;
if (swapped == 0)
break; }//End outer for

Data Structures & Algo- Dr Ahmar Rashid 12
Bubble Sort Example (Second Pass)
i = 0
j = 1
j = 2
j = 3
j = 4
j = 5
j = 6
swap(A[0] , A[1])
swap(A[1] , A[2])
swap(A[2] , A[3])
swap(A[3] , A[4])
swap(A[4] , A[5])
int n =N; // N is the size of the array;
for (int i = 0; i < N; i++){
int swapped = 0;
for (int j = 1; j < n; j++)
if (A[j] < A[j-1]) {
swap(j-1 , j); swapped = 1;
}//end if
n = n-1;
if (swapped == 0)
break; }//End outer for

58
(0)
55
(1)
10
(2)
45
(3)
44
(4)
6
(5)
62
(6)
90
(7)
55
(0)
58
(1)
10
(2)
45
(3)
44
(4)
6
(5)
62
(6)
90
(7)
55
(0)
10
(1)
58
(2)
45
(3)
44
(4)
6
(5)
62
(6)
90
(7)
55
(0)
10
(1)
45
(2)
58
(3)
44
(4)
6
(5)
62
(6)
90
(7)
55
(0)
10
(1)
45
(2)
44
(3)
58
(4)
6
(5)
62
(6)
90
(7)
55
(0)
10
(1)
45
(2)
44
(3)
6
(4)
58
(5)
62
(6)
90
(7)
55
(0)
10
(1)
45
(2)
44
(3)
6
(4)
58
(5)
62
(6)
90
(7)
Data Structures & Algo- Dr Ahmar Rashid 13
Bubble Sort Example (Third Pass)
i = 0
j = 1
j = 2
j = 3
j = 4
j = 5
int n =N; // N is the size of the array;
for (int i = 0; i < N; i++){
int swapped = 0;
for (int j = 1; j < n; j++)
if (A[j] < A[j-1]) {
swap(j-1 , j); swapped = 1;
}//end if
n = n-1;
if (swapped == 0)
break; }//End outer for

55
(0)
10
(1)
45
(2)
44
(3)
6
(4)
58
(5)
62
(6)
90
(7)
10
(0)
45
(1)
44
(2)
6
(3)
55
(4)
58
(5)
62
(6)
90
(7)
Data Structures & Algo- Dr Ahmar Rashid 14
Bubble Sort Example (Fourth Pass)
i = 0
j = 1
j = 2
j = 3
j = 4
int n =N; // N is the size of the array;
for (int i = 0; i < N; i++){
int swapped = 0;
for (int j = 1; j < n; j++)
if (A[j] < A[j-1]) {
swap(j-1 , j); swapped = 1;
}//end if
n = n-1;
if (swapped == 0)
break; }//End outer for

10
(0)
45
(1)
44
(2)
6
(3)
55
(4)
58
(5)
62
(6)
90
(7)
10
(0)
44
(1)
6
(2)
45
(3)
55
(4)
58
(5)
62
(6)
90
(7)
Data Structures & Algo- Dr Ahmar Rashid 15
Bubble Sort Example (Fifth Pass)
i = 0
j = 1
j = 2
j = 3
int n =N; // N is the size of the array;
for (int i = 0; i < N; i++){
int swapped = 0;
for (int j = 1; j < n; j++)
if (A[j] < A[j-1]) {
swap(j-1 , j); swapped = 1;
}//end if
n = n-1;
if (swapped == 0)
break; }//End outer for

10
(0)
6
(1)
44
(2)
45
(3)
55
(4)
58
(5)
62
(6)
90
(7)
10
(0)
44
(1)
6
(2)
45
(3)
55
(4)
58
(5)
62
(6)
90
(7)
Data Structures & Algo- Dr Ahmar Rashid 16
Bubble Sort Example (Sixth Pass)
i = 0
j = 2
int n =N; // N is the size of the array;
for (int i = 0; i < N; i++){
int swapped = 0;
for (int j = 1; j < n; j++)
if (A[j] < A[j-1]) {
swap(j-1 , j); swapped = 1;
}//end if
n = n-1;
if (swapped == 0)
break; }//End outer for

6
(0)
10
(1)
44
(2)
45
(3)
55
(4)
58
(5)
62
(6)
90
(7)
10
(0)
6
(1)
44
(2)
45
(3)
55
(4)
58
(5)
62
(6)
90
(7)
j = 1
Data Structures & Algo- Dr Ahmar Rashid 17
Bubble Sort Example (Seventh Pass)
i = 0
j = 1
int n =N; // N is the size of the array;
for (int i = 0; i < N; i++){
int swapped = 0;
for (int j = 1; j < n; j++)
if (A[j] < A[j-1]) {
swap(j-1 , j); swapped = 1;
}//end if
n = n-1;
if (swapped == 0)
break; }//End outer for

6
(0)
10
(1)
44
(2)
45
(3)
55
(4)
58
(5)
62
(6)
90
(7)
6
(0)
10
(1)
44
(2)
45
(3)
55
(4)
58
(5)
62
(6)
90
(7)
Data Structures & Algo- Dr Ahmar Rashid 18
Bubble Sort
int n =N; // N is the size of the array;
for (int i = 0; i < N; i++){
for (int j = 1; j < n; j++) {
if (A[j] < A[j-1]) {
swap(j-1 , j);
}//end if
} //end inner for
} //end inner for
Complexity ?
O(N
2
)

Data Structures & Algo- Dr Ahmar Rashid 19

Algorithm does not exit
until all the data is checked
Bubble Sort output(1) worst case
Data Structures & Algo- Dr Ahmar Rashid 20
Bubble Sort output(2) mixed data
Data Structures & Algo- Dr Ahmar Rashid 21
Bubble Sort output(3) best case
Data Structures & Algo- Dr Ahmar Rashid 22
Bubble Sort
int n =N; // N is the size of the array;
for (int i = 0; i < N; i++){
int swapped = 0;
for (int j = 1; j < n; j++) {
if (A[j] < A[j-1]) {
swap(j-1 , j);
swapped = 1;
}//end if
} //end inner for
// n = n-1;
if (swapped == 0)
break;
} //end inner for
Complexity ?
O(N
2
)
Data Structures & Algo- Dr Ahmar Rashid 23

Algorithm exits if no swap done
in previous (outer loop) step
Bubble Sort output (1) best case
Data Structures & Algo- Dr Ahmar Rashid 24
Insertion Sort
On the ith pass we insert the ith element
A[i] into its rightful place among
A[1],A[2],A[i-1] which were placed in
sorted order.

After this insertion A[1],A[2],A[i] are in
sorted order.



Data Structures & Algo- Dr Ahmar Rashid 25
Insertion Sort
for (int i =1, i < n, i++){
temp = A[i];
for (int j = i, j>0 && A[j-1] > temp, j--)
A[j] = A[j-1]
A[j] = temp;
}// end outer for

Complexity ?
O(N
2
)

Data Structures & Algo- Dr Ahmar Rashid 26
j > 0 && A [j -1] > temp
62 > 58
A[1] = A[0]
Insertion Sort Example (First Pass)
62
(0)
58
(1)
55
(2)
10
(3)
45
(4)
44
(5)
6
(6)
90
(7)
for (int i =1, i < n, i++){
temp = A[i];
for (int j = i, j > 0 && A[j-1] > temp, j--)
A[j] = A[j-1]
A[j] = temp;
}// end outer for
i = 1
j = 1
62
(0)
62
(1)
55
(2)
10
(3)
45
(4)
44
(5)
6
(6)
90
(7)
temp = 58
A[1] = A[0]
j = 0
58
(0)
62
(1)
55
(2)
10
(3)
45
(4)
44
(5)
6
(6)
90
(7)
A[0] = temp
= 58;
Data Structures & Algo- Dr Ahmar Rashid 27
j = 0 exit
A[j] = temp (= 58)
Insertion Sort Example (Second Pass)
58
(0)
62
(1)
55
(2)
10
(3)
45
(4)
44
(5)
6
(6)
90
(7)
i = 2
j = 2
58
(0)
62
(1)
62
(2)
10
(3)
45
(4)
44
(5)
6
(6)
90
(7)
temp = 55
A[2] = A[1]
j = 1
58
(0)
58
(1)
62
(2)
10
(3)
45
(4)
44
(5)
6
(6)
90
(7)
for (int i =1, i < n, i++){
temp = A[i];
for (int j = i, j > 0 && A[j-1] > temp, j--)
A[j] = A[j-1]
A[j] = temp;
}// end outer for
A[1] = A[0]
j = 0
55
(0)
58
(1)
62
(2)
10
(3)
45
(4)
44
(5)
6
(6)
90
(7)
A[0] = temp
= 55;
Data Structures & Algo- Dr Ahmar Rashid 28
62 > 55
58 > 55
Insertion Sort Example (Third Pass)
55
(0)
58
(1)
62
(2)
10
(3)
45
(4)
44
(5)
6
(6)
90
(7)
i = 3
j = 3
55
(0)
58
(1)
62
(2)
62
(3)
45
(4)
44
(5)
6
(6)
90
(7)
temp = 10
A[3] = A[2]
j = 2
55
(0)
58
(1)
58
(2)
62
(3)
45
(4)
44
(5)
6
(6)
90
(7)
for (int i =1, i < n, i++){
temp = A[i];
for (int j = i, j > 0 && A[j-1] > temp, j--)
A[j] = A[j-1]
A[j] = temp;
}// end outer for
A[2] = A[1]
j = 1
55
(0)
55
(1)
58
(2)
62
(3)
45
(4)
44
(5)
6
(6)
90
(7)
A[0] = temp
= 55;
A[1] = A[0]
j = 0
10
(0)
55
(1)
58
(2)
62
(3)
45
(4)
44
(5)
6
(6)
90
(7)
Data Structures & Algo- Dr Ahmar Rashid 29
Insertion Sort Example (Fourth Pass)
10
(0)
55
(1)
58
(2)
62
(3)
45
(4)
44
(5)
6
(6)
90
(7)
i = 4
j = 4
10
(0)
55
(1)
58
(2)
62
(3)
62
(4)
44
(5)
6
(6)
90
(7)
temp = 45
A[4] = A[3]
j = 3
10
(0)
55
(1)
58
(2)
58
(3)
62
(4)
44
(5)
6
(6)
90
(7)
for (int i =1, i < n, i++){
temp = A[i];
for (int j = i, j > 0 && A[j-1] > temp, j--)
A[j] = A[j-1]
A[j] = temp;
}// end outer for
A[3] = A[2]
j = 2
10
(0)
55
(1)
55
(2)
58
(3)
62
(4)
44
(5)
6
(6)
90
(7)
A[1] = temp
= 45;
A[2] = A[1]
j = 1
10
(0)
45
(1)
55
(2)
58
(3)
62
(4)
44
(5)
6
(6)
90
(7)
Data Structures & Algo- Dr Ahmar Rashid 30
Insertion Sort Example (Fifth Pass)
10
(0)
45
(1)
55
(2)
58
(3)
62
(4)
44
(5)
6
(6)
90
(7)
i = 5
j = 5
10
(0)
45
(1)
55
(2)
58
(3)
62
(4)
62
(5)
6
(6)
90
(7)
temp = 44
A[5] = A[4]
j = 4
10
(0)
45
(1)
55
(2)
58
(3)
58
(4)
62
(5)
6
(6)
90
(7)
for (int i =1, i < n, i++){
temp = A[i];
for (int j = i, j > 0 && A[j-1] > temp, j--)
A[j] = A[j-1]
A[j] = temp;
}// end outer for
A[4] = A[3]
j = 3
10
(0)
45
(1)
55
(2)
55
(3)
58
(4)
62
(5)
6
(6)
90
(7)
A[1] = temp
= 44;
A[3] = A[2]
j = 2
10
(0)
45
(1)
45
(2)
55
(3)
58
(4)
62
(5)
6
(6)
90
(7)
A[2] = A[1]
j = 1
10
(0)
44
(1)
45
(2)
55
(3)
58
(4)
62
(5)
6
(6)
90
(7)
Data Structures & Algo- Dr Ahmar Rashid 31
Insertion Sort Example (Sixth Pass)
10
(0)
44
(1)
45
(2)
55
(3)
58
(4)
62
(5)
6
(6)
90
(7)
i = 6
j = 6
10
(0)
44
(1)
45
(2)
55
(3)
58
(4)
62
(5)
62
(6)
90
(7)
temp = 6
A[6] = A[5]
j = 5
10
(0)
44
(1)
45
(2)
55
(3)
58
(4)
58
(5)
62
(6)
90
(7)
for (int i =1, i < n, i++){
temp = A[i];
for (int j = i, j > 0 && A[j-1] > temp, j--)
A[j] = A[j-1]
A[j] = temp;
}// end outer for
A[5] = A[4]
j = 4
10
(0)
44
(1)
45
(2)
55
(3)
55
(4)
58
(5)
62
(6)
90
(7)
A[0] = temp
= 6;
A[4] = A[3]
j = 3
10
(0)
44
(1)
45
(2)
45
(3)
55
(4)
58
(5)
62
(6)
90
(7)
A[3] = A[2]
j = 2
10
(0)
44
(1)
45
(2)
45
(3)
55
(4)
58
(5)
62
(6)
90
(7)
j = 1
10
(0)
10
(1)
45
(2)
45
(3)
55
(4)
58
(5)
62
(6)
90
(7)
j = 0
6
(0)
10
(1)
45
(2)
45
(3)
55
(4)
58
(5)
62
(6)
90
(7)
A[2] = A[1]
A[1] = A[0]
Data Structures & Algo- Dr Ahmar Rashid 32
Insertion Sort Example (Sixth Pass)
6
(0)
10
(1)
45
(2)
45
(3)
55
(4)
58
(5)
62
(6)
90
(7)
i = 7
temp = 90
A[j-1] >temp?
No
Quit
for (int i =1, i < n, i++){
temp = A[i];
for (int j = i, j > 0 && A[j-1] > temp, j--)
A[j] = A[j-1]
A[j] = temp;
}// end outer for Data Structures & Algo- Dr Ahmar Rashid 33
Insertion Sort Example (Sixth Pass)
6
(0)
10
(1)
45
(2)
45
(3)
55
(4)
58
(5)
62
(6)
90
(7)
i = 7
temp = 90
A[j-1] >temp?
No
Quit
for (int i =1, i < n, i++){
temp = A[i];
for (int j = i, j > 0 && A[j-1] > temp, j--)
A[j] = A[j-1]
A[j] = temp;
}// end outer for Data Structures & Algo- Dr Ahmar Rashid 34
Analysis of Insertion Sort
Because of the nested loops, each of which can take
n iterations, insertion sort is O(n
2
).
Furthermore, this bound is tight, because input in
reverse order can actually achieve this bound.
A precise calculation shows that the test at line 3 can
be executed at most i times for each value of i.
Summing over all i gives a total of



If the input is presorted, the running time is O(n)
because the test in the inner for loop always fails
immediately
The average running time also O(n
2
)

1
1
) ( 1 ... 2 1
2
n
i
n n i
Data Structures & Algo- Dr Ahmar Rashid 35
Selection Sort
Find the minimum value in the list
Swap it with the value in the first position
Repeat the steps above for the remainder
of the list (starting at the second position
and advancing each time)

http://en.wikipedia.org/wiki/Selection_sort
Data Structures & Algo- Dr Ahmar Rashid 36
Selection Sort
for (int i =0, i < n, i++){
min = i;
for (int j = i+1, j < n , j++){
if ( A[j] < A[min]){
min = j
} // end if
}// end inner for
swap(i, min)
}// end outer for

Complexity ?
O(N
2
)

// Swap function assumes that
// A[n] is a globally declared array
swap(i, min) {
int temp = A[i];
A[i] = A[min];
A[min] = temp;
}
Data Structures & Algo- Dr Ahmar Rashid 37
The Selection Sort might swap an
array element with itself--this is
harmless.
7 2 8 5 4
2 7 8 5 4
2 4 8 5 7
2 4 5 8 7
2 4 5 7 8
Selection Sort - Example
for (int i =0, i < n, i++){
min = i;
for (int j = i+1, j < n , j++){
if ( A[j] < A[min]){
min = j
} // end if
}// end inner for
swap(i, min)
}// end outer for


Selection Sort Example
62
(0)
58
(1)
55
(2)
10
(3)
45
(4)
44
(5)
6
(6)
90
(7)
for (int i =0, i < n, i++){
min = i;
for (int j = i+1, j < n , j++){
if ( A[j] < A[min]){
min = j
} // end if
}// end inner for
swap(i, min)
}// end outer for


i = 0
min = 6
A[min] = 62 A[min] = 58 A[min] = 55 A[min] = 10 A[min] = 10 A[min] = 10 A[min] = 6 A[min] = 6
6
(0)
58
(1)
55
(2)
10
(3)
45
(4)
44
(5)
62
(6)
90
(7)
i = 1
min = 3
A[min] = 58 A[min] = 55 A[min] = 10 A[min] = 10 A[min] = 10 A[min] = 10 A[min] = 10
6
(0)
10
(1)
55
(2)
58
(3)
45
(4)
44
(5)
62
(6)
90
(7)
i = 2
min = 5
A[min] = 55 A[min] = 58 A[min] = 45 A[min] = 44 A[min] = 44 A[min] = 44
6
(0)
10
(1)
44
(2)
58
(3)
45
(4)
55
(5)
62
(6)
90
(7)
i = 3
min = 4
A[min] = 58 A[min] = 45 A[min] = 45 A[min] = 45 A[min] = 45
Data Structures & Algo- Dr Ahmar Rashid 39
A[min] = 6
A[min] = 10
A[min] = 44
A[min] = 45
Selection Sort Example - continued
62
(0)
58
(1)
55
(2)
10
(3)
45
(4)
44
(5)
6
(6)
90
(7)
for (int i =0, i < n, i++){
min = i;
for (int j = i+1, j < n , j++){
if ( A[j] < A[min]){
min = j
} // end if
}// end inner for
swap(i, min)
}// end outer for


i = 0
min = 6
6
(0)
58
(1)
55
(2)
10
(3)
45
(4)
44
(5)
62
(6)
90
(7)
i = 1
min = 3
6
(0)
10
(1)
55
(2)
58
(3)
45
(4)
44
(5)
62
(6)
90
(7)
i = 2
min = 5
6
(0)
10
(1)
44
(2)
58
(3)
45
(4)
55
(5)
62
(6)
90
(7)
i = 3
min = 4
6
(0)
10
(1)
44
(2)
45
(3)
58
(4)
55
(5)
62
(6)
90
(7)
i = 4
min = 55
A[min] = 58 A[min]= 55 A[min] = 55 A[min] = 55
6
(0)
10
(1)
44
(2)
45
(3)
55
(4)
58
(5)
62
(6)
90
(7)
i = 5
min = 5
6
(0)
10
(1)
44
(2)
45
(3)
55
(4)
58
(5)
62
(6)
90
(7)
i = 6
min = 6
6
(0)
10
(1)
44
(2)
45
(3)
55
(4)
58
(5)
62
(6)
90
(7)
i = 7
min = 6
Data Structures & Algo- Dr Ahmar Rashid 40
A[min] = 6
A[min] = 10
A[min] = 44
A[min] = 45
A[min] = 45
A[min] = 58
A[min] = 62
A[min] = 62
Selection Sort vs Insertion Sort
Selection sort's advantage is that
While insertion sort typically makes fewer
comparisons than selection sort,
Insertion sort requires more writes than the
selection sort because the inner loop of the
insertion sort can require shifting large sections
of the sorted portion of the array.
In general, insertion sort will write to the array O(n
2
) times
Whereas selection sort will write/swap only O(n) times
For this reason selection sort may be preferable
in cases where writing to memory is significantly
more expensive than reading,
such as with EPROM or flash memory
Data Structures & Algo- Dr Ahmar Rashid 41
Comparisons of different sorting
algorithms
Bubble Sort Insertion Sort Selection Sort
(n
2
) comparisons (n
2
) comparisons (n
2
) comparisons
(n
2
) swaps (n
2
) writes (n) swaps
Adaptive: O(n)
running time when
nearly sorted (Best
case running time)
Adaptive: O(n)
running time when
nearly sorted (Best
case running time)
Not adaptive (n
2
)
running time when
nearly sorted (Best
case running time)
Data Structures & Algo- Dr Ahmar Rashid 42
Merge Sort
The fundamental operation in this algorithm is merging
two sorted lists.
Because the lists are sorted, this can be done in one
pass through the input, if the output is put in a third list.
The basic merging algorithm takes
two input arrays: a and b,
an output array: c
three counters: aptr, bptr, and cptr,
which are initially set to the beginning of their respective arrays.
The smaller of a[aptr] and b[bptr] is copied to the next
entry in c, and the appropriate counters are advanced.
When either input list is exhausted, the remainder of
the other list is copied to c.
Data Structures & Algo- Dr Ahmar Rashid 43
Merge Sort : Example
1
(0)
13
(1)
24
(2)
26
(3)
2
(0)
15
(1)
27
(2)
38
(3)
aptr
bptr
cptr
Data Structures & Algo- Dr Ahmar Rashid 44
Merge Sort : Example
1
(0)
13
(1)
24
(2)
26
(3)
1
(0)
2
(0)
15
(1)
27
(2)
38
(3)
aptr
bptr
cptr
Data Structures & Algo- Dr Ahmar Rashid 45
Merge Sort : Example
1
(0)
13
(1)
24
(2)
26
(3)
1
(0)
2
(1)
2
(0)
15
(1)
27
(2)
38
(3)
aptr
bptr
cptr
Data Structures & Algo- Dr Ahmar Rashid 46
Merge Sort : Example
1
(0)
13
(1)
24
(2)
26
(3)
1
(0)
2
(1)
13
(2)
2
(0)
15
(1)
27
(2)
38
(3)
aptr
bptr
cptr
Data Structures & Algo- Dr Ahmar Rashid 47
Merge Sort : Example
1
(0)
13
(1)
24
(2)
26
(3)
1
(0)
2
(1)
13
(2)
15
(3)
2
(0)
15
(1)
27
(2)
38
(3)
aptr
bptr
cptr
Data Structures & Algo- Dr Ahmar Rashid 48
Merge Sort : Example
1
(0)
13
(1)
24
(2)
26
(3)
1
(0)
2
(1)
13
(2)
15
(3)
24
(2)
2
(0)
15
(1)
27
(2)
38
(3)
aptr
bptr
cptr
Data Structures & Algo- Dr Ahmar Rashid 49
Merge Sort : Example
1
(0)
13
(1)
24
(2)
26
(3)
1
(0)
2
(1)
13
(2)
15
(3)
24
(2)
26
(3)
2
(0)
15
(1)
27
(2)
38
(3)
aptr
bptr
cptr
Data Structures & Algo- Dr Ahmar Rashid 50
Merge Sort : Example
1
(0)
13
(1)
24
(2)
26
(3)
1
(0)
2
(1)
13
(2)
15
(3)
24
(4)
26
(5)
27
(6)
38
(7)
2
(0)
15
(1)
27
(2)
38
(3)
aptr
bptr
cptr
Data Structures & Algo- Dr Ahmar Rashid 51
Merge Sort
void mergesort( input_type a[], unsigned int n )
{
input_type *tmp_array;
tmp_array = (input_type *) malloc ( (n) * sizeof (input_type) );
if( tmp_array != NULL )
{
m_sort( a, tmp_array, 0, n-1 );
free( tmp_array );
}
else
cout<<"No space for tmp array!!!;
}
Data Structures & Algo- Dr Ahmar Rashid 52
tmp_array = new input_type [ n];
delete [](temp_array);
Merge Sort
void m_sort( input_type a[], input_type tmp_array[ ], int left, int right )
{
int center;
if( left < right )
{
center = (left + right) / 2;
m_sort( a, tmp_array, left, center );
m_sort( a, tmp_array, center+1, right );
merge( a, tmp_array, left, center+1, right );
}
}
Data Structures & Algo- Dr Ahmar Rashid 53
Calculate the centre index of the input list
Recursively call the m_sort procedure
for the left-half of the input data
Recursively call the m_sort procedure
for the right-half of the input data
Merge the two sorted lists
Merge procedure
void merge( input_type a[ ], input_type tmp_array[ ], int l_pos, int r_pos, int right_end )
{
int i, left_end, num_elements, tmp_pos;
left_end = r_pos - 1;
tmp_pos = l_pos;
num_elements = right_end - l_pos + 1;
/* main loop */
while( ( 1_pos <= left_end ) && ( r_pos <= right_end ) )
if( a[1_pos] <= a[r_pos] )
tmp_array[tmp_pos++] = a[l_pos++];
else
tmp_array[tmp_pos++] = a[r_pos++];
while( l_pos <= left_end )
tmp_array[tmp_pos++] = a[l_pos++];
while( r_pos <= right_end )
tmp_array[tmp_pos++] = a[r_pos++];
/* copy tmp_array back to the main list a */
for (i=1; i <= num_elements; i++, right_end-- )
a[right_end] = tmp_array[right_end];
}
Data Structures & Algo- Dr Ahmar Rashid 54
merge( a, tmp_array, left, center+1, right );
Mark the end of the left list
Initialize the index pointer of the new
temp_array to the start of the left array
Calculate the total number of elements
in the current merged list
While both the input lists still
have elements to be processed
Copy the smaller of the two elements
Pointed by l_pos and r_pos, respectively,
into the temp_array.
Increment the respective index pointers
If only the left input list has elements to be processed
If only the right input list has elements to be processed
/* copy rest of first half */
/* copy rest of second half */
1
(0)
2
(1)
13
(2)
15
(3)
24
(4)
26
(5)
27
(6)
38
(7)
Merge Sort Example (recursive Function Calls)
62
(0)
58
(1)
55
(2)
10
(3)
45
(4)
44
(5)
6
(6)
90
(7)
Mergesort(a, 8)
m_sort( a, tmp_array, 0, 7 )
m_sort( a, tmp_array, 0, 3) m_sort( a, tmp_array, 4, 7)
m_sort(0, 1) m_sort(2, 3 ) m_sort(4, 5) m_sort(6, 7 )
Merge(a, tmp_array, 0, 4, 7 )
m_sort(0,0) m_sort(1,1) m_sort(2,2) m_sort(3,3) m_sort(4,4) m_sort(5,5) m_sort(6,6) m_sort(7,7)
Merge(a, tmp_array, 0, 2, 3 ) Merge(a, tmp_array, 4, 6, 7 )
Merge(0, 1, 1 ) Merge(2, 3, 3 ) Merge(4, 5, 5 ) Merge(6, 7, 7 )
62
(0)
58
(1)
55
(2)
10
(3)
45
(4)
44
(5)
6
(6)
90
(7)
62
(0)
58
(1)
55
(2)
10
(3)
45
(4)
44
(5)
6
(6)
90
(7)
62
(0)
58
(1)
55
(2)
10
(3)
45
(4)
44
(5)
6
(6)
90
(7)
Data Structures & Algo- Dr Ahmar Rashid 55
Merge Sort Example (Merging process)
62
(0)
58
(1)
55
(2)
10
(3)
45
(4)
44
(5)
6
(6)
90
(7)
Mergesort(a, 8)
m_sort( a, tmp_array, 0, 7 )
m_sort( a, tmp_array, 0, 3) m_sort( a, tmp_array, 4, 7)
m_sort(0, 1) m_sort(2, 3 )
m_sort(4, 5) m_sort(6, 7 )
Merge(a, tmp_array, 0, 4, 7 )
m_sort(0,0) m_sort(1,1) m_sort(2,2) m_sort(3,3) m_sort(4,4) m_sort(5,5) m_sort(6,6) m_sort(7,7)
Merge(a, tmp_array, 0, 2, 3 ) Merge(a, tmp_array, 4, 6, 7 )
Merge(0, 1, 1 ) Merge(2, 3, 3 ) Merge(4, 5, 5 ) Merge(6, 7, 7 )
62
(0)
58
(1)
55
(2)
10
(3)
45
(4)
44
(5)
6
(6)
90
(7)
62
(0)
58
(1)
55
(2)
10
(3)
45
(4)
44
(5)
6
(6)
90
(7)
62
(0)
58
(1)
55
(2)
10
(3)
45
(4)
44
(5)
6
(6)
90
(7)
58
(0)
62
(1)
10
(2)
55
(3)
10
(0)
55
(1)
58
(2)
62
(3)
6
(4)
44
(5)
45
(6)
90
(7)
44
(4)
45
(5)
6
(6)
90
(7)
6
(0)
10
(1)
44
(2)
45
(3)
55
(4)
58
(5)
62
(6)
90
(7)
Data Structures & Algo- Dr Ahmar Rashid 56
Merge Sort : Example
aptr
bptr
cptr
10
(0)
55
(1)
58
(2)
62
(3)
6
(4)
44
(5)
45
(6)
90
(7)
62
(0)
58
(1)
55
(2)
10
(3)
45
(4)
44
(5)
6
(6)
90
(7)
Data Structures & Algo- Dr Ahmar Rashid 57
Merge Sort : Example
6
(0)
aptr
bptr
cptr
10
(0)
55
(1)
58
(2)
62
(3)
6
(4)
44
(5)
45
(6)
90
(7)
Data Structures & Algo- Dr Ahmar Rashid 58
Merge Sort : Example
6
(0)
10
(1)
aptr
cptr
10
(0)
55
(1)
58
(2)
62
(3)
6
(4)
44
(5)
45
(6)
90
(7)
bptr
Data Structures & Algo- Dr Ahmar Rashid 59
Merge Sort : Example
6
(0)
10
(1)
44
(2)
aptr
cptr
10
(0)
55
(1)
58
(2)
62
(3)
6
(4)
44
(5)
45
(6)
90
(7)
bptr
Data Structures & Algo- Dr Ahmar Rashid 60
Merge Sort : Example
6
(0)
10
(1)
44
(2)
45
(3)
aptr
cptr
10
(0)
55
(1)
58
(2)
62
(3)
6
(4)
44
(5)
45
(6)
90
(7)
bptr
Data Structures & Algo- Dr Ahmar Rashid 61
Merge Sort : Example
6
(0)
10
(1)
44
(2)
45
(3)
55
(4)
aptr
cptr
10
(0)
55
(1)
58
(2)
62
(3)
6
(4)
44
(5)
45
(6)
90
(7)
bptr
Data Structures & Algo- Dr Ahmar Rashid 62
Merge Sort : Example
6
(0)
10
(1)
44
(2)
45
(3)
55
(4)
58
(5)
aptr
cptr
10
(0)
55
(1)
58
(2)
62
(3)
6
(4)
44
(5)
45
(6)
90
(7)
bptr
Data Structures & Algo- Dr Ahmar Rashid 63
Merge Sort : Example
6
(0)
10
(1)
44
(2)
45
(3)
55
(4)
58
(5)
62
(6)
aptr
cptr
10
(0)
55
(1)
58
(2)
62
(3)
6
(4)
44
(5)
45
(6)
90
(7)
bptr
Data Structures & Algo- Dr Ahmar Rashid 64
Merge Sort : Example
6
(0)
10
(1)
44
(2)
45
(3)
55
(4)
58
(5)
62
(6)
90
(7)
aptr
cptr
10
(0)
55
(1)
58
(2)
62
(3)
6
(4)
44
(5)
45
(6)
90
(7)
bptr
Data Structures & Algo- Dr Ahmar Rashid 65
Quick Sort
As its name implies, quicksort is the fastest
known sorting algorithm in practice.
It is very fast, mainly due to a very tight and highly
optimized inner loop
Its average running time is O(n log n)
It has O(n
2
) worst-case performance,
but this can be made exponentially unlikely with a little
effort
The quicksort algorithm is simple to understand
and prove correct
Like mergesort, quicksort is a divide-and-conquer
recursive algorithm
Data Structures & Algo- Dr Ahmar Rashid 66
Quick Sort
The basic algorithm to sort an array S consists
of the following four easy steps:
If the number of elements in S is 0 or 1, then return
2. Pick any element v in S. This is called the pivot.
3. Partition S - {v} (the remaining elements in S)
into two disjoint groups: S
1
= {x S - {v}| x v}
and S
2
= {x S - {v}| x v}
4. Return { quicksort(S
1
) followed by v followed by
quicksort(S
2
) }
Data Structures & Algo- Dr Ahmar Rashid 67
Quick Sort
Since the partition step ambiguously
describes what to do with elements equal to
the pivot, this becomes a design decision.
Part of a good implementation is handling this
case as efficiently as possible.
Intuitively, we would hope that
about half the keys that are equal to the pivot go
into S
1

while the other half into S
2
, much as we like binary
search trees to be balanced.
Data Structures & Algo- Dr Ahmar Rashid 68
Quick Sort
Selecting
the Pivot
Data Structures & Algo- Dr Ahmar Rashid 69
Quick Sort Selecting the Pivot
1- The popular, uninformed choice:
Use the first element as the pivot
This is acceptable if the input is random
But, if the input is presorted or in reverse order, then the pivot
provides a poor partition, because virtually all the elements go
into S
1
or S
2

It happens consistently throughout the recursive calls
Quicksort will take quadratic time to do essentially
nothing at all, which is quite embarrassing
2- A Safe Maneuver
A safe course is merely to choose the pivot randomly
This strategy is generally perfectly safe, unless the random
number generator has a flaw
However, random number generation is generally an
expensive commodity and does not reduce the average
running time of the rest of the algorithm at all
Data Structures & Algo- Dr Ahmar Rashid 70
Quick Sort Selecting the Pivot
The best choice of pivot would be the median of the
file.
The median of a group of n numbers is the (n/2)-th
largest number
Unfortunately, this is hard to calculate and would slow
down quicksort considerably
A good estimate can be obtained by picking three
elements randomly and using the median of these
three as pivot.
The randomness turns out not to help much
So, the common course is to use as pivot the
median of the left, right and center elements
A = {8, 1, 4, 9, 6, 3, 5, 2, 7, 0}
pivot: v = A[left + right)/2]
= A [(0+9)/2] = A[4] = 6
Data Structures & Algo- Dr Ahmar Rashid 71
QuickSort :Partitioning strategy
Example
8
(0)
1
(1)
4
(2)
9
(3)
6
(4)
3
(5)
5
(6)
2
(7)
7
(8)
0
(9)
i
The basic algorithm
While i is to the left of j,
we move i right, skipping over elements smaller than the pivot
We move j left, skipping over elements larger than the pivot
When i and j have stopped,
i is pointing at a large element, and
j is pointing at a small element
If i is to the left of j,
those elements are swapped
The effect is to push a large element to the right and a small
element to the left.
In the example above, i would not move and j would slide over
one place
pivot
j
Data Structures & Algo- Dr Ahmar Rashid 72
QuickSort :Partitioning strategy
Step 1
8
(0)
1
(1)
4
(2)
9
(3)
6
(4)
3
(5)
5
(6)
2
(7)
7
(8)
0
(9)
i
j pivot
Data Structures & Algo- Dr Ahmar Rashid 73
QuickSort :Partitioning strategy
Step 1
8
(0)
1
(1)
4
(2)
9
(3)
0
(4)
3
(5)
5
(6)
2
(7)
7
(8)
6
(9)
i
j
Start by swapping the pivot with the right, starting j at right -1
A[i] = 8 > pivot
Stop i right over here
pivot
Data Structures & Algo- Dr Ahmar Rashid 74
QuickSort :Partitioning strategy
Step 1
8
(0)
1
(1)
4
(2)
9
(3)
0
(4)
3
(5)
5
(6)
2
(7)
7
(8)
6
(9)
i
j
A[j] = 7 > pivot
Move Left
A[j] = 2 < pivot
Stop j right over here

Swap A[i] and A[j]
pivot
j
Data Structures & Algo- Dr Ahmar Rashid 75
QuickSort :Partitioning strategy
Step 1
2
(0)
1
(1)
4
(2)
9
(3)
0
(4)
3
(5)
5
(6)
8
(7)
7
(8)
6
(9)
i
A[j] = 7 > pivot
Move Right
A[j] = 2 < pivot
Stop j right over here

Swap A[i] and A[j]
pivot
j
Data Structures & Algo- Dr Ahmar Rashid 76
QuickSort :Partitioning strategy
Step 2
2
(0)
1
(1)
4
(2)
9
(3)
0
(4)
3
(5)
5
(6)
8
(7)
7
(8)
6
(9)
i
A[i] = 2 < pivot
Move Right
A[i] = 1 < pivot
Move Right
A[i] = 4 < pivot
Move Right
A[i] = 9 > pivot
Stop i right over here
pivot
j i
i i
A[j] = 8 > pivot
Move Left
A[j] = 5 < pivot
Stop j right over here
j
Data Structures & Algo- Dr Ahmar Rashid 77
QuickSort :Partitioning strategy
Step 2
2
(0)
1
(1)
4
(2)
5
(3)
0
(4)
3
(5)
9
(6)
8
(7)
7
(8)
6
(9)
pivot
i j
A[i] = 2 < pivot
Move Right
A[i] = 1 < pivot
Move Right
A[i] = 4 < pivot
Move Right
A[i] = 9 > pivot
Stop i right over here
A[j] = 8 > pivot
Move Left
A[j] = 5 < pivot
Stop j right over here

Swap A[i] and A[j]
QuickSort :Partitioning strategy
Step 3
2
(0)
1
(1)
4
(2)
5
(3)
0
(4)
3
(5)
9
(6)
8
(7)
7
(8)
6
(9)
A[i] = 5 < pivot
Move Right
A[i] = 0 < pivot
Move Right
A[i] = 3 < pivot
Move Right
A[i] = 9 > pivot
Stop i right over here
pivot
i j i i i
Data Structures & Algo- Dr Ahmar Rashid 79
QuickSort :Partitioning strategy
Step 3
2
(0)
1
(1)
4
(2)
5
(3)
0
(4)
3
(5)
9
(6)
8
(7)
7
(8)
6
(9)
pivot
A[j] = 9 > pivot
Move Left
A[j] = 3 < pivot
Stop j right over here
i and j have crossed
So no swap for A[i] and A[j]
Instead Swap A[i] and A[pivot]
j j
i
Data Structures & Algo- Dr Ahmar Rashid 80
A[i] = 5 < pivot
Move Right
A[i] = 0 < pivot
Move Right
A[i] = 3 < pivot
Move Right
A[i] = 9 > pivot
Stop i right over here
QuickSort :Partitioning strategy
Step 3
2
(0)
1
(1)
4
(2)
5
(3)
0
(4)
3
(5)
6
(6)
8
(7)
7
(8)
9
(9)
pivot j
i
Data Structures & Algo- Dr Ahmar Rashid 81
A[i] = 5 < pivot
Move Right
A[i] = 0 < pivot
Move Right
A[i] = 3 < pivot
Move Right
A[i] = 9 > pivot
Stop i right over here
A[j] = 9 > pivot
Move Left
A[j] = 3 < pivot
Stop j right over here
i and j have crossed
So no swap for A[i] and A[j]
Instead Swap A[i] and A[pivot]
QuickSort : Recursive calls
2
(0)
1
(1)
4
(2)
5
(3)
0
(4)
3
(5)
6
(6)
8
(7)
7
(8)
9
(9)
i
2
(0)
1
(1)
4
(2)
5
(3)
0
(4)
3
(5)
6
(6)
8
(7)
7
(8)
9
(9)
i + 1 i -1
0
N -1
Data Structures & Algo- Dr Ahmar Rashid 82
QuickSort : Left Recursive call
2
(0)
1
(1)
4
(2)
5
(3)
0
(4)
3
(5)
i pivot j
Data Structures & Algo- Dr Ahmar Rashid 83
QuickSort : Left Recursive call
First Swap
2
(0)
1
(1)
3
(2)
5
(3)
0
(4)
4
(5)
i pivot j
Data Structures & Algo- Dr Ahmar Rashid 84
QuickSort : Left Recursive call
Step 1: Movements
2
(0)
1
(1)
3
(2)
5
(3)
0
(4)
4
(5)
i pivot j
Data Structures & Algo- Dr Ahmar Rashid 85
QuickSort : Left Recursive call
Step 1: Swap
2
(0)
1
(1)
3
(2)
5
(3)
0
(4)
4
(5)
i
pivot j
Data Structures & Algo- Dr Ahmar Rashid 86
QuickSort : Left Recursive call
Step 1: Swap
2
(0)
1
(1)
3
(2)
0
(3)
5
(4)
4
(5)
i
pivot j
Data Structures & Algo- Dr Ahmar Rashid 87
QuickSort : Left Recursive call
Step 2: Movements
2
(0)
1
(1)
3
(2)
0
(3)
5
(4)
4
(5)
i
pivot j
Data Structures & Algo- Dr Ahmar Rashid 88
QuickSort : Left Recursive call
Step 2: Swap with the pivot
2
(0)
1
(1)
3
(2)
0
(3)
5
(4)
4
(5)
i
pivot
Data Structures & Algo- Dr Ahmar Rashid 89
QuickSort : Left Recursive call
Step 2: Swap with the pivot
2
(0)
1
(1)
3
(2)
0
(3)
4
(4)
5
(5)
i
pivot
Data Structures & Algo- Dr Ahmar Rashid 90
QuickSort :Recursive Calls
2
(0)
1
(1)
3
(2)
0
(3)
4
(4)
5
(5)
i
2
(0)
1
(1)
3
(2)
0
(3)
4
(4)
5
(5)
i -1 0
i + 1
Right
Data Structures & Algo- Dr Ahmar Rashid 91
QuickSort
void quick_sort( input_type a[ ], unsigned int n )
{
q_sort( a, 0, n-1 );
}
Data Structures & Algo- Dr Ahmar Rashid 92
QuickSort: Core Function
void q_sort( input_type a[], int left, int right )
{
int i, j; int pivot;
if( left + CUTOFF <= right )
{
pivot = median3( a, left, right );
i = left; j = right - 1;
for ( ; ; )
{
while( a[++i] < pivot );
while( a[--j] > pivot );
if( i < j )
swap( &a[i], &a[j] );
else
break;
} //end for loop
swap( &a[i], &a[right-1] ); /*restore pivot*/
q_sort( a, left, i -1 ); // left recursive call
q_sort( a, i +1, right ); // lright recursive call
}
}
Data Structures & Algo- Dr Ahmar Rashid 93
QuickSort: Medians
/* Return median of left, center, and right. */
/* Order these and hide pivot */
int median3( input_type a[], int left, int right )
{
int center;
center = (left + right) / 2;
if ( a[left] > a[center] )
swap( &a[left], &a[center] );
if ( a[left] > a[right] )
swap( &a[left], &a[right] );
if ( a[center] > a[right] )
swap( &a[center], &a[right] );
/* a[left] <= a[center] <= a[right] */
swap( &a[center], &a[right-1] );
return a[right-1]; /* return pivot */
}
Data Structures & Algo- Dr Ahmar Rashid 94
QuickSort: Medians
/* Return median of left, center, and right. */
/* Order these and hide pivot */
int median3( input_type a[], int left, int right )
{
int center;
center = (left + right) / 2;
if ( a[left] > a[center] )
swap( &a[left], &a[center] );
if ( a[left] > a[right] )
swap( &a[left], &a[right] );
if ( a[center] > a[right] )
swap( &a[center], &a[right] );
/* a[left] <= a[center] <= a[right] */
swap( &a[center], &a[right-1] );
return a[right-1]; /* return pivot */
}
Data Structures & Algo- Dr Ahmar Rashid 95
8
(0)
1
(1)
4
(2)
9
(3)
6
(4)
3
(5)
5
(6)
2
(7)
7
(8)
0
(9)
left
center
right
8 > 6
8
(0)
6 > 0
0
(9)
8 > 6
6
(4)
QuickSort: Medians
/* Return median of left, center, and right. */
/* Order these and hide pivot */
int median3( input_type a[], int left, int right )
{
int center;
center = (left + right) / 2;
if ( a[left] > a[center] )
swap( &a[left], &a[center] );
if ( a[left] > a[right] )
swap( &a[left], &a[right] );
if ( a[center] > a[right] )
swap( &a[center], &a[right] );
/* a[left] <= a[center] <= a[right] */
swap( &a[center], &a[right-1] );
return a[right-1];
}
Data Structures & Algo- Dr Ahmar Rashid 96
8
(0)
1
(1)
4
(2)
9
(3)
6
(4)
3
(5)
5
(6)
2
(7)
7
(8)
0
(9)
center
right -1
8 > 6
0
(0)
6 > 0
8
(9)
8 > 6
6
(4)
/* hide pivot */
6
(4)
7
(8)
pivot
/* return pivot */
QuickSort: Core Function
void q_sort( input_type a[], int left, int right )
{
int i, j; int pivot;
if( left + CUTOFF <= right )
{
pivot = median3( a, left, right );
i=left; j=right-1;
for ( ; ; )
{
while ( a[++i] < pivot );
while ( a[--j] > pivot );
if ( i < j )
swap( &a[i], &a[j] );
else
break;
} //end for loop
swap( &a[i], &a[right-1] );
q_sort( a, left, i-1 );
q_sort( a, i+1, right );
}
}
Data Structures & Algo- Dr Ahmar Rashid 97
8
(0)
1
(1)
4
(2)
9
(3)
7
(4)
3
(5)
5
(6)
2
(7)
6
(8)
0
(9)

i = left
j = right -1
0
(0)
8
(9)
6
(4)
pivot
/* while i is to the left of j,
move i right, skipping over elements
smaller than the pivot
move j left, skipping over elements
larger than the pivot. */
/* If i is to the left of j,
those elements are swapped */
center
7
(4)
6
(8)
QuickSort: Core Function
void q_sort( input_type a[], int left, int right )
{
int i, j; int pivot;
if( left + CUTOFF <= right )
{
pivot = median3( a, left, right );
i=left; j=right-1;
for ( ; ; )
{
while ( a[++i] < pivot );
while ( a[--j] > pivot );
if ( i < j )
swap( &a[i], &a[j] );
else
break;
} //end for loop
swap( &a[i], &a[right-1] );
q_sort( a, left, i-1 );
q_sort( a, i+1, right );
}
}
Data Structures & Algo- Dr Ahmar Rashid 98
8
(0)
1
(1)
4
(2)
2
(3)
5
(4)
3
(5)
7
(6)
9
(7)
6
(8)
0
(9)

i
right -1
0
(0)
8
(9)
pivot
/*restore pivot*/
// left recursive call
// lright recursive call
7
(4)
6
(8)

left
right
j
/* loop terminated when ( i >j ) i.e., i and j
have crossed so no swap is performed */
Linear Order Sorting Algorithms
Counting Sort
Radix Sort
Counting Sort
Suppose that there are N integers to be sorted,
A[1N], such that the range of the integers is
from 1 to M
Start by defining an array B[1M] and initializing
all its elements to 0. ( O(M) )
Scan through A[ i ], for i = 1,,N, and copy each
A[ i ] into B[ A[ i ] ]. ( ON) )
Scan through B[ i ], for i = 1,,M , and retrieve all
the non-zero elements of B[ i ]. ( O(M) )
Complexity: O(M + N)
If M ~ N Complexity: O(N)
If M ~ N
2
Complexity: O(N
2
)
Example
Sort 5 7 9 2 4 6 1
Counting Sort
5
(0)
7
(1)
9
(2)
2
(3)
4
(4)
6
(5)
1
(6)
A
B

(0)

(1)

(2)

(3)

(4)
5
(5)

(6)

(7)

(8)

(9)

(0)

(1)

(2)

(3)

(4)
5
(5)

(6)
7
(7)

(8)

(9)

(0)

(1)

(2)

(3)

(4)
5
(5)

(6)
7
(7)

(8)
9
(9)

(0)

(1)
2
(2)

(3)

(4)
5
(5)

(6)
7
(7)

(8)
9
(9)

(0)

(1)
2
(2)

(3)
4
(4)
5
(5)

(6)
7
(7)

(8)
9
(9)

(0)

(1)
2
(2)

(3)
4
(4)
5
(5)
6
(6)
7
(7)

(8)
9
(9)

(0)
1
(1)
2
(2)

(3)
4
(4)
5
(5)
6
(6)
7
(7)

(8)
9
(9)
B[ A[ 0] ] = A[ 0]
B[ 5 ] = 5
B[ A[ i ] ] = A[ i ]
B[ 7 ] = 7
B[ 9 ] = 9
B[ 2 ] = 2
B[ 4 ] = 4
B[ 6 ] = 6
B[ 1 ] = 1
Final Output: 1 2 4 5 6 7 9
How to cope up with duplicates?
B becomes a an array of pointers, (rather than
an array of integers). Each element has a
head pointer (points to the head of a linked list)
tail pointer (points to the tail of the linked list)
A[j] is added at the tail of the list B[ A [j]]
Finally, the array B is sequentially traversed
and each nonempty list retrieved.
Complexity: O(M + N)
Counting Sort
Counting Sort Duplicate keys
5
(0)
7
(1)
9
(2)
2
(3)
4
(4)
6
(5)
7
(6)

1
(7)
2
(8)
7
(9)
A
B
Final Output: 1 2 2 4 5 6 7 7 7 9

(0)
Null

(1)

head

(2)

head

(3)

Null

(4)

head

(5)

head

(6)

head

(7)

head

(8)

Null

(9)

head

1

2

4

5

6

7

9

2

7

7

Counting sort becomes expensive if M is large
compared to N
Radix sort may be built upon counting sort
Basic Idea: every integer can be represented
by at most k digits
d
1
d
2
d
k
where d
i
are digits in base r
Radix Sort
most significant digit
least significant digit
Algorithm
Take the least significant digit (or group of bits) of each
key.
Group the keys based on that digit, but otherwise keep
the original order of keys
Repeat the grouping process with each more significant
digit.
The sort in step 2 is usually done using bucket
sort or counting sort, which are efficient in this case
since there are usually only a small number of digits.


Radix Sort
Example
Sort: 170, 45, 275, 190, 802, 24, 2, 66

First pass
Radix Sort
(0) (1) (2) (3) (4) (5) (6) (7) (8)

(9)
170

802

24

45

66

275

190

2

First pass
Radix Sort
(0) (1) (2) (3) (4) (5) (6) (7) (8)

(9)
Second pass
(0) (1) (2) (3) (4) (5) (6) (7) (8)

(9)
170

802

24

45

66

275

190

2

Second pass
Radix Sort
(0) (1) (2) (3) (4) (5) (6) (7) (8)

(9)
Third pass
(0) (1) (2) (3) (4) (5) (6) (7) (8)

(9)
170

802

24

45

66

275

190

2

Final Output: 2 24 45 66 170 190 275 802
Algorithm RadixSort(A, N, d)
for p = 0 to 9
Q[p] =Null
D = 1
for k = 1 to d
D = D * 10
for i = 0 to N
t = (A[i] mod D) div (D/10)
enqueue(A[i], Q[t])
j = 0
for p = 0 to N
do while Q[p] is not empty
A[j] = dequeue(Q[p])
j = j +1



Radix Sort
D = 10
i = 0
t = (A[0] mod 10) div (1)
= 170 mod 10 = 0
enqueue(170, Q[0])
170, 45, 275, 190, 802, 24, 2, 66
(0) (1) (2) (3) (4) (5) (6) (7) (8)

(9)
170

Algorithm RadixSort(A, N, d)
for p = 0 to 9
Q[p] =Null
D = 1
for k = 1 to d
D = D * 10
for i = 0 to N
t = (A[i] mod D) div (D/10)
enqueue(A[i], Q[t])
j = 0
for p = 0 to N
do while Q[p] is not empty
A[j] = dequeue(Q[p])
j = j +1



Radix Sort
D = 10
i = 1
t = (A[1] mod 10) div (1)
= 45 mod 10 = 5
enqueue(45, Q[5])
170, 45, 275, 190, 802, 24, 2, 66
(0) (1) (2) (3) (4) (5) (6) (7) (8)

(9)
170

45

Algorithm RadixSort(A, N, d)
for p = 0 to 9
Q[p] =Null
D = 1
for k = 1 to d
D = D * 10
for i = 0 to N
t = (A[i] mod D) div (D/10)
enqueue(A[i], Q[t])
j = 0
for p = 0 to N
do while Q[p] is not empty
A[j] = dequeue(Q[p])
j = j +1



Radix Sort
170, 45, 275, 190, 802, 24, 2, 66
D = 10
i = 2
t = (A[2] mod 10) div (1)
= 275 mod 10 = 5
enqueue(245, Q[5])
(0) (1) (2) (3) (4) (5) (6) (7) (8)

(9)
170

802

24

45

66

275

190

2

i = 3, 4, 5, 6, 7, 8, 9
Algorithm RadixSort(A, N, d)
for p = 0 to 9
Q[p] =Null
D = 1
for k = 1 to d
D = D * 10
for i = 0 to N
t = (A[i] mod D) div (D/10)
enqueue(A[i], Q[t])
j = 0
for p = 0 to N
do while Q[p] is not empty
A[j] = dequeue(Q[p])
j = j +1



Radix Sort
170, 45, 275, 190, 802, 24, 2, 66
D = 10
(0) (1) (2) (3) (4) (5) (6) (7) (8)

(9)
170

802

24

45

66

275

190

2

A = { 170, 190, 802, 2, 24, 45, 275, 66}
Algorithm RadixSort(A, N, d)
for p = 0 to 9
Q[p] =Null
D = 1
for k = 1 to d
D = D * 10
for i = 0 to N
t = (A[i] mod D) div (D/10)
enqueue(A[i], Q[t])
j = 0
for p = 0 to N
do while Q[p] is not empty
A[j] = dequeue(Q[p])
j = j +1



Radix Sort
D = 100
(0) (1) (2) (3) (4) (5) (6) (7) (8)

(9)
170

A = { 170, 190, 802, 2, 24, 45, 275, 66}
i = 0
t = (A[0] mod 100) div (10)
= (170 mod 100) div (10) =70 / 10=7
enqueue(170, Q[7])
Algorithm RadixSort(A, N, d)
for p = 0 to 9
Q[p] =Null
D = 1
for k = 1 to d
D = D * 10
for i = 0 to N
t = (A[i] mod D) div (D/10)
enqueue(A[i], Q[t])
j = 0
for p = 0 to N
do while Q[p] is not empty
A[j] = dequeue(Q[p])
j = j +1



Radix Sort
D = 100
(0) (1) (2) (3) (4) (5) (6) (7) (8)

(9)
190

A = { 170, 190, 802, 2, 24, 45, 275, 66}
i = 1
t = (A[1] mod 100) div (10)
= (190 mod 100) div (10) =90 / 10=9
enqueue(190, Q[9])
170

Algorithm RadixSort(A, N, d)
for p = 0 to 9
Q[p] =Null
D = 1
for k = 1 to d
D = D * 10
for i = 0 to N
t = (A[i] mod D) div (D/10)
enqueue(A[i], Q[t])
j = 0
for p = 0 to N
do while Q[p] is not empty
A[j] = dequeue(Q[p])
j = j +1



Radix Sort
D = 100
i = 2, 3, 4, 5, 6, 7, 8, 9
(0) (1) (2) (3) (4) (5) (6) (7) (8)

(9)
170

802

24

45

66

275

190

2

A = {802, 2, 24, 45, 66, 275, 170, 190}
A = { 170, 190, 802, 2, 24, 45, 275, 66}
Algorithm RadixSort(A, N, d)
for p = 0 to 9
Q[p] =Null
D = 1
for k = 1 to d
D = D * 10
for i = 0 to N
t = (A[i] mod D) div (D/10)
enqueue(A[i], Q[t])
j = 0
for p = 0 to N
do while Q[p] is not empty
A[j] = dequeue(Q[p])
j = j +1



Radix Sort
D = 1000
(0) (1) (2) (3) (4) (5) (6) (7) (8)

(9)
i = 0
t = (A[0] mod 1000) div (1000)
= (802 mod 1000) div (1000) =802 /100=8
enqueue(802, Q[8])
802

A = {802, 2, 24, 45, 66, 275, 170, 190}
Algorithm RadixSort(A, N, d)
for p = 0 to 9
Q[p] =Null
D = 1
for k = 1 to d
D = D * 10
for i = 0 to N
t = (A[i] mod D) div (D/10)
enqueue(A[i], Q[t])
j = 0
for p = 0 to N
do while Q[p] is not empty
A[j] = dequeue(Q[p])
j = j +1



Radix Sort
D = 1000
(0) (1) (2) (3) (4) (5) (6) (7) (8)

(9)
802

A = {802, 2, 24, 45, 66, 275, 170, 190}
170

24

45

66

275

190

2

i = 1, 2, 3, 4, 5, 6, 7, 8, 9
A = {2, 24, 45, 66, 170, 190, 275 ,802}
Final Output