Data Structures and Algorithms

PLSD210
Sorting
Sorting
Card players all know how to sort
First card is already sorted
With all the rest,
Scan back from the end until you find the first card larger
than the new one,
gMove all the lower ones up one slot
insert it
E
Q
-
2
-
9
*
A
*
K
*
10
+
J
+
2
*
2
-
9



Sorting - Insertion sort
Complexity
For each card
Scan O(n)
Shift up O(n)
Insert O(1)
Total O(n)
First card requires O(1), second O(2),
For n cards operations + O(n
2
)

S i
i=1
n
Sorting - Insertion sort
Complexity
For each card
Scan O(n) O(log n)
Shift up O(n)
Insert O(1)
Total O(n)
First card requires O(1), second O(2),
For n cards operations + O(n
2
)

S i
i=1
n
Unchanged!
Because the
shift up operation
still requires O(n)
time
Use binary search!
Insertion Sort - Implementation
A challenge for you
The code in the notes (and on the Web) has an error
First person to email a correct version
gets up to 2 extra marks added to their final mark if that
would move them up a grade!
ie if you had x8% or x9%, it goes to (x+1)0%
To qualify, you need to point out the error in the original,
as well as supply a corrected version!
Sorting - Bubble
From the first element
Exchange pairs if theyre out of order
Last one must now be the largest
Repeat from the first to n-1
Stop when you have only one element to check
Bubble Sort
/* Bubble sort for integers */
#define SWAP(a,b) { int t; t=a; a=b; b=t; }

void bubble( int a[], int n ) {
int i, j;
for(i=0;i<n;i++) { /* n passes thru the array */
/* From start to the end of unsorted part */
for(j=1;j<(n-i);j++) {
/* If adjacent items out of order, swap */
if( a[j-1]>a[j] ) SWAP(a[j-1],a[j]);
}
}
}
Bubble Sort - Analysis
/* Bubble sort for integers */
#define SWAP(a,b) { int t; t=a; a=b; b=t; }

void bubble( int a[], int n ) {
int i, j;
for(i=0;i<n;i++) { /* n passes thru the array */
/* From start to the end of unsorted part */
for(j=1;j<(n-i);j++) {
/* If adjacent items out of order, swap */
if( a[j-1]>a[j] ) SWAP(a[j-1],a[j]);
}
}
}
O(1) statement
Bubble Sort - Analysis
/* Bubble sort for integers */
#define SWAP(a,b) { int t; t=a; a=b; b=t; }

void bubble( int a[], int n ) {
int i, j;
for(i=0;i<n;i++) { /* n passes thru the array */
/* From start to the end of unsorted part */
for(j=1;j<(n-i);j++) {
/* If adjacent items out of order, swap */
if( a[j-1]>a[j] ) SWAP(a[j-1],a[j]);
}
}
} Inner loop
n-1, n-2, n-3, , 1 iterations
O(1) statement
Bubble Sort - Analysis
/* Bubble sort for integers */
#define SWAP(a,b) { int t; t=a; a=b; b=t; }

void bubble( int a[], int n ) {
int i, j;
for(i=0;i<n;i++) { /* n passes thru the array */
/* From start to the end of unsorted part */
for(j=1;j<(n-i);j++) {
/* If adjacent items out of order, swap */
if( a[j-1]>a[j] ) SWAP(a[j-1],a[j]);
}
}
}
Outer loop n iterations
Bubble Sort - Analysis
/* Bubble sort for integers */
#define SWAP(a,b) { int t; t=a; a=b; b=t; }

void bubble( int a[], int n ) {
int i, j;
for(i=0;i<n;i++) { /* n passes thru the array */
/* From start to the end of unsorted part */
for(j=1;j<(n-i);j++) {
/* If adjacent items out of order, swap */
if( a[j-1]>a[j] ) SWAP(a[j-1],a[j]);
}
}
}
Overall



S i
i=n-1
1
=
n(n+1)
2
= O(n
2
)
n outer loop iterations
inner loop iteration count
Sorting - Simple
Bubble sort
O(n
2
)
Very simple code
Insertion sort
Slightly better than bubble sort
Fewer comparisons
Also O(n
2
)
But HeapSort is O(n

log n)
Where would you use bubble or insertion sort?

Simple Sorts
Bubble Sort or Insertion Sort
Use
when n
is small
Simple code
compensates
for low
efficiency!
n^2 and n log n
0
500
1000
1500
2000
2500
0 10 20 30 40 50 60
n
T
i
m
e
n log n
n^2
Quicksort
Efficient sorting algorithm
Discovered by C.A.R. Hoare
Example of Divide and Conquer algorithm
Two phases
Partition phase
Divides the work into half
Sort phase
Conquers the halves!
Quicksort
Partition
Choose a pivot
Find the position for the pivot so that
all elements to the left are less
all elements to the right are greater
< pivot > pivot pivot
Quicksort
Conquer
Apply the same algorithm to each half
< pivot > pivot
pivot < p p > p < p p > p
Quicksort
Implementation

quicksort( void *a, int low, int high )
{
int pivot;
/* Termination condition! */
if ( high > low )
{
pivot = partition( a, low, high );
quicksort( a, low, pivot-1 );
quicksort( a, pivot+1, high );
}
}
Divide
Conquer
Quicksort - Partition
int partition( int *a, int low, int high ) {
int left, right;
int pivot_item;
pivot_item = a[low];
pivot = left = low;
right = high;
while ( left < right ) {
/* Move left while item < pivot */
while( a[left] <= pivot_item ) left++;
/* Move right while item > pivot */
while( a[right] >= pivot_item ) right--;
if ( left < right ) SWAP(a,left,right);
}
/* right is final position for the pivot */
a[low] = a[right];
a[right] = pivot_item;
return right;
}
Quicksort - Partition
int partition( int *a, int low, int high ) {
int left, right;
int pivot_item;
pivot_item = a[low];
pivot = left = low;
right = high;
while ( left < right ) {
/* Move left while item < pivot */
while( a[left] <= pivot_item ) left++;
/* Move right while item > pivot */
while( a[right] >= pivot_item ) right--;
if ( left < right ) SWAP(a,left,right);
}
/* right is final position for the pivot */
a[low] = a[right];
a[right] = pivot_item;
return right;
}
This example
uses ints
to keep things
simple!
23 12 15 38 42 18 36 29 27
low high
Any item will do as the pivot,
choose the leftmost one!
Quicksort - Partition
int partition( int *a, int low, int high ) {
int left, right;
int pivot_item;
pivot_item = a[low];
pivot = left = low;
right = high;
while ( left < right ) {
/* Move left while item < pivot */
while( a[left] <= pivot_item ) left++;
/* Move right while item > pivot */
while( a[right] >= pivot_item ) right--;
if ( left < right ) SWAP(a,left,right);
}
/* right is final position for the pivot */
a[low] = a[right];
a[right] = pivot_item;
return right;
}
Set left and right markers
23 12 15 38 42 18 36 29 27
low high pivot: 23
left right
Quicksort - Partition
int partition( int *a, int low, int high ) {
int left, right;
int pivot_item;
pivot_item = a[low];
pivot = left = low;
right = high;
while ( left < right ) {
/* Move left while item < pivot */
while( a[left] <= pivot_item ) left++;
/* Move right while item > pivot */
while( a[right] >= pivot_item ) right--;
if ( left < right ) SWAP(a,left,right);
}
/* right is final position for the pivot */
a[low] = a[right];
a[right] = pivot_item;
return right;
}
Move the markers
until they cross over
23 12 15 38 42 18 36 29 27
low high pivot: 23
left right
Quicksort - Partition
int partition( int *a, int low, int high ) {
int left, right;
int pivot_item;
pivot_item = a[low];
pivot = left = low;
right = high;
while ( left < right ) {
/* Move left while item < pivot */
while( a[left] <= pivot_item ) left++;
/* Move right while item > pivot */
while( a[right] >= pivot_item ) right--;
if ( left < right ) SWAP(a,left,right);
}
/* right is final position for the pivot */
a[low] = a[right];
a[right] = pivot_item;
return right;
}
Move the left pointer while
it points to items <= pivot
23 12 15 38 42 18 36 29 27
low high pivot: 23
left right
Move right
similarly
Quicksort - Partition
int partition( int *a, int low, int high ) {
int left, right;
int pivot_item;
pivot_item = a[low];
pivot = left = low;
right = high;
while ( left < right ) {
/* Move left while item < pivot */
while( a[left] <= pivot_item ) left++;
/* Move right while item > pivot */
while( a[right] >= pivot_item ) right--;
if ( left < right ) SWAP(a,left,right);
}
/* right is final position for the pivot */
a[low] = a[right];
a[right] = pivot_item;
return right;
}
Swap the two items
on the wrong side of the pivot
23 12 15 38 42 18 36 29 27
low high
pivot: 23
left right
Quicksort - Partition
int partition( int *a, int low, int high ) {
int left, right;
int pivot_item;
pivot_item = a[low];
pivot = left = low;
right = high;
while ( left < right ) {
/* Move left while item < pivot */
while( a[left] <= pivot_item ) left++;
/* Move right while item > pivot */
while( a[right] >= pivot_item ) right--;
if ( left < right ) SWAP(a,left,right);
}
/* right is final position for the pivot */
a[low] = a[right];
a[right] = pivot_item;
return right;
}
left and right
have swapped over,
so stop
23 12 15 18 42 38 36 29 27
low high pivot: 23
left right
Quicksort - Partition
int partition( int *a, int low, int high ) {
int left, right;
int pivot_item;
pivot_item = a[low];
pivot = left = low;
right = high;
while ( left < right ) {
/* Move left while item < pivot */
while( a[left] <= pivot_item ) left++;
/* Move right while item > pivot */
while( a[right] >= pivot_item ) right--;
if ( left < right ) SWAP(a,left,right);
}
/* right is final position for the pivot */
a[low] = a[right];
a[right] = pivot_item;
return right;
}
Finally, swap the pivot
and right
23 12 15 18 42 38 36 29 27
low high pivot: 23
left right
Quicksort - Partition
int partition( int *a, int low, int high ) {
int left, right;
int pivot_item;
pivot_item = a[low];
pivot = left = low;
right = high;
while ( left < right ) {
/* Move left while item < pivot */
while( a[left] <= pivot_item ) left++;
/* Move right while item > pivot */
while( a[right] >= pivot_item ) right--;
if ( left < right ) SWAP(a,left,right);
}
/* right is final position for the pivot */
a[low] = a[right];
a[right] = pivot_item;
return right;
}
Return the position
of the pivot
18 12 15 23 42 38 36 29 27
low high
pivot: 23
right
Quicksort - Conquer
pivot
18 12 15 23 42 38 36 29 27
pivot: 23
Recursively
sort left half
Recursively
sort right half
Quicksort - Analysis
Partition
Check every item once O(n)
Conquer
Divide data in half O(log
2
n)
Total
Product O(n log n)
Same as Heapsort
quicksort is generally faster
Fewer comparisons
Details later (and assignment 2!)
But theres a catch .
Quicksort - The truth!
What happens if we use quicksort
on data thats already sorted
(or nearly sorted)
Wed certainly expect it to perform well!
Quicksort - The truth!
Sorted data
1 2 3 4 5 6 7 8 9
pivot
< pivot
?
> pivot
Quicksort - The truth!
Sorted data
Each partition
produces
a problem of size 0
and one of size n-1!
Number of partitions?
1 2 3 4 5 6 7 8 9
> pivot
2 3 4 5 6 7 8 9
> pivot
pivot
pivot
Quicksort - The truth!
Sorted data
Each partition
produces
a problem of size 0
and one of size n-1!
Number of partitions?
n each needing time O(n)
Total nO(n)
or O(n
2
)
? Quicksort is as bad as bubble or insertion sort
1 2 3 4 5 6 7 8 9
> pivot
2 3 4 5 6 7 8 9
> pivot
pivot
pivot
Quicksort - The truth!
Quicksorts O(n log n) behaviour
Depends on the partitions being nearly equal
+there are O( log n ) of them
On average, this will nearly be the case
and quicksort is generally O(n log n)

Can we do anything to ensure O(n log n) time?
In general, no
But we can improve our chances!!
Quicksort - Choice of the pivot
Any pivot will work
Choose a different pivot



so that the partitions are equal
then we will see O(n log n) time
1 2 3 4 5 6 7 8 9
pivot
< pivot
> pivot
Quicksort - Median-of-3 pivot
Take 3 positions and choose the median
say First, middle, last


+median is 5
+perfect division of sorted data every time!
+O(n log n) time
+ Since sorted (or nearly sorted) data is common,
median-of-3 is a good strategy
especially if you think your data may be sorted!
1 2 3 4 5 6 7 8 9
Quicksort - Random pivot
Choose a pivot randomly
Different position for every partition
+On average, sorted data is divided evenly
+O(n log n) time

Key requirement
Pivot choice must take O(1) time
Quicksort - Guaranteed O(n log n)?
Never!!
Any pivot selection strategy
could lead to O(n
2
) time

Here median-of-3 chooses 2
One partition of 1 and
One partition of 7
Next it chooses 4
One of 1 and
One of 5

1 4 9 6 2 5 7 8 3
1 2 4 9 6 5 7 8 3
Lecture 8 - Key Points
Sorting
Bubble, Insert
O(n
2
) sorts
Simple code
May run faster for small n,
n ~10 (system dependent)
Quick Sort
Divide and conquer
O(n log n)
Lecture 8 - Key Points
Quick Sort
O(n log n) but .
Can be O(n
2
)
Depends on pivot selection
Median-of-3
Random pivot
Better but not guaranteed


Quicksort - Why bother?
Use Heapsort instead?
Quicksort is generally faster
Fewer comparisons and exchanges
Some empirical data

n Quick Heap Insert
Comp Exch Comp Exch Comp Exch
100 712 148 2842 581 2595 899
200 1682 328 9736 9736 10307 3503
500 5102 919 53113 4042 62746 21083
Quicksort - Why bother?
Reporting data
Normalisation works when you have a hypothesis to work
with!


n Quick Heap Insert nlogn n^2
Comp Exch Norm Comp Exch Norm Comp Exch Norm Norm
100 712 148 0.74 2842 581 2.91 2595 899 4.50 0.09
200 1682 328 0.71 9736 1366 2.97 10307 3503 7.61 0.09
500 5102 919 0.68 53113 4042 3.00 62746 21083 15.62 0.08
Divide by n log n
Divide by n
2
Quicksort vs Heap Sort
Quicksort
Generally faster
Sometimes O(n
2
)
Better pivot selection reduces probability
Use when you want average good performance
Commercial applications, Information systems
Heap Sort
Generally slower
Guaranteed O(n log n) Can design this in!
Use for real-time systems
Time is a constraint
Quicksort - library implementation
Quicksort
POSIX standard






void qsort( void *base, size_t n, size_t size,
int (*compar)( const void *, const void * ) );
base address of array
n number of elements
size size of an element
compar comparison function
Quicksort - library implementation
Quicksort
POSIX standard







Comparison function
C allows you to pass a function to another function!
void qsort( void *base, size_t n, size_t size,
int (*compar)( const void *, const void * ) );
base address of array
n number of elements
size size of an element
compar comparison function