Sorting and Searching Algorithms

This is a collection of algorithms for sorting and searching. Descriptions are brief and intuitive,
with just enough theory thrown in to make you nervous. I assume you know a high-level
language, such as C, and that you are familiar with programming concepts including arrays and
pointers.
The first section introduces basic data structures and notation. The next section presents several
sorting algorithms. This is followed by a section on dictionaries, structures that allow efficient
insert, search, and delete operations. The last section describes algorithms that sort data and
implement dictionaries for very large files. Source code for each algorithm, in ANSI C, is
included.
Most algorithms have also been coded in Visual Basic. If you are programming in Visual Basic, I
recommend you read Visual Basic Collections and Hash Tables, for an explanation of hashing
and node representation. A search for Algorithms at amazon.com returned over 18,000 entries.
Here is one of the best.
Insertion Sort

One of the simplest methods to sort an array is an insertion sort. An example of an insertion sort
occurs in everyday life while playing cards. To sort the cards in your hand you extract a card,
shift the remaining cards, and then insert the extracted card in the correct place. This process is
repeated until all the cards are in the correct sequence. Both average and worst-case time is
O(n2). For further reading, consult Knuth [1998].

Theory
Starting near the top of the array in Figure 2-1(a), we extract the 3. Then the above elements are
shifted down until we find the correct place to insert the 3. This process repeats in Figure 2-1(b)
with the next number. Finally, in Figure 2-1(c), we complete the sort by inserting 2 in the correct
place.

Figure 2-1: Insertion Sort

Assuming there are n elements in the array, we must index through n- 1 entries. For each entry,
we may need to examine and shift up to n- 1 other entries, resulting in a O(n2) algorithm. The
insertion sort is an in-place sort. That is, we sort the array in-place. No extra memory is required.
The insertion sort is also a stable sort. Stable sorts retain the original ordering of keys when
identical keys are present in the input data.

Implementation in C
An ANSI-C implementation for insertion sort is included. Typedef T and comparison operator
compGT should be altered to reflect the data stored in the table.
/* insert sort */
#include <stdio.h>
#include <stdlib.h>
typedef int T;
typedef int tblIndex;

/* type of item to be sorted */

/* type of subscript */

#define compGT(a,b) (a > b)

void insertSort(T *a, tblIndex lb, tblIndex ub) {
T t;
tblIndex i, j;
/**************************
* sort array a[lb..ub] *
**************************/
for (i = lb + 1; i <= ub; i++) {
t = a[i];
/* Shift elements down until */

/* insertion point found.

*/
for (j = i-1; j >= lb && compGT(a[j], t); j--)
a[j+1] = a[j];
/* insert */
a[j+1] = t;
}

void fill(T *a, tblIndex lb, tblIndex ub) {

tblIndex i;
srand(1);
for (i = lb; i <= ub; i++) a[i] = rand();
}
int main(int argc, char *argv[]) {
tblIndex maxnum, lb, ub;
T *a;
/* command-line:
*
*
ins maxnum
*
*
ins 2000
*
sorts 2000 records
*
*/
maxnum = atoi(argv[1]);
lb = 0; ub = maxnum - 1;
if ((a = malloc(maxnum * sizeof(T))) == 0) {
fprintf (stderr, "insufficient memory (a)\n");
exit(1);
}
fill(a, lb, ub);
insertSort(a, lb, ub);
}

return 0;

Shell Sort

Shell sort, developed by Donald L. Shell, is a non-stable in-place sort. Shell sort improves on the
efficiency of insertion sort by quickly shifting values to their destination. Average sort time is
O(n7/6), while worst-case time is O(n4/3). For further reading, consult Knuth [1998].

Theory
In Figure 2-2(a) we have an example of sorting by insertion. First we extract 1, shift 3 and 5
down one slot, and then insert the 1, for a count of 2 shifts. In the next frame, two shifts are

required before we can insert the 2. The process continues until the last frame, where a total of 2
+ 2 + 1 = 5 shifts have been made.
In Figure 2-2(b) an example of shell sort is illustrated. We begin by doing an insertion sort using
aspacingof two. In the first frame we examine numbers 3-1. Extracting 1, we shift 3 down one
slot for a shift count of 1. Next we examine numbers 5-2. We extract 2, shift 5 down, and then
insert 2. After sorting with a spacing of two, a final pass is made with a spacing of one. This is
simply the traditional insertion sort. The total shift count using shell sort is 1+1+1 = 3. By using
an initial spacing larger than one, we were able to quickly shift values to their proper destination.

Figure 2-2: Shell Sort

Various spacings may be used to implement a shell sort. Typically the array is sorted with a large
spacing, the spacing reduced, and the array sorted again. On the final sort, spacing is one.
Although the shell sort is easy to comprehend, formal analysis is difficult. In particular, optimal
spacing values elude theoreticians. Knuth recommends a technique, due to Sedgewick, that
determines spacinghbased on the following formulas:
hs= 92s- 92s/2+ 1, if s is even
hs= 82s- 62(s+1)/2+ 1, if s is odd

These calculations result in values (h0,h1,h2,) = (1,5,19,41,109,209,). Calculate h until 3ht >=
N, the number of elements in the array. Then choose ht-1 for a starting value. For example, to sort
150 items, ht = 109 (3109 >= 150), so the first spacing is ht-1, or 41. The second spacing is 19,
then 5, and finally 1.

Implementation in C
An ANSI-C implementation for shell sort is included. Typedef T and comparison operator
compGT should be altered to reflect the data stored in the array. The central portion of the
algorithm is an insertion sort with a spacing of h.

/* shell sort */
#include <stdio.h>
#include <stdlib.h>
typedef int T;
typedef int tblIndex;

/* type of item to be sorted */

/* type of subscript */

#define compGT(a,b) (a > b)

void shellSort(T *a, tblIndex lb, tblIndex ub) {
tblIndex i, j;
unsigned int n;
int t;
static const unsigned int h[] = {
1, 5, 19, 41, 109, 209, 505, 929, 2161, 3905, 8929,
16001, 36289, 64769, 146305, 260609, 587521, 1045505,
2354689, 4188161, 9427969, 16764929, 37730305, 67084289,
150958081, 268386305, 603906049, 1073643521, 2415771649U
};
/* find t such that 3*h[t] >= n */
n = ub - lb + 1;
for (t = 0; 3*h[t] < n; t++);
/* start with h[t-1] */
if (t > 0) t--;
while (t >= 0) {
unsigned int ht;
/* sort-by-insertion in increments of h */
ht = h[t--];
for (i = lb + ht; i <= ub; i++) {
T tmp = a[i];
for (j = i-ht; j >= lb && compGT(a[j], tmp); j -= ht)
a[j+ht] = a[j];
a[j+ht] = tmp;
}
}

void fill(T *a, tblIndex lb, tblIndex ub) {

tblIndex i;
srand(1);
for (i = lb; i <= ub; i++) a[i] = rand();
}
int main(int argc, char *argv[]) {
tblIndex maxnum, lb, ub;
T *a;
/* command-line:
*
*
shl maxnum
*

*
*
*/

shl 2000
sort 2000 records

maxnum = atoi(argv[1]);
lb = 0; ub = maxnum - 1;
if ((a = malloc(maxnum * sizeof(T))) == 0) {
fprintf (stderr, "insufficient memory (a)\n");
exit(1);
}
fill(a, lb, ub);
shellSort(a, lb, ub);
}

return 0;

Quicksort

Although the shell sort algorithm is significantly better than insertion sort, there is still room for
improvement. One of the most popular sorting algorithms is quicksort. Quicksort executes in
O(n lgn) on average, and O(n2) in the worst-case. However, with proper precautions, worst-case
behavior is very unlikely. Quicksort is a non-stable sort. It is not an in-place sort as stack space is
required. For further reading, consult Cormen [2009].

Theory
The quicksort algorithm works by partitioning the array to be sorted, then recursively sorting
each partition. In Partition (Figure 2-3), one of the array elements is selected as a pivot value.
Values smaller than the pivot value are placed to the left of the pivot, while larger values are
placed to the right.
int function Partition (Array A, int Lb, int Ub);
begin
select a pivot from A[Lb]...A[Ub];
reorder A[Lb]...A[Ub] such that:
all values to the left of the pivot are <= pivot
all values to the right of the pivot are >= pivot
return pivot position;
end;
procedure QuickSort (Array A, int Lb, int Ub);
begin
if Lb < Ub then
M = Partition (A, Lb, Ub);
QuickSort (A, Lb, M - 1);
QuickSort (A, M, Ub);

end;

Figure 2-3: Quicksort Algorithm

In Figure 2-4(a), the pivot selected is 3. Indices are run starting at both ends of the array. One
index starts on the left and selects an element that is larger than the pivot, while another index
starts on the right and selects an element that is smaller than the pivot. In this case, numbers 4
and 1 are selected. These elements are then exchanged, as is shown in Figure 2-4(b). This
process repeats until all elements to the left of the pivot <= the pivot, and all elements to the right
of the pivot are >= the pivot. QuickSort recursively sorts the two subarrays, resulting in the array
shown in Figure 2-4(c).

Figure 2-4: Quicksort Example

As the process proceeds, it may be necessary to move the pivot so that correct ordering is
maintained. In this manner, QuickSort succeeds in sorting the array. If we're lucky the pivot
selected will be the median of all values, equally dividing the array. For a moment, let's assume
that this is the case. Since the array is split in half at each step, and Partition must eventually
examine all n elements, the run time is O(n lgn).
To find a pivot value, Partition could simply select the first element (A[Lb]). All other values
would be compared to the pivot value, and placed either to the left or right of the pivot as
appropriate. However, there is one case that fails miserably. Suppose the array was originally in
order. Partition would always select the lowest value as a pivot and split the array with one
element in the left partition, and Ub-Lb elements in the other. Each recursive call to quicksort
would only diminish the size of the array to be sorted by one. Therefore n recursive calls would
be required to do the sort, resulting in a O(n2) run time. One solution to this problem is to
randomly select an item as a pivot. This would make it extremely unlikely that worst-case
behavior would occur.

Implementation in C
An ANSI-C implementation of quicksort is included. Typedef T and comparison operator
compGT should be altered to reflect the data stored in the array. Two version of quicksort are
included: quickSort, and quickSortImproved. Enhancements include:

The center element is selected as a pivot in partition. If the list is partially

ordered, this will be a good choice. Worst-case behavior occurs when the
center element happens to be the largest or smallest element each time
partition is invoked.

For short arrays, insertSort is called. Due to recursion and other overhead,
quicksort is not an efficient algorithm to use on small arrays. Consequently,
any array with fewer than 50 elements is sorted using an insertion sort.
Cutoff values of 12-200 are appropriate.

Tail recursion occurs when the last statement in a function is a call to the
function itself. Tail recursion may be replaced by iteration, resulting in a
better utilization of stack space.

After an array is partitioned, the smallest partition is sorted first. This results
in a better utilization of stack space, as short partitions are quickly sorted and
dispensed with.

Included is a version of quicksort that sorts linked-lists. Also included is an ANSI-C

implementation, of qsort, a standard C library function usually implemented with quicksort.
Recursive calls were replaced by explicit stack operations. Table 2-1, shows timing statistics and
stack utilization before and after the enhancements were applied.

/* quicksort */
#include <stdio.h>
#include <stdlib.h>
typedef int T;
typedef int tblIndex;

/* type of item to be sorted */

/* type of subscript */

#define compGT(a,b) (a > b)

#define compLT(a,b) (a < b)
void sortByInsertion(T *x, tblIndex lb, tblIndex ub) {
tblIndex i, j;
for (i = lb + 1; i <= ub; i++) {
T t = x[i];
/* shift down until insertion point found */
for (j = i-1; j >= 0 && compGT(x[j], t); j--)

x[j+1] = x[j];

/* insert */
x[j+1] = t;

}
tblIndex partition(T *x, tblIndex lb, tblIndex ub) {
/* select a pivot */
double pivot = x[(lb+ub)/2];
/* work from both ends, swapping to keep
*/
/* values less than pivot to the left, and */
/* values greater than pivot to the right */
tblIndex i = lb - 1;
tblIndex j = ub + 1;
while (1) {
T t;
while (compGT(x[--j], pivot));
while (compLT(x[++i], pivot));
if (i >= j) break;

/* swap x[i], x[t] */

t = x[i];
x[i] = x[j];
x[j] = t;

return j;
}
void quickSort(T *x, tblIndex lb, tblIndex ub) {
tblIndex m;

if (lb >= ub) return;

m = partition(x, lb, ub);
quickSort(x, lb, m);
quickSort(x, m + 1, ub);

void quickSortImproved(T *x, tblIndex lb, tblIndex ub) {

while (lb < ub) {
tblIndex m;
/* quickly sort short lists */
if (ub - lb <= 50) {
sortByInsertion(x, lb, ub);
return;
}
m = partition(x, lb, ub);
/* eliminate tail recursion and */
/* sort the smallest partition first */

/* to minimize stack requirements

*/
if (m - lb <= ub - m) {
quickSortImproved(x, lb, m);
lb = m + 1;
} else {
quickSortImproved(x, m + 1, ub);
ub = m;
}

}
void fill(T *a, tblIndex lb, tblIndex ub) {
tblIndex i;
srand(1);
for (i = lb; i <= ub; i++) a[i] = rand();
}
int main(int argc, char *argv[]) {
tblIndex maxnum, lb, ub;
T *a;
/* command-line:
*
*
qui maxnum
*
*
qui 2000
*
sorts 2000 records
*
*/
maxnum = atoi(argv[1]);
lb = 0; ub = maxnum - 1;
if ((a = malloc(maxnum * sizeof(T))) == 0) {
fprintf (stderr, "insufficient memory (a)\n");
exit(1);
}
fill(a, lb, ub);
quickSortImproved(a, lb, ub);
return 0;
}

External Sort

One method for sorting a file is to load the file into memory, sort the data in memory, then write
the results. When the file cannot be loaded into memory due to resource limitations, an external
sort applicable. We will implement an external sort using replacement selection to establish
initial runs, followed by a polyphase merge sort to merge the runs into one sorted file. I highly

recommend you consult Knuth [1998], as many details have been omitted.

Theory
For clarity, I'll assume that data is on one or more reels of magnetic tape. Figure 4-1 illustrates a
3-way polyphase merge. Initially, in phase A, all data is on tapes T1 and T2. Assume that the
beginning of each tape is at the bottom of the frame. There are two sequential runs of data on
T1: 4-8, and 6-7. Tape T2 has one run: 5-9. At phase B, we've merged the first run from tapes
T1 (4-8) and T2 (5-9) into a longer run on tape T3 (4-5-8-9). Phase C is simply renames the
tapes, so we may repeat the merge again. In phase D we repeat the merge, with the final output
on tape T3.
Phase

T1

T2

7
6
8
4

9
5

9
8
5
4

7
6
9
8
5
4

T3

7
6
9
8
7
6
5
4

Figure 4-1: Merge Sort

Several interesting details have been omitted from the previous illustration. For example, how
were the initial runs created? And, did you notice that they merged perfectly, with no extra runs
on any tapes? Before I explain the method used for constructing initial runs, let me digress for a
bit.
In 1202, Leonardo Fibonacci presented the following exercise in his Liber Abbaci (Book of the

Abacus): "How many pairs of rabbits can be produced from a single pair in a year's time?" We
may assume that each pair produces a new pair of offspring every month, each pair becomes
fertile at the age of one month, and that rabbits never die. After one month, there will be 2 pairs
of rabbits; after two months there will be 3; the following month the original pair and the pair
born during the first month will both usher in a new pair, and there will be 5 in all; and so on.
This series, where each number is the sum of the two preceeding numbers, is known as the
Fibonacci sequence:
0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, ... .

Curiously, the Fibonacci series has found wide-spread application to everything from the
arrangement of flowers on plants to studying the efficiency of Euclid's algorithm. There's even a
Fibonacci Quarterly journal. And, as you might suspect, the Fibonacci series has something to
do with establishing initial runs for external sorts.
Recall that we initially had one run on tape T2, and 2 runs on tape T1. Note that the numbers
{1,2} are two sequential numbers in the Fibonacci series. After our first merge, we had one run
on T1 and one run on T2. Note that the numbers {1,1} are two sequential numbers in the
Fibonacci series, only one notch down. We could predict, in fact, that if we had 13 runs on T2,
and 21 runs on T1 {13,21}, we would be left with 8 runs on T1 and 13 runs on T3 {8,13} after
one pass. Successive passes would result in run counts of {5,8}, {3,5}, {2,3}, {1,1}, and {0,1},
for a total of 7 passes. This arrangement is ideal, and will result in the minimum number of
passes. Should data actually be on tape, this is a big savings, as tapes must be mounted and
rewound for each pass. For more than 2 tapes, higher-order Fibonacci numbers are used.
Initially, all the data is on one tape. The tape is read, and runs are distributed to other tapes in
the system. After the initial runs are created, they are merged as described above. One method
we could use to create initial runs is to read a batch of records into memory, sort the records,
and write them out. This process would continue until we had exhausted the input tape. An
alternative algorithm, replacement selection, allows for longer runs. A buffer is allocated in
memory to act as a holding place for several records. Initially, the buffer is filled. Then, the
following steps are repeated until the input is exhausted:

Select the record with the smallest key that is >= the key of the last record
written.

If all keys are smaller than the key of the last record written, then we have
reached the end of a run. Select the record with the smallest key for the first
record of the next run.

Write the selected record.

Replace the selected record with a new record from input.

Figure 4-2 illustrates replacement selection for a small file. To keep things simple, I've allocated
a 2-record buffer. Typically, such a buffer would hold thousands of records. We load the buffer
in step B, and write the record with the smallest key (6) in step C. This is replaced with the next
record (key 8). We select the smallest key >= 6 in step D. This is key 7. After writing key 7, we
replace it with key 4. This process repeats until step F, where our last key written was 8, and all
keys are less than 8. At this point, we terminate the run, and start another.
Step

Input

Buffer

Output

5-3-4-8-6-7

5-3-4-8

6-7

5-3-4

8-7

5-3

8-4

6-7

3-4

6-7-8

5-4

6-7-8 | 3

6-7-8 | 3-4

6-7-8 | 3-4-5
Figure 4-2: Replacement Selection

This strategy simply utilizes an intermediate buffer to hold values until the appropriate time for
output. Using random numbers as input, the average length of a run is twice the length of the
buffer. However, if the data is somewhat ordered, runs can be extremely long. Thus, this method
is more effective than doing partial sorts.
When selecting the next output record, we need to find the smallest key >= the last key written.
One way to do this is to scan the entire list, searching for the appropriate key. However, when
the buffer holds thousands of records, execution time becomes prohibitive. An alternative
method is to use a binary tree structure, so that we only compare lgnitems.

Implementation in C
An ANSI-C implementation of an external sort is included. Function makeRuns calls readRec to
read the next record. Function readRec employs the replacement selection algorithm (utilizing a

binary tree) to fetch the next record, and makeRuns distributes the records in a Fibonacci
distribution. If the number of runs is not a perfect Fibonacci number, dummy runs are simulated
at the beginning of each file. Function mergeSort is then called to do a polyphase merge sort on
the runs.
/* external sort */
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
/****************************
* implementation dependent *
****************************/
/* template for workfiles (8.3 format) */
#define FNAME "_sort%03d.dat"
#define LNAME 13
/* comparison operators */
#define compLT(x,y) (x < y)
#define compGT(x,y) (x > y)
/* define the record to be sorted here */
#define LRECL 100
typedef int keyType;
typedef struct recTypeTag {
keyType key;
#if LRECL
char data[LRECL-sizeof(keyType)];
#endif
} recType;

/* sort key for record */

/* other fields */

/******************************
* implementation independent *
******************************/
typedef enum {false, true} bool;
typedef struct tmpFileTag {
FILE *fp;
char name[LNAME];
recType rec;
int dummy;
bool eof;
bool eor;
bool valid;
int fib;
} tmpFileType;

/*
/*
/*
/*
/*
/*
/*
/*

file pointer */
filename */
last record read */
number of dummy runs */
end-of-file flag */
end-of-run flag */
true if rec is valid */
ideal fibonacci number */

static
static
static
static

/*
/*
/*
/*

array of file info for tmp files */

number of tmp files */
input filename */
output filename */

tmpFileType **file;
int nTmpFiles;
char *ifName;
char *ofName;

static int level;

static int nNodes;

/* level of runs */
/* number of nodes for selection tree */

void deleteTmpFiles(void) {
int i;
/* delete merge files and free resources */
if (file) {
for (i = 0; i < nTmpFiles; i++) {
if (file[i]) {
if (file[i]->fp) fclose(file[i]->fp);
if (*file[i]->name) remove(file[i]->name);
free (file[i]);
}
}
free (file);
}
}
void termTmpFiles(int rc) {
/* cleanup files */
remove(ofName);
if (rc == 0) {
int fileT;
/* file[T] contains results */
fileT = nTmpFiles - 1;
fclose(file[fileT]->fp); file[fileT]->fp = NULL;
if (rename(file[fileT]->name, ofName)) {
perror("io1");
deleteTmpFiles();
exit(1);
}
*file[fileT]->name = 0;
}
deleteTmpFiles();
}
void cleanExit(int rc) {

/* cleanup tmp files and exit */

termTmpFiles(rc);
exit(rc);

void *safeMalloc(size_t size) {

void *p;
/* safely allocate memory and initialize to zero */
if ((p = calloc(1, size)) == NULL) {
printf("error: malloc failed, size = %d\n", size);
cleanExit(1);
}
return p;

}
void initTmpFiles(void) {
int i;
tmpFileType *fileInfo;
/* initialize merge files */
if (nTmpFiles < 3) nTmpFiles = 3;
file = safeMalloc(nTmpFiles * sizeof(tmpFileType*));
fileInfo = safeMalloc(nTmpFiles * sizeof(tmpFileType));
for (i = 0; i < nTmpFiles; i++) {
file[i] = fileInfo + i;
sprintf(file[i]->name, FNAME, i);
if ((file[i]->fp = fopen(file[i]->name, "w+b")) == NULL) {
perror("io2");
cleanExit(1);
}
}
}
recType *readRec(void) {
typedef struct iNodeTag {
/* internal node */
struct iNodeTag *parent;/* parent of internal node */
struct eNodeTag *loser; /* external loser */
} iNodeType;
typedef struct eNodeTag {
/*
struct iNodeTag *parent;/*
recType rec;
/*
int run;
/*
bool valid;
/*
} eNodeType;

external node */
parent of external node */
input record */
run number */
input record is valid */

typedef struct nodeTag {

iNodeType i;
eNodeType e;
} nodeType;

/* internal node */
/* external node */

static nodeType *node;

static eNodeType *win;
static FILE *ifp;
static bool eof;
static int maxRun;
static int curRun;
iNodeType *p;
static bool lastKeyValid;
static keyType lastKey;

/*
/*
/*
/*
/*
/*
/*
/*
/*

array of selection tree nodes */

new winner */
input file */
true if end-of-file, input */
maximum run number */
current run number */
pointer to internal nodes */
true if lastKey is valid */
last key written */

/* read next record using replacement selection */

/* check for first call */
if (node == NULL) {
int i;
if (nNodes < 2) nNodes = 2;

node = safeMalloc(nNodes * sizeof(nodeType));

for (i = 0; i < nNodes; i++) {
node[i].i.loser = &node[i].e;
node[i].i.parent = &node[i/2].i;
node[i].e.parent = &node[(nNodes + i)/2].i;
node[i].e.run = 0;
node[i].e.valid = false;
}
win = &node[0].e;
lastKeyValid = false;

if ((ifp = fopen(ifName, "rb")) == NULL) {

printf("error: file %s, unable to open\n", ifName);
cleanExit(1);
}

while (1) {
/* replace previous winner with new record */
if (!eof) {
if (fread(&win->rec, sizeof(recType), 1, ifp) == 1) {
if ((!lastKeyValid || compLT(win->rec.key, lastKey))
&& (++win->run > maxRun))
maxRun = win->run;
win->valid = true;
} else if (feof(ifp)) {
fclose(ifp);
eof = true;
win->valid = false;
win->run = maxRun + 1;
} else {
perror("io4");
cleanExit(1);
}
} else {
win->valid = false;
win->run = maxRun + 1;
}
/* adjust loser and winner pointers */
p = win->parent;
do {
bool swap;
swap = false;
if (p->loser->run < win->run) {
swap = true;
} else if (p->loser->run == win->run) {
if (p->loser->valid && win->valid) {
if (compLT(p->loser->rec.key, win->rec.key))
swap = true;
} else {
swap = true;
}
}
if (swap) {

/* p should be winner */
eNodeType *t;
t = p->loser;
p->loser = win;
win = t;
}
p = p->parent;
} while (p != &node[0].i);
/* end of run? */
if (win->run != curRun) {
/* win->run = curRun + 1 */
if (win->run > maxRun) {
/* end of output */
free(node);
return NULL;
}
curRun = win->run;
}

/* output top of tree */

if (win->run) {
lastKey = win->rec.key;
lastKeyValid = true;
return &win->rec;
}

}
void makeRuns(void) {
recType *win;
int fileT;
int fileP;
int j;

/*
/*
/*
/*

winner */
last file */
next to last file */
selects file[j] */

/* Make initial runs using replacement selection.

* Runs are written using a Fibonacci distintbution.
*/
/* initialize file structures */
fileT = nTmpFiles - 1;
fileP = fileT - 1;
for (j = 0; j < fileT; j++) {
file[j]->fib = 1;
file[j]->dummy = 1;
}
file[fileT]->fib = 0;
file[fileT]->dummy = 0;
level = 1;
j = 0;
win = readRec();

while (win) {
bool anyrun;
anyrun = false;
for (j = 0; win && j <= fileP; j++) {
bool run;
run = false;
if (file[j]->valid) {
if (!compLT(win->key, file[j]->rec.key)) {
/* append to an existing run */
run = true;
} else if (file[j]->dummy) {
/* start a new run */
file[j]->dummy--;
run = true;
}
} else {
/* first run in file */
file[j]->dummy--;
run = true;
}
if (run) {
anyrun = true;

/* flush run */
while(1) {
if (fwrite(win, sizeof(recType), 1, file[j]->fp) != 1) {
perror("io3");
cleanExit(1);
}
file[j]->rec.key = win->key;
file[j]->valid = true;
if ((win = readRec()) == NULL) break;
if (compLT(win->key, file[j]->rec.key)) break;
}

}
/* if no room for runs, up a level */
if (!anyrun) {
int t;
level++;
t = file[0]->fib;
for (j = 0; j <= fileP; j++) {
file[j]->dummy = t + file[j+1]->fib - file[j]->fib;
file[j]->fib = t + file[j+1]->fib;
}
}
}

void rewindFile(int j) {
/* rewind file[j] and read in first record */
file[j]->eor = false;

file[j]->eof = false;
rewind(file[j]->fp);
if (fread(&file[j]->rec, sizeof(recType), 1, file[j]->fp) != 1) {
if (feof(file[j]->fp)) {
file[j]->eor = true;
file[j]->eof = true;
} else {
perror("io5");
cleanExit(1);
}
}

void mergeSort(void) {
int fileT;
int fileP;
int j;
tmpFileType *tfile;
/* polyphase merge sort */
fileT = nTmpFiles - 1;
fileP = fileT - 1;
/* prime the files */
for (j = 0; j < fileT; j++) {
rewindFile(j);
}
/* each pass through loop merges one run */
while (level) {
while(1) {
bool allDummies;
bool anyRuns;
/* scan for runs */
allDummies = true;
anyRuns = false;
for (j = 0; j <= fileP; j++) {
if (!file[j]->dummy) {
allDummies = false;
if (!file[j]->eof) anyRuns = true;
}
}
if (anyRuns) {
int k;
keyType lastKey;
/* merge 1 run file[0]..file[P] --> file[T] */
while(1) {
/* each pass thru loop writes 1 record to file[fileT] */
/* find smallest key */
k = -1;

for (j
if
if
if
(k

>rec.key)))

= 0; j <= fileP; j++) {

(file[j]->eor) continue;
(file[j]->dummy) continue;
(k < 0 ||
!= j && compGT(file[k]->rec.key, file[j]-

k = j;
}
if (k < 0) break;
/* write record[k] to file[fileT] */
if (fwrite(&file[k]->rec, sizeof(recType), 1,
file[fileT]->fp) != 1) {
perror("io6");
cleanExit(1);
}

/* replace record[k] */
lastKey = file[k]->rec.key;
if (fread(&file[k]->rec, sizeof(recType), 1,
file[k]->fp) == 1) {
/* check for end of run on file[s] */
if (compLT(file[k]->rec.key, lastKey))
file[k]->eor = true;
} else if (feof(file[k]->fp)) {
file[k]->eof = true;
file[k]->eor = true;
} else {
perror("io7");
cleanExit(1);
}

/* fixup dummies */
for (j = 0; j <= fileP; j++) {
if (file[j]->dummy) file[j]->dummy--;
if (!file[j]->eof) file[j]->eor = false;
}
} else if (allDummies) {
for (j = 0; j <= fileP; j++)
file[j]->dummy--;
file[fileT]->dummy++;
}
/* end of run */
if (file[fileP]->eof && !file[fileP]->dummy) {
/* completed a fibonocci-level */
level--;
if (!level) {
/* we're done, file[fileT] contains data */
return;
}
/* fileP is exhausted, reopen as new */
fclose(file[fileP]->fp);

if ((file[fileP]->fp = fopen(file[fileP]->name, "w+b"))

== NULL) {
perror("io8");
cleanExit(1);
}
file[fileP]->eof = false;
file[fileP]->eor = false;
rewindFile(fileT);
/* f[0],f[1]...,f[fileT] <-- f[fileT],f[0]...,f[T-1] */
tfile = file[fileT];
memmove(file + 1, file, fileT * sizeof(tmpFileType*));
file[0] = tfile;
/* start new runs */
for (j = 0; j <= fileP; j++)
if (!file[j]->eof) file[j]->eor = false;
}

}
}
void extSort(void) {
initTmpFiles();
makeRuns();
mergeSort();
termTmpFiles(0);
}
int main(int argc, char *argv[]) {
/* command-line:
*
*
ext ifName ofName nTmpFiles nNodes
*
*
ext in.dat out.dat 5 2000
*
reads in.dat, sorts using 5 files and 2000 nodes, output to
out.dat
*/
if (argc != 5) {
printf("%s ifName ofName nTmpFiles nNodes\n", argv[0]);
cleanExit(1);
}
ifName = argv[1];
ofName = argv[2];
nTmpFiles = atoi(argv[3]);
nNodes = atoi(argv[4]);
printf("extSort: nFiles=%d, nNodes=%d, lrecl=%d\n",
nTmpFiles, nNodes, sizeof(recType));
extSort();

return 0;

Binary Search Trees

In the introduction we used the binary search algorithm to find data stored in an array. This
method is very effective, as each iteration reduced the number of items to search by one-half.
However, since data was stored in an array, insertions and deletions were not efficient. Binary
search trees store data in nodes that are linked in a tree-like fashion. For randomly inserted data,
search time is O(lgn). Worst-case behavior occurs when ordered data is inserted. In this case the
search time is O(n). See Cormen [2001] for a more detailed description.

Theory
A binary search tree is a tree where each node has a left and right child. Either child, or both
children, may be missing. Figure 3-2 illustrates a binary search tree. Assuming krepresents the
value of a given node, then a binary search tree also has the following property: all children to
the left of the node have values smaller than k, and all children to the right of the node have
values larger than k. The top of a tree is known as the root, and the exposed nodes at the bottom
are known as leaves. In Figure 3-2, the root is node 20 and the leaves are nodes 4, 16, 37, and
43. The height of a tree is the length of the longest path from root to leaf. For this example the
tree height is 2.

Figure 3-2: A Binary Search Tree

To search a tree for a given value, we start at the root and work down. For example, to search
for 16, we first note that 16 < 20 and we traverse to the left child. The second comparison finds
that 16 > 7, so we traverse to the right child. On the third comparison, we succeed.
Each comparison results in reducing the number of items to inspect by one-half. In this respect,
the algorithm is similar to a binary search on an array. However, this is true only if the tree is
balanced. For example, Figure 3-3 shows another tree containing the same values. While it is a
binary search tree, its behavior is more like that of a linked list, with search time increasing
proportional to the number of elements stored.

Figure 3-3: An Unbalanced Binary Search Tree

Insertion and Deletion

Let us examine insertions in a binary search tree to determine the conditions that can cause an
unbalanced tree. To insert an 18 in the tree in Figure 3-2, we first search for that number. This
causes us to arrive at node 16 with nowhere to go. Since 18 > 16, we simply add node 18 to the
right child of node 16 (Figure 3-4).
Now we can see how an unbalanced tree can occur. If the data is presented in an ascending
sequence, each node will be added to the right of the previous node. This will create one long
chain, or linked list. However, if data is presented for insertion in a random order, then a more
balanced tree is possible.
Deletions are similar, but require that the binary search tree property be maintained. For
example, if node 20 in Figure 3-4 is removed, it must be replaced by node 18 or node 37.
Assuming we replace a node by its successor, the resulting tree is shown in Figure 3-5. The
rationale for this choice is as follows. The successor for node 20 must be chosen such that all
nodes to the right are larger. Therefore we need to select the smallest valued node to the right of
node 20. To make the selection, chain once to the right (node 38), and then chain to the left until
the last node is found (node 37). This is the successor for node 20.

Figure 3-4: Binary Tree After Adding Node 18

Figure 3-5: Binary Tree After Deleting Node 20

Implementation in C
An ANSI-C implementation for a binary search tree is included. Typedefs recType, keyType,
and comparison operators compLT and compEQ should be altered to reflect the data stored in the
tree. Each Node consists of left, right, and parent pointers designating each child and the
parent. The tree is based atroot, and is initially NULL. Function insert allocates a new node
and inserts it in the tree. Function delete deletes and frees a node from the tree. Function find
searches the tree for a particular value.
/* binary search tree */
#include <stdio.h>
#include <stdlib.h>
#define compLT(a,b) (a < b)
#define compEQ(a,b) (a == b)
/* implementation dependent declarations */
typedef enum {
STATUS_OK,
STATUS_MEM_EXHAUSTED,
STATUS_DUPLICATE_KEY,
STATUS_KEY_NOT_FOUND
} statusEnum;
typedef int keyType;
/* user data stored in tree */
typedef struct {
int stuff;
} recType;

/* type of key */

/* optional related data */

typedef struct nodeTag {

struct nodeTag *left;
struct nodeTag *right;
struct nodeTag *parent;
keyType key;
recType rec;
} nodeType;

/*
/*
/*
/*
/*

nodeType *root = NULL;

/* root of binary tree */

left child */
right child */
parent */
key used for searching */
user data */

statusEnum insert(keyType key, recType *rec) {

nodeType *x, *current, *parent;
/***********************************************
* allocate node for data and insert in tree *
***********************************************/
/* find future parent */
current = root;
parent = 0;
while (current) {
if (compEQ(key, current->key))
return STATUS_DUPLICATE_KEY;
parent = current;
current = compLT(key, current->key) ?
current->left : current->right;
}
/* setup new node */
if ((x = malloc (sizeof(*x))) == 0) {
return STATUS_MEM_EXHAUSTED;
}
x->parent = parent;
x->left = NULL;
x->right = NULL;
x->key = key;
x->rec = *rec;
/* insert x in tree */
if(parent)
if(compLT(x->key, parent->key))
parent->left = x;
else
parent->right = x;
else
root = x;
}

return STATUS_OK;

statusEnum delete(keyType key) {

nodeType *x, *y, *z;
/***************************
* delete node from tree *
***************************/
/* find node in tree */
z = root;
while(z != NULL) {
if(compEQ(key, z->key))
break;
else
z = compLT(key, z->key) ? z->left : z->right;
}

if (!z) return STATUS_KEY_NOT_FOUND;

/* find tree successor */
if (z->left == NULL || z->right == NULL)
y = z;
else {
y = z->right;
while (y->left != NULL) y = y->left;
}
/* point x to a valid child of y, if it has one */
if (y->left != NULL)
x = y->left;
else
x = y->right;
/* remove y from the parent chain */
if (x) x->parent = y->parent;
if (y->parent)
if (y == y->parent->left)
y->parent->left = x;
else
y->parent->right = x;
else
root = x;
/* if z and y are not the same, copy y to z. */
if (y != z) {
z->key = y->key;
z->rec = y->rec;
}
free (y);
return STATUS_OK;
}
statusEnum find(keyType key, recType *rec) {
/*******************************
* find node containing data *
*******************************/

nodeType *current = root;

while(current != NULL) {
if(compEQ(key, current->key)) {
*rec = current->rec;
return STATUS_OK;
} else {
current = compLT(key, current->key) ?
current->left : current->right;
}
}
return STATUS_KEY_NOT_FOUND;

int main(int argc, char **argv) {

int i, maxnum, random;

recType *rec;
keyType *key;
statusEnum status;
/* command-line:
*
*
bin maxnum random
*
*
bin 5000
// 5000 sequential
*
bin 2000 r
// 2000 random
*
*/
maxnum = atoi(argv[1]);
random = argc > 2;
if ((rec = malloc(maxnum * sizeof(recType))) == 0) {
fprintf (stderr, "insufficient memory (rec)\n");
exit(1);
}
if ((key = malloc(maxnum * sizeof(keyType))) == 0) {
fprintf (stderr, "insufficient memory (key)\n");
exit(1);
}
if (random) { /* random */
/* fill "key" with unique random numbers */
for (i = 0; i < maxnum; i++) key[i] = rand();
printf ("ran bt, %d items\n", maxnum);
} else {
for (i=0; i<maxnum; i++) key[i] = i;
printf ("seq bt, %d items\n", maxnum);
}
for (i = 0; i < maxnum; i++) {
status = insert(key[i], &rec[i]);
if (status) printf("pt1, i=%d: %d\n", i, status);
}
for (i = maxnum-1; i >= 0; i--) {
status = find(key[i], &rec[i]);
if (status) printf("pt2, i=%d: %d\n", i, status);
}
for (i = maxnum-1; i >= 0; i--) {
status = delete(key[i]);
if (status) printf("pt3, i=%d: %d\n", i, status);
}
return 0;
}

Merge Sort
Merge sort is based on the divide-and-conquer paradigm. Its worst-case running time has a
lower order of growth than insertion sort. Since we are dealing with subproblems, we state each
subproblem as sorting a subarray A[p .. r]. Initially, p = 1 and r = n, but these values change as
we recurse through subproblems.
To sort A[p .. r]:
1. Divide Step
If a given array A has zero or one element, simply return; it is already sorted. Otherwise, split
A[p .. r] into two subarrays A[p .. q] and A[q + 1 .. r], each containing about half of the elements
of A[p .. r]. That is, q is the halfway point of A[p .. r].
2. Conquer Step
Conquer by recursively sorting the two subarrays A[p .. q] and A[q + 1 .. r].
3. Combine Step
Combine the elements back in A[p .. r] by merging the two sorted subarrays A[p .. q] and A[q +
1 .. r] into a sorted sequence. To accomplish this step, we will define a procedure MERGE (A, p,
q, r).

Note that the recursion bottoms out when the subarray has just one element, so that it is trivially
sorted.

Algorithm: Merge Sort

To sort the entire sequence A[1 .. n], make the initial call to the procedure MERGE-SORT (A,
1, n).
MERGE-SORT (A, p, r)
1.
2.
3.
4.
5.

IF p < r
THEN q = FLOOR[(p + r)/2]
MERGE (A, p, q)
MERGE (A, q + 1, r)
MERGE (A, p, q, r)

// Check for base case

// Divide step
// Conquer step.
// Conquer step.
// Conquer step.

Example: Bottom-up view of the above procedure for n = 8.

Merging
What remains is the MERGE procedure. The following is the input and output of the MERGE
procedure.
INPUT: Array A and indices p, q, r such that p q r and subarray A[p .. q] is sorted and
subarray A[q + 1 .. r] is sorted. By restrictions on p, q, r, neither subarray is empty.
OUTPUT: The two subarrays are merged into a single sorted subarray in A[p .. r].
We implement it so that it takes (n) time, where n = r p + 1, which is the number of elements
being merged.

Idea Behind Linear Time Merging

Think of two piles of cards, Each pile is sorted and placed face-up on a table with the smallest
cards on top. We will merge these into a single sorted pile, face-down on the table.
A basic step:

Choose the smaller of the two top cards.

Remove it from its pile, thereby exposing a new top card.

Place the chosen card face-down onto the output pile.

Repeatedly perform basic steps until one input pile is empty.

Once one input pile empties, just take the remaining input pile and place it face-down
onto the output pile.

Each basic step should take constant time, since we check just the two top cards. There are at
most n basic steps, since each basic step removes one card from the input piles, and we started
with n cards in the input piles. Therefore, this procedure should take (n) time.
Now the question is do we actually need to check whether a pile is empty before each basic
step?
The answer is no, we do not. Put on the bottom of each input pile a special sentinel card. It

contains a special value that we use to simplify the code. We use , since that's guaranteed to
lose to any other value. The only way that cannot lose is when both piles have exposed as
their top cards. But when that happens, all the nonsentinel cards have already been placed into
the output pile. We know in advance that there are exactly r p + 1 nonsentinel cards so stop
once we have performed r p + 1 basic steps. Never a need to check for sentinels, since they
will always lose. Rather than even counting basic steps, just fill up the output array from index
p up through and including index r .
The pseudocode of the MERGE procedure is as follow:
MERGE (A, p, q, r )
1.
n1 q p + 1
2.
n2 r q
3.
Create arrays L[1 . . n1 + 1] and R[1 . . n2 + 1]
4.
FOR i 1 TO n1
5.
DO L[i] A[p + i 1]
6.
FOR j 1 TO n2
7.
DO R[j] A[q + j ]
8.
L[n1 + 1]
9.
R[n2 + 1]
10. i 1
11. j 1
12. FOR k p TO r
13.
DO IF L[i ] R[ j]
14.
THEN A[k] L[i]
15.
ii+1
16.
ELSE A[k] R[j]
17.
jj+1

Example [from CLRS-Figure 2.3]: A call of MERGE(A, 9, 12, 16). Read the following figure
row by row. That is how we have done in the class.

The first part shows the arrays at the start of the "for k p to r" loop, where A[p . . q] is
copied into L[1 . . n1] and A[q + 1 . . r ] is
copied into R[1 . . n2].

Succeeding parts show the situation at the start of successive iterations.

Entries in A with slashes have had their values copied to either L or R and have not had a
value copied back in yet. Entries in L and R with slashes have been copied back into A.

The last part shows that the subarrays are merged back into A[p . . r], which is now
sorted, and that only the sentinels () are exposed in the arrays L and R.]

Running Time
The first two for loops (that is, the loop in line 4 and the loop in line 6) take (n1 + n2) = (n)
time. The last for loop (that is, the loop in line 12) makes n iterations, each taking constant time,

for (n) time. Therefore, the total running time is (n).

Analyzing Merge Sort

For simplicity, assume that n is a power of 2 so that each divide step yields two subproblems,
both of size exactly n/2.
The base case occurs when n = 1.
When n 2, time for merge sort steps:

Divide: Just compute q as the average of p and r, which takes constant time i.e. (1).

Conquer: Recursively solve 2 subproblems, each of size n/2, which is 2T(n/2).

Combine: MERGE on an n-element subarray takes (n) time.

Summed together they give a function that is linear in n, which is (n). Therefore, the
recurrence for merge sort running time is

Solving the Merge Sort Recurrence

By the master theorem in CLRS-Chapter 4 (page 73), we can show that this recurrence has the
solution
T(n) = (n lg n).
Reminder: lg n stands for log2 n.
Compared to insertion sort [(n2) worst-case time], merge sort is faster. Trading a factor of n for
a factor of lg n is a good deal. On small inputs, insertion sort may be faster. But for large enough
inputs, merge sort will always be faster, because its running time grows more slowly than
insertion sorts.

Recursion Tree
We can understand how to solve the merge-sort recurrence without the master theorem. There is
a drawing of recursion tree on page 35 in CLRS, which shows successive expansions of the
recurrence.
The following figure (Figure 2.5b in CLRS) shows that for the original problem, we have a cost
of cn, plus the two subproblems, each costing T (n/2).

The following figure (Figure 2.5c in CLRS) shows that for each of the size-n/2 subproblems, we
have a cost of cn/2, plus two subproblems, each costing T (n/4).

The following figure (Figure: 2.5d in CLRS) tells to continue expanding until the problem sizes
get down to 1.

In the above recursion tree, each level has cost cn.

The top level has cost cn.

The next level down has 2 subproblems, each contributing cost cn/2.

The next level has 4 subproblems, each contributing cost cn/4.

Each time we go down one level, the number of subproblems doubles but the cost per
subproblem halves. Therefore, cost per level stays the same.

The height of this recursion tree is lg n and there are lg n + 1 levels.

Mathematical Induction

We use induction on the size of a given subproblem n.

Base case: n = 1
Implies that there is 1 level, and lg 1 + 1 = 0 + 1 = 1.
Inductive Step
Our inductive hypothesis is that a tree for a problem size of 2i has lg 2i + 1 = i +1 levels.
Because we assume that the problem size is a power of 2, the next problem size up after 2i is 2i
+ 1. A tree for a problem size of 2i + 1 has one more level than the size-2i tree implying i + 2
levels. Since lg 2i + 1 = i + 2, we are done with the inductive argument.
Total cost is sum of costs at each level of the tree. Since we have lg n +1 levels, each costing cn,
the total cost is
cn lg n + cn.
Ignore low-order term of cn and constant coefcient c, and we have,
(n lg n)
which is the desired result.

Implementation
void mergeSort(int numbers[], int temp[], int array_size)
{
m_sort(numbers, temp, 0, array_size - 1);
}

void m_sort(int numbers[], int temp[], int left, int right)

{
int mid;
if (right > left)
{

mid = (right + left) / 2;

m_sort(numbers, temp, left, mid);
m_sort(numbers, temp, mid+1, right);

merge(numbers, temp, left, mid+1, right);

}
}

void merge(int numbers[], int temp[], int left, int mid, int right)
{
int i, left_end, num_elements, tmp_pos;

left_end = mid - 1;
tmp_pos = left;
num_elements = right - left + 1;

while ((left <= left_end) && (mid <= right))

{
if (numbers[left] <= numbers[mid])
{
temp[tmp_pos] = numbers[left];

tmp_pos = tmp_pos + 1;
left = left +1;
}
else
{
temp[tmp_pos] = numbers[mid];
tmp_pos = tmp_pos + 1;
mid = mid + 1;
}
}

while (left <= left_end)

{
temp[tmp_pos] = numbers[left];
left = left + 1;
tmp_pos = tmp_pos + 1;
}
while (mid <= right)
{
temp[tmp_pos] = numbers[mid];

mid = mid + 1;
tmp_pos = tmp_pos + 1;
}

for (i = 0; i <= num_elements; i++)

{
numbers[right] = temp[right];
right = right - 1;
}
}

Merge sort
From Wikipedia, the free encyclopedia
Jump to: navigation, search
Merge sort

An example of merge sort. First divide the list into the

smallest unit (1 element), then compare each element
with the adjacent list to sort and merge the two
adjacent lists. Finally all the elements are sorted and
merged.
Class
Sorting algorithm
Data structure
Array
Worst case performance
O(n log n)
O(n log n) typical,
Best case performance
Average case performance
Worst case space complexity

O(n) natural
variant
O(n log n)
O(n) auxiliary

Merge sort (also commonly spelled mergesort) is an O(n log n) comparison-based sorting
algorithm. Most implementations produce a stable sort, which means that the implementation
preserves the input order of equal elements in the sorted output. Merge sort is a divide and
conquer algorithm that was invented by John von Neumann in 1945.[1] A detailed description
and analysis of bottom-up mergesort appeared in a report by Goldstine and Neumann as early as
1948.[2]

Merge Sorting
The fourth class of sorting algorithm we consider comprises algorithms that sort by merging .
Merging is the combination of two or more sorted sequences into a single sorted sequence.
Figure illustrates the basic, two-way merge operation. In a two-way merge, two sorted
sequences are merged into one. Clearly, two sorted sequences each of length n can be merged
into a sorted sequence of length 2n in O(2n)=O(n) steps. However in order to do this, we need
space in which to store the result. I.e., it is not possible to merge the two sequences in place in
O(n) steps.

Figure: Two-Way Merging

Sorting by merging is a recursive, divide-and-conquer strategy. In the base case, we have a
sequence with exactly one element in it. Since such a sequence is already sorted, there is nothing
to be done. To sort a sequence of n>1 elements:

1. Divide the sequence into two sequences of length

and

2. recursively sort each of the two subsequences; and then,

3. merge the sorted subsequences to obtain the final result.
Figure

illustrates the operation of the two-way merge sort algorithm.

Figure: Two-Way Merge Sorting

Implementation

Program

declares the TwoWayMergeSorter<T> class template. A single member variable,

tempArray, is declared. This variable is a pointer to an Array<T> instance. Since merge
operations cannot be done in place, a second, temporary array is needed. The tempArray variable
is keeps track of that array. The TwoWayMergeSorter constructor simply sets the tempArray
pointer to zero.

Program: TwoWayMergeSorter<T> Class Definition

In addition to the constructor, three protected member functions are declared, Merge and two
versions of DoSort. The purpose of the Merge function is to merge sorted subsequences of the
array to be sorted. The two DoSort routines implement the sorting algorithm itself.

7.5 Radix Sorting

The bin sorting approach can be generalised in a technique that is known as radix sorting.
An example
Assume that we have n integers in the range (0,n2) to be sorted. (For a bin sort, m
= n2, and we would have an O(n+m) = O(n2) algorithm.) Sort them in two phases:
1. Using n bins, place ai into bin ai mod n,

2. Repeat the process using n bins, placing ai into bin floor(ai/n), being careful
to append to the end of each bin.
This results in a sorted list.

As an example, consider the list of integers:

36 9 0 25 1 49 64 16 81 4

n is 10 and the numbers all lie in (0,99). After the first phase, we will have:
Bin
Content

1
81

2
-

3
-

64
4

25

36
16

9
49

Note that in this phase, we placed each item in a bin indexed by the least significant decimal
digit.
Repeating the process, will produce:
Bin

16

25

36

49

64

81

Content

0
1
4
9

In this second phase, we used the leading decimal digit to allocate items to bins, being careful to
add each item to the end of the bin.
We can apply this process to numbers of any size expressed to any suitable base or radix.
7.5.1 Generalised Radix Sorting
We can further observe that it's not necessary to use the same radix in each phase,
suppose that the sorting key is a sequence of fields, each with bounded ranges, eg
the key is a date using the structure:
typedef
int
int
int
} date;

struct t_date {
day;
month;
year;

If the ranges for day and month are limited in the obvious way, and the range for
year is suitably constrained, eg 1900 < year <= 2000, then we can apply the same
procedure except that we'll employ a different number of bins in each phase. In all
cases, we'll sort first using the least significant "digit" (where "digit" here means a

field with a limited range), then using the next significant "digit", placing each item
after all the items already in the bin, and so on.

Assume that the key of the item to be sorted has k fields, fi|i=0..k-1, and that each fi has si
discrete values, then a generalised radix sort procedure can be written:
radixsort( A, n ) {
for(i=0;i<k;i++) {
for(j=0;j<si;j++) bin[j] = EMPTY;

O(si)

for(j=0;j<n;j++) {
move Ai
to the end of bin[Ai->fi]
}

O(n)

for(j=0;j<si;j++)
concatenate bin[j] onto the end of A;
}

O(si)

Total

Now if, for example, the keys are integers in (0,bk-1), for some constant k, then the keys can be
viewed as k-digit base-b integers.
Thus, si = b for all i and the time complexity becomes O(n+kb) or O(n). This result depends on
k being constant.
If k is allowed to increase with n, then we have a different picture. For example, it takes log2n
binary digits to represent an integer <n. If the key length were allowed to increase with n, so that
k = logn, then we would have:

.
Another way of looking at this is to note that if the range of the key is restricted to (0,bk-1), then
we will be able to use the radixsort approach effectively if we allow duplicate keys when n>bk.
However, if we need to have unique keys, then k must increase to at least logbn. Thus, as n
increases, we need to have logn phases, each taking O(n) time, and the radix sort is the same as
quick sort!

Sample code
This sample code sorts arrays of integers on various radices: the number of bits
used for each radix can be set with the call to SetRadices. The Bins class is used in
each phase to collect the items as they are sorted. ConsBins is called to set up a set
of bins: each bin must be large enough to accommodate the whole array, so
RadixSort can be very expensive in its memory usage!