CHAPTER

4
Analysis of Simple Sorting
Algorithms

neof the fundamental problems of computer science is ordering a list of items. There's
Va surplus of solutions to this problem, known as sorting algorithms. Some sorting
algorithms are simple, such as the bubble sort. Others, such as the quick sort, are extremely
complicated, but produce lightning-fast results.
The common sorting algorithms can be divided into two classes by the complexity of their
algorithms. Algorithmic complexity is generally written in a form known as Big-O notation, where
the 'O represents the complexity of the algorithm and a value nrepresents the size of the setthe
algorithm is run agaínst.
For example, O(n) meansthat an algorithm has a linear complexity. In other words, it takes
ten times longer to operate on aset of 100 items than it does on a set of 10 items (10+10 =100) If the
complexity was O(n) (quadratic complexity), then it would take 100 times longer to operate on a
Set of 100 items than it does on a set of 10items.
The two classes of sorting algorithms are O(u') which inchudes the bubble, insertion,
ection, and shellsorts ; and O(nlog n) which includes the heap, merge, and quick sorts.
In addition to algorithmic complexity, the speed of the various sorts can be compared with
pirical data. Since the speed of asort can vary greatly depending on what data set it sorts, accurate
empirical results require several runs of the sort be made and the results averaged together.
Here, we are going to look at three simple sorting techniques : Bubble Sort, Selection Sort, and
Insertion Sort.
75)
76 ALGORITHMS DESIGN AND
ANALYSIS
4.1 Bubble Sort

Bubble sort, also known as exchange sort, is a simple sorting algorithm. It worksby if
stepping through the list tobe sorted, comparing two items at a time and swapping them whi
repeatedly
are in the wrong order. The pass through the list is repeated until no swaps are needed,
means the list is sorted. The algorithm gets its name from the way smaller elements "bubble"
the top (i.e., the beginning) of the list via the swaps. Because it only uses comparisons to opera
on elements, it is a comparison sort. This is the easiest comparison sort to implement.
Compare each element (except the last one) with its neighbour to the right
If they are out of order, swap them
4 This puts the largest element at the very end
The last element is now in the correct and final place
Compare each element (except the last tuo) with its neighbour to the right
If they are out of order, swap them
This puts the second largest element next to last
The last two elements are nowin their correct and final places
Compare each element (except the last three) with its neighbour to the right. Continue as
above until you have no unsorted elements on the left.
The bubble sort is generally considered to the most inefficient sorting algorithm in common
usage. Under best-case conditions (the list is already sorted), the bubble sort can approach a
constant O(n) level of complexity. General-case is an extremely bad O(n)
Bubble Sort (A)
1. for i 4-1 to length [A]
2. for j ength [A] downto i+1
3. if AL] <A[j-1]
4. exchange (A], ALj -1])
The outer loop is executed n-1times. Each time the outer loop is executed, the inner loop is
executed. Inner loop executes n-1 times at first, linearly dropping to just once. On average, inner
loop executes about n/2 times for each execution of the outer loop. In the inner loop, the comparison
is always done (constant time), the swap might be done (also constant time). Thus result 1s
n*n/2 *k, that is O(n)
Cxample. Illustrate the operation of Bubble sort on the array A=(5,2,1, 4,3,7,6).
Solution. A=(5,2,1, 4,3,7, 6)
Here length [A]=7
i=1to7 and j=7 to 2 i.e., A[ ]=
i=1 j=7 L521 4 3 6 7

A7]=6 and A|6] =7. So A7]< A(6]

Now exchange (A|71, A|6])
 SORTING ALGORITHMS 77
ANALYSIS OF SIMPLE

Now, i=l, j =6 then A(6]=6

A|5]=3 and A[5]< A(6]
Now, i=l j=5 then A5]=3
A|4]=4 and A(5]< A|4]
So, exchange (A[5], A4)
6
and A(]= 5 2 1 7

Now, i=l,j=4 then A[4]=3

A|3]=1 and A4]> A|3]
Now, i =lj=3then A3]=1
A(2]=2 and A|3]< A2]
So, exchange (A|3), A2|)

then A|]= 5 1 2 3 4 6 7

Now, i =2,j=2 then A2] =1

A1] =5 and A|2]< A[1]
So, exchange (A(2;, A(1)

then A]= 23 467

Now, i-2j=7 then A7]=7
A[6]=6 and A(7]> A[6) No exchange
Similarly, i=2, j =6,5,4. No change
then i=2, j=3
A(3]=2
A(2]=5 and A(3]< A2]
So, exchange (A3], A2))
and AJ= 1 2 5 3 |467
Now, i=3, j =7,6,5 No change
then i =3, j=4
A<4]=3
A(3]= 5 and A[4]< A(3]
So, exchange (A{4), A(3)
then A|]= 11 2 3 5 4 6 7
78 ALGORITHMS DESIGN AND ANALYSIS

Now i=4 j=7,6 No change

Now i= 4, j=5 then A[5]=4
A|4]=5 and A[5]< A|4]
So exchange (A|5], A|4))
and A]= 3 4 5 6 7 is the sorted array.

Example. (a) Let A denote the output of the procedure. Toprove that it is correct, we need toprove that
it terminates and A'(1]sA'2]s ... sA' [n,
What else must be proved to show that it actually sorts ?
loop.
(b) State explicitly a loop invariant for the for loop in lines 2-4, and prove that it holds for the
for
(c) Using the termination condition proved in the last part, state a loop invariant for the
loop in lines 1-4 that will allowyou to prove the property as stated in the above part a.
of the
(d) What is the worst-case running time for bubble sort ? How does it compare to that
insertion sort ?

Solution. (a) We need to show that the elements of A' form a permutation of the elements of
A i.e., they only come from A.

(b) The loop invariant may be stated as follows :

At the start of each iteration of the for loop of lines 2 -4,
A[j]= min {A[k]| jsksn}
the time that the
and the sub array A[j..n] is a permutation of the values that were in A[j.. n] at
loop started.
Now the proof :
Initially,j=, and the sub array A[j..n] consists of single elements A[n, Thus, the loop
invariant trivially holds.
For the maintenance part, consider an iteration for a given value of j =io, by the loop
invariant, A[jo]is the smallest value in A[jo.n Lines 3-4 exchange AjoJand A[jo -1]
if A[jo]is less than A[jo -1) and so A[jo -1]will be the smallest value in Ajo -1..n]
afterward.
Since the only change to the subarray A[jo -1..n] is the possible exchange, and the sub
array Aljo..n] is a permutation of the values that were in A[io ..n] at the time that the
loop started, we see that A[io -1..n] is a permutation of the values that were in
Ao -1..n] at the time that the loop started. Decrementing jto j, -1 for the next
iteration maintains the invariant.
4Finally, the loop terminates when jreaches . By the statement of the loop invariant,
A[i] = min{A[k]|isksn)
and A[i.. n]is a permutation of the values that were in A[i..n] at the time that the loop
started.
 ANALYSIS OF SIMPLE SORTING ALGORITHMS 79

(c) We can have the following loop invariant for the loop in lines 1-4.
At the start of cach iteration of the this for loop, the subarray A[1..i-1]consists of the i-1
smallest values originally in A|1. n) in sorted order, and A[i..n] consists of the n-i+1remaining
values originally in A[1.n]
Now the prof:
Before the first iteration of the loop, i=1. The subarray A[1..i-1] is empty, and so the
loop invariant trivially holds.
Consider an iteration for a given value of i=i,. By the loop invariant, A[1..i, -1]
consists to the i, smallest values in A[1..n) in sorted order.
The above part bshowed that after executing the for loop of lines2-4,A[i, Jis the smal
est value in Ali, -n) and so A[1..i,]is now the i, smallest values originally in A[1..n)
in sorted order. Morever, since the for loop of lines 2-4permutes Ali .n, the subarray
A[i, +1..n] consists of the n-i, remaining values originally in A[1..n) Increment of ito
i, +1 makes the loop invariant holds at the beginning of the next loop.
Finally, the for loop of lines 1-4 terminates when i=n+1 so that i-1=n. By the
statement of the loop invariant, A[1..i-1]is the entire array A[1..n, and it consists of
the original array A[1..n) in sorted order.
() The rurning time depends on the number of iteration of the for loop of lines 2-4. For a
given value of i this loop makes n-iiterations, and ie[1,n]
Thus the total number of iterations is

B(n) =Ei =1" (n-)=n(n-1)

2

4.2 Selection Sort

The idea of selection sort is rather simple : we repeatedly find the next largest (or smallest)
element in the array and move it to its final position in the sorted array. Assume that we wish to
Sort the array in increasing order, i.e., the smallest element at the beginning of the array and the
largest element at the end. We begin by selecting the largest element and moving it to the highest
index position. We can do this by swapping the element at the highest index and the largest
element. We then reduce the effective size of the array by the one element and repeat the process on
he smaller sub array. The process stops when the effective size of the array becomes 1 (an array of
Ielement is already sorted). Thus, the selection sort works by selecting the smallest unsorted item
Temaining in the list, and then swapping it with the item in the next position to be filled. The
selection sort has a complexity of O(n*)
80 ALGORITHMS DESIGN AND ANALYSIS

Selection Sort (A)

1. n+ length [A]
2. for j 1 to n-1
3. smallest +j
for i 4-j+1 to n
5. if A]<A [smallest]
6. then smallest - i
7. exchange (ALJ A[smallest])
Selection sort is very easy to analyzesince none of the loops depends on the data in the
array. Selecting the lowest elements requires scanning all nelements (this takes n-1 comparisons)
and then swapping it into the first position. Finding the next lowest element requires scanning the
remaining n-1 elements and so on, for a total of (n-1) +(n-2) +...+2 +1=0(n*)comparisons. Each
of these scans requires one swap for a total of n-1 swaps (the final element is already in place).
Thus,the comparisons dominate the running time, which is (n')

Example. Sort the following array using selection sort: A]=(5,2,1,4,3).

Solution.
1 2 4 5
A[]= 5 1 4

Here n=5
For j=1 to 4.
j=1 smallest =1
For i =2 to 5
i=2, smnallest = 1
A2] =2 A|1]=5 A|2]< A[1]
then smnallest =2
Now i=3, smallest =2
A|3] =1 A|2]=2 A|3]< A|2]
then smnallest =3
Now i=4 smallest = 3
A|4] =4
A(3]=1 A[4]> A3) No change
Now i=5, smallest = 3
A|5] =3
A(3]=1 A(5] >A(31 So,No change
 ANALYSIS OF SIMPLE SORTING ALGORITHMS 81

then
exchange (A|1], A{smallest)
exchange (5, 1)
ie.,
Now
1
A]= 5 4 3

Now j=2,, smallest =2

For i =3 to 5
Now i=3, smallest =2
A(3] =5
A|2]=2 A(3] > A(2] No change
Now i=4,smallest =2. No change
i=5, smallest =2. No change
Now j=3, smallest -3
For i =4 to 5
i=4 smallest -3
A|4] =4
A|3]=5 A[4]< A3] then smallest =4
Now i=5 smallest =4
A|5]=3
A[4]=4 A[5]< A|4] then smallest =5
Now exchange [A|3], A5]]
then
A[]= 2 3 4 5

Now j=4, smallest =4, i=5

A[5] =5
A|4]=4 A(5]> A[4) No change
Hence sorted array is
2 3 4 5

Lample. Write a pseudocode jor the Selection sort algorith1m. What loop invariant does it maintain ?Why
does it need to run for only the first n-1l elements, rather than for all nelements ? Give the best
and worst-case analysis of this sort.
82 ALGORITHMS DEIGN AND ANALYSIS
Soution. The seudonde tor seletion aort in
Selection sort (A)
1. length (A)
for 1to n-1
smallest e)
for )+1to n
if A]<A [smallest)
then smallest - i
exchange (Ai], A[smallest])

each iteration of the for loop

The algorithm maintains the loop invariant that at the start of elements in the array
hetween ìne 2 and line 7, the sub array A[1..j -1] consists of the j-1smallest
Al..n] and thìs sub array is in sorted order.
invariant, the sub array
Morever, after the first n-1elements, according to the previous
element A[n] must be the largest
A[1..n-1] contains the smallest n-l elements, sorted. Hence,
element
that, for allcases, we have to do
Regarding the running time of the algorithm, it is easy to see
the following amount of work :
17

S(n) ) -Xj-1-"(n-j) =n(n-1)-j-1 (n-1)

j=l i=j+1

=n(n-1)-n(n-1)=n(n-1)=e(1')
2
2
bubble sort, but the insertion sort is over
Ityields a60% performance improvement over the
implement as the selection sort. In short, there
twice as fast as the bubble sort and is just as easy to
use the insertion sort instead.
really isn't any reason to use the selection sort -

4.3 Insertion Sort

inserts each item into itsproper plae

The insertionsort works just like its name suggests - it
the final list. The simplest implementation of this requires two list structures - the source list and
in memory, most implementations use an
the list into which sorted items are inserted. To save
the already sorted items and repeatedly
in-place sort that works by moving the current item pass
swapping it with the preceding item until it is in place.
O(n')Although it has the sane
Like the bubble sort, the insertion sort has acomplexity ofas the bubble sort. It has various
cornpiexity, the insertion sort is alittle over twice as efficient
advantages :
4 It is simple to implement
4It is efficient on small data sets