Sort Algorithms

Efficiency and Analysis

Analysis of Simple Sorting Algorithms

Selection sort
Bubble sort

2
Analysis of selection sort
Assume n element list
Makes n-1 passes through the list
Each pass has a sequential search of
some portion of the n elements
Analysis: n * n = O(n2)

3
 The maxSelect function
int maxSelect(int a[], int n)
{
int maxPos(0), currentPos(1);
while (currentPos < n) {
// Invariant: a[maxPos] >= a[0] ... a[currentPos-1]
if (a[currentPos] > a[maxPos])
maxPos = currentPos;
currentPos++;
}
return maxPos;
}
4
The operation of Selection Sort

all items in here less than shaded items sorted items

a
last n-1

5
 Selection Sort
void selectionSort(int a[], int n)
{
int last(n-1);
int maxPos;
while (last > 0) {
// invariant: a[last+1] ... a[n-1] is sorted &&
// everything in a[0] ... a[last] <= everything in a[last+1] ... a[n-1]
maxPos = maxSelect(a, last+1); // last+1 is length from 0 to last
swapElements(a, maxPos, last);
last--;
}
}
6
 Selection Sort example

A[0] 2 0 maxPos 5 n
A[1] 7
A[2] 3 1 currPos
A[3] 5
A[4] 6 4 last

7
 Selection Sort example

A[0] 2 1 maxPos 5 n
A[1] 7
A[2] 3 1 currPos
A[3] 5
A[4] 6 4 last

8
 Selection Sort example

A[0] 2 1 maxPos 5 n
A[1] 7
A[2] 3 2 currPos
A[3] 5
A[4] 6 4 last

9
 Selection Sort example

A[0] 2 1 maxPos 5 n
A[1] 7
A[2] 3 3 currPos
A[3] 5
A[4] 6 4 last

10
 Selection Sort example

A[0] 2 1 maxPos 5 n
A[1] 7
A[2] 3 4 currPos
A[3] 5
A[4] 6 4 last

11
 Swap, after one pass

A[0] 2 1 maxPos 5 n
A[1] 6
A[2] 3 4 currPos
A[3] 5
A[4] 7 4 last

12
 Selection Sort

A[0] 2 0 maxPos 5 n
A[1] 6
A[2] 3 1 currPos
A[3] 5
A[4] 7 3 last

13
 Selection Sort

A[0] 2 1 maxPos 5 n
A[1] 6
A[2] 3 1 currPos
A[3] 5
A[4] 7 3 last

14
 Selection Sort

A[0] 2 1 maxPos 5 n
A[1] 6
A[2] 3 2 currPos
A[3] 5
A[4] 7 3 last

15
 Selection Sort

A[0] 2 1 maxPos 5 n
A[1] 6
A[2] 3 3 currPos
A[3] 5
A[4] 7 3 last

16
 Selection Sort

A[0] 2 1 maxPos 5 n
A[1] 5
A[2] 3 3 currPos
A[3] 6
A[4] 7 3 last

17
 Selection Sort

A[0] 2 0 maxPos 5 n
A[1] 5
A[2] 3 1 currPos
A[3] 6
A[4] 7 2 last

18
 Selection Sort

A[0] 2 1 maxPos 5 n
A[1] 5
A[2] 3 1 currPos
A[3] 6
A[4] 7 2 last

19
 Selection Sort

A[0] 2 1 maxPos 5 n
A[1] 5
A[2] 3 2 currPos
A[3] 6
A[4] 7 2 last

20
 Selection Sort

A[0] 2 1 maxPos 5 n
A[1] 3
A[2] 5 2 currPos
A[3] 6
A[4] 7 2 last

21
 Selection Sort

A[0] 2 0 maxPos 5 n
A[1] 3
A[2] 5 1 currPos
A[3] 6
A[4] 7 1 last

22
 Selection Sort

A[0] 2 1 maxPos 5 n
A[1] 3
A[2] 5 1 currPos
A[3] 6
A[4] 7 1 last

23
 Selection Sort

A[0] 2 0 maxPos 5 n
A[1] 3
A[2] 5 1 currPos
A[3] 6
A[4] 7 0 last

24
 Result after each pass

A[0] 2 2
A[1] 7 6
A[2] 3 3
A[3] 5 5
A[4] 6 7

25
 Result after each pass

A[0] 2 2 2
A[1] 7 6 5
A[2] 3 3 3
A[3] 5 5 6
A[4] 6 7 7

26
 Result after each pass

A[0] 2 2 2
A[1] 7 6 5
A[2] 3 3 3
A[3] 5 5 6
A[4] 6 7 7

27
 Result after each pass

A[0] 2 2 2 2
A[1] 7 6 5 3
A[2] 3 3 3 5
A[3] 5 5 6 6
A[4] 6 7 7 7

28
 Result after each pass

A[0] 2 2 2 2
A[1] 7 6 5 3
A[2] 3 3 3 5
A[3] 5 5 6 6
A[4] 6 7 7 7

29
 Result after each pass

A[0] 2 2 2 2 2
A[1] 7 6 5 3 3
A[2] 3 3 3 5 5
A[3] 5 5 6 6 6
A[4] 6 7 7 7 7

30
 Result after each pass

A[0] 2 2 2 2 2
A[1] 7 6 5 3 3
A[2] 3 3 3 5 5
A[3] 5 5 6 6 6
A[4] 6 7 7 7 7

31
 Result after each pass

A[0] 2 2 2 2 2 2
A[1] 7 6 5 3 3 3
A[2] 3 3 3 5 5 5
A[3] 5 5 6 6 6 6
A[4] 6 7 7 7 7 7

32
Analysis
Number of passes: n-1
the first pass guaranteed to place 1 item
the second guarantees a second
the third a third, etc.
After n-1 passes we have them all in place

33
Analysis: comparisons
In the first pass we compared n-1 pairs
In the second, n-2
In the third, n-3, etc.
Actual number of comparisons made
across all passes, for this example was:
4+3+2+1 = 10

34
Order of complexity
The number of passes is O(n)
The number of comparisons in each pass
is also O(n)
Therefore, the order of complexity of the
sort is O(n*n) = O(n2)

35
Analysis of selection sort (cont)
What if the list is sorted to begin with?
This selection sort is a mindless one
it would not know that it should stop.
Best case and worst case are the same
(except for the swaps)
The best, worst and average cases are all
quadratic algorithms with O(n2)

36
Bubble Sort
Also n-1 passes
Also n-pass comparisons in each pass
Order of complexity is therefore n-squared
The same as the selection sort
The main difference is that swapping may
occur as many as n-1 times on a single
pass, with the selection sort it only occurs
once, at the end of the pass.

37
Example of one phase of
Bubble Sort
compare These items are sorted

17 9 21 6 3 32 37 41 45
compare

9 17 21 6 3 32 37 41 45
compare

9 17 21 6 3 32 37 41 45
compare

9 17 6 21 3 32 37 41 45

9 17 6 3 21 32 37 41 45 38
 bubbleSortPhase
// void swapElements(int a[], int maxPos, int last);

void bubbleSortPhase(int a[], int last)

{
// Precondition: a is an array indexed from a[0] to a[last]
// Move the largest element between a[0] and a[last] into a[last],
// by swapping out of order pairs
int pos;

39
 bubbleSortPhase

for (pos = 0; pos < last - 1; pos++)

if (a[pos] > a[pos+1]) {
swapElements(a, pos, pos+1);
}
// Postconditions: a[0] ... a[last]
// contain the same elements,
// possibly reordered; a[last] >= a[0] ... a[last-1]
}

40
 Bubble Sort

void bubbleSortPhase(int a[], int last);

void bubbleSort(int a[], int n)
{ // Precondition: a is an array indexed from a[0] to a[n-1]
int i;
for (i = n - 1; i > 0; i--)
bubbleSortPhase(a, i);
// Postcondition: a is sorted
}

41
Equation 5-7

42
Version 1 of bubble sort
This version uses n-1 passes
During each pass, n-1 pairs are compared
Every time a pair needs to be swapped
this is done before the pass can continue

43
 Bubble Sort: difficult example

A[0] 7 0 Pos 5 n
A[1] 6
Pos+1
A[2] 5
A[3] 3
A[4] 2 4 last

44
 Bubble Sort: difficult example

A[0] 6 0 Pos 5 n
A[1] 7
Pos+1
A[2] 5
A[3] 3
A[4] 2 4 last

45
 Bubble Sort: difficult example

A[0] 6 0 Pos 5 n
A[1] 5
Pos+1
A[2] 7
A[3] 3
A[4] 2 4 last

46
 Bubble Sort: difficult example

A[0] 6 0 Pos 5 n
A[1] 5
Pos+1
A[2] 3
A[3] 7
A[4] 2 4 last

47
 Bubble Sort: difficult example

A[0] 6 0 Pos 5 n
A[1] 5
Pos+1
A[2] 3
A[3] 2
A[4] 7 4 last

48
 Bubble Sort: difficult example

A[0] 6 0 Pos 5 n
A[1] 5
Pos+1
A[2] 3
A[3] 2
A[4] 7 4 last

49
 Bubble Sort: difficult example

A[0] 5 1 Pos 5 n
A[1] 6
Pos+1
A[2] 3
A[3] 2
A[4] 7 4 last

50
 Bubble Sort: difficult example

A[0] 5 2 Pos 5 n
A[1] 3
Pos+1
A[2] 6
A[3] 2
A[4] 7 4 last

51
 Bubble Sort: difficult example

A[0] 5 3 Pos 5 n
A[1] 3
Pos+1
A[2] 2
A[3] 6
A[4] 7 4 last

52
 Bubble Sort: difficult example

A[0] 5 0 Pos 5 n
A[1] 3
Pos+1
A[2] 2
A[3] 6
A[4] 7 4 last

53
 Bubble Sort: difficult example

A[0] 3 1 Pos 5 n
A[1] 5
Pos+1
A[2] 2
A[3] 6
A[4] 7 4 last

54
 Bubble Sort: difficult example

A[0] 3 1 Pos 5 n
A[1] 2
Pos+1
A[2] 5
A[3] 6
A[4] 7 4 last

55
 Bubble Sort: difficult example

A[0] 3 0 Pos 5 n
A[1] 2
Pos+1
A[2] 5
A[3] 6
A[4] 7 4 last

56
 Bubble Sort: difficult example

A[0] 2 0 Pos 5 n
A[1] 3
Pos+1
A[2] 5
A[3] 6
A[4] 7 4 last

57
 Result after each pass

A[0] 7 6 5 3 2 2
A[1] 6 5 3 2 3 3
A[2] 5 3 2 5 5 5
A[3] 3 2 6 6 6 6
A[4] 2 7 7 7 7 7

58
Analysis
For this example, we made n-1 passes
through the array
Each time we looked at n-1 pairs
(we should have only looked at n-pass
pairs. Why?)
Either way, the number of passes is O(n)
The number of pairs processed in each
pass is O(n)
So, overall we get O(n*n) = O(n2)
59
Selection and Bubblesort
comparison
Both are O(n2) sorts
If we count up the number of critical
operations for both sorts, handling n-1, n-
2, etc. pairs for each pass, using the data
in the last example, we get
Selection sort: 10 comparisons
Bubble sort: 10 comparisons

60
What about swaps?
The same swap function can be used for
both programs.
Lets say it takes 3 operations, then the
actual number of operations is
Selection sort: 10 + 3(n-1) = 22
Bubble sort: 10 + 3(10) = 40
The selection sort uses roughly half the
number of operations as the bubble sort

61
Moral
Two sorts of the same order O(n2) may not
be the same speed.
It depends on the data sets they are
sorting.
It also depends on the way they are
implemented.

62
Data sets
The data set we chose for the bubble sort
was the worst one possible.
What happens if we run it on the data set
we originally used for the selection sort?

63
 Result after pass 1

A[0] 2 2
A[1] 7 3
A[2] 3 5
A[3] 5 6
A[4] 6 7

64
 Result after pass 2

A[0] 2 2 2
A[1] 7 3 3
A[2] 3 5 5
A[3] 5 6 6
A[4] 6 7 7

65
Improvement needed
Neither the selection sort, nor the bubble
sort know enough to stop if the list is
sorted early!
We should be able to come up with a
smart version of the bubble sort that can
do this.
Instead of the outer for loop, lets use a
while loop that runs until the array is
sorted.

66
Further improvements
We will also make sure the inner loop runs
the minimum number of times (n-pass)
and that we keep track of whether a swap
was needed for the pass we are on.
If a swap was needed then we cannot
assume the array is sorted.
If a swap was not needed, then the array is
sorted and we should send that signal to
the outer loop.
67
Bubble sort improvements
What improvements can you suggest for
the bubble sort that we have seen?

68
Improvements
Do not look at portions of the array that
are already in sorted order
Leave when the array is sorted
this is something the insertion and
selection sorts we have seen could not do.

69
Improvements
How much of an operational difference
does the smart bubble sort have over its
dumb form?
How much of an operational difference
does the smart bubble sort have over the
selection sort?
Do these improvements make a difference
in Big-O?
Which form of sort is best?
70
 Bubble Sort
void bubbleSort(int a[], int n)
{ // Precondition: a is an array indexed from a[0] to a[n-1]
bool sorted(false); int pass(0);
while (!sorted) {
sorted = true;
for (pos = 0; pos < last - 1; pos++)
if (a[pos] > a[pos+1])
{ swapElements(pos, pos+1); sorted = false; }
pos++;
} // Postcondition: a is sorted
}
71
 Result after pass 1

A[0] 2 2
A[1] 7 3
A[2] 3 5
A[3] 5 6
A[4] 6 7

72
 Result after pass 2

A[0] 2 2 2
A[1] 7 3 3
A[2] 3 5 5
A[3] 5 6 6
A[4] 6 7 7

73
Analysis
Only two passes are made
4 pairs are checked on pass 1
swap is called 3 times on pass 1
3 pairs are checked on pass 2, no swaps
Total operations:
4*3+3 = 15
Better than the selection sort.
The savings would be even bigger for
larger lists that were sorted early.
74
Bubble sort: final analysis
Not counting swapping, the best case is
only n-1 operations!
Worst case is still n-squared
Order of complexity same as selection
sort no matter what.

75