Sorting and Searching

Desired learning outcome

To study several sorting and searching algorithms
To appreciate that algorithms for the same task can
differ widely in performance
To understand the big-Oh notation
To estimate and compare the performance of
algorithms
Selection Sort
A sorting algorithm rearranges the elements of a
collection so that they are stored in sorted order.
Selection sort sorts an array by repeatedly finding
the smallest element of the unsorted tail region and
moving it to the front.
Slow when run on large data sets.
Example: sorting an array of integers

11 9 17 5 12
Sorting an Array of Integers
1. Find the smallest and swap it with the first element
5 9 17 11 12

2. Find the next smallest. It is already in the correct place

5 9 17 11 12

3. Find the next smallest and swap it with first element of

unsorted portion

5 9 11 17 12
4. Repeat
5 9 11 12 17

5. When the unsorted portion is of length 1, we are done

5 9 11 12 17
Selection Sort

In selection sort, pick the smallest element

and swap it with the first one. Pick the
smallest element of the remaining ones
and swap it with the next one, and so on.
s ectio
n_1/ Se
lection
Sorter.
java
section_1/SelectionSortDemo.java
section_1/
ArrayUtil.j
ava
Selection Sort on Various Size
Arrays
n Milliseconds

10,000 786

20,000 2,148

30,000 4,796

40,000 9,192

50,000 13,321
Figure 1 Time Taken by Selection Sort
60,000 19,299

Doubling the size of the array more than doubles the time
needed to sort it.
The curve resembles a parabola.
Analyzing the Performance of
the Selection Sort Algorithm
In an array of size n, count how many times an array
element is visited:
To find the smallest, visit n elements + 2 visits for the swap
To find the next smallest, visit (n - 1) elements + 2 visits for
the swap
The last term is 2 elements visited to find the smallest + 2
visits for the swap
Analyzing the Performance of
the Selection Sort Algorithm
The number of visits:
n + 2 + (n - 1) + 2 + (n - 2) + 2 + . . .+ 2 + 2
This can be simplified to n2 /2 + 5n/2 - 3
5n/2 - 3 is small compared to n2 /2 when n gets large – so let's ignore it
Also ignore the 1/2 – it cancels out when comparing ratios
Analyzing the Performance of
the Selection Sort Algorithm

The number of visits is of the order n2.

Computer scientists use the big-Oh notation to
describe the growth rate of a function.

Using big-Oh notation: The number of visits is O(n2).

Multiplying the number of elements in an array by 2
multiplies the processing time by 4.
To convert to big-Oh notation: locate fastest-growing
term, and ignore constant coefficient.
Common Big-Oh Growth Rates
Insertion Sort
§ Assume initial sequence a[0] . . . a[k] is
sorted (k = 0):

11 9 16 5 7

§ Add a[1]; element needs to be inserted before 11

9 11 16 5 7

§ Add a[2]

9 11 16 5 7

§ Add a[3]

5 9 11 16 7

§ Finally, add a[4]

5 7 9 11 16
Insertion Sort
public class InsertionSorter
{
/**
Sorts an array, using insertion sort.
@param a the array to sort
*/
public static void sort(int[] a)
{
for (int i = 1; i < a.length; i++)
{
int next = a[i];
// Move all larger elements up
int j = i;
while (j > 0 && a[j - 1] > next)
{
a[j] = a[j - 1];
j--;
}
// Insert the element
a[j] = next;
}
}
}
Insertion Sort
Insertion sort is the method that many people use to
sort playing cards. Pick up one card at a time and
insert it so that the cards stay sorted.

Insertion sort is an O(n2) algorithm.

 Merge Sort

§ Sorts an array by
§ Cutting the array in half
§ Recursively sorting each half
§ Merging the sorted halves

§ Dramatically faster than the

selection sort
§ In merge sort, one sorts each half,
then merges the sorted halves.
Merge Sort Example
Divide an array in half and sort each half

Merge the two sorted arrays into a single sorted

array
Merge Sort

public static void sort(int[] a)

{
if (a.length <= 1) { return; }
int[] first = new int[a.length / 2];
int[] second = new int[a.length - first.length];
// Copy the first half of a into first, the second half into second
. . .
sort(first);
sort(second);
merge(first, second, a);
}
section_4/MergeSorter.java
Analyzing the Merge Sort Algorithm

In an array of size n, count how many times an array

element is visited.

Assume n is a power of 2: n = 2m.

Calculate the number of visits to create the two sub-
arrays and then merge the two sorted arrays:
3 visits to merge each element or 3n visits
2n visits to create the two sub-arrays
total of 5n visits
Analyzing the Merge Sort Algorithm

Let T(n) denote the number of visits to sort an array

of n elements then
T(n) = T(n / 2) + T(n / 2) + 5n or
T(n) = 2T(n / 2) + 5n

The visits for an array of size n / 2 is: T(n / 2) = 2T(n

/ 4) + 5 n / 2
So T(n) = 2 × 2T( n /4) +5n + 5n

The visits for an array of size n / 4 is: T(n / 4) = 2T(n

/ 8) + 5 n / 4
So T(n) = 2 × 2 × 2T(n / 8) + 5n + 5n + 5n
Analyzing Merge Sort Algorithm

Repeating the process k times: T(n) = 2 kT( n / 2k)

+5nk
Since n = 2m, when k = m: T(n) = 2mT(n / 2m) +5nm
T(n) = nT(1) +5nm
T(n) = n + 5nlog2(n)
Analyzing Merge Sort Algorithm

To establish growth order:

Drop the lower-order term n
Drop the constant factor 5
Drop the base of the logarithm since all logarithms are
related by a constant factor
We are left with n log(n)

Using big-Oh notation: number of visits is O(n

log(n)).
Merge Sort V s Selection Sort

Selection sort is an O(n2) algorithm.

Merge sort is an O(n log(n)) algorithm.
The n log(n) function grows much more slowly than
n2.
Merge Sort Timing vs.
Selection Sort

Merge Sort Selection Sort

n
(milliseconds) (milliseconds)
10,000 40 786

20,000 73 2,148

30,000 134 4,796

40,000 170 9,192

50,000 192 13,321

Figure 2 Time Taken by Merge vs.

Selection Sort 60,000 205 19,299

Searching
Linear search: also called sequential search
Examines all values in an array until it finds a match
or reaches the end
Number of visits for a linear search of an array of n
elements:
The average search visits n/2 elements
The maximum visits is n

A linear search locates a value in an array in O(n)

steps
section_6_1/LinearSearcher.java
Binary Search
A binary search locates a value in a sorted array by:
Determining whether the value occurs in the first or second half
Then repeating the search in one of the halves

The size of the search is cut in half with each step.

Binary Search
§ Searching for 15 in this array

§ The last value in the first half is 9

So look in the second (darker colored) half

§ The middle element of this sequence is 20, so the value must be

in this sequence
Look in the darker colored sequence

§ The last value of the first half of this very short sequence is
12,
This is smaller than the value that we are searching, so we
must look in the second half

§ 15 ≠ 17: we don't have a match

section_6_2/BinarySearcher.java
1 /**
2 A class for executing binary searches in an array.
3 */
4 public class BinarySearcher
5 {
6 /**
7 Finds a value in a range of a sorted array, using the binary
8 search algorithm.
9 @param a the array in which to search
Binary Search
Count the number of visits to search a sorted array
of size n
We visit one element (the middle element) then search
either the left or right subarray
Thus: T(n) = T(n/2) + 1

If n is n / 2, then T(n / 2) = T(n / 4) + 1

Substituting into the original equation: T(n) = T(n / 4)
+2
This generalizes to: T(n) = T(n / 2k) + k
Binary Search

Assume n is a power of 2, n = 2m
where m = log2(n)

Then: T(n) = 1 + log2(n)

A binary search locates a value in a sorted array in
O(log(n)) steps.
Binary Search
Should we sort an array before searching?
Linear search - O(n)
Binary search - O(n log(n))

If you search the array only once

Linear search is more efficient

If you will make many searches

Worthwhile to sort and use binary search
Sorting and Searching in the
Java Library - Sorting
You do not need to write sorting and searching
algorithms.
Use methods in the Arrays and Collections classes.

The Arrays class contains static sort methods.

To sort an array of integers:
int[] a = . . . ;
Arrays.sort(a);

That sort method uses the Quicksort algorithm (see

Special Topic 14.3 in the textbook if interested).

To sort an ArrayList, use Collections.sort

ArrayList<String> names = . . .;
Collections.sort(names);

Uses merge sort algorithm

Sorting and Searching in the
Java Library - Binary Search
Arrays and Collections classes contain static
binarySearch methods.
If the element is not found, returns -k - 1
Where k is the position before which the element should be
inserted

For example
int[] a = { 1, 4, 9 };
int v = 7;
int pos = Arrays.binarySearch(a, v);
// Returns –3; v should be inserted before position 2
Comparing Objects
Arrays.sort sorts objects of classes that
implement Comparable interface:
public interface Comparable<T>
{
int compareTo(T other);
}

The call a.compareTo(b) returns

A negative number if a should come before b
0 if a and b are the same
A positive number otherwise
Comparing Objects
Several classes in Java (e.g. String and Date)
implement Comparable.
You can implement Comparable interface for your
own classes.
The Country class could implement Comparable:
public class Country implements Comparable<Country>
{
public int compareTo(Country other)
{
return Double.compare(area, other.area);
}
}

You could pass an array of countries to

Arrays.sort
Country[] countries = new Country[n];
// Add countries
Arrays.sort(countries); // Sorts by increasing area