Lecture:

Sorting – Searching - Big O

*I’d like to acknowledge Dr. Philip Heller and Dr. Chakarov from the Computer Science Department for sharing their resources for the
course. The majority of the material we use this semester will be based their work.
Outline

• Big O
• Selection sort
• Insertion sort
– Big O
• Merge sort
• Binary Search
– Big O
Big O
Algorithms need to be correct and efficient

• Just correct ➔ not good enough

• You want to still be alive when
execution terminates
 Complexity Analysis
• What I want to know:
– How much time to sort n ints using Selection Sort on my
computer?
• Why I can’t know that:
– Every computer is different: different OS, different Java
compiler, different JVM, different load.
– Different implementations of algorithms are fast or slow
 Complexity Analysis
• What I can know:
– How will increasing the input size affect execution time?
– This mathematical function is called the time complexity of
the algorithm.
– Usually expressed as big-O
• The complexity of SelectionSort is O(n2)
– Complexity is an inherent principle of an algorithm,
independent of implementation, OS, hardware
 O notation
• Big O notation is a mathematical way of describing how a function
(running time of an algorithm) generally behaves in relation to the input
size.
• Given a function that describes the running time of an algorithm, the Big
O notation for that function can be determined using the following rules:
1. If f(x) is a sum of several terms, the highest order term (the one with the fastest
growth rate) is kept and others are discarded.
2. If f(x) has a term that is a product of several factors, all constants (those that are
not in terms of x) are omitted.
• To analyze how runtime of an algorithm scales as the input size increases:
• we first determine how many operations the algorithm executes for a specific input size, N.
– Then, the big-O notation for that function is determined.
– Algorithm runtime analysis often focuses on the worst-case runtime complexity.
 O notation
• The worst-case runtime of an algorithm is the runtime complexity for an
input that results in the longest execution.
• Other runtime analyses include best-case runtime and average-case
runtime. Determining the average-case runtime requires knowledge of
the statistical properties of the expected data inputs.
• For algorithm analysis, the definition of a single operation does not need
to be precise. An operation can be any statement (or constant number of
statements) that has a constant runtime complexity, O(1). Since constants
are omitted in big-O notation, any constant number of constant time
operations is O(1). So, precisely counting the number of constant time
operations in a finite sequence is not needed.
 Big-Oh practice

• Compute T(n) in units of operations

• Drop everything except the fastest growing term, then
drop that term’s coefficient

12n + 1000n + 65
3 2

➔ O(n ) 3
 More on complexity
• Important in algorithms that process all data in a data
structure (array, ArrayList, HashSet, TreeSet, …)
• Look for loops that process each member of the data
structure

for (Movie m: theArchive) {

someRoughlyConstantTimeMethod(m);
}
Actually, ignore the data processing and just look at the
loop

for (int i=0; i<n; i++) {

someRoughlyConstantTimeMethod(whatever);
}
T(n) = some constant * n
Loop = O(n)

for (Thing t: anyCollection) {

someRoughlyConstantTimeMethod(whatever);
}
T(n) = some constant * n
Loop = O(n)
 Nested loops
for (int i=0; i<n; i++)
for (int j=0; j<m; j++)
someRoughlyConstantTimeMethod(whatever);

Executes n*m times

T = const * n * m
Loop = O(mn)
Sorting
 java.util.Arrays sort() method

• “Sorts the array into ascending numerical order”

• “Dual-Pivot Quicksort” ????
• “O(n log(n)) performance”
• ????
“Typically faster than traditional (one-pivot) Quicksort implementations”

????
 Sorting Algorithms
• The easy / obvious algorithms are slow
– Selection Sort: many visits, few moves
– Insertion Sort: many moves, few visits
• The smart algorithms are fast and recursive
– Merge Sort
– Quick Sort (you’ll see this in 146)
Selection Sort
 Selection sort
• Selection sort is a sorting algorithm that treats the input as two parts, a sorted part and an unsorted part, and repeatedly
selects the proper next value to move from the unsorted part to the end of the sorted part.
• The index variable i denotes the dividing point. Elements to the left of i are sorted, and elements including and to the
right of i are unsorted. All elements in the unsorted part are searched to find the index of the element with the smallest
value. The variable indexSmallest stores the index of the smallest element in the unsorted part. Once the element with
the smallest value is found, that element is swapped with the element at location i. Then, the index i is advanced one
place to the right, and the process repeats.
• The term "selection" comes from the fact that for each iteration of the outer loop, a value is selected for position i.
• Selection sort has the advantage of being easy to code, involving one loop nested within another loop
• Selection sort may require a large number of comparisons. The selection sort algorithm runtime is O().
 Selection Sort
• Sorts array members
• We’ll look at sorting ints
• Could be any numeric primitive type, or any Comparable
class type
• The big idea: swap array members until the array is
sorted
– Find smallest member, put it in a[0]
– Find next smallest member, put it in a[1]
– etc.
 A simple example

Find the Smallest 7 10 4 1

 A simple example

Find the Smallest 7 10 4 1

Swap

1 10 4 7
 A simple example

7 10 4 1

Find the Smallest 1 10 4 7

Swap

1 4 10 7
 A simple example

7 10 4 1

1 10 4 7

Find the Smallest 1 4 10 7

Swap

1 4 7 10
 A simple example

7 10 4 1

1 10 4 7

1 4 10 7

Find the Smallest 1 4 7 10

A simple example: done

1 4 7 10
 Pseudocode

• Almost looks like code, but missing explicit syntax pieces

• Should not be vague (e.g., given an input, anyone tracing your code
should produce the same output)
• Should not be too wordy (this isn’t a story about your code)
• Can and should be refined over time as you build up your solution.
• Guidelines are availabe on Canvas.
 Pseudocode
Selection Sort

selectionSort(arr)

for startOfUnsorted in 0 to arr.length

// Find index of smallest member of unsorted region.

smallestInUnsorted = max value
indexOfSmallest = -1
for i in startOfUnsorted to arr.length
if arr[i] < smallestInUnsorted
smallestInUnsorted = arr[i]
indexOfSmallest = i

swap arr[indexOfSmallest] and arr[startOfUnsorted]

public class SelectionSorter {
private int[] a;

public SelectionSorter(int[] a) {
this.a = a;
}

// Sorts a
public void sortInPlace() { . . . }
}
 The Actual Sorting Code
public void sortInPlace(int[] a) {
for (int startOfUnsorted=0;
startOfUnsorted<a.length;
startOfUnsorted++) {
// Find index of smallest member of unsorted region.
int smallestInUnsorted = Integer.MAX_VALUE;
int indexOfSmallest = -1;
for (int i=startOfUnsorted; i<a.length; i++) {
if (a[i] < smallestInUnsorted) {
smallestInUnsorted = a[i];
indexOfSmallest = i;
}
}

// Swap smallest/first members of unsorted region.

int temp = a[indexOfSmallest];
a[indexOfSmallest]= a[startOfUnsorted];
a[startOfUnsorted] = temp;
}
 The Actual Sorting Code
public void sortInPlace(int[] a) {
for (int startOfUnsorted=0; The part of the Array we
startOfUnsorted<a.length; haven’t sorted yet
startOfUnsorted++) {

// Find index of smallest member of unsorted region.

int smallestInUnsorted = Integer.MAX_VALUE;
int indexOfSmallest = -1;
for (int i=startOfUnsorted; i<a.length; i++) {
if (a[i] < smallestInUnsorted) {
smallestInUnsorted = a[i];
indexOfSmallest = i;
}
}

// Swap smallest/first members of unsorted region.

int temp = a[indexOfSmallest];
a[indexOfSmallest]= a[startOfUnsorted];
a[startOfUnsorted] = temp;
 The Actual Sorting Code
public void sortInPlace(int[] a) {
for (int startOfUnsorted=0;
startOfUnsorted<a.length;
startOfUnsorted++) {
This part finds the minimum in
the unsorted part
// Find index of smallest member of unsorted region.
int smallestInUnsorted = Integer.MAX_VALUE;
int indexOfSmallest = -1;
for (int i=startOfUnsorted; i<a.length; i++) {
if (a[i] < smallestInUnsorted) {
smallestInUnsorted = a[i];
indexOfSmallest = i;
}
}

// Swap smallest/first members of unsorted region.

int temp = a[indexOfSmallest];
a[indexOfSmallest]= a[startOfUnsorted];
a[startOfUnsorted] = temp;
 The Actual Sorting Code
public void sortInPlace(int[] a) {
for (int startOfUnsorted=0;
startOfUnsorted<a.length;
startOfUnsorted++) {

// Find index of smallest member of unsorted region.

int smallestInUnsorted = Integer.MAX_VALUE;
int indexOfSmallest = -1;
for (int i=startOfUnsorted; i<a.length; i++) {
if (a[i] < smallestInUnsorted) {
smallestInUnsorted = a[i];
indexOfSmallest = i;
} Swap the minimum element with the first element in the
} unsorteded part of the Array

// Swap smallest/first members of unsorted region.

int temp = a[indexOfSmallest];
a[indexOfSmallest]= a[startOfUnsorted];
a[startOfUnsorted] = temp;
 Complexity example: Selection Sort

• Unit of computation is the Array Visit

– 1 read from, or 1 write to, an array member
• For input array a[] with length n, how
many visits are required to sort a[]?
 startOfUnsorted = 0:

7 # of visits = 4
Ar
ey
10 ou
Ar

sm sm
ey
Ar
al al
ou
4 ey
les les
ou
t? t?
sm
1
Ar

al
ey

les
ou

t?
sm
al
les
t?
 startOfUnsorted = 0:

7 # of visits = 4
Ar
ey
10 ou
Ar

sm sm
ey
Ar
4 al al
ou
ey
les les
ou
t? t?
sm
1
Ar

al
ey

les
ou

t?
sm
al
les
t?
 startOfUnsorted = 0:

1 # of visits = 4
10 +4
SW
4 AP
7
 startOfUnsorted = 1

1 # of visits = 4
10 +4
4
7
 startOfUnsorted = 1

1 # of visits = 4
10 +4
Ar

+3
ey
Ar
4
ou
ey
sm
ou
al
7 sm
les
Ar

al
ey

t?
les
ou

t?
sm
al
les
t?
 startOfUnsorted = 1

1 # of visits = 4
10 +4
Ar

+3
ey
Ar
4
ou
ey
sm
ou
al
7 sm
les
Ar

al
ey

t?
les
ou

t?
sm
al
les
t?
 startOfUnsorted = 1

1 # of visits = 5
4 +4
SW +3
10 AP
+4
7
 startOfUnsorted = 2

1 # of visits = 4
4 +4
+3
10
+4
7
 startOfUnsorted = 2

1 # of visits = 4
4 +4
Ar +3
10 ey
ou +4
7 sm
+2
Ar

al
ey

les
ou

t?
sm
al
les
t?
 startOfUnsorted = 2

1 # of visits = 4
4 +4
Ar +3
10 ey
ou +4
7 sm
+2
Ar

al
ey

les
ou

t?
sm
al
les
t?
 startOfUnsorted = 2

1 # of visits = 4
4 +4
+3
10
+4
SW
AP
7 +2
+4
 startOfUnsorted = 2

5 # of visits = 4
9 +4
+3
7
+4
SW
AP
10 +2
+4
 startOfUnsorted = 3

1 # of visits = 4
4 +4
+3
7
+4
10 +2
+4
 startOfUnsorted = 3

1 # of visits = 4
4 +4
+3
7
+4
10 +2
Ar
ey
ou

+4
sm
al

+1
les
t?
 startOfUnsorted = 3

1 # of visits = 4
4 +4
+3
7
+4
10 +2
+4
+1
 startOfUnsorted = 3

1 # of visits = 4
4 +4
+3
7
+4
10 +2
SW
AP

(Algorithm doesn’t care that it’s swapping +4

10 with itself)
+1
+4
 DONE

1 # of visits = 4
4 +4
+3
7
+4
10 +2
+4
+1
+4
 # of visits =
4 + 4 + 3 + 4 + 2+ 4 + 1 + 4

Find smallest
 # of visits =
4+4+3+4+2+4+1+4

Find smallest Swap

 # of visits =
4+4+3+4+2+4+1+4

Find smallest Swap

 # of visits =
4+4+3+4+2+4+1+4
=
4+3+2+1 + 4+4+4+4
 # of visits =
4+4+3+4+2+4+1+4
=
4+3+2+1 + 4+4+4+4
=

Σ
i=1
i + 4*4
1-minute algebra review
x

Σ
i=1
i = x * (x+1) / 2

Check: 1 + 2 + 3 + 4 = 10 = 4*5/2
For array of size n, # of visits =
n

Σ
i=1
i + 4*n

= ½ * n * (n+1) + 4n
For array of size n, # of visits =
n

Σ
i=1
i + 4*n

= ½ * n * (n+1) + 4n

= n /2 + n/2 + 4n = n /2 + 4.5n
2 2
For array of size and, # of visits =
= ½ * n * (n+1) + 4n

= n /2 + n/2 + 4n = n /2 + 4.5n
2 2

Call this T(n) where n=array size

and units are array visits.
 Selection Sort: T(n) = n /2 + 4.5n 2

• Because n is large, we can ignore all terms except the one

that grows fastest
• ➔ T(n) ~= n2/2 (simpler!)
• Because we use “proportional to…”, we can ignore
coefficient of that term:
• ➔ T(n) α n2 (even simpler!)
• α means “is proportional to”
• ➔ T(n) is proportional to (after dropping slow-growing
terms) n2
• ➔ T(n) is Big-O of n2
• We write T(n) = O(n2)
• Big-O has a more rigorous definition (CS 146)
 Selection Sort is O(n^2) … What it means

• Measure execution time for some n

– On your computer, under your OS, with some
implementation
• Double n ➔ 4x execution time (because 22 = 4)
• Triple n ➔ 9x execution time (because 32 = 9)
• n x 1000 ➔ 1 million x execution time
(10002 = 1 million)
Insertion Sort
 Insertion sort
• Insertion sort is a sorting algorithm that treats the input as two parts, a sorted part and an unsorted part, and repeatedly inserts
the next value from the unsorted part into the correct location in the sorted part.
• The index variable i denotes the starting position of the current element in the unsorted part. Initially, the first element (i.e.,
element at index 0) is assumed to be sorted, so the outer for loop initializes i to 1. The inner while loop inserts the current
element into the sorted part by repeatedly swapping the current element with the elements in the sorted part that are larger. Once
a smaller or equal element is found in sorted part, the current element has been inserted in the correct location and the while
loop terminates.
• Insertion sort's typical runtime is O()
• For sorted or nearly sorted inputs, insertion sort's runtime is O(N). A nearly sorted list only contains a few elements not in sorted order. Ex: {4, 5,
17, 25, 89, 14} is nearly sorted having only one element not in sorted position.
• or each outer loop execution, if the element is already in sorted position, only a single comparison is made. Each element not in sorted position
requires at most N comparisons. If there are a constant number, C, of unsorted elements, sorting the N - C sorted elements requires one comparison
each, and sorting the C unsorted elements requires at most N comparisons each. The runtime for nearly sorted inputs is O((N - C) * 1 + C * N) =
O(N).
 Insertion Sort
Slowly builds up a sorted array by choosing unsorted elements and inserting
them into the sorted array

Where does 7 belong? 7 10 4 1

 Insertion Sort
Slowly builds up a sorted array by choosing unsorted elements and inserting
them into the sorted array

Where does 7 belong? 7 10 4 1

Where does 10 belong? 7 10 4 1

Insert 10 into the blue array where it belongs
 Insertion Sort
Slowly builds up a sorted array by choosing unsorted elements and inserting
them into the sorted array

Where does 7 belong? 7 10 4 1

Where does 10 belong? 7 10 4 1

Where does 4 belong? 7 10 4 1

Insert 4 into the blue array where it belongs
 Insertion Sort
Slowly builds up a sorted array by choosing unsorted elements and inserting
them into the sorted array

Where does 7 belong? 7 10 4 1

Where does 10 belong? 7 10 4 1

Where does 4 belong? 7 10 4 1

Where does 1 belong? 4 7 10 1

Insert 1 into the blue array where it belongs
 Insertion Sort
Slowly builds up a sorted array by choosing unsorted elements and inserting
them into the sorted array

Where does 7 belong? 7 10 4 1

Where does 10 belong? 7 10 4 1

Where does 4 belong? 7 10 4 1

Where does 1 belong? 4 7 10 1

1 4 7 10 Done!
 Pseudocode Practice
Insertion Sort

insertionSort(arr)
 Pseudocode
Insertion Sort Solution

insertionSort(arr)

for i in 0 to arr.length
toPlace = arr[i]
j = i-1 //last index of sorted part
while j>=0 and arr[j]>toPlace
arr[j+1] = arr[j]
j--
arr[j+1]=toPlace
Merge Sort
 Merge sort
• Merge sort is a sorting algorithm that divides a list into two halves, recursively sorts each half, and then merges
the sorted halves to produce a sorted list. The recursive partitioning continues until a list of 1 element is reached,
as list of 1 element is already sorted.
• The merge sort algorithm uses three index variables to keep track of the elements to sort for each recursive
method call. The index variable i is the index of first element in the list, and the index variable k is the index of
the last element. The index variable j is used to divide the list into two halves. Elements from i to j are in the left
half, and elements from j + 1 to k are in the right half.
• Merge sort merges the two sorted partitions into a single list by repeatedly selecting the smallest element from
either the left or right partition and adding that element to a temporary merged list. Once fully merged, the
elements in the temporary merged list are copied back to the original list.
• The merge sort algorithm's runtime is O(N log N). Merge sort divides the input in half until a list of 1 element is
reached, which requires log N partitioning levels. At each level, the algorithm does about N comparisons
selecting and copying elements from the left and right partitions, yielding N * log N comparisons.
• Merge sort requires O(N) additional memory elements for the temporary array of merged elements. For the final
merge operation, the temporary list has the same number of elements as the input. Some sorting algorithms sort
the list elements in place and require no additional memory, but are more complex to write and understand.
 Merge Sort
• Not in-place: uses extra memory
• “Divide-and-conquer” algorithm
• Recursion simplifies by splitting data set in half,
rather than reducing its size by 1

Why do you think we call this

5 7 1 9 6 5 2 3 divide and conquer?
 Merge Sort (divide)

5 7 1 9 6 5 2 3

5 7 1 9 6 5 2 3

5 7 1 9 6 5 2 3

5 7 1 9 6 5 2 3
Merge Sort (put it back together)

5 7 1 9 6 5 2 3

5 7 1 9 5 6 2 3

1 5 7 9 2 3 5 6

1 2 3 5 5 6 7 9
 Complexity of Merge Sort
How many times is the entire data set split?

5 7 1 9 6 5 2 3

5 7 1 9 6 5 2 3

5 7 1 9 6 5 2 3

5 7 1 9 6 5 2 3

log2(n) = log2(8) = 3
 Complexity of Merge Sort
How do we put the data back together?

5 7 1 9 6 5 2 3

5 7 1 9 5 6 2 3

1 5 7 9 2 3 5 6

1 2 3 5 5 6 7 9
O(n) for each step
 Merge Sort Complexity
• There are log2(n) “levels”
• For each “level”
• O(n) work is done to split and merge
• = O(n * log(n))
If you forget
everything else
about Merge Sort
When computer scientists say
log they mean log base 2. complexity,
remember this!
 Pseudocode Practice
Merge Sort

mergeSort(arr,start,end)

merge(arr,start,mid,end)
 Pseudocode Practice
Merge Sort

mergeSort(arr,start,end)

if start>=end
return

mid = midpoint of arr

mergeSort(arr,start,mid)
mergeSort(arr,mid+1,end)

//merge the two parts

merge(arr,start,mid,end)
 Pseudocode Practice
Merge Sort
merge(arr,start,mid,end)

leftSize = mid - start + 1;

rightSize = end - mid;

// temp arrays to hold values as we move them around

left[]
right[]

copy values from arr into left and right

// Initial indexes of first and second subarrays

leftIndex = 0
rightIndex = 0

mergedIndex = start

while leftIndex<leftsize and rightIndex < rightSize

if left[leftIndex] <= right[rightIndex]
arr[mergedIndex] = left[leftIndex]
leftIndex++
else
arr[mergedIndex] = right[rightIndex]
rightIndex++
mergedIndex ++

while leftIndex < leftSize

arr[mergedIndex] = left[leftIndex]
leftIndex++
mergedIndex++

while rightIndex < rightSize

arr[mergedIndex] = right[rightIndex]
rightIndex++
mergedIndex++
 Merge Sort 5 7 1 9 6 5 2 3
start=0 end=3
mid=1
merge( 57196523 ,0,1,3)

leftSize = mid - start + 1;

rightSize = end - mid;

// temp arrays to hold values as we move them around

left[]
right[]

copy values from arr into left and right

// Initial indexes of first and second subarrays

leftIndex = 0
rightIndex = 0

mergedIndex = start

while leftIndex<leftsize and rightIndex < rightSize

if left[leftIndex] <= right[rightIndex]
arr[mergedIndex] = left[leftIndex]
leftIndex++
else
arr[mergedIndex] = right[rightIndex]
rightIndex++
mergedIndex ++

while leftIndex < leftSize

arr[mergedIndex] = left[leftIndex]
leftIndex++
mergedIndex++

while rightIndex < rightSize

arr[mergedIndex] = right[rightIndex]
rightIndex++
 Merge Sort 5 7 1 9 6 5 2 3
start=0 end=3
mid=1
merge( 57196523 ,0,1,3)
leftSize = 1 - 0 + 1 = 2 rightSize = 3 - 1 = 2

leftSize = mid - start + 1;

rightSize = end - mid;

// temp arrays to hold values as we move them around

left[]
right[]

copy values from arr into left and right

// Initial indexes of first and second subarrays

leftIndex = 0
rightIndex = 0

mergedIndex = start

while leftIndex<leftsize and rightIndex < rightSize

if left[leftIndex] <= right[rightIndex]
arr[mergedIndex] = left[leftIndex]
leftIndex++
else
arr[mergedIndex] = right[rightIndex]
rightIndex++
mergedIndex ++

while leftIndex < leftSize

arr[mergedIndex] = left[leftIndex]
leftIndex++
mergedIndex++

while rightIndex < rightSize

arr[mergedIndex] = right[rightIndex]
rightIndex++
 Merge Sort 5 7 1 9 6 5 2 3
start=0 end=3
mid=1
merge( 57196523 ,0,1,3)
leftSize = 1 - 0 + 1 = 2 rightSize = 3 - 1 = 2

leftSize = mid - start + 1;

rightSize = end - mid;

// temp arrays to hold values as we move them around

left[]
right[] left right

copy values from arr into left and right

// Initial indexes of first and second subarrays

leftIndex = 0
rightIndex = 0

mergedIndex = start

while leftIndex<leftsize and rightIndex < rightSize

if left[leftIndex] <= right[rightIndex]
arr[mergedIndex] = left[leftIndex]
leftIndex++
else
arr[mergedIndex] = right[rightIndex]
rightIndex++
mergedIndex ++

while leftIndex < leftSize

arr[mergedIndex] = left[leftIndex]
leftIndex++
mergedIndex++

while rightIndex < rightSize

arr[mergedIndex] = right[rightIndex]
rightIndex++
 Merge Sort 5 7 1 9 6 5 2 3
start=0 end=3
mid=1
merge( 57196523 ,0,1,3)
leftSize = 1 - 0 + 1 = 2 rightSize = 3 - 1 = 2

leftSize = mid - start + 1;

rightSize = end - mid;

// temp arrays to hold values as we move them around

5 7 1 9
left[]
right[] left right

copy values from arr into left and right

// Initial indexes of first and second subarrays

leftIndex = 0
rightIndex = 0

mergedIndex = start

while leftIndex<leftsize and rightIndex < rightSize

if left[leftIndex] <= right[rightIndex]
arr[mergedIndex] = left[leftIndex]
leftIndex++
else
arr[mergedIndex] = right[rightIndex]
rightIndex++
mergedIndex ++

while leftIndex < leftSize

arr[mergedIndex] = left[leftIndex]
leftIndex++
mergedIndex++

while rightIndex < rightSize

arr[mergedIndex] = right[rightIndex]
rightIndex++
 Merge Sort 5 7 1 9 6 5 2 3
start=0 end=3
mid=1
merge( 57196523 ,0,1,3)
leftSize = 1 - 0 + 1 = 2 rightSize = 3 - 1 = 2

leftSize = mid - start + 1;

rightSize = end - mid;

// temp arrays to hold values as we move them around

5 7 1 9
left[]
right[] left right

copy values from arr into left and right

leftIndex=0 rightIndex=0
// Initial indexes of first and second subarrays
leftIndex = 0
rightIndex = 0
mergeIndex = 0

mergedIndex = start

while leftIndex<leftsize and rightIndex < rightSize

if left[leftIndex] <= right[rightIndex]
arr[mergedIndex] = left[leftIndex]
leftIndex++
else
arr[mergedIndex] = right[rightIndex]
rightIndex++
mergedIndex ++

while leftIndex < leftSize

arr[mergedIndex] = left[leftIndex]
leftIndex++
mergedIndex++

while rightIndex < rightSize

arr[mergedIndex] = right[rightIndex]
rightIndex++
 Merge Sort 5 7 1 9 6 5 2 3
1
start=0 end=3
mid=1
merge( 57196523 ,0,1,3)
leftSize = 1 - 0 + 1 = 2 rightSize = 3 - 1 = 2

leftSize = mid - start + 1;

rightSize = end - mid;

// temp arrays to hold values as we move them around

5 7 1 9
left[]
right[] left right

copy values from arr into left and right

leftIndex=0 rightIndex=1
// Initial indexes of first and second subarrays
leftIndex = 0
rightIndex = 0
mergeIndex = 1

mergedIndex = start

while leftIndex<leftsize and rightIndex < rightSize

if left[leftIndex] <= right[rightIndex]
arr[mergedIndex] = left[leftIndex]
leftIndex++
else
arr[mergedIndex] = right[rightIndex]
rightIndex++
mergedIndex ++

while leftIndex < leftSize

arr[mergedIndex] = left[leftIndex]
leftIndex++
mergedIndex++

while rightIndex < rightSize

arr[mergedIndex] = right[rightIndex]
rightIndex++
 Merge Sort 1 57 1 9 6 5 2 3
start=0 end=3
mid=1
merge( 57196523 ,0,1,3)
leftSize = 1 - 0 + 1 = 2 rightSize = 3 - 1 = 2

leftSize = mid - start + 1;

rightSize = end - mid;

// temp arrays to hold values as we move them around

5 7 1 9
left[]
right[] left right

copy values from arr into left and right

leftIndex=1 rightIndex=1
// Initial indexes of first and second subarrays
leftIndex = 0
rightIndex = 0
mergeIndex = 2

mergedIndex = start

while leftIndex<leftsize and rightIndex < rightSize

if left[leftIndex] <= right[rightIndex]
arr[mergedIndex] = left[leftIndex]
leftIndex++
else
arr[mergedIndex] = right[rightIndex]
rightIndex++
mergedIndex ++

while leftIndex < leftSize

arr[mergedIndex] = left[leftIndex]
leftIndex++
mergedIndex++

while rightIndex < rightSize

arr[mergedIndex] = right[rightIndex]
rightIndex++
 Merge Sort 1 57 1
7 9 6 5 2 3
start=0 end=3
mid=1
merge( 57196523 ,0,1,3)
leftSize = 1 - 0 + 1 = 2 rightSize = 3 - 1 = 2

leftSize = mid - start + 1;

rightSize = end - mid;

// temp arrays to hold values as we move them around

5 7 1 9
left[]
right[] left right

copy values from arr into left and right

leftIndex=2 rightIndex=1
// Initial indexes of first and second subarrays
leftIndex = 0
rightIndex = 0
mergeIndex =3

mergedIndex = start

while leftIndex<leftsize and rightIndex < rightSize

if left[leftIndex] <= right[rightIndex]
arr[mergedIndex] = left[leftIndex]
leftIndex++
else
arr[mergedIndex] = right[rightIndex]
rightIndex++
mergedIndex ++

while leftIndex < leftSize

arr[mergedIndex] = left[leftIndex]
leftIndex++
mergedIndex++

while rightIndex < rightSize

arr[mergedIndex] = right[rightIndex]
rightIndex++
 Merge Sort 1 57 1
7 9 6 5 2 3
start=0 end=3
mid=1
merge( 57196523 ,0,1,3)
leftSize = 1 - 0 + 1 = 2 rightSize = 3 - 1 = 2

leftSize = mid - start + 1;

rightSize = end - mid;

// temp arrays to hold values as we move them around

5 7 1 9
left[]
right[] left right

copy values from arr into left and right

leftIndex=2 rightIndex=1
// Initial indexes of first and second subarrays
leftIndex = 0
rightIndex = 0
mergeIndex =3

mergedIndex = start

while leftIndex<leftsize and rightIndex < rightSize

if left[leftIndex] <= right[rightIndex]
arr[mergedIndex] = left[leftIndex]
leftIndex++
else
arr[mergedIndex] = right[rightIndex]
rightIndex++
mergedIndex ++

while leftIndex < leftSize

arr[mergedIndex] = left[leftIndex]
leftIndex++
mergedIndex++

while rightIndex < rightSize

arr[mergedIndex] = right[rightIndex]
rightIndex++
 Merge Sort 1 57 1
7 9 5 6 2 3
start=4 end=7
mid=5
merge( 57196523 ,4,5,7)
leftSize = 5 - 4 + 1 = 2 rightSize = 7 - 5 = 2

leftSize = mid - start + 1;

rightSize = end - mid;

// temp arrays to hold values as we move them around

left[]
right[] left right

copy values from arr into left and right

leftIndex=0 rightIndex=0
// Initial indexes of first and second subarrays
leftIndex = 0
rightIndex = 0
mergeIndex =0

mergedIndex = start

while leftIndex<leftsize and rightIndex < rightSize

if left[leftIndex] <= right[rightIndex]
arr[mergedIndex] = left[leftIndex]
leftIndex++
else
arr[mergedIndex] = right[rightIndex]
rightIndex++
mergedIndex ++

while leftIndex < leftSize

arr[mergedIndex] = left[leftIndex]
leftIndex++
mergedIndex++

while rightIndex < rightSize

arr[mergedIndex] = right[rightIndex]
rightIndex++
 Merge Sort 1 57 1
7 9 2 3 5 6
start=0 end=7
mid=5
merge( 57196523 ,0,3,7)
leftSize = 3 - 0 + 1 = 4 rightSize = 7 - 3 = 4

leftSize = mid - start + 1;

rightSize = end - mid;

// temp arrays to hold values as we move them around

left[] right
right[]
left

copy values from arr into left and right

leftIndex=0 rightIndex=0
// Initial indexes of first and second subarrays
leftIndex = 0
rightIndex = 0
mergeIndex =0

mergedIndex = start

while leftIndex<leftsize and rightIndex < rightSize

if left[leftIndex] <= right[rightIndex]
arr[mergedIndex] = left[leftIndex]
leftIndex++
else
arr[mergedIndex] = right[rightIndex]
rightIndex++
mergedIndex ++

while leftIndex < leftSize

arr[mergedIndex] = left[leftIndex]
leftIndex++
mergedIndex++

while rightIndex < rightSize

arr[mergedIndex] = right[rightIndex]
rightIndex++
Search
 Searching algorithms
• An algorithm is a sequence of steps for accomplishing a task.

• Linear search is a search algorithm that starts from the beginning of a list, and checks each element until the
search key is found or the end of the list is reached.

• An algorithm's runtime is the time the algorithm takes to execute.

• For a list with N elements, linear search thus requires at most N comparisons. The algorithm is said to require
"on the order" of N comparisons.
 Searching algorithms
• An algorithm is a sequence of steps for accomplishing a task.

• Linear search is a search algorithm that starts from the beginning of a list, and checks each element until the
search key is found or the end of the list is reached.

• An algorithm's runtime is the time the algorithm takes to execute.

• For a list with N elements, linear search thus requires at most N comparisons. The algorithm is said to require
"on the order" of N comparisons.
 I want to know if my array contains some element

• Linear Search

int linearSearch(int[] a, int k) {

for(int i=0;i<a.length;i++) {
if(a[i]==k) {return i;}
}
return -1;
}
 Complexity of linear search in an array

• If the array doesn’t contain k (worst case scenario)

– T(n) = n visits, where n = length of the array
• If the array does contain k (average scenario)
– Over many executions, k is equally likely to be in any array slot
– Half of the time, k is in the lower half of the array, so T(n) < n/2
– Half of the time, k is in the upper half of the array, so T(n) > n/2
– On average, T(n) = n/2
• In both cases, O(n) complexity
 What if the array is sorted?
• What is the complexity of linear search on a sorted
array?
• Still O(n) :(
• Shouldn’t there be a benefit for sorting the array?
Can you think of some smarter
strategies?
 Binary search

• Binary search is a faster algorithm for searching a list if the list's elements are sorted and directly
accessible (such as an array). Binary search first checks the middle element of the list. If the search
key is found, the algorithm returns the matching location. If the search key is not found, the
algorithm repeats the search on the remaining left sublist (if the search key was less than the middle
element) or the remaining right sublist (if the search key was greater than the middle element).

• Binary search is incredibly efficient in finding an element within a sorted list. During each iteration or step of
the algorithm, binary search reduces the search space (i.e., the remaining elements to search within) by half.
The search terminates when the element is found or the search space is empty (element not found).

• For a 32 element list, if the search key is not found, the search space is halved to have 16 elements, then 8, 4,
2, 1, and finally none, requiring only 6 steps. For an N element list, the maximum number of steps required to
reduce the search space to an empty sublist is . Ex: .
 Binary Search:
Cut the problem in half each time

1 2 3 4 12 33 67 99

If it’s here, which half is it in?

12 33 67 99
If it’s here, which half is it in?

12 33
If it’s here, which half is it in?
 Complexity of binary search
• Assume n is a power of 2 (i.e., you can keep dividing it in half until you get 1)
• Iteration 1: Array size n
• Iteration 2: Array size n/2
• Iteration 3: Array size (n/2)/2 = n /4
• …
• Iteration k (one element array):
• Array size n/2^k-1 = 1,
• n = 2^k-1,
• log(n) = log(2^k-1) Smart recursive algorithm is able to benefit
• log(n) = (k-1)log 2= k - 1. from sorted array. Simple linear search is
• log(n) = k - 1 not.
• Thus the number of iterations is log(n) + 1
• Each iteration takes constant time
• Binary search is O(log(n))