Bubble Sort Selection Sort Insertion Sort Merge Sort

Code: Code: Code: Code:

for i=1 to n-1 { for i = 2 to n { MergeSort (A, p, r)
min = i j=i if p < r then {
for j = i+1 to n { while (aj < aj-1 && j > 1) { q = floor (p+r)/2
if aj < amin , then min = j swap aj with aj-1 MergeSort (A, p, q)
} j = j-1 MergeSort (A, q+1, r)
swap aj with amin } MergeSort (A, p, q, r)
} } }

Runtime Analysis: (stable) Runtime Analysis: (not stable) Runtime Analysis: (stable) Runtime Analysis: (stable)
# of Comparisons = based off of inner # of Comparisons = based off of inner # of Comparisons = based off of inner # of Comparisons = each level of the
loop -> loop -> loop -> recursive tree takes CN time, there are
(n-1) + (n-2) + … + 1 = ∑ k=1 -> n = (n-1) + (n-2) + … + 1 = ∑ k=1 -> n = (n-1) + (n-2) + … + 1 = ∑ k=1 -> n = logn levels in the tree
n(n-1)/2 n(n-1)/2 n(n-1)/2
Total Cost = ∑ i=1 -> lgn (cn) = cn(logn)
Runtime Complexity: Θ(n2) Runtime Complexity: Θ(n2) Runtime Complexity: cn(logn)
- Worst Case: Θ(n2)
Explanation: In a typical iteration, each Explanation: The Selection Sort is a very - Best Case: Θ(n) (n-1) Runtime Complexity: Θ(nlogn)
adjacent pair of elements is compared, basic sort. It works by finding the
starting with the first two elements, then smallest element in the array and putting Explanation: In general, in the jth step of Explanation: To understand the
second and third, and all the way to the it at the beginning of the list and then the insertion sort, the jth element of the efficiency of the mergesort algorithm it
final two elements. -> Each time two repeating that process on the unsorted list is inserted into the correct position is useful to separate the merging from
elements are compared, if they are remainder of the data -> Rather than in the list of the previously sorted j − 1 the sorting. The sorting takes place
already in sorted order, nothing is done making successive swaps with adjacent elements. indirectly, by repeatedly splitting the
to them, and the next pair of elements is elements like bubble sort, selection sort data in half until sorted singleton sets
compared. In the case where the two makes only one, swapping the smallest are created. The merging then rebuilds
elements are not in sorted order, the number with the number occupying its Outer loop: n-1 iterations the complete, original data set by
two elements are swapped, putting correct position. Inner loop: at most j-1 iterations splicing together the sorted mini-lists.
them in order. (will be exactly (n-1) - breaking down of the data occurs
iterations in the outer loop) Outer loop: n-1 iterations with efficiency (log n)
Inner loop: n-i iterations - the merging process is linear each
time two lists have to be merged (n)

Loop Invariant: 2 ways to phrase it: Loop Invariant: 2 ways to phrase it: Loop Invariant: Loop Invariant:
1) At the start of the kth iteration (when 1) At the start of the kth iteration (when 1) At the start of the kth iteration (when 1) Initialization:
i = k), the largest k-1 elements are in i = k), the first k-1 elements are the i = k+1), the first k elements are in 2) Maintenance:
the last positions in sorted order smallest elements and are sorted sorted order 3) Termination:
2) At the end of the kth iteration, the 2) Geeks: two loop invariant conditions
most right k elements are sorted - in the outer loop, array is sorted
and in place for first i elements (n-1 iterations)
- in the inner loop, min is always the
minimum value in

Proof of Correctness: Proof of Correctness: Proof of Correctness: Proof of Correctness:

1) Let P(k) be the proposition of (loop 1) Let P(k) be the proposition of (loop 1) Let P(k) be the proposition of (loop 1) Let P(k) be the proposition that
invariant) invariant) invariant) merge sort correctly sorts any array
2) Base Case: 2) Base Case: 2) Base Case: of length (k)
- 1st Iteration = at the start of the - 1st Iteration = at the start of the - 1st Iteration = at the start of the 2) Base Case:
first iteration, the largest (0) first iteration, the first (0) elements first iteration, when i=2, the first (1) - Consider an array of 1 element.
elements are in the last position are the smallest elements and are element is in sorted order This array is already sorted
3) Inductive Step: sorted 3) Inductive Step: 3) Inductive Step: assume that P(k) is
- As I.H. we assume P(k) is true at 3) Inductive Step: - As I.H. we assume P(k) is true at true for all 1 <= k <= n. we now
the start of the kth iteration (k >= 1) - As I.H. we assume P(k) is true at the start of the kth iteration (k >= 1) need to show that merge sort works
Now we must show P(k+1) is true the start of the kth iteration (k >= 1) Now we must show P(k+1) is true correctly on any array of sign n
for the (k+1) iteration Now we must show P(k+1) is true for the (k+1) iteration Suppose we call mergesort on an array
During the kth iteration the last k-1 for the (k+1) iteration During the kth iteration, the relative of size (n). It will then recursively call
elements will not be touched, so they During the kth iteration the first k-1 order of the first k elements will remain mergesort on two arrays of size n/2. By
will hold the largest k-1 elements in elements will not be touched, so they the same and the (k+1) element in the I.H. these calls will correctly sort these
sorted order. In addition, the largest of will still hold the smallest k-1 elements list will be properly inserted into this list arrays
the remaining element will sink to the kth in sorted order. In addition, the smallest suck that the first k+1 elements will be
position from the bottom (index n-k+1). of the remaining elements will be placed in sorted order by the start of the (k+1)st Once the two halves are correctly
Therefore, at the start of the (k+1)st at index k. Therefore, at the start of the iteration sorted, we will use the merge function
iteration, the largest k elements are in (k+1)st iteration, the smallest k elements that correctly combines the two sorted
the last positions in sorted order. There are in the first positions in sorted order. There will be exactly n-1 iterations in the subarrays. Therefore, after the
will be exactly n-1 iterations in the outer This concludes. outer loop, so by the correctness of the completion of the merge operation, we
loop, so by correctness of L.I., the loop invariant, the first n elements are will have a correct soring of the original
largest n-1 elements are in the last There will be exactly n-1 iterations in the sorted, i.e. the whole list is sorted array of size (n).
indices and in sorted order (so smallest outer loop, so by the correctness of the
element is first in the list, sorted) loop invariant, the first k-1 elemenets
are the smallest elements and are in
sorted order. So the nth element must
be the largest and the whole list is
sorted.
Quick Sort Counting Sort Radix Sort Bucket Sort

Code: For QUICK SORT: Code: Applies to the case where all the elements Code:
QuickSort (A, p, r) have multiple keys each within a fixed range, e.g. beats Ω(nlogn)
if r > p then { -if your list consists of 1 element, you are done Dates (year, month, day)
q = Partition (A, p, r) - otherwise, choose any element in your list to be the ==============================
QuickSort(A, p, q-1) “pivot” ==============================
Quick Sort(A, q+1, r)
} ============================== BinarySearch (value, Array, low, high)
LinearSearch (Array, length, x) if (low > high) { return -1 }
z

it.si I
Partition (A, p, r) for i = 1 to length mid = (low + high)/2
leftp = p
rightp = r - 1
if (Array[ i ] = x) {return i} if (value == Array[ mid ]) { return mid }
if (value < Array[ mid ] {
orouthof
Functions
pivot = A[ r ] return -1 return binarySearch( value, Array, low, mid - 1)
while True do { return binarySearch (value, Array, mid + 1, high)

a
while A [leftp] < pivot do { Best Case Scenario:
leftp = leftp + 1 c1 * 1 + c2 = c1 + c2 problem
arrangesanctions
u naeronorationosinson manners.mn
}
while A[rightp] > pivot && rightp > p{ Worst Case Scenario:
equivalent or
castrictsubsero
eosin
nniziognexampies
remember nnioszznciog.nl
right p = rightp - 1 c1 * length + c2
snoumatnisnoton
} into classify exponential
polynomiall ogarithm

i ii
exists.at mecnunenenn.n
if leftp >= right p {break}
else { swap A[leftp] and A[rightp] } room.comia one
}
swap A[leftp] and A[r]
sumoeneeareconstantscaanamianea
uneven
cases see
return leftp
n unan.o
Enisstatematnaera.is noses
Runtime Analysis: Runtime Analysis: nomatteranatcasnae.itunn Runtime Analysis: Runtime Analysis:
Best Case Scenario:
- At each step we divide the list into two equal Runtime Complexity: Θ(n)
n Runtime Complexity: Θ(d * (n+k)) where k is the
largest range of any of the fields involved in the sort
Runtime Complexity: Θ(n)

parts -Takes Θ(n + k) and if k = Θ(n), then it simply takes Explanation: Sorting algorithm in which the elements
- Performance in this case would be the same as Θ(n) Runtime will be Θ(dn) if k = O(n) are separated into several groups that are called
MergeSort -> Θ(nlogn) (ALSO random runtime) buckets. Each bucket is then sorted individually using
Explanation: Applies to the case where all the Explanation: Basic idea is that we use a stable linear any other algorithm or recursively using bucket sort
Worst Case Scenario: elements of the array are integers in the range 1 to k. search to sort the array d times -> d is the number of itself, Then the sorted buckets are gathered together
- At each step if the pivot element is the largest Basic idea is to go through the list and count the fields involved in the sort. - mainly useful when input is uniformly distributed
element, thene there will only be one subproblem number of occurrences of each element. Then, over a range
which is only one element smaller -> Θ(n2) construct a sorted list based on this count -> we divide the range into (n) equal sized buckets.
information. beats Ω(nlogn) we then distribute the n elements into the buckets (in
Explanation: Good choice of a pivot element is key one linear time pass). Since the elements have a
for performance. Strategies to choose pivot: 1) beats Ω(nlogn) uniform distribution, there wont be many elements
median of first, middle, and last elements 2) median within each buckets. So we can quickly sort the
of first middle, last, n/4th, 3n/4th elements 3) choose elements within each bucket and then go through the
pivot element randomly buckets in order. (the buckets can be sorted using
insertion sort)

Loop Invariant:
The pivot is in the exact correct position.

Proof of Correctness: Proof of Correctness: Proof of Correctness: Proof of Correctness:

1) Let P(k) be the proposition of (loop invariant) 1) Let P(k) be the proposition of (loop invariant) 1) Let P(k) be the proposition of (loop invariant) 1) Let P(k) be the proposition of (loop invariant)
2) Base Case: 2) Base Case: 2) Base Case: 2) Base Case:
3) Inductive Step: 3) Inductive Step: 3) Inductive Step: 3) Inductive Step:

constant lineasiogspoisnomiasen when isorgansandnexisasunctionthat

haslargeabsolutevaluesthangasa
Asymptotic Notation and Time Complexity:
wasresonating
mother intherelationshipsooasescanbereplaced
two
example tosowingapromen Growthoffunctions
a lgorithms
with
function largeabsolutevalues
and
oneusingioonstanta operations thegrowthoffunctionsisoftendescribedusingaspecial
the notation isaniecigan iron
overusing
n 'operations
anaisinaisigoisoranx athen
www.n.gg
noontiantausesearrewe
une
operations and thesetofintegersone
from
functions iron
no isuseseven
large though
moreoperation setofreal tothesetofrealnumberswe
numbers Hence isoonan
forsmanvanesan whenbig onotationisuseatherunctiongintherelationshipsasorganisortenon
saythat is organ ifthereareconstantscan thesmallestgrowthrateofthe
have belongingtoasetofreferencefunction
functions
remarkintuitivelythedefinition suchthat assonationsorthesormxsunerenisapositivereainombe
ate is ourssays that go basiscigar
owerthansomerixeamutineorgasas
bound
growswithout e
retranasber anabis
o megan otation
neconstantscanarinthedefinitionofbigonotationarecalledwitnessestothe
relationships ornesetorreainumberstonesetotreainuses
inparticular is.us ieanaoni
organs wesastnateasisregassie thereare
toestablishthat isorganweneedonly onepairofwitnessestothis
relationship canonwithapositivesuch
constants that
thatistosnouthat isorgan weneeasinaon onepairof constantscananthe taxi cigar
witnesses thatieanecigalwhenever x a
such
n
manpairsofwitnesses
ceanorreainumber.to
infinitely ceteanasberonaionseronmesetorintese.stnatsasis
mesetorreainumberswesastnat
tosee
thisnotetnatiecananareonepairorwitnessesthenanspairamanwhereccc
raison
is
scxsieanaonsitananai.isaisoapairo
wotetnatea.is
witnessesb ecauseransciganecigan whenever x visa neasisorgasana easisocasanagasisoca
warning withthedefinition Big onatation areissues
ausernapproacusoreinaing apairofwitnessesistofirst selectaraneousorunicnthesizeor asanitarons
mm orgastnateasisooraergasanamateasandapositivereanonsensatnat
fcsorwnichisaslecigaslsorx.rs aeottnesameo.ae aiganaanecagan
ampielisnouthateasetaxtiisons
whenever
n
wecan that
o bserve wecan these
e stimate
readily caananiaswitnesses
ecausexaxand icewhen x i
n that
abigoestimatesornicanbeontaineasnoting each
osxtzxtiextzxax.ae unenevers Because earnortheintesersininesomorthesian
Hence
ermintneproactaoesnotexceean
now consequentiswecantanecuanan aswitnesses integers
positive
aoesnotexceean.itsonoustnatnti.zs.onen.n.n.in
mastinentntinen
ni misono
annalogntisocniogn