Sorting2

Heap: array implementation13275648109Is it a good idea to store arbitrary binary trees as arrays? May have many empty spaces!

Array implementation1632514The root node is A[1].The left child of A[j] is A[2j]The right child of A[j] is A[2j + 1]The parent of A[j] is A[j/2] (note: integer divide)25436Need to estimate the maximum size of the heap.

Heapsort(1) Build a binary heap of N elements the minimum element is at the top of the heap(2) Perform N DeleteMin operationsthe elements are extracted in sorted order(3) Record these elements in a second array and then copy the array back

Heapsort – running time analysis(1) Build a binary heap of N elements repeatedly insert N elements O(N log N) time (there is a more efficient way)(2) Perform N DeleteMin operationsEach DeleteMin operation takes O(log N)  O(N log N)(3) Record these elements in a second array and then copy the array backO(N)Total: O(N log N)Uses an extra array

Heapsort: no extra storageAfter each deleteMin, the size of heap shrinks by 1We can use the last cell just freed up to store the element that was just deleted  after the last deleteMin, the array will contain the elements in decreasing sorted orderTo sort the elements in the decreasing order, use a min heapTo sort the elements in the increasing order, use a max heapthe parent has a larger element than the child

HeapsortSort in increasing order: use max heapDelete 97

Heapsort: A complete exampleDelete 16Delete 14

Example (cont’d)Delete 10Delete 9Delete 8

Lower bound for sorting,radix sort

Lower Bound for SortingMergesort and heapsortworst-case running time is O(N log N)Are there better algorithms?Goal: Prove that any sorting algorithm based on only comparisons takes (N log N) comparisons in the worst case (worse-case input) to sort N elements.

Lower Bound for SortingSuppose we want to sort N distinct elementsHow many possible orderings do we have for N elements?We can have N! possible orderings (e.g., the sorted output for a,b,c can be a b c, b a c, a c b, c a b, c b a, b c a.)

Lower Bound for SortingAny comparison-based sorting process can be represented as a binary decision tree.Each node represents a set of possible orderings, consistent with all the comparisons that have been madeThe tree edges are results of the comparisons

Decision tree forAlgorithm X for sortingthree elements a, b, c

Lower Bound for SortingA different algorithm would have a different decision treeDecision tree for Insertion Sort on 3 elements:

Lower Bound for SortingThe worst-case number of comparisons used by the sorting algorithm is equal to the depth of the deepest leafThe average number of comparisons used is equal to the average depth of the leavesA decision tree to sort N elements must have N! leavesa binary tree of depth d has at most 2d leaves  the tree must have depth at least log2 (N!)Therefore, any sorting algorithm based on only comparisons between elements requires at least log2(N!)  comparisons in the worst case.

Lower Bound for SortingAny sorting algorithm based on comparisons between elements requires (N log N) comparisons.

Linear time sortingCan we do better (linear time algorithm) if the input has special structure (e.g., uniformly distributed, every numbers can be represented by d digits)? Yes.Counting sort, radix sort

Counting SortAssume N integers to be sorted, each is in the range 1 to M.Define an array B[1..M], initialize all to 0  O(M)Scan through the input list A[i], insert A[i] into B[A[i]]  O(N)Scan B once, read out the nonzero integers  O(M)Total time: O(M + N)if M is O(N), then total time is O(N)Can be bad if range is very big, e.g. M=O(N2)N=7, M = 9, Want to sort 8 1 9 5 2 6 3 5893612Output: 1 2 3 5 6 8 9

Counting sortWhat if we have duplicates?B is an array of pointers.Each position in the array has 2 pointers: head and tail. Tail points to the end of a linked list, and head points to the beginning.A[j] is inserted at the end of the list B[A[j]]Again, Array B is sequentially traversed and each nonempty list is printed out.Time: O(M + N)

M = 9, Wish to sort 8 5 1 5 9 5 6 2 7 125678955Output: 1 2 5 5 5 6 7 8 9Counting sort

Radix SortExtra information: every integer can be represented by at most k digitsd1d2…dkwheredi are digits in base rd1: most significant digitdk: least significant digit

Radix SortAlgorithmsort by the least significant digit first (counting sort) => Numbers with the same digit go to same binreorder all the numbers: the numbers in bin 0 precede the numbers in bin 1, which precede the numbers in bin 2, and so on sort by the next least significant digitcontinue this process until the numbers have been sorted on all k digits

Radix SortLeast-significant-digit-firstExample: 275, 087, 426, 061, 509, 170, 677, 503

Radix SortDoes it work?Clearly, if the most significant digit of a and b are different and a < b, then finally a comes before bIf the most significant digit of a and b are the same, and the second most significant digit of b is less than that of a, then b comes before a.

Radix SortExample 2: sorting cards2 digits for each card: d1d2d1 = : base 4      d2 = A, 2, 3, ...J, Q, K: base 13A  2  3  ...  J  Q  K 2  2  5  K

// base 10// FIFO// d times of counting sort// scan A[i], put into correct slot// re-order back to original array

Radix SortIncreasing the base r decreases the number of passesRunning timek passes over the numbers (i.e. k counting sorts, with range being 0..r)each pass takes O(N+r)total: O(Nk+rk)r and k are constants: O(N)

IntroductionFastest known sorting algorithm in practiceAverage case: O(N log N)Worst case: O(N2)But, the worst case seldom happens.Another divide-and-conquer recursive algorithm like mergesort

QuicksortSDivide step: Pick any element (pivot) v in S Partition S – {v} into two disjoint groups S1 = {x  S – {v} | x v} S2 = {x  S – {v} | x  v}Conquer step: recursively sort S1 and S2Combine step: combine the sorted S1, followed by v, followed by the sorted S2vvS1S2

Pseudocode Input: an array A[p, r]Quicksort (A, p, r) { if (p < r) { q = Partition (A, p, r) //q is the position of the pivot elementQuicksort (A, p, q-1)Quicksort (A, q+1, r) }}

PartitioningPartitioning Key step of quicksort algorithmGoal: given the picked pivot, partition the remaining elements into two smaller setsMany ways to implement Even the slightest deviations may cause surprisingly bad results.We will learn an easy and efficient partitioning strategy here.How to pick a pivot will be discussed later

Partitioning Strategy196jiWant to partition an array A[left .. right]First, get the pivot element out of the way by swapping it with the last element. (Swap pivot and A[right])Let i start at the first element and j start at the next-to-last element (i = left, j = right – 1)swap656431219564312pivot

Partitioning Strategy191966jjiiWant to haveA[p] <= pivot, for p < iA[p] >= pivot, for p > jWhen i < jMove i right, skipping over elements smaller than the pivotMove j left, skipping over elements greater than the pivotWhen both i and j have stoppedA[i] >= pivotA[j] <= pivot564312564312

Partitioning Strategy191966jjiiWhen i and j have stopped and i is to the left of jSwap A[i] and A[j]The large element is pushed to the right and the small element is pushed to the leftAfter swappingA[i] <= pivotA[j] >= pivotRepeat the process until i and j crossswap564312534612

Partitioning Strategy191919666jjjiiiWhen i and j have crossedSwap A[i] and pivotResult:A[p] <= pivot, for p < iA[p] >= pivot, for p > i534612534612534612

Small arraysFor very small arrays, quicksort does not perform as well as insertion sorthow small depends on many factors, such as the time spent making a recursive call, the compiler, etcDo not use quicksort recursively for small arraysInstead, use a sorting algorithm that is efficient for small arrays, such as insertion sort

Picking the PivotUse the first element as pivotif the input is random, okif the input is presorted (or in reverse order)all the elements go into S2 (or S1)this happens consistently throughout the recursive callsResults in O(n2) behavior (Analyze this case later)Choose the pivot randomlygenerally saferandom number generation can be expensive

Picking the PivotUse the median of the arrayPartitioning always cuts the array into roughly halfAn optimal quicksort (O(N log N))However, hard to find the exact mediane.g., sort an array to pick the value in the middle

Pivot: median of threeWe will use median of threeCompare just three elements: the leftmost, rightmost and centerSwap these elements if necessary so that A[left] = SmallestA[right] = LargestA[center] = Median of threePick A[center] as the pivotSwap A[center] and A[right – 1] so that pivot is at second last position (why?)median3

66521365261352135213pivotpivot19Pivot: median of threeA[left] = 2, A[center] = 13, A[right] = 66431219Swap A[center] and A[right]64312196431219Choose A[center] as pivotSwap pivot and A[right – 1]64312Note we only need to partition A[left + 1, …, right – 2]. Why?

Main Quicksort RoutineChoose pivotPartitioningRecursionFor small arrays

Partitioning PartWorks only if pivot is picked as median-of-three. A[left] <= pivot and A[right] >= pivotThus, only need to partition A[left + 1, …, right – 2]j will not run past the endbecause a[left] <= pivoti will not run past the endbecause a[right-1] = pivot

Quicksort Faster than MergesortBoth quicksort and mergesort take O(N log N) in the average case.Why is quicksort faster than mergesort?The inner loop consists of an increment/decrement (by 1, which is fast), a test and a jump. There is no extra juggling as in mergesort.inner loop

AnalysisAssumptions:A random pivot (no median-of-three partitioningNo cutoff for small arraysRunning timepivot selection: constant time O(1)partitioning: linear time O(N)running time of the two recursive calls T(N)=T(i)+T(N-i-1)+cN where c is a constanti: number of elements in S1

Worst-Case AnalysisWhat will be the worst case?The pivot is the smallest element, all the timePartition is always unbalanced

Best-case AnalysisWhat will be the best case?Partition is perfectly balanced.Pivot is always in the middle (median of the array)

Average-Case AnalysisAssumeEach of the sizes for S1 is equally likelyThis assumption is valid for our pivoting (median-of-three) and partitioning strategyOn average, the running time is O(N log N) (covered in comp271)

Topological Sort3680729154Topological sort is an algorithm for a directed acyclic graphIt can be thought of as a way to linearly order the vertices so that the linear order respects the ordering relations implied by the arcsFor example:0, 1, 2, 5, 90, 4, 5, 90, 6, 3, 7 ?

Topological SortIdea:Starting point must have zero indegree!If it doesn’t exist, the graph would not be acyclicA vertex with zero indegree is a task that can start right away. So we can output it first in the linear orderIf a vertex i is output, then its outgoing arcs (i, j) are no longer useful, since tasks j does not need to wait for i anymore- so remove all i’s outgoing arcsWith vertex i removed, the new graph is still a directed acyclic graph. So, repeat step 1-2 until no vertex is left.

Topological SortFind all starting pointsReduce indegree(w)Place new startvertices on the Q

ExampleIndegreestart3680729154Q = { 0 }OUTPUT: 0

ExampleIndegree-1368-107291-154Dequeue 0 Q = { } -> remove 0’s arcs – adjust indegrees of neighborsDecrement 0’sneighborsOUTPUT:

ExampleIndegree3680729154Dequeue 0 Q = { 6, 1, 4 } Enqueue all starting pointsEnqueue allnew start pointsOUTPUT: 0

ExampleIndegree3-168-1729154Dequeue 6 Q = { 1, 4 } Remove arcs .. Adjust indegrees of neighbors Adjust neighborsindegreeOUTPUT: 0 6

ExampleIndegree38729154Dequeue 6 Q = { 1, 4, 3 } Enqueue 3Enqueue newstartOUTPUT: 0 6

ExampleIndegree-138729154Dequeue 1 Q = { 4, 3 } Adjust indegrees of neighborsAdjust neighborsof 1OUTPUT: 0 6 1

ExampleIndegree3872954Dequeue 1 Q = { 4, 3, 2 } Enqueue 2Enqueue new starting pointsOUTPUT: 0 6 1

ExampleIndegree38729-154Dequeue 4 Q = { 3, 2 } Adjust indegrees of neighborsAdjust 4’s neighborsOUTPUT: 0 6 1 4

ExampleIndegree387295Dequeue 4 Q = { 3, 2 } No new start points foundNO new startpointsOUTPUT: 0 6 1 4

ExampleIndegree387295-1Dequeue 3 Q = { 2 } Adjust 3’s neighborsOUTPUT: 0 6 1 4 3

ExampleIndegree87295Dequeue 3 Q = { 2 } No new start points foundOUTPUT: 0 6 1 4 3

ExampleIndegree8729-1-15Dequeue 2 Q = { } Adjust 2’s neighborsOUTPUT: 0 6 1 4 3 2

ExampleIndegree8795Dequeue 2 Q = { 5, 7 } Enqueue 5, 7OUTPUT: 0 6 1 4 3 2

ExampleIndegree8795-1Dequeue 5 Q = { 7 } Adjust neighbors OUTPUT: 0 6 1 4 3 2 5

ExampleIndegree879Dequeue 5 Q = { 7 } No new starts OUTPUT: 0 6 1 4 3 2 5

ExampleIndegree879-1Dequeue 7 Q = { } Adjust neighbors OUTPUT: 0 6 1 4 3 2 5 7

ExampleIndegree89Dequeue 7 Q = { 8 } Enqueue 8 OUTPUT: 0 6 1 4 3 2 5 7

ExampleIndegree89-1Dequeue 8 Q = { } Adjust indegrees of neighborsOUTPUT: 0 6 1 4 3 2 5 7 8

ExampleIndegree9Dequeue 8 Q = { 9 } Enqueue 9Dequeue 9 Q = { } STOP – no neighborsOUTPUT: 0 6 1 4 3 2 5 7 8 9

Example3680729154OUTPUT: 0 6 1 4 3 2 5 7 8 9Is output topologically correct?

Topological Sort: ComplexityWe never visited a vertex more than one timeFor each vertex, we had to examine all outgoing edgesΣ outdegree(v) = mThis is summed over all vertices, not per vertexSo, our running time is exactlyO(n + m)

Sorting2

More Related Content

What's hot (20)

Similar to Sorting2 (20)

More from Saurabh Mishra (8)

Recently uploaded (20)

Sorting2