Randomized Algorithms CSE 312 Spring 21

Lecture 28
Announcements
Three Concept checks this week (to get to a total of 30)
Concept Check 28 is a “standard” one for today’s lecture
Concept Check 29 asks you to fill out the official UW course evals.
Please fill these out! It’s super useful to know what you found valuable, what you found
frustrating, what took too long,…we are always working to improve the course.
Concept Check 30 asks you to fill out a different anonymous form with
specific feedback about the real world assignments.
They were new this quarter – asking about logistics (e.g., would you rather
have had these as part of regular homeworks like the programming or as
separate work like we did?), for any examples to add to suggested lists (like
in real world 1/2), etc.
Final Logistics
What’s Fair Game?
All the content/theorems we learned.
Principles behind the programming questions, but not the code itself.
Applications lectures won’t be tested directly (that is to say we won’t give you
a problem that would only make sense if you followed the application lectures.
But the principles behind the applications are absolutely fair game.
And we reserve the right to give a problem inspired by the applications
lectures, as long as you could do it even if you didn’t see the applications.
E.g. we could ask a question about covariance (because you know the formula
from main lectures) but we wouldn’t ask you to think about the ML
implications of a picture of covariances.
What’s a randomized algorithm?
A randomized algorithm is an algorithm that uses randomness in the
computation.
Well, ok.
Let’s get a little more specific.
Two common types of algorithms
Las Vegas Algorithm
Always tells you the right answer
Takes varying amounts of time.

Monte Carlo Algorithm

Usually tells you the right answer.
A classic Las Vegas Algorithm
Remember Quick Sort?
Pick a “pivot” element
Move all the elements smaller than the pivot to the left subarray (in no particular order)
Move all elements greater than the pivot element to the right subarray (in no particular
order)
Make a recursive call

It’s sometimes implemented as a Las Vegas Algorithm.

That is, you’ll always get the same answer (there’s only one sorted array) but
the time can vary.
 https://www.youtube.com/watch?v=ywWBy6J5gz8

Quick Sort 0 1 2 3 4 5 6
20 50 70 10 60 40 30
0 0 1 2 3 4
10 50 70 60 40 30
0 1 0 1
40 30 70 60
0 0
30 60

0 1 0 1
30 40 60 70

0 1 2 3 4
30 40 50 60 70

0 1 2 3 4 5 6
10 20 30 40 50 60 70
 Total time:
How long does it take?
Well…it depends on what pivots you pick.

About
levels

…
work when elements remaining.
For Simplicity
We’ll talk about how quicksort is really done at the end.
For now an easier-to-analyze version:

if(elements remaining > 1)

pick a pivot uniformly at random
split based on pivot
sortedLeft = QuickSort(left half)
sortedRight = QuickSort(right half)
return (sortedLeft pivot sortedRight)
What leads to a good time?
Pivots closer to the middle would be better.

Cut in
half is
ideal.
levels.

work when elements remaining. -- per level

Work at each level
How much work do those splits do?
Each call choose a pivot ( total per level)
Each element is compared to the pivot and possibly swapped ( per element
so per level)
So as long as we need at most levels, we’ll get the running time we need.

We only get the perfect pivot with probability . That’s not very likely…
maybe we can settle for something more likely.
Focus on an element
Let’s focus on one element of the array .
The recursion will stop when every element is all alone in their own
subarray.
Call an iteration “good for ” if the array containing in the next step is at
most the size it was in the current step.

Pivot here: might leave in a Pivot here: both subarrays size. Pivot here: might leave in a
big subarray (if is big) Must be good for . big subarray (if is small)
Good for
At least half of the potential pivots guarantee ends up with a good iteration.
So we’ll use

It’s actually quite a bit more than half for large arrays – one of the two red
subarrays might be good for (just bad for the others in the array)
might be our pivot, in which case it’s totally done.

To avoid any tedious special cases for small arrays, just say at least .
How many levels?
How many levels do we need to go?
Once is in a size subarray, it’s done. How many iterations does it take?
If we only had good iterations, we’d need
.
I want (at the end of our process) to say with probability at least <blah> the
running time is at most .
What’s the probability of getting a lot of good iterations…what’s the tool we
should use?
Fill out the poll everywhere so Robbie
knows how long to explain
Go to pollev.com/cse312
Needed iterations
is done after good for iterations.
Let’s imagine we do iterations. Let be the number of good for iterations. Let

Set up for Chernoff

Applying Chernoff

So, the probability that is not done after iterations is at most

Finishing The bound
So is done with probability at least
But being done doesn’t mean the whole algorithm is done…

This argument so far does apply to any other -- but they aren’t independent,
so….union bound!

algorithm not done) done) done)

algorithm done).
The Theorem
Quicksort
With probability at least , Quicksort runs in time

This kind of bound (with probability as is called a “high probability bound”

we say quicksort needs time “with high probability”
Better than finding a bound on the expected running time!
Want a different bound?
Want an even better probability? You just have to tweak the constant factors!
Be more careful in defining a “good iteration” or just change to or .
It all ends up hidden in the big-O anyway.

That’s the power of concentration – the constant coefficient affects the

exponent of the probability.
Common Quicksort Implementations
A common strategy in practice is the “median of three” rule.
Choose three elements (either at random or from specific spots). Take the
median of those for your pivot
Guarantees you don’t have the worst possible pivot.
Only a small constant number of extra steps beyond the fixed pivot (find the
median of three numbers is just a few comparisons).

Another strategy: find the true median (very fancy, very impractical: take
421)
Algorithms with some probability of failure
There are also algorithms that sometimes give us the wrong answer. (Monte
Carlo Algorithms)
Wait why would we accept a probability of failure?

Suppose your algorithm succeeds with probability only .

But given two runs of the algorithm, you can tell which is better.
E.g. “find the biggest <blah>” – whichever is bigger is the better one.

How many independent runs of the algorithm do we need to get the right
answer with high probability?
Small Probability of Failure
How many independent runs of the algorithm do we need to get the right
answer with high probability?
Probability of failure

Choose , and we get high probability of success.

So (for example) independent runs gives you the right answer with high
probability.
Even with very small chance of success, a moderately larger number of
iterations gives high probability of success. Not a guarantee, but close
enough to a guarantee for most purposes.