Scribd
Upload
55 views2 pages

Philosophy of Analyzing Randomized Algorithms

This document discusses analyzing randomized algorithms. It introduces the concept of identifying a "bad event" when analyzing an algorithm. As an example, it analyzes the BlindMatching algorithm for finding a perfect matching in a bipartite graph. It then discusses the techniques of union bounding and amplification to improve the analysis. Finally, it covers Markov's and Chebyshev's inequalities and using them to analyze estimating an unknown probability from samples.

Uploaded by

Karin Angelis
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
55 views2 pages

Philosophy of Analyzing Randomized Algorithms

This document discusses analyzing randomized algorithms. It introduces the concept of identifying a "bad event" when analyzing an algorithm. As an example, it analyzes the BlindMatching algorithm for finding a perfect matching in a bipartite graph. It then discusses the techniques of union bounding and amplification to improve the analysis. Finally, it covers Markov's and Chebyshev's inequalities and using them to analyze estimating an unknown probability from samples.

Uploaded by

Karin Angelis
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

CS 170, Spring 2020 DIS 13 A. Chiesa & J.

Nelson

Note: Your TA may not get to all the problems. This is totally fine, the discussion worksheets are
not designed to be finished in an hour. The discussion worksheet is also a resource you can use to
practice, reinforce, and build upon concepts discussed in lecture, readings, and the homework.

Philosophy of analyzing randomized algorithms. The first step is to always identify a bad
event. I.e. identify when your randomness makes your algorithm fail. We will review some techniques
from class using the following problem as our “test bed”.
Let G be a bipartite graph with n left vertices, and n right vertices on n2 − n + 1 edges.
• Prove that G always has a perfect matching.
• Give a polynomial in n time algorithm to find this perfect matching.
We will analyze the following algorithm BlindMatching:
• Let π and σ be independent and uniformly random permutations of [n].
• If {π(1), σ(1)}, {π(2), σ(2)}, . . . , {π(n), σ(n)} is a valid matching output it.
• Else output failed.

Union Bound. Suppose X 1 , . . . , X n are (not necessarily independent) {0, 1} valued random vari-
ables, then

Pr[X 1 + · · · + X n ≥ 1] ≤ Pr[X 1 = 1] + Pr[X 2 = 1] + · · · + Pr[X n = 1].

Now we analyze our algorithm using union bound. An output M = ({π(1), σ(1)}, . . . , {π(n), σ(n)})
is a valid perfect matching exactly when all edges of the form {π(i), σ(i)} are present in G. A “bad
event” happens if any of those pairs are not edges in G.
Let X i be the indicator of the event that {π(i), σ(i)} is not present in our graph.
1. What is the probability that X i = 1?
2. Use the union bound to upper bound the probability that M is not a valid perfect matching.
3. Conclude that G has a valid perfect matching.
The upper bound obtained on the probability of our bad event, i.e. of M not being a valid perfect
matching, is fairly high. In light of this, we introduce the technique of amplification.

Amplification. The philosophy of amplification is that if we have a randomized algorithm that


fails with probability p, we can repeat the algorithm many times and aggregate the output of all
the runs to produce a new output such that the failure probability of the randomized algorithm is
significantly smaller. Now consider the following algorithm SpamBlindMatching.
• Run BlindMatching independently T times.
• If at least one of the runs outputted a valid perfect matching, return the output of such a run.
• Else output failed.

1. What is the failure probability of SpamBlindMatching?


2. How large should we set T if we want a failure probability of δ?

Now we switch gears and turn our attention to concentration phenomena and its usefulness in
analyzing randomized algorithms.

1
CS 170, Spring 2020 DIS 13 A. Chiesa & J. Nelson

Markov’s inequality. Let X be a nonnegative valued random variable, then for every t ≥ 0:
1
Pr[X ≥ tE[X]] ≤ .
t
1. Markov’s inequality is false for random variables that can take on negative values! Give an
example.
2. Give a tight example for Markov’s inequality. In particular, given µ and t, construct a random
variable X such that µ = E[X] and Pr[X ≥ tµ] = 1t .

Chebyshev’s inequality. Let X be any random variable with well-defined variance1 , then
h p i 1
Pr |X − E[X]| > t Var[X] ≤ 2 .
t
To see the above inequality in action, consider the following problem:
Let B be a bag with n balls, k of which are red and n−k of which are blue. We do not have
knowledge of k and wish to estimate k from ` independent samples (with replacement)
drawn from B.
Let X be the number of red balls sampled.
1. What is E[X]?
2. What is Var[X]?
3. Choose a value√for ` and√give an algorithm that takes in n and X and outputs a number e
k such
that k ∈ [k − ε k, k + ε k] with probability at least 1 − δ.
e
Solution:
1. The probability that a random (u, v) pair is not an edge where u is a left vertex and v is a right
vertex is n1 − n12 .
2. By union bounding over all n edges chosen, the probability that M is not a perfect matching is
at most 1 − n1 .
1
3. The previous part implies that M has at least n probability of being a perfect matching, which
means a perfect matching exists in G.
1 T

4. The failure probability of SpamBlindMatching is bounded by 1 − n .
n
5. Setting T = n ln(1/δ) works because 1 − n1 ≤ 1e .
6. Uniform ±1 has expected value 0 but half chance of exceeding 0.
7. Consider the random variable that is tµ with probability 1/t and 0 with probability (t − 1)/t.
k
8. E[X] = n `.
9. Defining X i as the random variable that the i-th sample is red, and using independence of the
X i we have
 
k k
Var[X] = Var[X 1 + · · · + X ` ] = Var[X 1 ] + · · · + Var[X ` ] = ` 1− .
n n

10. The algorithm is to output n` X. E n` X = k and Var n` X = kn 1 − nk ≤ kn


    
q ` ` . Thisq quantity
1 kn 1 kn
deviates from k by √δ ` with probability at most δ. We wish to choose ` so that δ
√
` < ε.
n
This happens when ` = ε2 δ .

1
In this course, all random variables will have well-defined variance

You might also like