Probabilistic Method
Probabilistic Method
2 Linearity of Expectations
Linearity of expectations refers to the following basic fact about the expectation: given
random variables 𝑋1 , . . . , 𝑋𝑛 and constants 𝑐 1 , . . . , 𝑐 𝑛 ,
This identity does not require any assumption of independence. On the other hand,
generally E[𝑋𝑌 ] ≠ E[𝑋]E[𝑌 ] unless 𝑋 and 𝑌 are uncorrelated (independent random
variables are always uncorrelated).
Here is a simple application (there are also much more involved solutions via enumer-
ation methods).
Solution. Let 𝑋𝑖 be the event that element 𝑖 ∈ [𝑛] is fixed. Then E[𝑋𝑖 ] = 1/𝑛. The
expected number of fixed points is
17
MIT OCW: Probabilistic Methods in Combinatorics — Yufei Zhao
2 Linearity of Expectations
The minimization problem is easier. The transitive tournament (i.e., respecting a fixed
linear ordering of vertices) has exactly one Hamilton path. On the other hand, every
tournament has at least one Hamilton path (Exercise: prove this! Hint: consider a
longest directed path).
The maximization problem is more difficult and interesting. Here we have some
asymptotic results.
Proof. Consider a random tournament where every edge is given a random orientation
chosen uniformly and independently. Each of the 𝑛! permutations of vertices forms a
directed path with probability 2−𝑛+1 . So that expected number of Hamilton paths is
𝑛!2−𝑛+1 . Thus, there exists a tournament with at least this many Hamilton paths. □
This was considered the first use of the probabilistic method. Szele conjectured that the
maximum number of Hamilton paths in a tournament on 𝑛 players is 𝑛!/(2 − 𝑜(1)) 𝑛 .
This was proved by Alon (1990) using the Minc–Brégman theorem on permanents (we
will see this later in Chapter 10 on the entropy method).
where {·} denotes fractional part. Then 𝐴𝜃 is sum-free since (1/3, 2/3) is sum-free in
R/Z.
For 𝜃 uniformly chosen at random, {𝑎𝜃} is also uniformly random in [0, 1], so P(𝑎 ∈
𝐴𝜃 ) = 1/3. By linearity of expectations, E| 𝐴𝜃 | = 𝑛/3. □
18
MIT OCW: Probabilistic Methods in Combinatorics — Yufei Zhao
Remark 2.2.2 (Additional results). Alon and Kleitman (1990) noted that one can
improve the bound to ≥ (𝑛 + 1)/3 by noting that | 𝐴𝜃 | = 0 for 𝜃 close to zero (say,
|𝜃| < (3 max 𝑎 ∈ 𝐴 |𝑎|) −1 ), so that | 𝐴𝜃 | < 𝑛/3 with positive probability, and hence
| 𝐴𝜃 | > 𝑛/3 with positive probability. Note that since | 𝐴𝜃 | is an integer, being > 𝑛/3 is
the same as being ≥ (𝑛 + 1)/3.
Bourgain (1997) improved it to ≥ (𝑛 + 2)/3 via a difficult Fourier analytic argument.
This is currently the best lower bound known.
It remains an open problem to prove ≥ (𝑛 + 𝑓 (𝑛))/3 for some function 𝑓 (𝑛) → ∞.
In the other direction, Eberhard, Green, and Manners (2014) showed that there exist
𝑛-element sets of integers whose largest sum-free subset has size ≤ (1/3 + 𝑜(1))𝑛.
Taking the complement of a graph changes its independent sets to cliques and vice
versa. So the problem is equivalent to one about graphs without large independent
sets.
The following result, due to Caro (1979) and Wei (1981), shows that a graph with
small degrees much contain large independent sets. The probabilistic method proof
shown here is due to Alon and Spencer.
Proof. Consider a random ordering (permutation) of the vertices. Let 𝐼 be the set of
vertices that appear before all of its neighbors. Then 𝐼 is an independent set.
1
For each 𝑣 ∈ 𝑉, P(𝑣 ∈ 𝐼) = 1+𝑑 𝑣
(this is the probability that 𝑣 appears first among
{𝑣} ∪ 𝑁 (𝑣)). Thus E|𝐼 | = 𝑣∈𝑉 (𝐺) 𝑑 𝑣1+1 . Thus with positive probability, |𝐼 | is at least
Í
this expectation. □
19
MIT OCW: Probabilistic Methods in Combinatorics — Yufei Zhao
2 Linearity of Expectations
By taking the complement of the graph, independent sets become cliques, and so we
obtain the following corollary.
Corollary 2.3.5
Every 𝑛-vertex graph 𝐺 contains a clique of size at least
∑︁ 1
.
𝑛 − 𝑑𝑣
𝑣∈𝑉 (𝐺)
𝑇10,3 = 𝐾3,3,4 =
20
MIT OCW: Probabilistic Methods in Combinatorics — Yufei Zhao
2.4 Sampling
1 𝑛2
1− .
𝑟 2
Proof. Let 𝑚 be the number of edges. Since 𝐺 is 𝐾𝑟+1 -free, by Corollary 2.3.5, the
size 𝜔(𝐺) of the largest clique of 𝐺 satisfies
∑︁ 1 𝑛 𝑛
𝑟 ≥ 𝜔(𝐺) ≥ ≥ = .
𝑛 − 𝑑𝑣 𝑛 − 𝑛 𝑣 𝑑𝑣 𝑛 − 2𝑚
1 Í
𝑣∈𝑉 𝑛
1 𝑛2
Rearranging gives 𝑚 ≤ 1 − 𝑟 2. □
Remark 2.3.7. By a careful refinement of the above argument, we can deduce Turán’s
theorem that 𝑇𝑛,𝑟 maximizes the number of edges in a 𝑛-vertex 𝐾𝑟+1 -free graph, by
Í 1 Í
noting that 𝑣∈𝑉 𝑛−𝑑 𝑣
is minimized over fixed 𝑣 𝑑 𝑣 when the degrees are nearly equal.
Also, Theorem 2.3.6 is asymptotically tight in the sense that the Turán graph 𝑇𝑛,𝑟 , for
fixed 𝑟 and 𝑛 → ∞, as (1 − 1/𝑟 − 𝑜(1))𝑛2 /2 edges.
For more on this topic, see Chapter 1 of my textbook Graph Theory and Additive
Combinatorics and the class with the same title.
2.4 Sampling
By Turán’s theorem (actually Mantel’s theorem, in this case for triangles, the maximum
number of edges in an 𝑛-vertex triangle-free graph is 𝑛2 /4 .
How about the problem for hypergraphs? A tetrahedron, denoted 𝐾4(3) , is a complete 3-
uniform hypergraph (3-graph) on 4 vertices (think of the faces of a usual 3-dimensional
tetrahedron).
This turns out to be a notorious open problem. Turán conjectured that the answer is
5 𝑛
+ 𝑜(1) ,
9 3
21
MIT OCW: Probabilistic Methods in Combinatorics — Yufei Zhao
2 Linearity of Expectations
𝑉1
𝑉2 𝑉3
Above, the vertices are partitioned into three nearly equal sets 𝑉1 , 𝑉2 , 𝑉3 , and all the
edges come in two types: (i) with one vertex in each of the three parts, and (ii) two
vertices in 𝑉𝑖 and one vertex in 𝑉𝑖+1 , with the indices considered mod 3.
Let us give some easy upper bounds, in order to illustrate a simple yet important
technique of bounding by sampling.
Proof. Let 𝑆 be a 4-vertex subset chosen uniformly at random. If the graph has
𝑝 𝑛3 edges, then the expected number of edges induced by 𝑆 is 4𝑝 by linearity of
expectations (why?).
Since the 3-graph is tetrahedron-free, 𝑆 induces at most 3 edges. Therefore, 4𝑝 ≤ 3.
Thus the total number of edges is 𝑝 𝑛3 ≤ 34 𝑛3 .
□
Why stop at sampling four vertices? Can we do better by sampling five vertices? To
run the above argument, we will know how many edges can there be in a 5-vertex
tetrahedron-free graph.
Lemma 2.4.3
A 5-vertex tetrahedron-free 3-graph has at most 7 edges.
We can improve Proposition 2.4.2 by sampling 5 vertices instead of 4 in its proof. This
yields (check):
22
MIT OCW: Probabilistic Methods in Combinatorics — Yufei Zhao
Proposition 2.4.4
7 𝑛
Every tetrahedron-free 3-graph on 𝑛 ≥ 4 vertices has at most 10 3 edges.
By sampling 𝑠 vertices and using brute-force search to solve the 𝑠-vertex problem,
we can improve the upper bound by taking larger values of 𝑠. In fact in principle, if
we had unlimited computational power, we can arbitrarily close to optimum by taking
sufficiently large 𝑠 (why?). However, this is not a practical method due to the cost of
the brute-force search. There are more clever ways to get better bounds (also with the
help of a computer). The best known upper bound notably via a method known as flag
algebras (using sums of squares) due to Razborov, which can give ≤ (0.561 · · · ) 𝑛3 ).
For more on the Hypergraph Turán problem, see the survey by Keevash (2011).
Theorem 2.5.1
Let 𝑎𝑖 𝑗 ∈ {−1, 1} for all 𝑖, 𝑗 ∈ [𝑛]. There exists 𝑥𝑖 , 𝑦 𝑗 ∈ {−1, 1} for all 𝑖, 𝑗 ∈ [𝑛] such
that √︂ !
𝑛
∑︁ 2
𝑎 𝑖 𝑗 𝑥𝑖 𝑦 𝑗 ≥ + 𝑜(1) 𝑛3/2 .
𝑖, 𝑗=1
𝜋
23
MIT OCW: Probabilistic Methods in Combinatorics — Yufei Zhao
2 Linearity of Expectations
and set 𝑥𝑖 ∈ {−1, 1} to be the sign of 𝑅𝑖 (arbitrarily choose 𝑥𝑖 if 𝑅𝑖 = 0. Then the LHS
sum is
𝑛
∑︁ 𝑛
∑︁
𝑅𝑖 𝑥𝑖 = |𝑅𝑖 | .
𝑖=1 𝑖=1
For each 𝑖, 𝑅𝑖 has the same distribution as a sum of 𝑛 i.i.d. uniform {−1, 1}: 𝑆𝑛 =
𝜀1 + · · · + 𝜀 𝑛 (note that 𝑅𝑖 ’s are not independent for different 𝑖’s). Thus, for each 𝑖
√︂ !
2 √
E[|𝑅𝑖 |] = E[|𝑆𝑛 |] = + 𝑜(1) 𝑛,
𝜋
(one can also use binomial sum identities to compute exactly: E[|𝑆𝑛 |] = 𝑛21−𝑛 𝑛−1
⌊(𝑛−1)/2⌋ ,
though it is rather unnecessary to do so.) Thus
𝑛
√︂ !
∑︁ 2
E |𝑅𝑖 | = + 𝑜(1) 𝑛3/2 .
𝑖=1
𝜋
√︃
2
Thus with positive probability, the sum is ≥ 𝜋 + 𝑜(1) 𝑛3/2 . □
The next example is tricky. The proof will set up a probabilistic process where the
parameters are not given explicitly. A compactness argument will show that a good
choice of parameters exists.
Theorem 2.5.2
Let 𝑘 ≥ 2. Let 𝑉 = 𝑉1 ∪ · · · ∪ 𝑉𝑘 , where 𝑉1 , . . . , 𝑉𝑘 are disjoint sets of size 𝑛. The
edges of the complete 𝑘-uniform hypergraph on 𝑉 are colored with red/blue. Suppose
that every edge formed by taking one vertex from each 𝑉1 , . . . , 𝑉𝑘 is colored blue.
Then there exists 𝑆 ⊆ 𝑉 such that the number of red edges and blue edges in 𝑆 differ
by more than 𝑐 𝑘 𝑛 𝑘 , where 𝑐 𝑘 > 0 is a constant.
Proof. We will write this proof for 𝑘 = 3 for notational simplicity. The same proof
works for any 𝑘.
Let 𝑝 1 , 𝑝 2 , 𝑝 3 be real numbers to be decided. We are going to pick 𝑆 randomly by
24
MIT OCW: Probabilistic Methods in Combinatorics — Yufei Zhao
Then
E[#{blue edges in 𝑆} − #{red edges in 𝑆}]
equals to some polynomial
∑︁
𝑓 ( 𝑝1, 𝑝2, 𝑝3) = 𝑎𝑖, 𝑗,𝑘 𝑝𝑖 𝑝 𝑗 𝑝 𝑘 = 𝑛3 𝑝 1 𝑝 2 𝑝 3 + 𝑎 1,1,1 𝑝 31 + 𝑎 1,1,2 𝑝 21 𝑝 2 + · · · .
𝑖≤ 𝑗 ≤𝑘
Lemma 2.5.3
Let 𝑃 𝑘 denote the set of polynomials 𝑔( 𝑝 1 , . . . , 𝑝 𝑘 ) of degree 𝑘, whose coefficients
have absolute value ≤ 1, and the coefficient of 𝑝 1 𝑝 2 · · · 𝑝 𝑘 is 1. Then there is a
constant 𝑐 𝑘 > 0 such that for all 𝑔 ∈ 𝑃 𝑘 , there is some 𝑝 1 , . . . , 𝑝 𝑘 ∈ [0, 1] with
|𝑔( 𝑝 1 , . . . , 𝑝 𝑘 )| ≥ 𝑐.
Proof of Lemma. Set 𝑀 (𝑔) = sup 𝑝1 ,...,𝑝 𝑘 ∈[0,1] |𝑔( 𝑝 1 , . . . , 𝑝 𝑘 )| (note that sup is achieved
as max due to compactness). For 𝑔 ∈ 𝑃 𝑘 , since 𝑔 is nonzero (its coefficient of
𝑝 1 𝑝 2 · · · 𝑝 𝑘 is 1), we have 𝑀 (𝑔) > 0. As 𝑃 𝑘 is compact and 𝑀 : 𝑃 𝑘 → R is continuous,
𝑀 attains a minimum value 𝑐 = 𝑀 (𝑔) > 0 for some 𝑔 ∈ 𝑃 𝑘 . ■ □
25
MIT OCW: Probabilistic Methods in Combinatorics — Yufei Zhao
2 Linearity of Expectations
(It is not too hard to show that Wagner’s theorem and Kuratowski’s theorem are
equivalent)
If a graph has a lot of edges, is it guaranteed to have a lot of crossings no matter how
it is drawn in the plane?
Question 2.6.1
What is the minimum possible number of crossings that a drawing of:
• 𝐾𝑛 ? (Hill’s conjecture)
• 𝐾𝑛,𝑛 ? (Zarankiewicz conjecture; Turán’s brick factory problem)
• a graph on 𝑛 vertices and 𝑛2 /100 edges?
|𝐸 | 3
cr(𝐺) ≳ .
|𝑉 | 2
Corollary 2.6.4
In a graph 𝐺 = (𝑉, 𝐸), if |𝐸 | ≳ |𝑉 | 2 , then cr(𝐺) ≳ |𝑉 | 4 .
Proof. The proof has three steps, starting with some basic facts on planar graphs.
26
MIT OCW: Probabilistic Methods in Combinatorics — Yufei Zhao
Thus |𝐸 | ≤ 3|𝑉 | for all planar graphs. Hence cr(𝐺) > 0 whenever |𝐸 | > 3|𝑉 |.
Step 2: From one to many.
The above argument gives us one crossing. Next, we will use it to obtain many
crossings.
By deleting one edge for each crossing, we get a planar graph, so |𝐸 | − cr(𝐺) ≤ 3|𝑉 |,
that is
cr(𝐺) ≥ |𝐸 | − 3|𝑉 |.
This is a “cheap bound.” For graphs with |𝐸 | = Θ(𝑛2 ), this gives cr(𝐺) ≳ 𝑛2 . This is
not a great bound. We next will use the probabilistic method to boost this bound.
Step 3: Bootstrapping.
Let 𝑝 ∈ [0, 1] to be decided. Let 𝐺 ′ = (𝑉 ′, 𝐸 ′) be obtained from 𝐺 by randomly
keeping each vertex with probability 𝑝. Then
cr(𝐺 ′) ≥ |𝐸 ′ | − 3|𝑉 ′ |.
So
E cr(𝐺 ′) ≥ E|𝐸 ′ | − 3E|𝑉 ′ |
We have E cr(𝐺 ′) ≤ 𝑝 4 cr(𝐺), E|𝐸 ′ | = 𝑝 2 |𝐸 | and E|𝑉 ′ | = 𝑝E|𝑉 |. So
𝑝 4 cr(𝐺) ≥ 𝑝 2 |𝐸 | − 3𝑝|𝑉 |.
Thus
cr(𝐺) ≥ 𝑝 −2 |𝐸 | − 3𝑝 −3 |𝑉 |.
Setting 𝑝 = 4 |𝑉 | /|𝐸 | ∈ [0, 1] (here is where we use the hypothesis that |𝐸 | ≥ 4 |𝑉 |)
so that 4𝑝 −3 |𝑉 | = 𝑝 −2 |𝐸 |, we obtain cr(𝐺) ≳ |𝐸 | 3 /|𝑉 | 2 . □
Remark 2.6.5. The above idea of boosting a cheap bound to a better bound is an
important one. We saw a version of this idea in Section 2.4 where we sampled a
constant number of vertices to deduce upper bounds on the hypergraph Turán num-
ber. In the above crossing number inequality application, we are also applying some
preliminary cheap bound to some sampled induced subgraph, though this time the
sampled subgraph has super-constant size.
It is tempting to modify the proof by sampling edges instead of vertices, but this does
not work.
27
MIT OCW: Probabilistic Methods in Combinatorics — Yufei Zhao
2 Linearity of Expectations
Exercises
1. Let 𝐴 be a measurable subset of the unit sphere in R3 (centered at the origin)
containing no pair of orthogonal points.
a) Prove that 𝐴 occupies at most 1/3 of the sphere in terms of surface area.
b) ★ Prove an upper bound smaller than 1/3 (give your best bound).
2. ★ Prove that every set of 10 points in the plane can be covered by a union of
disjoint unit disks.
3. Let r = (𝑟 1 , . . . , 𝑟 𝑘 ) be a vector of nonzero integers whose sum is nonzero.
Prove that there exists a real 𝑐 > 0 (depending on r only) such that the following
holds: for every finite set 𝐴 of nonzero reals, there exists a subset 𝐵 ⊆ 𝐴 with
|𝐵| ≥ 𝑐| 𝐴| such that there do not exist 𝑏 1 , . . . , 𝑏 𝑘 ∈ 𝐵 with 𝑟 1 𝑏 1 + · · · + 𝑟 𝑘 𝑏 𝑘 = 0.
4. Prove that every set 𝐴 of 𝑛 nonzero integers contains two disjoint subsets 𝐵1 and
𝐵2 , such that both 𝐵1 and 𝐵2 are sum-free, and |𝐵1 | + |𝐵2 | > 2𝑛/3.
5. Let 𝐺 be an 𝑛-vertex graph with 𝑝𝑛2 edges, with 𝑛 ≥ 10 and 𝑝 ≥ 10/𝑛.
Prove that 𝐺 contains a pair of vertex-disjoint and isomorphic subgraphs (not
necessarily induced) each with at least 𝑐 𝑝 2 𝑛2 edges, where 𝑐 > 0 is a constant.
6. ★ Prove that for every positive integer 𝑟, there exists an integer 𝐾 such that the
following holds. Let 𝑆 be a set of 𝑟 𝑘 points evenly spaced on a circle. If we
partition 𝑆 = 𝑆1 ∪ · · · ∪ 𝑆𝑟 so that |𝑆𝑖 | = 𝑘 for each 𝑖, then, provided 𝑘 ≥ 𝐾,
there exist 𝑟 congruent triangles where the vertices of the 𝑖-th triangle lie in 𝑆𝑖 ,
for each 1 ≤ 𝑖 ≤ 𝑟.
7. ★ Prove that [𝑛] 𝑑 cannot be partitioned into fewer than 2𝑑 sets each of the form
𝐴1 × · · · × 𝐴𝑑 where 𝐴𝑖 ⊊ [𝑛].
28
MIT OpenCourseWare
https://ocw.mit.edu
For information about citing these materials or our Terms of Use, visit: https://ocw.mit.edu/terms.