Yao’s minimax principle

 Game Tree Evaluation

1
Yao’s Minimax Principle
 Yao’s Minimax Principle

Assume
• Consider a problem over the inputs X and let A be the set of all
possible deterministic algorithms that correctly solve the problem.
• For any algorithm 𝑎 ∈ 𝐴 and input 𝑥 ∈ 𝑋, let c(a,x) be the cost of
algorithm a running on x.
• Let p be be a probability distributions over the algorithms A and let
A’ denote a random algorithm chosen according to p.
• Let q be a probability distribution over the inputs X and let X’
denote a random input chosen according to q.

Then

max 𝐸(𝑐 𝐴′, 𝑥 ) ≥ min 𝐸(𝑐 𝑎, 𝑋′ )

𝑥∈𝑋 𝑎∈𝐴

3
 Yao’s Minimax Principle

Pf.
• Let 𝐶 = max 𝐸(𝑐 𝐴′, 𝑥 ).
𝑥∈𝑋
• For every input x, we have 𝑎 𝑝𝑎 𝑐(𝑎, 𝑥) ≤ 𝐶, Therefore,

𝑞𝑥 𝑝𝑎 𝑐(𝑎, 𝑥) ≤ 𝑞𝑥 𝐶 = 𝐶 𝑞𝑥 = 𝐶
𝑥 𝑥 𝑥
𝑎

• we can switch the order of summation, giving

𝑝𝑎 𝑞𝑥 𝑐(𝑎, 𝑥) ≤ 𝐶
𝑎
𝑥

• By the Pigeonhole principle, there must exist an algorithm a so that

𝑞𝑥 𝑐(𝑎, 𝑥) ≤ 𝐶
𝑥

4
 Lower Bound For Sorting

Problem. The expected of #comparisons by any randomized sorting

algorithm is Ω(𝑛 log 𝑛).

• Any randomized sorting algorithm S can be seen as a set A of all

possible deterministic algorithms that correctly sort the input
together with a probability distributions over A.
• Let c(a,x) be #comparisons made by algorithm a over input x.
• Let X be the set of inputs, each containing n numbers. let X’ denote
a random input chosen uniformly at random. Therefore,

max 𝐸(𝑐 𝑆, 𝑥 ) ≥ min 𝐸 𝑐 𝑎, 𝑋 ′ = Ω(𝑛 log 𝑛)

𝑥∈𝑋 𝑎∈𝐴

5
Game Tree Evaluation
 Definition of Game Tree

• A Game Tree Td,k is uniform tree in which the root and the internal
nodes has d children and every leaf is at distance 2k from the root.

• Internal nodes at even distance from the root are labeled MIN and
at odd distance are labeled MAX.

• Each leaf is associated with a value.

 Example of a Game Tree T2,2

MIN

MAX MAX

MIN MIN MIN MIN

MAX MAX MAX MAX MAX MAX MAX MAX

0 1 0 0 0 1 0 1 0 1 0 0 0 1 0 1
 Observations

• Every root-to-leaf path goes through the same number of MIN and
MAX nodes (including the root)

• If the depth of the tree is 2k, there are 22k = 4k leaves

 Game Tree Evaluation

MIN (AND) Node

returns the lesser of the two children

0 0 0 1

MIN MIN MIN MIN

0 0 0 1 1 0 1 1
 Game Tree Evaluation

MAX (OR) Node

returns the greater of the two children

0 1 1 1

MAX MAX MAX MAX

0 0 0 1 1 0 1 1
 Returned Value by the Root?

AND

OR OR

AND AND AND AND

OR OR OR OR OR OR OR OR

0 1 0 0 0 1 0 1 0 1 0 0 0 1 0 1
 A Deterministic Algorithm

Depth-first manner
always visit the left child before the right child

AND AND

OR OR OR OR

0 1 0 1 1 0 1 0

worst case – need to visit ALL 4k leaves a better case – visit 2 leaves is enough
 A Randomized Algorithm

Coin toss
 0.5 probability choosing the left child and 0.5 probability choosing the
right child

AND

OR OR

0 1 0 1

Expected cost (number of leaves visited)  3

 Design Rationale

Suppose AND node were to return 0

 at least one of the leaves is 0
 if deterministic algorithm is used, your opponent can always “hide” this
0 and make your algorithm visit both leaves
 if randomized algorithm is used, you foils your opponent’s strategy. The
expected number of steps (leaf visits) is 3

Similar for OR node were to return 1

 Design Rationale

Expected cost
 EAND_0 = EOR_1 = 3

What if AND(OR) node were to return 1(0)?

both children are 1(0), it seems that the randomized algorithm
doesn’t improve much since we need to visit both children anyway
however, it benefits the parent level
 Analysis of the Randomized Algorithm

Theorem. The expected cost of the randomized algorithm for

evaluating any T2,k game tree is at most 3k

Proof by induction:
 consider k = 1
 expected cost  3
 Analysis of the Randomized Algorithm

Case I – root evaluated to 0

 at least one of the subtrees (OR nodes) gives 0
 you have 0.5 probability that this particular
node is checked first
return 0
 E(T) = ½  2 + ½  (3/2 + 2)
= 2.75

AND

OR OR

0 0 0 1
 Analysis of the Randomized Algorithm

Case II – root evaluated to 1

 both subtrees give 1
 E(T) = 2  3/2
=3
return 1

Both cases give  3 expected cost, so the theorem

is true for k=1
AND

OR OR

0 1 0 1
 Analysis of the Randomized Algorithm

Assume that for all T2,k-1, the expected cost  3k-1

First, consider the OR-root tree

either gives 1 or 0

OR

T2,k-1 T2,k-1
 Analysis of the Randomized Algorithm

Case I: OR-root gives 1

 at least one subtree gives 1
 0.5 probability we use it first
 E(T)  ½  3k-1 + ½  2  3k-1
= 3/2  3k-1

either 1 or 0
Case II: OR-root gives 0
 both subtrees give 0
 E(T)  2  3k-1
OR

T2,k-1 T2,k-1
 Analysis of the Randomized Algorithm

Now, consider the AND-root game tree, T2,k

OR AND
T2,k

T2,k-1 T2,k-1

OR OR

T2,k-1 T2,k-1 T2,k-1 T2,k-1

 Analysis of the Randomized Algorithm

Case I: AND-root gives 0

 at least one subtree gives 0
 0.5 probability we use it first
 E(T2,k)  ½  2  3k-1
either 1 or 0
+ ½  (3/2  3k-1 + 2  3k-1)
= 2.75  3k-1  3k

AND
Case II: AND-root gives 1
 both subtrees give 1
 E(T2,k)  2  3/2  3k-1
= 3k OR OR

T2,k-1 T2,k-1 T2,k-1 T2,k-1

 Analysis of the Randomized Algorithm

Proved the theorem:

• The expected cost of the randomized algorithm for evaluating any T2,k
game tree is at most 3k

Remark. A tree has n = 4k leaves, then k = log4 n. Substitute log4 n for k

in the expected cost, then the cost  3log4n. By xlogab = blogax, the cost 
nlog43 = n0.793
 Lower Bound

we will use Yao’s Minimax Principle and show a lower bound of n0.694

• choose a random input obtained by giving each leave value 1 independently

with probability 𝑝 = (3 − 5)/2.
• replace all AND- and OR-nodes by NOR-nodes. A NOR-node has value 1 if
and only if both its children have value 0.
• Exercise: the value of a T2,k is equal to the value of a binary tree all of
whose nodes are NOR-nodes.
• The probability that a NOR-node whose two children are leaves, gets
value 1 is the probability that both leaves have value 0, which is
2 2
3− 5 5−1 3− 5
1− = = =𝑝
2 2 2

• This implies that every NOR-node in the tree has value 1 with probability
p, and independently from all other nodes at the same level.
 Replacing AND and OR gates with NOR gates

AND NOR
OR OR NOR NOR

AND AND AND AND NOR NOR NOR NOR

OR OR OR OR OR OR OR OR NOR NOR NOR NOR NOR NOR NOR NOR

0 1 0 0 0 1 0 1 0 1 0 0 0 1 0 1 0 1 0 0 0 1 0 1 0 1 0 0 0 1 0 1
 Lower Bound

Theorem. Let T be a NOR-tree each of whose leaf-values are set equal to 1

with probability q, independently, for some fixed q. Let W(T) be the
minimum expected number of leaves to be read for evaluating T over all
deterministic algorithms. Then there exists a depth-first pruning algorithm
having W(T) as the expected number of leaves to be read.

Depth-first pruning algorithm is an algorithm that for every node, starting

at the root, tries to evaluate one if its children first, before doing anything
else. It may happen that the child has value 1 in which case the other child
does not need to be considered anymore; the other child, i.e., the subtree
having this child as root is then pruned.

The above theorem says instead of considering all deterministic algorithm,

we can only consider depth-first pruning algorithms in Yao’s minimax
principle.
 Lower Bound

We use Yao’s minimax principle and compute min 𝐸(𝑐 𝑎, 𝑋 ) where X is a random
𝑎∈𝐴
input where each leave is 1 with probability p and c(a,X) is the number of leaves
read by first-depth pruning algorithm a.

• Let W(h) denote the expected number of leaves to be read for evaluating a
node at distance h from the leaves. Then

𝑊 ℎ = 𝑝𝑊 ℎ − 1 + 1 − 𝑝 𝑊 ℎ − 1 + 𝑊 ℎ − 1 = 𝑊 ℎ − 1 + 1 − 𝑝 W(h − 1)

• The above comes from the fact that one of the two children has to be
evaluated for sure and with probability (1−p) it gets value 0, in which case
also the second child needs to be evaluated.
• Substituting h by log2 n, we get 𝑊(ℎ) = 𝑛0.694
References
 References

 Some parts are form here.

30