15-780 – Numerical Optimization

J. Zico Kolter

April 23, 2014

1
 Outline

Introduction: Some (possibly) familiar games

Basic game theory

Computing equilibria

Special cases and extensions

2
 Prisoner’s dilemma
• Two prisoners being interrogated, can either stay silent or
implicate the other one

• If both stay silent, each sentenced to a year in jail; if only one

implicates another, he goes free and other gets 5 years in jail; if
both implicate each other, both get 3 years
Silent Implicate

Silent -1,-1 -5,0

Implicate 0,-5 -3,-3

• Even though Silent/Silent is best for both, each one strictly

benefits from implicating the other, regardless of other’s actions
3
 Guess 2/3 the mean

• All of you will play the game

• Pick a number between 1 and 10 (inclusive)

• The student whose number is closest to 2/3 of the mean of all

the guesses wins (and breaking ties randomly)

4
 General ideas

• Both these games differ slightly from what we have seen so far
in class: in order to decide our action we need to account for
other agents that are also acting (and trying to account for our
actions, ad infinitum)

• We focus here on the special cases of noncooperative game

theory and games in normal form

– Non-cooperative doesn’t mean that agents don’t cooperate, just

that they are self-interested

– Normal form here just means “one-shot” games, as opposed to

turn-based games

5
 Outline

Introduction: Some (possibly) familiar games

Basic game theory

Computing equilibria

Special cases and extensions

6
 Games in normal form

• A normal form game is defined by (N, A, u), where

– N is a number of players, each indexed by i

– A = A1 × A2 × . . . × An is a set of actions, where each Ai is a

finite set of actions available to player i

– u : A → Rn is a utility function that maps each set of actions

a ∈ A to a set of N utilities, one for each agent; i.e., ui (a)
denotes the utility of agent i for the actions a

7
• Example: Prisoner’s dilemma
Silent Implicate

Silent -1,-1 -5,0

Implicate 0,-5 -3,-3

– N =2

– A = {Silent, Implicate} × {Silent, Implicate}

 
−1

 if a = (Silent, Silent)
−1



  
−5




 if a = (Silent, Implicate)
0

– u(a) =  
 0

 if a = (Implicate, Silent)



 −5
 
−3


if a = (Implicate, Implicate)


8
−3

• Example: Guess 2/3 the mean

– N arbitrary

– A = {1, 2, . . . , 10}N

( 1
2 
if ai = 3 mean(a)
k=1 1{ak =[ 3 mean(a)]}
Pm 2
– ui (a) =
0 otherwise

– Note that utilities here refer to expected utilities: although we

are not guaranteed to win if we pick the average, we still have a
chance proportional to one over the number of others who pick
the same mean

9
• Battle of the sexes

– Husband and wife planning movie for the evening: husband

wants to see Wondrous Love (WL) wife wants to see Lethal
Weapon (LW)

– Different utilities for each movie, but both equally unhappy if

they end up seeing different movies

WL LW

WL 2,1 0,0

LW 0,0 1,2

– Like prisoner’s dilemma, points where there is no incentive for

either player to deviate, but here there are two such points

10
• Rock, paper, scissors

– Rock beats scissors, scissors beats paper, paper beats rock

Rock Paper Scissors

Rock 0,0 -1,1 1,-1

Paper 1,-1 0,0 -1,1

Scissors -1,1 1,-1 0,0

– If we play a fixed strategy, other player will always be able to

beat us

11
• Some special cases:
– Zero-sum game: two player game where u1 (a) = −u2 (a),
∀a ∈ A (e.g., rock paper scissors)

– Coordination game: payoffs for all players are the same

ui (a) = uj (a), ∀i, j ∈ {1, . . . , N }, a ∈ A

L R

L 1,1 -1,-1

R -1,-1 1,1

12
 Pure and mixed strategies

• A strategy for player i, denoted si : Ai → [0, 1] is a probability

distribution over actions: si (ai ) denotes probability that player i
takes action action ai (think of si as a vector in [0, 1]|Ai | that
must sum to one)

• A strategy profile s is a set of strategies for each player

s = (s1 , . . . , sN )

• The support of a strategy si is the set of actions that have

non-zero probability

• Strategy with a support of size one is called pure strategy,

otherwise mixed strategy

13
• The probability of set of actions a under strategy profile s is
N
Y
s(a) = si (ai )
i=1

i.e., the actions are all chosen independently

• The expected utility for a strategy profile s is given by

X
u(s) = u(a)s(a)
a∈A

• This can also be written elementwise, for example

X N
Y
ui (s) = ui (a) sj (aj )
a∈A j=1

14
 Best response

• Best response refers to a best (potentially mixed) strategy that

a player can play given the strategies of all opponents

• Define s−i be be strategy profile s omitting the strategy of the

ith player

• Formally, best response for player i given strategy profile s−i is a

strategy s?i such that u(s?i , s−i ) ≥ u(si , s−i ) for all strategies si

• Of course, in general we don’t know the strategies of the other

opponents

15
 Nash equilibrium
• Key definition: a strategy profile s is a Nash equilibrium if si is
a best response to s−i for all i = 1, . . . , N

• Intuitively corresponds to a “rational” set of strategies for each

agent: no agent gains an advantage by switching their strategy

• Can be one or more Nash equilibria for a game

• Strict Nash if for all i and s0i 6= si

u(si , s−i ) > u(s0i , s−i )
i.e., si is strictly preferable to all other strategies

• Weak Nash otherwise, i.e., can have s0i such that

u(si , s−i ) = u(s0i , s−i )
16
• Prisoner’s dilemma

Silent Implicate

Silent -1,-1 -5,0

Implicate 0,-5 -3,-3

What are NE? Are they strict?

17
• Rock paper scissors

Rock Paper Scissors

Rock 0,0 -1,1 1,-1

Paper 1,-1 0,0 -1,1

Scissors -1,1 1,-1 0,0

What are NE? Are they strict?

18
• Battle of sexes
WL LW

WL 2,1 0,0

LW 0,0 1,2

What are NE? Are they strict?

19
• In 1950, John Nash proved that
every game has at least one
equilibrium point (important,
requires mixed strategies)

• 27 pages, typeset like on the right

(probably about 5 pages in dense
latex, the same as your class
project writeup ... hmmm); two
references, one to his own paper

• Work won the Nobel prize in

economics

20
 Outline

Introduction: Some (possibly) familiar games

Basic game theory

Computing equilibria

Special cases and extensions

21
 Can we compute Nash equilibria?

• Since this is a computer science course after all...

• How do we actually compute the Nash equilibria of a game (for

now, let’s just consider two-player games)?

• In 2005, shown to be a PPAD-complete problem (not quite like

NP, since every game has a Nash equilibrium, but main intuition
is similar, thought to require solvable exponential time in game
size in the worst case)

• But “hard” problems don’t faze us in this course (see search,

mixed integer programming, etc)

22
Computing an equilibrium with known support

• If we just want to look at pure strategies (again,in two player

case) this is easy: just check all |A1 | × |A2 | possible strategies

• But, a game may not have a pure strategy equilibrium

• Key idea: For a given support, we can compute NE (if one

exists) by solving a set of linear equations

• Thus, problem really becomes one of searching for the correct

support

23
• Battle of sexes, let’s guess that the support for a mixed strategy
contains both WL and LW for husband and wife, and say that
husband chooses WL with probability p
WL LW

WL 2,1 0,0

LW 0,0 1,2

• Key idea: In order for strategy to be a NE, wife must be

indifferent between alternatives
u2 (WL) = u2 (LW)
p · 1 + (1 − p) · 0 = p · 0 + (1 − p) · 2
p = 2/3

• So s1 = (2/3, 1/3), s2 = (1/3, 2/3) is mixed strategy NE

24
• The general case

– Hypothesize some supports A1 ⊆ A1 , A2 ⊆ A2 for players

– Utilities for all actions in support must be equal for both players

u1 (a) = u1 (a0 ), ∀a, a0 ∈ A1 =⇒ |A1 | − 1 linear equations

u2 (a) = u2 (a0 ), ∀a, a0 ∈ A2 =⇒ |A2 | − 1 linear equations

X X
s1 (a) = 1, s2 (a) = 1 =⇒ |A2 | − 1 linear equations
a∈A1 a∈A2

Variables s1 (a), ∀a ∈ A1 , s2 (a), ∀a ∈ A2 , =⇒ |A1 |+|A2 | unknowns

25
• What happens when we try a support that does not produce a
NE?

• Prisoner’s dilemma, mixed strategy with full support (S,I)

Silent Implicate

Silent -1,-1 -5,0

Implicate 0,-5 -3,-3

26
 Finding the support

• But, 2|Ai | − 1 possible supports for each player

• Do we have to try them all? In the worst case, yes (unless

PPAD = P)

• But, many times we will find a solution much faster (c.f. search,
mixed integer programming, etc)

• In fact, a procedure that looks a lot like local hill-climbing

search is guaranteed to find a solution for the two-player case

27
• Lemke-Howson algorithm (stated very imprecisely)

• Start with some initial support A1 , A2 and repeat:

1. Choose (according to a specific rule), to add, drop, or swap
action from support

2. Solve resulting linear systems, if they are consistent with mixed

strategy, we have found a NE

3. Otherwise, continue

• Essentially the same procedure as the simplex algorithm for

linear programming, for those who may be familiar with that

28
 Outline

Introduction: Some (possibly) familiar games

Basic game theory

Computing equilibria

Special cases and extensions

29
 N -player games
• For more than two players, precise analogue of Lemke-Howson
algorithm doesn’t exist, but local search can still be effective

• Can also formulate as optimization problem

N X
X
minimize max{ui (ai , s−i ) − ui (s), 0}2
s
i=1 ai ∈Ai

subject to 1T si = 1, s≥0

• At any NE, objective value will be zero (no incentive to any

other pure strategy)

• Of course, a non-convex problem, with potential local optima

30
 Special case: zero-sum games
• Two-player zero-sum games can be solved efficiently (in
polynomial time) by formulating it as a linear program
minimize maximize sT1 Cs2
s1 s2

subject to s1 ≥ 0, 1T s1 = 1, s2 ≥ 0, 1T s2 = 1

• Requiring player 2 to play a pure strategy, equivalent to

minimize max (C T s1 )i
s1 i=1,...,|A2 |

subject to s1 ≥ 0, 1T s1 = 1
which is equivalent to linear program
minimize t
s1 ,t

subject to s1 ≥ 0, 1T s1 = 1, C T u ≤ t1
31
• Somewhat surprisingly, this is actually the optimal strategy for
player 1, even if player 2 can play mixed strategies (proof
involves an optimization concept called duality)

• Key aspect of zero-sum game is that we could express game as

minimization and maximization over the same objective terms
by the two agents, can’t do this in general case

32