Game-theoretic analysis

tools

Tuomas Sandholm
Professor
Computer Science Department
Carnegie Mellon University
 Terminology
• In a 1-agent setting, agent’s expected utility maximizing strategy is well-
defined
• But in a multiagent system, the outcome may depend on others’ strategies
also
• Game theory analyzes stable points in the space of strategy profiles =>
allows one to build robust, nonmanipulable multiagent systems
• Agent = player
• Action = move = choice that agent can make at a point in the game
• Strategy si = mapping from history (to the extent that the agent i can
distinguish) to actions
• Strategy set Si = strategies available to the agent
• Strategy profile (s1, s2, ..., s|A|) = one strategy for each agent
• Agent’s utility is determined after each agent (including nature that is
used to model uncertainty) has chosen its strategy, and game has been
played: ui = ui(s1, s2, ..., s|A|)
 Game representations
Matrix form
Extensive form (aka normal form
aka strategic form)

player 2’s strategy

Left 1, 2
Left, Left, Right, Right,
player 2 Left Right Left Right
Up
Right 3, 4 Up 1, 2 1, 2 3, 4 3, 4
player 1’s
player 1 strategy
Left 5, 6
Down 5, 6 7, 8 5, 6 7, 8

Down player 2
Right 7, 8

Potential combinatorial explosion

 Dominant strategy “equilibrium”
• Best response si*: for all si’, ui(si*,s-i) ≥ ui(si’,s-i)
• Dominant strategy si*: si* is a best response for all s-i
– Does not always exist
– Inferior strategies are called dominated
• Dominant strategy equilibrium is a strategy profile where
each agent has picked its dominant strategy
– Does not always exist
– Requires no counterspeculation

Prisoners’ Dilemma cooperate defect

Pareto optimal?
cooperate 3, 3 0, 5
Social welfare
maximizing?
defect 5, 0 1, 1
 Nash equilibrium [Nash50]

• Sometimes an agent’s best response depends on others’

strategies: a dominant strategy does not exist
• A strategy profile is a Nash equilibrium if no player has
incentive to deviate from his strategy given that others do
not deviate: for every agent i, ui(si*,s-i) ≥ ui(si’,s-i) for all si’
– Dominant strategy equilibria are Nash equilibria but not
vice versa
– Defect-defect is the only Nash eq. in Prisoner’s Dilemma
– Battle of the Sexes (has no dominant strategy equilibria):

Woman
boxing ballet
boxing 2, 1 0, 0
Man
ballet 0, 0 1, 2
Criticisms of Nash equilibrium
• Not unique in all games, e.g. Battle of the Sexes
– Approaches for addressing this problem
• Refinements (= strengthenings) of the equilibrium concept
– Eliminate weakly dominated strategies first
– Choose the Nash equilibrium with highest welfare
– Subgame perfection
– …
• Focal points
• Mediation
• Communication
• Convention
• Learning 1, 0 0, 1

• Does not exist in all games 0, 1 1, 0

Existence of (pure strategy) Nash equilibria

• Thrm.
– Any finite game,
– where each action node is alone in its information set
• (i.e. at every point in the game, the agent whose turn it is to
move knows what moves have been played so far)
– is dominance solvable by backward induction (at
least as long as ties are ruled out)
• Constructive proof: Multi-player minimax search
Rock-scissors-paper game

Sequential moves
Rock-scissors-paper game

Simultaneous moves
 Mixed strategy Nash equilibrium
Mixed strategy = agent’s chosen probability
distribution over pure strategies from its strategy set
rock 0, 0
move of Each agent has a
agent 2 best response strategy
scissors and beliefs
1, -1
paper (consistent with each
rock other)
-1, 1
rock -1, 1 Symmetric mixed
strategy Nash eq:
move of scissors scissors Each player
0, 0 plays each pure
agent 1 paper
paper strategy with
probability 1/3
1, -1

rock 1, -1 In mixed strategy

equilibrium, each
Information set strategy that occurs in
scissors
(the mover does not -1, 1 the mix of agent i has
know which node of the paper
equal expected utility to i
set she is in)
0, 0
 Existence & complexity of
mixed strategy Nash equilibria
• Every finite player, finite strategy game has at
least one Nash equilibrium if we admit mixed
strategy equilibria as well as pure [Nash 50]
– (Proof is based on Kakutani’s fix point theorem)

• May be hard to compute

– Even in 2-agent matrix games
• Finding one is PPAD-complete (even with 0/1 payoffs)
[Cheng&Deng FOCS-06; Abbott, Kane & Valiant FOCS-05]
• Finding an approximately good one is NP-hard
[Conitzer&Sandholm]
 Ultimatum game

(for distributional bargaining)

 Subgame perfect equilibrium & credible threats
• Proper subgame = subtree (of the game tree) whose root is alone in its information set
• Subgame perfect equilibrium = strategy profile that is in Nash equilibrium in every proper
subgame (including the root), whether or not that subgame is reached along the equilibrium
path of play
• E.g. Cuban missile crisis

- 100, - 100
Nuke

Kennedy
Arm
• Pure strategy Nash equilibria: (Arm,Fold), (Retract,Nuke)
Fold 10, -10
• Pure strategyKhrushchev
subgame perfect equilibria: (Arm,Fold)
• Conclusion: Kennedy’s Nuke threat was not credible
Retract -1, 1
Ultimatum game, again
 Thoughts on credible threats
• Could use software as a commitment device
– If one can credibly convince others that one cannot change
one’s software agent, then revealing the agent’s code acts as
a credible commitment to one’s strategy
– E.g. nuke in the missile crisis
– E.g. accept no less than 60% as the second mover in the
ultimatum game
• Restricting one’s strategy set can increase one’s utility
– This cannot occur in single agent settings
• Social welfare can increase or decrease
 Conclusions on game-theoretic
analysis tools
• Tools for building robust, nonmanipulable systems for
self-interested agents
• Different solution concepts
– For existence, use strongest equilibrium concept
– For uniqueness, use weakest equilibrium concept
Strong Nash eq
Strength
against
collusion Coalition-Proof Nash eq

Nash eq Subgame perfect eq Dominant

Bayes-Nash eq Perfect Bayesian eq Sequential eq strategy eq

Strength

Ex post equilibrium  Nash equilibrium for all priors