A Short Tutorial on Game Theory

EE228a, Fall 2002

Dept. of EECS, U.C. Berkeley
EE228a, Fall 2002
2
Outline
Introduction
Complete-Information Strategic Games
Static Games
Repeated Games
Stackelberg Games
Cooperative Games
Bargaining Problem
Coalitions

EE228a, Fall 2002
3
Outline
Introduction
What is game theory about?
Relevance to networking research
Elements of a game
Non-Cooperative Games
Static Complete-Information Games
Repeated Complete-Information Games
Stackelberg Games
Cooperative Games
Nashs Bargaining Solution
Coalition: the Shapley Value

EE228a, Fall 2002
Introduction 4
What Is Game Theory About?
To understand how decision-makers interact
A brief history
1920s: study on strict competitions
1944: Von Neumann and Morgensterns book
Theory of Games and Economic Behavior
After 1950s: widely used in economics, politics, biology
Competition between firms
Auction design
Role of punishment in law enforcement
International policies
Evolution of species

EE228a, Fall 2002
Introduction 5
Relevance to Networking Research
Economic issues becomes increasingly important
Interactions between human users
congestion control
resource allocation
Independent service providers
Bandwidth trading
Peering agreements
Tool for system design
Distributed algorithms
Multi-objective optimization
Incentive compatible protocols
EE228a, Fall 2002
Introduction 6
Elements of a Game: Strategies
Decision-makers choice(s) in any given situation
Fully known to the decision-maker
Examples
Price set by a firm
Bids in an auction
Routing decision by a routing algorithm
Strategy space: set of all possible actions
Finite vs infinite strategy space
Pure vs mixed strategies
Pure: deterministic actions
Mixed: randomized actions
EE228a, Fall 2002
Introduction 7
Elements of a Game: Preference and Payoff
Preference
Transitive ordering among strategies
if a >> b, b >> c, then a >> c
Payoff
An order-preserving mapping from preference to R
+
Example: in flow control, U(x)=log(1+x) px


payoff action
EE228a, Fall 2002
Introduction 8
Rational Choice
Two axiomatic assumptions on games
1. In any given situation a decision-maker always
chooses the action which is the best according to
his/her preferences (a.k.a. rational play).
2. Rational play is common knowledge among all
players in the game.




EE228a, Fall 2002
Introduction 9
Example: Prisoners Dilemma
Prisoner A
Prisoner B
mum fink
mum
fink
1, 1 9, 0
0, 9
6, 6
strategies
payoffs
As move
Bs move
9 6
9
6
outcome of
the game
EE228a, Fall 2002
Introduction 10
Different Types of Games
Static vs multi-stage
Static: game is played only once
Prisoners dilemma
Multi-stage: game is played in multiple rounds
Multi-round auctions, chess games
Complete vs incomplete information
Complete info.: players know each others payoffs
Prisoners dilemma
Incomplete info.: other players payoffs are not known
Sealed auctions

EE228a, Fall 2002
Introduction 11
Representations of a Game
Normal- vs extensive-form representation
Normal-form
like the one used in previous example
Extensive-form
Prisoner A
Prisoner B
mum
mum mum
fink
fink fink
EE228a, Fall 2002
12
Outline
Introduction
Complete-Information Strategic Games
Static Games
Repeated Games
Stackelberg Games
Cooperative Games
Nashs Bargaining Problem
Coalitions: the Shapley Value

EE228a, Fall 2002
13
Static Games
Model
Players know each others payoffs
But do not know which strategies they would choose
Players simultaneously choose their strategies
Game is over and players receive payoffs based on
the combination of strategies just chosen
Question of Interest:
What outcome would be produced by such a game?
EE228a, Fall 2002
14
Example: Cournots Model of Duopoly
Model (from Gibbons)
Two firms producing the same kind of product in
quantities of q
1
and q
2
, respectively
Market clearing price p=A q
1
q
2
Cost of production is C for both firms
Profit for firm i
J
i
= (A q
1
q
2
) q
i
C q
i

= (A C q
1
q
2
) q
i

define B A C

Objective: choose q
i
to maximize profit
q
i
*
= argmax
qi
(B q
1
q
2
) q
i


EE228a, Fall 2002
15
A Simple Example: Solution
Firm is best choice, given its competitors q
q
1
*
= (B q
2
)/2

q
2
*
= (B q
1
)/2

q
1
q
2
best-reply function
B/2
B
q
1
*
B/2
B
q
2
*
equilibrium: q
1
=q
2
=B/3
fixed-point solution
to the equations
EE228a, Fall 2002
16
Solution to Static Games
Nash Equilibrium (J. F. Nash, 1950)
Mathematically, a strategy profile (s
1
*

, , s
i
*
,, s
n
*

) is a
Nash Equilibrium if for each player i
U
i
(s
1
*

, , s
*
i-1
, s
i
*
, s
*
i+1
,, s
n
*

)
> U
i
(s
1
*

, , s
*
i-1
, s
i
, s
*
i+1
,,s
n
*

),
for each feasible strategy s
i
Plain English: a situation in which no player has incentive to
deviate
Its fixed-point solution to the following system of equations
s
i
=argmax
s
U
i
(s
1
, , s
i-1
, s, s
i+1
,,s
n
), i
Other solution concepts (see references)
EE228a, Fall 2002
17
An Example on Mixed Strategies
Pure-Strategy Nash Equilibrium may not exist
Player A
Player B
Head (H) Tail (T)
H
T
1, 1 1, 1
1, 1
1, 1
Cause: each player tries to outguess his opponent!
EE228a, Fall 2002
18
Example: Best Reply
Mixed Strategies
Randomized actions to avoid being outguessed
Players strategies and expected payoffs
Players plays H w.p. p and play T w.p. 1 p
Expected payoff of Player A
p
a
p
b
+ (1 p
a
) (1 p
b
) p
a
(1 p
b
) p
b
(1 p
a
)
= (1 2 p
b
) + p
a
(4p
b
2)
So
if p
b
>1/2, p
a
*
=1 (i.e. play H);
if p
b
>1/2, p
a
*
=0 (i.e. play T);
if p
b
=1/2, then playing either H or T is equally good
EE228a, Fall 2002
19
Example: Nash Equilibrium
p
b
p
a 0
1

1/2

1/2

1

EE228a, Fall 2002
20
Existence of Nash Equilibrium
Finite strategy space (J. F. Nash, 1950)
A n-player game has at least one Nash equilibrium,
possibly involving mixed strategy.
Infinite strategy space (R.B. Rosen, 1965)
A pure-strategy Nash Equilibrium exists in a n-player
concave game.
If the payoff functions satisfy diagonally strict concavity
condition, then the equilibrium is unique.
(s
1
s
2
) [ r
j
VJ
j
(s
1
) ] + (s
2
s
1
) [ r
j
VJ
j
(s
2
) ]<0
EE228a, Fall 2002
21
Distributed Computation of Nash Equilibrium
Nash equilibrium as result of learning
Players iteratively adjust their strategies based on locally
available information
Equilibrium is reached if there is a steady state
Two commonly used schemes
s
1
s
2
s
1
*
s
2
*
s
1
s
2
s
1
*
s
2
*
Gauss-Siedel Jacobian
EE228a, Fall 2002
22
Convergence of Distributed Algorithms
Algorithms may not converge for some cases
0
S
1
S
*
2
S
*
1
S
2
EE228a, Fall 2002
23
Suggested Readings
J.F. Nash. Equilibrium Points in N-Person Games. Proc.
of National Academy of Sciences, vol. 36, 1950.
A must-read classic paper
R.B. Rosen. Existence and Uniqueness of Equilibrium
Points for Concave N-Person Games. Econometrica, vol.
33, 1965.
Has many useful techniques
A. Orda et al. Competitive Routing in Multi-User
Communication Networks. IEEE/ACM Transactions on
Networking, vol. 1, 1993.
Applies game theory to routing
And many more
EE228a, Fall 2002
24
Multi-Stage Games
General model
Game is played in multiple rounds
Finite or infinitely many times
Different games could be played in different rounds
Different set of actions or even players
Different solution concepts from those in static games
Analogy: optimization vs dynamic programming
Two special classes
Infinitely repeated games
Stackelberg games
EE228a, Fall 2002
25
Infinitely Repeated Games
Model
A single-stage game is repeated infinitely many times
Accumulated payoff for a player
J=t
1
+ot
2
++o
n1
t
n
+=E
i
o
i1
t
i
discount factor payoff from stage n
Main theme: play socially more efficient moves
Everyone promises to play a socially efficient move in
each stage
Punishment is used to deter cheating
Example: justice system
EE228a, Fall 2002
26
Cournots Game Revisited. I
Cournots Model
At equilibrium each firm produces B/3, making a profit of
B
2
/9
Not an ideal arrangement for either firm, because
If a central agency decides on production quantity q
m
q
m
=argmax (B q) q = B/2
so each firm should produce B/4 and make a profit of B
2
/8
An aside: why B/4 is not played in the static game?
If firm A produces B/4, it is more profitable for firm B
to produce 3B/8 than B/4
Firm A then in turn produces 5B/16, and so on
EE228a, Fall 2002
27
Cournots Game Revisited. II
Collaboration instead of competition
Q: Is it possible for two firms to reach an agreement to
produce B/4 instead of B/3 each?
A: That would depend on how important future return is
to each firm

A firm has two choices in each round:
Cooperate: produce B/4 and make profit B
2
/8
Cheat: produce 3B/8 and make profit 9B
2
/64
But in the subsequent rounds, cheating will cause
its competitor to produce B/3 as punishment
its own profit to drop back to B
2
/9
EE228a, Fall 2002
28
Cournots Game Revisited. III
Is there any incentive for a firm not to cheat?
Lets look at the accumulated payoffs:
If it cooperates:
S
c
= (1+o+ o
2
+ o
3
+

) B
2
/8 =B
2
/8(1o)
If it cheats:
S
d
= 9B
2
/64 + (o+ o
2
+ o
3
+

) B
2
/9
={9/64 + o/9(1o)} B
2
So it will not cheat if S
c
> S
d
.




This happens only if o>9/17.
Conclusion
If future return is valuable enough to each player, then
strategies exist for them to play socially efficient moves.
EE228a, Fall 2002
29
Strategies in Repeated Games
A strategy
is no longer a single action
but a complete plan of actions
based on possible history of plays up to current stage
usually includes some punishment mechanism
Example: in Cournots game, a players strategy is

Produce B/4 in the first stage. In the n
th
stage,
produce B/4 if both firms have produced B/4
in each of the n1 previous stages; otherwise,
produce B/3.
history
punishment
EE228a, Fall 2002
30
Equilibrium in Repeated Games
Subgame-perfect Nash equilibrium (SPNE)
A subgame starting at stage n is
identical to the original infinite game
associated with a particular sequence of plays from
the first stage to stage n1
A SPNE constitutes a Nash equilibrium in every subgame
Why subgame perfect?
It is all about creditable threats:
Players believe the claimed punishments indeed will
be carried out by others, when it needs to be evoked.
So a creditable threat has to be a Nash equilibrium for
the subgame.
EE228a, Fall 2002
31
Known Results for Repeated Games
Friedmans Theorem (1971)
Let G be a single-stage game and (e
1
,, e
n
) denote the
payoff from a Nash equilibrium of G.
If x=(x
1
, , x
n
) is a feasible payoff from G such that
x
i
> e
i
,i, then there exists a subgame-perfect Nash
equilibrium of the infinitely repeated game of G which
achieves x, provided that discount factor o is close
enough to one.
Assignment:
Apply this theorem to Cournots game on an agreement other
than B/4.
EE228a, Fall 2002
32
Suggested Readings
J. Friedman. A Non-cooperative Equilibrium for Super-
games. Review of Economic Studies, vol. 38, 1971.
Friedmans original paper
R. J. La and V. Anantharam. Optimal Routing Control:
Repeated Game Approach," IEEE Transactions on
Automatic Control, March 2002.
Applies repeated game to improve the efficiency of
competitive routing
EE228a, Fall 2002
33
Stackelberg Games
Model
One player (leader) has dominate influence over another
Typically there are two stages
One player moves first
Then the other follows in the second stage
Can be generalized to have
multiple groups of players
Static games in both stages
Main Theme
Leader plays by backwards induction, based on the
anticipated behavior of his/her follower.
EE228a, Fall 2002
34
Stackelbergs Model of Duopoly
Assumptions
Firm 1 chooses a quantity q
1
to produce
Firm 2 observes q
1
and then chooses a quantity q
2
Outcome of the game
For any given q
1
, the best move for Firm 2 is
q
2
*

= (B q
1
)/2
Knowing this, Firm 1 chooses q
1
to maximize
J
1
= (B q
1


q
2
*

) q
1
= q
1
(B q
1
)/2

which yields
q
1
*

= B/2, and q
2
*

= B/4
J
1
*

= B
2
/8, and J
2
*

= B
2
/16
EE228a, Fall 2002
35
Suggested Readings
Y. A. Korilis, A. A. Lazar and A. Orda. Achieving
Network Optima Using Stackelberg Routing Strategies.
IEEE/ACM Trans on Networking, vol.5, 1997.
Network leads users to reach system optimal equilibrium
in competitive routing.
T. Basar and R. Srikant. Revenue Maximizing Pricing
and Capacity Expansion in a Many-User Regime.
INFOCOM 2002, New York.
Network charges users price to maximize its revenue.


EE228a, Fall 2002
36
Outline
Introduction
Complete-Information Strategic Games
Static Games
Repeated Games
Stackelberg Games
Cooperative Games
Nashs Bargaining Problem
Coalitions: the Shapley value

EE228a, Fall 2002
37
Cooperation In Games
Incentive to cooperate
Static games often lead to inefficient equilibrium
Achieve more efficient outcomes by acting together
Collusion, binding contract, side payment
Pareto Efficiency
A solution is Pareto efficient if there is no other
feasible solution in which some player is better
off and no player is worse off.
Pareto efficiency may be neither socially optimal nor fair
Socially optimal Pareto efficient
Fairness issues
Reading assignment as an example
A
B
mum fink
mum
fink
1, 1 9, 0
0, 9 6, 6
Nash equilibrium
more efficient
outcome
EE228a, Fall 2002
38
Nashs Bargaining Problem
Model
Two players with interdependent payoffs U and V
Acting together they can achieve a set of feasible payoffs
The more one player gets, the less the other is able to get
And there are multiple Pareto efficient payoffs
Q: which feasible payoff would they settle on?
Fairness issue
Example (from Owen):
Two men try to decide how to split $100
One is very rich, so that U(x)~ x
The other has only $1, so V(x)~ log(1+x)log1=log(1+x)
How would they split the money?

EE228a, Fall 2002
39
v

u

100
log(101)
Intuition
Feasible set of payoffs
Denote x the amount that the rich man gets
(u,v)=(x, log(101x)), xe[0,100]
Av
Au
A
B
Au
Av
Au
Av
C
A fair split should satisfy
| Au/u | = | Av/v |
Let A 0, du/u = dv/v
Or du/u + dv/v = 0, or
vdu+udv=0, or d(uv)=0.
Find the allocation which
maximizes UV
x
*
=76.8!
EE228a, Fall 2002
40
Nashs Axiomatic Approach (1950)
A solution (u
*
,v
*
) should be
Rational
(u
*
,v
*
) > (u
0
,v
0
), where (u
0
,v
0
) is the worst payoffs that the
players can get.
Feasible
(u
*
,v
*
)eS, the set of feasible payoffs.
Pareto efficient
Symmetric
If S is such that (u,v)eS (v,u)eS, then u
*
=v
*
.
Independent from linear transformations
Independent from irrelevant alternatives
Suppose Tc S. If (u
*
,v
*
)eT is a solution to S, then (u
*
,v
*
)
should also be a solution to T.
EE228a, Fall 2002
41
Results
There is a unique solution which
satisfies the above axioms
maximizes the product of two players additional payoffs
(uu
0
)(vv
0
)
This solution can be enforced by threats
Each player independently announces his/her threat
Players then bargain on their threats
If they reach an agreement, that agreement takes effort;
Otherwise, initially announced threats will be used
Different fairness criteria can be achieved by
changing the last axiom (see references)

EE228a, Fall 2002
42
Suggested Readings
J. F. Nash. The Bargaining Problem. Econometrica,
vol.18, 1950.
Nashs original paper. Very well written.
X. Cao. Preference Functions and Bargaining Solutions.
Proc. of the 21th CDC, NYC, NY, 1982.
A paper which unifies all bargaining solutions into a single
framework
Z. Dziong and L.G. Mason. FairEfficient Call Admission
Control Policies for Broadband Networks a Game
Theoretic Framework, IEEE/ACM Trans. On Networking,
vol.4, 1996.
Applies Nashs bargaining solution to resource allocation
problem in admission control (multi-objective optimization)
EE228a, Fall 2002
43
Coalitions
Model
Players (n>2) N form coalitions among themselves
A coalition is any nonempty subset of N
Characteristic function V defines a game
V(S)=payoff to S in the game between S and N-S, S c N
V(N)=total payoff achieved by all players acting together
V() is assumed to be super-additive
S, T c N, V(S+T) > V(S)+V(T)
Questions of Interest
Condition for forming stable coalitions
When will a single coalition be formed?
How to distribute payoffs among players in a fair way?
EE228a, Fall 2002
44
Core Sets
Allocation X=(x
1
, , x
n
)
x
i
> V({i}), ieN; E
ieN
x
i
= V(N).
The core of a game
a set of allocation which satisfies E
ieS
x
i
> V(S), S c N
If the core is nonempty, a single coalition can be formed
An example
A Berkeley landlord (L) is renting out a room
Al (A) and Bob (B) are willing to rent the room at $600
and $800, respectively
Who should get the room at what rent?
EE228a, Fall 2002
45
Example: Core Set
Characteristic function of the game
V(L)=V(A)=V(B)=V(A+B)=0
Coalition between L and A or L and B
If rent = x, then Ls payoff = x, As payoff = 600 x
so V(L+A)=600. Similarly, V(L+B)=800
Coalition among L, A and B: V(L+A+B)=800
The core of the game
x
L
+x
A
> 600
x
L
+x
B
> 800
x
L
+x
A
+x
B
=800
core={(y,0,800 y), 600s y s 800}
EE228a, Fall 2002
46
Fair Allocation: the Shapley Value
Define solution for player i in game V by P
i
(V)
Shapleys axioms
P
i
s are independent from permutation of labels
Additive: if U and V are any two games, then
P
i
(U+V) = P
i
(U) + P
i
(V), ieN
T is a carrier of N if V(ST)=V(S),S c N. Then for
any carrier T, E
ieT
P
i
= V(T).
Unique solution: Shapleys value (1953)
P
i
= E
ScN
(|S|1)! (N|S|)!
N!
[V(S) V(S {i})]
Intuition: a probabilistic interpretation
EE228a, Fall 2002
47
Suggested Readings
L. S. Shapley. A Value for N-Person Games.
Contributions to the Theory of Games, vol.2, Princeton
Univ. Press, 1953.
Shapleys original paper.
P. Linhart et al. The Allocation of Value for Jointly
Provided Services. Telecommunication Systems, vol. 4,
1995.
Applies Shapleys value to caller-ID service.
R. J. Gibbons et al. Coalitions in the International
Network. Tele-traffic and Data Traffic, ITC-13, 1991.
How coalition could improve the revenue of international
telephone carriers.
EE228a, Fall 2002
48
Summary
Models
Strategic games
Static games, multi-stage games
Cooperative games
Bargaining problem, coalitions
Solution concepts
Strategic games
Nash equilibrium, Subgame-perfect Nash equilibrium
Cooperative games
Nashs solution, Shapley value
Application to networking research
Modeling and design

EE228a, Fall 2002
49
References
R. Gibbons, Game Theory for Applied Economists,
Princeton Univ. Press, 1992.
an easy-to-read introductory to the subject
M. Osborne and A. Rubinstein, A Course in Game
Theory, MIT Press, 1994.
a concise but rigorous treatment on the subject
G. Owen, Game Theory, Academic Press, 3
rd
ed., 1995.
a good reference on cooperative games
D. Fudenberg and J. Tirole, Game Theory, MIT Press,
1991.
a complete handbook; the bible for game theory
http://www.netlibrary.com/summary.asp?id=11352