Security and Cooperation

in Wireless Networks
http://secowinet.epfl.ch/

Appendix B: A tutorial on game theory for

wireless networks

static games;
dynamic games;
repeated games;
strict and weak
dominance;
Nash equilibrium;
Pareto optimality;
Subgame perfection;
…
© 2007 Levente Buttyán and Jean-Pierre Hubaux
 Chapter outline

B.1 Introduction
B.2 Static games
B.3 Dynamic games
B.4 Repeated games

Security and Cooperation in Wireless Networks

Appendix B: A tutorial on game theory for wireless networks
2/37
 Brief introduction to Game Theory
 Discipline aiming at modeling situations in which actors have
to make decisions which have mutual, possibly conflicting,
consequences
 Classical applications: economics, but also politics and
biology
 Example: should a company invest in a new plant, or enter a
new market, considering that the competition may make
similar moves?
 Most widespread kind of game: non-cooperative (meaning
that the players do not attempt to find an agreement about
their possible moves)

Security and Cooperation in Wireless Networks

B.1 Introduction 3/37
Appendix B: A tutorial on game theory for wireless networks
 Classification of games

Non-cooperative Cooperative

Static Dynamic (repeated)

Strategic-form Extensive-form

Perfect information Imperfect information

Complete information Incomplete information

Perfect info: each player knows the identity of other players and, for each
of them, the payoff resulting of each strategy.

Complete info: each player can observe the action of each other player.
Security and Cooperation in Wireless Networks
B.1 Introduction 4/37
Appendix B: A tutorial on game theory for wireless networks
 Cooperation in self-organized wireless networks

D2 D1

S2
S1

Usually, the devices are assumed to be cooperative.

But what if they are not?

Security and Cooperation in Wireless Networks

B.1 Introduction 5/37
Appendix B: A tutorial on game theory for wireless networks
 Chapter outline

B.1 Introduction
B.2 Static games
B.3 Dynamic games
B.4 Repeated games

Security and Cooperation in Wireless Networks

Appendix B: A tutorial on game theory for wireless networks
6/37
 Example 1: The Forwarder’s Dilemma

?
Blue Green

Security and Cooperation in Wireless Networks

B.2 Static games 7/37
Appendix B: A tutorial on game theory for wireless networks
 From a problem to a game

 users controlling the devices are rational = try to

maximize their benefit
 game formulation: G = (P,S,U)
– P: set of players
– S: set of strategy functions • Reward for packet reaching
the destination: 1
– U: set of payoff functions
• Cost of packet forwarding:
c (0 < c << 1)
 strategic-form representation
Green
Blue Forward Drop
Forward (1-c, 1-c) (-c, 1)
Drop (1, -c) (0, 0)
Security and Cooperation in Wireless Networks
B.2 Static games 8/37
Appendix B: A tutorial on game theory for wireless networks
 Solving the Forwarder’s Dilemma (1/2)
Strict dominance: strictly best strategy, for any strategy of the other player(s)

Strategy si strictly dominates if

ui ( si' , si )  ui ( si , si ), si  Si , si'  Si
where: ui  U payoff function of player i
s i  S i strategies of all players except player i

In Example 1, strategy Drop strictly dominates strategy Forward

Green
Blue Forward Drop
Forward (1-c, 1-c) (-c, 1)
Drop (1, -c) (0, 0)
Security and Cooperation in Wireless Networks
B.2 Static games 9/37
Appendix B: A tutorial on game theory for wireless networks
 Solving the Forwarder’s Dilemma (2/2)

Solution by iterative strict dominance:

Green
Blue Forward Drop
Forward (1-c, 1-c) (-c, 1)
Drop (1, -c) (0, 0)

}
Drop strictly dominates Forward
BUT Dilemma
Forward would result in a better outcome

Result: Tragedy of the commons ! (Hardin, 1968)

Security and Cooperation in Wireless Networks
B.2 Static games 10/37
Appendix B: A tutorial on game theory for wireless networks
Example 2: The Joint Packet Forwarding Game

? ?
Source Blue Green Dest

Green
• Reward for packet reaching Blue Forward Drop
the destination: 1
• Cost of packet forwarding: (1-c, 1-c) (-c, 0)
Forward
c (0 < c << 1)
Drop (0, 0) (0, 0)

No strictly dominated strategies !

Security and Cooperation in Wireless Networks
B.2 Static games 11/37
Appendix B: A tutorial on game theory for wireless networks
 Weak dominance
Weak dominance: strictly better strategy for at least one opponent strategy
Strategy s’i is weakly dominated by strategy si if
ui ( si' , s i )  ui ( si , si ), s i  S i
with strict inequality for at least one s-i

? ?
Source Blue Green Dest

Green
Blue Forward Drop
Iterative weak dominance
Forward (1-c, 1-c) (-c, 0)
BUT
The result of the iterative weak Drop
(0, 0) (0, 0)
dominance is not unique in general !
Security and Cooperation in Wireless Networks
B.2 Static games 12/37
Appendix B: A tutorial on game theory for wireless networks
 Nash equilibrium (1/2)

Nash Equilibrium: no player can increase its payoff by deviating unilaterally

Green
Blue Forward Drop
E1: The Forwarder’s
Dilemma Forward (1-c, 1-c) (-c, 1)
Drop (1, -c) (0, 0)

Green
Blue Forward Drop
E2: The Joint Packet
Forwarding game Forward (1-c, 1-c) (-c, 0)
Drop (0, 0) (0, 0)

Security and Cooperation in Wireless Networks

B.2 Static games 13/37
Appendix B: A tutorial on game theory for wireless networks
 Nash equilibrium (2/2)

Strategy profile s* constitutes a Nash equilibrium if, for each player i,

ui ( si* , s* i )  ui ( si , s* i ), si  Si

where: ui  U payoff function of player i
si  Si strategy of player i

The best response of player i to the profile of strategies s-i is

a strategy si such that:

bi ( s i )  arg max ui ( si , s i )
si Si

Nash Equilibrium = Mutual best responses

Caution! Many games have more than one Nash equilibrium

Security and Cooperation in Wireless Networks
B.2 Static games 14/37
Appendix B: A tutorial on game theory for wireless networks
 Example 3: The Multiple Access game

Time-division channel

Green
Reward for successful Blue Quiet Transmit
transmission: 1
Quiet (0, 0) (0, 1-c)
Cost of transmission: c
(0 < c << 1) Transmit (1-c, 0) (-c, -c)

There is no strictly dominating strategy

There are two Nash equilibria
Security and Cooperation in Wireless Networks
B.2 Static games 15/37
Appendix B: A tutorial on game theory for wireless networks
 Mixed strategy Nash equilibrium

p: probability of transmit for Blue

q: probability of transmit for Green

ublue  p(1  q )(1  c)  pqc  p (1  c  q )

u green  q (1  c  p )
objectives
– Blue: choose p to maximize ublue
– Green: choose q to maximize ugreen

p  1  c, q  1  c
is a Nash equilibrium

Security and Cooperation in Wireless Networks

B.2 Static games 16/37
Appendix B: A tutorial on game theory for wireless networks
 Example 4: The Jamming game

transmitter transmitter:
two channels: • reward for successful
C1 and C2 transmission: 1
jammer • loss for jammed
transmission: -1
Green jammer:
Blue C1 C2 • reward for successful
jamming: 1
C1 (-1, 1) (1, -1) • loss for missed
jamming: -1
C2 (1, -1) (-1, 1)
There is no pure-strategy
p: probability of transmit
Nash equilibrium on C1 for Blue
1 1 q: probability of transmit
p  , q  is a Nash equilibrium on C1 for Green
2 2
Security and Cooperation in Wireless Networks
B.2 Static games 17/37
Appendix B: A tutorial on game theory for wireless networks
 Theorem by Nash, 1950

Theorem:
Every finite strategic-form game has a mixed-
strategy Nash equilibrium.

Security and Cooperation in Wireless Networks

B.2 Static games 18/37
Appendix B: A tutorial on game theory for wireless networks
 Efficiency of Nash equilibria

Green
Blue Forward Drop
E2: The Joint
Packet Forwarding Forward (1-c, 1-c) (-c, 0)
game
Drop (0, 0) (0, 0)

How to choose between several Nash equilibria ?

Pareto-optimality: A strategy profile is Pareto-optimal if it is not
possible to increase the payoff of any player without decreasing the
payoff of another player.

Security and Cooperation in Wireless Networks

B.2 Static games 19/37
Appendix B: A tutorial on game theory for wireless networks
 How to study Nash equilibria ?

Properties of Nash equilibria to investigate:

 uniqueness
 efficiency (Pareto-optimality)
 emergence (dynamic games, agreements)

Security and Cooperation in Wireless Networks

B.2 Static games 20/37
Appendix B: A tutorial on game theory for wireless networks
 Chapter outline

B.1 Introduction
B.2 Static games
B.3 Dynamic games
B.4 Repeated games

Security and Cooperation in Wireless Networks

Appendix B: A tutorial on game theory for wireless networks
21/37
 Extensive-form games
 usually to model sequential decisions
 game represented by a tree
 Example 3 modified: the Sequential Multiple Access game:
Blue plays first, then Green plays.

Time-division channel

Blue
Reward for successful T Q
transmission: 1 Green Green
Cost of transmission: c T Q T Q
(0 < c << 1)
(-c,-c) (1-c,0) (0,1-c) (0,0)
Security and Cooperation in Wireless Networks
B.3 Dynamic games 22/37
Appendix B: A tutorial on game theory for wireless networks
 Strategies in dynamic games

 The strategy defines the moves for a player for every

node in the game, even for those nodes that are not
reached if the strategy is played.
Blue
strategies for Blue: T Q
T, Q Green Green
strategies for Green: T Q T Q
TT, TQ, QT and QQ
(-c,-c) (1-c,0) (0,1-c) (0,0)

TQ means that player p2 transmits if p1 transmits and remains quiet if p1 remains quiet.

Security and Cooperation in Wireless Networks

B.3 Dynamic games 23/37
Appendix B: A tutorial on game theory for wireless networks
Backward induction
 Solve the game by reducing from the final stage
 Eliminates Nash equilibria that are increadible threats

Blue
T Q
Incredible threat: (Q, TT) Green Green
T Q T Q

(-c,-c) (1-c,0) (0,1-c) (0,0)

Backward induction solution: h={T, Q}

Security and Cooperation in Wireless Networks
B.3 Dynamic games 24/37
Appendix B: A tutorial on game theory for wireless networks
Subgame perfection

 Extends the notion of Nash equilibrium

One-deviation property: A strategy si conforms to the one-deviation
property if there does not exist any node of the tree, in which a
player i can gain by deviating from si and apply it otherwise.

Subgame perfect equilibrium: A strategy profile s constitutes a

subgame perfect equilibrium if the one-deviation property holds for
every strategy si in s.
Blue
Finding subgame perfect T Q
equilibria using backward Green Green
induction Q Q
T T
Subgame perfect equilibria:
(T, QT) and (T, QQ) (-c,-c) (1-c,0) (0,1-c) (0,0)

Stackelberg games have one leader and one or several followers

Security and Cooperation in Wireless Networks
B.3 Dynamic games 25/37
Appendix B: A tutorial on game theory for wireless networks
 Chapter outline

B.1 Introduction
B.2 Static games
B.3 Dynamic games
B.4 Repeated games

Security and Cooperation in Wireless Networks

Appendix B: A tutorial on game theory for wireless networks
26/37
 Repeated games
 repeated interaction between the players (in stages)
 move: decision in one interaction
 strategy: defines how to choose the next move, given the
previous moves
 history: the ordered set of moves in previous stages
– most prominent games are history-1 games (players consider only
the previous stage)
 initial move: the first move with no history
 finite-horizon vs. infinite-horizon games
 stages denoted by t (or k)

Security and Cooperation in Wireless Networks

B.4 Repeated games 27/37
Appendix B: A tutorial on game theory for wireless networks
 Utilities: Objectives in the repeated game
 finite-horizon vs. infinite-horizon games
 myopic vs. long-sighted repeated game

myopic: ui  ui  t  1
T
long-sighted finite: ui   ui  t 
t 0

long-sighted infinite: ui   ui  t 
t 0

payoff with discounting: ui   ui  t    t

t 0

0    1 is the discounting factor

Security and Cooperation in Wireless Networks
B.4 Repeated games 28/37
Appendix B: A tutorial on game theory for wireless networks
 Strategies in the repeated game

 usually, history-1 strategies, based on different inputs:

– others’ behavior: mi  t  1  si  mi  t  
– others’ and own behavior: mi  t  1  si  mi  t  , m i  t  

– payoff: mi  t  1  si ui  t  

Example strategies in the Forwarder’s Dilemma:

Blue (t) initial F D strategy name
move
Green (t+1) F F F AllC
F F D Tit-For-Tat (TFT)
D D D AllD
F D F Anti-TFT
Security and Cooperation in Wireless Networks
B.4 Repeated games 29/37
Appendix B: A tutorial on game theory for wireless networks
 The Repeated Forwarder’s Dilemma

?
Blue Green

?
Green
Blue Forward Drop
Forward (1-c, 1-c) (-c, 1)
Drop (1, -c) (0, 0)

stage payoff
Security and Cooperation in Wireless Networks
B.4 Repeated games 30/37
Appendix B: A tutorial on game theory for wireless networks
Analysis of the Repeated Forwarder’s Dilemma (1/3)


infinite game with discounting: ui   ui  t    t

t 0

Blue strategy Green strategy Blue payoff Green payoff

AllD AllD 0 0
AllD TFT 1 -c
AllD AllC 1/(1-ω) -c/(1-ω)
AllC AllC (1-c)/(1-ω) (1-c)/(1-ω)
AllC TFT (1-c)/(1-ω) (1-c)/(1-ω)
TFT TFT (1-c)/(1-ω) (1-c)/(1-ω)

Security and Cooperation in Wireless Networks

B.4 Repeated games 31/37
Appendix B: A tutorial on game theory for wireless networks
 Analysis of the Repeated Forwarder’s Dilemma (2/3)
Blue strategy Green strategy Blue payoff Green payoff
AllD AllD 0 0
AllD TFT 1 -c
AllD AllC 1/(1-ω) -c/(1-ω)
AllC AllC (1-c)/(1-ω) (1-c)/(1-ω)
AllC TFT (1-c)/(1-ω) (1-c)/(1-ω)
TFT TFT (1-c)/(1-ω) (1-c)/(1-ω)

 AllC receives a high payoff with itself and TFT, but

 AllD exploits AllC
 AllD performs poor with itself
 TFT performs well with AllC and itself, and
 TFT retaliates the defection of AllD

TFT is the best strategy if ω is high !

Security and Cooperation in Wireless Networks
B.4 Repeated games 32/37
Appendix B: A tutorial on game theory for wireless networks
 Analysis of the Repeated Forwarder’s Dilemma (3/3)

Blue strategy Green strategy Blue payoff Green payoff

AllD AllD 0 0
TFT TFT (1-c)/(1-ω) (1-c)/(1-ω)

Theorem: In the Repeated Forwarder’s Dilemma, if both

players play AllD, it is a Nash equilibrium.

Theorem: In the Repeated Forwarder’s Dilemma, both

players playing TFT is a Nash equilibrium as well.

The Nash equilibrium sBlue = TFT and sGreen = TFT is

Pareto-optimal (but sBlue = AllD and sGreen = AllD is not) !

Security and Cooperation in Wireless Networks

B.4 Repeated games 33/37
Appendix B: A tutorial on game theory for wireless networks
 Experiment: Tournament by Axelrod, 1984

 any strategy can be submitted (history-X)

 strategies play the Repeated Prisoner’s Dilemma (Repeated
Forwarder’s Dilemma) in pairs
 number of rounds is finite but unknown

 TFT was the winner

 second round: TFT was the winner again

R. Axelrod The Evolution of Cooperation

Basic Books, 1984

Security and Cooperation in Wireless Networks

B.4 Repeated games 34/37
Appendix B: A tutorial on game theory for wireless networks
 An Example beyond Engineering

Country 2
Reduce military Increase military
Country 1 investment investment
Reduce
military (1, 1) (-1, 2)
investment
Increase
military (2, -1) (0, 0)
investment

Payoffs:
2: I have weaponry superior to the one of the opponent
1: We have equivalent weaponry and managed to reduce it on both sides
0: We have equivalent weaponry and did not managed to reduce it on both sides
-1: My opponent has weaponry that is superior to mine
Security and Cooperation in Wireless Networks
35/38
Appendix B: A tutorial on game theory for wireless networks
 Discussion on game theory
 Rationality
 Payoff function and cost
 Pricing and mechanism design (to promote
desirable solutions)
 Infinite-horizon games and discounting
 Reputation
 Cooperative games
 Imperfect / incomplete information

Security and Cooperation in Wireless Networks

B.5 Discussion 36/38
Appendix B: A tutorial on game theory for wireless networks
 Who is malicious? Who is selfish?

Harm everyone: viruses,… Big brother

Selective harm: DoS,… Spammer

Cyber-gangster:
phishing attacks, Greedy operator
trojan horses,…

Selfish mobile station

Both
 Bothsecurity
securityand
andgame
gametheory
theorybackgrounds
backgroundsare
areuseful
usefulininmany
manycases
cases!!!!

Security and Cooperation in Wireless Networks

B.5 Discussion 37/38
Appendix B: A tutorial on game theory for wireless networks
 Conclusions
 Game theory can help modeling greedy behavior in wireless
networks
 Discipline still in its infancy
 Alternative solutions
– Ignore the problem
– Build protocols in tamper-resistant hardware

Security and Cooperation in Wireless Networks

Appendix B: A tutorial on game theory for wireless networks
38/37