An Introduction to Game Theory and MANETS

1
An Introduction to Game Theory and MANETS

• A Brief Review of MANETS , Critical Concepts From
  Game Theory, Application of Game Theory to
  MANETS, Key Game Models
• December 8, 2003

2
A Brief Review of MANETS

• Concepts, Applications, Challenges

3
Mobile Ad Hoc Network (MANET)
Backbone
Mobile nodes
Access points
MANET
Infrastructure based network
4
Fundamental Concepts

• Ad hoc network is an autonomous network operating
  either in isolation or as stub network
  connecting to a fixed network
• Do not necessarily rely on existing
  infrastructure
• No access point
• Topology of the network continuously changes

5
Fundamental Concepts

• Multi hop communication
• Self-organizing capability

6
Advantages of a MANET Over Infrastructure- Based
Network

• Ease of deployment
• Quick deployment
• Reduced need for any infrastructure
• Potentially more robust to attack

7
Applications

• Military
• Disaster management

8
Applications

• Neighborhood area networks (NANs)
• Impromptu communications among groups of people

9
Characteristics

• Dynamic topology
• Mobility of nodes and unpredictability of
  wireless link results in frequent topology
  changes
• Heterogeneity
• Based on functionality of each node
• Some nodes route packets for a group of nearby
  nodes (cluster head)
• Nodes act as server providing service to other
  nearby nodes
• Based on capability of each node
• Nodes have varying battery life
• Nodes with radios of different range
• Nodes with differing storage capabilities

10
Characteristics

• Bandwidth-constrained variable-capacity links
• Bandwidth limited by the modulation scheme used
  (typically, IEEE 802.11x)
• Unreliability due to the wireless medium
• Limited physical security

11
Analytic Challenges

• De-centralized nature of nodes
• Independently adapting its operation based on
  perceived or measures statistics
• Interactive decision makers
• Decision taken by one node affects and influences
  the other nodes
• Network dynamics
• Mobility of nodes ensures that topology is
  constantly changing
• Context of node is changing dynamically
• Network scalability

12
Example Adaptive Behavior

• Benefits of Adaptation
• Improved Spectrum Utilization
• Improve QoS
• Many decisions may have to be localized
• Distributed Behavior
• Adaptations of one node can impact adaptations of
  others
• Interactive Decisions
• Difficult to Predict Performance

13
Critical Concepts From Game Theory

• Defining Game Theory, Basic Game Forms,
• Key Concepts, and Simple Examples

14
Game Theory

• Game Theory is a part of (applied) mathematics
  that describes and studies interactive decision
  problems.
• In an interactive decision problem the decisions
  made by each decision maker affect the outcomes
  and, thus, the resulting situation for all
  decision makers involved.
• The study of mathematical models of conflict and
  cooperation between intelligent rational
  decision-makers Myerson (1991)

15
A Little Game Theory History

• Game theory evolved from recreational mathematics
  (Borel, Zermelo) and the mathematical economics
  of general equilibrium analysis (von Neumann,
  Wald, Menger). The foundations were provided by
  von Neumann (1928), von Neumann and Morgenstern
  (1944), and Nash (1950, 1951 and 1953).

16
Mathematical Models

• Modeling some interactive process
• Necessitates some abstraction (sometimes quite a
  lot of abstraction)
• Many different models exist to analyze the same
  situation
• Formal analysis greatly aided by rigorous
  mathematical descriptions of models

17
Key Concepts

• Decision Makers
• Agents who can choose an action
• Must be multiple decision makers
• Conflict and Cooperation
• Key is interaction, i.e. actions of one agent
  (player) must impact others either negatively or
  positively
• Effect of interaction is dependent on selected
  model

18
Intelligent Rational Decision Making

• Actions taken by an agent must be in that agents
  self-interest
• Clearly defined objectives
• Expectation on how actions impact objectives

19
Game

• A game is model (mathematical representation) of
  an interactive decision situation.
• Its purpose is to create a formal framework that
  captures the relevant information in such a way
  that is suitable for analysis.
• Different situations indicate the use of
  different game forms.

20
Critical Components of a Game

• A (well-defined) set of 2 or more players
• A set of actions for each player.
• A set of preference relationships for each player
  for each possible action tuple.

More elaborate games exist with more components
but these three must always be there. Some also
introduce an outcome function which maps action
tuples to outcomes which are then valued by the
preference relations. Games with just these three
components (or a variation on the preference
relationships) are said to be in Normal form or
Strategic Form
21
Set of Players

• N set of n players consisting of players
  named 1, 2, 3,,i, j,,n
• Note the n does not mean that there are 14
  players in every game.
• Other components of the game that belong to a
  particular player are normally indicated by a
  subscript.
• Generic players are most commonly written as i or
  j.
• Usage N is the SET of players, n is the COUNT of
  players.
• N \ i 1,2,,i-1, i1 ,, n All players in N
  except for i

22
Actions
Example Two Player Action Space
Ai Set of available actions for player i ai A
particular action chosen by i, ai ? Ai A Action
Space, Cartesian product of all Ai AA1? A2?
? An a Action tuple a point (vector) in the
Action Space A-i Another action space A formed
from A-i A1? A2? ?Ai-1 ? Ai1 ? ?
An a-i A point from the space A-i A Ai ? A-i
A1 A2 0 ?)
AA1? A2
A2 A-1
A1 A-2
a1 a-2
23
Preference Relations
Preference Relation expresses an individual
players desirability of one outcome over another
(A binary relationship)
Preference Relationship (prefers at least as much
as)
a is preferred at least as much as a by player i
Strict Preference Relationship (prefers strictly
more than)
iff
but not
Indifference Relationship (prefers equally)
iff
and
24
Utility Functions
A mathematical description of preference
relationships.
Maps action space to set of real numbers.
Preference Relation then defined as
iff
iff
iff
25
Nash Equilibrium
A steady-state where each player holds a correct
expectation of the other players behavior and
acts rationally. - Osbourne
An action vector from which no player can
profitably unilaterally deviate.
Definition
An action tuple a is a NE if for every i ? N
for all bi ?Ai.
Note showing that a point is a NE says nothing
about the process by which the steady state is
reached. Nor anything about its uniqueness. Also
note that we are implicitly assuming that only
pure strategies are possible in this case.
26
How do the players find the Nash Equilibrium?

• Preplay Communication
• Before the game, discuss their options. Note
  only NE are suitable candidates for coordination
  as one player could profitably violate any
  agreement.
• Rational Introspection (Best Response)
• Based on what each player knows about the other
  players, reason what the other players would do
  in its own best interest. Points where everyone
  would be playing correctly are the NE.
• Focal Point
• Some distinguishing characteristic of the tuple
  causes it to stand out. The NE stands out
  because its every players best response.
• Trial and Error (Better Response with Errors)
• Starting on some tuple which is not a NE a player
  discovers that deviating improves its payoff.
  This continues until no player can improve by
  deviating. Only guaranteed to work for Potential
  Games (later).

27
Friend or Foe Example
(Friend, Friend)??
No
(Friend, Foe)??
(Foe, Friend)??
Yes
(Foe, Foe)??
Yes
28
A Cournot Duopoly Example
Coke and Pepsi each independently choose to
produce quantities of cola, QCoke and QPepsi.
These products are virtually indistinguishable
(for the purposes of this example), so the total
quantity of cola supplied to the market is Q
QCoke QPepsi. The price P that each can charge
is an inverse function of demand given by P 0.5
0.01 Q. It costs both firms 0.01 per unit
production.
What quantity should each firm produce?
Is this the optimal quantities from the point of
view of the firms?
29
Coke-Pepsi Duopoly as a Game
N Coke, Pepsi
ACoke APepsi 0, 49
Revenue
Price
Profit
30
Best Response Function

• For each a-i, player i has an action or set of
  actions such that the utility of i is maximized.
• We call this player is best response to a-i.
• Best response function (correspondence)

31
Nash Equilibrium and Best Responses

• Recall the definition of a Nash Equilibrium
• Note that the best response function for i
  identifies the ai such that for all bi ?Ai only
  the for all i portion is not satisfied.
• An action tuple a is a NE iff

An action tuple a is a NE if for every i ? N
for all bi ?Ai.
32
Coke-Pepsi Best Response Functions
33
Nash Equilibrium
34
What is the Best the Firms Could Do?
Suppose the two companies worked together to
maximize their total profits and each agreed to
make the same quantity. What would they earn?
35
Comments

• A game may have no NE (Paper-Rock-Scissors)
• A game may have many NE (Friend-Foe Example)
• A NE says nothing about efficiency or optimality
  of the result. In fact, a NE may be very bad.

36
General Application of Game Theory to MANETs

• Key Issues and How Game Theory Approaches The
  Issues

37
Cognitive Radio Operation

• Level
• 0 SDR
• 1 Goal Driven
• 2 Context Aware
• 3 Radio Aware
• 4 Planning
• 5 Negotiating
• 6 Learns Environment
• 7 Adapts Plans
• 8 Adapts Protocols

Establish Priority
Generate Alternate Goals
Immediate
Normal
Urgent
Determine Best Known Waveform
Generate Best Waveform
Negotiate
Negotiate Protocols
38
Minimum MANET Node Operation
Valuate
Decide
Decision Update
Observe
Metrics
Outside World
Act
Alter Waveform
39
Game Theory Serves As An Analytic Framework for
Modeling MANET Behavior
MANET Component
Game Component
Nodes in Network
Player Set
Available Adaptations
Action Set
Adaptation Algorithm
Valuation Function (Preference Relations)
Utility Function
Decision Update Algorithm
Learning Process
40
Game Theory and Wireless Networks
Solution of games Nash equilibriums yields
information on networks convergence and steady
states
41
Key Issues

• Steady State Existence
• Steady State Optimality
• Convergence
• Stability
• Scalability

Convergence How do initial conditions impact
the system steady state? How long does it
take to reach the steady state?
Stability How does system variations impact
the system? Do the steady states change?
Is convergence affected
Steady State Existence Is it possible to
predict behavior in the system? How many
different outcomes are possible?
Optimality Are these outcomes desirable?
Do these outcomes maximize the system target
parameters?
Scalability As the number of devices
increases, How is the system impacted?
Do previously optimal steady states remain
optimal?
42
How Issues Fit Into the Project
Fixed Portion April 03-Oct 04
Stochastic Portion Oct 04-April 06

• Steady State Existence
• Steady State Optimality
• Convergence
• s
• Scalability

• Steady State Existence
• Steady State Optimality
• Convergence
• Stability
• Scalability

43
Key Game Models

• Extensive Form Games, Repeated Games, Potential
  Games, and Supermodular Games

44
Motivating Extensive Form Games

• Models attempt to capture as much information as
  possible in as simple a format as possible.
• Often this requires abstracting away certain
  system details that are deemed unimportant to the
  analysis.
• To this point, we have only considered systems
  where all decision-making occurs simultaneously
  and where action sets are defined independently
  of the decision making process.
• Although many systems can be abstracted to fit
  these conventions, it is sometimes clumsy and if
  the analyst is not careful, may eliminate
  information that would alter the expected outcome.

45
Extensive Form Game Components

• A set of players.
• The actions available to each player at each
  decision moment (state).
• A way of deciding who is the current decision
  maker.
• Outcomes on the sequence of actions.
• Preferences over all outcomes.

46
Sequences of Actions
(ak)k1,2,,K
Action Sequence
A sequence of k actions where ak-1 immediately
preceded ak.
Example
a1 Jack goes up the hill
a2 Jill goes up the hill
(ak)k1,2 Jack goes up the hill, then Jill goes
up the hill (technically preceded
by the null history)
47
Histories
History Set

• H A set of action sequences with the following
  properties
• The empty sequence ? is a member of H
• If (ak)k1,2,,K ?H and L lt K then
  (ak)k1,2,,L ?H
• If an infinite sequence satisfies
  (ak)k1,2,,L ?H
• for every possible positive L then

History
h a member of H (an action sequence
(ak)k1,2,,K)
(h, a) The history h followed by the action a
A history (ak)k1,2,,K is said to be terminal
if it is infinite or if there is no aK1 such
that (ak)k1,2,,K1?H.
Z the set of terminal histories in H
? is the starting point of the game, and ? is a
member of all h ?H
48
Functions of History
Player Function
P H \ Z ? N
Maps each nonterminal history to a player set.
Action Set
A(h)a (h,a)?H
The set of available actions after h.
Preference Relations
Defined over Z for each i
h, h ? Z
Utility Functions
49
Example Extensive Form Game
Abe picks a number 1 or 2. Then Bill chooses a
number 1 or 2. Abe then pays Bill the absolute
difference between the two numbers.
Player Set
N A,B
H ?, 1, 2, (1,1), (1,2), (2,1), (2,2)
History Set
Terminal
Z (1,1), (1,2), (2,1), (2,2)
Player Fcn
Action Fcn
Utility Fcns
50
Game Tree
A compact representation of an extensive form
game.
Components Vertices (Nodes) correspond to
a particular
history Edges (Paths between nodes) -
correspond to actions
h ? ?
h ?
Vertices are normally labeled by P(h)
Edges are normally labeled by an element of A(h)
When a player is uncertain of the history,
vertices are grouped by an ellipse. Useful for
modeling simultaneous moves. Also represented by
linking nodes with a dashed line.
51
Extensive Form Game Example
Centipede Game
1
1
C
2,4
4,6
3,1
0,2
5,3
S
S
1,0
52
Repeated Games

• A specialized form of an extensive form game
  wherein at each stage, the same game is played.
• A particular stage of the repeated game is called
  a stage game.
• In each stage game, a subset of the players
  modify their behavior according to some
  predefined rule.
• Repeated Game may continue indefinitely
  Infinite Horizon
• Repeated Game may end after an expected number of
  stages Finite Horizon

53
Comments on Repeated Game Components

• Player Set
• Remains the same in each stage
• Actions
• Action space is the same in each stage
• Players play an action in each stage (need not
  change from previous stage)
• Determination of action in each stage is based on
  each players strategy, observations, insights
  about other players, and possibly negotiations
  (threats, promises), may learn
• Utility Function
• Players have the same utility function for each
  stage.
• The utility for the entire repeated game is some
  combination of the stage game utilities.
• Time Discounting More weight for more immediate
  payoffs

54
Comments on Play

• Myopic Processes
• Players have no knowledge about utility
  functions, or expectations about future play,
  typically can observe or infer current actions
• Best response dynamic maximize individual
  performance presuming other players actions are
  fixed
• Better response dynamic improve individual
  performance presuming other players actions are
  fixed

55
More Complex Play

• Strategies that punish or reward other players to
  influence their actions
• Grim Trigger
• Once a player deviates from agreed upon action
  tuple, remaining players attempt to minimize
  deviating players utility for remainder of game
• Tit-for-Tat
• Reward good behavior, punish bad behavior

56
Repeated Games

• Depending on structure and decision updating
  algorithm, play may or may not lead to NE of the
  stage game.
• By Folk Theorem, play can be forced to any
  feasible payoff vector with proper selection of
  punishment strategy and discount factor.

57
Example Repeated Game

• Stage Game
• 2 Players
• 2 Actions
• Stage Game Utility as Shown
• 3 Stages
• Repeated Game Utility sum of utility at each stage

Foe
Friend
Friend
-10,1000
500, 500
Foe
0,0
1000,-10
58
Better Response Dynamic
Foe
Friend
Game starts in (Friend, Friend)
Friend
-10,1000
500, 500
Foe
0,0
1000,-10
In stage 2, Player 2 improves his payoff (Friend,
Friend)
Note Convergence would have happened from any
point The convergence of the better response
dynamic is due to special properties of the stage
game.
In stage 3, Player 1 improves his payoff (Foe,
Foe)
59
Potential Games

• Key Properties
• Nash Equilibrium Exists
• Better Response (Myopic) Dynamic Converges
• Why We Care
• Steady state exists
• Virtually every decision updating process
  converges
• Minimal level of network complexity
• Once Modeled, Steady States Easy to Identify
  (Potential Function Maximizers)
• How to Identify

Or find an ordinal transformation for which this
works.
60
Exact Potential Games
Definition Exact Potential Game
A normal form game whose objective functions are
structured such that there exists some function
P A ?? which satisfies the following property
for all players
In other words it must be possible to construct a
single-dimensional function whose change in value
is exactly equal to the change in value of the
deviating player.
61
EPG Property 1
(Voorneveld)
A game G ltN, Aii?N , uii?Ngt is an exact
potential game iff there exist functions cii?N
and dii?N such that

• ui ci di
• ltN, Aii?N , cii?Ngt is a coordination game
• ltN, Aii?N , dii?Ngt is a dummy game

Outline of proof if The characteristic
function of the coordination game
is an exact potential function of G Only if Let
P be an exact potential of G. Clearly P forms
a coordination game. Now consider a
game with objective fcns given by
ui P. As the value of
deviating in this game is now 0 at all points,
this is a dummy game.
62
EPG Property 2
The NE of an exact potential game are coincident
with the NE of its constituent coordination game.
Outline of Proof
Any unilateral deviation in a dummy game yields
the same payoff. Adding a dummy game D to
another game G preserves Gs NE. All exact
potential games can be expressed as the sum of a
coordination game and a dummy game (EPG Property
1). Therefore the NE of the potential game must
be the same as the NE of the coordination game.
63
EPG Property 3
(Voorneveld)
For an EPG, the maximizers of the EPF are NE of
the EPG.
Outline of Proof
The NE of an EPG are the NE of its coordination
game (CG). By CG Property 3, the maximizers of
its characteristic functions (V) are NE. All EPF
can be expressed as V constant c. Since the
addition of the constant does not change which
tuples yield maximum payoffs, the maximizers of
the EPF are coincident with the maximizers of V,
thus coincident with the NE of the CG, thus
coincident with the NE of the EPG.
64
EPG Property 4
(Voorneveld)
Let the EPG be finite (finite action space,
finite player set), then the EPG has at least one
pure-strategy NE.
Outline of Proof
Note these conditions mean that the EPF must have
at least one maximum. By CG Property 4, this
must be a NE.
65
Continuous Action Sets
EPG Property 5 (Shapley)
Let G be a game in which the strategy sets are
closed intervals of ?. Suppose the objective
functions are continuously differentiable. A
function P is a potential iff P is continuously
differentiable and
for every i ? N
EPG Property 6 (Shapley)
If objective functions are twice differentiable
then a game is a EPG iff
for every i, j ? N
66
Ordinal Potential Games
Definition Ordinal Potential Game (OPG)
A normal form game whose objective functions are
structured such that there exists some function
P A ?? which satisfies the following property
for all players
In other words it must be possible to construct a
single-dimensional function where the sign of the
change in value is the same as the sign of the
change in value of the deviating player.
Note that an EPG also satisfies this definition.
67
Properties Shared with EPG
(Shapley)
For an OPG, the maximizers of the OPF are NE of
the OPG.
An OPG has at least one pure-strategy NE.
A finite OPG has FIP.
An OPG with continuous bounded action sets has
AFIP.
A repeated game with the same OPG stage also
converges with a better response dynamic.
68
Cycles
Consider a cycle ?, the sum of the changes in
value seen by the deviating players in an OPG is
not always 0.
a2
b2

• ((a1, a2), (b1, a2),
• (b1, b2), (a1, b2), (a1, a2))

a1
0,0
1,1
b1
I(?, u) 2 1 1 1 3
0,1
2,0
69
Weak Improvement Cycles
No known simple necessary and sufficient
condition like the second derivative condition of
EPG.
Non-deteriorating path
A path is an non-deteriorating path if for all k
? 1
Weak improvement cycle
A finite non-deteriorating path ? (a0, a1,,ak)
where ak a0
(Voorneveld) All OPG lack weak improvement cycles.
70
Ordinal Transformation
Definition Ordinal Transformations An ordinal
transformation is a one-to-one mapping of the
utility functions ui to a new set of utility
functions ui in such a way that the ordinality
of the preference and indifference relationships
for all players are maintained. This can be
restated as
71
OT Property 1
An ordinal transformation of an ordinal potential
game is itself an ordinal potential game.
Proof Outline
Since an OT preserves the ordering of all
preference relationships, if the original game
lacks weak improvement cycles, then the
transformed game must also lack weak improvement
cycles.
72
OT Property 2
If an ordinal transformation of a game yields an
ordinal potential game, then the original game
must also be an ordinal potential game.
Proof Outline
Since an OT preserves the ordering of all
preference relationships, if the transformed game
lacks weak improvement cycles, then so must the
original game.
73
OT Property 3
An ordinal transformation of an exact potential
game is an ordinal potential game (will remain an
EPG if OT is linear).
Proof Outline
An EPG is also an OPG. Apply the OT property 1.
74
A Quick Review
75
Brief Review of What Weve Covered

• Basic MANET Concepts
• Multihop, Self-organizing, Decentralized
  Infrastructure
• Analytic Difficulties
• Basic Game Theory Concepts
• Fundamental Modeling Concepts
• Players, Actions, Utility Functions
• Fundamental Solution Concepts
• Nash Equilibrium
• Relation of Game Theory and MANETs
• Nodes as Players
• Adaptations as Actions
• Adaptation Criteria as Utility (Preference
  Relations)
• Relevant Models
• Extensive Form Games
• Repeated Games
• Potential Games