Game Theory

1
Game Theory Cognitive Radiopart A
Hamid Mala
2
Presentation Objectives

1. Basic concepts of game theory
2. Modeling interactive Cognitive Radios as a game
3. Describe how/when game theory applies to
   cognitive radio.
4. Highlight some valuable game models.

3
Interactive Cognitive Radios

• Adaptations of one radio can impact adaptations
  of others
• Interactive Decisions
• Difficult to Predict Performance

4
Interactive Cognitive Radios

• Scenario Distributed SINR maximizing power
  control in a single cluster.
• Final state All nodes transmit at maximum
  power.
• (1) the resulting SINRs are unfairly distributed
  (the closest node will have a far superior SINR
  to the furthest node)
• (2) battery life would be greatly shortened.

Power
SINR
5
traditional analysis techniques

• Dynamical systems theory
• optimization theory
• contraction mappings
• Markov chain theory

6
Research in a nutshell

• Applying game theory and game models (potential
  and supermodular) to the analysis of cognitive
  radio interactions
• Provides a natural method for modeling cognitive
  radio interactions
• Significantly speeds up and simplifies the
  analysis process
• Permits analysis without well defined decision
  processes

7
Game Theory

• Definition, Key Concepts

8
Exaple
Same color winner
opposite color winner
card number of winner
9
Exaple
Same color winner
opposite color winner
card number of winner
10
Exaple
Matrix representation
(2,-2)
(-8,8)
(-1,1)
(7,-7)
11
Games

• A game is a model (mathematical representation)
  of an interactive decision situation.
• Its purpose is to create a formal framework that
  captures the relevant information in such a way
  that is suitable for analysis.
• Different situations indicate the use of
  different game models.

Normal Form Game Model

1. A set of 2 or more players, N
2. A set of actions for each player, Ai
3. A set of utility functions, ui, that describe
   the players preferences over the outcome space

12
Nash Equilibrium
An action vector from which no player can
profitably unilaterally deviate.
Definition
An action tuple a is a NE if for every i ? N
for all bi ?Ai.
13
Friend or Foe Example
(Friend, Friend)??
No
(Friend, Foe)??
(Foe, Friend)??
Yes
(Foe, Foe)??
Yes
14
Modeling and Analysis Review
15
Modeling a Network as a Game
Network
Game
Nodes
Players
Power Levels
Actions
Algorithms
Utility Functions
Structure of game is taken from the algorithm and
the environment
Laboratoire de Radiocommunications et de
Traitement du Signal
16
Modeling Review

• The interactions in a cognitive radio network
  can be represented by the tuple
• ltN, A, ui, di,Tgt
• Timings
• Synchronous
• Round-robin
• Random
• Asynchronous

Dynamical System
17
Key Issues in Analysis

1. Steady state characterization
2. Steady state optimality
3. Convergence
4. Stability
5. Scalability

Steady State Characterization Is it possible
to predict behavior in the system? How many
different outcomes are possible?
Optimality Are these outcomes desirable?
Do these outcomes maximize the system target
parameters?
Convergence How do initial conditions impact
the system steady state? What processes will
lead to steady state conditions? How long
does it take to reach the steady state?
Stability How does system variations impact
the system? Do the steady states change?
Is convergence affected?
Scalability As the number of devices
increases, How is the system impacted?
Do previously optimal steady states remain
optimal?
18
How Game Theory Addresses These Issues

• Steady-state characterization
• Nash Equilibrium existence
• Identification requires side information
• Steady-state optimality
• In some special games
• Convergence
• in some cases
• Stability, scalability
• No general techniques
• Requires side information

19
Nash Equilibrium Identification

• Time to find all NE can be significant
• Let tu be the time to evaluate a utility
  function.
• Search Time
• Example
• 4 player game, each player has 5 actions.
• NE characterization requires 4x625 2,500 tu
• Desirable to introduce side information.

20
Example(1) The Cognitive Radios Dilemma
Example The Cognitive Radios Dilemma

• Two cognitive radios
• Each radio can implement two different waveforms
• low-power narrowband
• higher power wideband

Frequency domain representation of waveforms
The Cognitive Radios Dilemma in Matrix
NE?
21
Repeated Games and Convergence

• Finite Improvement Path (FIP)
• From any initial starting action vector, every
  sequence of round robin better responses
  converges.
• Weak FIP
• From any initial starting action vector, there
  exists a sequence of round robin better responses
  that converge.

• Repeated Game Model
• Consists of a sequence of stage games which are
  repeated a finite or infinite number of times.
• Most common stage game normal form game.

22
Better Response Dynamic

• During each stage game, player(s) choose an
  action that increases their payoff, presuming
  other players actions are fixed.
• Converges if stage game has FIP.

B
A
a
1,-1
0,2
b
-1,1
2,2
23
Best Response Dynamic

• During each stage game, player(s) choose the
  action that maximizes their payoff, presuming
  other players actions are fixed.
• converge if stage game has weak FIP.

B
A
C
a
-1,1
1,-1
0,2
1,-1
b
-1,1
1,2
c
2,1
2,0
2,2
24
Supermodular Games

• Key Properties
• Best Response (Myopic) Dynamic Converges
• Nash Equilibrium Generally Exists
• Why We Care
• Low level of network complexity
• How to Identify

25
Supermodulaar Games

• NE Existence have at least one NE.
• NE Identification all NE for a game form a
  lattice. While this does not particularly aid in
  the process of initially identifying NE, from
  every pair of identified
• Convergence have weak FIP, so a sequence of best
  responses will converge to a NE.
• Stability if the radios make a limited number
  of errors or if the radios are instead playing a
  best response to a weighted average of
  observations from the recent past, play will
  converge.

26
Example outer loop power control

• Parameters
• Single Cluster
• Pi Pj 0, Pmax ? i,j ?N
• Utility target SINR
• Supermodular best response convergence

27
Summary

• When we use game theory to model and analyse
  interactive CRs, it should address
• steady state existense and identification
• convergence
• stability
• desirability of steady states
• Supermodular games to some extent

28
Questions?
29
Game Theory Cognitive Radiopart B
Mahdi Sadjadieh
30
Overview

• Potential Game Model
• Type of Potential Game
• Example of Exact Potential Game
• FIP and Potential Games
• How Potential Games handle the shortcomings
• Physical Layer Model Parameters and Potential
  Game

31
Potential Game Model

• Existence of a potential function V such that

• Identification
• NE Properties (assuming compact spaces)
• NE Existence All potential games have a NE
• NE Characterization Maximizers of V are NE
• Convergence
• Better response algorithms converge.

• Stability
• Maximizers of V are stable
• Design note
• If V is designed so that its maximizers are
  coincident with your design objective function,
  then NE are also optimal.

32
Potential Games

• Existence of a function (called the potential
  function, V), that reflects the change in utility
  seen by a unilaterally deviating player.

E1 E2 E3 E4
G?PG?GOPG (Gilles) OPG ? G?PG (finite A)
33
Potential Games
34
Ordinal Potential Game Identification

• Lack of weak improvement cycles Voorneveld_97
• FIP and no action tuples such that
• Better response equivalence to an exact potential
  game Neel_04

Not an OPG
An OPG
35
Ordinal Potential Game Identification

• Lack of weak improvement cycles Voorneveld_97
• FIP and no action tuples such that
• Better response equivalence to an exact potential
  game Neel_04

Not an OPG
An OPG
36
Other Exact Potential Game Identification
Techniques

• Linear Combination of Exact Potential Game Forms
  Fachini_97
• If ltN,A,uigt and ltN,A,vigt are EPG, then
  ltN,A,?ui ?vigt is an EPG
• Evaluation of second order derivative
  Monderer_96

37
Exact Potential Game Forms

• Many exact potential games can be recognized by
  the form of the utility function

38
Example Identification

• Single cluster target SINR
• Better Response Equivalent

39
FIP and Potential Games

• GOPG implies FIP (Monderer_96)
• FIP implies GOPG for finite games
  (Milchtaich_96)
• Thus we have a non-exhaustive search method for
  identifying when a CRN game model has FIP.
• Thus we can apply FIP convergence (and noise)
  results to finite potential games.

40
Steady-states

• As noted previously, FIP implies existence of NE

41
Optimality

• If ui are designed so that maximizers of V are
  coincident with your design objective function,
  then NE are also optimal.

• () Can also introduce cost function to utilities
  to move NE.
• In theory, can make any action tuple the NE
• May introduce additional NE
• For complicated NC, might as well completely
  redesign ui

V
a
42
Convergence in Infinite Potential Games

• ?-improvement path
• Given ? gt0, an ?-improvement path is a path such
  that for all k?1, ui(ak)gtui(ak-1) ? where i is
  the unique deviator at step k.
• Approximate Finite Improvement Property (AFIP)
• A normal form game, ?, is said to have the
  approximate finite improvement property if for
  every ?gt0 there exists an such that the length of
  all ?-improvement paths in ? are less than or
  equal to L.
• Monderer_96 shows that exact potential games
  have AFIP, we showed that AFIP implies a
  generalized ?-potential game.

43
Convergence Implications
44
How potential games handle the shortcomings

• Steady-states
• Finite game NE can be found from maximizers of V.
  
• Optimality
• Can adjust exact potential games with additive
  cost function (that is also an exact potential
  game)
• Sometimes little better than redesigning utility
  functions
• Game convergence
• Potential game assures us of FIP (and weak FIP)
• DV satisfy Zangwills (if closed)
• Noise/Stability
• Isolated maximizers of V have a Lyapunov function
  for decision rules in DV
• Remaining issue
• Can we design a CRN such that it is a potential
  game for the convergence, stability, and
  steady-state identification properties
• AND ensure steady-states are desirable?

45
More Examples
46
Physical Layer Model Parameters
47
SINR Power Control Games
Assume that there is a radio network wherein each
radio can alter their power.
Assume each radio reacts to some separable
function of SINR, e.g. log ratio
Each radio would also like to minimize power
consumption
Decentralized Power Control Using a dB Metric
Thus game is a potential game and convergence is
assured and we can quickly find steady states.
48
Example Power Control Game

• Parameters
• Single Cluster
• DS-SS multiple access
• Pi Pj 0, Pmax ? i,j ?N
• Utility target BER

Also a potential game.
49
Snapshot inner outer loop power control

• Parameters
• Single Cluster
• DS-SS multiple access
• Pi Pj 0, Pmax ? i,j ?N
• Utility target SINR
• Supermodular best response convergence

50
Game Models, Convergence, and Complexity

• Determining the kind of game required to
  accurately model a RRM algorithm yields
  information about what updating processes are
  appropriate and thus indicates expected network
  complexity.
• In Neel04 the following relation between power
  control algorithms, game models, and network
  complexity was observed.

51
Summary

• Distributed dynamic resource allocations have the
  potential to provide performance gains with
  reduced overhead, but introduce a potentially
  problematic interactive decision process.
• Game theory is not always applicable.
• Can generally be applied to distributed radio
  resource management schemes.

52
Questions?
53
ExampleExact Poential Game
return
54
return
55
Example Ordinal Poential Game
return
56
Example Generalized Ordinal Poential Game
return
57
Exact Potential Game Forms

• Many exact potential games can be recognized by
  the form of the utility function