Game Models for Cognitive Radio Analysis

1
Game Models for Cognitive Radio Analysis

• James O. Neel, Jeffrey H. Reed,
  Robert P. Gilles

2
Work Sponsors

• Office of Naval Research
• Grant Number N00014-03-1-0629
• National Science Foundation
• Integrated Research and Education in Advanced
  Networking an IGERT program
• MPRG Affiliates Program

Lucent Technologies Motorola Qualcomm SBC
Laboratories Texas Instruments
Analog Devices Army Research Office DRS
Technologies General Dynamics Huawei Technologies
Company
3
Presentation Objectives

• Describe how/when game theory applies to
  cognitive radio.
• Convey the value of adopting a model based
  approach instead of an ad-hoc approach.
• Disabuse the research community of the habit of
  solely relying on Pareto efficiency
• Highlight some valuable game models.

4
Games

• A game is a model (mathematical representation)
  of an interactive decision situation.
• Its purpose is to create a formal framework that
  captures the relevant information in such a way
  that is suitable for analysis.
• Different situations indicate the use of
  different game models.

Normal Form Game Model

• A set of 2 or more players, N
• A set of actions for each player, Ai
• A set of utility functions, ui, that describe
  the players preferences over the outcome space

5
How a Normal Form Game Works
Player 1
Player 2
Actions
Actions
Action Space
Decision Rules
Decision Rules
Outcome Space
u1
u2
-1
1
1 WINS!
2 WINS!
TIE!
6
Cognition Cycle

• Level
• 0 SDR
• 1 Goal Driven
• 2 Context Aware
• 3 Radio Aware
• 4 Planning
• 5 Negotiating
• 6 Learns Environment
• 7 Adapts Plans
• 8 Adapts Protocols

Select Alternate Goals
Generate Alternate Goals
Establish Priority
Immediate
Normal
Urgent
Determine Best Known Waveform
Generate Best Waveform
Negotiate
Negotiate Protocols
Adapted From Mitola, Cognitive Radio for
Flexible Mobile Multimedia Communications , IEEE
Mobile Multimedia Conference, 1999, pp 3-10.
7
The Cognition Cycle is a Player

• Level
• 0 SDR
• 1 Goal Driven
• 2 Context Aware
• 3 Radio Aware
• 4 Planning
• 5 Negotiating
• 6 Learns Environment
• 7 Adapts Plans
• 8 Adapts Protocols

Utility Function
Utility function Arguments
Select Alternate Goals
Generate Alternate Goals
Establish Priority
Immediate
Normal
Urgent
Outcome Space
Decision Rules
Determine Best Known Waveform
Generate Best Waveform
Action Sets
Negotiate
Negotiate Protocols
Adapted From Mitola, Cognitive Radio for
Flexible Mobile Multimedia Communications , IEEE
Mobile Multimedia Conference, 1999, pp 3-10.
8
Cognitive Radio Network as a Game
Radio 1
Radio 2
Actions
Actions
Action Space
Decision Rules
Decision Rules
Informed by Communications Theory
u2
Outcome Space
u1
9
When Game Theory can be Applied

• Level
• 0 SDR
• 1 Goal Driven
• 2 Context Aware
• 3 Radio Aware
• 4 Planning
• 5 Negotiating
• 6 Learns Environment
• 7 Adapts Plans
• 8 Adapts Protocols

Select Alternate Goals
Establish Priority
Generate Alternate Goals
Immediate
Normal
Urgent
Generate Best Waveform
Determine Best Known Waveform
Game Theory applies to 1. Adaptive aware radios
2.
Cognitive radios that learn about
their environment
Negotiate
Negotiate Protocols
10
Key Issues in Analysis

• Steady state characterization
• Steady state optimality
• Convergence
• Stability
• Scalability

Steady State Characterization Is it possible
to predict behavior in the system? How many
different outcomes are possible?
Optimality Are these outcomes desirable?
Do these outcomes maximize the system target
parameters?
Convergence How do initial conditions impact
the system steady state? What processes will
lead to steady state conditions? How long
does it take to reach the steady state?
Stability How does system variations impact
the system? Do the steady states change?
Is convergence affected?
Scalability As the number of devices
increases, How is the system impacted?
Do previously optimal steady states remain
optimal?
11
How Game Theory Addresses These Issues

• Steady-state characterization
• Nash equilibrium existence
• Identification requires side information
• Steady-state optimality
• Pareto optimality
• Convergence
• Learning processes
• Stability, scalability
• No general techniques
• Requires side information

12
Nash Equilibrium Existence

• Frequently shown with the aid of fixed point
  theorems.
• Given
• is nonempty, compact, and
  convex
• ui is continuous in a, and quasi-concave in ai
  (implies BR A?A is upper-semi continuous)
• Then the game has a Nash Equilibrium

13
Nash Equilibrium Identification

• Time to find all NE can be significant
• Let tu be the time to evaluate a utility
  function.
• Search Time
• Example
• 4 player game, each player has 5 actions.
• NE characterization requires 4x625 2,500 tu
• Desirable to introduce side information.

14
Pareto Efficiency
Almost Worthless!

• Formal Definition An action vector a is Pareto
  efficient if there exists no other action vector
  a, such that
• with at least one player strictly greater.
• Informal Definition An action tuple is Pareto
  efficient if some players must be hurt in order
  to improve the payoff of other players.
• Important Notes
• An action tuple which is a NE need not be Pareto
  optimal.
• An action tuple which is Pareto optimal need not
  be a NE.
• Generally, it is desirable for a NE to be Pareto
  optimal, but this need not be the case..

15
Being Mislead by Pareto Optimality

• Scenario Distributed SINR maximizing power
  control in a single cluster.
• Unique NE All nodes transmit at maximum power.
• Though clearly undesirable, NE is Pareto optimal.

Power
SINR
Preferable Approach Demonstrate NE maximizes
a design objective function.
16
Repeated Games and Convergence

• Repeated Game Model
• Consists of a sequence of stage games which are
  repeated a finite or infinite number of times.
• Most common stage game normal form game.

• Finite Improvement Path (FIP)
• From any initial starting action vector, every
  sequence of round robin better responses
  converges.
• Weak FIP
• From any initial starting action vector, there
  exists a sequence of round robin better responses
  that converge.

17
Better Response Dynamic

• During each stage game, player(s) choose an
  action that increases their payoff, presuming
  other players actions are fixed.
• Converges if stage game has FIP.

B
A
a
1,-1
0,2
b
-1,1
2,2
18
Best Response Dynamic

• During each stage game, player(s) choose the
  action that maximizes their payoff, presuming
  other players actions are fixed.
• May converge with weak FIP.

B
A
C
a
-1,1
1,-1
0,2
1,-1
b
-1,1
1,2
c
2,1
2,0
2,2
19
Potential Game Model

• Existence of a potential function V such that

• Identification
• NE Properties (assuming compact spaces)
• NE Existence All potential games have a NE
• NE Characterization Maximizers of V are NE
• Convergence
• Better response algorithms converge.

• Stability
• Maximizers of V are stable
• Design note
• If V is designed so that its maximizers are
  coincident with your design objective function,
  then NE are also optimal.

20
Supermodular Game Model

• A game such that
• Action space is a lattice
• Utility functions are supermodular
• Identification
• NE Properties (assuming compact spaces)
• NE Existence All supermodular games have a NE
• NE Characterization NE form a lattice
• Convergence
• Best response algorithms converge.
• Stability
• NE Lattice is stable

21
Example Potential Games

• Menon Fair Interference Avoidance (Session 1.4)
• Neel SDR02 specialized ad-hoc power control
  and waveform adaptations
• Single Cell Power Control target SINR
• Ad-hoc power control target SINR (fixed
  assignment)
• Hicks Globecom04 Littoral combat interference
  avoidance
• Lau - Aloha

22
Example Supermodular Games

• Yates Standard Interference Function (Altman)
• Goodman et al linear priced power control

23
Why Use Game Models?

• Introduces valuable side information for simply
  establishing important analytic results.
• NE existence
• NE characterization
• Convergence
• Stability
• Can also be used to establish network complexity
  Neel WCNC04.

24
Conclusions

• Game theory applies to cognitive radio levels
  1-6.
• Demonstrate that a system objective function is
  maximized, rather than showing Pareto efficiency.
• Use of game models can greatly simplifies
  analysis.
• Choice of goal and allowable adaptations largely
  determine applicable models.
• When possible, try to design algorithms that are
  potential games or supermodular games.