Application of Game Theory to the Analysis of Radio Resource Management

1
Application of Game Theory to the Analysis of
Radio Resource Management

• James Neel
• October 13, 2004

2
Presentation Objectives

• At the end of this presentation (hopefully)
  youll
• Be able to read most game theory papers
• Understand how (and when) game theory applies to
  radio resource management
• Have a feel for where game theory research is
  headed
• Think our group is really cool

3
Presentation Overview

• Overview of Game Theory
• Applying Game Theory to Radio
  Resource Management
• Example GPRS Analysis
• Ad-hoc approach
• Model based approach
• Example Ad-hoc Network Analyses
• Power control
• Sensor network formation

(30 minutes)
(10 minutes)
(30 minutes)
(20 minutes)
4
What is Game Theory?

• Game Theory is a part of (applied) mathematics
  that describes and studies interactive decision
  problems.
• In an interactive decision problem the decisions
  made by each decision maker affect the outcomes
  and, thus, the resulting situation for all
  decision makers involved.
• The study of mathematical models of conflict and
  cooperation between intelligent rational
  decision-makers Myerson (1991)

5
Games

• A game is model (mathematical representation) of
  an interactive decision situation.
• Its purpose is to create a formal framework that
  captures the relevant information in such a way
  that is suitable for analysis.
• Different situations indicate the use of
  different game models.
• Important questions
• Does the game have a steady state? (NE Existence)
• What are those steady states? (NE
  Characterization)
• Is the steady state(s) desirable? (NE Optimality)
• When will play reach the steady state(s)?
  (Convergence)

6
Normal Form Game Model
Components

• A set of 2 or more players
• A set of actions for each player
• A set of utility functions that describe the
  players preferences over the action space

Player 1 Actions a, b Player 2 Actions A, B
States from which no player can unilaterally
deviate and improve are Nash Equilibriums
7
Nash Equilibrium
A steady-state where each player holds a correct
expectation of the other players behavior and
acts rationally. - Osborne
An action vector from which no player can
profitably unilaterally deviate.
Definition
An action tuple a is a NE if for every i ? N
for all bi ?Ai.
Note showing that a point is a NE says nothing
about the process by which the steady state is
reached. Nor anything about its uniqueness. Also
note that we are implicitly assuming that only
pure strategies are possible in this case.
8
Red River Shootout

• Consider a single play. Abstractly, Mack Brown
  can call a run or a pass play, and Bob Stoops can
  call a run or a pass defense.
• Game Model
• Player Set Mack Brown, Bob Stoops
• Action Sets R,P
• Utility Functions Yards Gained (Lost)
• Notes
• No pure Nash equilibrium
• Nash equilibrium does exist in mixed strategies
• Example of a zero-sum game

Bob Stoops
R
P
2
8
r
Mack Brown
p
15
0
9
More Nash Equilibrium

• Best Response Set

Best Response Function (Correspondence)
Nash Equilibrium
An action tuple a is a NE iff
A NE is a point a in action space where the best
response functions map back onto a. Note that
this is a fixed point.
Determining if a game has a NE is equivalent to
determining if B() has a fixed point.
10
When will there be a fixed point?

• Given a mapping a point
• is said to be a fixed point of f if

In 2-D fixed points for f can be found by
evaluating where and
intersect.
1
f(x)
How much information do we need to have to know
that a function has a fixed point/Nash
equilibrium?
x
1
0
11
Fixed Point Theorems

• Brouwers fixed point theorem
• Let f X? X be a continuous function from a
  non-empty compact convex set X ? ?n, then there
  is some x?X such that f(x) x.

You Can't Comb the Hair on a Coconut Without
There Being a Whorl.

• Note originally written as f B? B where B x ?
  ?n x?1 the unit n-ball
• Kakutanis fixed point theorem
• Let f X? X be a upper semi-continuous convex
  valued correspondence from a non-empty compact
  convex set X ? ?n, then there is some x?X such
  that x ? f(x)

12
Glicksberg-Fan (Debreu) NE Existence

• Given
• is nonempty, compact, and
  convex
• ui is continuous in a, and quasi-concave in ai
  (implies BR A?A is upper-semi continuous)
• Then the game has a Nash Equilibrium

Note that these conditions are just an
application of Kakutanis. There also exists a
non Euclidean GF NE theorem.
13
Quasi-concavity
A function f X ?X, X?? is quasi-concave if
for all ??0,1, x, y ? X
f(x)
Upper Level Set
where a??
Pu
Equivalent definition of quasi-concave
a
x
f is quasi-concave if Pu is convex for all a
14
Ok No More Topology I promise
15
Nash Equilibrium Characterization

• Consider a normal form game with 4 players, with
  each player having 5 actions.
• How long does it take to identify all NE in the
  game?
• To evaluate if a particular action tuple is a NE,
  we must evaluate and compare utility of 4(5-1)
  16 action vectors or Nx(Ai-1) a polynomial
  time problem assuming ui is polynomial
• Size of action space is 54625 or AiN
  evaluations

• Thus to finding all NE is given by AiN
  Nx(Ai-1) x tu NxAiN1x tu where
  tu is the time to evaluate a utility function.
• In the general case, NE characterization is a
  NP-complete problem.
• When possible, we would like to introduce side
  knowledge to limit the search space.

16
Nash Equilibrium Desirability

• Typically determined by demonstrating NE is
  Pareto Efficient.
• An action tuple a is pareto efficient if there
  exists no other action vector a, such that
• with at least one player strictly greater.
• However, this is a very weak concept as lots of
  very undesirable outcomes are pareto efficient.
  Neel SDR Forum 2004
• Generally preferable to demonstrate that NE
  maximizes some objective function.

17
Convergence

• Requires that game models include the notion of
  time.
• Issues such as order of play, memory, and the
  ability to predict the actions of other players
  can be important.

18
Extensive Form Game Model
Components

• A set of players.
• The actions available to each player at each
  decision moment (state).
• A way of deciding who is the current decision
  maker.
• Outcomes on the sequence of actions.
• Preferences over all outcomes.

1
1
C
Example of backwards induction
2,4
4,6
3,1
0,2
5,3
S
S
1,0
19
Repeated Game Model

• A stage game (normal form game) is played
  repeatedly with payoffs after each stage
• Game can continue indefinitely (infinite horizon)
  or end at a specified time (finite horizon)
• Players may consider future payoffs when playing
  and may consider when future payoffs will occur

Example stage game
20
Comments on Play in Repeated Games

• Common Player Functions
• Simultaneous (Synchronous)
• Round Robin
• Random Subsets (Asynchronous)
• Decision Update Processes
• Players have no knowledge about utility
  functions, or expectations about future play,
  typically can observe or infer current and past
  actions
• Best response dynamic maximize individual
  performance presuming other players actions are
  fixed
• Better response dynamic improve individual
  performance presuming other players actions are
  fixed

21
Better Response Dynamic

• During each stage game, player(s) choose an
  action that increases their payoff, presuming
  other players actions are fixed

B
A
a
1,-1
0,2
b
-1,1
2,2
22
Best Response Dynamic

• During each stage game, player(s) choose the
  action that maximizes their payoff, presuming
  other players actions are fixed

B
A
C
a
-1,1
1,-1
0,2
b
1,-1
-1,1
1,2
c
2,1
2,0
2,2
23
Key Myopic Convergence Properties

• Finite Improvement Path (FIP)
• From any initial starting action vector, every
  sequence of round robin better responses
  converges.
• Weak FIP
• From any initial starting action vector, there
  exists a sequence of round robin better responses
  that converge.

24
Repeated Game Communication

• Players agree to play in a certain manner
• Threats can force play to almost any state
• Breaks down with finite horizon

C
N
Nada
-5,5
0,0
nada
-100,0
c
-1,1
5,-5
-100,-1
-100,-100
n
-1,-100
0,-100
25
Potential Games

• How to identify
• Or find an ordinal (monotonic) transformation
• Key properties
• Nash equilibrium exists
• Better response dynamic converges (FIP)
• Stable steady states given by maximizers of
  potential function
• Why we care
• Steady state exists
• Virtually every decision updating process
  converges (no communication required) as
  game has FIP
• Once modeled, steady states easy to identify
  (potential function maximizers)

26
Supermodular Games

• How to identify
• Key properties
• Best response dynamic converges (gives the game
  weak FIP)
• Nash equilibrium generally exists
• Why we care
• Most decision updating processes converge
• (no communication required)

27
Why we care about game models?

• Cognitive radio networks can be modeled as a game
• Game model provides insights into network
  complexity
• When possible, strive for potential games

Game Model
Implied Complexity
CRN
Repeated
MacKenzie
High
Altman
S-modular
Low
Yates
S-modular
Low
OPG
Minimal
Target SINR
OPG
Minimal
Goodman
S-modular
Low
Sung
Repeated
High
28
My Fun Tricks (1/3)

• Best Response Transformations
• Suppose ui are modified such that for all
  action vectors, the best response sets remain the
  same. Then the set of NE remain the same and best
  response convergence properties remain the same.

29
My Fun Tricks (2/3)

• Monotone Transformations
• Suppose ui are modified such that ui(a)gtui(b)
  iff ui(a)gtui(b). Then the better response sets
  remain the same. Then the set of NE remain the
  same and best and better response convergence
  properties remain the same.

30
My Fun Tricks (3/3)

• Better Response Transformations
• Suppose ui are modified such that
  ui(ai,a-i)gtui(bi, a-i) iff ui(ai,a-i)gtui(bi,
  a-i) for all a. Then the better response sets
  remain the same. Then the set of NE remain the
  same and best and better response convergence
  properties remain the same.

31
Application of Game Theory to Wireless Networks
32
Application of Game Theory

• Game Theory is a part of (applied) mathematics
  that describes and studies interactive decision
  problems.
• Modeling
• Describe existing interactive decision algorithms
• Analysis
• Analyze (study) and design interactive decision
  algorithms

33
Modeling a Network as a Game
Network
Game
Nodes
Players
Adaptations
Actions
Algorithms
Utility Functions Learning
34
Conditions for Applying Game Theory to RRM

• Conditions for rationality
• Well defined decision making processes
• Expectation of how changes impacts performance
• Conditions for a nontrivial game
• Multiple interactive decision makers
• Nonsingleton action sets
• Conditions generally satisfied by distributed
  dynamic RRM schemes

35
Example Application Appropriateness

• Inappropriate applications
• Cellular Downlink power control (single cell)
• Site Planning
• Appropriate applications
• Distributed power control on non-orthogonal
  waveforms
• Ad-hoc power control
• Cell breathing
• Adaptive MAC
• Network formation (localized objectives)

36
Analyzing Distributed Dynamic Behavior

• Dynamic benefits
• Improved spectrum utilization
• Improve QoS
• Many decisions may have to be localized
• Distributed behavior
• Adaptations of one radio can impact adaptations
  of others
• Interactive decisions
• Difficult to predict performance

37
Key Issues in Analysis

• Steady state existence
• Steady state optimality
• Convergence
• Stability
• Scalability

Convergence How do initial conditions impact
the system steady state? What processes will
lead to steady state conditions? How long
does it take to reach the steady state?
Steady State Existence Is it possible to
predict behavior in the system? How many
different outcomes are possible?
Optimality Are these outcomes desirable?
Do these outcomes maximize the system target
parameters?
Stability How does system variations impact
the system? Do the steady states change?
Is convergence affected?
Scalability As the number of devices
increases, How is the system impacted?
Do previously optimal steady states remain
optimal?
38
Example Applicationsand Results
39
GPRS Networks

• Define a basic game
• Krishnaswamy analysis
• Ginde analysis
• and design

40
Basic GPRS Game Model

• Stage Game
• Player Set Co-channel interfering
  links
• Action Sets
• Power (Closed Interval)
• Rate (discrete set)
• Utility Functions
• Some form of throughput (really goodput)

41
Sigmoid Approx. of Throughput
Throughput L(x)
SINR hence throughput is a function of all
players powers gt Interaction
42
Krishnaswamy Game

• Considered in D. Krishnaswamy, Game Theoretic
  Formulations for Network-assisted Resource
  Management in Wireless Networks, Fall VTC 2002.
• Utility Function
• x is actual SINR, and Ku is a target SINR
  (actually the SINR that maximizes a curvature
  function)

43
Krishnaswamy Game

• Asserts (rather weakly) that game has a NE and
  that NE is Pareto efficient
• Basic proof
• There exists a SINR such that
• At this point player i would have no incentive to
  change its power/rate choice.
• Therefore point is a NE (and must be Pareto
  optimal).

44
Krishnaswamy Game

• Problem with proof
• Just because there is a xi such that each player
  i has no incentive to change doesnt mean that
  there is a feasible power/rate vector that
  corresponds to that xi. To demonstrate NE
  existence, application of first derivative
  conditions must be satisfied simultaneously for
  all players.
• However, the NE result does hold assuming a
  compact convex action space.
• Paper doesnt deal with convergence.

45
Ginde Game

• Considered in S. Ginde, Game Theoretic Analysis
  of Joint Link Adaptation and Distributed Power
  Control in GPRS, Fall VTC 2003.
• Utility Function

Penalty function
Throughput
46
Ginde Throughput Details
47
Ginde NE Existence (1/2)

• Applies Glicksburg-Fan-Debreu (ok I lied, heres
  a little topology again) for when rates are held
  constant.
• Action Set (closed interval of powers) is
  nonempty, compact, and convex
• ui is continuous in a, and quasi-concave in ai
• Thus the game has a Nash Equilibrium

48
Ginde NE Existence (2/2)

• For non constant rates, demonstrates existence of
  NE through simulation, shows that the NE are not
  unique, though tightly packed.

49
Algorithm LAG

• Iteratively calculates new P,r until there is a
  negligible change in P and r remains constant
  between iterations
• When P is unchanged, utility will be unchanged
  from previous iteration, hence r cannot change
  either.
• Otherwise, P and r are chosen to improve utility
  over previous value
• Note this is a best response dynamic

50
Algorithm LAG Convergence

• Algorithm LAG can be proved to converge to a NE
  starting from any initial vector (P,r)
• Proof Ginde Thesis Chapter 4 is based on
  convergence theory in Chapter 7 of Bazaraa,
  M.S., et al., Non Linear Programming Theory and
  Algorithms, 2ed, John Wiley and Sons, 1993.
• Note ordinal (monotonic) transformations preserve
  convergence.

51
Optimality

• We use the FOMs to evaluate the effect of
  different initial rates, and different values of
  q and K
• FOM1 and FOM2 trade off throughput and power
  efficiency
• FOM3 is simply the system throughput

System Throughput
Peak throughput scaling
Power Efficiency
52
Simulation Configuration

• GPRS Downlink, Frequency re-use factor 3
• First tier of interferers only
• Log-distance path loss, No shadowing
• Static simulation, MS positions fixed
• Maximum transmit power 10 mW
• Noise same for all players

53
An Example Simulation
K1, q 0.7, Initial Rates CS-1 for all links
Throughput Assignment Intuitive
Nash Equilibrium
54
Throughput Power Tradeoff

• The value of q may be selected to find the best
  trade-off between system throughput and power
  consumption

55
Effect of q on Figures of Merit
Optimal value of q
FOM1 q 4.9 FOM2 q 3.6 FOM3 q 0.9
56
Related Paper

• M. Hayajneh, C. T. Abdallah, Distributed Joint
  rate and Power Control Game-Theoretic Algorithms
  for Wireless Data, IEEE Comm. Letters, Aug 2004,
  pp 511-513.
• Not GPRS, but considers joint power and rate
  adaptation (assumes convex rates).
• Shows that power only game (penalized throughput)
  is a standard interference function.
• Shows that rate only game (penalized throughput)
  is a standard interference function.
• By Altman, both games are supermodular.
• Proposes a decision update algorithm like LAG
  recursively computes best power and then best
  rate.
• Shows convergence through simulation.
• Note that as its really just a ordinal
  transformation of Ginde convergence proof would
  hold.

57
Key Ideas from GPRS Examples

• GPRS networks with joint power/rate adaptations
  can be modeled as a game.
• Network has a steady state.
• LAG algorithm converges.
• Behavior can be influenced through the
  introduction of a penalty function.

58
Ad-hoc Power Control

• Neel, WOW 2004

59
Ad-hoc Power Control

• Goal Manage interference to achieve desired QoS
  and capacity
• Maximum capacity achieved when equal received
  powers from all nodes (except SIC)
• No central decision maker
• Distributed decision process
• Nodes independently adjust power level according
  to an objective function, ui, performance
  metrics, and a power update algorithm

• Basic requirements for any distributed RRM scheme
• Convergent behavior
• A necessity for network design
• Stability
• Insensitivity to noise
• Fairness
• Example uses
• Bluetooth
• 802.11
• Sensor networks

60
Ad-hoc Power Control as a Game

• Player Set N
• Set of decision making radios
• Individual nodes i, j ? N
• Actions
• Pi power levels available to node i
• May be continuous or discrete
• P power space
• p power tuple (vector)
• pi power level chosen by player i
• Nodes of interest
• Each node has a node or set of nodes at which it
  measures performance
• ?i the set of nodes of interest of node i.
• Utility function
• Target SINR at node of interest

1
?5
5
?0
2
0
?1
?4
?3
?2
4
3
61
Yatess Fixed Assignment Scenario

• Distributed power control algorithm in cellular
  system
• Performance for node j is measured by SINR at
  base station k, with path gain from j to k hjk
  and noise ?k
• Each node j attempts to achieve a target SINR ?j.
• Decision update algorithm

• Fixed point exists for decision update algorithm
• Fixed point is unique
• Decision update algorithm converges synchronously
• Decision update algorithm converges
  asynchronously

62
Equivalence Results

• Ad-hoc power control is equivalent to Yates
  fixed assignment scenario (just a lot more base
  stations)
• Yates as a game
• Decision update algorithm is a best response
  algorithm
• Has a Nash equilibrium
• Has a unique NE
• Game has weak FIP (given by best response
  convergence and Altman)
• Best response algorithm converges

63
Establishing FIP Target SINR

• Define
• Neel04 Suppose
  then game has FIP.
• Define
• Note that ui is concave, thus also quasi-concave.
  Means upper level sets are convex.
• Thus better response sets are convex
• convex - Assuming finite P. If p?BR(p),
  p?BR(p) and p?p, then all p?P such that
  p? p?p are also in BR(p)

64
Establishing FIP Target SINR

• Due to convex upper level sets
• Thus if the recursions p(k1)BRmin(p(k)) and
  p(k1)BRmax(p(k)) converge, then all better
  response recursions must converge to the region
  bounded by
• Note BRmin and BRmax are monotonic sequences
• As BRmin is nondecreasing, and BRmax is
  nonincreasing, both must converge
  (pseudo-squeeze) though not necessarily to the
  same point.
• Note that convergent points are NE.
• However, NE is unique, therefore BRk converges
  and game has FIP.

65
Implications

• Any synchronous directional improvement algorithm
  will converge
• Steady state is stable (potential maximizers are
  stable)
• Target throughput, target BER, target FER, target
  QoS converge (ordinal transformation converges)
• Spacing between power levels can be made
  arbitrarily fine.

• Note that BR(p(k1))? BR(p(k)) for all kgt0
• Note that we just established synchronous
  convergence.
• Note that box condition holds as well (Cartesian
  product)
• Thus asynchronous convergence theorem (Bertsekas)
  holds, and updates can also occur with any subset
  of nodes updating at the same time.

66
Simulation Scenario

• Two cluster ad-hoc network
• 11 nodes
• DS-SS N 63
• Path loss exponent n 4
• Power levels -120, 20 dBm
• Step size 0.1 dBm
• Synchronous updating
• Target SINR ? 8.4 dB
• Objective Function

67
Simulation Results
Noiseless Simulation
Noisy Simulation
Identical values for ui Implies fairness
Statistically Identical values for ui Implies
fairness
Cluster heads
Cluster heads
Steady state Steady state exists Attractor and
steady state the same point Attractor is
stable Steady state is fair
Convergence Both scenarios converge Noise has
little impact on convergence rate Implies that
outside of region immediately around NE, PE ltlt PC
68
Power Control Thoughts

• Game has a unique NE
• Distributed ad-hoc power control algorithms
  converge asynchronously if nodes adapt in the
  right direction (making the game a potential
  game)
• Target SINR
• Target BER,FER
• Target Throughput
• Target QoS
• Convergence rate only weakly influenced by noise
  implies convergence rate can be estimated from
  noiseless analysis

69
Things to Take Away

• Game theory is used to model and analyze
  interactive decision problems.
• Numerous distributed algorithms are interactive
  decision problems.
• Game theory can address
• Existence of a steady state (fixed point
  theorems)
• Characterization of fixed points (depends on
  model)
• Desirability of steady states (but dont trust
  pareto efficiency analyses)
• Convergence (FIP, weak FIP)

70
Game Theory Group at MPRG

• Four areas of emphasis
• Power control
• Adaptive interference avoidance
• Network formation
• Node participation
• Initial efforts
• Model identification
• Static analysis
• Current efforts
• Convergence
• Stochastic issues (noise)
• Papers, presentations, tutorials available at
• www.mprg.org/people/gametheory/index.shtml

71
Referenced Papers

• E. Altman and Z. Altman. S-Modular Games and
  Power Control in Wireless Networks IEEE
  Transactions on Automatic Control, Vol. 48, May
  2003, 839-842.
• D. Bertsekas and J. Tsitsikis, Parallel and
  Distributed Computation Numerical Methods,
  Athena Scientific, 1997.
• S. Ginde, A Game-theoretic Analysis of Link
  Adaptation in Cellular Radio Networks MS Thesis
  Virginia Tech May 2004.
• S. Ginde, R. Buehrer, and J. Neel, A Game
  Theoretic Analysis of the GPRS Adaptive
  Modulation Schemes Fall VTC 2003.
• M. Hayajneh, C. T. Abdallah, Distributed Joint
  rate and Power Control Game-Theoretic Algorithms
  for Wireless Data, IEEE Comm. Letters, Aug 2004,
  pp 511-513.
• D. Krishnaswamy, Game Theoretic Formulations for
  Network-assisted Resource Management in Wireless
  Networks, Fall VTC 2002.
• J. Neel, J. Reed, and R. Gilles, Convergence of
  Cognitive Radio Networks WCNC2004, March 25,
  2004.
• J. Neel, J. Reed, Convergence Conditions for
  Distributed Power Control Algorithms in Ad-hoc
  Networks, CWT WOW 2004.
• J. Neel, J. Reed, and R. Gilles, Game Models for
  Cognitive Radio Algorithm Analysis, SDR Forum
  2004. (to appear)
• R. Yates, A Framework for Uplink Power Control
  in Cellular Radio Systems, IEEE Journal on
  Selected Areas in Communications, Vol. 13, No 7,
  September 1995, pp. 1341-1347.