On Learning in Policy Space by Oligopolists in Electricity Markets

1
On Learning in Policy Space by Oligopolists in
Electricity Markets

• by
• Steve Kimbrough
• Fred Murphy

2
Organization of Presentation

• Bidding process in day-ahead market
• Supply function equilibria
• Agent-based modeling
• Bid representation
• Results
• Conclusions

3
Day-ahead Market

• Each firm bids a step-function supply curve
• LSEs offer demand curves
• The supply and demand curves are put into an LP
  with transmission capacity constraints and the LP
  solved for the equilibrium

4
Our Day-ahead Market

• Demand curves are continuous
• No transmission constraints
• Firms bid step-function supply curves
• Combine individual supply curves into single
  supply curve
• Find market equilibrium, where curves cross

5
Alternative models of Equilibria

• Cournot - bid quantities
• Problem utilities bid prices as well
• Bertrand - bid prices
• Problem oddly discontinuous behavior
• Add capacity game and same as Cournot
• Supply function bid a supply curve

6
Supply Function Equilibria

• Klemperer and Meyer (1989), Green and Newbery
  (1993)
• Literature uses continuous curves to derive
  results
• Models have multiple equilibria unless severely
  restricted in functional form and/or domain

7
A Problem with all of the Standard Models

• Firms are part of a community
• The models assume myopic optimization and pure
  non-cooperative behavior
• There are institutions where firms can talk
  without breaking the law
• Leading companies look to the interests of the
  industry as well as the firm

8
Agent-based Modeling

• Two approaches
• Agents are decisions/policies and fittest survive
• Agents try different decisions and mostly go with
  the decisions that have better outcomes
• Our approach is to
• Give agents alternative objective functions
• Let agents make decisions based on the values in
  the assigned objective function
• Evaluate outcomes based on firm profitability

9
The Model of the Market

• Player objective functions
• Firm profitability
• Industry profitability
• Industry profitability subject to a fair share
  constraint
• Supply functions
• n plants, n steps
• n plants, n1 steps

10
The Model of the Market, cont.

• Episode - one round of play
• Players randomly adjust prices and quantities
  around trial values on steps
• Epoch - a collection of episodes
• Players evaluate the outcomes from the random
  trials using the assigned objective function
• They adjust the trial values in the direction of
  the increased objective function

11
Monopolist, N
File elec-Own-N.png
12
Monopolist, N1
File elec-Own-N1.png
13
Duopoly, Own-Own,N-N
Players find Cournot outcome. High-bidding
player is exploited.
File elec-Own-Own-N-N.png
14
Duopoly, Own-Own, N1-N1
Players find Cournot-plus outcome. Neither
player is exploited.
File elec-Own-Own-N1-N1.png
15
Duopoly, Ind-Ind, N1-N1
Players individually maximize industry profits
and jointly find monopoly outcome.
File elec-Industry-Industry-N1-N1.png
16
Duopoly, Ind-Own, N1-N1
Players find near-monopoly outcome. Cooperative,
Industry Returns player, is exploited.
File elec-Industry-Own-N1-N1.png
17
Duopoly, IndOwn-Own, N1-N1
Players find Cournot-plus outcome. Cooperative,
Industry Returns, s.t. Own Returns player, is
not exploited.
File elec-IndustryOwn-Own-N1-N1.png
18
Duopoly, IndOwn-IndOwn, N1-N1
Players find monopoly outcome. Neither player is
exploited. (But the customers are.)
File elec-IndustryOwn-IndustryOwn-N1-N1.png
19
A Pattern

• Classical results hold for learning agents who
  are very myopic.
• Agents learn to collude tacitly by being less
  myopic and trading off exploration and
  exploitation, tilting more towards exploration.
  (Exploring rationality)
• Simple tilts towards exploration are subject to
  exploitation.
• There exist simple constraints on exploration
  that greatly reduce exposure to exploitation. In
  this mode agents may mutually achieve tacit
  collusion safely.

20
Conclusions

• In repeated play the structure of the supply
  curves can be exploited to find the Pareto
  optimum
• At the same time it is possible for a firm to
  protect itself from self-serving players
• To get its fair share, each player has to make
  the other see the consequences of its actions on
  aggregate demand in its choices of prices and
  quantities