Testing Strategic Models of Firm Behavior in Restructured Electricity Markets: A Case Study of ERCOT

1
Testing Strategic Models of Firm Behavior in
Restructured Electricity MarketsA Case Study of
ERCOT

• Ali Hortacsu, University of Chicago
• Steve Puller, Texas AM

2
Motivation

• Empirical auction literature
• Bid data equilibrium model ? valuation
• New Empirical IO
• Eqbm (p,q) data demand elasticity behavioral
  assumption ? MC
• Can equilibrium models be tested?
• Laboratory experiments
• Electricity markets are a great place to study
  firm pricing behavior
• This paper measures deviations from theoretical
  benchmark explores reasons

3
Texas Electricity Market

• Largest electric grid control area in U.S.
  (ERCOT)
• Market opened August 2001
• Incumbents
• Implicit contracts to serve non-switching
  customers at regulated price
• Various merchant generators

4
Electricity Market Mechanics

• Forward contracting
• Generators contract w/ buyers beforehand for a
  delivery quantity and price
• Day before production fixed quantities of
  supply and demand are scheduled w/ grid operator
• (Generators may be net short or long on their
  contract quantity)
• Spot (balancing) market
• Centralized market to balance realized demand
  with scheduled supply
• Generators submit supply functions to increase
  or decrease production from day-ahead schedule

5
Balancing Energy Market

• Spot market run in real-time to balance supply
  (generation) and demand (load)
• Adjusts for demand and cost shocks (e.g. weather,
  plant outage)
• Approx 2-5 of energy traded (up and down)
• up ? bidding price to receive to produce more
• down ? bidding price to pay to produce less
• Uniform-price auction using hourly portfolio bids
  that clear every 15-minute interval
• Bids monotonic step functions with up to 40
  elbow points (20 up and 20 down)
• Market separated into zones if transmission lines
  congested we focus on uncongested hours

6
Who are the Players?
Generator of Installed Capacity
TXU Electric 24
Reliant Energy 18
City of San Antonio Public Service 8
Central Power Light 7
City of Austin 6
Calpine 5
Lower Colorado River Authority 4
Lamar Power Partners 4
Guadalupe Power Partners 2
West Texas Utilities 2
Midlothian Energy 2
Dow Chemical 1
Brazos Electric Power Coop 1
Others 16
7
Incentives to Exercise Market Power

• Suppose no further contract obligations upon
  entering balancing market
• INCremental demand periods
• Bid above MC to raise revenue on inframarginal
  sales
• Just monopolist on residual demand
• DECremental demand periods
• Bid below MC to reduce output
• Make yourself short but drive down the price of
  buying your short position (monopsony)

8
(No Transcript)
9
Methods to Test Expected Profit Maximizing
Behavior

• Difficult to compare actual to ex-ante optimal
  bids
• Wolak (2000,2001) ? solving ex-ante optimal bid
  strategy (under equilibrium beliefs about
  uncertainty) is computationally difficult
• Options
• Restrict economic environment so ex-post optimal
  ex-ante optimal
• Intuitively, uncertainty and private information
  shift RD in parallel fashion
• Check (local) optimality of observed bids (Wolak,
  2001)
• Do bids violate F.O.C. of Eep(p,e)?
• Can simple trading rules improve upon realized
  profits?

10
Overview of Model

• Setup
• Static game, N firms
• Marginal Costi is public information
• Contract quantity (QCi) and price (PCi) are
  private information
• Generators bid supply functions Si(p,QCi)
• Sources of uncertainty
• Total demand D(p) stochastic
• Rivals bids S-i(QC-i)
• Market clearing price (pc) is uncertain
• (application of Wilson 1979 share auction)

11
Sample Genscape Interface
12
Overview of Model (contd)
13
Overview of Model (contd)
14
Computing Ex Post Optimal Bids

• Ex post best response is Bayesian Nash Eqbm
• Uncertainty shifts residual demand parallel in
  out
• Can trace out ex post optimal / equilibrium bids

15
Data (Sept 2001 thru July 2002)

• Bids
• Hourly firm-level bids
• Demand in balancing market assumed perfectly
  inelastic
• Marginal Costs for each operating fossil fuel
  unit
• Fuel efficiency average heat rates
• Fuel costs daily natural gas spot prices
  monthly average coal spot prices
• Variable OM
• SO2 permit costs
• Each units daily capacity day-ahead schedule

16
Measuring Marginal Cost in Balancing Market

• Use coal and gas-fired generating units that are
  on and the daily capacity declaration
• Calculate how much generation from those units is
  already scheduled Day-Ahead Schedule

17
Reliant (biggest seller) Example
18
TXU (2nd biggest seller) Example
19
Guadalupe (small seller) Example
20
Calculating Deviation from Optimal Producer
Surplus
21
Measures of Foregone Profits
22
Percent of Potential Gains from Not Bidding
23
Learning by Larger Players?
24
Testing Expected Profit Maximizing Behavior

1. Restrict economic environment so ex-post optimal
   ex-ante optimal
2. No restrictions -- Check (local) optimality of
   observed bids (Wolak, 2001)
3. Can simple trading rules improve upon realized
   profits?

25
Generators Ex-Ante Problem

• Max Eep(p,e)
• s.t. (1) monotonicity of bids
• (2) transmission congestion
• (3) physical operating contraints
• Restrict our sample ? ignore constraints
• Wolak test for (local) optimality
• Ho Each bidpoint chosen optimally
• Changing the price of each (pk,qk) will not
  incrementally increase profits

26
Reliant (biggest seller) Example
27
Guadalupe (small seller) Example
28
Test for (Local) Optimality of Bids
Moments for GMM
29
Test for (Local) Optimality of Bids
Firm J-stat d.o.f. p-value
Reliant 0.131 9 0.99
TXU 0.302 5 0.99
Guadalupe 0.005 2 0.99

• Fail to reject (even for Guadalupe!)
• Test is lower power in our setting
• Future work use quantity moments

30
Naïve Best Reply Test of Optimality

• Bidders can see aggregate bids with a few day lag
• Simple trading rule use bid data from t-3,
  assume rivals dont change bids, and find ex post
  optimal bids (under parallel shift assumption)
• Does this outperform actual bidding?

31
How Much Does Trading Rule Increase Profits?
Bryan 200/hr18
Calpine 1,325/hr18
City of Austin 1,129/hr18
Reliant 957/hr18
TXU 1,770/hr18
Preliminary
32
Learning in Second Year?
Sep01-Jul02 Sep02-May03
Reliant/Texas Genco 82 27
TXU 53 77
Bryan 44 56
Calpine 34 44
City of Austin 27 28
Fair bit of month to month variability by firm.
Note Second year excludes Aug02, Dec02 (data
not clean yet) and Feb 03 (weather incident)
33
What the Traders Say about Suboptimal Bidding

• Lack of sophistication at beginning of market
• Some firms bidders have no trading experience
  are employees brought over from generation
  distribution
• Heuristics
• Most dont think in terms of residual demand
• Rival supply not entirely transparent b/c
• Eqbm mapping of rival costs to bids too
  sophisticated
• Some firms do not use lagged aggregate bid data
• Bid in a markup have guess where price will be
• Newer generators
• If a unit has debt to pay off, bidders follow a
  formula of markup to add

34
What the Traders Say (contd)

• TXU
• old school would prefer to serve its
  customers with own expensive generation rather
  than buy cheaper power from market
• Anecdotal evidence that relying more on market in
  2nd year of market
• Small players (e.g. munis)
• scared of market afraid of being short w/
  high prices
• Dont want to bid extra capacity into market
  because they want extra capacity available in
  case a unit goes down

35
Counterfactual Welfare Calculations (Not Yet
Completed)

• Productive inefficiencies under alternative
  bidding
• (1) Actual vs. Competitive (Vickrey multiunit)
• (2) Actual vs. Unilateral Best-Reply
  (Uniform-Price)
• (3) Actual vs. "Large Unilateral" and "Small
  Competitive"

36
Conclusion

• Electricity markets are a great field setting
  to understand firm behavior under uncertainty and
  private information
• Stakes appear to matter in strategic
  sophistication
• Both sophistication (market power) and lack of
  sophistication (avoid the market) contribute to
  inefficiency in this market

37
The End
38
(No Transcript)
39
(No Transcript)
40
Dispersion of Money on the Table
Reliant
41
Dispersion of Money on the Table
Reliant
TXU
Calpine
Bryan
42
Quantity Traded in Balancing Market
Mean -257 Stdev 1035 Min -3700 25th Pctile
-964 75th Pctile 390 Max 2713
Sample Sept 2001-July 2002, 600-615pm,
weekdays, no transmission congestion
43
Zones in ERCOT 2002
Source Public Utility Commission of Texas, MOD
Annual Report (2003)
44
Sample Bidding Interface
45
Do We Expect to See Optimal Bidding?

• First year of market
• Some traders experienced while others brought
  over from generation and transmission sectors
• Many bidding optimization decisions being made
• Real-time information?
• Frequency charts Genscape sensor data ? rival
  costs
• Aggregate bid stacks with 2-3 day lag ? adaptive
  best-response bidding?
• Is there enough at stake in balancing market?
• Several hundred to several thousand per hour
• Bounded rationality

46
Smaller Players

• Appear to bid to withhold capacity to avoid the
  balancing market
• ? productive inefficiencies
• Not unilateral market power because
  markups/markdowns are too large given their small
  inframarginal sales
• Policy implications
• Fixed costs to participation?
• But some small players are closer to optimal
• Bidders lacking trading experience?
• Sticky market for managerial efficiency?

47
Testing Explanations for Suboptimal Bidding

• Not enough at stake ? avoid the balancing
  market
• Potential profits for each 6-7pm
• Reliant 6,165
• Lamar Power Partners 1,391
• But Bryan 315!!
• Learning
• Exercising market power on DEC side (monopsony)
  may not be obvious
• Bid to DEC low so youre short but at a low price
• Decrease in bid-ask spread
• Profitability over time
• Use more bid points over time

48
Testing Explanations (contd)

• Adjustment costs?
• Marginal generating unit most often is gas (very
  flexible)
• Transmission congestion is important
• We analyze only periods with no interzonal
  transmission congestion
• Congestion changes residual demand
• If cannot perfectly forecast congestion, the
  bidding strategy under congestion may spillover
  to uncongested times
• Collusion?
• Would be small(!) players - unlikely

49
Sample Bidders Operations Interface
50
Residual Contract Positions
51
Difference in average system loads INC 33GW
DEC29GW Can marginal costs differ by that much?
52
(No Transcript)
53
(No Transcript)
54
(No Transcript)
55
Medians of Reduced-form conduct measures
56
(No Transcript)
57
Example of Data We See
Sept 14, 2001 600-615pm Total Balancing Demand
-996 MW
One Firms Bids and MC
Aggregate Bids and MC
58
(No Transcript)
59
Calpine (3rd biggest seller) Example
60
Test for (Local) Optimality of Bids
61
(No Transcript)
62
Who are the Players?
Generator Average Balancing Sales (MWh) of Installed Capacity
TXU Electric 156 24
Reliant Energy 473 18
City of San Antonio Public Service 8
Central Power Light 28 7
City of Austin 40 6
Calpine 78 5
Lower Colorado River Authority 4
Lamar Power Partners 23 4
Guadalupe Power Partners 8 2
West Texas Utilities 10 2
Midlothian Energy 2
Dow Chemical 1
Brazos Electric Power Coop 5 1
Others 16
Cannot uniquely identify the bids Sales
in zones where bids can be uniquely identified