Session 17 IV' Methods of incentive regulation: Techniques for improving utility efficiency

1
Session 17IV. Methods of incentive regulation
Techniques for improving utility efficiency
PURC/World Bank International Training
Programme June 10-21, 2002 - Gainesville, Florida

• William Emery
• Office of Water Services UK

2
Techniques for improving utility efficiency

• Three parts drawn from UK experience
• Incentives
• Scope for direct market competition
• Comparative competition
• Statistical techniques, use of benchmarks, data
  issues, burden of proof, step by step model

3
1. Incentives - some key questions...

• What are the incentives for efficient behaviour
  in the regime?
• What are the sticks and carrots that drive
  managers in the enterprise?
• How can they be strengthened within the
  regulatory constraints?
• Will the incentives be understood?

4
Strong incentives?

• Some positive characteristics...
• Some market competition?
• Clear and consistent regulatory rules?
• Stability in medium term requirements
• Minimal political or regulatory intervention
• Clear pay-off to enterprise and owners from
  efficiency initiatives...
• Shareholder corporate control pressures

5
Weak incentives?

• No market competition - monopoly utilities
• Uncertainty in requirements - frequent changes
• No benefit from innovation
• Inconsistency in rules over time
• Intervention in management by regulators,
  politicians etc..
• Gaming and regulatory skills gives better pay-off
  for managers

6
medium term incentive based price-cap
regulation (1/3)

• Utility companies own the assets long term
  licences to supply designated areas
• Separation of roles
• Price limits set for 5 year periods with clear
  objectives / expectations limited risks carried
  by customers
• Incentives for companies to out-perform
  assumptions in price limits generate higher
  returns for owners

7
medium term incentive based price-cap
regulation (2/3)

• Where no market competition use benchmarking and
  relative efficiency analysis to inform price
  setting
• Clear rules on retention of efficiency savings
  for at least 5 years
• Lower cost structure feeds through into lower
  bills for customers review to review
• Rewards for good service

8
medium term incentive based price-cap
regulation (3/3)

• The sticks and carrots
• Must deliver service outputs or severe
  regulatory penalties
• Out-performance high returns
• Meeting assumptions earns cost of capital
• Under-performance trouble on all fronts

9
2. Scope for direct market competition

• Introducing direct market competition the
  preferred option but scope?
• Telecoms?
• Energy?
• Water Sewerage?
• Essential facilities - shift burden of proof on
  shared access to assets and systems
• In absence of market competition promote use of
  comparative competition

10
3. Comparative competition

• Theory of yardstick competition (Scheifer) using
  average industry costs but
• Statistical techniques?
• unit costs, data envelopment analysis, regression
  analysis and econometrics etc..
• Relative efficiency analysis versus forecasting
  analysis?
• Process or technique?

11
I will use the UK water experience to illustrate
some of the issues and draw out some lessons
12
scene setting ...

• Substantial variations in costs (operating,
  capital maintenance, capital investment) between
  companies ...
• Possible reasons?
• Different environments?
• Different inheritance?
• Different requirements?
• Data inconsistencies?
• Different efficiency?

13
benchmarks frontiers

• Stimulating a truly competitive market where the
  performance of the best drives all others to
  raise their game or go out of business
• Aim of analysis is to locate these benchmarks /
  frontiers and then use information in regulation
• Regular publication
• In price setting

14
analysis informed by

• comprehensive guidelines for company data and
  estimates
• independent scrutiny and challenge of company
  data and estimates
• comparative analysis or benchmarking
• assessments of the general scope for future
  efficiency
• link analysis to approach taken on incentive
  balance

15
two components of judgement

• minimum improvement element to expect for the
  best performers (not necessarily efficient in
  absolute terms)
• catch-up element based on comparative efficiency
  studies to assess Relative Efficiency and
  judgements on pace of required improvements

16
assessing relative efficiency ...

• Use systematic comparative analysis Find the best
  cost performers and use them as benchmark for
  others to strive to achieve
• Opex - econometrics plus
• Capex - COST BASE and econometrics plus
• Focus on good audited data
• Open and transparent about analysis etc..
• Shift burden of proof onto poorer performers to
  justify why their costs are above yardstick levels

17
Touching on two of the Ofwat analyses, firstly
econometrics and this is best illustrated by
use of a worked example
18
Step by step analysis
Steps 1 to 4 - Expert review of potential cost
drivers, data collection, validation, removal of
atypicals to create valid data set for
statistical analysis
Step 5 to 7 - Generate plausible conceptual
models, carry out econometrics, expert review
many iterations
Steps 8 9 - Preliminary assessments of relative
cost followed by review of company specific
factors
Steps 10 11 - Test results against parallel
analysis and finalise judgements on relative
efficiency
19
Why use econometrics?

• Econometrics uses economics and statistics to
  model real life situations
• We use it to measure the relative efficiency of
  the water companies.
• Isnt the lowest cost company the most efficient?
• For simple comparisons between companies we can
  use unit costs BUT Unit costs are not
  measures of company efficiency.

20
For instance...

• The current unit cost of pence per cubic metre
  of water delivered for 3 companies
• Three Valleys (tvn) 58p/m3
• Bristol (brl) 69p/m3
• North West (NWT) 74p/m3
• Three Valleys delivers the cheapest water of the
  three companies but is it the most efficient?
• Costs can vary because of differences in
  operating environments which are outside the
  companies control (e.g. population density,
  terrain)

21
Econometric models?
COST DRIVERS
COSTS
22
Methodology

• We use regression analysis to derive a range of
  econometric models for each area of the business.
  
• Each model describes an industry representative
  relationship between costs and cost drivers

23
A Simple RegressionBusiness Activities Model
COSTS
COST DRIVERS
24
What makes a good model?

• We have consulted the industry, obtained expert
  advice and published to expose them to scrutiny
• We consider our models stand up -
• on economic grounds
• on engineering grounds
• on statistical grounds
• and are as simple as possible

25
Statistical tests

• To be statistically robust, the relationship
  between the cost drivers and opex must have
• A high R2, given the number of data points.
• R2 is a measure of the closeness of fit of data
  points to a line.
• Supported by a P value of lt 0.05, showing that
  the relationship is valid within a confidence
  limit of 95.
• P is the probability that there is no
  relationship between two given measures.
• Other supporting statistics, e.g. t tests

26
Our models

• Water service - 4 main models
• Business Activities, Distribution, Power
  Resources and Treatment
• Sewerage service - 5 main models
• Business Activities, Network, Large sewage
  treatment works, Small sewage treatment works
  Sludge treatment and disposal
• I am going to use the water service - resources
  and treatment model in this worked example...

27
For this model we need to know...

• What drives costs?
• How can we measure these costs?
• Which measures should we use as scale variables?
• What form should the model take?
• But first we need some data...

28
Collecting data

• We collect data so that we can monitor the
  companies performance.
• Each year we require the companies to submit
  financial and non-financial data (June Returns).
• Periodically we also require additional sewerage
  service data so that we can create more
  statistically robust models for the limited
  number of sewerage companies.

29
Collecting data ...

• The June Return has 39 tables, providing
  financial, demographic and technical data.
• Our opex models use data from Tables 7, 10, 12,
  13, 21, and 22.
• We also use unpublished data from Tables 17a-b,
  d, f-g of the periodic submissions mains diameter
  data that companies provide separately.
• All the data are reviewed by independent
  Reporters and Auditors

30
What drives resources and treatment costs?

• The spreadsheet shows actual costs and potential
  cost drivers
• We want to establish a robust correlation
• by looking at relationships between cost drivers
  and
• opex
• log opex
• unit cost
• log unit cost

31
What drives resources and treatment costs?

• Companies abstract from different types of source
  in different proportions.
• Waters from some sources are cheaper to treat
  than others (treatability).
• Companies will need to spend less if they only
  need to use simple treatment methods and more if
  they use complex methods.
• Companies will spend less if they have a few
  large sources than if they have many small
  sources and treatment works (source size).

32
Potential cost drivers?

• Treatability
• Proportion of distribution input (DI) from river
  abstraction
• Proportion of DI subject to simple treatment
• Proportion of non-river sources with complex
  treatment
• Source Size
• Number of sources/DI or number of
  sources/(DI-leakage)
• DI/number of sources or (DI - leakage)/number of
  sources
• and the data on these...

33
(No Transcript)
34
Treatability Cost Driver
Statistically, proportion of DI from rivers is a
good cost driver and we include it in our model.
35
Simple Treatment Cost Driver
Adding a term for simple disinfection into the
model failed to increase (or decrease)
significantly the fit to industry data. We did
not include this cost driver in our model.
36
Additional Treatability Cost Driver
The relationship between unit cost and complex
treatment was not significant, and we did not
include it in our model.
37
Source Size Cost Driver

• Should we deduct leakage from distribution input
  to take account of differing levels of leakage?
• How should we measure source size?
• Should it be DI/number of sources or number of
  sources/DI?

38
Source Size Cost Driver
Sources/DI Adjusted R2 is
0.3793, P is 0.001, SE is
0.0047175. Sources/(DI -leakage) Adjusted
R2 is 0.3702, P is 0.001 SE is
0.003769. Number of sources/DI is the
statistically stronger measure.
39
Source Size Cost Driver
Sources/DI Adjusted R2 is
0.3793, P is 0.001, SE is
0.0047175. DI/sources Adjusted R2 is
0.1208, P is 0.064 SE is
0.0000669. Number of sources/DI is the
statistically stronger measure.
40
Source Size Cost Driver
Statistically, number of sources/DI is a good
cost driver and we included it in our model.
41
Choice of Cost Drivers

• From the regression analysis the strongest,
  statistically significant cost drivers were
• Number of sources/DI
• Proportion of supply from rivers
• Now we need a scale variable...

42
Scale variables?

• Scale variables describe company size in unit
  costs and cost drivers.
• Business activities model uses number of
  properties billed
• But we could use a number of variables to
  describe resources and treatment opex.
• For resources and treatment we considered
• Winter population, Number of properties billed,
  Distribution input (DI), DI-leakage Number of
  sources

43
Scale variable data look like this...
44
Correlation matrix for potential scale variables
45
Relationship between scale variables and opex
The regression lines show that the linear
relationship is less strong for ln(total number
of sources) than for the other variables.
46
Form of the model?

• Models must make economic and engineering sense
  be statistically robust
• Potential model forms?
• linear, log, linear unit cost and log unit cost
  models.
• Unit costs give statistically strong results
• Both the linear unit cost and the log unit cost
  models were statistically robust (though the
  linear unit cost model was slightly stronger)
• We chose the linear unit cost model, taking
  account of statistical, engineering and economic
  factors.

47
Example - Treatability Cost Driver
Linear unit cost model Adjusted R2
0.3793, P0.000, SE0.0018015. Ln(unit cost)
model Adjusted R2 0.3459, P0.000,
SE0.3241256. The statistics show that the
linear model is slightly stronger.
48
Example - Source Size Cost Driver
Linear unit cost model Adjusted R2
0.3702, P0.001, SE0.0047175. Ln(unit
cost) model Adjusted R2 0.3459,
P0.003, SE0.1308723. The statistics show that
the linear model is slightly stronger.
49
Comparing the Log and Linear Unit Cost Models
Comparing the predicted and actual values in the
alternative models. The models are statistically
similar and are both acceptable when we use
t-tests.
50
The final RT model ...

• Resources and treatment expenditure (less
  Environment Agency charges and power) (m) /
  resident winter population (millions)
• 0.81 (16.7 x (number of sources/distribution
  input (Ml/d)) (6.4 x proportion of supply from
  rivers)

51
Relative unit costs...

• The econometric model is used to determine what
  each company could be spending on Resources and
  Treatment after taking account of the cost
  drivers.
• We then work out the difference between what we
  believe it could be spending compared with its
  actual spend.
• How do these calculations look?

52
We use the predicted unit cost to calculate the
predicted opex. The residual in m is the actual
minus the predicted opex. The percentage residual
is the residual expressed as a percentage of
predicted opex.
53
Once we have chosen the frontier we can calculate
each companys efficiency relative to that
frontier.
54
The right hand column shows this
calculation. Company 6 is the only company more
efficient (-ve) than the frontier. Company 26
is the frontier, - its efficiency is set to
zero. The larger the efficiency number, the
more inefficient the company is relative to the
frontier. For example, company 25 is the least
efficient company in RT.
55
Relative cost to efficiency

• We combine the results from all the models for
  water and do the same for all the sewerage
  models.
• We then take account of any company specific
  special factors (those that are outside the
  companies control and that cannot be be
  incorporated into the models)
• Examples of special factors include
• Salary costs in the South East of England
• High meter penetration
• Poor quality sources
• Special Legal Requirements

56
Relative cost to efficiency

• We then derive an efficiency banding for each
  company (A being the most efficient, and E being
  the least efficient), relative to the most
  efficient company
• We use these bandings together with other
  information to produce the catch-up efficiency
  assumptions for each company when setting price
  limits...
• We publish the bandings annually in our Report
  on Water and Sewerage service unit costs and
  relative efficiency

57
Earlier we looked at the pence per cubic metre
of water delivered unit costs for 3 companies,
tvn, brl and NWT. We can now compare these costs
with our current assessments for their relative
efficiency...
D
C
B
80
60
p/m3
40
20
0
tvn
brl
NWT
Unit Cost
58
Econometrics - resource inputs?

• Water Service Opex Econometrics..
• Steps 1 to 4 5yrs effort, 7 studies, 2 rounds
  of proposals lots of guidance, JR97-99, 80
  atypicals etc..
• Steps 5 to 7 100 models, 21 reports
  man-years of effort
• Steps 8 to 11 Publications, consultation, 3
  sets of special factor submissions, many letters,
  views of Reporter, extra data analysis

59
Improving efficiency...
60
and secondly comparative capital costs
61
COST BASE ...

• Approach based on comparative analysis of
  specified comparative unit capital costs and
  project estimates
• Standardised specifications of 100 items
• Companies to use same unit costs and company
  practices as used to forecast CAPEX programmes
  (checked and verified by Reporters)

62
COST BASE ...

• Step-by-step approach 2 iterations
• Credible benchmarks chosen for each standard cost
  following expert advice, analysis review
• Expectation that scope for savings based on
  moving at least halfway to benchmarks (more for
  quality programmes)
• Publication, extensive dialogue, challenge

63
and a typical histogram ...
64
Typical standard cost histogram
Benchmark
50 adjustment
65
However it was not all sweetness and light
66
COST BASE ...

• Huge attack on approach by industry
• Inadequate coverage of standard costs...
• too much inconsistency...
• lack of openness re analysis, selection of
  benchmarks...
• funded critical analysis by other consultants
• did not offer any real alternative ...
• Most issues raised were addressed in second round
  of data submissions...

67
showing the need for the step by step approach

68
Review and consult on standard cost specification
Collect data, validate, review for compliance
with specification and company specific factors
to produce revised data for comparative analysis
Feedback repeat cycle
Compare contrast, identify robust
benchmark companies for groups of standard costs
Calculate relative unit cost of each company for
each area of investment activity
The Cost Base
Use factors in draft determinations together with
explanation of findings
Review company written / face to face
representations Finalise judgements of scope for
improvements
69
and the timetable
70
PR99 Cost Base timetable

• 03/97 - publish paper on PR94 approach
• 1997 - work with expert group on definitions
• 11/97 - consult on draft reporting requirements
• 03/98 - issue reporting requirements
• 06/98 - data submission Reporters report

• 10/98 - feedback to companies use of draft
  results
• 12/98 - publish analysis
• 12/98 - issue revised requirements for BP
• 04/99 - updated data
• 07/99 - feedback use Draft Determinations
• 11/99 - final results in Final Determinations

71
and what were the results
72
Cost Base catch up judgements- water service

• Based on 57 standard capital costs and industry
  benchmarks
• Band A Most efficient companies 2
  adjustment
• Band C Least efficient companies up to 19
  adjustment

73
To finish with a few observations
74
Some lessons from Ofwat experience...

• Benchmarking is not easy .
• Needs a sound and systematic basis ...
• Needs clear guidelines for comparative data and
  credible audit/technical scrutiny
• Needs considerable time and at least two
  iterations
• Feedback and reasonable disclosure important
  throughout to shift burden of proof onto poor
  performers

75
To conclude ...

• Systematic benchmarking of water company
  efficiency has worked in water sector
• Both MMC (1995) and CC (2000) endorsed the
  comparative approaches used by Ofwat
• Ofwat will be using these techniques again for
  the 2004 periodic review...
• Is there a credible alternative for monopoly
  utilities?

76
Thank you ...
77
(No Transcript)