Analysis of revisions for shortterm economic statistics

1
Analysis of revisions for short-term economic
statistics
OECD Short Term Economic Statistics Working Party

• Richard McKenzie OECD

2
Analysis of revisions for short-term economic
statistics

• Main results from OECD revisions analysis study
  for the index of industrial production
• Country comments and reactions
• Preliminary results from a similar OECD study on
  Retail Trade Volume
• OECD Main Economic Indicators Original Release
  Data and Revisions Database
• Providing the tools for countries to do their own
  analysis

3
Revisions analysis for IIP

• An attempt to analyse 3 dimensions of statistical
  quality and make comparisons across countries
• Timeliness
• Accuracy (assessed through revisions)
• Coherency (between IIP and GDP in industry)
• Also able to study the interactions between these
  quality measures
• Monthly snapshots of data published in MEI CD-Rom
  from February 1999 March 2006

4
Main reasons for revisions in IIP

• Late data or revised data from respondents later
  correction of errors or imputations
• Different estimation methodologies for early
  estimates and lower sample sizes
• Updating of seasonal factors
• Benchmarking to other sources (e.g. National
  Accounts or annual structural surveys)
• Changes to base period and changes in statistical
  methodology, classifications etc.

5
Mean absolute revision to first estimates of
year-on-year growth rates
6
Mean absolute revision to first estimates of
year-on-year growth rates
7
Are revisions statistically significant?

• Mean absolute size of revisions is important
• Revisions analysis quantifies this and provides a
  basis to review our compilation methods
• Ideally revisions should centre around zero over
  time (i.e. equally likely to be or - )
• If this is not true then reasons causing this
  tendency in the compilation process must be found

8
Mean revision between first estimates and
published one year later
9
Mean revision between first estimates and
published one year later
10
Which short term measures should we advise users
to focus on?

• Are first estimates of month-on-previous-month
  growth rates from the IIP reliable enough to
  enable informed decision making?
• Not in most countries! First estimates of MoM
  growth rates revised by 2/3 initial value on
  average after one year (95 countries revised by
  more than 2/5)
• Shows the value that revisions analysis can
  provide to users
• How to extract the latest signals in estimates
  without being misleading?

11
Reliability of MoM growth rates to signal
expansion or contraction (sign of first estimate
compared to that one year later)
12
Do revisions affect identification of turning
points?

• Very difficult to study
• As future turning points interact with past
  revisions
• OECD Composite Leading Indicators dates turning
  points in IIP using the Bry-Boschan routine
• Compared series of first estimates of YoY growth
  rates with those published two years later, at
  the BB turning point spots
• Where turning points were evident in the YoY
  growth rates series, preliminary estimates and
  those 2 years later usually coincided or were
  very close

13
Timeliness and accuracy

• Is there a trade-off between timeliness and size
  of revisions for the index of industrial
  production?
• Might expect those countries which publish
  earlier to have higher revisions at least in
  the following two months
• E.g. due to estimation based on lower response
  rates, less editing of errors, flash estimation
  techniques

14
(No Transcript)
15
Mean absolute revision to year-on-year growth
rates before and after an improvement in
timeliness.
16
Coherence analysis

• In concept the Index of industrial production
  should have a strong relationship to Value added
  in industry from the National Accounts
• Coherency test compare annual growth rates of
  IIP (most recent series) to annual growth rates
  in value added in industry from the national
  accounts
• Maybe the countries with high revisions are more
  coherent (e.g. due to revisions caused by
  benchmarking?)

17
Correlation between annual growth rates for the
IIP and GDP in industry from the annual national
accounts
18
Review of country comments to IIP revisions
analysis study

• Comparisons with similar studies done by the
  national institute (or comments on results /
  methods used)
• Generally similar results for countries that had
  done their own studies (some exceptions)
• Need to look at other time windows and remove
  outliers
• Sign change analyse of MoM revisions
• Impact of seasonal effects should be separated
• Few comments on the validity of the timeliness
  analysis

19
Review of country comments to IIP revisions
analysis study

• Comments on reasons for revisions and national
  revisions analysis policies
• About 1/3 countries reported aspects of their
  revisions analysis policy (some with recent
  changes and efforts to be clear to users in this
  area)
• Need to be wary of revisions analysis as
  definitive measure of quality (one user
  emphasised the preference for transparency on
  statistical methods)
• Reasons for revisions covered earlier, all in
  line with those presented in the paper

20
Review of country comments to IIP revisions
analysis study

• Comments on the importance of performing
  revisions analysis for Short term statistics
• 2/3 countries stressed the importance of
  performing revisions analysis
• Many countries welcomed the provision of tools
  and a standardised approach which can save their
  resources

21
Preliminary results from Retail Trade Volume
revisions analysis

• Very similar results to analysis of IIP
• Mean absolute revisions increase the longer the
  revision interval
• Revision size similar across countries with
  some countries noticeably higher and with mean
  revision statistically significant
• First estimate of MoM growth rates subject to
  large revision within one year

22
Mean absolute revision to first estimates of
year-on-year growth rates
23
Mean absolute revision to first estimates of
year-on-year growth rates
24
Mean revision between first estimates and
published one year later
25
Mean revision between first estimates and
published one year later
26
OECD Main Economic Indicators Original Release
Data and Revisions Database

• Full time series of data published every month
  starting from the February 1999 edition of the
  Main Economic Indicators for 21 key economic
  variables
• Access OECD revisions analysis studies for GDP,
  Index of Industrial Production and Retail Trade
  Volume
• Automated programs and detailed user guide
  allowing users to perform there own revisions
  analysis for any country / variable combination
  available in the database
• Information on reasons for revisions and
  references to international recommendations for
  establishing revisions analysis policies (IMF
  SDDS and OECD)

27
(No Transcript)
28
Live version

• http//stats.oecd.org/mei/default.asp?rev1

29
Conclusions / Future work

• Engaging users on results of revisions analysis
  studies
• Especially the expected reliability of different
  measures from first estimates should we do more
  work to promote more appropriate measures?
• How to promote the facilities developed by the
  OECD
• Large saving of resources for NSIs to perform
  revisions analysis, with OECD or their own data
• How to use the results from revisions analysis to
  improve compilation methods?

30
THE END