Diapositiva 1

1
Comments on Common Biases in OECD and IMF
Forecasts Who Dares to be Different? by H.
Glück and S.P. Schleicher.
Antoni
Espasa,

Universidad Carlos III,

Madrid.
2
The paper aims to give evidence of - the degree
of accuracy of the OECD and IMF
forecasts for the G7 countries, and - the
existence of common forecast errors through
countries
3
This is done by calculating the forecast errors
for different horizons for 20 years. These
errors are then smoothed. I do not see why
smoothing could be a good idea for the purposes
of this paper.
4
The quality of the forecasts for a particular
horizon is evaluated considering the bias
(non-zero mean in the forecast errors) and the
correlation of the dated forecasts with
last-reported data.
5

• This rises several questions on which I will
  comment below
• -1.- is first or last-reported data the data
  which practicioners follow and forecasters try to
  forecast?
• -2.- What matters to practioners ,point
  forecasts or trace forecasts?

6

• The measure used to detect for common factors
  between forecasts for different countries is the
  sample correlation of the forecast errors.
• Some additional comments can be made in this
• respect
• -3.-Since these errors have their density
  functions it would be better to use measures
  which compare both densities.
• -4.-a deeper analysis is needed to really
  learn from different forecasts

7

• Results in the paper
• Only the first forecast made in the target year
  provides reasonable accuracy.
• The quality of forecasts deteriorates along the
  sample.
• There are important biases which persist in data
  revision errors.
• There are common forecasts errors between
  conected countries as US and Canada and Germany,
  France and Italy and they increase with the
  economic integration.

8

• The paper provides an interesting evaluation of
  forecasts done by two prestigious institutions
  pointing out important pitfalls.
• I WILL FOCUS MY COMMENTS ON THE TWO QUESTIONS
  AND THE TWO REMARKS MENTIONED ABOVE .

9

• 1.- Is first or last-reported data, the data
  which practicioners follow and forecasters try to
  forecast?

10
The motivation of this paper is the concern about
the implications for policy of the quality of
real-time data and forecasts. Starting with
forecasts a question which inmediately arises
refers to the magnitude which forecasters want to
forecast. a) the first-reported data, b) the
last-reported data or c) some other between.
11
The authors mention that the forecasters look
for gaining reputation between practioners.
Practiociners take decisions based on forecasts
and then ajust them according how first-reported
data diverge from forecast. Other possible
future adjustments on previous practioners
decisions would be done comparing nth-reported
data with the previous one but not with the
forecast initially used.
12
Then,it can be said that the reputation of the
forecasters depends on the accuracy in
forecasting first-reported data. This does not
seem to be a good thing but it is inevitable as
far as nth-reported data are published as fix
values and not as estimates with their
corresponding standard deviations and even with
their density functions.
13
This could be a claim to do to the statistical
offices if they are going to publish several
releases of data before the last one,it is
important that they provide confidence
intervals. This matters because the authors
show in the paper that real time data have not a
narrow density around the final-reported values.
14
Another implication of publishing several
releases of data without confidence
intervals. Question should policy makers base
their decisions on real-time data or on an
estimation of final-released data?
15
The effects of a policy measure depend, between
other things, on the agents expectations and
on how these agents evaluate the present state of
the economy. If agents evaluate the present
taking real-time data as fixed and good, then it
is not clear on which type of data real-time or
final-reported data - policy should be based on.
16

• 2.- What matters to practioners ,point
  forecasts or trace forecasts?
• In the context of this paper,
• What matters, next year forecast or the path of
  current, one-period next and two-period next
  years?

17
It seems to be a fact that generally agents do
not decide taking into account a point, or
interval or density forecast, but a congruent
trace forecast. Ideally considering a density
trace forecast as the one provided by the fan
charts. This is particulary true for quarterly
GDP or monthly inflation.
18
(No Transcript)
19
If we acept the previous remarks the forecast
evaluations should be done along the whole
trace. It is not easy to discover how policy
makers and practitioners actually evaluate the
trace forecasts. Cecchetti et al. (2000) and
Banerjee et al. (2003) adopt an evaluation
procedure based on the RMSE for horizons from to
one the light quarters.
20
If we acept the previous remark the forecast
evaluations should be done along the whole
trace. It is not easy to discover how policy
makers and practitioners actually evaluate the
trace forecasts. Cecchetti et al. (2000) and
Banerjee et al. (2003) adopt an evaluation
procedure based on the RMSE for horizons from to
one the eight quarters.
21
The elements of this RMSE statistic are highly
correlated and a reliable statistical test to
discriminate between different forecasting
performances is not available.
22

• 3.-Since errors from different forecasts have
  their density functions, it would be better in
  comparing the different forecasts to use
  measures which compare both densities.

23

• 4.- A deeper analysis is needed to really learn
  from different forecasts.

24
In the paper the comparsison of OECD and IMF
forecasts is performed by using statistical
measures as biases and correlations. This
provides a useful description of the two
forecasting performances. But a deeper analysis
is needed to really learn from different forecasts
25
In discovering the reasons of the differences
between forecasts it could be useful to consider
the following scheme last observed value (tn)
for the year-on-year rate of growth Xn p
long term forecast of Xn the
rate of growth for a given base Linear or
quasi-linear models forecast constant long term
annual rates of growth for a given forecast base,
which could depend or not on the conditions at
the base point.
26
Then it turns to be very useful to understand why
the correponding forecasting procedures (1)
generate, if this is the case, different
long-term rates of growth and (2) Different
dynamic paths to the long-term rate of
growth. Becasue - Different structural dynamic
in the models - Different exogenous variables or
differ- ent forecasts of them.
27
WHO DARES TO BE DIFFERENT?

• Perhaps there is a previous questionWho does
  not dare to make economic forecasts?
• Almost nobody.One only needs to give just point
  figures.
• If we ask the forecasters to produce
• - trace forecasts with fan charts and
• - to describe the main features of their
  forecasting procedures to estimate the
  medium-term rates of growth and the dynamic path
  to them ,

28

• Then only forecasters with reliable procedures
  will publish their results .In this context they
  perhaps dare to be different.
• Act as a copycat in point forecasts is very easy
  but in fan charts is not so.

29

• CONCLUSIONS
• The paper shows that forecasts and real-time data
  for GDP in G7 countries made by OECD and IMF have
  not a narrow density around the final-reported
  values.
• Consequently the research on real-time data and
  forecast should take into consideration the whole
  density functions.

30
3) This implies that comparisons of forecasts
should be based on meausures which compare
densities. 4) Elaborating on the argument that
forecasters look for reputation,it is possible to
conclude that forecasters try to forecast
first-released data and not final-reported data.
31
5) The different releases of data possibly also
the final release should be published with
confidence intervals. 6) Since real-time data is
not published with confidence intervals economic
agents could tend to consider them as good fixed
values. If economic agents really behave in that
way it is not clear on which type of data, first
or final-released data, should be based on the
economic policy.
32
7) The paper basically analyzes point forecasts,
but as the authors point out by passing,
congruent trace forecasts could be more relevant
because possibly this is the type of forecasts
considered by agents in their decisions. In this
respect trace forecasts should be published with
their corresponding fan chart.
33
8) It is not easy to discover how policy makers
and practicionaers evaluate trace forecasts. In
any case,reliable statistical tests to
discrimante between procedures generating trace
forecasts are not available and research on this
topic is needed.
34
Summing up, we have a stimulating paper which
evaluates the accuracy of real time data and
forecasts produced by prestigious institutions on
macro magnitudes of the special relevance at the
world level. The results point out important
pitfalls which, between other things,show the
interest of publishing and considering real-time
date and forecasts with confidence intervals.
More research on comparing trace forecasts in
needed.