FRONTIERS OF REAL-TIME DATA ANALYSIS

1
FRONTIERS OF REAL-TIME DATA ANALYSIS

• Dean Croushore
• Associate Professor, University of Richmond
• Interim Director, Real-Time Data Research Center,
  Federal Reserve Bank of Philadelphia
• October 2008

2
Introduction

• First paper to do real-time data analysis
• Gartaganis-Goldberger, Econometrica (1955)
• Statistical properties of the statistical
  discrepancy between GNP and gross national income
  changed after data were revised in 1954

3
Research Categories

• Data Revisions
• Forecasting
• Monetary Policy
• Macroeconomic Research
• Current Analysis

4
Introduction

• Data sets
• Real-Time Data Set for Macroeconomists
• Philadelphia Fed University of Richmond
• Need for good institutional support
• Club good non-rival but excludable

5
Introduction

• Data sets
• Unrestricted access
• U.S. Philadelphia Fed, St. Louis Fed, BEA
• OECD
• Bank of England (recently updated)
• Restricted access
• EABCN
• Fate unclear
• Canada
• One-time research projects
• Many, most not continuously updated

6
Introduction

• Analysis of data revisions is not criticism of
  government statistical agencies!
• May help agencies improve data production process
• Revisions reflect limited resources devoted to
  data collection
• Revised data usually superior to unrevised data
  (U.S. CPI vs. PCE price index)

7
Introduction

• Structure of data sets
• The data matrix
• Columns report vintages (dates on which data
  series are observed)
• Rows report dates for which economic activity is
  measured
• Moving across rows shows revisions
• Main diagonal shows initial releases
• Huge jumps in numbers indicate benchmark
  revisions with base year changes

8
REAL OUTPUT Vintage 11/65 2/66 5/66 . .
. 11/07 2/08 Date 47Q1 306.4 306.4 306.4 .
. . 1570.5 1570.5 47Q2 309.0 309.0 309.0 . .
. 1568.7 1568.7 47Q3 309.6 309.6 309.6 . .
. 1568.0 1568.0 . . . . . . . . . . . . . . . .
. . . . . 65Q3 609.1 613.0 613.0 . .
. 3214.1 3214.1 65Q4 NA 621.7 624.4 . .
. 3291.8 3291.8 66Q1 NA NA 633.8 . .
. 3372.3 3372.3 . . . . . . . . . . . . . . . .
. . . . . 07Q1 NA NA NA . . .
11412.6 11412.6 07Q2 NA NA NA . .
. 11520.1 11520.1 07Q3 NA NA NA
. . . 11630.7 11658.9 07Q4 NA NA NA
. . . NA 11677.4
9
Data Revisions
10
Data Revisions

• What Do Data Revisions Look Like?
• Are They News or Noise?
• Is the Government Using Information Efficiently?
• Are Revisions Forecastable?
• How Should We Model Data Revisions?
• Key issue are data revisions large enough
  economically to worry about?

11
Data Revisions

• What Do Data Revisions Look Like?
• Short Term (example)
• Long Term (example)
• What Do Different Types of Data Revisions Look
  Like?
• Short run revisions based on additional source
  data
• Benchmark revisions based on structural changes
  or updating base year

12
(No Transcript)
13
(No Transcript)
14
Data Revisions

• Are Data Revisions News or Noise?
• Data Revisions Add News Data are optimal
  forecasts, so revisions are orthogonal to early
  data revisions are not forecastable
• Data Revisions Reduce Noise Data are measured
  with error, so revisions are orthogonal to final
  data revisions are forecastable

15
Data Revisions

• Are Data Revisions News or Noise?
• Mankiw-Runkle-Shapiro (1984) Money data
  revisions reduce noise
• Mankiw-Shapiro (1986) GDP data revisions
  contain news
• Mork (1987) GMM results show final NIPA data
  contain news other vintages are inefficient and
  neither noise nor noise
• UK Patterson-Heravi (1991) revisions to most
  components of GDP reduce noise

16
Data Revisions

• Is the Government Using Information Efficiently?
• Theoretical Issue Should the government report
  its sample information or project an unbiased
  estimate using extraneous information?

17
Data Revisions

• Is the Government Using Information Efficiently?
• Key Issue What is the trade-off the government
  faces between timeliness and accuracy?
• Zarnowitz (1982) evaluates quality of different
  series
• McNees (1989) found within-quarter estimate of
  GDP to be as accurate as estimate released 15
  days after quarter end

18
Data Revisions

• Findings of bias and inefficiency based on
  ex-post tests
• UK Garratt-Vahey (2003)
• US Aruoba (2008)

19
Data Revisions

• Findings of bias and inefficiency of seasonally
  revised data
• Kavajecz-Collins (1995)
• Swanson-Ghysels-Callan (1999)
• Revisions to seasonals may be larger than
  revisions to NSA data Fixler-Grimm-Lee (2003)
• Key question Are seasonal revisions
  predictable? Who cares if that is an artifact of
  construction?

20
Data Revisions

• Key Issue If early government data are
  projections, then state of business cycle may be
  related to later data revisions.
• Dynan-Elmendorf (2001) GDP is misleading at
  turning points
• Swanson-van Dijk (2004) volatility of revisions
  to industrial production and producer prices
  increases in recessions

21
Data Revisions

• Are Revisions Forecastable?
• Conrad-Corrado (1979) use Kalman filter to
  improve governments monthly data on retail sales
• Aruoba (2008) revisions to many U.S. variables
  are forecastable

22
Data Revisions

• Are Revisions Forecastable?
• Key Issue can revisions be forecast in
  real-time (or just ex-post)?
• Guerrero (1993) combines historical data with
  preliminary data on Mexican industrial production
  to get improved estimates of final data
• Faust-Rogers-Wright (2005) Examines G-7
  countries output forecasts find Japan U.K.
  output revisions forecastable in real time

23
Data Revisions

• How Should We Model Data Revisions?
• Howrey (1978)
• Conrad-Corrado (1979)
• UK Holden-Peel (1982)
• Harvey-McKenzie-Blake-Desai (1983)
• UK Patterson (1995)
• UK Kapetanios-Yates (2004)

24
Data Revisions

• How Should We Model Data Revisions?
• Is there any scope for new research here?
• Show predictability between different vintages to
  help data agencies improve methods
• Ex US data on PCE inflation

25
Forecasting
26
Forecasting

• Forecasts are only as good as the data behind
  them
• Literature focuses on model development trying
  to build a better forecasting model, especially
  comparing forecasts from a new model to other
  models or to forecasts made in real time
• Details Croushore (2006) Handbook of Economic
  Forecasting

27
Forecasting

• Does the fact that data are revised matter
  significantly (in an economic sense) for
  forecasts?

28
Forecasting

• EXAMPLE THE INDEX OF LEADING INDICATORS
• Leading indicators seem to predict recessions
  quite well.
• But did they do so in real time? The evidence
  suggests skepticism.
• Diebold and Rudebusch (1991) investigated the
  issue, using real-time data
• Their conclusion The leading indicators do not
  lead and they do not indicate!
• The use of revised data gives a misleading
  picture of the forecasting ability of the leading
  indicators.

29
(No Transcript)
30
Forecasting

• EXAMPLE THE INDEX OF LEADING INDICATORS
• Chart shows not much problem
• But recession started in November 1973
• Subsequently, leading indicators were revised
  ex-post they do much better

31
(No Transcript)
32
Forecasting

• Why Are Forecasts Affected by Data Revisions?
• Change in data input into model
• Change in estimated coefficients
• Change in model itself (number of lags)
• See experiments in Stark-Croushore (2002)

33
Forecasting

• What Do We Use as Actuals?
• Answer Depends on purpose
• Best measures are probably latest-available data
  for truth (though perhaps not in
  fixed-weighting era)
• But forecasters would not anticipate
  redefinitions and generally forecast to be
  consistent with government data methods (example
  pre-chain-weighting period 2013 capitalization
  of RD)

34
Forecasting

• What Do We Use as Actuals?
• Real-Time Data Set many choices
• first release (or second, or third)
• four quarters later (or eight or twelve)
• Date of annual revision (July for U.S. data)
• last benchmark (the last vintage before a
  benchmark revision)
• latest available

35
Forecasting

• How Should Forecasts Be Made When Data Are
  Revised?
• Key issue temptation to cheat!
• Try method it doesnt work but thats because
  of one outlier dummy out that observation the
  method works!
• If data are not available, use a real-time proxy,
  dont peak at future data
• Cheating is inherent because you know the history
  already

36
Forecasting

• Forecasting with Real-Time versus
  Latest-Available Data
• Denton-Kuiper (1965) first to compare forecasts
  with real-time vs revised data
• Cole (1969) data errors reduce forecast
  efficiency may lead to biased forecasts
• Trivellato-Rettore (1986) data errors in a
  simultaneous equations model affect everything
  estimated coefficients and forecasts but for
  small model of Italian economy, addition to
  forecast errors were not large

37
Forecasting

• Forecasting with Real-Time versus
  Latest-Available Data
• Faust-Rogers-Wright (2003) research showing
  forecastability of exchange rates depended on a
  particular vintage of data other vintages show
  no forecastability
• Molodtsova (2007) combining real-time data with
  Taylor rule allows predictability of exchange
  rate
• Moldtsova-Nikolsko-Rzhevskyy-Papell (2007)
  dollar/mark exchange rate predictable only with
  real-time data

38
Forecasting

• Summary for forecasting, sometimes data vintage
  matters, other times it doesnt

39
Forecasting

• Levels versus Growth Rates
• Howrey (1996) level forecasts of GNP more
  sensitive than growth forecasts so policy should
  feed back on growth rates, not levels
• Kozicki (2002) choice of forecasting with
  real-time or latest-available data is important
  for variables with large level revisions

40
Forecasting

• Model Selection and Specification
• Swanson-White (1997) explores model selection
• Robertson-Tallman (1998) real-time data affect
  model specification for industrial production but
  not for GDP
• Harrison-Kapetanios-Yates (2005) it may be
  optimal to estimate a model without using most
  recent preliminary data
• Summary model choice is sometimes affected by
  data revisions

41
Forecasting

• Evidence on Predictive Content of Variables
• Croushore (2005) consumer confidence indicators
  have no predictive power in real time, even when
  they appear to when using latest-available data

42
Forecasting

• Optimal Forecasting When Data Are Subject to
  Revision
• Howrey (1978) adjusts for differing degrees of
  revision using Kalman filter in forecasting, use
  recent data but filter it
• Harvey-McKenzie-Blake-Desai (1983) use
  state-space methods with missing observations to
  account for irregular data revisions large gain
  in forecast efficiency compared with ignoring
  data revisions

43
Forecasting

• Optimal Forecasting When Data Are Subject to
  Revision
• Howrey (1984) use of state-space models to
  improve forecasts of inventory investment yields
  little improvement
• Patterson (2003) illustrates how to combine
  measurement process with data generation process
  to improve forecasts for income consumption

44
Forecasting

• Optimal Forecasting When Data Are Subject to
  Revision
• What information set to use?
• Koenig-Dolmas-Piger (2003), Kishor-Koenig (2005)
  focus on diagonals to improve forecasting treat
  data the same that have been revised to the same
  degree

45
Forecasting

• Optimal Forecasting When Data Are Subject to
  Revision
• Summary There are sometimes gains to accounting
  for data revisions but predictability of
  revisions (today for US data) is small relative
  to forecast error (mainly seasonal adjustment)

46
Forecasting

• A Troublesome Issue
• Specifying a process for data revisions
• Some papers specify an AR process
• But research on revisions suggests that benchmark
  revisions are not so easily characterized

47
Forecasting

• Key Issue What are the costs and benefits of
  dealing with real-time data issues versus other
  forecasting issues?

48
Monetary Policy
49
Monetary Policy Data Revisions

• How Much Does It Matter for Monetary Policy that
  Data Are Revised?
• How Misleading Is Monetary Policy Analysis Based
  on Final Data Instead of Real-Time Data?
• How Should Monetary Policymakers Handle Data
  Uncertainty?

50
Monetary Policy Data Revisions

• How Much Does It Matter for Monetary Policy that
  Data Are Revised?
• Example Feds favorite inflation measure is the
  Personal Consumption Expenditures Price Index
  Excluding Food Energy Prices (PCEPIXFE)
• But it has been revised substantially

51
(No Transcript)
52
(No Transcript)
53
Monetary Policy Data Revisions

• How Much Does It Matter for Monetary Policy that
  Data Are Revised?
• Croushore (2008) PCE revisions could mislead the
  Fed
• Maravall-Pierce (1986) The Fed optimally signal
  extracts from the noise in money data, so data
  revisions would not significantly affect monetary
  policy
• Kugler et al. (2005) Monetary policy shojuld be
  less aggressive because of data revisions

54
Monetary Policy Data Revisions

• How Misleading Is Monetary Policy Analysis Based
  on Final Data Instead of Real-Time Data?
• Croushore-Evans (2006) Data revisions do not
  significantly affect measures of monetary policy
  shocks in recursive systems, but they make
  identification of simultaneous systems
  problematic

55
Monetary Policy Data Revisions

• How Should Monetary Policymakers Handle Data
  Uncertainty?
• Coenen-Levin-Wieland (2001) use money as an
  indicator when GDP data are uncertain
• Bernanke-Boivin (2003) use factor model to
  incorporate much data results do not depend on
  using real-time data instead of revised data

56
Monetary Policy Data Revisions

• How Should Monetary Policymakers Handle Data
  Uncertainty?
• Giannone-Reichlin-Sala (2005) extract real-time
  information to determine a real shock and a
  nominal shock, which represent fundamental
  dynamics of US economy

57
Monetary Policy Data Revisions

• How Should Monetary Policymakers Handle Data
  Uncertainty?
• Aoki (2003) without certainty equivalence,
  policymakers need to react less aggressively
  theoretical view
• Similar results hold with uncertainty about
  potential output or other analytical concepts

58
Monetary Policy Analytical Revisions

• What Happens When Economists or Policymakers
  Revise Conceptual Variables?
• Output gap
• Natural rate of unemployment
• Equilibrium real interest rate
• Concepts are never observed, but are centerpiece
  of macroeconomic theory

59
Monetary Policy Analytical Revisions

• Orphanides (2001) Fed overreacted to perceived
  output gap in 1970s, causing Great Inflation but
  output gap was mismeasured

60
Monetary Policy Analytical Revisions

• One strand of literature plug alternative data
  vintages into Taylor rule
• Kozicki (2004) on U.S. data
• Kamada (2005) on Japanese data
• Other Taylor rule work
• Rudebusch (2001) reverse engineer Taylor rule
  it would be more aggressive if data werent
  uncertain
• Orphanides (2003) if policy rules are based on
  revised data, they are too aggressive

61
Monetary Policy Analytical Revisions

• Other real-time models of policy rules
• Cukierman-Lippi (2005) Fed was too aggressive in
  1970s, appropriately conservative in 1990s
• Boivin (2006) Fed changed policy parameters in
  1970s and temporarily reduced response to
  inflation causing Great Inflation

62
Monetary Policy Analytical Revisions

• Other natural rate issues
• Orphanides-Williams (2002) large costs to
  ignoring mismeasurement of natural rate of
  unemployment and natural rate of interest
• Staiger-Stock-Watson (1997) tremendous
  uncertainty about natural rate of unemployment
• Clark-Kozicki (2005) ditto for natural rates of
  interest

63
Monetary Policy Analytical Revisions

• Output gap uncertainty
• U.S. Orphanides-van Norden (2002)
• UK Nelson-Nikolov (2003)
• Germany Gerberding-Seitz-Worms (2005)
• Euro area Gerdesmeieir-Roffia (2005)
• Norway Bernhardsen (2005)
• Canada Cayen-van Norden (2005)
• Germany Döpke (2005)

64
Monetary Policy Analytical Revisions

• Policy models may change
• Tetlow-Ironside (2007) changes in FRB-US model
  changed the story the model was telling to
  policymakers

65
Monetary Policy Analytical Revisions

• What Happens When Economists or Policymakers
  Revise Conceptual Variables?
• Key issue end-of-sample inference for
  forward-looking concepts (filters)
• Key issue optimal model of evolution of
  analytical concepts
• Most work is statistical perhaps a theoretical
  breakthrough is needed

66
Macroeconomic Research
67
Macroeconomic Research

• How Is Macroeconomic Research Affected By Data
  Revisions?
• Croushore-Stark (2003) how results from key
  macro studies are affected by alternative
  vintages
• Boschen-Grossman (1982) testing neutrality of
  money under rational expectations support for RE
  with revised data, but not with real-time data

68
Macroeconomic Research

• How Is Macroeconomic Research Affected By Data
  Revisions?
• Amato-Swanson (2001) the predictive content of
  money for output is not clear in real time only
  in revised data

69
Macroeconomic Research

• Should Macroeconomic Models Incorporate Data
  Revisions?
• Aruoba (2004) business-cycle dynamics are
  captured better by a DSGE model that accounts for
  data revisions than one that does not
• Edge, Laubach, Williams (2004) learning
  explains long-run productivity growth forecasts
  helps explain cycles in employment, investment,
  long-term interest rates

70
Macroeconomic Research

• Do Data Revisions Affect Economic Activity?
• Oh-Waldman (1990) false (positive) announcements
  increase economic activity with leading
  indicators and industrial production in real time
• Bomfim (2001) improving the signal in data
  would exacerbate cyclical fluctuations if agents
  performed optimal signal extraction but if
  agents ignore data revisions, then improving data
  quality would reduce cyclical fluctuations

71
Macroeconomic Research

• Overall literature in its infancy more work
  needed in all three areas (robustness of research
  results, incorporating data revisions into macro
  models, examining how or whether data revisions
  affect economic activity)

72
Current Analysis

• How Do Financial Markets React to Data Revisions?
• Christoffersen-Ghysels-Swanson (2002) need
  real-time data to properly determine announcement
  effects in financial markets

73
Current Analysis

• How Is Business Cycle Dating Affected By Data
  Revisions?
• Economists like to argue about the state of the
  business cycle . . .

74
(No Transcript)
75
Current Analysis

• How Is Business Cycle Dating Affected By Data
  Revisions?
• Chauvet-Piger (2003, 2005) test algorithms to
  identify turning points in real time
• Chauvet-Hamilton (2006) develop alternative
  recession indicators and forecasts in real time
• Nalewaik (2007) using real-time gross domestic
  income helps forecast recessions better than just
  using GDP

76
Current Analysis

• Overall much additional research needed in
  current analysis in real time

77
Summary

• Field of real-time data analysis offers many
  opportunities for new research
• Most promising areas
• Macroeconomic research incorporating data
  revisions into macro models
• Current analysis of business and financial
  conditions
• Other areas are more mature need more
  sophisticated analysis