Econometric Analysis

1
Econometric Analysis
Lecture 3 Further aspects of cross-section data
econometric models
2
Objectives

• To consider some issues relating to modelling
  strategy
• Modelling strategies - General-to-specific
  modelling
• Non-nested models
• Out-of-sample model predictive performance and
  testing
• Criteria for model selection

3
Frequently asked questions!

• Should I include all the variables in the
  database in my model?
• How many explanatory variables do I need in my
  model?
• How many models do I need to estimate?
• What functional form should I be using?
• What are interactive dummies do I need them?
• Which regression model will work best and how do
  I arrive at it?

4
Modelling strategies
The three golden rules of econometrics are test,
test and test. David F. Hendry (1980)
5
General to specific modelling

• Begin with a general model which nests the
  restricted model and so allows any restrictions
  to be tested
• These restrictions may be suggested either by
  theory or by empirical results

6
General to specific modelling (2) diagnostic
testing of the general model

• TEST 1
• First ensure that the general model does not
  suffer from any diagnostic problems. Examine the
  residuals in the general model to ensure that
  they possess acceptable properties.
• (Test for problems of heteroskedasticity,
  non-normality, incorrect functional form etc.)

7
General to specific modelling
General to specific modelling (3) testing
restrictions on the general model

• TEST 2
• Now test the restrictions implied by the
  specific model against the general model either
  by exclusion tests or other tests of linear
  restrictions.

8
General to specific modelling
General to specific modelling (4) diagnostic
testing of the simple model

• TEST 3
• If the restricted model is accepted, test its
  residuals to ensure that this more specific model
  is still acceptable on diagnostic grounds

See Peter Kennedy (2003) A Guide to Econometrics.
Fifth Edition - especially chapters 5
(Specification) and 21 (Applied Economics)
9
non-nested models and tests (1)

• We know how to use the F test for testing zero or
  linear restrictions on a model, but sometimes you
  may have two rival models to choose between,
  where neither can be nested within the other
  (i.e. neither is a restricted version of the
  other).
• An example might be
• and
• see Wooldridge 3rd edition p225
• Wooldridge suggests that, so long as the
  dependent variable is the same in both models (as
  is the case here) we can simply use R squared (or
  adjusted R squared if the number of parameters in
  the two models differs) to rank the models.
  (Some text books mention other criteria that can
  be used such as Akaikes Information Criterion
  (AIC) and Schwarzs Information Criterion (SIC)
  and most modern econometric software can give you
  these values.)

10
non-nested models and tests (2)

• An alternative approach would be to form a
  composite or encompassing model1 that nests both
  rival models and then test the relevant
  restrictions of each rival model against it.
  Assuming that the restrictions are accepted we
  would prefer (other things being equal) the model
  with the lower F statistic for the test of
  restrictions.
• So here, with a suitable adjustment to the
  notation, the encompassing model is
• We now test, separately, the hypotheses (1)
  (2)
• This kind of approach is due to Mizon and Richard
  (1986) - see Wooldridge 3rd edition, p 309-310.
• 1 Kennedy, P (2003) A Guide to Econometrics Fifth
  Edition p 100, calls this an artificial nesting
  model

11
non-nested models and tests (3)

• The Davidson-MacKinnon J test see W3 p310, W4
  p304
• This test is based on the observation that if the
  first equation is true then the fitted values
  from the second equation, when added to the first
  equation, should be insignificant. This gives us
  a procedure to follow.
• Estimate equation 1 and obtain the fitted values
  of the dependent variable. Add this variable to
  the list of regressors in equation 2. A
  significant t value for this regressor would be
  evidence against equation 1 and in favour of
  equation 2. Repeat the procedure for the
  equations the other way round. Rank the models on
  the basis of this test.
• Of course neither of these methods may give a
  clear ranking, and in any case the method cannot
  be used if the dependent variables differ between
  the two rival models. One would also want to
  examine the diagnostic test results when choosing
  between two models.

12
the out-of-sample performance of a model tests
and measures of performance

• examination of a models predictive accuracy
  for individual out-of-sample observations
• examination of a models predictive accuracy
  for a group of out-of-sample observations
• Chow prediction test

13
the out-of-sample performance of a model
Suppose that (based on a sample data set of
i1,,n) you have arrived at a preferred model
specification of the form
or in matrix form
You can now generate m out-of-sample predictions
based on this model using the data points in1,
n2,.,nm. You just substitute into this
equation the values of X1, X2 and X3 for these m
observations and generate m values of

We can analyse the m prediction errors in various
ways.
14
Analysis of out of sample predictions and
prediction errors

• Summary measures of out-of-sample (forecast)
  accuracy

Mean Error
Mean Absolute Prediction Error (MAPE)
Root Mean Square Error (RMSE)
15
Analysis of out of sample predictions and
prediction errors

• CHOW prediction test

F(m,n-k)
Example A wage equation based on n500
observations and 3 regressors plus a constant
intercept gives an RSS of 78.8769257. For an
extended sample adding a further m95
observations the RSS 94.4958041 So Fcal
(94.4958041- 78.8769257)/95
78.8769257/(500-4) 1.03385097 Using PcGive's
tail probability tool F(95, 496) 1.0339
0.4026 so we accept H0 of parameter
constancy.
16
Criteria for model selection weighing up
everything
In the end the researchers judgment must be used
in weighing up various criteria 1 The Economic
Criterion are the estimated parameters
plausible? (Economic Significance) 2 The First
Order Statistical Criterion does the model
provide a good fit (in-sample) with statistically
significant parameter estimates? 3 The Second
Order Statistical Criterion - Is the model
generally free of misspecification problems as
evidenced in the diagnostic tests? 4 The Out of
Sample Predictive Criterion does the model
provide good out of sample predictions? (See also
the five criteria for a congruent model stated
in Kennedys book)