General Estimation Approaches Nonlinear

1
General Estimation Approaches(Nonlinear)

• Course Applied Econometrics
• Lecturer Zhigang Li

2
Purpose

• Our purpose is to obtain good estimates of some
  parameters. There are generally many different
  ways of achieving this. We have been using OLS
  mainly, but there are many occasions (mostly
  nonlinear models) in which have better
  alternatives.
• Generalized Method of Moments (GMM)
• Maximum likelihood Estimators

3
Method of Moments

• The idea of the Method of Moments estimators is
  to find parameter estimates that let the actual
  moments and model-based moments match the best.
• Moments Mean, variance, correlation,
• Actual moments are typically functions of actual
  data and the chosen parameter estimates.

4
Example 1

• E(Y)µ
• Y is actual data
• µ is the parameter to be estimated
• Rewrite the equation as E(Y-µ)0
• Actual Moments S(Y-µ)/n
• Model-based moment 0
• We try to find a value for µ such that
• S(Y-µ)/n0

5
Example 2

• IV Estimation
• The model is f(Y, X, ?)e
• The function f(.) could be nonlinear in ?. The
  linear model Y ?Xe is a special case.
• X could be endogneous.
• Suppose that an instrumental variable Z is
  available (if X is exogenous, then Z can be X
  itself).
• What is the moment condition we could use?
• EZf(Y, X, ?)0

6
Example 3

• Intertemporal Optimization
• Eßu(ct1)(1rt)-u(ct)Ot0
• Where Ot indicates the information set available
  at time t.
• This suggests that any variables zt in the
  information set Ot can be an instrument because
• Eztßu(ct1)(1rt)-u(ct)0

7
Generalized Method of Moments (GMM)

• In MM, the number of moment equations is the same
  as the number of parameters to be estimated (this
  is called exactly identified).
• In GMM, the number of moment equations is more
  than the number of parameters to be estimated
  (this is called over-identified). In this case,
  parameters satisfying all the moment equations
  may not be available.
• Solution Instead of solving the equation system
  for parameter estimates, we search for parameter
  values that can minimize some reasonable
  criterion function.
• For example, in example 2, if we have only one
  parameter ? to estimate but we have n
  instrumental variables, then we can minimize
• SEZif(Y, X, ?)2

8
GMM Framework

• Moment Conditions
• Eml(y,x,z,?)0, l1,2,,L.
• GMM
• Search for values of ? to minimize
• Where A is a positive definite matrix to produce
  a consistent estimator of ? and m-bar is the
  sample average of the moment equations

9
Key Assumptions of GMM

• Y, X, and Z are stationary.

10
Maximum Likelihood

• Idea Choose parameter values such that the
  theoretical likelihood for the observed data to
  happen is the largest.

11
A Simple Example

• For example, suppose we have a couple of
  observations of a variable X with normal
  distribution N(µ,1). What is a good estimate of
  µ?
• Least Square Choose µ to minimize S(xi-µ)2.
• Maximum Likelihood Assume that the observations
  are independent, then the likelihood of observing
  all of them is the product of the density of them
  each f(x1µ)f(x2µ) f(xnµ), where
  f(xiµ)exp-(xi-µ)2/(2p)1/2. Therefore, we may
  choose µ to minimize
• -Slogf(xiµ)

12
Example 2 Estimating Autoregressive Model

• xt?xt-1ut
• f(x1, x2,, xn)g(xTxT-1)g(x2x1)g(x1)
• Where f(x1, x2,, xn) is the density of observing
  the actual time series x1, x2,, xn.
• Assume the variance of ut is one and the
  distribution is normal, then g(xtxt-1)
  exp-(xt-?xt-1)2/2 /(2p)1/2
• We can estimate the value of ? by minimizing
  logf(x1, x2,, xn).

13
Example 3 Binary Response Models

• P(y1X)G(ßX)
• Y is either 0 or 1.
• P(y1X) is the probability for y to be one given
  the actual value of X.
• For example, y may be whether to get higher
  education and X may be the characteristics of an
  individual and his/her family.
• Compared with the linear probability model we
  have discussed before, the binary response model
  does not predict probabilities greater than one
  or less than zero.
• In the logic model, G is the logistic function
  G(z)exp(z)/1exp(z)

14
How to estimate the binary response models?

• Assume that we have a random sample of size n
  (i.e. we have n observations each of them
  contains information on binary variable y and
  usual variables X).
• Similar to example one, since we have random
  sample, the density of observing y1, y2, , yn is
  simply the product of the density of each of the
  ys.
• f(yiXi)G(ßXi)y1-G(ßXi)1-y, y0,1.