EC3090 Econometrics Junior Sophister 20092010

1
EC3090 Econometrics Junior Sophister 2009-2010
Topic 2 The Simple Regression Model
Reading Wooldridge, Chapter 2 Gujarati, Chapters
1, 2 and 3
2
Topic 2 The Simple Regression Model

• 1. Definition of the Simple Regression Model
• Regression analysis is concerned with the study
  of the dependence of one variable (the dependent
  variable) on one or more other variables (the
  explanatory variables) with a view to estimating
  or predicting the population mean average value
  of the dependent variable in terms of the known
  values of the independent variables.
• Bivariate Example Explaining an individuals
  average wages given the individuals education
  level.
• (Illustrate using a scattergram)

3
Topic 2 The Simple Regression Model

• Definition of the Simple Regression Model
• Scattergram of distribution of wages
  corresponding to fixed education levels
• Note
• Variability in wages for each education level
• Despite variability, average wages increase as
  education level increases

4
Topic 2 The Simple Regression Model

• 1. Definition of the Simple Regression Model
• The population model
• Mean of Y for a given X is known as the
  conditional expected value
• E(YX)
• Note The unconditional expected value, E(Y),
  is just the mean of the population
• The population regression is the locus of the
  conditional means of the dependent variable for
  the fixed values of the explanatory variables
  (illustrate)

5
Topic 2 The Simple Regression Model

• Definition of the Simple Regression Model
• Scattergram of distribution of wages
  corresponding to fixed education levels
• Note
• Variability in wages for each education level
• Despite variability, average wages increase as
  education level increases

E(YX)
6
Topic 2 The Simple Regression Model

• 1. Definition of the Simple Regression Model
• The population model
• Mean of Y for a given X is known as the
  conditional expected value
• E(YX)
• Note The unconditional expected value, E(Y),
  is just the mean of the population
• The population regression is the locus of the
  conditional means of the dependent variable for
  the fixed values of the explanatory variables
  (illustrate)
• Population regression function
• E(YXi) f(Xi)

7
Topic 2 The Simple Regression Model

• 1. Definition of the Simple Regression Model
• The population model
• Assume linear functional form
• E(YXi) ß0ß1Xi
• ß0 intercept term or constant
• ß1 slope coefficient - quantifies the linear
  relationship between X and Y
• Fixed parameters known as regression
  coefficients
• For each Xi, individual observations will vary
  around E(YXi)

8
Topic 2 The Simple Regression Model

• Definition of the Simple Regression Model
• Scattergram of distribution of wages
  corresponding to fixed education levels
• Note
• Variability in wages for each education level
• Despite variability, average wages increase as
  education level increases

E(YX)
9
Topic 2 The Simple Regression Model

• 1. Definition of the Simple Regression Model
• The population model
• Assume linear functional form
• E(YXi) ß0ß1Xi
• ß0 intercept term or constant
• ß1 slope coefficient - quantifies the linear
  relationship between X and Y
• Fixed parameters known as regression
  coefficients
• For each Xi, individual observations will vary
  around E(YXi)
• Consider deviation of any individual observation
  from conditional mean
• ui Yi - E(YXi)
• ui stochastic disturbance/error term
  unobservable random deviation of an observation
  from its conditional mean

10
Topic 2 The Simple Regression Model

• 1. Definition of the Simple Regression Model
• The linear regression model
• Re-arrange previous equation to get
• Yi E(YXi) ui
• Each individual observation on Y can be
  explained in terms of
• E(YXi) mean Y of all individuals with same
  level of X systematic or deterministic
  component of the model the part of Y explained
  by X
• ui random or non-systematic component
  includes all omitted variables that can affect Y
• Assuming a linear functional form
• Yi ß0ß1Xi ui

11
Topic 2 The Simple Regression Model

• 1. Definition of the Simple Regression Model
• A note on linearity Linear in parameters vs.
  linear in variables
• The following is linear in parameters but not in
  variables
• Yi ß0ß1Xi2 ui
• In some cases transformations are required to
  make a model linear in parameters

12
Topic 2 The Simple Regression Model

• 1. Definition of the Simple Regression Model
• The linear regression model
• Yi ß0ß1Xi ui
• Represents relationship between Y and X in
  population of data
• Using appropriate estimation techniques we use
  sample data to estimate values for ß0 and ß1
• ß1 measures ceteris paribus affect of X on Y
  only if all other factors are fixed and do not
  change.
• Assume ui fixed so that ?ui 0, then
• ? Yi ß1 ? Xi
• ? Yi /? Xi ß1
• Unknown ui require assumptions about ui to
  estimate ceteris paribus relationship

13
Topic 2 The Simple Regression Model

• 1. Definition of the Simple Regression Model
• The linear regression model Assumptions about
  the error term
• Assume E(ui) 0 On average the unobservable
  factors that deviate an individual observation
  from the mean are zero
• Assume E(uiXi) 0 mean of ui conditional on Xi
  is zero regardless of what values Xi takes, the
  unobservables are on average zero
• Zero Conditional Mean Assumption
• E(uiXi) E(ui) 0

14
Topic 2 The Simple Regression Model

• 1. Definition of the Simple Regression Model
• The linear regression model Notes on the error
  term
• Reasons why an error term will always be
  required
• Vagueness of theory
• Unavailability of data
• Measurement error
• Incorrect functional form
• Principle of Parsimony

15
Topic 2 The Simple Regression Model

• 1. Definition of the Simple Regression Model
• Statistical Relationship vs. Deterministic
  Relationship
• Regression analysis is concerned with
  statistical relationships since it deals with
  random or stochastic variables and their
  probability distributions the variation in which
  can never be completely explained using other
  variables there will always be some form of
  error
• Deterministic relationships are exact e.g. Ohms
  law
• For metallic conductors over a limited range of
  temperature the current C is proportional to the
  voltage C(1/k)V, where 1/k is the constant of
  proportionality

16
Topic 2 The Simple Regression Model

• 1. Definition of the Simple Regression Model
• Regression vs. Correlation
• Correlation analysis measures the strength or
  degree of linear association between two random
  variables
• Regression analysis estimating the average
  values of one variable on the basis of the fixed
  values of the other variables for the purpose of
  prediction.
• Explanatory variables are fixed, dependent
  variables are random or stochastic.

17
Topic 2 The Simple Regression Model

• 2. Ordinary Least Squares (OLS) Estimation
• Estimate the population relationship given by
• using a random sample of data i1,.n
• Least Squares Principle Minimise the sum of the
  squared deviations between the actual and the
  sample values.
• Define the fitted values as
• OLS minimises
• Illustrate graphically

18
Topic 2 The Simple Regression Model

• Ordinary Least Squares Estimation
• First Order Conditions (Normal Equations)
• Solve to find
• Assumptions?

19
Topic 2 The Simple Regression Model

• 2. Ordinary Least Squares Estimation
• Method of Moments Estimator
• Replace population moment conditions with sample
  counterparts
• Assumptions?

20
Topic 2 The Simple Regression Model

• 2. Ordinary Least Squares Estimation
• Algebraic Properties
• 1.
• 2.
• 3. is always on the regression line
• 4.
• Using 1-4 show that SSTSSESSR
• SSTTotal Sum of Squares
• SSEExplained Sum of Squares
• SSRResidual Sum of Squares

21
Topic 2 The Simple Regression Model

• 3. Properties of OLS Estimator
• Gauss-Markov Theorem
• Under the assumptions of the Classical Linear
  Regression Model the OLS estimator will be the
  Best Linear Unbiased Estimator
• Linear estimator is a linear function of a
  random variable
• Unbiased
• Best estimator is most efficient estimator,
  i.e., estimator has the minimum variance of all
  linear unbiased estimators

22
Topic 2 The Simple Regression Model

• 3. Properties of OLS Estimator
• Assumptions required to prove unbiasedness
• A1 Regression model is linear in parameters
• A2 X are non-stochastic or fixed in repeated
  sampling
• A3 Zero conditional mean
• A4 Sample is random
• A5 Variability in the Xs
• Proof

23
Topic 2 The Simple Regression Model

• 3. Properties of OLS Estimator
• Assumptions required to prove efficiency
• A6 Homoscedasticity
• Illustrate
• A7 Autocorrelation
• Illustrate
• Illustrate efficiency

24
Topic 2 The Simple Regression Model

• 4. Interpretation of coefficients units of
  measurement
• Estimate impact that average return on equity
  () has on salary of CEOs (in thousands of euros)
• ß0 963.191 ? when ROE 0, predicted salary
  963.191
• Interpret as 963,161
• ß1 18.501 ? when ?ROE 1, ? predicted salary
  18.501
• Interpret as 18,501
• Use equation to compared predicted salaries for
  different ROEs, e.g. if ROE 20
• Interpret as 1,333,191
• Note Importance of units of measurement in
  interpretation of results

25
Topic 2 The Simple Regression Model

• 4. Interpretation of coefficients units of
  measurement
• Measure CEO salary in euros
• ß0 963,191 ? when ROE 0, predicted salary
  963,191
• ß1 18,501 ? when ?ROE 1, ? predicted salary
  18,501
• No effect on results but effect on
  interpretation
• In general If dependent variable is multiplied
  by a constant c, then the intercept and slope
  estimates are also multiplied by c.
• Measure ROE in decimals
• ß0 963.191 ? same as original model
• ß1 1,850.1 ? when ?ROE 0.01, ? predicted
  salary 18.501 same as original model
• In general If independent variable is
  multiplied (divided) by a nonzero constant c,
  then OLS slope coefficient is also multiplied
  (divided) by c

26
Topic 2 The Simple Regression Model

• 5. Goodness of Fit
• How well does regression line fit the
  observations?
• R2 (coefficient of determination) measures the
  proportion of the sample variance of Yi explained
  by the model where variation is measured as
  squared deviation from sample mean.
• Illustrate
• Recall SST SSE SSR ? SSE ? SST and SSE gt 0
• ? 0 ? SSE/SST ? 1
• If model perfectly fits data SSE SST and R2
  1
• If model explains none of variation in Yi then
  SSE0 since
• and R2 0

27
Topic 2 The Simple Regression Model

• 6. Functional Form
• Incorporate non-linearities into model
• Regress wages (measured in euro per hour) on
  years of schooling
• ? same return of ß1 0.54 (54 cent) for each
  each additional year of schooling
• hourly return more realistic regress
  ln(wagei) on years of schooling
• ? wagei ? (1000.083)?educi
• Actual wage function estimated
• In general
• Estimate
• 100 ? lnYi /?Xi percentage change in Y as a
  result of a one unit change in X
• Estimate
• ? lnYi /?lnXi percentage change in Y as a
  result of a percentage change in X

28
Topic 2 The Simple Regression Model

• 7. Estimating the variance of the OLS estimator
• Need to know dispersion (variance) of sampling
  distribution of OLS estimator in order to show
  that it is efficient (also required for
  inference)
• Show that
• Note depends on the error variance (reduces
  accuracy of estimates) and variation in X
  (increases accuracy of estimates)
• Show that
• What about the variance of the error terms ?2?
• Show that

29
Topic 2 The Simple Regression Model

• 8. Regression through the origin
• May wish to impose the restriction that when
  X0, Y0
• Estimate
• Regression through the origin.