MULTIPLE REGRESSION OF TIME SERIES - PowerPoint PPT Presentation

Loading...

PPT – MULTIPLE REGRESSION OF TIME SERIES PowerPoint presentation | free to view - id: eea6d-NzA3Z



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

MULTIPLE REGRESSION OF TIME SERIES

Description:

12 ADVERT 2.28 0.198 11.500 0.0000. b) Linear Regression Sales=f(Competition)-Eq.10-3 ... 12 ADVERT 2.5248 0.1295 19.495 0.0000. 13 COMP -0.5449 0.1230 -4.432 0.0016 ... – PowerPoint PPT presentation

Number of Views:162
Avg rating:3.0/5.0
Slides: 52
Provided by: mong2
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: MULTIPLE REGRESSION OF TIME SERIES


1
CHAPTER 10
  • MULTIPLE REGRESSION OF TIME SERIES
  • LINEAR MULTIPLE REGRESSION MODEL
  • General Multiple Regression Model

2
BIG CITY BOOKSTORE EXAMPLE
  • Multiple Regression Model
  • Coefficient of Multiple Determination-R2
  • Partial Regression Coefficients
  • Describing the Regression Plane

3
THE MULTIPLE REGRESSION MODELING PROCESS
  • MULTICOLLINEARITY
  • Collinearity Among Variables
  • Solutions Multicoll. Problems
  • An Example Solution
  • PARTIAL F-TEST FOR INCLUDING VARIABLES SERIAL
    CORRELATION PROBLEMS Forecasting with Serially
    Correlated Errors

4
ANALYSIS OF STOCK INDEXES USING COILS
  • ELASTICITIES AND LOGARITHMIC RELATIONSHIPS
  • HETEROSCEDASTICITY
  • Wrong Functional Form-UK and US Stock Indexes
    Goldfeld-Quandt Test

5
Interpretation of Elasticities
  • WEIGHTED LEAST SQUARES
  • GENERALIZED LEAST SQUARES
  • BETA COEFFICIENTS
  • DICHOTOMOUS (DUMMY) VARS. FOR MODELING EVENTS
  • CONSTRUCTING CONFIDENCE AND PREDICTION INTERVALS
  • PARSIMONY AND REGRESSION ANALYSIS
  • AUTOMATED REGRESSION METHODS

6
CHAPTER 10 MULTIPLE REGRESSION OF TIME SERIES
  • "Theorize longer, analyze shorter. Don't be in
    a rush to run the program. Think about the model
    from every angle, hypothesize how different
    variables affect each other. When you have a
    theory, then try it. Impatience is the enemy of
    valid models. Contemplation is productive work."
    The Author"Measure twice, cut once." The
    Carpenter's Rule

7
GENERAL LINEAR MULTIPLE REGRESSION
  • General Multiple-Regression Model Y a
    b1X1 b2X2...bnXn e (10-1)"n" is
    rarely above 6 to 7.

8
  • BIG CITY BOOKSTORE EXAMPLE
  • Table 10-1. Big City Bookstore
  • Year Sales(Y) Advertising(X1) Competition
    (X2)
  • (1000) (1000) (1000sq.ft.) 1
    27 20 10 2
    23 20 15 3
    31 25 15 4
    45 28 15 5
    47 29 20 6 42
    28 25 7 39
    31 35 8 45
    34 35 9 57
    35 20 10 59
    36 30 11 73
    41 20 12 84 45
    20

9
Multiple Regression Model
  • Correlation Matrix Sales
    Advertising CompetitionSales 1
    .964 .221Advertising .964
    1 .426 Competition
    .221 .426 1
  • SALES f(ADV.) IGNORING COMPETITION y
    -23.02 2.280X1 (10-2)
    (-3.64) (11.5) (X1 advertising)
    Syx 5.039 R2 .923 n12
    F132.26 DW 1.13676

10
  • SALES f(COMP.) IGNORING ADVERTISING
  • y 37.34 .477X2 (10-3)
    (2.339) (.687) (X2competition)
    Syx 18.574 R2 -.045 n 12 F
    .507 DW .3767
  • SALES f(ADV. , COMP.) SIMULTANEOUSLY
  • y -18.80 2.525X1 - .545X2 (10-4)
    (-4.879) (19.50) (-4.432) Syx 2.978
    R2 .973 n 12 F 199.21 DW
    1.7705

11
  • Table 10-2. Simple and Multiple Regression for
    Big City Bookstorea) Linear
    Regression Salesf(Advertising)-Eq.10-2
    1 Dependent Variable SALES 2 Usable
    Observations 12 Degs of Freedom 10 3 R
    Bar2
    0.9227 4 Std Error of Dependent Variable
    18.1225 5 Standard Error of Estimate
    5.0394 6 Sum of Squared Residuals
    253.9530 7 Regression F(1,10)
    132.26 8 Significance Level
    of F 0.00000044 9
    Durbin-Watson Statistic
    1.13710 Variable Coeff Std Error T-Stat
    Signif
    11 Constant -23.02 6.316 -3.644
    0.004512 ADVERT 2.28 0.198
    11.500 0.0000

12
  • b) Linear Regression Salesf(Competition)-Eq.10-3
    1 Dependent Variable
    SALES 2 Usable Observations 12 Degs of Freedom
    10 3 R Bar2
    -0.050 4 Std Deviation of Dependent
    Variable 18.123 5 Standard Error of
    Estimate 18.574 6 Sum of
    Squared Residuals 3449.780 7
    Regression F(1,10) 0.472
    8 Significance Level of F 0.5076
    9 Durbin-Watson Statistic
    0.37710 Variable Coeff Std Error T-Stat
    Signif
    11 Constant 37.3372 15.960 2.339
    0.041412 COMP 0.4767 0.694
    0.687 0.5076

13
  • c) Mult Regression Salesf(Adver.
    Comp.)-Eq.10-4 1 Dependent
    Variable SALES 2 Usable Observations 12 Degs
    of Freedom 9 3 R Bar2
    0.9730 4 Std Deviation of Dependent
    Variable 18.1225 5 Standard Error of Estimate
    2.978 6 Sum of Squared
    Residuals 79.803 7 Regression
    F(2,9) 199.2155 8
    Significance Level of F
    0.00000004 9 Durbin-Watson Statistic
    1.77110 Variable Coeff
    Std Error T-Stat Signif
    11 Constant -18.7958
    3.8520 -4.880 0.000912 ADVERT 2.5248
    0.1295 19.495 0.000013 COMP
    -0.5449 0.1230 -4.432 0.0016

14
  • Multiple Coefficient of Determination - R2
    Expl Variance Unexp Var.
    Syx2R2 1 - 1-
    Total Variance Total Var. Sy2
  • Partial Regression Coefficients Y -18.80
    2.52530 - .545X2 e Y
    -18.80 75.75 - .545X2 e 56.86 -
    .545X2 e (10-5)

15
  • Figure 10-2 Here-Deviations About a Plane or
    HyperspaceDescribing The Regression
    PlaneFigure 10-3 Here Regression Plane for
    Equation 10-3.Figure 10-4 Here Several Reg
    Lines on the Reg PlaneFigure 10-5 The Multiple
    Regression Modeling Process

16
MULTIPLE REGRESSION MODELING-PLOTS
  • Res. VS Included Indep Vars. Detect
    heteroscedasticity, misspecification (Nlin)
  • Res. VS Excluded Indep Vars. Detect variable to
    be included, misspecification (Nlin)
  • Res. VS Y. Detect serial correlation,
    heteroscedasticity,misspecifications
  • Residualst VS Residualst-1. Detect serial
    correlation, out of sample projections,
    unreasonable forecasts.

17
MULTICOLLINEARITY
  • Highly Related Independent Variables may or may
    not be a problem if a problem coefficient be
    wrong also its standard error
    Syx Sb1
    (10-6) ?x2(1 -
    r122) With r121, Impossible to fit unique
    model because of their redundancy.

18
  • Multicollinearity Problems (MCP)
  • May not be evident to the analyst
  • Yields the wrong sign or insignificant t-values
  • Avoid MCPs By
  • Good theory
  • Large sample sizes
  • Good diagnostic procedures
  • Sometimes MCP are simply an artifact of the sample

19
DETECTING
  • Insignificant/Incorrect regression coefficients
  • Some strange regression results from MCP
  • Assume correlation between X1 and X2 is high
  • Each is highly correlated with Y
  • Often one regression coefficient is negative
    despite the positive relationship
  • Often one variable is highly significant the
    other not
  • Often the sum of the regression coefficients
    equals true, single variable regression
    coefficient.

20
COLLINEARITY AMONG MORE THAN TWO VARIABLES
  • What is the problem? Consider that some for of
    Linear Transformation of X2 and X3 perfectly
    defines X1
  • X1 a b2X2 b3X2
  • with Syx 0, R2 1, and r123 1 Thus, when
    an attempt is made to fit the following
    relationship, a solution is not possible.

21
  • Y a b1X1 b2X2 b3X3 eWhen perfect
    collinearity the estimation procedure aborts.
    Often with Dichotomous Variable
    t Yt d1
    d2 d3 d4 1 10 1
    0 0 0 2 20 0
    1 0 0 3 30
    0 0 1 0 4 5
    0 0 0 1

22
  • Thus, d1 1 - d2 - d3 - d4
  • Solution is impossible.
  • Avoid-always defining one-less variable
  • The last var. is part of constant
  • Dummy vars. are studied here later

23
Solutions to Multicollinearity Problems
  • With redundant measures delete the redundant
    variable. Good theory precludes most redundant
    variables.
  • Some MCP are an artifact of a specific sample.
    Then additional obs. may eliminate the problem.
  • MCP from flawed theories. When vars. represent
    different dimensions of an influence then they
    might be combined using factor analysis

24
  • When MCP is caused by a unique sample use ridge
    regression.
  • When Theory dictates that both variables should
    be included, then include them.While MCP
    affects regression coefficients and their
    interpretability, it might not alter the
    predictive power of the regression model. That
    is, the overall relationship may still be useful
    in predictive power, this being confirmed by a
    low standard error of estimate and high F-value.
    To better understand this, consider the example
    below.

25
  • An Example of MCPs (MULT.DAT)
  • Table 10-3. Correlation Matrix
    X1 X2 X3
    X4 Y X1 1.0000 -0.1067
    0.1821 0.9998 0.4622 X2 -0.1067 1.0000
    0.1031 -0.1053 0.7479 X3 0.1821 0.1031
    1.0000 0.1830 0.5334 X4 0.9998 -0.1053
    0.1830 1.0000 0.4638 Y 0.4622 0.7479
    0.5334 0.4638 1.0000

26
  • Table 10-4. Models Illustrating
    Multicollinearity Problems
  • X1
    X2 X3 X4 R2
    F-value Syx t-values under each
    coefficient) (significance)
    M1
    9.16 6.06 76.51
    162.2 2166.5 (14.3)
    (9.4) (.0000) M2 14.82
    9.96 4.84 98.51
    2188.9 544.9 (37.9) (61.26)
    (29.33) (.0000) M3
    9.95 4.83 14.83 98.53 2217.0
    541.4 (61.61) (29.47)
    (38.17) (.0000) M4 -7.07 9.94
    4.83 21.89 98.52 1648.3 543.8
    (-.38) (61.18) (29.30) (1.17)
    (.0000)

27
Partial F-test for Including Variables
  • DETERMINING IF VARS. SHOULD BE IN A RELATIONSHIP
    TEST WHETHER M-VARIABLES SHOULD BE INCLUDED
    (SSER - SSEU)/m Fcalculated
    (10-7)
    SSEU/(n-k-1)

28
  • where SSEU Sum of Squared Errors with all
    variables
  • in the relationship, called the
  • unrestricted SSE SSER Sum of Sqed
    Errors with m vars.
  • excluded, called the restricted SSE
    k-1 Total no. of unrestricted indep.variables
    m Number of restricted independent variables
    a Chosen level of sign., typically .01 or .05

29
  • This test is used as follows
  • Estimate a full, unrestricted k-var.
    modelCapture the SSEU,
  • Estimate a partial, restricted model, k-m
    var. Capture the SSER.
  • Calculate F using eq. 10-6 Compare to F-table
    with df of (i.e., m, n-K-1) and alpha value,
    that is Fm,n-k-1,a.
  • If F-cal gtF-table, then SSER is significantly
    greater than the SSEU.

30
  • Denotes that
  • unexpl. Var. Res. gt unexpl. Var. Unres.
  • If F-cal lt F-table then SSER SSEu thus no
    significant additional explained variance from
    unrestricted model.
  • Again, If F-cal. gt F-table then SSER gt SSEu ,
    there is additional explained variance from the
    unrestricted model.
  • Consider Big City Bookstore W and W/O Competition

31
  • (SSER - SSEU) / m F-cal
    SSEU / (n-K-1)
    (253.9 - 79.8)/1
    19.64 (10-7a)
    79.8/(12-2-1)
  • F-cal 19.64 gtF-table Fm,n-K-1,a
    F1,9,a.055.12
  • F-cal 19.64 gt F-table F1,9,a .01 10.56
    We infer include COMP. This is a powerful test

32
SERIAL CORRELATION PROBLEMS
  • An Assumption of OLS - residuals are independent.
    That is, ACF(k) 0 for all k gt 0
  • When et have ACF(k) 0 then there may be a
    deficiency in model/estimationConsider Table
    10-2a), b), and c).
  • Serial Correlation denotes the following may be
    incorrectR2, Syx, Sb, b-t-values

33
  • First order serial correlation denotes
    Yt a bXt ret-1 et
    (10-8)where r is rho, the first-order
    coefficient.In ARIMA terms r is actually q1

34
How to estimate r?
  • One of Several Iterative Processes-Including
    Cochrane-Orcutt Iterative Least Squares
    (COILS), Hildreth-Lu method, and Prais-Winston
    methods. We Illustrate the COILS Method
    COILS Given Yt a bXt
    ret-1 et (10-9)

35
  • Therefore from et-1 Yt-1 -Y t-1 ret-1
    r(Yt-1-Yt-1) r(Yt-1-(abXt-1
    ret-2et-1)) (10-10)
  • substituting equation 10-10 into 10-9 yields
    Yt abXtr(Yt-1-(a bXt-1ret-2 et-1))et
  • expanding and combining a's into a new term
  • Yt a bXt rYt-1 - ra rbXt et
    Yt - rYt-1 a bXt - rbXt-1 et (10-11)
  • reintroducing backshift operator
    (1-B)Yt (Yt-Yt-1)

36
  • and therefore (1- rB)Yt
    (Yt- rYt-1)therefore equation 10-11 can be
    simplified to (1- rB)Yt a b(1- rB)Xt
    et (10-12)
  • This is estimated iteratively by trial and error
    using different values of r , called the
    Cochrane-Orcutt Iterative Least Squares (COILS)
    procedure.

37
  • COILS can be used with OLS Software
  • Run OLS to determine first r (i.e., ACF(1) of et
    ).
  • Using r transform Yt and Xt to Yt and Xt
    Yt Yt - rYt-1 (1- rB) Yt Xt Xt - rXt-1
    (1- rB) Xt
  • Save these new variables for use in Yt
    a b Xt et (10-13)(We lose one
    observation in backshifting. The
    Prais-Winston method does not.)

38
  • Estimate a and b using OLS in Eq. 10-11
  • Iteratively Search for r with MIN(SSE)
  • Using this r, use coef. of eq. 10-13 in eq. 10-8
    However, remember that the a
    a (1- r)aFigures 10-5 and 10-6
    illustrate Xt and Yt

39
  • Table 10-5. OLS Between Y and X,
    AR1DAT.DAT Usable
    Observations 100 Degrees of Freedom 98R
    Bar2
    0.5924Std Error of Dependent Variable
    2.898Standard Error of Estimate
    1.850Sum of Squared Residuals
    335.476Regression F(1,98)
    144.895Significance Level of F
    0.00000000Durbin-Watson Statistic
    0.905Q(25)
    61.009Significance
    Level of Q 0.00008
    Variable Coeff Std Error T-Stat
    Signif
    1. Constant 79.894 9.899 8.071
    0.000000002. X 0.681 0.057
    12.037 0.00000000

40
  • Table 10-6. ACFs of et for OLS of Table
    10-5. 1 0.547 0.239
    0.242 0.147 0.084 0.049 7 0.005 -0.003
    -0.025 -0.141 -0.103 0.001
    2Approx. 2SeACF
    .20 100
  • Using the ACF(1) of .547 yields Yt
    yt - .55Yt-1 Xt xt -
    .55Yt-1Regressing these two variables yields
    Table 10-7.

41
  • Table 10-7. Y f(X) for r.55
    Dependent Variable
    Y-Estimation by Least SquaresUsable Obs. 99
    Degs. of F. 97R Bar2
    0.456 Std Error of
    Dependent Variable 2.0089 Standard
    Error of Estimate 1.4823
    Sum of Squared Residuals
    213.145 Regression F(1,97)
    82.99 Significance Level of F
    0.00000000Durbin-Watson Statistic
    1.637 Q(24)
    22.958
    Significance Level of Q
    0.5223 Variable Coeff Std Error
    T-Stat Signif
    1. Constant 49.731
    4.3745 11.368 0.000000002. X
    0.506 0.0555 9.110 0.00000000

42
  • Now, let's try r.45 and r.65
    Yt yt - .45Yt-1 and Xt
    xt - .45Yt-1Table 10-8. Y f(X) for
    r.45
  • Dependent Variable Y-Estimation by Least
    SquaresUsable Obs 99 Degrees of
    Freedom 97R Bar2
    0.483 Standard Error of Estimate
    1.519 Sum of Squared Residuals
    223.879 Regression F(1,97)
    92.44
    Significance Level of F
    0.00000000Durbin-Watson Statistic
    1.478 Q(24)
    26.128 Significance
    Level of Q 0.34668
    Variable Coeff Std Error T-Stat
    Signif
    1. Constant 57.403 5.4164 10.598
    0.000000002. X 0.541 0.0563
    9.615 0.00000000 This
    is worse than r .55

43
  • Consider the r in the opposite direction, r
    .65 Yt yt - .65Yt-1
    and Xt xt -
    .65Yt-1Table 10-9. Y f(X) for r.65
  • Dependent Variable Y-Estimation by Least
    SquaresUsable Obs 99 Degrees of
    Freedom 97R Bar2
    0.432 Standard Error of Estimate
    1.465 Sum of Squared Residuals
    208.201 Regression F(1,97)
    75.52
    Significance Level of F
    0.00000000Durbin-Watson Statistic
    1.795 Q(24)
    23.328 Significance
    Level of Q 0.5005
    Variable Coeff Std Error T-Stat
    Signif
    1. Constant 40.549 3.353 12.09
    0.000000002. X 0.476 0.055 8.69
    0.00000000

44
  • Table 10-10. Iterations of r to Minimum SSE.
    r SSE
    D-W Statistic.00 535.5
    .9053.45 223.88
    1.478 .55 213.145
    1.637.65 208.20
    1.795.75 209.79
    1.9331.85 218.626
    2.033.95 235.22
    2.082 denotes optimal
    value of r in manual search.

45
Forecasting With Serially Correlated Errors
  • Yt a bXt r et-1 et Yt
    40.55/(1- .65) .476Xt .65et-1 et
    Yt 115.85 .476Xt .65et-1 et
    (10-14)
  • Yt made at the end of period t-1 Yt 115.85
    .476Xt .65et-1 Yt1 made at the
    end of period t-1 Yt1115.85 .476Xt1
    .65(0) (10-15)where et-1 is unknown in
    period t1.
  • Cochrane-Orcutt Iterative Least Squares (COILS)

46
  • Table 10-11. Y f (X) COILS
    Usable Obs 99
    Degrees of Freedom 96R Bar2
    0.744 Std Error of
    Dependent Variable 2.910 Standard
    Error of Estimate 1.472 Sum
    of Squared Residuals 207.954
    Durbin-Watson Statistic 1.835
    Q(24)
    24.037 Significance Level of Q
    0.4018 Variable Coeff Std Error
    T-Stat Signif
    1. Constant 117.105 9.592
    12.208 0.000000002. Xt 0.468 0.055
    8.550 0.00000000
    3. RHO(r) 0.677 0.077
    8.815 0.00000000

47
  • Because r is so high, only a fraction of the
    explained variance is attributed to Xt.
  • This R2 and RSE are indicative of one-period
    forecast. After one period, the influence of r
    declines to zero,The RSE (Standard Error of
    Estimate) for Yt1 for Kgt1

48
REVIEWING COILS
  • OLS Yt 79.894 .681Xt et
    RSE 1.850 DW0.905
  • Correct Coefficients from COILS
    Yt 117.105 .468Xt et RSE
    1.472 DW1.835Figures 10-7 and 10-8 Here

49
  • Table 10-12. Y f(X) by OLS, ARDAT.DAT
    Usable Obs 100 Degrees of
    Freedom 98R Bar2
    0.368 Std Error of Dependent Variable
    1.644 Standard Error of Estimate
    1.306 Sum of Squared
    Residuals 167.243
    Regression F(1,98)
    58.70 Significance Level of F
    0.00000000Durbin-Watson Statistic
    0.211 Q(25)
    310.561 Significance
    Level of Q 0.00000000 Variable
    Coeff Std Error T-Stat
    Signif
    1. Constant 192.602 12.077 15.948
    0.000000002. Xt -0.907 0.118
    -7.661 0.00000000

50
  • Table 10-13. Yf(X)-Estimation by
    COILS
  • Usable Obs 99 Degrees of Freedom
    96R Bar2
    0.910Std Error of Dependent Variable
    1.652 Standard Error of Estimate
    0.497 Sum of Squared Residuals
    23.6708 Durbin-Watson Statistic
    2.012 Q(24)
    25.823
    Significance Level of Q
    0.30927 Variable Coeff Std Error
    T-Stat Signif
    1. Constant 114.298 12.331
    9.269 0.000000002. Xt -0.136
    0.120 -1.133 0.25960000
    3.
    RHO(r) 0.952 0.032 29.582
    0.00000000

51
  • This was generated using a random number
    generator X0 100 Y0 100Xt
    Xt-1 (1.5 - (RAN1 RAN2 RAN3))Yt Yt-1
    (1.5 - (RAN4 RAN5 RAN6))ANALYSIS OF STOCK
    INDEXES USING COILS
About PowerShow.com