# MULTIPLE REGRESSION OF TIME SERIES - PowerPoint PPT Presentation

PPT – MULTIPLE REGRESSION OF TIME SERIES PowerPoint presentation | free to view - id: eea6d-NzA3Z

The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
Title:

## MULTIPLE REGRESSION OF TIME SERIES

Description:

### 12 ADVERT 2.28 0.198 11.500 0.0000. b) Linear Regression Sales=f(Competition)-Eq.10-3 ... 12 ADVERT 2.5248 0.1295 19.495 0.0000. 13 COMP -0.5449 0.1230 -4.432 0.0016 ... – PowerPoint PPT presentation

Number of Views:162
Avg rating:3.0/5.0
Slides: 52
Provided by: mong2
Category:
Tags:
Transcript and Presenter's Notes

Title: MULTIPLE REGRESSION OF TIME SERIES

1
CHAPTER 10
• MULTIPLE REGRESSION OF TIME SERIES
• LINEAR MULTIPLE REGRESSION MODEL
• General Multiple Regression Model

2
BIG CITY BOOKSTORE EXAMPLE
• Multiple Regression Model
• Coefficient of Multiple Determination-R2
• Partial Regression Coefficients
• Describing the Regression Plane

3
THE MULTIPLE REGRESSION MODELING PROCESS
• MULTICOLLINEARITY
• Collinearity Among Variables
• Solutions Multicoll. Problems
• An Example Solution
• PARTIAL F-TEST FOR INCLUDING VARIABLES SERIAL
CORRELATION PROBLEMS Forecasting with Serially
Correlated Errors

4
ANALYSIS OF STOCK INDEXES USING COILS
• ELASTICITIES AND LOGARITHMIC RELATIONSHIPS
• HETEROSCEDASTICITY
• Wrong Functional Form-UK and US Stock Indexes
Goldfeld-Quandt Test

5
Interpretation of Elasticities
• WEIGHTED LEAST SQUARES
• GENERALIZED LEAST SQUARES
• BETA COEFFICIENTS
• DICHOTOMOUS (DUMMY) VARS. FOR MODELING EVENTS
• CONSTRUCTING CONFIDENCE AND PREDICTION INTERVALS
• PARSIMONY AND REGRESSION ANALYSIS
• AUTOMATED REGRESSION METHODS

6
CHAPTER 10 MULTIPLE REGRESSION OF TIME SERIES
• "Theorize longer, analyze shorter. Don't be in
a rush to run the program. Think about the model
from every angle, hypothesize how different
variables affect each other. When you have a
theory, then try it. Impatience is the enemy of
valid models. Contemplation is productive work."
The Author"Measure twice, cut once." The
Carpenter's Rule

7
GENERAL LINEAR MULTIPLE REGRESSION
• General Multiple-Regression Model Y a
b1X1 b2X2...bnXn e (10-1)"n" is
rarely above 6 to 7.

8
• BIG CITY BOOKSTORE EXAMPLE
• Table 10-1. Big City Bookstore
(X2)
• (1000) (1000) (1000sq.ft.) 1
27 20 10 2
23 20 15 3
31 25 15 4
45 28 15 5
47 29 20 6 42
28 25 7 39
31 35 8 45
34 35 9 57
35 20 10 59
36 30 11 73
41 20 12 84 45
20

9
Multiple Regression Model
• Correlation Matrix Sales
1 .426 Competition
.221 .426 1
• SALES f(ADV.) IGNORING COMPETITION y
-23.02 2.280X1 (10-2)
Syx 5.039 R2 .923 n12
F132.26 DW 1.13676

10
• y 37.34 .477X2 (10-3)
(2.339) (.687) (X2competition)
Syx 18.574 R2 -.045 n 12 F
.507 DW .3767
• SALES f(ADV. , COMP.) SIMULTANEOUSLY
• y -18.80 2.525X1 - .545X2 (10-4)
(-4.879) (19.50) (-4.432) Syx 2.978
R2 .973 n 12 F 199.21 DW
1.7705

11
• Table 10-2. Simple and Multiple Regression for
Big City Bookstorea) Linear
1 Dependent Variable SALES 2 Usable
Observations 12 Degs of Freedom 10 3 R
Bar2
0.9227 4 Std Error of Dependent Variable
18.1225 5 Standard Error of Estimate
5.0394 6 Sum of Squared Residuals
253.9530 7 Regression F(1,10)
132.26 8 Significance Level
of F 0.00000044 9
Durbin-Watson Statistic
1.13710 Variable Coeff Std Error T-Stat
Signif
11 Constant -23.02 6.316 -3.644
11.500 0.0000

12
• b) Linear Regression Salesf(Competition)-Eq.10-3
1 Dependent Variable
SALES 2 Usable Observations 12 Degs of Freedom
10 3 R Bar2
-0.050 4 Std Deviation of Dependent
Variable 18.123 5 Standard Error of
Estimate 18.574 6 Sum of
Squared Residuals 3449.780 7
Regression F(1,10) 0.472
8 Significance Level of F 0.5076
9 Durbin-Watson Statistic
0.37710 Variable Coeff Std Error T-Stat
Signif
11 Constant 37.3372 15.960 2.339
0.041412 COMP 0.4767 0.694
0.687 0.5076

13
Comp.)-Eq.10-4 1 Dependent
Variable SALES 2 Usable Observations 12 Degs
of Freedom 9 3 R Bar2
0.9730 4 Std Deviation of Dependent
Variable 18.1225 5 Standard Error of Estimate
2.978 6 Sum of Squared
Residuals 79.803 7 Regression
F(2,9) 199.2155 8
Significance Level of F
0.00000004 9 Durbin-Watson Statistic
1.77110 Variable Coeff
Std Error T-Stat Signif
11 Constant -18.7958
0.1295 19.495 0.000013 COMP
-0.5449 0.1230 -4.432 0.0016

14
• Multiple Coefficient of Determination - R2
Expl Variance Unexp Var.
Syx2R2 1 - 1-
Total Variance Total Var. Sy2
• Partial Regression Coefficients Y -18.80
2.52530 - .545X2 e Y
-18.80 75.75 - .545X2 e 56.86 -
.545X2 e (10-5)

15
• Figure 10-2 Here-Deviations About a Plane or
HyperspaceDescribing The Regression
PlaneFigure 10-3 Here Regression Plane for
Equation 10-3.Figure 10-4 Here Several Reg
Lines on the Reg PlaneFigure 10-5 The Multiple
Regression Modeling Process

16
MULTIPLE REGRESSION MODELING-PLOTS
• Res. VS Included Indep Vars. Detect
heteroscedasticity, misspecification (Nlin)
• Res. VS Excluded Indep Vars. Detect variable to
be included, misspecification (Nlin)
• Res. VS Y. Detect serial correlation,
heteroscedasticity,misspecifications
• Residualst VS Residualst-1. Detect serial
correlation, out of sample projections,
unreasonable forecasts.

17
MULTICOLLINEARITY
• Highly Related Independent Variables may or may
not be a problem if a problem coefficient be
wrong also its standard error
Syx Sb1
(10-6) ?x2(1 -
r122) With r121, Impossible to fit unique
model because of their redundancy.

18
• Multicollinearity Problems (MCP)
• May not be evident to the analyst
• Yields the wrong sign or insignificant t-values
• Avoid MCPs By
• Good theory
• Large sample sizes
• Good diagnostic procedures
• Sometimes MCP are simply an artifact of the sample

19
DETECTING
• Insignificant/Incorrect regression coefficients
• Some strange regression results from MCP
• Assume correlation between X1 and X2 is high
• Each is highly correlated with Y
• Often one regression coefficient is negative
despite the positive relationship
• Often one variable is highly significant the
other not
• Often the sum of the regression coefficients
equals true, single variable regression
coefficient.

20
COLLINEARITY AMONG MORE THAN TWO VARIABLES
• What is the problem? Consider that some for of
Linear Transformation of X2 and X3 perfectly
defines X1
• X1 a b2X2 b3X2
• with Syx 0, R2 1, and r123 1 Thus, when
an attempt is made to fit the following
relationship, a solution is not possible.

21
• Y a b1X1 b2X2 b3X3 eWhen perfect
collinearity the estimation procedure aborts.
Often with Dichotomous Variable
t Yt d1
d2 d3 d4 1 10 1
0 0 0 2 20 0
1 0 0 3 30
0 0 1 0 4 5
0 0 0 1

22
• Thus, d1 1 - d2 - d3 - d4
• Solution is impossible.
• Avoid-always defining one-less variable
• The last var. is part of constant
• Dummy vars. are studied here later

23
Solutions to Multicollinearity Problems
• With redundant measures delete the redundant
variable. Good theory precludes most redundant
variables.
• Some MCP are an artifact of a specific sample.
Then additional obs. may eliminate the problem.
• MCP from flawed theories. When vars. represent
different dimensions of an influence then they
might be combined using factor analysis

24
• When MCP is caused by a unique sample use ridge
regression.
• When Theory dictates that both variables should
be included, then include them.While MCP
affects regression coefficients and their
interpretability, it might not alter the
predictive power of the regression model. That
is, the overall relationship may still be useful
in predictive power, this being confirmed by a
low standard error of estimate and high F-value.
To better understand this, consider the example
below.

25
• An Example of MCPs (MULT.DAT)
• Table 10-3. Correlation Matrix
X1 X2 X3
X4 Y X1 1.0000 -0.1067
0.1821 0.9998 0.4622 X2 -0.1067 1.0000
0.1031 -0.1053 0.7479 X3 0.1821 0.1031
1.0000 0.1830 0.5334 X4 0.9998 -0.1053
0.1830 1.0000 0.4638 Y 0.4622 0.7479
0.5334 0.4638 1.0000

26
• Table 10-4. Models Illustrating
Multicollinearity Problems
• X1
X2 X3 X4 R2
F-value Syx t-values under each
coefficient) (significance)
M1
9.16 6.06 76.51
162.2 2166.5 (14.3)
(9.4) (.0000) M2 14.82
9.96 4.84 98.51
2188.9 544.9 (37.9) (61.26)
(29.33) (.0000) M3
9.95 4.83 14.83 98.53 2217.0
541.4 (61.61) (29.47)
(38.17) (.0000) M4 -7.07 9.94
4.83 21.89 98.52 1648.3 543.8
(-.38) (61.18) (29.30) (1.17)
(.0000)

27
Partial F-test for Including Variables
• DETERMINING IF VARS. SHOULD BE IN A RELATIONSHIP
TEST WHETHER M-VARIABLES SHOULD BE INCLUDED
(SSER - SSEU)/m Fcalculated
(10-7)
SSEU/(n-k-1)

28
• where SSEU Sum of Squared Errors with all
variables
• in the relationship, called the
• unrestricted SSE SSER Sum of Sqed
Errors with m vars.
• excluded, called the restricted SSE
k-1 Total no. of unrestricted indep.variables
m Number of restricted independent variables
a Chosen level of sign., typically .01 or .05

29
• This test is used as follows
• Estimate a full, unrestricted k-var.
modelCapture the SSEU,
• Estimate a partial, restricted model, k-m
var. Capture the SSER.
• Calculate F using eq. 10-6 Compare to F-table
with df of (i.e., m, n-K-1) and alpha value,
that is Fm,n-k-1,a.
• If F-cal gtF-table, then SSER is significantly
greater than the SSEU.

30
• Denotes that
• unexpl. Var. Res. gt unexpl. Var. Unres.
• If F-cal lt F-table then SSER SSEu thus no
unrestricted model.
• Again, If F-cal. gt F-table then SSER gt SSEu ,
there is additional explained variance from the
unrestricted model.
• Consider Big City Bookstore W and W/O Competition

31
• (SSER - SSEU) / m F-cal
SSEU / (n-K-1)
(253.9 - 79.8)/1
19.64 (10-7a)
79.8/(12-2-1)
• F-cal 19.64 gtF-table Fm,n-K-1,a
F1,9,a.055.12
• F-cal 19.64 gt F-table F1,9,a .01 10.56
We infer include COMP. This is a powerful test

32
SERIAL CORRELATION PROBLEMS
• An Assumption of OLS - residuals are independent.
That is, ACF(k) 0 for all k gt 0
• When et have ACF(k) 0 then there may be a
deficiency in model/estimationConsider Table
10-2a), b), and c).
• Serial Correlation denotes the following may be
incorrectR2, Syx, Sb, b-t-values

33
• First order serial correlation denotes
Yt a bXt ret-1 et
(10-8)where r is rho, the first-order
coefficient.In ARIMA terms r is actually q1

34
How to estimate r?
• One of Several Iterative Processes-Including
Cochrane-Orcutt Iterative Least Squares
(COILS), Hildreth-Lu method, and Prais-Winston
methods. We Illustrate the COILS Method
COILS Given Yt a bXt
ret-1 et (10-9)

35
• Therefore from et-1 Yt-1 -Y t-1 ret-1
r(Yt-1-Yt-1) r(Yt-1-(abXt-1
ret-2et-1)) (10-10)
• substituting equation 10-10 into 10-9 yields
Yt abXtr(Yt-1-(a bXt-1ret-2 et-1))et
• expanding and combining a's into a new term
• Yt a bXt rYt-1 - ra rbXt et
Yt - rYt-1 a bXt - rbXt-1 et (10-11)
• reintroducing backshift operator
(1-B)Yt (Yt-Yt-1)

36
• and therefore (1- rB)Yt
(Yt- rYt-1)therefore equation 10-11 can be
simplified to (1- rB)Yt a b(1- rB)Xt
et (10-12)
• This is estimated iteratively by trial and error
using different values of r , called the
Cochrane-Orcutt Iterative Least Squares (COILS)
procedure.

37
• COILS can be used with OLS Software
• Run OLS to determine first r (i.e., ACF(1) of et
).
• Using r transform Yt and Xt to Yt and Xt
Yt Yt - rYt-1 (1- rB) Yt Xt Xt - rXt-1
(1- rB) Xt
• Save these new variables for use in Yt
a b Xt et (10-13)(We lose one
observation in backshifting. The
Prais-Winston method does not.)

38
• Estimate a and b using OLS in Eq. 10-11
• Iteratively Search for r with MIN(SSE)
• Using this r, use coef. of eq. 10-13 in eq. 10-8
However, remember that the a
a (1- r)aFigures 10-5 and 10-6
illustrate Xt and Yt

39
• Table 10-5. OLS Between Y and X,
AR1DAT.DAT Usable
Observations 100 Degrees of Freedom 98R
Bar2
0.5924Std Error of Dependent Variable
2.898Standard Error of Estimate
1.850Sum of Squared Residuals
335.476Regression F(1,98)
144.895Significance Level of F
0.00000000Durbin-Watson Statistic
0.905Q(25)
61.009Significance
Level of Q 0.00008
Variable Coeff Std Error T-Stat
Signif
1. Constant 79.894 9.899 8.071
0.000000002. X 0.681 0.057
12.037 0.00000000

40
• Table 10-6. ACFs of et for OLS of Table
10-5. 1 0.547 0.239
0.242 0.147 0.084 0.049 7 0.005 -0.003
-0.025 -0.141 -0.103 0.001
2Approx. 2SeACF
.20 100
• Using the ACF(1) of .547 yields Yt
yt - .55Yt-1 Xt xt -
.55Yt-1Regressing these two variables yields
Table 10-7.

41
• Table 10-7. Y f(X) for r.55
Dependent Variable
Y-Estimation by Least SquaresUsable Obs. 99
Degs. of F. 97R Bar2
0.456 Std Error of
Dependent Variable 2.0089 Standard
Error of Estimate 1.4823
Sum of Squared Residuals
213.145 Regression F(1,97)
82.99 Significance Level of F
0.00000000Durbin-Watson Statistic
1.637 Q(24)
22.958
Significance Level of Q
0.5223 Variable Coeff Std Error
T-Stat Signif
1. Constant 49.731
4.3745 11.368 0.000000002. X
0.506 0.0555 9.110 0.00000000

42
• Now, let's try r.45 and r.65
Yt yt - .45Yt-1 and Xt
xt - .45Yt-1Table 10-8. Y f(X) for
r.45
• Dependent Variable Y-Estimation by Least
SquaresUsable Obs 99 Degrees of
Freedom 97R Bar2
0.483 Standard Error of Estimate
1.519 Sum of Squared Residuals
223.879 Regression F(1,97)
92.44
Significance Level of F
0.00000000Durbin-Watson Statistic
1.478 Q(24)
26.128 Significance
Level of Q 0.34668
Variable Coeff Std Error T-Stat
Signif
1. Constant 57.403 5.4164 10.598
0.000000002. X 0.541 0.0563
9.615 0.00000000 This
is worse than r .55

43
• Consider the r in the opposite direction, r
.65 Yt yt - .65Yt-1
and Xt xt -
.65Yt-1Table 10-9. Y f(X) for r.65
• Dependent Variable Y-Estimation by Least
SquaresUsable Obs 99 Degrees of
Freedom 97R Bar2
0.432 Standard Error of Estimate
1.465 Sum of Squared Residuals
208.201 Regression F(1,97)
75.52
Significance Level of F
0.00000000Durbin-Watson Statistic
1.795 Q(24)
23.328 Significance
Level of Q 0.5005
Variable Coeff Std Error T-Stat
Signif
1. Constant 40.549 3.353 12.09
0.000000002. X 0.476 0.055 8.69
0.00000000

44
• Table 10-10. Iterations of r to Minimum SSE.
r SSE
D-W Statistic.00 535.5
.9053.45 223.88
1.478 .55 213.145
1.637.65 208.20
1.795.75 209.79
1.9331.85 218.626
2.033.95 235.22
2.082 denotes optimal
value of r in manual search.

45
Forecasting With Serially Correlated Errors
• Yt a bXt r et-1 et Yt
40.55/(1- .65) .476Xt .65et-1 et
Yt 115.85 .476Xt .65et-1 et
(10-14)
• Yt made at the end of period t-1 Yt 115.85
.476Xt .65et-1 Yt1 made at the
end of period t-1 Yt1115.85 .476Xt1
.65(0) (10-15)where et-1 is unknown in
period t1.
• Cochrane-Orcutt Iterative Least Squares (COILS)

46
• Table 10-11. Y f (X) COILS
Usable Obs 99
Degrees of Freedom 96R Bar2
0.744 Std Error of
Dependent Variable 2.910 Standard
Error of Estimate 1.472 Sum
of Squared Residuals 207.954
Durbin-Watson Statistic 1.835
Q(24)
24.037 Significance Level of Q
0.4018 Variable Coeff Std Error
T-Stat Signif
1. Constant 117.105 9.592
12.208 0.000000002. Xt 0.468 0.055
8.550 0.00000000
3. RHO(r) 0.677 0.077
8.815 0.00000000

47
• Because r is so high, only a fraction of the
explained variance is attributed to Xt.
• This R2 and RSE are indicative of one-period
forecast. After one period, the influence of r
declines to zero,The RSE (Standard Error of
Estimate) for Yt1 for Kgt1

48
REVIEWING COILS
• OLS Yt 79.894 .681Xt et
RSE 1.850 DW0.905
• Correct Coefficients from COILS
Yt 117.105 .468Xt et RSE
1.472 DW1.835Figures 10-7 and 10-8 Here

49
• Table 10-12. Y f(X) by OLS, ARDAT.DAT
Usable Obs 100 Degrees of
Freedom 98R Bar2
0.368 Std Error of Dependent Variable
1.644 Standard Error of Estimate
1.306 Sum of Squared
Residuals 167.243
Regression F(1,98)
58.70 Significance Level of F
0.00000000Durbin-Watson Statistic
0.211 Q(25)
310.561 Significance
Level of Q 0.00000000 Variable
Coeff Std Error T-Stat
Signif
1. Constant 192.602 12.077 15.948
0.000000002. Xt -0.907 0.118
-7.661 0.00000000

50
• Table 10-13. Yf(X)-Estimation by
COILS
• Usable Obs 99 Degrees of Freedom
96R Bar2
0.910Std Error of Dependent Variable
1.652 Standard Error of Estimate
0.497 Sum of Squared Residuals
23.6708 Durbin-Watson Statistic
2.012 Q(24)
25.823
Significance Level of Q
0.30927 Variable Coeff Std Error
T-Stat Signif
1. Constant 114.298 12.331
9.269 0.000000002. Xt -0.136
0.120 -1.133 0.25960000
3.
RHO(r) 0.952 0.032 29.582
0.00000000

51
• This was generated using a random number
generator X0 100 Y0 100Xt
Xt-1 (1.5 - (RAN1 RAN2 RAN3))Yt Yt-1
(1.5 - (RAN4 RAN5 RAN6))ANALYSIS OF STOCK
INDEXES USING COILS