Title: MA and ARCH Time Series model inference using Minimum Message Length
1MA and ARCH Time Series model inference using
Minimum Message Length
- By
- Mony Sak - 13080512
- Supervisors
- Assoc. Prof. David Dowe, Dr Sid Ray
2Contents
- The Problem
- Time Series Concepts
- Minimum Message Length (MML)
- MML applied to Time Series
- My Project
- Results
- Conclusion Future Work
31. The Problem
- Which model fits the data best?
41. The Problem
- Which model fits the data best?
?
51. The Problem
- Which model fits the data best?
?
?
?
?
?
?
62. Time Series Concepts
- What is a Time Series (TS)?
Observation value
time
72. Time Series Concepts
- What is a Time Series (TS)?
- Some examples (1 of 4)
-
- Light Curve of Beta Persei, also known as Algol,
or demon star1
82. Time Series Concepts
- What is a Time Series (TS)?
- Some examples (2 of 4)
-
- Closing stock price of Apple Computer Inc. (AAPL)
(1984-2005)2
92. Time Series Concepts
- What is a Time Series (TS)?
- Some examples (3 of 4)
-
- Global temperature difference vs. Years3
102. Time Series Concepts
- What is a Time Series (TS)?
- Some examples (4 of 4)
-
- Average monthly busridership (weekdays) in Iowa
city (1971-1982)4
112. Time Series Concepts
- Description
- The best method of conveying information
- Explanation
- A good model good understanding of the
underlying process generating that data
- Prediction
- Predict future observation values
- Control
- If we can predict future values, we are able to
control the time series to our benefit
122. Time Series Concepts
- Some TS models (1 of 3)
- Autoregressive, order p AR(p)
- Current observation value is a sum of weighted
past observation values random error5
132. Time Series Concepts
- Some TS models (2 of 3)
- Moving Average, order q MA(q)
- Current observation value is a sum of weighted
past error values random error5
142. Time Series Concepts
- Some TS models (3 of 3)
- Autoregressive Conditional Heteroskedastic, order
q ARCH(q) - Current variance value is a sum of weighted past
squared error values5
152. Time Series Concepts
- 1 set of data and many many models
?
?
?
?
?
?
162. Time Series Concepts
- Partial solution to the The Problem
- The Model Selection Criterion (MSC)
- An equation, based on parsimony
- Objective scoring of different models
101.21
i am a criterion!
99.90
172. Time Series Concepts
- Some popular Model Selection Criteria
- Akaikes Information Criterion (AIC)6
- Bayesian Information Criterion (BIC)7
- many more incl. HQ8, RCL9, MML10
183. Minimum Message Length
- Information-theoretic criterion for model
selection and point estimation
- Developed here at Monash University by Wallace
Boulton in 196811
- Has been applied to mixture modelling (snob),
decision tree/graph induction, generalized
Bayesian networks, and more
193. Minimum Message Length
- A message can be encoded in 2 parts
- Part 1 Model,
- Part 2 Data (given the Model in Part 1)
- Combined Message Length Part 1 Part 2
- We choose the model that yields the smallest
Combined Message Length
203. Minimum Message Length
model 1
datamodel 1
model 2
datamodel 2
model 3
datamodel 3
model 4
datamodel 4
213. Minimum Message Length
- Developed by Wallace Freeman in 198713
224. MML87-based MSC
- Past Research
- MML87-based MSC for
- AR model inference10,
- Stock market simulation of AR traders14
- ARMAX models15
- Results
- MML does very well when compared to the other
Model Selection Criteria
234. MML-based MSC
- Results from Fitzgibbon, Dowe, Vahid (2004)10
- Motivation for my project
- How well does MML-based MSCs perform with other
models?
245. My Project
- Moving Average (MA) models?
- How well does an MML-based MSC perform with
- Autoregressive Conditional Heteroskedastic (ARCH)
models?
- We need to derive 2 MSCs, 1 for each model
- MA is a conditional mean model, whereas ARCH is a
conditional variance model - quite different
- Complex math regarding Fisher Information matrix.
We resort to approximations
255. My Project
- MML87 equation we will be using
266. Results
- Results (simulations)
- Moving Average (MA) models
-
- (Results from Sak, Dowe, Ray (2005). Accepted
for inclusion in proceedings of Advanced
Computing in Financial Markets 05. Istanbul,
Turkey. Dec 15-17, 2005.)16
276. Results (MA simulations)
286. Results (MA simulations)
296. Results (MA simulations)
306. Results (MA simulations)
316. Results (MA simulations)
326. Results (MA simulations)
336. Results (ARCH simulations)
347. Conclusion Future Work
- MML-based MSC for MA models performs very well
- MML-based MSC for ARCH models.
- Try other MML approximations such as MMLD17
- Other Time Series models Generalized ARCH
(GARCH)18, Generalized/Indexed AR (GAR)18
- Other parameter estimation methods Maximum
Likelihood Estimation (MLE) is very very slow!
35Thanks!
36References (1 of 2)
- J. Stebbins. The measurement of the light of
stars with a selenium photometer, with an
application to variations of Algol. The
Astrophysical Journal, 32(3)185-214, 1910. - Data obtained from http//finance.yahoo.com/q?saa
pl - Data obtained from http//www.elmhurst.edu/chm/vc
hembook/globalwarmA.html - Hyndman, R.J. (n.d.) Time Series Data Library,
http//www-personal.buseco.monash.edu.au/hyndman/
TSDL/. Accessed on 24 Oct., 2005. - J. D. Hamilton. Time Series Analysis. Princeton
University Press, 1994. - H. Akaike. Information theory as an extension of
the Maximum Likelihood principle. In Second
International Symposium on Information Theory,
pages 267-281, 1973. Petrov, B.N. and Csaki, F.
(editors). Akademiai Kiado, Budapest. - G. Schwarz. Estimating the dimension of a model.
The Annals of Statistics, 6(2)46-464, 1978. - E.J. Hannan and B.G. Quinn. The determination of
the order of an autoregression. Journal of the
Royal Statistical Society, Series B
(Methodological), 41(2)190-195, 1979. - H. Mitchell and D.M McKenzie. GARCH model
selection criteria. Quantitative Finance,
3262-284, 2003. - L.J. Fitzgibbon, D.L. Dowe, and F. Vahid. Minimum
Message Length Autoregressive Model Order
Selection. In M. Palanaswami, C. Chandra Sekhar,
G. Kumar Venayagamoorthy, S. Mohan and M. K.
Ghantasala (eds.), International Conference on
Intelligent Sensing and Information Processing
(ICISIP), pages 439-444, 2004. Chennai, India,
4-7 January 2004, (ISBN 0-7803-8243-9, IEEE
Catalogue Number 04EX783), www.csse.monash.edu.au
/dld/Publications/2004/FitzgibbonDoweVahid2004.
ref.
Want a copy of these slides? Send requests to
monys_at_csse.monash.edu.au
37References (2 of 2)
- C.S. Wallace and D.M. Boulton. An information
measure for classification. Computer Journal,
11(2)185-194, 1968. - L.J. Fitzgibbon. Message from Monte Carlo A
Framework for Minimum Message Length Inference
using Markov Chain Monte Carlo Methods. PhD
thesis, Monash University, Clayton Campus.
Wellington Rd, Clayton. Victoria 3800, Australia,
2004. - C.S. Wallace and P.R. Freeman. Estimation and
inference by compact encoding. Journal of the
Royal Statistical Society. Series B
(Methodological), 49(3)240-265, 1987. - M. J. Collie, D. L. Dowe, and L. J. Fitzgibbon.
Stock market simulation and inference technique,
2005. Accepted for inclusion in proceedings of
the 5th international conference on Hybrid
Intelligent Systems (HIS05), Rio de Janeiro,
Brazil, November 6-9, 2005. - Schmidt
- M. Sak, D.L. Dowe, and S. Ray. Minimum Message
Length Moving Average Time Series Data Mining. In
Computational Intelligence Methods and
Applications. First International ICSC Symposium
on Advanced Computing in Financial Markets
(ACFM2005), 2005. Accepted for inclusion in
proceedings of Advanced Computing in Financial
Markets (ACFM2005), Istanbul, Turkey. Dec. 15-17,
2005. - E. Lam. Improved Approximations in MML. Honours
Thesis, Monash University, School of Computer
Science and Software Engineering (CSSE), Monash
University, Clayton 3168, Australia, 2000. - T. Bollerslev. Generalized Autoregressive
Conditional Heteroskedasticity. Journal of
Econometrics, 31307-27, 1986. - M.S. Peiris. Improving the Quality of Forecasting
using Generalized AR Models An Application to
Statistical Quality Control. Statistical Methods,
5(2)156-171, 2003.
38Negative Log Likelihood
- Takes into account the estimated variance
396. Results
- Simulate data sets for 200 models for each model
order (i.e. MA(1) - MA(8)) for a total of 1,600
MA data sets
- Estimate model parameters using Maximum
Likelihood (MLE)
- Pass to each Model Selection Criterion (MSC) the
same 1,600 data sets and parameter estimates (for
each data set), and let them choose the model
they think best represents the data
- Assessment is on correct model order selection
accuracy and negative log likelihoood
- Repeat experiment for ARCH models (again 1,600
data sets)