MA and ARCH Time Series model inference using Minimum Message Length - PowerPoint PPT Presentation

1 / 39
About This Presentation
Title:

MA and ARCH Time Series model inference using Minimum Message Length

Description:

MA and ARCH Time Series model inference using Minimum Message Length ... Other Time Series models: Generalized ARCH (GARCH)18, Generalized/Indexed AR (GAR)18 ... – PowerPoint PPT presentation

Number of Views:109
Avg rating:3.0/5.0
Slides: 40
Provided by: rei9
Category:

less

Transcript and Presenter's Notes

Title: MA and ARCH Time Series model inference using Minimum Message Length


1
MA and ARCH Time Series model inference using
Minimum Message Length
  • By
  • Mony Sak - 13080512
  • Supervisors
  • Assoc. Prof. David Dowe, Dr Sid Ray

2
Contents
  • The Problem
  • Time Series Concepts
  • Minimum Message Length (MML)
  • MML applied to Time Series
  • My Project
  • Results
  • Conclusion Future Work

3
1. The Problem
  • Which model fits the data best?

4
1. The Problem
  • Which model fits the data best?

?
5
1. The Problem
  • Which model fits the data best?

?
?
?
?
?
?
6
2. Time Series Concepts
  • What is a Time Series (TS)?
  • Observations over time

Observation value
time
7
2. Time Series Concepts
  • What is a Time Series (TS)?
  • Some examples (1 of 4)
  • Light Curve of Beta Persei, also known as Algol,
    or demon star1

8
2. Time Series Concepts
  • What is a Time Series (TS)?
  • Some examples (2 of 4)
  • Closing stock price of Apple Computer Inc. (AAPL)
    (1984-2005)2

9
2. Time Series Concepts
  • What is a Time Series (TS)?
  • Some examples (3 of 4)
  • Global temperature difference vs. Years3

10
2. Time Series Concepts
  • What is a Time Series (TS)?
  • Some examples (4 of 4)
  • Average monthly busridership (weekdays) in Iowa
    city (1971-1982)4

11
2. Time Series Concepts
  • Why study Time Series?
  • Description
  • The best method of conveying information
  • Explanation
  • A good model good understanding of the
    underlying process generating that data
  • Prediction
  • Predict future observation values
  • Control
  • If we can predict future values, we are able to
    control the time series to our benefit

12
2. Time Series Concepts
  • Some TS models (1 of 3)
  • Autoregressive, order p AR(p)
  • Current observation value is a sum of weighted
    past observation values random error5

13
2. Time Series Concepts
  • Some TS models (2 of 3)
  • Moving Average, order q MA(q)
  • Current observation value is a sum of weighted
    past error values random error5

14
2. Time Series Concepts
  • Some TS models (3 of 3)
  • Autoregressive Conditional Heteroskedastic, order
    q ARCH(q)
  • Current variance value is a sum of weighted past
    squared error values5

15
2. Time Series Concepts
  • 1 set of data and many many models

?
?
?
?
?
?
16
2. Time Series Concepts
  • Partial solution to the The Problem
  • The Model Selection Criterion (MSC)
  • An equation, based on parsimony
  • Objective scoring of different models

101.21

i am a criterion!
99.90

17
2. Time Series Concepts
  • Some popular Model Selection Criteria
  • Akaikes Information Criterion (AIC)6
  • Bayesian Information Criterion (BIC)7
  • many more incl. HQ8, RCL9, MML10

18
3. Minimum Message Length
  • What it is History
  • Information-theoretic criterion for model
    selection and point estimation
  • Developed here at Monash University by Wallace
    Boulton in 196811
  • Has been applied to mixture modelling (snob),
    decision tree/graph induction, generalized
    Bayesian networks, and more

19
3. Minimum Message Length
  • Theory
  • A message can be encoded in 2 parts
  • Part 1 Model,
  • Part 2 Data (given the Model in Part 1)
  • Combined Message Length Part 1 Part 2
  • We choose the model that yields the smallest
    Combined Message Length

20
3. Minimum Message Length
  • Theory (example)

model 1
datamodel 1
model 2
datamodel 2
model 3
datamodel 3
model 4
datamodel 4
21
3. Minimum Message Length
  • MML87 Approximation
  • Developed by Wallace Freeman in 198713
  • Part 1 (model)
  • Part 2 (datamodel)

22
4. MML87-based MSC
  • Past Research
  • MML87-based MSC for
  • AR model inference10,
  • Stock market simulation of AR traders14
  • ARMAX models15
  • Results
  • MML does very well when compared to the other
    Model Selection Criteria

23
4. MML-based MSC
  • Results from Fitzgibbon, Dowe, Vahid (2004)10
  • Motivation for my project
  • How well does MML-based MSCs perform with other
    models?

24
5. My Project
  • Moving Average (MA) models?
  • How well does an MML-based MSC perform with
  • Autoregressive Conditional Heteroskedastic (ARCH)
    models?
  • We need to derive 2 MSCs, 1 for each model
  • MA is a conditional mean model, whereas ARCH is a
    conditional variance model - quite different
  • Complex math regarding Fisher Information matrix.
    We resort to approximations

25
5. My Project
  • MML87 equation we will be using

26
6. Results
  • Results (simulations)
  • Moving Average (MA) models
  • (Results from Sak, Dowe, Ray (2005). Accepted
    for inclusion in proceedings of Advanced
    Computing in Financial Markets 05. Istanbul,
    Turkey. Dec 15-17, 2005.)16

27
6. Results (MA simulations)
28
6. Results (MA simulations)
29
6. Results (MA simulations)
30
6. Results (MA simulations)
31
6. Results (MA simulations)
32
6. Results (MA simulations)
33
6. Results (ARCH simulations)
34
7. Conclusion Future Work
  • Conclusion
  • MML-based MSC for MA models performs very well
  • MML-based MSC for ARCH models.
  • Future Work
  • Try other MML approximations such as MMLD17
  • Other Time Series models Generalized ARCH
    (GARCH)18, Generalized/Indexed AR (GAR)18
  • Other parameter estimation methods Maximum
    Likelihood Estimation (MLE) is very very slow!

35
Thanks!
36
References (1 of 2)
  • J. Stebbins. The measurement of the light of
    stars with a selenium photometer, with an
    application to variations of Algol. The
    Astrophysical Journal, 32(3)185-214, 1910.
  • Data obtained from http//finance.yahoo.com/q?saa
    pl
  • Data obtained from http//www.elmhurst.edu/chm/vc
    hembook/globalwarmA.html
  • Hyndman, R.J. (n.d.) Time Series Data Library,
    http//www-personal.buseco.monash.edu.au/hyndman/
    TSDL/. Accessed on 24 Oct., 2005.
  • J. D. Hamilton. Time Series Analysis. Princeton
    University Press, 1994.
  • H. Akaike. Information theory as an extension of
    the Maximum Likelihood principle. In Second
    International Symposium on Information Theory,
    pages 267-281, 1973. Petrov, B.N. and Csaki, F.
    (editors). Akademiai Kiado, Budapest.
  • G. Schwarz. Estimating the dimension of a model.
    The Annals of Statistics, 6(2)46-464, 1978.
  • E.J. Hannan and B.G. Quinn. The determination of
    the order of an autoregression. Journal of the
    Royal Statistical Society, Series B
    (Methodological), 41(2)190-195, 1979.
  • H. Mitchell and D.M McKenzie. GARCH model
    selection criteria. Quantitative Finance,
    3262-284, 2003.
  • L.J. Fitzgibbon, D.L. Dowe, and F. Vahid. Minimum
    Message Length Autoregressive Model Order
    Selection. In M. Palanaswami, C. Chandra Sekhar,
    G. Kumar Venayagamoorthy, S. Mohan and M. K.
    Ghantasala (eds.), International Conference on
    Intelligent Sensing and Information Processing
    (ICISIP), pages 439-444, 2004. Chennai, India,
    4-7 January 2004, (ISBN 0-7803-8243-9, IEEE
    Catalogue Number 04EX783), www.csse.monash.edu.au
    /dld/Publications/2004/FitzgibbonDoweVahid2004.
    ref.

Want a copy of these slides? Send requests to
monys_at_csse.monash.edu.au
37
References (2 of 2)
  • C.S. Wallace and D.M. Boulton. An information
    measure for classification. Computer Journal,
    11(2)185-194, 1968.
  • L.J. Fitzgibbon. Message from Monte Carlo A
    Framework for Minimum Message Length Inference
    using Markov Chain Monte Carlo Methods. PhD
    thesis, Monash University, Clayton Campus.
    Wellington Rd, Clayton. Victoria 3800, Australia,
    2004.
  • C.S. Wallace and P.R. Freeman. Estimation and
    inference by compact encoding. Journal of the
    Royal Statistical Society. Series B
    (Methodological), 49(3)240-265, 1987.
  • M. J. Collie, D. L. Dowe, and L. J. Fitzgibbon.
    Stock market simulation and inference technique,
    2005. Accepted for inclusion in proceedings of
    the 5th international conference on Hybrid
    Intelligent Systems (HIS05), Rio de Janeiro,
    Brazil, November 6-9, 2005.
  • Schmidt
  • M. Sak, D.L. Dowe, and S. Ray. Minimum Message
    Length Moving Average Time Series Data Mining. In
    Computational Intelligence Methods and
    Applications. First International ICSC Symposium
    on Advanced Computing in Financial Markets
    (ACFM2005), 2005. Accepted for inclusion in
    proceedings of Advanced Computing in Financial
    Markets (ACFM2005), Istanbul, Turkey. Dec. 15-17,
    2005.
  • E. Lam. Improved Approximations in MML. Honours
    Thesis, Monash University, School of Computer
    Science and Software Engineering (CSSE), Monash
    University, Clayton 3168, Australia, 2000.
  • T. Bollerslev. Generalized Autoregressive
    Conditional Heteroskedasticity. Journal of
    Econometrics, 31307-27, 1986.
  • M.S. Peiris. Improving the Quality of Forecasting
    using Generalized AR Models An Application to
    Statistical Quality Control. Statistical Methods,
    5(2)156-171, 2003.

38
Negative Log Likelihood
  • Takes into account the estimated variance

39
6. Results
  • Empirical Comparison
  • Simulate data sets for 200 models for each model
    order (i.e. MA(1) - MA(8)) for a total of 1,600
    MA data sets
  • Estimate model parameters using Maximum
    Likelihood (MLE)
  • Pass to each Model Selection Criterion (MSC) the
    same 1,600 data sets and parameter estimates (for
    each data set), and let them choose the model
    they think best represents the data
  • Assessment is on correct model order selection
    accuracy and negative log likelihoood
  • Repeat experiment for ARCH models (again 1,600
    data sets)
Write a Comment
User Comments (0)
About PowerShow.com