View by Category

Loading...

PPT – Maximum Likelihood ML Estimation PowerPoint presentation | free to download - id: 228359-OGIwM

The Adobe Flash plugin is needed to view this content

About This Presentation

Write a Comment

User Comments (0)

Transcript and Presenter's Notes

Maximum Likelihood(ML) Estimation

The basic idea

Assume a particular model with unknown

parameters. Determine how the likelihood of a

given event varies with model model

parameters Choose the parameter values that

maximize the likelihood of the observed event

A general mathematical formulation

Consider a sample (X1, ..., Xn) which is drawn

from a probability distribution P(X?) where ?

are parameters. If the Xs are independent with

probability density function P(Xi?) then the

joint probability of the whole set is Find

the parameters that maximize this function

- Consider a sample (X1, ..., Xn) which is drawn

from a probability distribution P(XA) where A

are parameters. - If the Xs are independent with probability

density function P(XiA) the joint probability of

the whole set is

The likelihood function for the general

non-linear model

Assume that Then the likelihood function

is Note that the ML-estimator of ? is

identical to the mean square estimator if ?

?2I, where I is the identity matrix.

Large sample properties of ML estimators

Consistency As the sample size increases, the

ML estimator converges to the true parameter

value Invaríance If f(?) is a function of the

unknown parameters of the distribution, then the

ML estimator of f(?) is f( ) Asymptotic

normality As the sampe size increases, the

sampling distribution of an ML estimator

converges to a normal distribution Variance For

large sample sizes, the variance of an ML

estimator (assuming a single unknown parameter)

is approximately the negative of the reciprocal

of the second derivative of the log-likelihood

function evaluated at the ML estimate. Note

that the ML-estimator of ? is identical to the

mean square estimator if ? ?2I, where I is the

identity matrix.

The information matrix (Hessian)

- The matrix
- is a measure of how pointy' the likelihood

function is. - The variance of the ML estimator is given by the

inverse Hessian

The Cramer-Rao lower bound

- The Cramer-Rao lower bound is the smallest

theoretical variance which can be achieved. - ML gives this, so any other estimation technique

can at best only equal it. - Do we need estimators other than ML estimators?

ML estimators for dynamic models

- A general decomposition technique for the log

likelihood function allows us to extend standard

ML procedures to dynamic models (time series

models). - From the basic definition of conditional

probability - This may be applied directly to the likelihood

function

Prediction error decomposition

- Consider the decomposition
- The first term is the conditional probability of

Y given all past values. We can then condition

the second term and so on to give - that is, a series of one step ahead prediction

errors conditional on actual lagged Y.

Numerical optimization

- In simple cases (e.g. OLS) we can calculate the

maximum likelihood estimates analytically. - But in many cases we cannot, then we resort to

numerical optimisation of the likelihood

function. - This amounts to hill climbing in parameter space.

- set an arbitrary initial set of parameters.
- determine a direction of movement
- determine a step length to move
- examine some termination criteria and either stop

or go back to 2

L

Lu

Gradient methods for determining the maximum of a

function

- These methods base the direction of movement on

the first derivatives of the likelihood function

with respect to the parameters. - Often the step length is also determined by (an

approximation to) the second derivatives. So - The class of gradient methods include Newton,

Quasi Newton, Steepest descent etc.

Qualitative response models

- Assume that we have a quantitative model
- but we only observe certain limited information,

e.g. - Then we can group the data into two groups and

form a likelihood function with the following

form - where F is the cumulative distribution function

of the error terms ut

About PowerShow.com

PowerShow.com is a leading presentation/slideshow sharing website. Whether your application is business, how-to, education, medicine, school, church, sales, marketing, online training or just for fun, PowerShow.com is a great resource. And, best of all, most of its cool features are free and easy to use.

You can use PowerShow.com to find and download example online PowerPoint ppt presentations on just about any topic you can imagine so you can learn how to improve your own slides and presentations for free. Or use it to find and download high-quality how-to PowerPoint ppt presentations with illustrated or animated slides that will teach you how to do something new, also for free. Or use it to upload your own PowerPoint slides so you can share them with your teachers, class, students, bosses, employees, customers, potential investors or the world. Or use it to create really cool photo slideshows - with 2D and 3D transitions, animation, and your choice of music - that you can share with your Facebook friends or Google+ circles. That's all free as well!

For a small fee you can get the industry's best online privacy or publicly promote your presentations and slide shows with top rankings. But aside from that it's free. We'll even convert your presentations and slide shows into the universal Flash format with all their original multimedia glory, including animation, 2D and 3D transition effects, embedded music or other audio, or even video embedded in slides. All for free. Most of the presentations and slideshows on PowerShow.com are free to view, many are even free to download. (You can choose whether to allow people to download your original PowerPoint presentations and photo slideshows for a fee or free or not at all.) Check out PowerShow.com today - for FREE. There is truly something for everyone!

You can use PowerShow.com to find and download example online PowerPoint ppt presentations on just about any topic you can imagine so you can learn how to improve your own slides and presentations for free. Or use it to find and download high-quality how-to PowerPoint ppt presentations with illustrated or animated slides that will teach you how to do something new, also for free. Or use it to upload your own PowerPoint slides so you can share them with your teachers, class, students, bosses, employees, customers, potential investors or the world. Or use it to create really cool photo slideshows - with 2D and 3D transitions, animation, and your choice of music - that you can share with your Facebook friends or Google+ circles. That's all free as well!

For a small fee you can get the industry's best online privacy or publicly promote your presentations and slide shows with top rankings. But aside from that it's free. We'll even convert your presentations and slide shows into the universal Flash format with all their original multimedia glory, including animation, 2D and 3D transition effects, embedded music or other audio, or even video embedded in slides. All for free. Most of the presentations and slideshows on PowerShow.com are free to view, many are even free to download. (You can choose whether to allow people to download your original PowerPoint presentations and photo slideshows for a fee or free or not at all.) Check out PowerShow.com today - for FREE. There is truly something for everyone!

presentations for free. Or use it to find and download high-quality how-to PowerPoint ppt presentations with illustrated or animated slides that will teach you how to do something new, also for free. Or use it to upload your own PowerPoint slides so you can share them with your teachers, class, students, bosses, employees, customers, potential investors or the world. Or use it to create really cool photo slideshows - with 2D and 3D transitions, animation, and your choice of music - that you can share with your Facebook friends or Google+ circles. That's all free as well!

For a small fee you can get the industry's best online privacy or publicly promote your presentations and slide shows with top rankings. But aside from that it's free. We'll even convert your presentations and slide shows into the universal Flash format with all their original multimedia glory, including animation, 2D and 3D transition effects, embedded music or other audio, or even video embedded in slides. All for free. Most of the presentations and slideshows on PowerShow.com are free to view, many are even free to download. (You can choose whether to allow people to download your original PowerPoint presentations and photo slideshows for a fee or free or not at all.) Check out PowerShow.com today - for FREE. There is truly something for everyone!

For a small fee you can get the industry's best online privacy or publicly promote your presentations and slide shows with top rankings. But aside from that it's free. We'll even convert your presentations and slide shows into the universal Flash format with all their original multimedia glory, including animation, 2D and 3D transition effects, embedded music or other audio, or even video embedded in slides. All for free. Most of the presentations and slideshows on PowerShow.com are free to view, many are even free to download. (You can choose whether to allow people to download your original PowerPoint presentations and photo slideshows for a fee or free or not at all.) Check out PowerShow.com today - for FREE. There is truly something for everyone!

Recommended

«

/ »

Page of

«

/ »

Promoted Presentations

Related Presentations

Page of

Home About Us Terms and Conditions Privacy Policy Contact Us Send Us Feedback

Copyright 2017 CrystalGraphics, Inc. — All rights Reserved. PowerShow.com is a trademark of CrystalGraphics, Inc.

Copyright 2017 CrystalGraphics, Inc. — All rights Reserved. PowerShow.com is a trademark of CrystalGraphics, Inc.

The PowerPoint PPT presentation: "Maximum Likelihood ML Estimation" is the property of its rightful owner.

Do you have PowerPoint slides to share? If so, share your PPT presentation slides online with PowerShow.com. It's FREE!