K. Kinoshita

1
evaluating quality of fit inUnbinned Maximum
Likelihood Fitting
Statistical distribution of l?- zero free
parameters impact of free parameters some
speculations

• K. Kinoshita
• University of Cincinnati
• Belle Collaboration

2
Motivation
Unbinned Maximum Likelihood (UMxL) fitting
preferred for determining parameter(s) a via
parameter- dependent shape f(x a) of
distribution in measured x maximizes use of x
information, esp. w. limited statistics used in
many current analyses - CP, lifetime, Dalitz
plot, Goodness-of-fit to answer - are data
statistically consistent with fitted shape
(not easily visualized within binless context,
esp. in multiple dim.)? - is f(x a) a valid
parametrization? Straightforward for
least-squares To date, no good test in UMxL -
why not?
3
Unbinned Maximum Likelihood (UMxL) fit
Brief outline have experiment w N measurements
of x - Maximize under variations in a (f(x
a)normalized PDF) Equivalent to
maximizing Max. at Wish to examine fit
quality - questions How are ????????distributed
in ensemble, if root is ? 0 free
parameters effect of free parameters at what
level can other distributions be ruled out?
4
Distribution with zero free parameters
Mean limit for large N expected mean for finite
N Variance of over PDF
(Statistical Methods in Experimental
Physics, Eadie, Drijard, James, Roos,
Sadoulet)
Summary Ensemble expts w N measurements of
x 0 free parameters
5
UMxL with free parameter(s)
(1) in each experiment, is maximized -
(2) J. Heinrich note (CDF/MEMO/BOTTOM/CDFR/5639)
toy MCs for 2 different PDFs by UMxL
found confirmed in analytic calculation -gt
conjecture CL/goodness is always 100 First,
examine (2)
6
Does fitted always give expected
?
Rewrite parametrized PDF -gt measured
distribution To maximize, gt
expectation value over PDF
lt-
7
The bottom line
Just 2 measured numbers characterize data
vis-a-vis f (averages, not highly correlated w
shape of data distribution -gt no GoF) Look at
PDFs examined by Heinrich (a) Note (b) Note
(maximization of l constrains 1)
(1 param - 2 measured s, 1 constraint)
gt 0 DoF in lmax
(2 params - 3 measured s, 2 constraints)
gt 0 DoF
i.e. these are special cases where
fixes
8
Illustrate
lmax
????????
Seen in lmax vs amax - Always get lmax
El(amax)
amax
100 correlation is special case
However there is often a partial correlation
9
example
(-1ltxlt1)
lmax
gt 2 largest terms are highly correlated
????????
amax

• ?max is at least partially correlated w measured
  amax,
• INDEPENDENTLY from actual distribution in
  data-gtno GoF

10
Can lmax be used within parametrization to set
confidence interval?
?????????????????????lmax - how much is mean ?
shifted by fit?
a
?
If we assume 100 correlation,
determines
N10
max D?O(??)-gtconjecture D?O(0.5) per fitted
parameter
11
Test on the same suspects
N10 ?1.0 Dl0.510.01
Mean -1.6420.001
Mean -1.6930.001
N100 ?1.0 Dl0.50.1
Mean -1.6900.001
Mean -1.6950.001
N1000 ?0.5 Dl0.50.1
Mean -0.68510.0001
Mean -0.68460.0001
12
can lmax be used within parametrization to set
confidence interval?
Maybe - with stronger demo of distribution
shift, width shift, extension to multiple
parameters nice for multi-parameter fits -
reduce to 1-d
Other speculations tests of fit quality using
information generated in UMxL without
resorting to binning Test on subsets of fitted
sample, e.g. sin2f1 result from simultaneous
fit over many decay modes - compare ?max in
different sets w. expectation - ?2
event-by-event distribution ??(?max) - moments,
or K-S test
13
Summary
Goodness-of-fit for UMxL sorry, not possible
with ?max alone Other measures of fit
quality Desirable, especially for multiparameter
fitting steps toward definition of ?max
distribution for general PDF speculation -
exploit info in ??(?max)