Information Geometry of

1
Bernoulli 2000 Conference at Riken on 27 October,
2000
Information Geometry of Self-organizing
maximum likelihood
Shinto Eguchi ISM, GUAS
This talk is based on joint research with
Dr Yutaka Kano, Osaka Univ
2
Consider a statistical model
Maximum Likelihood Estimation (MLE)
( Fisher, 1922),
Consistency, efficiency sufficiency, unbiasedness
invariance, information
Take an increasing function .
-MLE
3
Normal density
-MLE
given data
-MLE
MLE
4
Normal density
MLE
outlier
-MLE
5
(No Transcript)
6
Examples
KL-divergence
(1)
(2)
-divergence
-divergence
(3)
7
(No Transcript)
8
Pythagorian theorem
(0,1)
(1,1)
.
( t, s )
(1,0)
(0,0)
9
(Pf)
10
Differential geometry of
Riemann metric
Affine connection
Conjugate affine connection
Ciszsars divergence
11
-divergence
Amaris -divergence
12
-likelihood function
Kullback-Leibler and maximum likelihood
M-estimation ( Huber, 1964, 1983)
13
Another definition of Y-likelihood
Take a positive function k(x, q) and define
Y-likelihood equation is a weighted score with
integrabity.
14
Consistency of Y-MLE
15
Fisher consistency
e -contamination model of
Influence function
Asymptotic efficiency
Robustness or Efficiency
16
Generalized linear model
Regression model
Estimating equation
17
Bernoulli regression
Logistic regression
18
Misclassification model
MLE
MLE
19
Logistic Discrimination
Group I from
Group II from
Mislabel
5
Group I
Group II
35
Group I
Group II
20
Misclassification
5 data
Group II
Group I
35 data
21
Poisson regression
-likelihood function
-contamination model
Canonical link
22
Neural network
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
(No Transcript)
27
Classic procedure
Self-organizing procedure
28
Independent Component Analysis (Minami
Eguchi, 2000)
F
F
29
Theorem (Semiparametric consistency)
S
F
S
(Pf)
30
-likelihood satisfies the semiparametric
consistency
31
(No Transcript)
32
(No Transcript)
33
(No Transcript)
34
Concluding remark
Bias potential function
Y-sufficiency Y-factoriziable Y-exponential
family Y-EM algorithm
Y-Regression analysis Y-Discriminant
analysis Y-PCA Y-ICA
?
!