A%20Null-space%20Algorithm%20for%20Overcomplete%20ICA

1
A Null-space Algorithm for Overcomplete ICA
Ying Nian Wu UCLA Department of
Statistics Joint with Ray-Bing Chen National
Kaohsiung University
2

• Plan
• Independent component analysis
• Overcomplete ICA
• Null space algorithm
• Experiments

3
Independent Component Analysis (ICA)
Blind source separation (Comon,1989,94)
4
Statistical modeling inference
Coding/mutual information
5
Unsupervised learning (Bell Sejnowski,1997)
6
Overcomplete ICA
7
Olshausen Field (1996)
Lewicki Sejnowski (2000), Lee et al (1999)
8
Comparing ICA and Overcomplete ICA
ICA OICA
Noise
Source
Our work null-space representation
9
A simple case
General situation singular value decomposition
10
Bayesian modeling inference
11
Data augmentation algorithm (Tanner Wong, 1987)
12
AR coefficient known
Mixture 1
Mixture 2
Speech
Recovered speech
Rain
Recovered rain
Wind
Recovered wind
13
AR coefficient known
Mixture1
Mixture 2
Ocean wave
Recovered wave
Rain
Recovered rain
Wind
Recovered wind
14
AR coefficient known
Mixture 1
Mixture 2
Ocean wave
Recovered wave
Rain
Recovered rain
Engine
Recovered engine
15
AR coefficient unknown
Mixture 1
Mixture 2
Ocean wave
Recovered wave
Rain
Recovered rain
Wind
Recovered wind
16
Parameter estimation
Parameter estimation
True coefficient ?1 ?2 ?3 ?4 ?5 ?6 ?7 ?8
Source 1 0.613 -0.264 0.235 -0.202 0.171 -0.188 0.139 -0.068
Source 2 1.015 -0.610 0.461 -0.331 0.340 -0.196 0.176 0.011
Source 3 0.849 -0.448 0.353 -0.216 0.244 -0.133 0.083 -0.034
Estimation ?1 ?2 ?3 ?4 ?5 ?6 ?7 ?8
Recover 1 0.568 -0.202 0.156 -0.226 0.113 -0.182 0.173 -0.179
Recover 2 1.135 -0.855 0.717 -0.520 0.490 -0.266 0.137 0.096
Recover 3 0.810 -0.424 0.321 -0.216 0.231 -0.142 0.079 -0.046