Open Issues in Constrained Blind Source Separation - PowerPoint PPT Presentation

Loading...

PPT – Open Issues in Constrained Blind Source Separation PowerPoint presentation | free to download - id: 47762a-NzQ1Y



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Open Issues in Constrained Blind Source Separation

Description:

Open Issues in Constrained Blind Source Separation Jonathon Chambers Cardiff Professorial Research Fellow Cardiff School of Engineering Cardiff University, Wales, U.K. – PowerPoint PPT presentation

Number of Views:1476
Avg rating:3.0/5.0
Slides: 44
Provided by: JenniferM53
Learn more at: http://www.ewh.ieee.org
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Open Issues in Constrained Blind Source Separation


1
Open Issues in Constrained Blind Source
Separation
  • Jonathon Chambers
  • Cardiff Professorial Research Fellow
  • Cardiff School of Engineering
  • Cardiff University, Wales, U.K.
  • E-mail chambersj_at_cf.ac.uk

2
Summary of Talk
  • Acknowledgement
  • Historical background motivation
  • BSS with matrix constraints
  • Penalty functions in FD-BSS
  • Exploiting periodicity in BSS
  • Future application-driven challenges

3
Acknowledgements
  • Jonathon Chambers wishes to express his
    sincere thanks for the support of Professor
    Andrzej Cichocki, Riken Brain Science Institute,
    Japan
  • The invitation from the organising committee
    of the workshop to give this talk.
  • His co-researchers Drs Saeid Sanei, Maria
    Jafari and Wenwu Wang.

4
LMS Algorithm
  • B. Widrow, and M.E. Hoff, Jr.,
  • Adaptive switching circuits, IRE Wescon
    Conv. Rec., pt. 4, pp. 96-104, 1960.
  • LMS Update

5
Historical Background
  • The field of conventional adaptive signal
    processing has been greatly enhanced by the
    exploitation of constrained optimisation
  • Constraints on the error, and/or structure or
    some norm of the weights via, for example,
    Lagrange multipliers and/or Karush-Khun-Tucker
    conditions

6
Historical Background
  • Certain key papers
  • O.L. Frost, III, An algorithm for linearly
    constrained adaptive array processing, Proc.
    IEEE, Vol. 60(8), pp. 926-925, 1972
  • R.P. Gitlin et al. The tap-leakage algorithm an
    algorithm for the stable operation of a digitally
    implemented fractionally spaced equalizer, Bell
    Sys. Tech. Journal, Vol. 61(8), pp. 1817-1839,
    1982.
  • D.T.M. Slock, Convergence behavior of the LMS
    and Normalised LMS Algorithms, IEEE Trans.
    Signal Processing, Vol. 41(9), pp. 2811-2825,
    1993.

7
Historical Background Cont.
  • S.C. Douglas, A family of normalized LMS
    algorithms, IEEE Signal Processing Letters, Vol.
    1(3), pp. 49-51, 1994.
  • S.C. Douglas, and M. Rupp, A posteriori updates
    for adaptive filters, Asilomar Conference on
    Signals, Systems and Computers, Vol. 2, pp
    1641-1645, 1997.
  • T. Gänsler, et al., A robust proportionate
    affine projection algorithm for network echo
    cancellation, Proc. ICASSP 2000, Vol. 2, pp.
    793-796, 2000.
  • O. Vainia, Polynomial constrained LMS adaptive
    algorithm for measurement signal processing,
    Proc. IECON 2002, Vol. 2, pp. 1479-1482, 2002.

8
Motivation
  • In many applications of Independent Component
    Analysis (ICA) and Blind Source Separation (BSS)
    estimated source signals and the mixing or
    separating matrices have some special structure
    or some constraints are imposed for the
    matrices, Cichocki and Georgiev, 2003

9
Fundamental Model for Instantaneous Blind Source
Separation
10
Certain BSS Books
  • Andrzej Cichocki and Shun-Ichi Amari, Adaptive
    Blind Signal and Image Processing, Wiley, 2002
  • Simon Haykin Unsupervised Adaptive Filtering,
    Vols. I and II, Wiley, 2000
  • Aapo Hyvärinen, Juha Karhunen and Erkki Oja,
    Independent Component Analysis, Wiley, 2001
  • Te-Won Lee, Independent component analysis
    theory and applications, Kluwer, 1998

11
BSS References
  • A. Mansour and M. Kawamoto, ICA Papers
    Classified According to their Applications and
    Performances, IEICE Trans. Fundamentals, Vol.
    E86-A, No. 3, March 2003, pp. 620-633.
  • In 2002, 800 different papers have been
    published, these are downloadable at
    http//ali.mansour.free/REF.htm

12
BSS With Matrix Constraints
With a symmetric mixing matrix CG,2003-
13
BSS With Matrix Consts. Cont.
Stable Frobenius norm of the separating matrix
Theorem CG 2003 The learning rule
where ß gt 0 is a scaling factor and ?(t)
trace(WT(t)F(y(t))W(t)) gt 0, stabilizes the
Frobenius norm of W(t) such that
14
BSS With Matrix Consts. Cont.
Consequence The modified NG descent learning
algorithm, with a forgetting factor, described
as
with ?(t) -trace(WT(t)?J(W)/ ? WWT(t)W(t)) gt
0 has a W(t) with bounded Frobenius norm
throughout the learning process.
15
BSS With Matrix Consts. Cont.
Prof. Amaris Leaky NG Algorithm becomes
where 0 ltlt (1-ß?(t)?(t)) lt 1 is the leakage
factor
16
BSS With Matrix Consts. Cont.
Introducing a semi-orthogonality constraint so
that it is possible to extract an arbitrary group
of sources, say e, 1 ? e ? N. Assuming
pre-whitened data
and the mixing matrix A QH, the demixing
matrix We should satisfy WeA Ie,0N-e
17
BSS With Matrix Consts. Cont.
A natural gradient algorithm to find We
becomes-
With initial conditions which satisfy
18
Real Convolutive Mixing Env. Cocktail Party
Problem
19
Convolutive BSS Model
Convolution
Compact form
Expansion form
20
Taxonomy of Existing Sols. To Convolutive BSS
  • Performing blind separation in the time domain
    by extending the existing instantaneous methods
    to conv. case
  • Transforming the convolutive BSS problem into
    multiple instantaneous (complex) problems in the
    frequency domain
  • Decomposing the system into smaller problems
    using, for example, a subband approach
  • Hybrid frequency and time domain approaches

21
Transform Convolutive BSS into the Frequency
Domain
DFT
Convolutive BSS problem
Multiple complex-valued instantaneous BSS
problems
22
Mathematical Formulation
In the frequency domain-
23
De-mixing Operation
24
Constrained Optimisation and Joint Diagonalisation
25
Joint Diagonalisation Criterion
Exploiting the non-stationarity of speech signals
measured by the cross-spectrum of the output
signals,
26
Exterior Penalty Function Approach
27
Exterior Penalty Function Approach
Typical exterior penalty functions, and the
shadow area represents the feasible set.
28
Proposed General Cost Function
With a factor vector ? to incorporate exterior
penalty functions, our cost function becomes-
29
Numerical Experiments
  • Use an exterior penalty function
  • Employ a variant of gradient adaptation
  • Utilize the filter length constraint to address
    the permutation problem (Parra Spence)
  • System with two inputs and two outputs (TITO!)
  • H(z) 1 1.9 -0.75, z-50.5 0.3 0.2
    z-5-0.7 -0.3 -0.2, 0.8 -0.1 D 7, T
    1024, K 5.

30
Convergence Performance of the New Criterion as a
function of ?
31
Room Environment Experiment
  • Use roommix function due to Westner
  • Room 10x10x10m3 cube
  • Wall reflections calculated up to fifth order,
    atten. factor 0.5
  • SIR is plotted as a function of length of the
    separating system

32
Room Environment
33
Room Environment SIR
34
Permutation Problem in FD-CBSS
35
Summary of Existing Solutions to Permut. Problem
in FD-CBSS
  • Constraints on the filter models in the
    frequency domain
  • Using special structure contained in signals
  • Merging beamforming view to align solutions
  • Exploiting the continuity of the spectra of the
    recovered signals could coupled hidden Markov
    Models be used?
  • What happens when the sources move,
    enter/re-enter the environment? What is the way
    forward?

36
Exploiting Source (Pseudo) -Periodicity
  • W. Wang, M.G. Jafari, S. Sanei, and J.A.
    Chambers, Blind source separation of convolutive
    mixtures of cyclostationarity, to appear in the
    Special Issue on BSS, International Journal of
    Adaptive Control and Signal Processing, Guest
    Editor Mike Davies, Queen Marys College,
    University of London
  • H. Swada, R. Mukai, S. Araki, and S. Makino, A
    robust and precise method for solving the
    permutation problem of frequency-domain blind
    source separation, ICA 2003, Nara, Japan, 2003,
    pp. 505-510.

37
A natural gradient update exploiting
cyclostationarity
  • The Cyclostationary NGA uses the update equation
  • where
  • and ?p is the cycle frequency of the p-th source

38
A natural gradient update exploiting periodicity
  • The Periodic NGA type update equation
  • where

39
Emerging Applications
Biomedical- ECG, EEG, MEG and their
integration Microarray time courses
Measurements from the nano-lab http//www.nmrc.i
e/research/transducers-group/trends.html http//ww
w.nanospace.systems.org/ns_2000/NS00_Sessions.htm
http//nanomed.ncl.ac.uk/m2l.htm
Star Trek The Tri-corder
40
Emerging Applications
  • T. Bowles, J. Chambers, and A. Jakobsson,
    Advanced spectral estimation for the
    identification of cell-cycle regulated genes,
    IEEE EMBS UK and RI Postgraduate Conf in
    Biomedical Engineering and Medical Physics, 2003.
  • X. Liao, and L. Carin, Constrained independent
    component analysis of DNA microarray signals,
    IEEE Workshop on Genomic Signal Processing and
    Statistics, 2002.
  • S-I, Lee, and S. Batzoglou, Discovering
    biological processes from microarray data using
    independent component analysis, Dept EE/CS,
    Stanford Univ.

41
Summary
  • The exploitation of constrained optimisation
    has been fundamental to the development and
    application of adaptive signal processing this
    process is, however, very much in its infancy in
    blind source separation (BSS).
  • Utilisation of certain a priori knowledge on the
    mixing matrices and the properties of the sources
    is likely to yield solutions to real-life SP
    problems.
  • As such, the challenge for DSP engineers in the
    21st Century, is to advance the application of
    BSS methods in line with methods from adaptive
    signal processing.

42
Other References
  • A. Cichocki, and P. Georgiev, Blind source
    separation algorithms with matrix constraints,
    IEICE Trans. Fundamentals, Vol. E86-A(3), March
    2003, pp. 522-531.
  • J.G. McWhirter, Mathematics and signal
    processing, Mathematics Today, April 2003, pp
    47-54.
  • W. Wang, S. Sanei, and J. Chambers, Penalty
    function based joint diagonalization approach for
    convolutive blind source separation, submitted
    to IEEE T-SP, Sept 2003.

43
Close ???
Mark Twain A man who swings a cat by its tail
learns things he can learn no other way
About PowerShow.com