Independent Component Analysis For Time Series Separation - PowerPoint PPT Presentation

1 / 16
About This Presentation
Title:

Independent Component Analysis For Time Series Separation

Description:

An orthonormal matrix has n(n-1)/2 degrees of freedom. ... Demo for BBS with 'CoBliSS' (wav-files) http://www.esp.ele.tue.nl/onderzoek/daniels/BSS.html ... – PowerPoint PPT presentation

Number of Views:342
Avg rating:3.0/5.0
Slides: 17
Provided by: ahtasha
Category:

less

Transcript and Presenter's Notes

Title: Independent Component Analysis For Time Series Separation


1
Independent Component AnalysisFor Time Series
Separation
  • Ahtasham Ashraf

2
ICA
  • Blind Signal Separation (BSS) or Independent
    Component Analysis (ICA) is the identification
    separation of mixtures of sources with little
    prior information.
  • Applications include
  • Audio Processing
  • Medical data
  • Finance
  • Array processing (beamforming)
  • Coding
  • and most applications where Factor Analysis and
    PCA is currently used.
  • While PCA seeks directions that represents data
    best in a Sx0 - x2 sense, ICA seeks such
    directions that are most independent from each
    other.
  • We will concentrate on Time Series separation of
    Multiple Targets

3
The simple Cocktail Party Problem
Mixing matrix A
x1
s1
Observations
Sources
x2
s2
x As
n sources, mn observations
4
Motivation
Two Independent Sources
Mixture at two Mics
aIJ ... Depend on the distances of the
microphones from the speakers
5
Motivation
Get the Independent Signals out of the Mixture
6
ICA Model (Noise Free)
  • Use statistical latent variables system
  • Random variable sk instead of time signal
  • xj aj1s1 aj2s2 .. ajnsn, for all j
  • x As
  • ICs s are latent variables are unknown AND
    Mixing matrix A is also unknown
  • Task estimate A and s using only the observeable
    random vector x
  • Lets assume that no. of ICs no of observable
    mixtures
  • and A is square and invertible
  • So after estimating A, we can compute WA-1 and
    hence
  • s Wx A-1x

7
Illustration
2 ICs with distribution Zero mean and v
ariance equal to 1
Mixing matrix A is
The edges of the parallelogram are in the
direction of the cols of A So if we can Est joint
pdf of x1 x2 and then locating the edges, we
can Est A.
8
Restrictions
  • si are statistically independent
  • p(s1,s2) p(s1)p(s2)
  • Nongaussian distributions
  • The joint density of unit variance s1 s2 is
    symmetric. So it doesnt contain any information
    about the directions of the cols of the mixing
    matrix A. So A cannt be estimated.
  • If only one IC is gaussian, the estimation is
    still possible.

9
Ambiguities
  • Cant determine the variances (energies) of the
    ICs
  • Both s A are unknowns, any scalar multiple in
    one of the sources can always be cancelled by
    dividing the corresponding col of A by it.
  • Fix magnitudes of ICs assuming unit variance
    Esi2 1
  • Only ambiguity of sign remains
  • Cant determine the order of the ICs
  • Terms can be freely changed, because both s and A
    are unknown. So we can call any IC as the first
    one.

10
ICA Principal (Non-Gaussian is Independent)
  • Key to estimating A is non-gaussianity
  • The distribution of a sum of independent random
    variables tends toward a Gaussian distribution.
    (By CLT)
  • f(s1)
    f(s2) f(x1) f(s1 s2)
  • Where w is one of the rows of matrix W.
  • y is a linear combination of si, with weights
    given by zi.
  • Since sum of two indep r.v. is more gaussian than
    individual r.v., so zTs is more gaussian than
    either of si. AND becomes least gaussian when its
    equal to one of si.
  • So we could take w as a vector which maximizes
    the non-gaussianity of wTx.
  • Such a w would correspond to a z with only one
    non zero comp. So we get back the si.

11
Measures of Non-Gaussianity
  • We need to have a quantitative measure of
    non-gaussianity for ICA Estimation.
  • Kurtotis gauss0 (sensitive to outliers)
  • Entropy gausslargest
  • Neg-entropy gauss 0 (difficult to estimate)
  • Approximations
  • where v is a standard gaussian random variable
    and

12
Data Centering Whitening
  • Centering
  • x x Ex
  • But this doesnt mean that ICA cannt estimate the
    mean, but it just simplifies the Alg.
  • ICs are also zero mean because of
  • Es WEx
  • After ICA, add W.Ex to zero mean ICs
  • Whitening
  • We transform the xs linearly so that the x are
    white. Its done by EVD.
  • x (ED-1/2ET)x ED-1/2ET Ax As
  • where Exx EDET
  • So we have to Estimate Orthonormal Matrix A
  • An orthonormal matrix has n(n-1)/2 degrees of
    freedom. So for large dim A we have to est only
    half as much parameters. This greatly simplifies
    ICA.
  • Reducing dim of data (choosing dominant Eig)
    while doing whitening also help.

13
Noisy ICA Model
  • x As n
  • A ... mxn mixing matrix
  • s ... n-dimensional vector of ICs
  • n ... m-dimensional random noise vector
  • Same assumptions as for noise-free model, if we
    use measures of nongaussianity which are immune
    to gaussian noise.
  • So gaussian moments are used as contrast
    functions. i.e.
  • however, in pre-whitening the effect of noise
    must be taken in to account
  • x (ExxT - S)-1/2 x
  • x Bs n.

14
Simulation Results
I have used the Synthetic data with without
noise to separate the time series of DW AAV
which are moving fairly close to each other
15
Simulation Results
16
References
  • Feature extraction (Images, Video)
  • http//hlab.phys.rug.nl/demos/ica/
  • Aapo Hyvarinen ICA (1999)
  • http//www.cis.hut.fi/aapo/papers/NCS99web/node11.
    html
  • ICA demo step-by-step
  • http//www.cis.hut.fi/projects/ica/icademo/
  • Lots of links
  • http//sound.media.mit.edu/paris/ica.html
  • object-based audio capture demos
  • http//www.media.mit.edu/westner/sepdemo.html
  • Demo for BBS with CoBliSS (wav-files)
  • http//www.esp.ele.tue.nl/onderzoek/daniels/BSS.ht
    ml
  • Tomas Zemans page on BSS research
  • http//ica.fun-thom.misto.cz/page3.html
  • Virtual Laboratories in Probability and
    Statistics
  • http//www.math.uah.edu/stat/index.html
Write a Comment
User Comments (0)
About PowerShow.com