To Understand, Survey and Implement Neurodynamic Models

1
To Understand, Survey and Implement Neurodynamic
Models

• By
• Farhan Tauheed
• Asif Tasleem

2
Project Progress

• Literature Review
• Temporal Networks
• Specific Problem for Implementation
• Implications
• Architectural Plan for Implementation
• Formal definition

3
Motivation

• Machine Perception
• Biological aspects of Traditional Neural Network
  Models
• Summation neuron
• Non Linear Activation function
• Non biological aspects
• Static
• Continuous Input
• Back propagation learning algorithm

4
Temporal Neural Networks

• Biologically Inspired
• Continuous data feed is operated on
• Dynamic Model
• Long term Memory
• Short term Memory
• Tapped delay line
• Distributed Time Lagged Feed forward NNs
• Different Back Propagation algorithm

5
Literature Review

• Universal Myopic Mapping theorem
• Any uniform fading memory mapped behind a static
  network can simulate just as well
• Fontine and Shastri 1993. have demostrated that
  certain tasks not having an explicit temporal
  aspect can also be processed advantageously by
  Temporal Networks
• Thompson(1996) Completeness of BSS

6
Related Problems

• Time Series Data Prediction
• Blind Signal Separation
• Cocktail Party Problem
• Attention Based Search Optimization
• Visual Pattern Recognition

7
Blind Signal Separation Implication

• Speech Recognition (phoneme recognition)
• Multimedia Compression
• MM database sound based retrieval
• Noise Removal
• Audio Analysis and Visualization
• Sonar and Radar
• Cache Hit Algorithms

8
Problem Decomposition

• Blind Signal Separation
• General problem
• No knowledge about the constituents
• Cocktail Party problem (Specific case)
• Much restricted
• Few sources
• Can be many sensors
• Source positioning can also be used as a cue

9
Continued

• Melody Decomposition (Specific Case)
• Repetition in constituent signals (Cue)
• Signals usually periodic
• Difficulty (Scale invariant)
• Basic Keyword DECONVULUTION

10
Cocktail Party Problem

• Formal Problem Description
• Given N signal sensors receiving N convolved
  signals made up of d original signals such that
  dltN
• We have to design an adaptive filter that masks
  each original signal from the rest

11
Our Solution

• Assumptions
• Ideal environment .. No noise .. No other signals
  other than d
• We have prior knowledge of number of d
• Number of sources is known (MIC) were
  experimenting with two
• Research being followed
• COMBINING TIME-DELAYED DECORRELATION AND
  ICATOWARDS SOLVING THE COCKTAIL PARTY PROBLEM
• By Te-Won Lee Andreas Ziehe

12
Solution details

• Network Architecture
• Single layer
• Feed forward
• Feedback (stability issues )
• Sigmoid activation function
• Learning rule (Maximizing joint entropy)
• Frequency domainFFT 1024 point
• Tapped delay lines for short term memory

13
Example
14
Current progress

• Things Done
• Obtained binaural audio files
• Implementation done in MATLAB
• Using Neural Network toolbox
• FFT function
• Problem in training time due to FFT in training
  rule.
• Things TODO
• Implementation complete / Optimize
• Look into oscillatory networks