BSA, a Fast and Accurate Spike Train Encoding Scheme

1
BSA, a Fast and Accurate Spike Train Encoding
Scheme

• Benjamin Schrauwen

2
Overview

• Introduction
• Reading the spiking language
• Speaking the spiking language
• Results
• Conclusions and further work

3
Introduction

• NN as engineering tools
• Operate on analog quantities
• Analog input, analog output

4
Introduction

• When using NN as engineering tools
• Operate on analog quantities
• Analog input, analog output
• Spiking neural networks
• More powerful than classic NN
• Interesting to implement in hardware
• Problem internally work with spike trains ?
  analog

E
D
5
Introduction

• When using NN as engineering tools
• Operate on analog quantities
• Analog input, analog output
• Spiking neural networks
• More powerful than classic NN
• Interesting to implement in hardware
• Problem internally work with spike trains ?
  analog
• Use spiking NN as engineering tool
• Communication problem must be solved
• Convert spike trains into analog function
  decoding
• Convert analog function into spike trains
  encoding

6
Overview

• Introduction
• Reading the spiking language
• Speaking the spiking language
• Results
• Conclusions and further work

7
Reading the spiking language

• Has been studied thoroughly by neuroscientists
• Classic methods
• Average rate
• Analog value sum of spikes in fixed time window
  of spike train
• Exact spike time, ea. Phase coding
• Analog value time from periodic event to first
  spike
• Stimulus estimation or filter coding
• Analog output filtered version of spike train
• Filter low-pass

8
Speaking the spiking language

• How does it work

Filter decoding
Filter f
output
t
input
t
9
Overview

• Introduction
• Reading the spiking language
• Speaking the spiking language
• Results
• Conclusions and further work

10
Speaking the spiking language

• Classic decoding almost trivial encoding
• Filter decoding not so simple
• Hough et al. introduced spike train encoding
  algorithm
• HSA (Houghs spike algorithm)
• Tries to reverse the filtering decoding
• Based on a heuristic
• Deterministic
• Filter used for decoding is also used for
  encoding
• We use slightly different algorithm than the one
  used in original paper (allows threshold
  optimization)

11
Speaking the spiking language

• How does it work

Filter encoding using HSA
input
t
output
t
12
Speaking the spiking language

• How does it work

Filter encoding using HSA
Heuristic used by HSA min(x,f)-f if heuristic
gt threshold then fire
input
t
negative !
output
t
13
Speaking the spiking language

• How does it work

Filter encoding using HSA
input
t
output
t
14
Speaking the spiking language

• How does it work

Filter encoding using HSA
Bigger than threshold -0.0685
input
t
output
t
15
Speaking the spiking language

• How does it work

Filter encoding using HSA
Subtract f from Input signal
input
t
FIRE !
output
t
16
Speaking the spiking language

• How does it work

Filter encoding using HSA
input
t
output
t
17
Speaking the spiking language

• How does it work

Filter encoding using HSA
input
t
output
t
18
Speaking the spiking language

• How does it work

Filter encoding using HSA
input
t
output
t
19
Speaking the spiking language

• How does it work

Filter encoding using HSA
input
t
output
t
20
Speaking the spiking language

• How does it work

Filter encoding using HSA
input
t
output
t
21
Speaking the spiking language

• How does it work

Filter encoding using HSA
input
t
output
t
22
Speaking the spiking language

• How does it work

Filter encoding using HSA
input
t
output
t
23
Speaking the spiking language

• How does it work

Filter encoding using HSA
input
t
output
t
24
Speaking the spiking language

• How does it work

Filter encoding using HSA
Bigger than threshold
input
t
output
t
25
Speaking the spiking language

• How does it work

Filter encoding using HSA
Subtract f from Input signal
input
t
FIRE !
output
t
26
Speaking the spiking language

• How does it work

Filter encoding using HSA
Remaining error signal
Spike train
input
t
output
t
27
Speaking the spiking language

• HSA problems
• Better performance ?
• What must the threshold be ?
• What's the meaning of the heuristic ?
• Filters with negative taps not supported
• BSA
• Basic algorithm same as HSA, but other heuristic
• Calculate two error functions
• Error 1 mean error if we dont fire a spike now
• Error 2 mean error if we fire a spike now
• Mean absolute error (MSE possible but performs
  worse)
• Subtract, compare to threshold and according to
  this fire

28
Speaking the spiking language

• How does it work

Filter encoding using BSA
Heuristic used by BSA x-x-f
-
input

t
output
t
29
Speaking the spiking language

• How does it work

Filter encoding using BSA
Heuristic used by BSA x-x-f
-
input

t
output
t
30
Speaking the spiking language

• How does it work

Filter encoding using HSA
Bigger than threshold 0.9550
Heuristic used by BSA x-x-f
input

t
FIRE !
output
t
31
Speaking the spiking language

• What about the threshold
• Actual threshold largely determines performance
• Original HSA publication threshold 0

BSA
Average over different test functions
SNR in dB
threshold
32
Overview

• Introduction
• Reading the spiking language
• Speaking the spiking language
• Results
• Conclusions and further work

33
Results
Impulse response

• The filter used in the results has
• 24 taps
• 10 bit quantized coefficients
• Was used in original HSA publications
• All results
• Input encoded into spikes
• Spikes decoded into output
• Difference between input and output
• Paper
• Unipolar and bipolar case
• Here only bipolar

amplitude
samples
34
Results
SNR (dB)
SNR (dB)
BSA
HSA
Optimal threshold
Optimal threshold
Fixed threshold
8dB
5dB
Fixed threshold
Original publication
Normalized input frequency
Normalized input frequency
Input amplitude 0.25
35
Results

• HSA and BSA are non-linear systems
• Not only characterized by SNR for different
  frequencies
• Also give SNR for different amplitudes

36
Results
SNR (dB)
SNR (dB)
BSA
HSA
Overloading
Dynamic range
Optimal threshold
10dB
Optimal threshold
Fixed threshold
5dB
Original publication
Fixed threshold
Input amplitude (dB)
Input amplitude (dB)
Input frequency 0.0033
37
Overview

• Introduction
• Reading the spiking language
• Speaking the spiking language
• Results
• Conclusions and further work

38
Conclusions and further work

• Threshold optimization 5 dB increase for
  original HSA
• BSA 15 dB increase over published HSA
• Frequency and amplitude characteristics are
  smoother
• Still very limited calculation overhead
• Delay introduced equal to filter length

39
Conclusions and further work

• Analyze why these algorithms work
• Even better algorithm possible ? Optimal ?
• How does the filter change the characteristics of
  the algorithms ?
• How does performance compare to other coding
  schemes ?
• Is there any biological relevance ?