Functional Link Network

1
Functional Link Network
2
Support Vector Machines
3
(No Transcript)
4
(No Transcript)
5
Support Vector Machines
6
(No Transcript)
7
Support Vector Machines
8
Support Vector Machines
9
Support Vector Machines
10
Support Vector Machines
11
Support Vector Machines
12
Support Vector Machines
13
Support Vector Machines
14
Support Vector Machines
15
Two Spiral Problem
16
SVM architecture
17
Application text classification

• Reuters newswire messages
• Bag-of-words representation
• Dimension reduction
• Training SVM

18
Results
Break-even point precision value at which
precision and recall are nearly equal
19
Results
20
Application 2 face recognition
21
False detections
22
System architecture
23
Results
24
Results
25
Skin detection and real-time recognition
26
Neural Networks
27
Ccortex is a massive spiking neuron network
emulation and will mimic the human cortex, the
outer layer of gray matter at the cerebral
hemispheres, largely responsible for higher brain
functions. The emulation covers up to 20 billion
layered neurons and 2 trillion 8-bit connections.
28
Spiking Neural Networks

• From neurones to neurons
• Artificial Spiking Neural Networks (ASNN)
• Dynamic Feature Binding
• Computing with spike-times

29
Neural Networks

• Artificial Neural Networks
• (neuro)biology -gt Artificial Intelligence (AI)
• Model of how we think the brain processes
  information
• New data on how the brain works!
• Artificial Spiking Neural Networks

30
Real Neurons

• Real cortical neurons communicate with spikes or
  action potentials

31
Real Neurons

• The artificial sigmoidal neuron models the rate
  at which spikes are generated
• artificial neuron computes function of weighted
  input

32
Artificial Neural Networks

• Artificial Neural Networks can
• approximate any function
• (Multi-Layer Perceptrons)
• act as associative memory
• (Hopfield networks, Sparse Distributed Memory)
• learn temporal sequences
• (Recurrent Neural Networks)

33
ANNs

• BUT....
• for understanding the brain the neuron model is
  wrong
• individual spikes are important, not just rate

34
Binding Problem

• When humans view a scene containing a red circle
  and a green square, some neurons
• signal the presence of red,
• signal the presence of green,
• signal the circle shape,
• Signal the square shape.
• The binding problem
• how does the brain represent the pairing of color
  and shape?
• Specifically, are the circles red or green?

35
Binding

• Synchronizing spikes?

36
New Data!

• neurons belonging to same percept tend to
  synchronize (Gray Singer, Nature 1987)
• timing of (single) spikes can be remarkably
  reproducible
• Spikes are rare average brain activity lt 1Hz
• rates are not energy efficient

37
Computing with Spikes

• Computing with precisely timed spikes is more
  powerful than with rates.
• (VC dimension of spiking neuron models)
• W. Maass and M. Schmitt., 1999
• Artificial Spiking Neural Networks??W. Maass
  Neural Networks, 10, 1997

38
Artificial Spiking Neuron

• The state ( membrane potential) is a weighted
  sum of impinging spikes
• spike generated when potential crosses threshold,
  reset potential

39
Artificial Spiking Neuron

• Spike-Response Model
• where e(t) is the kernel describing how a single
  spike changes the potential

40
Artificial Spiking Neural Network

• Network of spiking neurons

41
Error-backpropagation in ASNN

• Encode X-OR in (relative) spike-times

42
XOR in ASNN

• Change weights according to gradient descent
  using error-backpropagation (Bohte et al,
  Neurocomputing 2002)
• Also effective for unsupervised learning(Bohte
  etal, IEEE Trans Neural Net. 2002)

43
Oil Application