Model and largescale simulator of a biological retina, with contrast gain control

1
Model and large-scale simulator of a biological
retina,with contrast gain control

• Adrien Wohrer
• May 6, 2008

i3S
Odyssée Team
Sophia Antipolis
2
The retina ?
Light
to the brain
Spikes
3
The retina ?
Light
Spikes
4
The retina is?

• 1. The retina is the brains camera.

• 2. The retina is an edge detector.

5
The retina is?

• 1. The retina is the brains camera.

• Simple transmission of the light information.

• 2. The retina is an edge detector.

• Linear filtering by a Difference-of-Gaussians.

6
The retina is the brains camera

• Yes, but

• a camera has static gains, leading easily
• to saturation.

7
The retina is the brains camera
Retinal illumination (log td)
Possible damage
8.5
2.6
Cone system
Rod system
-2
Absolute threshold (dark noise)
-4.4
Middleton 52 Lance Hahn
8
The retina is the brains camera

• Contextual processing of the stimulus.

9
The retina is an edge detector
Input image I(x,y)
Retinal output A(x,y)
norm
Marr, 82
10
The retina is an edge detector
Center-Surround architecture

• Response to sinusoidal gratings ? Fourier
  analysis
• Problems
• Uniform across cells ?
• Temporal properties ?
• Nonlinearities ?
• Really an edge detection ? (see next slide)

Hubel and Wiesel 60, Rodieck 65, Enroth-Cugell
and Robson 66 Croner and Kaplan, 95
11
The retina is an edge detector

• Primate midget cells
• wS/wC 0.8
• sC 0.03, sS 0.18
• (Croner and Kaplan 95)

8
12
The retina is?

• 1. The retina is the brains camera.

• Correct analogy. But also
• Ongoing, dynamic adaptations.
• Spatio-temporal pre-processing.

• 2. The retina is an edge detector.

• Rather Band-pass sensitivity.
• Temporal properties of filtering ?

13
Goals

• Build a biologically plausible retina model.
• Review and integrate a wide bibliography.
• Focus on functionality.
• Large-scale simulator, from light to spikes.
• Customizable, easy to use, open-source.
• Generic computer tools (for other models).
• Tool to understand low-level visual processing.
• Input to models of higher-level cortical areas.
• Study retinal processing per se.

14
Overview

• Introduction
• Classical Anatomy and Models
• Biological Retina Model
• Gain Control Mechanism
• Virtual Retina Large-Scale Simulator
• Contributions, Perspectives

15
Overview

• Introduction
• Classical Anatomy and Models
• Biological Retina Model
• Gain Control Mechanism
• Virtual Retina Large-Scale Simulator
• Contributions, Perspectives

16
Anatomy of retinal cells
Light
Spikes
17
Anatomy of retinal cells
18
Layers of retinal cells
Receptors
Beta (X) cells
Horizontal
Bipolar
Amacrine
Ganglion
Alpha (Y) cells
Masland 01
Wässle 01
19
Radial repartition of cells

• Fovea (area centralis) with higher density
• Primates (humans), (cats, dogs, )

Ganglion cells
Receptors
Osterberg 35
Dacey and Petersen 92
20
Synaptic interactions
Receptors
Horizontal
OUTER PLEXIFORM LAYER
Bipolar
INNER PLEXIFORM LAYER
Amacrine
Ganglion
21
Ganglion cells General characteristics

• For linear analysis, retinal output is the
  average firing rate.
• The responses to static stimuli are transient.
• ON and OFF cells.

22
Temporal filters
Experimental Measures
K(t)
I(t)
A(t) KI (t)
A(t)
Keat et al. 01
23
Temporal filters
The difference-of-Exponentials (DOE) model
Impulse response
Transfer function
24
Temporal filters
The difference-of-Exponentials (DOE) model

• A transient filter

I(t)
1
A(t) KI (t)
0
t
K(t)
25
Spatio-temporal filters
I(x,y,t)
!
A(t) KI (x0,y0,t)
measured
model
Cai et al. 95
(x0,y0)
A(t)

• Temporalized difference-of-Gaussians

• IMPORTANT ! Short delay (4 ms) of surround
  signal as compared to center.
• (Enroth-Cugell et al 83, Bernadete and Kaplan 99)

26
LN models (Linear-Nonlinear)
KI
K(t)
I(x,y,t)
N(.)
N(.)
A(t) N ( KI (x0,y0,t) )
(x0,y0)
A(t)

• Reverse correlation analysis -gt best (K,N).
• The State-of-the-art model.
• Enhancements LNP (spikes), LNL (Y cells), . . .

Chichilnisky 01
27
Overview

• Introduction
• Classical Anatomy and Models
• Biological Retina Model
• Gain Control Mechanism
• Virtual Retina Large-Scale Simulator
• Contributions, Perspectives

28
III. Biological Retina Model

• The OPL Filter
• Contrast Gain Control
• Ganglion Cells

29
Three Stages of the Model
OUTER PLEXIFORM LAYER
INNER PLEXIFORM LAYER
30
Three Stages of the Model
OPL Filter
OUTER PLEXIFORM LAYER
OPL Current
Contrast Gain Control
INNER PLEXIFORM LAYER
31
Three Stages of the Model
OPL Filter
OUTER PLEXIFORM LAYER
OPL Current
Contrast Gain Control
Bipolar Current
INNER PLEXIFORM LAYER
32
Three Stages of the Model
OPL Filter
OUTER PLEXIFORM LAYER
OPL Current
Contrast Gain Control
Bipolar Current
INNER PLEXIFORM LAYER
Ganglion cells
33
Three Stages of the Model
OPL Filter
OUTER PLEXIFORM LAYER
OPL Current
Contrast Gain Control
Bipolar Current
INNER PLEXIFORM LAYER
Ganglion cells
34
STAGE The OPL Filter
OPL Filter
OUTER PLEXIFORM LAYER
OPL Current
Contrast Gain Control
Bipolar Current
INNER PLEXIFORM LAYER
Ganglion cells
35
STAGE The OPL Filter

• Linear spatio-temporal filtering structure of the
  retina.
• Phototransduction in receptors.
• Center-Surround opposition in the OPL.

36
Receptors and phototransduction

• Phototransduction creates a circulating
  electrical current.

• Complex chemical cascade.
• Linear approximation low-pass temporal filter.

Impulse response, Van Hateren and Lamb 02

• Calcium feedbacks provide adaptation to luminance.

37
STAGE The OPL Filter
Receptors
t
Receptors
Horizontal
OPL Current
Bipolar
38
Origin of the center-surround opposition ?
Receptors
?
Horizontal

-
Bipolar

• Strong coupling gap junctions in receptors and
  horizontal cells
• Synaptic transmission with different signs to
  bipolar cells
• Possible role of amacrine cells ?

Models Mahowald and Mead 91, Herault 96
Lamb 76
39
STAGE The OPL Filter
Receptors
t
Receptors
Horizontal Cells
Horizontal
OPL Current
optional
OPL Current
Bipolar
Undershoot
Tw,t (t)
KOPL KC w KS
w
40
STAGE The OPL Filter
Temporalized difference of Gaussians

• Spatially band-pass
• Temporally band-pass

Receptors
Horizontal
OPL Current
Bipolar
41
STAGE Contrast Gain Control
OPL Filter
OUTER PLEXIFORM LAYER
OPL Current
Contrast Gain Control
Bipolar Current
INNER PLEXIFORM LAYER
Ganglion cells
42
STAGE Contrast Gain Control

• Definition Nonlinear adaptation to the levels of
  contrast in the stimulus.
• Temporal expression recent levels of contrast
• Shapley and Victor 78, Baccus and Meister 02,
• Models Victor 87, Van Hateren 02,
• Spatial expression neighboring levels of
  contrast
• Bonin et al. 05,

• Our model New contribution
• Common framework spatial and temporal.
• Simple, functional expression.
• Biological interpretation.

43
Temporal expression of contrast gain control
t
Shapley and Victor, 78

• Static grating modulated by a temporal sum of 8
  sinusoidal signals.
• Repeated for 4 different levels of contrast
  (contrast being doubled each time)

44
Temporal expression of contrast gain control
Log(2)
t
Shapley and Victor, 78

• Observations
• Band-pass behavior
• Contrast gain control
• Underlinearity with contrast
• Phase advance at high contrasts

Amplitude
Phase
45
Spatial expression of contrast gain control
Bonin et al. 05

• Divisive influence of the spatial surround
  (Mask) on the central responses (Test).

46
STAGE Contrast Gain Control
OPL Current
Wohrer et al., Neurocomp 06 Wohrer et al. 08,
JCNS.
Bipolar Cells
Feedback through inhibitory conductance
Pooling
Delay
Q(V)
y
t
x
Adaptation Conductance
Q (
47
STAGE Ganglion Cells
OPL Filter
OUTER PLEXIFORM LAYER
OPL Current
Contrast Gain Control
Bipolar Current
INNER PLEXIFORM LAYER
Ganglion cells
48
STAGE Ganglion Cells

• Different types of ganglion cells
• X cells, Y cells.
• Y cells have a very typical spatial nonlinearity.
• (Enroth-Cugell and Robson 66, Enroth-Cugell and
  Freeman 87, Demb 01)

• Spike generation
• Random process
• Not Poisson
• Better modeled by Integrate-and-Fire (LIF)
  processes

• Many other subtleties !
• Spike sychrony, rare ganglion cells, amacrine
  cells, etc.
• See thesis !, or Wohrer 08, in preparation

49
STAGE Ganglion Cells
Bipolar cells

• Additional temporal high-pass on bipolar current
• Displaced amacrine cells. Nirenberg et al, 97
• Effect stronger in Y cells.

t

• Synaptic rectification from Bipolar signal to
  excitatory current in ganglion cells.
• Possible pooling of Bipolar current (for Y
  cells). Demb et al, 01

Ganglion cells
50
STAGE Ganglion Cells
Bipolar cells

• Additional temporal high-pass on bipolar current
• Displaced amacrine cells. Nirenberg et al, 97
• Effect stronger in Y cells.

t

• Synaptic rectification from Bipolar signal to
  excitatory current in ganglion cells.
• Possible pooling of Bipolar current (for Y
  cells). Demb et al, 01

Ganglion cells
51
STAGE Ganglion Cells
Bipolar cells

• Additional temporal high-pass on bipolar current
• Displaced amacrine cells. Nirenberg et al, 97
  
• Effect stronger in Y cells.

t

• Synaptic rectification from Bipolar signal to
  excitatory current in ganglion cells.
• Possible pooling of Bipolar current (for Y
  cells). Demb et al, 01

Spike when , and refractory
period.
Ganglion cells
Two possible noise sources to reproduce the
statistics of (T,N) in ganglion cells. Keat et
al, 01
52
Spikes

• Firing events a coding principle?

Berry and Meister 99

• Spiking responses are generally concentrated in
  firing events
• One event (T,N) ( time of first spike ,
  number of spikes )
• T is a reliable variable gt not Poisson !!
• Better explained by a noisy LIF model (Reich and
  Victor 98, Keat et al. 01)

53
Conclusion

• Functional model with
• Precise spatio-temporal filter
• Plausible contrast gain control
• X and Y cells
• Plausible spike generation
• State-of-the art
• Stages 1 and 3
• Original formulations
• Stage 2
• ASSEMBLED MODEL
• Bio-plausible (simulations).

OPL Current
Bipolar Current
Wohrer et al. 08, JCNS.
54
(First) Simulations
Static grating apparitions (X and Y cells)
X CELL
Y CELL
Cat
Model
Cat
Model
55
(First) Simulations
Drifting gratings
Cat X cell Enroth-Cugell Robson, 66
Model X cell

• Contrast gain control is mandatory to amplify
  third line response (1.1 c/deg)

56
Overview

• Introduction
• Classical Anatomy and Models
• Biological Retina Model
• Gain Control Mechanism
• Virtual Retina Large-Scale Simulator
• Contributions, Perspectives

57
IV. Gain Control Mechanism

• Experimental Validations
• Mathematical Study
• Spatial Equalization

58
Multi-sinus experiment
Log(2)
t
Shapley and Victor, 78

• Observations
• Band-pass behavior
• Contrast gain control
• Underlinearity with contrast
• Phase advance at high contrasts

Amplitude
Phase
59
Multi-sinus experiment

• In our model
• Band-pass behavior from the temporal linear
  filters (amacrine cells, etc.)
• Contrast gain control from the variable
  conductance feedback
• Underlinearity at low temporal frequencies
• Phase advance at high contrasts

Amplitude
Phase
60
IV. Gain Control Mechanism

• Experimental Validations
• Mathematical Study
• Spatial Equalization

61
Mathematical studyof the gain control mechanism

• Forget spatial structure (to reduce
  dimensionality).
• Mathematical analysis for sinusoidal stimulation.
• Theorems see Thesis !

62
Purely temporal model
OPL Current
?
Bipolar Cells
Delay
Q(V)
Adaptation Conductance
63
Formal definition
Can we prove
?
I0
Gain control system ?
Growth with input ?I0Vmax gt 0 Under-linearity
?I0 (Vmax/I0) lt 0 Phase advance ?I0fmax lt 0
system
fmax
Low-pass system ?
Vmax
??Vmax lt 0 and ??fmax gt 0
64
As I0 increases
V
G
t
V
65
As I0 increases
Vmax
Gain control is observed

• Growth with input ?I0Vmax gt 0
• Under-linearity ?I0 (Vmax/I0) lt 0
• Phase advance ?I0fmax lt 0

fmax
V
Mathematical proof ?

• Oscillating system -gt difficult.
• Gronwall
• Theorems, in asymptotic cases.

t
66
As b varies
V
G
V
t
67
As b varies
1D asymptotic limits

• When b -gt 0
• G(t) cst. G0
• Theorem.

G

• When b -gt 8
• G(t) q(V(t))
• V(t) I0 cos(?t) q(V(t))V(t)
• Theorem.

V
Wohrer 08, in preparation. (preliminary INRIA
RR 6733).
68
IV. Gain Control Mechanism

• Experimental Validations
• Mathematical Study
• Spatial Equalization

69
Spatial equalization
OPL Current
Wohrer et al., Neurocomp 06 Wohrer et al. 08,
JCNS.
Bipolar Cells
Feedback through inhibitory conductance
Pooling
Delay
Q(V)
y
t
x
Adaptation Conductance
Q (
70
Spatial equalization
Prediction of our model

• Back to computer vision !
• Experimental validation ?

71
Overview

• Introduction
• Classical Anatomy and Models
• Biological Retina Model
• Gain Control Mechanism
• Virtual Retina Large-Scale Simulator
• Contributions, Perspectives

72
Example of result
On
Off
X
Y
73
Example of result
On
Off
X
Y
74
Virtual Retina

• Up to 100,000 neurons in 100 real time
• Recursive filtering
• Event-driven spike simulation
• Open-source software (APP, CeCILL C license)
• 3 levels of use
• Direct code (C)
• Download/Install full software
• Run it on a web-service

75
C library
Enter the code !

• Synaptic ports and transmissions
• Log-polar schemes and filtering
• Spatio-temporal filtering
• Spiking arrays
• XML serialization

• Generic tools.
• Model extensions.
• Other models.

76
Using Virtual Retina
Choose the level of complexity

• Radial scheme (fovea),
• or rectangular (uniform).
• Contrast gain control,
• or not.
• X/Y cells.
• Spikes, or not.
• Etc.

77
On-line web service
Do it more simply

• For tests / introduction
• to the software
• Beta-version available
• Nicolas Debeissat
• Pierre Kornprobst
• Thierry Vieville

78
Overview

• Introduction
• Classical Anatomy and Models
• Biological Retina Model
• Gain Control Mechanism
• Virtual Retina Large-Scale Simulator
• Contributions, Future Work

79
Contributions

• Wide bibliography
• Detailed retina model
• Mathematical study of the Gain Control loop
• Virtual Retina Tool to study retinal
  processing.

Wohrer et al. 06, INRIA RR 5648 Wohrer 08, in
preparation.
Wohrer et al. 08, JCNS (preliminary version
INRIA RR 6327) (conferences ECVP 05-06, IJCNN
06, Neurocomp 06)
Wohrer 08, in preparation (preliminary version
INRIA RR 6733)
80
Spike profile at image onset
!
measured
model
t30 ms
t90 ms
81
Spike profile at image onset

• Short delay (4 ms) of surround signal as compared
  to center.
• Prediction
• First spikes (one or two) code for the luminance
  signal.
• The next spikes code for edges.

82
Spike profile at image onset
Recent experimental validation !
Gollisch and Meister 08

• T reliably transmits the luminance signal.
• N is more sensitive to image edges (delayed
  surround signal).

83
Future work (I3S and after)

• Study coding / reconstruction / compression
• in the retinal spike representation.

84
Future work (I3S and after)

• Study coding / reconstruction / compression
• in the retinal spike representation.

On
Off
T
N
85
Future work (I3S and after)

• Filter bank approach for reconstruction?

reconstruction
86
Thank you!
This work was partially supported by the EC IP
project FP6-015879, FACETS
Odyssée Lab
87
Future work

• Finish validation of the retina model
• Quantitative links with other models (Bonin 05,
  Keat 01).
• Quantitative links with real data.
• Enhance the retina model
• Light adaptation.
• Color.
• Spike phase locking in populations of cells.
• Other ganglion cells (sluggish).

88
Future work

• Spikes Phase locking in a population of cells

Neuenschwander and Singer 96

• Phase locking only for continuous spatial
  regions.
• Functional implications Threshold detectors,
  segmentation,

89
Future work

• Diversity of ganglion cells

(very) approximate percentages

• Brisk cells
• Brisk sustained ( cat X cell, primate
  Parvocellular? )
• Brisk transient ( cat Y cell, primate
  Magnocellular )
• Small bistratified ( blue-yellow, primate
  Koniocellular )
• Sluggish cells (primate Koniocellular?)
• Concentric sustained (cat Q cell? ), concentric
  transient.
• Local edge detector
• Orientation-selective
• Suppressed-by-contrast ( Uniformity detector
  )
• Direction-selective (DS) cells
• ON-OFF DS cell, ON DS cell,
• Photosensitive

50(cat), 75(primate)
3-8 (all species)
5 ?
gt 10 (cat)
gt 15 (rabbit)
?
?
gt10 (rabbit)
lt 3
90
STAGE Contrast Gain Control

• Functional equation
• Conductance associated to the resting Nernst
  potential.
• Symmetrical, convex function Q.

• Biological plausibility
• Fast CGC is already present in bipolar cells
  (Rieke 01).
• Amacrine cells are not involved in CGC (Rieke 01,
  Beaudoin 07).

• Two possible interpretations of our model
• Voltage-dependent conductances in bipolar cell
  membranes.
• Calcium adaptation in bipolar dendrites.

91
LN Analysis and contrast gain control
Stimulus White noise (reverse correlation
analysis)

• Observations
• Band-pass behavior
• Contrast gain control
• Underlinearity with contrast
• Phase advance at high contrasts
• Fast and slow components

Baccus and Meister, 02
92
LN Analysis and contrast gain control

• Our feedback loop can reproduce fast contrast
  gain control

93
Receptors and phototransduction
94
Receptors and phototransduction

• Phototransduction and light adaptation

Normann and Perlman 79

• Calcium feedback, etc.
• Instantaneous range of 2-3 log units only.
• Time scales of adaptations 100 ms -gt several
  minutes.

95
Linear analysis of ganglion cells

• Electrical models for the OPL

Receptors (outer)

• Receptors, horizontal cells
• Coupled networks of leaky condensators
• Bipolar cells
• Difference between
• Receptors (center signal) and
• Horizontal (surround signal)

Receptors (inner)
Horizontal cells
-

Bipolar cells
Herault 96
Mahowald and Mead 91
96
Contrast gain control

• Temporal model

Van Hateren et al. 02

• Divisive influence on contrast gain.
• Change on the temporal scale of filtering.
• Fast and Slow gain controls.

Shapley and Victor 81, Victor 87
97
Contrast gain control

• Spatial model

Bonin et al. 05

• Standard Deviation of luminance in a spatial
  neighborhood.
• Divisive influence on contrast gain.

98
Spikes

• Recent, LIF-inspired model

• Integrated version of a LIF model.
• 2 sources of noise
• Gaussian auto-correlated process a(t) gt first
  spike T.
• Noisy refractory kernel P(t)
• gt number of spikes N.

T
N
Keat et al. 01
99
Spikes

• Phase locking in a population of cells

Neuenschwander et al. 99

• Correlogram between cell populations i and j
• Cij (t) P( tj ti t ti ) (possibly,
  ij autocorrelogram)
• Phase locking in the population ? Oscillations in
  the autocorrelogram
• Phase locking depends on the size of the light
  stimulus.

100
Spikes

• Phase locking in a population of cells

Neuenschwander and Singer 96

• Phase locking only for continuous spatial
  regions.
• Functional implications Threshold detectors,
  segmentation,

101
Different output pathways

• X cells and Y cells

X CELL
Y CELL
X CELL
Y CELL
Sustained
Transient
Spatially linear
Spatially nonlinear
FORM PATHWAY
MOTION PATHWAY
Enroth-Cugell and Robson 66
102
Different output pathways

• Model of Y cells spatial nonlinearity

Hochstein and Shapley 76 Enroth-Cugell and
Freeman 87
103
Different output pathways

• Y cells spatial nonlinearity

Demb et al 01
104
The Y cell non-linearity

• To reproduce the correct responses of Y cells to
  grating apparitions, three elements are required
  in the right order (fig.B)
• Temporal high-pass
• Rectification
• Spatial pooling

105
Different output pathways

• Diversity of ganglion cells

Roska et al 06
106
When b8
When , Hence the 1D system
with
107
When b8