the hippocampus is thought to be critical for the rapid for

1

Caching and replay of place sequences in a
Temporal Restricted Boltzmann Machine model of
the hippocampus Sue Becker, Dept. of Psychology
Neuroscience Behaviour, McMaster Univ.,
becker_at_mcmaster.ca and Geoff Hinton, Dept. of
Computer Science, Univ. of Toronto
1. Summary
3. Learning in RBMs and TRBMs
5. Training patterns
Place tuning in linear track environment
The hippocampus is thought to be critical for the
rapid formation and cued recall of complex event
memories. For example, a set of hippocampal place
cells that fires in a particular sequence during
exploration may fire in the same sequence during
sleep 1,2,3. We propose a novel model of the
hippocampal memory system that reconciles a wide
range of neurobiological data. Here we address
the question of how the hippocampus encodes
sequences rapidly, and what is the function of
sequence replay. Our model draws upon two recent
developments of the Restricted Boltzmann Machine
(RBM) 1) a hierarchy of sequentially trained
RBM's 4, and 2) the extension to sequential
inputs employed in the Temporal RBM 5. The top
two layers of the model, representing the dentate
gyrus (DG) and CA fields, are connected via
undirected links to form an autoassociator,
allowing the model to generate memories of
coherent events, rather than generating
top-level unit states independently. The CA
region also has directed connections from
previous CA states, representing the CA3
recurrent connections. Thus the probability
distribution learned over the visible and hidden
units is conditioned on earlier states of the
autoassociator. The model is trained by
contrastive Hebbian learning, with data-driven
and generative phases providing the statistics
for the positive and negative Hebbian updates
respectively. This is broadly consistent with
Hasselmo's proposal that the hippocampus
oscillates between encoding and recall modes
within each theta cycle 6. When trained on a
variety of spatial environments (fig 5), the
hippocampal units develop place fields (fig 6),
while the CA time-delayed recurrent collaterals
encode the sequential structure of the data (fig
7). It has been widely assumed that sequence
replay is required for memory consolidation, in
which rapidly stored hippocampal memories are
gradually transferred to cortex. Alternatively,
the hippocampus may always be required to recall
complex associative memories, while replay allows
the brain to maintain consistent forward and
generative models in the hippocampal-cortical
system 7.
3.1 Restricted Boltzmann Machines
Figure 4 Spatial locations are coded by Boundary
Vector Cells (BVCs) 11. Each BVCs tuning curve
is a product of 2 Gaussians, tuned to distance
from and direction to environmental boundaries
Probabilities of states P(V,H) are proportional
to their Boltzmann free energies. In an RBM,
conditional probabilities of H V can be
computed efficiently
Figure 6 Place tuning of several CA units
Sequence replay
X
Figure 5 The model was trained on 3
environments, a linear track, a U-shaped maze and
a rectangular box. Left U-maze, with training
locations shown by blue dots. Right the
reconstruction of the boundaries from the BVC
representation of the location marked X in the
U-maze.
Time
3.2 Contrastive divergence learning 10
Maximizing likelihood requires settling to
equilibrium. Instead, minimize diff. between 2
K.L. divergence terms
2. Data not explained by most HC models
6. Simulation results
Figure 7 Models reconstruction of a sequence of
spatial locations along the linear track.

• Anatomy (fig 1)
• Cascaded architecture
• Reciprocal connectivity btw DGlt--gtCA3 and
  CAlt--gtEC
• Activation dynamics
• (fig 2)
• Theta vs sharp waves
• Sequence replay (fig 3)
• slow during REM sleep
• fast during SWS, sharp waves
• Learning dynamics
• LTP in phase w theta rhythm
• LTD in anti-phase w theta rhythm

Input reconstruction
3.3 Temporal Restricted Boltzmann Machines
7. Discussion
C2
This is one example of a class of models called
TRBM discussed in 5.
The model learns spatial environments gradually,
but can rapidly cache state sequences in its
temporal generative model (CA recurrent
collaterals) Making the temporal connections
bi-directional requires propagation of states
backwards in time, accounting for reverse
sequence replay. The addition of sparseness
constraints sharpens place-tuning and leads to a
grid-like representation
C1
Figure 1. Cascaded architecture of the HC.
W
Figure 2. Left Theta-modulated gamma ripples
(fig 2A from Chrobak et al 2000) 8. Right
Sharp waves (fig 1 A, C from Chrobak Buszaki,
1994). 9 Reproduced with permission.
8. References
1 Wilson, M.A. and McNaughton, B.L. (1994),
Science 265(5172)676-679 2 Louie, K Wilson,
MA (2001), Neuron 145-156 3 Lee, A.K. and
Wilson, M.A. (2002) Neuron 36(6)1183-1194 4
Hinton, G. E., Osindero, S. and Teh, Y. (2006),
Neural Computation 18(7)1527-1554. 5
Sutskever, I. and Hinton, G. E. (2006) Technical
Report UTML TR 2006-003. 6 Hasselmo, M.E. et
al (2002), Neural Computation, 14(4)
793-817. 7 Kali, S. and Dayan, P. (2004)
Nature Neuroscience 7(3)286-294 8
Chrobak,J.J., Lorincz,A. Buszaki,G. (2000)
Hippocampus 10(4)457-65 9 Chrobak Buszaki,
(1994) J Neurosci 146160-6170 10 Hinton, G.E.
(2002) Neural Computation, 14(8)17111800 11
Hartley, T., Burgess, N., Lever, C., Cacucci, F.
and O'Keefe, J. (2000), Hippocampus 10(4)369-379
4. A TRBM model of the hippocampus
Reproduced with permission from Fig 2 Lee
Wilson 2002.3
Figure 3. Top Smoothed raster plots of spikes vs
sequentially traversed spatial locations
(5-second time scale). Bottom Same sequence of
6 place cells fires during slow wave sleep
(150msec time scale).
Figure 5 Examples of boundary reconstructions
from BVC input patterns (left) and from models
reconstruction of input (right) averaged over 5
binary stochastic samples
Figure 4. Simplified HC architecture used here.
This work was supported by funding from the
Natural Sciences and Engineering Research Council
of Canada (SB, GEH) and CIAR (GEH)