Navigation and Cognitive Map Formation using Aperiodic Neurodynamics

1
Navigation and Cognitive Map Formation using
Aperiodic Neurodynamics

• Derek Harter, Robert Kozma
• University of Memphis
• July 15, 2004

Simulation of Adaptive Behavior 2004
2
K-Sets

• Neural Population Model
• Model aperiodic dynamics observed in olfactory
  system.
• K-Sets
• KA-Sets
• discretization of K-Set ODE
• speed, analyzability,

Skarda Freeman (1987). How brains make chaos in
order to make sense of the world. Behavioral and
Brain Sciences, 10 161-195. Freeman (1991). The
physiology of perception. Scientific American,
264(2)78-85.
Harter Kozma (under revision). Chaotic
neurodynamics for autonomous agents. IEEE
Transactions on Neural Networks.
3
Modeling Olfactory Dynamics Perceptual
Categorization
(Freeman 1986)
4
K Set Hierarchy
KA-Ie/i
KA-II
E1
E2
I1
I2
KA-III
5
K-Ie
KI Populations of Fixed point convergence connecte
d (excitatory) excitatory or to zero or nonzero
value, periglomerular cells inhibitory
units positive feedback
KII Interacting Periodic, limit cycle connections
of excitatory populations of oscillations,
frequency in populations with inhibitory excitato
ry and the gamma band, populations, such as in
the inhibitory units negative feedback olfactory
bulb.
K0 Excitatory
E1




E2
K0 Excitatory
6
K-II
E1
I1
KI Populations of Fixed point convergence connecte
d (excitatory) excitatory or to zero or nonzero
value, periglomerular cells inhibitory
units positive feedback

-
-

E2

KII Interacting Periodic, limit cycle connections
of excitatory populations of oscillations,
frequency in populations with inhibitory excitato
ry and the gamma band, populations, such as in
the inhibitory units negative feedback olfactory
buld.
I2
-
-


-
7
K-III
KI Populations of Fixed point convergence connecte
d (excitatory) excitatory or to zero or nonzero
value, periglomerular cells inhibitory
units positive feedback
KII Interacting Periodic, limit cycle connections
of excitatory populations of oscillations,
frequency in populations with inhibitory excitato
ry and the gamma band, populations, such as in
the inhibitory units negative feedback olfactory
bulb.
KIII Several interacting Aperiodic,
chaotic Cortex, Hippocampal KII and KI
sets oscillations Formation, Midline Forebrain
8
K-III
KI Populations of Fixed point convergence connecte
d (excitatory) excitatory or to zero or nonzero
value, periglomerular cells inhibitory
units positive feedback
KII Interacting Periodic, limit cycle connections
of excitatory populations of oscillations,
frequency in populations with inhibitory excitato
ry and the gamma band, populations, such as in
the inhibitory units negative feedback olfactory
bulb.
KIII Several interacting Aperiodic,
chaotic Cortex, Hippocampal KII and KI
sets oscillations Formation, Midline Forebrain
9
K-III
receptors
OB
KI Populations of Fixed point convergence connecte
d (excitatory) excitatory or to zero or nonzero
value, periglomerular cells inhibitory
units positive feedback
d
KII Interacting Periodic, limit cycle connections
of excitatory populations of oscillations,
frequency in populations with inhibitory excitato
ry and the gamma band, populations, such as in
the inhibitory units negative feedback olfactory
bulb.
KIII Several interacting Aperiodic,
chaotic Cortex, Hippocampal KII and KI
sets oscillations Formation, Midline Forebrain
AON
d
PC
10
KA, K-Set and Rat Power Spectra
KA-III model Layer 1
11
Summary

• K and KA sets are a neural population model
• originally developed to model and understand
  chaotic dynamics observed in olfactory perceptual
  system
• Aperiodic dynamics are normal background state of
  olfactory sense
• Upon recognition of an order (categorization)
  falls into a new (lower dimensional) chaotic
  attractor
• Aperiodic dynamics offer advantages in forming
  perceptual categorization
• speed of convergance
• flexibility (randomness)
• Use aperiodic dynamics to form perceptual
  categorization and action selection in agents.

12
Hippocampal Simulation and Formation of Place
Cells using a KA-III
13
(No Transcript)
14
Hebbian Modification
CA2 (KA-II) (1)
L1
DG (KA-0) (8x8)
L2
CA3 (KA-II) (8x8)
CA1 (KA-II) (8x8)
Orientation Beacons
L3
L4
15
Formation of AM Pats in KA-III
c
a
d
b
0.5s of CA1
(Loc 1) (Loc 2) (Loc 3) (Loc 4) (Loc 5) (Loc
6) (Loc 7) (Loc 8)
(Test d) (Test c) (Test b) (Test a)
16
Comparison of Closest AM Pattern
1-a 1-b 1-b 1-a 1-c 1-b 1-d 1-a 2-a 2-d 2-b 2-c 2-
c 2-b 2-d 2-b 3-a 3-b 3-b 3-a 3-c 3-b 3-d 3-a 4-a
4-c 4-b 4-d 4-c 4-a 4-d 4-b 5-a 5-b 5-b 5-a 5-c 5-
d 5-d 5-c 6-a 6-b 6-b 6-d 6-c 6-b 6-d 6-b 7-a 7-d
7-b 7-d 7-c 7-d 7-d 7-a 8-a 8-c 8-b 8-d 8-c 8-a 8-
d 8-c
a
c
d
b
b
c
d
a
17
Appetitive/Aversive Behavior using a KA-III

• Harter, D. and Kozma, R. (2004).   Aperiodic
  Dynamics for Appetitive/Aversive Behavior in
  Autonomous Agents. Accepted in  Proceedings of
  the 2004 IEEE International Conference on
  Robotics and Automation (ICRA).

18
3
1
Environment Key
Edible food source
1
1
Poisonous food source
Agent Morphology
1
3
Distance sensor Light sensor
Front
Wheel Motor
2
2
19
3
1
Environment Key
Edible food source
1
1
Poisonous food source
Agent Morphology
1
3
Distance sensor Light sensor
Front
Wheel Motor
2
2
20
Architecture of Appetitive/Aversive Experiment
Mapp
Mave
DS0
DS1
DS2
DS3
DS4
DS5
LS0
LS1
LS2
LS3
LS4
LS5
LS6
LS7
Left Obs
Right Obs
No Obs
Left Grad
Right Grad







-
-
-
Turn Left
Turn Right
Move Fwd
Left App
Right App
Left Ave
Right Ave
-
-
-
-
-
-



-


-



-
Front
2
3
1
4
0
5
LM
RM
Wheel Motor
7
6
Distance sensor Light sensor
21
Architecture of Appetitive/Aversive Experiment
Touch (KA-I 5)
Distance (IR) (KA-I 8)
Smell (Light) (KA-0 10)
KA-III
OB (8x8 KA-II)
AON (8x8 KA-II)
V
PC (8x8 KA-II)
Mapp
Mave
Msearch
Valence
Hebbian Modification
Tasteapp
Tasteave
22
Agent PathNo Learn (left) and Learn (Right)
Environment Key
Edible food source
1
1
Poisonous food source
1
3
1
3
1
3
1
3
2
2
2
2
23
App/Ave Results
24
Experiments Summary

• Aperiodic dynamics can be shaped to form location
  categorization in a Hippocampus-like
  architecture.
• Consistent with so called place cell phenomena
  of hippocampus.
• Aperiodic dynamics can be used to build plastic
  control systems that learn categories and
  associate them with behavioral responses.
• How aperiodic dynamics in hippocampal formation
  of remembered locations might be used to do
  goal-directed navigation still an open question.

25
Evolution of Intentional Deliberative Behavior

• 3.5 billion years single-cell entities
• 550 million years fish vertebrates
• 430 million years insects
• 370 million years reptiles
• 330 million years dinosaurs
• 250 million years mammals
• 120 million years primates
• 18 million years great apes
• 2.5 million years man
• 5000 years writing

• Basic Limbic System
• Primitive Hippocampus
• Long-term memory
• Beyond stimulus/response
• Episodic Memory
• Cognitive Maps

26
Limbic System Simplest Complete Intentional
System

• Far from equilibrium, therodynamic systems
• Mechanisms of self-organization
• competition
• cooperation
• autocatalytic loops
• hierarchy mesh
• Aperiodic dynamics
• Expectation or reafference
• Embodiment
• environment/organism coupling
• Small worlds type divergent-convergent systems
  connections

(Harter Kozma 2004)
27
Talk Conclusions

• Aperiodic dynamics are the norm in biological
  brains.
• Result of intrinsic population effects (as well
  as external noisy stimulation).
• May be useful properties for perception.
• Such dynamics are being explored to determine
  usefulness in biological and artificial systems.

28
?0.058
?0.-044
KA-III
?0.0109
?0.152