Animal Cognition I

1
Vision-Based Place Recognition and Cognitive
Mapping
Hanspeter A. Mallot Faculty of Biology
andWerner-Reichardt-Center for Integrative
NeuroscienceUniversity of Tübingen
2
Spatial knowledge and problem solving The
"Means-Ends-Field"
Edward C. Tolman (1886-1959)

• Nodes
• Places, goal, intermediate goals
• Sensory cues characterizing places
• States
• Edges
• Actions
• "Means-Ends-Expectations"
• Schemata lts,r,s'gt

E.C. Tolman, Purposive Behavior in Animals and
Men, The Century Comp. 1932, p 177
3
Place Recognition
4
Landmarks and Places
Snapshot approach Landmarks are image features
that characterize the place from which they are
viewed. They need not correspond to objects in
the world (view axes!).
Localized cues approach Landmarks are object
features that have a position (coordinates) in
the environment.
5
Snapshot Homing Accuracy
sensor 3
sensor 2
sensor 1
Image manifold I(u,vx,y,f) Local image variance
Catchment area homing possible Area of
uncertainty no further approach possible Size of
both areas proportional to local image variance
6
Circular Color Texture
Sinusoidal intensity modulation per color channel
Image manifold
7
Snapshot-based homing in humans

• Subject with HMD in 5.2 x 6 m tracked walking
  arena
• Circular room with homogeneous color gradient
• Task
• Subject at position 1
• View scene at position 2
• Walk to position 2
• View scene at position 3
• Dependent measure trajecory, homing error

8
Sample Trajectories and Viewing Directions

• a-d subjects head towards goal from the
  beginning
• right heading directions
• top small room, subject quickly finds goal
  directions
• bottom large room, subjects looks around and
  then starts moving towards the right direction.
• General performance is good. Residual homing
  error is well below chance level.

Gillner S, Weiß A, Mallot HA, Cognition
9
Place recognition models

• g goal
• p current place
• c place code (vector of local position
  information)
• S(g,p) comparison function (radial basis
  function)
• confusion area

10
Image Comparison Model c(p) I(?p)
tangential var
radial variance

• Circular confusion areas
• Size decreases with l.i.v.
• Weak dependence on eccentricity

11
Closest Wall Segment Model c(p)
(1-p,I(arg(p))
tangential var
radial variance

• Shape of confusion areas depends on l.i.v.
• Size decreases with contrast
• Size depends on eccentricity

12
Boundary Vector Cell Model

• Geocentric room axis (e.g., red cyan)
• "Boundary vectors" sample wall distance in a
  number of directions
• Radial error distance uncertainty
• Tangential error axis (color) uncertainty

Barry C, et al. (2006) Rev. Neurosciences 1771-79
13
Boundary Vector Model
tangential var
radial variance

• Confusion areas do not depend on contrast
• Shape depends on eccentricity

14
Dependence on Color Modulation Gillner S, Weiß
A, Mallot HA, submitted
homing in 6 subjects, 4 repetitions
prediction from squared image difference algorithm
Color modulation 10
Color modulation 100
15
Dependence on Color Modulation Gillner S, Weiß
A, Mallot HA, submitted
Model Prediction

• Overall signifi-cant effect of color modulation
• Per point, effect is significant only for the
  three peripheral points
• Threshold effect for higher modulations

Experimental Data
16
Dependence on Room Size Gillner S, Weiß A,
Mallot HA, submitted
Model Prediction
S room (diameter 4.5 m)
Experimen-tal Data
XL room (diameter 27 m)

• Human visual homing in featureless environment
  depends on contrast and room size, i.e. on local
  image variation, l.i.v.

17
Metric Embedding of Place Graphs
18
Metric Embedding of Place Graphs
path integration
metric embedding
actual position
represented position

• place recognition by panoramic view comparison
• view graph for topological navigation
• egomotion estimates from odometry or optical flow
  
• metric embedding of view graph by modified MDS

19
MDS and Metric Embedding
Embedding X
Optimization Q
x2
x3
x4
x1
dot product match leg projection
vector productmatch triangle area
Hübner Mallot, Autonomous Robots 2007
20
Embedded Place Graph
residual error
route planning on subgraph
Hübner Mallot, Autonomous Robots 2007
21
Route following
Path integration error corrected by visual homing
Replanning after getting lost at an obstacle
Hübner Mallot, Autonomous Robots 2007
22
Metric Knowledge in Human Longterm Memory

• Foo P, Warren WH, Duchon A, Tarr MJ. J Exp.
  Psychol LMC 31195-215,2005
• Little evidence for metric embedding of locally
  learnt segments.

23
Metric Knowledge in Human Longterm Memory

• Foo P, Warren WH, Duchon A, Tarr MJ. J Exp.
  Psychol LMC 31195-215,2005
• Little evidence for metric embedding of locally
  learnt segments.
• Ishikawa T, Montello D, Cognitive Psychology
  5293-129 (2006)
• No improvement of metric performance over
  sessions.

24
Scaleable model of spatial memory The
view-graph approach

• Place recognition
• Topological navi-gation Routes (chains) and
  maps (graphs)
• Local metric information
• Metric embedding?
• Regions and route planning

25
Cognitive Neuroscience Lab

• Rodent behaviour
• Johannes Thiele
• Alexandar Jovalekic
• Okuary Osechas
• Phillip Schwedhelm
• Berna Ertas
• Martin Seitz
• Human behaviour
• Gregor Hardiess
• Dagmar Schoch
• Wolfgang Röhrig
• Stephan Storch
• Geraldine Hopf

• Robots and Models
• Chunrong Yuan
• Kai Basten
• Fabian Recktenwald
• Stefan Blazcek
• Deniz Bahadir
• Isabelle Schwab
• Hansjürgen Dahmen
• Staff
• Heinz Bendele
• Martina Schmöe-Selich
• Annemarie Kehrer