Neural Visuomotor Controller for a Simulated Salamander Robot - PowerPoint PPT Presentation

Loading...

PPT – Neural Visuomotor Controller for a Simulated Salamander Robot PowerPoint presentation | free to view - id: 74481-ZDc1Z



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Neural Visuomotor Controller for a Simulated Salamander Robot

Description:

Neural Visuomotor Controller for a Simulated Salamander Robot. Biljana Petreska ... The salamander has difficulties with following the prey stimulus as the amount ... – PowerPoint PPT presentation

Number of Views:47
Avg rating:3.0/5.0
Slides: 41
Provided by: petr5
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Neural Visuomotor Controller for a Simulated Salamander Robot


1
Neural Visuomotor Controller for a Simulated
Salamander Robot
  • Biljana Petreska
  • Diploma Thesis March 2004
  • Responsible
  • Prof. Auke Jan Ijspeert

2
Goals of the Project
  • Investigate through simulations tightly coupled
    with neurobiological data, the neural mechanisms
    underlying visually guided behaviour in
    amphibians
  • Implement a closed-loop with the environment onto
    the existing neuromechanical simulation developed
    by Ijspeert, by adding biologically inspired
    models for parts of the salamander brain
  • Develop a controller that accounts for
    observations in feeding behaviour, including prey
    localization and prey recognition
  • Study a model proposed by Ijspeert of structured
    mapping between the optic tectum (primary visual
    processing center) and the brain stem (motor
    centers) as a solution to the visuomotor
    coordination

3
Interests
  • Relevant for perceptual robotics
  • decoding the brain processes, assigning meaning
    to complex patterns of sensor stimuli may lead to
    the solution of many robotics tasks
  • Test bed for probing neurobiological
    contributions
  • ideal for the validation or refutation of new
    theories

4
Overview
  • Short introduction on relevant topics and
    previous works
  • Implemented Models
  • Respective Results
  • Conclusion and Future Work

5
Everything youve always wanted to know on
Salamanders
  • Amphibians
  • Great variety of species (3924 indexed so far),
    sizes (from 16mm to 1.5m), aspects and lifestyles
    (terrestrial and/or aquatic).
  • A relatively simple neural circuitry that
    presents all main vertebrate features
  • Tractable from an experimental point of view an
    important amount of behavioral, biological and
    neurological data exists

6
Visually Guided Behavior
  • Vision is by far the most important feeding
    guiding sense. Under good visual conditions the
    other signals such as olfactory are overridden
  • Feeding strategies (some species can switch from
    one to another)
  • hunter strategy active search for prey.
    Prerequisites are a short massive tongue and poor
    visual capacities.
  • ambush strategy wait until prey comes close.
    Prerequisites are a highly specialized projectile
    tongue (up to 80 body length), evolved visual
    system and frontally oriented eyes

7
Visually Guided Behaviour
  • Sequence of feeding behavior
  • orienting
  • approach
  • olfaction tests
  • gaze stabilization
  • snapping
  • Prey preferences (in order of importance)
  • stimulus size
  • stimulus velocity
  • stimulus-background contrast
  • stimulus shape
  • movement pattern
  • experience-dependant

8
Morphology of the Salamander Brain
  • Functional differentiation of the brain
    structurally different regions accomplish
    different tasks
  • Global top to bottom visual information
    processing
  • Principal components photoreceptors, retina,
    optic tectum, nucleus isthmi, pretectum,thalamus,
    medulla oblongata and brain stem

9
Retinal Ganglion Cells
  • First layer of visual processing, transfers
    visual signals to the brain via the optic nerve
  • 3 Types of retinal ganglion cells that project to
    particular layers in the optic tectum

10
Optic Tectum
  • Main visual processing center. Integrates also
    multimodal perception, such as ascending
    somatosensory, auditory, olfaction and vestibular
  • Stratification in 9 layers, first three are
    retinal afferents
  • Six morphological neuron types identified (one
    interneuron)
  • Topographic representation of the visual field
  • Viewed as a set of partial overlapping maps due
    to the different types of tectal projection
    neurons
  • Number of tectal cells in Hydromantes Italicus
    92 000 and 3300 out of 5000 projection neurons
    are descending
  • Projection patterns from and to the retina,
    pretectum, thalamus, nucleus isthmi and medulla
    (reaching the spinal cord)

Distribution and Receptive Field Sizes of Tectal
Neurons in H.Italicus
11
Pretectum
  • Has been ascribed a role in optokinetic
    nystagmus, figure-background discrimination,
    pupillary reflex, fixation, phototaxis, and
    prey-enemy distinction.
  • Properties of pretectal neurons
  • Homogenous arborisation
  • (no classification was possible)
  • Divergent projections
  • (including to the tectum and spinal cord)
  • Large receptive fields
  • Receive direct and indirect (from tectum) retinal
    input
  • Direction-sensitive neurons (predominantly in
    temporonasal direction)
  • Respond to stimuli in the contralateral visual
    field

12
Lesion Experiments
  • Give insight of the function of the destroyed
    brain region
  • Lesion of the optic tectum both visual
    prey-catching and predator avoidance fail to
    occur. Local lesions produce scotoma, total
    blindness for a part of the visual field
    corresponding to the size of the lesion.
  • Lesion of the pretectum locally facilitates
    feeding and abolishes prey-predator
    discrimination, attack everything that moves
    including their own extremities and threatening
    stimuli
  • Lesion of the thalamus unable to avoid collision
    to a vertically stripped barrier, affects the
    binocular field
  • Lesion of the medulla oblongata affects
    distance, elevation or horizontal eccentricity
    estimates, overshoots prey or snaps only in
    frontal positions gt different components of the
    stimulus position are handled through different
    pathways
  • Difficulty in some cases the animal recovers
    shortly after the lesion and the relative
    precision of lesions may induce errors

13
Previous Works
  • Based upon the principle of coarse coding (Eurich
    et al, 1997)
  • Motivation the high sensory resolution observed
    in nature seems incompatible with the large size
    of receptive fields of tectal neurons
  • Definition population-coding using mapping
    combinatorics of intersecting receptive fields
  • A non-firing neuron conveys as much information
    as a firing neuron. All neurons participate at
    the information coding
  • Weakness likely to suffer from metamery
    (convergence of information channels)
  • Simulander I
  • Feedforward network with only 100 neurons,
    trained by an evolution strategy for the specific
    task of head orienting (implies prey
    localization)
  • Distribution and sizes of receptive fields of
    tectal neurons are respected and firing rates
    have been adapted
  • Unstructured mapping follows the prey with high
    accuracy
  • Simulander II
  • Similar to Simulander I, but trained for the
    specific task of frontal tongue projection
    (implies depth perception)

14
Addressed Questions
  • How can the stimulus location and depth estimates
    be extracted from the tectum maps?
  • What sensorimotor transformations occur at the
    level of the optic tectum, the brainstem and the
    pathways between them? Can a structured mapping
    provide an accurate visual tracking?
  • Which type of a tectum-brainstem mapping explains
    the typical curved approach in monocularized
    salamanders?
  • How is the visual perception influenced by head
    motion during the approach toward a stimulus? Are
    additional mechanisms necessary for dealing with
    the remaining shifts in the visual background?
  • Which mechanism implements the release of the
    snapping behavior? And how is the tongue
    controlled?

15
Neural Networks
  • Restrictions (performance motivated)
  • Uniformly distributed neuron units
  • Square receptive fields
  • Specification
  • Center receptive field (in degrees of visual
    field) determines the size of the neural
    network
  • Surround receptive field (in degrees of visual
    field) determines the overlap and redundancy
    feature
  • Weights matrix, activation function and
    thresholds
  • Features
  • Reduction (biologically motivated)
  • Visualization (extremely practical)

16
Eyes of the Simulated Salamander
  • Virtual cameras extract views using provided
    OpenGL functions
  • Correct the view using a spherical projection
  • Photoreceptors are equivalent to pixel grey
    values
  • Scalable visual field

17
Retinal Ganglion Cells of Type 1
  • Properties
  • small size excitatory (2-3) and strong
    inhibitory (12-16) receptive fields
  • no response to change in light
  • involved in local contrast calculation gt edge
    detector
  • project to the contralateral spinal cord gt
    obstacle avoidance?
  • give rise to a fine grained representation of the
    visual field in the retina
  • Modelled with the laplacian of a gaussian filter
  • Classic edge detector in computer vision and
    confirmed by the study of a larval tiger
    salamander retina receptive field

18
Retinal Ganglion Cells of Type 2
  • Type 2 retinal ganglion cells respond only to
    moving objects gt motion detectors
  • Detection of change compare the corresponding
    pixels at different times, using a linear
    difference function
  • where t is a threshold, j and k are moments in
    time, x and y are the pixel positions in the
    frame
  • Biological inspirations
  • Reflects signals with delayed pathways that give
    rise to a simultaneous representation of the same
    object at different times in the brain
  • Flat weights the activity sharply increases when
    an object enters the receptive field variation
  • Tectal neurons are contrast-sensitive linear
    function

19
Retinal Ganglion Cells of Type 3
  • Properties
  • large receptive fields (10-20)
  • tonic response to change in light intensity
  • respond to overall luminosity (dimming detectors)
  • respond also at low contrast and velocity
  • Model
  • flat weights
  • simple summing network
  • Predator detectors among other

20
Optic Tectum Model
  • Principal biological inspirations
  • Retinotopic map in the optic tectum electrical
    stimulations result in turning movements that
    roughly correspond to this map
  • Only two synapses between the retina and the
    brain stem the tectum directly projects onto the
    brain stem
  • Input retinal ganglion cells of type 2. Motion
    is a necessary prerequisite for a stimulus to be
    interpreted as prey
  • Structured mapping different strengths along the
    rostro-caudal axis, reflects the stimulus
    eccentricity
  • Motoneuron activation function (integrating
    weighted tectal activity)
  • where x is the change in light intensity of the
    pixel at positions i and j
  • Linear weights function
  • where a and ß are parameters

21
Optic Tectum Model II
  • Version with ipsilateral input
  • (contribution from both eyes)

22
Optic Tectum Model III
  • Normalizing tectal activity. The modified model
    is robust to changes in the stimulus parameters
    and visual scene
  • Biological reference
  • TO4 neurons, arborize in RGC2
  • TO2 neurons, large receptive fields
  • both project to the nucleus isthmi

23
Pretectum Model
  • Large stimuli based on RGC3 gt dimming detectors
    with three times larger receptive fields
  • Motion compare direct and indirect (via tectum)
    RGC3 responses
  • Direction-sensitive neurons
  • Why temporonasal sensitivity?
  • Based upon separating the ON and OFF channels
  • Hypothesis only sensitive to dark objects
    (biologically consistent)

24
Snapping Model
  • Relevant for depth estimation
  • Tongue mechanism (biologically consistent) 4
    muscles, protraction and retraction times
    modulated by the stimulus position
  • We proposed a mechanism for frontal snapping
    based upon divergent projections of the tectal
    neurons

25
Results
  • Find optimal a and ß parameters of the linear
    weights function through an exhaustive search of
    the parameter space
  • Cost function difference between the stimulus
    direction and the salamander orienting movement
  • Experimental conditions
  • Ewert experiment the stimulus is moved on a
    semi-circular trajectory with a constant speed in
    front of the animal
  • (task of head orienting)
  • Body muscles were inhibited (only neck muscles)
  • The stimulus parameters (size, speed, distance,
    ...) and network parameters (number of neurons)
    were fixed according to values found in
    literature. Both single stimulus and complex
    background were used

26
Optimal Values
  • Many combinations of values give similar results
  • Good results are also achieved without
    ipsilateral input

27
a and ß Parameters
  • Regular parameter space. With different fixed
    values the aspect is conserved and the minimal
    error area (in black) is shifted
  • ß parameters are not essential, the minimal error
    area is centered in point (0,0)

28
Performance Results
  • An accuracy of less than 3 (real value) is
    achieved for small stimulus velocity values with
    20000 RGC2 and 2000 (less than 3300) tectal
    neurons.
  • Robustness stable reaction to change in
    stimulus parameters and visual scene

29
With Complex Background
  • The salamander has difficulties with following
    the prey stimulus as the amount of noise is
    considerable. It discriminates between objects
    with same apparent angular size, however orients
    at "average flies"
  • The model should be coupled with a selective
    visual attention mechanism (enhanced retinal
    signals in the area containing the prey stimulus)
    and/or optokinetic or vestibucollic image
    stabilization reflexes (antagonistic head
    movements that compensate for body undulations)
  • Integrating approach is trivial with a unique
    prey stimulus

30
Pretectum
  • The salamander discriminates between a small prey
    object and a large predator object
  • When the pretectum is abolished, escape behavior
    fails to occur
  • Delayed response the salamander escapes for a
    longer time than the predator is visible
  • Weakness based upon angular size, close prey may
    be interpreted as predator. Therefore the
    threshold is essential (arbitrary as no data
    exists on predation)

31
Snapping
  • No additional neurons, based upon divergent
    patterns of tectal neurons projections
  • Consistent with biological lesion data
  • codes for closeness
  • realistic precision (about 30)
  • Depends on the movement direction

32
Reproduced Phenomena
  • Lesion and stimulation experiments
  • Lesion and stimulation of the optic tectum
  • Lesion of the pretectum
  • Generation of saccadic movements
  • Monocularized salamanders
  • Prey preferences

33
Saccadic Movements
  • Pursuit movements such as the head accelerates
    for a few seconds, until maximum velocity is
    reached, and then is released
  • We attribute them to the tectal cells resolution

34
Monocularized Salamander
  • With one eye covered, H.Italicus shows a
    conspicuous approach behavior toward a prey
    stimulus. It takes a curved path and bends its
    body toward the side of a seeing eye,
    compensating by turning the head between 60 and
    90

35
Monocularized Salamander
36
Prey preferences
  • All preferences are inherent to the network!!

37
Comparison to Previous Works
  • Simulander I
  • More neurons, but still biologically plausible
    (2000 vs. 100)
  • Less accurate, more realistic (2-6 vs. 1)
  • Inherent preferences vs. a function reflecting
    the stimulus size and velocity (corresponds to
    the observers knowledge)
  • No real distribution of tectal neurons, respected
    in Simulander
  • Faster reaction
  • No positions in which stationery prey elicit
    orienting behavior
  • Simulander II
  • Lower precision, but more realistic (90, real
    success rate 40)
  • In Simulander far objects elicit more activity,
    double inconsistency (should code for closeness
    and further objects seem smaller)

38
Response to questions
  • Extraction of stimulus localization and depth
    estimates can be achieved with a structured
    mapping between the optic tectum and the brain
    stem
  • The sensorimotor transformation of the horizontal
    angular distance of the tectum neurons to muscle
    activity can provide an accurate prey
    localization.
  • Direct observation of the influence of head
    motion during the approach is provided.
    Additional mechanisms for dealing with the
    self-motion visual shifts are necessary
  • The investigated tectum model accounts for the
    typical curved approach in monocularized
    salamanders
  • A plausible mechanism that acts as a releaser for
    the snapping behavior is proposed

39
Conclusion
  • We have implemented models of the three types of
    retinal ganglion cells, the optic tectum, the
    pretectum and a tongue projection mechanism that
    account for the typical feeding sequence and
    escape behavior
  • The optic tectum model reproduces many
    experimental data
  • Everything is observable
  • Warning data and methodology dependant
  • Our salamander resembles a newly born salamander
    thrown in the world

40
Future Work
  • Study a tectum model with nonlinear weights
    functions
  • Use the real distribution and receptive fields
    sizes of tectal neurons
  • Time-dynamics vs. discrete time steps
  • Study the effect of overlapping fields
    (redundancy gt error resistant, maybe emerging
    properties)
  • Implement a visual attention model
  • Implement experience-based models such as
    habituation
  • Further development of the pretectum model
  • Extend the model to other brain areas such as the
    nucleus isthmi or thalamus (obstacle avoidance)
  • Develop a more elaborate model for depth
    estimation (not only frontal)
  • Work on an object-background discrimination with
    respect to self-motion shifts of the visual input
About PowerShow.com