Feedback, Adaptation, Learning or Evolution: How Does the Brain Coordinate and Time Movements?

1
Feedback, Adaptation, Learning or Evolution
How Does the Brain Coordinate and Time Movements?

• Amir Karniel
• Department of Biomedical Engineering
• Ben Gurion University of the Negev

The studies presented were done in collaboration
with Gideon Inbar, Ronny Meir, and Eldad
Klaiman - Technion Sandro Mussa-Ivaldi -
Northwestern University
The first workshop of THE CENTER FOR MOTOR
RESEARCH December 18-21, 2003
2
Outline

• The Hierarchy of Wide Sense Adaptation Equilib
  rium trajectories and internal models Reaching
  movements muscle models and adaptation
• Adaptation to Force Perturbations Time
  representation Sequence learning and switching
• Bimanual Coordination Symmetry at the
  perceptual level as an invariant feature
  Tapping experiments and first indications for
  internal models
• Summary and Future Research

3
Two Important Concepts in the Theory of Motor
Control
Equilibrium
Inverse Model
Feldman Bizzi et al. Minimum Jerk, Flash and
Hogan Force fields, primitives, Mussa-Ivaldi
Albus (cerebellum) Inbar and Yafe (signal
adaptation) feedback error, Kawato distal
teacher, Jordan
Adaptation Change of Impedance Change
of the inverse
4
Reaching movements

• Feed-Forward Control
• Invariant Features Roughly straight line,
  bell shaped speed profile (Flash Hogan
  1985)
• Key Questions
• What is the origin of the invariance ?
• How do we handle external perturbations ?

5
A Hill-type mechanical muscle model The viscose
element B is not a constant !
6
Linear Vs. Nonlinear Muscle Model

• Linear model

The nonlinear Hill-type model
The physiologically plausible nonlinear model can
produce the typical speed profile with a simple
control signals Karniel and Inbar (1997) Biol.
Cybern. 77173-183
7
Other typical features of rapid movements are
also facilitated by the nonlinear muscle
properties
In this set of simulations the one-fifth power
law model was used.
Karniel and Inbar (1999) J. Motor Behav.
31203-206
8
Adaptation to force perturbations

• Force field exposure ? recovery of
  unperturbed pattern
• Removal of field ? after-effects
• (Shadmehr Mussa-Ivaldi 1994)

Modified with permission from Patton and
Mussa-Ivaldi
9
Hierarchical system with feedback adaptation and
learning
Internal models for control
Desired Target
Adaptation
Learning
Dynamics determine the control signal (e.g., EPH,
CPG, )
Feedback
Musculoskeletal system
Actual Performance
10
The hierarchy of wide sense adaptationKarniel
and Inbar (2001), Karniel (In preparation)
Change Scale
Structural Change
Evolution
Learning
Functional Change
Parameters Change
Adaptation
Time Scale
Feedback
No Change
mSec Minutes Years Myears
11
Outline

• The Hierarchy of Wide Sense Adaptation Equilib
  rium trajectories and internal models Reaching
  movements muscle models and adaptation
• Adaptation to Force Perturbations Time
  representation Sequence learning and switching
• Bimanual Coordination Symmetry at the
  perceptual level as an invariant feature
  Tapping experiments and first indications for
  internal models
• Summary and Future Research

12
What are the limitations of adaptation?
Key Questions
13
Time Representation
?

• These systems are indistinguishable therefore
• The existence of time variable isnt sufficient
  to define time representation.
• It is sufficient to consider the following form

14
Time Representation - Definition

• The system is said to be capable of time
  representation if there exists a deterministic
  function h(x) such that for any
  u(t).
• The system is said to be capable of time
  representation of up to T seconds with e accuracy
  if there exists a deterministic function h(x)
  such that for tltT and for
  any u(t).

15
The experiment
Number of movements 100 500
100
Null Learning Generalization
No external field External Force
field time/state/sequence dependent
16
Time Varying Force Field
The force field is not correlated with the
movement initiation, therefore there is no way to
use state information. Only time representation
would allow adaptation and after-effects for this
field.
17
Result No adaptation to this TV force field
The maximum distance from a straight line during
learning
A control experiment with the viscous curl field
Karniel and Mussa-Ivaldi (2003) Biol. Cybern.
18
Viscous Curl Force Field
19
Result There is Significant Adaptation with This
Sequence of Force Fields
The maximum distance from a straight line during
learning
A control experiment with the viscous curl field
20
Direction Error Calculation
1. Find the Euclidean distance from a straight
line at the point of maximum velocity(The
feed-forward part of the movement)
2. If the deviation is to the right multiply by 1
B
3. If the curl field in the sequence is B-
multiply by 1
Therefore Positive DE Yielding to the
field Negative DE Over resisting the field
DE is Positive
21
Catch trials After Effects
A few trials without force field were introduced
unexpectedly. The left bar is the mean of the
error (DE) during these trials in the first part
of the learning. The right bar is in the last
part.
Significant expectation to the correct field
after learning i.e., learning of an internal
model of the force field
22
Mid Summary

• No adaptation in the case of the time dependent
  force field
• Adaptation in the case of the simplest sequence
  of curl viscous fields with four targets.

What is learned in the second case?
23
Odd and Even Movement

• During the learning it is possible to assign a
  unique force field to each movement instead of
  learning the sequence of force fields.
• The generalization phase would violate this
  representation.

B B-
Force Field
24
Refuting the Sequence Learning Assumption

• 1. Analysis of errors in the last part where
  diagonal movements are introduced

The same sequence is applied in this part
sequence learning predicts similar errors
Force Field
B B-
25
Distance Error Analysis of movements in part 1
and part 5
Left bar Catch trials in part 1. Middle bar
Movements in part 5 that are inconsistent with
the learning phase. Right bar Movements in part
5 that are consistent with the learning phase.
All movements are consistent with the sequence
of force field.
The sequence learning assumption predicts similar
errors in the right two bars that is smaller than
the first, left bar
However, ANOVA of the data shows similar error in
the first two bars and significantly smaller
error in the right bar!
26
Refuting the Sequence Learning Assumption

• We found that when the perturbation can be
  modeled both as a function of sequence and as a
  function of the state, the brain generates a
  state dependent model.

Can we design an experiment where only sequence
representation would allow adaptation? Would the
brain adapt to this perturbation?
We tried to train subject with the same sequence
but with three targets. In this case one needs to
follow the temporal sequence in order to adapt
27
Result No Adaptation to the Sequence of Force
Fields!
The maximum distance from a straight line during
learning
A control experiment with the viscous curl field
Karniel and Mussa-Ivaldi (2003) Biol. Cybern.
8910-21
28
Catch trials No After Effects
A few trials without force field were introduced
unexpectedly. The left bar is the mean of the
error (DE) during these trials in the first part
of the learning. The right bar is in the last
part.
No significant expectation to the correct field
after learning i.e., no learning of an internal
model to the sequence!
29
Mid Summary (2)

• No adaptation in the case of time dependent force
  field
• Adaptation when the temporal sequence coincide
  with single state mapping
• No adaptation in the case of sequence of force
  fields

Karniel and Mussa-Ivaldi (2003) Biol. Cybern.
8910-21
Maybe it is too difficult to construct two
internal models simultaneously
Multiple Models Conjecture (soft version) If
each force field is experienced separately and
enough time is given for consolidation of each
model, then the multiple model would be
constructed
30
Day 1 Day 2 Day 3
Day 4
Karniel and Mussa-Ivaldi EBR 2002
Early Training
Late Training
Late Training Catch-Trials
31
Result Clear learning of each perturbation, but
No evidence for ability to utilize multiple
models and context switching
Error DE, mm during early and late training
Day 1 Day 2 Day 3
Day 4
Error DE, mm during catch trials
(Subject E)
32
Does the brain employs clocks counters or
switches ?

• In contrast to artificial devices that are based
  on clock counters and switches the brain seems to
  prefer state dependent maps

33
Outline

• The Hierarchy of Wide Sense Adaptation Equilib
  rium trajectories and internal models Reaching
  movements muscle models and adaptation
• Adaptation to Force Perturbations Time
  representation Sequence learning and switching
• Bimanual Coordination Symmetry at the
  perceptual level as an invariant feature
  Tapping experiments and first indications for
  internal models
• Summary and Future Research

34
Bimanual Coordination (1)

• Preference for in-phase symmetry
• Stable vs. Unstable
• Homologous muscles
• Figure from Kelso and Schöner (1988)

35
Bimanual Coordination (2)

• It was recently shown that the preference for
  symmetry in bimanual coordination is perceptual
• Figure from Mechsner et al. (2001)

36
Bimanual Coordination (3)

• Untrained individuals are unable to produce
  non-harmonic polyrhythms
• However, with altered feedback (gear) they are
  able to generate symmetrical movement of the
  flags and non-symmetrical movements of the hands.
  
• Again The preference for symmetry is perceptual
• Figure from Mechsner et al. (2001)

37
Bimanual Coordination (4)

• The preference for symmetry was explained in
  terms of stable solution of dynamic system
  without employing internal models.
• Following the vast literature about reaching
  movements we propose an alternative Hypothesis
• The brain contains internal representation of the
  transformation between the perceptual level and
  the execution level in order to maintain the
  symmetry invariance in face of altered feedback
  or other external perturbations.
• Predictions 1. Learning curves, 2. After
  effects

38
Bimanual Index Tapping Experiment

• The right hand received slower feedback such that
  when the display shows rotation at equal speeds
  the subject eventually produces a non-harmonic
  polyrhythm, with a left/right tapping frequency
  ratio of 2/3

39
Learning Curve Regression (Standardized Data)
From Karniel A, Klaiman E, and Yosef V, Society
for Neuroscience 2003  
40
After-Effect IndicationsThe last 60 seconds of
each half in the experiment
41
Bimanual Adaptation Hypothesis

• Symmetry Invariance
• Adaptable transformation from the perception
  level to the execution level
• After effects
• The structure, learning rates and generalization
  capabilities are subjects for future research

42
Future Research

• Relative role of each level, muscles, spinal
  cord, central nervous system
• The structure of internal models (learning
  capabilities and generalization capabilities)
• Virtual Haptic Reality
• The Robo-Sapiens age

Mathematical Analysis, Simulation, Experiments
43
Turing-like test for motor intelligence The
Robo-Sapiens age Building a robot that would be
indistinguishable from human being