Controlling a Neuroprosthetic Arm: Dynamic Estimation and Prediction

1
Controlling a Neuroprosthetic Arm Dynamic
Estimation and Prediction

• Advanced Data Analysis Project,
• Cari Kaufman
• Advisors Valérie Ventura (CMU),
• Dawn Taylor (Case Western)

2
Neuroprosthetic devices

• Aim to restore lost function.
• Use signals from the brain to control a
  mechanical prosthesis.
• Require detailed signal recording, so electrodes
  are implanted directly into the brain.

Source U. Pittsburgh Motor Control Laboratory
3
Goal of the project

• Design a dynamic algorithm allowing the user of
  the prosthetic device to learn to control its
  movement.

4
Goal of the project

• Design a dynamic algorithm allowing the user of
  the prosthetic device to learn to control its
  movement.
• Dynamic algorithm should
• Incorporate new data as it arrives.
• Do predictions in real time.

5
Overview

• Neuronal data and how to use it
• The current paradigm - Arm control
• The brain control case
• Data description
• Methods and results
• Future work

6
How do neurons communicate?

• Neurons communicate using rapid voltage changes
  called spikes.

7
How do neurons communicate?

• Neurons communicate using rapid voltage changes
  called spikes.

8
Direction
Source Georgopoulos et al., 1982
9
Modelling the spiking rate

• SpikesMovement Poisson(f(Movement))

10
Modelling the spiking rate

• SpikesMovement Poisson(f(Movement))
• Direction take f to be cosine of angle between
  actual and preferred direction.
• 2D

11
Predicting movement

• Can estimate P(spikes movement).
• For prediction want P(movement spikes).
• P(movement spikes)
• P(spikes movement) P(movement)
• Use posterior mean or median for prediction.

12
Arm control Current paradigm

• Training data

Actual arm movement Position, direction, speed,
etc.
Spike times for N neurons
13
Arm control Current paradigm

• Training data

Actual arm movement Position, direction, speed,
etc.
Spike times for N neurons

P(spikes movement)
14
Arm control Current paradigm

• Training data

• New data

Actual arm movement Position, direction, speed,
etc.
Actual arm movement Position, direction, speed,
etc.
Spike times for N neurons
Spike times for N neurons

P(spikes movement)
15
Arm control Current paradigm

• Training data

• New data

Actual arm movement Position, direction, speed,
etc.
Actual arm movement Position, direction, speed,
etc.
Spike times for N neurons
Spike times for N neurons

P(spikes movement), P(movement)

P(spikes movement)
Predicted movement
16
Arm control Current paradigm

• New data

Actual arm movement Position, direction, speed,
etc.
COMPARE
Predicted movement
17
Brain control The problem

• Without a real arm, data will look like
• But we tell the prosthetic where to go.
• What should the algorithm be now?

PROSTHETIC movement
Spike times for N neurons
18
Data and experimental design

• Taylor et al. (2002) conducted experiments with
  rhesus monkeys.
• 3D version of center-out task
• 65 neurons
• Both hand and brain control data

Source Taylor et al., 2002
19
Brain control data

• Firing times of 65 neurons
• Cursor position - determined by 65 recorded
  neurons and Taylors algorithm
• Target position

20
The current algorithm

• Taylor used an dynamic algorithm to eliminate
  need for training data.

21
(No Transcript)
22
Modelling the spiking rate

• What do the neurons encode under brain control?
• Arm control

23
Modelling the spiking rate

• What do the neurons encode under brain control?
• Arm control

24
Modelling the spiking rate

• What do the neurons encode under brain control?
• Brain control

25
Modelling the spiking rate

• What do the neurons encode under brain control?
• Brain control

26
Modelling the spiking rate

• Idea Use the direction needed to reach the
  target as the monkeys intended direction.
• Model P(spikes intended direction)

27
Brain control Dynamic training

• Data up to time t-1

Intended direction(to the target)
Spike times for N neurons
28
Brain control Dynamic training

• Data up to time t-1

Intended direction(to the target)
Spike times for N neurons

P(spikes intended direction)
29
Brain control Dynamic training

• Data up to time t-1

• Data at time t

Intended direction (to the target)
Intended direction (to the target)
Spike times for N neurons
Spike times for N neurons

P(spikes intended direction)
30
Brain control Dynamic training

• Data up to time t-1

• Data at time t

Intended direction (to the target)
Intended direction (to the target)
Spike times for N neurons
Spike times for N neurons

P(spikes intended direction), P(intended
direction)

P(spikes intended direction)
Predicted direction
31
Brain control Dynamic training

• Data at time t

Intended direction(to the target)
COMPARE
Predicted direction
32
Brain control Dynamic training

• In order to create natural looking movements, do
  this every 30 ms.
• This allows us to compare our predictions
  directly to those of the current algorithm.

33
Details Estimating the model

• For intended direction, can use 3D cosine model.
• Fit using standard GLM software.
• Model parameters are allowed to vary with time, a
  learning effect.

34
Details Predicting direction

• Simplifying assumptions
• Intended movements form a Markov chain.
• Firing rates only depend on the current intended
  movement.
• Then p(MtY1,,t) p(YtMt)
  p(MtY1,...,t-1)

35
Details Predicting direction

• Simplifying assumptions
• Intended movements form a Markov chain.
• Firing rates only depend on the current intended
  movement.
• Then p(MtY1,,t) p(YtMt)
  p(MtY1,...,t-1)p(MtY1,...,t-1) ? p(MtMt-1)
  p(Mt-1Y1,...,t-1) dMt-1

36
Modelling issues

• p(MtY1,,t) p(YtMt) p(MtY1,...,t-1)p(MtY1
  ,...,t-1) ?p(MtMt-1) p(Mt-1Y1,...,t-1) dMt-1
• Need Likelihood, initial prior p(M1),transition
  density

37
Modelling issues

• p(MtY1,...,t-1) ?p(MtMt-1)p(Mt-1Y1,...,t-1)
  dMt-1
• Transition density For direction, can use a
  spherical Fisher distribution centered at the
  current direction.

38
Computational Issues

• P(MtY1,,t) p(YtMt) p(MtY1,...,t-1)p(MtY1
  ,...,t-1) ?p(MtMt-1) p(Mt-1Y1,...,t-1) dMt-1
• No closed form expression for the integral.
• Approximate the posteriors using a Monte Carlo
  method called particle filtering, which is
  similar to importance sampling.

39
Particle filter One iteration
40
Particle filter One iteration
41
Particle filter One iteration
42
Particle filter One iteration
43
Particle filter One iteration
44
Comparison to current algorithm

• Cursor position still given by current algorithm.
• Show where our algorithm would have gone at each
  point in time.

45
(No Transcript)
46
Angle of discrepancy
47
Extensions

• Modelling
• Likelihood independent Poisson
• Other covariates intended speed, attention
• Time lags
• Real time implementation