Moving by Thinking: Towards a Cortical Neural Prosthetic

1
Moving by Thinking Towards a Cortical Neural
Prosthetic
E R C
M. Jarvis, D. Meeker B.Pesaran,
P. Mitra S. Cao , J.
Burdick R. Andersen
Bell Labs, Lucent H. Mak, E.
Branchaud Biology
Physics
Engineering Applied Science

Proposed Cortical Prosthetic System
2 mm
2
Parietal Reach Region (PRR)
PRR
Monkey
Human
3
Potential Advantages of PRR Neurons for
Prosthetic Systems

• PRR neurons encode
• The plan to reach to a target
• The plan for the upcoming reach
• The plan with respect to the eyes

• PRR neurons may
• not encode muscle forces
• reorganize little following injury
• adapt quickly to calibrate the system

120 spikes/s
Cue
Reach
1 second
Plan
Batista, Buneo, Snyder, Andersen (1999) Science
285.
4
Recording from Many NeuronsChronic Electrode
Array in PRR
Courtesy Bionic Tech.
Courtesy Bionic Tech.
5
Arm Control Systems
target
Artificial Sensors

• Key variables
• intended reach location
• intentional and cognitive mind state
• external sensor variables

6
Key Challenges and Research Agenda

• 1) What control signals can be decoded?
• arm reach direction
• logical variables corresponding to intent
• Target/no target, go, scrub, replan, path
  sequence, via point
• 2) Best decode method accuracy, robustness, SNR?
• 3) How many neural signals needed?
• 4) Construct a dynamic model of human intent?
• 5) Adaptive Algorithms?
• 6) System latency?
• 7) Safe arm control algorithms? (incorporating
  external sensors?)

7
Estimating the Planned Reach Direction
Treatment follows that of Zhang, Ginzburg,
McNaughton, Sejnowski (1998) J. Neurophysiol. 79,
Brown et. al, (1998)
8
Reconstruction Performance
Reach tuning in 49 PRR neurons
Error vs. Population Size
Reconstruction Error ()
Neurons recorded one at a time(Monkey CKY)
Reconstruction Cells ()
9
Decoding Logical Signals
Reach Target
Direction 1 Direction 2 Direction n
Move
Go ?
Stop
Scrub ?
Plan
No Move
Target ?
Spike train
No Plan

• Planning involves a sequence of logical decisions
• Decoding logical states and transitions is key
  to
• accurate decoding of reach
• purposeful and effective control of prosthetic

10
Simple Finite State Machine (FSM) model

• Logical planning sequence can be idealized as a
  FSM (this one is crude). Need to
• Detect transitions
• Determine current State

Continue Planning
No Go ?
Done
Target Achieved

• During this period we have
• demonstrated target, go decoding
• shown how very simple FSM model can improve
  decoding

11
A Go Signal in the LFP
Single Trial Data
Data from array implant
12 electrodes
Average power
Reach Onset
target

• Power in 15-25 Hz band
• averaged over channels
• average over trials

std. dev.
mean
The go signal
target
target
12
Naïve Classification of State Evolution (i.e.,
decoding without benefit of FSM model) (250 msec
windows)
Cue
Reach
Plan
White 100 Black 0
Average
13
Go
State Prediction Performance for different FSM
models (250 msec windows) Moral performance
improves with better FSM models
Plan Direction
Baseline
0 3
0 3
0 3
time
14
Prosthetic-System Testbed Physical Setup
Move animated arm to green target.
X , Y

X , Y
Animate Virtual Arm
Amplify, Filter, Digitize
Determine Controls
Extract Neural Metrics
Estimate Planned Movement
E.g., spike times spectral power
Extract Logical State
15
Prosthetic-System Testbed Architecture
Minimize estimation error
Pre-amp / filter
Main-amp / filter
32 12-bit A/D Converters
DSP Spike sorters

Head-stage
X , Y
32
32
32
Spike Path
X , Y
32
Pre-amp / filter
Main-amp
12-bit A/D Converter
MUX
High-speed serial BUS
32
32
32
LFP Path
PCI BUS
Behavioral Control PC
100 Mbit/s TCP/IP
Neural Data Server PC
Eye-position Client
Reward Client
Graphics Client
Graphics Client
Spectral Analysis Client PC 1
Trial Events Client
Spectral Analysis Client PC 2
Arm-position Client
Display PC
Spectral Anal. Client/Server
Spike-sort Client
Animation Client
X , Y

X , Y
Eye/Arm Target Client/Server
Trial Status Client
Spectral Analysis Client PC n
100 Mbit/s TCP/IP


X , Y
Decode Client
Decode Client
Hybrid Controller

X , Y
100 Mbit/s TCP/IP
Reach Estimate Server
(courtesy K. Shenoy)
16
Future Integrated/Implantable Systems(M.
Mojaridi et. al, JPL)
17
Movable Tetrode Arrays
18
Generalization

• Future implantable human sensors will
• measure many signals in parallel
• have wireless telemetry
• have low-power on-board processing circuitry
• be able to continually adjust their geometry
  (via miniature on-board actuators) to optimize
  signal quality
• CNSE/Lee have expertise in MEMS, wireless,
  low-power VLSI, sensor processign