Cognitive Control Signals for Neural Prosthetics

1
Cognitive Control Signals for Neural Prosthetics

• Sam Musallam
• The Andersen Lab
• sam_at_vis.caltech.edu

2
What is a cognitive signal?

• Signals that lie along the sensory to motor
  pathway but away from sensory and away from motor
• Not visual
• Not motor
• Encode higher order variables
• Intention
• Motivation
• Value of a reward

3
Cognitive-based rather thanmotor-based

• May require fewer cells, less invasive
• Cognitive variables-expected value
• Hierarchical control using smart machines

PMC
4
Arrays are Implanted in the Medial Intraparietal
Sulcus and Dorsal Premotor Cortex
A5
5
Arrays are Implanted in Medial Intraparietal
Sulcus
6
We are currently implanting 160 electrodes in
PRR and in DPMC

• - 32 electrode arrays
• - Made from platinum and iridium
• - 80 microns at the shaft
• 2-3 micron tip

7
Decoding Goals E
Reach Trials
Brain control trials
8
Database Built on Memory Period Activity

• Monkeys are in the dark and are not allowed to
  move
• their
• Hands
• Eyes
• This memory period activity is cognitive

9
Example of feedback performance6 Parietal
neurons
Fixation maintained throughout trial. Decoding
900 ms of memory period
10
More neurons also improve the decode(dPMC)
8 targets decoded with 16 neurons
11
At 100 ms, predictions are still significantly
above chance (25)
12
2 types of databases used

• Adaptive
• Database updated after every successful feedback
  trial
• Frozen
• Database frozen after the end of the reach
  trials

13
Frozen or adaptive database? Doesnt seem to
matter
14
Type of reward changes the tuning of neurons in
PRR
Blackorange juice Red water
15
Tuning is also enhanced for increased probability
and magnitude of reward
16
Our decoding ability improves with preferred
reward condition
Decode on 5 single units
17
Overall decode improvement
18
Decoding the value of reward
19
Decoding Direction and Reward from a monkey with
no single units
Without the reward schedule barely above
chance. Reward schedule improves decode
20
Advantages of a Cognitive Prosthetics

• Cognitive signals will allow us to directly
  determine the mood of patients
• This can be done without requiring patients to
  indicate this preference using overt behavior.
• Optimize control of prosthetic devices
• counter neuronal sample biases.(surgical
  placement of electrodes, etc.)
• allow multiple tool use.(update or variable
  functionality of prosthesis)

21
Conclusion

• Cognitive neural activity not directly related to
  visual input or motor output can be used for as
  prosthetic control signals.
• Cognitive signals can also give us information
  about the patients preference or mood.

22
Acknowledgment

• Kelsie Pejsa
• Lea Martel
• Viktor Shcherbatyuk
• Tessa Yao

• Richard Andersen
• Brian Corneil
• Bradley Greger
• Hans Scherberger
• Grant Mulliken
• Rajan Bhattacharyya
• Igor Fineman
• Bijan Pesaran
• Daniella Meeker