1MPI for Biological Cybernetics 2Stanford University 3Brain-Computer Interface Laboratory, Wadsworth Center 3Werner Reichardt Centre for Integrative Neuroscience Eberhard Karls University Tuebingen

1
1MPI for Biological Cybernetics 2Stanford
University 3Brain-Computer Interface Laboratory,
Wadsworth Center 3Werner Reichardt Centre for
Integrative NeuroscienceEberhard Karls
University Tuebingen
Closing the Sensorimotor Loop Haptic Feedback
Facilitates Decoding of Arm Movement Imagery
M. Gomez-Rodriguez1,2 J. Peters1 J. Hill3
B. Schölkopf1 A. Gharabaghi4 M. Grosse-Wentrup1
SMC Workshop in Shared-Control for BMI, October
2010
2
BCI robot-assisted therapy

• Brain Computer Interface (BCI) robot-assisted
  physical therapy for neurorehabilitation of
• Hemiparetic syndromes due to brain damage
• may outperform traditional therapy

Our approach to rehabilitation
Traditional rehabilitation
Sensorimotor loop is broken They do not help for
severe motor impairment
We close the sensorimotor loop Synchronize
subjects attempt and robot arm
3
Stand-alone BCI and robot-assisted therapy

• It has been shown that

Motor imagery
Robot-assisted physical therapy
are beneficial for rehabilitation as stand-alone
therapies 2,3but loop is still broken!
Next logical step is to combine both in an
integrated rehabilitation to close the loop
4
Hebbian plasticity Why closing the loop?

• Closing artificially the sensorimotor loop is
• likely to result in increased cortical plasticity
• because we induce Hebbian plasticity.

Hebbian plasticity 1
A positive feedback-mediated plasticity in which
synapses between presynaptic and postsynaptic
neurons that are coincidently active are
strengthened.
Requirements
Instantaneous feedback Delays in the order of ms.
High accuracy On-line decoding of arm movement
intention.
High specificity Focus on motor and sensorimotor
cortex.
5
BCI decoding Effect of closing the loop
Combining BCI and robot-assisted physical therapy
opens many research questions.

• Our work builds on analyzing the effect of
  artificially closing the sensorimotor loop on
  BCI-decoding.

Previous studies Passive and active movements
induce patterns in the brain similar to those
induced by motor imagery 2, 3. Random haptic
feedback has been shown to be beneficial for
BCI-decoding 4.
In our work We show how haptic feedback (closing
the sensorimotor loop) influences BCI-decoding.
6
Outline

• Experimental Design
• Human subjects, recording and task feedback
  conditions.
• Methods
• Signal processing, on-line decoding and
  conditions comparison.
• Results
• Analysis of the haptic feedback effect on
  decoding performance and spatial/frequency
  features.
• Conclusions

7
Subjects and recordings
Human subjects 6 right-handed healthy
subjects between 22 and 32 years old.
Recording 35 EEG channels250 Hz sampling
rateQuickamp with built-in CARBCI2000 BCPy2000
Pre-motor, primary motor and somatosensory cortex
are covered
8
Task and haptic feedback conditions
Subjects task Think about moving the right
arm forward (extension) or backward (flexion) in
the same way the robot does.
Condition Training Test


Condition I

Condition II

Condition III
Condition IV
9
Haptic feedback conditions

Rest 3sMI 5s
Rest 3sMI min(5s, robot hits border)
Trial duration
X
Robot moves while motor imagery
Training(25s per condition)
Arrow in a screen robot moves-stops according
to classifier while motor imagery
Arrow in a screen moves-stops according to
classifier while motor imagery
Test(consecutively after training)
10
Outline

• Experimental Design
• Human subjects, recording and task feedback
  conditions.
• Methods
• Signal processing, on-line decoding and
  conditions comparison.
• Results
• Analysis of the haptic feedback effect on
  decoding performance and spatial/frequency
  features.
• Conclusions

11
Signal Processing
Preprocessing
Surface Laplacian Filter
Band-pass filtering (2-115Hz)
Notch filtering (50 Hz)
Features Computation
Power spectral densities over 2Hz frequency bins
for each electrode are used as features. Welchs
method over overlapping incrementally bigger time
segments each 5-s movement or 3-s resting periods.
Larger segments ? Less noise and more reliable
estimates.
Shorter segments ? Necessary for on-line feedback.
12
On-line Decoding

• During the test periods, on-line classification
  between movement and resting using spectral
  features
• Every 300 ms,
• One classifier output.
• Visual on-line feedback and depending on the
  condition also haptic feedback is updated.

A linear support vector machine (SVM) is
generated each run on-line after the training
period and its outputs are mapped to
probabilistic outputs by fitting a sigmoid.
13
Conditions comparison
To discover how haptic feedback influences the
BCI
we compare the BCI performance for each
condition of haptic feedback by computing

1. Two-way analysis of variance (ANOVA) over
   probabilistic outputs in each condition.
2. Average accuracy per condition.
3. Area under the receiving operating characteristic
   (AUC) per condition

We expect all three to support the same
conclusions to strengthen the empirical evidence.
14
ANOVA and AUC
ANOVA
AUC

1. For each condition, we group M probabilistic
   outputs from all subjects.
2. Compute ANOVA at significant level a 0.05 with
   Bonferroni multiple-comparison correction.
3. ANOVA tell us if we can reject the hypothesis
   that the probabilistic outputs means are equal
   between conditions.

1. For each subject and condition, we have N
   probabilistic outputs.
2. We sweep over different thresholds in (0, 1) to
   classify mov/rest and compute the accuracy for
   each.
3. The area under the curve threshold versus
   accuracy is our AUC.

15
Outline

• Experimental Design
• Human subjects, recording and task feedback
  conditions.
• Methods
• Signal processing, on-line decoding and
  conditions comparison.
• Results
• Analysis of the haptic feedback effect on
  decoding performance and spatial/frequency
  features.
• Conclusions

16
ANOVA

• ANOVA confidence intervals

Training
Test



Conditions

Average probabilistic on-line (every 300ms) output
Condition I outperforms the rest, very clearly
condition IV!
17
Average accuracy

1. In group average, Condition I outperforms the
   rest.
2. Condition I outperforms Conditions III and IV for
   all subjects, and it outperforms II for all
   subjects except two.

Average accuracy

Test

The results are coherent with ANOVA!
Training


Conditions
18
AUC

1. In group average, Condition I outperforms the
   rest.
2. Condition I outperforms the rest for all subjects.

AUC

Test

The results are coherent with ANOVA and average
accuracy!
Training


Conditions
19
Spatial and spectral features
Average classifiers weights for each electrode
over the frequency band (2 40 Hz)
Condition I and II(Robot moves during training)
Condition III and IV(Robot does not move during
training)
When the robot moves, we have higher weights in
the motor/somatosensory area
20
Outline

• Experimental Design
• Human subjects, recording and task feedback
  conditions.
• Methods
• Signal processing, on-line decoding and
  conditions comparison.
• Results
• Analysis of the haptic feedback effect on
  decoding performance and spatial/frequency
  features.
• Conclusions

21
Conclusions

• Artificially closing the sensorimotor feedback
  loop facilitates decoding of movement intention
  in healthy subjects.
• Our results indicate the feasibility of future
  integrated rehabilitation therapy that combines
  robot-assisted physical therapy with decoding of
  movement intention by a BCI.
• We assume that the results presented here with
  healthy subjects can be transferred to stroke
  patients.
• We speculate that haptic feedback support
  subjects in initiating a voluntary modulation of
  their SMR.
• In a shared-control scenario in BMIs, we may
  improve performance by means of haptic feedback.