Hidden Process Models: Decoding Overlapping Cognitive States with Unknown Timing

1
Hidden Process ModelsDecoding Overlapping
Cognitive States with Unknown Timing

• Rebecca A. Hutchinson
• Tom M. Mitchell
• Carnegie Mellon University
• NIPS Workshops New Directions on Decoding
  Mental States from fMRI Data
• December 8, 2006

2
Overview

• Open questions we address
• Treating fMRI as the time series that it is.
• Allowing the testing of hypotheses.
• Open questions we do NOT address
• Interpretability of time series or spatial
  representation of activity.
• This talk
• Motivation
• HPMs (in 1 slide!)
• Preliminary results

3
Motivation

• Goal connect fMRI to cognitive modeling.
• Cognitive Model
• Set of cognitive processes hypothesized to occur
  during a given fMRI experiment.
• Cognitive Process
• Spatial-temporal hemodynamic response function.
• Timing distribution relative to experiment
  landmarks (like stimulus presentations and
  behavioral data).

4
Study Pictures and Sentences
Press Button
View Picture
Read Sentence
Read Sentence
View Picture
Fixation
Rest
4 sec.
8 sec.
t0

• Task Decide whether sentence describes picture
  correctly, indicate with button press.
• 13 normal subjects, 40 trials per subject.
• Sentences and pictures describe 3 symbols , ,
  and , using above, below, not above, not
  below.
• Images are acquired every 0.5 seconds.

5
One Cognitive Model
Press Button
View Picture
Read Sentence
Read Sentence
View Picture
Fixation
Rest
4 sec.
8 sec.
t0
ViewPicture or ReadSentence
ViewPicture or ReadSentence

• ViewPicture
• begins when picture stimulus is presented
• ReadSentence
• begins when sentence stimulus is presented
• Decide
• begins within 4 seconds of 2nd stimulus

Decide
6
(No Transcript)
7
ViewPicture in Visual Cortex
8
ReadSentence in Visual Cortex
9
ViewPicture
10
ReadSentence
11
Seconds following the second stimulus
Multinomial probabilities on these time points
Decide
12
Comparing Models
5-fold cross-validation, 1 subject P
ViewPicture S ReadSentence S
ReadAffirmativeSentence S- ReadNegatedSentence
D Decide D DecideAfterAffirmative D-
DecideAfterNegated Dy DecideYes Dn
DecideNo Dc DecideConfusion B Button
- This HPM can also classify Dy vs. Dn with
92.0 accuracy. GNBC gets 53.9. (using the
window from the second stimulus to the end of the
trial)
13
Conclusions

• Simultaneous estimation of spatial-temporal
  signature (HRF) and temporal onset of cognitive
  processes.
• Framework for principled comparison of different
  cognitive models in terms of real data.