Modeling fMRI data generated by overlapping cognitive processes with unknown onsets using Hidden Pro

1
Modeling fMRI data generated by overlapping
cognitive processeswith unknown onsets using
Hidden Process Models

• Rebecca A. Hutchinson (1)
• Tom M. Mitchell (1,2)
• (1) Computer Science Department, Carnegie Mellon
  University
• (2) Machine Learning Department, Carnegie Mellon
  University
• Statistical Analyses of Neuronal Data (SAND4),
  May 30, 2008

2
Hidden Process Models

• HPMs are a new probabilistic model for time
  series data.
• HPMs are designed for data that is
• generated by a collection of latent processes
  that have overlapping spatial-temporal
  signatures.
• high-dimensional, sparse, and noisy.
• accompanied by limited prior knowledge about when
  the processes occur.
• HPMs can simultaneously recover the start times
  and spatial-temporal signatures of the latent
  processes.

3
Process 1
Process P
d1 dN
d1 dN
t
t
Example



t
t
Prior knowledge
There are a total of 6 processes in this window
of data.
An instance of Process 1 begins in this window.
An instance of Process P begins in this window.
An instance of either Process 1 OR Process P
begins in this window.
d1 dN
4
Simple Case Known Timing
Apply the General Linear Model YXW
D
p1
p3
p2
D
1 0 0 0 0 0 0 0 0 0 1 0 1 0 0 1 0 0 0 0 1
0 1 0 0 1 0 0 0 0 0 0 1 0 0 1
W(1)
p1

p2
W(2)
Y
T
W(3)
p3
Convolution Matrix X
Unknown parameters W
Data Y
Dale 1999
5
Challenge Unknown Timing
D
p1
p3
p2
D
1 0 0 0 0 0 0 0 0 0 1 0 1 0 0 1 0 0 0 0 1
0 1 0 0 1 0 0 0 0 0 0 1 0 0 1
W(1)
p1

p2
W(2)
Y
T
W(3)
p3
Uncertainty about the processes essentially makes
the convolution matrix a random variable.
6
fMRI Data
Hemodynamic Response
Features 10,000 voxels, imaged every
second. Training examples 10-40 trials (task
repetitions).
Signal Amplitude
Neural activity
Time (seconds)
7
Goals for fMRI

• To track cognitive processes over time.
• Estimate process hemodynamic responses.
• Estimate process timings.
• Allowing processes that do not directly
  correspond to the stimuli timing is a key
  contribution of HPMs!
• To compare hypotheses of cognitive behavior.

8
Our Approach

• Model of processes contains a probability
  distribution over when it occurs relative to a
  known event (called a timing landmark).
• When predicting the underlying processes, use
  prior knowledge about timing to limit the
  hypothesis space.

9
Study Pictures and Sentences
Press Button
View Picture
Read Sentence
Read Sentence
View Picture
Fixation
Rest
4 sec.
8 sec.
t0

• Task Decide whether sentence describes picture
  correctly, indicate with button press.
• 13 normal subjects, 40 trials per subject.
• Sentences and pictures describe 3 symbols , ,
  and , using above, below, not above, not
  below.
• Images are acquired every 0.5 seconds.

Keller et al, 2001
10
Process 1 ReadSentence Response signature
W Duration d 11 sec. Offsets W 0,1
P(?) q0,q1
Process 2 ViewPicture Response signature
W Duration d 11 sec. Offsets W 0,1
P(?) q0,q1
Processes of the HPM
v1 v2
v1 v2
Input stimulus ?
sentence
picture
Timing landmarks ?
Process instance ?2 Process h 2 Timing
landmark ?2 Offset O 1 (Start time ?2 O)
?1
?2
One configuration c of process instances
?1, ?2, ?k (with prior fc)
?1
?2
?
Predicted mean
N(0,s1)
v1 v2
N(0,s2)
11
HPM Formalism

• HPM ltH,C,F,Sgt
• H lth1,,hHgt, a set of processes (e.g.
  ReadSentence)
• h ltW,d,W,Qgt, a process
• W response signature
• d process duration
• W allowable offsets
• Q multinomial parameters over values in W
• C ltc1,, cCgt, a set of configurations
• c ltp1,,pLgt, a set of process instances
• lth,l,Ogt, a process instance (e.g.
  ReadSentence(S1))
• h process ID
• timing landmark (e.g. stimulus presentation of
  S1)
• O offset (takes values in Wh)
• ltf1,,fCgt, priors over C
• S lts1,,sVgt, standard deviation for each voxel

Hutchinson et al, 2006
12
Encoding Experiment Design
Processes
Input stimulus ?
Constraints Encoded h(p1) 1,2 h(p2)
1,2 h(p1) ! h(p2) o(p1) 0 o(p2) 0 h(p3)
3 o(p3) 1,2
ReadSentence 1
ViewPicture 2
Timing landmarks ?
?2
?1
Decide 3
Configuration 1
Configuration 2
Configuration 3
Configuration 4
13
Inference

• Over configurations
• Choose the most likely configuration, where

• Cconfiguration, Yobserved data, Dinput
  stimuli, HPMmodel

14
Learning

• Parameters to learn
• Response signature W for each process
• Timing distribution Q for each process
• Standard deviation s for each voxel
• Expectation-Maximization (EM) algorithm to
  estimate W and Q.
• E step estimate a probability distribution over
  configurations.
• M step update estimates of W (using reweighted
  least squares) and Q (using standard MLEs) based
  on the E step.
• After convergence, use standard MLEs for s.

15
Uncertain Timings

• Convolution matrix models several choices for
  each time point.

Configurations for each row
P
D
S
1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1
0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0
0 0 1 0 0 0 0 1 0 0 0 1 0 0 0 0
0 1 0 0 0 0 0 1 0 1 0 0 0 0 0 0 1 0 0
1 ...
t1 t1 t2 t2 t18 t18 t18 t18
3,4 1,2 3,4 1,2 3 4 1 2
TgtT
16
Uncertain Timings

• Weight each row with probabilities from E-step.

P
D
S
Configurations
Weights
e1 e2 e3 e4
3,4 1,2 3,4 1,2
1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1
0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0

Y

W
e1 P(C3Y,Wold,Qold,sold) P(C4Y,Wold,Qold,s
old)
17
(No Transcript)
18
(No Transcript)
19
(No Transcript)
20
Potential Processes
Can group these many ways to form different HPMs
21
Comparing HPMS
Participant
Cross-validated data log-likelihood. All values
are 106.
22
Are we learning the right number of processes?
For each training set, the table shows the
average (over 30 runs) test set log-likelihood of
each of 3 HPMs (with 2, 3, and 4 processes) on
each of 3 synthetic data sets (generated with 2,
3, and 4 processes). Each cell is reported as
mean standard deviation. NOTE All values in
this table are 105.
23
Ongoing Research

• Regularization for process response signatures
  (adding bias for temporal and/or spatial
  smoothness, spatial priors, spatial sparsity).
• Modeling process response signatures with basis
  functions.
• Allowing continuous start times (decoupling
  process starts from the data acquisition rate)
• A Dynamic Bayes Net formulation of HPMs.

24
References

• Dale, A.M., Optimal experiment design for
  event-related fMRI, 1999, Human Brain Mapping, 8,
  109-114.
• Hutchinson, R.A., Mitchell, T.M., Rustandi, I.,
  Hidden Process Models, 2006, Proceedings of the
  23rd International Conference on Machine
  Learning, 433-440.
• Keller, T.A., Just, M.A., Stenger, V.A.,
  Reading span and the time-course of cortical
  activation in sentence-picture verification,
  2001, Annual Convention of the Psychonomic
  Society.