fMRI Techniques to Investigate Neural Coding: fMRA and MVPA

1
fMRI Techniques to Investigate Neural Coding
fMRA and MVPA
Jody Culham Brain and Mind Institute Department
of Psychology University of Western Ontario
http//www.fmri4newbies.com/
Last Update January 18, 2012 Last Course
Psychology 9223, W2010, University of Western
Ontario
Last Update January 18, 2012 Last Course
Psychology 9223, W2010, University of Western
Ontario
2
Limitations of Subtraction Logic

• Example We know that neurons in the brain can be
  tuned for individual faces

Jennifer Aniston neuron in human medial
temporal lobe Quiroga et al., 2005, Nature
3
Limitations of Subtraction Logic

• fMRI resolution is typically around 3 x 3 x 6 mm
  so each sample comes from millions of neurons.
  Lets consider just three neurons.

Neuron 1 likes Jennifer Aniston
Neuron 2 likes Julia Roberts
Neuron 3 likes Brad Pitt
Even though there are neurons tuned to each
object, the population as a whole shows no
preference
4
Two Techniques with Subvoxel Resolution

• subvoxel resolution the ability to
  investigate coding in neuronal populations
  smaller than the voxel size being sampled
• fMR Adaptation (or repetition suppression or
  priming)
• Multivoxel Pattern Analysis (or decoding)

5
fMR Adaptation(or repetition suppression or
priming)
6
fMR Adaptation

• If you show a stimulus twice in a row, you get a
  reduced response the second time

Hypothetical Activity in Face-Selective Area
(e.g., FFA)
Unrepeated Face Trial
?
Activation
Repeated Face Trial
?
Time
7
fMRI Adaptation
different trial
500-1000 msec
same trial
Slide modified from Russell Epstein
8
Block vs. Event-Related fMRA
9
Why is adaptation useful?

• Now we can ask what it takes for stimulus to be
  considered the same in an area
• For example, do face-selective areas care about
  viewpoint?

• Viewpoint selectivity
• area codes the face as different when viewpoint
  changes

Repeated Individual, Different Viewpoint
Activation
?

• Viewpoint invariance
• area codes the face as the same despite the
  viewpoint change

Time
10
Actual Results
LO
pFs (FFA)
Grill-Spector et al., 1999, Neuron
11
And more

• We could use this technique to determine the
  selectivity of face-selective areas to many other
  dimensions

Repeated Individual, Different Expression
Repeated Expression, Different Individual
12
Models of fMR Adaptation
Grill-Spector, Henson Martin, 2006, TICS
13
Evidence for Fatigue Model
Data from Li et al., 1993, J Neurophysiol Figure
from Grill-Spector, Henson Martin, 2006, TICS
14
Evidence for Facilitation Model
James et al., 2000, Current Biology
15
Caveats in InterpretingfMR Adaptation Results
16
(No Transcript)
17
fMRA Does Not Accurately Reflect Tuning

• MT most neurons are direction-selective (DS),
  high DS in fMRA
• V4 few (20?) neurons are DS, very high DS in
  fMRA
• perhaps fMRA is more driven by inputs than
  outputs?

Tolias et al., 2001, J. Neurosci
18
(No Transcript)
19
Basic Assumption/Hypothesis

• if a neuronal population responds equally to two
  stimuli, those stimuli should yield
  cross-adaptation

Neural Response
Predicted fMRI Response
A-A
A-B
A
B
C
B-B
C-A
20
Experimental Question

• the human lateral occipital complex (LOC) is
  arguably analogous/homologous to macaque
  inferotemporal (IT) cortex
• both human LOC and macaque IT show fMRI
  adaptation to repeated objects
• Does neurophysiology in macaque IT show object
  adaptation at the single neuron level?

21
Design
Experiment 1 Block Design Adaptation
Experiment 2 Event-Related Adaptation
Sawamura et al., 2006, Neuron
22
Yes, neurons do adapt
Sawamura et al., 2006, Neuron
23
but cross-adaptation is less clear
A-A ADAPT AB
B-A ADAPT AB
WHOLE POPULATION
EXAMPLE
BLOCK
A-A B-B C-A B-A
EVENT- RELATED
Sawamura et al., 2006, Neuron
24
Sawamura et al. Conclusions

• Evidence for adaptation at the single neuron
  level is clear
• Cross-adaptation is not as strong as expected,
  particularly for event-related designs
• They dont think its just attention
• Something special about repeated stimuli

25
(No Transcript)
26
Design
Task press button for inverted face
REP BLOCK (75 rep trials, 25 alt trials) AA
BB CD EE FF GH II JJ ALT BLOCK (25
rep trials, 75 alt trials) AB CC DE FG
HI JK LM NN
Summerfield et al., 2008, Nat Neurosci
27
Results
22 plt.001
9 plt.05
SIG INTERACTION stronger fMRA in blocks with
freq. reps
Individual FFA ROIs
Summerfield et al., 2008, Nat Neurosci
28
Replication
Task press button for small face

• results were replicated with a different task

Summerfield et al., 2008, Nat Neurosci
29
New Explanation of fMRA

• repetition suppression reflects a reduction in
  perceptual prediction error
• mismatch between expectations and stimulus
  increases fMRI activation
• mismatch is higher on novel trials than
  repetition trials

30
Additional Caveats

• Adaptation effects can be quite unreliable
• variability between labs and studies
• even effects that are well-established in
  neurophysiology and psychophysics dont always
  replicate in fMRA
• e.g., orientation selectivity in primary visual
  cortex
• David Heeger suggests that it may be critical to
  control attention
• The effect may also depend on other factors
• e.g., time elapsed from first and second
  presentation
• days, hours, minutes, seconds, milliseconds?
• number of intervening items
• attention (especially in block designs)
• memory encoding
• Different areas may demonstrate fMRA for
  different reasons
• reflected in variety of terms repetition
  suppression, priming

31
So is fMRA dead? No.

• Criticism fMRA may reflect inputs rather than
  outputs
• Response This is a general caveat of all fMRI
  studies. Inputs are interesting too, just harder
  to interpret. Focus on outputs oversimplifies
  neural processing when presumably feedback loops
  are an essential component.
• Criticism fMRA may not reveal cross-adaptation
  even in populations that do show cross-coding
• Response This suggests that caution is
  especially warranted when there is a failure to
  find adaptation (or a finding of recovery from
  adaptation). However, cross-adaptation can
  occur and is meaningful when it does. Many past
  fMRA studies have found it.

32
So is fMRA dead? No.

• Criticism None of the basic models of fMRA seem
  to work.
• Response In some ways, it doesnt matter. The
  essential use of fMRA is to determine whether
  neural populations are sensitive to stimulus
  dimensions. The exact mechanism for such
  sensitivity may not be critical.
• Criticism fMRA, and maybe fMRI in general, is
  just responding to predictions.
• Response Prediction is interesting too.
  Regarding fMRA, why do some brain areas make
  predictions about a stimulus while others dont?

33
Multivoxel Pattern Analyses(or decoding or mind
reading)
34
Voxels
3 mm
3 mm
3 mm
lowactivity
highactivity

• Modern scanner can collect 150,000 voxels in 2 s

35
Difficulty with Standard fMRI analysis
Movement 1 or Movement 2
Beep
Light
Next trial
Preview
Plan
Execute
ITI
Movement 1
R
L
Movement 2
36
Voxel Pattern Information
Movement 1
Movement 2
3 mm
R
L
3 mm
3 mm
37
Standard Analysis
Movement 1
Movement 2
trial 1
trial 1
VoxelwiseActivityin ROI
trial 2
trial 2
trial 3
trial 3
AverageSummed Activation
38
Spatial Smoothing
No smoothing

• most conventional fMRI studies spatially smooth
  (blur) the data
• increases signal-to-noise
• facilitates intersubject averaging
• loses information about the patterns across voxels

39
Effect of Spatial Smoothingand Intersubject
Averaging
3 mm
3 mm
3 mm
40
Perhaps voxels contain useful information

• In traditional fMRI analyses, we average across
  the voxels within an area, but these voxels may
  contain valuable information
• In traditional fMRI analyses, we assume that an
  area encodes a stimulus if it responds more, but
  perhaps encoding depends on pattern of high and
  low activation instead
• But perhaps there is information in the pattern
  of activation across voxels

41
Multi-voxel pattern analysis (MVPA)
Movement 1
Movement 2
trial 1
trial 1
TrainingTrials
trial 2
trial 2
trial 3
trial 3
Can an algorithm correctly guess trial identity
better than chance (50)?
TestTrials (not in training set)
42
Coding in Voxel Patterns

• Simple experiment Show subjects pictures of
  different objects (e.g., shoes vs. bottles) on
  different trials of different runs

43
Simple Correlation Analysis

• Measure within-category correlations
• within bottles (B1B2)
• within shoes (S1S2)
• Measure between-category correlations
• between bottles shoes (B1 S2 S1 B2)
• If within-category correlations gt
  between-category correlations, conclude that area
  encodes different stimuli

44
First Demonstration
45
Haxby et al., 2001, Science
46
Haxby et al., 2001, Science
47
Haxby et al., 2001, Science
48
Decoding Algorithms

• Train algorithm to distinguish two object
  categories on a training set
• Test success of algorithm on distinguishing two
  object categories on a test set
• If algorithm succeeds better than chance,
  conclude that area encodes different stimuli

Norman et al., 2006, Trends Cogn. Sci.
49
MVPA Methods

• block or event-related data
• resolution
• works even with moderate resolution (e.g., 3 mm
  isovoxel)
• tradeoff between resolution and coverage, SNR
• 2 mm isovoxel recommended at 3 T
• preprocessing
• usually steps apply (slice scan time correction,
  motion correction, low pass temporal filter)
• EXCEPT No spatial smoothing!
• Model single subjects, not combined group data
  (at least initially)

50
MVPA Methods

• separate data into independent training and test
  sets
• e.g., even and odd runse.g., iterate sequence of
  leave one run out
• pick the area to analyze
• ROI localizer
• contrast in training set
• train the classifier
• input beta weights from each voxel in area
• variety of classifiers available
• e.g., linear support vector machine
• test the classifier
• does classifier perform better than chance?
• e.g., chi-squared test

Summarized from Mur et al., 2009, Social
Cognitive and Affective Neuroscience
51
Haynes Rees, 2006, Nat Rev Neurosci
52
How can MVPA see patterns lt 1 voxel?
Data from Kamitami Tong, 2005, Nat
Neurosci Figure from Norman et al., 2006, TICS
53
MVPA Searchlight

• define a spherical searchlight
• optimal searchlight has radius 4 mm
• contains 33 2-mm-isovoxel voxels
• compute multivariate effect within all possible
  locations within brain volume
• calculate voxelwise p values and threshold them
  at false discovery rate q values

Kriegeskorte, Goebel Bandettini, 2006, PNAS
54
MVPA Searchlight
Kriegeskorte, Goebel Bandettini, 2006, PNAS
55
MVPA Searchlight
Kriegeskorte, Goebel Bandettini, 2006, PNAS
56
Does MVPA (decoding) make fMRA obsolete?

• MVPA allows us to address similar questions about
  what is coded in an area.
• MVPA may have some advantages (e.g., less
  susceptible to attentional confounds)
• MVPA utility depends on numerous factors (e.g.,
  region size are there enough voxels to get a
  meaningful pattern)
• MVPA requires clustering of neural populations
  and is sensitive to scanning parameters (voxel
  size) fMRA does not
• MVPA has the same problem as fMRA its very hard
  to draw conclusions from a null result

57
Activation vs. Patterns
Mur et al., 2009, Social Cognitive and Affective
Neuroscience