Brain Waves and Button Presses: The Role for Experiments in Theoretical Linguistics

1
Brain Waves and Button Presses The Role for
Experiments in Theoretical Linguistics

• Alec Marantz
• Department of Linguistics Philosophy, MIT
• KIT/MIT MEG Joint Research Lab

2
Competence vs. Performance?Standard View

• Performance
• active
• computational
• accounts for reaction times and error rates in
  comprehension and production
• language use

• Competence
• static
• representational
• accounts for generalizations about structures and
  relations between sentences and words
• language knowledge

3
Data are DataTheory of linguistic knowledge
the grammar should be involved in an account of
all data

• Judgments of grammaticality
• Judgments of meaning (e.g., synonymy, entailment)
• Reaction times in lexical decision
• Reaction times in phoneme monitoring
• Amplitude of N400 response to content words

4
Non-issues in competence vs. performance
IFrequency

• CAT PORCUPINE
• Is frequency part of these representations?
• (Font size would be proportional to word
  frequency and time to access/use these
  representations would be some function of
  frequency)

5
Non-issues in competence vs. performance
IICategorical distinctions in grammar

• Categorical distinctions from grammatical
  representations do not imply categorical
  decisions, judgments, or behavior
• N Adj N glori os ity
• N Adj N glori ous ness
• Categorical grammatical/ungrammatical distinction
  along one linguistic dimension does not
  immediately imply categorical judgment from
  speakers.

6
New Competence/Performance DistinctionDual
Route Theories

• Generative grammars instantiate a computational
  system that maps between sound and meaning
• These grammars accurately describe speakers
  knowledge of the connection between sound and
  meaning
• However, suppose speakers have alternative
  strategies (computational systems) for
  connecting sound and meaning for particular
  performance needs (comprehension, production)?

7
Dual Route Theories

• raise the question
• What special data does a linguist have that allow
  him/her to develop a true account of linguistic
  computations independent of strategies used in
  language comprehension or production?
• In practice
• Linguists must take strategies seemingly
  supported by psycholinguistic data as competing
  theories of linguistic computation
• That is
• Data are data and the linguist is responsible for
  all the data

8
Impact of Experiments on Linguistic Theory

• A symbolic importance, a reminder of the
  potential testability of competing analyses
• A constraint on linguistic theory from what might
  be called the logical problem of language use
• Clarification of the concrete mechanisms of
  language processing in the brain that allows
  straightforward interpretation of brain and
  behavioral data

9
KIT/MIT MEG Lab
Symbol of potential predictions of linguistic
hypotheses
10
MEG as symbol taming the slovenly linguistic
wilderness

• I placed a jar in Tennessee,
• And round it was, upon a hill.
• It made the slovenly wilderness
• Surround that hill.
• The wilderness rose up to it,
• And sprawled around, no longer wild.
• The jar was round upon the ground
• And tall and of a port in air.

• It took dominion every where.
• The jar was gray and bare.
• It did not give of bird or bush,
• Like nothing else in Tennessee.
• --Wallace Stevens

11
The logical problem of language use

• Linguistic computations for structures that
  speakers use must be computable in real time and
  from the information available to
  speakers/listeners
• Considerations of this logical problem support
  strong locality constraints on information
  dependencies in linguistic representations, for
  example

12
The more we know, the more we can discover

• Mapping linguistic computation in time and in
  brain space increases the relevance of
  psycholinguistic and neurolinguistic
  experimentation for linguistic theory

13
MEG

• Similar to EEG but measures the magnetic field
  around the electric current source (instead of
  electric potentials).

14
Magnetoencephalography (MEG) study of the
brains magnetic fields
http//www.ctf.com/Pages/page33.html
15
Magnetoencephalography (MEG)
EEG
MEG
http//www.ctf.com/Pages/page33.html
16
Magnetoencephalography (MEG)
Distribution of magnetic field at 93 ms (auditory
M100)
Averaged epoch of activity in all sensors
overlapping on each other.
17
Visual Word Recognition (Lexical Decision)
HBM 2003, poster 1345
18
M350
(i) 1st component sensitive to
lexical factors (such as lexical frequency
and sound probability) (ii) not affected
by form competition, e.g. from phonological
neighbors
19
M350
(i) 1st component sensitive to
lexical factors (such as lexical frequency)
(ii) not affected by competition
20
Effect of sound probability/ neighborhood density
(n10)
(Pylkkänen, Stringfellow, Marantz, Brain and
Language, 2002)
21
Auditory and visual M350 for S1
Visual
Auditory
RMS (all left hemisphere sensors)
M350 field pattern
Sagittal view
Sagittal view
M350 location with respect to auditory M100
A
P
A
P
M100
M350
22
Competition effects counteract morphological
priming (in cross-modal priming)

• gave-GIVE no priming
• taught-TEACH yes priming
• walked-WALK robust priming

23
(No Transcript)
24
MEG Evidence for the morphological complexity of
the English Irregular Past Tense

• Linnaea Stockall, Priya Singh, Pranav Anand,
  Justin Fitzpatrick and Alec Marantz
  
  Dept. of
  Linguistics and Philosophy, MIT KIT/MIT MEG Lab
  

25
Prediction

• gave-GIVE
• taught-TEACH
• M350 priming, followed by RT inhibition

26
Method

• Stimuli 4 comparisons
• Identity ghost-ghost vs. trick-ghost
• Irregulars
• high form overlap gave-give vs. plum-give
• low form overlap taught-teach vs. warp-teach
• Orthographic Overlap stiff-staff vs. clap-staff
  
• 20 test and 20 control items per condition
• 320 prime-target pairs
• Plus 320 fillers (NW-NW, W-NW NW-W)

27
Method

• Design
• Visual-visual immediate priming
• (see Pastizzo and Feldman 2002 )

prime

target
450 50
200 0 2500ms
Duration of trial (ms)
28
Results
Behavioral Data (n14)



n.s.
29
Results

• Behavioral Data
• Significant priming for
• Identity condition (p0.0009)
• GAVE-GIVE vs. PLUM-GIVE (p0.03)
• Significant inhibition for
• STIFF-STAFF vs. CLAP-STAFF (p0.01)
• No reliable effect for
• TAUGHT-TEACH vs. WARP-TEACH (p0.21)
• (but trend towards inhibition)

30
Results
M350 Priming (advantage of prime condition over
control) (n8)



n.s.
31
Results

• MEG Data
• Significant priming for
• Identity condition (p0.01)
• GAVE-GIVE vs. PLUM-GIVE(p0.05)
• TAUGHT-TEACH vs. WARP-TEACH (p0.04)
  
• No reliable effect for
• STIFF-STAFF vs. CLAP-STAFF (p0.13)
  
• But trend towards priming

32
What do these results say about the kind of stem
allomorphy involved in English irregular verbs?

• The observed form competition requires that
  allomorphs compete for recognition are these
• Stored allomorphs not explicitly related to each
  other?
• Allomorphs derived via special morpholexical
  rules?
• Allomorphs derived via morphologically triggered
  phonological rules?
• Brain data provides new empirical texture to
  these questions.
• Differential time-course of stem activation and
  form competition allows one to distinguish levels
  and types of stored information.

33
HBM 2003, poster 1345
Magnetoencephalographic indices of the effects of
morphological family frequency

• Liina Pylkkänen
• Department of Linguistics/ Center for
  Neuromagnetism
• New York University

Alec Marantz Department of Linguistics and
Philosophy, KIT/MIT MEG Laboratory Massachusetts
Institute of Technology
34
Effect of lexical frequency

• High frequency words are processed faster than
  low frequency words.
• Prediction of decompositional theories of
  morphology cumulative root frequency effects.

35
Effect of lexical frequency

• High frequency words are processed faster than
  low frequency words.
• Prediction of decompositional theories of
  morphology cumulative root frequency effects.

Same number of derivates
High frequency derivatives
Low frequency derivatives
- ist ize -ism
- ic ize ism
terror
magnet
Matched for surface frequency
36
Cumulative root frequency effects for inflection

• Response times to a noun depend on the cumulative
  frequency of the singular and plural (Schreuder
  Baayen, JML, 1997)
• CAT
• CATS

37
But NO cumulative root frequency effects for
derivation
Schreuder Baayen (1997)

• Family frequency

HIGH
LOW
Family frequency does not affect lexical decision
times.
- ic ize ism
- ist ize -ism
terror
SB Therefore, no decomposition in derivation.
magnet
High family size speeds up lexical decision times.
SB this is a late post-lexical effect.
38
M350
(i) 1st component sensitive to
lexical factors (such as lexical frequency)
(ii) not affected by form competition
39
HBM 2003, poster 1345
M350 not sensitive to interlexical or
allomorphic form competition


SUBLEXICAL FREQUENCY EFFECT
COMPETITION EFFECT
40
Hypothesis
HBM 2003, poster 1345

• High morphological family frequency is associated
  with

null behavioral effect BUT morphological
competition should be distinguishable from
phonological competition
41
Materials 2 categories of singular nouns
HBM 2003, poster 1345
terror
magnet

• Task Lexical decision

42
M350 source analysis
HBM 2003, poster 1345

• Equivalent current dipole analysis
• Latencies and amplitudes measured at points where
  the source amplitude reached 25, 50, 75 and
  100 of the maximum source strength.

43
HBM 2003, poster 1345
Results Lexical decision times (n 10)No
behavioral culmulative root frequency
effect(trend toward inhibitory effect of higher
frequency)
44
Results M350 (S1)
HBM 2003, poster 1345
45
Results M350 (S1)
HBM 2003, poster 1345
46
HBM 2003, poster 1345
Results M350 (S1)
Low family frequency

• Morphological competition at the M350

47
HBM 2003, poster 1345
Results M350 amplitude (n10)
48
HBM 2003, poster 1345
Results M350 amplitude (n10)

49
1. Difference in the time course of competition
High frequency morphological family
High density phonological neighborhood
(frequency-weighted)

• Relationship between target and competitors
  qualitatively different difference is due to
  morphology.

DECOMPOSITION

• Difference is due to the different phonological
  and/or semantic properties of the competitors.

terrorism
TERROR
NO DECOMPOSITION DUAL ROUTE THEORY (DECOMPOSITION
FOR REGULARINFLECTION)
terrorist
terrorize
50
1. Difference in the time course of competition

• Non-decompositional account also predicts
  interference effects in priming for pairs such as
  TERRORISM TERROR.
• BUT this is completely unsupported by data
  effect is robustly facilitory (e.g. a-d).

• Difference is due to the different phonological
  and/or semantic properties of the competitors.

terrorism
TERROR
NO DECOMPOSITION
terrorist
terrorize

• (a) Marslen-Wilson, W. D., Tyler, L., Waksler,
  R., Older, L. (1994). Morphology and meaning in
  the English mental lexicon. Psychological Review
  101, 3-33.
• (b) Pylkkänen, L. Stringfellow, A., Gonnerman,
  L., Marantz, A. 2002. Magnetoencephalographic
  indices of identity and similarity in lexical
  access. In preparation.
• Gonnerman, L. 1999, Morphology and the lexicon
  exploring the semantics-phonology interface, PhD
  thesis, University of Southern California.
• Rastle, K., Davis, M., Marslen-Wilson, W.,
  Tyler, L.K. (2000). Morphological and semantic
  effects in visual word recognition A time course
  study. Language and Cognitive Processes, 15,
  507-538.

51
1. Difference in the time course of competition
High frequency morphological family
High density phonological neighborhood
(frequency-weighted)
DECOMPOSITION

• Competition between morphological family members
  appears to precede competition between
  phonological neighbors.
• An account of the phenomenon needs to make a
  distinction between morphological and
  phonological competitors.

52
Conclusion
HBM 2003, poster 1345
Decomposition
Morphological competition effects
Phonological competition effects
53
Summary

• A misunderstanding of the competence/performance
  distinction has perhaps led to the
  underutilization within linguistics of evidence
  from experiments
• Any retreat from obvious processing implications
  of linguistic theories involves a dual or
  multi-route claim that there might be more than
  one computational path to the connection between
  sound and meaning
• Supporting such multi-route theories would put
  extreme pressure on linguists to explain the
  special source of data for privileged
  linguistic computation
• Rather, any processing strategies proposed in the
  literature should be taken as competing
  linguistic theories and refuted via standard
  theory comparison/linguistic argumentation

54
Comfortable position for linguistics, represented
by the research of Uil-OTS

• Take every bit of linguistic theory as a claim
  about necessary computation in linguistic
  performance (no independent strategies for
  sound/meaning connections no multiple routes to
  linguistic representations)
• Take every bit of behavioral and brain data as
  potentially decisive between competing linguistic
  hypotheses