Language electrified

1
Language electrified

• Department of Psychology
• Maastricht University
• Bernadette Schmitt
• Blok 3.2 Neurocognition
• October 1999

2
Language

• Speaking is one of the most
• complex cognitive skills in humans
• Automatic
• Fast
• 150 words per minute
• 6 syllables per second
• 1 out of 1000 words incorrect

3
Language

• ERPs in Comprehension
• ERPs in Production
• Language theories

4
(No Transcript)
5
Oh boy, gimme a big hug!!
6
Oh boy, gimme a big hug!!
Oh man, thats what wea already doin, but I
wont tellya.
7
Speech Production (Levelt)
8
Speech Comprehension (Cutler)
9
Units of speech perception

• Sound waves
• Phonemes
• Syllables
• Words
• Sentences

10
Example of a syllable detection experiment

• present words to participants
• task detect phonemes group /bal/ in
• balcony
• ballet
• detection time shorter in balcony -gt syllable
• but where is the syllable in the signal?
• and when is it processed?

11
Difficulties

• physical signal does not show these units
• segmentation problem
• variability problem

12
Variability

• Real world
• speech
• slow vs. fast
• within/between
• speakers
• Segmentation
• no start/end

13
Alternative methods

• high temporal resolution methods
• ERPs
• phonological encoding
• meaning encoding
• syntactic encoding
• discourse processing

14
Phonological encoding and ERP

• When is phonological information processed?
• Subject hears pairs of words
• phonologically related at the beginning cap -
  cat
• phonologically related at the end hat - cat
• unrelated control sun - cat
• see where ERP signals are different
• Praamstra, Meyer, and Levelt (1994)

15
Phonological encoding estimated by ERPs
16
Conclusion ERP on phonology

• negativity around 400 ms
• N400 reduced for related words
• 250 - 450 ms for onset related pairs
• 450 - 700 ms for rhyme related pairs
• time course of phonological encoding
• distribution of N400 similar to standard N400
• Phonology Semantics ???

17
Integration of meaning
18
Integration of meaning

• Present sentences to subjects word by word
• one with normal meaning
• one with weird meaning
• Kutas and Hillyard (1980)
• He was stung by a bee
• He was stung by a mile

19
Integration of meaning

• Present sentences to subjects word by word
• one with normal meaning
• one with weird meaning
• Kutas and Hillyard (1980)
• He was stung by a bee
• He was stung by a mile

20
(No Transcript)
21
19 years of N400 research

• Review by Kutas and Van Petten, 1994
• integration component because all words in a
  sentence generate N400
• word frequency larger for low frequency words
• concreteness larger for abstract words
• repetition reduced N400 if word is repeated
• semantic relation reduced N400 if two words are
  related

22
19 years of N400 research

• integration of words into sentence
• N400 gets smaller the further into the sentence
  it occurs
• Control study syntactic prose
• Colorless green ideas sleep furiously
• no N400 reduction across the sentence

23
N400 and discourse (Van Berkum et al., 1997)
24
N400 and cognitive development
25
Conclusion ERP and Meaning

• negativity around 400 ms
• N400 reduced for expected words both in sentence
  and discourse
• sensitive to semantic relations
• integration of meaning takes place around 400 ms
  after word onset

26
Integration of syntax
27
Integration of syntax

• Present sentences word by word
• one with normal syntax
• one with incorrect syntax
• My pet aardvark prefers to eat potatoes
• My pet aardvark prefer to eat potatoes

28
Integration of syntax

• Present sentences word by word
• one with normal syntax
• one with incorrect syntax
• My pet aardvark prefers to eat potatoes
• My pet aardvark prefer to eat potatoes

29
Integration of syntax

• Present sentences word by word
• one with normal syntax
• one with incorrect syntax
• My pet aardvark prefers to eat potatoes
• My pet aardvark prefer to eat potatoes

30
Integration of syntax Osterhout et al., 1992
Hagoort et al., 1993)
31
Ambiguity detection (Hagoort and Brown, 1994)
32
P600 in syntactic prose(Muente et al., 1997)
33
Conclusion ERP and Syntax

• P600/SPS
• Syntactic anomalies
• Ambiguities
• in normal sentences and prose
• not in pseudo-word sentences
• P600/SPS related to syntax in the context of
  available mental representations

34
Sentence processing and Memory
35
Sentence processing and Memory

• Let subjects read sentence
• easy ones
• difficult ones
• King and Kutas, 1995 Mueller et al., 1997
• The aardvark that really scared the cop ran into
  the bushes.
• The aardvark that the cop scared ran into the
  bushes.

36
Sentence processing and Memory

• Let subjects read sentence
• easy ones
• difficult ones
• King and Kutas, 1995 Mueller et al., 1997
• The aardvark that really scared the cop ran into
  the bushes.
• The aardvark that the cop scared ran into the
  bushes.

37
Sentence processing and Memory

• Let subjects read sentence
• easy ones
• difficult ones
• King and Kutas, 1995 Mueller et al., 1997
• The aardvark that really scared the cop ran into
  the bushes.
• The aardvark that the cop scared ran into the
  bushes.

38
Language and Memory
Left anterior negativity (LAN)
39
Conclusions on relation of language and working
memory

• Left anterior negativity (LAN)
• more negative if difficult process
• sensitive to syntactic structures
• sensitive to semantic integration
• if something has to keep activated for a while

40
Take home message ERPs in comprehension

• Phonological component (N400 like)
• Meaning integration component (N400)
• Syntactic component (P600/SPS)
• Memory component (LAN)
• Open Interaction of these processes

41
BREAK
42
ERPs in language production

• The theory
• Some examples of RT experiments
• ERP I LRP
• ERP II N200

43
A psycho-linguistic view of language

• Intention to say or understand something
• Meaning
• Syntax
• Sound
• Articulation

44
(No Transcript)
45
Speaking

• a picture -gt speech planning -gt
  naming

46
Speaking

• a picture -gt speech planning -gt
  a response

DOCTOR
47
Speaking

• a picture -gt speech planning -gt
  a response

DOCTOR
800 ms
48
Manipulate meaning access

• Picture-Word Interference Paradigm
• naming Doctor, hearing Nurse 880 ms
• naming Doctor, hearing Sun 830 ms
• --------
• slow down 50 ms
• semantic interference effect

49
Manipulate syntactic access

• Picture-Word Interference Paradigm
• is Doctor a de or het word,
• seeing het 680 ms
• is Doctor a de or het word,
• seeing de 620 ms
• --------
• slow down 60 ms
• syntactic incongruency effect

50
Manipulate phonological access

• Picture-Word Interference Paradigm
• naming Doctor, hearing Doll 800 ms
• naming Doctor, hearing Sun 830 ms
• --------
• speed up 30 ms
• phonological facilitation effect

51
Time course
Meaning
Phonology
Visual
Levelt et al., 1991
52
Time course
Meaning
Phonology
Visual
word
SOA
Levelt et al., 1991
53
Time course
Meaning
Phonology
Visual
word
SOA
Levelt et al., 1991
54
Time course
Meaning
Phonology
Visual
word
word
SOA
Levelt et al., 1991
55
Time course
Meaning
Phonology
Visual
word
word
SOA
Levelt et al., 1991
56
Time course
Semantics
Phonology
Effect size
SOA
Word second Picture first
Word first Picture second
Schriefers et al., 1990
57
Dry cognitive theory

• Test the model in reaction time experiments
• Problem
• reaction times only show the end of the process

58
Other methods?

• Look into the process during its run
• Event related potentials

59
Todays ERP components

• LRP (lateralized readiness potential)
• N200

60
LRP

• derivative from the Bereitschaftspotential
• button press tasks
• ramp-shape activation above the motor cortex
• planning of movement (before actual movement)
• in go and nogo responses
• maximum contra lateral to movement
• maximum at electrode sites C3 and C4

61
LRP
62
LRP in language production

• preparation to respond
• indicates when specific information becomes
  available
• dual choice go/nogo paradigm (Van Turennout et
  al., 1997, 1998)
• two decisions
• one is based on semantics
• one is based on phonology

63
LRP
design
64
LRP two major conditions

• hand semantics
• left/right hand response preparation on semantics
• go/nogo decision contingent on phonology
• hand phonology
• left/right hand response preparation on phonology
• go/nogo decision contingent on semantics

65
LRP Hypothesis

• hand semantics
• if semantics precedes phonology

GO
mV
NOGO
time
Meaning
Phonology
Visual
66
LRP Hypothesis

• hand phonology
• if semantics precedes phonology

GO
mV
NOGO
time
Meaning
Phonology
Visual
67
(No Transcript)
68
LRP Conclusions

• nogo LRP in hand semantics
• no nogo LRP in hand phonology
• data fit hypothesis
• semantic encoding precedes phonological encoding
  by about 120 ms

69
Second ERP component N200

• go/nogo paradigm
• enhanced negativity for nogos compared to gos
• maximum at frontal sites
• related to response inhibition
• Sasaki and Gemba, 1989, 1993
• Single cell recordings in monkeys

70
N200 in language processing

• nogo - go difference wave
• onset and peak of the effect
• moment in time when specific information is
  available

71
N200 Meaning vs. N200 Sound

• two kinds of information processes
• a meaning process and a sound process
• use pictures to trigger the encoding
• ask participants to decide
• Is there an animal on the picture or an object?
• Does the pictures name start with a vowel or a
  consonant?

72
N200
design
73
N200 Meaning vs. N200 Sound

• go and nogo responses based on sound information
• reverse the instruction
• show the same pictures again
• go and nogo responses based on meaning information

74
Meaning vs. Sound
- 4 mV
400
NOGO
GO
75
- 4 mV
Semantics
Phonology
400
Difference
NOGO
GO
76
N200 Conclusions

• N200 peak in go/nogo semantics around 380 ms
• N200 peak in go/nogo phonology around 470 ms
• 90 ms head start for semantics
• semantic information available earlier than
  phonological information
• N200 data go hand in hand with LRP results

77
Is the N200 general or specific?

• test in other go/nogo decision tasks
• localize the effect with fMRI

78
fMRI