Prosody-driven Sentence Processing: An Event-related Brain Potential Study Ann Pannekamp, Ulrike Toepel, Kai Alter, Anja Hahne and Angela D. Friederici Presented by Laura Matzen, 9/1/2005

1
Prosody-driven Sentence Processing An
Event-related Brain Potential StudyAnn
Pannekamp, Ulrike Toepel, Kai Alter, Anja Hahne
and Angela D. FriedericiPresented by Laura
Matzen, 9/1/2005
2
Pannekamp et al. (2005)

• Goal of study
• Determine what causes the closure positive shift
  (CPS)
• Is this effect driven by prosody alone or by
  other factors?

3
Pannekamp et al. (2005)

• Basic Design
• Systematically reduce linguistic content of
  sentences
• Record ERPs in each condition to see if CPS is
  present in all cases or to see how it changes

4
Background onEvent-related Potentials (ERPs)
5
Background onEvent-related Potentials (ERPs)
6
(No Transcript)
7
(No Transcript)
8
(No Transcript)
9
(No Transcript)
10
(No Transcript)
11
Background

• Closure Positive Shift (CPS)
• seen in response to normal spoken sentences
• somehow related to intonation contour
• Intonation contour
• sentence melody
• provides information about syntactic structure,
  sentence mode
• fundamental frequency F0

12
Frequency
Actually, we were the ones who said we didnt
want a regular cake, so you can blame it on us.
13
Frequency
Actually, we were the ones who said we didnt
want a regular cake, so you can blame it on us.
14
Intonational Phrases (IPh)

• One or more in each sentence
• Group and organize words into phrases
• Structure is usually determined by syntax

15
Intonational Phrases (IPh)

• ...defined as containing at least one nuclear
  accent and a boundary tone at their right edges
• Pitch contour drops at the end, resets at start
  of the next IPh
• Last syllable usually lengthened
• Often followed by a pause

16
Examples

• She went to the store, then she picked up the dry
  cleaning on the way home.
• She went to the store, then she dropped the kids
  off at school, then she went home.

17
Intonational Phrases (IPh)

• Seem to be secondary to syntax in some ways
• supporting role
• BUT, listeners can identify prosodic boundaries
  in the absence of semantic and syntactic
  information (de Rooij, 1975)

18
Intonational Phrases (IPh)

• Beckman (1996)
• the prosodic structure of the utterance has to
  be seen as a full grammatical property also
  requiring its own parsing
• However, it might be harder to process prosody in
  absence of other linguistic information
• Off-line behavioral studies cant address this
  issue

19
Steinhauer et al. (1999)

• Studied prosody with ERPs
• Two sentence types 1 or 2 IPh boundaries
• Saw positive-going waveform in response to all
  boundaries CPS
• Possible Confound
• Close relationship between prosody and syntax
• CPS could still be related to processing
  syntactic boundaries, not just prosodic boundaries

20
Steinhauer Friederici (2001)

• ERP study with delexicalized speech
• Filtered to removed phonemic, semantic and
  syntactic information
• Only prosodic info left (pitch, amplitude,
  rhythm)
• ? CPS in this case could only be caused by prosody

21
Steinhauer Friederici (2001)

• Results
• Strange-looking CPS at first boundary
• No CPS at second boundary
• Contingent negative variation (CNV) across whole
  sentence
• Is this because the sentences are so unnatural?
• Is this task completely different from
  language processing?

22
Meyer et al. (2002)

• fMRI study
• Very different responses to natural speech and
  delexicalized speech
• Response to prosody stronger in right hemisphere
• strong evidence that pitch processing in the
  absence of additional linguistic information such
  as syntax and/or semantics takes place in right
  hemisphere regions

23
Current Experiments

• Motivations
• Investigate partial replication of CPS findings
  by Steinhauer Friederici (2001)
• Why didnt they get the expected results?
• Try using more natural stimuli
• See if CPS shifts to the right hemisphere as
  segmental information is removed

24
Experiments

• Experiment 1 Normal sentences
• Normal semantic, syntactic and phonemic info
• Experiment 2 Jabberwocky sentences
• Remove semantic information
• Experiment 3 Pseudo sentences
• Remove semantic and syntactic information
• Experiment 4 Hummed sentences
• Remove semantic, syntactic, and phonemic info
  (only prosody left)

25
Expt 1 Normal Sentences

• A1 Kevin promises mom to sleepIPh1
• and to be a good boy for a while.
• B1 Kevin promisesIPh1 mom to kissIPh2
• and to be a good boy for a while.

26
Expt 1 Normal Sentences

• A1 one IPh boundary at 1950 msec
• B1 two IPh boundaries at 950 and 2700 msec
• First part (subject verb) of sentence longer
  than A1
• High boundary tone at end of first part
• Longer pause after first part
• Both conditions have IPh boundary after second
  verb (marked by high boundary tone)

27
(No Transcript)
28
(No Transcript)
29
Expt 2 Jabberwocky Sentences

• All content words replaced with pseudo words
• A2 The bater rabels Onna to lubolIPh1
• and the rado to nupe.
• B2 The bater rabels IPh1 Onna to lubolIPh2
• and the rado to nupe.

30
Expt 2 Jabberwocky Sentences

• A2 one IPh boundary at 2100 msec
• B2 two IPh boundaries at 1100 and 2600 msec
• First part (subject verb) of sentence longer
  than A2
• High boundary tone at end of first part
• Longer pause after first part
• Both conditions have IPh boundary after second
  verb (marked by high boundary tone)

31
(No Transcript)
32
Expt 3 Pseudo Sentences

• All function and content words replaced with
  pseudo words
• A3 Bater saklimm Onna ko labei keg IPh1
• nug som Rado lie nupes.
• B3 Bater saklimmIPh1 Onna ko labei kegIPh2
  nug som Rado lie nupes.

33
Expt 3 Pseudo Sentences

• A3 one IPh boundary at 2000 msec
• B3 two IPh boundaries at 920 and 2400 msec
• First part (subject verb) of sentence longer
  than A3
• High boundary tone at end of first part
• Longer pause after first part
• Both conditions have IPh boundary after second
  verb (marked by high boundary tone)

34
(No Transcript)
35
Expt 4 Hummed Sentences

• A4 mm mmm mmmm mm mmmmIPh1
• mmmm mmm mmm mmm mmmmm.
• B4 mm mmmIPh1 mmmm mm mmmIPh2 mmmm mmm
  mmm mmm mmmmm.

36
Expt 4 Hummed Sentences

• A4 one IPh boundary at 1850 msec
• B4 two IPh boundaries at 850 and 2150 msec
• Longer pause after first part
• High boundary tone at end of first part
• (First part itself is NOT longer in this case)
• Both conditions have IPh boundary after second
  verb (marked by high boundary tone)

37
(No Transcript)
38
Procedure

• Subjects only saw one type of sentence
• Auditory presentation for all sentences
• Trial followed by a probe word
• Subjects had to say whether word was in sentence
  or not
• (Words were placed randomly into hummed filler
  sentences)

39
Data Collection

• 23 Ag/AgCl electrodes
• 200 msec prestimulus baseline
• Averages computed over whole sentences 4500
  msec segments

40
Results
41
A1 IPh at 1950 ms Response at 2000
ms ------------------------------ B1 IPh1 at 950
ms Response at 1500 ms IPh2 at 2700
ms Response at 2700 ms
42
A2 IPh at 2100 ms Response at 2200
ms ------------------------------ B2 IPh1 at
1100 ms Response at 1500 ms IPh2 at 2600
ms Response at 2800 ms
43
A3 IPh at 2000 ms Response at ms ----------------
-------------- B3 IPh1 at 920 ms Response at
1500 ms IPh2 at 2400 ms Response at 2500 ms
44
A4 IPh at 1850 ms Response at 2200
ms ------------------------------ B4 IPh1 at 850
ms Response at 1000 ms IPh2 at 2150
ms Response at 2000-2500 ms Negative peak from
500-1000 ms
45
Waveforms measured from offset of 1st sentence
fragment
46
Their interpretation...

• Positivity resembles CPS
• Observed in all experimental conditions
• Related to processing of prosodic boundaries
• Scalp distribution changes across conditions

47
Their interpretation...

• CPS associated with first IPh seems related to
  the amount of segmental content in sentences
• Moved forward and rightward as info decreased
• CPS associated with second IPh seems UNrelated to
  amount of segmental content

48
Scalp Distributions IPh1

• Experiment 1 CPS over whole head
• Experiment 2 CPS at midline and lateral sites,
  also moves to anterior sites
• Experiment 3 Moves to right anterior sites
• Experiment 4 CPS broadly distributed

49
Scalp Distributions

• First IPh
• CPS moved anterior sites as linguistic
  information decreased
• CPS for psuedo sentences moved rightward, but not
  for hummed sentences
• (possible explanation left hemisphere
  processing timing of hums?)

50
Scalp Distributions

• Second IPh
• CPS distributed broadly over midline sites
• also over lateral sites for normal and pseudo
  sentence
• (why not for other types of sentences?)
• CPS moved rightward for hummed sentences ONLY
• (CPS not seen here at all in previous study-
  conclude that naturalness is important)

51
Negativity

• Early negativity in hummed sentences
• Different processing mechanism for pure prosody?
• Search for early accent (that isnt there)?
• Different task demands?

52
Their Conclusions

• CPS is independent of expt manipulations
• Seems to be related to perception of prosodic
  boundaries
• Dependent exclusively on prosody
• The observed differences in the scalp
  distribution of the CPS as a function of the
  segmental content of the acoustic speech stream
  suggest that prosodic processing interacts with
  other information types involving different
  systems.

53
Questions

• Why did scalp distribution change?
• (It doesnt seem to change in a systematic way)
• Why are there different results for the 1st and
  2nd IPh boundaries?
• Are the differences in timing meaningful?
• Why not compare sentences with no IPh boundaries?
  (one phrase)
• Is it reasonable to say that these are natural
  manipulations?

54
(No Transcript)