The role of the prenuclear F0 region in the perception of German questions and statements

1
The role of the prenuclear F0 region in the
perception of German questions and statements

• Caterina Petrone Oliver Niebuhr
• Zentrum für Allgemeine Sprachwissenschaft,
  Berlin
• Laboratoire Parole et Langage, Aix-en-Provence

2
Whats a tune?

• Sequence of static tones (cf. AM model,
  Pierrehumbert,1980) or contour elements (KIM,
  Kohler, 1991)

- For pitch accents nuclear prenuclear
- No structural (only semantic) restrictions
for combinations of pitch accents edge
components

• Compositionality -gt Intonational meaning given by
  the independent contribution of individual tones
  (Pierrehumbert Hirschberg, 1990, inter alia)

Nucleus (nuclear accent terminal
edge components) essential in conveying meaning
BUT
English
Tunes
Sentence mode
Statement
H
L-
L



Yes/No question
L
H-
H



Prenuclear region?
3
Q/S in Neapolitan
Introduction Experiment I Discussion Expe
riment II Discussion Summary Conclusion

• Intonation only mean to distinguish Yes/No Q
  vs.S
• - Late vs. Early nuclear rise (DImperio,
  2000)
• - Convex vs. Concave prenuclear fall
  (Petrone, 2008)

LH
L-L
(LH)
shallow concave
Q
mam
vuo
La
le
ve
de
ra
na?
ma
la
re
(LH)
LH
L-L
S
steep convex
La
ra
na
mam
vuo
le
de
re
la
ma
ve
Tr. The mom wants to see the frog
4
Results

• Q/S effect already in the prenuclear accent region

• Score decreases for statement-base when a steep
  region is heard

• Score increases for question-base stimuli only
  when the nuclear accent is present

5
Q/S in German

• Marked by syntactic (subject-verb inversion),
  lexical (e.g., wh-words) and/or intonational
  means (final F0 rise/fall)

Statement
Yes/No question

AND

• Questions with final fall Dialogue partner is
  supposed to give a short answer according to the
  speakers expectation (cf. Stock 1996 Kohler
  2004 Peters 2005).

6
Q/S in German

• Peters (2005)

(L)H
L-
L
H
Sind sie Heidelbergerin
(Are you from Heidelberg)
Give me just a short answer (Yes/No), please
7
Q/S in German

• Peters (2005)

(L)H
L-
L
H
Sind sie Heidelbergerin
(Are you from Heidelberg)
Give me just a short answer (Yes/No), please
H
(L)H
H-
H
Sind sie Heidelbergerin
(Are you from Heidelberg)
Answer Yes/No Tell me a bit more about you!
8
Q/S in German

• Peters (2005)

(L)H
L-
L
H
Sind sie Heidelbergerin
(Are you from Heidelberg)
Give me just a short answer (Yes/No), please
H
(L)H
H-
H
Sind sie Heidelbergerin
(Are you from Heidelberg)
Answer Yes/No Tell me a bit more about you!

• Kohler (2004)

(L)H
L-
L
H
Würde Ihnen das passen
(Would that suit you)
Say yes, please
9
Q/S in German

• Peters (2005)

(L)H
L-
L
H
Sind sie Heidelbergerin
(Are you from Heidelberg)
Give me just a short answer (Yes/No), please
H
(L)H
H-
H
Sind sie Heidelbergerin
(Are you from Heidelberg)
Answer Yes/No Tell me a bit more about you!

• Kohler (2004)

(L)H
L-
L
H
Würde Ihnen das passen
(Would that suit you)
Say yes, please
(L)H
H-
H
H
Würde Ihnen das passen
(Would that suit you)
The final choice is up to you!
10
Q/S in German

• Also questions with declarative syntax are
  possible

11
Q/S in German

• Also questions with declarative syntax are
  possible

(L)H
H
H-
H
Katherina sucht ne Wonhung
(Katherina searches for a flat)

• In such cases the intonation is crucial, i.e. It
  MUST be raising. Isacenko Schädlich (1970)
• If an utterance contains no other syntactic or
  lexical cue to identify it as a question then
  only the last rising tone-switch provides the
  necessary information to allow the hearer to
  identify it as a question" (p.32)
• (Cf. also Huddleston (1994) for English and
  Haeseryn et al. (1997) for Dutch)
• gt known as intonation question or
  queclarativ (Sadock 1974)

12
Q/S in German

• Characteristic function of such questions
  request for confirmation

BUT

• with regard to the assumed functional differences
  between final rises/falls in German questions

Q1 Is the final rise actually mandatory in
German intonation questions, i.e. does
it mean question ?
(b) and in view of the domain and the cues for
question in Neapolitan Italian
Q2 What role plays the prenuclear region?
Our study started from observations in
natural utterances
13
Q/S in German
H
H
L-L
Statement
steep convex
Ka
the
ri
na
sucht
ne
Woh
nung
H
LH
H-H
Yes/no Question
shallow concave
Ka
the
ri
sucht
Woh
nung?
na
ne
14
Q/S in German
H
H
L-L
Statement
steep convex
Ka
the
ri
na
sucht
ne
Woh
nung
H
LH
H-H
Yes/no Question
shallow concave
Ka
the
ri
sucht
Woh
nung?
na
ne
H
Intonation Question?
LH
L-L
shallow concave
Ka
the
ri
sucht
Woh
nung?
na
ne
15
Experiment I

• Similarly to Neapolitans, German listeners are
  able to identify Q/S sentences, i.e. asserting
  and questioning speech acts, well before hearing
  the nucleus
• Differences in the prenuclear rise/fall cue
  questionhood independent of the presence of a
  terminal rise and interrogative syntax

 Questionhood 
 Assertion 
late rise-fall alignment concave fall
early rise-fall alignment convex fall
16
Corpus

• Natural Utterances Katherina sucht ne Wohnung

• 5 intonation Bases

(1)
H HL L-
early rise-fall alignment convex fall
S
(2)
H H L-
(3)
H LH L-
late rise-fall alignment concave fall
(4)
H LH H-
Q
(5)
H LH L-

• (1)-(3) nuclear pitch-accent difference with
  final fall. Known to signal meaning differences
  within statements, i.e. settled, open,
  astonished (Niebuhr 2007 Grice Baumann 2000
  Kohler 1987)
• (4) tune with final rise. Known to signal
  questions. The contour preceding the terminal
  mouvement is constant for
• (3) (4) gt clear Q/S difference
• - (3)(5) difference in prenuclear region (H
  alignment and shape/alignment of subsequent fall)
  gt yields Q/S difference ?
• (4)(5) difference in final fall vs. final rise
  with constant preceding tune gt yields Q/S
  difference ?

17
Corpus

• Natural Utterances Katherina sucht ne Wohnung

• 5 intonation Bases

(1)
H HL L-
early rise-fall alignment convex fall
S
(2)
H H L-
(3)
H LH L-
late rise-fall alignment concave fall
(4)
H LH H-
Q
(5)
H LH L-

• 3 tune fragments short

Ka
the
ri
sucht
Woh
nung?
na
ne
18
Corpus

• Natural Utterances Katherina sucht ne Wohnung

• 5 intonation Bases

(1)
H HL L-
early rise-fall alignment convex fall
S
(2)
H H L-
(3)
H LH L-
late rise-fall alignment concave fall
(4)
H LH H-
Q
(5)
H LH L-

• 3 tune fragments medium

Ka
the
ri
sucht
Woh
nung?
na
ne
19
Corpus

• Natural Utterances Katherina sucht ne Wohnung

• 5 intonation Bases

(1)
H HL L-
early rise-fall alignment convex fall
S
(2)
H H L-
(3)
H LH L-
late rise-fall alignment concave fall
(4)
H LH H-
Q
(5)
H LH L-

• 3 tune fragments long

Ka
the
ri
sucht
Woh
nung?
na
ne
20
Methods

• Semantic differential task
• Three 7-point scales aiming at the Q-S
  difference
• (1) astonished - not astonished
• (2) questioning - not questioning
• (3) uncertain - certain

The speaker sounds
3
2
1
0
-1
-2
-3
certain
uncertain
21
Methods

• Semantic differential task
• Three 7-point scales aiming at the Q-S
  difference
• (1) astonished - not astonished
• (2) questioning - not questioning
• (3) uncertain - certain

• Procedure
• One randomized block containing short and medium
  tune fragments
• Long sentences at the end of the session
• 11 German subjects X 9 repetitions (1782 obs.)

22
Methods

• Semantic differential task
• Three 7-point scales aiming at the Q-S
  difference
• (1) astonished - not astonished
• (2) questioning - not questioning
• (3) uncertain - certain

• Procedure
• One randomized block containing short and medium
  tune fragments
• Long sentences at the end of the session
• 11 German subjects X 9 repetitions (1782 obs.)

• Statistical analysis Linear mixed model
• - Fixed Scales Tune fragments
• - Random Subjects
• - p lt .05

23
Results
(1)
(2)
H H L-
H HL L-



n.s.
n.s.
n.s.
In line with Niebuhr (2007), stimuli with early
pren. rise/fallconvex fall
(3)
H LH L-

• Judged as assertive already in the prenuclear
  region

• Assertiveness increases as the terminal fall is
  heard with HL (1), but NOT with H (2)

• In (3), the LH accent conveys more astonishment

24
Results
(4)
(5)
H LH H-
H LH L-






Stimuli with late prenuclear rise concave fall

• Judged as questioning already in the prenuclear
  region, i.e. independent of the presence the
  terminal rise

• Adding the nucleus increases questioning in
  long stimuli

• Effects stronger in intonation base H-H (4) than
  in L-L (5)

• Overall significant difference in responses
  between (1)-(2)-(3) vs. (4)-(5) base types for
  both short and long stimuli

25
Discussion
The intonational signalling of pragmatic
functions of sentences in German is not bound to
the nucleus and the subsequent final F0
pattern
1. Questioning vs. assertive sentences are well
discriminable when only the prenuclear accent
region is left in the stimulus
This cannot depend on the phonological
specification of the prenuclear accent -(H) for
the 5 intonation types- though phonetic factors
such as speech rate or intensity might have
affected listeners judgements
26
Discussion
The intonational signalling of pragmatic
functions of sentences in German is not bound to
the nucleus and the subsequent final F0
pattern
1. Questioning vs. assertive sentences are well
discriminable when only the prenuclear accent
region is left in the stimulus
This cannot depend on the phonological
specification of the prenuclear accent -(H) for
the 5 intonation types- though phonetic factors
such as speech rate or intensity might have
affected listeners judgements
2. Contrast in listeners judgement for stimuli
(1)-(2)-(3) vs. (4)-(5) already in the short
condition
This might be due to the differences in the
alignment of the prenuclear rise and/or the shape
of the fall between the two intonation groups
27
Discussion
3. Terminal falling intonations can signal a
question, even when not marked syntactically
Stimuli with patterns (5), ie. containing a
terminal fall but preceded by a late
rise-fall/concave fall, shifted the judgements
towards more astonished, uncertain and
questioning
28
Discussion
3. Terminal falling intonations can signal a
question, even when not marked syntactically
Stimuli with patterns (5), ie. containing a
terminal fall but preceded by a late
rise-fall/concave fall, shifted the judgements
towards more astonished, uncertain and
questioning
4. The perception of intonation modality is
improved when the nucleus is also available,
especially in questioning stimuli (4)-(5)
This suggests that prosodic cues in the
prenuclear region are less stronger for Q than
for S modality, so that listeners have to rely
more on the nuclear pattern when perceiving Q
(see also Petrone DImperio, 2008 for
Neapolitan)
29
Experiment II
German listeners capitalize on differences in the
prenuclear F0 region in the perception of Q/S
utterances
30
Experiment II
German listeners capitalize on differences in the
prenuclear F0 region in the perception of Q/S
utterances
Rise timing?
31
Experiment II
German listeners capitalize on differences in the
prenuclear F0 region in the perception of Q/S
utterances
Rise timing?
Fall timing?
32
Experiment II
German listeners capitalize on differences in the
prenuclear F0 region in the perception of Q/S
utterances
Rise timing?
Fall timing?
Fall slope?
33
Experiment II
German listeners capitalize on differences in the
prenuclear F0 region in the perception of Q/S
utterances
Rise timing?
Fall timing?
Fall slope?
Fall shape?
34
Corpus

• 2 base types Resyntheses based on astonished
  statement intonation question (H LH L-)
• F0 (nuclear) F0 peaks of the 2 base types merged
  into a constant, intermediate pattern for all
  stimuli.
• F0 (prenuclear) 2 rise alignments (early/late) X
  4 fall slopes (or fall alignments) X 3 fall
  shapes (linear/concave/convex)

80 ms
Late peak
Early peak
time
35
Corpus

• 2 base types Resyntheses based on astonished
  statement intonation question (H LH L-)
• F0 (nuclear) F0 peaks of the 2 base types merged
  into a constant, intermediate pattern for all
  stimuli.
• F0 (prenuclear) 2 rise alignments (early/late) X
  4 fall slopes (or fall alignments) X 3 fall
  shapes (linear/concave/convex)

Late peak
Early peak
50 ms
time
T4
T3
T1
T2
36
Corpus

• 2 base types Resyntheses based on astonished
  statement intonation question (H LH L-)
• F0 (nuclear) F0 peaks of the 2 base types merged
  into a constant, intermediate pattern for all
  stimuli.
• F0 (prenuclear) 2 rise alignments (early/late) X
  4 fall slopes (or fall alignments) X 3 fall
  shapes (linear/concave/convex)

Late peak
Early peak
50 Hz
(see DImperio Cangemi, PAPI 2009)
time
T4
T3
T1
T2
37
Methods

• Procedure
• - Indirect identification test Does it match?

Test sentence
Katherina wants to become a painter
Really? Thats a risky step
Context
YES astonished statement
NO intonational question
- 11 German listeners x 5 repetitions (2460 obs.)
38
Methods

• Procedure
• - Indirect identification test Does it match?

Test sentence
Katherina wants to become a painter
Really? Thats a risky step
Context
YES astonished statement
NO intonational question
- 11 German listeners x 5 repetitions (2460 obs.)

• Statistical analysis Generalized Mixed Model
• - Fixed rise and fall alignment, slope, shape,
  base type
• - Random Subjects
• - p lt .05

39
Results Timing Slope
0.5

• Perception of astonished statement decreases
  around the chance level as the end of the fall is
  shifted later

• This effect is stronger for earlier peak
  alignment at T3 T4

• Effects of the slope indistinguishable from those
  of the timing

• Small Base Type effect

40
Results Shape
Late
Early
Early
Late
0.5

• Strong interaction shape by fall alignment in
  early peak the perception of astonished
  statement goes much below the chance level for
  concave shape stimuli at late fall alignments

• Small shape effects in late peak Why?

41
Discussion

• German listeners exploit the F0 prenuclear region
  for
• Q/S perception, even in absence of a clear
  information
• from the nucleus
• Alignment and dynamic cues distributed in the
  prenuclear F0 region seems to be at work when
  perceiving the Q/S contrast in German
• The early fall alignment is a robust cue for
  astonished statements perception. The late fall
  alignment is more ambiguous, and additional
  differences in the shape of the fall become
  crucial for question perception.

S
Early fall

Q
Late concave fall
42
Discussion
2. The fall alignment/shape manipulation affected
listeners judgment only when the prenuclear rise
is early This can be due to our manipulation
the shift from the early to the late alignment
could have been too far. The shape difference
is more salient when the rise is earlier
43
Discussion

• 2. The fall alignment/shape manipulation affected
  listeners judgment only when the prenuclear rise
  is early
• This can be due to our manipulation the shift
  from the early to the late alignment could have
  been too far. The shape difference is more
  salient when the rise is earlier
• The base stimulus produce a small but significant
  effect.
• This means that cues other than F0 (voice
  quality, speech rate, etc.) might have been
  exploited by listeners

44
Conclusion

• Results from (Neapolitan and) German indicate
  that the prenuclear F0 region is relevant in
  conveying pragmatic functions, and thus it should
  be taken into account by theories of intonational
  meaning

45
Conclusion

• Results from (Neapolitan and) German indicate
  that the prenuclear F0 region is relevant in
  conveying pragmatic functions, and thus it should
  be taken into account by theories of intonational
  meaning
• The influence of the prenuclear F0 region is
  accounted for by the interaction of multiple F0
  dimensions (alignment, shape), thus suggesting
  that dynamic properties might help in
  interpreting linguistic information

46
Conclusion

• Results from (Neapolitan and) German indicate
  that the prenuclear F0 region is relevant in
  conveying pragmatic functions, and thus it should
  be taken into account by theories of intonational
  meaning
• The influence of the prenuclear F0 region is
  accounted for by the interaction of multiple F0
  dimensions (alignment, shape), thus suggesting
  that dynamic properties might help in
  interpreting linguistic information

• Our results suggest that intonation meaning is
  defined by the contour as a whole the
  interrelation between tones in a tune cannot be
  captured by a strict compositional approach

47
Grazie/Danke!
48
Corpus

• 2 Base Types Resynthesized stimuli from an
  astonished statement an intonational question
  (H LH L-)

Ka
ri
the
na
will
ne
Ma
le
rin
werden
ri
Ka
the
na
will
ne
Ma
le
rin
werden?
49
Whats a tune?
Tunes of utterances can be decomposed into
sequenc- es of static tones (cf. AM model,
Pierrehumbert,1980) or contour elements (KIM,
Kohler 1991)

• For pitch accents nuclear prenuclear

• No structural (only semantic) restrictions for
  com-binations of pitch accents edge components

For example AM approach (Pierrehumbert 1980)
50
Meaning compositionality
(Pierrehumbert Hirschberg, 1990)

• Intonational meaning given by the independent
  contribution of individual tones

English
The accented item is instantiated in the
open expression to be added to Hearers mutual
belief
H

The interpretation of the current ip does
not depend on that of subsequent ips
L-

•  Nucleus  (nuclear accent terminal edge
  components) is essential for conveying meaning

Sentence mode
Tunes
Statement
H
L-
L



Yes/No question
L
H-
H



Prenuclear region?
51
Tones-F0 mapping

• In their phonetic manifestation tonal targets are
  characterized by temporal alignment and F0 scaling

H
F0
L
A
M
r
Time

• F0 between targets defined by interpolation rules

Shape
Slope
Duration
H
H
H
H
H
H
L
L
L
L
L
L
52
Tunes in Neapolitan Italian
Introduction Experiment I Discussion Expe
riment II Discussion Summary Conclusion

• Intonation contrast AP-tone

Speaker OM
H
(LH)
Q
L
na?
mam
ve
re
Di
vuo
le
de
la
ma
La
S
Tr. The mom wants to see (the) Dina

• Same phrasing tonal composition but
• different slope!