Segmental encoding of prosodic categories: A perception study through speech synthesis - PowerPoint PPT Presentation

Loading...

PPT – Segmental encoding of prosodic categories: A perception study through speech synthesis PowerPoint presentation | free to download - id: c5540-ZDc1Z



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Segmental encoding of prosodic categories: A perception study through speech synthesis

Description:

A diphone-based speech synthesis system for Korean ... Tone & Break Indices. August 19, 2005. Segmental encoding of prosodic categories. 6 ... – PowerPoint PPT presentation

Number of Views:186
Avg rating:3.0/5.0
Slides: 25
Provided by: Sim998
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Segmental encoding of prosodic categories: A perception study through speech synthesis


1
Segmental encoding of prosodic categories A
perception study through speech synthesis
  • Kyuchul Yoon, Mary Beckman Chris Brew

2
Contents
  • An overview
  • Allophonic variations
  • Segmental positions
  • Word-initial vs. word-internal positions in
    K-ToBI framework
  • Allophonic variations an extended view
  • Production studies on Korean and other languages
  • Need for a perception study, but how?
  • A diphone-based speech synthesis system for
    Korean
  • Conventional diphones vs. prosodically-sensitive
    ones
  • A listening experiment
  • Design and synthesis of test stimuli
  • Results conclusion

3
Allophonic variations
An overview
  • Defined mostly in terms of neighboring segments.
  • e.g. Allophones of /t/ in English
  • /t/
  • t th ? ?
  • stop top kitten
    little

4
Segmental positions
An overview
  • Determined in most cases within a word
  • by its
  • 1. neighboring segments and
  • 2. word boundaries, i.e. word-initial/final

5
Word-initial vs. word-internal positions in
K(orean)-ToBI (Prosody labeling conventions)
An overview
  • IP Intonational Phrase H high tone
  • AP Accentual Phrase L low tone
  • W Prosodic Word (PW) T tone (could be H or L)
  • s syllable boundary tone (e.g. H, L, HL,
    etc.)

Tone Break Indices
6
Word-initial positions in K-ToBI
An overview
7
Word-initial positions in K-ToBI
An overview
word-final
word-initial
8
Word-initial positions in K-ToBI
An overview
Three types of word-initial positions in K-ToBI !
9
Allophonic variations an extended view
An overview
  • Defined mostly in terms of neighboring segments.
  • Need to be examined with respect to its prosodic
    constituency in K-ToBI.

10
Productions studies on Korean and other languages
An overview
  • Korean
  • Jun (93,98) lenis stop voicing, obstruent
    nasalization, VOT of /ph/
  • Cho Keating (01) segmental properties of /t,
    th, t, n/
  • Kim (01) segmental properties of /sh, s/
  • Yoon (03) subsegmental durations of /sh, s/
  • Other languages
  • Smith (97) American /z/
  • Pierrehumbert Talkin (92), Pierrehumbert
    (95) English /h/ and /?/
  • Fougeron (01) French segments /t, k, s, l, n,
    i, a/
  • Keating et al. (98) /t, n/ of Korean, English,
    French Taiwanese

11
Productions studies on Korean and other languages
summary of results
An overview
  • Korean
  • AP is the domain of lenis stop voicing,
    post-obstruent tensing (Jun).
  • IP is the domain of obstruent nasalization
    (Jun).
  • VOT of /ph/ AP-initial gt PW-initial gt PW-medial
    (Jun).
  • Consonants initial to higher prosodic domains
    are stronger (Cho, Keating, Kim).
  • Non-uniform variations in durations of
    subsegmental units (Yoon).
  • Other languages
  • American English /z/ is devoiced differently in
    different positions (Smith).
  • English /h/ and /?/ produced differently in
    different word-/phrase-level prosody. (P T)
  • Articulation of initial segments varied
    depending on the prosodic level of
    the constituent, i.e. initial to an IP, AP, W or
    syllable. (Fougeron)
  • There is phrasal/prosodic conditioning of
    articulation across the four languages. (Keating
    et al.)

12
Need for a perception study, but how?
An overview
  • As the production studies show, Korean speakers
    seem to encode prosodic categories, i.e. IP, AP,
    PW, etc., in domain-initial segments.
  • Then what about listeners? Do they decode the
    encodings? Are the encodings perceptible?
  • How do we test it?
  • One way to test it is to use a concatenative TTS
    system so that one can synthesize sentences by
    manipulating phone-sized units, i.e. diphones.

13
Need for a perception study, but how?
An overview
  • Key idea Synthesize a set of two sentences,
    differing only in terms of their domain-initial
    segment compositions.

14
Need for a perception study, but how?
An overview
  • Test stimuli 1st set good AP composed of
    prosodically appropriate synthetic units bad AP
    composed of prosodically inappropriate units
    (Replace ? with ?)
  • 2nd set good PW composed of prosodically
    appropriate synthetic units bad PW composed of
    prosodically inappropriate units (Replace ? with
    ?)

15
Diphones
A diphone-based speech synthesis system
  • Text-to-speech (TTS) synthesis systems
  • Diphones, prosodically sensitive
  • Festival Speech Synthesis System (University of
    Edinburgh)

16
Text-to-speech (TTS) synthesis systems
A diphone-based speech synthesis system
  • A system that automatically generates speech
    given a particular natural language text the
    speech produced should be both comprehensible and
    natural sounding.
  • Two main components NLP module (natural
    language processing) DSP module (digital signal
    processing)
  • NLP module an elaborate text analysis system
  • input text ? sequences of phones prosodic
    organization.
  • DSP module symbolic input from NLP ? natural
    sounding speech

17
Diphones
A diphone-based speech synthesis system
  • Phone-sized synthesis units.
  • Parametric representations of short chunks
    (usually extending from the middle of one phone
    to the middle of the immediately following one)
    of audio signal extracted from a cache of
    recorded sentences that can be re-combined by a
    TTS system to produce a novel synthesized word or
    sentence.
  • Avoid the need for modeling phone-to-phone
    transitions of natural speech signal. For
    example, a diphone i-u contains the second half
    of i and the first half of u. A p-a diphone
    contains the second half of p and the first
    half of a.
  • Prosodically sensitive diphones Each diphone is
    stored as four different versions, i.e. three
    versions initial to an IP, AP or PW, and one
    version medial to a PW. (NB A conventional
    diphone is stored as one version)

18
Diphones
A diphone-based speech synthesis system
?) lt??? ????gt -lt?, lt?-?, ?-?, ?-?, ?-?,
?-?, ?-?, ?-?,
6,503 prosodic diphones needed to synthesize any
Korean utterance.
19
Festival Speech Synthesis System (University of
Edinburgh, http//www.festvox.org)
A diphone-based speech synthesis system
  • A free software multi-lingual speech synthesis
    workbench.
  • An open architecture for research in speech
    synthesis.
  • Primarily developed under Unix/Linux/FreeBSD/Solar
    is and ported to Windows.
  • Developed for conventional diphone-based systems,
    but can be modified to accommodate our
    prosodically sensitive diphones.
  • Consult Yoon (05) for how we created a prototype
    system.

20
Design synthesis of test stimuli
A listening experiment
  • 96 stimuli (phrases) synthesized from the
    Festival system (Durations and F0 contours copied
    from natural utterances).
  • All were composed of either two APs or two PWs.
  • All contained one target site, where an
    AP/PW-initial segment was replaced with a
    PW-medial segment.
  • 24 good AP phrases with intact diphones. 24 bad
    AP phrases whose target site segment
    (AP-initial segment) was replaced
    with a PW-medial segment 24 good PW phrases
    with intact diphones 24 bad PW phrases whose
    target site segment (PW-initial segment)
    was replaced with a PW-medial segment

21
Design synthesis of test stimuli
A listening experiment
  • Prototype system lacks duration F0 generation
    module ? Get help from natural utterances.
  • Synthesis of a sample stimulus (Praat script)
  • lt???? ???gt

natural utterance
diphone sequences from Festival
fundamental frequency (F0) contour and segmental
durations copied from natural utterance
intensity contour copied from natural utterance
22
Design synthesis of test stimuli
A listening experiment
  • Sample stimuli
  • lt?? ???gt target site segment /p/

23
Design synthesis of test stimuli
A listening experiment
  • More sample stimuli

target segment
good AP
bad AP
good PW
bad PW
/p/
/t/
/k/
/ph/
/th/
/t/
/t?/
/t?h/
/sh/
24
Results conclusion
A listening experiment
  • 80 listeners (37 women and 43 men) native
    speakers of Korean, average age of 30.6, grew up
    in Korea until at least 18 years old.
  • Two types of tests in three tasks Intelligibility
    dictation task ? wrote down what they heard in
    hangul Naturalness rating preference task ?
    rate one version wrt/ the other and choose one
    over the other
  • Statistical analyses showed that listeners
    performed better in the dictation task with
    good versions of the stimuli. They also
    liked/rated better the good versions.
  • Segmental encoding of prosodic domains/categories
    is perceptible to Korean listeners.
About PowerShow.com