CS626-449: Speech, NLP and the Web/Topics in AI

1
CS626-449 Speech, NLP and the Web/Topics in AI

• Pushpak Bhattacharyya
• CSE Dept., IIT Bombay
• Lecture-31 Phonology ASR, Speech Synthesis
• (courtesy Ankit Agarwal for part of material on
  phonology)

2
What is Phonology

• Phonetics Study of sounds produced by the
  articulatory system (place and manner of
  articualtion)
• Phonology Study of sound units combine to form
  bigger units like syllables

3
Ancient 5 x 5 Indian Classification of Consonants
Group
? ???? ? ? ? ? ? Velar
? ???? ? ? ? ? ? Palatal
? ???? ? ? ? ? ? Alveolar
? ???? ? ? ? ? ? Dental
? ???? ? ? ? ? ? Labial
4
Vowels (1/2)
5
Vowels (2/2)
6
Phonology Syllables
7
The concept of schwa

• First alphabet of IAL a
• Unstressed and Toneless neutral vowel
• Sanskrit is phonetically perfect no neutral
  vowels
• Hindi, Bengali etc. allow schwa to be neutral
• Some schwas deleted and some are not
• Schwa deletion important issue for grapheme to
  phoneme conversion

8
Schwa deletion contexts

• Saphalya and Amantrana (??????, ???????)
• Priya and Tritiya (?????, ?????)
• Kavya and Ashva (?????, ????)
• Badhai (????)

Deleted only at the end of ???????
Not Deleted for rest of the examples
A difficult problem in case of transliteration
9
English Phonology

• Phonology
• Study of the structure and systematic patterning
  of sounds in human language.
• Refers to a description of the sounds of a
  particular language and the rules governing the
  distribution of these sounds.
• English Phonology
• No. of speech sounds in English varies from
  dialect to dialect.
• Longman Dictionary 24 consonant phonemes (c.p.),
  23 vowel phonemes (v.p.), additionally 2 c.p. 4
  v.p. for foreign words.
• American Heritage Dictionary 25 c.p., 18 v.p.,
  additionally 1 c.p. 5 v.p. for foreign words.

10
Consonant Phonemes

• 25 consonant phonemes found in most dialects of
  English.
• Categorized under six different categories (on
  the basis of their sonority level, stress, way of
  pronunciation etc.)
• Nasal Acoustically, nasal stops are sonorants,
  meaning they do not restrict the escape of air
  and cross-linguistically are nearly always
  voiced.
• Plosive Produced by stopping the airflow in the
  vocal tract (the cavity where sound is filtered).
• Affricate Affricate consonants begin as stops
  (such as /t/ or /d/) but release as a fricative
  (such as /s/ or /z/) rather than directly into
  the following vowel.

11
Consonant Phonemes

• Fricative Produced by forcing air through a
  narrow channel made by placing two articulators
  close together. These are the lower lip against
  the upper teeth in the case of /f/.
• Approximant In the articulation of approximants,
  articulatory organs produce a narrowing of the
  vocal tract, but leave enough space for air to
  flow without much audible turbulence. Examples
  /l/, as in lip, and approximants like /j/ and
  /w/ in yes and well which correspond closely
  to vowels.
• Lateral Laterals are L-like consonants
  pronounced with an occlusion made somewhere along
  the axis of the tongue, while air from the lungs
  escapes at one side or both sides of the tongue.

12
Consonant Phonemes

13
Vowel Phonemes

• 20 vowel phonemes found in most dialects of
  English.
• Categorized under different categories (on the
  basis of their sonority level).

14
Vowel Phonemes

• Monophthong monophthongos single note.
  pure vowel sound.
• Articulation at both beginning and end is
  relatively fixed.
• Does not glide up or down towards a new position
  of articulation.
• Categorized in Short and Long vowels.
• Short Perceived for a shorter duration. For
  e.g., /?/, /e/ etc.
• Long Comparatively longer duration. For e.g.,
  /i/, /u/ etc.
• Diphthong two tones. Vowel combination
  involving quick but smooth movement from one
  vowel to another.
• Often interpreted by listeners as a single vowel
  sound.
• Two target tongue positions.
• Represented by two symbols. For e.g., /e?/

15
Syllable Structure

• Count of no. of syllables in a word is
  roughly/intuitively the no. of vocalic segments
  in a word.
• Thus, presence of a vowel is an obligatory
  element in the structure of a syllable. This
  vowel is called nucleus.
• Basic Configuration (C)V(C).
• Part of syllable preceding the nucleus is called
  the onset.
• Elements coming after the nucleus are called the
  coda.
• Nucleus and coda together are referred to as the
  rhyme.

S Syllable, O Onset R Rhyme, N Nucleus Co
Coda
16
Syllable Structure Examples

• word
• sprint

17
Syllable Structure Examples

• may
• opt
• air

? No Coda.
? No Onset.
? No Coda, No Onset.
18
Syllable Structure

• Open Syllable ends in vowel
• Closed syllable ends in consonant or consonant
  cluster
• Light Syllable A syllable which is open and ends
  in a short vowel
• General Description CV.
• Example, air.
• Heavy Syllable Closed syllables or syllables
  ending in diphthong
• Example opt
• Example, may

19
Syllabification Determining Syllable Boundaries

• Given a string of syllables (word), what is the
  coda of one and the onset of another?
• In a sequence such as VCV, where V is any vowel
  and C is any consonant, is the medial C the coda
  of the first syllable (VC.V) or the onset of the
  second syllable (V.CV)?
• E.g., ari (??? enemy)
• To determine the correct groupings, there are
  some rules, two of them being the most important
  and significant
• Maximal Onset Principle,
• Sonority Hierarchy

20
Maximal Onset Principle

• The consonants that form a word-internal onset
  are the maximal sequence that can be found at the
  beginning of words.
• English permits only 3 consonants to form an
  onset.
• Once 2nd and 3rd consonants are determined, only
  1 consonant can appear in the 1st position.
• Second /p/, Third /r/. Then First can only be
  /s/. E.g., spring.
• More illustrative example constructs
• Consonant sequence n-s-t-r
• Either con structs OR cons tructs OR const
  ructs OR constr ucts.
• As, str can serve as the onset of a syllable,
  thats why the correct syllabification will be
  con structs.

21
Sonority Hierarchy

• Sonority A perceptual property referring to the
  loudness of a sound relative to that of other
  sounds with the same length.
• Sonority Hierarchy Ranking of speech sounds (or
  phonemes) by amplitude.
• For e.g., if you say the vowel /e/, you will
  produce louder sound than if you say the plosive
  /t/.
• It suggests that nuclei are the peaks of sonority
  and segments on either side of the peak show a
  decrease in sonority w.r.t. peak.
• Plosives ? Affricates ? Fricatives ? Nasals ?
  Laterals ? Approximants ? Vowels (Increasing
  order of sonority).

22
Constraints Phonotactics

• Phonotactics
• Determines possible comb. of onsets and codas
  which can occur.
• Deals with restriction on the permissible
  combination of phonemes.
• Defines permissible syllable structure, consonant
  clusters and vowel sequence by means of
  phonotactical constraints.
• In general, rules operate around the sonority
  hierarchy.
• Fricative /s/ is lower on the sonority hierarchy
  than the lateral /l/, so the combination /sl/ is
  permitted in onsets and /ls/ is permitted in
  codas. Opposite is not allowed.
• Thus, slips and pulse are possible English
  words.
• lsips and pusl are not possible.

23
Constraints on Onsets

• One-consonant Only /?/ cant be distributed in
  syllable-initial position.
• Two-consonant We refer to the scale of sonority.
• Sequence rn is ruled out since there is a
  decrease of sonority.
• Minimal Sonority Distance Distance in sonority
  between the first and the second element in the
  onset must be of at least 2 degrees.
• Thus, on the basis of Sonority Hierarchy and
  Minimal Sonority Distance, only a limited no. of
  possible two-consonant clusters.
• Three-consonant
• Restricted to licensed two-consonant onsets
  preceded by /s/.
• Also, /s/ can only be followed by a voiceless
  sound.
• Therefore, only /spl/, /spr/, /str/, /skr/,
  /spj/, /stj/, /skj/, /skw/, /skl/, /smj/ will be
  allowed. (splinter, spray, strong etc.)
• While /sbl/, /sbr/, /sdr/, /sgr/, /s?r/ will be
  ruled out.

24
Constraints on Onsets
Possible 2-consonat clusters in an Onset
25
Constraints on Coda
26
Constraints on Coda
27
Other Constraints

• Nucleus The following can occur as nucleus
• All vowel sounds (monophthongs as well as
  diphthongs).
• Syllabic
• Both the onset and the coda are optional (as seen
  previously).
• /j/ at the end of an onset (/pj/, /bj/, /tj/,
  /dj/, /kj/, /fj/, /vj/, /?j/, /sj/, /zj/, /hj/,
  /mj/, /nj/, /lj/, /spj/, /stj/, /skj/) must be
  followed by /u?/ or /??/.
• Long vowels and diphthongs are not followed by
  /?/.
• /?/ is rare in syllable-initial position.
• Stop /w/ before /u?, ?, ?, a?/ are excluded.

28
Phonteic Symbols and IPA notation
29
IPA vowels
30
Parallel Corpus

• Phoneme Example Translation
• ------- ------- -----------
• AA odd AA D
• AE at AE T
• AH hut HH AH T
• AO ought AO T
• AW cow K AW
• AY hide HH AY D
• B be B IY

Left?Right Speech Synthesis (Grapheme to
Phoneme) Right?Left Speech Recognition (Phoneme
to Grapheme)
31
Parallel Corpus cntd

• Phoneme Example Translation
• ------- ------- -----------
• CH cheese CH IY Z
• D dee D IY
• DH thee DH IY EH Ed EH D
• ER hurt HH ER T
• EY ate EY T
• F fee F IY
• G green G R IY N
• HH he HH IY
• IH it IH T
• IY eat IY T
• JH gee JH IY

Left?Right Speech Synthesis (Grapheme to
Phoneme) Right?Left Speech Recognition (Phoneme
to Grapheme)