DO MOUTHS SIGN DO HANDS SPEAK - PowerPoint PPT Presentation

1 / 49
About This Presentation
Title:

DO MOUTHS SIGN DO HANDS SPEAK

Description:

Gesture systems (home sign) can appear in the absence of ... the source of the first words: imitative sounds mimicking the sounds of the world around them. ... – PowerPoint PPT presentation

Number of Views:99
Avg rating:3.0/5.0
Slides: 50
Provided by: benc158
Category:

less

Transcript and Presenter's Notes

Title: DO MOUTHS SIGN DO HANDS SPEAK


1
DO MOUTHS SIGN? DO HANDS SPEAK?
  • Bencie Woll
  • University College London
  • NIAS Workshop May 19-20, 2006

2
Introduction
  • Gesture systems (home sign) can appear in the
    absence of linguistic input
  • The sign languages of Deaf communities have
    sometimes been regarded as primitive
    communication systems and, the reasoning follows,
    as a precursor to spoken languages
  • Sign languages are not primitive and are in fact
    full natural languages with complex grammars
  • Nevertheless it is possible that sign languages,
    have features in common with manual precursors of
    spoken language
  • Is there a plausible mechanism by which primarily
    manual actions could have transformed themselves
    into vocal actions?

3
Theories of language origins
  • Ding-Dong onomatopoeia is the source of the
    first words imitative sounds mimicking the
    sounds of the world around them.
  • Bow-wow imitation of animal sounds.
  • Pooh-pooh - sighs of pleasure, moans of pain, and
    other semi-involuntary cries, which then became
    the names of the phenomena
  • Uh-oh - warnings to other members of the human
    band.

4
Theories (cont.)
  • Yo-he-ho - rhythmic chants and vocalisms uttered
    by people engaged in communal labour.
  • Watch the Birdie - communication became
    elaborated because humans found it advantageous
    to be able to deceive other humans. Since
    exclamations and vocalisms can involuntarily
    reveal your true mental state, humans learned to
    feign them in order to deceive others for selfish
    advantage.
  • Ta-ta - oral gestures that began in imitation of
    hand gestures that were already in use for
    communication.

5
Historical Perspectives
  • there are actions of the mouth which are
    commonly performed under certain circumstances
    and which seem to be due to imitation or some
    sort of sympathy. Thus persons cutting anything
    may be seen to move their jaws simultaneously
    with the blades of the scissors. Children
    learning to write often twist about their tongues
    as their fingers move, in a ridiculous fashion.
  • Charles Darwin, The Expression of Emotions in Man
    and Animals (1872)

6
Henry Sweet
  • "Gesture .. helped to develop the power of
    forming sounds while at the same time helping to
    lay the foundation of language proper. When men
    first expressed the idea of 'teeth', 'eat',
    'bite', it was by pointing to their teeth. If the
    interlocutor's back was turned, a cry for
    attention was necessary which would naturally
    assume the form of the clearest and most open
    vowel. A sympathetic lingual gesture would then
    accompany the hand gesture which later would be
    dropped as superfluous so that ADA or more
    emphatically ATA would mean 'teeth' or 'tooth'
    and 'bite' or 'eat', these different meanings
    being only gradually differentiated".

7
Richard Paget
  • The earliest human language was a language of
    gestures, in which features originally made by
    hand were unconsciously copied by movements or
    positions of the mouth, tongue or lips.
  • Human Speech some observations, experiments and
    conclusions as to the nature, origin, purpose and
    possible improvement of human speech (1930)

8
Morris Swadesh
  • The shape of objects is imitated in human
    gestures and from there passes into vocalisation.
    This is due to two circumstances. One is that in
    humans as in other primates, the lips are
    flexible and can be used to copy shapes, such as
    round or flat. The other is that the passage of
    air through spaces gives a resonance that is
    related to their shape e.g. a word like the
    Latin CAPIO I TAKE or English CAPTURE
  • the formation of the K sound at the back of the
    mouth, while the lips are open, is comparable to
    the open hand. The closing of the lips then is
    analogous to the fingers closing with the thumb
    as one takes hold of an object. Thus the
    pronunciation of the root CAPIO is like the
    action of taking. Of course not all words are to
    be explained in this way, in fact only a few. And
    yet the possibility that some words developed in
    this way is not denied by other qualities also
    evident in language.

9
Allotts tests for these theories
  • Evidence that gesture historically preceded
    speech and that the gesture-language fairly
    precisely expressed, by shape and movement, the
    objects and events referred to
  • Evidence for a historical process by which those
    overt gestures were reflected, reproduced in
    miniature, in gestures particularly of the tongue
    and lips which were associated with the
    production of speech-sounds
  • There can be no evidence for this as a
    historical process and Paget presents little to
    explain or justify the hypothesis as a reasonable
    physiological speculation

10
Allotts tests for these theories
  • Evidence that gesture historically preceded
    speech and that the gesture-language fairly
    precisely expressed, by shape and movement, the
    objects and events referred to
  • Evidence for a historical process by which those
    overt gestures were reflected, reproduced in
    miniature, in gestures particularly of the tongue
    and lips which were associated with the
    production of speech-sounds
  • There can be no evidence for this as a
    historical process and Paget presents little to
    explain or justify the hypothesis as a reasonable
    physiological speculation
  • Is there any contemporary evidence?

11
Summary
  • Most of these writers suggest that the mouth
    actions themselves would share underlying imagery
    with the iconic manual gesture
  • In the absence of any plausible mechanism or
    historical evidence, the notion of a hand-mouth
    link remains as speculative as any other theory

12
Signs and gestures
  • full human languages
  • used by modern humans with language-ready
    brains
  • But they share some features with gestures
  • common articulators
  • greater iconicity than spoken languages
  • How could a hypothesised highly iconic manual
    communication system have led to the creation of
    a vocal communication system in which the links
    between symbol and referent are for the most part
    arbitrary?

13
The articulators of sign language
  • Not just manual there is a rich and complex
    role for other articulators body, face, and, in
    particular, the mouth.
  • The research reported here focuses on one
    subgroup of these mouth actions 'echo phonology
  • a repertoire of mouth actions not derived from
    spoken language
  • form an obligatory accompaniment to some manual
    signs in a range of sign languages
  • characterised by 'echoing' on the mouth certain
    of the articulatory actions of the hands.

14
Data
  • narratives in 3 different European sign languages
  • evidence from bilinguals
  • functional imaging studies (fMRI) with deaf
    signers and hearing non-signers
  • These provide examples of a possible mechanism
    in the evolution of language by which the units
    of an iconic manual communication system could
    convert into a largely arbitrary vocal
    communication system.

15
Neurobiological Perspectives
  • Studies of neurons in the monkey brain by
    Rizzolatti and colleagues since 1996 have
    identified mirror neurons, which fire when the
    animal observes another individual making
    specific movements (primarily for reaching and
    grasping)
  • The mirror system, in temporal, parietal and
    frontal regions, is part of a system specialised
    for perceiving and understanding biological motion

16
Hand and mouth - Gentilucci
  • when humans are asked to open their mouths while
    grasping objects, the size of the mouth opening
    increases with the size of the grasped object
  • Grasping larger objects and bringing them to the
    mouth induces increases in the size of mouth
    opening and voice spectra of syllables pronounced
    simultaneously
  • Observing another individual grasping or bringing
    different sizes of objects to the mouth also
    affects the articulation of syllables

17
The hand and the mouth shared actions
  • the anatomical closeness of hand and mouth
    related neurons in the premotor cortex may relate
    evolutionarily to the involvement of both in
    common goals
  • The relationship between mouth actions related to
    eating, and those found in spoken language, have
    been discussed in detail by MacNeilage

18
Non-manual actions in sign languages (after Woll,
2001)
Mouth actions
Eyes, brows, head, body...
Mouthings derived from spoken language
Mouth gestures not derived from spoken language
Echo phonology
Adverbials
in loans e.g. to disambiguate
Enactions
in contact signing
19
The Mouth in Sign Language
  • extensive use of the mouth for a variety of
    functions
  • Mouthings borrowed from spoken words used to
    disambiguate manually homonymous forms
  • Adverbials used to signal manner and degree
  • Enaction (mouth-for-mouth) the mouth
    represents an action directly (e.g. CHEW, BITE
  • Echo phonology

20
Echo Phonology
  • obligatory in the citation forms of lexical signs
  • not derived from spoken language
  • specifiers of meaning, not adverbials added to
    citation forms
  • not enactions
  • the mouth action includes movement either the
    exhalation or inhalation of breath, or a change
    in mouth configuration during the articulation of
    the sign.
  • features based on the articulatory features of
    the manual movement
  • 3 BSL signs meaning succeed or win
  • Three different oral patterns occur in these
    signs one cannot be substituted for the other
  • the action of the mouth, while echoing that of
    the hands, is not in itself iconic

21
Examples of EP syllable types
  • pa (SUCCEED)
  • associated with one or two active hands, movement
    consists of hand separation and twisting, with
    single sharp action
  • ?? (EXIST)
  • wriggling or fingers, repeated shaking or
    twisting of wrists(s), no path movement
  • hww or hyy (WIN)
  • repeated shaking of wrist, no path movement
  • ?p (THANK-GOD)
  • closing and twisting of hand(s), sharp movement
  • am or ?m (TRUE)
  • hand closes and contacts passive hand, sharp
    movement
  • ?p (DISAPPEAR)
  • hand(s) close, sharp movement with abrupt stop
  • pipipi (VARIED)

22
Cross-linguistic study
  • Comparison of narratives in three sign languages
  • the occurrence of echo phonology was compared
    with other types of mouth action
  • of Echo phonology is low information-adding
    functions have priority over those where
    information is redundant

23
Some evidence from bilinguals
  • Data from CODAs (hearing native signers)
  • Written texts in CODA-CODA correspondence (ASL)
  • Father fork in throat (father is frustrated)
  • Echo phonology

24
Echo Phonology without the hands
  • Have you done that poster?
  • ??? (NOT-YET), Ill do it tomorrow
  • It was terrible. It was like ?mp
    (END/absolutely over)
  • I couldnt get a straight answer from anyone. It
    was completely pipip (VARIED/inconsistent)

25
Functional Imaging
  • sign languages embody the structural and
    communicative properties of spoken language,
    while existing entirely within a wholly
    visuo-gestural medium
  • Among other insights, they enable investigators
    to clarify the core components of language
    processing in distinction to those that reflect
    input or action characteristics of the language
    system.

26
  • An fMRI study was undertaken to explore the
    following conditions
  • silent speech (SS)
  • BSL signs that require mouthings for
    disambiguation (DM)
  • BSL signs that exhibit echo phonology (EP)
  • BSL signs that require no specific mouth action
    (NM)

27
Stimulus characteristics
28
Research Questions
  • to what extent does the pattern of activation
    during speech perception and sign language
    perception differ?
  • does the perception of mouthings (DM) differ from
    signs without mouth (NM)?
  • does echo phonology (EP) generate distinctive
    activation compared with mouthings (DM)?
  • how do hearing non-signers differ from deaf
    signers?

29
Study participants
  • Thirteen (6 female mean age 27.4 age range
    18-49) right handed participants
  • congenitally, severely or profoundly deaf native
    signers, having acquired BSL from their deaf
    parents
  • comparator group of 13 hearing English-speakers,
    with no knowledge of BSL, matched for age,
    gender, IQ and educational background.

30
Experimental stimuli
31
To what extent do the patterns of activation for
speech perception and sign language perception
differ?
32
Deaf native signers
No mouth
Echo phonology
Silent speech
Disambiguating mouth
33
Speechreading
  • The major area of activation was perisylvian
    (superior temporal and inferior frontal) , with
    somewhat more extensive activation on the left
    than the right
  • These findings conform with other recent studies
    and confirm that silent speech can activate
    regions in deaf peoples brains that have been
    identified as auditory speech processing regions
    in hearing people.

34
Deaf native signers
No mouth
Echo phonology
Silent speech
Disambiguating mouth
35
Sign language
  • In all three sign language conditions, there is
    also activation in perisylvian regions
  • This affirms that sign language in Deaf native
    signers activates core language regions that are
    typically found when hearing people listen to
    speech.
  • Although both sign language and speech involve
    perisylvian regions, sign language perception
    activated more posterior and inferior regions

36
Does the perception of signs with mouthings (DM)
differ from signs with no mouth (NM)?
  • If language (speech vs. sign) is the crucial
    reason for the more posterior activation found in
    BSL perception, then signs with disambiguating
    mouth and signs without mouth should be processed
    identically
  • On the other hand, if the articulators used
    determine the areas of activation, then DM and NM
    signs should differ, with more posterior
    activation for the NM signs
  • The data support the second alternative anterior
    activation characterised DM and posterior
    activation NM
  • These findings are very similar to those
    exploring distinctive patterns of activation when
    viewing hand and mouth gestures that are
    unrelated to sign language.

37
Deaf native signers
No mouth
Echo phonology
Silent speech
Disambiguating mouth
38
Does echo phonology (EP) generate distinctive
activation compared with other mouthings (DM)?
  • The contrast between DM and EP provides further
    insight into the cortical correlates associated
    with observing specific articulators within sign
    language
  • DM generated relatively greater activation in a
    circumscribed region of the left middle and
    posterior portions of the superior temporal
    gyrus, while EP produced relatively greater
    posterior activation
  • This can be considered to reflect the fact that
    DM is more speech-like than EP
  • Thus EP appears to occupy an intermediate
    position between signs without mouth and signs
    with mouth actions derived from spoken language.

39
Deaf native signers
No mouth
Echo phonology
Silent speech
Disambiguating mouth
40
Implications
  • Obligatory mouth actions differentially activate
    a circumscribed region within the middle and
    posterior portions of the superior temporal
    gyrus.
  • But generally, a strong similarity between the
    patterns of distinctive activation for mouth
    actions and for hand actions
  • core language processes appear to be similar for
    sign language and speechreading in Deaf native
    signers.

41
How do hearing non-signers differ from deaf
signers?
  • Silent speech Hearing participants extensively
    activated the regions associated with listening
    to and articulating speech
  • Sign language Knowledge of SL generates greater
    activation in posterior superior temporal
    regions, including Wernickes area, and greater
    activation in left than right cortical regions.
    Only when material can be linguistically
    processed does differential activation extend to
    the superior parts of the temporal lobe
  • Echo phonology The only condition to generate
    differential activation (Deafgt hearing) in
    frontal regions, including premotor regions. In
    line with mirror neuron theory, this condition
    requires signers to engage motor circuitry
    possibly because the mouth actions observed
    reflect those of the hands in their dynamic
    patterning.

42
Speech
Deaf signers Hearing
nonsigners
43
No mouth
Deaf signers Hearing
nonsigners
44
Echophonology
Deaf signers Hearing
nonsigners
45
Conclusions
  • One issue for those concerned with suggesting a
    link between gesture and word has always been how
    visually-motivated gestures could have been
    transformed into the largely arbitrary words of
    spoken language
  • Echo phonology provides evidence for a possible
    mechanism

46
  • the phenomenon appears to be fairly common across
    different sign languages (needs research in
    non-European sign languages)
  • the oral activities in echo phonology are
    themselves not visually motivated.
  • the inventory of elements looks very much like a
    system of maximal contrasts in a spoken language
    phonology
  • The oral components can be used in speech
  • fMRI suggests that echo phonology occupies an
    interestingly intermediate position between signs
    and words

47
  • This paper represents only a very preliminary
    exploration of echo phonology Further research
    may provide more insights into the origins of
    phonological structure in spoken language, and
    from that into the evolution of human language.

48
Collaborators on the ECHO project
  • Nijmegen University
  • Els van der Kooij, Onno Crasborn, Annika
    Nonhebel, Wim Emmerik
  • Stockholm University
  • Johanna Mesch, Brita Bergman
  • UCL/City University London
  • Dafydd Waters, Bencie Woll, Rachel Sutton-Spence

49
Collaborators on Imaging the deaf Brain
  • Cheryl Capek
  • Dafydd Waters
  • Bencie Woll
  • Mick Brammer
  • Ruth Campbell
  • Tony David
  • Philip McGuire
  • Mark Seal
  • Mairéad MacSweeney
  • Jordan Fenlon
  • Tyron Woolfe
  • Zoë Hunter
  • Steve Williams and the imaging team
Write a Comment
User Comments (0)
About PowerShow.com