Cues to Emotion: Face, Body, Brain - PowerPoint PPT Presentation

Loading...

PPT – Cues to Emotion: Face, Body, Brain PowerPoint presentation | free to download - id: 5791cb-MDY0M



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Cues to Emotion: Face, Body, Brain

Description:

Cues to Emotion: Face, Body, Brain Bob Coyne (cs6998) * 56 Happy Sad Fear Anger Surprise Disgust * 79 Happy Sad Fear Anger Surprise Disgust * 102 Happy Sad Fear Anger ... – PowerPoint PPT presentation

Number of Views:69
Avg rating:3.0/5.0
Slides: 53
Provided by: RobertC195
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Cues to Emotion: Face, Body, Brain


1
Cues to Emotion Face, Body, Brain
  • Bob Coyne (cs6998)

2
Outline
  • Emotion in voice and brain
  • The voice of emotion an FMRI study of neural
    responses to angry and happy vocal expressions
    (2006) Tom Johnstone et al
  • Facial Expressions and emotion
  • http//en.wikipedia.org/wiki/Paul_Ekman
  • Facial Action Coding System (FACS) Ekman
    Friesen
  • Microexpressions - Ekman
  • Poker tells The Body Language of Poker (1984)
    Mike Caro
  • 3D graphics facial expressions FaceGen software
  • www.facegen.com

3
The voice of emotion an FMRI study of neural
responses to angry and happy vocal expressions
(2006) Tom Johnstone et al
  • Range of issues
  • Emotions in general vs specific emotions
  • General regions of brain (left/right) vs
    localized
  • Perception vs production
  • Prosody vs semantic content in speech
  • Facial expressions vs speech
  • Attention vs inattention
  • Type of experiment, controls, and measurement
    (FRMI, MEG, PET, EEG, other)
  • What function does vocal affect have and how to
    test

4
Importance of response to vocal affect
  • From the earliest stages of development, infants
    respond to affect-laden vocal expressions from
    their mothers (Fernald, 1989 Fernald and
    Morikawa, 1993)
  • Why might this be?
  • Relevance to the 4 approaches (darwinian,
    jamesian)?
  • Still important/relevant after?
  • When?
  • Emoticons?
  • Emotion in voice recognized as well as faces
    across cultures (Scherer and Wallbott, 1994)
  • True?

5
Early Prior work
  • Production/comprehension experiments
  • Production vs perception. Makes a difference?
  • Production in Right hemisphere (Hughlings-Jackson
    - 1915)
  • How did they know?
  • Neurological evidence for comprehension (Tucker
    et al., 1977)
  • Reduced affective speech comprehension in
    right-hemisphere-damaged listeners (bowers 1987,
    Heilman et al., 1984 Peper and Irle, 1997 Ross,
    1981)
  • Whats missing?
  • Where processed?
  • Controlled experiments?
  • How else to test/identify region?

6
More recent Prior work
  • Specific regions for facial emotion processing
    (e.g. amygdala for fear, insula for disgust)
  • Little prior work on localized regions for
    emotional speech for specific emotions.
  • Propositional content vs emotional prosody
  • Right prefrontal activation (PET - George et al.
    1996)
  • EEG scalp electroencephalography Pihan et al.
    1997
  • Accented syllable duration and fundamental
    frequency
  • Are those correlated with emotional speech?
  • Magnetoencephalography (MEG) (Imaizumi et al.
    1998)
  • Also found some left-hemisphere processing of
    propositional and prosodic content

7
Recent Prior work
  • non-differentiated per emotion
  • Selective activation
  • Congruence effect with facial and vocal
  • Experiments
  • Mitchell et al. (2003) found areas of posterior
    middle temporal gyrus (MTG) and superior temporal
    sulcus (STS) that activated more when attending
    to affective prosody as compared with semantic
    content of spoken words
  • Grandjean et al. (2005) and Sander et al. (2005)
    have reported FMRI data that revealed a region in
    STS that showed greater activation in response to
    angry speech as compared with neutral speech
  • Wildgruber et al. (2005) identified a right
    hemisphere network consisting of posterior STS,
    and dorsolateral and orbitobasal prefrontal
    cortex that showed selective activation during an
    emotion recognition task. Differential
    activations for the five emotions were not
    observed.
  • Ethofer et al. (2006) identified regions in the
    right posterior MTG and STS and bilateral
    inferior/middle frontal gyrus that activated more
    when individuals identified affective prosody
    than when identifying the emotional content of
    the spoken words. No distinction was made between
    responses to the different expressed emotions
    studied
  • Pourtois et al. (2005) demonstrated an area in
    left MTG that showed heightened activation when
    congruent vocal expressions and facial
    expressions of happiness or fear were
    simultaneously presented, as compared with when
    only one expressive modality was presented

8
The Brain
Whole Brain Atlas Top 100 Brain Structures
http//www.med.harvard.edu/AANLIB/cases/caseM/cas
e.html
9
Loci of Sulci and Gyri
  1. Central sulcus
  2. Precentral gyrus
  3. Postcentral gyrus
  4. Precentral sulcus
  5. Postcentral sulcus
  6. Middle frontal gyrus
  7. Superior frontal gyrus
  8. Superior parietal lobule
  9. Occipital gyri
  10. Longitudinal fissure

10
Highly specialized brain function?
11
Hypotheses
  • The question remains, therefore, of whether
    specific neural regions are more engaged in the
    processing of some emotions than others.
  • Based on the hypothesis that affiliative social
    vocal signals are prevalent throughout our lives
    and serve a fundamental purpose in social binding
    we hypothesize that emotional expressions of
    happiness would preferentially engage parts of
    the temporal cortex and inferior frontal regions
    previously shown to be involved in the processing
    of affective prosody.
  • A further question concerning the perception of
    vocal expressions of emotion is how directed
    attention towards or away from the expressed
    emotion affects the associated neural response.
  • How to test this?
  • Activation for face versus voice most studies
    on face
  • What types of emotional speech detected? All?

12
Experimental approach
  • FMRI to detect localized regions for processing
    happiness vs anger in speech
  • Why those two emotions?
  • Reasons
  • Control for similar acoustic signal.
  • Common/basic. Easier to characterize.
  • social utility hypothesis regarding happiness
  • Which theory of emotion (jamesian etc)
  • What was experimental setup?
  • Some factors
  • 40 subjects
  • Facial stimuli consisted of 16 grayscale images
    of posed expressions of anger and 16 expressions
    of happiness, half of them females.
  • Vocal stimuli consisted of short phrases (dates
    and numbers) lasting on average 1 s, spoken with
    either angry or happy prosody.
  • Half the participants make their decision on the
    basis of the facial expression, while the other
    half on the basis of the vocal expression
  • FMRI scan during happy/angry speech
  • With corresponding or opposite facial expression

13
Experiment Issues/problems?
  • Brain regions necessarily responding to emotion?
  • Quality of audio? Affects results?
  • FMRI measures blood flow. Relevant to neural?
  • Right handed subjects. Relevant?
  • Faces not correlated to the speaker.
  • Why normalized audio amplitude?
  • Why no semantic emotional content?
  • Why short phrases?

14
FRMI
Functional MRI or functional Magnetic Resonance
Imaging (fMRI) is a type of specialized MRI scan.
It measures the haemodynamic response related to
neural activity in the brain or spinal cord of
humans or other animals. It is one of the most
recently developed forms of neuroimaging. Since
the early 1990s, fMRI has come to dominate the
brain mapping field due to its low invasiveness,
lack of radiation exposure, and relatively wide
availability.
15
Experiment - results
  • Happy voices elicited significantly more
    activation than angry voices in right anterior
    and posterior middle temporal gyrus (MTG), left
    posterior MTG and right inferior frontal gyrus.
  • Attention-dependent results In contrast with
    brain regions showing a main effect of vocal
    emotion regardless of attentional focus, a
    network of brain regions including the left
    insula, left amygdala and hippocampus, and
    rostral ACC responded more to happy voices than
    to angry voices when attending to the voice, but
    showed either no difference or greater activation
    to angry voices than to happy voices when
    attending to the face.
  • Do these results support the social binding
    hypothesis?

16
Results - RT and accuracy
  • Why longer RT for voice case?
  • Why measure RT and accuracy at all?

17
Results
How do these graphs support the stated results?
18
Facial Expressions and Emotion - Paul Ekman
http//en.wikipedia.org/wiki/Paul_Ekman
  • Found facial expressions of emotion are not
    culturally determined, but universal across human
    cultures and thus biological in origin.
  • How did he determine this?
  • Fully true? Examples?
  • How does this square with the four approaches
    Darwinian? Jamesian? Social constructivist?
    Appraisal?
  • Expressions he found to be universal included
    those indicating anger, disgust, fear, joy,
    sadness, and surprise. Findings on contempt are
    less clear, though there is at least some
    preliminary evidence that this emotion and its
    expression are universally recognized.
  • examples of non-universal emotions? of
    culture-dependent expression of same emotion? Of
    individual-dependent expression of same emotion?
  • Facial Action Coding System (FACS) taxonomize all
    human facial expression. (Ekman and Friesen)
  • 48 action descriptors, which are a contraction or
    relaxation of one or more muscles.
  • http//en.wikipedia.org/wiki/Facial_Action_Coding_
    System
  • http//www.cs.cmu.edu/afs/cs/project/face/www/facs
    .htm
  • Microexpressions detect lying
  • How does this phenomenon square with the four
    approaches?

19
FACS action descriptors
1 Inner Brow Raiser -- Frontalis (pars
medialis) 2 Outer Brow Raiser -- Frontalis (pars
lateralis) 4 Brow Lowerer -- Depressor glabellae,
Depressor supercilii, Corrugator supercilii 5
Upper Lid Raiser -- Levator palpebrae
superioris 6 Cheek Raiser -- Orbicularis oculi
(pars orbitalis) 7 Lid Tightener -- Orbicularis
oculi (pars palpebralis) 9 Nose Wrinkler --
Levator labii superioris alaeque nasi 10 Upper
Lip Raiser -- Levator labii superioris, caput
infraorbitalis 11 Nasolabial Deepener --
Zygomaticus minor 12 Lip Corner Puller --
Zygomaticus major 13 Cheek Puffer -- Levator
anguli oris (also known as Caninus) 14 Dimpler --
Buccinator 15 Lip Corner Depressor -- Depressor
anguli oris (also known as Triangularis) 16 Lower
Lip Depressor -- Depressor labii inferioris 17
Chin Raiser -- Mentalis 18 Lip Puckerer --
Incisivii labii superioris and Incisivii labii
inferioris 19 Tongue Out 20 Lip stretcher --
Risorius w/ platysma 21 Neck Tightener --
Platysma 22 Lip Funneler -- Orbicularis oris 23
Lip Tightener -- Orbicularis oris 24 Lip Pressor
-- Orbicularis oris
24 Lip Pressor -- Orbicularis oris 25 Lips part
-- Depressor labii inferioris or relaxation of
Mentalis, or Orbicularis oris 26 Jaw Drop --
Masseter, relaxed Temporalis and internal
pterygoid 27 Mouth Stretch -- Pterygoids,
Digastric 28 Lip Suck -- Orbicularis oris 29 Jaw
Thrust 30 Jaw Sideways 31 Jaw Clencher --
Masseter 32 Lip Bite 33 Cheek Blow 34 Cheek
Puff 35 Cheek Suck 36 Tongue Bulge 37 Lip Wipe 38
Nostril Dilator 39 Nostril Compressor 41 Lid
Droop 42 Slit 43 Eyes Closed -- Relaxation of
Levator palpebrae superioris Orbicularis oculi
(pars palpebralis) 44 Squint 45 Blink --
Relaxation of Levator palpebrae superioris
Orbicularis oculi (pars palpebralis) 46 Wink --
Relaxation of Levator palpebrae superioris
Orbicularis oculi (pars palpebralis)
20
Facial Action Coding System - FACS
  • FACS can be used to distinguish two types of
    smiles
  • insincere and voluntary Pan American smile
  • contraction of zygomatic major alone
  • sincere and involuntary Duchenne smile
  • contraction of zygomatic major and inferior part
    of orbicularis oculi.

21
Pictures of Facial Affect (resource) Ekman and
Friesen
  • 110 pictures of posed facial emotions
  • 10 second display to subjects (male and female
    50 mix)
  • 70 accurately identified
  • Mark each example as one of
  • Happy
  • Sad
  • Fear
  • Anger
  • Surprise
  • Disgust

22
89
  • Happy
  • Sad
  • Fear
  • Anger
  • Surprise
  • Disgust

23
56
  • Happy
  • Sad
  • Fear
  • Anger
  • Surprise
  • Disgust

24
79
  • Happy
  • Sad
  • Fear
  • Anger
  • Surprise
  • Disgust

25
102
  • Happy
  • Sad
  • Fear
  • Anger
  • Surprise
  • Disgust

26
54
  • Happy
  • Sad
  • Fear
  • Anger
  • Surprise
  • Disgust

27
76
  • Happy
  • Sad
  • Fear
  • Anger
  • Surprise
  • Disgust

28
52
  • Happy
  • Sad
  • Fear
  • Anger
  • Surprise
  • Disgust

29
34
  • Happy
  • Sad
  • Fear
  • Anger
  • Surprise
  • Disgust

30
94
  • Happy
  • Sad
  • Fear
  • Anger
  • Surprise
  • Disgust

31
108
  • Happy
  • Sad
  • Fear
  • Anger
  • Surprise
  • Disgust

32
Tally your results!
  • Should match most of the time

33
89
  • Happy
  • Sad
  • Fear
  • Anger
  • Surprise
  • Disgust

34
56
  • Happy
  • Sad
  • Fear
  • Anger
  • Surprise
  • Disgust

35
79
  • Happy
  • Sad
  • Fear
  • Anger
  • Surprise
  • Disgust

36
102
  • Happy
  • Sad
  • Fear
  • Anger
  • Surprise
  • Disgust

37
54
  • Happy
  • Sad
  • Fear
  • Anger
  • Surprise
  • Disgust

38
76
  • Happy
  • Sad
  • Fear
  • Anger
  • Surprise
  • Disgust

39
52
  • Happy
  • Sad
  • Fear
  • Anger
  • Surprise
  • Disgust

40
34
  • Happy
  • Sad
  • Fear
  • Anger
  • Surprise
  • Disgust

41
94
  • Happy
  • Sad
  • Fear
  • Anger
  • Surprise
  • Disgust

42
108
  • Happy
  • Sad
  • Fear
  • Anger
  • Surprise
  • Disgust

43
Ekman - Microexpressions
  • Play video
  • Impact on Darwinian, jamesian, etc?
  • What is deception w/r emotion?

44
Poker Tells The Body Language of Poker (1984)
Mike Caro
  • Poker players exhibit tells that reveal
    valuable information about their hands or
    intentions to bet
  • Some tells are unconscious and directly related
    to the internal emotion (excitement at a good
    hand, disappointment at a bad one)
  • Some tells are the result of acting the
    opposite to try to deceive the opponents.
  • Tells involve facial expressions, gestures, hand
    positions, body posture, and other behaviors

45
Poker Tells instant reaction
The Body Language of Poker (1984) Mike Caro
46
Poker Tells acting the opposite
47
Poker Tells guarding the hand
48
Expressions in virtual 3D faces FaceGen
Neutral Pose
www.facegen.com
49
Expressions in virtual 3D faces FaceGen Anger
50
Expressions in virtual 3D faces FaceGen
Disgustsmile(open)
51
Expressions in virtual 3D faces FaceGen blend
other expression, phonemes, etc.
52
Next Week
  • Hand in either
  • A list of possible topics for your course
    project, with a paragraph on each, or (if you
    have already discussed the project with me and
    decided on one)
  • A one page project specification with
  • Project goal
  • What you will do to achieve
  • What resources you will need
  • What prior work you can draw on
About PowerShow.com