Affective Computing

1
Affective Computing

• Dr. Mark Brosnan
• 2 South M.J.Brosnan_at_bath.ac.uk

2
Picard (1997)

• Affective Computing Computing that relates to,
  arises from, or deliberately influences emotions
  (p.3)
• Recognise emotions
• Express emotions
• Have emotions

3
What are emotions? Paul Ekman

• The 6 basic emotions are happy, sad, angry,
  afraid, surprise and disgust.

4

• These six emotions are universally recognized and
  universally expressed through the same facial
  features.

5
Sex differences

• Recognition of Emotional Expressions (Thayer
  Johnsen, 2000)
• Females are better than males recognizing
  emotional expressions.
• Females recognize emotions equally well from male
  and female faces.

6
Electronic emotions encyclopaedic Baron-Cohen
et al. (2005)

• Used a thesaurus to identify every word in the
  English language that describes an emotion.
• There were 412 human emotions (excluding
  synonyms).
• virtually all emotions could be assigned to one
  of 24 different groups.

7
24 emotions

• Afraid, angry, bored, bothered, disbelieving,
  disgusted, excited, fond, happy, hurt,
  interested, kind, liked, romantic, sad, sneaky,
  sorry, sure, surprised, thinking, touched,
  unfriendly, unsure, wanting.

8
Examples of Afraid emotion

• Afraid, consternation, cowardly, cowed, daunted,
  desperate, discomforted, disturbed, dreading,
  frantic, intimidated, jumpy, nervous, panicked,
  shaken, terrified, threatened, uneasy,
  vulnerable, watchful, worried.

9
Implications

• There are different ways to categorise emotions
• A basic, universal set is ideal for AI
• Transmitted through the face is also useful for
  AI to detect emotion
• Implications for transmission of emotion
• There are deficits in emotional processing in
  some humans (and sex differences). What if you
  cant experience emotion?

10
(No Transcript)
11
Is Mr. Spock intelligent?

• Spock is only rational
• Descartes Error (Damasio, 1994)
• Elliot searches unlimited search space to make a
  rational decision
• Missing somatic markers that associate feelings
  with decisions

12
Artificial Intelligence?

• AI is like Elliot
• Turing Test (1950 French 2000)
• Jabberwacky.com
• Emotion is required for artificial intelligence
  (Hofstadter, 1981)
• Emotional Intelligence?

13
Professor Jefferson Lister (1949)

• Not until a machine can write a sonnet or compose
  a concerto because of thoughts and emotions felt,
  and not by the chance fall of symbols, could we
  agree that machine equals brain.

14
Affective communication

• Social rules extended to computers
• Media Equation (Reeves and Nass, 1996)
• Anthropomorphism
• Mechanomorphism
• Ethopoeic perceptions

15
Ethopoeic Perceptions

• People have social relationships with computers
  (Turkle, 1984/ 2004)
• Ascribe characteristics to computers, gender,
  demonstrate politeness, anger, have a favourite
  in the lab (Reeves and Nass, 1996)

16
Trigger scripts

• Anthropomorphism
• - Ascribing feelings and purpose
• Ethopoeic perceptions
• - Human-like characteristics but not purposeful

17
Computer Perceptions Scale (Charlton, 2006)

• No sex differences, less experience related to
  greater ethopoeic perceptions
• more experience alters perceptions
• certain sorts of people have more experience

18
Recognise Emotions

• Vision to recognise facial expression
• Multimodal
• GSR polygraph
• Which emotion happiness, guilt
• Emotional Turing test
• Person dependent
• Person independent

19
Criteria for recognition

• Input
• Pattern recognition
• Reasoning

• Learning
• Bias
• Output

20
Express emotions

• Kismet (Breazeal and Scassellati, 2002)
• Emotional expression for communication and social
  co-ordination
• Emotion for organisation of behaviour (action
  selection, attention and learning)
• Arbib and Fellous (2004)

21
http//www.ai.mit.edu/projects/humanoid-robotics-g
roup/ kismet/kismet.html
22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25

• More effective expression than humans
• Human expression identified 50 of the time
• Computer expression identified 70 of the time
• (Elliott, 1997)
• Computers having non-human emotion?

26
Criteria for expression

• Input
• Intentional vs. spontaneous pathways
• Feedback

• Bias exclusion
• Social display rules
• Output

27
Have emotions

• Can machines feel?
• How would we know?

28
Criteria for having emotions

• System has behaviour that appears to arise from
  emotions
• System has fast primary emotional responses to
  certain inputs
• System can cognitively generate emotions
• System can have emotional experience
• Systems emotions interact with other processes
  (e.g. memory)

29
Do computers need bodies to have emotions?

• Robot emotions? Arbib (2005)
• Recognition of own emotion
• Recognition of other computers emotions
• Consciousness?
• Real or simulation?
• Sci Fi 2001!

30
Design questions

• 1. Should computers be allowed to keep their
  emotions from their designers?
• 2. Should what is considered good and bad be
  hard-wired or learned?
• 3. Should a computers mood be affected by others
  moods?
• 4. Do computers need negative emotions, anger,
  fear, misery?

31
Interacting with Computers

• Special Issue (2002, 14(2))
• Affective Computing

32
Scheirer et al. (2002)

• Frustration
• Slow computer game
• Mouse clicking behaviour

33
Klein and Picard (2002)

• This computer responds to user frustration
• Affect-support agent
• Text and buttons in a GUI
• Demonstrate empathy to support user
• Control 1 Emotions ignored
• Control 2 Vent frustration

34
Experiment

• Game 1
• Agent intervention
• Game 2
• Affect support agent lead to greater involvement
  in longer play with Game 2

35
Picard and Klein (2002)

• Emotion skill needs
• Emotional self awareness
• Manage emotions
• Self-motivate
• Affect perception
• Empathy
• Experiential emotional needs

36
Hone (2006)

• Empathetic agents more effective
• Embodied
• Female embodied agents more effective

37
(No Transcript)
38
Tractinsky (2004)

• Affective HCI is difficult to study
• Affective HCI is hard to do
• Design interactive technologies that help users
  help themselves

39
Muller (2004)

• 2 Criticisms
• Computers Are Social Actors (CASA)?
• Other technologies are anthropomorphised too
  (boats, cars, toys etc)
• Need to better understand emotions

40
Affect and the user experience
41

• Usability is viewed as one of the cornerstones of
  HCI. The term has traditionally been synonymous
  with simplicity, ease of use and learning
  (Carroll, 2004).
• Definitions commonly used in HCI, such as within
  standards, e.g. ISO 9241-11 (ISO, 1998 e.g.
  Mahlke, 2005 MacFarlane, Sim Horton, 2005),

42

• These focus on the efficient achievement of tasks
  and goals and the associated cognitive
  information processes.
• However, there has been a significant shift in
  how HCI is perceived and this has brought with it
  the need to either broaden the definition of
  usability (Carroll, 2004) or accept that it is no
  longer the only consideration when designing a
  user-centred system (e.g. De Angeli, Lynch
  Johnson, 2002 Hassenzahl, 2003).

43

• Affective qualities such as engagement and fun
  are increasingly regarded as forming a vital
  aspect of the user experience (Swallow, Blythe
  Wright, 2005).
• The functional vision of computers as tools for
  cognition now includes an experiential vision of
  interactive systems as a medium for emotions,
  sociability and pleasure (De Angeli, Sutcliffe
  Hartmann, 2006).

44
Affective interaction

• concerns non-instrumental quality aspects
  (Hassenzahl, 2005, cited Mahlke, 2005) of the
  user experience
• hedonics, aesthetics and pleasure/fun.

45
Hedonics

• Quality dimensions with no obvious relation to
  the task the user wants to accomplish with the
  system (Hassenzahl et al, 2000, p.202).
• Correlations have been found between hedonic
  qualities and user satisfaction (Hassenzahl et
  al., 2000 Hassenzahl, 2001).

46
Pleasure/fun

• Inherent enjoyment in the activity for its own
  sake (Draper, 1999 p. 118)
• Things are fun when they attract, capture, and
  hold our attention by provoking new or unusual
  perceptions, arousing emotions in contexts that
  typically arouse none, or arousing emotions not
  typically aroused in a given context. (Carrol,
  2004 p. 39)

47
Aesthetics

• Attractive things work better (Norman, 2004
  p.17)

48
Evaluating affective interaction

• Is it possible to meaningfully measure such
  concepts as enjoyment and fun? (Sampson, 2006)
• Or to find the relationships between design
  features and emotional responses? (Desmet,
  Overbeeke Tax, 2001).

49
Subjective self-reporting

• This usually consists of administering a
  questionnaire after the event to elicit ratings
  of user feelings (Isbister et al., 2006 Picard
  Daily, 2005).
• Butrequires accurate recall
• Butonly generate data when a question is asked

50
Observation

• Think-aloud commentary
• Coding of cues of emotion, such as facial and
  verbal expressions, body language and gesture
  (Hazlett, 2006 Mandryk, Atkins Inkpen, 2004
  Isbister et al., 2006).
• Butcan only be event-based (e.g. participant
  smiles)

51
Biometrics

• Heart rate or Galvanomic Skin Response (GSR)
• Facial Electromyography (EMG)
• non-intrusive (relatively) objective,
  quantitative .
• Butmeasure stimulation, difficulty
  distinguishing between specific affects .
• Butcomplex and costly

52
Indirect task assessment

• If positive affect increases creativity (e.g.
  Norman, 2004), then users should be more creative
  after a positive task than after a negative task.
• Perceived time on task relative to actual time

53
Self report

• Adam Hardy (2006) evaluating a browser for
  digital photos
• After Morris (1995)

54
valence
55
arousal
56
dominance