Learning From and About Others: Towards Using Imitation to Bootstrap the Social Understanding of Oth

1
Learning From and About Others Towards Using
Imitation to Bootstrap the Social Understanding
of Others by Robots

• Cynthia Breazeal et al
• MIT Media Lab

2
Goals

• Improving robot-human communications
• Social Interaction
• Interpret
• Predict
• React

3
Simulation Theory

• to know a man is to walk a mile in his shoes
• Perceiving similarities
• Mental state
• Simulate thoughts, feelings, etc.
• Apply same idea to our problem

4
Imitation

• Infants learn to simulate others by imitation
• A. Meltzoff
• Requires similarity
• Connection between physical and emotional state
• Mirror neurons

5
Imitation by Infants

• Four stages
• Stage 1 Body babbling
• Build a movement directory
• Stage 2 Imitate body movement
• Organ identification

6
Imitation by Infants

• Stage 3 Deferred imitation
• Long term memory retention
• Intermodal space
• Stage 4 Imitate novel actions
• Infer purpose of another persons action

7
Meet Leo

• 64 DoF
• 24 DoF for the face

8
Leonardo the Robot

• Facial recognition
• Axiom ffT software (Nevengineering Inc.)
• Maps 22 points

9
Perception System

• Percept tree
• Node percept
• Recognize and extract features
• Deeper gt more specific
• Movement percepts
• Contingency percepts

10
Motor System

• Posegraph
• Directed, weighted graph
• Each node is a joint configuration
• Edges are allowed transitions
• Motor programs
• Generate path
• Perform action

11
Basis Poses

• Three motor subsystems
• Blend poses

12
Imitation Games

• Two stage process
• Stage 1 Human imitates Leo
• Motor babbling
• Presence awareness
• Map expressions to intermodal space

13
Contingency

• Train neural nets
• Input/output pairs
• Detect imitation
• Contingent motion based on time

14
Organ Identification

• Humans only imitate part of expression
• Partition facial feature data
• Right eyebrow, left eyebrow, mouth
• Separate neural network for each
• 2 layer, 7 hidden nodes
• Input facial feature data
• Output joint position
• Intermodal representations

15
Imitation Games

• Stage 2 Leo imitates human
• Stops motor babbling
• Motion cues
• Imitation action
• Represents pose in intermodal space

16
Goal-Directed Search

• Search posegraph
• Best match of intermodal representation
• Strikes that pose
• Blending
• Search for better weights
• Uses hill climbing algorithm
• Distance metric Angular and translational
  distance
• Quality weighted average of angular distance

17
Goal-Directed Search
18
(No Transcript)
19
Results

• All using virtual Leo
• Same intermodal representation for all people
• Avoids impossible join configurations

20
Results
21
Results
22
Social Referencing

• Four stage approach
• Stage 1 recognize facial expression
• Based on intermodal space
• Stage 2 tie expression to an emotional state
• Stage 3 shared attention
• Stage 4 affective tagging

23
Remarks Reactions

• Interesting and accessible
• Theoretical background gt implementation
• More info on imitation-emotion mapping
• Working physical robot