Towards Human-Realistic Animation by Observing Real Human Dynamics

1
Towards Human-Realistic Animation by Observing
Real Human Dynamics

• Darren Cosker
• Department of Computer Science,
• University of Bath

2
Motivation
3
Overview

• Analysis and Re-Synthesis of Facial Dynamics
• Dynamics gt CG Puppet
• Dynamics gt us
• Dynamics gt Computer Vision
• Intuitive/Automatic methods for animation
• Voice Driven Animation/Analysis
• Reactive Avatars

4
Analysing/Resynthesising Dynamics
Parameter Magnitude
Time
5
Analysing/Resynthesising Dynamics

• Study effect of manipulating temporal
  characteristics of emotions, e.g. smiles

Emotion 2007
6
Analysing/ResynthesisingDynamics
7
Analysing/ResynthesisingDynamics
8
Analysing/ResynthesisingDynamics

• What did we learn?
• Genuine Dynamics More likeable/Trustworthy/Emp
  loyable
• Fake Dynamics Less Likeable/Trustworthy/Employ
  able
• Relationship between Attractiveness and Facial
  Dynamics
• To Do! - Mock Video Game Scenario

9
Analysing Multiple Actions

• Define facial regions local
• to sub-facial actions
• Construct new training
• set of artificial images

10
Analysing Multiple Actions

1. Create a corresponding artificial training set of
   shape vectors

11
Performance Analysis
12
Image Based Transfer
13
Performance Analysis
14
3D Model Transfer

• Animation of a 3D facial model

15
3D Model Transfer
Dynamic knowledge easily applied to existing
pipelines
16
3D Facial Modelling

• 4D Camera
• Continuous 3D Scanning (Cardiff/Surrey)

17
3D Facial Modelling
(Link to 3D Example Expression)
18
3D Facial Modelling
19
Voice Driven Animation
Visual Speech Synthesis
Non-Verbal Action Synthesis
20
Perceptual Evaluation

• Motivation How best to evaluate facial
  animation?
• Perceptual test based on McGurk effect

ACM TAP 2005
21
Perceptual Evaluation

• Record participant responses
• Compare real and synthetic
• Subtle - participants blind to real motivation
• Isolate strengths and weaknesses

22
Reactive Full Body Avatar Animation

• Motivation
• Animate based on an input performance
• Key avatar drives other avatar behaviour

23
Conclusions and Future Directions

• Humans are very sensitive to different dynamics
• Different interpretation depending on timing
• Human realism will depend on understanding
  dynamic behaviours
• Realism will also depend on the accurate
  modelling and resynthesis of dynamics - e.g.
  McGurk effect
• Technology now allows study of 3D dynamics - our
  work has only just started!

24

• Thanks for listening!
• Acknowledgements
• Eva Krumhuber, Antony Manstead, Lanthao Benedikt,
  David Marshall, Paul Rosin, Alistair Calvert,
  Cathy Holt, Daniel Mason, Gemma Whatling, Simon
  Rushton, Susan Paddock
• RAEng, EPSRC.