Converting 3D Facial Animation with Gouraud shaded SVG

1
Converting 3D Facial Animation with Gouraud
shaded SVG

• A method to extract 3D animated files and to
  convert it in SVG allowing Gouraud shading

Michel Gaudry, Thomas Di Giacomo, Nadia
Magnenat-Thalmann
2
Outline

• Introduction
• Context
• Purpose
• Related work
• 3D on mobile devices
• Conversion of 3D to other media
• 3D to SVG converter
• Context and requirements
• Process
• Structure
• 2D data extraction
• Generation of SVG
• SVG
• Optimization
• Gouraud shading
• Experiments and evaluation
• The pyramid
• The talking head
• Conclusion

3
Introduction

• Context and motivation
• Developement of wearable devices
• User's growing interest for web graphics and 3D
• Hardware limitations
• Increasing availability of SVG for web and
  mobiles devices
• Possibility to convert 3D animated faces to SVG

4
Introduction

• Purpose
• Extend the authoring of SVG
• Enable new SVG authoring by creating scenes in a
  3D environment and converting it in SVG
• Extend SVG rendering with motion captured
  sequence or highly realistic 3D rendering
• Extends availlability of 3D graphics with other
  medias
• Generate animated and Gouraud shaded SVG to
  simulate 3D scenes
• Extend 3D possibilities on mobile devices
• Method to convert generic 3D to SVG
• Specific for large 3D scenes and focus on face
  animation and deformation
• Simulation of light shading with Gouraud

5
Related work

• 3D on mobile devices
• Adaptations
• Hybrid approaches
• VRML
• Simplification of 3D
• Conversion of 3D to other media
• To image and video
• Current growing technics
• Java 2 Micro Edition and JSR-184
• OpenGL ES
• X3D
• 3D with SVG
• Advantages
• Conversion from other languages

6
3D to SVG Converter

• Purpose
• Convert 3D animations
• Animation data from external file
• Allow animation optimization and Gouraud shading
• Main structure
• Open a 3D animation
• Identification and extraction of 2D data
• Generation of SVG
• Optimized animation
• Gouraud shaded image

7
Context and requirements

• Formats
• VRML chosen as input
• FAP for face deformation
• Animation inputs
• User actions
• Predefined
• Not additional software or scripts
• Preserve
• Pixel spatial information
• Pixel color information
• Gouraud shading
• Non animated objects
• Real time conversion
• Ease authoring of SVG animation with 3D

8
Structure

• 3 main modules
• Extraction
• Animation
• Generation
• User interface
• 2 optional generation modules

3D rendering with Open GL
User menu
GUI manager
VRML FAP
Animation record
Extraction
SVG
9
Process

• 3D to SVG
• Extraction from VRML file
• Get dynamic data from user or file input
• 2D data
• 2D mesh initialized with a list of path and
  colors per triangles
• Conversion of triangles animation with gradient,
  position and visibility
• SVG generator
• Pre-generation of SVG file
• Generation

10
2D data extraction

• Algorithm
• Initialization of
• the 2D "copy" topology, index, colors
• static data camera parameters
• Update and Extraction
• 3D vertices dynamic positions
• Culling of 3D faces
• Projection of 3D to viewpoint plane
• Storage of computed 2D coordinates

11
2D data extraction

• Gouraud shading
• Method
• In SVG
• Projection
• Visibility

12
Generation of SVG

• From application cache
• Animation

rotate(X)" ... --
13
Generation of SVG
style"fillrgb(c1,c2,c3)" attributeName"d" begin"abs" dur"ass"
fill"freeze" vallues" M x00 y00 L x10 y10 L
x20 y20 z M x00 y00 L x10 y10 L x20 y20
z ... M x0n y0n L x1n y1n L x2n y2n z
"/ to"hidden" begin"hb1 s" dur"hs1 s"/ begin"hbn s" dur"hsn s"/

• To represent triangles
• Coordinates
• Visibility
• Filling
• Single color
• Gradients

14
Generation of SVG

• Optimization
• Problem
• Generated files can be big
• Module
• Can be integrated in an application or can be
  used independently to optimize any SVG animation
  file
• Processed before or after generation
• Complementary to gzip compression
• Methods
• Reduce length of static sequences
• Remove invisibles sequences

15
Generation of SVG

• Optimization methods
• Reduce static sequences
• Remove repeated coordinates when the polygon is
  not moving
• Remove invisible sequences
• Remove the tags and cut main animation in
  parts
• Others possibilities
• Use relative coordinates
• Simplify animation path

16
Generation of SVG

• Gouraud shading
• Purpose
• Improve realism, from flash-shaded triangles to
  interpolated colors vertex
• Method
• Adaptation of the method presented in
   Interactive 3D Viewer Written in SVG  (SVG
  Open 2003)
• In large scale
• For a 3D converted mesh
• Gouraud with SVG
• Use of gradients and filters
• Superposition of triangles
• Gradients
• 3 gradients per triangle, one from each vertex
• Blending of the gradients with
  attribute
• Filters

Alpha value  100
Gradient area
Gradients path
Alpha value  0
Triangle area
17
Generation of SVG
feColorMatrix
Element canvas

• Gouraud shading
• Structure of a triangle

T2
T1
path
g
T3
T1

path
T4
path
T1
Sub-triangle
18
File structure
in"SourceGraphic" in2"BackgroundImage"
operator"arithmetic" k1"0" k2"1" k3"1"
k4"0"/ x type"matrix" values" 1 0 0 0 0 0 1 0 0 0 0 0
1 0 0 1 1 1 1 0 0 0 0 0 1 "/ id"t0" d"M263.245 379.216 227.297 173.235
219.885 164.915 z" / gradientUnits"userSpaceOnUse" x1"263.245"
y1"379.216" x2"345.509" y2"305.932" offset"0" style"stop-colorrgb(47.098, 30.779,
23.575) stop-opacity1"/ style"stop-colorrgb(47.098, 30.779, 23.575)
stop-opacity0"/

• Header
• SVG
• Viewport
• Definitions
• Filters
• Triangles
• Gradients

19
File structure

• Construction of the triangles
• Subtriangle
• Merge of the 3 gradient's triangles
• Adjust opacity

67)" fill-opacity"1"/
e xlinkhref"t0" fill"url(g0_0)" /
xlinkhref"t0" fill"url(g0_1)"
filter"url(colorAdd)"/
fill"url(g0_2)" filter"url(colorAdd)"/
20
Experiments and evaluation

• Files used for test
• The pyramid
• A simple rotating polygon with user interaction
• The face
• Simple face with 484 triangles
• Mesh deformation using MPEG-4 3D facial animation
  as input

21
The pyramid

• Results
• Visual
• Reproduction of 3D scene
• Faces visibility
• Size
• Acceptable for playback with standard SVG viewers
  on light devices

22
The talking head

• Results
• Animation
• Reproduction of the original animation
• Gouraud shading
• Results
• Drawbacks

23
Results table
24
Conclusion

• Conversion
• Improvements
• Optimization
• Interpolation of animation's path
• Gouraud shading