Geometric Algorithms in 3D RealTime Rendering and Facial Expression Modeling Part II - PowerPoint PPT Presentation

Loading...

PPT – Geometric Algorithms in 3D RealTime Rendering and Facial Expression Modeling Part II PowerPoint presentation | free to download - id: 53882-MTQ1Y



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Geometric Algorithms in 3D RealTime Rendering and Facial Expression Modeling Part II

Description:

One for every animated character created! NPR vs. PR ... Methodology: Portrait generator. Producing black-and-white images using binarization ... – PowerPoint PPT presentation

Number of Views:109
Avg rating:3.0/5.0
Slides: 60
Provided by: russela
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Geometric Algorithms in 3D RealTime Rendering and Facial Expression Modeling Part II


1
Geometric Algorithms in 3D Real-Time Rendering
and Facial Expression Modeling (Part II)
  • Prof. Marina L. Gavrilova
  • University of Calgary

2
Where we are…
  • Brief Overview of the Area
  • Geometric Approaches for mesh reconstruction and
    image morphing
  • Real-Time Optimally Adapting Mesh Algorithms
  • Non Photo-realistic Facial Expression Modeling
    and Morphing Algorithms
  • Swarm Real-time Visualization System

3
Part II
  • NPR Face Rendering

4
Terminology
  • DT Delaunay triangulation
  • VD Voronoi Diagram
  • FT Feature Transform
  • DT Distance Transform

5
Preliminaries
  • Transformation from a photograph into an artistic
    work

Li and Huang 03
6
Preliminaries
  • Transformation from an Excel into Artistic Work ?

7
NPR vs. PR philosophy
  • PHOTOREALISTIC
  • GOALS imitate as realistically as possible
    facial expression and details
  • Methodology
  • complex geometry fitting,
  • generating high density meshes,
  • recovering camera positions,
  • utilizing data interpolation,
  • simulating light conditions,
  • applying texturing and other techniques
  • Outcome
  • Very costly processing, highly realistic
    visualization
  • NON-PHOTOREALISTIC
  • GOALS imitate emotions as closely as possible,
    while at the same time allowing the model its own
    identity and artistic representation
  • Methodology
  • Utilize artistic drawings
  • Feature line extraction
  • Expression morphing
  • Model partitioning onto regions
  • Outcome
  • Expression and emotion preservation, efficient
    storage, fast processing, artistic visualization

8
Animation Trivia
  • Face models used in Pixars Toy Story had
    approximately how many control points each?
  • 100
  • 1000
  • 10000
  • None they were made by hand out of clay!

9
Animation Trivia
  • TOY STORY" used how many SUN workstations to
    render images?
  • 1
  • 100
  • 1000
  • None they were all IBM workstations!

10
Animation Trivia
  • Pixar have received how many Academy Awards
    from the Academy of Motion Picture Arts and
    Sciences, including an Oscar for Best Short
    Animated Film for 'Tin Toy' and a Technical
    Achievement Award for Pixar's RenderMan
    software?
  • 2
  • 6
  • 12
  • One for every animated character created!

11
NPR vs. PR
  • PHOTOREALISTIC - Final Fantasy (from Final
    Fantasy website)

NON-PHOTOREALISTIC - Shrek 10 to 1 preference
Crawford (Shrek movie web site)
12
Oscar for ACM Fellow
  • ACM Bulletin Service Today's Topic ACM Fellow Ed
    Catmull Receives Oscar for Contributions to
    Computer Graphics February 25, 2009
  • ACM Fellow Ed Catmull was presented with an Oscar
    statuette for the Gordon E. Sawyer Award by the
    Board of Governors of the Academy of Motion
    Picture Arts and Sciences.
  • Conferred at the Scientific and Technical Awards
    Presentations earlier this month, the award is in
    recognition of his lifetime of technical
    contributions and leadership in the field of
    computer graphics for the motion picture
    industry.

13
Objectives of our Research
  • To develop efficient and realistic portrait
    generation method from 2D photographs
  • To develop fast and easy 2D morphing algorithm
  • To create appropriate 3D mesh representation for
    model morphing

14
Developed Methodology
  • Proposed to use NPR modeling by example for
    portrait generation
  • Applied binarization method to produce black and
    white images from photograph
  • Developed segmentation technique for detailed
    feature extraction
  • Used combined edge eextraction strategy
  • Used application-dependent style for rendering
  • Developed Euclidean distance transform based
    algorithm for morphing

15
Facial Modeling Research
  • Focus on Human Faces

Lin and Hong 02
Li and Kobatake 97
16
Facial Modeling Research
  • Other examples

Lin and Hong 02
Gooch 04
Hong 04
17
Motivation
  • Shortcoming of some previous works
  • Facial creases, wrinkles and freckles are hard to
    represent
  • Detail feature lines of different expressions and
    which lines?
  • Efficient morphing

18
Contribution
  • Task Generate a smooth line-drawing animation
    from two input photographs
  • Input/Output two human face images as input and
    output of a NPR facial animation
  • Methodology
  • Create NPR style portrait from a frontal facial
    photograph to generate a digital artwork.
  • Use segmentation and tracking methods to map
    expressive lines representing expressive
    wrinkles and creases into the image.
  • Utilize morphing algorithm based on Euclidean
    distance transform that does not require control
    points or control lines, and finds pixel
    correspondence automatically.
  • Use the developed system as a prototype for 3D
    morphing based on controllable mesh deformations
    utilizing Delaunay triangulation.

19
System FlowChart
20
Methodology Use artists rendering as a template
Six Primary Facial Expressions drawn by
artistsanger, disgust, fear, happiness, sadness
and surprise
21
Methodology Portrait generator
  • Producing black-and-white images using
    binarization

22
Methodology Details Extraction
Enlarge certain areas
Edge Detection
Delete normal contours
Detail feature lines tracking
Save feature Lines information
23
Methodology region subdivision
  • Five interested sub-regions are used to extract
    details

24
Methodology edge detection
  • Edge Detection Process Sobel, Prewitt, Roberts,
    LoG, Canny operators (from (b) to (f)), g
    combination of e and f

25
Methodology details extraction
  • Close-up Details Extraction feature

26
Methodology feature line tracking
  • Feature Lines Tracking

27
Methodology feature line tracking
  • Tracking Strategy

28
Methodology feature line tracking
  • Data Structure to store the feature lines

29
Methodology Distance transform for Morphing
Image
Template
30
Distance Transform
Given an n x m binary image I of white and black
pixels, the distance transform of I is a map that
assigns to each pixel the distance to the nearest
black pixel (a feature).
31
Feature Transform
The feature transform of I is a map that assigns
to each pixel the feature that is nearest to it.
32
A Fast Algorithm for Computing the Euclidean
Distance Transform
  • Works in Euclidean Metric
  • Optimal linear in the number of pixels
  • Proceed in two passes over the image, top down
    and bottom up
  • For each row
  • Maintain a polygonal chain - a set of pixels in
    which the nearest feature may lies
  • Updates the chain for next row by pruning - it
    becomes the minimal set of pixels
  • Prune Endpoints and Internal Vertices

33
Chain Definition
  • One Chain for the row
  • Contains at most one pixel per column
  • Contains the lowest pixel in the column
  • After pruning, it contains only those points that
    will be nearest features to some points in the
    row
  • Dynamically updated from row to row

34
Example of a a Chain
35
Pruning Endpoints
  • Identifying superfluous endpoints

36
Pruning Internal Points
  • Point q is removed

37
Pruning Internal Points
  • Point q remains

38
Results on NPR Rendering
39
Experimental results for distance transform
algorithm
Algm 1 with pruning was tested Saturation levels
5, 15, 30 Results Linear Running Time,
Regardless of Saturation ( of black pixels in
image)
40
Morphing Direction
41
Results of utilizing Distance Transform for
Morphing
Starting Frame
Ending Frame
42
Results of feature lines extraction
43
  • Line Simplification and Rendering Results

44
  • Line Simplification and Rendering

45
Animation
46
3D Facial Model Synthesis
47
Brief Preliminaries
  • Over the last 30 years, thousands of papers were
    devoted to 3D face animation and modeling.
  • Different classification and registration
    techniques have been developed, including those
    based on 2D feature matching, multiple point of
    view registration (using calibrated cameras),
    nearest-neighbor feature comparison,
    segmentation-based feature extraction, ICP
    (Iterative Closest Point-based Matching), TPS
    (Thin Plates Spline) algorithms and others.

48
3D Facial System Classification (by Akarun,
Gokberk, Salah)
49
3D Facial System Classification (by Akarun,
Gokberk, Salah) cont.
50
3D Facial System Classification (by Akarun,
Gokberk, Salah) cont.
51
3D Facial System Classification (by Akarun,
Gokberk, Salah) cont.
52
Voronoi Diagram for 3D face modeling
  • Voronoi Diagram is a partitioning of the space
    into convex regions such that any point within
    the region is closer to the site that originates
    this region than to any oethr site in the space.
  • Delaunay Triangulation is the dual structure that
    connects those sites whose Voronoi Regions share
    an edge (in 2D) or a face (in 3D).

53
Methodology
  • Utilize geometric concept Voronoi diagram in
    application to FFD (free-form deformation) model.
  • Develop a method for 3D facial model synthesis
    that combines the traditional free-form
    deformation (FFD) model with techniques of data
    interpolation based on Voronoi diagrams.
  • With 18 feature points extracted from 2D facial
    images in two orthogonal views, Dirichlet Free
    Form Deformation (DFFD) is utilized for modifying
    a generic 3D face to produce the individual face.
  • The main advantages of this approach over former
    extensions of FFD is in removing the constraints
    on control lattice and control points location.
  • By assigning different weights to those control
    points, we make the method more adaptable to the
    facial deformation. The reconstructed 3D faces
    can be used to generate different facial
    animations.

54
Methodology
  • Build DFFD model
  • Compute Sibson coordinates using space
    partitioning introduced by Voronoi diagram
  • Compute deformation in 3D using weighted DFFD
    relationship based on computed Sibson coordinates
  • Adapt approach for face synthesis using Delaunay
    triangulation

55
Illustration
  • 28 zones (below).
  • Control points (above).

56
Illustration
  • FaceGen generic mesh and texturing result

57
Illustration
  • Original face (left) and generated face (right)

58
Extensions
  • The developed methodology is currently being
    utilized for 3D face morphing and expression
    synthesis.

59
Acknowledgements NSERC Granting Agency,
Canada GEOIDE Network of Excellence,
Canada Canadian Foundation for Innovation All
students of SPARC LAB and BT Lab, University of
Calgary, in particularly Russel Apu and Luo Yuan
All collaborators and of course the audience
About PowerShow.com