Advanced Programming for 3D Applications CE00383-3

1
Advanced Programming for 3DApplicationsCE00383-3
Motion Capture Lecture 9
Bob Hobbs Staffordshire university
2
Definition of Motion Capture

• Motion capture is the recording of
• human body movement (or other
• movement) for immediate or delayed
• analysis and playback. MoCap

3
History

• Late 1970s Rebecca Allen created the first
  primitive form of motion capture. Rotoscoping
• 1980-1983- Tom Calvert invented Geniometers
• 1982-1983- Ginsberg Maxwell invented
  Graphical Marionette
• 1988- deGraf/Wahrman invented Mike the Talking
  Head

4

• 1988- Pacific Data Images invented Waldo C.
  Graphic
• 1989- Kleiser- Walczak invented Dozo
• 1991- Videosystem invented Mat the Ghost
• 1992- SimGraphics invented Mario
• 1992- Brad deGraph invented Alive
• 1993- Acclaim invented two character animation
  motion capture

5
Applications

• Entertainment
• Medicine
• Arts / Education
• Science / Engineering

6
Entertainment Live Action Films

• Computer generated characters in live action
  films (e.g. Battle Droids and many others in Star
  Wars Prequels, Gullum in The Lord of the Rings,
  King Kong in King Kong)

7
Entertainment 3D computer animations

• Characters in computer animated files (e.g. Polar
  Express, Monster House)

8
Entertainment Video Games

• Video games by Electronic Arts, Gremlin, id,
  RARE, Square, Konami, Namco, and others, (e.g.
  Enemy Territory)

9
Medicine

• Medicine (e.g., gait analysis, rehabilitation)
• Sports medicine (e.g. injury prevention,
  performance analyses, performance enhancement)

Gait Analysis Service
10
Arts / Education

• Dance and theatrical performances
• Archiving (e.g., Marcel Marceau)

OSU/ACCAD
11
Science / Engineering

• Computer Science (e.g., human motion database,
  recognitions)
• Engineering (e.g., Biped robot developments)
• Ergonomic product design
• Military (e.g., field exercises, virtual
  instructors, and role-playing games)

12
Types of mocap equipment

• Magnetic systems
• Mechanical systems
• Optical systems

13
Magnetic systems

• Utilize sensors placed on the body to measure the
  magnetic field generated by a transmitter source.

14
Magnetic systems

• Advantages
• Require no special lighting condition.
• Sensors are never occluded.
• Disadvantages
• Require a metal-fee environment.
• Electro-magnetic interference

15
Mechanical systems

• Exoskeleton with angle sensors.

16
Mechanical systems

• Advantages
• Measure joint angles (no marker ID problems).
• Sensors are never occluded.
• Disadvantages
• Breakable!
• Configuration of sensors is fixed.
• Constrains on joints.

17
Optical systems

• The cameras are equipped with infrared LED's and
  filters. (Filters enhance the contrast of the
  image.)
• The cameras see reflector markers.

18
Optical systems

• Advantages
• Higher sampling rate.
• Larger capture space.
• Disadvantages
• Markers are sometimes occluded -gt marker ID
  problems.
• Provide only positional data -gt joint angles need
  to be computed.

19
Typical Mocap system

• Vicon optical system - Best system in Academia!
• 8 high-speed MX 13 (up to 1000 fps) and 8
  high-resolution MX 40 (4 million pixels) cameras.
• Capture up to 5 performers at once.

20
Mocap animation

• Motion capture animation is different from
  keyframe animation in terms of how motion is
  created.
• Same principles apply to mocap animation
  keyframe animation!
• A combination of motion capture animation and
  keyframe animation is often used.

21
Keyframe animation

• A keyframe is a drawing of a key moment in an
  animated sequence, where the motion is at its
  extreme.
• Inbetweens fill the gaps between keyframes.
• Every motion is created by animators.

22
Advantages of mocap animation

• Faster to create (only if an established
  production pipeline exists.)
• Secondary motions and all the subtle motions are
  captured -gt more realism.
• Physical interactions between performers and
  props can be captured.

23
Disadvantages of mocap animation

• Cost.
• Manipulating mocap data is often difficult -gt
  Re-capturing or key framing a shot with bad data
  is often easier.
• Mapping mocap data of a performer to a character
  with a different proportion often causes problems.

24
Process of MoCap

• Data needs to be manipulated
• Transform data into file format
• Build specific models
• Attaching the mesh

25
movement flowchart for games

• Planning and Directing Motion Capture For
  GamesBy Melianthe Kines Gamasutra January 19,
  2000URL http//www.gamasutra.com/view/feature/34
  20/planning_and_directing_motion_.php

26
processing passive markers

• each camera records capture session
• extraction markers need to be identified in the
  image
• determines 2d position
• problem occlusion, marker is not seen
• use more cameras
• markers need to be labeled
• which marker is which?
• problem crossover, markers exchange labels
• may require user intervention
• compute 3d position if a marker is seen by at
  least 2 cameras then its position in 3d space
  can be determined

27
Direct Linear Transformation
BcA
28
Distortion Effects
The DLT model of Abdel-Aziz/Karara only accounts
for errors in perpendicularity between axes

• Achieves accuracy of 12000

Other types of distortion

• Radial

• Pin Cushion
• Barrel

• Tangential

• De-centering

29
16 parameter DLT

• Equations representing non-linear distortions

• L12-L14 represent symmetrical lens distortion

• L15 16 represent asymmetrical/ de-centering
  distortion

30
The Problem

• The original DLT method contained 11 unknowns

Uo,Vo
2 scaling factors
Xo,Yo,Zo
d
3 rotation angles

• Only 10 are actually independent
• d and the 2 scaling factors are mutually
  dependent variables

31
Modified Direct Linear Transformation

• The Non-Linear constraint is added to the system
  to ensure orthogonality

• To solve
• Compute the 11 parameters normally
• Use the value of one parameter to remove it from
  the equation and estimate the 10 independent
  parameters
• Solve for the value of the removed parameter
  using the values of the 10 independent parameters

32
Non-Linear MDLT

• Adds correction for lens distortion to the MDLT
  method

33
retargeting the character

• the character is controlled by skeleton
• to control the skeleton, need to specify joint
  rotations
• muscles?

34
retargeting

• capture motion on performer
• positions of markers are recorded
• retarget motion on a virtual character
• motion is usually applied to a skeleton
• a skeleton is hierarchical
• linked joints
• need rotation data!
• need to convert positions to rotations

35
performer? actor ? character

• the actor is used to convert marker positions to
  rotational data
• markers are handles on the actor
• actor should have similar proportions as the
  performer
• joint rotations of the actor are applied to the
  character
• there are still issues with proportions
• http//www.youtube.com/watch?vfrNpkIH8_vo

Alias Motionbuilder actor and markers
36
retargeting problems
37
retargeting problems
38
Production pipeline overview

• Calibrate the system.
• Fit a generic skeleton to the subjects
  proportion (subject calibration).
• Capture shots reconstruct 3D trajectories using
  the calibrated subject.
• Link the subject specific skeleton to a CG
  characters skeleton and edit motion (in
  MotionBuilder).
• Add skin to the CG character, edit motion, and
  render (in Maya).http//atec.utdallas.edu/midori
  /Handouts/mocap_pipeline.htm

39
2D Image 2D Camera Data
3D Markers Positions
Trajectories
System Calibration and Capturing and Processing
Data
C3D
Triangulation
Reconstruction
Circle fitting
Positional Data
IQ
Subject Calibration

VSK
Actor Setup
VST (subject template)
MOTION BUILDER

• Capture Range of Motion (ROM)
• Reconstruct trajectories of ROM
• Label markers

• Place markers on Actor ( intermediate skeleton )

Calibrated Subject
Character Setup
The processes that you go through for each
character
Mocap Pipeline Flow Chart
HIK

• Correlate
• Actor
• (C3D, HIK)
• Character
• (FBX)
• Edit motion
• Bake motion data to the skeleton

Markers Set
Character Setup

• Create a skeleton
• Bind skin to the skeleton
• Rig the character

FBX
MAYA
Skeleton Only
The processes that you repeat for each shot
Rendering
FBX

• Marge the rigged character (MB) and the skeleton
  with motion data (FBX)
• Edit motion (IK/FK blend, Trax)
• Render

MB
Skeleton with motion data ( joint rotation
angles )
Maya Scene File
40
References

• Sturman, David. A Brief History of Motion
  Capture for Computer Character Animation. 13
  March 1999. MEDIALAB. 25 Nov. 2005.
  http//www.siggraph.org/education/materials/HyperG
  raph/animation/character_animation
• When Motion Capture Beats Keyframing. Sept.
  1997. Game Developer. http//www.gdmag.com
• http//www.kwon3d.com/theory/dlt/dlt.html
• http//www.kwon3d.com/theory/dlt/mdlt.html
• http//express.howstuffworks.com/gollum3.htm

41
Related Work

• Motion capture search
• Arikan and Forsyth 2002 Kovar et al. 2002 Lee
  et al. 2002 Li et al. 2002 Metoyer 2002
• Physics and motion capture
• Rose et al. 1996
• Popovic and Witkin 1999 Pollard 1999 Pollard
  and Behmaram-Mosavat 2000
• Shapiro et al. 2003
• Physically based reactions
• Oshita and Makinouchi 2001 Zordan and Hodgins
  2002
• Komura et al. 2004 Mandel 2004
• Industry Software
• Natural Motions Endorphin