Monocular Human Motion Tracking

1
Monocular Human Motion Tracking

• C. Barron and Ioannis Kakadiaris
• University of Houston
• ECE 285 Presentation
• Presented by Zachary Bauer
• 2/2/2005

2
Problem Statement

• Given an image sequence (from a fixed monocular
  camera) of a moving human, estimate his or her
  motion by estimating the corresponding pose of an
  essentially determined (up-to-scale) Virtual
  Human model (VHM) at each frame of the image
  sequence.

3
Goals

• Use a detailed VHM to model human movement.
• Develop an algorithm to do 3D motion capture from
  a monocular, uncalibrated camera.
• Ensure algorithm is efficient, simple, and robust.

4
Proposed Solution

• Semiautomatically initialize VHM model to first
  acceptable image frame.
• Fit VHM to image by calculating the
  Sum-of-Squared Intensity Differences (SSD)
  between the VHM projection and the image.

5
Proposed Solution

• Use penalty factors to make SSD minimization
  problem convex and tractable.
• Decompose minimization problem into smaller
  problem subsets
• chest-neck-head
• left arm
• right arm
• left leg
• right leg.

6
Sum-of-Squared Differences (SSD)

• VHM(0)Cx,y is a projection of the VHM into 2D.
• P is the coordinates of a pixel
• Vp is the intensity value at p in the projected
  VHM image.
• Vp is the intensity value at p in the current
  image.

7
Virtual Human model (VHM)

• 18 joints
• 57 degrees of freedom
• Represented as a set of arrays of position and
  appearance (xsi,ysi,zsi,rsi,gsi,bsi)

8
System

• Input from uncalibrated monocular camera or
  synthetic computer graphics images.
• Use color information for segmentation.
• Use intensity information to track.
• Computer system used? Unknown
• Run-time? - Unknown

9
STEP 1 Semiautomatic Initialization

• User selects landmarks on figure in first
  image.
• Algorithm fits VHM to landmarks.
• User adjusts girth of VHM segments to fit the
  segmented person.

10
STEP 2 Update VHM

• Update VHM by assuming the pixels values using
  the overlap between the VHM and the current
  image.
• Map visible 3D VHM points onto 2D image of the
  camera plane.

11
STEP 3 Estimate Position

• Estimate position of each segment
  (i.e.-CHEST-NCK-HD, LARM, etc.) separately to
  reduce problem size.
• Work from the hips out.

12
STEP 3 (cont.)

• Map each segment into spherical coordinates.
• Check a small arc of rotations and perform
  Penalized SSD.

13
STEP 4 PSSD Minimization

• Use the penalizing factors to weight the SSDs to
  make them convex.
• Find the global minima in the PSSD (Penalized
  SSD). This point corresponds to the VHM segment
  rotation that best matches the image segment.

14
STEP 4 (cont.)
15
Penalty Factors

• Lamda(Vp) Penalty factors from VHM segments
  under C.
• Lamda(vp) Penalty factors for values in the
  image.
• Lamda 1 for pixels in a region in the image
  that correspond to the subject of interest.
• Lamda gtgt 1 otherwise.

16
Penalty Factors (cont.)

• Within a region of interest
• Pixels near proximal joint weighted less in SSD.
• Pixels near distal joint/segment given more
  weight.

17
Penalty Factors ???

• How are these penalty factor regions determined?
• Why are the areas to be emphasized actually give
  lower penalty factors?

18
Results

• The algorithm successfully tracked 3D motion the
  image with high accuracy
• (1 for non occluded areas).
• Robust to partial self occlusion, rapid movement,
  low image quality, large displacement.
• Penalty factors and minimization problem
  breakdown succeeded in making the nonconvex
  minimization problem tractable.
• Works for monocular image, independent of camera.

19
Results (cont.)
20
Results (cont.)
Successfully tracks despite moving background,
occlusion, and low image quality.
21
Notes

• Does not focus much on segmentation
• Option between adaptive background subtraction or
  color segmentation.
• No discussion of shadows
• Tests generally done in uniformly lit areas.
• Very reliant on manual initialization

22
Additional Information

• Ioannis Kakadiaris
• University of Houston
• http//www.vcl.uh.edu/ioannisk/
• Other projects Optical Tracking for
  Telepresence/Teleoperation Space Applications
• Robust real-time 3D tracking

23
Related Information

• I. Mikic, E. Hunter, M. Trivedi, P. Cosman,
  "Articulated Body Posture Estimation from
  Multi-Camera Voxel Data", IEEE International
  Conference on Computer Vision and Pattern
  Recognition, Kauai, Hawaii, December 2001
• K. Huang and M. Trivedi, "Omni and Rectilinear
  Video Arrays for Real-TimePerson Tracking,"
  Submitted to PETS2001 Workshop, CVPR2001
  Conference, Hawaii,Dec. 2001