Title: Outdoor Motion Capturing of Ski Jumpers using Multiple Video Cameras
1Outdoor Motion Capturing of Ski Jumpers using
Multiple Video Cameras
- Atle Nes
- atle.nes_at_hist.no
Faculty of Informatics and e-Learning Trondheim
University College
Department of Computer and Information
ScienceNorwegian University of Science and
Technology
2General description
- Task
- Create a cheap and portable video camera system
that can be used to capture and study the 3D
motion of ski jumping during take-off and early
flight. - Goals
- More reliable, direct and visual feedback
- More effective outdoor training
- ?Longer ski jumps!
32D?3D solution
- Multiple video cameras have been placed
strategically around in the ski jumping hill
capturing image sequences from different views
synchronously. - Allows us to reconstruct 3D coordinates if the
same physical point is detected in at least two
camera views.
4Camera equipment
- 3 x AVT Marlin F080B (CCD-based)
- FireWire/1394a (no frame grabber card needed)
- 640 x 480 x 30 fps
- 8-bit / 256 grays (color cameras not chosen
because of intensity interpolating bayer
patterns) - Exchangeable C-mount lenses (fixed and zoom)
5Camera equipment (cont.)
- Video data (3 x 9MB/s 27 MB/s)
- 2 GB RAM (5 seconds buffered to memory)
- 2 x WD Raptor 10.000 rpm in RAID-0 (enables
continuous capture) - Extended range
- 3 x 400 m optical fibre (full duplex firewire)
- Power from outlets around the hill
- 400 m BNC synchronization cable
6Camera setup
Synch pulse
Video data Control signals
7Direct Linear Transformation
- Based on the pinhole model - Linear image
formation
W
Z
image space(U, V, W)
object pointO (x, y, z)
principal point P (u0, v0, 0)
U
object space(X, Y, Z)
V
X
Y
image point I (u, v, 0)
projection centreN (u0, v0, d) (x0, y0, z0)
image plane(U, V)
8DLT Fundamentals
- Classical collinearity equations
- Standard DLT equations (aka 11 parameter
solution)
Abdel-Aziz and Karara 1971
9DLT Camera Calibration
- Minimum n 6 calibration points for each camera
(2n equations)
DLT parameters (unknowns)
10DLT Point reconstruction
- Minimum m 2 camera views of each reconstructed
image point (2m equations) - Usually a redundant set (more equations than
unknowns) ? Linear Least Squares Method
object coordinates (unknowns)
11Direct Linear Transform
- Loved by the computer vision community -
simplicity - Hated by the photogrammetrists - lack of accuracy
- DLT indirectly solves both the
- Intrinsic/Interior parameters (- 3 -)
- principal distance (d)
- principal point (u0,v0)
- Extrinsic/Exterior parameters (- 6 -)
- camera position (x0,y0,z0)
- pointing direction R(?, f, ?)
12Lens distortion / Optical errors
- Non-linearity is commonly introduced by imperfect
lenses (straight lines are no longer straight) - Should be taken into account for improved
accuracy - Additional parameters (- 7 -)
- radial distortion (K1,K2,K3)
- tangential distortion (P1,P2)
- linear distortion (AF,ORT)
13Radial distortion (symmetric)
14Lens distortion / optical errors
- Tangential distortion (decentering)
- Linear distortion (affinity, orthogonality)
U
U
V
V
Skewed image / Non-Orthogonality
Non-Square Pixels / Affinity
15Added nonlinear terms
- Extended collinearity equations
Brown 1966, 1971
16Bundle Adjustment
- Requires a good initial parameter guess (for
instance from a DLT Calibration) - Non-linear search - Iterative solution using the
Levenberg Marquardt Method - Basically Update one parameter, keep the rest
stable, see what happens Do this systematically - Calibration points and intrinsic/extrinsic
parameters can be separated blockwise - The matrix has a sparse structure which can be
exploited for lowering the computation time
17Detection of outliers
- Calibration points with the largest errors are
removed automatically/manually resulting in a
more stable geometry. - Both image and object point coordinates are
considered.
18Overview
- Direct Linear Transformation is used to estimate
the initial intrinsic and extrinsic parameter
values for the 2D?3D mapping. - Bundle Adjustment is used to refine the
parameters and geometry iteratively, including
the additional parameters. - Intrinsic Additional parameters off-site (focal
length, principal point, lens distortion) - Extrinsic parameters on-site (camera position
direction)
19Calibration frame
- Was used for finding estimates of theintrinsic
parameters. - Exact coordinates in the hill was measured using
differential GPS and a land survey robot station. - Points made visible in the camera views using
white marker spheres.
20Video processing
- Points must be automatically detected, identified
and tracked over time and accross different
views. - Reflective markers are placed on the ski jumpers
suit, helmet and skies.
21Video processing (cont.)
- Blur caused by fast moving jumpers (100 km/h) is
avoided by tuning aperture and integration time. - Three cameras gives a redundancy in case of
occluded/undetected points (epipolar lines). - Also possible to use information about the
structure of human body to identify relative
marker positions.
22Granåsen ski jump arena
23Granåsen ski jump arena
24Visualization
- Moving feature points are connected back onto a
dynamic 3D model of a ski jumper. - Model is allowed to be moved and controlled in a
large static model of the ski jump arena.
25Results
- Reconstruction accuracy
- Distance 30-40 meters
- Points in the hill 3 cm xyz
- Points on the ski jumper 5 cm xyzD
26Future work
- Real-Time Capturing and Visualization
- Direct Feedback to the Jumpers
- Time Efficient Algorithms
- Linear Closed-Form Solutions
27Questions?