Omnidirectional Vision

1
Omnidirectional Vision
CSc I6716 Fall 2005

• Topic 2 of Part II
• Omnidirectional Cameras

Zhigang Zhu, City College of New York
zhu_at_cs.ccny.cuny.edu
2
Lecture Outline

• Applications
• Robot navigation, Surveillance, Smart rooms
• Video-conferencing/ Tele-presence
• Multimedia/Visualization
• Page of Omnidirectional Vision (Many universities
  and companies.)
• http//www.cis.upenn.edu/kostas/omni.html
• Design Requirements
• 360 degree FOV, or semi-sphere or full sphere in
  one snapshot
• Single effective viewpoint
• Image Resolutions one or more cameras?
• Image Sharpness optics as well as geometry
• Several Important Designs
• Catadioptric imaging mirror (reflection) lens
  ( refraction)
• Mirrors Planar, Conic, Spherical, Hyperboloidal,
  Ellipsoidal, Paraboloidal
• Systematic design ( S. Nayars group)

3
Sensor Design

• Catadioptric imaging
• mirror (reflection) lens ( refraction)
• Theory of Catadioptric Image Formation ( S.
  Nayars group)
• "A Theory of Single-Viewpoint Catadioptric Image
  Formation" , Simon Baker and Shree K. Nayar
  ,International Journal of Computer Vision, 1999.
• Mirrors
• Planar
• Conic, Spherical
• Hyperboloidal, Ellipsoidal
• Paraboloidal
• Cameras (Lens)
• Perspective (pinhole) or orthogonal (tele-centric
  lens) projection
• One or more?
• Implementations
• Compactness - size, support, and installation
• Optics Image sharpness, reflection, etc.

4
Planar Mirror

• Panoramic camera system using a pyramid with four
  (or more) planar mirrors and four (or more)
  cameras (Nalwa96) has a single effective
  viewpoint

Mirror pyramid
6 cameras
4 camera design and 6 camera prototype FullView
- Lucent Technology http//www.fullview.com/
5
Planar Mirror

• Panoramic camera system using a pyramid with four
  (or more) planar mirrors and four (or more)
  cameras (Nalwa96) has a single effective
  viewpoint

Geometry of 4 camera approach four separate
cameras in 4 viewpoints can generate images with
a single effective viewpoint
6
Planar Mirror Approach

• A single effective viewpoint
• More than one cameras
• High image resolution

7
Planar Mirror Approach

• A single effective viewpoint
• More than one cameras
• High image resolution

8
Conic Mirror

• Viewpoints on a circle
• semispherical view except occlusion
• Perspective projection in each direction
• Robot Navigation (Yagi90, Zhu96/98)

9
Spherical Mirror

• Viewpoints on a spherical-like surface
• Easy to construct (Hong91 -UMass )

10
Hyperboloidal Mirror

• Single Viewpoint
• if the pinhole of the real camera and the virtual
  viewpoint are located at the two loci of the
  hyperboloid
• Semi-spherical view except the self occlusion

11
Hyperboloidal Mirror

• ACCOWLE Co., LTD, A Spin-off at Kyoto University
• http//www.accowle.com/english/ 
• Spherical Mirror
• Hyperbolic Mirror

Image High res. in the top
12
Ellipsoidal Mirror

• Single Viewpoint
• if the pinhole of the real camera and the virtual
  viewpoint are located at the two loci of the
  ellipsoid
• Semi-spherical view except the self occlusion

13
Panoramic Annular Lens
- geometric mathematical model for image
transform calibration
panoramic annular lens (PAL) - invented by Pal
Greguss 40 mm in diameter, C-mount view H
360, V -15 20 single view point (O)
14
Panoramic Annular Lens

• panoramic annular lens (PAL)
• - invented by P. Greguss
• 40 mm in diameter, C-mount
• view H 360, V -15 20
• single view point (O)
• C-Mount to CCD Cameras

Image High res. In the bottom
15
Cylindrical panoramic un-warping
Two Steps (1). Center determination (2)
Distortion rectification 2-order polynomial
approximation
16
Paraboloidal Mirror

• Semi-spherical view except the self occlusion
• Single Viewpoint at the locus of the paraboloid,
  if
• Tele-lens - orthographic projection is used
• Mapping between image, mirror and the world
  invariant to translation of the mirror. This
  greatly simplifies calibration and the
  computation of perspective images from
  paraboloidal images

17
Paraboloidal Mirror

• Remote Reality A Spin-off at Columbia
  University
• http//www.remotereality.com/

Camcorder
Web Camera
Back to Back Full Spherical View
18
Paraboloidal Mirror

• Remote Reality A Spin-off at Columbia
  University
• http//www.remotereality.com/

19
Catadioptric Camera Calibration

• Omnidirectional Camera Calibration Harder or
  Easier?
• In general, the reflection by the 2nd order
  surface makes the calibration procedure harder
• However, 360 view may be helpful
• Paraboloidal mirror orthogonal projection
• Mapping between image, mirror and the world
  invariant to translation of the mirror.
• Projections of two sets of parallel lines suffice
  for intrinsic calibration from one view
• C. Geyer and K. Daniilidis, "Catadioptric Camera
  calibration", In Proc. Int. Conf. on Computer
  Vision, Kerkyra, Greece, Sep. 22-25, pp. 398-404,
  1999.

20
Image Properties of Paraboloid System
(Assuming aspect ratio 1)

• The Image of a Line
• is a circular arc if the line is not parallel to
  the optical axis
• Is projected on a (radial) line otherwise
• Dual Vanishing Points
• There are two VPs for each set of parallel lines,
  which are the intersections of the corresponding
  circles
• Collinear Centers
• The center of the circles for a set of parallel
  lines are collinear
• Vanishing Circle
• The vanishing points of lines with coplanar
  directions lie on a circle ( all the lines
  parallel to a common plane)

21
Image Properties of Paraboloid System
(with aspect ratio)

• The Image Center
• Is on the (vanishing) line connecting the dual
  vanishing points of each set of parallel lines
• Can be determined by two sets of parallel lines
• Projection of a Line with unknown aspect ratio
• Is an elliptical arc in the general case
• The Aspect Ratio
• Is determined by the ratio of the lone-short axes
  of the ellipse corresponding to a line
• Intrinsic Calibration
• Estimate aspect ratio by the ratio of ellipse
• Estimate the image center by the intersection of
  vanishing lines of two sets of parallel lines in
  3-D space

22
Calibration of Paraboloid System

• The Image Center
• Is on the (vanishing) line connecting the dual
  vanishing points of each set of parallel lines
• Can be determined by two sets of parallel lines

23
Calibration of Paraboloid System

• The Image Center
• Yellow vanishing line of horizontal set of
  parallel lines
• Pink vanishing line of vertical set of parallel
  lines
• The Vanishing Circle (Red dotted)
• The vanishing points of lines with coplanar
  directions ( on a plane in this example)

Projected to the plane of the calibration pattern
24
Next

• Turn in your projects and schedule meetings with
  me

END