Title: Survey: Visionbased Model generation of 3D real world scene
1Survey Vision-based Model generation of 3-D
real world scene
- ???, Marc Nguyen, ???
- Nov. 05
2Introduction
- Building 3-D Model without using wrestling with
CAD tools for months ? - Labor-intensive
- time-consuming
- resulting models is apparently computer-generated
- cant be sure about the accuracy of the model
- The Alternatives
- vision-based approach
- take some photos, process them, ready-to-go
- 3-D scanning
- not suitable for outdoor scene
3Modeling Approaches
Geometry(CAD) -based
Vision-based
4Decisions to make
- Model polygon or image?
- Tightly coupled to utilization of the model
- 3-D polygonal model
- conventional VR walk-thru / fly-by
- image-based model
- image-based renderering/VR (sort of QuickTime
VR) - User input?
- Fully automatic
- User input as needed
5Getting Polygonal models
- Existing works
- Depth map textures
- Hybrid approach Devebec96
- Issues
- shape-from
- Stereo
- Motion
- something else ?
- which feature to use
- pixel, line, face,
6Basic PrinciplesStereo Vision(1/5)
- Basic formula
- reconstruction of the 3-D coordinates of a number
of points in a scene for given 2 (or more) images
obtained by cameras of known relative positions
and orientations - Correspondence problem
- given a token in image 1, what is the
corresponding token in image 2?
7Basic PrinciplesStereo Vision(2/5)
- Constraints
- epipolar constraint
- for a given point in the plane 1, its possible
matches in the plane 2 all lie on a line,
therefore search space is reduced from 2D to 1D
8Basic PrinciplesStereo Vision(3/5)
- Ordering constraints
- the orders of tokens in one image is preserved in
the other image (not true when one token is in
the forbidden region of the other token)
9Basic PrinciplesStereo Vision(4/5)
- Planarity constraint
- if the surfaces of the objects are planar, there
exists an analytic transformation from the left
image coordinates to the right image coordinates.
10Basic PrinciplesStereo Vision(5/5)
- Limitation
- still exist ambiguity
- the distance between of the two camera must be
sufficiently small
11Basic PrinciplesModel-based Vision(1/3)
- Basic principle
- to recognize 3D objects, compare a scene model
(constructed by processing images obtained from
sensors) against entities in a model database
(containing a discription of each object the
system is expected to recognize).
12Basic PrinciplesModel-based Vision(2/3)
- Related works
- Hanson and HendersonHans89
- the automatic synthesis of a specialized
recognition scheme, called a strategy tree based
on CAGD(computer aided geometric design) model. - Strategy tree
- describe the search process used for recognition
and localization of a particular objects in the
given scene - consist of selected 3D features which satisfy
system constraints and corroborating evidence
subtrees which are used in the formation of
hypothesis.
13Basic PrinciplesModel-based Vision(3/3)
- Flynn and Jain Flyn91
- develop a system which uses 3D object
descriptions created on a commercial CAD system - express in both the industry-standard IGES
(initial graphics exchange specification) form
and a polyhedral approximation - perform geometric inferencing to obtain a
relational graph representation of the object
which can be stored in a database of models for
object recognition
14Depth map Textures
- Not provide polygonal representation
- need further processing(e.g mesh construction)
- Need special H/W
- 3D scanner
- laser range finder
- video-rate stereo machine
Http//www.cyberware.com
15Depth map Texture T.Kanade at CMU (1/3)
- MBV(Modeling by Videotaping)
- Walking around the room with camcorder, and get
the 3-D model of the room and the trajectory of
camera - Based on shape-from-motion
- factorization technique
Terrain
House
http//www.ius.cs.cmu.edu/IUS/mbvc0/www/modeling.h
tml
16Related works at CMU (2/3)
- Z-key
- generation of depth map in real time using
special purpose H/W
http//www.cs.cmu.edu/afs/cs/project/stereo-machin
e/www/StereoMachine.html
17Related works at CMU (3/3)
- Virtualized Reality
- create virtual models of real-world events (e.g.
sports)
http//www.cs.cmu.edu/virtualized-reality/
18Hybrid approach for architectural scene
- Modeling and Rendering Architecture from
Photographs A Hybrid Geometry- and Image-based
Approach, Proc. SIGGRAPH 96 - For architectural scene
- Hierarchy of Block primitives
- parameter reduction in first phase
- affordable level in amount of user input
- Model-based stereo
- refine rough model to recover the details
SIGGRAPH 96 Conference Proceedings
19Hybrid approach???, SNU
- Road and surrouding environments
- Simplified case of Debevec96
- Face feature instead of edge
?????????,97??
20Image-based model
- Existing works
- Hirose95
- View mosaicing
- Issues
- how to acquire / store the 2D images ?
- How to generate seamless image sequence
- morphing, stitching
- tightly related to image-based rendering
21Hirose 95 (1/4)
- Purpose
- generation of virtual words by processing 2D real
images taken by video cameras - Basic concept
- image recording
- position recording
- image generation for users viewpoint
Presence, Vol 5, No 1, http//ghidorah.t.u-tokyo.a
c.jp/Projects/IBR/
22Hirose 95 (2/4)
Image Recording H/W
23Hirose 95 (3/4)
- Image synthesis system
- Search for nearby images in the database
- Basic operations shift, scale, rotation of
recorded image - Combination of basic operations
- Enhanced system
- Use multi-images interpolation
- Reduce the the feeling of abrupt switch from one
image to another
24Hirose 95 (4/4)
- Advantages of the method
- easy way to generate virtual worlds
- very realistic appearance
- Drawbacks
- No possibility of users interaction
- Archiving volume very large
- Image processing problems (speed, distortion,)
- Future works
- Use of new technology for archiving virtual
worlds - Generation of wide virtual worlds (world
database) - evolved into CABIN?
25View mosaicing
- process of registering several images to obtain a
single coherent image - Suitable for looking around style VR
http//falcon.postech.ac.kr/people/narziss/image_m
osaic/mosaic.html http//www.cs.cornell.edu/Info/P
eople/kleong/mosaic.html
26Szeliski96 (1/2)
- Purpose
- Set of techniques for building image mosaics
- Virtual reality applications
- Planar image mosaics
- Different pictures are used to generate one wide
planar image - Panoramic image mosaics
- Set of images taken from one viewpoint with a
rotation of the camera - Panoramic effect -gt illusion of real view and
scene - Used for outdoor scenic view, building interior
in virtual reality applications
IEEE CGA 1996
27Szeliski96 (2/2)
- Projective depth recovery
- Necessary for illusion of 3D
- Conclusions
- These techniques can be used as vision based
generation of virtual worlds - Photorealistic appearance and try to restore 3D
effect - But now only static applications
- May be used as a part of more complete
vision-based system
28Summary
- Getting realistic model of real world object /
scene without CAD - indoor, human-scale object 3D scanning
- outdoor scene vision-based approach
- Based on computer vision techniques
- human input to some degree might be very helpful
- Hybrid approach
- towards moving objects / realtime modeling