Instructor: Zhigang Zhu

1
Introduction
CSc 83300 Spring 2006 Reading in 3D Computer
Vision and Video Computing

• Instructor Zhigang Zhu
• zzhu_at_ccny.cuny.edu

Lecture 1 Introduction
http//www-cs.engr.ccny.cuny.edu/zhu/GC-Spring200
6/CSc83300-Video-Computing-2006.html
2
Research at CcvcL

• Research Projects at
• The City College Visual Computing Lab
• 3D video mosaicing, representation and rendering
• Multimodal human signature detection
• Robotized Sensor networks
• 3D cargo and vehicle inspection
• The NSF PI Meeting Talk
• The UTRC Hi-Tech Meeting Talk

http//www-cs.engr.ccny.cuny.edu/zhu/research.htm
l
3
Purpose-Driven Reading

• A slightly new approach purpose-driven reading
• Goals come first
• Readings come second
• Projects come last
• Some more details
• Get to know what you want to do
• Figure out potential projects and the goals
• Find related papers to read and present
• Try out something new and/or interesting
• Write out something
• Ideally publications for a conference or journal

4
Homework 1

• Submit your resume with
• Education (where/what/how good)
• Skills (what you are good at GUI, algorithm,
  hardware)
• Work Experiences ( RD)
• Publications (if any)
• Projects related to imaging, vision and graphics
  (IVG)
• Your advisors (if any)
• Courses taken
• Particularly math, CS, EE related to IVG
• Algorithms, Signal processing, etc

5
Course Organization

• Lectures by me (4 lectures)
• Basics in 3D computer vision
• Camera models, calibration, stereo, motion
• Talks by others
• CCNY lecture series, GC CS Colloquium
• Project Ideas by us
• 3D vision, video and robotics
• Some ongoing work at the CCNY Visual Computing
  Lab
• Your research interests (2nd level up)
• Presentations by you ( 3 times each, 2 3 each
  class meet)
• Others work (12)
• Your work proposals and work reports (12)
• Office Hours
• Tuesday 430 600 pm

6
Course Web Page
http//www-cs.engr.ccny.cuny.edu/zhu/GC-Spring200
6/CSc83300-Video-Computing-2006.html

• Lectures available in Powerpoint format
• Reading schedule will be posted on the web
• All assignments will be distributed over the web
• Additional materials and pointers to other web
  sites

March 6-10 A talk by Prof. Harvey Rhody _at_ RIT
Monday or Tuesday ?
7
Book

• Textbook
• Introductory Techniques for 3-D Computer Vision
  Trucco and Verri, 1998
• Additional readings when necessary
• Computer Vision A Modern Approach Forsyth and
  Ponce, 2003
• Three-Dimensional Computer Vision A Geometric
  Viewpoint O. Faugeras, 1998
• On-Line References and Reading Materials

8
C and Matlab

• C
• For some simple computation, you may use C
• Matlab
• An interactive environment for numerical
  computation
• Available on Computer Labs machines (both Unix
  and Windows)
• Matlab primer available on line (web page)
• Pointers to on-line manuals also available
• Good rapid prototyping environment
• You should use C and/or Matlab for your
  homework assignments and project(s) Java will
  also be fine

9
Grading

• Homework (about 23) 20
• Course Work 40
• Reading Related work by others
• Experiments Results by yourselves
• Written Report Papers or TRs
• Presentations (23 per student) 40

10
3D Computer Vision

• What makes (3D) Computer Vision interesting ?
• Image Modeling/Analysis/Interpretation
• Interpretation is an Artificial Intelligence
  Problem
• Sources of Knowledge in Vision
• Levels of Abstraction
• Interpretation often goes from 2D images to 3D
  structures
• since we live in a 3D world
• Image Rendering/Synthesis/Composition
• Image Rendering is a Computer Graphics problem
• Rendering is from 3D model to 2D images

2D images
CV
CG
3D world
11
IP vs CV

• Image processing (mainly in 2D)
• Image to Image transformations
• Image to Description transformations
• Image Analysis - extracting quantitative
  information from images
• Size of a tumor
• distance between objects
• facial expression
• Image restoration. Try to undo damage
• needs a model of how the damage was made
• Image enhancement. Try to improve the quality of
  an image
• Image compression. How to convey the most amount
  of information with the least amount of data

12
Video Computing

• Some Examples on Video Computing
• Video compression
• Video surveillance
• Video manipulation (video texture, video
  composition)
• Video mosaicing
• Video segmentation
• 3D video

13
Approaches

• Three interesting approaches
• Computational Vision Image Structure
• David Marr (MIT)
• Knowledge-Based Vision Image Structure
• Active Vision
• Applied Vision Images Function(Control)
• many others
• Different methodological assumptions
• Different methods
• Different results
• Where is Video Computing?
• an example.... draw your own conclusions!

general
specific
14
Related Fields

• Image Processing image to image
• Computer Vision Image to model
• Computer Graphics model to image
• Pattern Recognition image to class
• image data mining/ video mining
• Artificial Intelligence machine smarts
• Machine perception
• Photogrammetry camera geometry, 3D
  reconstruction
• Medical Imaging CAT, MRI, 3D reconstruction (2nd
  meaning)
• Video Coding encoding/decoding, compression,
  transmission
• Physics Mathematics basics
• Neuroscience wetware to concept
• Computer Science programming tools and skills?

All three are interrelated!
AI
Applications
basics
15
Applications

• Visual Inspection ()
• Robotics ()
• Intelligent Image Tools
• Image Compression (MPEG 1/2/4/7)
• Document Analysis (OCR)
• Image Libraries (DL)
• Virtual Environment Construction ()
• Environment ()
• Media and Entertainment
• Medicine
• Astronomy
• Law Enforcement ()
• surveillance, security
• Traffic and Transportation ()
• Tele-Conferencing and e-Learning ()

16
Job Markets

• Homeland Security
• Port security cargo inspection, human ID,
  biometrics
• Facility security Embassy, Power plant, bank
• Surveillance military or civil
• Media Production
• Cartoon / movie/ TVs/ photography
• Multimedia communication, video conferencing
• Research in image, vision, graphics, virtual
  reality
• 2D image processing
• 3D modeling, virtual walk-through
• Consumer/ Medical Industries
• Video cameras, Camcorders, Video phone
• Medical imaging 2D -gt 3D

17
Research at CcvcL

• Research Projects at
• The City College Visual Computing Lab
• 3D video mosaicing, representation and rendering
• Multimodal human signature detection
• Robotized Sensor networks
• 3D cargo and vehicle inspection
• The NSF PI Meeting Talk
• The UTRC Hi-Tech Meeting Talk

http//www-cs.engr.ccny.cuny.edu/zhu/research.htm
l
18
Research at CcvcL

• Research Projects at
• The City College Visual Computing Lab
• 3D video mosaicing, representation and rendering
• Multimodal human signature detection
• Robotized Sensor networks
• 3D cargo and vehicle inspection
• The NSF PI Meeting Talk
• The UTRC Hi-Tech Meeting Talk

http//www-cs.engr.ccny.cuny.edu/zhu/research.htm
l