Course Projects

1
Course Projects

• This presentation will probably involve audience
  discussion, which will create action items. Use
  PowerPoint to keep track of these action items
  during your presentation
• In Slide Show, click on the right mouse button
• Select Meeting Minder
• Select the Action Items tab
• Type in action items as they come up
• Click OK to dismiss this box
• This will automatically create an Action Item
  slide at the end of your presentation with your
  points entered.

• Topics and Plans
• Martin Jagersand

2
Today

• Some tips on carrying out your projects
• Literature search and readings
• Quick prototyping (e.g. Matlab) , then final
  implememntation (e.g. c, c), or combining
  Matlab with c, c mex files.
• Balance between reading and doing
• Labs and resources available
• Win free trip?
• Short presentation and discussion of your topic
  and plans

3
Preliminary project topics

• Individual and group aspect
• Every person has some individual focus and all
  individual pieces combine to a whole.
• Main topics Vision for 3D modeling and robotics
• Real time tracking
• Integrating tracking with 3D modeling
• Integration of a-priori knowledge in 3D modeling
• Predictive display and visualization
• Visual servoing for robots (manipulator or
  mobile)
• Visual specification and planning of robot tasks

4
Resources

• SW I/we will install and try to help support
• Real time video input (under linux, video
  pipeline done)
• Basic tracking, XVision
• Geometry, Hand-eye, Robotics code
• HW Access to machines in Robotics/Vision lab
• Cameras Linux IEEE 1394 in both grad labs and
  course labs csc235.
• Web cams, 200Hz high speed cams, 1600x1200 Hi-Res
  cam available
• VZ motion tracker (3000Hz special device)
• Could also use digital still camera, camcorder.
• Vision for motion control Robot arms, hands,
  mobile robot
• WAM, Barrett hand, Segway (one dedicated more
  coming), old Pumas
• Vision and haptics 3 Phantom omnis.
• Visualization Ok in lab (HW acc graphics), In
  research lab (new ATI and nvidia, SGI HW,
  projectors and CAVE)
• Anything else? Some resources for buying
  available

5
Martins tips

• Plan incremental progress and checkpoints.
• Makes it easier to identify promising directions
  as well as difficulties and redefine plans as
  needed.
• Find balance between reading and doing
• It is difficult to fully grasp methods by only
  reading
• Some experiments are incomplete, results wrong
• Practical trying out can add a lot of insight.
• Learn how to quickly prototype in e.g. matlab

6
Literature search

• Goal Find the 10-15 most relevant and recent
  papers in a subarea.
• Method
• Seed with a few relevant papers.
• Do internet search. e.g. research index or
  google scholar
• Do citation search backwards and forwards.
• Find common buzz words. Do title and abstract
  text search.
• Check most recent proceedings manually. (They
  wont be indexed yet)

7
Literature search 2

• Expect to
• Read the titles of hundreds of papers (and web
  pages)
• Read the abstract of 20-40 of papers
• Skim through dozens of papers
• In order to find the 10-15 or so relevant papers.
  Read these in detail to understand the topic.
• Of these select a handful of the most closely
  related to benchmark your project to.

8
Report

• Review
• Summarize the main contributions and comparing
  the results in the papers.
• Your contribution and experiments.
• Methods
• Results
• Discussion
• Where does it fit into the bigger picture
• Future work

9
Schedule

• Now good time to think about and refine project
  plans
• Late Oct
• Written project proposal.
• Include reference list and a start at literature
  review, ie. Read some papers and write a few
  pages summary
• Throughout course in class
• Keep up to date on your project progress.
• In class presentation of project readings and
  analysis
• End of semester Project reports.

10
Preliminary project topics

• Objective Vision for 3D modeling and robotics
• Individual parts
• Real time tracking
• Integrating tracking with 3D modeling
• Integration of a-priori knowledge in 3D modeling
• Predictive display and visualization
• Mapping and navigation for mobile robots
• Visual servoing for manipulators
• Visual specification and planning of robot tasks
• Singularity and obstacle avoidance
• Subgroups Vision, Robotics

11
GPU accelerated visual tracking

• Tracking readings Color, Feature, SSD, SIFT.
• Investigate what maps naturally to GPU, CPU
• Make incremental plan, e.g.
• Video pipeline Cam-gtVideo RAM or CPU RAM?
• Basic image processing on GPU Lin alg, conv,
  filt, im deriv
• Implement and test various tracking Color, SSD,
  SIFT
• Integrate with other system parts
• Design and carry out experiments Test sequences
  robustness, accuracy etc.

12
Tracking and 3D modeling

• Make tracking more robust by restricting 2D image
  tracker movement to those consistent with a 3D
  interpretation
• Rigid constraints
• Loose constraints
• Language for partial constraints (Collab with
  visual spec)
• Convergence tradeoff
• Restricted tracker may not reach elliptical point
• Unrestricted tracker may track wrong points
• Experiments What are good test sequences? How
  accurate is tracking? Captured 3D? How robust?

13
Use of a-priori knowledge in modeling

• Can make 3D capture easier
• Tells if recovered scene is probable
• Orth, plan etc and gives Euclidean structure w/o
  cumbersome self calibration
• Types of a-priori knowledge
• Generic orthogonality, parallelism, planes.
• Specific architectural housesfeatures(doors
  windows) Indoors Furniture on floor, lamps on
  wall, scales room, furn, items
• How to mathematically incorporate
• Hard constraints
• Probabilistic
• How to practically add
• Image editor collaborate with Visual spec project
• Experiments
• Scenes from photos,
• Indoor scenes from video

14
Predictive DisplayVisual User Interface

• Systems oriented project
• How to modify 3D capture to incrementally detect
  changes in 3D remote scene, send and incorporate
  in model
• How to display 3D model in HMD
• Minimize latencies
• How to track and interpret human motions. Control
  robot motion based on these.
• Could also be set up in CAVE

15
Visual servoing of manipulators

• Manipulators Arms hands, high DOF devices
• Investigate properties of measurement
• E-functions, properties of robot.
• How to estimate visual-motor
• kinematics? Dynamics?
• Local, global?
• Design controllers
• Combine joint and visual feedback
• Integrate with tracking and visual space
  specifications
• Experiements.

16
Visual specification

• What are natural visual task primitives?
• What tasks do the solve? Completeness under
  particular geometry?
• How smooth E-functions do they give?
  (Collaboration with visual servoing).
• How to have human enter
• Pointing 2D image editor
• and gesturing in 3D
• Experiments and test in vision-based manip

17
Singularity and Obstacle avoidance

• Investigate what aspects of calibrated Euclidean
  approach carry over
• How to find and characterize difficult regions
• Find Computer vision, contact, other sensing
• Region representation Points, Regions, Potential
  function
• How to combine with visual servoing controller
• Primary or secondary controller objective
• Effects on convergence
• Experiments with synthetic data, real sequences

18
Next steps

• Firm up project ideas, specification
• Identify readings
• Discuss plans
• Write project proposal
• Investigate hardware needs and plan use
• Plan interaction with other people/groups
• Iterate Devise method, implement, test
• Final integration with other parts
• Write final report