The Office of the Past

1
The Office of the Past

• Jiwon Kim
• Steve Seitz
• Maneesh Agrawala

2
Where is my W-2?
3
Unify physical andelectronic desktops
Video camera

• Recognize video of paper on physical desktop
• Tracking
• Recognition
• Linking

Desktop
4
Demo Remote desktop
5
System overview
Video camera
Computer
User
Desk
6
System overview
Video of desk
7
System overview
Images from PDF
Video of desk
8
System overview
Images from PDF
Video of desk
Track recognize
9
System overview
Internal representation
Images from PDF
Video of desk
Track recognize
T
T1
10
System overview
Internal representation
Images from PDF
Video of desk
Track recognize
T
T1
Scene Graph
11
System overview
Where is my W-2?
Internal representation
Images from PDF
Video of desk
Track recognize
T
T1
12
System overview
Where is my W-2?
Answer
Internal representation
Images from PDF
Video of desk
Track recognize
Desk
Desk
T
T1
13
Document tracking example
before
after
14
Document tracking example
before
after
15
Document tracking example
SIFT Lowe 04
before
after
16
Document tracking example
before
after
17
Document tracking example
Motion (x,y,?)
before
after
18
Document Recognition

• Match against PDF image database

File2.pdf
File3.pdf
File4.pdf
File5.pdf
File6.pdf
File1.pdf
19
Demo Paper tracking
20
Photo sorting example
21
Photo sorting example
22
Demo Photo sorting
23
Future work

• Enhance realism
• Handle more realistic desktops
• Real-time performance
• More applications
• Support other document tasks
• E.g., attach reminder, cluster documents
• Beyond documents
• Other 3D desktop objects, books/CDs