User Interface Design Tools for the Future Multimodal UI Research in the HCC

1
User Interface Design Toolsfor the Future
Multimodal UI Research in the HCC

• James A. Landay
• Jason Hong, Scott Klemmer,
• Jimmy Lin, Mark Newman, Anoop Sinha
• HCC Retreat
• July 5, 2000

2
Several Projects on Design UIs of the Future

• Designers Outpost
• SUEDE
• Multimodal Design Assistant

3
The Future Information-centric UIs
4
Designers Outpost Tangible Tools for
Information Design
5
Designers Outpost Virtual Feedback on Physical
Layouts
6
Designers Outpost Virtual Feedback on Physical
Layouts
7
Designers Outpost Virtual Linking of Related
Information
8
Designers Outpost Export to Next Stage Tools
(DENIM)
9
Designers Outpost Infrastructure for Next
Prototype

• Two video/still cameras
• still camera above desk/wall
• captures ink / IDs
• video camera below desk/wall
• occlusion free
• captures structure
• SmartBoard
• 1280 x 1024 resolution
• looking for a projector donation
• direct pen input
• Cross iPen tablet
• high-res ink capture

10
One Future Our Access to Information will be via
Speech
Read my important email
11
SUEDE Low-fidelity Prototyping for Speech-based
UIs
12
A Better Future Our Access to Info will be via
Multimodal UIs

• Take advantage of the best mode of input/output
  for the task/context at hand
• How do we design so that these new devices will
  not seem impoverished relative to desktop
  computers?

13
A Better Future Our Access to Info will be via
Multimodal UIs

• How do we combine speech, gesture, etc. into a UI
  design?
• rapid production of rough cuts
• informal (sketching / Wizard of Oz)
• iterative design (user testing/fast mods)
• generate initial code
• UIs for multiple devices
• designer adds detail / improve UI
• Study
• uses of these novel modes alone
• e.g., SUEDE
• construction of multimodal apps
• e.g., a multimedia notebook
• Early stages so far
• based on inferring models

14
More information on these design tools at
http//guir.berkeley.edu