Title: User Interface Design Tools for the Future Multimodal UI Research in the HCC
1User Interface Design Toolsfor the Future
Multimodal UI Research in the HCC
- James A. Landay
- Jason Hong, Scott Klemmer,
- Jimmy Lin, Mark Newman, Anoop Sinha
- HCC Retreat
- July 5, 2000
2Several Projects on Design UIs of the Future
- Designers Outpost
- SUEDE
- Multimodal Design Assistant
3The Future Information-centric UIs
4Designers Outpost Tangible Tools for
Information Design
5Designers Outpost Virtual Feedback on Physical
Layouts
6Designers Outpost Virtual Feedback on Physical
Layouts
7Designers Outpost Virtual Linking of Related
Information
8Designers Outpost Export to Next Stage Tools
(DENIM)
9Designers Outpost Infrastructure for Next
Prototype
- Two video/still cameras
- still camera above desk/wall
- captures ink / IDs
- video camera below desk/wall
- occlusion free
- captures structure
- SmartBoard
- 1280 x 1024 resolution
- looking for a projector donation
- direct pen input
- Cross iPen tablet
- high-res ink capture
10One Future Our Access to Information will be via
Speech
Read my important email
11SUEDE Low-fidelity Prototyping for Speech-based
UIs
12A Better Future Our Access to Info will be via
Multimodal UIs
- Take advantage of the best mode of input/output
for the task/context at hand - How do we design so that these new devices will
not seem impoverished relative to desktop
computers?
13A Better Future Our Access to Info will be via
Multimodal UIs
- How do we combine speech, gesture, etc. into a UI
design? - rapid production of rough cuts
- informal (sketching / Wizard of Oz)
- iterative design (user testing/fast mods)
- generate initial code
- UIs for multiple devices
- designer adds detail / improve UI
- Study
- uses of these novel modes alone
- e.g., SUEDE
- construction of multimodal apps
- e.g., a multimedia notebook
- Early stages so far
- based on inferring models
14More information on these design tools at
http//guir.berkeley.edu