Visual Design for 3D User Interfaces

1
Visual Design for 3D User Interfaces
_at_ somewhere

• Daniel C. Robbins dcr_at_microsoft.com

2
The Ideal?

• Why is it so hard to use computers?

3
Structure of This Talk

• Who am I?
• Problems with current UIs
• What to do about it?
• Examples from our research
• Lessons we learned
• Metaphors
• Future directions

4
Who Am I?

• Fine-arts major
• Furniture, sculpture, jewelry, etc.
• UI Designer for 13 years
• 7 years at MS, 6 in MSR
• Combining academic and product focus

5
Contact Info

• dcr_at_microsoft.com
• http//research.microsoft.com/dcr

6
What Is a User-interface?

• Means by which a person uses a computer
• The sensory connection between a user and the
  functionality of a computer-based application
• Human-computer interaction
• Human-computer-human interaction

7
UI Puzzlers

• Has the Web killed innovative UI?
• Video what is truth?

8
Problems w/ Current UIs

• Too hard to use
• Too many options
• Unreliable
• Inconsistent

Too Vague
9
Specific Problems

• Multiple representations for an object
• Objects look completely different in different
  places
• Low-level interaction
• Manipulation centric vs. task-centric
• Should be
• Just do what I wanted you to do!

10
Specific Problems (2)

• Discretely shifting spatial configurations
• The rug is pulled out from under the user
• Multiple simultaneous views
• Divided attention
• Homogeneous representations
• Everything looks the same
• No context
• How do changes to one thing affect another?

11
So What? Why Fix It?

• Increase customer base
• Different ages
• Different abilities
• New ways of using computers
• On-the-go
• One-function-at-a-time
• Step-up use
• Over the Net

12
How to Fix It?
13
Mantra Cause and Effect
What can the user do?
What is the user doing?
What has the user done?
14
Specifics

• Take advantage of human perception
• 3D, animation, constancy, richer cues
• Take advantage of human experience
• Appropriate use of metaphors
• In-situ assisted learning / hard-fun
• Video games, apprenticeship, sports
• Not the office-assistant!

15
What is a 3D UI?
NOT
16
Examples From Our Research

• Data Mountain, TaskGallery, PhotoMountain

17
The TaskGallery

• Lets change everything

18
Task Gallery Functionality

• UI exploration for new desktop UI
• 1) Windows management
• Comparing multiple windows is hard
• 2) Task management
• Switching between user tasks is hard
• Use rooms concept, but use 3D to engage spatial
  memory
• Push the technology

19
The Task Gallery Video

• Runs on commodity hardware
• Uses unmodified applications
• Runs on Windows 2000 with Application
  Redirection
• Results

20
Good UI Practices

• Show cause-and-effect
• Combine related user actions, params.
• Distinguish UI from data
• Combine UI and data (Direct Manip.)
• Use appropriate channels
• Appropriate metaphor
• Support current functionality

21
Spatial Memory the Bet!

• Evolution perceptual abilities
• Remember (generally) where stuff is (?)
• Engage navigational memory
• Reinforce with landmarks proximity
• Requires spatial stability/constancy

22
Spatial Cues 2

• Kevin Lynch The Image of the City
• Urban Design
• Boundaries, neighborhoods, topography

23
The DataMountain

• Back to basics

24
The Data Mountain

• What is the role of landmarks?
• Is the arrangement of the data itself enough of a
  cue?

25
Cues Neighborhood
26
Boundaries
27
The PhotoMountain

• Adapting the DataMountain to Photographs

28
PhotoMountain Functionality

• Import / Add photos
• Categorize photos
• Auto, manual
• Find photos by
• Time
• People
• Place
• Create slide shows

29
Design Refinements
30
Lessons We Learned

• How do we actually design this stuff?

31
Principles

• Get the Big Picture
• Channels
• Geometry
• Metaphor

32
Focus Context

• Scarcest resource is human attention!
• Hierarchical regions for organization
• Broaden or narrow attention
• Vs switching attention
• Parallel attention
• 3D is a natural way to do this
• Get interactions cheaply

33
High-level Interaction

• Make one click do lots of stuff
• Free-form Navigation vs. High-level
• Doom vs. chauffer

34
Channels Bag of Tricks

• Animation
• Rendering style , Transparency
• Abstraction
• Audio
• Harder in 3D
• Channels already taken (size, color, etc.)

35
Methodology

• How to free up design

36
Basic Geometries

• Carousel
• P-Wall
• Bowl
• Landscape
• 3D Star (chandelier)
• Layering
• Embedding multiple dimensions
• Endocentric vs. Exocentric

37
Carousel
38
Perspective Wall
39
Bowl

• Natural fisheye

40
Landscape
41
3D Star
42
Data Layering (1)
43
Data Layering (2)
44
Other?
45
All at Once
46
Using These Principles

• The next version of the PhotoMountain

47
Axes / Dimensions

• Identify Axes / Dimensions
• Prioritize dimensions (rankings)
• Which is primary?
• Bounded vs. Unbounded
• If bounded, how big?

• Extra Credit -gt
• Does there have to be a Primary Axis?

48
Axes / Dimensions for the PhotoMountain
49
Draw Some Geometries
50
Cleanup an Image
51
Try a First Pass Mapping
Cluster based on?
Current Time
Width of items
Time
52
Add Some Real Data
53
Metaphors

• Is there anything new?

54
Why Use a Metaphor

• Capitalize on previous knowledge
• Make a UI seem friendlier (more approachable
• Provide a set of rules that explain and predict
  the behavior of objects
• Under computer control
• Under user control

55
Inspiration for Metaphors?

• Nature
• The machine
• The spiritual
• Is everything in our world designed?

56
Example Metaphors
57
Problems With Metaphors

• Mapping can be arbitrary
• Scalability
• False impressions
• High expectations
• Question
• Can we use real-world objects as landmarks in
  an abstract world?

58
Future Directions
59
Other Geom 1
60
Future Directions

• 3D UI and the connected world

61
Wall Displays
62
Working Together
63
Working from Home
64
Dynamic Tracking
65
Sharing and Revising
66
Shared Environments
67
The Way-out

• Lets get wacky

68
Public Kiosks 1
69
Public Kiosk 2
70
Reading List

• Principles of Three-Dimensional Design, Luecking
• Principles of Three-Dimensional Design, Wong
• The Image of the City, Lynch
• Design and Form (Bauhaus), Itten
• Metaphors We Live By, Lakoff et al.
• Tufte, Shneiderman, Laurel, Wurman, etc

71
END
72
Window Comparison

• Rearrange
• Peel-back
• Transparency

• Flash Mockup

73
User Studies

• Four iterative usability tests completed
• Numerous usability problems identified and fixed
• Very high satisfaction ratings

74
What Did Users Like?

• Transparency (to deal with occlusion)
• Glancing (spatial awareness)
• Context menus
• Overview of tasks

75
Problems

• Occlusion
• Discoverability
• Can it scale?
• Target audience?

76
Future Work

• Other metaphors
• Other geometric arrangements
• Peripheral notification
• Semantic zooming for thumbnails

77
Multiple Representations
? Back
78
Low-level interaction
79
Discrete Changes
80
Multiple Views
5R x 3C
81
Homogeneous Reps.
82
Context and Relationships
? Back
83
Animation

• No animation 1, 2?
• If its in the way, make it go away
• Shift load from cognitive to perceptual
• Dont force user to figure out what changed
• Animation styles

84
Discrete Switching

• But with animation!

85
Discrete Broaden
But with animation!
Animation
86
Rendering Styles 1

• 2D Wireframe for navigation UI elements

87
Rendering Styles 2

• Icons for Window Manipulation controls
• Different styles

Widgets mockup
88
Abstraction

• Icons vs. Thumbnails
• Depends on
• Degree of attention
• Perceptibility
• Confusability
• Content (high freq. Vs. low freq.)
• Need semantic zooming!
• Tied to metaphor? ?

89
2D vs. 3D Channels Size
90
2D vs. 3D Channels Color
91
Criteria Fixing the Problems

• Spatial memory
• Context human attention
• Visual recognition
• Stability
• Scalability
• Good UI practices

Perception vs. Cognition
92
Output Redirection
93
Input Redirection
94
The Guts

• Commodity hardware
• Unmodified applications
• Windows 2000 with Application Redirection

95
Audio Comparison

• No audio

• With audio

96
Scalability
Many tasks

• A few tasks

97
Spatial Cues Landmarks

• Shadows aid in depth perception

Landmarks aid in spatial memory
98
Visual Recognition (1)

• Objects represented by thumbnails
• User selectable wallpaper

99
Visual Recognition (2)

• Live applications
• (Dynamic updating)

100
Task Gallery Evaluation

• Different interactions
• User Studies
• Problems
• Future work

101
Body Centric

• Left, Right Hands
• Feet
• Look Up
• Glancing
• Lightweight, ephemeral interaction

102
Spatial Stability
103
Transparency Video
104
Animation Video
105
New Task Video
106
Drag Task Video (Direct manip)
107
Rename Task Video
108
Switch Tasks Video
109
Show All Tasks Video
110
Transparency Video
111
Wallpaper Video
112
Freeform Nav Video
113
Hands Video
114
Glancing Video
115
TG Full Video
116
DM Short Video
117
Broaden Video
118
Narrow Video