User Interfaces for 3D

1
User Interfacesfor 3D
K. Gatland and D. Jefferis The World of the
Future Robots. 1979, Usborne Hayes.
2
Outline

• Why bother with 3D ?
• Some Basic Tasks
• Input devices
• Output devices
• Properties and pitfalls of 3D
• Scientific Visualization
• Information Visualization in 3D
• Interaction techniques and interface schemes
• Demos

3
Why bother with 3D ?

• Displaying objects or environments that are
  naturally 3D (architectural plans, industrial
  designs, ). Examples
• using VR to shop for a new kitchen
• designing an automobile
• visualizing CT scans or MRI data

http//www.tjhsst.edu/TechLabs/CAD/cad98/rciszek/a
home.htm
4
Why bother with 3D ?

• Scientific Visualization
• Where the data has a natural 3D spatial structure

Weyl scalar fields from orbiting binary neutron
stars. http//jean-luc.ncsa.uiuc.edu/Movies/
Two colliding black holes. http//jean-luc.ncsa.ui
uc.edu/Exhibits/exhibits.html
sphere eversion

• Sometimes, 3D isnt enough !

5
Why bother with 3D ?

• To harness natural human abilities ?
• Pre-conscious processing by the human visual
  system
• Spatial memory

6
Example
The moon is the largest natural satellite of the
earth, and is composed of 30 cheddar, 40
mozzarella, 25 star dust, and 5 Elmers glue.
Yesterday, at 1215 pm, the cow owned by Mrs.
Farmwell jumped over the moon.
The cow jumped over the moon.
http//www.angelfire.com/pa2/klb01/spheregallery2.
html

• To not use 3D seems like a waste of bandwidth !
• But how do we represent abstract data in 3D ?

7
Why bother with 3D ?

• We can pack more information, and more complex
  relationships, into 3D
• Information visualization
• Where abstract data is embedded into 3D
• Example below Anemone (Benjamin Fry)

Using the process of organic information design
to visualize the changing structure of a web
site, juxtaposed with usage information. http//ac
g.media.mit.edu/people/fry/anemone/
8
Why not bother with 3D ?

• Added complexity
• Many more degrees of freedom to handle
• Standard input/output devices not designed for
  3D
• Interfaces may be difficult to learn

The benefits shouldoutweigh the costs !
9
Some Basic Tasks

• Specify a point (3 DOF)
• Specify an orientation (3 DOF)
• Specify a path
• Selecting an object or region in space
• Create an object or surface
• Navigation (6 DOF)
• Finding out where something is

10
Navigation Camera Control
11
Zoom vs Dolly (Translation)
From slides by Chris North
12
Metaphors for Camera Navigation

• World-in-hand
• Eyeball-in-hand (a.k.a. egocentric)
• Walking
• Flying

13
Some Input Devices

• Plain old 2D mouse
• Use picking ray to select objects
• Translate a 3D cursor, 1 or 2 dimensions at a
  time
• 21 D pointers
• E.g. mouse thumbwheel lever that can be
  pushed/pulled
• 3D pointers
• E.g. trackers, floating mice,
• Higher DOF devices
• Data glove, shape tape,

14
Spaceball Logitech Magellan
http//www.alsos.com/Products/Devices/SpaceBall.ht
ml

• 6 DOF, but
• Rate control rather than position control

15
Rockin Mouse
R. Balakrishnan, T. Baudel, G. Kurtenbach, G.
Fitzmaurice (1997). The Rockin Mouse Integral
3D manipulation on a plane. CHI97.
16
Shape Tape
17
Dolls Head Props-based interface for 3D
Cutting Plane

• Ken Hinckley

From slides by Chris North
18
Output devices

• Flat screens
• Stereoscopic displays
• Red/green glasses, LCD shutters, head-mounted
  displays
• Autostereoscopic displays
• Re-imaging displays
• Parallax displays (e.g. holographic displays)
• Volumetric displays

For more info on autostereoscopic
displays http//web.media.mit.edu/halazar/autost
ereo/autostereo.html
19
High Fidelity 3D Output

• Stereoscopic
• Convergence
• Accommodation

20
Boom Chameleon(G. Fitzmaurice et al.)

• Navigation is easy to learn
• 3D view can be shared
• Not stereoscopic

G. Fitzmaurice and fakespacesystems.com
21
Virtual Reality
Head-mounted display
High DOF input device

• Immersive, but also cumbersome ?
• Stereoscopic, but no ocular accommodation

22
Volumetric Display
The Perspecta display. www.actuality-systems.com

• Stereoscopic, and ocular accommodation !
• No occlusion, and no view independent shading ?
• Interesting property no perspective projections
  possible

23
Volumetric Display
Elizabeth Downing, www.3dtl.com
24
Properties and Pitfalls of 3D
25
What is this ?

• Whats behind it ? Whats on the other side ?
• Whats behind me ?
• 3D is inherently subjective

26
Where am I ?
27
Where am I ?
28
Where am I ?
29
Cues (visual, and depth)

• Occlusion
• Gives ordinal information
• Transparency
• Perspective
• Relative size, foreshortening, converging lines
• Stereopsis
• Motion parallax
• Contour, shading, specular highlights,
  reflections
• Shadows (e.g. drop shadows)
• Ground plane grid, coloured sky
• Landmarks, compass arrows

30
Example use of cues
Vida Dujmovi\'c, Pat Morin, David R.
Wood Path-Width and Three-Dimensional
Straight-Line Grid Drawings of Graphs GD 2002
31
Example use of cues card readers
32
Example use of cuesShading with surface normals
(images by Michael McGuffin)
33
Example use of cues
Product Logo http//www.cri-mw.co.jp/products/prod
uct_adx_e.htm
34
Example use of cues
Plumb Designs Visual Thesaurus http//www.visualt
hesaurus.com/
35
Scientific Visualization
36
From an ad for GRAFTOOL software, made by 3-D
Visions, appearing in the January 1992 issue of
Scientific American.
37
From an ad for GRAFTOOL software, made by 3-D
Visions, appearing in the January 1992 issue of
Scientific American.
38
From an ad for GRAFTOOL software, made by 3-D
Visions, appearing in the January 1992 issue of
Scientific American.
39
Cutting Planes Isosurfaces
http//www.slicerdicer.com/2astrolg.html
40
Geological Data
http//www.slicerdicer.com/6geolg.html
41
Volumetric Data
Janet Haswell Visualizing Electromagnetic
Data in G. Grinstein and H. Levkowitz
(Eds.) Perceptual Issues in Visualization pp.
109--125 1995 Springer
42
Textures for enhacing cues
L. M. de la Cruz, I. Garcia, V. Godoy, E. Ramos,
Case study parallel lagrangian visualization
applied to natural convective flows, ACM PVG 2001
43
Sphere Eversion
http//www.geom.umn.edu/munzner/ieee94/ieee/node2
5.html
44
Sphere Eversion
http//www.geom.umn.edu/graphics/pix/Video_Product
ions/Outside_In/blue-red-alpha.html
45
Hyperbolic Space
http//www.geom.umn.edu/munzner/ieee94/ieee/node2
5.html
46
Hierarchical Flow Diagram
PhD thesis of H. Loeffelmann http//www.vrvis.at/v
is/
47
Medical Vis
C. Balazs et al. http//www.vrvis.at/vis/research/
npvr/
48
Medical Vis
C. Balazs et al. http//www.vrvis.at/vis/research/
npvr/
49
Information Visualization in 3D
If Keanu Reeves does it, it must be cool, no ?
50
3D message board

• Looks cool, but
• User spends most of their time navigating ?
• How to fix ?

51
3D message board
52
A static diagram of theProthos Application
Framework
What problems would arise if this diagram were
dynamic/interactive ?
http//www.cs.ualberta.ca/hoover/cmput660/reading
s/SoftwareArch/section/prothos.htm http//www.cs.u
alberta.ca/hoover/cmput660/readings/SoftwareArch/
image/framework.gif
53
Collapsible Cylindrical Trees(Dachselt and
Ebert, IEEE InfoVis 2001)
54
ThreeDFM
http//www.icewalkers.com/Linux/Software/516470/Th
reeDFM.html
55
File System Navigator (FSN) by SGI
From slides by Chris North
http//www.sgi.com/fun/freeware/3d_navigator.html
56
File System Visualizer (FSV) http//fsv.sourceforg
e.net/
57
Information Pyramids
Andrews and Wolte and Pichler IEEE Visualization
1997, Late Breaking Hot Topics
58
Cone Trees (G. Robertson et al.)
G. Robertson, J. Mackinlay, and S. Card. Cone
trees Animated 3d visualizations of hierarchical
information. CHI '91. Image courtesy of G.
Fitzmaurice.
59
Kodiseins file browser
http//www.mulle-kybernetik.com/software/kodisein/
manual/filebrowser.html
60
Innolab 3D File Manager(Ferris wheel-like
arrangement)
Adam Miezianko, Kristopher Rambish, Karen Fung,
Zavnura Pingkan http//www.sciencemag.org/cgi/cont
ent/full/301/5639/1476 http//www.sciencemag.org/f
eature/data/vis2003/illus_first.html http//www.sc
iencemag.org/feature/data/vis2003/images/fileman_l
arge.jpg
61
Hierarchical Net
Hierarchy based 3D Visualization of Large
Software Structures Michael Balzer and Oliver
Deussen poster at Vis 2004
62
PolyPlane (Hong and Murtagh,IEEE InfoVis 2003
poster)
63
Information Cube (Rekimoto Green)
Adapted from slides by Chris North
64
Cube a 3D visual programming language (Marc
Najork)
http//www.research.compaq.com/SRC/personal/najork
/cube.html
65
SAM An Animated 3D Programming Language
(Christian Geiger et al.)
66
3D-PP Visual Programming System (Oshiba and
Tanaka)
67
Force-directed layout of graphs in 3D

• Pseudo-physical simulation of forces leads to
  automatic layout
• Nodes are mutually repelled by an electrical
  force
• Edges are springs

http//www.dgp.toronto.edu/mjmcguff/research/grap
h3D/
68
Valence (Ben Fry)

• Nodes are words in a text
• Edges connect words that appear consecutively in
  the text (thus, the text is a path through the
  graph)
• Words that appear many times are pushed outward
• Pairs of words that appear many times have
  shorter edges connecting them
• Graph is constructed dynamically as text is read
  in

Image from Ben Frys masters thesis
69
Valence (Benjamin Fry)
http//acg.media.mit.edu/people/fry/valence/
70
Gradus (Matt Grenby)

• Scatter plot of dictionary words
• x, y, z axes correspond to time, familiarity,
  alphabetic ordering
• Overall form reveals something about underlying
  data

http//acg.media.mit.edu/people/grenby/gradus/
71
Examples ofInteraction Techniques,Interface
Schemes
72
(Very) Indirect ManipulationThrough Traditional
Widgets

• Creates a divided attention problem
• User must switch between mouse and keyboard

Thomas Strothotte. Computational Visualization
Graphics, Abstraction, and Interactivity, p. 318
73
Indirect ManipulationThrough 3D Widgets
http//www.refractions.net/terrainserver/screensho
ts/snapshot_vrml.jpg http//www.mmu.ac.uk/art-des/
arc/people/sforestiero/diss/FIG05.JPG
http//hackberry.chem.trinity.edu/IJC/Text/adam.gi
f
Cosmo Player VRML browser, by SGI
74
(Almost) Direct Manipulation Through 3D Widgets
Manipulators in Maya for translating, rotating,
and scaling a cube

• All operations performed with a regular mouse

75
Traditional WidgetsEmbedded in the 3D scene
http//www.ssec.wisc.edu/billh/sc95.gif
76
Smarter Cameras

• Camera always stays upright
• Camera detects surrounding geometry
• Glances (J. Pierce et al.)
• StyleCam (N. Burtnyk et al.)

J. S. Pierce, M. Conway, M. Van Dantzich, G.
Robertson. Toolspaces and Glances. 1999 Symposium
on Interactive 3D Graphics (I3D '99).
77
Voodoo Dolls (Jeff Pierce)

• Crushing head selection
• Other possible schemes casting fishing line

J. S. Pierce, B. Stearns, R. Pausch. Two Handed
Manipulation of Voodoo Dolls in Virtual
Environments. 1999 Symposium on Interactive 3D
Graphics (I3D '99).
78
Haptic feedback The Phantom
R. Jagnow and J. Dorsey. Virtual sculpting with
haptic displacement maps. Proceedings of Graphics
Interface, 2002.
http//www.sensable.com
79
3D desktop (3dna.net)

• Improved harnessing of spatial memory ?

80
Win3D, by clockwise3d.com
81
3D OS X
http//www.acm.uiuc.edu/macwarriors/projects/3dosx
/screenshots.html
82
Project Looking Glass(Sun Microsystems)
http//wwws.sun.com/software/looking_glass/
83
A 3D Window Manager(www.3dwm.org)
84
www.3dwm.org
85
www.3dwm.org
86
www.3dwm.org
87
Ferris Wheel layout
http//www.lego.com/eng/create/designer/default.as
p?xxid4100
88
Metisse (Olivier Chapuis and Nicolas Roussel)
http//insitu.lri.fr/chapuis/metisse/screenshots/
89
Data Mountain (G. Robertson et al.)
"Our pre-attentive ability to recognize spatial
relationships ... makes it possible to place
pages at a distance (thereby using less screen
space) and understand their spatial relationships
without thinking about it."
G. Robertson et al. Data Mountain Using spatial
memory for document management. UIST 98.
90
Task Gallery (G. Robertson et al.)
G. Robertson et al. The Task Gallery A 3D Window
Manager. CHI 2000.
91
2D versus 3D (Cockburn et al.)
A. Cockburn and B. McKenzie. Evaluating the
effectivenessof spatial memory in 2D and
3D physical and virtual environments. CHI 2002.
92
Proprioception and VR
Reference for above pictures Mine et al.,
"Moving objects in space exploiting
proprioception in virtual-environment
interaction", SIGGRAPH '97. For related work,
see also Pierce, Conway, van Dantzich, Robertson
(1999), Toolspaces and Glances, I3D99
93
CSCW in 3D

• Goal awareness of location view of other users
  in a common 3D space
• Avatars not enough, because
• When avatar is distant or facing away, cant see
  what user is looking at
• When avatar is out of view, cant tell anything
  about the user

94
Groupspace

• 7 techniques to increase awareness of location,
  perspective, and proximity of others
• Nose ray
• View cone
• Head light
• Awareness slider
• Rotating participant
• WYSIWIS participant
• Grand tour

95
Groupspace
J. Dyck and C. Gutwin. Groupspace a 3D workspace
supporting user awareness. CHI 2002.
96
Lessons ?

• For things that are naturally 3D (e.g. cars,
  scientific visualization)
• 3D is necessary, but still not simple
• Good design will make it better
• For information visualization,
• 3D can be worse than 2D
• Good design is essential
• Good design includes
• Use of constraints where possible, to simplify
  navigation and manipulation
• Use of cues, to ease interpretation of visual
  information

97
Demos
98
Extra Material
99
http//dform1shiftfunc.net
100
http//dform1shiftfunc.net
101
Doom as a tool for system administration(Dennis
Chao) http//www.cs.unm.edu/dlchao/flake/doom/
102
Doom as a tool for system administration(Dennis
Chao) http//www.cs.unm.edu/dlchao/flake/doom/
103
Doom as a tool for system administration

• Advantages
• The machine load is immediately apparent to the
  player, who can see how crowded a room is.
• A new sysadmin can be given less power by
  providing her with a smaller weapon.
• Drastic action takes work. In a command line
  interface, all actions take approximately the
  same amount of effort. One can ls just as easily
  as rm -rf In a cyberspace environment
  performing large actions takes time and effort.
• Important processes can be instantiated as more
  powerful monsters. They can then defend
  themselves against inexperienced sysadmins.

104
Doom as a tool for system administration

• Disadvantages
• Mapping processes to appropriate monsters is
  difficult. Should large processes be mapped to
  large monsters? Should the monster type reflect
  the CPU as well as memory usage? Should processes
  and their children look alike?
• It is difficult to tell if your employees are
  doing real work or just goofing off when tools
  and games have the same GUI.