Interaction in Virtual Environments

1
Interaction in Virtual Environments

• Benjamin Lok

This Lecture contains notes created by Doug
Bowman Virginia Tech Sherman Craig, pp. 332-363
2
3D Interfaces

• We live in a 3D world
• Natural interfaces are better
• Therefore 3D interfaces will be the ultimate
• Whats wrong with the above?
• Natural interfaces arent always better!
• Making the interface simple (thus unnatural)
  often aids performance
• Constrains movement
• Limiting possible actions
• Depends on application and goal of the user
  interface
• Surgery simulation
• Military simulation (general vs. soldier
  training)
• Architecture, education, product design
• Video games

3
3D Interfaces

• What we really want are enhanced interfaces
• Give us powers we dont normally have
• Flying, x-ray vision, teleportation, undo, etc.
• Be careful we dont become overzealous
• Air traffic control 3D display
• Library interfaces using a books on shelves (what
  is it good for? What is it poor for?)
• Hurts performance
• Study results 3D Bar charts dont help
• So what is helped by 3D?

4
Good 3D

• Social interfaces 3D can be very powerful
• MMORPG (EveQuest)
• ActivedWorlds
• The Sims Online
• Experiences
• Art gallary
• 3D Desktops (Macs latest)
• Office metaphors did not take off (BOB, Task
  Gallary)
• 3D Webbrowsing. Sure you can arrange 16 web
  pages spatially, but why?
• Compromises to provide 3D interfaces might be
  undermine usability
• Think RTS games
• Discussion Is the interface holding back 3D?

5
3D Interfaces

• Use occlusion, shadows, perspective carefully
• Improves use of spatial memory (Ark 98)
• Distracting and confusing
• Minimize navigation steps
• Keep text readable (good contrast, 30 degree tilt
  max)
• Simple user movement (why lock to a floor?)
  Descent vs Quake
• Prevent Errors (put in guides to help)
• Simplify object movement (connecting two parts,
  for example, can be abstracted most of the time)
• Organize groups of items into alignments that
  facilitate visual search and recall (allow user
  choice)

6
3D Interface Development

• Developments that show promise
• 3D sound
• Stereo display (Ware and Frank 96)
• Haptic feedback (mouse)
• 3D can help by
• Provide overviews to see big picture
• Rapid teleportation (context shifts)
• Zooming (aid disabled)
• Multiple coordinated views (3dsmax)
• 3D icons can represent abstract or recognizable
  concepts

7
Teleoperation

• Combines
• Direct Manipulation
• Process Control
• Human operators control physical processes in
  complex environments
• Example applications Mars rover control, flying
  airplanes (Predator), manufacturing, medicine
  (surgery)
• Supervisory control (Sheridan 92)
• Different levels of human control (automation)

8
Teleoperation

• Direct Manipulation Issues
• Adequate feedback (data quality, latency
  (transmission and operation delays), incomplete,
  interference)
• Presence
• Point and click or more natural interaction vs.
  typing
• Example project Nanomanipulator (show video)

9
VR Interaction

• Trying to simulate reality or an experience
• Training, Learning, Exploring
• Expensive
• Dangerous
• Logistically Difficult
• Best interaction?
• Flight simulators (they can cost 10 mil, but
  thats still a good deal!)
• Why?
• Why do video game flight sims not cut it? (only
  40!)
• Okay, we have monitors that show 3D worlds, what
  else do we possibly need?

10
Why 3D interaction?

• 3D / VE apps. should be useful
• Immersion
• Natural skills
• Immediacy of visualization
• But, current VE apps are
• Not complex interactively
• Have serious usability problems

11
What makes 3D interaction difficult?

• Spatial input
• Lack of constraints
• Lack of standards
• Lack of tools

• Lack of precision
• Fatigue
• Layout more complex
• Perception

Youve seen this in creating Project 2!
12
Lets look at this example (ISAAC)
13
Universal interaction tasks

• Navigation
• Travel motor component
• Wayfinding cognitive component
• Selection
• Manipulation
• System control

14
Goals of interaction design

• Performance (efficiency, accuracy, productivity)
• Usability (ease of use, ease of learning, user
  comfort)
• Usefulness (users focus on tasks, interaction
  helps users meet system goals)

15
Philosophies of interaction design

• Artistic approach
• Intuition about users, tasks
• Heuristics, metaphors
• Aesthetics
• Adaptation

• Scientific approach
• Formal analysis
• Formal evaluation
• Performance requirements

16
Travel

• Motor component of navigation
• Movement between 2 locations, setting the
  position (and orientation) of the users
  viewpoint
• Most basic and common VE interaction technique
• Used in almost any large-scale VE

17
Types of Travel Tasks

• Exploration
• Travel which has no specific target
• Build knowledge of environment
• Search
• Naive travel to find a target whose position is
  not known
• Primed travel to a target whose position is
  known
• Build layout knowledge
• Move to task location
• Maneuvering
• Travel to position viewpoint for task
• Short, precise movements

18
Steering metaphor

• Continuous specification of direction of motion
• Gaze-directed
• Pointing
• Physical device (steering wheel, flight stick)

19
Target-based metaphor

• Discrete specification of goal
• Point at object
• Choose from list
• Enter coordinates
• E.g. Google Earth

20
Route-planning metaphor

• One time specification of path
• Place markers in world
• Move icon on map
• Manual manipulation of viewpoint
• Camera in hand
• Fixed object manipulation

21
Natural travel metaphors

• Walking techniques
• Treadmills
• Bicycles
• Other physical motion
• VMC / Magic carpet
• Disneys river raft ride
• Simulation of flying

22
Technique classification
Target specification Route specification Continuou
s specification
Start to move
position
Indicate position
velocity
Travel
acceleration
Indicate orientation
Stop moving
23
Alternate Technique classification
gaze-directed pointing physical props
Direction/Target Selection
gesture slow in, slow out physical props
Velocity/Accel. Selection
Travel
start/stop buttons automatic start/stop constant
movement
Conditions of Input
Ontologies by Bowman99 and Hand97
24
Evaluation results

• Steering techniques have similar performance on
  absolute motion tasks
• Non-head-coupled steering better for relative
  motion
• Teleportation can lead to significant
  disorientation
• Environment complexity affects information
  gathering
• Travel interaction technique and users
  strategies affect spatial orientation

25
Evaluation results 3

• Manipulation-based techniques efficient for
  relative motion
• Manipulation-based techniques not requiring an
  object efficient for search, but tiring
• Steering techniques best for naïve and primed
  search
• Map-based techniques not effective in unfamiliar
  environments, or when any precision is required

26
Myths

• There is one optimal travel technique for VEs.
• A natural technique will always be better than
  another technique.
• Desktop 3D, workbench, and CAVE applications
  should use the same travel ITs as HMD-based VEs.

27
Design guidelines

• Make simple travel tasks simple (target-based
  techniques for motion to an object, steering
  techniques for search).
• Provide multiple travel techniques to support
  different travel tasks in the same application.

28
More design guidelines

• Use transitional motions if overall environment
  context is important.
• Train users in sophisticated strategies to help
  them acquire survey knowledge.
• Consider integrated (cross-task) ITs if travel
  is used in the context of another task (e.g.
  manipulation).

29
Wayfinding

• Cognitive process of defining a path through an
  environment using and acquiring spatial knowledge
• 6DOF makes wayfinding hard
• Human beings have different abilities to orient
  themselves in an environment
• Observing wayfinding as a decision making process

30
Wayfinding tasks

• General, explorative search Search without
  target
• Naïve search target position unknown
• Primed search target seen before (known)
• Specified trajectory movement Predefined path

31
Cognitive Map

• During wayfinding, a person makes use of three
  kinds of knowledge to built up a cognitive map of
  the environment
• Landmark knowledge
• Procedural knowledge
• Survey knowledge

32
Reference frames

• Egocentric reference frame position,
  orientation, movement of object with respect to
  position and orientation of the
• eyes
• head
• body
• Exocentric reference frame position, orientation
  and movement are defined in coordinates external
  to body
• object shape
• object orientation
• object motion

33
Travel technique effects

• Steering technique with good strategy helps
  spatial orientation
• A good travel technique will integrate aids to
  wayfinding
• Jumping between points disturbs spatial
  orientation

34
Support of spatial knowledge acquisition

• Allow a wide field of view
• Provide motion cues for judging depth and
  direction of movement
• Audio could enhance visual spatial perception
• Support sense of presence it could strengthen
  the construction of a cognitive map

35
Support of spatial knowledge acquisition

• Design legible environments - allow the user to
  easily see the spatial organisation of an
  environment, enabling the establishment of a
  cognitive map.
• Divide a large-scale environment into parts with
  a distinct character

36
Support of spatial knowledge acquisition

• Create a simple spatial organisation in which the
  relations between the parts are clear
• Support the matching process between the
  egocentric and exocentric frames of reference by
  (visual) cues, including directional cues

37
Support of Spatial Knowledge Acquisition

• Use real-world wayfinding principles to build up
  your environment
• Natural environment principles
• Urban design principles Lynch
• Architectural design principles
• Artificial cues

38
Examples of wayfinding aids
39
Map usage guidelines

• Provide you are here marker
• Provide grid
• Choose either north-up or forward-up map,
  depending on task
• Example World-in-Miniature

40
Manipulation Techniques

• Wand-based
• Go-Go
• Ray Casting
• HOMER (Hand-centered Object Manipulation
  Extending Ray-casting)

41
Two Handed and Body-Centered Interaction

• What can you do with two hands?
• What if you use your body as a reference point?
• Mine, Mark, Frederick P. Brooks Jr., and Carlo
  Sequin (1997). Moving Objects in Space
  Exploiting Proprioception in Virtual-Environment
  Interaction. Proceedings of SIGGRAPH 97, Los
  Angeles, CA. (133K pdf version)

42
Two Handed and Body-Centered Interaction

• What can you do with two hands?
• What if you use your body as a reference point?
• Mine, Mark, Frederick P. Brooks Jr., and Carlo
  Sequin (1997). Moving Objects in Space
  Exploiting Proprioception in Virtual-Environment
  Interaction. Proceedings of SIGGRAPH 97, Los
  Angeles, CA. (133K pdf version)

43
Other Manipulation Approaches

• Tablet/hand-held
• Pinch Glove