Issues on 3D Interaction

1
Issues on 3D Interaction

• ? ? ? (Namgyu Kim)
• ngkim_at_postech.ac.kr

2
Overview

• I/O devices for 3D interaction
• performance and applicability
• constraints each devices have for interaction
  design
• Selection/Manipulation techniques
• performance, usability and applicability
• state-of-the art selection/manipulation
  techniques
• Navigation techniques
• performance, usability and applicability
• travel and way-finding techniques

3
I/O for 3D Interaction

• Output devices
• visual displays
• auditory output
• tactile and haptic output
• Input devices
• discrete event devices
• continuous event devices
• combination devices
• speech input

4
Visual Displays

• Fully immersive displays
• head mounted display(HMD) many HMDs
• Semi immersive displays
• arm mounted display BOOM
• surround screen virtual environment CAVE
• Non-immersive displays
• projection-based ImmersaDesk, WorkBench
• stereo monitor use shutter glasses

5
Interface Design (Visual Displays)

• HMD
• physical objects require a graphical
  representation
• limits the types of input devices that can be
  used
• BOOM-type
• physical objects require a graphical
  representation
• limits two-handed interaction
• Other types
• do not want the user to get too close to the
  screens (stereo vision)
• possible to use physical props
• more flexibility in interface mappings from input
  device to task
• physical/graphical object conflicts

6
Auditory Output

• Types
• simple beep sound
• iconic simple wave sound
• natural sound recorded and synthesized(sound
  rendering)
• Issues
• localization and sonification
• 3D/stereo sound, HRTF(Head Related Transfer
  Functions)
• graphic and audio synchronization
• setup methods headphone, speakers
• Interface design
• secondary user feedback
• when to happen and where to look for
• substitution effects

7
Tactile/Haptic Output

• Main types
• ground referenced Phantom
• body referenced CyberGrasp
• tactile CyberTouch
• Issues
• produce mechanical devices
• Interface design
• useful for manipulation
• very intimidating
• mimic real world interaction

8
Discrete Input Devices

• Main features
• generate one event at a time based on the user
• examples
• keyboard, pinch glove
• Interface design
• useful for discrete command interface

9
Continuous Input Devices

• Trackers
• types magnetic, mechanical, acoustic, inertial,
  vision/camera, hybrids
• interface design
• correspondence between physical and virtual
  worlds
• Dataglove
• gesture and posture communication

10
Combination Devices and Speech

• Combination/Hybrid devices
• mouse, joysticks, tablets, space mouse, ring
  mouse, fly mouse, BAT, Wand, Flex and Pinch
• interface design
• consider DOF for users interaction
• Speech input
• require more speech recognition technology
• word-level user-independent
• sentence-level more training, noise
• interface design
• very convenient input
• ideal for multimodal interaction

11
Guidelines for choosing I/O

• Think about what interaction techniques are
  required
• Think about inter-restriction of input devices
  and output devices
• Practically money is big factor
• design restricted interaction techniques with
  given devices?
• buy more advanced devices for implementing
  interaction techniques?
• make new devices for accomplishing interaction
  techniques?
• assume that
• constraints of VR devices are disappeared in near
  future
• basic 3D devices are proposed like 2D devices
• design interaction techniques in basic 3D devices

12
Selection/Manipulation Techniques

• Importance
• using hand is only way to affect natural
  environment
• major method of interaction in immersive VE
• effectiveness of manipulation affects quality of
  whole VE interface
• development of manipulation interfaces and
  techniques is difficult
• Basic tasks
• select, position and rotate(orientation)
• Complex specific tasks
• object size, required translation, amount of
  rotation, distance to object, etc

13
Design Objectives

• Design techniques that
• conform to input and output devices used
• are effective for manipulation in required task
  conditions
• object distance within/outside the reach
• object sizes small objects, large objects, flat
  objects, etc
• required manipulation accuracy
• allow high user performance and usability
• are natural and intuitive
• conform to external constraints (usefulness)

14
Manipulation Techniques (1)

• the most natural
• limited area of reach

• at any distance
• natural, requires little effort
• difficult to select small objects and far away
  objects
• position and rotation constrained

Simple Virtual Hand
Ray-casting (Bolt, 1980)
Advantages
Disadvantages
Poupyrev et al. 1996 P-37
Bowman et al. 1996 P-4
15
Manipulation Techniques (2)

• selection of small objects is easier at any
  distance
• disambiguation is needed if several objects fall
  in spotlight

• interactive object disambiguation
• manipulation is still difficult

Spotlight (Liang, 1994)
Aperture (Forsberg, 1996)
Advantages
Disadvantages
Liang et al. 1994 C-16
Forsberg et al. 1996 P-23
16
Manipulation Techniques (3)

• easy selection (on image plane)
• manipulation is difficult

• controls distance in manipulation
• needs extra control devices

Sticky Finger (Pierce, 1994)
Fishing reel (Bowman, 1997)
Advantages
Disadvantages
Pierce et al. 1996 P-25
Bowman et al. 1996 P-4
17
Manipulation Techniques (4)

• a large range of distances
• intuitive
• manipulation range is still limited
• overshoot with large distance

• seamless 6DOF manipulation with a large range of
  distance
• natural manipulation within reach
• do not notice actual scaling
• outward object movement is difficult

Go-Go (Poupyrev , 1996)
Scaled-world grab (Mine , 1997)
Advantages
Disadvantages
Mine et al. 1997 P-44
Poupyrev et al. 1996 P-37
18
Manipulation Techniques (5)

• allows 6DOF manipulation at any distance
• difficult to precisely manipulate small objects
  in WIM

World-in-Miniature (Stoakley, 1995)
Others(?, 1999)
Advantages
Disadvantages
Stoakley et al. 1995 P-34
19
Design Guidelines

• Practice
• eliminate extraneous degree of freedom by
  implementing only those dimensions which users
  perceive as being related to given tasks
• free moving tracked gloves may be more useful
  when implemented as position controllers
• Considerations
• create new and re-examine existing techniques for
  emerging input and output devices
• Integrate techniques
• how to combine multiple techniques in complex
  manipulation scenarios?
• create standards and guidelines

20
Navigation Techniques

• Way-Finding
• cognitive process of defining path through an
  environment, using acquired spatial knowledge,
  helped by artificial cues
• Travel
• motor component of navigation
• Problem
• way-finding different abilities to orient
  oneself in an environment
• travel exploration, search, maneuvering
• way-finding ? search task in travel ?
• cognitive issues are important in way-finding

21
Navigation Tasks

• Types
• exploration no specific target
• build knowledge of environment
• search
• naïve search without knowing the position of
  the target
• primed search with knowing the position of the
  target
• build layout knowledge
• maneuvering
• travel to position viewpoint for task
• short movements

22
Cognitive Map

• Survey knowledge
• topological knowledge of an environment
• object locations, inter-object distances, object
  orientations
• map-like, hierarchical nature
• Procedural knowledge
• the sequences of actions required to follow a
  certain path
• Landmark knowledge
• visual attributes of an environment
• shape, size and texture

23
Reference frames

• Egocentric reference frame
• being in the center of space
• position, orientation, movement of object with
  respect to position and orientation of the eyes,
  head, body, proprioceptive
• non-visual sense of body parts
• visual visual location of body parts
• intersensory combination of two sense
• Exocentric reference frame
• looking from the outside, objects as center
• position, orientation and shape are defined to
  coordinates external to body

24
Navigation Techniques (1)

• viewpoint person in context
• good for path planning
• occludes environment
• less useful for large-scale environ.

• 3D landmarks world guidebooks
• avoiding occlusion
• additive viewpoint and world selection

World-in-Miniature (Stoakley, 1995)
Worldlet (Elvins, 1997)
Advantages
Disadvantages
Elvins et al. 1995 p-84
Stoakley et al. 1995 P-34
25
Navigation Techniques (2)

• Steering direction and velocity
• hand-directed
• gaze-directed
• physical devices (steering wheel, flight sticks)
• Target-based
• point at object, list of coordinates
• Route planning
• place markers in world

Mine, 1995 P-40
26
Design Guidelines

• Practice
• make simple travel tasks simple
• provide multiple techniques to support different
  tasks
• use graceful transitional motions if overall
  environment context is important
• train users in sophisticated strategies to help
  them acquire survey knowledge
• consider integrated interaction techniques if
  travel is used in the context of another task
• Considerations
• there is one optimal travel technique for VEs?
• a natural technique will always exhibit more
  performance, usability, and applicability than
  other technique?
• way-finding is dependent on exposure time?