Errors in Perceiving Depth with Stereoscopic Displays and Mixed Reality

1
Errors in Perceiving Depth with Stereoscopic
Displays and Mixed Reality

• David Drascic Paul Milgram
• Ergonomics in Teleoperation Control Lab
• Mechanical Industrial Engineering Dept.
• University of Toronto
• http//etclab.mie.utoronto.ca

2
Overview

• Classification of Mixed Reality Displays
• Practical Context / Motivation
• Review of Perceptual Issues
• Experimental InvestigationContext Ergonomics

3
What is Mixed Reality?
Mixed Reality (MR)
Reality - Virtuality Continuum
Real Environment
Virtual Environment
Augmented Reality (AR)
Augmented Virtuality (AV)
Local Remote
4
Classes of Mixed Reality (MR)Visual Display Media

• Monitor Based (Window-on-World)
• Dynamic Head Mounted Virtual Window
• Optical See-Through HMD
• Video See-Through HMD
• Large Screen Projection

5
Some Ergonomic Issues with MR Stereoscopic
Displays

• Must ensure compatibility among different means
  of generating images
• Direct Viewing (DV)
• Stereo Video (SV)
• Stereo Graphics (SG)
• Must provide for accurate spatial alignment of
  virtual objects with real objects

6
Example 1 Virtual Tape Measure for Minimally
Invasive Surgery (SVSG)
7
Example 2 Mixed Reality Interactions (DVSG)

8
Example 3 Large Screen Projection Displays

• High immersion via large screen size
• Virtual graphics with Direct Viewing (DV)
  Overlays
• High resolution reality (DV) Variable res
  graphics
• Head tracking gtViewpoint Dependent Imaging
• Real-world registration critical

Courtesy of ATR Communication Systems Research
Lab, Kyoto
9
Implementation Problems

• Video Calibration Errors
• errors in measuring actual stereo video
  parameters
• errorsgt warp visual space distort perceived
  velocities
• displays should be completely orthoscopic
• Video/Graphic Mismatches
• differences in video graphics parameters
• affects overlay compatibility
• Interpupillary Distance (IPD) Errors
• affects perception of absolute distances

10
(Current) Technology Limitations

• Static / Dynamic Registration Mismatches
  (Tracking)
• Restricted Fields of View
• Display Resolution Limitations Mismatches
• Display Luminance Limitations Mismatches
• Contrast Mismatches
• Depth Resolution Limitations
• Vertical Alignment Mismatches

11
Fundamental Perceptual Factors

• Interposition Inconsistencies

12
Interposition/Transparency Effects

• Video-based displays SV never occludes SG
• Depending on texture gt surface effects
• SG images sometimes break down at surfaces
• other times real objects appear transparent

(SG interposition is possible, but with model
of real world)
13
Interposition/Transparency Effects

• Video-based displays SV never occludes SG
• Optical HMD displays Combined images always
  transparent
• DV never occludes SG
• SG never occludes DV (without model)
• Large-screen displays DV always occludes
  projected SG image

(e.g. immersive CAVE-type environments)
14
Fundamental Perceptual Factors

• Interposition Inconsistencies
• Accommodation - Vergence Conflicts

15
Accommodation - Vergence Conflicts

• Common problem with many Stereo displays
• Accommodation distance to screen differs from the
  convergence distance to fused image
• Cause of eye strain?
• Perceived distance at intermediate point?

16
Fundamental Perceptual Factors

• Interposition Inconsistencies
• Accommodation - Vergence Conflicts
• Accommodation Mismatches

17
Accommodation Mismatches

• Example Real hand touching virtual object
• Accommodation to real hand (fDV) ? accommodation
  to graphic box (fSG) on screen
• If box perceived at position p, where will hand
  point?

18
Fundamental Perceptual Factors

• Interposition Inconsistencies
• Accommodation - Vergence Conflicts
• Accommodation Mismatches
• Absence of Shadow Cues
• Image Quality Differences

19
Image Quality Differences

• Real objects appear sharp, in perfect focus, with
  high contrast
• Virtual objects appear fuzzy, with poor focus and
  low contrast
• Problem (with all else being equal)
• Fuzzy images appear to be further away
• Low contrast images appear further away
• Darker images appear further away
  Accidental Depth Cues?

20
Hypothesised Perception - Action Coupling

• Computer draws object at Position 1
  (Vergence)
• User perceives object at 2 (Accom-Verg
  conflict)
• User reaches for object, but perceives
  (Disparity) error due to binocular disparity
• User adjusts hand to Position 3,
  (Accom. balancing disparity with other cues
  Mismatch)
• As hand grasps object, perceived
  (Occlusion) position moves to 4

21
Summary

• Perceptual factors influence design of Mixed
  Reality displays
• Accurate perception of objects in MR may be
  impossible, due to number of factors and
  intra/inter user variability
• Computer aiding (computational vision) may
  alleviate some factors
• MR systems without DV avoid many problems because
  SV and SG affected equally

22
Experiment 1 SVSG Alignment Performance
Pointer Target Shape
Pointer
Target

• Subjects aligned real and virtual pointers with
  real and virtual targets
• Found no bias, similar accuracy

23
Experiment 1 Principal Results
24
Experiment 2 Real/Virtual DV-SG Alignment
25
Design of Experiment 2 Real/Virtual DV-SG
Alignment

• Subjects used same CoRD (Computerised Rope
  Device) input for all conditions
• Condition VP-VT Alignment of Virtual Pointer
  (top) with Virtual Target (bottom)
• Condition RP-VT Alignment of Real Pointer (top)
  with Virtual Target (bottom)
• Null Hypothesis Zero alignment errors for all
  conditions

26
Experiment 2 Results Real/Virtual DV-SG
Alignment
Alignment Error vs Target Distance
Virtual Pointer / Virtual Target
Alignment Error
Real Pointer / Virtual Target
27
Experiment 2 Results Sample Pointer Placement
Trajectories
Virtual Pointer
Real Pointer
Alignment Error
Time
Time
28
Implications

• Lack of significant differences in alignment
  errors gt no evidence of placement problems for
  monitor based alignment of VP-VT and RP-VT tasks
• Not necessarily true for large screen immersive
  MR applications
• Evidence of more efficient placement performance
  for real-pointer (RP) control