Visualization in Teleimmersive Environments J' Leigh, A' Johnson, M' Brown, D' Sandin, T' DeFanti El

1
Visualization in Teleimmersive EnvironmentsJ.
Leigh, A. Johnson, M. Brown, D. Sandin, T.
DeFantiElectronic Visualization Lab, University
of Illinois-ChicagoRef IEEE Computer Dec
1999Presenter K. Palaniappan
2
Presentation Overview

• Teleimmersion
• Display technologies
• Networking issues
• Cavernsoft software framework
• Sample applications

3
Teleimmersion

• Collaborators at remote sites share the details
  of a virtual world that can autonomously control
  and evolve computation, query databases and
  gather results
• Collaborative VR - reproduce face-to-face meeting
  environment
• Teleimmersion - collaborative VR audio- and
  video-conferencing, data mining, heavy
  computation

4
Shared Virtual Enviornments

• Examples
• part of car design - standing inside engine block
• visualization of severe storm weather simulation
  - standing inside storm,
• participate in non-physical 3D environments
• Avatars
• Lifelike computer generated representations (of
  people, perhaps agents)
• Collaborators enter teleimmersive environment as
  avatars
• Environment transmits gestures as well as audio
  and video
• Steering
• Change parameters of supercomputing simulation
  interactively and collectively study the impact
  collaboratively

5
CAVE

• Cave Automated Virtual Environment (1992)
• Cavern - CAVE Research Network
• A collection of participating industrial and
  research institutions with CAVE, ImmersaDesk VR
  systems, high performance computing and high
  speed networks
• Around 100 CAVE and ImmersaDesk installations
  worldwide
• Support for effective collaboration requires
• New display devices for teleimmersion
• International networking infrastructure
• Software framework for teleimmersion applications
• Seamless integration of CAVEs, ImmersaDesks,
  other VR devices, workstations

6
Synchronity

• Synchronous vs Asynchronous collaboration
• Nationally dispersed teleimmersive sites vs
• International collaboration with significant time
  differences
• Continuing virtual world
• One that persists even when the collaborators are
  not present
• May evolve autonomously
• Different groups of collaborators may share the
  virtual world at different times

7
Display Technologies

• CAVE vs head mounted displays for VR
• Heavy helmets with liquid crystal displays or
  CRTs mounted in front of eyepieces
• CAVE VR -
• 10 x 10 sq ft room with CRT projectors for
  projecting stereoscopic images on walls, floor,
  ceiling
• Up to 10 users
• Illusion that objects surround user
• Light weight liquid crystal shutter glasses
• Three button wand for 3D interaction
• Electromagnetic tracking system attached to
  glasses and wand to determine location and
  orientation of users head and hands
• SGI Onyx to render images from users point of
  view
• Well suited for panoramic views

8
Display Technologies

• ImmersaDesk, ImmersaDesk 2
• Drafting table like systems with stereoscopic
  projection
• Objects that fit on a desktop - CAD models,
  biomedical, network topologies, battlefield
  simulation, etc.
• Prototypes of 3 new devices being investigated
• ImmersaDesk3
• 42 plasma screen mounted on a conventional
  office desk
• Vertical to horizontal positions
• Plasma screen generates too much EM noise for
  head tracking and stereo synchronization

9
Display Technologies

• PARIS - Personal Augmented Reality Immersive
  System
• Desktop device around an office desk
• Prototyped in a VR environment before
  construction to study user preferences and
  application requirements for viewing angle
  geometry
• Semitransparent mirror enables user to see hands
  and computer rendering simultaneously
• Future Place cameras under the mirrors to track
  hand location and gesture

10
Display Technologies

• AccessBot
• Teleconference device that uses life-sized plasma
  display screen
• High fidelity video- and audio-conferencing
• Robotic camera pan-tilt-zoom camera that lets
  remote user look around meeting room (zoom gives
  augmented vision capability)
• High speed (wireless) networking
• Equal telepresence with local participants in a
  meeting
• Prototype NCSA Access Center in Washington, DC

11
Networking

• Teleimmersive environments provide testbed for
  high performance networking
• Transmission of extremely large scientific and
  multimedia datasets
• Latency and bandwidth issues
• Complex distributed data handling - real-time
  audio and video streams, real-time tracker
  information on position and orientation of
  participants, updates to large multidimensional
  datasets, rendered geometry
• Multiple connections
• Realtime data network performance instruments -
  low latency, low bandwidth
• Audio and video - multicast over high bandwidth,
  low latency
• Models - high bandwidth
• Smooth interaction - guaranteed QoS

12
Networking

• Participants in
• MREN - Metropolitan Research and Education
  Network midwestern US
• NSF vBNS - Very high performance backbone network
  service
• Star Tap - Science, technology, and research
  transit access point for international
  infrastructure
• Teleimmersive applications are only beginning to
  use these networks

13
Software Framework

• Current software framework for collaborative VR
• Low-bandwidth commodity Internet
• Massive connectivity involving thousands of
  simultaneous participants - gaming, military
  simulations
• VR for scientific, manufacturing, information
  visualization
• Smaller, focused groups
• More demanding realtime streaming data
• Higher volume of engineering, scientific datasets

14
Software Framework

• Cavernsoft framework
• Many data classes
• Information Resource Broker (IRB) like CORBAs
  Object Request Broker (ORB)
• Lower overhead than ORB
• Emulate distributed shared memory, message
  passing system but maintain them separately
• TCP, UDP, or multicast
• Persistence of shared memory segments so that
  client programs can cache frequently used data at
  local sites
• Client-server symmetry using IRB - collection of
  clients can easily form arbitrary collaborative
  topologies

15
Software Framework

• Cavernsoft framework
• IRB-based libraries - avatars, audio-conferencing,
  collaborative interfaces, virtual email, model
  importing, data distribution, 3D manipulation
• Limbo and Tandem - higher level tool
  organization, source code available
• Example - Users import 3D models into
  teleimmersive env., Limbo distributes datasets to
  all participants for collective manipulation

16
Applications

• VisualEyes
• General Motors
• Import 3D CAD models from Alias into CAVE
• Quick visual inspection then do life-sized design
  review
• Changes in lighting and materials propagated
  automatically
• GM extension for transglobal design and
  manufacturing
• Synchronous and asynchronous access to design
  that persist, evolve

17
Applications

• Virtual Temporal Bone
• UIC Virtual Reality Medicine Lab
• Remote physician teaches medical students about
  3D structure and function of inner ear
• 3D interaction with model between students and
  instructor
• Remove organs and tissues to reveal inner anatomy

18
Applications

• ALIVE
• Architecture Linked Immersive Virtual Environment
  (Amsterdam Univ)
• Architects in Amsterdam and clients in Chicago
  use StarTap and teleimmersion for navigating 3D
  CAD models
• Network performance instruments - bandwidth
  usage, latency, jitter

19
Applications

• CAVE6D
• Center for Coastal and Physical Oceanography
• Uses CAVE5D and Cavernsoft
• Vis5D for visualization methods for
  multidimensional numerical data - isosurfaces,
  contour slices, volume visualization,
  wind/trajectory vectors, image projections
• Extensions - salinity, circulation vectors,
  temperature, etc.
• Distribute multiple observable dimensions to
  different users - coordinated vs independent
  displays

20
Applications

• TIDE
• Teleimmersive Data Explorer, UIC National Center
  for Data Mining
• Let collaborators query and access terascale
  datasets
• CAVE, Immersadesk, workstation users with avatars
  linked
• Streaming audio between geographically separated
  sites
• Data flow model on workstation used to construct
  visualization
• 3D tools to manipulate visualizations online
• Offline generation of high resolution stereo
  animation sequence on remote rendering server in
  background while teleimmersion continues then
  users see animation stream
• Need 3D annotation and recording of discoveries
  during session
• US DoE and NOAA applications being investigated