TeleImmersive Collaborative Tools for Artistic and Scientific Work

1
Tele-ImmersiveCollaborative Tools for Artistic
and Scientific Work 
Department of Computing Science University of
Alberta

• Dr. Pierre Boulanger
• Advanced Man-Machine Interface Laboratory
• University of Alberta
• http//www.cs.ualberta.ca/ammi

2
Lets Start With A Real Artistic Collaboration

• Collaboration
• Catherine Richard and One of her student Sean
  Cleasen, Fine Art Department, University of
  Ottawa
• Pierre Boulanger, Advanced Man Machine Interface
  Lab., Department of Computing Science, UofA
• The Problem
• Exploration of a Virtual Space Using a Low Cost
  Hand-Held Display System

3
Work Statement
4
Performance Room
5
Entering the Room
6
Display System
7
Low Cost Hand-Held Display System
Local Network
Graphic Rendering Machine
3D Tracker
PDA Display
8
Model Creation
NRC 3D Laser Scanner
9
Final Model
10
Some Conclusions

• Next generation will allow collaborative
  observation over a network
• Collaboration with artist is not always easy
• Different vocabulary and technical expertise on
  both sides
• Requirement for low budget solutions
• Need for a performance ready solution
• But it is always fun, challenging, and mutually
  beneficial

11
Western Canada Grid
Partially Funded by The Canadian Fund for
Innovation 38M Project
12
MACI Cluster Located at the UofA
13
A WestGrid Example

• Collaboration
• Moritz Heimpel, Institute for Geophysical
  Research, Department of Physics, UofA
• Pierre Boulanger, Advanced Man Machine Interface
  Lab., Department of Computing Science, UofA
• The Problem
• Simulation and 3D Visualization of The Planetary
  Dynamo Problem

14
Core Geometry The Radius Ratio c ri/ro
Earth (c 0.35) Mercury (c 0.75) Ganymede (c
0.2) Io (c 0.50 ) Jupiter (c 0.85)
15
Equations of Motion
16
Comparison of Earth Magnetic Field With Dynamo
Model
(U. Christensen et al., 1999)
17
The Essential 3D Display
18
And Now Lets Meet Over the Grid
19
Towards Wide Area Teleimmersion

• Convergence of Virtual Reality, Collaboration
  Technology and Active Spaces

Visual
Audio
VR
Haptics
Visual
Visual
Audio
Audio
VR
VR
Haptics
Haptics
20
Grid Computing and Visualization
3D Graphic Rendering
New VizRoom
Massive Storage
Haptic Rendering
High Speed Network CanNet 4
3D Sound Rendering
Input Sensors
21
UofA Passive Stereo Projection Unit
22
Passive Stereo

• Two projectors are used for the single screen
  one for each eye's view.
• Differently polarizing filters are placed in
  front of each projector lens.
• Users wear polarizing glasses where each lens
  only admits the light from the corresponding
  projector.

23
The Access Grid
24
The Access Grid

• Access Grid does for people what the
  computational Grid does for machines
• The Access Grid project focus is to enable
  groups of people to interact with Grid resources
  and to use the Grid technology to support group
  to group collaboration at a distance
• Distributed Lectures and seminars
• Remote participation in panel discussions
• Virtual site visits meetings
• Complex distributed grid based demonstrations

25
What is the Access Grid?
Access Grid Movie
26
Components of an AG Node
SGI Sizserver 3.0
RGB Video
Digital Video
Display Computer
Network
Shared App, Control
NTSC Video
Video Capture Computer
Digital Video
Analog Audio
Audio Capture Computer
Digital Audio
Mixer
Control Computer
RS232 Serial
Echo Canceller

• Software, Production Issues

27
Banff/UofA Collaborative Laboratory (Third Floor
Clyde Hall)
28
Banff/UofA Collaborative Laboratory (Projection
System)
Screen
Speaker
Speaker
Computer Rack
Projectors Ceiling Mount
M3
M4
Sofa 1
Sofa 2
Table
M1
M2
29
Banff/UofA Collaborative Room Extension
Access Grid
PC Based Passive VR Display System
Immersive Communication Device
30
Banff Advanced Visualization Laboratory
31
Banff System Configuration
Passive Stereo Projectors
32
UofA Advanced Collaborative Immersive Environments
33
UofA System Configuration
Master Control Computer
3D Audio Server
AG Audio Capture Server
Interface Server
Front Display Pipe
Left Display Pipe
S1
M1
Joystick
Active Stereo Projectors
S2
M2
Inertial Tracker
Right Display Pipe
S3
M3
3D Tracker
M4
SGI ONYX 2 with 6 CPUs and 3 Graphic Pipes
Sn
Other Interface
Microphones
Speakers
34
Canadian National Optical Network and
Collaborative Environment
www.virtualizedreality.org
CAnet 3
GigaPOP
RAN
SRnet
WURCnet
MRnet
OC3
DS3
OC12
ACORN
BCnet
St Johns
Calgary
OC3
Regina
RISQ
Winnipeg
Charlottetown
ONet
OC48
Fredericton
OC12
Teleglobe
Montreal
Halifax
Vancouver
Ottawa
STAR TAP
Toronto
Chicago
35
Immersive Video Display and Visualization over
Satellite

• Project Goal To demonstrate that true immersive
  video conferencing and scientific visualization
  is possible over digital satellite communication
  using state of the image processing and 3D
  display system
• Project in collaboration with Telesat Canada

36
Immersive Video Display and Visualization over
Satellite
Anik E2
Telesat Ottawa
University of Alberta Edmonton
Encoder MPG-2
Decoder MPG-2
Network Controller
Encoder MPG-2
Decoder MPG-2
Network Controller
Side By Side MPX
Side By Side MPX
Control PC
Control PC
DTI Glassless Stereo Display
DTI Glassless Stereo Display
37
The UofA Virtual Meeting Place Project

• Goal the main goal of this project is to create
  a general man-machine interface allowing
  engineers and scientists to communicate their
  design and visualize their data over the
  internet, producing the equivalent of a virtual
  meeting place.

Live Stereo Texture and sound
Virtual Actuators
Collaborative Object And Data Manipulations and
Interactions
CAD Model or Scientific Data
38
UofA Virtual Meeting Project
39
UofA Virtual Meeting Project
40
UofA Virtual Meeting Project
41
Virtual Avatars Based on Stereo Textures
Left Image
Stereo Texture
Background Extraction Movie
Right Image
42
Network issues with Immersive Collaborative
Environments

• Timing is essential in the operation of
  distributed virtual environment applications,
  since the perception of changes in the virtual
  environment is based on the timely delivery of
  messages, informing all the participants of the
  changes made by a user.
• Any action issued by any participant must reach
  the other participants within 200 ms. This task,
  challenging as it is, becomes even more
  challenging, when dealing with virtual
  environments with a large number of participants.

43
Network issues with Immersive Collaborative
Environments

• The high level of dynamicity in group structure
  and topology increases the complexity of the
  problem. Participants might join and leave the
  session dynamically.
• The requirement that the developed DVE
  applications will be supporting collaborative
  functions, makes timing even more important.

44
What Next Collaborative Environment Using AR For
Training
Work in Collaboration with the University of
Ottawa SITE Laboratory
45
AR Training Session
46
What Next? Emotional State Transmission and
Display Over the Network Using EEG
EEG Analysis System LORETA Algorithm
EEG Analysis System LORETA Algorithm
Emotion State Display System
Emotion State Display System