Network Virtual Environment

1
Network Virtual Environment

• ? ??
• VR Lab.
• CSD, KAIST
• ujsung_at_vr.kaist.ac.kr

2
Talk Outline - Network VE

• 1. Current activity standards, projects,
  applications
• 2. NVR System architecture and Distribution
  issues
• 3. Projects at KAIST and Future applications
• 4. Group Communication

3
Network Virtual Environment

• Multiple participants
• world wide distribution - Internet
• Shared environment
• 3D virtual environment
• interaction through VR interface
• Human - Human Interaction
• aware each other (Avatar)
• communicate each other (virtual societies)
• competition / collaboration (net game / team
  training)

4
(No Transcript)
5
(No Transcript)
6
1. Current activity standards, projects,
applications

• Projects, Standards
• SIMNET, DIS, HLA
• DiamondPark, SPLINE
• CAVE, CAVERN
• COVEN, DIVE, MASSIVE
• WWW - VRML, X3D, H-Anim
• Commercial applications
• Simulator MAK, BDI, ...
• Internet 3D chatting - AlphaWorld, Worlds,
  Blaxxun, Sony, ...
• Game multi-user 3D games - BattleNet,
  UltimaOnline, ...

7
Timeline
1984 Neuromancer
1965 HMD
1985 VPL
1992 SGI - RE
1994 Voodoo
Mid 1970s VIDEOPLACE
1999
1990
1970
1980
1991 DIS starts
1995 HLA
1988 SIMNET
1993 DIVE
1995 COVEN
1992 CAVE
1997 CAVERN
1996 ActiveWorld
1988 IRC
1980 MUD
1998 X3D(VRML-NG)
1995 VRML
19911994 WWW, HTML
8
SIMNET, DIS, HLA

• Military simulation / training

9
(No Transcript)
10
SIMNET, DIS, HLA

• SIMNET - 1980s
• interconnection of low-end simulators
• Ethernet broadcast less then 200 participants
• homogeneous platform, the simplest solution
• DIS (Distributed Interactive Simulation) - 1991
• evolved from SIMNET IEEE standard
• Internet multicast 1,00010,000 participants
• PDU packet (heterogeneity)
• message loss, latency problem
• best-effort multicast, receive order processing
• dead-reckoning

11
SIMNET, DIS, HLA

• HLA (High Level Architecture) - 1995
• evolved from DIS, proposal to IEEE standard
• Http//hla.dmso.mil/
• A general purpose architecture
• goal interoperability, reusability
• DIS PDUs are specialized for special simulation.
• general time management
• HLA Basic components
• HLA compliance rules
• Object model templates
• Run time infrastructure

12
DI-Guy

• Http//www.mak.com, http//www.bdi.com
• DI-Guy Dismounted Infantry Option for MAK Stealth

13
DiamondPark, SPLINE

• MERL(Mitsubishi Electric Research Laboratory)
• http//www.merl.com/
• Electronic Meeting Place (1994)
• five participants four human, a
  computer-simulated robot
• MIT, Boston Dynamics Inc., Carnegie Mellon Univ.
  U. Penn.

14
DiamondPark, SPLINE

• Social Virtual Reality
• Diamond Park (19951996) SPLINE Architecture

15
CAVE, CAVERN

• CAVE(1992), ImmersaDesk(1995)
• EVL / UIC (http//www.evl.uic.edu/cavern)
• CAVERN (CAVE Research Network)
• alliance of industrial and research institutions
• CAVE, ImmersaDesk
• supercomputer
• interconnected by (very) high-speed networks
• CAVERNUS (CAVERN) Users Society
• organized by NCSA

16
CAVERNsoft Users

• Car Styling (General Motors)
• Product Drop Test Visualization (Motorola)
• Heliac Plasma Coils (ACSys/ANU)
• CAVE Quake (NCSA)
• VR Tele-Collaborative Manufacturing Environment
  (Searle)
• CAVE6D- Environmental Hydrology (CCPO, ODU)
• Data-Mining Health-Care Data (NCDM)
• Virtual Temporal Bone (VRML, UIC)
• LIMBO/VTK (EVL,UIC)
• Virtual Mail (EVL, UIC)
• Virtual Whiteboard (Tohwa Univ)
• CAVE Virtual Console (VT)
• Silk Road Shrine (EVL,UIC)
• Future Camp (IUPUI)
• Tele-Immersion Performance Studies (EVL,UIC)
• Battlefield Visualization (NRL)

17
COVEN Project

• COllaborative Virtual ENvironments
• 19951999 European Project Consortium
• http//chinon.thomson-csf.fr/projects/coven/
• Thomson-CSF LCR, France, (Project Coordinator)
• Division Ltd., UK
• EPFL, Swiss Federal Institute of Technology,
  Switzerland
• Integrated Information Systems Ltd, Greece
• KPN. Royal PTT Netherland, Netherlands
• Swedish Institute of Computerr Science, Sweden
• Arax Ltd., UK
• Lancaster University, UK
• TNO Physics and Electronics, Netherlands
• University College London, UK
• University of Geneva (MIRALab), Switzerland
• University of Nottingham, UK

18
COVEN

• Application Citizen and business applications
• citizen-oriented scenario - large virtual travel
  agency
• professional scenarios - virtual conferencing,
  business game
• System (COVEN platform) enhancement of the
• development system dVS(DIVISION)
  http//www.division.com/
• research system DIVE(SICS), MASSIVE(Nottingham
  Univ.)
• Network
• Europe-wide ISDN and ATM network
• network performance - trial and experimentation
• Usability study
• human factors - trial and experimentation

19
DIVE

• Distributed Interactive Virtual Environment
• Swedish Institute of Computer Science
• http//www.sics.se/dive/

20
Internet and VR

• VRML standard - http//www.vrml.org
• VRML 1.0
• VRML 97
• Multi-user VR over Internet
• ActiveWorld (Circle of Fire Inc.)
• CommunityPlace (Sony)
• Virtual Place (Ubique, Ltd.)
• Worlds Chat (Worlds, Inc.)
• Passport (Blaxxun Inc.)
• X3D (VRML-NG)
• Web3D - http//www.vrml.org/fs_specifications.htm

21
Active World
22
Active World

• Http//www.activeworlds.com -
• _at_Mart 3D virtual shopping mall

23
Web3D Consortium X3D Mission Goals

• Broad range of 3D applications, not just
  immersive worlds
• Components approach building blocks
• Most crucial goals interoperability
• VRML 97
• Other scene graph APIs
• Applications
• Streaming broadcast
• SIGGRAPH 99 demos?
• Web3D approval 1999 (maybe joint with W3C) then
  ISO ratification

http//www.vrml.org/fs_technicalinfo.htm
24
2. NVE System Architecture and Distribution
issues

• NVE System Architecture
• walk-through system
• CVE (Collaborative Virtual Environment) system
• Distribution issues
• Scalability
• High level issues
• Middle level issues
• (Low level issues)
• group communication

25
Network Virtual Environment

• Objects
• rigid 3D geometry
• Avatar
• set of objects behavior
• Interaction
• aware
• navigation
• object manipulation
• (verbal communication)
• gesture communication

Verbal communication (through network)
aware
avatar
avatar
manipulate
object
object
navigate
26
Object, Avatar

• Object
• rigid 3D geometry, unique identifier (URL)
• Avatar
• Jack - U.penn, DI-Guy (http//www.bdi.com/)
• EPFL, MIRALab (http//ligwww.epfl.ch/)
• DIVE
• H-Anim (http//ece.uwaterloo.ca/h-anim)
• Avatar in NVE systems
• representation of human
• user interface
• state of avatar controls view, audio, network,
  session, ...

27
Navigation in NVE

• Local response
• change my view, move my body, perform body motion
• Remote notify
• send avatar update message to others
• , and
• download some objects - incremental loading of VE
• ActiveWorld, MASSIVE, ...
• portal - change current VE (and change of session
  server)
• DiamondPark Locale, ActiveWorld,
• enable/disable connection with others (hear/see)
• MASSIVE SIM, NPSNET AOIM,
• enable/disable manipulate

28
VR system Architecture

• Single user
• Walk-through (ex. VRML 1.0)
• Dynamic world (ex. VRML 97)
• Multi-user Walk-through
• battlefield simulation (ex. DIS)
• virtual conferencing, virtual society
• Multi-user Collaborative VE
• CAD design / layout
• virtual factory
• virtual mall

29
Basic models Single user
sensing
Navigator
3D models
Human
view position
geometry
Single user walk-through
Renderer
Sensory data
coordinator
Manipulator
Navigator
Human
simulator
Avatar
3D Objects
Single user dynamic world
Renderer
30
Basic Models Multi-user walk-through
Human
Human
Navigator
Renderer
Navigator
Renderer
Avatar
3D model
Avatar
Navigator
Navigator
network
31
Basic Models Collaborative VE (Centralized
approach)
Human
Human
Navigator
Renderer
Manipulator
Navigator
Renderer
Manipulator
Avatar
3D objects
Avatar
Navigator
Manipulator
Manipulator
Navigator
network
Coordinator
32
Basic Models Collaborative VE (Distributed
approach)
Human
Human
Navigator
Renderer
Manipulator
Navigator
Renderer
Manipulator
Navigator
Coordinator
Navigator
Coordinator
network
33
Communication Architecture

• 3D Model replication
• client-server easy, persistency
• distributed scalability
• Update message communication
• communication server
• multicast
• Coordination of concurrency
• client-server
• distributed

34
Physical limits

• Bandwidth
• packet size 1Kbyte, update 10fps - 10Kbyte /
  second
• 10Mbit Ethernet 100 participants(multicast),RMP,
  10 (unicast)
• Latency
• LAN - 1ms, WAN - 100ms 500ms
• light speed limit - on earch? 1/7.5 second
• within 1/30 second latency - ? km
• message loss and retransmit
• 130 (one-to-one inter-continental)
• Message processing
• need multi-thread rendering, user interface and
  network

35
Scalability

• Replication and Update multicasting
• SIMNET
• Filtering
• DIS dead-reckoning
• MASSIVE Spatial Interaction Model
• RING visibility test
• HLA Data Distribution Management
• Partitioning and portal
• MUD rooms and uni-directional links
• NPSNET-IV AOI
• large scale participants
• DiamondPark Locale
• large scale environment
• ...

36
Dead-reckoning

• Dead-reckoning
• Each host runs dead-reckoning on its own
  object(player) and other hosts objects(ghosts)
• Periodically sends Entity State PDU to correct
  its objects state in other hosts

Send Entity State PDU if Deviation gt Threshold
Actual Path
Dead-Reckoning Path
37
NPSNET-IV

• Region based communication
• multicast group per AOI(area-of-interest)
• Each entity may join multiple groups

AOI
Virtual world
38
SIM - MASSIVE2

• Entity based communication
• multicast group per media, group
• group space based interest group
• media text, audio, graphic

39
Locale - DiamondPark

• Locale based communication
• locale small chunk of replicated world model
• multicast group per locale

40
Interactability

• NVE characteristics
• real-time interaction
• trade-offs between responsiveness and consistency
• Concurrency control
• message ordering
• ISIS (causal ordering) - DIVE 1
• Centralized global ordering - CoCAD
• locking
• Distributed locking - DIVE 2
• Locking server - dVS, Bricknet,
• Per object token passing - DIVE 3,
• optimistic
• optimistic token multicasting - CIAO

41
Concurrency control in DIVE

• DIVE version 1
• ISIS reliable multicast, causal/total ordering
• DIVE version 2
• distributed locking (coarse-grained)
• RMP
• DIVE version 3
• pessimistic token passing (fine-grained)
• SRM

42
NVE Issues

• Bandwidth
• Latency
• Reliability

• Scalability
• Interactability

• Architecture
• Sharing Mechanism
• Multi-user Interface

• Awareness
• Responsiveness
• Consistency

43
High level issues

• Acceptable
• realistic / synchronous is better then
• concurrent, real-time interaction
• Questionable
• awareness more and more rich awareness
• interaction / coordination as we do in
  real-life
• Problems
• Have you been in the NVE?
• multi-user interface, metaphor
• Is it better then 2D interface application?
• What is the requirements of NVE system?

44
Middle level issues

• Acceptable
• replicated virtual environment
• multicast communication
• Questionable
• concurrency transparency
• large scale and scalability
• distributed design is better than centralized
• bandwidth, latency, message loss/ordering is the
  most important ...
• Problems - performance!
• session management and world partitioning
• consistency and concurrency control
• synchronous interaction over asynchronous network

45
3. Projects at KAIST VR Lab. and Future
applications
Landscape (9597)
Mult-User 3D Web Browser VRML / HTTP
CVRAT V4(94)
Collaborative VE Authoring Distributed VR system
VRAT V3(9395)
Single Node VR Kernel
CIAO (98)
Collaborative, Immersive Architectural layOut
(98)
46
LandScape
47
LandScape System configuration
World Server
Participants
2D Browser
World Server
User A
CGI
MIME
CCI
3D Browser
HTTPD
VRML
HTTP
2D Browser
User B
MIME
CCI
Standard Protocol / Existing System
3D Browser
48
CVRAT 4 Users Dialog with Gestures
Whos Next?
You !!
I dont know.
Not Me !
49
CVRAT Authoring
50
CIAO System Architecture
Participant nodes
Session Management Server
Object Information Server
multicast
Object Information Server
unicast
51
CIAO node Software Architecture
CVRAT
Avatar Management
Concurrency Control
Session Management
VRAT
Multicast Message Handler
Graphic Object (scene-graph) Management
Rendering System
UNIX Socket
VR Sensor drivers
GUI
OpenGL, IRIS GL, glide
RS232c
X/Motif
52
Concurrency control in CIAO

• Version number
• Semi optimistic approaches
• Validation
• conflict multiple tentative tasks with same
  version number
• token per object fine-grained locking
• validate one task among multiple conflicting tasks

Taski
Taski1
Versioni-1
Versioni
Versioni1
Task'i1
53
Concurrency control in CIAO

• Token multicast based validation

Actor
Observer
1
1
2
2
1 notification/request 2 validation/token
transfer
Token owner

• Network latency L
• response time - 0
• notification time - L

54
CIAO User Interface Cloning object
Wire-frame original object
1
2
3
4
After validation
Translucent cloned ghost
55
On the LAN? 0.001 sec I can not see the ghost
L 0.2 sec
L 0.5 sec
L 2 sec
56
CIAO Ongoing projects
Rutgers
KAIST DSG
KAIST VRL
98
99
Upenn
ICU

• Setting up the CIAO based Network VR System
• Application developments
• Multi-user collaborative design
• Virtual Factory / Prototyping
• Networked Virtual Office
• Virtual Model House at home

57
Virtual Factory

• 1999 (KAIST VR Lab.)
• Collaborative Virtual Car Design
• Concurrent Engineering for Car Manufacturing

58
Lifecycle Model ofCar Manufacturing
Concept Verification (styling)
Design Verification
Production Verification
Packaging
Styling
Fairing
Testing
Drawing
Tooling
Prototyping
Analytic Simulation
Production Drawing
Production Tooling
59
Virtual Mockup
Concept Portfolio
Concept Car in 3D
Mockup
Virtual Mockup
60
Blaxxun
61
Virtual Factory
Primitive Browser
62
Future applications

• What is the major use of nowadays?
• Play? Networked Games
• Chat? Virtual Societies
• Work? Cooperated Working Environment
• Future applications - Old Idea, Modern
  technology?
• Telepresense/Teleoperation
• Virtual Communities
• Simultaion/Training
• Medical
• Scientific/Visualization
• Design/Engineering
• Entertainment
• ...