Enhancing Neighborship Consistency for Peer-to-Peer Distributed Virtual Environments

1
Enhancing Neighborship Consistency for
Peer-to-PeerDistributed Virtual Environments

• Jehn-Ruey Jiang, Jiun-Shiang Chiou and Shun-Yun
  Hu
• Department of Computer Science and Information
  Engineering
• National Central University

2
Outline

• Introduction
• Background
• P2P DVEs
• Factors affecting Neighborship Consistency
• Proposed Solutions
• Simulation Results
• Conclusion

3
Outline

• Introduction
• Background
• P2P DVEs
• Factors affecting Neighborship Consistency
• Proposed Solutions
• Simulation Results
• Conclusion

4
DVE (1)

• Distributed Virtual Environments (DVEs) are
  computer-generated virtual world where multiple
  geographically distributed users can assume
  virtual representatives (or avatars) to
  concurrently interact with each other
• A.K.A. Networked Virtual Environments (NVEs)

5
DVE (2)

• Examples of DVEs include early DARPA SIMNET and
  DIS systems, as well as currently booming
  Massively Multiplayer Online Games (MMOGs).

6
Massively Multiplayer Online Games

• MMOGs are growing quickly
• 8 million registered users for World of Warcraft
• Over 100,000 concurrent players
• Billion-dollar business

Adaptive Computing and Networking Lab, CSIE, NCU
7
Adaptive Computing and Networking Lab, CSIE, NCU
8
Adaptive Computing and Networking Lab, CSIE, NCU
9
(No Transcript)
10
(No Transcript)
11
Adaptive Computing and Networking Lab, CSIE, NCU
12
DVE (3)

• 3D virtual world with
• People (avatar)
• Objects
• Terrain
• Agents
• Each avatar can do a lot of operations
• Move
• Chat
• Using items

13
Issues for DVE

• Scalability
• To accommodate as many as participants
• Consistency
• All participants have the same view of object
  states
• Persistency
• All contents (object states) in DVE need to exist
  persistently
• Reliability
• Need to tolerate H.W and S.W. failures
• Security
• To prevent cheating and to keep user information
  and game state confidentially.

14
The Scalability Problem (1)

• Client-server has inherent resource limit

Resource limit
Funkhouser95
Adaptive Computing and Networking Lab, CSIE, NCU
15
The Scalability Problem (2)

• Peer-to-Peer Use the clients resources

Resource limit
Keller Simon 2003
Adaptive Computing and Networking Lab, CSIE, NCU
16
You only need to know some participants
Area of Interest(AOI)

• ? self
• ? neighbors

Adaptive Computing and Networking Lab, CSIE, NCU
17
Neighborship Consistency (1)

• Definition
• current AOI neighbors observed
• current AOI neighbors

18
Neighborship Consistency (2)

• An example

is actual neighbor
is observed neighbor
Neighborship Consistency 4 / 5 80
19
Outline

• Introduction
• Background
• P2P DVEs
• Factors affecting Neighborship Consistency
• Proposed Solutions
• Simulation Results
• Conclusion

20
Related Work (1)DHT-based SimMUD

• B. Knutsson, H. Lu, W. Xu and B. Hopkins,
  Peer-to-peer Support for Massively Multiplayer
  Games, in Proceedings of INFOCOM 2004.
• Authors are fromDepartment of Computer and
  Information Science, University of Pennsylvania

21
Related Work (1)DHT-based SimMUD

• Knutsson et al. 2004 (UPenn)
• Pastry (DHT mapping) Scribe (Multicast)
• Fixed-Sized Regions
• Coordinators

22
SimMUD -- Introduction

• Proposes use of P2P overlays to support Massively
  multiplayer games (MMG)
• Primary contribution of paper
• Architectural (P2P for MMG)
• Evaluative

23
SimMUD -- Introduction
MMG GAME
SCRIBE (Multicast support)
PASTRY (P2P overlay)
24
SimMUD -- Introduction

• Players contribute memory, CPU cycles and
  bandwidth for shared game state
• Persistent user state is centralized
• Example payment information, character
• Allows central server to delegate to peers the
  dissemination and the process of intensive game
  states

25
Distributed Game Design

• GAME STATES
• Game world divided into connected regions
• Regions are controlled by different coordinates

26
Distributed Game Design
27
Distributed Game Design

• Game design based on fact that
• Players have limited movement speed
• Limited sensing capability
• Hence data shows temporal and spatial localities
• Use Interest Management
• Limit amount of state player has access to

28
Distributed Game Design

• Players in same region form interest group
• State updates relevant to group disseminated only
  within group
• Player changes group when going from region to
  region

29
Distributed Game Design

• GAME STATE CONSISTENCY
• Must be consistent among players in a region
• Basic approach employ coordinators to resolve
  update conflicts
• Split game state management into three classes to
  handle update conflicts
• Player state
• Object state
• The Map

30
Distributed Game Design

• Player state
• Single writer multiple reader
• Position change is most common event
• Use best effort multicast to players in same
  region
• Use dead reckoning to handle loss or delay

31
Distributed Game Design

• Object state
• Use coordinator-based mechanism for shared
  objects
• Each object assigned a coordinator
• Coordinator resolves conflicting updates and
  keeps current value

32
Distributed Game Design

• Map
• Maps are considered read-only because they remain
  unchanged during the game play.
• They can be created offline and inserted into the
  system dynamically.
• Dynamic map elements are handled as objects.

33
Game on P2P overlay

• Map game states to players
• Group players objects by region
• Map regions to peers using pastry Key
• Each region is assigned ID
• Live Node with closest ID becomes coordinator
• Random Mapping reduces chance of coordinator
  becoming member of region (reduces cheating)
• Currently all objects in region coordinated by
  one Node
• Could assign coordinator for each object

34
Game on P2P overlay

• Shared state replication
• Lightweight primary- backup to handle failures
• Failure detected using regular game events
• Dynamically replicate coordinator when failure
  detected
• Keep at least one replica at all times
• Uses property of P2P (route message with key K to
  node ID, say N , closest to K)

35
Game on P2P overlay

• Shared state replication (contd..)
• The replica kept at M which is the next closest
  to key K
• If new node T added which is closer to K than
  coordinator N
• Forwards messages to coordinator N until all
  states of K are transferred from N to T
• Takes over as coordinator and N becomes a replica

36
Game on P2P overlay

• Catastrophic failure
• Both coordinator and replica dead
• Problem solved by cached information from nodes
  interested in the area

37
Experimental Results

• Prototype Implementation of SimMud
• Used FreePastry (open source)
• Maximum simulation size constrained by memory to
  4000 virtual nodes
• Players eat and fight every 20 seconds
• Remain in a region for 40 seconds
• Position updates every 150 millisec by multicast

38
Experimental Results

• Base Results
• No players join or leave
• 300 seconds of game play
• Average 10 players per region
• Link between nodes have random delay of 3-100 ms
  to simulate network delay

39
Experimental Results(Base results)
40
Experimental Results(Base results)
41
Experimental Results(Base results)
42
Experimental Results(Base results)

• 1000 to 4000 players with 100 to 400 regions
• Each node receives 50 120 messages
• 70 update messages per second
• 10 players 7 position updates
• Unicast and multicast message take around 6 hops
  (but 50 hops in the worst case)

43
Experimental Results

• Breakdown of type of messages
• 99 messages are position updates
• Region changes take most bandwidth
• Message rate of object updates higher than
  player-player updates
• Object updates multicast to region
• Object update sent to replica
• Player player interaction effects only players

44
Experimental Results

• Effect of Population Growth
• As long as average density remains same,
  population growth does not make difference
• Effect of Population Density
• Ran with 1000 players , 25 regions
• Position updates increases linearly per node
• Non uniform player distribution hurts
  performance

45
Experimental Results

• Three ways to deal with population density
  problem
• Allow max number of players in region
• Different regions have different size
• System dynamically repartitions regions with
  increasing players

46
Experimental Results

• Effect of message aggregation
• Since updates are multicast, aggregate them at
  root
• Position update aggregated from all players
  before transmit
• Cuts bandwidth requirement by half
• Nodes receive less messages

47
Experimental Results
48
Experimental Results
49
Experimental Results

• Effect of network dynamics
• Nodes join and depart at regular intervals
• Simulate one random node join and depart per
  second
• Per-node failure rate of 0.06 per minute
• Average session length of 16.7 minutes (close to
  18 minutes for a FPS game -- Half Life)
• Average message rate increased from 24.12 to 24.52

50
Related Work (2)Neighbor-list Exchange

• Kawahara et al. 2004 (Univ. of Tokyo)
• Fully-distributed
• Nearest-neighbors
• List exchange
• High transmission
• Overlay partition

51
Related Work (3) Mutual Notification Solipsis

• Keller Simon 2003 (France Telecomm RD)
• Links with AOI neighbor
• Mutual cooperation
• Inside convex hull
• Potentially slow discovery
• Inconsistent topology

52
Related Work (4) P2P Hybrid (DHT,
super-nodes,neighbor list exchange MOPAR

• Yu, A. and Vuong, S. T., MOPAR a mobile
  peer-to-peer overlay architecture for interest
  management of massively multiplayer online
  games, In Proceedings of the international
  Workshop on Network and Operating Systems Support
  For Digital Audio and Video (Stevenson,
  Washington, USA, June 13 - 14, 2005). NOSSDAV '05.

53
Related Work (5) Voronoi-based Overlay Network
VON

• SY Hu, JF Chen, TH Chen, VON a scalable
  peer-to-peer network for virtual environments,
  IEEE Network, 2006

54
Design Goals

• Observation
• for virtual environment applications, the
  contents we want are messages from AOI neighbors
• Content discovery is a neighbor discovery problem
• Solve the Neighbor Discovery Problem in a
  fully-distributed, message-efficient manner.
• Specific goals
• Scalable ? Limit minimize message traffics
• Responsive ? Direct connection with AOI neighbors

55
Voronoi Diagram

• 2D Plane partitioned into regions by sites, each
  region contains all the points closest to its site

Neighbors
Region
Site
56
Design Concepts
Use Voronoi to solve the neighbor discovery
problem

• Each node constructs a Voronoi of its neighbors
• Identify enclosing and boundary neighbors
• Mutual collaboration in neighbor discovery

? node i and the big circle is its AOI
enclosing neighbors ? boundary neighbors ? both
enclosing and boundary neighbors ? normal AOI
neighbors ? irrelevant nodes
57
Procedure (JOIN)

• 1) Joining node sends coordinates to any existing
  node
• Join request is forwarded to acceptor
• 2) Acceptor sends back its own neighbor list
• joining node connects with other nodes on the
  list

Joining node
Acceptors region
58
Procedure (MOVE)

• 1) Positions sent to all neighbors, mark messages
  to B.N.
• B.N. checks for overlaps between movers AOI and
  its E.N.
• 2) Connect to new nodes upon notification by B.N.
  

Boundary neighbors
New neighbors
59
Procedure (LEAVE)

• 1) Simply disconnect
• 2) Others then update their Voronoi
• new B.N. is discovered via existing B.N.

Leaving node (also a B.N.)
New boundary neighbor
60
Outline

• Introduction
• Background
• P2P DVEs
• Factors affecting Neighborship Consistency
• Proposed Solutions
• Simulation Results
• Conclusion

61
Factors affecting Neighborship Consistency

• Mobility
• Speed
• AOI-radius
• Node failure
• Possibly causes overlay partition

62
Mobility
63
Overlay Partition (1/2)

• Nodes divide into two or more groups
• Each group can not be aware each other
• May destroy neighborship consistency
• In pure P2P DVE
• May be impossible to detect and recover
• Have to avoid it

64
Overlay Partition (2/2)






65
Outline

• Introduction
• Background
• P2P DVEs
• Factors affecting Neighborship Consistency
• Proposed Solutions
• Simulation Results
• Conclusion

66
Proposed Solutions

• Adaptive AOI
• AOI radius increases with speed increasing
• Detecting critical nodes
• Critical nodes are the nodes in some regions
  where the density is low
• It is used to avoid overlay partition

67
Adaptive AOI
68
Critical Nodes (1/4)
69
Critical Nodes (2/4)

• Periodically detect by node itself
• AOI
  neighbors Neighbor_level
• Average AOI
  neighbors
• If a node detects that its neighbor level is
  lower than a pre-specified threshold, it regards
  itself as a critical node.
• A critical node will perform the backup operation
  to send to all connected neighbors the backup
  message.

70
Critical Nodes (3/4)

• A Backup message
• Sent by critical nodes
• Contains a neighbor list of the critical node
• Stock neighbor list
• Stores nodes from backup message
• Uses to periodically check
• Check operation
• Check all nodes in its stock neighbor list
  periodically
• To discovery missed neighbors

71
Critical Nodes (4/4)
Stock neighbor list of node A
Neighbor list of node A
C
B
A
Neighbor list of node B
C
B
C
72
Outline

• Introduction
• Background
• P2P DVEs
• Factors affecting Neighborship Consistency
• Proposed Solutions
• Simulation Results
• Conclusion

73
Simulation Parameters

• 1000 nodes
• in a 1200-unit by 1200-unit plane
• constant velocity of 5 units per time-step
• AOI radius is 100 units (the number of directly
  connected neighbors is limited to be 20)
• Clustered distribution nodes cluster into three
  groups in the first 300 steps and can move
  randomly after 300 steps.
• Random distribution nodes can move randomly in
  all simulation steps.

74
Mobility (1/2)
75
Mobility (2/2)
76
Node Failure (1/8)
77
Node Failure (2/8)
78
Node Failure (3/8)
79
Node Failure (4/8)
80
Node Failure (5/8)
81
Node Failure (6/8)
82
Node Failure (7/8)
83
Node Failure (8/8)
84
Outline

• Introduction
• Background
• P2P DVEs
• Factors affecting Neighborship Consistency
• Proposed Solutions
• Simulation Results
• Conclusion

85
Conclusion

• Address two factors
• Node mobility
• Node failure
• Improve neighborship consistency by
• Adaptive AOI buffer
• Critical node detection

86

• QA