A Network Communication System for Large-Scale, High-Speed Games

1
A Network Communication System for Large-Scale,
High-Speed Games
Ashwin Bharambe, Jeff Pang, Srini Seshan, Xinyu
Zhang Carnegie Mellon
John Douceur, Jacob R. Lorch, Thomas
Moscibroda Microsoft Research
2
P2P Games
3
P2P Games
4
P2P Games
Goal 1000 players in the same battle at the same
time
5
Problem 1 Insufficient Capacity

• Update Size 35 bytes/frame
• Update Rate 20 frames/sec
• ? Need 14 Mb/s upload BW per peer
• Real P2P topologies
• 800 Kb/s to 1.5Mb/s per peer (on average)

6
Solution 1 Subscriber Set

• Players can only focus on 5 objects at the same
  time
• Each player divides other peers into two classes
• Subscriber Set players focused on me
• Update every frame
• Best Effort everyone else
• Update with guidance once per second

• Who goes in my Subscriber Set?
• Attention(i,j) how much a player i is focused
  on player j
• Attention(i,j) sent from peer i to peer j
  periodically
• Peers with the highest Attention(i,j) go in peer
  js Set

7
Solution 1 Subscriber Set

• Who goes in my Subscriber Set?
• Attention(i,j) how much a player i is focused
  on player j
• Attention(i,j) sent from peer i to peer j
  periodically
• Peers with the highest Attention(i,j) go in peer
  js Set

8
Projected Scalability
With Donnybrook
Without Donnybrook
9
Problem 2 Bandwidth Heterogeneity

• Best Effort updates still scale quadratically
• Still need 1 Mb/s per peer
• ? Limited by low bandwidth peers

10
Solution 2 Guidance Forwarding

• Static Overlay Multicast
• Need low latency ? limit tree depth to 2
• Cant queue updates ? have senders send to
  forwarders at 20 updates/sec
• Generally have centralized match making ?
  (replicated) centralized coordinator picks
  forwarders

11
Projected Scalability
12
Problem 3 Interest Heterogeneity

• Some players more interesting than others
• More players focus on them
• They need to send more frequent updates
• Subscriber sets change dynamically (60 turnover
  per second!)
• ? Cant use static overlay multicast
• Limited by BW of popular players

13
Solution 3 Update Dissemination

• Alterative 1
• Static forwarder groups
• Alternative 2
• Centralized forwarder assignment
• Alternative 3
• Uncoordinated forwarder assignment

I need BW!
Use B
A
B
A
Pick random
B
14
Projected Scalability
15
Evaluation

• Detailed packet level simulation
• Empirical simulation models
• E2E latency
• Jitter
• Access Bandwidth
• Packet Loss
• Quake III-like CTF workload
• ... many other details
• But no churn

16
Alternative Comparison

• Metric updates delivered roughly on time
• updates delivered too late are useless
• 3-5 loss rate acceptable for games

17
Performance Scaling
18
Summary