Structure Management for Scalable Overlay Service Construction

1
Structure Management for Scalable Overlay Service
Construction

• Kai Shen
• Department of Computer Science
• University of Rochester

2
Motivations

• Structure set of overlay links that data flow
  through
• link selection is important for performance
• low latency, high bandwidth,
• link selection can be costly
• large selection base
• high cost of link property probing
• Existing link selection are mostly
  service-specific
• unicast overlay path selection (e.g., RON)
• end-system multicast (e.g., Narada, Overcast, and
  NICE)
• substrate-aware DHT (e.g., CAN, Chord, and Pastry)

3
A Common Structure Layer

• A service-independent structure layer Saxons
• Substrate-Aware Connectivity Support for Overlay
  Network Services
• Potential benefits
• simplify service design implementation
• modularity
• allow runtime overhead sharing across multiple
  services (not yet addressed in this paper)
• Questions
• Performance?
• How can services utilize a common structure layer?

4
Design Objectives

• A common structure layer must meet the quality
  requirements of a wide range of services
• overlay latency
• hop-count distance
• overlay bandwidth on the shortest path, or on
  the widest path
• Best effort no guarantee on structure quality
• Other design objectives
• scalability
• extremely simple API
• stability

5
Saxons Design Overview
Node bootstrap
Partition detection repair
Structure quality maintenance
Like property probing
Membership management

• Scalability
• functional-symmetric architecture
• per-node management overhead only depends on the
  number of attached links not the overlay size
• do not maintain complete system view at any
  single node

6
Structure Quality Maintenance

• Configurable node degree range ltdactive, dtotalgt
• High-level description
• periodically check random links replace existing
  ones if better
• employ adjustment threshold to avoid oscillation
• Three quality maintenance approaches
• AllShort maintain all short links
• tend to create grid-like structure ? high
  hop-count distance ( vs. O(log
  n) produced by random structure)
• ShortLong Ratnasamy et al 2002 half short,
  half random links
• ShortWide half short, half wide links (high adj.
  threshold)

7
Random Membership Subsets

• Membership subset service
• dynamically changing subsets with uniform
  randomness
• for tree-like overlay structures Kostic et al
  2003
• Each node maintains a member-subset
• Periodically, each node informs its neighbors a
  randomly selected update-set
• To ensure equal representation
• the node itself is selected into each update-set
  at probability
• (update-set size) / (overlay size)

8
Implementation

• Saxons runtime prototype
• stand-alone daemon communicating with local
  overlay application instances through IPC or
• linked and run inside the application process
  space
• Basic API for overlay applications
• directly query the Saxons runtime for directly
  attached links
• provide a callback function to the Saxons
  runtime, invoked by Saxons whenever neighbor
  links change
• Advanced API
• control protocol parameters

9
Link Bandwidth Measurement

• Requirements robustness, overhead, accuracy
• Many techniques were proposed in the past
• Our goal a simple scheme that works
• based on the packet bunch CarterCrovella 1996

4.8MB/measurement
2
10
480KB/measurement
All-to-all measurement results on 61 Planetlab
sites
1
Bandwidth (in Mbps)
10
0
10
10
Evaluation

• Simulation
• evaluation on large-scale overlays (up to 12,800
  nodes)
• use 3 kinds of Internet backbones
• BGP routing dumps from NLANR and RouteViews
• synthetic backbones generated using Inet and
  GT-ITM
• based on all-to-all measurement results from
  NLANR AMP
• PlanetLab experimentation
• performance assessment on a particular real-world
  environment
• most nodes are on Internet2
• most nodes have 10Mbps bandwidth limit

11
Overall Structure Quality(55 PlanetLab sites)
CDF of overlay path latency
CDF of widest path bandwidth
100
100
80
80
60
60
40
40
20
20
0
0
1.25
2.5
5
10
20
40
80
0
100
200
300
Latency (in millisecond)
Bandwidth (in Mbps)

• All three schemes outperform Random by over 18
  on latency
• ShortWide provides gt10Mbps bandwidth for over 3
  times more site pairs

12
Structure Stability During Node Churn(55
PlanetLab sites)
Overlay link adjustment during node join/departure
60

5 sites fail
50

Site 1 rejoins

All sites complete bootstrap

Site 2 rejoins
40

Site 3 rejoins
Adjustment per hour per node
30

Site 4 rejoins

Site 5 rejoins
20
10
0
0
20
40
60
80
100
120
Time after all sites have joined (in minutes)

• Five nodes fail at the 60th minute and rejoin one
  by one at three-minute intervals
• Small disturbances as the result of node
  join/departure

13
Saxons-based Overlay Multicast(52 PlanetLab
sites)

• Compared with Random, Saxons-based multicast
  provides small-loss (lt5) data delivery to over 4
  times more receivers
• Performance close to Independent Direct Unicast

14
Related Work

• Structure-first overlay multicast Narada Chu et
  al 2000
• Utilities/infrastructures for overlay service
  construction
• Topology probing Nakao et al 2003, MACEDON
  Rodriguez et al 2004
• Service-specific link selection
• Unicast routing RON Andersen et al 2001,
  Savage et al 1999
• Multicast routing Narada Chu et al 2001,
  Overcast Jannotti et al 2000, NICE Banerjee et
  al 2002
• Substrate-aware DHT Binning Ratnasamy et al
  2002, Brocade Zhao et al 2002, Pastry Castro
  et al 2002
• Related work for various Saxons components
• Membership management Kostic et al 2003,
  lpbcast Eugster et al 2003
• Bandwidth measurement CarterCrovella 1996,
  Paxson 1997, LaiBaker 2000
• Scalable latency estimation Hotz 1994, IDMaps
  Francis et al 1999, GNP NgZhang 2002

15
Conclusion and Future Work

• Saxons - a common structure management layer
  supporting scalable overlay service construction
• simplify the construction of many overlay
  services
• still allow many services (e.g., overlay
  multicast, Gnutella-style query flooding, DHT) to
  achieve high-performance
• Future work
• support runtime overhead sharing when overlay
  nodes host multiple services
• best effort structure quality ? soft structure
  quality guarantee