ShapeShifter: Scalable, Adaptive End-System Multicast

1
ShapeShifter Scalable, Adaptive End-System
Multicast

• John Byers, Jeffrey Considine, Nicholas
  Eskelinen, Stanislav Rost, Dmitriy Zavin
• Listed alphabetically

2
Problem

• Problem efficient delivery of popular bulk
  content
• Existing Approaches
• Single-source unicast
• Forward caching/Content Delivery Networks
• Reliable IP Multicast
• End-system multicast
• Our approach
• Improving end-system multicast through the use of
  forward error correction and better topologies

3
Network-supported (IP) Multicast

• Optimal solution duplicates and disseminates
  data only when necessary
• Relies on network support in the real world, IP
  Multicast lacks deployment
• Scalability concerns per-group accounting and
  topology management do not scale due to limited
  router resources
• Reliability many proposals, few solutions

4
End-System Multicast

• Does not rely on network support builds and
  manages a virtual, overlay topology of unicast
  links on top of the networks physical topology
• Flexibility optimization of the tree according
  to a richer set of metrics (perhaps specified by
  the application), ability to change topology
  on-demand
• Improved scalability end-systems are richer
  than routers in terms of dedicated resources
• Problems increased network resource
  requirements compared to IP Multicast,
  non-optimal mapping of the virtual topology onto
  physical topology

5
Related Work

• Narada/End-System Multicast Build and maintain
  a mesh of low-latency unicast links and use its
  minimal spanning tree for distribution. Also
  showed costs relative to IP Multicast are not
  excessive.
• Overcast A core group of well-placed nodes uses
  end-system multicast to distribute bulk content
  internally, in order to eventually provide it to
  clients. A node chooses a parent based on
  bandwidth through the candidate nodes using the
  number of network hops as a tie breaker.

6
Improvements in ShapeShifter

• Erasure codes improved overlay management, more
  connected graph structure, increased adaptivity
• Scalable group management a node need only be
  aware of a small portion of the graph but
  achieves coverage through continuous discovery
• Measurements metrics crucial to optimization of
  the overlay graph, such as shared-link congestion
  (refer to Khaleds presentation)

7
Forward Error Correcting Codes

• FEC codes a well-known solution to dealing with
  packet loss without using feedback instead of
  retransmitting packets, redundant packets are
  sent combining the original packets to recover
  from losses. e.g. x, y, xy, a, b, abx
• Efficient codes instead of the traditional slow
  Reed-Solomon codes, we use a variant of Tornado
  codes. This allows fast decoding while only
  requiring a small number of extra packets.
• Strategy the original server sends out FEC
  packets along the end-system multicast graph (à
  la Digital Fountain).

8
Recoding

• Problem
• Correlation client nodes may have a high degree
  of correlation in information received due to
  common sources
• Duplication given correlation, duplicate
  packets received from client nodes can be
  ineffectual
• Solution
• Recoding nodes blend received packets to
  generate new, high utility symbols for other
  nodes
• Beyond trees recoding allows non-tree
  topologies since duplication is avoided

9
Uncorrelated Loss Compensation
Loss Rate30
Loss Rate5

• A neighbor node with greater throughput may
  supplement the flow of content to another node,
  circumventing the problematic physical link.

10
Download from Multiple Nodes in Parallel
1 MB/s
1 MB/s
1 MB/s
1 MB/s
Non-uniform dissemination of content

• Well-connected
• newcomer scenario

11
Resilience To Partitioning
Partition avoidance through discovery and
awareness of multiple candidates

• Collaborative
• Reconstruction

12
Future Work

• Implementation underway
• More analysis of
• Codes and correlation
• Graph management
• Security issues
• Testing
• WAN, simulated and real
• Mobile environments
• Extensions to streaming media