Scattercast: Taming Internet Multicast An InfrastructureService Architecture for Internet Broadcast

1
Scattercast Taming Internet MulticastAn
Infrastructure-Service Architecture for Internet
Broadcast Distribution
11/22/2000
2
Internet broadcasting is hereAlmost

• The web has revolutionized the Internet
• Next step Internet broadcasting
• Internet radio/TV
• Software distribution
• Can the Internet handle large-scale broadcasting?
• Access bandwidth problem is almost solved
• But what about efficient distribution to millions
  of viewers?

3
The Problem
I want to say a special welcome to everyone
thats climbed into the Internet tonight and has
got into the MBoneand I hope it doesnt all
collapse. Mick Jagger (Nov 18, 1994)
The huge numbers of people unable to watch
Wednesdays live Victorias Secret fashion show
webcast showed the Internet isnt ready to handle
mass market video. cnet News (Feb 5, 1999)

• Traditional unicast model does not scale
• IP multicast is not the right solution

4
10,000 foot view

• Our solution scattercast
• Move complexity up the protocol stack
• Explicit infrastructure support
• Application-level multicast distribution
• Application-specific customization

Scattercast Infrastructure-service-based
transport as opposed to global IP multicast
5
Outline

• Motivation
• Introduction The Scattercast Architecture
• Gossamer Application-level Multicast
• Semantic Transport Application-aware
  customization
• Summing up

6
The Problem

• Traditional unicast model does not scale
• Millions of clients ? server and network meltdown

7
Traditional solution IP Multicast

• IP Multicast to the rescue

• Global broadcast distribution primitive
• Source sends single stream
• Routers split stream towards all clients

8
Problems with IP Multicast

• Complex network protocol
• No access control
• Scaling issues state explosion and inter-domain
  routing
• Difficult to manage and debug
• Heterogeneity
• Single stream cannot satisfy all clients/networks
• Different applications have different
  requirements
• Reliable multicast
• Much harder than TCP
• No scalable loss recovery, congestion control

• End-to-end argument fails for multicast
• network layer is no longer simple and robust

9
Scattercast Broadcasting as an Infrastructure
Service
Unicast connections
Infrastructure proxies (SCXs) provide the
broadcast service
10
Benefits of this approach

• Localize hard multicast problems
• Bandwidth allocation, congestion control, loss
  recovery are tractable
• Simplify network layer via intelligent
  infrastructure
• No inter-domain multicast routing required
• Impose access restrictions within SCXs
• Leverage well-understood wide-area unicast
  protocols
• Incorporate app-specific semantics within SCXs to
  address heterogeneity
• App-specific reliability and data scheduling
• On-the-fly content and bandwidth adaptation

11
New challenges

• How do you distribute data efficiently across the
  infrastructure?
• Gossamer Application Level Multicast
• How do you incorporate application-specific
  intelligence into the distribution
  infrastructure?
• Application-customizable scattercast transport
• How do you manage the service and ensure fault
  tolerance, availability, and scalability of SCXs?
• Cluster-based SCX implementation

12
Outline

• Motivation
• Introduction The Scattercast Architecture
• Gossamer Application-level Multicast
• Semantic Transport Application-aware
  customization
• Summing up

13
Overview

• The problem
• Source multicasts data to local SCX
• How do SCXs forward data across wide area
  efficiently

14
Overview

• Our solution Gossamer
• Dynamically build overlay distribution tree
  across SCXs with unicast interconnections
• Forward data from SCX to SCX on top of this tree

15
Goals

• Minimize latency from source to all receivers
• SCXs are not routers ? Overlay tree not as
  optimal as IP multicast
• Optimize overlay to reflect underlying Internet
  topology
• Limit number of duplicate packets traversing any
  physical Internet link
• Each SCX transmits to handful of nearby SCXs ?
  Restrict degree of each SCX based on its
  bandwidth capabilities

16
A Possible Approach

• Construct a tree directly
• Each SCX explicitly picks its appropriate parent

• Problems with this approach
• Fragile single edge/node failure causes
  partition
• Requires loop avoidance and detection mechanisms
• For source-rooted trees, unnecessary duplication
  of mechanism

17
Our Approach

• 2-step process
• Construct strongly connected graph mesh

• Run application-level routing protocol on top of
  mesh
• build source-rooted shortest path spanning trees

• Advantages
• Redundancy ? resilience against partitions

• Built-in loop detection and avoidance
• Re-use for multiple trees

18
SCX Discovery

• Bootstrap using list of well-known rendezvous
  SCXs
• Gossip-style discovery
• Pick random SCX Xj send it our membership list
• Xj merges this into its own list
• Xj responds with part of its own list
• Gradually all SCXs discover each other

Summary well-known rendezvous gossip to
disseminate session membership
19
Mesh Construction

• Set up connections with up to k other SCXs
• k degree restriction
• Periodically probe a random SCX, Xj
• Measure unicast distance to Xj
• Use local optimization algorithm to determine
  suitability for picking as a neighbor
• If Xj has better route towards source than a
  current neighbor, then replace that neighbor with
  Xj

Summary Local optimization based on unicast
distances to choose mesh neighbors
20
Application-level Routing

• Variant of distance vector routing
• shortest path routing protocol
• routing table entries only for source SCXs
• to detect loops, store entire path in routing
  table
• Build distribution trees from routing tables
• source-rooted trees
• reverse shortest path
• forward data using reverse path forwarding

Summary Shortest path routing to build
source-rooted trees
21
Evaluation

• Metric for evaluation
• C sum of path lengths between source SCX and
  all other SCXs

• Cstar sum of unicast distances between source
  SCX and all other nodes

• Cost Ratio C/Cstar ? 1.0
• Cost Ratio of 1.7 ? average latency using
  Gossamer is 1.7
  times worse than unicast

edge weight unicast
distance
22
Variation of Cost Ratio with Session Size
Cost ratio remains low (size increases
23
Time to Stability
Most mesh changes occur early on in the protocol
24
Packet Duplication Overhead
Most heavily loaded link for Gossamer 14 copies
Most heavily loaded link for unicast 99 copies
Load on physical links is lower for Gossamer than
for vanilla unicast
25
Gossamer Summary

• Application-level multicast is feasible
• Mesh routing approach results in stable overlay
  distribution structure
• Gossamer is but one approach for
  application-level multicast

26
Outline

• Motivation
• Introduction The Scattercast Architecture
• Gossamer Application-level Multicast
• Semantic Transport Application-aware
  customization
• Summing up

27
Overview

• Different applications have different transport
  requirements
• Reliability, bandwidth management, congestion
  control, etc.
• One-size-fits-all solution will not work
• Single data stream cannot satisfy all
  heterogeneous clients

• Our solution Application-awareTransport
  Framework
• Delivery of information rather than the
  representation of the information
• Expose underlying overlay topology to
  applications
• Allow applications to define their own forwarding
  policies

28
An Example Application

• Web-page Dissemination
• Distribute web pages for on-line presentations
• Requires eventual reliability
• High-bandwidth image data
• Four levels of customization
• Customizable data forwarding
• Customizable data reliability
• Transmission scheduling
• Data transformation

29
Customizable Data Forwarding

• Expose underlying overlay topology to transport
  layer
• Local view of the distribution tree upstream
  link towards source list of downstream links
• Allows transport layer to assist in end-to-end
  reliability protocol
• Transmit nacks/acks upstream towards source
• Transmit data/retransmissions towards receivers

30
Customizable Data Reliability

• Reliability constraints vary
• Ordered/unordered delivery, best effort,
  periodically updating data
• Different types of reliability within the same
  app
• Apps define their own reliability policies
• Application Data Units (ADUs)
• Group related ADUs into containers
• e.g. html in one container, images in another
• Assign reliability policies to containers
• e.g. ignore losses in image containerallow
  out-of-order delivery of ADUs

31
Customizable Transmission Scheduling

• Customized bandwidth management
• Buffer data to avoid congestion
• Notify upstream SCX to slow down
• Prioritize important ADUs over others

HTML high priority
images low priority
32
Customizable Data Transformation

• Transform ADUs on the fly
• Bandwidth adaptation discard redundant
  information
• Format conversionadapt content to suit client
  devices
• Feedback from scheduler drives transformation
  decisions
• e.g. convert images to P-JPEGprioritize base
  scan
• limit P-JPEG size based on available bandwidth

33
Real Applications

• Electronic whiteboard
• Shared drawing space
• Adaptive reliability

• Whiteboard for PalmPilot
• Extreme client heterogeneity
• Split application PalmPilot app is simple
  smarts in SCX

• Streaming MP3 broadcast server
• Radio over the Internet
• Interface to standard clients e.g. WinAmp

34
Outline

• Motivation
• Introduction The Scattercast Architecture
• Gossamer Application-level Multicast
• Semantic Transport Application-aware
  customization
• Summing up

35
Scattercast Broadcasting as an Infrastructure
Service

• End-to-end is not the right answer
• Use intelligent infrastructure to simplify the
  network layer
• Divide-and-conquer localizes hard problems
• Use multicast only in local area where it is
  tractable robust unicast across wide-area
• Application-level intelligence is crucial
• Adapt to heterogeneity by leveraging application
  hints in transport protocol

36
The Longer TermEvolving Scattercast

• Scaling issues as session size increases, mesh
  takes longer to stabilize

• Multi-level scattercast

• Provides incremental expansion path
• Deploy in core infrastructure push towards edges
  as system scales

37
Conclusions

• Contributions
• A new approach for Internet content distribution
• An architecture for leveraging infrastructure
  support in transport protocols
• Real applications that work in heterogeneous
  environments
• Infrastructure services are a powerful approach
  for building network protocols
• Can solve problems that are intractable at the
  network layer

38
The Longer TermBeyond Scattercast

• Pervasive Application-aware Internet Middleware
• Generalize scattercast concepts
• Middleware layer that sits on top of IP and
  provides application-specific computation
• New applicationse.g. global network proximity
  service application-aware congestion
  management

39
Customizable Data Reliability

• Reliability constraints vary
• Ordered/unordered delivery, best effort,
  periodically updating data
• Different types of reliability within the same
  app
• Apps define their own reliability policies
• Application Data Units (ADUs)
• Group related ADUs into containers
• e.g. html in one container, images in another
• Assign reliability policies to containers
• e.g. ignore losses in image containerallow
  out-of-order delivery of ADUs

40
What went wrong

• End-to-end argument fails
• Keep network layer simple robust
• Layer richer services on top
• Not the right answer for multicast
• Instead, move complexity up the protocol stack
• Define multi-point delivery as an infrastructure
  service

41
Cost Ratio Distribution
Bell curve distribution Avg cost ratio 1.7 ?
0.016 (95 confidence interval)
42
SCX Discovery Progress
43
Cost Ratio v/s Node Degree
44
Scaling Behavior
45
Putting it all together

• Basic mechanisms
• ALF-based reliability
• Customizable transmission scheduler
• Custom transformation engines
• Isolate application policies in a policy module
  inside the SCX
• This module coordinates the flow and adaptation
  of data through the SCX

46
Related Work

• Application-level multicast
• EndSystem Multicast
• Yallcast
• AMRoute
• Self-organizing protocols
• Self-organizing Transcoders (SOT)
• Group Formation Protocol (GFP)
• Infrastructure proxies for heterogeneity
• TranSend/TACC
• Active Services/MeGa