The SAHARA Project: Composition and Cooperation in the New Internet

1
The SAHARA ProjectComposition and
Cooperationin the New Internet

• Randy H. Katz, Anthony Joseph, Ion Stoica
• Computer Science Division
• Electrical Engineering and Computer Science
  Department
• University of California, Berkeley
• Berkeley, CA 94720-1776

2
Presentation Outline

• Service Architecture Opportunity
• SAHARA Project and Architecture
• Routing as Service Composition
• Summary and Conclusions

3
Presentation Outline

• Service Architecture Opportunity
• SAHARA Project and Architecture
• Routing as Service Composition
• Summary and Conclusions

4
The New Opportunity

• New things you can do inside the network
• Connecting end-points to services with
  processing embedded in the network fabric
• Not protocols but agents, executing in places
  in the network
• Location-aware, data format aware
• Controlled violation of layering necessary!
• Distributed architecture aware of network
  topology
• No single technical architecture likely to
  dominate think overlays, system of systems

5
Services in Converged Networks
6
Services in Converged Networks
7
Presentation Outline

• Service Architecture Opportunity
• SAHARA Project and Architecture
• Routing as a Service Composition
• Summary and Conclusions

8
The SAHARA Project

• Service
• Architecture for
• Heterogeneous
• Access,
• Resources, and
• Applications

9
Composition ScenarioUniversal In-box

• Message type (phone, email, fax)
• Access network (data, telephone, pager)
• Terminal device (computer, phone, pager, fax)
• User preferences rules
• Message translation storage

Separate end device andnetwork from
end-to-endcommunications serviceindirection
via compositionof translators with access
10
SAHARA Focus

• New mechanisms, techniques for end-to-end
  services w/ desirable, predictable, enforceable
  properties spanning potentially distrusting
  service providers
• Tech architecture for service composition
  inter-operation across separate admin domains,
  supporting peering brokering, and diverse
  business, value-exchange, access-control models
• Functional elements
• Service discovery
• Service-level agreements
• Service composition under constraints
• Redirection to a service instance
• Performance measurement infrastructure
• Constraints based on performance, access control,
  accounting/billing/settlements
• Service modeling and verification

11
The Network Effect

• Creation and deployment of new services
• Achieving desirable end-to-end properties,e.g.,
  by controlling the end-to-end path
• Deploying computation and storage INSIDE the
  network
• BUT new networks are expensive evolving existing
  networks virtually impossible
• E.g., Cost of 3G licenses and networks
• Even if I had 1 billion and set up 1000s of
  locations, I could never in my network have a
  completely ubiquitous footprint.Sky Dayton,
  founder of Boingo
• QoS IntServ, DiffServ New Function Multicast,
  
• Approaches
• Composition, Overlays, Peering
• Cooperation, Brokering

12
Internet Connectivity and Processing
13
Interconnected WorldAgile or Fragile?

• Baltimore Tunnel Fire, 18 July 2001
• The fire also damaged fiber optic cables,
  slowing Internet service across the country,
• Keynote Systems says the July 19 Internet
  slowdown was not caused by the spreading of Code
  Red. Rather, a train wreck in a Baltimore tunnel
  that knocked out a major UUNet cable caused it.
• PSINet, Verizon, WorldCom and AboveNet were some
  of the bigger communications companies reporting
  service problems related to peering, methods
  used by Internet service providers to hand
  traffic off to others in the Web's
  infrastructure. Traffic slowdowns were also seen
  in Seattle, Los Angeles and Atlanta, possibly
  resulting from re-routing around the affected
  backbones.
• The fire severed two OC-192 links between
  Vienna, VA and New York, NY as well as an OC-48
  link from, D.C. to Chicago. Metromedia routed
  traffic around the fiber break, relying heavily
  on switching centers in Chicago, Dallas, and
  D.C.

14
Internet Routing Realities

• Provider-customer vs. peer-to-peer
• Relationships established by BGP protocol
• Charging based on traffic volumes

ISP A
Hot Potato Routing
ISP B
15
Mobile Virtual Network OperatorComposition and
Cooperation
16
PeeringPolicy-Based Routing

• Multi-homing
• Reliability of network connectivity
• Traffic discrimination

Primary Transit Network
End Network
Berkeley Campus
Dorm Traffic
Alternative Transit Network
Research Traffic
Fail-over
Peer Network
Peer Network
Peer Network
Peer Networks
CalREN
17
OverlaysCreating New Interdomain Services

• Deploy new services above the routing layer
• E.g., interdomain multicast management and
  peering
• E.g., alternative connectivity for performance,
  resilience

Isolated Intra-cloud service
Traditional unicast peering
Steve McCanne
18
Wireless ISP Composition
Billing, ECommerce Authentication Inter-site
Mobility
Full Service Network Operator
Premises-based Access
19
Layered Reference Modelfor Service Composition

• Connectivity Plane
• End-to-end network with desirable properties
  composed on top of commodity IP network
• Enhanced Links Paths QoS and protocol
  verification within and between connectivity
  service providers
• Applications Plane
• Services strategically placed and actively
  managed within the network topology
• Applications and Middleware Services end-client
  oriented vs. infrastructure oriented

20
Layered Reference Model for Service Composition
End-User Applications
Applications Services
Application Plane
Middleware Services
End-to-End Network With Desirable Properties
Enhanced Paths
Connectivity Plane
Enhanced Links
IP Network
21
Presentation Outline

• Service Architecture Opportunity
• SAHARA Project and Architecture
• Routing as Service Composition
• Summary and Conclusions

22
Routing as a Composed Service

• Routing as a Reachability Service
• Implementing paths between composed service
  instances,e.g., links within an overlay
  network
• Multi-provider environment, no centralized
  control
• Desirable Properties
• Trust verify believability of routing
  advertisements
• Agility converge quickly in response to global
  routing changes to retain good reachability
  performance (e.g., latency)?
• Reliability detect service composition path
  failures quicklyto enable fast recomposition to
  maintain reachability
• Scalability and Interoperability Adapt protocols
  via processing at impedance matching points
  between administrative domains

23
Characterizing the Internet Hierarchy from
Multiple Vantage Points

• Customer-Provider Relationships
• Customer pays provider for Internet access
• AS exports customers routes to all neighbors
• AS exports providers routes only to its
  customers
• Peer-to-Peer Relationships
• Peers exchange traffic between their customers
• Free of charge (assumption of even traffic load)
• AS exports a peers routes only to its customers

Sharad Agarwal. Lakshmi Subramanian, Jennifer
Rexford
24
Knowing These Relationships Matters!

• Useful for
• Placement of servers for content distribution
• Selection of new peers or providers for an AS
• Analyzing convergence properties of BGP
• Installing route filters to protect against
  misconfiguration
• Understanding basic structure of the Internet
• Knowing the AS graph is Not Enough
• Interdomain routing is not shortest-path routing
• Some paths not allowed (e.g., transit through a
  peer)
• Local preference of paths (e.g., prefer customer
  path)
• Node degree does not define the Internet
  hierarchy
• Need to Know Relationship between AS Pairs

25
Revealed Structure

• Peer-peer relationships hard to infer
• Mislabeling peer-peer edge as provider-customer
  does not change valid path into invalid
• Heuristics to detect peer-peer edges
• Some AS pairs unusually related
• Siblings providing mutual transit
• Backup relationship for connectivity under
  failure
• Misconfiguration of conventional relationship
• Detect such cases by analyzing invalid paths
• Access to large path set is hard
• Exploit BGP routing tables from multiple vantage
  points (10 public BGP tables)

26
Policy Management for BGP

• Integrate BGP with a new Policy Agent control
  plane
• Improved BGP convergence through explicit fail
  over policies
• Constrained routing for performance or trust
  reasons
• Traffic discrimination, low quality vs. high
  quality connectivity or fair use issues
• Load balancing outbound and inbound flows for
  multi-homed ASs
• Sharad Agarwals Ph.D. thesis, currently
  interning at Sprint ATL

27
Agility in Response to Route ChangesInternet
Converges Slowly

• Convergence Times Labovitz et al.
• Theory O(n!) (n number of ASes)
• Practice linear with the longest backup path
  length
• Measurement up to 15 minutes
• Why so slow?
• BGP protocol effects path exploration
• Route flap damping!?
• Delay convergence of relatively stable routes
• Unexpected interaction between flap damping and
  convergence

Morley Mao, Ramesh Govindan, George Varghese
28
How Does Flap Damping Work?

• RFC2439
• For each peer, per destination, keep penalty
  value, increase it for each flap
• Flap is a route change
• Penalty decays exponentially
• Parameters
• Fixed Penalty increment
• Configurable half-life, suppress-,
  reuse-threshold, max suppressed time

29
A Better WaySelective Route Flap Damping

• Flaps happen because of certain topologies among
  routers, causing triggered announcements and
  withdrawalsthese are not toy scenarios
• Approach ignore flap sequences indicating path
  explorationthese are likely to trigger more
  changes in near future
• In essence, we redefine what constitutes a flap
• From any route change is considered a flap to
  must alter direction of route preference value
  change, relative to flaps
• Flaps due to withdrawal increasing ASPath
  lengths, route value keeps decreasing
• Morley Mao Ph.D. dissertation, currently
  interning at ATT Labs

30

• Stability achieved through flap damping RFC2439
• BUT unexpectedflap damping delaysconvergence!

Topology clique of routers

• Selective flap damping
• Duplicate suppression ignore flaps caused by
  transient convergence instability
• Eliminates undesired interaction without
  sacrificing stability

31
Trusting the Routing InfrastructureBGP Route
Verification

• BGP protocol vulnerable
• Single misconfigured router can cause long
  outages
• Malicious routers can cause larger damage
• Pretend to be a genuine end-host!!!
• Misroute or sniff on traffic
• Potential collusion with other malicious nodes?
• Verify BGP routes without PKI-based
  authentication?
• Secure-BGP, tier-1 ISP proposal, yet to be
  deployed
• Assumed an Internet wide PKI with ICANN as root!

32
ApproachDetection and Containment

• Misconfiguration affects reachability
• Roughly 6 of misconfigurations cause
  reachability problems Mahajan02
• Passive TCP-probing modified nodes watch TCP
  traffic to detect reachability problems
• No modifications to BGP, incrementally
  deployable, but ineffective for detecting
  malicious hosts
• Contain malicious nodes
• Without authentication, cant distinguish between
  genuine and malicious hosts
• Two BGP enhancements--hash chains, loop-testing
• Avoid routes through nodes (misconfigured/maliciou
  s) affecting routes to multiple destinations
• Lakshmi Subramanian Ph.D. Dissertation

33
Overlay Approach for Achieving Desirable
Performance OverQoS

• Embed QoS functionality in Internet via overlays
• Overlay nodes implement QoS functions
• No support needed from IP routers
• Virtual Links
• Underlying path between two OverQoS routers
• Characterized by three time-varying parameters
• Available bandwidth, b(t), using fairness
  criterion(e.g., N TCP flows) or by explicit SLA
  with ISP
• Loss rate, p(t)
• Delay, d(t)
• Challenges
• Nodes not connected to congested points, have no
  control on cross-traffic, cannot avoid losses
  (reducing sending rate doesnt help!)

Lakshmi Subramanian, Hari Balakrishnan, Ion Stoica
34
Architecture
AS
AS
AS
AS
IP
IP
IP
IP
Virtual links
AS
AS
AS
OverQoS routers
35
Controlled-Loss Virtual Link (CLVL)

• Control losses if you cant avoid them
• Aggregate a set of flows along a virtual link in
  a bundle
• Protect the bundles traffic against losses
• Redistribute b/w and loss across flows in a
  bundle at entry node
• Two parameters
• Statistical bound on loss rate, q (lt p
  typically ltlt p)
• Capacity, c(t), possibly time-varying
• Can prove if offered load lt c(t), then loss rate
  lt q
• c provided in two ways
• Implicit b is bundles bandwidth c is some part
  of b
• Explicit via provisioning in underlying Internet
  path

Flow 1
b(t), p(t)
Flow 2
Flow n
OverQos Node
36
Reliability in Wide-AreaService Composition
Text to audio

• Wide-area/multi-provider composition
• Fast recovery improves service availability

Text to audio

• gt 15 s outage
• BGP recovery much worse! Labovitz00

• Detect recover from failures via service
  replicas
• Aggressive heartbeat msgs
• Quick detection (2 s)
• Scalable messaging for recovery (1000s of
  clients)
• Load balancing slack service provisioning to
  handle fast path fall-over

• End-to-end recovery in 3.6 s 2 s detect, 600 ms
  signaling, 1 s state restoration

Wide-area Experiment UCB, Berk. (Cable), SF
(DSL), Stan., CMU, UCSD, UNSW (Aus), TU-Berlin
(Germany)
Bhasker Raman
37
Scalability and Interoperability Multicast
Broadcast Federation
Source
Broadcast Domains
CDN

• Compose non-interoperablem/c domains to provide
  e2e m/c service
• IP and App-layer protocols
• Overlays of Broadcast Gateways (BGs)
• Peering between domains
• Internal m/c inside domain
• Clustered gateways for scalability across domains
• Independent data flows and control flow

IP Mul
SSM
Clients
BG
Peering
Data

• Implementation
• Linux/C event-driven program
• Customizable interface to local multicast (700
  lines)
• 1 Gbps BG thruput with 6 nodes
• 2500 sessions with 6 nodes

Mukund Seshadri, Yatin Chawathe
38
Presentation Outline

• Service Architecture Opportunity
• SAHARA Project and Architecture
• Routing as Service Composition
• Summary and Conclusions

39
SAHARA Project

• Evolve Internet architecture to better support
  multi-network/multi-service provider model
• Dynamic environment, large numbers of service
  providers service instances
• Achieve desirable properties across multiple,
  potentially distrusting (Internet) service
  providers
• Exploit PlanetLab infrastructure to construct
  wide-area prototype
• Routing as a composed service
• Trust BGP Verification/Detection Containment
• Agility Fast Convergence
• Reliability Keep-Alive Messaging
• Scalability Clustered Gateways
• Interoperability M/C Protocol Transformation
• New Policy/Control Planes

40
New Service ArchitectureIntegrated
Communications and Processing

• Increasing diversity of interconnected devices
• Increasing importance of services to mitigate
  diversity/provide new functionality and
  customization
• Enabled by processing embedded in the network
  interconnect, locally and globally
• Active networking is real
• Global services via managed composition
• Role of multiple service providers and
  administrative domains
• Separation of services from connectivity via
  overlays
• No single operator deploys the global service

41
The SAHARA ProjectComposition and
Cooperationin the New InternetRandy H.
KatzThank You!