Structure of the Internet

1
Structure of the Internet

• Update for 1st H/Wk
• We will start lab next week
• Paper presentation at the end of the session
• Next Class MPLS

2
Other Attempts at the problem

• IDPR (and not IDRP)
• Link state!
• Prunes the ASes and keeps only the transit ones
• Assumes information is very static
• Topology databases need not be synchronized
• Use source routing in order to avoid loops
• Establish a connection through the ASes in the
  Path
• Between Border routers
• And repair it when it breaks
• More complex than OSPF and BGP-4 together
• Did not go anywhere

3
How is the Internet in reality?

• Provider relationships
• PoPs and Internet structure?
• Provider internal structure
• Traffic Egnineering
• BGP monitoring tools
• Examples

4
AS relationships

• The internet is a BIG AS graph
• How does it look?
• Ideally, we would like to see a nice hierarchy
  customer, local ISP, regional ISP, national ISP,
  transit ISP
• Not really
• Locality is determined based on cost
• Reliability requires multiple redundant paths
• To whom I talk can have important business
  implications

5
Types of relationships

• Customer Provider
• Customer pays money for the service
• Customer is usually smaller than the provider
• Paid Transit
• ISP A and B pay ISP C to connect them
• Transit ISPs have big global networks (tier-1)
• Peering
• Two ISPs exchange routes that they originate into
  the internet
• I.e. their own customer routes
• No upstream routes
• Nobody pays

6
Things are complicated

• Network connectivity does not imply reachability
• Policies may prevent it, for example a
  multi-homed customer can not transit traffic
  between its providers
• Need to know the relationships between ASes and
  this is not easy
• Policies are not widely advertised
• Treated as sensitive business information

7
Tiers

• Large transit ISPs are Tier-1
• (MCI, ATT)
• They have no parent provider
• Smaller national/regional ISPs are Tier-2
• GEANT
• And small local ISPs are Tier-3 or edge
• ForthNet
• It is possible to find more structure in Tier-1
• See characterizing the internet hierarchy from
  multiple vantage points
• Some tier-1 form the dense core of the Internet
  
• Almost fully connected graph, tier-0
• Then tier-1 and tier-2, less connected large ISPs
  
• Then small ISPs and customers
• In 2001, 20 ISPs in the dense core

8
ISPs need to talk to each other

• Depends on the relationship
• Customer provider over a single link
• Transit and peering?
• Do it in Internet Exchanges
• Also known as Network Access Points (NAPs) and
  Points of Presence (PoPs)
• No need for n2 connections
• Exchange provides a stable environment for
  peering
• Backup power, administration etc
• Providers need to co-locate in the exchange
• Exchanges are not free
• Although peering is
• Can always have private peerings between two ISPs

9
Exchange architectures

• Centralized
• A single or multiple routers
• Router may have to enforce policies, not too good
  
• Switched
• Just connectivity, BGP enforces the policies
• Need to co-locate
• More expensive
• Co-location costs and cost to send traffic to the
  exchange
• Distributed
• No need to co-locate
• Not so stable as the centrally administered
  exchange

10
Peering Costs

• Peering
• How to share the cost of an end-to-end path
• Cost of sending a packet is almost 0
• Try to split the cost down the middle between the
  sender and the receiver
• Zero cost peering
• Slowly emerging paid peering

11
Peering economics

• When is it better to peer?
• How much traffic I will be able to send through
  the peering
• So I will not pay for it anymore?
• Hard to measure how must traffic goes behind
  certain peers
• How much will I have to pay for the exchange
  peering
• Transit costs, exchange costs, operational costs
  

12
How to charge?

• Charging models in customer-provider
• Say I have a OC-12 (622 Mbit/sec) connection
• Pay flat rate for the whole thing
• Expensive probably
• Pay for a fraction of it (say 200 Mbit/sec only)
• Can not send more
• Burstable fractional
• Pay for a fraction but I can send more
• Extra traffic charged per Mbyte
• 95 charging
• Drop the 5 highest samples and use the next one
  to charge
• For the whole month!
• How often do I sample the traffic? Usually 5 min
  
• Volume based charging
• And flat rate (DSL style)

13
Structure of provider networks

• Three levels (example)
• Aggregation
• Distribution
• Core
• Make sure IGP scales
• Do not send it full BGP routes
• Neither customer prefixes
• Aggregation and distribution may not run iBGP
• Core has to run iBGP
• In transit ISPs core carries full BGP routes
• In edge ISPs core may not have to run iBGP

14
Routing policy best practices

• Do not re-advertise to provider B routes you
  learn from provider A
• Customers should not allow transit
• Do not advertise internal networks
• Do not advertise prefixes that are aggregated
• If you have a single provider no need for full
  routes
• Always check routes you get for bogons
• Limit the maximum number of routes you receive
  from other so that their errors do not kill you

15
Multi-homing

• Two types
• Provider assigned prefix
• Secondary provider has to agree to advertise it
• Provider independent prefix
• Both providers will advertise it
• But connectivity is only part of the problem
• How to I use this multi-homing effectively?
• How do I decide where I send traffic?
• How can I control how I receive traffic?

16
Traffic Engineering and BGP

• BGP conveys only connectivity information
• Can not tell me which is the best/cheapest/least
  load path to use
• I have minimum influence on what paths are used
  to reach me from other providers
• There are some hacks to do something about it
• AS prepending make some paths I export longer so
  they are not used too much
• Selectively advertise my external networks
• Breaks aggregation
• Use help from my providers
• They may advertise communities that allow me to
  have little bit of control on the incoming path
• By telling provider where to advertise my paths
• These are not real solutions
• Spawned a market for route analytics
• But these only address my outbound traffic

17
Traffic Engineering inside the AS

• Need to be able to control how transit and
  customer traffic flows in my network
• It was believed that it is necessary to have
  circuit based transit to achieve this
• ATM
• Now MPLS
• But IGP may be sufficient if I set the weights in
  a smart way
• More to come

18
Hot Potato Routing

• IGP cost to reach a BGP next-hop can make all the
  difference
• May affect a lot of traffic and cause
  instabilities
• And cause BGP forwarding loops
• BGP routers compute their paths on a timer, in
  between route computations there may be
  inconsistencies
• IGP cost is low in the BGP path selection process
• Paths have to be otherwise the same
• Common in tier-1 providers
• Also rule for preferring eBGP over iBGP can
  result in asymmetric paths

19
BGP Tools

• WHOIS
• Registration information for AS
• Some examples, show the community stuff too
• Looking glass
• A provider opens ups its routing tables
• I can see how my routes look from there
• RADB
• Route policy registry, some providers do not
  accept announcements that do not have correct
  entries there
• BGP reports for scaling, CIDR etc
• http//bgp.potaroo.net
• http/www.cidr-report.org

20
General Tools

• Ping
• Trace route