Evolution 0.9: The Evolution of the U.S. Peering Ecosystem

1
Evolution 0.9The Evolution of the U.S. Peering
Ecosystem

• William B. Norton
• Co-Founder Chief Technical Liaison
• Equinix, Inc.

Gigabit Peering Forum VII Herndon, VA September
9, 2003
2
Internet Researcher

• Four Years of Research
• Document Internet Operations Practices
  Interconnect/Peering/IXes/etc.
• Research Process Focus with Community and
• Write White Paper version M.m
• Walk throughs Feedback
• Fix/Update Paper
• 200 Walkthroughs later a document reflecting
  community Internet Operations knowledge

So far 6 White Papers
3
Internet Operations White Papers

1. Interconnection Strategies for ISPs
2. Internet Service Providers and Peering
3. A Business Case for Peering
4. The Art of Peering The Peering Playbook
5. The Peering Simulation Game
6. Do ATM-based Internet Exchanges Make Sense
   Anymore?

Freely available. Send e-mail to wbn_at_equinix.com
New white Paper The Evolution of the U.S.
Peering Ecosystem, Agenda
4
Agenda

• The Evolution of the Internet Peering Ecosystem
• The Internet Peering Ecosystem Environment
• Each Player, their Motivations, their Behavior
• Four Major Events, led to
• Three Major Evolutions in the Peering Ecosystem

5
The Evolution of the U.S. Peering Ecosystem

• Act I 1998-2000

6
The Internet Peering Ecosystem

• The Environment
• The Players
• Their role in the Peering Ecosystem
• Their behavior in the Peering Ecosystem
• Their relationship to others in the Peering
  Ecosystem
• The Evolution of the Peering Ecosystem

7
The Internet Peering Ecosystem

• Many Internet Regions (usually defined by
  geopolitical boundaries)
• Each with its own Peering Ecosystem consisting of
• Tier 1 ISPs
• Tier 2 ISPs
• Content Players
• and Enterprises

8
Interconnect Regions

• Large Internet Regions have multiple geographical
  areas where ISPs peer

9
Player Tier 1 ISPs
Def A Tier 1 ISP is an ISP that has access to
the entire Regional Internet routing table solely
through Peering Relationships (I.e. doesnt buy
transit from anyone).
10
Tier 1 Peering

• Full Mesh Peering among Tier 1 ISPs within each
  Interconnect Region

11
Some Tier 1 ISPs in the U.S.
12
Tier 1 Peering Across the U.S.
8 Interconnect Regions
13
Tier 1 Peering Motivations

• Generally not interested to peer with anyone else
• They already get the traffic for free
• They may be foregoing revenue (they could sell
  transit)
• They dont want to give away that which
  differentiates them

14
Player Tier 2 ISP

• Def A Tier 2 ISP is an ISP that purchases (and
  therefore resells) transit within an Internet
  Region.

15
Tier 2 ISP Peering Motivations

• Reduce transit costs
• Improve Performance
• Greater control over routing

16
Tier 2 Peering

• Sparse Mesh Peering in each Interconnect Region

17
Tier 2 ISPs

• Widely varying characteristics and behaviors
• Scope Global, National, Regional, Local
• Underfunded to well funded
• Food fights to solid relationships within the
  community
• Big Peoples Table and Little Peoples Table
  perhaps a bad generalization

18
The Content Provider and Enterprise Player

• Always buy transit
• Focus on Content
• Focus on Service
• Few Network Engineers
• SLAs valuable

19
The Internet Peering Ecosystem
20
Peering Inclinations and Peering Policies

• Def A Peering Inclination is a predilection
  towards or against peering as demonstrated by
  Peering behavior in a Peering Ecosystem
• Def A Peering Policy documents and defines the
  prerequisites to peering.

21
Four types of Peering Inclination

• Open Peering Anyone, Anywhere
• Selective Peering Some Requirements
• Restrictive Peering No
• No Peering No, prefer buying transit

22
The Evolution of the U.S. Peering Ecosystem

• Act II 2000-2003

23
Evolution of the U.S. Peering Ecosystem

• Four key events leading to drastic disruption in
  the Peering Ecosystem
• The 1999/2000 Telecom Collapse
• The growth of the Used Equipment Market
• The Upstream Provider for the Cable Companies
  (_at_Home) went bankrupt
• Peer-to-Peer file sharing systems (like Kazaa)
  grow in popularity, traffic grows exponentially
  between Access Providers (1.5MB MP3 ? 700MB AVI
  files)

Leads to 3 Major Evolutions
24
Evolution 1 Cable Companies are Peering

• 40 of traffic Kazaa
• 3-5 Gbps of transit traffic-gt Gbps of peering
  potential !
• Significant because
• Volume
• Open Peering
• Kazaa Effect

25
U.S. Cable Companies Eyeballs
26
AOL
27
1998-2000
Tier 1 ISPs

Transit
Tier 2 ISPs
Thickness represents In-Group Peering
Content
28
2000-2003 Evolution 1 Cable Companies Peering
Tier 1 ISPs
CableCos
Peering
Tier 2 ISPs
Content
29
Evolution 2 Network Savvy Large Scale Content
Companies get into Peering

• To reduce transit costs
• To improve the end user experience
• They need to move out of bankrupt colo anyway
• Significant because
• Volume of traffic is huge
• Content Providers have Open Peering
• Leading players pave the way

30
1998-2000
Tier 1 ISPs

Transit
Tier 2 ISPs
Thickness represents In-Group Peering
Content
31
2000-2003 Evolution 2 Large Scale Network
Savvy Content Peers
Tier 1 ISPs
Tier 2 ISPs
LSNSC
Content
32
Other Broadband Players

• Significant, but
• Not Open Peers so disincentive to peer
• So less volume and less disruptive

33
Evolution 3 Cable Companies Freely Peer with
Content Companies

• Content literally on the Cable Company Network
• Lowest possible latency from content to eyeballs
• Internet Gaming, Broadband Streaming, etc.

34
1998-2000
Tier 1 ISPs

Transit
Tier 2 ISPs
Thickness represents In-Group Peering
Content
35
2000-2003 Evolution 1 Cable Companies Peering
Tier 1 ISPs
CableCos
Tier 2 ISPs
Content
36
2000-2003 Evolution 2 Large Scale Network
Savvy Content Peers
Tier 1 ISPs
Tier 2 ISPs
LSNSC
Content
37
2000-2003 Evolution 3 Cable Peers with Large
Scale Network Savvy Content
Tier 1 ISPs
CableCos
Tier 2 ISPs
LSNSC
Content
38
International Dynamics Separate White Paper
JP
U.S.
U.S. Tier 1 ISPs are Tier 2 ISPs in Japan
Internet Region Japan Tier 1 ISPs are Tier 2 ISPs
in the U.S. Internet Region
39
Summary
2003 Peering Ecosystem Evolving
NoPeering Open Selective Restrictive
Lg.Content
Tier2 ISPs
CableCos
RBOCs
Tier 1 ISPs
Foreign ISPs
Excite_at_Home