The Next Wave of

1
The Next Wave of Massive Disruptions to the
Peering Ecosystem
William B. Norton Co-Founder Chief Technical
Liaison Equinix, Inc.
Asia Pacific Peering Forum Sydney, Nov. 8, 2006
Slide set v1.1
2
Definitions of Peering and Transit
Def Peering is the business relationship whereby
ISPs reciprocally announce reachability to each
others transit customers.
Peering
Peering
USNet
EastNet
WestNet
Routing Table

• (Note Peering is a Non-Transitive Relationship)

Def Transit is the business relationship whereby
one ISP sells access to all destinations in its
routing table.
Game of relationships.
3
Peering is a game of relationships
White Papers.My job these days
4
On the Internet Everyone is a Publisher
5
Internet Operations White Papers

1. Interconnection Strategies for ISPs
2. Internet Service Providers and Peering
3. A Business Case for Peering
4. The Art of Peering The Peering Playbook
5. The Peering Simulation Game
6. Do ATM-based Internet Exchanges Make Sense
   Anymore?
7. Evolution of the U.S. Peering Ecosystem
8. The Asia Pacific Internet Peering Guidebook
9. The Folly of Peering Traffic Ratios?

Freely available. See Web site or send e-mail to
wbn_at_equinix.com
Internet makes anyone a publisher, similar effect
now emerging for video
6
Massive Disruption in U.S. Peering Ecosystem ?
Short Videos

• YouTube founded 2005
• Short video clips 50 million view per day!
• 20Gbps of peering traffic Feb 2006
• 1M/month in Sept 2006!
• Entering Peering Ecosystem
• 30 Other competitors?600Gbps peerable?
• DoveTail
• Video may dwarf current peered traffic
• 2010 80-90 Internet is Video
• Inculcate video guys into peering ecosystem

On the Internet Everyone is a Broadcaster
Short video clipsFull TV shows
Source http//digg.com/tech_news/YouTube_Gets_Ban
dwidth_Boost_from_Level_3
Source http//www.nanog.org/mtg-0606/norton.html
7
(No Transcript)
8
Massive Disruption in U.S. Peering Ecosystem ?
Full Episodes

• Desperate Housewives 210MB/hour
• For 320x240 H.264 Video iTunes image
• 10,000,000 households
• 2,100,000,000 MB 2.1 peta-Bytes
• How long will that take to download?

3 days _at_ 64Gbps non-stop ! Just one show Try
250M180 ChannelsHDTV
Historical Perspectivereview 5yr disruptions
Source http//www.pbs.org/cringely/pulpit/pulpit2
0060302.html
9
Research Topic

• Massive Wave of Internet Traffic
• 90 of all Internet bits by 2010
• How will Video Service Providers distribute this
  massive amount of Video Traffic over the Internet?

How will Video Service Provider distribute?
10
Modeling the Video Service Provider Distribution
Networks

• Four Models
• Commodity Transit
• CDN
• Transit/Peering/DIY CDN
• Peer2Peer

Four Load Models A Small Load B Medium Load C
Large Load
Goal estimate cost /video downloaded
11
Model 1 Commodity Transit

• Business Premise
• VSP focuses on core competence
• Transit Providers handle traffic better and
  cheaper
• Economies of scale, Aggregation, Expertise,
  Billing, Peering, etc.

12
Model 1A Transit Light Load
13
Model 1B Transit Medium Load
14
Model 1C
Upstream ISPs
Router4
Router2
10G
Router2
8 10GE to upstreams each
Server1
GigE Switch1
Router1

Server24

10G

GigE Switch14

Server262
Server263
Server264

Distribution GigE Switch 48 port GigE for
servers 2 10GE for upstream 10,000 Add another
every 24 servers
Routers Cisco 6509Sup720-3bxl w/44-port 10GE,
150,000 80Gbps from switches, 80Gbps to upstreams
15
Model 2 Content Delivery Networks (CDNs) for the
Distribution of Video Content

• Business Premise Single-site transit traffic
  traverses potentially many network devices,
  increasing latency and the potential of packet
  loss
• By spreading web objects closer to the eyeball
  networks latency is reduced
• Fewer network elements are traversed so
  reliability is improved
• Congestion points in the core of the Internet are
  avoided
• CDNs have the expertise, deployed infrastructure,
  economies of scale from aggregation efficiencies.

16
Assumption CDN Price Points
17
Model 2A CDN Light Load
18
Model 2A CDN Light Load
19
Model 2B CDN Medium Load
20
Model 2C CDN Large Load
21
Model 3 Transit/Peering/DIY CDN

• Business Model Premise Operation of the Internet
  distribution is seen as strategic to the VSP
• End-user experience is mission-critical so
  outsourcing the end user experience to a transit
  provider or CDN puts the VSP at risk.
• The VSP has visibility into what video are being
  released, which ones are likely to be hot and
  which ones dont require special infrastructure
  adjustments.
• Internet Video distribution is so new that the
  VSP prefers control. This is a strategic focus of
  the VSP ensuring reliability, scalability,
  through the constant monitoring and evolution of
  the infrastructure to ensure the end user
  experiences during these early phases of Internet
  Video Distribution.
• The traditional CDN may be ill-suited to
  distribute very large video object, therefore we
  have to do it yourself.

22
Model 3 Transit/Peering Light Load
23
Model 3A Peering/Transit Light Load
24
Model 3B Transit/Peering Medium Load
25
Model 3C Transit/Peering Heavy Load
26
Model 4 Peer2Peer

• Business Model Premise The current Internet
  Service Providers and CDNs at the core can not
  handle the load across single or even multiple
  locations
• Backbone, peering interconnects, and the hundreds
  of thousands of routers deployed can not handle
  the load of today and tomorrows video.
• the leaf nodes (i.e. Grandmas PCs left on) in
  aggregate have the cycles and network capacity,
  if shared, to distribute popular content today.
• Popular content can be chopped up into small
  chunks such that many downloaders become sources,
  and topologically close downloaders will prefer
  the topologically close sources. This
  swarmcasting requires only a source seed, and
  a lookup mechanism for the first downloaders to
  find the seed, and then to direct future
  downloaders to topologically closer sources.

27
Model 4A P2P Light Load
28
Model 4A P2P Light Load
29
Model 4B P2P Medium Load
30
Model 4B P2P Medium Load
31
Model 4C P2P Large Load
32
Model 4C P2P Large Load
33
Summary
Per Video Cost Of delivery
34
Observation/Implications

• Internet Transit Supply ?
• Internet Transit Price ?
• Internet Transit Model ? src/dst specific
• Bottlenecks
• IX Power, Router Capacity, Peers Capacity,
• Last Mile bottleneck
• Do I need to upgrade gear to support my
  competitor (peer)?
• Geoff Huston P2P has won. Telco/Cable co trying
  to keep its 1998 biz plan relevant.

I look forward to discussing this over the next
few days.