Locality Aware Network Solutions

1
Locality Aware Network Solutions

• Dahlia Malkhi
• The Hebrew University of Jerusalem

2
A Brief Overview of Distributed Computing

• The 90s
• Internet activity Web browsing
• Paradigm Client-server
• Techniques cluster computing, Paxos, group
  communication

3
A Brief Overview of Distributed Computing

• The 90s

• 2000-
• Internet activity File sharing
• Paradigm P2P, grid, web-services
• Techniques overlay networks, content
  distribution networks, resource location

4
Application IPv6 Routing over IPv4van Renesse
02
Distribute Hash Tables (DHT)
5
Application Content Delivery / Finding Nearest
Copies of Data
?
?
?
6
Application HyperencryptionMaurer 92, Ding
Rabin 02
Random bits
Adversary bits
7
Application A Hyperencryption P2P NetworkRabin
03
Distributed Hash Table (DHT)
8
Application A Distributed Google?
?
9
Scalable Network Solutions

• Overlay networks provide added functionality at
  the application level
• Search, routing, location services
• Network theory provides the foundations
• Possibilities, impossibilities, lower/upper
  bounds
• Practical solutions require flexible deployment

10
Distributed Data Structures (DDS)

• Peers jointly implement a data structure, e.g.,
  hash table
• Route queries based on data-name (key)

11
DDS Problem Reduced to Routing
?? 00001111
Responsible for 00001111
12
Why classic routing network designs dont help
111
011

• Static
• of nodes a priori known
• Node labels designated by network designer

101
001
110
010
000
100
13
DDS Reduced to Routing

• The problem Overlay routing network
• Variants labeled routing, name-independent
  routing, finding nearest copies
• Dynamic emulation

14
Distributed Hash Tables
Malkhi, Naor, Ratacjzak, PODC 2002
Abraham, Awerbuch, Azar, Bartal, Malkhi, Pavlov,
IPDPS 2003
15
Tree View of Dynamic Graphs
111
011

• Leafs of the tree represent current nodes

101
001
110
010
00
000
100
00

• Inner nodes in the tree represent nodes that were
  split

110
111
000
001
010
011
100
101

• Example merge of 000, 001 into 00

16
Locality awareness
17
Locality awareness
18
Locality Awareness in Overlay Networks

• Model the network as a weighted undirected graph
• c(x, y) cost of shortest path from x to y
• c() is a metric
• An overlay network is a sub-graph
• Let xx0 , x1, , xty be a route in the overlay
  network
• Stretch Ratio between overlay route cost and
  shortest path cost( c(x, x1) c(x1,x2)
  c(xt-1, y) ) / c(x,y)

19
Overlay Networks inGrowth-Bounded Metrics

• Previous work
• Plaxton, Rajaraman, Rica 1997, Tapestry
  (Berkeley), Pastry (MS UK)
• Expected (large) constant stretch
• Logarithmic node degree
• LAND Abraham, Malkhi, Dobzinski, SODA 2004
• Guaranteed stretch (1e)
• Expected logarithmic node degree, constant
  depends on growth-bound
• Simple, intuitive construction and proofs

20
Overlay Networks in Geometric Spaces

• Modeling the Internet as a geometric space is
  practical
• Ubiquitous GPS devices
• Successful embeddings in virtual coordinate-space
• Problem 1 Locate nodes
• Problem 2 Route to known coordinates

21
Location Services and Routingin Geometric Spaces

• LLS First fully-locality aware location service
  Abraham Dolev Malkhi 2004
• bounded stretch lookup
• bounded stretch update
• First constant-degree routing scheme (to known
  coordinates)Abraham Malkhi, PODC 2004
• constant node degree, logarithmic hops, 1e
  stretch

22
Routing in Arbitrary Graphs Lower and upper
bounds

• Name-independent routing node names are
  independent of routing scheme
  Awerbuch, Bar Noy, Linial, Peleg 1989
• Lower bounds Gavoille
  Gengler 2001
• Stretch lt 3 ? O(n) routing information
• Stretch lt 5 ? vn routing information
• Upper bound Abraham, Gavoille, Malkhi,
  Nisan, Thorup, SPAA 2004
• stretch-3 routing with O(vn ) routing information
• Stretch 3 is indeed attainable!
• General upper bound Abraham Gavoille, Malkhi,
  DISC 2004
• Stretch-k routing with memory O(k2 kvn )

23
infrastructure and new methods and systems
devoted to measurement, mock-up and and analysis
of present and future network traffic, topology
and logical structure, to bridge the gap in
theory, protocols and understanding to what the
Internet can be in 2025.
An EC project. Coordinators SICS (Sweden) and
HUJI (Jerusalem)
24
Locality-Aware, Robust Overlay for Information
Lookup and Content Delivery

• Degree O(vn)
• Locality awareness
• Formally stretch 3
• For far-apart nodes, lower stretch
• Mostly two-hop
• Whenever full connectivity exists
• Flexibility
• Estimate vn roughly
• Cache information on many vicinity nodes
• Store information about any known node of same
  color
• Fault tolerance
• Multiple route choices
• Quick repair
• Maintain QoS in face of churn

25
Thank you!
26
Large Scale Content Delivery

• Initially split the content
• Then cross-exchange data pieces
• Solutions build on top of overlay routing networks

27
FastReplica Cherkasova,Lee -HP Labs - 2003

• Phase 1

Source
Clients
28
FastReplica Cherkasova,Lee -HP Labs - 2003

• Phase 2

Source
Clients
29
Locality motivation Tree example
30
Julia algorithm motivation Divide and conquer
First phase
31
Julia algorithm motivation Divide and conquer
32
Network nodes
33
Nodes random identifiers
1001001
0101001
0111010
0001000
0011110
1111110
34
Coloring and Vicinities
35
Coloring and Vicinities
?
?
?
?
36
Stretch 3
?
d
2d
d
37
The Full Routing Scheme