Vasilios Darlagiannis, EPFL - PowerPoint PPT Presentation

1 / 33
About This Presentation
Title:

Vasilios Darlagiannis, EPFL

Description:

Connectivity. Constraint topology (Ring) P-Grid. DKS. Oscar. Jupp. Fuzzynet. Routing table ... Study overlay networks as Complex System ... – PowerPoint PPT presentation

Number of Views:35
Avg rating:3.0/5.0
Slides: 34
Provided by: scottkir
Learn more at: http://www.evergrow.org
Category:

less

Transcript and Presenter's Notes

Title: Vasilios Darlagiannis, EPFL


1
SP3 Overlay Networks and ServicesWP3k
Algorithms for Overlay Networks
  • Vasilios Darlagiannis, EPFL
  • Partners KTH, SICS, EPFL, HUJI, UCL

2
Contents
  • Introduction
  • Highlights of previous years
  • Achievements during the last year of Evergrow
  • Low maintenance structured overlays
  • Scalable routing schemes and latency
    approximation
  • Overlay network merging
  • Self management of large-scale systems
  • Conclusions

3
Conclusions
Connectivity
  • Overlay Networks
  • Significant dimensions

P-Grid
DKS
Oscar
Flexibility for application developers
P-Grid
Jupp
Chord
Routing table lt-gt load balancing
Chord
DKS
Fuzzynet
P-Grid
Constraint topology (Ring)
4
Conclusions
  • Services over SONs

P-Grid DKS Chord Oscar Fuzzynet Jupp
Broadcast
DistanceEstimation
BeliefPropagation
Semantics
5
Motivation
  • Study overlay networks as Complex System
  • Overlay networks already produce more than 60 of
    the Internet traffic
  • Overlay networks are complex systems
  • Complex evolving structures
  • Complex dynamics
  • Traditional approaches for distributed algorithms
    and distributed services are not effective
  • No ability to largely control the system state
  • No ability to enforce constraints or centralized
    coordination
  • gt Novel Paradigms inspired by Complex Systems

6
Highlights in 2005
  • Extension of Kleinbergs model
  • Novel type of structured overlays which support
    load-balancing for imbalanced key and work-load
    distributions
  • Case study to evaluate standard methodologies for
    studying stochasticity of probabilistic systems
  • Mean value analysis
  • Evolution of probability mass distribution
    function
  • P2PKit middleware enabling the implementation of
    overlays
  • P2PS library for enabling overlay deployment in
    hostile environments
  • Julia content distribution network (theoretical
    foundations and deployment)
  • Measurement of global properties in complex
    systems with low overhead
  • Efficient and adaptive estimation of network size
  • MyriadStore design document

7
Highlights in 2006
  • Metric Embedding techniques
  • Approximation of Internet latencies
  • Utilizing labels to estimate distances
  • Investigating scalable routing schemes
  • Oscar Load-balancing for heterogeneous peer
    populations
  • Release simplification assumptions on key
    distributions
  • Consideration of heterogeneous peer capabilities
  • Small-world principle
  • Gradual PDF approximation with local information

8
Highlights in 2006
  • Distributed k-ary System (DKS)
  • Atomic Lookup Consistency
  • Reliable Lookup Algorithms
  • Group Communication
  • Proximity Routing Schemes
  • Usage of index replicas
  • Proximity Route Selection (PRS)
  • Proximity Neighbor Selection (PNS)
  • Parallel lookups

9
Accomplishments during last year
10
Fuzzynet Zero maintenance ringless overlay (1)
  • Motivation
  • Advantages of the ring
  • Easy Navigation (greedy routing)
  • Clear responsibility ranges
  • Easy to bootstrap long-range links
  • BUT!
  • Keeping the ring invariant is a difficult task
  • Expensive maintenance (periodic, eager)
  • Non-transitivity effect (A?B, B? C, but not A? C
    )
  • Firewalled peers, NATs
  • Routing anomalies

11
Fuzzynet Zero maintenance ringless overlay (2)
  • Fuzzynet
  • No ring structure (only Small-World long-range
    links)
  • No predefined responsibility ranges
  • Data is probabilistically stored in the data-key
    vicinity
  • Compatible with any Small-World network
  • Typical DHT replication rate
  • Network construction without the help of the ring
    (peer order is considered)
  • Lookup (Read) simple greedy routing
  • Publish (Write) greedy routing write burst

12
Write phase 1 Greedy-Approach
  • Routing from the originator peer (P0.56) to the
    greedy-closest peer (P0.21) where the greedy
    approach towards the target key 0175
    (actual-closest peer P0.17) is no further
    possible.

13
Write phase 2 Write-Burst
  • The greedy-closest peer (P0.21) seeds the
    replicas in the cluster vicinity of the key 0.175
    using the Write-Burst.

14
Lookup (read)
  • After writing the data in the vicinity of the key
    0.175, the lookup (read) from any node will have
    very high chance finding at least one of the data
    replicas.

15
Fuzzynet Results
  • Theoretical analysis (lower bound on success
    rate)
  • Planet Lab Simulation results
  • Fuzzynet Zero-maintenance Ringless Overlay.
    Submitted to ACM TAAS.

16
Routing in DHTs
  • A novel efficient routing table construction
    algorithm
  • arbitrary skews in the peer distribution
  • cheap construction
  • low routing cost
  • easy to implement
  • no maintenance
  • lost routing entries do not have to be replaced
  • hop counts are never updated
  • Jupp On Routing in Distributed Hash Tables. The
    Seventh IEEE International Conference on
    Peer-to-Peer Computing

17
Solution Hop Space
  • Separate the identifier space from the routing
    topology
  • With each link, associate distance in direct
    neighbor hops
  • Hop space only used for table construction
  • No changes of the routing protocol
  • Peers route connection requests in the hop space
    (independent of ID space)
  • Table construction
  • Create links with hops distances 2, 4, 8, 16,
    etc.

2
4
8
16
1
A
B
C
D
E
0
1
0.5
17
18
Routing Range Queries
  • Recursive
  • Choose all routing entries that fall into the
    queried range, plus the one before the range.
  • Split range query into sub-range queries
  • n number of peers
  • m number of peers in the range
  • Cost O((log n) m)
  • Time O(log n)

A
id2
F
id1
E
D
C
B
18
19
Simulation Results
20
Scalable routing schemes and latency approximation
  • Several theoretical results
  • Algorithm with formal guarantees for finding
    faithful and low dimensional representation of
    data in high dimensional space
  • Measure to assign a value to every metric space
  • Quantify how close is to a tree metric
  • Embeddings with distortion
  • depending solely in local structure of the space
  • Distributed name-independent routing scheme
  • Embedding Metric Spaces in their Intrinsic
    Dimension, (SODA 2008)
  • Reconstructing Approximate Tree Metrics, (PODC
    2007)
  • Local Embeddings of Metric Spaces, (STOC 2007)
  • Strong-Diameter Decompositions of Minor Free
    Graphs, (SPAA 2007)

21
Merging two similar structured overlays
  • Multiple networks can result from
  • Network partitioning
  • Separate construction
  • Challenging issues
  • How can they merge?
  • What is the complexity?
  • What is the operational cost?
  • Use cases
  • Ring based overlay networks
  • Structurally replicated overlay networks

22
Merging Illustration on Chord
23
Results on overlay merging
  • Ring based overlay networks
  • Cannot function at all until merging is complete
  • Data transfer complexity
  • Structurally replicated overlay networks
  • Merging process is graceful and system can
    operate
  • Location of all structural replicas can become
    tricky
  • Complex replica management
  • The challenges of merging two similar structured
    overlays A tale of two networks, International
    Workshop on Self-Organizing Systems (IWSOS)

24
Network Partitions in Rings
  • Two algorithms for merging two similar ring-based
    SONs
  • Simple Ring Unification
  • Periodic process
  • Builds on top of Chord lookup
  • O(N) time complexity
  • Unable to recover from pathological scenarios
  • Gossip-based Ring Unification
  • Extension of the Simple Ring Unification
  • Triggering multiple merging processes (fan-out
    parameter F)
  • Handling Network Partitions and Mergers in
    Structured Overlay Networks, IEEE P2P2007

25
r
r.succ
q
SON 1
SON 2
26
Results
  • Simulations over SicsSim
  • Time complexity with multiple nodes detecting
    the merger
  • Message complexitywith multiple nodes detecting
    the merger
  • Loopy network cases have beenalso investigated

27
Self management of large-scale systems
  • Relaxed Ring Architecture
  • Lookup consistency in self-managed SONs
    (ring-based)
  • Handling multiple joins and leaves/failures
  • Self-organization and Self-healing
  • Applied on P2PS
  • Feedback loops
  • Improving the Peer-to-Peer Ring for Building
    Fault-Tolerant Grids, CoreGRID Workshop on Grid
    Programming Model, Grid and P2P Systems
    Architecture, Grid Systems, Tools, and
    Environments
  • A Relaxed-Ring for Self-Organising and
    Fault-Tolerant Peer-to-Peer Networks, XXVI
    International Conference of the Chilean Computer
    Science Society (SCCC 2007)

28
Self management of large-scale systems
  • Key-Based Consistency and Availability in SONs
  • Evaluation of inconsistency
  • Inaccurate failure detectors is the major
    inconsistency factor
  • Trade-off between availability and consistency
  • The probability of a responsibility inconsistency
    is very low
  • Majority-based quorums have been considered
  • Key-Based Consistency and Availability in
    Structured Overlay Networks, High-Performance
    Distributed Computing (HPDC)

29
Results
  • Majority based quorums
  • Probability of getting disjoint majority sets
    for a replica set
  • Number of replicas
  • Number of inconsistencies
  • Probability of disjoint majority sets for two
    majority based operations in one configuration
    of a SON
  • Accuracy of the failure detector

30
Conclusions
  • Services over SONs
  • Broadcast
  • Semantic overlays
  • Distance estimation
  • Belief propagation

31
Summary of achievements
  • The research done in WP3l which has been
    completed in the final year of Evergrow produced
    many interesting results including
  • Ringless overlay network
  • Consistency in Lookup operations
  • Merging of overlay networks
  • Scalable routing schemes
  • Latency approximation mechanisms
  • Several publications in Journals, conferences and
    workshops

32
Dissemination
  • Publications
  • 1 Fuzzynet Zero-maintenance Ringless Overlay.
    Submitted to ACM TAAS.
  • 2 On Routing in Distributed Hash Tables. The
    Seventh IEEE International Conference on
    Peer-to-Peer Computing
  • 3 Embedding Metric Spaces in their Intrinsic
    Dimension, ACM-Siam Symposium on Discrete
    Algorithms (SODA 2008)
  • 4 Reconstructing Approximate Tree Metrics, 26th
    ACM Symposium on Principles of Distributed
    Computing (PODC 2007)
  • 5 Local Embeddings of Metrics Spaces, 39th ACM
    Symposium on Theory of Computing (STOC 2007)
  • 6 Strong-Diameter Decompositions of Minor Free
    Graphs, 19th ACM Symposium on Parallelism in
    Algorithms and Architectures (SPAA 2007)
  • 7 The challenges of merging two similar
    structured overlays A tale of two networks,
    International Workshop on Self-Organizing Systems
    (IWSOS)
  • 8 Dealing with Network Partitions in Structured
    Overlay Networks,
  • 9 Improving the Peer-to-Peer Ring for Building
    Fault-Tolerant Grids, CoreGRID Workshop on Grid
    Programming Model, Grid and P2P Systems
    Architecture, Grid Systems, Tools, and
    Environments
  • 10 A Relaxed-Ring for Self-Organising and
    Fault-Tolerant Peer-to-Peer Networks, XXVI
    International Conference of the Chilean Computer
    Science Society (SCCC 2007)
  • 11 Key-Based Consistency and Availability in
    Structured Overlay Networks, High-Performance
    Distributed Computing (HPDC)
  • 6 Ph.D. Thesis

33
Plans for the Future
  • P-Grid (http//www.p-grid.org )
  • Code is used in 5 other EU projects
  • Bricks
  • Alvis
  • Nepomuk
  • TEAM
  • OCCAM
  • DKS http//dks.sics.se
  • Grid4All
  • SelfMan
Write a Comment
User Comments (0)
About PowerShow.com