The Cost of Inconsistency in Chord

1
The Cost of Inconsistency in Chord

• Shelley Zhuang, Ion Stoica, Randy Katz
• OASIS/i3 Retreat, January 2005

2
Introduction

• DHTs support a hashtable-like lookup interface
• Each node maintains a routing table about a few
  other (typically O(log n)) nodes
• A node communicates with other nodes to perform a
  lookup
• The routing state may become inconsistent as
  nodes continuously join and leave
• What is the impact of routing state
  inconsistencies on lookup performance?

3
Outline

• Introduction
• Cost of Inconsistency
• Performance Evaluation
• Conclusion

4
Routing State Inconsistencies
Inconsistency Generated by Corrected by
False negative a node falsely thinks a neighbor is up Leaves Failure detection algorithms
False positive a node falsely removes a neighbor that is up Failure detection algorithms Failure recovery algorithms
Join hole a new node joins but has not been fully incorporated into the routing state Joins Join algorithms
Leave recovery a node leaves and disrupts the routing state Leaves Failure recovery algorithms
5
Possible Outcomes of a Lookup
6
Chord Protocol

• Each node has an identifier
• Nodes are ordered on a circular identifier space
• Key k is assigned to the first node whose
  identifier is equal to or follows k on the
  circular identifier space
• Each node maintains a predecessor, successors,
  and fingers
• Correctness of lookup depends only on predecessor
  and successor pointers

7
Cost of Inconsistency - Loss

• Network loss
• False negative a node forwards a lookup to a
  neighbor that has failed

A
C
B
8
Cost of Inconsistency - Loop

• A forwards lookup to B, B forwards it on because
  B's pred is between key and B

A
key
B
9
Cost of Inconsistency - Loop

• A forwards lookup to B, B forwards it on because
  B's pred is between key and B

A

• A's succ is wrong, B's pred is correct or wrong

key
B
10
Cost of Inconsistency - Loop

• A forwards lookup to B, B forwards it on because
  B's pred is between key and B

A

• A's succ is wrong, B's pred is correct or wrong

key

• A's correct succ should be some node between A
  and key

B
? The above two cases can be caused by FP, join,
leave
11
Cost of Inconsistency - Loop

• A forwards lookup to B, B forwards it on because
  B's pred is between key and B

A

• A's succ is wrong, B's pred is correct or wrong

key

• A's correct succ should be some node between A
  and key

B

• A's correct succ should be some node between key
  and B

? The above two cases can be caused by FP, join,
leave

• A's succ is correct, B's pred is wrong

12
Cost of Inconsistency - Incorrect

• A forwards lookup to B, B accepts it
• A's succ must be wrong because there is some node
  between key and B
• B's pred must be wrong because B only accepts a
  lookup if its pred is before key, but there is a
  node between key and B

A
key
Y
B
13
Cost of Inconsistency - Incorrect

• A forwards lookup to B, B accepts it
• A's succ must be wrong because there is some node
  between key and B
• B's pred must be wrong because B only accepts a
  lookup if its pred is before key, but there is a
  node between key and B

A
key
Y
B

• A's succ must be wrong, B's pred must be wrong

14
Cost of Inconsistency - Incorrect

• A forwards lookup to B, B accepts it
• A's succ must be wrong because there is some node
  between key and B
• B's pred must be wrong because B only accepts a
  lookup if its pred is before key, but there is a
  node between key and B

A
key
Y
B

• A's succ must be wrong, B's pred must be wrong

15
Cost of Inconsistency - Incorrect

• A forwards lookup to B, B accepts it
• A's succ must be wrong because there is some node
  between key and B
• B's pred must be wrong because B only accepts a
  lookup if its pred is before key, but there is a
  node between key and B

A
key
Y
B

• A's succ must be wrong, B's pred must be wrong

• A's correct succ should be some node between A
  and key

? the above cases can be caused by FP, join, leave
16
Outline

• Introduction
• Cost of Inconsistency
• Performance Evaluation
• Conclusion

17
Lookup Timeouts vs. Network Loss Rate
18
Lookup Timeouts vs. Churn Rate
19
Incorrect Lookups vs. Churn Rate
20
Outline

• Introduction
• Cost of Inconsistency
• Performance Evaluation
• Conclusion

21
Conclusion

• Routing invariant has a first order impact on the
  relative cost of different types of
  inconsistencies
• Cost of false negatives is higher than false
  positives ? more important to ensure timely
  failure detection than low probability of false
  positive
• Cost of join holes is higher than false positives
  and leave recoveries due to the routing invariant
  of Chord

22
Ongoing / Future Work

• Lookup losses
• Perhop retry
• End to end retry
• What is the best waypoint to try?
• Incorrect lookups
• Predecessor list
• Cost of inconsistency in other DHTs
• Correctness in Chord depends only on predecessor
  and successor pointers
• Many other protocols do not have similar entries
  in the routing state that is sufficient for
  correctness