Storage management and caching in PAST, a large-scale, persistent peer-to-peer storage utility

1
Storage management and caching in PAST, a
large-scale, persistent peer-to-peer storage
utility

• Antony Rowstron, Peter Druschel

2
Introduction

• PAST is an Internet-based, peer-to-peer global
  storage utility.
• It aims to provide strong persistence, high
  availability, scalability and security.
• PAST employs Pastry as the routing scheme.
• Pastry is an efficient routing scheme used to
  reliably route client requests to the appropriate
  nodes.

3
PAST Operations

• fileIdInsert(name, owner-credentials, k, file)
• fileId is 160-bit, while nodeId is 128-bit
• fileId
• nodeId
• fileLookup(fileId)
• Reliably retrieve a copy of the file
• Reclaim(fileId, owner-credentials)
• Notice Reclaim doesnt guarantee that the file
  is no longer available after it was reclaimed.

128 Bits
32 Bits
M
L
128 Bits
4
Storage Management

• Design Goals
• Allow high global storage utilization
• Gracefully degrade as the system approaches the
  maximal utilization
• Load balancing
• Replica diversion
• File diversion

5
Replica Diversion

• Replica diversion is to accommodate differences
  in the storage capacity and utilization of nodes
  within a leaf set.

File is stored in B due to the less capacity in A
B
C
C is the k1th closest node, which stores the
pointer in case of the failure of node A
An entry points to B
A
A is the right node to store the file, but has
insufficient space
6
File Diversion

• The k closest nodes to the fileId could not
  accommodate the file nor divert the replicas
  then, a file diversion is invoked.
• The client generates a new fileId, and retries to
  insert the file.
• Repeat the second step up to 3 times.
• If it still fails, an error is reported to the
  application.

7
Caching

• PAST nodes use the unused portion of their
  advertised disk space to cache files.
• Caching in PAST is based on GreedyDual-Size
  (GD-S) policy, which was originally developed for
  caching Web proxies.

8
Experimental Results
This graph is from the paper.
9
Question?