DEBAR: A Scalable HighPerformance Deduplication Storage System for Backup and Archiving

1
DEBAR A Scalable High-Performance
De-duplication Storage System forBackup and
Archiving

• Tianming Yang, Hong Jiang, Dan Feng, ZhongyingNiu
• College of Computer Science and Technology, HUST
• Wuhan National Laboratory for Optoelectronics,
  Wuhan, 430074, China
• Department of Computer Science and Engineering
• University of Nebraska-Lincoln Lincoln, NE 68588,
  USA

2
Motivation

• Scalability
• Throughput
• Distributed

3
Outline

• Architecture
• Design
• Debar Disk Index
• Two-Phase De-duplication Scheme
• Evaluation

4
Architecture
5
Design
6
Design Director

• User Interface Job Object, specifies what,
  where, how and when for tasks to do.
• Metadata Manager manage job metadata, such as
  job ID, job size, and file metadata and indices
  (fingerprint list).
• Job Scheduler.

7
Design Backup Client

• Job Interface Control interface used by
  Director.
• Backup Engine Run CDC, cooperate with Backup
  Server.

8
Design Backup Server

• File Store (Phase I) File indexing, preliminary
  filtering, cache new chunk in log.
• Chunk Store (Phase II) Sequential index lookup,
  chunk store, sequential index update.
• Container Manager Interface for store chunks in
  Chunk repository.

9
Design Chunk Repository

• Container Fix size, stream informed segment
  layout (SISL), locality preserved caching (LPC).

10
Debar Disk Index
11
Debar Disk Index
12
Two-Phase De-duplication Scheme

• Preliminary filtering use previous version of
  same file.

13
Two-Phase De-duplication Scheme
14
Evaluation Compression ratio
15
Evaluation Throughput
16
Evaluation Capacity
17
Evaluation Capacity
18
Evaluation Performance
19
Evaluation Scalability
20
Conclusion

• A scalable and high throughput de-duplication
  storage system for backup and archiving.
• TPDS cache and batch.
• Debar Disk Index SIL, SIU.
• Distributed implemention.

21

• Thank you!