Distributed File System: Data Storage for Networks Large and Small - PowerPoint PPT Presentation

About This Presentation
Title:

Distributed File System: Data Storage for Networks Large and Small

Description:

Distributed File System: Data Storage for Networks Large and Small Pei Cao Cisco Systems, Inc. Review: DFS Design Considerations Name space construction AAA Operator ... – PowerPoint PPT presentation

Number of Views:317
Avg rating:3.0/5.0
Slides: 22
Provided by: DanB68
Learn more at: https://zoo.cs.yale.edu
Category:

less

Transcript and Presenter's Notes

Title: Distributed File System: Data Storage for Networks Large and Small


1
Distributed File System Data Storage for
Networks Large and Small
  • Pei Cao
  • Cisco Systems, Inc.

2
Review DFS Design Considerations
  1. Name space construction
  2. AAA
  3. Operator batching
  4. Client caching
  5. Data consistency
  6. Locking

3
Summing it Up CIFS as an Example
  • Network transport in CIFS
  • Use SMB (Server Message block) messages over a
    reliable connection-oriented transport
  • TCP
  • NetBIOS over TCP
  • Use persistent connections called sessions
  • If a session is broken, client does the recovery

4
Design Choices in CIFS
  • Name space construction
  • per-client linkage, multiple methods for server
    resolution
  • file//fs.xyz.com/users/alice/stuff.doc
  • \\cifsserver\users\alice\stuff.doc
  • E\stuff.doc
  • CIFS also offers redirection method
  • A share can be replicated in multiple servers or
    moved
  • Client open ? server reply STATUS_DFS_PATH_NOT_C
    OVERED ? client issues TRANS2_DFS_GET_REFERRAL
    ? server reply with new server

5
Design Choices in CIFS
  • AAA Kerberos
  • Older systems use NTLM
  • Operator batching supported
  • These methods have AndX variations
    TREE_CONNECT, OPEN, CREATE, READ, WRITE, LOCK
  • Server implicitly takes results of preceding
    operations as input for subsequent operations
  • First command that encounters an error stops all
    subsequent processing in the batch

6
Design Choices in CIFS
  • Client caching
  • Cache both file data and file metadata,
    write-back cache, can read-ahead
  • Offers strong cache consistency using an
    invalidation-based approach
  • Data access consistency
  • Oplocks similar to tokens in AFS v3
  • level II oplock read-only data locks
  • exclusive oplock exclusive read/write data
    lock
  • batch oplock exclusive read/write open lock
    and data lock and metadata lock
  • Transition among the oplocks
  • Observation can have a hierarchy of lock
    managers

7
Design Choices in CIFS
  • File and data record locking
  • Offer shared (read-only) and exclusive
    (read/write) locks
  • Part of the file system Mandatory
  • Can lock either a whole file or byte-range in the
    file
  • Lock request can specify a timeout for waiting
  • Enables atomic writes with the ANDX batching
    with Writes
  • Lock/write/unlock as a batched command sequence
  • Additional capability directory change
    notification

8
DFS for Mobile Networks
  • What properties of DFS are desirable
  • Handle frequent connection and disconnection
  • Enable clients to operate in disconnected state
    for an extended period of time
  • Ways to resolve/merge conflicts

9
Design Issues for DFS in Mobile Networks
  • What should be kept in client cache?
  • How to update the client cache copies with
    changes made on the server?
  • How to upload changes made by the client to the
    server?
  • How to resolve conflicts when more than one
    clients change a file during disconnected state?

10
Example System Coda
  • Client cache content
  • User can specify which directories should always
    be cached on the client
  • Also cache recently used files
  • Cache replacement walk over the cached items
    every 10 min to reevaluate their priorities
  • Updates from server to client
  • The server keeps a log of callbacks that couldnt
    be delivered and deliver them upon client
    connection

11
Coda File System
  • Upload the changes from client to server
  • The client has to keep a replay log
  • Contents of the replay log
  • Ways to reduce the replay log size
  • Handling conflicts
  • Detecting conflicts
  • Resolving conflicts

12
Performance Issues in File Servers
  • Components of server load
  • Network protocol handling
  • File system implementation
  • Disk accesses
  • Read operations
  • Metadata
  • Data
  • Write operations
  • Metadata
  • Data
  • Workload characterization

13
DFS for High-Speed Networks DAFS
  • Proposal from Network Appliance and companies
  • Goal eliminate memory copies and protocol
    processing
  • Standard implementation network buffers ? file
    system buffer cache ? user-level application
    buffers
  • Designed to take advantage of RDMA (Remote DMA)
    network protocols
  • Network transport provides direct memory ? memory
    transfer
  • Protocol processing is provided in hardware
  • Suitable for high-bandwidth, low-error-rate,
    low-latency network

14
DAFS Protocol
  • Data read from the client
  • RDMA request from the server to copy file data
    directly into application buffer
  • Data write from the client
  • RDMA request from the server to copy application
    buffer into server memory
  • Implementation
  • as a library linked to user application interface
    with RDMA network library directly
  • Eliminate two data copies
  • as a new file system implementation in the kernel
  • Eliminate one data copy
  • Performance advantage
  • Example 90 usec/op in NFS vs. 25 usec/op in DAFS

15
DAFS Features
  • Session-based
  • Offer authentication of client machines
  • Flow control by server
  • Stateful lock implementation with leases
  • Offers atomic writes
  • Offers operator batching

16
Clustered File Servers
  • Goal scalability in file service
  • Build a high-performance file service using a
    collection of cheap file servers
  • Methods for Partitioning the Workload
  • Each server can support one subtree
  • Advantages
  • Disadvantages
  • Each server can support a group of clients
  • Advantages
  • Disadvantages
  • Client requests are sent to server in round-robin
    or load-balanced fashion
  • Advantages
  • Disadvantages

17
Non-Subtree-Partition Clustered File Servers
  • Design issues
  • On which disks should the data be stored?
  • Management of memory cache in file servers
  • Data consistency management
  • Metadata operation consistency
  • Data operation consistency
  • Server failure management
  • Single server failure fault tolerance
  • Disk failure fault tolerance

18
Mapping Between Disks and Servers
  • Direct-attached disks
  • Network-attached disks
  • Fiber-channel attached disks
  • iSCSI attached disks
  • Managing the network-attached disks volume
    manager

19
Functionalities of a Volume Manager
  • Group multiple disk partitions into a logical
    disk volume
  • Volume can expand or shrink in size without
    affecting existing data
  • Volume can be RAID-0/1/5, tolerating disk
    failures
  • Volume can offer snapshot functionalities for
    easy backup
  • Volumes are self-evident

20
Implementations of Volume Manager
  • In-kernel implementation
  • Example Linux volume manager, Veritas volume
    manager, etc.
  • Disk server implementation
  • Example EMC storage systems

21
Serverless File Systems
  • Serverless file systems in WAN
  • Motivation peer-to-peer storage never lose the
    file
  • Serverless file system in LAN
  • Motivation client powerful enough to be like
    servers use all clients memory to cache file
    data
Write a Comment
User Comments (0)
About PowerShow.com