Distributed File System

1
Distributed File System

• By Manshu Zhang

2
Outline

• Basic Concepts
• Current project
• Hadoop Distributed File System
• Future work
• Reference

3
DFS

• A distributed implementation of the classical
  time sharing model of a file system, where
  multiple users share files and storage resources.

4
Key Characteristics of DFS

• Dispersion
• Clients and files
• Multiplicity
• Clients and files

5
Primary issues of DFS

• Naming and Transparency
• Fault Tolerance

6
Naming

• Naming mapping between logical and physical
  objects.
• Multilevel mapping.
• Transparent replicas and location

7
Naming Schemes Three Main Approaches

• Host name local name
• guarantees a unique system wide name.
• Mount remote directories to local directories
• once mounted, files can be referenced in a
  location-transparent manner
• Total integration of the component file systems.
• A single global name structure
• If a server is unavailable, some arbitrary set of
  directories on on different machines also becomes
  unavailable

8
Transparency(1)

• Login Transparency User can log in at any host
  with uniform login procedure and perceive a
  uniform view of the file system.
• Access Transparency Client process on a hots has
  uniform mechanism to access all files in system
  regardeless of files are on local/remote host.
• Location Transparency The names of the files do
  not reveal their physical location.

9
Transparency(2)

• Concurrency Transparency An update to a file
  should not have effect on the correct execution
  of other process that is concurrently sharing a
  file.
• Replication Transparency Files may be
  replicated to provide redundancy for availability
  and also to permit concurrent access for
  efficiency.

10
Fault Tolerance

• Stateful Vs. Stateless
• Maintain information on client
• File Replication

11
Distinctions Between Stateful Stateless Service

• Failure Recovery.
• A stateful server loses all its volatile state in
  a crash.
• With stateless server, the effects of server
  failure and recovery are almost unnoticeable.

12
File Replication

• Several copies of a file's contents at different
  locations enable multiple servers to share the
  load of providing the service
• Naming scheme maps a replicated file name to a
  particular replica.
• Updates

13
Current Project

• HDFS Hadoop Distributed File System
• Distributed parallel fault tolerant file system.
  It is designed to reliably store very large files
  across machines in a large cluster.
• Efficient, reliable, and open source

14
(No Transcript)
15

• Naming central metadata server
• Synchronization write-once-read-many, give locks
  on objects to clients, using leases
• Consistency and replication server side
  replication, asynchronous replication, checksum
• Fault tolerance failure as norm
• Security no dedicated security mechanism

16
Future Work

• Robustness of data sharing model
• The preceding section, architecture, naming,
  synchronization, availability, heterogeneity and
  support for databases
• Security

17
Reference

• 1 Thanh, T.D. Mohan, S. Choi, E. SangBum
  Kim Pilsung Kim. 2008Networked Computing and
  Advanced Information Management. A Taxonomy and
  Survey on Distributed File Systems
• 2 Randy chow,1997,Distributed operating systems
  Algorithms
• 3 Eliezer Levy, Abraham Silberschatz. December
  1990 Computing Surveys (CSUR) , Volume 22 Issue
  4. Distributed file systems concepts and
  examples.
• 4http//hadoop.apache.org/common/docs/current/hd
  fs_design.htmlIntroduction
• 5http//www.snia.org/events/wintersymp2009/cloud
  /dhruba_hadoop_snia.pdf

18

• 6http//en.wikipedia.org/wiki/List_of_file_syste
  msDistributed_file_systems
• 7http//en.wikipedia.org/wiki/HadoopHadoop_Dist
  ributed_File_System
• 8http//www.cs.gsu.edu/cscyqz/courses/aos/slide
  s08/ch6.1-Fall08.pptx

19
QA?
20
Thank you!