Unionfs: User and CommunityOriented Development of a Unification File System

1
Unionfs User- and Community-Oriented Development
of a Unification File System
David Quigley, Josef Sipek, Charles P. Wright,
Erez Zadok Stony Brook University
2
Quick Summary

• Unionfs is a stackable file system
• Merges contents of several directories
• Licensed under GPL
• XXX word nicely copy-up is good
• Available at www.unionfs.org

3
Overview

• Introduction
• Design Issues
• Performance
• Usage Case Studies
• Status
• Future Work
• Conclusion

4
Motivation

• Merge contents of several directories (branches)
• Keep their physical contents separate
• Maintain Unix semantics
• Allow arbitrary branch manipulations
• Add branch
• Remove branch
• Change branch permission mode
• rw, ro

5
Background
User Level
User Process
read()
sys_read()
vfs_read()
Virtual File System
Kernel Level
ext2_read()
wrapfs_read()
Ext2
Wrapfs
ext2_read()
6
Sample Stackable File systems
7
Unionfs Fan-out Stacking
User Process
User Level
read()
sys_read()
vfs_read()
Kernel Level
Virtual File System
ext2_read()
unionfs_read()

• Add/remove branch
• Duplicates
• File changes on RO file systems

Ext2
Unionfs
Upper FS
rw
rw
ro
rw
Ext2
NFS
ISO9660

Lower FS
Left-most (highest priority)
Right-most (lowest priority)
8
Branches Example
Unionfs
alice
bob
carol
alice
bob
carol
fsl/dave
fsl/jeff
fsl/jeff
fsl/dave
branch 3
branch 1
branch 2
9
Overview

• Introduction
• Design Issues
• Performance
• Usage Case Studies
• Status
• Future Work
• Conclusion

10
Design Issues

• Duplicates
• Whiteouts
• Copy-up

11
Duplicates

• Problem A file with the same name may exist on
  several branches
• Solution
• readdir simply eliminate duplicate names
• lookup stop at the first instance

12
Duplicates Example
Unionfs
alice
bob
bob
alice
bob
bob
fsl/dave
fsl/jeff
fsl/jeff
fsl/dave
branch 3
branch 1
branch 2
13
Whiteouts

• Problem What happens if one wants to remove a
  file which resides on a read-only branch?
• Solution
• File is masked out by a whiteout
• Whiteout created in a higher priority branch
• readdir remove all instances of the name do
  not list the whiteout
• lookup return -ENOENT

14
Whiteouts Example
Unionfs
alice
bob
carol
alice
bob
carol
.wh.carol
.wh.carol
fsl/dave
fsl/jeff
fsl/jeff
fsl/dave
Whiteout!
branch 3
branch 1
branch 2
15
Copy-up

• Problem What if user wants to modify a file on a
  read-only branch?
• Solution
• Copy the original file up to a higher priority
  branch
• Let the write through on the new copy

16
Copy-up Example
bob
Unionfs
Read-only branch
Read-only branch write fails!
Duplicate eliminated
rw
ro
rw
alice
bob
carol
alice
bob
carol
bob
fsl/dave
fsl/jeff
fsl/jeff
fsl/dave
bob
branch 3
branch 1
branch 2
17
Overview

• Introduction
• Design Issues
• Performance
• Usage Case Studies
• Status
• Future Work
• Conclusion

18
Performance

• I/O-intensive benchmark
• Postmark simulates the operations of electronic
  mail servers
• Benchmark configuration
• 20,000 files
• 512-10,240K byte files
• 200,000 transactions
• 200 subdirectories
• CPU-intensive benchmark
• OpenSSH compile

19
Postmark
20
OpenSSH Compile (dup)
1.16 Elapsed time
2.41 Elapsed time
10.7 Elapsed time
5.26 Elapsed time
3.25 System time
14.0 System time
5.63 System time
10.3 System time
21
OpenSSH Compile (dist)
0.99 Elapsed time
1.12 Elapsed time
1.66 Elapsed time
2.10 Elapsed time
4.42 System time
4.85 System time
6.69 System time
7.59 System time
22
Overview

• Introduction
• Design Issues
• Performance
• Usage Case Studies
• Status
• Future Work
• Conclusion

23
Usage Case Studies

• Live CDs
• SLAX, KNOPPIX, and many more
• NFS/disk-less environment
• Bluetopia (IBM)
• Many other sysadmins
• Snapshotting
• snapmerge

24
Live CDs

• Provide user the illusion of being able to write
  to the files stored on the CD-ROM
• Generally, made up of one writable branch (tmpfs)
  and one read-only branch (squashfs, iso, etc.)

mount -t unionfs \ -o dirs/ramdiskrw/KNOPPIXr
o \ /UNIONFS /UNIONFS
25
NFS/disk-less environment

• Each client uses a union of a read-only base
  system and a writeable storage
• Any changes are saved to the writable branch
• Allows the base system to be shared between many
  clients

mount -t nfs server/bin /mnt/nfsbins mount -t
nfs server/store/hostname -s\ /mnt/persist mou
nt -t unionfs none /bin o \ dirs/mnt/persist/m
nt/nfsbinsnfsro
26
Snapshotting

• Create a consistent image of an entire file
  system
• When a snapshot is taken at time n
• Mark existing branches file system read-only
• Add a new high-priority branch (tn)
• All writes take place on tn
• Transparently migrate open files on tiltn to tn
• To view snapshot i union ti through t0

• Issues
• Dynamic branch management
• Scalable to many branches

Unionfs

t0
t1
t2
t3
tn
27
Overview

• Introduction
• Design Issues
• Performance
• Usage Case Studies
• Status
• Future Work
• Conclusion

28
Status

• Out-of-kernel module
• 11,627 lines of code
• Tracking Linux kernel 2.6
• Include user-level utilities and documentation
• Licensed under GPL

29
Popularity

• Included in Debian stable
• Included in Gentoo Testing/Masked
• Downloaded by more than 2800 users from 66
  countries
• Underestimates actual number of end-users

30
Users

• Live CD Distributions
• Knoppix 3.8 and newer
• SLAX
• At least two dozen more
• Assorted projects ranging from clusters to
  security applications
• IBM Bluetopia

31
Overview

• Introduction
• Design Issues
• Performance
• Usage Case Studies
• Status
• Future Work
• Conclusion

32
Future Work

• Persistent inode map
• Needed by NFS v2 v3
• invariant a file and a whiteout for that file
  cannot exist on the same branch
• unionfs_fsck would be useful
• Lower file systems cannot be modified directly
• current workaround uniondbg g
• CANNOT be solved without changes to the VFS
• unified cache manager (Heidenmen SOSP '95)

33
Overview

• Introduction
• Design Issues
• Performance
• Usage Case Studies
• Status
• Future Work
• Conclusion

34
Conclusion

• Unionfs provides a merged view of an arbitrary
  number of disparate directories
• Used by thousands of end-users

35
Questions?

• Unionfs User- and Community-Oriented Development
  of a Unification File System

David Quigley, Josef Sipek, Charles P. Wright,
Erez Zadok Stony Brook University
36
Opaque Directories

• normally, the contents of directories are merged
  and the user is provided with a unified view
• what happens if we create a new directory
  (unionfs_mkdir)? There might exist a directory on
  a lower priority branch whose contents should
  _NOT_ be merged (mkdir is supposed to create an
  _empty_ directory)
• The naive solution is to white out the "contents"
• this is inefficient (many foo_create() calls
  make inodes wasted)
• The better approach is to make the newly created
  directory "opaque" - preventing unionfs from
  going further to lower-priority branches

37
Opaque Directories Example
Unionfs
Whiteout!
Not-visible
Whiteout!
bob
carol
bob
carol
alice
alice
.wh.fsl
fsl
fsl
fsl/dave
fsl/jeff
fsl/jeff
fsl/dave
fsl/.wh._opaque
Not-visible
branch 3
branch 1
branch 2
38
Configuration

• Configfs
• Another dependency
• Already written
• Sysfs
• No new dependency
• No kobjects to represent VFS structures
• Procfs
• Sysfs is prefered
• Custom pseudofs
• No limitations
• Must be implemented from scratch
• IOCTL
• Evil
• Remount
• Complex parser in kernel-space

39
nfsro

• NFS returns -EACCES instead of
• -EROFS
• nfsro branch mode was introduced to work around
  this issue

40
TODO

• Extra slides
• Animation for common operations
• Rename
• Remove