Operating Systems: Exokernel 2001 Fall

1
Operating Systems Exokernel2001 Fall

• Instructor Joonwon Lee

2
Overview

• What kind of paper is this?
• More than just a system description.
• Justifying a big, new idea.
• A bit of a retrospective.
• Performance study as well.
• They really packed a lot in!
• The Big Idea
• Proposed Exokernels in 1995 (export hardware
  interfaces).
• Now have experience with them have built several
  and know what the virtues and limitations are.
• Built a UNIX-compatible library OS.
• Demonstrate that it runs regular UNIX apps as
  fast (or faster) than a native implementation.
• Shows that the interfaces are flexible enough to
  do very cool stuff!
• Key is letting the kernel provide protection, but
  letting applications do everything else.

3
Application Control

• why kernel/privileged servers are inadequate?
• servers in kernel is slow
• servers in user space may enhance performance but
• they do not help applications control the way
  they access system resources
• Exokernel accommodates traditional techniques
  like
• fast IPC techniques
• move server code to library less code sharing
• mapping read-only shared data
• batching system call
• Kernel extension techniques are adequate for
  flexible kernel
• code can be downloaded into the kernel, the issue
  is the safely
• type safety
• software fault isolation

4
Exokernel Principles

• Separate protection (exokernel) from management
  (library OS).
• Expose allocation (make applications explicitly
  allocate resources).
• buffer cache, pages, network buffer,..
• Expose names (use physical names).
• name translation is costly and race-prone
• Expose revocation (applications give up resources
  at their discretion).
• application has control over which resources to
  release
• Expose information (collect data that is
  difficult for applications to obtain and expose
  it to them).
• network buffer
• file caching pages
• RLU information of pages used by an application

5
Support for protection

• All access control is uniform.
• Bind hardware together with software abstractions
  (e.g., tie together buffer cache and physical
  memory).
• Allow downloaded code where necessary, and
  protect it.

6
Protected Sharing

• Mechanisms
• software regions protected areas of memory
• read/write via syscall
• fault isolation
• on the fly hierarchically-named capabilities
• present capability on every access
• wakeup predicates downloaded predicates used to
  wakeup processes when conditions become true
• prevent processes from being hung by malicious
  ones
• robust critical sections disable software
  interrupts
• robust than locks

7

• Optimizations
• Mutual trust (common case) applications that
  share resources usually trust each other (e.g.,
  shared memory of Unix process).
• fault isolation is good enough
• Unidirectional trust (client/server case)
• e.g.) child process is not allowed to change the
  privilege of a shared page to write
• may increase page faults
• Mutual distrust unrelated processes.
• libOS must enforce defensive protection
• this case is infrequent

8
XN Disk Subsystem

• Goals
• multiplex disks among multiple libFS(part of
  libOS)
• multiple libFS may share the same file with
  different semantics
• allow libFS as much as control over file
  management
• protect files from unauthorized accesses
• Requirements
• creation of new file format should be
  light-weight
• multiple libFS can safely share at the bock or
  meta data level
• this subsystem should be as fast as raw disk
• allow libFSs to share buffer cache
• cache coherence
• security
• concurrency

9
XN

• Challenges safe sharing at block level
• Unix knows the files metadata format but
• exokernel allows application-defined metadata
  format
• UDFs (Untrusted Deterministic Functions)
• Should be prepared for each type of meta-data.
• Functions have no internal state, simply
  transform meta-data.
• Kernel can test that the UDFs are following the
  rules (i.e., reporting only blocks that have been
  allocated to an object). .
• Three key functions per template
• owns-udfT returns list of blocks allocated
• acl-ufT returns T/F indicating if the requested
  operation is allowed
• size-ufT returns size of the meta-data
  structure.

10
UDF Usage

• LibFS asks xok for a permission for a file
  operation with its own metadata type which xok
  does not understand
• xok finds a template for the kind of metadata
• templates contains one UDF and two
  nondeterministic functions
• e.g.) Unix has templates for data blocks, i-node
  blocks, i-nodes, indirect blocks,
• xok calls an appropriate function in the template

11
XN

• Design and Implementation
• Never reuse on-disk structure while retaining
  pointers to it. (defer deallocation until all
  references are removed.)
• Never create pointers to objects until the
  objects are initialized.
• During a move, do not delete old pointer before
  setting new pointer. (Crash time reference
  counting.)
• Buffer Cache Registry
• Allows distrustful library file systems to share
  a cache.
• handles coherence problems
• Access control is checked when a cache page is
  mapped into a library FS.
• for efficiency
• Replacement is on virtual pages, so applications
  can implement application-specific page
  replacement.
• C-FFS is a colocated FFS, implemented on top of
  XN.
• colocation of metadata and data, etc

12
Xok

• x86-based.
• Scheduling is round-robin time slices.
• Page table modifications must go through system
  calls (because page tables are implemented in
  hardware).
• Use wake-up predicates for user-level events.
• bulk of OS functions are in user-level
• applications inject predicates into the kernel

13
ExOS

• Mostly 4.4-compatible.
• UNIX kernel state is split into shared among all
  UNIX processes and private per process.
• Makes extensive use of shared memory.
• Fork implemented via copy-on-write.
• Signals on top of Xok ipc
• The kernel is essentially a shared library that
  gets linked into a reserved portion of a process
  address space.

14
Performance

• Benchmarks are standard UNIX utilities for the
  most part.
• Base System
• Common utilities perform as well as those on a
  native UNIX (OpenBSD).
• Some utilities perform noticeably better on
  three applications (but don't know why).
• File System Benefits
• C-FFS provides better performance (in OpenBSD as
  well as ExOS).
• The cost of protection
• Not everything is implemented, so the authors add
  3 extra system calls per write to shared tables.
• Compare software-region protection to no
  protection.
• still much cheaper than OpenBSD
• Protection is a function of the amount of data
  (although it seems odd that the protected 8KB
  test is faster than the unprotected one).

15
Experience and Lessons

• Exposing the hardware makes it easier to add
  system optimizations.
• Downside of low-level interfaces means more
  recompiling for important applications.
• Building/debugging application OS's is easier
  than building/debugging real OS's.
• Interface design is hard (this lesson is found in
  many of the related projects as well).
• Poorly chosen abstractions lose informaton.
• Leave room in kernel structures for
  application/OS information.
• Microbenchmark performance is not necessarily
  important.
• Turning off interrupts for critical sections can
  be a performance win.
• Putting page tables at user-level is tricky.
• In general, downloaded code is powerful.