Cyclone Memory Management

1
Cyclone Memory Management

• Greg Morrisett
• Cornell University Microsoft Research

2
Joint Work

• Trevor Jim, Dan Grossman, Mike Hicks,
• Yanling Wang, James Cheney.

3
The Cyclone Project

• Cyclone is a type-safe dialect of C
• Started with C syntax and semantics.
• Threw out things that could lead to type errors
• Unsafe casts, unions, pointer arithmetic,
  deallocation,
• Yields a safe but very small language. ?
• Making up for missing features through
• An advanced type system (polymorphism, regions,
  )
• An intra-procedural flow analysis (def.
  assignment,)
• New language features (tagged unions, fat
  pointers,)
• Run-time checks (array bounds checks,)

4
Cyclone in Practice

• Cyclone compiler, tools, libraries
• Over 100 KLOC
• Eating our own dog food ensures some degree of
  quality, usability.
• Windows and Linux device drivers
• UPenn In-kernel Network Monitoring
• Cornell Maryland MediaNet
• Leiden Institute Open Kernel Environment
• Moving on to embedded systems.
• Utah RBClick Router

5
This Talk

• Region-based Memory Management
• Type-safe stack allocation
• Interaction with ADTs and polymorphism
• Adding lexical arena allocation
• Adding dynamic arena allocation
• Integrating unique pointers
• Flow analysis
• Sharing unique objects
• Temporary aliasing

6
Spot the problem

• char itoa(int i) char buf20sprintf(buf,"d"
  ,i)return buf
• Most compilers will warn you of this

7
But Consider...

• list_t global NULL
• // inserts item into global list
• void insert(char item)
• list_t e (list_t)malloc(sizeof(struct List))
• e-gthd item
• e-gttl global
• global e
• void foo(int x) char buf20sprintf(buf,
  "d", x)insert(buf)

8
Regions

• We use a region-based type system to prevent
  dereferencing a dangling pointer.
• Based on work by Tofte Talpin plus others.
• Each lexical block is given a unique
  (compile-time) name called a region.
• Each pointer type indicates the region into which
  the value points.
• Only pointers into live regions can be
  dereferenced.
• Region polymorphism ensures reusable code.
• Region inference and default region annotations
  minimize the burden on the programmer.

9
Points what you think you wrote

• typedef struct Point int x,y pt
• void addTo(pt a, pt b)
• a-gtx b-gtx
• a-gty b-gty
• return a
• pt p 1,2
• void main()
• pt q 3,4
• pt pptr p
• pt qptr q
• addTo(pptr, qptr)

10
Points what you really wrote

• typedef struct Point int x,y pt
• void addToltr1,r2gt(pt r1 a, pt r2 b r1, r2)
  
• a-gtx b-gtx
• a-gty b-gty
• pt p 1,2
• void main() main
• pt q 3,4
• pt Heap pptr p
• pt main qptr q
• addToltHeap,maingt(pptr, qptr)

point p is allocatedin the heap region (Heap)
point q is allocatedin main's region (main)
the region of an object is reflected in a
pointer's type
11
Points Continued

• typedef struct Point int x,y pt
• void addToltr1, r2gt(pt r1 a, pt r2 b r1, r2)
  
• a-gtx b-gtx
• a-gty b-gty
• pt p 1,2
• void main() main
• pt q 3,4
• pt Heap pptr p
• pt main qptr q
• addToltHeap,maingt(pptr, qptr)

This function is parameterized by two regions
corresponding to the two pointers passed in. By
default, we assume such regions are live across
the call.
Any caller has to prove that the regions will be
live.
12
Points Inferred

• typedef struct Point int x,y pt
• void addTo(pt a, pt b)
• a-gtx b-gtx
• a-gty b-gty
• pt p 1,2
• void main()
• pt q 3,4
• pt pptr p
• pt qptr q
• addTo(pptr, qptr)

Missing regions in prototypes are replaced with
a fresh type variable (just like
type-polymorphism). We generalize over
the region variables for function arguments.
We perform local type inference. Result very
few region annotations.
13
Region Subtyping

• Because blocks are allocated in a LIFO fashion,
  we can safely treat a pointer into region r1 as a
  pointer into r2 if r1 was defined before r2.
• int foo(int r1 x, int r2 y)
• foo int foo z
• if (rand()) z x // r1 lt
  foo
• else
• z y // r2 lt foo
• ...

14
Region Subtyping, contd.

• Programmers can specify region ordering relations
  as pre-conditions to functions.
• int r1 foo(int r1 x, int r2 y r2ltr1)
• if (rand()) return x
• else
• return y
• Note the Heap outlives all regions

15
Dangling Pointers Revisited

• char ??? itoa(int i)
• itoa char buf20 sprintf(buf,"d",i)
  return buf // buf charitoa
• The buffer lives in region itoa.
• But itoa is not in scope for the return type.
• And itoa does not outlive any other region.
• Ergo, theres no way to make the example
  type-check.
• It seems as though pointers cannot escape the
  scope defining their lifetimes.
• Or can they?

16
Ensuring Soundness

• With out a mechanism for hiding regions in types,
  it would be sufficient to check if a region is in
  scope to determine whether or not it is still
  live.
• But Cyclone includes support for polymorphism and
  abstract data types (?, ?) which provide ways to
  hide regions within abstracted types.
• Closures and objects also hide types.
• ? can be used to encode closures, objects
• We must somehow ensure these pointers aren't
  dereferenced after the region is deallocated.

17
Live Region Sets

• We keep track of the set of live regions (always
  a subset of those in scope).
• Analogous to the effects used by Tofte Talpin.
• To access an ADT that hides some abstract set of
  regions S, you must present evidence that S is a
  subset of the currently live regions.
• Achieved by lifting region subtype constraints to
  sets of regions r lt S meaning if r is live,
  then so are all the regions in S.
• For details, see PLDI02 paper.

18
Beyond Stack Allocation

• Thus far, the only way to allocate an object is
  by declaring it as a local variable and taking
  its address (stack allocation).
• Pros
• No run-time checks -gt all errors caught at
  compile time.
• Covers caller allocates callee reads/writes.
• Easy to determine space bounds (w/o recursion).
• Constant time deallocation.
• Sufficient for lots of systems code!
• Cons
• Caller doesnt know how much to allocate --
  gets().
• Lifetimes of objects are constrained to LIFO.

19
Lexical Arena Allocation

• char r rgets(handle_tltrgt)
• char r append(handle_tltrgt, char
  r1, char r2)
• void bar(int n)
• regionltrgt r
• char r str1 rmalloc(r,6)
• strncpy(str1, Hello , 5)
• regionltsgt s
• char s str2 rgets(s)
• str1 append(r,str1,str2)
• printf(s,str1)

20
Lexical Arena Allocation

• char r rgets(handle_tltrgt)
• char r append(handle_tltrgt, char
  r1, char r2)
• void bar(int n)
• regionltrgt r
• char r str1 rmalloc(r,6)
• strncpy(str1, Hello , 5)
• regionltsgt s
• char s str2 rgets(s)
• str1 append(r,str1,str2)
• printf(s,str1)

Declares a new region r, whose lifetime
corresponds to the block. The variable r is a
handle for allocating in r.
21
Lexical Arena Allocation

• char r rgets(handle_tltrgt)
• char r append(handle_tltrgt, char
  r1, char r2)
• void bar(int n)
• regionltrgt r
• char r str1 rmalloc(r,6)
• strncpy(str1, Hello , 5)
• regionltsgt s
• char s str2 rgets(s)
• str1 append(r,str1,str2)
• printf(s,str1)

rmalloc(r, n) allocates n bytes in the region
for which r is a handle.
22
Lexical Arena Allocation

• char r rgets(handle_tltrgt)
• char r append(handle_tltrgt, char
  r1, char r2)
• void bar(int n)
• regionltrgt r
• char r str1 rmalloc(r,6)
• strncpy(str1, Hello , 5)
• regionltsgt s
• char s str2 rgets(s)
• str1 append(r,str1,str2)
• printf(s,str1)

Declares a region s with handle s.
23
Lexical Arena Allocation

• char r rgets(handle_tltrgt)
• char r append(handle_tltrgt, char
  r1, char r2)
• void bar(int n)
• regionltrgt r
• char r str1 rmalloc(r,6)
• strncpy(str1, Hello , 5)
• regionltsgt s
• char s str2 rgets(s)
• str1 append(r,str1,str2)
• printf(s,str1)

rgets allocates its result in s by using the
handle it is passed.
24
Lexical Arena Allocation

• char r rgets(handle_tltrgt)
• char r append(handle_tltrgt, char
  r1, char r2)
• void bar(int n)
• regionltrgt r
• char r str1 rmalloc(r,6)
• strncpy(str1, Hello , 5)
• regionltsgt s
• char s str2 rgets(s)
• str1 append(r,str1,str2)
• printf(s,str1)

Same for append, except the result for this call
goes in r.
25
Lexical Arena Allocation

• char r rgets(handle_tltrgt)
• char r append(handle_tltrgt, char
  r1, char r2)
• void bar(int n)
• regionltrgt r
• char r str1 rmalloc(r,6)
• strncpy(str1, Hello , 5)
• regionltsgt s
• char s str2 rgets(s)
• str1 append(r,str1,str2)
• printf(s,str1)

Storage for s is deallocated here (i.e., str2).
26
Lexical Arena Allocation

• char r rgets(handle_tltrgt)
• char r append(handle_tltrgt, char
  r1, char r2)
• void bar(int n)
• regionltrgt r
• char r str1 rmalloc(r,6)
• strncpy(str1, Hello , 5)
• regionltsgt s
• char s str2 rgets(s)
• str1 append(r,str1,str2)
• printf(s,str1)

Storage for r is deallocated here.
27
Runtime Organization
Regions are linked lists of pages. Arbitrary
inter-region references. Similar to
arena-style allocators.
runtime stack
28
Lexical Arenas

• Based on Tofte-Talpin's MLregions
• Regions introduced with lexical scope (i.e., have
  LIFO lifetimes)
• Can support O(1) memory management operations.
• Adds dynamic allocation.
• Supports callee allocates idioms.
• Arenas are often used in C programs (e.g., LCC
  Apache)
• Beats the crap out of Real-time Java proposals.
• Unlike the ML-Kit
• Programmer controls where objects are allocated
  instead of implicit inference.
• In practice, ML-Kit requires coding idioms.
• More code, but more control, easier to reason
  about space requirements.

29
But Still Too Limited

• LIFO arena lifetimes is still too strict for many
  programs.
• No notion of tail-call for regions.
• Makes it hard to code iterative algorithms where
  state must be transferred from the previous
  iteration to the next (e.g., a copying
  collector).
• Data lifetimes are statically determined
• Consider a server or gui that creates some state
  upon a request, and only deallocates that state
  upon a subsequent request.
• Creating/destroying a region is relatively
  expensive.
• Must install exception handler/deconstructor to
  ensure memory is reclaimed upon an uncaught
  exception.
• NB real-time Java has same troubles

30
To Address these Issues

• Dynamic arenas
• Can allocate or deallocate the arena at will.
• But an extra check is required for access.
• (Checks can be amortized.)
• Unique pointers
• Lightweight (single-object) regions
• Arbitrary lifetimes.
• But restrictions on making copies
• (Restrictions can be temporarily lifted.)

31
Dynamic Arenas

• dynhandle_tltrgt
• Think a possibly NULL pointer to a region.
• it will be NULL if the region has been freed.
• Three operations
• dynregion()returns ? r.dynhandle_tltrgt. Note
  that r is not in the live set.
• region h open(dynhandleltrgt) ... dynamically
  checks that r has not been freed, records that
  the region is opened, and then grants access to
  the region for the specified scope.
• free(dynhandleltrgt)checks that the region r is
  not open and has not been freed, and then
  deallocates the storage.

32
Notes on Dynamic Arenas

• Weve traded some potential for dynamic failure
  for greatly increased flexibility.
• So flexible, in fact, its possible to write a
  copying garbage collector within the language
  (see Wang Appel, Monnier Shao, etc.)
• You can amortize the dynamic checks.
• Can think of opening a dynamic region as trying
  to map a page (put it in your TLB) for some
  scope. Subsequent access within the scope is
  then cheap.
• Opening a dynamic region is a convenient
  synchronization point for shared data among
  threads.
• See D. Grossmans TLDI paper and up-coming
  thesis.
• Its easy to support asynchronous revocation.
• See C. Hawblitzels thesis.

33
Unique Pointers

• Arenas are relatively expensive
• Must install/tear-down exception handler.
• Good for batching up allocation/access.
• Too heavyweight for individual objects.
• Unique pointers provide very lightweight
  anonymous regions.
• TU ??r.Tr.
• Can create and destroy at will.
• But there are strong restrictions on creating
  copies of a unique pointer to ensure uniqueness.

34
Copying Kills

• void foo(int U x)
• int U y x
• y 3
• free(y)
• x // oops!
• Flow analysis prevents old copies of unique
  pointers from being used, so this is flagged as a
  compile-time error.

35
Flow Analysis

• Tracks whether a value is defined at each point.
• Needed to ensure you dont use an uninitialized
  value.
• Copying a unique value makes the source undefined
• void foo(int UU x)
• int U y// y undefined
• y x // y defined, but x undefined
• y 3 // okay, y is defined
• f(y) // y now undefined
• y // compile-time error!

36
Joins

• void foo(int U x)
• if (rand())
• free(x)
• x // x undefined
• The analysis conservatively considers a value
  undefined if its undefined on any path.
• This can lead to leaks, so we warn if its
  undefined on one path, but not another.
• Considered making it an error, but too many false
  positives, and it prevents the next feature

37
Sharing Unique Pointers

• void foo(int Ur x)
• int Ur y x
• free(x) // x undefined
• // so, y undefined
• To be sound, it seems we must have completely
  accurate, global alias information or else
  prevent unique pointers from being placed in
  shared objects.
• The latter approach is the norm for linear types.

38
Swap to the Rescue

• We allow placing unique objects in shared ones,
  but the only way to extract the unique object is
  by using an atomic swap.
• int foo(int Ur x)
• int Ur y x
• int U temp malloc(sizeof(int))
• temp 3
• y temp
• free(temp)
• return x

39
Unifying U and r

• With swap, unique pointers are great!
• Supports sharing (even among threads).
• Yet also supports grabbing a unique object.
• In turn, provides fine-grained memory mgmt.
• But uniqueness is still a strong constraint.
• Have to write functions so they return the
  arguments they dont consume.
• Example list length or list map.
• I should be able to code these once, and use them
  for unique lists or shared ones.
• But to traverse a unique list, you have to use
  swaps and reverse it as you crawl over it, and
  then reverse it again on the way back up.

40
Alias Declarations

• alias ltrgt x y
• When y has type TU and r is fresh
• Makes a copy of y and binds it to x.
• The type of x is Tr (i.e., shareable)
• So x can be freely copied or traversed
• But r isnt in scope outside so no copy can
  escape.
• y is undefined within so it cant be freed.
• After , y becomes defined again.

41
Notes on Alias Declarations

• Generalization of Wadlers Let-!
• Closer to Walker Watkins Let-region
• But we support a form of deep aliasing
• TU can be treated as Tr as long as T is a
  covariant type constructor.
• E.g., listltUgt can be treated as listltrgt
  throughout the scope of the alias declaration.
• This makes it possible to write libraries where
  much code can be shared.

42
From the List Library

• mlist_tlta,rgt, // mutable list lt
• list_tlta,rgt // immutable lists
• void freelist(mlist_tlta,Ugt x)
• int length(listlta,rgt x)
• void foo(mlist_tltint,Ugt x)
• int i
• aliasltrgt list_tltint,rgt y x in
• i length(x)
• freelist(x)

43
Compiler Can Often Infer Alias

• mlist_tlta,rgt, // mutable list lt
• list_tlta,rgt // immutable lists
• void freelist(mlist_tlta,Ugt x)
• int length(listlta,rgt x)
• void foo(mlist_tltint,Ugt x)
• int i
• i length(x) // alias inferred!
• freelist(x)

44
Summary

• Cyclone provides flexible, real-time,
  user-controlled memory management with static
  type safety guarantees.
• Stack lexical arenas require no checks, but
  only support static lifetimes.
• Dynamic arenas support arbitrary lifetimes, but
  require some run-time checks.
• Unique pointers support lightweight regions and
  arbitrary lifetimes, but have restrictions on
  sharing.
• Both dynamic arena and unique pointers can be
  temporarily treated as lexical pointers.
• Crucial for building re-usable libraries.
• The region-based type system provides a unifying
  framework.

45
What Next?

• Reference counting
• Some preliminary support based on unique
  pointers.
• Bounded arenas
• I really think the real-time and embedded guys
  will like this stuff.
• Better type error messages!
• Very, very, very, very hard
• Region constraints (instead of parameters)
• Like where clauses in ML modules

46
For more info

• Download the code
• www.cs.cornell.edu/projects/Cyclone
• www.research.att.com/projects/Cyclone