The Impact of Programming Language Theory on Computer Security

1
The Impact of Programming Language Theory on
Computer Security

• Drew Dean
• Computer Science Laboratory
• SRI International

2
What Im not Talking About

• Cryptographic Protocol Verification
• See, e.g., Computer Security Foundations Workshop
• Type Systems for Non-Interference
• See, e.g., POPL

3
Much of security is

• Program P exactly implements Specification S and
  no more.
• For this talk, we assume that the specification
  is correct

4
Security Tripos
Correctness wrt critical requirements
No undefined user mode behavior
Proper system call use
5
Correctness wrt Security

• No system that misses security checks can be
  secure
• Program Verification
• Architectural Support
• Stack inspection
• Security Passing Style WAF

6
Program Verification

• Obvious connections
• Lambda calculus, Curry-Howard
• Hoare Logic

7
Architectural Support

• Stack Inspection
• Access control based on endorsement of code
  answers Who called me?
• Designed to prevent untrusted code from bypassing
  access controls, while allowing higher level code
  to assert that it knows what its doing

8
Stack Inspection Example

• Applet wants to use the Helvetica font
• May require JVM to read a file
• Solution
• Font handling code checks arguments
• If successful, asserts privilege
• Attempts to read file
• Which notes that font code (privileged) has
  asserted everythings OK

9
Stack Inspection Critique

• Exposes call stack
• Tail call elimination painful
• Function inlining also painful
• Goodbye, Church-Rosser, goodbye!

10
Security Passing Style

• Wallach, Appel, Felten, TOSEM 9/00
• A la CPS, pass security context as an extra
  (implicit) argument
• Restores tail call elimination and function
  inlining
• Doesnt restore Church-Rosser

11
Observation

• SPS is in closer analogy to CPS than its authors
  say
• Shivers Threads are paths through continuation
  space
• Continuations are the right semantic object to
  attach permissions to
• Would a dependent type system work out?

12
Properly Using System Calls

• If a program handles its own security, e.g.,
  ftpd, it better use system calls correctly
• Many programs dont
• Wu-ftpd
• Sendmail

13
How Can PLT help?

• Joint work with David Wagner and Hao Chen, UC
  Berkeley
• Given a program, morph control flow graph into an
  automaton that accepts language of system calls

14
IEEE SP 2001

• Take automaton, check runtime trace of system
  calls for anomaly detection
• (Most of) Benefits of specification-based
  intrusion detection without needing the
  non-existent spec

15
Current Work

• Take abstracted specification, throw it and
  library of security best practices (and known
  attacks) at (custom) model checker
• But this requires understanding system calls
• Usually the POSIX spec is reasonable
• But not for setuid()

16
Understanding setuid

• Absolutely necessary for writing secure setuid
  Unix programs
• Linux, FreeBSD, Solaris all subtly different
• Even if all POSIX compliant
• Kernel code unreadable
• Reverse engineer formal model
• Will appear at USENIX Security 2002

17
No Undefined User-mode Behavior

• Buffer overflows are still a problem in 2002
• PL people think this is stupid
• It is
• Like it or not, most of the world codes in C or
  unsafe C

18
Not Just Buffer Overflows

• Any corruption of program state can cause
  vulnerability
• Nearly science fiction attack based on a C
  program double freeing a pointer

19
Observation

• Memory comes in two colors
• Storage of variables
• Compiler/runtime support

20
Partition Property

• All variables only refer to memory locations
  that the compiler has mapped to program
  variables, not compiler/runtime support (e.g.,
  return addresses, temporaries for evaluating
  expressions, memory management overhead, etc.)

21
Partition Properties

• Note that this is weaker than non-interference
• Values obviously depend on program values
• Stronger than some forms of memory type safety
• Should be a theorem of modern (safe) languages

22
Conclusions

• This was a brief survey of a wide field
• and no more is hard to implement
• Hopefully, breaking it down helps

Correctness wrt critical requirements
No undefined behavior
Proper system call use