MOPS: an Infrastructure for Examining Security Properties of Software

1
MOPS an Infrastructure for Examining Security
Properties of Software

• Authors
• Hao Chen and David Wagner
• Appears in
• ACM Conference on Computer and Communications
  Security, 2002
• Presented by
• Peter Matthews

2
Software Vulnerabilities

• Several causes
• Certain bugs like buffer overflows may be avoided
  by use of a type-safe language or use of
  bounds-checking libraries
• Another class of security bugs involve misuse of
  higher level semantics
• Many OS system calls have implicit constraints on
  how they should be called
• If this is not done, vulnerabilities may be
  introduced
• This the class MOPS addresses

3
An example

• // Current directory is /var/ftp/
• chroot(/var/ftp/pub)
• chdir(/var/ftp/pub)
• filename read_from_network()
• fd open(filename, O_RDONLY)

/
/var
/etc
After chdir() call, current working directory is
still /var/ftp If filename //etc/passwd,
will not restrict access! Must call chdir(/)
after chroot() to ensure security
/var/ftp
/etc/paswd
/var/ftp/pub
4
Temporal Safety Properties

• Dictates the order of a sequence of
  security-relevant operations
• Can encode many rules of secure programming
• Violating such properties may render a program
  vulnerable to attack
• Manual checking difficult in large programs
• Detecting violations of these properties or
  verifying the satisfaction of them would
  significantly help reduce the frequency of
  software vulnerabilities

5
MOPS

• A program analysis tool that allows one to make
  temporal safety properties explicit and verify
  whether they are properly respected by source
  code
• Determines any execution path through a program
  that may violate a security property
• Uses techniques from model checking and program
  analysis to do so

6
(Very) Short Formal Model Review

• Finite State Automaton (FSA)
• Pushdown Automaton (PDA)
• Adds a stack, and allows transitions to interact
  with this stack

Input String
1
1
1
1
0
0
S1
S2
0
0
Stack
7
Formal Model

• The set of security-relevant operations
• All sequences of security
  operations that violate the security property
• The set of all feasible execution
  paths of the program
• Assume that B is a regular language
• There exists a FSA, M, that accepts B
• Assume that T is a context-free language
• There exists a PDA, P, that accepts T

8
Formal Model Continued

• Question is now
• This is also a context free language
• Efficient algorithms to compute intersection of
  PDA and FSA, and also to determine if the
  language accepted by a PDA is empty
• May give false alarms, but will not overlook a
  violation of the security property
• General problem is undecidable
• Authors claim that false alarms are minimized

9
Example FSA
chdir(/)
S3
S2
S1
Other
chroot
Other
10
Another example of a Temporal Security Property

• Since a privileged process has full access
  permission to the system, it should not make
  system calls that run untrusted programs without
  first dropping all privileges
• One such system call execl()
• Should first call seteuid(!0)

11
Modular Implementation

• Model process privilege
• Model execl()

Seteuid(0)
S2
S1
Other
Other
Seteuid(!0)
Other
S2
S1
Execl()
Other
Execl()
12
Modular Composition

• Mops automatically combines the two

13
Modeling Execution

• MOPS models control flow
• e.g. IF-THEN-ELSE
• MOPS pursues all paths
• MOPS does not model data flow
• Does not provide offending input
• Could possibly pursue impossible paths
• Pointer points to position of next statement
• Stack records return addresses of function calls

14
Example Violating Program
1.
2.
5.
3.
4.
15
Execution Path Output
16
Modeling Operating System Semantics

• Difficult to construct accurately because of
  increasing complexity, differences among
  operating systems, and incomplete/incorrect
  documentation
• Instead construct directly from kernel code
• First, determine the relevant kernel variables,
  create states in FSA based on these variables
• Done by hand
• Then, determine the transitions among these
  states
• Done by MOPS, state-space explorer automatically
  creates all transitions

17
Testing Real Applications

• Performance
• Sendmail 8.12.0 has 53k lines of code
• MOPS spent 110 seconds parsing the source code,
  and 95 seconds in model checking
• Checking proper dropping privilege
• sendmail 8.10.1 fails to drop privilege in user
  IDs
• sendmail 8.12.0 fails to drop privilege in group
  IDs
• Known vulnerabilities

18
Strengths of Approach

• Proposes concept of temporal security properties
  novel application of automata theory
• Provably sound approach guarantees violations
  will be found, given some constraints
• Scalable, efficient algorithm
• Can be used as a verification and bug-finding
  tool
• Expansible, customizable approach
• Comes with numerous example FSAs
• Missing string termination in context of strncat
  strncpy
• Insecure tempfile creation

19
Weaknesses of Approach

• Can only test portable, single-threaded
  applications without dynamic code generation
• Data flow insensitivity means does not handle
  function pointers, signal handlers, and non-local
  jumps
• Possibility of false positives due to modeling
  imprecision
• Construction of further FSAs is non-trivial

20
Possible Improvements

• Pursue incorporation of data flow analysis into
  model checking
• Creation of repository of temporal safety
  property models, allow access to and contribution
  by members of the public

21
Questions?