Bugs as deviant behavior: a general approach to inferring errors in systems code

1
Bugs as deviant behavior a general approach to
inferring errors in systems code

• Dawson Engler, David Yu Chen, Seth Hallem, Andy
  Chou and Benjamin Chelf (Stanford)
• SOSP 2001
• Presented by Rimon Barr
• Systems Lunch (Cornell)
• 3 September 2001

2
i n t r o d u c t i o n

• finding bugs
• bugs exist
• correct behavior known
• sophisticated techniques
• obstacle knowing which rules to check!
• goal
• automatically extract rules
• without a priori knowledge

3
a p p r o a c h

• a bug is -
• a contradiction in beliefs
• MUST - implied by the code
• a deviation from common behavior
• MAY - inferred from the code
• probability of coincidence

4
r e l a t e d w o r k

• type systems
• find greatest number of bugs
• expressiveness of type system
• temporal and context dependence
• requires recoding
• fixed type system throughout code
• vs. ad hoc inferences

5
r e l a t e d w o r k

• type systems
• specifications
• formal verification
• model check, theorem proof or check
• complex, difficult, costly
• is the model correct?
• less heavyweight techniques
• interface level specifications
• annotations
• cost proportional to code size

6
r e l a t e d w o r k

• type systems
• specifications
• high-level compilation
• hard-wired application-level information
• fixed set of errors
• can use automatically generated rules

7
r e l a t e d w o r k

• type systems
• specifications
• high-level compilation
• dynamic invariant inference
• monitor program execution
• simple algebraic invariants
• can only see executed paths
• use profiles to rank bugs

8
b e l i e f s

• rule template
• ltagt must be paired with ltbgt
• check for violated beliefs
• MUST
• internal consistency
• MAY
• statistical analysis

9
i n t e r n a l c o n s i s t e n c y

• checking for null pointers
• 1 if (cardNULL) 2 printk(KERN_ERR,
  capidrv-d ... d!\n,3 card-gtcontrnr,
  id)4
• 1 ? card is NULL, on true path (2-4)
• 3 ? card is not NULL
• ... inconsistent
• Incidentally 1 ( check ) ? card is NULL or
  not NULL

10
i n t e r n a l c o n s i s t e n c y

• checking for null pointers
• 1 int mxser_write(struct tty, ...) 2 struct
  mxser_struct infotty-gtdriver_data3 unsigned
  long flags45 if (!tty !info-gtxmit_buf)6
  return 07 ...
• 1 ? tty belief set is set to unknown
• 2 ? tty belief set set to not null
• 3, 4 ? no impact, belief set propagated onward
• 5 ? belief set null and not null ... contradiction

11
i n t e r n a l c o n s i s t e n c y

• checking for null pointers
• general internal consistency
• 1. Rule template T
• 2. Valid slot instances for T
• 3. Code actions that imply beliefs
• 4. Rules for combining beliefs contradictions
• 5. Rules for belief propagation

12
i n t e r n a l c o n s i s t e n c y

• checking for null pointers
• general internal consistency
• 1. Rule template T
• 2. Valid slot instances for T
• 3. Code actions that imply beliefs
• 4. Rules for combining beliefs contradictions
• 5. Rules for belief propagation

13
i n t e r n a l c o n s i s t e n c y

• checking for null pointers
• general internal consistency
• inferring MUST beliefs
• direct observation
• variable assignment
• conditional code path
• implied pre- and post-conditions
• divide by variable ? variable non-zero
• pointer dereference ? pointer non-NULL

14
i n t e r n a l c o n s i s t e n c y

• checking for null pointers
• general internal consistency
• inferring MUST beliefs
• relating code
• by implementation
• a() calls b() ? b() must be at least as careful
• also... ? a() fails if b() fails
• by abstraction
• common interface eg. drivers, POSIX
• different versions of same routine

15
s t a t i s t i c a l a n a l y s i s

• lock ltlgt protects variable ltvgt
• 1 lock l2 int a, b3 void foo() 4
  lock(l)5 aab6 unlock(l)7
  bb18
• 5 ? MAY a,b protected by l
• 7 ? MUST b not protected by l

16
s t a t i s t i c a l a n a l y s i s

• lock ltlgt protects variable ltvgt
• 1 lock l2 int a, b...9 void bar() 10
  lock(l)11 aa112 unlock(l)13
• 11 ? MAY a protected by l

17
s t a t i s t i c a l a n a l y s i s

• lock ltlgt protects variable ltvgt
• 1 lock l2 int a, b...14void baz() 15
  aa116 unlock(l)17 bb-1 18
  aa/b19
• 15 ? MAY a protected by l
• 17 ? MUST b not protected by l
• 18 ? MUST a not protected by l

18
s t a t i s t i c a l a n a l y s i s

• lock ltlgt protects variable ltvgt
• general statistical analysis
• 1. Check all potential slot instances (MUST)
• 2. Count positive and negative matches
• 3. Augment count with rank function
• issues
• need large sample of instances
• filter the universe of slot instances

19
s t a t i s t i c a l a n a l y s i s

• lock ltlgt protects variable ltvgt
• general statistical analysis
• ranking
• z-statistic for proportions (let p00.9)
• ranking augmentation
• code trustworthiness
• inversion property T useful ? T useful.
• non-spurious principle n?1.
• latent specifications lock and unlock

20
e v a l u a t i o n

• internal null consistency
• check for null pointer errors
• implementation
• check-then-use
• use-then-check
• redundant checks
• results

21
e v a l u a t i o n

• security checker
• do not (kernel) dereference user pointer
• implementation
• dereference ? safe kernel pointer
• paranoid check ? tainted user pointer
• simultaneously safe and tainted ? error
• results

22
e v a l u a t i o n

• inferring failure
• routines not correctly checked for failure
• implementation
• assume that all functions can fail
• emit error function result ignored / unchecked
• otherwise, emit checked
• results
• IS_ERR check ! NULL pointer check

23
e v a l u a t i o n

• deriving temporal rules
• deallocation no ltagt after ltbgt
• ltbgt must follow ltagt
• implementation
• assume that every function frees its arguments
• count checks on every function-arg pair
• segregate errors by pair
• issue restrict pair space to (de)alloc routines
• use code traces
• future work machine learning

24
e v a l u a t i o n

• flexible technique
• internal null consistency
• security checker
• inferring failure
• deriving temporal rules
• what else?

25
s u m m a r y

• a bug is -
• a contradiction in beliefs
• a deviation from common behavior
• finding bugs
• internal consistency
• statistical analysis
• discussion and questions

26
Bugs as deviant behavior a general approach to
inferring errors in systems code

• Dawson Engler, David Yu Chen, Seth Hallem, Andy
  Chou and Benjamin Chelf (Stanford)
• SOSP 2001
• Presented by Rimon Barr
• Systems Lunch (Cornell)
• 3 September 2001