An Enhanced Buffer Separation Scheme to Protect Security Sensitive Data against Buffer Overflow Atta

1
An Enhanced Buffer Separation Scheme to Protect
Security Sensitive Data against Buffer Overflow
Attacks

• 2006. 2. 20.
• Researcher Lee, Seung Min
• (Presenter Lee, Seung Ick)
• High Performance Computing Laboratory at POSTECH

2
Contents

• Introduction
• Related Works
• Motivation
• Problem Definition
• Proposed Idea
• Performance Analysis
• Conclusion and Future Works

3
Introduction

• Buffer overflow
• Occur when a program or process stores more data
  in buffer than the buffer size
• Security sensitive data
• Data to be a target of buffer overflow attack for
  a successful change of control flow
• Basic steps of buffer overflow attack
• The first step
• To find, discover and identify vulnerability of
  buffer overflow
• The second step
• To overflow and overwrite security sensitive data
  near buffer
• The third step
• To change control flow of process
• The fourth step
• To execute intended instruction or process

4
Protection Methods
Related Works ( 1/2 )

• Protection Methods are classified depending on
  the step the prevention method is applied.
• The first step
• To eliminate buffer overflow vulnerability
• The second step
• To prevent overflow or overwrite data
• The third step
• To prevent no intended control flow of a program
  or process
• The fourth step
• To apply that data of buffer cant be executed
  through hardware support

5
Problems of Previous Works
Related Works ( 2/2 )

• Problem of the prevention method on the first
  step
• Generate too many false warnings and miss errors
  in the code
• Problem of the prevention method on the second
  step
• Performance degradation through array bounds
  checking
• Not to provide complete protection against
  vulnerabilities in user defined or non standard
  library code
• Problem of prevention method on the third and
  fourth steps
• Occur an exception or termination because of
  attack failure

6
Motivation ( 1/2 )

• The prevention methods applied in these steps but
  the second have the problem of process
  availability, since the process stops its
  execution because of false warnings, exceptions
  or termination.
• Our focus is to provide a reliable process
  availability and more secure protection method.
• Buffer separation approach is the method applied
  on the second step for prevention.
• Remove buffers from stack
• Allow the occurrence of buffer overflow but
  prevent security sensitive data from being
  overwritten
• Gemini and DYBOC

7
Motivation ( 2/2 )

• Shortcoming of previous works
• Gemini
• Can generate heap overflow because of buffer
  using heap area instead of stack
• DYBOC
• Has a memory overhead problem because of using
  write- protected page

8
Problem Definition

• To prevent stack and heap overflow for using
  enhanced buffer separation approach
• To have minimal performance penalty

9
Basic Idea
Proposed Idea ( 1/3 )

• Assumption
• We can know the size of arrays at compile time.
• Enhanced buffer separation schemes
• Buffer stack
• Separate buffer from the security sensitive data
  on the stack area.
• Separated meta data
• Separate meta data from buffer on the heap area.

10
Buffer Stack Architecture
Proposed Idea ( 2/3 )

• Buffer stack is to prevent against stack
  overflow.
• Buffer stack will be determined as using heap or
  stack at the compile time.

11
Separated Meta Data Architecture
Proposed Idea ( 3/3 )

• Separated meta data is to prevent against heap
  overflow.

12
Performance Analysis

• Performance comparison with Gemini, DYBOC and our
  solutions
• Limitation
• If the security sensitive data exists inside
  buffers, it is very hard to split them from the
  buffers.
• Our solutions cant prevent data from pointer
  operations that use the primitive type variables.
• Compiler has to know the size of stack.

13
Experiments

• We chose two vulnerabilities which is similar to
  the source of finger demon and PCT SSL
  vulnerability.
• We applied single thread program but it can be
  applied in multi-threaded environment easily.
• In both cases, the security sensitive data of the
  original program is overwritten and make an
  exception.
• But, the program applied our approaches correctly
  executes because it protects the security
  sensitive data even though the buffer data is
  tainted.

14
Conclusion and Future Works

• Conclusion
• Buffer stack prevents the stack overflow attack.
• Separated meta data prevents the heap overflow
  attack.
• Future works
• We must find trade-off between performance and
  memory overhead on case 1 and 2 of the Buffer
  stack architecture, and then will choose one of
  cases.
• Throughout the implementation, we can provide
  more exact performance analysis.
• We can provide more powerful solutions to change
  the main assumption security sensitive data is
  located near the buffer and also in the buffer.

15

• Thank you!Questions?

16

• Backup slide

17
References

• 1 Hiroaki Etoh and Kunikazu Yoda. Protecting
  from Stack-Smashing Attacks. Published on
  World-Wide Web at URL http//www.trl.ibm.com/proje
  ts/security/ssp/main.html, June 2000
• 2 Donaldson, Mark E. Inside The Buffer Overflow
  Attack Mechanism, Method, Prevention. April 3,
  2002. URLhttp//www.sans.org/rr/paper.php?id386
• 3 Bharath Madhusudan, John Lockwood. Design of
  a System for Real-Time Worm
• 4 H. Wang, C. Guo, D. Simon, and A. Zugenmaier.
  Shield Vulnerability-driven network filters for
  preventing known vulnerability exploits. In
  Proceedings of ACM SIGCOMM, Portland, OR, Aug.
  2004 Detection, 12th Annual Proceedings of IEEE
  Hot Interconnects 2004
• 5 J. Newsome and D. Song. Dynamic taint
  analysis for automatic detection, analysis, and
  signature generation of exploits on commodity
  software. In Proceedings of the 12th Annual
  Network and Distributed System Security
  Symposium(NDSS05), Feb. 2005
• 6 Rinard. M., Cada. C., Dumitran. D., Roy. D.,
  Leu.T. A Dynamic Technique for Eliminating Buffer
  Overflow Vulnerabilities (and Other Memory
  Errors). In Proceedings 20th Annual Computer
  Security Applications Conference (ACSAC), 2004
• 7 StackShield. http//www.angelfire.com/sk/stack
  shield
• 8 A. Baratloo, T. Tsai, and N. Singh.
  Transparent Run-Time Defense Against Stack
  Smashing Attacks. In Proceedings. of the USENIX
  Annual Technical Conference, June 2000

18
Reference (Cont.)

• 9 Crispin Cowan, Steve Beattie, John Johansen,
  and Perry Wagle. Pointguard Protecting pointers
  from buffer overflow vulnerabilities. In
  Proceedings of the 12th USENIX Security
  Symposium, Washington, D.C., August 2003
• 10 BB. Madan, S. Phoha, G. NIST, KS Trivedi,
  StackOfence A Technique for Defending Against
  Buffer Overflow Attacks, In Proceedings of the
  International COnference on Information
  Technology Coding and Computing(ITCC05), 2005
• 11 J. Xu, Z. Kalbarczyk, S. Patel, and R. K.
  Iyer. Architecture support for defending against
  buffer overflow attacks. In 2nd Workshop on
  Evaluating and Architecting Systems for
  Dependability, 2002
• 12 S Bhatkar, DC DuVarney, R Sekar, Address
  obfuscation An efficient approach to combat a
  broad range of memory error exploits, In
  Proceedings of the 12th USENIX Security
  Symposium, 2003
• 13 Wilander, J. and M. Kamkar, Comparison of
  Publicly Available Tools for Dynamic Buffer
  Overflow Prevention, 10th Network and Distributed
  System Security Symposium, 2003
• 14 Stelios Sidiroglou, Giannis Giovanidis, and
  Angelos D. Keromytis, A Dynamic Mechanism for
  Recovering from Buffer Overflow Attacks
• 15 R Hieb, RK Dybvig, C Bruggeman, L Hall,
  Representing Control in the Presence of
  First-Class Continuations, In Proceedings of the
  ACM SIGPLAN 1990 conference on 1990 16 EG
  Barrantes, DH Ackley, S Forrest, D Stefanovic,
  Randomized Instruction Set Emulation, ACM
  Transactions on Information and System Security,
  2005
• 16 Christopher Dahn, Spiros Mancoridis, Using
  Program Transformation to Secure C Programs
  Against Buffer Overflows, in Proceedings of the
  10th Working Conference on Reverse Engineering
  (WCRE03)

19
Scenario
Modified Pseudo Assembly Code
Original Pseudo Assembly Code
Original Code
push ebpmov ebp, espsub exp, 10push
ebp4push expcall strcpyadd
exp,10leaveretpush ebpmov ebp, espsub esp,
4sub exp, 20mov exp5,4push expcall
f1add exp,20add esp,4leaveret
a
Return Address of main
push ebpmov ebp, espsub esp, 10push
ebp8push ebp-10call strcpyadd
esp,10leaveretpush ebpmov ebp, espsub esp,
24mov ebp-15,4push ebp-20call f1add
esp,24leaveret
Old EBP
void f(char a) char c10
strcpy(c,a)void main() char a20 int
i a5 4 f1(a)
Prologue
EBP
a
EXP
c
i
address of a
Return Address of f
ESP
Epilogue
Old EBP
c
Return Address of main
Old EBP
EBP
i
address of a
ESP
Return Address of f
Old EBP
20
Scenario (Cont.)
Modified Pseudo Assembly Code
Original Code
Return Address of main
push ebpmov ebp, espsub esp, 10push
ebp4Spush ebp-10Scall strcpyadd
esp,10leaveretpush ebpmov ebp, espsub esp,
24mov exp-15S,4push expcall f1add
esp,24leaveret
Old EBP
void f(char a) char c10
strcpy(c,a)void main() char a20 int
i a5 4 f1(a)
Prologue
EBP
a
i
address of a
Return Address of f
ESP
Epilogue
Old EBP
c
S Size of Stack
21
Vulnerable code of PCT SSL vulnerability
function(char packet, unsigned int N)
        char buf32         unsigned int
register i         if(N lt 32)         
             memcpy(buf,packet,N)
            for(i 0 i lt N i)
            bufiN bufi