A%20DoS%20Resilient%20Flow-level%20Intrusion%20Detection%20Approach%20for%20High-speed%20Networks

1
A DoS Resilient Flow-level Intrusion Detection
Approach for High-speed Networks

• Yan Gao, Zhichun Li, Yan Chen

Lab for Internet and Security Technology
(LIST) Northwestern University
2
Outline

• Motivation
• Background on sketches
• Design of the HiFIND system
• Evaluation
• Conclusion

3
The Spread of Sapphire/Slammer Worms
4
Existing Network IDSes Insufficient

• Signature based IDS cannot recognize unknown or
  polymorphic intrusions
• Statistical IDSes for rescue, but
• Flow-level detection unscalable
• Vulnerable to DoS attacks
• e.g. TRW IEEE SSP 04, TRW-AC USENIX Security
• Symposium 04, Superspreader NDSS 05 for
  port
• scan detection
• Overall traffic based detection inaccurate, high
  false positives
• e.g. Change Point Monitoring for flooding
  attack
• detection IEEE Trans. on DSC 04
• Key features missing
• Distinguish SYN flooding and various port scans
  for effective mitigation
• Aggregated detection over multiple vantage points

5
Our Solution HiFIND System

• Goal accurate High-speed Flow-level
• INtrusion Detection (HiFIND) system
• Leverage our data streaming techniques
  reversible sketches
• Select an optimal small set of metrics from
  TCP/IP headers for monitoring and detection
• Design efficient two-dimensional sketches to
  distinguish different types of attacks
• Aggregate compact sketches from multiple routers
  for distributed detection

6
Deployment of HiFIND

• Attached to a router/switch as a black box
• Edge network detection particularly powerful

Monitor each port separately
Monitor aggregated traffic from all ports
Original configuration
7
Outline

• Motivation
• Background on sketches
• Design of the HiFIND system
• Evaluation
• Conclusion

8
k-ary sketch
The first to monitor and detect flow-level heavy
changes in massive data streams at network
traffic speeds IMC 03
Update (k, v) Tj hj(k) v (for all j)
Estimate v(S, k) sum of updates for key k
SCombine(a,S1,b,S2)
9
Reversible Sketch

• Report keys with heavy changes
• Significantly improve its usage IMC 2004,
  INFOCOM 2006, ACM/IEEE ToN to appear
• Efficient data recording
• For the worst case traffic, all 40-byte packet
  streams
• Software 526Mbps on a P4 3.2Ghz PC
• Hardware 16 Gbps on a single FPGA broad
• INFERENCE(S,t)

?
?
10
Outline

• Motivation
• Background on sketches
• Design of the HiFIND system
• Architecture
• Sketch-based intrusion detection
• Intrusion classification with 2D sketches
• Feature analysis
• Evaluation
• Conclusion

11
Architecture of the HiFIND system
12
Architecture of the HiFIND system

• Threat model
• TCP SYN flooding (DoS attack)
• Port scan
• Horizontal scan
• Vertical scan
• Block scan
• Forecast methods
• EWMA
• Holt-Winter Forecasting Algorithm

13
Sketch-based Detection Algorithm
Keys SYN flooding Hscan Vscan Score
SIP, Dport non-spoofed Yes No 1.5
DIP, Dport Yes No No 1
SIP, DIP non-spoofed No Yes 1.5
SIP non-spoofed Yes Yes 2.5
DIP Yes No Yes 2
Dport Yes Yes No 2

• RS(DIP, Dport, SYN - SYN/ACK)
• Detect SYN flooding attacks
• RS(SIP, DIP, SYN - SYN/ACK)
• Detect any intruder trying to attack a particular
  IP address
• RS(SIP, Dport, SYN - SYN/ACK)
• Detect any source IP which causes a large number
  of uncompleted connections to a particular
  destination port

14
Intrusion Classification

• Major challenge
• Can not completely differentiate different types
  of attacks
• E.g., if destination port distribution unknown,
  it is
• hard to distinguish non-Spoofing SYN flooding
  attacks from vertical scans by
• RS(SIP, DIP, SYN - SYN/ACK)
• Bi-modal distribution

SYN floodings
SYN floodings
Vertical scans
Vertical scans
15
Two-dimensional (2D) Sketch

• For example differentiate vertical scan from
  SYN flooding attack
• The two-dimensional k-ary sketches
• An example of UPDATE operation
• Accuracy analysis
• Examples 5 hash tables, 3.2MB memory consumption
• Vertical scan detected at least 99.56
• SYN attack classified correctly at least 99.99

16
DoS Resilience Analysis

• HiFIND system is resilient to various DoS
  attacks as follows
• Send source spoofed SYN packets to a fixed
  destination
• Detected as SYN flooding attack
• Send source spoofed packet to random destinations
  
• Evenly distributed in the buckets of each hash
  table, no false positives
• Reverse-engineer the hash functions to create
  collisions
• Difficult to reverse engineering of hash
  functions
• Unknown hash output of each hash function
• Multiple hash tables and different hash functions
• Even know the hash functions of sketches
• Very hard to find collisions through exhaustive
  search
• E.g. given 6 hash functions, the probability of a
  collision of two random keys in 5 hash functions
  is 5.210-18

17
Distributed Intrusion Detection
SYN/ACK2
SYN2
SYN1
SYN/ACK1

• Naive solution
• Transport all the packet traces or connection
  states to the central site
• HiFIND
• Summarize the traffic with compact sketches at
  each edge router, and deliver them to the central
  site

18
Outline

• Motivation
• Background on sketches
• Design of the HiFIND system
• Evaluation
• Conclusion

19
Evaluation Methodology

• Router traffic traces
• Lawrence Berkeley National Laboratory
• One-day trace with 900M netflow records
• Northwestern University
• One day experiment in May 2005 with 239M netflow
  records, 1.8TB traffic and 11 packet samples
• Evaluation metrics
• Detection accuracy
• Online performance
• Speed
• Memory consumption
• Memory access per packet

20
Highly Accurate
21
Detection Validation

• SYN flooding
• Backscatter USENIX Security Symposium 2001
• Hscans and Vscans
• The knowledge of port number
• e.g. 5 major scenarios of the top 10 Hscans
• e.g. 5 major scenarios of the bottom 10 Hscans

Anonymized SIP Dport DIP Cause
204.10.110.38 1433 56275 SQLSnake scan
5.4.247.103 1433 54788 SQLSnake scan
109.132.101.199 22 45014 Scan SSH
95.30.62.202 3306 25964 MySQL Bot scans
15.192.50.153 4899 23687 Rahack worm
Anonymized SIP Dport DIP Cause
98.198.251.168 135 64 Nachi or MSBlast worm
3.66.52.227 445 64 Sasser and Korgo worm
2.0.28.90 139 64 NetBIOS scan
98.198.0.101 135 64 Nachi or MSBlast worm
165.5.42.10 5554 62 Sasser worm
22
Online performance evaluation

• Small memory access per packet
• 16 memory accesses per packet with parallel
  recording
• Small memory consumption
• Recording speed
• Worst case recording 239M items in 20.6 seconds
• i.e., 11M insertions/sec
• Detection speed
• Detection on 1430 minute intervals
• Average detection time 0.34 seconds
• Maximum detection time 12.91 seconds
• Stress experiments in each hour interval
• Detecting top 100 anomalies with average 35.61
  seconds and maximum 46.90 seconds

23
Conclusion

• Proposed the first online DoS resilient
  flow-level IDS for high-speed networks
• Scalable to highspeed networks
• Highly accurate
• DoS attack resilient
• Distinguish SYN flooding and various port scans
• Aggregate detection over multiple vantage points

24
Thank You !

• Questions?
• For more info
• http//list.cs.northwestern.edu

25
K-ary Sketch
Online data recording estimation IMC 2003
Update (k, u) Tj hj(k) u (for all j)
SCOMBINE(a,S1,b,S2)
26
Two-dimensional (2D) Sketch

• Accuracy analysis
• Given a key k of a vertical scan, the majority of
  the H hash matrices will classify k as a vertical
  scan attack with probability at least
  ,
• where . (
  )
• Given a key k of a SYN flooding, the majority of
  the H hash matrices will classify k as a SYN
  flooding attack with probability at least
  ,
• where .

27
Related work

• Threshold Random Walk (TRW) for port scan
  detection J. Jung et al. 2004
• Not DoS resilient
• TRW with approximate caches (TRW-AC)
• N. Weaver et al. 2004
• High false negatives under DoS attack
• Change Point Monitoring (CPM) H. Wang et al.
  2002
• Detecting port scans as SYN floodings
• Backscatter D. Moore et al. 2001
• Only targeting randomly spoofed DoS attacks
• Superspreader S. Venkataraman et al. 2005
• High false positives with P2P traffic
• Partial Completion Filters (PCF) R. Kompella et
  al. 2004
• Not reversible