Bitmap Algorithms for Counting Active Flows on High Speed Links

1
Bitmap Algorithms for Counting Active Flows on
HighSpeed Links

• Cristian Estan, George Varghese, Mike Fisk
• Computer Science and Engineering Department,
• University of California, San Diego

2
Why count flows?

• Detect port/IP scans
• Identify DoS attacks
• Estimate spreading rate of a worm
• Packet scheduling

Dave Plonkas FlowScan
3
Existing flow counting solutions
Traffic reports
Router
Server
NetFlow data
Analysis
Fast link
Network Operations Center
Network
4
Motivating question

• Can we count flows at line speeds at the router?
• Wrong solution counters
• Naïve solution use hash tables (like NetFlow)
• Our approach use bitmaps

5
Bitmap counting algorithms

• A family of algorithms that can be used as
  building blocks in various systems
• Algorithms can be adapted to application
• Low memory and per packet processing
• Generalize flows to distinct header patterns
• Count flows or source addresses to detect attack
• Count destination addressport pairs to detect
  scan

6
Talk structure

• Per packet processing for bitmap algorithms
• Computing flow count estimates from bitmaps
• Variance analysis of estimates
• Derived algorithms
• Related work
• Measurements
• Conclusions

7
Bitmap counting direct bitmap
Set bits in the bitmap using hash of the flow ID
of incoming packets
HASH(green)10001001
8
Bitmap counting direct bitmap
Different flows have different hash values
HASH(blue)00100100
9
Bitmap counting direct bitmap
Packets from the same flow always hash to the
same bit
HASH(green)10001001
10
Bitmap counting direct bitmap
Collisions OK, estimates compensate for them
HASH(violet)10010101
11
Bitmap counting direct bitmap
HASH(orange)11110011
12
Bitmap counting direct bitmap
HASH(pink)11100000
13
Bitmap counting direct bitmap
As the bitmap fills up, estimates get inaccurate
HASH(yellow)01100011
14
Bitmap counting direct bitmap
Solution use more bits
HASH(green)10001001
15
Bitmap counting direct bitmap
Solution use more bits
Problem memory scales with the number of flows
HASH(blue)00100100
16
Bitmap counting virtual bitmap
Solution a) store only a portion of the bitmap
b) multiply estimate by scaling
factor
17
Bitmap counting virtual bitmap
HASH(pink)11100000
18
Bitmap counting virtual bitmap
Problem estimate inaccurate when few flows active
HASH(yellow)01100011
19
Bitmap counting multiple bmps
Solution use many bitmaps, each accurate
for a different range
20
Bitmap counting multiple bmps
HASH(pink)11100000
21
Bitmap counting multiple bmps
HASH(yellow)01100011
22
Bitmap counting multiple bmps
Use this bitmap to estimate number of flows
23
Bitmap counting multiple bmps
Use this bitmap to estimate number of flows
24
Bitmap counting multires. bmp
Problem must update up to three bitmaps
per packet
Solution combine bitmaps into one
25
Bitmap counting multires. bmp
HASH(pink)11100000
26
Bitmap counting multires. bmp
HASH(yellow)01100011
27
Talk structure

• Per packet processing for bitmap algorithms
• Computing flow count estimates from bitmaps
• Variance analysis of estimates
• Derived algorithms
• Related work
• Measurements
• Conclusions

28
Basic estimates
Direct bitmap
Virtual bitmap
29
Multiresolution bitmap estimate
30
Talk structure

• Per packet processing for bitmap algorithms
• Computing flow count estimates from bitmaps
• Variance analysis of estimates
• Derived algorithms
• Related work
• Measurements
• Conclusions

31
Relative error in estimates
Direct bitmap
Virtual bitmap
Multiresolution bitmap
32
Error of virtual bitmap
Average (relative) error
Flow density (flows/bit)
33
Memory requirements
Direct bitmap lt N / ln (Ne21)
Virtual bitmap 1.5441/ e2
Multiresolution bitmap 0.9186 ln (Ne2) / e2ct.
34
100 million flows, error 1
Hash table 1.21 Gbytes
Direct bitmap 1.29 Mbytes
Virtual bitmap 1.88 Kbytes
Multiresolution bitmap 10.33 Kbytes
35
Talk structure

• Per packet processing for bitmap algorithms
• Computing flow count estimates from bitmaps
• Variance analysis of estimates
• Derived algorithms
• Related work
• Measurements
• Conclusions

36
Triggered bitmap

• Need multiple instances of counting algorithm
  (e.g. port scan detection)
• Many instances count few flows
• Triggered bitmap
• Allocate small direct bitmap to new sources
• If number of bits set exceeds trigger value,
  allocate large multiresolution bitmap

37
Talk structure

• Per packet processing for bitmap algorithms
• Computing flow count estimates from bitmaps
• Variance analysis of estimates
• Derived algorithms
• Related work
• Measurements
• Conclusions

38
Related work

• Flajolet, Martin (1985) probabilistic counting
• Memory use similar to multiresolution bitmap
• Whang et al (1990) introduce direct bitmap
• You, Chang (1996) use virtual bitmap
• Chauduri, Motwani, Narasayya (1998)
• Counting flows without bias impossible from
  sampled data
• Duffield, Lund, Thorup (2002)
• Accurate solutions based on counting TCP SYN
  flags

39
Talk structure

• Per packet processing for bitmap algorithms
• Computing flow count estimates from bitmaps
• Variance analysis of estimates
• Derived algorithms
• Related work
• Measurements
• Conclusions

40
Multires. bmp. vs. prob. counting
Average (relative) error
Number of flows (log scale)
41
Scan detection memory usage
Interval length Snort (naïve) Probabilistic counting Triggered bitmap
12 seconds 1.94 M 2.42 M 0.37 M
600 seconds 49.60 M 22,34 M 5.59 M
42
Talk structure

• Per packet processing for bitmap algorithms
• Computing flow count estimates from bitmaps
• Variance analysis of estimates
• Derived algorithms
• Related work
• Measurements
• Conclusions

43
A family of counting algorithms
Setting Algorithm Applications
General counting Multiresolution bmp. Track infections
Narrow range Virtual bitmap Triggers (e.g. DoS)
Small counts common Triggered bitmap Port scans
Stationarity Adaptive bitmap Measurement
Add and delete Increment-decrement Scheduling
44
Bitmap counting algorithms

• A family of algorithms that can be used as
  building blocks in various systems
• Algorithms can be adapted to application
• Low memory and per packet processing
• With 2Kbytes error around 1

45
The end

• Bitmap algorithms will be available at
• http//ial.ucsd.edu/bitmaps/
• Any questions?
• Acknowledgements Vern Paxson, David Moore,
  Philippe Flajolet, Marianne Durand, Alex Snoeren,
  K Claffy, Stefan Savage, Florin Baboescu, NIST,NSF

46
Adaptive bitmap

• Virtual bitmap measures accurately number of
  flows if range known in advance
• Often number of flows does not change rapidly
• Measurement repeated
• Can use previous measurement to tune virtual
  bitmap
• Combine a large virtual bitmap with a small
  multiresolution bitmap used for tuning

47
Adaptive bitmap accuracy
Average (relative) error
Number of flows (log scale)
48
With 2 kilobytes of memory
Adaptive bitmap (min avg max) Probabilistic counting (min avg max)
Trace1 -4.4 1.1 4.7 -9.5 2.8 13.3
Trace2 -1.9 0.7 2.0 -6.9 2.8 7.6
Trace3 -1.8 0.6 1.8 2.4 10.2 17.7
49
Increment-decrement algorithms

• Active flow defined as flow with packets in queue
• Must support additions and deletions
• Replace bits of bitmap with counters
• Increment when packet arrives
• Decrement when packet leaves
• Estimate number of flows based on zero counters