BotMiner: Clustering Analysis of Network Traffic for Protocol- and Structure-Independent Botnet Detection

1
BotMiner Clustering Analysis of Network Traffic
for Protocol- and Structure-Independent Botnet
Detection

• Presented by D Callahan

2
Outline

• Introduction
• Botnet problem
• Challenges for botnet detection
• Related work
• BotMiner
• Motivation
• Design
• Evaluation
• Conclusion

3
What Is a Bot/Botnet?

• Bot A malware instance that runs autonomously
  and automatically on a compromised computer
  (zombie) without owners consent
• Profit-driven, professionally written, widely
  propagated
• Botnet (Bot Army) network of bots controlled
  by criminals- A coordinated group of malware
  instances that are controlled by a botmaster via
  some CC channel
• Architecture centralized (e.g., IRC,HTTP),
  distributed (e.g., P2P)
• 25 of Internet PCs are part of a botnet! (
  - Vint Cerf)

4
Botnets are used for

• All DDoS attacks
• Spam
• Click fraud
• Information theft
• Phishing attacks
• Distributing other malware, e.g., spyware

5
How big is the Bot Problem?

• Computers were used for fun, now they are
  platforms
• Current top computing platforms
• http//www.top500.org/list/2008/11/100
• Storm worm-1-50 million computers infected
• -Massive computing power
• -Incredible bandwidth distributed world wide
• -Is the storm over?

6
Conflicker according to McAfee

• When executed, the worm copies itself using a
  random name to the Sysdir folder.
• obtains the public ip address of the affected
  computer.
• Attempts to download a malware file from the
  remote website
• Starts a HTTP server on a random port on the
  infected machine to host a copy of the worm.
• Continuously scans the subnet of the infected
  host for vulnerable machines and executes the
  exploit.

7
Challenges for Botnet Detection

• Bots are stealthy on the infected machines
• We focus on a network-based solution
• Bot infection is usually a multi-faceted and
  multi-phased process
• Only looking at one specific aspect likely to
  fail
• Bots are dynamically evolving
• Static and signature-based approaches may not
  be effective
• Botnets can have very flexible design of CC
  channels
• A solution very specific to a botnet instance
  is not desirable

8
Existing Techniques

• Traditional Anti Virus tools
• Bots use packer, rootkit, frequent updating to
  easily defeat Anti Virus tools
• Traditional IDS/IPS
• Look at only specific aspect
• Do not have a big picture
• Honeypot
• Not a good botnet detection tool

9
Related Work

• Binkley,Singh 2006 IRC-based bot detection
  combine IRC statistics and TCP work weight
• Rishi Goebel, Holz 2007 signature-based IRC
  botnickname detection
• Livadas et al. 2006, Karasaridis et al. 2007
  (BBN, ATT) network flow level detection of IRC
  botnets (IRCbotnet)
• BotHunter Gu etal Security07 dialog
  correlation to detect bots based on an infection
  dialog model
• BotSniffer Gu etal NDSS08 spatial-temporal
  correlation to detect centralized botnet CC
• TAMD Yen, Reiter 2008 traffic aggregation to
  detect botnets that use a centralized CC
  structure

10
Motivation

• Botnets can change their CC content (encryption,
  etc.), protocols (IRC, HTTP, etc.), structures
  (P2P, etc.), CC servers, infection models

11
Botnet again

• A coordinated group of malware instances that
  are controlled by a botmaster via some CC
  channel
• We need to monitor two planes
• C-plane (CC communication plane) who is
  talking to whom
• A-plane (malicious activity plane) who is
  doing what

12
Botminer Framework
13
C-Plane clustering
What characterizes a communication flow (C-flow)
between a local host and a remote service?
ltprotocol, srcIP, dstIP, dstPortgt
14

• Temporal related statistical distribution
  information in
• BPS (bytes per second)
• FPH (flow per hour)
• Spatial related statistical distribution
  information in
• BPP (bytes per packet)
• PPF (packet per flow)

15
Two-step Clustering of C-flows

• Why multi-step?
• Coarse-grained clustering
• Using reduced feature space mean and
• variance of the distribution of FPH, PPF, BPP,
  BPS for each C-flow (248)
• Efficient clustering algorithm X-means
• Fine-grained clustering
• Using full feature space (13452)
• Whats left?

16
A-plane Clustering

• Capture activities in what kind of patterns

17
Cross-plane Correlation

• Botnet score s(h) for every host h
• Similarity score between host hi and hj
• Hierarchical clustering
• Two hosts in the same A-clusters and
• in at least one common C-cluster are
• clustered together

18
Results
19
False Positive Clusters
20
Botnet detection
21
Overview
22
Conclusion

• Botminer
• - New botnet detection system based on
  Horizontal correlation
• - Independent of botnet CC protocol and
  structure
• -Real-world evaluation shows promising results
• -while it is possible to avoid detection of
  BotMiner the efficiency and convenience of the
  BotNet will also suffer