Data Mining and Intrusion Detection

1
Data Mining and Intrusion Detection

• Alan Hunt
• Will Fletcher
• Auburn University

2
Outline

• Intrusion Detection Systems
• Data Mining
• Data Mining and Intrusion Detection
• Data Mining Traffic Analysis to Determine and
  Predict User Behavior
• A Priest, a Rabbi, an Intrusion Detection System,
  a Data Miner and a Graduate Student Walk into a
  Bar
• The Bartender Says Im sorry, we dont serve
  miners
• Resources
• Questions?

3
Intrusions

• Intrusions are actions aimed to compromise the
  confidentiality, integrity, and/or availability
  of a computer or computer network.
• Solution Intrusion Detection Systems

4
Intrusion Detection Systems

• Monitors network traffic looking for suspicious
  activity.
• Various approaches
• Network based intrusion detection (NIDS)
  monitors network traffic
• Host based intrusion detection (HIDS) monitors
  a single host
• Signature based (similar to antivirus software),
  also known as misuse detection
• Anomaly detection

5
Intrusion Detection

• Limitations of Signature based IDS
• Signature database has to be manually revised for
  each new type of discovered intrusion
• They cannot detect emerging threats
• Substantial latency in deployment of newly
  created signatures
• Limitations of Anomaly Detection
• False Positives alert when no attack exists.
  Typically, anomaly detection is prone to a high
  number of false alarms due to previously unseen
  legitimate behavior.
• Data Overload
• The amount of data for analysts to examine is
  growing too large. This is the problem that data
  mining looks to solve.
• Lack of Adaptability

6
Data Mining

• Data Mining - Extraction of interesting
  (non-trivial, implicit, previously unknown and
  potentially useful) information or patterns from
  data in large databases Han and Kamber 2005.
• Data mining is used to sort through the
  tremendous amounts of data stored by automated
  data collection tools.
• Extracts rules, regularities, patterns, and
  constraints from databases.

7
Data Mining Techniques

• Association rule mining
• Finding frequent patterns, associations,
  correlations, or causal structures among sets of
  items or objects in transaction databases,
  relational databases, and other information
  repositories.
• Sequence or path analysis
• looking for patterns where one event leads to
  another later event
• Classification
• predicts categorical class labels
• classifies data (constructs a model) based on the
  training set and the values (class labels) in a
  classifying attribute and uses it in classifying
  new data

8
Data Mining Techniques

• Cluster analysis
• Grouping a set of data objects into clusters.
  Objects in same cluster are similar.
• Forecasting
• Discovering patterns in data that can lead to
  reasonable predictions about the future

9
Data Mining and Intrusion Detection

• Data mining can help automate the process of
  investigating intrusion detection alarms.
• Data mining on historical audit data and
  intrusion detection alarms can reduce future
  false alarms.

10
Data Mining and Intrusion Detection

• Julisch and Dacier 2002 apply data mining to
  historical intrusion detection alarms to gain
  new and actionable insights.
• Insights can be used to reduce the number of
  future alarms to be dealt with.
• Use clustering technique on previously mined
  knowledge to efficiently handle intrusion
  detection alarms

11
Data Mining and Intrusion Detection

• Method proposed by Lee, Stolfo, and Mok
• Process raw audit data into ASCII network events
• Summarize into connection records (attributes
  such as service, duration, flags, etc.)
• Apply data mining algorithms to connection
  records to compute frequent sequential patterns
• Classification algorithms then used to
  inductively learn the detection models

12
Data Mining and Behavior

• Detecting Behavior
• Data mining has been used to predict behavior
• Modify these techniques to identify anonymous
  users on a network
• Predict future needs based on past patterns

13
Data Mining and Behavior

• For Example
• User A typically creates a lot of ssh traffic to
  a particular server
• User B checks her email and receives large files
  via FTP after lunch
• User C refreshes the slashdot homepage 10 time
  per minute for 8 hours

14
Data Mining and Behavior

• Research Questions
• Can this behavior be correctly predicted?
• Can users be differentiated based solely on
  network traffic?

15
References

• Intrusion detection Specification-based anomaly
  detection a new approach for detecting network
  intrusions R. Sekar, A. Gupta, J. Frullo, T.
  Shanbhag, A. Twat, H. Yang, S. Zhou November 2002
   Proceedings of the 9th ACM conference on
  Computer and communications security
• Industry track papers Mining intrusion detection
  alarms for actionable knowledge Klaus Julisch,
  Marc Dacier July 2002  Proceedings of the eighth
  ACM SIGKDD international conference on Knowledge
  discovery and data mining
• Detecting intrusions using system calls
  alternative data modelsWarrender, C. Flicker,
  S. Pearlmutter, B. Security and Privacy, 1999.
  Proceedings of the 1999 IEEE Symposium on , 9-12
  May 1999 Pages133 - 145
• Mining in a data-flow environment experience in
  network intrusion detection Wenke Lee, Salvatore
  J. Stolfo, Kui W. Mok August 1999  Proceedings of
  the fifth ACM SIGKDD international conference on
  Knowledge discovery and data mining
• ADMIT anomaly-based data mining for intrusions.
  Karlton Sequeira and Mohammed Zaki Proceedings
  of the eighth ACM SIGKDD international conference
  on Knowledge discovery and data mining. Pages
  386 395. 2002
• www.cs.sfu.ca/han/bk/1intro.ppt
• http//netsecurity.about.com/cs/hackertools/a/aa03
  0504.htm
• http//www.sans.org/resources/idfaq/host_based.php
  
• http//www.symantec.com/symadvantage/016/pad.html