Ensemble-based Adaptive Intrusion Detection

1
Ensemble-based Adaptive Intrusion Detection Wei
Fan IBM T.J.Watson Research Salvatore J.
Stolfo Columbia University
2
Data Mining for Intrusion Detection
Connection Records
(telnet, 10,3,...)
Feature Construction
(ftp,10,20,...)
Training Data
Label Existing Connections
Intrusion Detection Model
Inductive Learner
3
Some interesting requirements ... ...

• New types of intrusions are constantly invented
  by hackers.
• Most recent coordinated attacks on many ebusiness
  websites in 2000.
• Hackers tend to use new types of intrusions that
  intrusion detection system is unaware of or weak
  at detecting them successfully.
• Data mining for intrusion detection is a very
  data-intensive process.
• very large data
• revolving patterns
• real-time detection

4
Question

• When new types of intrusions are invented, can we
  quickly adapt our existing model to be able to
  detect these new intrusions before they cause
  more damages?
• If we don't have a solution, the new attack will
  make significant damage.
• For this kind of problem, having a solution that
  is not completely satisfactory is better than
  having no solution.

5
Naive Approach - Complete Re-training
Existing Training Data
Merged Training Data
New Data
NEW Intrusion Detection Model
Inductive Learner
6
Problem with the Naive Approach

• Since data (existing plus new) will be very
  large, it takes a long time to compute a
  detection model.
• By the time, the model is constructed, the new
  attack probably will have already made enough
  damage to our system.

7
New Approach
New Data
NEW Model
Learner
Combined Model
Existing Model
Key point we only compute model from the data
on new types of intrusions only
8
How do we label connections?
a new connection
existing model
connection type unrecognized
NEW Model
normal or previously known intrusion types
normal or new intrusion types
9
Basic Idea

• Existing model is built to identify THREE classes
• normal
• some type of intrusions
• and anomaly some connection that is neither
  normal nor some known types of intrusions.
• anomaly detection - we use the artificial
  anomaly generation method (Fan et al, ICDM 2001)
  

10
Anomaly Detection

• Generate "artificial anomalies" from training
  data similar to "near misses".
• Artificial anomalies are data points that are
  different from the training data.
• The algorithm concentrates on feature values that
  are infrequent in the training data.
• Distribution-based Artificial Anomaly (Fan et al,
  ICDM2001)

11
Four Configurations

• H1(x) existing model.
• H2(x) new model.
• They differ in how H2(x) is computed.
• and how H1(x) and H2(x) are combined
• and how a connection is processed and classified.

12
Configuration I
13
Configuration II
14
Configuration III
15
Configuration IV
16
Experiment

• 1998 DARPA Intrusion Detection Evaluation Dataset
• 22 different types of intrusions.

17
Experiment

• Sequence to introduce intrusions into the
  training data to simulate new intrusions are
  being invented and launched by hackers
• 22! unique sequences
• we randomly used 3 unique sequences.
• The results are averaged.
• RIPPER
• unordered rulesets

18
3 Unique Sequences
19
Measurements

• All results on the new intrusion types
• Precision
• If I catch a potential thief, what is the
  probability that it is a real thief?
• Recall
• What is the probability that real thieves are
  detected?
• Anomaly Detection Rate classified as anomaly
• Other classified as other types of intrusions.

20
Precision Results
21
Recall Results
22
Anomaly Detection Rate
23
Other Detection Rate Results
24
Summary of results

• The most accurate is Configuration 1 where
• new model is trained from normal and the new
  intrusion type
• all predicted normal and anomalies by the old
  model is examined by the new model.
• Reason
• Existing model's precision to detect normal
  connection influences combined model's accuracy.
• New data is limited in amount. Artificial
  anomalies generated from new data is limited as
  well.

25
Training Efficiency
26
Related Work (incomplete list)

• Anomaly Detection
• SRI's IDES use probability distribution of past
  activities to measure abnormality of host events.
  We measure network events.
• Forrest et al uses absence of subsequence to
  measure abnormality.
• Lane and Brodley employ a similar approach but
  use incremental learning approach to update
  stored sequence from UNIX shell commands.
• Ghosh and Schwarzbard use neural network to learn
  profile of normality and distance function to
  detect abnormality.
• Generating Artificial Data
• Nigam et al assign label to unlabelled data using
  classifier trained from labeled data.
• Chang and Lippman applied voice transformation
  techniques to add artificial training talkers to
  increase variability.
• Multiple classifiers
• Asker and Macline "Ensembles as a sequence of
  classifiers"

27
Summary and Future Work

• Proposed a two-step two classifier approach for
  efficient training and fast model deployment.
• Empirically tested in the intrusion detection
  domain.
• Need to test if it works well for other domains.