The Roles of Intrusion Detection and Data Fusion in Cyber Security Situational Awareness

1
The Roles of Intrusion Detection and Data Fusion
in Cyber Security Situational Awareness

• A Review of the Published Literature and
  Discussion of Future Research Plans
• Nicklaus A. Giacobe

2
Cyber Security Situational Awareness
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion

• Intrusion Detection (ID) Plays and Important Role
  in Developing Situational Awareness
• Cyber Situational Awareness
• Network Security Situational Awareness
• Activities Performed on Behalf of an Organization
  Network Security Office
• Activities Performed by Computer/Network Security
  Analysts
• Difficult, Complex Work Lots of Data from IDS,
  Antivirus Systems, Firewall Logs, Server Security
  Logs, etc.
• Ever-Changing Landscape - New Threats, New
  Technologies, New Software, New Vulnerabilities

3
Cyber Security Situational Awareness
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion

• This Introduction
• Part 1 What is the Current State of ID
  Technology?
• Part 2 What are We Trying to
  Accomplish?
• Part 3 Future Research Recommendations
• Conclusion/Discussion

4
Part 1 The Current State of Technology in ID
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion

• History of ID
• Alert Correlation and Data Fusion
• Data Fusion Techniques
• Visualizations

5
Part 1 The Current State of Technology in ID
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion

• History of ID
• Alert Correlation and Data Fusion
• Data Fusion Techniques
• Visualizations

6
History of Intrusion Detection
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion

• Two Different Locations to Monitor
• Host-Based IDS (Denning)
• Log Files (C2 compliance) on Unix Machines
  (Denning 1987)
• IDES/NIDES Baseline normal user behavior
  (Javitz et al. 1994)
• Network-Based IDS (Mukherjee/Heberlein)
• NSM (LAN Monitor) history of previous
  connections, known bad actors lists, signatures
  of attack types (Mukherjee et al. 1994)
• NIDS (Multiple Network IDS and Host) (Snapp et
  al, 1991) (interesting JDL comparison)

7
History of Intrusion Detection
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion

• Two Different Methods of Analysis
• Pattern-Matching (Misuse) Detection (Spafford)
• Match activity to patterns of known undesired
• behavior (Kumar et al. 1994, 1995)
• Tripwire MD Hashing of files (Kim et al. 1994)
• DDoS prevention /SYN Floods / Active DoS
  prevention (Schuba et al. 1997)
• Anomaly Detection (Stolfo)
• Looking for abnormalities in network traffic (Lee
  et al. 1999)
• Qualitative evaluation of the data stream
  (statistical methods) (Portnoy, et al. 2001)
  alert on infrequent types of data
• Statistical Payload Evaluations for Worm
  Detection (Wang et al. 2004, 2006a, 2006b) and
  mitigation (Locasto et al., 2006)

8
History of Intrusion Detection
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion

• Testing and Evaluation of IDSs
• DARPA IDS Data Sets from 1998-2000
• 1999 Data Set Contained
• 2 Weeks of training data with labeled known
  intrusions
• 7 Weeks of unlabeled data
• Evaluate IDSs under design or in production
• Over-fit problem
• IDSs could be developed that find all of the
  problems in the training data, but could be
  very poor at alerting on novel intrusion methods

9
Part 1 The Current State of Technology in ID
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion

• History of ID
• Alert Correlation and Data Fusion
• Data Fusion Techniques
• Visualizations

10
Alert Correlation and Data Fusion
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion

• Correlate by Source, Destination or Attack Method
• Non-Trivial port-number vs. service name, IP
  address vs. hostname, etc. (Cuppens 2001)
• Need Adaptors Different systems not designed
  for fusion (Debar et al. 2001)
• Promise of better understanding see next slide

11
Understanding Through Correlation
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion
Adapted from (Debar et al. 2001)
12
Alert Correlation and Data Fusion
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion

• JDL Fusion Model (Hall and McMullen 2004)

13
Alert Correlation and Data Fusion
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion

• JDL Fusion Model (Hall and McMullen 2004)

Source Pre-Processing
Level 3 Threat Refinement
Level 2 Situation Refinement
Level 1 Object Refinement
14
Part 1 The Current State of Technology in ID
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion

• History of ID
• Alert Correlation and Data Fusion
• Data Fusion Techniques
• Visualization of Underlying and Fused Data

15
Data Fusion Techniques
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion

• Bayesian Inference
• Complete list of all possible states of the
  system
• Probabilities of current state
• Need for accurate historical data (Holsopple et
  al. 2006)
• D-S Theory
• No need for exact knowledge
• Sort out independent evidence and combine it
  using the Dempster Rule
• Very human-like logical combination
• Can combine evidence of non-similar sources/data
  types

16
Data Fusion Techniques
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion

• Data Mining Algorithms
• Support Vector Machines (SVMs) (Liu et al. 2007
  x3)
• Neural Networks (Wang et al. 2007)
• May be helpful in rapidly combining multiple
  sources of similar data
• Thomas and Balakrishnan (2008)
• Combined alert data from 3 different IDSs (PHAD,
  ALAD, Snort) using MLFF-NN
• Tested vs. DARPA 1999 data set
• Showed improved detection rates of the known data
  over each individual IDS (68 vs. 28, 32, 51)

17
Part 1 The Current State of Technology in ID
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion

• History of ID
• Alert Correlation and Data Fusion
• Data Fusion Techniques
• Visualizations

18
Visualizations
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion

• Based on Network Topology
• Based on Geopolitical Topology
• Network Traffic Representations
• Alert and Track-Based Displays

19
Hierarchical Network Map from Mansmann and Vinnik
(2006)
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion
20
Representation of Threats and Actors on a
Geopolitical Map from (Pike et al. 2008)
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion
21
Representation of host to port to remote port to
remote host of network traffic from (Fink et al.
2004)
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion
22
Panel Displaying Network Connections from a
Single Host from (Fischer et al. 2008)
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion
23
Representing the Three Ws from (Foresti et al.
2007)
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion
24
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion
25
Part 2 What are We Trying to Accomplish?
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion

• Definition of Computer Security
• Theory of Situational Awareness
• Cognitive Load Theory
• Cognitive Task Analysis

26
Part 2 What are We Trying to Accomplish?
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion

• Definition of Computer Security
• Theory of Situational Awareness
• Cognitive Load Theory
• Cognitive Task Analysis

27
Definitions
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion

• (Computer) Security is
• Manunta (1999)
• Security is interaction of Asset (A), Protector
  (P) and Threat (T) in a given Situation (Si)
• CIA Triad (Tipton et al. 2007)
• Confidentiality
• Integrity
• Availability
• Bishop (2003)
• Only authorized actions can be executed by
  authorized users

28
Part 2 What are We Trying to Accomplish?
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion

• Definition of Computer Security
• Theory of Situational Awareness
• Cognitive Load Theory
• Cognitive Task Analysis

29
Theory of Situational Awareness
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion

• Endsley (1995)
• State of Knowledge
• Elements
• Situation
• Future Projection
• Awareness Machine unlikely
• Focus instead on awareness support technologies

30
Theory of Situational Awareness
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion

• Endsley (1995)

31
Higher Levels of Fusion Situational Awareness
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion

• Mapping of IDS Fusion tasks between JDL Model and
  Endsley SA Model. From Yang et al. (2009)

32
Higher Levels of Fusion
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion

• INFERD
• Level 2 Fusion Engine Based on a priori
  knowledge from system experts pattern matching
  attack methods and known vulnerabilities of the
  system
• TANDI
• Level 3 Fusion Projection of future attacks
  based on knowledge of vulnerabilities of the
  system
• (Yang et al. 2009)

33
Part 2 What are We Trying to Accomplish?
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion

• Definition of Computer Security
• Theory of Situational Awareness
• Cognitive Load Theory
• Cognitive Task Analysis

34
Cognitive Load Theory
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion

• Sweller et al. (1998)
• Working Memory (limited capacity)
• Long Term Memory (unlimited capacity, based on
  schemas to represent complex, related
  information)
• Split Attention
• Conflicting, Repetitive
• Modality Effect

35
Part 2 What are We Trying to Accomplish?
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion

• Definition of Computer Security
• Theory of Situational Awareness
• Cognitive Load Theory
• Cognitive Task Analysis

36
Cognitive Task Analysis
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion

• Biros and Eppich (2001) CTA of IDS Analysts in
  the USAF - 5 capabilities required
• ID non-local addresses
• ID source addresses
• Develop mental image of normal behavior
• Create and maintain SA
• Knowledge sharing
• Killcrece et al. (2003) CTA of govt/military
  security specialists 3 general categories
• Reactive Work (majority of the work)
• Proactive Work
• Quality Management (training, etc)

37
Cognitive Task Analysis
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion

• DAmico et al. (2007) CTA of Network Security
  Professionals in the Department of Defense

38
Part 3 Where Do We Go From Here?
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion

• Model Building
• To understand the contributions of the algorithm
  builders
• CTA
• To understand the needs of the analyst
• Visualization Recommendations
• Based on the work above

39
Conclusion
Introduction Current State of ID
Technology Theory and Background Future
Research Conclusions Discussion

• Current State of ID
• History of ID
• Alert Correlation and Data Fusion
• Data fusion techniques
• Visualization of underlying and fused data
• Theoretical Basis for Understanding SA in the
  Cyber Security Domain
• Definition of Computer Security
• Theory of Situational Awareness
• Cognitive Load Theory
• Cognitive Task Analysis
• Recommendations for Future Work
• Model Building - To understand the contributions
  of the algorithm builders
• CTA - To understand the needs of the analyst
• Visualization Recommendations Based on Needs
  and Cognitive Capabilities of Analysts

40
Discussion and Questions

• Just in case you needed a prompt to ask questions
  here it is