Teaching A Course In Intrusion Detection Systems

1
Teaching A Course In Intrusion Detection Systems

• Deborah Frincke
• Director, Center for Secure and Dependable System
• University of Idaho

2
What Kind of a Course Will it Be?

• Who is in the target audience?
• Resources, abilities, what info they need
• What information do you want to get across?
• Particular topics
• Where will you be?
• Facilities facilities facilities!
• Why are you teaching this?
• Researchers, basic understanding, practitioners
• How will you present the material?
• Theoretical, lecture, lab, ?

3
Who and Where Target Audience Has Varied
Equipment

• Distance Education Students
• Access to corporate networks and data, and/or
• Home computer network, and/or
• Home computer
• Remote access to general campus facilities
• No access to testbed
• Local Campus Students
• Home computer network or home computer
• Physical and remote access to general facilities
• Access to testbed
• Graduate Students, One Campus

4
What One Possible Syllabus
This course will address intrusion detection
systems (IDS) from a theoretical and a practical
standpoint. Topics covered will include the
effectiveness and limitations, efficiency,
assessment, and new directions in the
field.   The course format will primarily be
seminar and discussion, with some lectures being
provided by the instructor and others by the
students. Student projects will include but not
be limited to work with formal models and/or
simulation of intrusion detection
systems development and assessment of a working
IDS and/or IDS component publishable quality
paper Two or three formal presentations of
research papers to their peers See Team
Presentation section

• Last Years Class
• http//www.cs.uidaho.edu/frincke/classes/classes-
  02-03/CS504IDS/SyllabusIDS.htm

5
Starting Point for A Syllabus

•  
• Weeks 1-4
• Background
• Data Sources
• Profiling Strategies and Applications
• Anomaly and Misuse Detection
• Host Versus Network
• Trust and Policy
• Architecture Styles Hierarchical, Centralized,
  Decentralized
•  
• Weeks 11-15
• Learning In IDS
• Agent Systems
• Immune System Strategies
• Data Mining
• Integrated Defense
• Intrusion Tolerance
• Specialized and/or Unusual Applications
• TBA

• Weeks 5-10
• Getting Practical Scalability, Manageability,
  and Reliability
• Tracing and Tracking
• Breaking and Entering
• Performance Evaluation Techniques
• Incident Response Active Response

6
Book Suggestions

• One on systems
• Intrusion Detection, Rebecca Bace  
• Inside Network Perimeter Security The Definitive
  Guide to Firewalls, Virtual Private Networks
  (VPNs), Routers, and Intrusion Detection
  Systemsby Stephen Northcutt, Lenny Zeltser,
  Scott Winters, Karen Fredrick, Ronald W. Ritchey
• Practical Intrusion Detection Handbookby Paul E.
  Proctor
• Intrusion Signatures and Analysisby Mark Cooper,
  Stephen Northcutt, Matt Fearnow, Karen Frederick

• One on attacks/testing
• Hacking Exposed Author McClure Scambray
  KurtzBD
• Newer titles sound promising!
• How to Break Software SecurityJames A.
  Whittaker, Florida Institute of
  TechnologyHerbert H. Thompson, Security
  Innovation
• Know Your Enemy Revealing the Security Tools,
  Tactics, and Motives of the Blackhat Communityby
  The Honeynet Project (Editor), Honeynet Project,
  Lance Spitzner (Preface), Bruce Schneier, The
  Honeynet Project

7
We did a lot of reading and teamwork

• Teams took responsibility for middle lectures
• Material based around Instructor-identified paper
• Teams developed a curriculum for their lecture,
  had it approved, presented it, provided
  supplemental material
• Roughly eight teams x 3-4 members for most

• Deliverables for teams
• Two meetings with instructor prior to
  presentation and one followup
• Development of a syllabus, class reading
  assignment, and outline
• Presentation in class of 90 minute
  lecture/discussion
• Development of sample test questions

8
Why are you teaching?

• My goals
• Focus on graduate students who will be doing
  research in the area
• Provide a balance of awareness of issues, history
  of the field, upcoming trends
• Enhance graduate school, researcher, and new
  prof survival skills such as writing, editing,
  public presentation, defending a perspective,
  devising a course

9
A key decisionDefense Assurance Versus
Attack Understanding
10
How Safe and Secure Lab Practices

• Picture this

• Local University Students
• Accidentally Release Worm!
• Thousands of Computers Crash!
• Millions of Dollars Lost!
• Students in a class taught by
• ltyour name heregt---------------------
• ------------------------------------------
• ------------------------------------------
• ------------------------------------------

11
Safe and Secure Lab Practices

• If you plan to run live exercises, I recommend
• Staff, Staff, Staff!
• What if you dont have one?
• Air-gapped testbed laboratory
• Controlled and documented experimentation
• Signoff sheet for obtaining attack code
• Signoff sheet for using attack code
• Pre-planned laboratory experiments
• Class code of ethics
• Class discussions about ethics, consequences,
• Meeting your helpful local campus support staff
  (more later)

12
Some Practical Suggestions

• Ensure that your department system administrator
  knows what you are doing
• Get acquainted with your campus Media Relations
  personnel
• Meet your campus legal advisor
• Make sure your department chair is aware of your
  laboratory

13
Back to Content Suggestions for Addressing
Concepts

• My approach successive refinement of
  understanding, defense-oriented
• Outline the key concepts of the area
• Raise awareness by hands on experiments
• Add depth with student-managed lectures
• Add breadth by expanding
• Add still more depth with publishable research
  papers
• Three accepted so far for summer conference (10)
• Two more submitted for publication and others
  pending
• About half the research projects will directly
  contribute to thesis/dissertation/project

14
Get them thinking about assessment and evaluation
early
Testbed?
Real Data?
                           7KB      
Coverage?
Repeatability?
Basis of Comparison?
15
Put Defense Options in Context
Making systems more secure
Lessons learned
What were doing with the IDS up to the point
where we detect or raise an alert
Everything after the alert
16
Make It Personal By Raising Awareness of Threat
Environment

• A good first exercise
• Download and install one or more personal IDS
  and/or firewall
• There are many free, shareware, trial versions
• Zone Labs ZoneAlarm
• Sygate Personal Firewall
• http//project.honeynet.org/
• but check on human subjects aspect
• see www.tucows.com

17
and Assessing Personal Defenses

• Exercise
• Devise and defend a strategy for assessing system
  defenses, tied to system policy/likely risks
• Use your methodology to assess the effectiveness
  of your own systems defenses (prior approval!)
• Methods
• Port scanners
• Suggestions
• I often pair this with the previous exercise
• Make sure students only scan systems that they
  have a right to scan
• Consider using your testbed for this

18
Determining the Local Threat Environment
CIAO, National Plan, cont.
19
When did we notice misuse?
Misuse occurring
Diagnosis And Response
We might have been able to detect and respond to
misuse while it is occurring
Misuse occurring
Diagnosis And Response
or we might not have started our core detection
activities until after the misuse ceases.
20
Appropriate Responses differ in these cases
Misuse occurring
Diagnosis And Response
Assume the gap between misuse and diagnosis is
relatively lengthy.
Primary response priorities are likely to be
be prevent vulnerability from being exploited
again identify extent of the damage
repair (f)orensic and perhaps (F)orensic
activities
There is generally less pressure to act
immediately than there is in the case of ongoing
misuse, although rapid response is generally
expected for for preventive measures and repairs
affecting current running of the system this
depends somewhat on the kind of system and kind
of damage anticipated.
21
Intrusion Detection Systems

• Why would we want to do intrusion detection? Why
  not just keep intruders out?

22
What others have said

• Stallings' and others.
• Second line of defense. Even the best perimeter
  defense method can fail.
• Weve met the enemy and s/he is us. Many
  intruders are really misusing insiders and they
  are already inside!
• Ejection. Catch misusers before they can do much
  damage.
• Deterrent. Intruders may stay out if they think
  they'll be caught.
• Educational. Learn how intruders do what they do
  and use this to improve both prevention and
  detection techniques.

23
Intrusion Management

• There are many ways of considering the problem of
  intrusion management. EICAR suggested these
• Avoidance
• Assurance
• Detection
• Investigation
• Modern IDS (research and commercial) are
  beginning to include
• Management of general system defense, perhaps
  including multiple enterprises
• Better integration with environment
• Active Response

24
Audit-Based Intrusion Detection

• Premise we can observe differences between
  intrusive/abnormal behavior and normal behavior.
• Requires ...
• that this activity information (audit data) is
  available
• We can characterize behavior

Audit Date
Profiles, Rules, or other
IDS System
decisions
25
Categories of Intrusion Detection Systems

• There are as many ways to divide intrusion
  detection systems as there are systems
  themselves!
• Useful distinctions include
• Where is the IDS based host, network, combined?
• What is the IDS made of software, hardware,
  ?
• What does the architecture look like
• agents, monolithic, components, ?
• Is the system dynamic or static?
• (continuously gather data, or look for snapshots,
  or react to events)
• What sort of data is available?
• Who manages attacks/response third party,
  internal, automated ?

• Is the system misuse or anomaly based?
• does it have an idea regarding what misuse looks
  like, or does it have an idea what anomalies look
  like? Or both?
• Is the system integrated with defenses, primarily
  investigatory, used for retaliation/response?
• Are rules used, or statistics, or a combination?
• Is the data gathered from the host, the network,
  or a combination?
• How well integrated is the IDS with the
  surrounding environment?
• Does the IDS look externally or internally?
  What is the criteria?

26
The Inputs
Domain Knowledge (rules, stats, behavior, )
Misuser Behavior
System Behavior
The Broader Environment
27
Add in Historical Results/Roots

• Jim Anderson papers in 72 and 80

Penetrator not authorized to use resource Penetrator authorized to use resource
Penetrator not authorized use of computer Case A External Penetration Not applicable
Penetrator authorized use of computer Case B Internal Penetration Case C Misfeasance (policy violation)
28
Andersons Early Report

• Anderson identified these goals for security
  audit mechanisms
• Enough information to localize problems, but not
  enough to enable (additional) attacks
• Collect information on a variety of system
  resources
• Discern some notion of normal activity for a
  given resource lt- especially for insider abuse
  detection
• Incorporate strategy of system attacker
• When violations allow attacker to obtain highest
  privilege, no remedy is reliable so suggestion
  was to instrument a system with embedded audit
  mechanisms that monitor CPU and other system
  internals lt- comment is that this isnt durable
• Suggests statistical analysis of user behavior,
  including unusual pattern detection, to identify
  masqueraders

29
What others say
CIAO, National Plan, cont.
30
Masquerading/Spoofing

• User enters under one name, then manages
  somehow to change names, or to enter the next
  system under another name.

Masquerader pretending to be Deb
31
Accuracy?
DNS Server

• Doing telnet

Client
Server
Step 4 Server logs incoming data, does a host
name lookup using DNS (could be done earlier),
stores info, and chooses whether to launch telnet.
32
Accuracy?

• If we believe this information is 100 accurate
  and comprehensive, it would mean that we believed
  the following
• All external telnets use the standard telnet
  port (not true if a local user has set up a
  private' telnet
• Packet source host (telnet client) and port
  information is correct
• Telnet client host can correctly identify the
  user, given the port
• Telnet client host chooses to send us the correc
  tidentity
• Our request and the telnet client host's reply is
  transferred correctly
• Also, the right programs must be present
• Logger program on the telnetd server side
• Identity program on the telnet client side
• And DNS must work.

33
False Negatives and Positives

• Given this premise - anomaly detection works
  because misuse behavior is observably abnormal
• Will we miss fewer behaviors that are
  abnormal/misuse if we become stricter (narrower)
  in our definition of what is normal?
• Will we falsely accuse more users of misuse if
  we become stricter (narrower) in our definition
  of what is normal?

of users
Average login duration
34
False Negatives and Positives

• Look at it this way

Our premise here is that individuals who misuse
the system will have either an unusually short,
or unusually long, login duration (thats why we
selected login time as a feature).
90
80
90
Circles include all users within a certain range
of average login time
35
Good questions for class discussion or assignment

• what kind of misuse behaviors are we most
  likely to uncover using a statistical approach to
  anomaly detection?
• what kind of misuse behaviors are we unlikely
  to uncover using a statistical approach to
  anomaly detection?
• how would you support your answer?

36
Profiling Who, What, How Detailed?

• Statistical/observable behavior of entities over
  time
• Individuals, groups, processes, system components
• Issues
• Modification fixed or variable schedule?
• Danger of training to ignore longterm misuse!
• Good exercises here
• Write programs to identify activity as typical of
  some individual and/or group behaviors
• See how well this works in terms of false
  positives/negatives

37
Centralized vs Distributed Collection

• and that gives us a different view

Centralized data collection it doesnt matter
how many computers are connected to the internal
networks, we always get our data from this point.
38
Examples

• Direct monitoring
• IDS that uses the output of the Unix ps command
  to see how much CPU time a process has consumed.
  Thats direct, because the ps command
  directly goes to the kernel to see what that
  value is
• Indirect monitoring
• Using the output of a CPU logger that works
  with an IDS
• In our example, the CPU logger takes snapshots of
  process usage (perhaps using ps!), and writes
  that to a file.
• This is indirect because the IDS is getting the
  information through an intermediary

IDS
IDS
39
Study Research Systems

• If possible, get some in the lab for practice as
  well
• SNORT is particularly good for rules
• We also use Hummer (potentially tailorable)
• Wide variety
• Emerald/IDES statistical analysis
• AAFID agents and some distribution
• Forrest et al immunological model
• STAT, NetStat state transitions
• AAIRS automated response

40
AAFID Spaf and Zamboni 00
41
Sharing information a multi-site attack
42
AAIRES
43
Sharing Data Hummer
Sharing data from multiple sources with
centralized and decentralized analysis options
44
Decision-Making What happens after Analysis?

• Similar to the analysis engine discussion, we
  primarily see systems that fall into these
  categories
• Centralized and hierarchical decision-making
• One main system makes a decision and the other
  systems implement it, sometimes fleshing out the
  details
• Group decision-making
• Gain consensus as to what to do, then all systems
  implement the plan
• Round robin or synthesized group think
• Distributed
• grass roots sometimes done based on a
  distributed analysis of the problem, sometimes
  done on centralized analysis
• Combined centralized/distributed
• Some decisions made locally, others made
  centrally (Hummer)

45
Distributed Collection plus (partially)
Centralized Analysis DIDS
DIDS Director (Expert Sys UI) Comm interface
Host Monitor (Host Agent Event Generator)
LAN Monitor (LAN rules plus Event Generator)
Host Monitor (Host Agent Event Generator)
Host Monitor (Host Agent Event Generator)
46
NetSTAT Vigna/Kemmerer 99
47
Interest-Driven Agents

• Gopalakrishna/Spafford, 2001
• Distributed plus hierarchy analysis at agent
  level, but there is info at higher levels about
  the events of interest to the agents.

48
IDES Measure Categories and Examples from Bace
Ordinal (continuous) Categorical (Discrete)
Binary CPU time used Num audit records produced Directory used? File accessed?
Linear times command used login failures last hour files modified
49
MIDAS Rule
50
(No Transcript)
51
Possibilities for Team Based Live Exercises
testbed essential!

• The ever popular attack-defense scenario
• Capture the flag/doc (everyone does everything)
• Trusted attackers, student defenders
• Devise/defend variation
• One team develops, the other team tries to
  break/evade. Then reverse.
• Combine with a Forensics class to analyze what
  happened, and a Law/ moot court for
  presentation
• Scientific Experimentation
• Revisit the Darpa Lincoln-Labs scenarios

52
What constitutes an effective defense?

• Manage response in the context of a
  pre-determined policy and current status
• Happens fast enough
• Provide for collaboration with other systems
  without giving up control
• Provide security quality of service

• Scale ( systems, volume of data, criticality of
  asset, )
• Plan for effects of real-world damage during
  attack and considers costs of defense options
• Allows easy modification of policy and
  response, and observation of system state

53
Putting together a defense

• Ideally, we would consider
• Those things we want to protect
• Vulnerabilities and Threats
• Possible partners
• External requirements
• Available defense mechanisms

• Wed then
• Evolve a security policy
• Design/purchase corresponding defenses that meet
  the policy in a cost effective way
• Install and maintain system defenses

54
Visualization Strategies
55
Data Reduction Classification Trees

• Reference Chapple paper, not yet submitted for
  publication
• Determine whether these areas are worth further
  exploration
• Classification at the packet level
• Classification with session aggregation

56
Edges classification decisions made based upon
the dataset features. Nodes of packets in
the training dataset correctly classified without
progressing any farther down the tree.
75 out of 11,245 false negative, 0.04 575 out
of 200K False positive, 0.27 Total mis 0.31