An Introduction to Intrusion Detection/Prevention, Vulnerability Assessment and related Technologies

1
An Introduction to Intrusion Detection/Prevention
, Vulnerability Assessment and related
Technologies
Network Security
2
Contents

• Lecture aims and learning outcomes
• Assumptions
• Motivation - Why Intrusion Detection and
  Vulnerability Assessment
• Attack Development
• Vulnerability Development
• Hacker Strategy
• Detection - Intrusion Detection Systems
• Host based IDS
• Network Based IDS
• Prevention - Vulnerability Assessment
• Software
• Services (Audits)
• Web-Based Services
• Counter attacks
• Honey Pots
• Appliances
• Summary

3
Lecture aims and learning outcomes

• The lecture aims are
• To describe the problems related to network based
  attacks
• To describe how some these problems may be
  addressed
• At the end of this lecture you will be able to
• Demonstrate an understanding of the main issues
  relating to threats in the context of network
  attacks
• Understand a number of basic design components
  for building a network security architecture
• Demonstrate an understanding of the importance of
  a security policy with reference to the security
  of a computer network
• Describe the features and security mechanisms
  which are generally used to implement security
  policies for dealing with the security of a
  computer network

4
Assumption

• Perimeter security devices (e.g. firewalls) and
  computer security mechanisms (e.g. application
  and OS security) can only offer best effort at
  preventing attacks.
• They may fail to do so
• a firewall may be misconfigured,
• a password may be sniffed off the network,
• a new attack type may emerge. (cf. Zero-day
  attacks)
• They do not detect when an attack is underway or
  has taken place.
• And they do not react to attacks.

5
Traditional Methods

• Example
• Imagine continuous inspection of a Unix system by
  hand (similar examples for NT, W2K)
• The following simplified checklist is taken from
  CERT (http//www.cert.org/tech_tips/intruder_detec
  tion_checklist.html)
• Examine log files for connections from unusual
  locations or other unusual activity. For example,
  look at your 'last' log, process accounting, all
  logs created by syslog, and other security logs.
• Look for setuid and setgid files (especially
  setuid root files) everywhere on your system.
  Intruders often leave setuid copies of /bin/sh or
  /bin/time around to allow them root access at a
  later time.

6
Ad Hoc Intrusion Detection

• Imagine the complexity and degree of expertise
  needed to carry out the tasks in this checklist
  for every host and every sensitive network link
  on a network every single day.
• The ad hoc approach is not recommended!
• Automated systems are needed
• monitor multiple hosts and network links for
  suspicious behaviour
• report this behaviour, possibly react to it.
• Hence Intrusion Detection Systems (IDS).

7
Definitions

• Detection
• Intrusion Detection Systems (IDS)
• Network based
• Host based
• Prevention
• Intrusion Prevention Systems (IPS)
• Vulnerability Assessment (VA)
• Counterattack
• Honeypots/nets

8
Motivation
4 Vulnerability Development
700
600
Linux (aggr.)
500
Solaris
Windows
400
Total
300
200
100
0
1999
2000
2001
2002
2003
Source SecurityFocus
9
Motivation
Auto Coordinated
Cross site scripting
Attack Sophistication
stealth / advanced scanning techniques
High
Staged
packet spoofing
denial of service
distributed attack tools
sniffers
sweepers
www attacks
automated probes/scans
GUI
back doors
network mgmt. diagnostics
disabling audits
hijacking sessions
burglaries
Attack Sophistication
exploiting known vulnerabilities
password cracking
self-replicating code
Intruder Knowledge
password guessing
Low
2000
1980
1985
1990
1995
Source Carnegie Mellon University
10
Motivation
Vulnerability Exploit Lifecycle
Vulnerability Scanners adding detection signature
Widespread Awareness
Selective Awareness
First Discovery
Advisory Release
11
Unauthorized Use of Computer Systems within the
Last 12 Months
12
Origin of the Attack
13
Which Type of Attacks ?
14
Dollar Amount of Losses by Type
15
Reactions to attacks
16
A Typical Hacker Strategy
PING
CORP
Internet
NETWORK
SWEEP
Primary Target Identification - Identify
Hosts ( ) with external visibility
denotes internal hosts with high value data but
no external view
17
A Typical Hacker Strategy
PORT
CORP
NETWORK
SWEEP
WEB
Primary Target Analysis - Identify services
running on visible hosts to prioritize further
probing activities
18
A Typical Hacker Strategy
Primary Target Selection - Determine
vulnerability state of weakest point and
concentrate further activities against this system
19
A Typical Hacker Strategy
Primary Target Exploitation - Gain privileges
control of primary target - attacker now
controls a trusted corporate system !
20
A Typical Hacker Strategy
Secondary Target Identification - Probing for
high value information or systems which are
then compromised and data stolen or trojan horses
planted, etc.
21
Animated Demo
22
Detection
23
Intrusion Detection Systems

• Popular second layer of network security
  enforcement
• Passive supervision of exiting network, analogues
  to intruder alarms
• Creates more work for personal
• There exist 2 different approaches to the
  implementation of Intrusion Detection Systems
  (IDS)
• Knowledge-based IDS
• Network based
• Host based
• Behaviour-based IDS
• Statistical anomaly detection

24
Intrusion Detection Systems

• An Intrusion Detection System (IDS) is a network
  security system designed to identify intrusive or
  malicious behaviour via monitoring of network
  activity. The IDS identifies suspicious patterns
  that may indicate an attempt to attack, break in
  to, or otherwise compromise a system. An IDS can
  be network-based or host-based, passive or
  reactive, and can rely on either misuse detection
  or anomaly detection. IDS vs Firewalls.  
  Firewalls specify policies about what traffic may
  or may not enter a particular computer network.
  An IDS monitors patterns of traffic and signals
  an alert once it deems that an attack has taken
  place.

25
Knowledge-based IDS

• ALL commercial IDS look for attack signatures
• specific patterns of network traffic or activity
  in log files that indicate suspicious behaviour.
• Called a knowledge-based or misuse detection IDS
• Example signatures might include
• a number of recent failed login attempts on a
  sensitive host
• a certain pattern of bits in an IP packet,
  indicating a buffer overflow attack
• certain types of TCP SYN packets, indicating a
  SYN flood DoS attack.

26
Knowledge-based IDS

• Knowledge-based IDS uses information such as
• Security policy
• Known vulnerabilities of particular OS and
  applications
• Known attacks on systems.
• They are only as good as the information in the
  database of attack signatures
• new vulnerabilities not in the database are
  constantly being discovered and exploited
• vendors need to keep up to date with latest
  attacks and issue database updates customers
  need to install these
• large number of vulnerabilities and different
  exploitation methods, so effective database
  difficult to build
• large database makes IDS slow to use.

27
Behaviour-based IDS

• Statistical Anomaly Detection (or behaviour-based
  detection) is a methodology where statistical
  techniques are used to detect penetrations and
  attacks.
• Begin by establishing base-line statistical
  behaviour what is normal for this system?
• Then gather new statistical data and measure the
  deviation from the base-line.
• If a threshold is exceeded, issue an alarm.

28
Behaviour-based IDS

• Example monitor the number of failed login
  attempts at a sensitive host over a period
• if a burst of failures occurs, an attack may be
  under way
• or maybe the admin just forgot his password?
• This raises the issue of false positives (an
  attack is flagged when one was not taking place
  a false alarm) and false negatives (an attack was
  missed because it fell within the bounds of
  normal behaviour).
• This issue does also apply to knowledge-based
  systems.

29
Behaviour-based IDS

• IDS does not need to know about security
  vulnerabilities in a particular system
• the base-line defines normality
• dont need to know the details of the
  construction of a buffer overflow packet.
• Normal behaviour may overlap with forbidden
  behaviour.
• Legitimate users may deviate from the baseline,
  causing false positives (e.g. user goes on
  holiday, or works late in the office, or forgets
  password, or starts to use new application).
• If the base-line is adjusted dynamically and
  automatically, a patient attacker may be able to
  gradually shift the base-line over time so that
  his attack does not generate an alarm.

30
Host-based and Network-based IDS

• When an IDS looks for attack signatures in
  network traffic, it is called a network-based IDS
  (NIDS).
• When an IDS looks for attack signatures in log
  files of hosts, it is called a host-based IDS
  (HIDS).
• Naturally, the most effective Intrusion Detection
  System will make use of both kinds of
  information.

31
IDS Architecture

• Distributed set of sensors either located on
  hosts or on network to gather data.
• Centralised console to manage sensor network,
  analyze data, report and react.
• Ideally
• Protected communications between sensors and
  console
• Protected storage for signature database/logs
• Secure console configuration
• Secured signature updates from vendor
• Otherwise, the IDS itself can be attacked and
  manipulated.

32
Network-based IDS

• Uses network packets as the data source.
• Typically utilizes a network adapter running in
  promiscuous mode to monitor and analyze all
  traffic in real-time as it travels across the
  network.
• The attack recognition module uses three common
  techniques to recognize attack signatures
• Pattern, expression or bytecode matching
• Frequency or threshold crossing (eg detect
  portscanning activity)
• Correlation of lesser events (in reality, not
  much of this in commercial systems).

33
Network-based IDS
Response Capability
UserConfigurable Policy
Attack Recognition
Filter Engine
Packet Grabber
Network Sensor
Adapter
34
Placement of Network-based IDS

• Deployment options
• Outside fire wall
• Just inside fire wall
• Combination of both will detect attacks getting
  through firewall and may help to refine firewall
  ruleset.
• Behind remote access server
• Between Business Units
• Between Corporate Network and Partner Networks

35
Placement of Network-based IDS

• Internet

Sensor
Mail server
Firewall
Perimeter Network
Sensor
Web server
Sensor
Console
Protected Network
36
Animated Demo
37
Host-based IDS

• Typically monitors system, event, and security
  logs on Windows and syslog in Unix environments.
• Checks key system files and executables via
  checksums at regular intervals for unexpected
  changes.
• Some products can use regular-expressions to
  refine attack signatures (e.g. passwd program
  executed AND .rhosts file changed).
• Some products listen to port activity and alert
  when specific ports are accessed limited NIDS
  capability.

38
Host-based IDS
Response Capability
UserConfigurable Policy
Attack Recognition
Filter Engine
Log files and file checksums
Host Sensor
39
Placement of Host-based IDS

• Deployment options
• Key servers that contain mission-critical and
  sensitive information
• Web servers
• FTP and DNS servers
• E-commerce database servers, etc.

40
Placement of Host-based IDS
Internet
Sensor
Mail server
Firewall
Perimeter Network
Web server
Sensor
Human Resources Network
Console
Sensor
41
IDS as a Response Tool

• Given the (near) real-time nature of IDS alerts,
  an IDS can be used as a response tool as well as
  for detection.
• NIDS and HIDS have different response
  capabilities because they detect different
  attacks, or the same attacks but in different
  ways.

42
HIDS and NIDS

• There are attack types that a HIDS can detect but
  a NIDS cannot
• SYN flood, Land, Smurf and Teardrop attacks,
  BackOrifice,
• And vice-versa
• Trojan login script, walk up to unattended
  keyboard attack, encrypted traffic,
• For more reliable detection, combine both types
  of IDS.

43
IDS Response Options
44
IDS Response Options

• Dangers of automated response
• Attacker tricks IDS to respond, but response
  aimed at innocent target (say, by spoofing source
  IP address)
• Users locked out of their accounts because of
  false positives
• Repeated e-mail notification becomes a denial of
  service attack on sysadmins e-mail account
• Repeated restoration of index.html from CD
  reduces website availability.

45
What is Snort?

• Snort is a fast, flexible, small-footprint,
  open-source NIDS developed by the security
  community and a benevolent dictator
• Lead coder Marty Roesch, now founder of
  Sourcefire (http//www.sourcefire.com)
• Initially developed in late 1998 as a sniffer
  with consistent output, unlike protocol-dependent
  output of TCPDump
• Licensed under GPL, but version 2.0 may change to
  a different license

46
Snort Rules

• Snort rules are extremely flexible and are easy
  to modify, unlike many commercial NIDS
• Sample rule to detect SubSeven trojan
• alert tcp EXTERNAL_NET 27374 -gt HOME_NET any
  (msg"BACKDOOR subseven 22" flags A content
  "0d0a5b52504c5d3030320d0a" referencearachnids,
  485 referenceurl,www.hackfix.org/subseven/
  sid103 classtypemisc-activity rev4)
• Elements before parentheses comprise rule
  header
• Elements in parentheses are rule options

47
Third-Party Enhancements

• Analysis Console for Intrusion Databases (ACID)
• http//acidlab.sourceforge.net/
• PHP-based analysis engine to search and process a
  database of security events generated by various
  IDSes, firewalls, and network monitoring tools
• Query-builder and search interface, packet viewer
  (decoder), alert management, chart and statistics
  generation
• Description and screenshots taken from ACID web

48
(No Transcript)
49
(No Transcript)
50
Third-Party Enhancements

• Demarc
• www.demarc.com
• NIDS management console, integrating Snort with
  the convenience and power of a centralized
  interface for all network sensors
• Monitor all servers / hosts to make sure network
  services such as a mail or web servers remain
  accessible at all times
• Monitor system logs for anomalous log entries
  that may indicate intruders or system
  malfunctions
• Description and screenshots taken from demarc web

51
(No Transcript)
52
(No Transcript)
53
IDS The Future

• Integrated approach to IDS
• Network and host-based in one system (some
  products already do this in a limited way)
• The strengths of both NIDS and HIDS (but maybe
  all of the weaknesses!)
• Better visualisation, management and reporting
  tools
• Event correlation
• Correlate a number of sub-events which
  individually do not indicate an attack but which
  when viewed in combination do
• Requires much more sophisticated software and
  data processing.
• Potentially much better attack detection.
• Commercial Statistical Anomaly Detection

54
Prevention

• Vulnerability Assessment
• Intrusion Prevention Systems

55
Vulnerability Assessment
56
Vulnerability Assessment

• An examination of the ability of a system or
  application, including current security
  procedures and controls, to withstand assault.
• A vulnerability assessment may be used to
• identify weaknesses that could be exploited
• predict the effectiveness of additional security
  measures in protecting information resources from
  attack.

57
Vulnerability Assessment

• Vulnerability Assessment Methods
• Software solutions (ISS Scanner, Stat, Nessus
  etc.)
• Audit Services (manual Penetration tests etc)
• Web based commercial (Qualys, Security Point
  etc.)
• Use a database of vulnerability signatures
• Usually perform a port scan to detect which
  services available
• Try to identify if service is vulnerable by
• Looking for banner information
• Sending a harmless request and analysing the
  response
• Actually performing the attack!
• Offer various reporting and management facilities

58
Animated Demo
59
Lesson learnt from VA

• Keep up-to-date with security (and other) patches
• Form Microsoft OS www.windowsupdate.com
• Enterprise version available Windows Update
  Services (WUS)
• Microsoft Baseline Security Advisor
• Includes hfnetcheck.exe (from Shavlik)
• Similar for SUN, HP, IBM, CISCO etc. OSs

60
Intrusion Prevention Systems

• Rate based
• Content based

61
Intrusion Prevention System - IPS

• Relatively new (marketing) term
• Essentially a combination of access control
  (firewall/router) and intrusion detection systems
• Often shared technologies between stateful
  inspection and signature recognition (looking
  deep into the packet)
• Inline network IDS allows for instant access
  control policy modification
• 2004 Gartner study claims by 2005 only integrated
  firewalls with IDS (i.e. IPS) will survive
• Most success to-date with flood (DoS) attacks

62
Definition of an IPS

• Can be defined as an in-line product that focuses
  on identifying and blocking malicious network
  activity in real time.
• Two general categories
• rate-based products
• content-based (also referred to as signature- and
  anomaly-based)
• Often look like firewalls and often have some
  basic firewall functionality.
• But firewalls block all traffic except that which
  they have a reason to pass
• IPSs pass all traffic except that which they have
  a reason to block.

63
Rate-based IPS

• Block traffic based on load
• too many packets,
• too many connects,
• too many errors.
• In the presence of too much of anything, the
  rate-based IPS kicks in and blocks, throttles or
  otherwise mediates the traffic.
• Most useful rate-based IPS include a combination
  of powerful configuration options with range of
  response technologies
• For example, limit queries to your DNS server to
  1,000 per second
• Other simple rules covering bandwidth and
  connection limiting

64
Disadvantages of Rate-based IPS

• Biggest problem with deploying rate-based IPS
  products is deciding what constitutes an
  overload.
• For any rate-based IPS to work properly, need to
  know not only what "normal" traffic levels are
  (on a host-by-host and port-by-port basis) but
  also other network details such as how many
  connections your Web servers can handle.
• Most products do not provide any help but require
  a trained system engineer
• Because rate-based IPSs require frequent tuning
  and adjustment, they will be most useful in very
  high-volume Web, application and mail server
  environments.

65
Content-based products

• Block traffic based on attack signatures and
  protocol anomalies
• Worms, e.g. Blaster and MyDoom, that match a
  signature can be blocked.
• Packets that do not comply to TCP/IP RFCs can be
  dropped.
• Suspicious behaviour such as port scanning
  triggers the IPS to block future traffic from a
  single host
• The best content-based IPSs offer a range of
  techniques for identifying malicious content and
  many options for how to handle the attacks,
• simply dropping bad packets to
• dropping future packets from the same attacker,
  and
• reporting and alerting strategies.
• IDS-like technology for identifying threats and
  blocking them, content-based IPSs can be used
  deep inside the network to complement firewalls
  and provide security policy enforcement.

66
Counter attacks

• The Problem of origin
• Honeypots/nets

67
Problem of origin

• Denial of Service attacks (DoS)
• In contrast to unauthorised access attacks a DoS
  attack does not need to contain method for
  communicating back to the attacker
• Distributed Denial of Service (DDoS) attacks
• Trin00/Stacheldraht (Feb 2000)
• Attacks on ebay, amazon.com and etrade.com
• MS.Blaster (August 2003)
• Problem of lack of metrics to measure the impact
  of Denial of Service attacks more research
  required

68
What is a DDoS Attack ?

• In a Denial of Service (DoS) attack,
• The attacker overwhelms a targeted system with a
  flood of packets to deny availability of services
  to legitimate users
• In a Distributed Denial of Service (DDoS) attack,
• The attacker uses dozens or even hundreds of
  zombie machines to multiply the force of the
  attack

69
Motives Behind DDoS Attacks

• Until recently attacks appear to be motivated by
• Desire for attention
• Notoriety
• Fun
• Long term, DDoS type attacks could become
  motivated by
• Economic warfare between competition
• Disgruntled employees/customers
• Monetary gains (i.e. stock market
  manipulation/online betting)
• Political sabotage and vandalism (party websites
  during election campaigns)

70
DDoS Components

• All DDoS attacks consist of three parts
• Client Program
• Master Server
• Agent (Zombie) Program

71
DDoS Attack Illustrated
72
DDoS Attack Illustrated
Zombies
Hacker secretly installs zombie
agent programs, turning unsecured computers into
zombies
2
73
DDoS Attack Illustrated
Zombies
Hacker selects a Master Server to
send commands to the zombies
3
74
DDoS Attack Illustrated
Zombies
Using Client program, Hacker sends
commands to Master Server to launch zombie attack
against a targeted system
4
75
DDoS Attack Illustrated
Zombies
Master Server sends signal to
zombies to launch attack on targeted system
5
76
DDoS Attack Illustrated
Zombies
Targeted system is overwhelmed by
bogus requests that shut it down for legitimate
users
6
User
77
Minimizing Risk

• Prevent yourself from being victimized
• Ensure your computers are not zombies
• Perform periodic assessments via automated
  scanning services
• Implement an early warning system
• Automated Intrusion Detection Response tools
• Collect forensic data to prosecute hackers later

78
Honeypots

• Technology used to track, learn and gather
  evidence of hacker activities
• Definition
• a resource whose value is being attacked or
  compromised
• Laurence Spitzner, The value of honeypots,
  SecurityFocus, October 2001
• Strategically placed systems designed to mimic
  production systems, but not reveal real data
• Modes of operation
• Baiting
• Waiting
• Collating
• Disseminating

79
Honeypot types of implementation

• Level of Involvement
• Low Involvement Port Listeners
• Mid Involvement Fake Daemons
• High Involvement Real Services
• Risk increases with level of involvement

80
Honeynet

• Network of honeypots
• Supplemented by firewalls and intrusion detection
  systems - Honeywall
• Advantages
• More realistic environment
• Improved possibilities to collect data

81
Honeynet
82
Sebek

• Sebek is a data capture tool designed to capture
  all of the attackers activities on a honeypot,
  without the attacker knowing it.
• 2 components.
• Client that runs on the honeypots, its purpose is
  to capture all of the attackers activities
  (keystrokes, file uploads, passwords) then
  covertly send the data to the server.
• Server which collects the data from the
  honeypots. The server normally runs on the
  Honeywall gateway.
• Since the Sebek client runs as a kernel module on
  the honeypots, it can capture all activity,
  including encrypted, such as SSH, IPSec

83
Honeynet using a Honeywall
84
Summary

• Threats are both internal and external.
• Prevention, detection and reaction are needed in
  combination.
• Intrusion detection systems are a very useful
  second line of defence (in addition to firewalls
  and other safeguards).
• IDS deployment, customisation and management is
  generally not straightforward.
• Vulnerability Assessment and Patch Management are
  King.
• Newer technologies such as IPS and Honeynets can
  remove the burden from over worked system and
  network administrators.

85
IDS Further Reading

• Stallings Chapter 9, pp.292-303 (possibly too
  much emphasis on statistical approach
  research-focussed rather than commercially
  focussed).
• An article The future of IDS by Matthew Tanase
  at SecurityFocus.com
• http//online.securityfocus.com/infocus/1518
• An evaluation of IDS products by Kathleen A.
  Jackson
• http//www.sekure.net/ids/00416750.pdf

86
Questions

• Thank You !
• Merry Christmas
• Happy New Year