ECE 526

1
ECE 526 Network Processing Systems Design

• Network Security
• 11/10-12/2008

2
What is network security

• Confidentiality only sender, intended receiver
  should understand message contents
• Authentication sender, receiver want to confirm
  identity of each other
• Message integrity sender, receiver want to
  ensure message not altered (in transit, or
  afterwards) without detection
• Access and availability services must be
  accessible and available to valid users only

2
3
Attack Types

• eavesdrop intercept messages
• actively insert messages into connection
• impersonation can fake (spoof) source address in
  packet (or any field in packet)
• hijacking take over ongoing connection by
  removing sender or receiver, inserting himself in
  place
• denial of service prevent service from being
  used by others (e.g., by overloading resources)

4
Security in Exiting Internet

• At least we understand cryptography now

4
5
Cryptography for Confidentiality
Alices encryption key
Bobs decryption key
encryption algorithm
decryption algorithm
ciphertext
plaintext
plaintext

• symmetric key crypto sender, receiver keys
  identical
• Pros and cons
• public-key crypto encryption key public,
  decryption key secret (private)
• Pros and cons

6
Cryptography for Message Integrity

• Alice verifies signature and integrity of
  digitally signed message
• Integrity using the hash function
• Signature is the encryption key, encrypt h(m)
  instead of messages.

7
Cryptography for Authentication

• number (R) used only once in-a-lifetime
• Potential problem in the middle attack

I am Alice
Bob computes
R
and knows only Alice could have the private key,
that encrypted R such that
send me your public key
8
What NP systems can Do

• to improve Access and availability

9
Firewalls
isolates organizations internal net from larger
Internet, allowing some packets to pass, blocking
others.


public Internet
administered network




firewall


10
Firewalls Why

• prevent denial of service attacks
• SYN flooding attacker establishes many bogus TCP
  connections, no resources left for real
  connections
• prevent illegal modification/access of internal
  data.
• e.g., attacker replaces CIAs homepage with
  something else
• allow only authorized access to inside network
  (set of authenticated users/hosts)
• three types of firewalls
• stateless packet filters
• stateful packet filters
• application gateways

11
Credential-based Networks

• to improve Access and availability

12
Setup Credentials
12
13
Credentials Data Structure

• m of bit in the array
• n of hash functions
• r of hops
• Two steps of bloom filter
• Programming
• Query

13
14
False Positive Probability

• false negative is impossible -gtlegal packet will
  be forwarded
• false positive is possible -gt how big the chance
• Refer to http//en.wikipedia.org/wiki/Bloom_filter

14
15
Intrusion detection systems

• multiple IDSs different types of checking at
  different locations

application gateway
firewall

Internet

internal network
Web server
IDS sensors
DNS server
FTP server
demilitarized zone
16
Intrusion detection systems

• packet filtering
• operates on TCP/IP headers only
• no correlation check among sessions
• IDS intrusion detection system
• deep packet inspection look at packet contents
  (e.g., check character strings in packet against
  database of known virus, attack strings)
• examine correlation among multiple packets
• port scanning
• network mapping
• DoS attack

17
NIDS Techniques

• Signature-based
• Anomaly-based
• Stateful detection
• Application-level detection

18
Signature-base NIDS

• Similar to the traditional anti-virus
  applications
• Example
• Martin Overton, Anti-Malware Tools Intrusion
  Detection Systems,
• European Institute for Computer Anti-Virus
  Research (EICAR), 2005
• Signature found at W32.Netsky.p binary sample
• Rules for Snort

19
Signature matching

• Used in intrusion prevention/detection,
  application classification, load balancing
• Input byte string from the payload of packet(s)
• Hence the name deep packet inspection
• Output the positions at which various signatures
  match.
• challenges
• thousand of possible signature
• high performance requirement
• easy to update the new patterns

20
DFA construction

• Example P he, she, his, hers

21
DFA Searching

• Matching String

• Input stream

• Scanning input stream only once
• Complexity linear time
• .

• h

• x

• h

• e

• r

• s

22
Network Attack Patterns
23
DFA mapped to Traditional Memory

• 256 entries for each state
• Snort Dec. 2005 has 2733 patterns
• Needs 27000 states
• Memory size 13 MB

24
SAM-FSM

• Traditional 13 MB Ours 16KB

25
Overall System
26
Anomaly-based NIDS

• Signature-based NIDS cant detect zero-day
  attacks
• Anomaly Operations deviate from normal behavior.
  
• What could cause anomaly?
• Malfunction of network devices
• Network overload
• Malicious attacks, like DoS/DDoS attacks
• Other network intrusions
• Two main kinds of network anomalies.
• 1. Related to network failures and performance
  problems.
• 2. Security-related problems
• (1) Resource depletion
• (2) Bandwidth depletion

26
27
Key Technical Challenges

• Large data size
• Millions of network connections are common for
  commercial network sites
• High dimensionality
• Hundreds of dimensions are possible
• Temporal nature of the data
• Data points close in time - highly correlated
• Skewed class distribution
• Interesting events are very rare ? looking for
  the needle in a haystack
• High Performance Computing (HPC) is critical for
  on-line analysis and scalability to very large
  data sets

28
Anomaly detection meets troubles

• There are many schemes based on checking abrupt
  traffic changes.
• E.g. apply signal processing technique to detect
  out traffics abrupt change
• However, this kind of anomaly does not always
  mean illegitimate.
• Abrupt change of traffic does not mean an attack
  has exactly happened
• We call this case as

Legitimately-abrupt-change (LAC)
28
29
Legitimately abrupt changes

• Example 1
• Famous information gateway websites, e.g. Yahoo.
• When bombastic news is announced, it would
  appear.
• Example 2
• Special information announce center, e.g. the
  website of national meteorological agency
• When a nature disaster is said to be coming, it
  would occur.
• Typhoon, Earthquake, Tsunami
• Important outdoor holidays

29
30
Anomaly Detection

• Already used by industry
• --Protocol Anomaly
• --Statistical/Threshold based
• In Research
• --Data mining

31
Protocol Anomaly Detection

• Based on the well established RFCs
• Focus on the packet header
• Example
• --All SMTP commands have a fixed maximum size. If
  the size exceeds
• the limit, it could be a buffer overflow or
  malicious code inserting
• attack
• --SYN flood attack attacker sends SYN with fake
  source address
• --Teardrop attack fragmented IP packets with
  overlapped offset

32
Threshold based

• Using training data to generate a statistical
• model, then select proper thresholds for
• network environment (traffic volume, TCP
• packet count, IP fragments count, etc.)
• -- usually used as an complementary tool

33
Stateful IDS

• No practical Solutions
• Very simple implementing
• Example
• Snort uses patter matching in continuous Packets.
  
• Traditional signature rules pattern1 pattern1
  pattern2
• The rule now can be defined as
  pattern1.pattern2

34
Application-level IDS

• Focus on specific services or programs
• (Web Server, Database, etc.)
• Example
• --Monitoring all invocation for Microsoft RPCs
• --Analyze HTTP request for malicious query
  strings
• Products
• --mod_security an optional IDS component for
  Apache
• Web Server

35
Current NIDS Challenges

• High false positives
• -- FP of 0.1 means a normal packet will be
  misclassified as an alert for every 1000 normal
  packets, which is about one error alert per
  minute on a 100M network
• Zero day attack (unknown attack)
• --Most current products rely on signature-based
  detection, difficult to detect new attacks.
• Poor at automatically preventing ability
• --Human interaction is required when attack is
  detected

36
IDS Today Products

• Snort
• McAfee Intrushield
• ISS RealSecure
• Cisco IPS
• Symantec IDS

37
Snort

• Open Source, since 1998
• Used by many major network security products
• Signature-based (more than 3000)
• Simple IP header protocol anomaly detection
• Simple stateful pattern matching

38
McAfee

• Profile-based anomaly detection
• --Manually create profile
• --Create profile by self-learning through a
  training period
• Using profile plus threshold for defending
  against DOS and DDOS
• Inspect encrypted traffic by collecting the
  server side private keys

39
ISS RealSecure

• About 2000 signatures
• Application-based approach
• --identifying any possible exploit to the
  published vulnerabilities of MS RPC, IIS, Apache,
  Lotus, etc.
• Additional support for P2P,Instant Messengers
• Virtual Prevention System
• --a virtual environment to examine the execution
  of a file in order to find any possible malicious
  behaviors
• Support for IPv6
• --Detect possible backdoors which enable the
  IPv6 of a system (usually off)

40
Cisco IPS produtcs

• Protocol decoding

• Threshold based property checking

• Signature matching

• Protocol Anomaly Detection

• Checking file behaviors by intercepting all calls
  to the system resources

41
Symantec

• Multi-steps (protocol, vulnerability, signature,
  DOS, traffic, evasion check)
• Unique feature evasion check
• e.g. request /index.html can be replace with
  /69nd65x.html to evade the signature matching

42
Summary of Current Products
Snort McAfee Intrushield ISS RealSecure Cisco IDS Symantec IMUNE
Signature General x x x x x
Signature Application based x
Anomaly Detection Profile-based x
Anomaly Detection Vulnerability-based x x
Anomaly Detection Statistical-based x x x
Anomaly Detection Protocol-based x x x
Anomaly Detection Self-learning x
Anomaly Detection Application specific x x
Stateful Stateful x x
Behavior Behavior x x
Encrypted Traffic Detection Encrypted Traffic Detection x
IPv6 Support IPv6 Support x
43
Academia on Anomaly Detection

• Columbia University
• --Data mining based (since 1997)
• University of California at Santa Barbara
• --Service Specific (HTTP)
• --Stateful IDS
• Florida Institute of Technology
• --Protocol Anomaly (Statistical based)
• University of Minnesota
• --MIND (Minnesota Intrusion Detection System)

44
Columbia Univ. IDS

• 1997, Applied RIPPER rule learning algorithm on
  UNIX system calls monitoring for malicious events
  detection
• 1998, Applied the algorithm on off-line network
  traffic data (clean training data)
• 2000, Applied EM and clustering algorithm for
  dealing with noisy dataset
• 2001, Developed an complete experiment NIDS based
  on those algorithms.
• 2004, New approach towards payload anomaly
  detection

45
Implementing Procedure

• Wenke Lee, Sal Stolfo, and Kui Mok., A Data
  Mining Framework for Building Intrusion Detection
  Models, Proceedings of the 1999 IEEE Symposium
  on Security and Privacy, Oakland, CA, May 1999

• Pre-Processing

• Process raw packet data

• Feature construction

• Create statistic features

• Apply RIPPER algorithm

• Rule learning

46
Pre Processing

• SYN flood attack

47
Feature Construction

• (servicehttp, flagS0, dst_hostvictim),
• (servicehttp, flagS0, dst_hostvictim)
• -gt (servicehttp, flagS0, dst_hostvictim)
• 0.93, 0.03, 2
• 93 of the time, after two http connections with
  S0
• flag are made to host victim, within 2 seconds
  from
• the first of these two, the third similar
  connection is
• made, and this pattern occurs in 3 of the data

48
RIPPLE Rules

• smurf - serviceecr_i, host_count gt 5,
• host_srv_countgt5
• ( if the service is icmp echo request, and
  connections with the same
• destination host are at least 5, and connections
  with the same service
• are at least 5,then it is a smurf/DOS attack)
• satan - host_REJ_gt83, host_diff_srv_ gt
• 87
• ( for connections with the same destination host,
  if the rejection rate is at least
• 83, and the percentage of different services is
  at least 87, then it is a
• santa/PROBING attack)

49
Experiment Results

• Applied on DARPA98 Intrusion Detection
  Evaluation Data Set

50
Payload based Approach

• K. Wang, S. J. Stolfo, Anomalous Payload-based
  Network
• Intrusion Detection, RAID 2004
• Construct the statistical model for all bytes in
  the header
• Use Mahananobis distance to measure the
  difference
• Problems
• Clean training data is required
• False positive (unacceptable)

51
Service Specific IDS by UCSB

• V.Giovanni et al at University of California at
  Santa Barbara
• Since 2002
• Application level
• Focuses on HTTP request
• HTTP request analyzing
• Constructing models for important fields in the
  request instead of all bytes of the payload
  (Columbia payload approach)

52
Sample Request

• Request
• GET /scripts/access.pl?userjohndoecredadmin
• Properties for Detection
• Request Type e.g. GET
• Request Length e.g. Length(GET
  /scripts/access.pl?userjohndoecredadmin)
• Payload Distribution

53
Request Type

• Assumption
• If a rare used request type was found, it is very
  possible it
• will initiate malicious activity
• Anomaly Score
• AStype-log2(ptype)
• Ptype stands for the probability of a certain
  type

54
Request Length

• Assumption
• The request length should not vary much of a
  certain type.
• Otherwise, it is probably caused by some attacks
• (e.g. overflow)
• Anomaly Score
• ASlen1.5(1-?)/(2.5?)
• Ptype stands for the probability of a certain
  type

55
Characters Distribution

• 256 ASCII Characters
• e.g. passwd -gt 112 97 115 115 119 100
• Distributions 0.33, 0.17, 0.17, 0.17, 0.17
• ?2f(Oi, Ei) (i corresponds from segment 0 to 5)
• Aspd ?2(15/L) (L stands for the payload length)

Segment 0 1 2 3 4 5
ASCII Value 0 1-3 4-6 7-11 12-15 16-255
56
Final Anomaly Score

• AS0.3AStype 0.3ASlen0.4ASpd

57
Later Research at UCSB

• Structure Inference with Markov Model

58
Other Properties Used

• Token Finder
• if the query parameter is drawn from known
  candidates
• Attribute Presence or absence
• malicious crafted request usually ignore the
  order of parameters
• Access Frequency
• Invocation order
• Request time interval

59
Experiment Results

• Tested at UCSB campus network and Google
• False positive 0.06
• Major cons
• Limited to HTTP service

60
Packet Header Anomaly Detection

• Packet Header Anomaly Detection (PHAD)
• developed by Florida Institute of Technology
  since 2001
• Basic Assumption
• If an event x happened n times with r different
  results in the
• training period, the probability of a novel data
  is r/n

61
Implementing

• Step 1
• Assign the novel data probability to important
  fields
• of the packet header (protocol type, flags, etc.)
• Step 2
• Adding all the novel data probability together as
  a
• threshold

62
MINDS

• MINDS (Minnesota Intrusion Detection System)
• Statistic outlier-based anomaly detection
• Compared 5 outlier-based scheme
• K-th nearest neighbor
• Nearest neighbor
• Mahalanobis-distance based
• Local Outlier Factor (LOF)
• Unsupervised SVMs

63
Comparison Result

• A. Lazarevic, et al, A Comparative Study of
  Anomaly Detection Schemes in Network Intrusion
  Detection, Proceedings of the 3rd SIAM
  Conference on Data Mining, San Francisco, 2003

64
Some Emerging Approaches

• SVMs
• (unsupervised and supervised)
• PCA
• PCA SVMs
• Neural Network

65
Conclusion

• Network is lacking of security
• Crypto is well understood and used
• NIDS
• Signature based approaches still play the major
  part in practical IDS
• Anomaly detection has only very limited success
• New approaches are proposed everyday, but false
  positive and detection rate are still the major
  problem
• Various mechanisms should work together for
  maximum success

66
Reference

• Jim Kurose Computer Networks
• Tilman Wolf Credential-based Networks

66