CANCCOM 2003

1
DNS-based Detection of Scanning Worms in an
Enterprise Environment
David Whyte School of Computer
Science Carleton University
2
Outline

• Internet Environment
• Scanning Worm Propagation Characteristics
• Domain Name System (DNS)-based Detection Approach
• Results
• Limitations
• Future Work
• Conclusions

3
Recent Examples

• Saphire/Slammer worm Jan 25, 2003
• Fastest spreading worm yet
• 90 compromised in first 10 minutes
• Doubled in size every 8.5 seconds (first minute)
• August 2003 the Month of Worms
• SoBig.F 1 mass mailing virus of all time
• Blaster/LovSan
• Welchia/Nachi
• Witty worm March 2004
• Buffer overflow in a suite of security products
• Use of a hit-list?

4
DShield Report(November 26, 2004)
                                                
                                                  
                             Geographic
Distribution of attack sources. Last
daysDShield, The Movie
5
Long-term Internet Impact

• Worm activity lingers
• Study by Arbor Networks A Snapshot of Global
  Internet Worm Activity
• Internet activity recorded between September and
  November 2001
• 5 Worms were active Code Red, Code Red v2, Code
  Red.d, Nimda, Nimda.E
• Nimda 5 billion scans per day recorded

6
Countermeasure Challenges

• Propagation speed renders human-based defensive
  strategies non-effective
• Security patches frequent, large and sometimes
  broken
• Slammer SQL Server 2000 SP2
• SQL2KASP2.exe 39335 KB, sql2kdeskfullsp2.exe 39033
  4 KB, SQL2KDeskSP2.exe, 26903 KB,
  SQL2KSP2.exe 49943 KB
• Active response risky (self imposed DoS)

7
Countermeasure Challenges

• Helpful Gray Hats
• Anti-Code Red default.ida pages launched Code
  Red counterattacks
• So called Blue or White Worms (i.e. Max Vision
  www.whitehats.org)
• The IDS that cried Worm!!
• Signatures lack sophistication
• (Snort) alert UDP any any -gt any 1434 (msg"SQL
  Slammer Worm" rev1 content"726e51686f756e746
  869636b43684765")
• (Dragon) NAMESMBDCOM-OVERFLOW SIGNATURET D A B
  2 0 135 SMBDCOM-OVERFLOW /5c/00/43/00/24/00/5c/00
  /00/00/00/00/00/00/00/00/00/00/00/00/
  00/00 , /01/10/08/00/cc/cc/cc/cc

8
Problem

• Scanning worm propagation can occur extremely
  fast
• Recall Slammer infected 90 of vulnerable
  Internet hosts lt 10 mins.
• Automated countermeasures are required for worm
  containment and suppression
• Current worm propagation detection methods are
  limited by
• Speed of detection
• Inability to detect zero-day worms
• Inability to detect slow scanning worms
• High false positive rate

9
Scanning Worm Characteristics

• Scanning worms can employ a variety of strategies
  to infect systems
• Topological scanning
• Slow scanning
• Fast scanning
• So far, all make use of a pseudo random generated
  32-bit numbers to determine their targets
• The use of numeric IP addresses does not require
  a DNS lookup
• Violation of typical network behavior (i.e. DNS)

10
DNS-based Scanning Worm Detection Approach

• Most legitimate traffic uses the alphanumeric
  equivalent of an IP address and thus requires a
  DNS lookup
• Hosts within a domain use their respective DNS
  servers for IP translations
• As the network traffic leaves the network
  boundary it can easily be determined if a DNS
  request was involved
• If no DNS query is detected for a connection
  attempt it is considered anomalous

11
DNS-based Scanning Worm Detection Approach

• Inline network device
• Divide network into cells
• Gather DNS requests, embedded IPs in HTTP
  requests
• Construct a candidate connection list (CCL)
  respecting Time to Live (TTL)s
• Observe outgoing connections
• Those outgoing connections not matching an entry
  in the CCL generate an alert

12
DNS-based Scanning Worm Detection Approach

• Technique can be used to rapidly determine if a
  host within an enterprise network is trying to
  infect external systems
• Anomaly-based training period required to
  generate whitelists
• Whitelists are valid non-DNS using
  protocols/activities
• Detects local to remote (L2R) and local to local
  (L2L) inter-cell propagation

13
Detecting Scanning Worms
14
Software Developed

• Prototypes are written in using Perl modules and
  libpcap
• High-level design
• Packet Processing Engine
• Extract features from network traffic
• DNS Correlation Engine
• Connection candidate list (DNS, HTTP, Whitelists)
• New connections
• Generate alerts

15
Testing Analysis

• Testing on two individual live networks
• Two cells within the Carleton University Class B
  network
• Lab network
• One quarter Class C network
• Small user population
• Closed network
• Interdepartmental Network (IDN)
• Traffic from multiple Class C networks
• Large user population
• Open network

16
Network Data Profile (Lab)

• 1 week of network traffic 3.3 GB
• 62 IPv4 Internet addresses
• Total number of TCP connections 18,634
• Total number of UDP packets 941,141
• Maximum number of DNS entries recorded 693
• Only a 3-hour training period to generate the
  whitelists

17
Initial Results Lab Monitoring

• TCP false alerts 52
• 16 alerts attributed to whitelist activity
• 36 true false positives
• UDP false alerts 0
• Worm infections detected 0
• False alert analysis
• Majority caused by low DNS reply TTLs coupled
  with improperly terminating TCP connections
• Solution
• Longer training period (more than 3 hours)
• Require observation of two scans 4 false
  positives

18
Initial Results IDN Monitoring

• 74,968 alerts
• False positives 0
• Three worm infections
• Sasser, Blaster, Gaobot
• Remote Access Trojans (RAT) scanning
• Optix Trojan
• Estimated internal infected hosts
• 195

19
Detected Worm Propagation
Sasser 49,425 Blaster 7,014 Gaobot
18,171
20
Anomaly-based Worm Detection Advantages

• Detection of zero-day worms / attack tools
• Detection of low and slow attacks no threshold
• Low maintenance
• Relies on observation of a protocol found in all
  networks

21
Limitations

• Will not detect intra-cell propagation
• Open networks cause large whitelists and the
  potential for false negatives
• Will not detect network share traversal
  propagation or mass mailing worms
• Automated scanning/attack tool false positives

22
Future Work

• ARP-based detection of scanning worms in an
  enterprise network
• Extend the technique to detect
• Mass-mailing worm detection
• DNS-based selection and filtering
• Automated attack tool detection
• Covert communication detection

23
Conclusions

• Has the potential to detect zero-day worms
• Fastest detection claim in the research community
  today detection within 10 scans with a scanning
  rate gt 1 scan per minute
• Our technique detection in a single scan no
  scanning rate constraint
• Technique could be applied to detect network
  scanning tools, mass-mailing worms, and covert
  communications

24

• Questions?
• DNS-based Detection of Scanning Worms in an
  Enterprise Network. Authors D. Whyte, E.
  Kranakis, P.C. van Oorschot.
• Conference Network and Distributed System
  Security (NDSS'05), Feb.2005, San Diego.