Bro: A System for Detecting Network Intruders in Real-Time

1
Bro A System for Detecting Network Intruders in
Real-Time

• Presented by
• Abdulmajid , Ameen, Yaseen ,mohammed
• Lecturer
• DR.Nur Izura

2
Bro Overview

• Bro is an open-source, Unix-based Network
  Intrusion Detection System (NIDS) that passively
  monitors network traffic and looks for suspicious
  activity.
• Bro was developed by Vern Paxson in the Network
  Research Group at Lawrence Berkley National Lab,
  and by the International Computer Science
  Institute.
• Bro Big Brother (watching and taking care of
  you and all your actions).

3
Original Goals

• High-speed, large volume monitoring
• No packet filter drops (Low packet loss rate)
• Would allow intruder to Avoid the NIDS.
• Real-time notification
• The monitor will be attacked
• Mechanism separate from policy

4
Bro Features

• Rich Application-Layer Analysis
• Custom Scripting Language
• Pre-written Policy Scripts
• Powerful Signature Matching Facility
• Network Traffic Analysis
• Detection Followed by Action
• Snort Compatibility Support (snort2bro)

5
Target Environments

• Bro is intended for use by sites requiring
  flexible, highly customizable intrusion
  detection.
• Development of bro is primarily driven by
  research for intrusion detection and traffic
  analysis
• Bro is specifically well-suited for scientific
  environments
• Extremely useful in networks with liberal
  (default allow) policies
• Supports intrusion prevention schemes
• High-performance on commodity hardware
• Runs on Unix-based systems (e.g., Linux,
  FreeBSD, MacOS)
• Open-source (BSD license)

6
Target Environments(cont)

• Bro is designed for use by Unix experts who place
  a premium on the ability to extend an intrusion
  detection system with new functionality as
  needed, which can greatly aid with tracking
  evolving attacker techniques as well as
  inevitable changes to a site's environment and
  security policy requirements.
• Some sites may wish to run a commercial IDS as
  their front-line of defense, and then also run
  Bro as a way to
• Verify the results of the commercial IDS /
  defense-in-depth
• Attain richer forensics capabilities
• Provide policy-checking capabilities not
  facilitated by the commercial IDS
• Experiment with new approaches and incorporate
  leading-edge research

7
Bro Deployment
8
Structure of the system

• Bro is layered into three major components . It
  uses the now-standard libpcap packet-capture
  library developed by LBNL's Network Research
  Group to filter the packet stream in the kernel
  for high performance.
• Bro is conceptually divided into an event
  engine that reduces a stream of (filtered)
  packets to a stream of higher-level network
  events.
• An interpreter for a specialized language that
  is used to express a site's security policy.
• The lower-most layers process the greatest volume
  of data, and hence must limit the work performed
  to a minimum.
• As we go higher up through the layers, the data
  stream diminishes, allowing for more processing
  per data item.
• This basic design reflects the need to conserve
  processing as much as possible, in order to meet
  the goals of monitoring high-speed, large volume
  traffic flows without dropping packets

9
Bro Architecture

• Policy Scripts in Bro Policy language processes
  event stream ...

Policy Interpreter

• ... and takes action!

• Distills filtered stream into high-level events.

Event Engine

• Kernel filters down high-volume stream via packet
  capture library.

libpcap

• Tap network link passively, send up a copy of all
  traffic.

Network
10
Architecture

• Taps network link passively, sends up a copy of
  all network traffic.

Network
11
Architecture
Filtered Packet Stream
Tcpdump Filter

• Kernel filters down high-volume stream via
  standard libpcap packet capture library.

libpcap
Packet Stream
Network
12
Architecture
Event Stream
Event Control

• Event engine distills filtered stream into
  high-level, policy-neutral events reflecting
  underlying network activity
• E.g., connection_attempt, http_reply,
  user_logged_in
• These span a range of semantic levels
• Currently about 300 different types

Event Engine
Filtered Packet Stream
Tcpdump Filter
libpcap
Packet Stream
Network
13
Architecture
Real-time Notification Record To Disk
Policy Script

• Policy script processes event stream,
  incorporates
• Context from past events
• Sites particular policies

Policy Script Interpreter
Event Stream
Event Control
Event Engine
Filtered Packet Stream
Tcpdump Filter
libpcap
Packet Stream
Network
14
Architecture
Real-time Notification Record To Disk
Policy Script

• Policy script processes event stream,
  incorporates
• Context from past events
• Sites particular policies

Policy Script Interpreter
Event Stream
Event Control
Event Engine
Filtered Packet Stream
Tcpdump Filter
and takes action Records to disk Generates
alerts via syslog or paging Sends events to other
Bros Executes programs as a form of response
libpcap
Packet Stream
Network
15
Packet Filtering

• capture via pcap API (libpcap)
• Filters expressed in same format as
  tcpdump(8).Also support off-line traces
• port finger or port ftp or (tcp13 7 ! 0)
• Using libpcap gains significant advantages it
  isolates Bro from details of the network link
  technology (Ethernet, FDDI, SLIP, etc.Network .
• it greatly aids in porting Bro to different Unix
  variants (which also makes it easier to upgrade
  to faster hardware as it becomes available).
  Berkeley Packet Filter (BPF), found on some Unix,
  it allows libpcap to bring Bros network filter
  into the kernel (filters arent in userland!).
• FreeBSD has a Zero-copy Buffer Extension, which
  makes it Bros OS of choice!

16
Event Engine

• Written in C.
• Distills filtered stream into high-level,
  policy-neutral events reflecting underlying
  network activity!
• E.g., connection_attempt, http_reply,
  user_logged_in, etc.
• Protocols supported 30 protocols
• ARP, IP, ICMP, TCP, UDP
• DCE-RPC, DNS, FTP, Finger, Gnutella, HTTP, IRC,
  Ident, NCP, NFS, NTP, NetBIOS, POP3, Portmapper,
  RPC, Rsh, Rlogin, SMB, SMTP, SSH, SSLCiphers,
  SunRPC, Telenet

17
Event Engine, cont.

• Those protocol analyzers generate over 300
  events!
• new_connection(c connection), new_packet(c
  connection, p pkt_hdr), connection_established(c
  connection), tcp_contents(c connection,
  is_orig bool, seq count, contents string),
  icmp_echo_reply(c connection, icmp icmp_conn,
  id count, seq count, payload string), ...
• Originally, event engine analyzers were written
  in C Class libraries.
• Now BinPAC (Binary Protocol Analyzer Compiler) is
  used.
• binpac generates the C code.

18
Event Engine, cont.

• When the packets arrive, the Event Engine checks
  if the headers of these packets are well formed
  or not. Packages with not well formed headers are
  discarded and events of this problem are created.
• The packets with well formed headers gets a
  connection state which can be connection_attempt,c
  onnection_established, connection_rejected or
  connection_finished for TCP and for UDP the state
  is defined as udp_request or udp_reply.
• The packets are then collected to their
  corresponding connection. These collections use
  different protocols e.g. HTTP, FTP, SMTP and is
  therefore handled by the analyzers specialized
  for each protocol.
• Whether or not the connection are handled by any
  analyzer depends on which analyzers that are
  enabled in the system.

19
Event Engine, cont.

• The analyzer for each protocol then examines the
  content of the connection.
• When this procedure is done, the Event Engine has
  created high-level events describing the activity
  and these are now ready for the Policy Script
  Interpreter.
• As analyzers observe activity, generate events
• Event name typed parameter list
• Queued for asynchronous execution
• All events triggered by a given packet executed
  prior to processing next packet (Along with
  timers that expire at that point).

20
Policy Script Interpreter

• The Policy Script Interpreter takes the
  high-level events generated by the Event Engine
  and compares these with the policy scripts in the
  system.
• The events are sorted in a FIFO list which means
  the first that comes along are the first that is
  processes.
• Policy Script Interpreter takes action if it
  detects any suspicious and dangerous actions or
  it discards other events not defined in the
  policy scripts.
• Traffic that seems like attacks but arent
  (false negatives), can be detected at this point,
  but if the policy scripts are good, this will be
  minimal.

21
Policy Language

• Policy analysis scripts which track network
  activity written in Bros language
• Procedural You write event handlers functions
  asynchronously executed that dont return a
  value).
• Strongly typed (to detect inconsistencies at
  compile-time).
• Rich in types
• USUAL bool, int, count, double, string, time,
  interval, port, addr, hostname, pattern.
• Domain-specific types, such as addresses, ports,
  subnets
• Aggregates record, table, set, file.
• Real-time communication with other Bro instances
  and Extensive state management support (Timers
  automatic expiration persistence

22
Policy Language

• Policy analysis scripts written in Bros
  language
• Procedural (you provide event-handlers that are
  executed asynchronously and that dont return a
  value).
• Strongly typed (to detect inconsistencies at
  compile-time).
• Rich in types
• bool, int, count, double, string, time, interval,
  port, addr, hostname, pattern.
• Aggregates record, table, set, file.
• Flex-like regular expressions built into language
  (pattern)!

23
Language, cont.

• Operators (, -, , /, , !, , , ?,
  relationals, infix, --, ).
• Static, global const scoping of variables.
• const allowed_services setaddr, port
• ftp_serv, ftp, smtp, 20/tcp,
• nttp.lbl.gov, nttp
• Statements (if-then-else, when, return, print,
  add, delete, and function).
• No for loops! But recursion through functions is
  allowed.

24
Notices Alarms

• Convention is to express much detection in terms
  of notices
• I.e., Ive noticed the following activity
• Notices do not necessarily convey a problem
• Distributed scripts detect 78 different types of
  notices
• Scripts provide number of ways to decide when to
  promote a notice to an alarm These can include
  syslog, sending email, paging
• When you see in a script something like
  NOTICE(noteOutboundTFTP, connu,
  msgfmt("outbound TFTP s -gt s", src,
  dst))its just a call to a function named
  NOTICE, using the languages mechanism for
  constructing a record
• Note Entire mechanism is add-on to language, not
  built in

25
Sample Bro Policy

• Using the Bro language, sites can write custom
  policy scripts to generate alarms on any policy
  violation.
• For example, if a site only allows external http
  and mail to a small, controlled lists of hosts,
  they could do this
• const web_servers www.lbl.gov,
  www.bro-ids.org,
• const mail_servers smtp.lbl.gov,
  smtp2.lbl.gov,
• redef allow_services_to setaddr, port
• mail_servers, smtp,
• web_servers, http,
• Bro can then generate an Alarm or even terminate
  the connection for policy violations
• if ( service !in allow_services)
  NOTICE(noteSensitiveConnection, connc,)
• if ( inbound service in terminate_successful_i
  nbound_service )
• terminate_connection(c)

26
Sample Bro Alarms
27
FTP buffer overflow attack
28
Bro in Practical Use

• Primary IDS for LBNL/NERSC since 1996
• Primary IDS for SC00-03 conferences
• No specialized hardware needed
• Low cost allows for multiple deployment
• Requirements
• FreeBSD
• Intel platform
• Fiber tap
• Disk space to archive data

29
Sites using Bro (LBNL)

• Use multiple Bros across a 10Gb/s link, where
  each Bro is only executing a minimal set of
  polices.
• Operational 24?7 since 1996. Monitors traffic for
  suspicious behavior or policy violations
  incoming/outgoing/internal
• In conjunction with blocking routers, Bro acts as
  a dynamic and intelligent firewall
• Blocks access from offending IP addresses
• Blocks high risk ports
• Blocks known high-risk activity
• Terminates connections and/or sends alarms
• Locates site policy violations (e.g. Kazaa and
  gnutella)

30
Sites using Bro

• IDS controls a blocking router
• IDS blocks dynamically when an intrusion attempt
  is detected or alerts upon suspicious activity
• Router blocks statically like a firewall
  Intrusion Prevention

31
Use of Bro Within NERSC
ESNet
Multiple Bro Systems

• Real Time Analysis
• Redundant Backup
• Test Box
• Bulk Traffic Recorder

Tapped Traffic
Network Traffic

Filtering Border Router
ACL Insertion
Multiple IDS

• Snort
• Bro Heavyweight Protocol Analysis
• Bro GRID / SSL Analysis

Tapped Traffic
Tapped Traffic

• Internal Traffic Bro Monitor
• Wireless Network Bro Monitor

NERSC
Wireless Network
32
Bro at (NERSC)

• Tied into a paging system for on-call security
  person
• Bro check pointed at set intervals (Reports
  generated whenever checkpointed)
• Clears out orphaned sessions
• Allows for offline data analysis
• Data archiving
• Maintain traffic data for about 3 months
• Maintain network
• Automatic ACL injection has very low false
  positive rate
• At NERSC average about 1 every 6 months

33
NERSC Network Traffic3 Week Period
Type of Traffic Number of Connections Overall Percentage of Traffic
Bulk Data Transfer 666,529 83.73
Grid Services 74,178 7.19
Web Related 288,3754 5.30
Database 620,1730 .27
Mail 200,484 .04
System Services 185,272 .04
Interactive 116 lt.1
34
Hardware Support

• Significant Bro research effort these days
  focuses on hardware to accelerate network
  analysis
• Note, quite different problem than fast signature
  matching!
• Approach 1 Bro Cluster
• Set of PCs use inter-Bro coordination for joint
  analysis
• Scales to x23 performance for 24 nodes (then
  flattens)
• Approach 2 Shunting
• Custom (simple) FPGA provides intelligent
  front-end
• Per-flow decisions on forward/block/divert
  through IDS/IPS
• 1 Gbps prototype working, undergoing upgrading
• Future Multicore Bro
• Will require extensive analysis to
  understand/tune memory locality intra-analysis
  messaging

35
Bro Future Directions

• Grid related technologies
• Ability to detect Grid related protocols
• X.509 Certificate Analyzer
• SSL Analyzer
• Verify certificates are legitimate
• Router Shunting
• Primary bottleneck in moving packets into user
  space
• Leverage router based hardware filtering to
  analyze packets of interest
• Proof of concept demo at SC01-03
• 1)Utilizing Bro and Juniper router
• 2) Hardware based BPF to filter traffic

36
Port Mirroring
External Network
Mirrored Traffic
Juniper
GigE Interface
Bro
Internal Network
37
Filter-based Forwarding
External Network

Filtered Traffic

Bro
GigE Interface
Filter
Juniper
Internal Network
38
Pros

• Wow, where do we begin?
• Separation of event generation and event handling
  (mechanism vs. policy).
• Code is/was written for a hostile environment.
• Bros language (to write policy scripts) is quite
  tractable.
• Resilience in the presence of flooding(Via
  connection compressor)
• An active community (mailing list
  Bro_at_bro-ids.org).
• Reaffirms that open-source solution are possible!
• Support for plugging in external analyses

39
Pros Bro Noteworthy Features

• Broccoli Bro Client Communications Library
• C interface for external programs to transmit
  receive values and events
• E.g., syslog reader sends log messages to Bro
• Dynamic adaptation to load
• Via policy/load-levels.bro
• Changes current packet filter to activate
  more/less analysis
• User-level state management
• Entries in tables, sets automatically expire T
  sec. after Creation / last read / last write
• Advanced detection algorithms (Backdoors
  protocols running off-port (obviated by DPD)
• )

40
Cons

• Encrypted Traffic is Bros bane
• Act like a SSL proxy (keep all necessary
  certificates?)
• IPv6 ... I think it (finally) made it into
  development code and asymmetry .
• Pretty complex, script-based system
• Requires understanding of the network
• No GUI, just ASCII logs
• Only partially documented
• Lacking resources to fully polish the system

41
The End

• Thank You