6' Traffic Monitoring R

1
6. Traffic Monitoring RD, Standards Activities
2
6. Traffic Monitoring RD, Standards Activities

• RD Groups
• NLANR, CAIDA, SLAC NMTF
• Standard Activities
• IETF RTFM (Real Time Flow Measurement)
• IETF RMONMIB (Remote Network Monitoring)
• IETF IPPM (IP Performance Metrics)
• IETF IPFIX (IP Flow Information Export)
• IETF PSAMP (Packet Sampling)
• Conferences Workshops
• Passive Active Measurement Workshop (PAM)
• PAM2000, PAM2001, PAM 2002
• PAM2003 (Apr. 6-8, 2003 in UC San Diego) is
  hosted by NLANR
• Internet Measurement Workshop (IMW)
• Sponsored by ACM SICCOMM
• IMW2001, IMW2002, IMC2003
• IEEE NOMS IM

3
NLANR

• The National Laboratory for Applied Network
  Research
• its primary goal is to provide technical,
  engineering, and traffic analysis support
• NSF High Performance Connection sites
• HPNSP (high-performance network service
  providers)
• such as vBNS and Abilene
• Measurement and Operations Analysis Team (MOAT)
• Network Performance Characterization
• lead by San Diego Supercomputing Center at UCSD
• National Center for Network Engineering Team
  (NCNE)
• End-to-end engineering support
• lead by PSC at CMU
• Distributed Applications Support Team (DAST)
• user applications support/optimization
• lead by NCSA at UIUC

4
NLANR MOAT

• The goal is to characterize the behavior of high
  performance connection networks with 2 major
  projects
• Passive Measurement and Analysis (PMA)
• OCXmon (OC3, OC12, OC48, OC192)
• Active Measurement Project (AMP)
• There are currently around 120 AMP monitors
  collecting data at NSF supported HPC sites.
• Four types of measurement are available
• Round trip times (RTT)
• Loss
• Topology
• on demand throughput tests

5
NLANR - NCNE

• National Internet Measurement Infrastructure
• A scalable and dynamic infrastructure for
  measuring performance on the global Internet
• Network Based TCP Tuning
• Tools and services to facilitate high-performance
  use of the Next-Generation Internet backbones
• PITAC Performance Measurement Project
• Work for the President's Information Technology
  Advisory Committee on quantifying the network
  performance of high-performance networks
• Traffic Analysis and Automatic Diagnosis (TAAD)
• TAAD tool is designed to facilitate the automatic
  detection and partial diagnosis of networking
  problems by scanning aggregate traffic for flows
  that bear the signatures of various problems
• Technology Integration
• Migrating new technologies - such as QoS, IPv6,
  instrumentation and measurement - from the
  network research lab to production networks

6
NLANR - DAST

• DAST offers support for researchers working with
  high-performance network applications and assists
  in the development of distributed applications
  and tools
• Advanced Applications Database (AAD) - provides
  information on applications and resources to
  research groups around the world
• Autobuf - Autotuning FTP client and server.
• Iperf - a TCP and UDP bandwidth testing tool,
  similar in function to the traditional ttcp tool
  but nicer.
• Multicast Beacon - is a multicast diagnostic
  tool, showing packet loss, delay, jitter,
  out-of-order packets, and duplicate packets for a
  given multicast group.
• Netlog - is a C library that can be linked into
  an existing network application to provide
  instrumentation of network performance.
• Viznet - Java application to visualize network
  bandwidth performance over time.

7
CAIDA www.caida.org

• The Cooperative Association for Internet Data
  Analysis
• provides tools and analyses promoting the
  engineering and maintenance of a robust, scalable
  global Internet infrastructure
• Goals
• Encourage the creation of Internet traffic
  metrics (in collaboration with IETF/IPPM and
  other organizations) and work with industry,
  consumer, regulatory, and other representatives
  to assure their utility and universal acceptance.
  
• Create a collaborative research and analytic
  environment in which various forms of traffic
  data can be acquired, analyzed, and shared.
• Foster the development of advanced methodologies
  and techniques for traffic performance and flow
  characterization, simulation, analysis, and
  visualization.

8
CAIDA Tools

• Traffic Measurement Tools
• cflowd - a flow analysis tool used for analyzing
  Cisco's NetFlow and includes the collections,
  storage, and basic analysis modules for cflowd
  and for arts libraries.
• CoralReef - a comprehensive software suite to
  analyze data collected by passive Internet
  traffic monitors, in real time or from trace
  files.
• Mantra - is a tool for monitoring various aspects
  of multicast on a global scale at the router
  level.
• NeTraMet - an open-source (GPL) implementation of
  the IETF RTFM (Realtime Traffic Flow Measurement)
  architecture
• Skitter - a tool for actively probing the
  Internet in order to analyze topology and
  performance.

9
CAIDA Tools

• Visualization Tools
• GeoPlot - a lightweight java applet that allows
  users to create a geographical image of a data
  set.
• GTrace - a graphical Java front-end to
  traceroute.
• Mapnet - a tool for visualizing the backbone
  infrastructure of major Internet Service
  Providers.
• Otter - a tool for visualizing nodes and arcs in
  connectivity data typical to Internet
  applications.
• Plankton - a tool for visualizing a global web
  caching hierarchy
• Plotpaths - displays forward and reverse network
  path data from a single source to one or more
  destinations.
• Walrus - a tool for interactively visualizing
  large directed graphs in three-dimensional space.

10
CAIDA Tools

• Utilities
• arts - a C class library and applications for
  storing, manipulating and analyzing Internet data
• dnsstat - watches for DNS queries on UDP port 53
  and counts numbers of messages and numbers of
  queries, aggregated by any of source IP,
  destination IP, opcode, query type, query class.
• dnstop - a libpcap application (ala tcpdump) that
  displays various tables of DNS traffic on your
  network
• FlowScan - analyzes and reports on NetFlow format
  data (indigenous to Cisco routers) collected
  using cflowd flow tool.
• NetGeo - a database and collection of Perl
  scripts used to map IP addresses, domain names
  and AS numbers to geographical locations.
• RRDtool - (Round Robin Database tool) is a system
  to store and display time-series data

11
SLAC NMTF

• SLAC (Stanford Linear Accelerator Center)
• NMTF at SLAC
• ESnet Network Monitoring Task Force
• http//www.slac.stanford.edu/xorg/nmtf/nmtf-tools.
  html
• NMTF Goals
• To act as a focus group/forum for ESnet (Energy
  Sciences Network, www.es.net) sites in the area
  of network monitoring
• Get a sense of what ESnet sites are doing in the
  area of network monitoring
• Share network monitoring information among
  participants.
• Determine what tools/applications are needed to
  perform network monitoring

12
IETF RTFM WG

• Real-time Traffic Flow Measurement (RTFM)
• Developed a generalized, distributed,
  asynchronous, reliable flow measurement
  architecture
• http//www.auckland.ac.nz/net/Internet/rtfm
• RFCs
• 2720 Applicability Statement
• 2721 Architecture
• 2722 Meter MIB
• 2723 SRL A Simple Ruleset Language
• 2724 New Attributes
• NeTraMet - an open-source implementation of RTFM
• Runs on Unix, DOS, Windows
• Ethernet, FDDI, OC3, NetFlow interface
• IPv4, IPv6, IPX, AppleTalk, DECnet, CLNS
  protocols
• http//www.auckland.ac.nz/net/NeTraMet

13
RTFM Architecture
14
RTFM Architecture

• Meters, Meter Readers and Managers
• Flows, each defined by a set of attribute values
• User specifies which flows to measure, and the
  level of detail, via a rule set
• Flows are bi-directional, user specifies the
  direction
• Meter does front-end data reduction to produce
  table of flows, Meter Reader reads flows and
  writes flow data files
• Meters configured in SRL, a high-level ruleset
  language

15
IETF RMONMIB WG

• Remote Network Monitoring MIB Working Group
• http//www.ietf.org/html.charters/rmonmib-charter.
  html
• Limited capability for network monitoring using
  SNMP
• SNMP only provides rudimentary local performance
  by devices
• Large network overhead Polling, Bulk transfer
• RMON WG Goals
• Define a set of managed objects for remote
  monitoring of networks
• To provide the ability to monitor network traffic
  in remote networks
• Consistent with the existing SNMP framework and
  standard
• RFCs
• RFC2819 RMON MIB (Internet Standard)
• RFC2021 RMON MIB II (Proposed Standard)

16
IETF RMONMIB WG

• Design Goals for RMON
• Off-line operation
• Continuously collects information without manager
  polling
• Proactive monitoring
• Notify management station and provide information
• Problem detection and reporting
• Value-added data
• Analysis data
• Support Multiple managers
• RMON MIB
• monitors MAC-level subnet traffic
• RMON MIB II
• can monitor traffic of packets at Network and
  Application layers

17
RMON MIB 1 2
rmon (mib-2 16)
statistics (1)
protocolDir (11)
history (2)
protocolDist (12)
alarm (3)
addressMap (13)
host (4)
nlHost (14)
hostTopN (5)
nlMatrix (15)
matrix (6)
alHost (16)
filter (7)
capture (8)
alMatrix (17)
event (9)
usrHistory (18)
tokenRing (10)
probeConfig (19)
RMON 2
RMON 1
18
IETF IPPM WG

• IP Performance Metrics (IPPM)
• http//www.ietf.org/html.charters/ippm-charter.htm
  l
• IPPM Goals
• Develop a set of standard metrics that can be
  applied to the quality, performance, and
  reliability of Internet data delivery service
• metrics do not represent a value judgment (i.e.,
  good or bad), but rather provide unbiased
  quantitative measures of performance
• Offer a forum for sharing information about the
  implementation and application of these metrics
• Metrics
• Connectivity, one-way delay and loss, round-trip
  delay and loss, delay variation, loss patterns,
  packet reordering, bulk transport capacity, link
  bandwidth capacity

19
IETF IPPM

• RFCs
• RFC2330 Framework for IP Performance Metrics
• RFC2678 IPPM Metrics for Measuring Connectivity
• RFC2679 A One-Way Delay Metric for IPPM
• RFC2680 A One-way Packet Loss Metric for IPPM
• RFC2681 A Round-trip Delay Metric for IPPM
• RFC3357 One-way Loss Pattern Sample Metrics
• RFC3393 IP Packet Delay Variation Metric for
  IPPM
• RFC3432 Network Performance Measurement with
  Periodic Streams

20
IETF IPFIX WG

• IP Flow Information eXport Working Group
• http//www.ietf.org/html.charters/ipfix-charter.ht
  ml
• http//ipfix.doit.wisc.edu/
• IPFIX WG Goals
• Define a "standard IP flow." 
• Devise data encodings that support analysis of
  IPv4 and IPv6 unicast and multicast flows
• Consider the IP flow information export based
  upon packet sampling
• Identify and address any security privacy
  concerns affecting flow data
• Specify the transport mapping for carrying IP
  flow information
• Ensure that the flow export system is reliable

21
IETF IPFIX WG

• Internet Drafts
• Requirements for IP Flow Information Export (Oct.
  2003)
• Evaluation of Candidate Protocols for IP Flow
  Information Export (IPFIX) (June 2003)
• Information Model for IP Flow Information Export
  (Aug. 2003)
• Architecture Model for IP Flow Information Export
  (Oct. 2003)
• IPFIX Protocol Specifications (Oct. 2003)
• IPFIX Applicability (Oct. 2003)

22
IETF PSAMP WG

• Packet Sampling Working Group
• http//www.ietf.org/html.charters/psamp-charter.ht
  ml
• http//psamp.ccrle.nec.de/
• PSAMP WG Goals
• To define a standard set of capabilities for
  network elements to sample subsets of packets by
  statistical and other methods.
• The capabilities should be simple enough that
  they can be implemented ubiquitously at maximal
  line rate.
• They should be rich enough to support a range of
  existing and emerging measurement-based
  applications, and other IETF working groups where
  appropriate.

23
IETF PSAMP WG

• Internet Drafts
• A Framework for Packet Selection and Reporting
  (Oct. 2003)
• Sampling and Filtering Techniques for IP Packet
  Selection (Oct. 2003)
• Definitions of Managed Objects for Packet
  Sampling (Oct. 2003)
• Packet Sampling (PSAMP) Protocol Specifications
  (Oct. 2003)
• Information Model for Packet Sampling Exports
  (Oct. 2003)