Standards Activities on Traffic Measurement

1
Standards Activities on Traffic Measurement
2
Outline

• Applications requiring traffic measurement
• Packet capturing and flow measurement
• Existing protocols
• IETF IPFIX and PSAMP working groups

3
Applications Requiring Traffic Measurement (1)

• Usage-based accounting
• input to charging and billing
• various business model
• time-based, volume-based, QoS class-based
• per application, per user, per user group
• Traffic engineering
• optimizing network usage
• traffic analysis on congested links
• origin of traffic
• type of traffic
• dynamic behavior (bursty, adaptive, )
• Traffic profiling

4
Applications Requiring Traffic Measurement (2)

• QoS monitoring
• (passive) measurement of QoS properties
• validating Service Level Agreements
• Attack detection and analysis
• detecting (high volume) traffic patterns
• investigation of origin of attacks
• Intrusion detection
• detecting unexpected or illegal packets

5
Outline

• Applications requiring traffic measurement
• Packet capturing and flow measurement
• Existing protocols
• IETF IPFIX and PSAMP working groups

6
The General Traffic Flow Measurement Process
Analysis by applications (TE, attack detect.,
QoS monitoring, accounting, )
Sampling
packets
Filtering
Observation Point
Visualize (FlowScan)
Classification Flow Recording
Store (TCPdump)
flow records
Sampling
Display (Ethereal)
Filtering
other
7
The General Traffic FlowMeasurement Process

• Packet capturing at observation point
• Packet sampling and filtering
• both steps may be trivial (11 sampling, no
  filtering)
• both steps may be applied repeatedly
• Packet classification, mapping to flow records,
  maintaining of flow records
• Flow record sampling and filtering
• both steps may be trivial (11 sampling, no
  filtering)
• both steps may be applied repeatedly
• Processing flow records in application

8
Packet Capturing Protocols
Capturing

• Packet Capturing Protocols
• Capture packets at observation point
• Optionally Sample and filter packets
• Export packets or parts of packets (e.g., first
  100 bytes)
• Packet classification, flow recording and
  processing after transfer
• Proprietary sFlow
• Standard (to be) PSAMP

Sampling
router or probe
Filtering
packet transfer
Classification Flow Recording
Application
9
Flow Monitoring Protocols
Capturing

• Flow Monitoring Protocols
• Capture packets at observation point
• Optionally Sample and filter packets
• Classify packets and update flow records
• Export flow records
• Flow record processing after transfer
• Proprietary NetFlow, LFAP, CRANE
• Standard Meter MIB, IPFIX

Sampling
Filtering
Classification Flow Recording
Sampling
router or probe
Filtering
flow record transfer
Application
10
Comparison

• Packet Capturing Protocols
• simple function on router or probe
• low cost on router or probe
• high data volume for packet transfer
  orunreliable recording because of sampling
• packet classification required after data transfer

• Flow Monitoring Protocols
• more complex functions on router or probe
• high resource requirement on router or probe
  fast memory for flow records
• low data volume for flow record transfer
• flow records available after data transfer

11
Outline

• Applications requiring traffic measurement
• Packet capturing and flow measurement
• Existing protocols
• IETF IPFIX and PSAMP working groups

12
Protocols

• Packet Capturing
• sFlow (InMon Corporation, HP spin-off)
• PSAMP (under standardization at IETF)
• Flow Monitoring
• LFAP (Riverstone)
• CRANE (XACCT)
• NetFlow (Cisco)
• IPFIX (under standardization at IETF)
• RTFM Meter MIB (IETF standard)
• RMONMIB (IETF standard)

13
sFlow
Application

• By InMon Corporation
• Includes packet capturing, sampling and packet
  transmission
• Statistical packet sampling
• Timestamping at data collector
• Configuration by sFlow MIB
• RFC 3176, www.sflow.org
• Applicable to high speed links when sampling is
  used
• Adopted by many vendors (HP, Hitachi, Alaxaia -
  by Hitachi and NEC, Extreme and more)

Data collector
Packets
sMon Meter
14
PSAMP
Application

• Under standardization at IETF Packet Sampling WG
• Very similar to sFlow
• Time stamping by meter
• Configuration by PSAMP MIB
• Intention to use IPFIX protocolfor packet
  transfer

Data collector
Packets
PSAMP Meter
15
LFAP
Application

• Light-weight Flow Accounting Protocol
• Proprietary by Riverstone (Cabletron)
• Just data transfer protocol
• Meter at Connection Control Entity (CCE)
  communicates to Flow Accounting Server (FAS)
• Tight and reliable interaction between CCE and
  FAS
• Reliable data transport
• Flexible TLV coding of transferred data
• Larger overhead than NetFlow
• More cost-intensive at meter/CCE and at data
  collector/FAS

FAS
Flow records
CCE
16
CRANE

• Common Reliable Accounting for Network Element
  (CRANE) Protocol
• Proprietary by XACCT
• Just data transfer protocol
• Template-based data model
• Focus on reliability
• Not yet in extensive commercial use

17
IPFIX
Application

• Under standardization at IETF IP Flow Information
  eXport WG
• Very similar to NetFlow version 9
• Will not use UDP, but use TCP or SCTP (Stream
  Control Transmission Protocol)
• Standardization close to completion
• Close collaboration with PSAMP WG

Data collector
Flow records
IPFIX Meter
18
NetFlow
Application

• Proprietary by Cisco, but de-facto standard
• Fast and efficient, implemented for IOS
• Configurable measurement per 5-tuple
• Unreliable data transport (UDP)
• Hardware-supported on some models
• Not well documented
• re-engineered by Juniper
• Versions 1, 3, 5, 7, 8
• fixed data model
• no support of IPv6 flows
• Version 9 (starting point for IPFIX standard)
• data model templates
• can report IPv6 flows
• optional reliable transport
• not related to older versions!
• RFC 3954

Data collector
Flow records
Router
Meter
19
Real-Time Flow Measurement (RTFM)
Application

• Very flexible and powerful meter
• programmable rule sets
• can serve several readers
• programmable overload behavior
• Reader polls meter
• Realization by SNMP Meter MIB
• Free software implementation NeTraMet
• No acceptance at manufacturers
• Complicated to use (too powerful)
• Specified by RFCs 2720 - 2724

Manager
Reader
Config.
Flow records
Meter
20
Remote Network Monitoring MIB (RMON)

• Very flexible and powerful
• Serves more general goals (analysis on layers
  2-4)
• Just a monitoring tool, no measurement
  architecture defined
• Suited for very specific analysis tasks
• High (hardware) performance requirements
• Too complicated and too expensive for massive
  usage in routers
• Specified by RFCs 2021(RMON2), 2613, 2819(RMON),
  2895, 2896, 3144, 3287, 3273, 3395, 3434, 3577,
  3729, 3737, 3919, 4149, 4150

21
Outline

• Applications requiring traffic measurement
• Packet capturing and flow measurement
• Existing protocols
• IETF IPFIX and PSAMP working group

22
IETF IPFIX Working Group

• IP Flow Information eXport (IPFIX)
• BoF sessions 12/00 and 08/01
• active since 10/01
• Successor of RTFM (Real-Time Flow Measurement) WG
• Target (official) standardizing current practice
• Target (unofficial) standardizing (something
  like) Cisco NetFlow
• Chairs
• Nevil Brownlee, CAIDA
• David Plonka, University of Wisconsin

23
IPFIX Scope and General Requirements

• Goal Find or develop a basic common IP Traffic
  Flow measurement technology to be available on
  (almost) all future routers
• Fulfilling requirements of many applications
• Low hardware/software costs
• Simple and scalable
• Metering to be integrated in general purpose IP
  routers and other devices (probes, middleboxes)
• Data processing to be integrated into various
  applications
• Interoperability by openness or standardization

24
IPFIX Architecture
Flow Information Export
Application
Exporting Process
Collecting Process
Flow Record
MeteringProcess
Observation Point
25
IPFIX Devices
Probe
Simple Router
Complex Router
Multiple Exporters
E
E
E
E
E
M
M
M
M
M
M
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
Concen-trator
Proxy
Protocol Converter
E

M
E
E
M
E
C
E
C
O
M
(Meter MIB)
E
O
M Meter E Exporter C Collector
M
M
O
O
26
IPFIX WG Expected Output

• Planned documents
• Requirements RFC (completed, RFC 3917)
• Evaluation RFC (completed, RFC 3955)
• Protocol specification (in progress)
• Data Model (in progress)
• Architecture RFC (in progress)
• Information model RFC (in-progress)
• Applicability RFC (in-progress)
• No new protocol development in working group
• Instead protocol selection and refinement
• Selected protocol NetFlow version 9
• Configuration of measurements will not (yet?) be
  standardized

27
IPFIX WG Current Status

• Good support from IESG (Internet Engineering
  Steering Group)
• High interest from equipment manufacturers
• Cisco designed NetFlow v9 compliant to IPFIX
  requirements and contributes to documents
• Riverstone/Enterasys contributing actively
• Juniper is closely monitoring progress
• Several accounting and billing system providers
  are monitoring and contributing
• HP, XACCT, InMon, ...
• More information at http//ipfix.doit.wisc.edu

28
IETF PSAMP Working Group

• Packet SAMPling (PSAMP)
• BoF session 03/02
• active since 07/02
• Initiated by Nick Duffield, ATT
• Target standardizing new technology for
  sampling, filtering and exporting packets
• can be interpreted as a component of the IPFIX
  measurement process
• but different to IPFIX, there is no current
  practice
• Chairs
• Juergen Quittek, NEC

29
PSAMP Scope and General Requirements

• Goal Develop effective but low-cost packet
  sampling technology
• Allowing measurements at high-speed links
• Fulfilling requirements of applications using per
  packet measurement
• QoS analysis, traffic profiling
• Very low hardware/software costs
• Much simpler than IPFIX
• Will use subset of IPFIX protocol
• Metering to be integrated in general purpose IP
  routers and other devices (probes, middleboxes)
• Configuration of sampling included (different
  than IPFIX)

30
PSAMP Architecture
Packet Information Export
Application
Exporting Process
Collecting Process
Packet Record
Sampling Filtering Process
Observation Point
31
PSAMP WG Expected Output

• Planned documents
• Architecture RFC (in progress)
• Packet Sampling and Filtering Spec. RFC (in
  progress)
• Report Format and Protocol specification (close
  to final document)
• PSAMP MIB RFC (close to final document)
• Applicability RFC (not started)
• Dependencies on IPFIX protocol development

32
PSAMP WG Current Status

• Good support from IESG (Internet Engineering
  Steering Group)
• Growing interest from equipment manufacturers
• Main drivers are ATT, Cisco and NEC
• Avaya is actively contributing
• Alcatel, Avici, InMon, Lucent are monitoring and
  joining discussions
• Cisco shows strong interest in having PSAMP close
  to IPFIX in order to re-use their existing IPFIX
  software