IPFIX Aggregation

1
IPFIX Aggregation

• draft-dressler-ipfix-aggregation-01.txt

2
Motivation

• Reduction of monitoring data
• Bandwidth savings and performance savings at the
  collector
• Speed-up of flow accounting
• Reduction of concurrent active streams in a
  monitor
• Concentrating multiple IPFIX streams
• Definition of concentrator functionality
• Transport of information about the aggregation
  rules
• For improved processing of IPFIX data

3
Architecture
exported monitoring data (IPFIX Protocol)
exported monitoring data (IPFIX Protocol)
EP
EP
EP
AP
AP
CP
CP
MP
MP
MP
exported monitoring data (IPFIX Protocol)
EP Exporting Process AP Aggregation
Process MP Metering Process
4
Aggregation Rules

• Specify
• which flow records to aggregate into a meta-flow
  record
• how the meta-flow record and the corresponding
  data template looks like
• Comprise aggregation instructions containing
• IPFIX field ID
• mandatory field for incoming records
• included in meta-flow record or data template
  depending on field modifier
• pattern (optional)
• restricts aggregated flow records to those that
  match this pattern
• field modifier (discard, keep, mask/n, or
  aggregate)
• specifies how this field is treated
• implicitly defines if the field appears in
  meta-flow or data template

5
Field Modifiers
Rule instruction Rule instruction Result Result
Field modifier Pattern exist Field in meta-flow record contains Fixed-value field in Data Template contains
discard no n/a n/a
discard yes n/a pattern
keep no original value n/a
keep yes original value, if pattern is range of values pattern
mask/n no IP network address n/a
mask/n yes IP network address pattern
6
Field Modifier contd

• Special field modifier aggregate for counters,
  timestamps etc.
• Result depends on field
• minimum in case of
• minimumPacketLength, minimumTtl,
  flowStartSeconds, flowStartMilliSeconds
• maximum in case of
• maximumPacketLenth, maximumTtl, flowEndSeconds,
  flowEndMilliSeconds
• binary OR (as suggested by IPFIX-INFO) in case of
  
• ipv6OptionHeaders, tcpControlBits
• sum in case of
• octetDeltaCount, packetDeltaCount

7
Example

• Goal
• monitor flows to web servers (http/https) in
  10.10.0.0/16
• aggregate sources addresses into /24 network
  addresses
• Aggregation Rule

discard protocolIdentifier discard
sourceTransportPort mask/24 sourceIpv4Address disc
ard destinationTransportPort in 80,443 keep
destinationIpv4Address in 10.10.0.0/16 aggregate
packetDeltaCount aggregate octetDeltaCount aggrega
te flowStartMilliSeconds aggregate
flowEndMilliSeconds
8
Example contd

• Data Template

-----------------------
--------- Template ID
Field Count 6
-------------------------
------- Data Count 2
Preceding Rule
-------------------------
------- Field 1 Type
sourceIpv4SourceNetwork
------------------------
-------- Field 2 Type
destinationIpv4Address
------------------------
-------- Field 3 Type
packetDeltaCount
------------------------
-------- Field 4 Type
octetDeltaCount
------------------------
-------- Field 5 Type
flowStartMilliSeconds
------------------------
-------- Field 6 Type
flowEndMilliSecondsess
------------------------
-------- Data 1 Type
destinationTransportPort
------------------------
-------- Data 1 Value 80,443

-------------------------
------- Data 2 Type
destinationIpv4Network
------------------------
-------- Data 2 Value
10.10.0.0/16
------------------------
--------
9
Example contd
pattern in data template

• Incoming flows
• Resulting meta-flow

Prot Src Port Src Addr Dst Port Dst Addr Pkt Oct Start End
TCP 64235 10.0.1.1 80 10.10.0.10 4 144 1055 1090
TCP 64236 10.0.1.1 80 10.10.0.10 3 56 1071 1103
TCP 6889 10.0.1.2 80 10.10.0.10 2 34 1083 1100
TCP 5555 10.0.2.1 80 10.10.0.10 6 155 1090 1201
TCP 6666 10.0.2.1 80 10.10.0.11 3 77 1095 1199
discarded
fixed-value in data template
Src Net Dst Addr Pkt Oct Start End
10.0.1.0/24 10.10.0.10 9 234 1055 1103
10.0.2.0/24 10.10.0.10 6 155 1090 1201
10.0.2.0/24 10.10.0.11 3 77 1095 1199
10
Cascading Aggregation Rules

• Goal
• Allows other semantics than match-any, i.e. may
  be used to avoid that an incoming flow
  contributes to more than one meta-flow
• Cascading aggregation rules
• Use preceding rule field in data template header

Get incoming flow
preceding rule
Apply rule 1?
no
preceding rule
Apply rule 2?
no
yes
Aggregate

yes
Aggregate
11
Conclusions

• IPFIX Aggregation -00 received only positive
  feedback
• -01 has reached a good state
• Already two implementations supporting
  aggregation
• IBM
• Erlangen University / Tuebingen University
• Next steps
• To be continued as an individual I-D?
• To be added to the IPFIX charter?