Licentiate Seminar: On Measurement and Analysis of Internet Backbone Traffic

1
Licentiate SeminarOn Measurement and Analysis
of Internet Backbone Traffic

• Wolfgang John Department of Computer Science and
  EngineeringChalmers University of
  TechnologyGöteborg, Sweden

2
Why measure Internet traffic? (1)

• The Internet is changing in size

Internet, 1983
Internet, 2005
ARPANET, 1969
3
Why measure Internet traffic? (2)

• The Internet is changing in application

4
Why measure Internet traffic? (3)

• The Internet
• is constantly developing
• is used differently in different locations
• is heterogeneous
• The Internet is not understood in its entirety!

INTERconnected NETworks
NET
INTER
5
Why measure Internet traffic? (4)

• Operational purpose
• Troubleshooting, provisioning, planning .
• Scientific purpose
• Protocols, infrastructure and services
• Performance properties
• Internet simulation models
• Security measures

6
Thesis Objectives

1. Guidelines for Internet measurement
2. Current traffic characteristics
3. Traffic decomposition
4. Inconsistent behavior

7
Outline

• Measurement approaches
• Internet measurement challenges
• The MonNet project
• Scientific contribution
• Results
• Four studies included
• Conclusions

Measurement
Analysis
8
Measurement approaches
Network traffic measurement
Passive
Active
Software
Hardware
Online
Offline
Packets
Flows
Statistical summaries
Complete
Headers
Different protocol levels
Transport layer
9
Internet measurement challenges (1)

• Legal considerations
• Ethical and moral considerations
• Operational considerations
• Technical considerations

10
Measurement challenges (3)

• Technical considerations
• Data amount
• Exhausting I/O and storage access speeds
• Data reduction techniques
• Filtering, sampling, packet truncation
• Timing
• Clock synchronization

11
The MonNet Project (1)

• Technical Solution

Processing Platform and Storage
Measurement Node 1
splitter
10 Gbps
Göteborg
Borås
10 Gbps
Measurement Node 2
12
The MonNet Project (2)

• Measurement location

• April 2006 148 traces (20 minutes) 11 billion
  packets, 7.6 TB of data
• Sept. Nov. 2006 554 traces (10 minutes) 28
  billion packets, 19.5 TB of data

Internet
Stockholm
Student-Net
Borås
Regional ISPs
Göteborg
Göteborgs Univ.
Chalmers Univ.
Other smaller Univ. and Institutes
13
Scientific Contribution
Level of complexity
Packet level
Flow level
Traffic classes
Traffic characterization
Study III
Study I
Study II
Study IV
Quantification of inconsistent behavior
Upcoming
14
Study I Packet Level Analysis

• Updated packet-level characteristics of Internet
  traffic
• Inconsistencies in headers will appear
• Network attacks and malicious traffic
• Active OS fingerprinting
• Buggy applications or protocol stacks

15
Study II Flow level analysis

• High level analysis does not necessarily show
  differences ? detailed analysis does!
• 2 main reasons for directional differences
• Malicious traffic
• the Internet is unfriendly
• P2P
• Göteborg is a P2P source
• P2P is changing traffic characteristicse.g.
  packet sizes, TCP termination, TCP option usage

16
Study III Classification Method (1)

• Classification of flow traffic without payload
• Heuristics to identify nature of endpoints
• Rules based on connection patterns and port
  numbers
• 5 rules for P2P traffic
• 10 rules to classify other types of traffic
• remove false positives from P2P

17
Study III Classification Method (2)

• Comparison of classification methods for P2P
  traffic

18
Study III Classification Method (3)

• Previous classification methods on packet header
  traces dont work well on backbone data
• Proposal of refined and updated heuristics
• Simple and fast method to decompose traffic
• No payload required
• Effectively used even on short traces (10 min)
• 0.2 of the data left unclassified

19
Study IV Classification Results (1)
Tuesday, 18.04.2006
20
Study IV Classification Results (2)

• Application breakdown April till Nov. 2006

21
Study IV Classification Results (3)

• Connection establishment for traffic classes

22
Study IV Classification Results (4)

• Behavior of P2P traffic
• Unsuccessful TCP connection attempts increasing
• Serving peers terminate with FIN and
  RSTDecreased from 20 to 8
• UDP overlay traffic doubled
• TCP options deployment differs
• P2P behaves as expected
• Web traffic shows artifacts of client-server
  pattere.g. popular web-servers neglecting SACK
  option

23
Summary

• Guidelines for Internet measurement
• Experiences of the MonNet project
• Current traffic characteristics
• Packet and flow level
• Traffic decomposition
• Traffic classification method
• Inconsistent behavior
• Packet header anomalies
• Malicious traffic flows

24
General remarks

• Internet today is essential, but still not
  understood entirely
• Large-scale traffic measurements uncommon
• A lot of analysis is done on outdated datasets
• Each study generated as much questions as answers
• Reconsider measurement process (duration,
  payload)
• A lot of open questions
• get more answers in two years