Title: Licentiate Seminar: On Measurement and Analysis of Internet Backbone Traffic
1Licentiate SeminarOn Measurement and Analysis
of Internet Backbone Traffic
- Wolfgang John Department of Computer Science and
EngineeringChalmers University of
TechnologyGöteborg, Sweden
2Why measure Internet traffic? (1)
- The Internet is changing in size
Internet, 1983
Internet, 2005
ARPANET, 1969
3Why measure Internet traffic? (2)
- The Internet is changing in application
4Why measure Internet traffic? (3)
- The Internet
- is constantly developing
- is used differently in different locations
- is heterogeneous
- The Internet is not understood in its entirety!
INTERconnected NETworks
NET
INTER
5Why measure Internet traffic? (4)
- Operational purpose
- Troubleshooting, provisioning, planning .
- Scientific purpose
- Protocols, infrastructure and services
- Performance properties
- Internet simulation models
- Security measures
6Thesis Objectives
- Guidelines for Internet measurement
- Current traffic characteristics
- Traffic decomposition
- Inconsistent behavior
7Outline
- Measurement approaches
- Internet measurement challenges
- The MonNet project
- Scientific contribution
- Results
- Four studies included
- Conclusions
Measurement
Analysis
8Measurement approaches
Network traffic measurement
Passive
Active
Software
Hardware
Online
Offline
Packets
Flows
Statistical summaries
Complete
Headers
Different protocol levels
Transport layer
9Internet measurement challenges (1)
- Legal considerations
- Ethical and moral considerations
- Operational considerations
- Technical considerations
10Measurement challenges (3)
- Technical considerations
- Data amount
- Exhausting I/O and storage access speeds
- Data reduction techniques
- Filtering, sampling, packet truncation
- Timing
- Clock synchronization
11The MonNet Project (1)
Processing Platform and Storage
Measurement Node 1
splitter
10 Gbps
Göteborg
Borås
10 Gbps
Measurement Node 2
12The MonNet Project (2)
- April 2006 148 traces (20 minutes) 11 billion
packets, 7.6 TB of data - Sept. Nov. 2006 554 traces (10 minutes) 28
billion packets, 19.5 TB of data
Internet
Stockholm
Student-Net
Borås
Regional ISPs
Göteborg
Göteborgs Univ.
Chalmers Univ.
Other smaller Univ. and Institutes
13Scientific Contribution
Level of complexity
Packet level
Flow level
Traffic classes
Traffic characterization
Study III
Study I
Study II
Study IV
Quantification of inconsistent behavior
Upcoming
14Study I Packet Level Analysis
- Updated packet-level characteristics of Internet
traffic - Inconsistencies in headers will appear
- Network attacks and malicious traffic
- Active OS fingerprinting
- Buggy applications or protocol stacks
15Study II Flow level analysis
- High level analysis does not necessarily show
differences ? detailed analysis does! - 2 main reasons for directional differences
- Malicious traffic
- the Internet is unfriendly
- P2P
- Göteborg is a P2P source
- P2P is changing traffic characteristicse.g.
packet sizes, TCP termination, TCP option usage
16Study III Classification Method (1)
- Classification of flow traffic without payload
- Heuristics to identify nature of endpoints
- Rules based on connection patterns and port
numbers - 5 rules for P2P traffic
- 10 rules to classify other types of traffic
- remove false positives from P2P
17Study III Classification Method (2)
- Comparison of classification methods for P2P
traffic
18Study III Classification Method (3)
- Previous classification methods on packet header
traces dont work well on backbone data - Proposal of refined and updated heuristics
- Simple and fast method to decompose traffic
- No payload required
- Effectively used even on short traces (10 min)
- 0.2 of the data left unclassified
19Study IV Classification Results (1)
Tuesday, 18.04.2006
20Study IV Classification Results (2)
- Application breakdown April till Nov. 2006
21Study IV Classification Results (3)
- Connection establishment for traffic classes
22Study IV Classification Results (4)
- Behavior of P2P traffic
- Unsuccessful TCP connection attempts increasing
- Serving peers terminate with FIN and
RSTDecreased from 20 to 8 - UDP overlay traffic doubled
- TCP options deployment differs
- P2P behaves as expected
- Web traffic shows artifacts of client-server
pattere.g. popular web-servers neglecting SACK
option
23Summary
- Guidelines for Internet measurement
- Experiences of the MonNet project
- Current traffic characteristics
- Packet and flow level
- Traffic decomposition
- Traffic classification method
- Inconsistent behavior
- Packet header anomalies
- Malicious traffic flows
24General remarks
- Internet today is essential, but still not
understood entirely - Large-scale traffic measurements uncommon
- A lot of analysis is done on outdated datasets
- Each study generated as much questions as answers
- Reconsider measurement process (duration,
payload) - A lot of open questions
- get more answers in two years