Internet Quality of Service

1
Internet Quality of Service
Michael Welzl michael.welzl_at_uibk.ac.athttp//come
.to/michael.welzl Institute of Computer
Science University of Innsbruck, Austria
2
Outline

• Motivation - why QoS?
• Brief introduction to networking, the Internet
  and QoS
• Internet dynamics congestion control traffic
  characteristics
• End2end QoS
• RTP/RTCP, XTP, RTSP, fairness / TCP friendliness,
  multimedia adptation issues
• QoS as managed unfairness
• IntServ / RSVP, DiffServ, QoS Routing, Traffic
  Engineering, MPLS, MP?S, IPv6, Internet2 QBone
• Lessons learned and the future of Internet QoS

3
Networking a brief introduction

• Network layers, ISO/OSI vs. DoD model,
  terminology, the Internet

4
Logical communication flow
draw a green rectangle
5
Physical communication flow
language (protocol)
draw a green rectangle
C H A O S
translation
translation
Network(Internet Cloud)
Choose a path
check for logical errors
check for bit errors
01001010111101010010101110100010101100001111010001
0111001011
6
Abstraction

• Programming Languages
• Machine code
• Assembler
• Low level languages
• High level languages (special purposes),OO
  Design, ..
• Simplification by abstraction
• Same for networks network layer models

7
ISO/OSI Reference Model

• THE famous layer model
• 7 layers
• precise terminology, huge amount of theoretical
  work
• layer provides service to upper layers
• strict rules (layers must not be skipped, but
  they are interchangeable)

8
OSI Architecture

• ISO Basic Reference Model for Open Systems
  Interconnection (OSI)
• OSI model is concept, architecture, common
  terminology.

9
OSI Layers

• Application oriented layers7. Application
  Layer actual communicating apps6. Presentation
  Layer semantics5. Session Layer structured
  dialogue - synchronisation, ..
• Transport oriented layers4. Transport Layer
  end2end msg. stream, no knowledge of routing3.
  Network Layer routing, packets between adjacent
  systems(LANs 2b Logical Link Control L.2a
  Medium Access Control)2. Data Link Layer
  error-recovery, frames (not packets) stream
  between adjacent systems1. Physical Layer
  unsecure bitstream between adjacent systems

10
The most successful (packet) network is...(and
the winner is...)
The Internet
11
How the Internet has changed

• Original goal robust communication on a
  long-term basis
• Original size ARPANET...

12
How the Internet has changed

• Currently 69 new hosts added each minute!
• IEEE Spectrum, Feb. 2001
• Commercial demand for new, accordingly priced
  services(VoIP, streaming audio/video,
  videoconferencing, ..)
• Overprovisioning does not suffice - demands
  increase
• Goal is not just speed / reliability
  anymoreInternet "best effort" service is not
  good enough

13
Common Internet myths misconceptions

• The network automatically finds the best path for
  my packets.
• IPv6 is brand new, provides QoS and will be
  widely deployed soon.
• The Internet changes with incredible speed.
• Overprovisioning can solve all QoS problems.
• Somewhat less commonInternet2 will soon replace
  the Internet as we know it.
• World Wide Web Internet.
• The Internet exists since ...
• TCP/IP is (are) the Internet protocol(s).

14
Primary reason for tremendous success
end2end argumentmove complexity to higher
layers in the stack...important Internet design
rule!
Example security Security provided by the
network may be too bad or unnecessary for some
applications. The network cannot provide
authentication only the application has access
to the necessary information
Today, scalability remains the no. 1 goal
15
DoD reference model

• DoD-model older (approx. 1969) than OSI (approx.
  1977)
• Only 4 layers
• Application Layer
• Transport Layer
• Internet Layer ( layer 3, Network layer)
• Network Access Layer ( everything underneath the
  Internet)
• Layers 5 and 6 missing
• Less restrictive - layers may be skipped

16
OSI, DoD and reality

• OSIMany service functions carried out in
  several layers / services? Overhead, even
  reversal!
• Why should I implement layers 5, 6 in my app /
  OS ?
• Commercial failure - but still useful to explain
  networks
• DODactually obsolete Internet defined by
  TCP/IP stack(defined by RFCs)

17
TCP/IP Protocol Stack

• IP addressing, routing, fragmentation/reassembly,
  TTL
• UDP ports, checksum
• TCP UDP connection-oriented service
  (retransmission/ACK),combined flow control /
  congestion control

18
A simple router model
Switching Fabric
In 1
Out 1
Queue 1
In 2
Out 2
In 3
Queue 2

• Switching fabric forwards a packet (dest.
  addr.)
• if no special treatment necessary fast path
  (hardware)
• Queues grow when traffic bursts arrive
• Packets are dropped when queues overflow
  ("DropTail queueing")

19
IP Routing Domain level (IGP's)

• Distance Vector Routing RIP, routers only
  keepinformation about adjacent routers
• Link State Routing OSPF, IS-IS - each router
  knows about the whole domain (less scalable!)
• based on routing metric, usually no. of hops -
  not always optimal

20
Interdomain routing (EGP's)

• BGP Distance Vector Routingbetween "Autonomos
  Systems"(registered unique AS number)
• Metrics often configuredmanually according to
  costs
• Peering relationshipsusually cheaper than
  thebackbone
• Implementation filteronly accept routes
  fromspecific sources

21
Flow Rate Control

• Remember refers to next msg-no to be
  treated!!
• Version described at left
• credit C-A is fixed-size window
• C-A pushed by ACKs
• S, R move within window
• sender keeps copies of A thru S-1
• Common design flaw
• Rcv. wants to slow down sender? holds back ACK
• ? timeouts, retransmits
• Remedy, cf. TCP
• TCP separates ACK / window size (credit) per
  packet
• _______________________________
• Rate control
• S, R (, routers) negociate data rate
• may be slowed down by either party
• No explicit feedback necessary

• Sliding window flow control
• most widely used, many variants

22
Who defines the Internet?

• Preliminary research in IRTF ( http//www.irtf.org
  )
• Standards (RFCs) defined by IETF (
  http//www.ietf.org ) -mostly Working Groups
• Decisions by IESG (as of Feb. 2001, 14 elected
  members)
• IAB stimulates IETF / IESG actions
• RFCs have different status standard, proposed
  standard, draft standard, experimental,
  informational, ..
• Internet-draft preliminary - may turn into RFC

23
Who really defines the Internet?

• RFCs define the Internet. Only standard RFCs
  mandatory
• Extremer views, but maybe closer to reality
• Jon Postel defined the Internet.
• Cisco defines the (core) Internet.
• Microsoft defines the (end2end) Internet.
• Some standards never made it.Some other things
  did (MBone, NAT, P2P, ..)

24
Standards that never really made it(and probably
never will)

• TTL as an actual time value
• The (originally planned usage of the) IP TOS
  field
• Several IP options Strict / Loose Source
  Routing Record Route Timestamp
• Source Quench

25
Who runs the Internet?

• (See http//www.caida.orgfor more pictures)
• Hierarchical structure
• Distributed ISP admins run it
• No Internet "police"
• Each ISP interested in his/her own services, but
  end2end paths may include the backbone
• CPU cycles scarce in core routers

26
If you really want to know how it works

• ... check out
• http//www.warriorsofthe.net

important links http//www.ietf.org/ http//www.i
rtf.org/
27
Quality of Service 1

• Concept, specification, architectures

28
QoS and network layers
29
QoS and network layers /2

• QoS fundamentally an end-to-end issue...
• QoS spec. must not be violated at any layer
• QoS request may originate from (almost) any layer
• QoS provisioning may be demanded at (almost) any
  layer
• There is no overall framework -demand for QoS
  often leads to layer violation

30
Important QoS parameters

• Latency - time to transfer an "empty" message -
  not directly relevant to an application
• Bandwidth (throughput, goodput) - how many
  bits/sec can be transferred ("how thick is the
  pipe")
• Consider modem connection vs. a van of mag tapes
  traveling an interstate highway
• End2end delay latency msg_length / min.
  path bandwidth
• queuing delay
• Jitter - delay fluctuations, very critical for
  most real-time applications

31
Delay / Bandwidth

• Delay / Bandwidth are related, but not
  interchangeable
• On the Internet, delay changes indicate (path
  changes or) congestion -gt TCP Vegas
• Congestion related to bandwidth -gt delay related
  to bandwidth!
• but

similar delay, different available bandwidth!
32
QoS below IP

• LAN Medium Access Control (MAC) Layer
• CSMA/CD (Ethernet) behaviour practically
  unpredictable(collisions lead to Binary
  Exponential Backoff, calculations too
  complicated)
• Token passing schemes bandwidth / delay
  predictable
• WAN ATM-Layer (ATM has its own 3-dimensional
  model)
• ATM was the first serious QoS attempt - "ATM to
  the desktop"
• Constant cell size of (548) bytes enables Time
  Division Multiplexing-gt predictable data rate!

33
ATM Services
34
ATM Services and Switching



35
ATM and reality

• ATM to the desktop dead
• Most often used for high-speed Internet links
  (backbone)
• Suboptimal for various reasons
• Cell size does not match packet sizes
• IP provides datagram service, no use for CBR
  etc.
• IP mostly used with UBR or ABR service in case
  of ABR,TCP is a control loop on top of a control
  loop!

36
Typical QoS requests
37
"Human Layer" QoS requirements
38
QoS Architectures

• Heidelberg QoS Model, OMEGA, int-serv, XRM
  (hierarchical), QoS-A and Tenet (3-dimensional),
  OSI, TINA, MASI, ..
• Various concepts related to layers (OSI, QoS-A),
  related to specific implementations (int-serv),
  ..
• Architectures identify fundamental concepts of
  QoS specification, provisioning, control and
  management
• No overall agreement on a single architecture

39
Internet congestion control

• TCP congestion control, Active Queue Management,
  ECN

40
The Congestion Control problem

• Congestion control necessary
• adding fast links does not help!

total throughput w/o cc. 20kb/s total throughput
w/ cc. 110kb/s
41
The old days

• 1968/69 dawn of the Internet
• 1986 first congestion collapse
• 1988 "Congestion Avoidance and Control"
  (Jacobson/Karels)Combined congestion/flow
  control for TCP
• Goal stability - in equilibrum, no packet is
  sent into the network until an old packet leaves
• ack clocking, conservation of packets principle
• made possible through window based stopgo -
  behaviour
• Superposition of stable systems stable
  -gtnetwork based on TCP with congestion control
  stable

42
TCP Congestion Control /1 Tahoe, 1988

• Distinguish
• flow control protect receiver against overload
• (receiver "grants" a certain amount of data
  ("receiver window") )
• congestion control protect network against
  overload
• ("congestion window" (cwnd) limits the amount of
  data TCP)
• Flow/Congestion Control combined in TCP. Several
  algorithms
• (SMSS Sender Maximum Segment Size,init
  cwndlt2SMSS, ssthresh usually 64k)
• Slow Start for each ack received, increase cwnd
  by 1(exponential growth) until cwnd gt ssthresh
• Congestion Avoidance each RTT, increase cwnd by
  SMSSSMSS/cwnd(linear growth - "additive
  increase")

43
TCP Congestion Control /2

• If a packet or ack is lost (timeout, roughly
  4rtt), set cwnd 1, ssthresh current
  bandwidth / 2(multiplicative decrease") -
  exponential backoff
• Several timers, based on RTT good estimation is
  crucial!
• Later additions(TCP Reno, 1990)Fast retransmit
  / fast recovery (notify sender of loss via
  duplicate acks)

44
Background AIMD
45
TCP Congestion Control /3

• Timeout interpreted as congestion
• was bad idea in times of error-prone networks
• seems reasonable in times of fibre networks
• absolutely insane for wireless links!
• TCP over long fat pipes large bandwidthdelay
  product
• long time to reach equilibrium, MD problematic!
• latest definition of TCP Congestion Control in
  RFC 2581, April '99
• Different implementations / extensions
• TCP SACK - explicit notification of missing
  segments
• TCP over wireless - checksum error -gt do not
  immediately drop packet-gt not interpreted as
  congestion
• TCP over sat (also "Interplanetary Internet")
  larger windows, ..

46
Active Queue Management

• Today, TCP behaviour dominates the Internet (WWW,
  ..)
• Recent backbone measurement 98 TCP traffic
• 1993 Random Early Detection ("Discard", "Drop")
  (RED)(now that end nodes back off as packets are
  dropped, drop packets earlier to avoid queue
  overflows)
• Another goal add randomization to avoid traffic
  phase effects!
• Qavg (1 - Wq) x Qavg Qinst x Wq(Qavg
  average occupancy, Qinst instantaneous
  occupancy, Wq weight - hard to tune,
  determines how aggressive RED behaves)

47
Active Queue Management /2

• Based on exponentially weighted moving average
  (EWMA) of instantaneous queue occupancy low
  pass filter
• recalculated every time a packet arrives
• Qavg below threshold min_th Nothing happens
• Qavg above threshold min_th Drop probability
  rises linearly
• Qavg above threshold max_th Drop packets
• RED expects all flows to behave like TCP - but is
  it fair?
• Variants drop from front, drop based on
  instantaneous queue occupancy, drop arbitrary
  packets, drop based on priorities...

48
Explicit Congestion Notification (ECN)

• 1999 Explicit Congestion Notification
  (ECN)Instead of dropping, set a bit
• End systems are expected to act as if packet was
  dropped-gt actual communication between end nodes
  and the network!
• ATM and Frame Relay not only ECN but also BECN
• Internet BECN often proposed and regularly
  discussed (ICMP SQ), but very unlikely - several
  reasons
• Idea charge more when ECN flag is set
• ECN cannot totally replace loss measurements!

49
Elephants Mice
50
Internet traffic characteristics

• MRTG trace (based on SNMP, accessing traffic
  counters in MIB)

51
Internet traffic characteristics /2

• Traditional traffic modelling queuing
  theorynotion traffic follows poisson
  distribution
• Internet traffic is bursty - intuitive reasons
• TCP is bursty by nature congestion avoidance,
  payload vs. acks
• ACK compression can cause payload bursts due to
  ACK-clocking
• various packet sizes
• Bursts from queues aggregate as traffic traverses
  the net
• Burstiness of one flow affects other adaptive
  flows

Still true for user arrival !
52
Internet traffic characteristics /3

• Overlapping of independent on-off sources leads
  to distribution with heavy-tailed autocorrelation
  function
• Long-range dependance "peaks sit on ripples
  which sit on waves"
• No "flattening" towards a mean as you zoom out -
  same structures may be found at different time
  scales, hence self similar
• Traffic characteristics sometimes modeled with
  time series (fARIMA models) or wavelets
• Measurement of the "degree of self similarity"
  Hurst parameter
• -gt model approximation involves Hurst parameter
  estimation
• Calculations extremely difficult -Internet
  analysis mostly based on simulation!

53
Internet measurements

• Topology traceroute, ping
• Bandwidth tcpdump, pathchar
• Popular technique packet pair approach
• send a large packet p1 followed by a small packet
  p2
• probability that p2 enqueued behind p1 extremely
  high
• at receiver calculate bottleneck bandwidth via
  time between p1 and p2
• minimize error via multiple probes
• problem different queueing mechanisms (e.g., WFQ
  at bottleneck)
• Lots of other measurement goals (geographic
  position estimation, ...) and methods
• Various tools results at http//www.caida.org !

54
Quality of Service 2

• End 2 end
• RTP/RTCP, XTP, RTSP, fairness / TCP friendliness,
  multimedia adptation issues

55
End2end real-time data transfer

• Assumption no special service available at
  application level
• (Definition of Internet "real-time" softer than
  usual)
• Different requirements
• reliable service may not be needed (no
  retransmission)
• Timely transmission important
• Different treatment
• no retransmission / waiting for ACKs
• no sliding window (stop go behaviour not
  feasible)
• but
• some kind of flow control still needed
• synchronization necessary
• often Multicast

56
End2end bandwidth adaptation example
Bandwidth
Time
T0
T1
57
Real Time Protocol (RTP)

• Can be used on top of anything, but usually UDP
• adds a timestamp
• supports payload type identification
• adds a counter to preserve the original packet
  order
• Intermediate systems
• Translator converts encodings, filtering
• Mixer synchronizes multiple streams based on RTP
  timestamps
• Real Time Control Protocol (RTCP)
• Receiver reports provide QoS feedback (packet
  loss,interarrival jitter)
• Sender reports similar to RRs, issued if receiver
  isalso a sender

Regular network end nodes -gt overlay network
model!
Other examplesMbone, VPNs
58
Other related protocols

• XTP
• rate-based protocol for multimedia
  transport(instead of allowing a certain amount
  of data, inform the sender about the allowed data
  rate)
• no time synchronization / timestamps
• Multicast (overlay)
• Application level protocols (mainly for ease of
  use)
• Real Time Streaming Protocol (RTSP)
• Application level protocol supporting VCR-like
  commands
• Play, Forward, Rewind, Pause, Stop, Record
• Just signaling - no payload transmission (usually
  done with RTP/UDP)
• Not connection-oriented, can be used on top of
  TCP or UDP
• Hypertext Transport Protocol (HTTP)
• Can be used to control streaming multimedia
  (browser plug-ins, ..)

59
Data encoding / application layer signaling

• RTP content type definitions encapsulate various
  formats
• H.323 protocol suite
• big, covers various layers (e.g., H.263 video)
• encoding formats for video, audio
• originally designed for ISDN (CBR)
• SDP, SAP, SIP
• newer than H.323, improved scalability
• session signaling similar to http request (ascii)
• designed to co-exist with RTP/RTCP in the
  Internet
• MPEG
• Fluctuating data stream designed for efficient
  encoding
• Semantics in MPEG7

Compare DivX downloads lt-gt streaming video on
demand
60
Why adapt?

• Popular recommendation from researchers
• primary goal stable network, avoid congestion
  collapse
• stability proved for AIMD congestion control
• Less popular among commercial application
  developers... why?
• QoS gain not clear / not good enough
• Often "Lower resolution but constant frame rate
  (less packet loss)"
• Specific QoS goals cope with
• congestion
• packet loss
• delay
• jitter
• transmission errors

Notevery critical effects for VoIP!
61
How adapt? Measure...
well studied- leads to misinterpretation
of transmission errors

• throughput ("goodput")
• ( mean, fluctuations,
• packet loss ratio..)

easy to measure independent of transmission
errors- not practical without throughput
delay( rtt, one way delay, jitter..)
similar delay, different available bandwidth!
62
... and change!

• lower layers
• throughput (gap between packets)
• flow control / congestion control
• well studied, many options - our main interest!
• packet size
• large recommended(Path MTU Discovery),less
  overhead
• small less impact of transmission
  errors,smaller latency!
• protocol (UDP Lite)

Note packet size granularity ofthroughput
measurements
VoIP, online games, ...

• content
• compression
• hierarchical encoding
• FEC

Common difficultiesbandwidth known (depending
on content)?granularity of rate changes
63
Multimedia adaptation

• Mistake
• adaptation schemes assume arbitrary data stream
  scalability
• Problem
• Data streams show fluctuations (example MPEG I-,
  B-, P-frames)
• Solution
• Special CBR design for communication - H.261
  designed for ISDN
• not possible or feasible in all cases
• Problem
• compression usually not deterministic - size
  depends on content!
• real-life distance learning example40kbps
  enough for streaming video (Smartboard) audio
  (speech), but speech suffers dramatically if
  teacher visible

64
Fairness

• ATM ABR Max-Min-fairness
• A (..) allocation of rates is max-min fair iff
  an increase of any rate (..) must be at the cost
  of a decrease of some already smaller rate.
• One resource mathematical definition satisfies
  "general" understanding of fairness - resource is
  divided equally among competitors
• Usually requires knowledge of flows in routers
  (switches)
• Internet
• TCP dominant, but does not satisfy
  Max-Min-fairness criterion!
• Ack-clocked - flows with shorter RTT react sooner
  (slow start, ..)and achieve better throughput
• Therefore, Internet definition of fairness
  TCP-friendliness"A flow is TCP-compatible
  (TCP-friendly) if, in steady state, it uses no
  more bandwidth than a conformant TCP running
  under comparable conditions."

65
Proportional Fairness
F. Kelly Network should solve a global
optimization problem (maximize log utility
function) Max-Min-fairness suboptimalS1 S2
S3 c/2
All link capacities c

• Proportional fairness
• An allocation of rates x is proportionallyfair
  iff, for any other (..) allocationy, we have
  (roughly approximated by
  AIMD!)

Proportionally fair allocationS1 c/3, S2 S3
2c/3
66
How to be TCP-friendly

• TCP-friendliness can be achieved by emulating the
  behaviourof TCP (or the desired parts of it)
• Simplified TCP AIMD (additive incr. ? ,
  multiplicative decr. ?)
• 0 lt ? , 0 lt ? lt 1 -gt stable and fair
  congestion control
• ? 4 x (1 - ?2) / 3 -gt TCP-friendly
  congestion control
• ? 1, ? 1/2 -gt TCP
• AIMD mechanisms for multimedia applications RAP,
  LDA
• Different approaches
• TCP Emulation At Receivers (TEAR)TCP
  calculations (cwnd calculation, fast recovery,
  ...) moved to receiver, do not ack every packet,
  smooth sending rate
• Binomial congestion control TCP-friendly,
  nonlinear control law

67
GAIMD congestion control

• Relationship between ? and ? for TCP-friendliness

more aggressive responsive
TCP
smoother
68
Equation based congestion control

• Based on TCP steady-state response function-
  gives upper bound for transmission rate T
  (bytes/sec)

• well known example TFRC - TCP-friendly rate
  control protocol
• smooth sending rate

69
Evaluation of TCP-friendly mechanisms

• Infocom 2001 paper by Yang, Kim, Lam examines
  simulations ofTCP (Reno) vs. GAIMD (?0.31,
  ?7/8) vs. TFRC vs. TEAR(bigger is better)
• Network simulator "ns-2
• Well known scenario compete on a single
  bottleneck link ("dumbbell")

70
...and QoS?

• Parameters not very QoS oriented
• Smoothness not very interesting if large buffers
  are available (one-way streaming)
• Despite efforts to identify non-adaptive
  flowsThe more you send, the more you get.
• Probably most interesting QoS question beside
  smoothnessHow closely does the mechanism follow
  the available bandwidth?
• measurable parameter packet loss ratio

71
Real life evaluation AVCS
Adaptive VideoCommunication SystemTestbed for
adaptationmechanismsRAP, TFRC
implemented Not yet finished, butalready
illustratesunsolved difficulties
72
Not-so-TCP-friendly solutions

• Overcome rate fluctuationslimit encodings (e.g.
  2 or 3 qualities), let user decide
• Cross-media-adaptationchoose from video, audio,
  single pictures, text(e.g. MPEG7)
• Limit by bottleneck bandwidth
• often "last mile" - e.g. RealMedia
• better determine actual bottleneck via packet
  pair approach
• If wireless link involved small packets, UDP
  Lite
• Another possibility send more (do FEC) in
  response to packet loss
• (very network-unfriendly behaviour, but may yield
  less data loss)

Network stability some adaptation better than
none!
73
Some thoughts

• How TCP-friendly are 8 web browsers?
• Congestion Manager congestion control for all
  flows in OS core
• MulTCP Emulate multiple TCPs to provide
  differentiated services
• How TCP-friendly are short-lived flows?
  (web-traffic, ..)
• Some day, UDP uncontrolled traffic may be
  forbidden!
• Solution Datagram Congestion Control Protocol
  (DCCP)
• Well-defined framework for TCP-friendly
  congestion control
• Sender app chooses an appropriate congestion
  control mechanism
• Core OS implementation of mechanisms
• Lots of additional features nonces, checksums
  like UDP Lite, ...

74
Quality of Service 3

• "Managed unfairness"
• Service rules behaviours
• IntServ / RSVP, DiffServ, QoS Routing, Traffic
  Engineering, MPLS, MP?S, IPv6, Internet2 QBone

75
Generic QoS-capable router
InputInterfaces
OutputInterfaces
Meter
Policing / Admission Control Marking
Queuing Scheduling / Shaping
PacketClassification
SwitchFabric
Building blocks of modern QoS architectures
76
QoS router building blocks

• Packet Classification
• Group packets according to header properties
• Multiple fields (MF classification) needed to
  detect individual flowsip source / destination,
  protocol and port numbersproblems packet
  fragmentation (port numbers),header compression,
  encryption (IPSec)
• Meter
• Monitor traffic characteristics (e.g., does flow
  741 hold its promises?), provide information to
  other block(s)
• Policing
• Drop packets if certain conditions are fulfilled
• Admission Control
• React (not necessarily drop packets) if certain
  conditions are fulfilled
• Marking
• Mark packets (change header) if certain
  conditions are fulfilled
• for later special treatment - maybe not even in
  the same router

77
QoS router building blocks /2

• Switch(ing) Fabric
• Do a query on the routing table, decide where to
  send the packet
• Queuing
• If a packet cannot be delivered immediately
  (congestion),put in queue(s) for later delivery
• Decision which queue? Active queue management?
• Scheduling
• When to take a packet from which queue (e.g.,
  round robin)
• Shaping
• Adjust traffic characteristics if certain
  conditions are fulfilled(usually implemented in
  scheduling)
• Useful even without QoS provisioning Do not
  exceed max. promised quality - customers will get
  accustomed and complain!

78
Integrated Services (IntServ)

• Notionhard guarantees desired, per-flow
  resource reservation needed
• Two services defined
• Guaranteed Serviceguaranteed bandwidth, firm
  bounds on end-to-end queuing delaysto be used
  by real-time applications
• Controlled Loadclosely approximates the
  behaviour seen when there is (almost) no
  congestion to be used by elastic applications
• Architecture, Services / Reservation signaling
  protocol("Resource Reservation Protocol" - RSVP)
  design separated

79
IntServ per-hop requirements

• Classification
• per-flow context established via multifield
  classification
• flow context used to drive token-bucket metering

TokenGenerator
Marker, Policer, ..
Token
Token
Threshold
Token
Token
Token
Packet
Packet

• implemented as byte counter goal detect
  various degrees of burstiness
• several thresholds (also empty) with
  associated treatment possible!

IntServ traffic specification contains token
generation rate, bucket size
80
IntServ per-hop requirements /2

• IntServ token bucket metering leads to remarking
  or dropping(admission control)
• Multiple queues, one for each flow
• Implementation virtual queues - only one real
  queue per service
• Scheduler takes packets based on priorities
  (airline analogy)
• e.g., 1, 1, 2, 1, 1, 2, .. but not priority
  queuing (q1 until empty) - may cause starvation
  of q2!

• No bandwidth guarantees because of packet
  sizes!
• Solution Weighted Fair Queuing (WFQ), Class
  Based Queuing (CBQ)

81
Resource ReserVation Protocol (RSVP)

• Signaling - routers must know which flows to
  choose
• state in routers is established via PATH messages
  from sender
• Sender advertises allowed traffic spec via adspec
  messages
• Receivers initiate reservation (resv messages
  containing flow spec.)
• Multicast support, state merging

82
IntServ / RSVP discussion

• RSVP requires support by all routers(if
  unsupported, RSVP is tunneled - but no more hard
  guarantees)
• Scaling per-flow state not feasible!RSVP
  protocol not scalable either (maybe due to bad
  implementation)
• Strict guarantees per customer complicated
  accounting
• Solution "softer" QoS, no per-flow state in core
  routers - DiffServ

83
Differentiated Services (DiffServ)

• Edge routers Classifier / Meter / Marker /
  Shaper / Dropper
• Core routers static forwarding according to
  DiffServ-class, implementation may vary
• SLA Service Level Agreement between DS Domains

84
DiffServ terminology

• SLA contains non-technical aspects
• Service Level Specification (SLS)
• Parameters which determine theservice provided
  by a DS domain
• contains Traffic Conditioning Spec. (TCS),and
  other properties such as encryptionand routing
  constraints
• DiffServ Codepoint (DSCP) - IPv4 Precedence / TOS
  Bytes
• DSCP mapped to Per Hop Behaviour (PHB)
• how are packets treated in the core?
• Aggregated flows with same DSCP Behaviour
  Aggregate (BA)
• Distinguish PHB specification / implementation
• PHB Group PHBs that call for similar spec. /
  implementation

85
DiffServ details

• Edge routers MF and BA classificationbased on
  signaling, metering .. or ideas such as simply
  UDP / TCP
• Expedited Forwarding (EF) PHB
• "Virtual Leased Line" Service
• Aggregated flows must not exceed peak bandwidth
• Ingress Router Policing (dropping) Egress
  Router shaping
• Small delay - real time apps simple service
  model
• Unused bandwidth used by best-effort traffic!
• Assured Forwarding (AF) PHB Group
• Supports bursty flows
• Packets are marked with AF Class and Drop
  Precedence
• non-conforming packets are remarked

86
DiffServ details /2

• DiffServ does not define
• End2end service models
• Implementation details (PHBs, traffic
  conditioners, ..)
• But hints
• As in ATM ABR, "open" spec. leads to a lot of
  research work
• Implementation examples
• schedulers for PHB WFQ, CBQ, WRR (Weighted Round
  Robin), ..
• policers for drop precedence Weighted RED, RIO -
  RED variants which drop according to priorities
• shapers for traffic conditioningLeaky Bucket -
  enforces CBR, may drop!
• meters for drop precedence marking
• Token Bucket(s) with various thresholds("A
  Single Rate Three Color Marker")

87
DiffServ extensions / ideas

• IntServ over DiffServ
• may be good idea fine granularity of IntServ /
  RSVP signaling at edge routers end systems,
  scalability through DiffServ core
• IntServ flows are aggregated for DiffServ
• DiffServ does not participate in RSVP signaling
• IntServ treats DS Domains (EF PHB!) as a leased
  line
• Bandwidth Broker
• additional network nodes for signaling and
  negotiation
• translation SLS -gt TCS
• explicit communication with edge routers, e.g.
  via COPS
• Open specification brought some chaos, tooRed /
  green / blue packets, assured / premium service,
  Gold / Silver / Bronze olympic services .. what
  is real?

88
QoS Routing

• IntServ and DiffServ assume shortest-path
  routing!Not always optimal some flows may
  prefer a "long, fat pipe"
• Solution classify / meter, then forward
  according to requirements
• Knowledge of a path's QoS properties additional
  routing metrics(increases routing protocol
  traffic!)
• Problems
• scalability / oscillation - if QoS Routing is
  done for many sourcesquality reduced by own
  payload! use old path again?
• when / how often is QoS measured / calculated?
• QoS Routing not yet a real issue in IETF(WG only
  produced framework, OSPF QoS extensions
  experimental)

89
Traffic Engineering

• Static configuration administrators want to move
  some traffic
• based on long-term measurements

• IP-in-IP tunneling example
• B encapsulates packets (new srcB, dstC), C
  removes new header

90
From traffic engineering to MPLS

• Layer violation
• If you are tunneling along a fixed path and your
  network is ATM, you could just as well set up a
  VC for the path - faster forwarding!
• Automatic variant Ipsilon IP Switching
• Switches identify flow (MF classification),
  establish ATM-VC "Short-Cut"
• Does not scale well - fine granularity
• Better Multiprotocol Label Switching (MPLS)
• Not just (but mostly) ATM, even LANs!
• based upon separation of forwarding and control
  functionality in routers
• Label put short info. (from layer 2) in front of
  IP (like IP encapsulation)
• Label Switching Routers (LSR) forward on Label
  Switched Path (LSP)
• At destination remove label, forward IP packet
  normally

91
Multi Protocol Label Switching (MPLS)

• Forwarding Equivalence Class (FEC)
• Group of packets with similar expected treatment
  (usually same label)
• Various forms of classification possible (MF, ..)
• What if labeled packets are labeled again?
• Labels are stacked (push, pop, swap poppush,
  connects two LSPs )

92
MPLS details

• Label designed for speed
• 32 bit
• S1 this is the last label
• TTL is the only IP header field that MUST be
  treated at each hop
• Labels distributed via Label Distribution
  Protocol (LDP)
• MPLS applications
• Traffic engineering (like IP-in-IP tunneling)
• Split load by establishing more LSPs for one FEC
• Virtual Private Networks (VPNs)
• First aid if links go down (switch to different
  LSP)
• QoS support

93
QoS support with MPLS

• Obvious choices
• IntServ over MPLS
• set up a label switched RSVP tree
• RSVP-TE protocol RSVP extension MPLS-specific
  information in messages (LABEL_REQUEST in PATH,
  LABEL in RSV, ..)
• DiffServ over MPLS
• CR-LDP protocol LDP extended with traffic
  parameters
• Group packets according to DSCP
• Set up E/L - LSPs via LDP or RSVP
• E-LSP EF and AF1 packets transmitted on same
  LSP, but differentqueues (based on experimental
  bits)
• L-LSP EF and AF1 packets on different LSPs,
  different queues

94
Multi Protocol Lambda Switching (MP?S)

• Wavelength Division Multiplexing (WDM)
• frequency multiplexing for optical tranmission
  media
• optical packet switches - switches based on
  colours
• problems contention (signals with same colour
  overlap), ...
• IP over WDM difficult to realizeeasier if
  connection oriented
• Achieved via MPLS (Wavelength label) -gt MP?S
• MP?S switches can be connected via ?'s for
  default-routing and signaling (similar to MPLS
  lt-gt ATM VC)
• Even heavier layer violation!

95
Internet Protocol Version 6 (IPv6, IPNG)

• Different addresses (much bigger! but makes
  migration hard)
• Some header fields removed
• Multicast - IGMP now part of ICMP
• New optional header extensions(IPSec problematic
  for MF classification!)
• QoS support
• DiffServ field / flow label instead of ToS /
  precedence...for easier flow classification (no
  further semantics defined)

96
Internet2 Qbone Initiative

• Internet2 - http//www.internet2.edu
• partnership of over 130 universities, 40
  corporations and 30 other organizations (numbers
  of fall '99)
• One Internet2 objective (there are others)
• engineer scalable, interoperable, and
  administrable interdomain QoS to support advanced
  networked applications
• Qbone - http//www.internet2.edu/qos/qbone
• an interdomain testbed to enable Internet2 QoS
  objective
• driven by input and work from
• Internet2 QoS Working Group
• Internet2/Qbone Bandwidth Broker Advisory Council
• To participate, you must make your network part
  of Internet2!

97
Lessons learned
98
Some QoS rules

• Scalability above everything!
• especially avoid per flow state
• avoid state alltogether
• consider hierarchical structures for state
  aggregation
• QoS guarantees need a consistent end2end service
  model
• If hard guarantees are impossible, consider
  "softer" QoS
• Consider interactions with end system congestion
  control!
• Layer violations may be necessary
• Either "manage unfairness" or be fair (Internet
  TCP-friendly)

99
The future
100
History again

• 1968 ARPAnet effort startet by BBN
• 1969 first protocols developed
• 1986 congestion collapse
• 1988 "Congestion Avoidance and Control"
• 1989? QoS discussions in the IRTF
• 1993 "Random Early Detection Gateways for
  Congestion Avoidance"
• 1994 IETF WGs on IntServ and RSVP
• 1995 IPv6
• 1998 RFC on Active Queue Management
• 1998 IETF WG on DiffServ
• 1998 MPLS WG
• 1999 RFC on Explicit Congestion Notification
• 2000 RFC 2990

101
RFC2990 (IAB) - open issues

• State and Stateless QoSIntServ DiffServ are
  endpoints of a continuum of control models
• Uncertain QoS-enabled applications or just
  transport layer?Each approach has its own
  advantages / disadvantages
• IntServ explicit signaling - but DiffServ?
• Signaling of resource availability in the network
  coreDiffServ lacks signaling, IntServ/RSVP too
  fine-granular
• Still no standardized Inter-Domain signaling

102
RFC2990 (IAB) - open issues /2

• Trouble with TCPbursty by nature (ACK-clocking
  problem mentioned in RFC)token bucket
  TCP-hostileshould be managed in TCP stack
• Missing QoS routing / resource management
  solutionIntServ and DiffServ assume regular
  shortest-path routing!Not feasible - traffic
  should be split accordingly(No comment about
  MPLS in the RFC - but might be a solution)
• QoS Accounting is still not solved
• Chicken (admins waiting for apps) /Egg (app
  developers waiting for admins) problem

103
RFC2990 (IAB) - open issues /3

• Still no clear objectivesapplication-centric vs.
  network-centric goals
• Unrespolved security issuesweighted fairness
  needs contract
• End-to-end architecture is neededPossibly
  IntServ over DiffServe.g., RSVP signaling at
  edge, traffic aggregate forwarding in core
• "It is extremely improbable that any single form
  of service differentiation technology will be
  rolled out across the Internet and across all
  enterprise networks."

104
RFC2990 (IAB) - actual prediction

• "The architectural direction that appears to
  offer the most promising outcome for QoS is not
  one of universal adoption of a single
  architecture, but instead use a tailored approach
  where scalability is a major design objective and
  use of per-flow service elements at the edge of
  the network where accuracy of the service
  response is a sustainable outcome."
• "Architecturally, this points to no single QoS
  architecture, but rather to a set of QoS
  mechanisms and a number of ways these mechanisms
  can be configured to ineroperate in a stable and
  consistent fashion."

105
Further issues

• Heterogeneous environments (convergence big
  issue!)
• Problems with TCP over wireless links
• Interactions with new underlying technologies
  (GPRS, UMTS, ..)
• Problems with TCP over satellite links
• Will TCP still be feasible, anyway?
• Congestion Control over "leased line"
• Security
• Today, we have all got best effort.
• Tomorrow, you may want to steal my service!
• DoS (Degradation-of-Service) attacks?

106
Charging, billing accounting

• Most tools are there ... but
• No significant progress in global standardization
  of charging, billing accounting areas
• Numerous complicated research efforts to
  calculate prices based on QoS, but the IETF is
  behind
• Good global set of regulations needed (how much
  is given to which domain admins so they can add
  more bandwidth? What about inter-domain links?,
  ..) - may be the most difficult part
• analogy still no global laws for the Internet!

107
My opinion...

• It will still take QUITE a while for things to
  settle
• it will take even longer to solve wireless,
  sat,.. issues
• We may never get hard guarantees for QoS-enabled
  apps because charging, billing accounting
  issues will not be solved
• Mechanisms probably a fully integrated mixture
  of
• per flow edge QoS (not necessarily, but maybe
  IntServ / RSVP)
• traffic aggregate forwarding core (DiffServ)
• MPLS routing / resource management combined w/
  DiffServ
• additional signaling
• May not always work end2end and give unreliable
  and location-dependant services - app developers
  will not (be able to) care
• May never work with PDA / cell phone

108
So this was March 2001 ... and now?

• Mainly refinement of existing QoS mechanisms
  more work on...
• Scalable QoS with fine granularities
• extra signaling (NSIS)
• dynamic resource allocation for DiffServ
• special mechanisms like Dynamic Packet State
  (DPS)
• Economic models for QoS
• congestion pricing
• Congestion control as an enabling QoS mechanism
• STILL NO GLOBALLY SUCCESSFUL QOS ARCHITECTURE!

109
References
110
Recommended reading

• QoSGrenville Armitage, Quality of Service in
  IP Networks, MTP (Macmillan Technical
  Publishing), USA, April 2000G. Huston, "Next
  Steps for the IP QoS Architecture", RFC 2990,
  November 2000Torsten Braun, "Internet Protocols
  for Multimedia Communications", IEEE Multimedia
  July-September 1997 (pt. 1) and October-December
  1997 (pt. 2)
• Christina Aurrecoechea, Andrew T. Campbell and
  Linda Hauw,"A Survey of QoS Architectures", ACM
  / Springer Verlag Multimedia Systems Journal,
  Special Issue on QoS Architecture, Vol. 6 No. 3,
  pg 138-151, May 1998

111
Recommended reading /2

• Internet Congestion ControlDah-Ming Chiu and
  Raj Jain, Analysis of the Increase and Decrease
  Algorithms for Congestion Avoidance in Computer
  Networks, Computer Networks and ISDN Systems 17
  (1989)Raj Jain, Congestion Control in Computer
  Networks Issues and Trends, IEEE Network
  Magazine, May 1990Van Jacobson, Congestion
  Avoidance and Control , ACM SIGCOMM 88Frank
  Kelly, Charging and rate control for elastic
  traffic, European Transactions on
  Telecommunications, volume 8 (1997)

112
Recommended reading /3

• B. Braden et. al., "Recommendations on Queue
  Management and Congestion Avoidance in the
  Internet", RFC 2309, April 1998
• Sally Floyd and Kevin Fall, "Promoting the Use of
  End-to-End Congestion Control in the Internet",
  IEEE/ACM Transactions on Networking, May 1999
• M. Allman, V. Paxson, and W. Stevens, "TCP
  Congestion Control",RFC 2581, April 1999
• Sally Floyd and Van Jacobson, "Random Early
  Detection Gateways for Congestion Avoidance",
  IEEE/ACM Transactions on Networking, August 1993

113
Recommended reading /4

• End2end Multimedia TransmissionSally Floyd,
  Mark Handley, Jitendra Padhye, and Jörg Widmer,
  "Equation-Based Congestion Control for Unicast
  Applications", ACM SIGCOMM 2000
• Y. Richard Yang, Simon S. Lam, "General AIMD
  Congestion Control", May 2000
• Elephants and MiceMark E. Crovella and Azer
  Bestavros, "Self-similarity in World Wide Web
  Trafic Evidence and Possible Causes", IEEE/ACM
  Transactions on Networking Vol. 5, No. 6,
  December 1997
• Andrew S. Tanenbaum, "Computer Networks",
  Prentice Hall, 1998

114
Acknowledgements

• Slides by / information from
• Max Mühlhäuser
• Gerhard Stummer
• Torsten Braun
• Golden B. Richard III
• Kathleen Nichols
• various participants of the QoS Summit '99
  conference