RTP Scalability in Large Multicast Groups

1
RTP Scalability in Large Multicast Groups

• Jonathan Rosenberg
• High Speed Networks Research
• 11/10/97

2
Contents

• RTP and RTCP Primer
• Large scale applications
• Problems
• Congestion
• State Storage
• Delay
• BYE Storms
• Premature Timeouts

• Reconsideration
• Conditional
• Unconditional
• Reverse
• BYE
• Polling
• Group Size Estimation
• Conclusions

3
RTP and RTCP

• RTP provides for
• Real time transport
• Resequencing
• Payload type identification
• Intra and Inter media synchronization
• Encryption
• Multicast
• Per User demultiplexing - SSRC
• RTP does not
• Provide QoS
• Require RSVP

• RTP is a framework
• Specific payload formats defined for H.263, MPEG
  etc.
• UDP Port numbers based on application

4
RTCP

• Real Time Control Protocol
• RTP port 1
• Used for
• QoS Reporting
• Sender reports packets sent, bytes sent
• Receiver reports (per sender) loss, delay,
  jitter observed instantaneous and cumulative
• Media Synchronization
• NTP and RTP Timestamp correlation
• Loose Session Control
• Hello, Bye messages
• SDES - email, username, CNAME, etc.

5
Large Scale Applications

• Cable Distribution
• one sender, tens of thousands of receivers
• QoS reporting critical
• Nielsen - group size estimation
• Distributed real time sensors
• Van Jacobson RLM - Receiver Driven Layered
  Multicast
• MBONE concerts, talks

6
Achieving Scalability

• RTP
• RTP scales if number of data senders is small or
  controlled
• Generally not a problem
• RTCP
• Both data senders AND receivers send reports
• Reports are periodic
• Period must scale with group size to avoid
  implosion

• Period is defined to scale linearly
• Each system listens to group and counts number of
  other users heard from - L(t) is group size
  estimate
• Period is set to CL(t) C depends on application
• Works fine if group size is static
• 10M question How does this work when L(t) is
  dynamic?

7
Other Applications

• Problem is general applies when
• Some unkown number of users wish to share a
  bandwidth resource equally
• The users are on a shared medium, and can hear
  each other, like multicast
• The number of users grows/shrinks over time
• Examples
• CBR traffic on Ethernet
• Route updates among BGP peers
• Distributed Sensors

8
RTCP Intervals

• Require RTCP reporting to scale with multicast
  group size
• Total RTCP bw should be no more than 5 of
  session bw
• Divide 5 among participants - 75 senders, 25
  receivers
• Algorithm
• At time t, emit packet
• Compute interval T
• Schedule packet for tT
• Repeat at tT
• TMIN is 2.5s for first packet, 5s for all others
• Add randomization to avoid synchronization

9
Problems

• Congestion
• Simultaneous Joins (ER at 10pm)
• RTCP Interval based on group size estimate
• N users, each think there is only ONE group
  member
• Multicast flood over initial 2.5s interval - all
  N users send
• Packet losses -gt further congestion
• State Storage
• To count group size, must store state for each
  other group member
• Set top boxes - memory size limitations

10
Problems

• Delay
• Frequency of reports from any user becomes small
• Average loss reports over large intervals useless
• Time to converge on group size exceeds membership
  times
• DVMRP cache times -gt EVERY RTCP packet is
  flooded!
• Premature Timeouts
• Must timeout members if you havent heard from
  them in M transmission intervals
• Drop in group membership reduces transmission
  interval
• Sharp drop can cause a user to prematurely
  timeout other users
• BYE Storms
• Many users simultaneously leave and send a BYE
  message

11
Reconsideration

• Avoid Congestion
• Last packet send time was tn-1
• About to send at time tn
• Reconsider if number of users changed since tn-1
• Compute interval based on new observation -gt T,
  add to tn-1. If result is less than current time,
  send now, else reschedule for tn-1 T
• Conditional As above
• Unconditional Recompute even if number of users
  unchanged
• Backwards compatible
• Effectiveness grows with usage
• Safe

3 new users observed
tn-1
tn
tn-1 T
12
Simulation Assumptions

• 28.8 kbps access links downstream
• Infinitely fast upstream links
• Access server buffers 100kB per user
• Network adds delay uniform between 0 and 600ms
• Full connectivity - multicast join latency not
  considered
• Step join 10K users at t0. Each have a
  homogeneous view of state

13
Packets Sent
14
Analytical Results

• Can derive a differential equation for L(t) given
  a step join!
• For small t, solution is linear when there is no
  network delays
• Can also show that packet rate is roughly
  derivative of L(t)
• Gives an estimate of network bandwidth

15
Add Delay

• Real network has delay - significant impact on
  performance
• Arrival of packets used to increase group size
  estimate and back-off -gt network delays cause
  initial bunch of senders not to see packets
• Result is a spike of packets
• After packets arrive, causes rapid increase in
  group size estimates
• Result is a cessation of packet transmission
  plateau effect
• What is size of spike?
• Again, analytic solutions exist!
• Let D be fixed network delays, m is modem rate in
  packets/s
• In limit of N large, distribution of delays
  doesnt matter!
• Conditional - first a linear growth in packets,
  then a growth as t2 - much smaller

16
Unconditional

• Unconditional works much better - growth in
  number of packets consists only of small t2
  component
• Total packets sent grows with square of delay to
  TMIN quotient
• Reason - unconditional skews distribution of
  packet transmissions towards later in time

17
State Storage

• Problem
• Must maintain group size estimate
• To distinguish new users from old, must maintain
  SSRC list for all users
• In a group of thousands, can be problematic

• Solution
• Dynamic sampling
• Given memory size M, store all SSRC until memory
  full
• Discard all SSRC with MSB1 (roughly 1/2)
• Sample probability is 2-k
• Group size estimate is Ns 2k

18
Delay

• May not desire RTCP in current distributed form
• When you know group size is large apriori
• Central monitor polls stations for reports
• Monitor sends SSRC mask, and interval T
• Stations whose SSRC match mask respond to poller
  within T seconds
• Must gradually decrease size of mask to avoid
  flood (Bolot, Wakeman)

• Standard sampling theorems allow group size
  estimation and confidence computations
• IDEA - Use non-uniform response distribution, and
  allow poller to cancel poll
• If poller gets lots of responses early, it can
  expect tons later, so cancel the poll, decrease
  probability
• Like the beach parking lot problem!

19
Premature Timeouts

• Subtle problem
• Timeout if no packet seen in MCL(t)
• Rapid leave causes L(t) to drop timeout interval
  drops
• Remaining users wont send packets immediately
  change in interval size takes effect after next
  packet transmission

RTCP Packets
BYE Flood
0
1
2
3
Original Timeout Window
Timeout Window after Flood
20
Solution

• Two-fold
• Reverse Reconsideration
• When BYE arrives, each user reschedules their
  next packet a little earlier
• Allows packet transmission rate to closely track
  group membership

• Group Size filtering
• Dont use L(t) for timeout - use filtered
  version!
• How to filter?
• Optimal timeout window size topt is solution to

21
BYE Storms

• Users simultaneously leave also!
• Usually send a BYE - flood of packets
• Solution Apply reconsideration to BYE packets
• At leave time, schedule BYE packet
• Count number of BYEs from other users since my
  leave time
• Use that as group size estimate in
  reconsideration equations
• Works very well

22
Conclusions

• RTP Scalability to large multicast groups has
  several components
• Step joins and leaves
• State storage
• Premature timeouts
• Problem is general
• Desire equal bandwidth sharing among users
• Number of users is known in a distributed way by
  observing transmission
• Number of users is very dynamic