QoS Support in High-Speed, Wormhole Routing Networks

1
QoS Support in High-Speed, Wormhole Routing
Networks

• Mario Gerla, B. Kannan, Bruce Kwan, Prasasth
  Palanti,Simon Walton

2
Overview

• Introduction
• QoS via separate subnets
• QoS via synchronous framework
• QoS via virtual channels
• Conclusions

3
Introduction

• Wormhole routing offers low latency, high speed
  interconnection for supercomputers and clusters.
• Its a modification of virtual cut-through
• -A packet is forwarded to output port once its
  head is received at the switch
• -If channel is busy, whole packet is buffered at
  input port
• -Wormhole packet composed of several flits is
  stored across several
• switches
• Used in high speed LANs like Myrinet
• -Asynchronous LAN
• -uses wormhole routing, source routing,
  backpressure flow control to
• achieve low latency and high bandwidth

4
Supercomputer SuperNet

• Two level architecture
• Optical backbone based on physical optical star
  topology
• High speed wormhole routing are Myrinet LANs
• Optical Channel Interface connects electronic
  LANs to optical backbone
• Provide support for distributed supercomputing.
• Scientific visualization, video display, parallel
  applications
• Different types of traffic (low latency datagram,
  high bandwidth connection oriented)
• Different types of QoS

5
Objective

• Want to provide connection oriented traffic with
  QoS parameters
• -reliable support no worm loss
• -scalable and deadlock free network
• Assumption
• -Traffic with QoS is connection oriented
• -QoS parameters specified at connection setup
• -Connection can be refused if no guarantee for
  Qos parameters
• -QoS parameters average bandwidth, end-to-end
  delay or jitter

6
QoS support via Separate Subnet

• Create two subnets
• One carries QoS traffic
• Another carries non QoS traffic
• Routing is independent for the subnets (Myrinet
  has support)
• Issues
• Call admission and control
• Source host behavior
• Number of interfaces at host

7
Call Admission Control

• Admission agent maintains state of QoS subnet
• Request for QoS traffic connection comes in
• Upon receiving request, agent decides a suitable
  route
• If route not available, host can retry or use
  other subnet
• If route exists, connection is accepted, host can
  send
• Once completed, host informs admission agent
• Admission agent update its view or state of subnet

8
Host Behavior

• Host must be responsible for amount of traffic
  injected in subnet according to QoS parameters it
  required
• Solution host uses pacing mechanism
• Allow only predetermined number of flits to be
  transmitted per time period

9
Number of Interfaces

• Suppose host has only one interface
• Sender side
• Host can schedule transmission into the network
• Receiver side
• Possible non-QoS worm may block QoS worm
• QoS worm encounters delay if non-QoS worm is
  large
• Solutions
• Two interfaces this double cost of network
• Account for the worst case non-QoS traffic delay
  on single host interface at call setup time

10
Alternative

• Subnets
• difficult to provide delay bounds due to delay
  dynamics from blocking at different cross points
• Alternative
• Impose synchronous structure on top of the
  asynchronous network
• Enables control over the blocking
• Delay bounds and message priorities may be
  implemented
• Trade off
• Network is no longer asynchronous
• Under low traffic load, messages suffer delay due
  to synchronous protocol overhead

11
QoS support via Synchronous framework

• Similar to dedicated traffic channels
• Use timed-token to control traffic streams
• Provides tighter delay bounds and bandwidth
  guarantees
• Target Token Rotation Time TTRT limit the amount
  of transmission
• Average delay TTRT
• Worst case delay 2 TTRT

12
How to support timed token

• Dedicated unidirectional ring is embedded in the
  network

• Attributes of Token scheme
• -fair
• -deadlock free

13
Issues

• Number of host interfaces
• Caused by interaction of QoS non-QoS traffic
• QoS traffic travel on core ring while non-QoS
  travel on other links not on ring
• If host with one interface is busy receiving
  non-QoS message, a QoS message will suffer delay
• QoS message must have preemptive priority
• To increase non-QoS throughput, embedded ring may
  be used if bandwidth is not completely taken by
  QoS traffic

14
Continued

• Scalability
• throughput performance maintained by increasing
  TTRT parameter
• Allows nodes to transmit for longer time when
  they have the token
• Causes less capability to provide tight delay
  bounds

15
Virtual Channel Based QoS

• Each link is split into two different sets of
  virtual channels used for datagram and QoS
  traffic
• Each input port buffer of switch is split into
  several disjoint buffers
• Link between node and input port of switch is a
  collection of virtual channels
• Allows worms to be interleaved
• Give QoS traffic priority in the network

16
Non-preemptive priority

• A worm arriving at QoS virtual channel does not
  get transmitted right away
• Current worm (datagram or QoS) being transmitted
  on outgoing link must either complete or get
  blocked
• Then, scan QoS virtual channels before datagram
  channels to schedule the worm for transmission on
  outgoing link
• Easy to apply preemptive priority by making
  arrived worm preempt datagram worm at the QoS
  virtual channel

17
Implementation

• Preemptive and non-preemptive implementation
  require intelligence switch
• At a switch
• monitor all traffic passing
• Schedule QoS non-QoS traffic according to
  protocol
• Harder to implement preemtive
• Switch must check arrival of QoS traffic at any
  input port before transmitting non-QoS flit from
  output port

18
Advantage of virtual channels

• Network appears the same for both traffic
• Intelligent switches allocate bandwidth as
  required to support QoS
• Can provide delay jitter bounds
• Bandwidth guarantee is provided by employing call
  admission agent

19
conclusions

• Wormhole routing networks provide low latency,
  high bandwidth support for datagram traffic
• To support QoS traffic is a challenge
• Dedicated QoS subnet with pacing and call
  admission control can support QoS
• Synchronous framework on top of asynchronous
  network provides guaranteed bandwidth and delay
• Virtual channels with priority mechanism also is
  effective way to support QoS