Using Prioritized Network Traffic to Achieve EndtoEnd Predictability

1
Using Prioritized Network Traffic to Achieve
End-to-End Predictability

• BBN Technologies OOMWorks LLC
• Cambridge, MA
  Metuchen, NJ
• 
  
• Craig Rodrigues
  Yamuna Krishnamurthy
• 
  Irfan Pyarali
• 
  Pradeep Gore
• Real-Time and Embedded Distributed Object
  Computing
• June 15-18, 2002
• Arlington, Virginia, USA

2
Managed Behavior for Computational Resources

• In order to preserve end-to-end Quality of
  Service (QoS) requirements, different
  technologies must be used to manage the behavior
  of computational resources
• Processor Resources
• RT-CORBA
• Network Resources
• RSVP
• DiffServ

3
Talk Outline

• Real-Time CORBA (RTCORBA) Overview
• RSVP and Diffserv Overviews
• Diffserv enhancements to RTCORBA middleware
• Performance results

4
Real-time CORBA Overview

• RT-CORBA adds QoS control to regular CORBA to
  improve the application predictability
• Bounding priority inversions
• Managing resources end-to-end
• Policies mechanisms for resource
  configuration/control in RT-CORBA include
• Processor Resources
• Thread pools
• Priority models
• Portable priorities
• Communication Resources
• Protocol policies
• Explicit binding
• Memory Resources
• Request buffering
• These capabilities address some (but by no means
  all) important real-time application development
  challenges

5
Preserving Priorities End-to-End

• RT-CORBA preserves end-to-end priorities by
• Mapping importance of activities to corresponding
  OS priorities
• Propagating priorities across the network as
  activity spans multiple hosts

• However , RT-CORBA specification is less explicit
  about
• Communication transport and underlying network
• Unless this behavior is carefully considered and
  modeled
• In-transit activities will be affected by
  network idiosyncrasies
• End-to-end predictability in the system
  difficult to achieve

6
What is RSVP?

• Resource Reservation Protocol, specified in IETF
  RFC 2205
• QoS properties are requested by an out-of-band
  signalling protocol
• Can be used to request a reserved bandwidth
  between a sender and receiver host
• Information about the reservation is stored in
  each intermediate router soft-state

7
Problems with RSVP

• Protocol is very complex to implement
• much complexity due to anticipated use-cases of
  large scale multicast trees multicast not widely
  used
• Storing reservation per-flow reservation state is
  costly
• some work being done into looking reservations
  for aggregate flows
• Microsoft implemented RSVP in Win2K, disabled it
  in WinXP
• equipment vendors have less incentive to
  implement it

8
Problems with RSVP (continued)

• IETF Next Steps in Signaling (NSIS) Working group
• realized that RSVP has it shortcomings, but a
  signaling protocol for QoS is useful
• developing new signaling mechanism (RSVP
  Lite?)
• specification due end of 2002
• Maybe worth looking at in future, but right
  nowindustry seems to be moving away RSVP

9
Solution Differentiated Services

• Use Differentiated Services to prioritize the
  RT-CORBA network traffic
• Differentiated Services (Diffserv) architecture
  provides different types or levels of service for
  network traffic
• Diffserv Code Points (DSCP) are added to data
  packet headers to specify the expected type of
  service
• Diffserv enabled routers and network elements use
  DSCP codepoints to differentiate the network
  traffic
• We are enhancing the existing TAO ORB's real-time
  implementation by making it Diffserv aware

10
What is Diffserv?
IP Datagram Header
Header Length (4-bits)
Version (4-bits)
Diffserv Field (8-bits)
Total Length (16-bits)
Flags (3-bits)
Identification (16-bits)
Fragment offset (13-bits)
TTL (8-bits)
Protocol (8-bits)
Checksum (16-bits)
Source Address (32-bits)
Destination Address (32-bits)

• Diffserv field 6 bits of Diffserv
  Codepoint, 2 bits ECN
• Each DSCP (0-63) specifies a Per-Hop-Behavior
  (PHB), which is a type of router-level QoS
  (RFC 2475)

11
Diffserv QoS Enhancements to the TAO ORB

• Provided an efficient and flexible way of setting
  the DSCP codepoints in the ORB data packets
• Done by extending the RTCORBA protocol properties
• Setting the DSCP codepoint in the message sent
  and the reply received
• Provided a mechanism to map the RT-CORBA
  priorities to the Diff-Serv network priorities

12
RTCORBA IDL Modification

• local interface TCPProtocolProperties
  ProtocolProperties
• attribute long send_buffer_size
• attribute long recv_buffer_size
• attribute boolean keep_alive
• attribute boolean dont_route
• attribute boolean no_delay
• attribute boolean enable_network_priority

13
Initialize Protocol Properties

• //Set the tcp protocol properties
• RTCORBATCPProtocolProperties_var tcp_properties
  
• rt_orb-gtcreate_tcp_protocol_properties
  (send_buffer_size,
• recv_buffer_size,
• 1, // keep_alive
• 0, // dont_route
• 1, // no_delay
• enable_network_priority)
  

14
Client Side Initializing the Policy List

• RTCORBAProtocolList protocols
• protocols.length (1)
• protocols0.protocol_type 0
• protocols0.transport_protocol_properties
• RTCORBAProtocolProperties_duplicate
  (tcp_properties.in ())
• protocols0.orb_protocol_properties
• RTCORBAProtocolProperties_nil ()
• poa_policy_list1
• rt_orb-gtcreate_client_protocol_policy
  (protocols )

15
Client Side - Change Transport Protocol Policy

• ORB Level
• object orb-gtresolve_initial_references
  ("ORBPolicyManager")
• CORBAPolicyManager_var policy_manager
• CORBAPolicyManager_narrow (object.in ()
  )
• policy_manager-gtset_policy_overrides
  (policy_list,
  
  CORBASET_OVERRIDE )

16
Client Side - Change Transport Protocol Policy

• Thread Level
• orb-gtorb_core () gtpolicy_current().set_policy_over
  rides
• (policy_list,
• CORBASET_OVERRIDE)
  
• Object Level
• CORBAObject_var object server-gt_set_policy_ov
  errides (policy_list,
• 
  CORBASET_OVERRIDE)
• server Test_narrow (object.in () )

17
Server Side - Initializing the Policy List

• RTCORBAProtocolList protocols
• protocols.length (1)
• protocols0.protocol_type 0
• protocols0.transport_protocol_properties
• RTCORBAProtocolProperties_duplicate
  (tcp_properties.in ())
• protocols0.orb_protocol_properties
• RTCORBAProtocolProperties_nil ()
• poa_policy_list1
• rt_orb-gtcreate_server_protocol_policy
  (protocols )

18
Server Side Set Policy on the POA

• //Create POA with Diffserv enabled
• PortableServerPOA_var poa_with_ds
• root_poa-gtcreate_POA ("POA_WITH_DS",
• 
  poa_manager.in (),
• 
  poa_policy_list)

19
RTCORBA to Network Priority Mapping

• Added a pluggable RTCORBA to Network Priority
  Mapping framework
• Similar to the RTCORBA to Thread Priority Mapping
  framework
• RTCORBA IDL modification
• typedef short NetworkPriority
• native NetworkPriorityMapping
• The framework allows the user to implement their
  own mapping algorithms and plug it in using the
  ACE Service Configurator

20
RTCORBA/Diffserv Testbed
192.168.1.
192.168.10.
Linux
Linux
Linux
Diffserv
Diffserv
T1
IIOP
IIOP
T2
Traffic
Congestion
Linux
Priority of T1 Priority of T2
21
Performance Results

• No network traffic congestion
• With network traffic congestion
• With network traffic congestion and DSCP set on
  Thread 1

22
Performance Results

• With congestion traffic and DSCP set on T1

23
Potential Use Case for RT-CORBA/Diffserv
Predator Squadron Operations Center ADSI with
JCPI PowerScene Equipped
Source UAV Battlelab, Eglin Air Force Base U.S.
Air Force
24
Summary

• End-to-end QoS requires resource management of
  CPU and network resources
• RT-CORBA today primarily manages CPU resources
• Our enhancement allows RT-CORBA middleware to
  manage network resourcesusing Diffserv, which is
  priority based

25
Obtaining Software

• http//deuce.doc.wustl.edu/Download.html
• Contacts
• Craig Rodrigues, crodrigu_at_bbn.com,
  1-617-873-4725
• Yamuna Krishnamurthy, yamuna_at_oomworks.com,
  1-732-205-0346

26
Further References

• Protocol Selection and Explicit
  Binding,Schmidt Vinoski http//www.cuj.com/ex
  perts/2005/vinoski.htm
• An Architectural for Differentiated Services,
  RFC 2475 http//www.ietf.org/rfc/rfc2475.txt