Managing SPP Resources: System Resource Manager SRM

1
Managing SPP ResourcesSystem Resource Manager
(SRM)

• Fred Kuhns
• fredk_at_arl.wustl.edu
• Applied Research Laboratory
• Washington University in St. Louis

2
Common Definitions and Types

• For all commands the message context ID is used
  to identify the context within which a command is
  to be executed.
• The special value of 0 (cid 0) indicates a
  privileged operation performed by the substrate.
• Otherwise the context ID is an identifier
  indicating a user specific context in which the
  command is to be interpreted and executing. In
  most cases this is the Global Fastpath ID.
• See presentation titled Types.ppt

3
NATD Interface

• egress_map, ingress_map get_sched_map(ipAddr_t
  link, hwaddr_t board)
• Message context ID 0.
• sched_map_t
• qmID_t qmID // 2-bit QM identifier
• schID_t schID // 3-bit qm relative scheduler
  identifier
• qid_t qid // 16-bit queue identifier
• statID_t sindx // Stats Index
• vlanID_t vid // VLAN identifier
• Return
• egress_map sched_map_t
• ingress_map sched_map_t
• Arguments
• link Interface IP address (aka external IP)
• board Board/Device MAC Address

4
Reserving Fastpath Resources

• See Types.ppt for type definition and mapping to
  substrate API, message context ID 0.
• Reserve resources for 0..N fastpaths (on NPEs)
  and 0..1 GPEs
• retCode_t make_reservation(plabID, rsvRecord_t)
• Update an existing reservation, note because all
  operations need to be idempotent the update
  argument includes all required resources.
• retCode_t update_reservation(plabID_t,
  rsvRecord_t)
• Cancel any reservation overlapping date_t
• retCode_t cancel_reservation(plabID_t, date_t)
• Return any reservation (belonging to slice) that
  overlaps with date_t
• rsvRecord_t? get_reservation(plabID_t, date_t)
• Return a list of all reservations associated with
  calling slice
• rsvRecord_t get_reservations(plabID_t)

5
Allocating Fast-path Instance

• fastpath_t alloc_fastpath(plabID_t, copt, bwspec,
  rcnts, mem)
• Message context ID 0
• plabID_t GPE/PlanetLab slice identifier
• uint16_t copt code option type. IPv41,I32
• struct bwspec bw_t bw-firm, bw-soft
• struct rcnts cnt_t fltrs, queues, buffers,
  stats
• struct mem cnt_t sram, dram // Requested
  size of SRAM and DRAM in Bytes
• struct fastpath_t
• fpid_t fpid // fastpath id assigned by srm,
  used as context ID in future messages vlan_t
  vlan // VLAN id to associate with fps logical
  interface ipAddr_t fpIP // IP address of
  logical fps interface on GPE ipAddr_t npeIP //
  IP address for fps data interface on
  NPE ipAddr_t fpNM // Netmask of fps logical
  interface ipAddr_t scdIP // IP address for fps
  assigned SCD ipPort_t scdPort // UDP Port
  number for fps assigned SCD hwaddr_t
  npeMAC // NPE MAC address
• bw-firm Expresses the required aggregate
  processing resources in terms of BW, units are
  Kbps.
• bw-soft This is just a hint as to the expected
  maximum processing resources required. Resources
  are not actually allocated to the fastpath,
  rather the system uses this number for load
  balancing best-effort fastpaths across the
  available NPEs..
• Actions required to allocate code option instance
  and resources
• Select NPE
• Load balance across available NPEs
• Must have sufficient resources (SRAM, TCAM space,
  queues, etc)
• Either allocates requested resources or returns
  error
• Keeps memory map of SRAM so can perform
  allocation, though the absolute starting address
  is not required.
• If compaction is necessary then must communicate
  with SCD directly.
• Allocate VLAN and configure switch.

Deprecated
6
Releasing a fast-path instance

• retCode_t free_fastpath()
• Message context ID Global fastpath ID (fpid)
• Send command to SCD on NPE
• set_sched_params(sid(gpe), GPE_MIID, GPE_BW_MAX,
  GPE_BW_MIN)
• retCode_t set_fpmi_bw(fpid, sid(gpe), GPE_MIID,
  0)
• set_encap_gpe(fpid, 0, 0)
• rem_fastpath(fpid)

Deprecated
7
Instantiate Fastpath and/or Slowpath reservations

• All resource allocation methods operate on the
  currently active reservation (i.e. must be used
  during the interval of time when a reservation is
  valid).
• If free is not called then all resources are
  automatically freed when reservation expires.
• allocate any GPE/Slice resources reserved in the
  current interval
• retCode_t alloc_plspec(plabID_t)
• instantiate the current fastpath reservation
  with name fpName
• fpInfo_t alloc_fpspec(plabID_t, name_t fpName)
• if only one fastpath then the fpName parameter
  can be omitted.
• free any GPE specific resources currently
  allocated to slice.
• retCode_t free_plspec(plabID_t)
• Destroys fastpath instance, releasing
  meta-interfaces and NPE resources.
• retCode_t free_fpspec()
• Message context ID set to the Global fastpath ID

8
Managing Interfaces

• Reserve Aggregate Interface BW for slice (traffic
  to/from GPE).
• retcode resrv_slice_ifbw(plabID, bw, ifn)
• Release slice interface BW reservation for GPE
• retcode reles_slice_ifbw(plabID, bw, ifn)
• Reserve aggregate Interface BW for indicated
  fastpath
• retcode resrv_fpath_ifbw(bw, ifn)
• message context ID fpid
• Release Aggregate Interface BW allocation for
  indicated fastpath
• retcode reles_fpath_ifbw(bw, ifn)
• message context ID fpid

Deprecated
9
System Interfaces, Addresses and Ports

• Get list of available interfaces
• if_list get_interfaces(plabID)
• Message context ID 0.
• if_list ifAttr_t, ...
• ifAttr_t
• ifn_t ifn // Interface number
• ifType_t type // Multi-Access 0,
  Point-to-Point 1
• ipAddr_t ipaddr // Interface/Link IP address
• ipAddr_t linkBW // Raw (Gross) interface
  (link) BW
• bw_t availBW // Available (Net) interface
  (link) BW
• if type point-to-point then may call
  get_ifpeer()
• Map interface IP address to interface number.
• ifn_t get_ifn(plabID_t, ipAddr_t)
• Get updated interface attributes
• ifAttr_t get_ifattrs(plabID_t, ifn_t)

10
Managing Externally Visible Address for GPE
Traffic

• Message context ID set to 0, plabID used to
  identify Slice making request
• Managing bandwidth
• SRM verifies slice meta-interfaces and queues do
  not exceed interface allocations
• SRM Notifies SCD of per Scheduler slice limits
• Allocate a local endpoint for slice (linecard
  directs matching traffic to the corresponding
  GPE)
• epInfo_t alloc_endpoint(plabID_t, epInfo_t)
• Message context ID 0.
• Release allocated endpoint (linecard filters
  removed and address added bac kto available pool
  of addresses tuples).
• retCode_t free_endpoint(plabID_t, epoint_t)
• Message context ID 0

11
Internal Interfaces for NAT

• mapbw_t bw Kbps
• ipAddr_t ipaddr local interface IP address
• ipPort_t port if TCP/UDP and 0 then allocate
• ipProto_t proto TCP/UDP,ICMP
• dw1_t smac low order 8 bits of source mac
• fid, port alloc_epmap(map)
• Message context ID 0.
• Allocate endpoint and add LC TCAM filter to send
  pkts to GPE.
• retCode_t free_epmap(fid)
• Message context ID 0.
• Allocate endpoint and add LC TCAM filter to send
  pkts to GPE.

Deprecated
12
Allocating Meta-Interfaces (tunnel endpoint)

• Common Types
• ep uint32_t bw, uint32_t ipaddr uint16_t port
  uint8_t proto
• dw2_t mi Meta-interface number for an endpoint.
  Interface numbers start at 0.
• mi, ep alloc_tunnel(bw, ipaddr, tunnel_type,
  params) Not implemented
• mi, ep alloc_udp_tunnel(bw, ipaddr, port)
• Message context Id fpid
• uint8_t tunnel_type must be one of UDP 0
• uint32_t bw (Minimum guaranteed) BW assigned to
  endpoint, Kbps
• uint32_t ipaddr External IP address of
  endpoint/tunnel. There is a 1-to-1 relationship
  between links and ip addresses.
• uint16_t port External UDP port number
• Actions required to allocate code option instance
  and resources
• verify requested address is available, assign
  port if necessary (port 0)
• Allocate new meta-interface ID
• allocate and assign filter id for ingress and
  egress TCAM
• Lookup existing or allocate qid for ingress
  and/or egress
• Send command to SCD on Line Card Ingress
• set_sched_params(sid(fabric,NPE), ifn(fabric),
  BW, BWmin)
• set_queue_sched(qid, sid(fabric,NPE))
• set_queue_params(qid, threshold, bw)

Deprecated
13
Releasing Meta-Interfaces

• Common Types
• ep uint32_t bw, uint32_t ipaddr uint16_t port
  uint8_t proto
• uint_t mi Meta-interface number for an endpoint.
  Interface numbers start at 0.
• retCode_t free_tunnel(ipaddr, tunnel_type,
  params)retCode_t free_udp_tunnel(ipaddr, port)
• Message context Id fpid.
• uint32_t ipaddr External IP address of
  endpoint/tunnel
• uint16_t port External UDP port number
• Actions required to allocate code option instance
  and resources
• Send command to SCD on Line Card Ingress
• set_sched_params(sid(fabric,NPE), ifn(fabric),
  BW, BWmin)
• set_queue_params(qid, threshold, bw)
• rem_fltr_byfid(dbid_ingress, fid)
• rem_fltr.fid(dbid_egress)
• Send commands to Line Card Egress
• set_sched_params(sid(link), ifn(link), BW,
  BWmin)
• set_queue_params(qid, threshold, bw)
• Send command to SCD on NPE
• set_sched_params(sid(link, miid), ifn(fabric),
  BW, BWmin)
• set_fpmi_bw(fpid, sid(link, miid), miid, bw)

Deprecated
14
Allocating Meta-Interfaces (tunnel endpoint)

• --- Simplified interface, see steps from
  deprecated slides ---
• Allocate local Address for Slices fastpath (MI)
  and forward to NPE, will also reserve sufficient
  aggregate BW on corresponding internal and
  external interfaces.
• miID_t, epInfo_t alloc_udp_tunnel(bw_t,
  ipAddr_t, ipPort_t)
• Message context ID set to the global fastpath ID
• It is an error if the sum of requested BW exceeds
  the reserved fastpath BW.
• Adds one end of a UDP tunnel, treated as a
  meta-interface for a fast path instance.
• if port 0 then one is allocated from the free
  pool
• Returns the meta-interface number representing
  the new endpoint
• Release BW and Address associated with
  corresponding meta-interface
• retCode_t free_udp_tunnel(ipAddr_t, ipPort_t)
• Map meta-interface ID to corresponding externally
  visible address tuple
• epInfo_t get_endpoint(miID_t)
• mi must be a valid meta-interface ID returned
  from a call to alloc_udp_tunnel

15
Classes and Tables
xsid
Slice Table
plabID
Mappings plabID to Entry xsid to Enry
Entry plabID, xsid, gpe, npe, MI
GPE GPE object reference, BW, EPs, ??
NPE NPE object reference, vlan bw, sram, fltrs,
qs, stats
MI EP object reference, ID