Stream Processing in PNEs

1
Stream Processing in PNEs

• George Porter
• Edge Services Session
• Winter Retreat - 2004

2
Overview

• Streams prevalent in the edge network
• PNEsgeneral platform for packet processing
  (roughly layers 2-4) in the edge
• We need a general mechanism for stream processing
  (roughly layer 7) that is integrated with the
  current PNE design and architecture
• I am proposing an extension to PNEs to specify
  streams and a mechanism for executing that
  specification at high speeds in a general way
  from the PNE interface
• This mechanism provides a clear separation
  between the transport of packets and the
  structure of the protocol

3
Packet-vs.-stream processing whats the
difference?

• Packet Processing
• Ethernet, IP, and TCP headers in every packet
• Decisions are stateless between packets
• Resilient to loss, reorder, duplication
• Stream Processing
• Assumes reliable transport
• Sequence of variable length data units over TCP
• Simple for endhosts, since O/S delivers in order
• Loss, reorder, dups a problem for in-network
  elements
• PNEs must track data unit boundaries

Ethernet

IP
TCP
Ethernet
Ethernet
IP
IP
TCP
TCP
iSCSI1
Data1
iSCSI2
Data
Data1
Data2
4
Examples of observed phenomenon

• (observed on OASIS testbed)

header1
data1
header2
data2
Original Stream
1)
2)
3)
(overlaps with first And last part of header)
5
Need for stream processing

• Necessary to track any layer-7 protocol that
  exists on top of TCP
• Storage
• iSCSI storage virtualization, load-balancing,
  security, caching, in-network optimizations
• Web/P2P
• HTTP pipelining load-balancing
• P2P HTTP often used as a transport protocol
• Measurement/Monitoring
• Tracking state of observed protocols over time

6
PNE mechanism for tracking ADUs
StreamTracker
Packet flow (fast path)
Packet(s) containing ADU boundary
Priority Queue
Computed ADU size
FixedLogic
(iscsi)
Paramaterized by specification
7
PNE mechanism for tracking ADUs
StreamTracker
Packet flow (fast path)
Packet(s) containing ADU boundary
Priority Queue
Computed ADU size
VariableLogic
(http)
Paramaterized by specification
8
Evaluation Platform

• Deployed onto testbed consisting of
• 20 Pentium-500 Linux machines
• 100 Mbit ethernet-based network
• iSCSI source and target through Intel drivers
• 100MB ramdisk on target / 30gig IDE Seagate drive
• Code deployment
• Baseline Click modular router running in Kernel
• Tracking Click router in kernel with custom
  elements

9
Results pkts examined

• Only a small number of packets are examined in
  detail

Workload Packets examined Total Packets Percent
Unpack and compile Apache 2272 42850 5.30
Resize two 80Kb images 15 266 5.64
Store/Read 20Mb MPEG file 595 37207 1.60
FS caching does not effect above measurements
10
Results overhead of packet examination

• Overhead of examining packets in depth is large,
  but infrequent

Workload Stock Click CPU cycles/pkt Tracking CPU Cycles/pkt Percent Increase
iSCSI regression Test 1430 1871 30.8
iSCSI regression Test (1 loss) 1328 1937 45.9
Unpack and compile Apache 1591 2104 32.2
11
Next Steps

• Implement in PNE software artifact
• Write compiler for specification language
• Deployment
• (see poster for deployment overview)
• VideoCollective, VideoCollective with
  sharing, VideoCollective with collaborative
  sharing
• Measurement of storage statistics in above
  deployment