C.%20Edward%20Chow%20Department%20of%20Computer%20Science%20University%20of%20Colorado%20at%20Colorado%20Springs

1

• C. Edward ChowDepartment of Computer
  ScienceUniversity of Colorado at Colorado
  Springs
• Sponsored by Computer Comm. Lab/ITRI

2
Content Switch Topics

• What is a Content Switch?
• What Services it Can Provide
• Content Switch Example
• Related Technologies
• Content Switch Architecture and Basic Operations
• TCP Delay Binding and Related Improvement
• Content Switch Rule and Conflict Detection
• Conclusion

3
Content Switch (CS)

• Route packets based on high layer (Layer 5/7)
  headers and content.
• Examples
• Direct Web traffic based on pattern of
• URLs, cookies URL Switching
• XML Tag Value Web Switching
• Can Route incoming email based on email
  addressConnect POP/IMAP based on login
• Web switches and Intel XML Director/accelerator
  are special cases of content switch.

4
What Services It Can Provide

• Enabling premium services for e-commerce, ISP,
  and Web hosting providers
• Load Balancing and High Available Server
  Clusters Web, E-commerce, Email, Computing,
  File, SAN
• Policy-based networking, differential/QoS
  services.
• Firewall, Strengthening DoS protection,
  cache/firewall load-balancing
• Flash-crowd' management
• Email Spam Protection, Virus Detection/Removal
• Applet Authentication/Filtering

5
F5 VRM Solution
6
Intel Netstructure XML Director 7280

• Example of RuleServer1 create /order.asp
  //AmountValue gt 10000

7
Phobos In-Switch

• Only load balancing switch in a PCI card form
  factor
• Plugs directly into any server PCI slot
• Supports up to 8,192 servers, ensuring
  availability and maximum performance
• Six different algorithms are available for
  optimum performance Round Robin, Weighted
  Percentage, Least Connections, Fastest Response
  Time, Adaptive and Fixed.
• Provides failover to other servers for
  high-availability of the web site
• U.S. Retail 1995.00

8
E-Commerce Example 1. Client

• Client submits via HTTP/Post (or SOAP) the
  following purchase in XML
• ltpurchasegt
• ltcustomerNamegtCCLlt/customerNamegt
• ltcustomerIDgt111222333lt/customerIDgt
• ltitemgtltproductIDgt309121544lt/productIDgt
• ltproductNamegtIBM Thinkpad T21lt/productNamegt
• ltunitPricegt5000lt/unitPricegt
• ltnoOfUnitsgt10lt/noOfUnitsgt
• ltsubTotalgt50000lt/subTotalgt
• lt/itemgt
• ltitemgtltproductIDgt309121538lt/productIDgt
• ltproductNamegtIntel wireless LAN PC
  Cardlt/productNamegt
• ltunitPricegt200lt/unitPricegt
• ltnoOfUnitsgt10lt/noOfUnitsgt
• ltsubTotalgt2000lt/subTotalgt
• lt/itemgt
• lttotalAmountgt52000lt/totalAmountgt
• lt/purchasegt

9
E-Commerce Example 2. Content Switch

• Content switch receives the packet.
• Recognize it is a http post request from http
  request line POST /purchase.cgi HTTP/1.1
• Recognize it is an XML document from the meta
  headercontent-type TEXT/XML
• Parsing XML content
• Extract values of tag sequences
  52000 purchase/totalAmount
  CCL
  purchase/customerName
• Rule 1 is matched and packet is routed to one of
  highSpeedServers.Rule 1 if (xml.purchase/totalAm
  ount gt 5000) routeTo(highSpeedServers)Rule 2
  if (xml.purchase/customerName CCL)
  routeTo(specialCustomerServers)

10
No Free LunchPenalty of Having Content Switch

• ? Increased packet processing time.
• For XML Director/Accelerator, it needs to parse
  XML document and match tag sequences.? 1-3?
  order of processing time

11
Related Technologies

• Application level solution Proxy server
  Apache/Tomcat/Servlet Microsoft NLB
• Kernel level layer 4 load balancing solution
  http//www.linuxvirtualserver.org/
• Joseph Marks presentation
• LVS-NAT(Network Address Translation) web page
• LVS-IP Tunnel web page
• LVS-DR (Direct Routing) web page
• Hardware solution Cisco 11000, F5 (Big IP),
  Alteon Web Systems, Foundry Networks
  (ServerIron),Excellent information on Foundry
  ServerIron Installation and Configuration Guide,
  May 2000.
• Routing table lookup Longest prefix
  (Gupta/McKeown)

12
Basic Operations of Content Switching
CS Content Switching
CS RuleEditor
CS Rules
Incoming Packets
Packet Classification
Header ContentExtraction
CS Rule Matching Algorithm
Forward Packet To Servers
Packet Routing(Load Balancing)
Network Path Info
Server Load Status
13
Content Switch Architecture

• Apostolopoulos Infocom 2000

14
Content Switch Architecture
Case A Controller finds there is an entry in
its Hash Table, Route request to sticky
connection outgoing port
Hash Table
15
Content Switch Architecture
Case B Step 1. Controller finds there is no
entry in Hash Table, Route request to content
switch processor
Hash Table
16
Content Switch Architecture
Step2. CS processora. Extract content/Match CS
rulesb.Route requestc. Setup Sequence
modification on server side port
Case B Step 1. Controller finds there is no
entry in Hash Table, Route request to content
switch processor
Hash Table
17
Content Switch Architecture
Step2. CS processora. Extract content/Match CS
rulesb.Route requestc. Setup Sequence
modification on server side port
Case B Step 1. Controller finds there is no
entry in Hash Table, Route request to content
switch processor
Step 3. At server side port, Return pkts are
modified Sequence/IP addr/ChksumRoute back to
client
Hash Table
18
Efficient Software Architecture

• Tasks Million packets with thousand of rules to
  match and load balancing algorithms to run.
• How to assign tasks to the (network) processors
  and threads?
• Packet Extraction (Understand header formats,
  XML parsing)
• Content Switching Rule Matching
• Packet Routing (Load Balancing, Bandwidth
  Control)
• How Much Packet Processing Should Controllers
  Do?
• What a controller can do?
• A Typical Parallel Processing Problem?

19
TCP Delay Binding (Splicing)
client

server

content switch


SYN(CSEQ)
step1


step2

SYN(DSEQ)


ACK(CSEQ1)
step4


SYN(CSEQ)

step5

SYN(SSEQ)

step6


ACK(CSEQ1)

step7

ACK(SSEQ1)


step8
DATA(CSEQ1)

ACK(SSEQ1)

DATA(SSEQ1)
DATA(DSEQ1)
step9



ACK(CSEQlenR1)
ACK(CSEQLenR1)
step10


ACK(DSEQ
lenD1)

ACK(SSEQlenD1)

step11
lenR size of http request.

.

lenD size of return document
20
Improve Content Switching

• Setup CS-Real Server connections ahead of time
  (Persistent HTTP Connections). NetScale? Reduce
  TCP 3-way handshake time
• Pre-allocate Server Scheme (Guess Real Server
  based on the TCP Sync)
• Sequence modification on every return pkt ? Need
  to recompute checksum also.
• Filter Scheme (Offload Sequence
  modification/rule matching to real servers).
• Buffering/Pipeline (aggregate) Requests

21
Pre-Allocate Server Scheme
Pre-allocated server
client


content switch


SYN(CSEQ)
SYN(CSEQ)

step1

SYN(SSEQ)
SYN(SSEQ)
step2




ACK(CSEQ1)

ACK(CSEQ1)

step4
DATA(CSEQ1)
DATA(CSEQ1)

• Guess routing decision based on IP/Port/History
• Advantage
• Faster than TCP delay binding.
• Possible direct route between client and server
• Reduce session processing overhead no need to
  convert server sequence

.
22
Degenerated to TCP Delayed Binding If Guess is
Wrong
Pre-allocated server
client


content switch


SYN(CSEQ)
SYN(CSEQ)
step1


SYN(SSEQ)/ ACK(CSEQ1)
step2
SYN(SSEQ)/ ACK(CSEQ1)








step4
DATA(CSEQ1)/ACK(SSEQ1)
DATA(CSEQ1)/ ACK(SSEQ1)

Server sent HTTP 404
FIN(CSEQlenR1))
step6
Right server
step7
SYN(CSEQ)


SYN(RSEQ)/ ACK(CSEQ1)

step8


Sequence conversion needed for right server now
ACK(RSEQ1)

step9



step10

DATA(SSEQ1)/ACK(CSEQLenR1)
DATA(RSEQ1)/ACK(CSEQlenR1)
step11

step12
ACK(SSEQlenD1
ACK(RSEQlenD1)
23
Filter Process Scheme
Filter Processrun on server
client

server

content switch


SYN(CSEQ)
step1





step3


DATA(CSEQ1)/ACK(DSEQ1)
step4
step5b
SYN(CSEQ)
Migrate(Data, CSEQ, DSEQ)

step5
a

SYN(SSEQ)/ ACK(CSEQ1)

step6


step7

DATA(CSEQ1)/ACK(SSEQ1)

step8






ACK(DSEQ
lenD1)

ACK(SSEQlenD1)

step10
24
Pre-allocate performance plot
Series 1 - Basic scheme with no rule matching
module inserted, i.e., using default
IPVS.Series 2 - Basic scheme with the rule
matching module inserted.Series 3 -
Pre-allocate scheme with all hits, i.e., where
all pre-allocate guesses were correct.Series 4
- Pre-allocate scheme with all misses, i.e.,
where all pre-allocate guesses were wrong.
25
Handling multiple requestsin a Keep-Alive
connection

• Determine when new request arrives
• Verify that previous request has been completely
  received
• Request data size is gt 0
• Key assumption is only one outstanding request is
  sent at a time by client, i.e., requests are not
  pipelined
• Reuse connections
• Store each connection control information in a
  hash table keyed by real server address, once it
  is established.

26
Quiz

• Web server keeps the TCP connection alive,
  expecting the browser to return for images and
  in-line media files.
• How many keep-alive connections are setup on IE5
  and Netscape 4.7 for web page with many .jpg/.gif
  images?
• Can these image requests be pipelined from client
  browser to web server?

27
Multiple HTTP Requests from One TCP Connection
NAT approach
server1
uccs.gif
ContentSwitch
server2
client
. .
cs.jpg
Index.htm
.
rocky.mid
server9

• A keep alive TCP connection may include multiple
  HTTP GET requests.
• Content Switch examines each GET request and
  makes new routing decision.
• Content Switch establishes another connection
  with a different server based on the
  routing decision.
• Those HTTP responses from different servers need
  to be interleaved and seen by the user as if
  from the same server.
• Solutions In order delivery (buffer
  requirement) Out of order delivery (seq
  tracking)?
• Problems Should we throw away earlier html
  requests if receive later requests?

28
Multiple HTTP Requests from One TCP Connection
server1
uccs.jpg
ContentSwitch
server2
client
. .
.
server9
rocky.mid
cs.gif

• Can servers return documents directly to client
  in keep-alive session case?
• Can equivalent VS-Tunnel or VS-DR be implemented
  using Content Switch?

29
Content Switch Rule Survey

• Survey shows that existing switches support
• rules in basic (condition action) or (action
  condition) form
• some define condition as class, then specify the
  action in separate statement or command
• simple single conditional term
• command line interface (to facilitate incremental
  update?)
• Actions can include reject, forward, put in queue
  (for bandwidth control, scheduling)

30
Content Switch Rule Design

• Rule syntax generic to support all Intended
  features.
• Use simple C if statement syntax rule if
  (condition) action
• Easy to read
• Allow optimization using c compiler
• Condition consists of multiple terms of
• variable relational_operator value e.g.
  xml.purchase/totalAmount gt 50000 smtp.to
  chow_at_cs.uccs.edu cookie.name
  servlet1 bitmatch(64, 8, 0xff) 64
  above mean TTL64 idea from netfilter
  universal filter
• suffix(variable, string) e.g. suffix(url,
  gif)
• regex(variable, pattern) e.g. regex(url,
  /purchase)
• Action consists of reject, forward(server
  queue)loadBalance(serverGroup,
  loadBalancingAlgorihtm)

31
Efficient CS Rule Matching

• Brute force, strict priority Rules are executed
  in sequential manner.
• Efficient Rule Matching Method
• Organize Rules so that rules can be skipped
  based on existing content types.
• Utilize compiler optimization technique.

32
Simple CS Rule Editor GUI
33
Conflict Detection on Content Switching Rules

• Detect conflicts among rules or rule set.
• Absolute conflict type r1 if
  (xml.purchase/customerName CCL)
  routeTo(r1)r2 if (xml.purchase/customerName
  CCL) routeTo(r2)
• Potential conflict type r1 if
  (xml.purchase/totalAmount gt 5000)
  routeTo(quickServers)r2 if (xml.purchase/total
  Amount gt20000) routeTo(superServers)
• Algorithm Build tree with the same variable,
  check operator and value to see if they are the
  same or lead to potential conflict, compare
  actions to decide conflict type or duplication.
• Developed conflict detection algorithm for rules
  with multiple term condition. Can be applied to
  policy-based rules conflict detection.
• Editor can build these trees while a user enters
  rules and warns about conflict right away.

34
XML Tag Value Extraction

• A xmlContentExtract() is built to extract the tag
  values of a list of unique tag sequences.
• It is based on clark coopers expat 1.0
  xmlparser.
• Its argument include the pointer to an XML
  document, the pointer to the array of strings
  (unique xml tag squences we follow the xsl
  selector syntax), and the number of sequences.
• It return the list of a structure node, with the
  tag sequence, its attribute, and its value.
• Currently, it supports one attribute and tag
  sequece needs to be unique.

35
Status of UCCS ACSD Project

• A Linux-based LVS content switch called LCS was
  developed
• Sponsored by CCL/ITRI.
• Based on Linux-2.2.16-3, current release LCS02.
• ip_forward.c, ip_masq.c, ip_vs.c are modified to
  implement basic TCP delay binding.
• ip_cs.c are added for most of the content
  switching functions with http header extraction
  and xml content extraction.
• A simple Java-based ruleEdit program was created
  for rule editing and conflict detection.
• Rule translate program to convert the rule set
  into a Linux kernel module and allow dynamic
  replacement of rule without restarting the
  system.
• LCS is being ported to Intel IXP 1200 network
  processor.

36
LCS Demo

• We set up viva.uccs.edu as a content switch and
  wait and ace as two real servers.
• URL Switching demohttp//viva.uccs.edu/lcs1/
  route to ace.uccs.eduhttp//viva.uccs.edu/lcs2/
  route to wait.uccs.edu
• XML Web Switching (E-commerce applications)http/
  /archie.uccs.edu/acsd/lcs/xmldemo.htmlWhen the
  2nd subtotal tag gt50000, route to ace.When the
  2nd subtotal tag lt50000, route to wait.
• Let us know if you have problem accessing
  them.My students may be working on LCS extension.

37
LCS Rule Example

• R4 if (atoi(rule_fields1.value) gt 50000)
• return route_to("ace", NON_STICKY,
  saddr)
• R5 if ((atoi(rule_fields1.value) gt 0)
• (atoi(rule_fields1.value) lt
  50000))
• IP_RULE_MSG("serevrwait\n")
• return route_to("wait", NON_STICKY,
  saddr)
• R10 if (strstr(url, "lcs1") ! NULL)
• IP_RULE_MSG("serverace\n")
• return route_to("ace", NON_STICKY,
  saddr)
• R11 if(strstr(url, "lcs2") ! NULL)
• IP_RULE_MSG("serverwait\n")
• return route_to("wait", NON_STICKY,
  saddr)

38
Related Load Balancing Research Results

• Modified Apache status module to report
• Total bytes to be transferred by child processes
• Average document transfer speed
• Modified LB-DNS to receive server status and
  bandwidth probing results.
• LB-DNS returns IP-address of the best server
  based a weight contributed by both server load
  and bandwidth.
• Modified WebStone benchmark to test the
  performance of load balancing web server clusters.

39
Load balancing Systems
Bandwidth Probe Results
Modified Web Server 1
Statistics Gathering Daemon
Server Delay
Server Ranking /tmp/StatFile
Modified Web Server n
LBA Modified DNS
Request for Web pages
40
Connection Rate LBA vs. Round-Robin
Round robin only run once
41
Conclusion

• Content Delivery Network improves internet
  content retrieval
• LVS provides a low cost layer 4 switching service
  for cluster.
• Linux Content Switch with generic rules can be
  easily configured for wide-variety of value-added
  services
• Premium services
• Load balancing/High Available server farm.
• Firewall
• Bandwidth control/Traffic shaping
• Require efficient SW/HW architecture and rule
  matching algorithms to reduce processing
  overhead.
• Content rule design/conflict detection are
  important and challenging.
• TCP delay binding can be improved.

42
References

• http//www.linuxvirtualserver.org/
• http//www.akamai.com/
• http//cs.uccs.edu/chow/pub/contentsw/talk/conten
  tswitching.ppt
• Aron2000 Aron, Mohit, Differential and
  predictable QoS in web server systems, Ph.D
  dissertation Rice University, Oct. 2000.
• Zhang97 Lixia Zhang, Sally Floyd, and Van
  Jacobson, Adaptive Web Caching, April 25, 1997.
  http//www-nrg.ee.lbl.gov/floyd/web.html
• Esi2001 Edge Side Includes, http//www.esi.org/.
  
• Chow2001a C. Edward Chow and Indira Semwal,
  Web Load Balancing Through More Accurate Server
  Report, Proceeding of PDCAT 2001, Taipei,
  Taiwan.
• Chow2001b C. Edward Chow, Ganesh Godavari, and
  Jianhua Xie, Content Switch Rules and their
  Conflict Detection, Proceeding of PDCAT 2001,
  Taipei, Taiwan.
• Chow2001c C. Edward Chow and Weihong Wang, The
  Design and Implementation of Linux LVS-based
  Content Switch, Proceeding of PDCAT 2001,
  Taipei, Taiwan.
• Aversa2000 Luis Aversa and Azer Bestavros,
  Load Balancing a Cluster of Web Servers Using
  Distributed Packet Rewriting, Proceedings of
  IPCCC 2000. 
• Cao98 PeiCao, Jin Zhang and Kevin Beach,
  Active Cache Caching Dynamic Contents on the
  Web http//www.cs.wisc.edu/cao/papers/active-cac
  he.ps