The State of the Art in Locally Distributed Webserver Systems

1
The State of the Art in Locally Distributed
Web-server Systems

• V. Cardellini, E. Casalicchio, M. Colajanni, P.S.
  Yu
• IBM Research Report

2
Contents

• Introduction
• Request routing mechanisms for cluster-based Web
  systems
• Request routing mechanisms for distributed Web
  systems
• Dispatching algorithms for cluster-based Web
  systems
• Integrated dispatching algorithms
• Placement of Web content and services
• Summary
• Research perspectives

3
Introduction (1/3)

• Needs for scalable Web server systems
• Web site is the only component to be controlled
  by a single organization
• In interne, other components are not controllable
  by a single organization
• The complexity of Web applications and services
  is growing
• Web servers contribute for 40 of the delay in a
  Web transaction
• As network speed improves faster than CPU, server
  will be bottleneck in Web
• Approaches for scalable Web server systems
• Scale-up
• Hardware scale-up more H/W resources
• Software scale-up improving Web server and O/S
  performance
• Scale-out
• Local scale-out server nodes reside at a single
  network location
• Global scale-out server nodes are located at
  different geographical locations

4
Introduction (2/3)

• Basic architecture of locally distributed Web
  systems
• Appear as a single host (transparency)
• Request routing mechanisms
• At the client / network / DNS / server level

5
Introduction (3/3)

• Classification of locally distributed
  architectures
• Cluster-based Web systems the server nodes are
  only visible by virtual IP
• Also known as Web cluster or Web farm
• Web switch (dispatcher) distributes the requests
  to the servers in LAN
• Distributed Web systems IP address of the server
  nodes are visible
• One-to-many mapping in authoritative DNS server

Cluster-based web system
Distributed web system
6
Request Routing Mechanisms forCluster-based Web
Systems (1/9)

• Layer-4 vs. Layer-7 switching
• Layer-4 switching (content-blind routing)
• Web switch determines the target server when the
  clients asks for establishing a TCP/IP
  connection, upon the arrival of the first TCP SYN
  packet
• Efficient and simple, but unaware of the content
  of the client request
• Layer-7 switching (content-aware routing)
• Web switch examines the HTTP request at
  application level and then replays it to the
  target server
• Inefficient, but more sophisticated dispatching
  policies are possible
• Two-way vs. one-way routing
• Two-way the target server returns its response
  to the Web switch
• One-way the target server responds directly to
  the client
• More complex but efficient

7
Request Routing Mechanisms forCluster-based Web
Systems (2/9)

• Layer-4 two way architecture
• Packet rewriting based on IP Network Address
  Translation (NAT)
• Web switch rewrites inbound packet by changing
  VIP to the target server IP
• Web switch rewrites outbound packet by changing
  the serve IP to VIP
• Web switch recalculates IP and TCP header
  checksum for both packets

8
Request Routing Mechanisms forCluster-based Web
Systems (3/9)

• Layer-4 one-way architecture
• Packet single-rewriting
• Packet tunneling
• Packet forwarding

9
Request Routing Mechanisms forCluster-based Web
Systems (4/9)

• Layer-4 one-way architecture (contd)
• Packet single-rewriting
• Web switch rewrites the destination IP address
  for each inbound packet
• Web server replaces the source IP address with
  VIP for each outbound packet

143.248.137.2
src
202.103.106.5
dst
143.248.137.2
172.16.0.3
202.103.106.5
143.248.137.2
10
Request Routing Mechanisms forCluster-based Web
Systems (5/9)

• Layer-4 one-way architecture (contd)
• Packet tunneling (IP encapsulation)
• Web switch encapsulates inbound IP packet
• Web server strips the IP header off and response
  directly to the client using VIP
• All servers should support IP tunneling and
  configured with VIP

dst
src
payload
VIP
client
encapsulation at Web switch
dst
src
payload
VIP
client
server
switch
IP network
dst
src
payload
VIP
client
server
switch
decapsulation at Web server
dst
src
payload
VIP
client
11
Request Routing Mechanisms forCluster-based Web
Systems (6/9)

• Layer-4 one-way architecture (contd)
• Packet forwarding (MAC address translation)
• VIP address is shared by Web switch and all of
  the servers
• Inbound packets reach the Web switch, even all
  nodes share VIP
• All servers disable ARP (Address Resolution
  Protocol) mechanism
• Web switch modifies MAC address of the inbound
  packet and retransmit it
• Web server can response directly to the client
  since it shares VIP

dst
src
payload
VIP
client
Web switch
VIP
client

server MAC
physical layer
VIP
client

server MAC
Web server
VIP
client
12
Request Routing Mechanisms forCluster-based Web
Systems (7/9)

• Layer-7 two-way architecture
• TCP gateway
• Web switch maintains both connections for the
  client and the server
• TCP splicing
• Once the client-to-server binding is determined,
  the two connections are spliced
• Packet rewriting for established connection

13
Request Routing Mechanisms forCluster-based Web
Systems (8/9)

• Layer-7 one-way architecture
• TCP handoff
• Web server hands off its endpoint of the TCP
  connection to the server
• Multiple handoff, back-end forwarding
• TCP connection hop (by Resonate)
• Send encapsulated IP packet to the server
• Web switch manages session information

14
Request Routing Mechanisms forCluster-based Web
Systems (9/9)

• Comparison of routing mechanisms for Web clusters
• Two-way vs. one-way
• Two-way is less scalable because of the overhead
  in the Web switch
• Layer-4 switching vs. Layer-7 switching
• Layer-7 switching is less scalable
• Layer-7 switching gains benefits from
  content-aware dispatching

approach
cons
data flow
routing
pros
two-way
switching throughput TCP connection grain control
flexible, portable
inbound/outbound
content-blind
Layer-4
one-way
Network topology, TCP connection grain control
simple, fast routing
inbound
content-blind
two-way
switching bottleneck, Slowest routing
simple, caching, server specialization, HTTP
grain control, content partition, SSL session
reuse
inbound/outbound
content-aware
Layer-7
one-way
switching bottleneck, Complex routing
server specialization, HTTP grain
control, content partition, SSL session reuse
inbound
content-aware
15
Request Routing Mechanisms forDistributed Web
Systems (1/2)

• DNS routing mechanisms
• Authoritative DNS server for the Web site selects
  a different server for every address resolution
  request ltIP, TTLgt
• Larger TTL address-caching causes load imbalance
• Smaller TTL DNS traffic increases ? response
  time increases
• Authoritative DNS server does not work for
  client-level address caching
• Non-cooperative DNS servers would override with
  their minimum TTL

16
Request Routing Mechanisms forDistributed Web
Systems (2/2)

• Web server routing mechanisms
• Triangulation
• The client continues to send packets to the first
  contacted server
• The first server re-routes requests to the other
  server by IP tunneling
• HTTP redirection
• HTTP status code 301, 302
• Replication management, content-aware routing
• Extra round-trip time
• Most browsers does not handle redirection well
• Bookmark to the redirected site
• URL rewriting
• The first contacted server changes
  dynamicallythe links for the embedded objects
• Used by Content Delivery Networks (CDN)
• Additional load on the redirecting Web server

HTTP redirection
17
Dispatching Algorithms forCluster-based Web
Systems (1/5)

• Content-blind dispatching policies
• Stateless (static) algorithms
• Random, RR, static WRR
• Client state aware algorithms
• Use client IP address and TCP port numbers to
  assign a request to a server
• Server state aware algorithms
• Least loaded least connections, fast response
• WRR
• Client and server state aware algorithms
• Considering client affinity (ex. consecutive SSL
  connections, FTP)
• Considerations for content-blind dispatching
• Stateless algorithms easy and fast dispatching,
  but poor performance
• State-aware algorithms extra overheads
• herd effect temporal saturation caused by
  long load information interval

18
Dispatching Algorithms forCluster-based Web
Systems (2/5)

• Content-aware dispatching policies
• Cache affinity improve reference locality in the
  server caches
• Specialized servers provide differentiated Web
  services
• Load sharing increase load sharing among the
  server nodes
• Client affinity
• HTTP/1.1 persistent connection
• HTTP requests in a same TCP connection can be
  handled by different servers
• Content-blind dispatching should control TCP
  connection granularity

content-aware dispatching
client-aware
client and server aware
cache affinity
load sharing
specialized servers
client affinity
cache affinity
load sharing
hash
CAP
SITA-E
service partitioning
session identifiers
cache manager
LARD
19
Dispatching Algorithms forCluster-based Web
Systems (3/5)

• Content-aware dispatching policies (contd)
• Client state aware algorithms
• Hash static partitioning of the files
• Improves cache hit rate, but can be used only for
  static contents
• Ignores load sharing
• Service partitioning different contents are
  serviced by different servers
• Size Interval Task Assignment with Equal load
  (SITA-E)
• Separating light tasks from heavy tasks
  (considering content size only)
• Client-Aware Policy (CAP)
• Classifies Web requests into several classes
• Estimates each requests impact on Web server
  resources
• Session identifier
• Layer-7 session identifier is better than Layer-4
  when Web proxies squeeze a large number of users
  into a small number of different IP addresses

20
Dispatching Algorithms forCluster-based Web
Systems (4/5)

• Content-aware dispatching policies (contd)
• Client and server state aware algorithms
• Locality-Aware Request Distribution (LARD)
• Considers both locality and load balancing
• Direct all requests for the same object to the
  same server below a threshold
• Cache manager
• Cache manager periodically gathers cache
  information of the servers

node A
node A
node A
requests
node B
node B
node B
Web switch
node C
node C
node C
Cache affinity only
Load sharing only
LARD
21
Dispatching Algorithms forCluster-based Web
Systems (5/5)

• Analysis of dispatching algorithms
• Content-blind vs. content-aware dispatching
• Content-aware dispatching can potentially
  outperform content-blind dispatching
• Content-aware dispatching is expensive
• Consider the heterogeneity of Web services
• Dynamic contents, secure services,
• Server state information
• Load indexes heavily impact on final performance
  of the dispatching algorithms
• Input indexes, server indexes, forward indexes
• Input indexes can be easily computed, but have
  limited information
• Server indexes provide detailed information, but
  take longer time to acquire
• Updating load information
• Long update interval poor performance because of
  the old load information
• herd effect
• Short update interval more overhead

22
Integrated Dispatching Mechanisms

• Improve scalability of Web systems
• Solutions that improve Web cluster scalability
• Multiple Web clusters authoritative DNS
• Web server accelerators
• The caches which store frequently accessed Web
  objects
• Web switch routes requests to the cache nodes
• Solutions that improve Web switch scalability
• Web switch is a potential bottleneck and a single
  point of failure
• Layer-7 switch is more critical
• Solution multi-level switching
• Layer-4 switch acts as frontend, and several
  Layer-7 switches are attached
• Layer-4 switch a distributor in each server
  node a single dispatcher (Aron et al.,
  UATC2000)
• Multi-Node Load Balancing (MNLB) by CISCO
• Separates packet-by-packet processing functions
  from sever selection

23
Placement of Web Contents and Services(1/2)

• Distribution of static information
• Distributed file system
• Communication overhead, need to be optimized for
  Web services
• Content replication
• No extra communication overhead
• High storage overhead
• Hard to maintain consistency among replicated
  objects
• Especially for highly volatile and frequently
  updated objects
• Service partition
• Layer-7 switching
• High secondary storage scalability, server
  specialization
• Load imbalance
• Partition partial replication can be a
  cost-effective solution
• Static placement has potential weakness as the
  access pattern changes
• Dynamic placement approaches

24
Placement of Web Contents and Services(2/2)

• Distribution of dynamic services
• Dynamic services are essential in modern Web
  sites
• Use Web as a standard interface for underlying
  network services
• Web-based information systems, or Web-based
  enterprise applications
• CGI, JSP, ASP, PHP, XML,
• Multi-tier architecture for a Web cluster
• Request routing and dispatching can
  beimplemented at different levels
• Second-level Web switch can beintegrated into
  each Web server
• Simple mechanisms are usedin the practical
  products
• Due to complexity
• RR, least loaded
• Application-level caching

25
Summary

• Locally distributed Web system architecture
• Multi-node scale-out to improve Web system
  performance and scalability
• Cluster-based Web system
• Layer-4 vs. Layer-7 switching
• Two-way vs. One-way routing
• Distributed Web system
• DNS based routing / Web server routing
• Dispatching algorithms
• Content-blind vs. content-aware routing
• Use client and server state information to
  improve performance
• Integrated dispatching mechanisms improves Web
  system scalability
• Placement of web contents and services
• Static contents distributed file system,
  replication, partition
• Dynamic services multi-tier architecture

26
Research Perspectives

• Web is becoming the standard interface for
  network services
• Web clusters will be the basic architecture for
  Web-based info. system
• Increasing complexity of Web-based information
  system will make performance of the Web clusters
  worse
• Heterogeneity, security, middleware complexity,
  high availability,
• Research directions
• End-to-end QoS support by Web server systems
• Layer-7 scalability improvement
• Optimal resource management in multi-tier
  architectures
• Cooperation of multiple servers and multiple
  caches