Scalability of Linux Event-Dispatch Mechanisms

1
Scalability of Linux Event-Dispatch Mechanisms

• Abhishek Chandra
• University of Massachusetts
• Amherst

David Mosberger Hewlett Packard Labs Palo Alto
2
Motivation
Web Server
Clients
WAN

• Large Web and Internet traffic
• Heavily Loaded/Accessed Web Servers
• cnn.com, britneyspearsfans.com,
• Starr Report, Napster ruling, ...
• Challenge Make Web Servers Scalable

3
Server Scalability Issues

• Large number of concurrent/idle connections
• Last-mile problem Slow end-connections
• High latency WAN traffic
• HTTP/1.1 Persistent Connections
• Heavy Request Loads
• Need for high throughput
• Pure Thread-based vs. Event-based servers
• Focus Scalability of Event-based servers on Linux

4
Outline

• Motivation
• Event-based Servers
• Linux Event-Dispatch Mechanisms
• Evaluation Handling concurrent connections
• RT signals and Signal-per-fd enhancement
• Evaluation Handling request load
• Concluding Remarks

5
Event-Based Servers
Server
Kernel
Connections

• Server specifies Interest Set to Kernel
• Kernel notifies Server of Event on a connection
• Server handles I/O on the connection

6
Linux Event-Dispatch Mechanisms

• select() system call
• poll() system call
• /dev/poll interface
• POSIX.4 Real-Time Signals

7
select() system call
Server
Kernel
Connections

• Interest Set specified on each call
• Notification requires scan of interest set

8
poll() and /dev/poll

• Interest Set
• List of pollfd structures
• Better for sparse interest sets, worse for dense
  sets
• Notification
• Requires scan of Interest Set
• /dev/poll
• Interest Set specified incrementally
• More compact ready set

9
POSIX.4 Real Time Signals

• RT signals are queued
• Multiple signals of same type can be delivered
• RT signals carry a data payload (siginfo)
• Provides the context of the signal
• sigwaitinfo() system call
• Dequeues signals
• Avoids overhead of calling signal handler
• Signal can be blocked

10
Using Real Time Signals for Network I/O
Server
Kernel
Signal Queue
Interest Set
Sockets

• Interest Set specified incrementally
• No scanning of Interest Set required

11
Outline

• Motivation
• Event-based Servers
• Linux Event-Dispatch Mechanisms
• Evaluation Handling concurrent connections
• RT signals and Signal-per-fd enhancement
• Evaluation Handling request load
• Concluding Remarks

12
Evaluation Handling Concurrent Connections

• Dispatch overhead and latency as a function of
  number of concurrent connections
• Experimental Setup
• 400 MHz P3 Linux 2.4.0-test7 server
• µ -server using select(), /dev/poll or RT signals
• 10 clients running httperf
• Fixed request rate, increasing number of
  connections

13
Server CPU Usage
500 req/s

• RT signal overhead independent of no. of
  concurrent connections

14
Response Time
500 req/s

• RT signal response time independent of no. of
  concurrent connections

15
Limitations of Real Time Signals
Server
3
2
3
Kernel
Signal Queue
Interest Set
Sockets
1
2
3
4

• Signal Queue Overflow
• New events lost
• Can lead to hung server
• Unfair Allocation of Signal Queue

16
Handling Signal Queue Overflow

• Fallback mechanism
• select(), poll(), etc.
• Reconstruct current state
• Issues
• Server complexity
• Overhead of maintaining explicit interest sets
• Potential performance penalty

17
RT Signal Enhancement Signal-per-fd

• Goals
• Avoid signal queue overflows
• Fair Allocation of signal queue
• Solution Enqueue only one signal per socket

Server
2
3
Kernel
Signal Queue
Interest Set
Sockets
1
2
3
4
18
Signal-per-fd

• Idea
• Signal queue length same as fdset size
• Bitmap used to efficiently determine
  presence/absence of signal in queue
• Advantages
• Simpler Server Implementation
• No signal queue overflows
• No need for fallback mechanisms
• Fair Allocation of Signal Queue Resource
• Avoids too fine-grained event notification
• Coalesce multiple events for a socket

19
Outline

• Motivation
• Event-based Servers
• Linux Event-Dispatch Mechanisms
• Evaluation Handling concurrent connections
• RT signals and Signal-per-fd enhancement
• Evaluation Handling request load
• Concluding Remarks

20
Server Throughput
6000 idle connections

• Linear scaling of RT signals, signal-per-fd

21
Server CPU Usage
6000 idle connections

• Linear Scaling of RT signals, signal-per-fd

22
Related Work

• Event-Delivery API BMD99
• Performance studies
• select() BM98, /dev/poll PL00
• RT signals PLT00
• Web Servers
• Event-based Flash PDZ99, phhttpd Brown99
• In-kernel TUX, khttpd, AFPA JKNRT01
• Future Linux 2.5 Asynchronous I/O?

23
Summary

• Scalability issues with Linux Event-dispatch
  mechanisms
• Real Time Signals are scalable
• Performance independent of number of concurrent
  connections
• Signal Queue Overflow Problems
• Signal-per-fd enhancement
• potentially improves performance
• reduces server complexity
• provides fairness
• Patch available at http//www.netli.com/links