Multithreading vs. Event Driven in Code Development of High Performance Servers

1
Multithreading vs. Event Driven in Code
Development of High Performance Servers
2
Goals

• Maximize server utilization and decrease its
  latency.
• Server process doesnt block!
• Robust under heavy loads.
• Relatively easy to develop.

3
Key Idea
Multi-threaded servers
Event driven servers
Programming level
Event Driven
Multi-Threaded
Event Driven
Multi-Threaded
Implementation level
4
Key Idea
Multi-threaded servers
Event driven servers
Programming level
Event Driven
Multi-Threaded
Event Driven
Multi-Threaded
Implementation level

• Get the best of the two worlds.

5
Backgound AIO

• Asynchronous I/O mainly used for disk
  reads/writes.
• Issue disk request and return immediately without
  blocking for results.
• On completion an event is raised.

6
Servers Architecturesa- Multi-threaded Servers

• Kernel thread per request executing the request
  handler.
• A thread blocking on I/O doesnt block the
  server.
• Ex Apache, MySql.

Request Handler
A
B
Function Calls
C
7
Multi-threaded Servers (Cont.)

• Advantages
• Relatively easy to develop.
• Disadvantages
• - Not good performance.
• Thread scheduling.
• Context switching.
• - Doesnt scale.
• Under heavy load , many threads would be created
  exhausting kernels memory, crashing the server.

8
Needed for Event-Driven servers
Asynchronous I/O

• Kernel event notification mechanism.
• Supported by FreeBSD Kernel.
• Events queued in a kernel queue, and delivered to
  user by kernel calls.
• Events usually correspond to AIO completions, or
  socket reads/writes.

9
b- Event Driven Servers

• Single thread (event loop).
• AIO operations.
• I/O completions / Requests (events) put in Event
  Queue.
• Continuations.
• ExFlash web server (slightly different
  architecture).
• High performance.
• Scalable/Robust.
• - Hard to write.

Event Loop
Events From Kernel
A1
A2
B2
B1
Event Queue (KQueue)
C2
C1
Blocking operation
10
Lazy Threading

• Similar design to event driven but no
  continuations.
• Threads yield control on fixed points (I/O).
• If request handler doesnt block
• Run handler to completion then use same thread to
  handle next event.
• Else
• Suspend current thread, create another user
  thread to handle next event.

11
Wrappers / Thread Suspension

• IO_Wrapper(..)
• uc save_user_context(..)
• / save the current user thread context to be
  able to resume it later /
• aio_func(..)
• / non-blocking I/O call /
• stackallocate_stack()
• switch_thread(event_loop,stack)
• / suspend current thread and create a new thread
  running the event loop using the newly allocated
  stack /
• .
• .

• Wrappers around blocking operations to create
  the blocking illusion.
• Uses AIO.
• Saves current stack instead of unwinding it.
• Creates a user thread by jumping to the event
  loop and using a new stack.

12
Thread Resumption

• Event_loop()
• .
• .
• .
• e get_event(..)
• / read new event from the kernels kqueue /
• if e is completion of aio_func(..)
• run_thread(uc)
• / dispose current thread and resume the
  IO_Wrapper from where it has stopped /
• .
• .

• Restores registers and jumps back to wrapper as
  if blocking has finished.
• Current thread is disposed as its useless.
• Wrappers thread eventually returns to the event
  loop after handling its request.

13
Summary
Lazy-Threaded
Event Driven
Multi-Threaded
Yes
Yes
No
Scalable
Yes
Yes
No
High Performance
Yes
No
Yes
Easy to build
Yes(user)
No
Yes(kernel)
gt1 thread
No
No
Yes
Thread / request?
No
N/A
Yes
Arbitrary thread yield