Communication in Shared Memory Systems LRPC

1
Communication in Shared Memory Systems - LRPC

2
LRPC
3
Takeaway

• Approach behind the study
• Start with some intuition about the system
• Measure the system components
• Prove or disprove the hypothesis
• Construct system to optimize the common case in
  the hypothesis
• Specifically in this paper
• RPC and cross domain calls in a single machine

4
RPC Vs. Cross domain calls

• Distributed system assumptions
• Large grained protection domains (since they
  cross machine boundaries)
• Large complex objects exchanged (e. g. media
  files, )
• Same machine cross domain
• Fine grained protection domains
• Small objects (args and return values in typical
  procedures)
• Interesting note
• Cross domain same machine calls dominate even in
  distributed systems such as NFS connected LANs

5
Upshot of poor structuring

• pack logically separate entities into one domain
  for performance
• bad.bad.bad
• How LRPC wins
• Simple control transfer
• Simple data transfer
• Simple stubs
• Design for concurrency in an SMP

6
Measurements

• Even distributed systems do mostly cross domain
  activities on the same machine most of the time
• Parameter distribution for cross domain calls
  imply small size transfers

7
Remote activity
8
RPC size distribution
9
Cross domain RPC

• Cost of client-server call through kernel
• 2 traps
• 2 context switches
• 1 procedure execution
• Sources of overhead
• Message copying through kernel (potentially 4)
• Access validation
• Message transfer
• Scheduling
• Context switch (memory system)
• Dispatching a thread

10
Cross domain measurements
11
LRPC ideas

• Basically we want to make the cross domain call
  look like a simple procedure call within the same
  address space
• Possible?
• Execution model of protected proc call
• Large-grained protection model of RPC (dynamic
  binding)

12
Binding

• Server registers entry points in name server
• Clerk in server waits for bind requests from
  kernel
• Client looks up name server for entry points
• Client issues bind call up call to clerk clerk
  validates returns PDL to kernel
• Kernel allocates A-stack mapped into client and
  server domains
• BO returned to client

13
Calling

• Client Stub prepares A-stack with args
• Args by ref copied into A-stack (why?)
• Traps to the kernel with BO, A-stack address,
  etc., in well known place
• Client validates the call using the BO
• Client thread is doctored by the kernel to start
  executing the server code in the server domain
• Server stub prepares A-stack for return
• Result by ref copied into A-stack (why?)
• No validation required on return from server
  (why?)
• A-stack and E-stack managed by the kernel

14
LRPC on MP

• Context switch is most expensive
• Pre-allocate server domains to processors
• Make the client execute on a processor that is
  pre-loaded with server domain
• Monitor and allocate more processors to heavily
  used domains

15
Performance

• Once again watch for trends
• Table IV and V

16
LRPC performance
17
LRPC breakdown
18
Uncommon cases

• Cross domain vs. cross machine call determination
  needs to be quick
• Variable sized args/results need to be handled in
  A-stack
• Domain termination needs to be handled