Monkey See, Monkey Do A Tool for TCP Tracing and Replaying

1
Monkey See, Monkey DoA Tool for TCP Tracing and
Replaying

• Yu-Chung Cheng
• Stefan Savage
• Geoffrey M. Voelker
• Dept. of Computer Science Engineering
• University of California, San Diego

Urs Hölzle Neal Cardwell
2
Motivation

• Server benchmarks the what if question
• Ex how will changing the server in some way
  impact the client experience?
• Current solutions
• Synthetic workload testing risk-free,
  convenient, not very realistic
• Real user testing realistic, risky, more hassle
• Best of both worlds?
• Realistic, convenient and risk-free

3
Goal

• Develop a tool, Monkey, that provides the best of
  both worlds
• Monkey See captures real traces, infers network
  and client parameters
• Monkey Do replays traces via emulation
• Applied Monkey to Google workload
• Captured Google search requests
• Replayed and evaluated against Google servers
• Evaluation
• How realistic is Monkeys replay?
• Can Monkey answer the what if questions?

4
Monkey Design
5
Monkey See

• Goal capturing Google search traces and infer
  client/network parameters
• Trace capture in front of server farm
• Random connection sampling
• Extracting info for each TCP connection
• Search query from HTTP header
• Path delays from RTT
• Bandwidth by counting ACK spacing
• Loss rate by counting retransmissions
• Path MTU from TCP MSS option
• Use of TCP delay ACKs by observing ACK sequences

6
Assumptions
3. Well-provisioned Google Web Server
1. Congestion/loss only happens on data path but
not on ACK path
client
Google Web server
Google DB
2. No congestion in Google intranet
TCP data packet
TCP ACK packet
7
Monkey Do

• Goal replay traces by emulating the client and
  network environment
• Network emulator (dummynet)
• Setup virtual dummynet pipes as data and ACK
  paths
• Client emulator
• Establishes connections to the server and send
  out search queries
• Server emulator
• Emulates the effect of caching on DB search delay
• Runs on same kernel (Linux) as the real Google
  web daemon does

8
Replay in action real Google search
Web server
DB server
Client
connect()
Static header
Search result
Time
9
Replay in action replay version
Client Emulator
Server Emulator
connect()
Static header
Search result
Time
10
Evaluation

• Replay Validation
• How well does Monkey reproduce the workload?
• Predictive Replay
• How well does Monkey predict?
• Metric Google search response time
• Time between the server receives the request and
  completes sending the result
• Trace taken a weekday of Nov 2003

11
Replay Validation Experiment

• How well does Monkey reproduce the workload?
• Replay the workload without any changes and
  compare the results

12
Replay Validation Result
connections
Relative error of search response time ms

• CDF of relative error in search response time per
  connection
• 86 of connections have response time within 20
  error

• CDF of relative error in search response time per
  connection
• Monkey tends to underestimate for long RTT
  connections

13
Predictive Replay

• Is Monkeys replay predictive?
• Compare replay result with result from real
  server changes
• Experiment
• Search response time vs. TCP initial congestion
  window size (cwin)

14
Predictive Replay Result
connections
Search response time ms
CDF of search response times for Tw3, Tw1, and
Treplay
15
Discussion

• Monkey can replay Google search traces well, but
  
• Can Monkey replay non-Google web or any TCP
  traffic?
• Possible, if the assumptions hold
• Caveats dependencies in disk/network IO events,
  multiple TCP connections
• Is server emulator required?
• No, if the server performance distribution is
  memory-less. Otherwise, yes.

16
Conclusion

• Monkey is a new tool for server benchmarking
  (what if questions)
• Currently is domain-specific
• Accurate for short flows
• Best of both worlds
• Realism of real user testing
• Risk-free and convenience of synthetic tools
• Possible to build such tools for specific domain

17
Q A

• Questions?

http//ramp.ucsd.edu/projects/monkey/