Title: Towards Virtual Networks for Virtual Machine Grid Computing
1Towards Virtual Networks for Virtual Machine Grid
Computing
- Ananth I. Sundararaj
- Peter A. Dinda
- Prescience Lab
- Department of Computer Science
- Northwestern University
- http//virtuoso.cs.northwestern.edu
2Outline
- Virtual machine grid computing
- Virtuoso system
- Networking challenges in Virtuoso
- Enter VNET
- VNET Adaptive virtual network
- Related Work
- Conclusions
- Current Status
3Deliver arbitrary amounts of computational power
to perform distributed and parallel computations
Aim
1
New Paradigm
Traditional Paradigm
5
2
Grid Computing using virtual machines
Resource multiplexing using OS level mechanism
Grid Computing
4
3a
6a
3b
Problem1
6b
Virtual Machines What are they?
Complexity from resource users perspective
Solution
Problem2
How to leverage them?
Complexity from resource owners perspective
4Virtual Machines
Virtual machine monitors (VMMs)
- Raw machine is the abstraction
- VM represented by a single
- image
- VMware GSX Server
5Virtual machine grid computing
- Approach Lower level of abstraction
- Raw machines, not processes, jobs, RPC calls
- R. Figueiredo, P. Dinda, J. Fortes, A Case For
Grid Computing on Virtual Machines, ICDCS 2003 - Mechanism Virtual machine monitors
- Our Focus Middleware support to hide complexity
- Ordering, instantiation, migration of machines
- Virtual networking
- remote devices
- Connectivity to remote files, machines
- Information services
- Monitoring and prediction
- Resource control
6The Simplified Virtuoso Model
Virtual networking ties the machine back to
users home network
Users LAN
Specific hardware and performance
VM
Basic software installation available
Orders a raw machine
Virtuoso continuously monitors and adapts
User
7Users View in Virtuoso Model
Users LAN
VM
User
8Outline
- Virtual machine grid computing
- Virtuoso system
- Networking challenges in Virtuoso
- Enter VNET
- VNET Adaptive virtual network
- Related Work
- Conclusions
- Current Status
9Why VNET? A Scenario
Foreign hostile LAN
Users friendly LAN
IP network
User has just bought
Virtual Machine
10Why VNET? A Scenario
VM traffic going out on foreign LAN
Foreign hostile LAN
X
Users friendly LAN
IP network
Virtual Machine
Host
- A machine is suddenly plugged into a foreign
network. What happens? - Does it get an IP address?
- Is it a routeable address?
- Does firewall let its traffic
- through? To any port?
Proxy
VNET A bridge with long wires
11Outline
- Virtual machine grid computing
- Virtuoso system
- Networking challenges in Virtuoso
- Enter VNET
- VNET Adaptive virtual network
- Related Work
- Conclusions
- Current Status
12A Layer 2 Virtual Network for the Users Virtual
Machines
- Why Layer 2?
- Protocol agnostic
- Mobility
- Simple to understand
- Ubiquity of Ethernet on end-systems
- What about scaling?
- Number of VMs limited (1024 per user)
- One VNET per user
- Hierarchical routing possible because MAC
addresses can be assigned hierarchically
13VNET operation
Host Only Network
VM
ethy
ethz
eth0
ethx
eth0
vmnet0
Client LAN
VNET
VNET
IP Network
Host
Proxy
Client
Ethernet Packet Injected Directly into VM
interface
Ethernet Packet Captured by Promiscuous Packet
Filter
Ethernet Packet Tunneled over TCP/SSL Connection
Traffic outbound from the users LAN
14Performance Evaluation
However
Main goal
- VNETs performance should be
- In line with physical network
- Comparable to other options
- Sufficient for scenarios
Convey the network management problem induced by
VMs to the home network of the user
Metrics
Latency
Bandwidth
Why?
How?
How?
Why?
- small transfer
- Interactivity
- ttcp
- socket buffer
- 1 GB of data
- Large transfer
- low throughput
15VNET test configuration
VM
100 mbit Switches
100 mbit Switch
100 mbit Switch
Router
Router
Firewall 1
Client
Host
Proxy
Local
IP Network (14 hops via Abilene)
Northwestern University, IL
Carnegie Mellon University, PA
Wide area configuration
VM
100 mbit Switches
100 mbit Switch
100 mbit Switches
100 mbit Switch
Router
Firewall 1
Firewall 2
Client
Host
Proxy
Local
Local area configuration
16Average latency over WAN
Host - VM
Proxy - Host
Client - Proxy
(Physical Network)
Northwestern University, IL
Carnegie Mellon University, PA
17Standard deviation of latency over WAN
What
VNET increases variability in latency
TCP connection between VNET servers trades packet
loss for increased delay
Why
(Physical Network)
18Bandwidth over WAN
Expectation
VNET to achieve throughput comparable to the
physical network
What do we see
VNET achieves lower than expected throughput
Why
VNETs is tricking TTCPs TCP connection
19Outline
- Virtual machine grid computing
- Virtuoso system
- Networking challenges in Virtuoso
- Enter VNET
- VNET Adaptive virtual network
- Related Work
- Conclusions
- Current Status
20VNET Overlay
Foreign hostile LAN 1
Users friendly LAN
VM 1
Host 1 VNET
IP network
Proxy VNET
Foreign hostile LAN 2
Host 3 VNET
VM 2
VM 4
Host 2 VNET
Host 4 VNET
VM 3
Foreign hostile LAN 3
Foreign hostile LAN 4
21Bootstrapping the Virtual Network
VM
VM
Host VNETd
Vnetd
Proxy VNETd
VM
- Star topology always possible
- Topology may change
- Links can be added or removed on demand
- Virtual machines can migrate
- Forwarding rules can change
- Forwarding rules can be added or removed on
demand
22Application communication topology and traffic
load application processor load
VM Layer
Vnetd layer can collect all this information as a
side effect of packet transfers and invisibly act
VNETd Layer
Network bandwidth and latency sometimes topology
Physical Layer
23Outline
- Virtual machine grid computing
- Virtuoso system
- Networking challenges in Virtuoso
- Enter VNET
- VNET Adaptive virtual network
- Related Work
- Conclusions
- Current Status
24Related Work
- Collective / Capsule Computing (Stanford)
- VMM, Migration/caching, Hierarchical image files
- Denali (U. Washington)
- Highly scalable VMMs (1000s of VMMs per node)
- SODA and VIOLIN (Purdue)
- Virtual Server, fast deployment of services
- VPN
- Virtual LANs, IEEE
- Overlay Networks RON, Spawning networks,
Overcast - Ensim
- Virtuozzo (SWSoft)
- Ensim competitor
- Available VMMs IBMs VM, VMWare, Virtual
PC/Server, Plex/86, SIMICS, Hypervisor, VM/386
25Conclusions
- There exists a strong case for grid computing
using virtual machines - Challenging network management problem induced by
VMs in the grid environment - Described and evaluated a tool, VNET, that solves
this problem - Discussed the opportunities, the combination of
VNET and VMs present, to exploit an adaptive
overlay network
26Current Status
- Application traffic load measurement and topology
inference Ashish Gupta - Support for arbitrary topologies and forwarding
rules - Dynamic adaptation to improve performance
27Current Status Snapshots
Pseudo proxy
28- For More Information
- Prescience Lab (Northwestern University)
- http//plab.cs.northwestern.edu
- Virtuoso Resource Management and Prediction for
Distributed Computing using Virtual Machines - http//virtuoso.cs.northwestern.edu
- VNET is publicly available from
- http//virtuoso.cs.northwestern.edu
29Isnt It Going to Be Too Slow?
Small relative virtualization overhead compute-in
tensive
Relative overheads lt 5
Experimental setup physical dual Pentium III
933MHz, 512MB memory, RedHat 7.1, 30GB disk
virtual Vmware Workstation 3.0a, 128MB memory,
2GB virtual disk, RedHat 2.0 NFS-based grid
virtual file system between UFL (client) and NWU
(server)
30Isnt It Going To Be Too Slow?
Synthetic benchmark exponentially arrivals of
compute bound tasks, background load provided by
playback of traces from PSC Relative overheads lt
10
31Isnt It Going To Be Too Slow?
- Virtualized NICs have very similar bandwidth,
slightly higher latencies - J. Sugerman, G. Venkitachalam, B-H Lim,
Virtualizing I/O Devices on VMware Workstations
Hosted Virtual Machine Monitor, USENIX 2001 - Disk-intensive workloads (kernel build, web
service) 30 slowdown - S. King, G. Dunlap, P. Chen, OS support for
Virtual Machines, USENIX 2003 - However May not scale with faster NIC or disk
32Average latency over WAN
P-H
Client -- C Proxy -- P Host -- H
Inline with Physical? Physical C-P P-H H-VM
0.34 36.993 0.189
37.522 ms VNET 37.535 ms
35.525 ms (with SSL)
H-VM
C-P
Comparison with options VNET 37.535 ms
35.525 ms (with SSL) VMware 35.625
(NAT) 37.435 ms (bridged)
Physical network
VMware options
VNET options
33Standard deviation of latency over WAN
What
VNET increases variability in latency
TCP connection between VNET servers trades packet
loss for increased delay
Why
Client -- C Proxy -- P Host -- H
H-VM
Inline with Physical? Physical C-P P-H H-VM
1.11 18.702 0.095
19.907 ms VNET 77.287 ms
40.763 ms (with SSL)
C-P
H-VM
34Bandwidth over WAN
VMWare bridged networking
Physical network
Expect
VNET to achieve throughput comparable to the
physical network
What
VNET achieves lower than expected throughput
VNETs is tricking TTCPs TCP connection
Why
Inline with Physical? Physical 1.93 MB/s VNET
1.22 MB/s 0.94 MB/s (with SSL)