On the Run Architecting and delivering Wintel Virtualisation with VMware Accurately

1
On the RunArchitecting and delivering Wintel
Virtualisation with VMwareAccurately

• Shmuel Markovits
• Performance Capacity Management Group,
• Datos Pty Ltd

2
Agenda

• Our Scenario Description
• Rule of thumb for estimating of VMs
• Any Qualifications?
• Virtualisation Candidates
• Classes of applications
• Snapshots of running LPARs
• Indicators of Current Performance of VMs
• Moving forward
• Measure Mixing?

3
Virtualisation

• Allows a homogenous environment to appear as
  multiple environments or vice versa.
• Virtualisation layer allows the
• Server
• A server to appears as an aggregation of servers
  (distrib network) or aggregation of servers to
  appears as one server
• Network
• Allows homogenous communications link appears as
  Mutiple VLANS with multiple QoS characteristics
• Storage
• Turns heterogenous network storage devices into
  homogenous storage device

4
What is your goal in Virtualisation?

• Consolidate resources
• CPU usage
• Memory optimisation
• I/O spread
• Network

• Reduced environmental footprint
• Power
• Space
• Cabling

• Other Benefits
• Availability
• Security
• QoS
• Disaster recovery

5
Bit of History

• Partitioning in mainframe host env
• Create testing scenarios
• Re-create failure scenarios
• X86 cheap to separate applications
• Different server types environments
• Payback-
• Initially labour costs caused re-evaluation
• Labour associated with physical resource is
  reduced while server administration cost is not
  reduced
• Easy setup for develop and test env

6
Motivation for Virtualisation Layer inX86 servers

• Mainframes -gt Minis -gt x86 based servers
• Move to isolate an application to a individual
  server
• Per unit capability x86 based servers has gone up
• Single Core -gt Dual -gt Quad
• Synergise
• Workload vs Time etc
• Need for a Virtualisation Layer

7
Virtualisation Layer
8
Virtualisation Layer

• What can we put in this layer ?
• All physical resources are available for
  virtualisation
• Historically made visible via the OS for
• Disk
• I/O
• Memory via paging
• Physical File (file sockets)
• Expanded to inter/intra computer Communications
• ip streams handled as file sockets
• Is this NEW technology?
• for X86 the answer is YES

9
Before u run

• There are some terms we need to take on board and
  feel comfortable about firstterms first
• VM (virtual machine)
• Guests (VMware)
• Logical PARtitions (LPARs IBM)
• Host machine
• from VM point of view

10
VMs within a Host (2x1vCPU and 2vCPU )
11
Agenda

• Rule of thumb for estimating of LPARS
• Any Qualifications?
• Current Performance of LPARs
• Snapshots of running LPARs
• Virtualisation Candidates
• Classes of applications
• Mix Match?
• Further investigation

12
Architecture Design Dependencies

• Types of applications to be deployed
• Type of OS to be deployed
• Qty of guests needed
• Hardware characteristics available

13
Sample Vendors?

• IBM
• AIX and technology imbedded in many of the
  servers and products
• In x86 market
• VMware eg ESX (market leader)
• SUN xVM Virtual Box on Mac OS X, Windows x86,
  Solaris Linux.
• Microsoft eg Virtual Server or Hyper-V (new to
  market)
• Xen Based (open source) eg XenSource or Virtual
  Iron
• Kvm (Kernel based VM) open source hypervisor
  (linux and s390).
• This talk will concentrate on performance using
  VMware on IBM Xseries

14
Workstation, ESX Server

• 4 layers or more
• Guest Layer eg Win OS, Linux OS
• Presented Virtual Machine (VM)
• Virtualisation Software
• Host Machine (OS and h/w or baremetal)

• Interrelation between layers underpins
  understanding of
• performance issues
• where to put effort.

15
Agenda

• Rule of thumb for estimating of LPARS
• Any Qualifications?
• Current Performance of LPARs
• Snapshots of running LPARs
• Virtualisation Candidates
• Classes of applications
• Mix Match?
• Further investigatio

16
How many Guests per CPU

• Std VMware disclaimer
• workload dependent
• application dependent
• Extremes can vary
• very busy Host may not allow more than lt2 VMs
  per core
• very lightly loaded Host may easily run lt8 VMs
  as per core.
• Your MILAGE will vary

17
So how many should I say?

• Again
• workload dependent
• Aggressive
• Go for 4
• Conservative
• Go for lt 3
• WHY?
• Standard usage of any single application use X86
  server is very low, well in the order of lt10
• Any VM uses approx 4ltltVMltlt15 per core as
  virtualisation overhead or Tax
• This acts as a practical limit on how aggressive
  you can be on count of VMs on a per core
• Tax dependent on ESX version

18
Lets do a simple calculation

• Few assumptions -
• Headroom 18 ie populate Core with VMs lt 82
  usage of Core
• Virtualisation Layer (Tax) per VM is approx 8
• very lightly loaded-
• VM has lt2 usage gtget up to 8 VMs per Core ie
  (28)x8 80 usage
• lightly loaded-
• VM has lt12 usage gtget up to 4 VMs per Core ie
  (128)x4 80 usage
• avg load-
• VM has lt32 usage gtget up to 2 VMs per Core ie
  (328)x2 80 usage

19
Is that all that is?

• Well no, but we are making a rule of thumb for
  our scenario described
• Lets get real then
• More depth into traditional factors affecting
  performance, how VMware handles them
• More importantly how this limits the VM cnt
• CPU
• Memory
• Disk
• I/O
• Comms

20
Deeper into VM architecture
21
Agenda

• Rule of thumb for estimating of LPARS
• Any Qualifications?
• Current Performance of LPARs
• Snapshots of running LPARs
• Virtualisation Candidates
• Classes of applications
• Mix Match?
• Further investigatio

22
CPU

• A VM can be defined as single or SMP
• VM will attempt to utilise the full cpu
  allocation given with a scheduling price
• Performance is dependent on virtualised resources
  available at the time on this set of CPUs
  assigned and other resources.

23
H/w considerations

• Core type
• Single Core vs Dual Core vs Quad Servers
• Special features for Virtualisation eg AMD- V,
  Intel VT
• Memory access time
• VM access time swap time
• Dedicated Cache per core vs L1, L2 cache
• NUMA technology
• Trade-off on qty vs speed of memory
• Recommended High L2 cache amts

24
MultiCore Issues

• HT Threading on core
• Dual Core HT allows 4 threads concurrently
• Appln coding w threading achieves parallelisation
• Req balancing of CPU loads
• CPU-bound applications can potentially run twice
  as fast on dual-core processors, vs memory-bound
  appls may 50 percent faster I/O-bound applns
  may not run any faster.
• Eg two CPU intensive threads and two UI
  intensive threads, one UI thread and one compute
  thread should be handled via HT on each core.

25
SMP

• Multiple vCPU iff
• Applications is genuinely multi-threaded and
• Can make use of multiple CPUs
• OR
• Designed as many single-threaded applications
  that run simultaneously
• Configure min. count vCPU for expected load. Over
  provisioning effects schedular

26
The number of possible scheduling opportunities
for a 2-vCPU machine on a 4-P or8-P physical ESX
Server host is described by combinatorial
mathematics (http//www.vmware.com/vmtn/resources
/409,)
27
Memory Techniques

• Memory is managed to be efficient using native OS
  techniques.
• ESX ballooning
• Homogenous OS will allow more share memory and
  higher VM count
• Real memory available acts a limit to of VMs
• Cache _at_ what speed available
• Need to consider how much ESX needs (console)
  plus per VM vs the cost of paging

28
I/O

• I/O is virtualised and is not nearly as efficient
  CPU and memory virtualisation.
• All I/O is trapped by Host and performed in
  software
• introduces a performance penalty by using more
  CPU cycles
• gt more I/O more cpu cycles gt reduce VM count
• Directly connected HDD and paging speed
• Paging files, swapping space and panic
  situations
• SAN speed
• LUN optimisation
• 2 Gig NIC as HBA are worthwhile
• All above acts as limitations to VM count

29
Agenda

• Rule of thumb for estimating of LPARS
• Any Qualifications?
• Current Performance of LPARs
• Snapshots of running LPARs
• Virtualisation Candidates
• Classes of applications
• Mix Match?
• Further investigation

30
Classes of applications

• More Lightweight apps more VMs per core
• Lightweight
• eg webserver eg apache, Print servers
• Middleweight
• eg SOA with a few network connections
• Heavyweight
• Eg database server little r/w activity few
  connections
• Very Heavyweight
• Eg Application compute intensive, high I/O

31
Classes of applications (ctd)

• Lightweight
• Web servers are usually not intensive and e2e
  timings are normally good.(gt4 per core)
• Middleweight
• Java based applications performance can be
  sensitive to the JVM heap size and reading in
  pages from disk.
• Can be effected by ballooning and memory pages
  swapped to disk by Host
• Heavyweight
• Uses many CPU cycles for computation plus many
  CPU cycles for I/O eg database server w many
  client connections eg MS SQL (lt3 per core)
• Very Heavyweight
• perhaps is not a candidate for Virtualisation
• VMware ESX version dependent
• 2.5, 3.0 vs 3.5 is said better

32
Agenda

• Rule of thumb for estimating of LPARS
• Any Qualifications?
• Current Performance of LPARs
• Snapshots of running LPARs
• Virtualisation Candidates
• Classes of applications
• Mix Match?
• Further investigation

33
CPU Normal with ready 2 examples
34
Memory - Normal
35
Contrained VM ( ready high)
36
ready higher than Run
37
SMP
38
Memory-Example of Ballooning in Host
39
Memory-Example of Ballooning in VM
40
Memory-and Schedular Examples Activity over few
days
41
Agenda

• Rule of thumb for estimating of LPARS
• Any Qualifications?
• Current Performance of LPARs
• Snapshots of running LPARs
• Virtualisation Candidates
• Classes of applications
• Mix Match?
• Further investigation

42
Mix and Match

• Initially balance workload on Host by knowing
  characteristics of your guests as initially
  measured when physical server
• Eg Attempt mix of compute intensive with little
  IO together with little compute I/O intensive
• Measure and refine distribution of workload over
  time as guest
• No guarantee performance characteristics
  displayed as physical will the same as guest
• Scheduling with resource needs of multiple guests
  

43
VMmotion

• Standardise on hardware chip sets.
• VMmotion needs to swap to exactly the same h/w
  chip set/config to work. ie Xeon -\-gt AMD, 32
  bit -\-gt 64 bit, single core -\-gt dual core
• Add servers in pairs

44
Summary other comments

• If data centre only provides homogenised virtual
  containers
• Better to be conservative with VM count
• If you have intimate knowledge of guests resource
  requirements
• can afford to be aggressive with VM count
• Future is managing Hypervisors and VMs, not the
  Hypervisors themselves.
• Extra VMware Trivia
• What does ESX/GSX stand for ?
• ESX Elastic Sky X
• GSX Ground Storm X

45
Agenda

• Rule of thumb for estimating of LPARS
• Any Qualifications?
• Current Performance of LPARs
• Snapshots of running LPARs
• Virtualisation Candidates
• Classes of applications
• Mix Match?
• Further investigation

46
Blogs

• VROOM -http//blogs.vmware.com/performance/
• http//www.virtualization.info/
• http//communities.vmware.com/index.jspa
• /www.petri.co.il/ (in virtulisation Vmware forum)