Virtualization%20Trends,%20Challenges%20and%20Solutions

1
Virtualization Trends, Challenges and Solutions

• Naresh Sehgal, Ph.D., MBA
• Lead SW Architect
• Enterprise Platforms and Services Division
• Intel Corp, Bangalore
• Email naresh.k.sehgal_at_intel.com

2
Robert X. Cringely on Computers

• "If the automobile had followed the same
  development cycle as the computer
• A Rolls-Royce would today cost 100,
• get a million miles per gallon, and
• explode once a year, killing everyone inside. ?

3
Hardware Virtual Machines (VMs)
...
App
App
App
VM1
VM0
...
...
App
App
App
App
App
App
Operating System
Guest OS0
...
Guest OS1
A new layer of software...
GFX
Physical Host Hardware
VM Monitor (VMM)
Memory
Processors
Graphics
Physical Host Hardware
Keyboard / Mouse
Storage
Network
Without VMs Single OS owns all hardware resources
With VMs Multiple OSes share hardware resources
Virtualization enables multiple operating systems
to run on the same platform
4

• How long has virtualization been around?
• Recent development 5 years
• A while 10 years
• Older than Microsoft 30 years
• A lot longer..gt40 years

Would you believe 45 - 50 years?
5
Strachey Time Sharing in Large Fast Computers
Virtualization
Open source Xen is released
MIT Project MAC
Connectix is founded
Intel introducesIntel Virtualization Technology
VMWare is founded
Goldberg Survey of Virtual Machines Research
Microsoft acquires Connectix
IBM M44/44X Project
IBM MIT Compatible Time Sharing System
6
Virtualization Challenges

• Complexity
• CPU virtualization requires binary translation or
  paravirtualization
• Must emulate I/O devices in software
• Functionality
• Paravirtualization may limit supported guest OSes
• Guest OSes see only simulated platform and I/O
  devices
• Reliability and Protection
• I/O device drivers run as part of host OS or
  hypervisor
• No protection from errant DMA that corrupts
  memory
• Performance
• Overheads of address translation in software
• Extra memory required (e.g., translated code,
  shadow tables)

7
Processor Virtualization
Guest OSes run at intended rings
With VT

• Without VT

RingCompression
VMM
CPU0
Processors(StandardIA-32 or IPF)
CPUn
8
Intel Virtualization Technology (VT)
1st VT base SW Solutions
..
Virtual Machine Monitor
Processors with Intel Virtualization Technology
and others

• Intel VT
• First to market with native virtualization
  support
• Broadest HW and SW ecosystem support

• CoreTM 2 Microarchitecture based systems
• Significant increase in performance and improved
  VT performance overall segments
• Mobile - Intel Core2 Duo Mobile Processor for
  Intel Centrino Duo Mobile Technology
• Desktop - Intel Core2 Duo Desktop Processor
  E6000 sequence -
• Server Dual-Core Intel Xeon Processor 5100
  series

Get More Done On Every Server Get More
Capabilities On Client
9
Todays Uses Servers Virtualization addresses
todays IT concerns
Server Consolidation
Test and Development
VM1
VMM
HW
101 in many cases
Enables rapid deployment
Virtualization increases server utilization,
simplifies legacy software migration
10
Emerging Server Usage ModelsTrue Lights Out
Datacenter
Dynamic Load Balancing
Disaster Recovery
VM1a
VM1b
VM1
VMM
VMM
VMM
HW
HW
HW

• Upholding high-levels of business continuity

• Balancing utilization with head room

Intel Virtualization Technology will play
anintegral role on the next generation of VMMs
11
Emerging Business Usage Models
Built-in Management
ProactiveSecurity
Professional Business Platform
Energy Efficient Performance
Intel Platform Software
12
vProTM Key Features
Remotely Manageability - Repair down
systems Securely update systems Audit
powered-down PCs Prevents malicious packets
from entering the OS
Supported by over 45 OEMs, ISVs, IT Outsourcers
More details in the IDF vProTM tracks
13
Intel Virtualization and Intel vPro
technology
Uses Intel VT for creating a separate
independent hardware-based environment inside of
the PC Service Partition Allowing IT
administrators to create a dedicated and tamper
resistant service environment or partition where
tasks can run independently and isolated from the
main operating system as well as from the end
user User partition - OS and application
.
VM0
VM1
User Partition
Service Partition
Stack owned and managed by IT dept protected
from users
Firewall Application
Management Application
App0
App1
Appn
Service OS(WinCE or Linux)
User OS (Win2K, XP)
Lightweight VMM (LWVMM)
VT AMT
Intel architecture Platform
Help desk or console access even when user
partition is down
Intel, the Intel logo, and Intel architecture are
trademarks or registered trademarks of Intel
Corporation or its subsidiaries in the United
States and other countries.
14
Intel Virtualization Technology Evolution

• Standards for IO-device sharing
• Multi-context I/O devices
• Endpoint device translation caching
• Under definition in the PCI-SIG IOV

Vector 3I/O Focus
Vector 2Platform Focus
Vector 1Processor Focus
VMMSoftware Evolution

• Software-only VMMs
• Binary translation
• Paravirtualization

Simpler and more Secure VMM through foundation of
virtualizable ISAs
PastNo HardwareSupport
Today
VMM software evolution over time with hardware
support
Other names and brands may be claimed as the
property of others
15
Options for I/O Virtualization

• Pro High Performance
• Pro I/O Device Sharing
• Pro VM Migration
• Con Large Hypervisor

• Pro Higher Performance
• Pro Rich Device Features
• Con Limited Sharing
• Con VM Migration Limits

• Pro Higher Security
• Pro I/O Device Sharing
• Pro VM Migration
• Con Lower Performance

VT Goal Support all 3 Models
16
VT-d Overview

• VT-d provides infrastructure for I/O
  virtualization
• Defines architecture for DMA and interrupt
  remapping
• Common architecture across IA platforms
• Will be supported broadly across Intel chipsets

Other names and brands may be claimed as the
property of others
17
How VTd works?

• Each VM thinks it is 0 address based
• GPA (Guest Physical Address)
• But mapped to a different address in the system
  memory
• HPA (Host Physical Address)
• VTd does the address mapping between GPA and HPA
• Catches any DMA attempt to cross VM memory
  boundary

18
DMA Remapping Hardware Overview
DMA Requests
Device ID
Virtual Address

Length
DMA Remapping Engine
Memory Access with Host Physical Address
19
VT-d Applied to Hypervisor Model

• Improved Reliability and Protection
• Hypervisor programs remap tables
• Errant DMA is detected by hardware and reported
  to hypervisor / device driver
• Bounce Buffer Support
• Limited DMA addressability in I/O devices limits
  access to high memory
• Bounce buffer is a software technique to copy
  I/O buffers into high memory
• VT-d eliminates need for bounce buffer
• Above equally useful for standard OSes
• VT-d does not require a VMM to function

Pro Higher Performance Pro I/O Device
Sharing Pro VM Migration Con Larger Hypervisor
20
VT-d Applied to Service VM Model

• Device Driver Deprivileging
• Device drivers run above hypervisor as part of a
  Service OS
• Guest device drivers program devices in
  DMA-virtual address space
• Service VM
• Forwards DMA API calls to hypervisor
• Hypervisor sets up DMA-virtual to host-physical
  translation
• Further Improvements in Protection
• Guest device driver cannot compromise hypervisor
  code or data

Pro High Security Pro I/O Device Sharing Pro
VM Migration Con Lower Performance
21
VT-d Applied to Pass-through Model

• Direct Device Assignment to Guest OS
• Guest OS directly programs physical device
• For legacy guests, hypervisor sets up guest- to
  host-physical DMA mapping
• For remapping aware guests, hypervisor involved
  in map/unmap of DMA buffers
• PCI-SIG I/O Virtualization Working Group
• Activity towards standardizing natively sharable
  I/O devices
• IOV devices provide virtual interfaces, each
  independently assignable to VMs

Pro Highest Performance Pro Smaller
Hypervisor Pro Device-assisted sharing Con VM
Migration Limits
22
DMA Remapping IOTLB Scaling

• Address Translation Services (ATS) extensions to
  PCIe enable IOTLB scaling
• ATS endpoint implements Device IOTLBs
• Device-IOTLBs can be used to improve performance
• E.g., Cache only static translations (e.g.
  command buffers)
• Pre-fetch translations to reduce latency
• Minimizes dependency on root-complex caching
• Support device-specific demand I/O paging

Other names and brands may be claimed as the
property of others
23
Address Translation Services (ATS)

• ATS Translation Flows
• Device issues Translation Requests to
  root-complex
• Root-complex provides Translation Response
• Device caches translation locally in Device
  IOTLB
• Devices can issue DMA with translated address
• Translated DMA from enabled devices bypass
  address translation

Translated DMA Request
Device IOTLB
DMA using Translated Address
VT-d supports per-device control of ATS
Other names and brands may be claimed as the
property of others
24
Invalidation Architecture

• Invalidation enforces consistency of caches
• Required when software updates translation
  structures
• Invalidation primitives
• Global, domain-selective, and page-range
  invalidations
• Support for Device-IOTLB invalidation (through
  ATS)
• Invalidation software interfaces
• Synchronous interface through MMIO registers
• Queued interface through invalidation queue

25
ATS Invalidations

• ATS Invalidation Flow
• Root-complex issues invalidation request to
  device
• Device invalidates specified mappings from Device
  IOTLB
• Device issues Invalidation response
• Invalidation details
• Invalidation request contains unique Invalidation
  Tag
• Invalidation Responses may be coalesced

Other names and brands may be claimed as the
property of others
26
Mapping to VMM Software Challenges
VM0
VM1
VM2
VMn
VM0
VM2
VirtualMachines(VMs)

Apps
Apps
Apps
Apps
OS
OS
OS
OS
Higher-level VMM FunctionsResource Discovery /
Provisioning / Scheduling / User Interface
VMM(a.k.a.,hypervisor)
Processor Virtualization
Memory Virtualization
I/O Device Virtualization
BinaryTranslation
RingDeprivileging
Page-tableShadowing
I/O DMARemapping
InterruptRemapping
I/O DeviceEmulation
CPU0
CPU0
Storage
PhysicalPlatformResources
CPUn
CPUn
Network
Memory
Processors
I/O Devices
27
Example 6 Virtualization overhead on Intel
experimental client VMM (vs. Native OS)

• Relatively low Virtualization overheads for
  client benchmark
• Targeting lt10 overhead with improved SW
  techniques
• Further VMM SW optimization and Next generation
  VT features to reduce virtualization overheads

Pre beta version
Source Intel Corporation Projections and
technical specifications are based on internal
analysis and subject to change
28
Summary A better IA platform

• First to Market Massive Ecosystem Support

• Choice Broadest virtualization software support
  in the industry

• Robust First x86 hardware assisted
  virtualization technology (Intel VT)
• Innovation common specification enhanced
  virtualization on x86 and will set the standard
• Flexibility Leverage Intel Xeon
  processor-based servers widely deployed
  infrastructure for advanced failover and dynamic
  load balancing

• Better Platform Reliability
• Critical for more applications on the same server
• More reliability features
• Proven Platform Architecture - almost 40X more IA
  based servers than AMD based since 19961

Choose the right basket

• Performance Headroom
• Intel Xeon processors have key performance
  features for virtualization dual-core,
  hyper-threading, I/O, memory, and larger caches

1 source Q405 IDC server Tracker, 1996-2005
total system shipped
Whitepaper on Virtualization benefits
http//www.intel.com/business/bss/products/server/
virtualization_wp.pdf
29
Backup

• Q A

30
Example 1 SysBench Running with VMwares ESX
Server
Significant performance lead over competition
Source Principled Technologies (PT) performance
report http//www.principledtechnologies.com/clie
nts/reports/Intel/VMSysBench0706.pdf
System configuration in backup foils
31
Example 2 SPECjbb Running with VMwares ESX
Server
performance lead over competition
Other names and brands may be claimed as the
property of others Source Principled
Technologies (PT) performance report Comparing
Dual-Core AMD Opteron 285 with Dual-Core Intel
Xeon Processor 5160
32
Example 3Microsoft Virtual Server
VMM Microsoft Virtual Server 2005 R2 SP1 Java
JFT workload Guest OS - Windows 2003 Enterprise
Edition R2 (32 bit) Benchmark - JVM BEA
WebLogic x (build R26.0.0-188-52875-1.5.0_04-200
5110-0920-linux-x86_64) Systems HP DL385 2
AMD Opteron 2.6GHz 2x1MB Intel Dual-Core Intel
Xeon Processor 3.0G SuperMicro SDP 16x1GB
Up to 53 gain
Source Intel Corporation Projections and
technical specifications are based on internal
analysis and subject to change
Other names and brands may be claimed as
property of others. System Configuration details
in backup. Performance tests and ratings are
measured using specific computer systems and/or
components and reflect the approximate
performance of Intel products as measured by
those tests. Any difference in system hardware or
software design or configuration may affect
actual performance. Buyers should consult other
sources of information to evaluate the
performance of systems or components they are
considering purchasing. For more information on
performance tests and on the performance of Intel
products, visit http//www.intel.com/performance/r
esources/limits.htm or call (U.S.) 1-800-628-8686
or 1-916-356-3104.
33
Example 4 Energy Efficient Performance
Intel CoreTM 2 Duo based system provide Energy
Efficient Performance (EEP ) Leadership in
virtualized environment
Source Intel Corporation Projections and
technical specifications are based on internal
analysis and subject to change
34
Example 5 MS VS SpecJBB 2005
Host OS Microsoft Server 2003 X64 Enterprise Edition SP1 RTM
Virtualization Microsoft Virtual Server R2 Beta SP1 ver. 1.1.512.0 EE Drop B1036 vmm.sys Microsoft Virtual Machine Windows Guest Editions ver. 13.705
Guest OS RedHat V9 2.4.20-8 kernel (32-bit)
Workload SpecJBB 2005
1.18X
1.66X
1.95X
Intel XEON SW Virtualized guest performance is
1.66x of Opteron
Intel XEON VT performance is 1.18x of Software
(no VT) Intel XEON
Intel XEON VT performance is 1.95x of Opteron
SW (no Pacifica)
Source Intel Corporation Projections and
technical specifications are based on internal
analysis and subject to change