Hypervisors

1
Hypervisors

• Orran Krieger

2
Hypervisors will be pervasive for commercial
systems

• Server consolidation.
• Incremental upgradeability of HW and SW.
• Testing for new deployments.
• Functions like RAS implemented just once.
• Security.
• Quality of service.
• Virtualization commodity and ubiquitous. PHYP,
  zSeries, VMWare, Microsoft Virtual PC Virtual
  Server, UML, coLinux, Xen, Denali, L4, Jaluna,
  rHype, Virtual IRON

3
My position in the mid 90s

• Demonstrates limitations in current OSes
• Server consolidation because the OSes cant
  isolate the workloads.
• Scalable apps on non-scalable OSes (Disco)
  Fault containment (Cellular Disco)
• Rt-Linux to isolate real time.
• Keeps the OS from direct access to the HW.
• Large grained partitioning of resources result in
  inefficiencies.
• Makes fine grained sharing difficult.
• Requires configuration and management of multiple
  OSes.
• Hypervisors are for weenies

4
Started dealing with Console vendor

• Game comes with its own customized OS.
• Absolutely deterministic performance.
• OS can never be upgraded.
• But. Need persistent storage, network access,
  general applications
• Solution Hypervisor with General purpose Linux
  and ability to start games in their own
  domain/partition.
• We implemented IBMs Research Hypervisor
  (rHype).
• Research platform for IBMs PERCS project.
• Supports IBMs PAPR interface used by PHYP on
  GP-UL Runs K42 and existing Linux distroes.
• Supports modified Linux on non-hypervisor mode
  PPC.
• Released Research Hypervisor in February under
  GPL and now working on Xen PPC.

5
Value for HEC/FASTOS (1)

• Enable at-scale evaluation and testing before
  production deployment.
• Multiple OSes on a platform
• App doesnt have to be written to least common
  denominator improved productivity, and improved
  performance.
• Enable ISV community.
• Can share machine with commercial environment.
• Enables high level of security, e.g., virtual
  cluster provided to public.
• Isolation from OS perturbations.
• Services can be implemented once for all OSes,
  e.g., checkpoint/restart, migration, power
  management, network virtualization
• Resource management, e.g., interactive
  supercomputing.
• Scalability and fault containment on SMMP.

6
Value for HEC/FASTOS (1)

• Enabling HW innovation.
• Perform once non-performance critical operations
  like HW initialization/configuraiton and
  virtualize non-performance critical HW.
• Security services, e.g., introspection, can be
  provided outside the OS, also verify OS
  invariants continuously.
• Out-of-body debugging.
• Enable wacky/library/customizable/light-weight
  OSes
• Can be deployed on real at-scale HW.
• Dont have to solve all the problems (e.g.,
  ACPI).
• Can be moved to new platforms.

7
Example 1 real mode application (PERCS HPCS)
8
Current Prototype (Parent/Child)
Ethernet
mpifs
application
Disk Partition
netfs
open read write close
File System
u9fs
devcons
libfs
tcp/ip
Shared Memory
in channel
out channel
9
Areas of Opportunity
App
9P Enabled Application/Service
Host A
Libraries
9P Library Interface/Interposer
File System

Sockets
Local FS
NFS
Direct 9P Sockets/Pipes

LVM
TCP
UDP
SCTP
OS N
9P Exported Networking Stack

IP
SCSI DD

Storage Adapter DD

NIC DD
9P Exported Device Drivers
Operating System 1
Hypervisor

PCI Bridge or Switch
9P exporting self-virtualizing hardware
Various StorageLinks Options(pSCSI, SAS,SATA,
iSCSI)
StorageHBAs
10
Example 2 Linux on compute nodes

• Limited system call support
• Must marshal arguments and send to linux
• Layer to convert to Linux system calls

Compute node
I/O node

• Map FASTOS memory into Linux app
• System calls reflected into Linux
• All system calls can be supported
• Continue to avoid TLB, get deterministic
  performance

Compute node
11
Implementation issues

• With right HW, system calls go directly to OS,
  otherwise trap reflection.
• Options for page tables
• Paravirtualized interface to hypervisor owned
  page table.
• Page tables cached by hypervisor.
• Writable page tables.
• Hypervisor needs to own some timer interrupt.
• Interrupts likely go through hypervisor and
  forwarded to right OS.

12
IO hosting
Logical Partition
Logical Partition
Logical Partition
Logical Partition
Hypervisor
Hardware Platform
13
Device Partitioning with IOMMU
Logical Partition
Logical Partition
Logical Partition
Logical Partition
Hypervisor
Hardware Platform
14
Self virtualizing devices
Logical Partition
Logical Partition
Logical Partition
Logical Partition
Hypervisor
Hardware Platform
15
What we need to do

• Xen is the obvious choice.
• We need to help drive definition of hypervisor
  before it becomes too mature.
• Investigate costs of their design decisions, and
  fix
• Need to drive definition of I/O virtualization
  and self-virtualizing devices.
• Determine set features we can use in common,
  e.g.
• One implementations of checkpoint/restart/migratio
  n,
• Gang scheduling of partitions.
• Hypervisor as a base is close to ready, making it
  first class platform for HEC will take
  investments..