Virtual Machine Monitors

1
Virtual Machine Monitors
2
Bibliography

• Virtual Machine Monitors Current Technology And
  Future Trends, Mendel Rosenblum and Tal
  Garfinkel, IEEE Computer, May 2005
• Xen and the Art of Virtualization, P. Barham,
  R. Dragovic, K. Fraser, S. Hand, T. Harris, A
  Ho, R. Neugebauer, I. Pratt, A. Warfield, SOSP
  03.
• The Definitive Guide to the Xen Hypervisor, David
  Chisnall, Prentice Hall, 2008.
• Scale and Performance in the Denali Isolation
  Kernel, Andrew Whitaker, Marianne Shaw, and
  Steven D. Gribble, in System Design and
  Implementation (OSDI), Boston, MA, Dec. 2002.
• Denali Lightweight virtual Machines for
  Distributed and Networked Applications, Andrew
  Whitaker, Marianne Shaw, and Steven D. Gribble,
  Proc. USENIX annual Technical Conference, June
  2002.
• Xen Homepage http//www.cl.cam.ac.uk/research/srg
  /netos/xen/
• VMWare http//www.vmware.com/products/esx/

3
Outline

• Overview
• What is a virtual machine?
• What is a virtual machine monitor (VMM)?
• System or application (process) virtual machines
• History of Virtual Machines
• Benefits of Virtual Machines
• Issues and Implementation
• Examples

4
What is it? (1)

• What is virtualization? an abstraction or
  simulation of hardware resources
• e.g., virtual memory
• A virtual machine is an isolated environment that
  appears to be a whole computer, but actually only
  has access to a portion of the computers
  resources.

5
What is it? (2)

• A virtual machine monitor (VMM) is the software
  layer that supports one or more virtual machines
• Each VM appears to run on bare hardware, giving
  the appearance of multiple instances of the same
  computer, but all run on a single machine.
• VMM is also called a hypervisor
• Guest operating system an operating system that
  runs on a VMM rather than directly on the
  hardware.

6
System Process VMshttp//en.wikipedia.org/wiki/
Virtual_machine

• System virtual machine (hardware virtual machine)
  See previous definitions
• Provides a complete system
• Each VM can run its own OS, which in turn can run
  multiple applications
• Process or application virtual machine e.g., JVM
• Runs inside (under the control of) a normal OS
• Provides a platform-independent host for a single
  application

7
System Virtual Machines

• Traditional VMM is a thin software layer that
  runs directly on the host machine hardware
• Main advantage/objective performance
• VMWare ESX, ESXi Servers, Xen, OS370, Denali
• Also called a bare metal VMM
• Hosted VMM runs on top of an existing OS.
• Main advantage easier to build easier to
  install
• Examples User-mode Linux
• Hybrid shares the hardware with existing OS
• Example VMWare Workstation

8
Application Guest OS1
Application Guest OS2
Application Guest OS3
VM1
VM2
VM3
Virtual machine layer - VMM Hardware layer
Traditional VMM
9
Hybrid Rosenblum Garfinkel Fig. 2
VM1
VM2
VMM
App
App
App
Operating system
I/O VMM
Guest OS
Hardware layer
Host OS
VMM
Hosted
Hardware Layer
10
Hosted/Hybrid versus Non-hosted VMM

• Hosted has 3 advantages 1
• VMM is no harder to install than any other
  application
• The VMM can use the host OS scheduler, pager,
  etc. and focus primarily on isolation
• I/O support is better the VMM can use the device
  drivers that are designed to work with the host
  OS rather than having to provide its own.

11
Hosted versus Non-hosted VMM

• Disadvantage 1
• I/O overhead is greatly increased requests go
  from guest OS to VMM to host OS and down
  eventually to the device driver.
• Too inefficient for servers
• More difficult to provide complete isolation, so
  not appropriate for servers from a security
  perspective.

12
Hosted v Non-hosted VMM

• Conclusion
• Hosting is a good approach for individual work
  stations reduces effort needed to get VMM up and
  running.
• Hosting is not advisable for servers. Security
  issues are the most important concern, followed
  by added overhead for I/O.

13
VM How They Work (1)

• VMM runs in kernel mode (replacing tradtional OS)
• Guest OS runs in user mode
• Some modern hardware has a third mode for the
  guest OS
• For the most part, applications run normally and
  execute machine code directly (direct execution)
• What about system calls?

14
VM How They Work (2)

• The guest OS runs in user mode how can it
  execute privileged code?
• It cant. When it tries to execute a privileged
  instruction, the VMM traps the operation, and
  performs the system call in place of the guest OS
  
• e.g., when a guest OS appears to execute an I/O
  system call, the VMM is actually in charge.

15
Virtualization versus Emulation

• Virtualization presents multiple copies of the
  same hardware system.
• Direct execution of code on the hardware
• Emulation presents a model of another hardware
  system
• Instructions are emulated in software much
  slower than virtualization
• Example Microsofts VirtualPC could run on other
  chipsets than the x86 family used on Mac
  hardware until Apple adopted Intel chips

16
Full Virtualization versus Paravirtualization

• Full virtualization each virtual machine runs on
  an exact copy of the actual hardware.
• Paravirtualization each virtual machine runs on
  a slightly modified copy of the actual hardware
• Because some aspects of the hardware cant be
  virtualized (see examples later)
• To present a simpler interface improve
  performance.

17
History - Why VMMs?

• Early computers were large (mainframes) and
  expensive
• VMM approach allowed the machine to be safely
  multiplexed among many different applications
• An alternative to multiprogramming

18
Virtual Machines - History

• Early example the IBM 370
• VM/370 is the virtual machine monitor
• As each user logs on, a new virtual machine is
  created
• CMS, a single-user, interactive OS was commonly
  run as the OS
• Separation of powers
• Virtual machine interacts with user applications
• Virtual machine monitor manages hardware resources

19
History 1980s 1990s

• As hardware got cheaper and operating systems
  became better equipped to handle multitasking,
  the original motivation went away.
• Hardware platforms gradually eliminated hardware
  support for virtualization.
• And then

20
History late 90s to today

• Massively parallel processors (MPPs) were
  developed during the 1990s they were hard to
  program and did not support existing operating
  systems
• Researchers at Stanford used virtualization to
  make MPPs look more like traditional machines
• Other research groups explored different
  approaches to VMs
• Result today, virtual machines are very common

21
Example Virtual Machine Systems

• VMware commercial products, derived from
  research done at Stanford
• Xen open source, Cambridge University, widely
  used in research and academia xen.org
• Denali University of Washington, focused on
  support for Internet services

22
VMware

• VMware, a publicly held company, founded by
  Stanford developers
• Two lines of products
• Desktop a range of products advertised as a
  way for corporations to migrate and upgrade
  operating systems from a centralized IT center
• VMware ESXi Server is the most recent product in
  this line is a bare-metal hypervisor

23
Xen

• Xen open-source VM system for x86, Itanium, ARM
  others
• Originated at Cambridge University Computer Lab
• Now supported as an open-source product that has
  destktop, server, and cloud capabilities (Amazon
  uses it for its cloud services.)
• Designed to support execution of Linux, other
  Unix-like systems (Solaris, BSD), Windows
  simultaneously on the same platform
• Objective of original project efficient hosting
  of up to 100 virtual machines

24
Denali

• Research project U of Washington
• Time frame 2001-2004.
• Problem addressed hosting Internet services
  economically
• Goal to allow new, untrusted, services to be
  hosted on third-party servers.
• Protection provided by VM concept lets servers
  safely host multiple different services.
• Encapsulation lets services be swapped in and out
  of memory easily so multiple services can share
  one machine

25
Reasons for Adopting VMMs

• Flexibility in choice of operating system
• Encapsulation A VM collects together an
  operating system, a complete (virtual) computer
  system, and one or more applications into a
  single unit that can be treated like any other
  software application.
• Can be saved to a file, for example
• Security and isolation provided by encapsulation

26
Security and Isolation

• Applications running on a virtual machine are
  more secure than those running directly on
  hardware machines.
• VMM controls how guest operating systems use
  hardware resources what happens in one VM
  doesnt affect any other VM.
• OS level security is more vulnerable than VM
  security

27
OS Flexibility

• Support several operating systems at the same
  time on a single hardware platform
• Ability to experiment with new operating systems,
  or modifications of existing systems, while
  maintaining backward compatibility with existing
  systems.

28
Encapsulation

• Conventionally, servers ran on dedicated
  machines.
• Protects against another server/application
  crashing the OS
• But wasteful of hardware resources
• VMM technology makes it possible to support
  multiple servers, each running on its own VM, on
  a single hardware platform
• Rosenblum and Garfinkel 1 point out that this
  makes it possible to suspend and resume entire
  virtual machines even move to other platforms
• For load balancing, system maintenance, etc.

29
Desirable Qualities

• A good VMM
• Doesnt require applications to be modified
• Doesnt severely affect performance
• Is not complex/error prone

30
Implementation Issues

• Virtualize CPU
• Guest OS runs as if it is executing directly on
  the hardware CPU, but it isnt
• Virtualize memory
• Guest OS thinks it is managing memory directly,
  but it isnt
• Paravirtualization versus binary translation
• Hardware-assisted virtualization

31
CPU Virtualization

• Basic technique direct execution
• As long as it is executing unprivileged
  instructions the virtual machine (guest OS
  applications) executes hardware instructions
  directly.
• If the guest OS tries to execute a privileged
  instruction the CPU traps to the VMM which
  executes the privileged operation.
• VMM runs in privileged (kernel) mode, guest OS
  runs in user mode.

32
Example Disable Interrupts 1

• If a guest OS tries to disable interrupts, the
  instruction is trapped by the VMM which makes a
  note that interrupts are disabled for that
  virtual machine
• If interrupts arrive for that machine, they are
  buffered at the VMM layer until the guest OS
  enables interrupts.
• Other interrupts are directed to VMs that have
  not disabled them.

33
Direct Execution Not Always Possible

• Modern CPUs, esp. x86 architectures, have not
  been designed for virtualization.
• Example POPF (pop CPU flags from stack)
• If executed in user mode, no trap its just
  ignored by the hardware
• In this case, direct execution fails Guest OS
  assumes flags have been popped, but they havent
  been because the VMM isnt notified.

34
Two Ways to Handle Non-virtualizable Instructions

• Paravitualization
• Xen, Denali
• Binary Translation
• VMware
• Both use the same basic approach catch
  non-virtualizable instructions and emulate them
  in software at the VMM level.

35
Paravirtualization

• Rewrite portions of the guest OS to replace
  non-virtualizable instructions with a trap the
  VMM, which emulates the instruction on behalf of
  the guest OS
• e.g., remove POPFs substitute something else
• Paravirtualization affects the guest OS, but not
  applications that run on it the API is
  unchanged
• Paravirtualization is also used sometimes to
  replace inefficient operations with more
  efficient ones.

36
Binary Translation

• Instead of modifying the OS, detect these
  instructions at runtime.
• VMwares approach The DBT (dynamic binary
  translator) controls execution of kernel code -
  replaces non-virtualizable instructions with
  equivalent code that can be virtualized.
• Once translated, code is saved and used again if
  needed.

37
Comparison

• Paravirtualization changes the source code of a
  guest OS binary translation changes the binary
  code as it executes.
• Paravirtualization is more efficient, but
  requires modification to the guest OS
• Paravirtualization also allows more efficient
  interfaces, in some cases
• Binary translation is backward-compatible but has
  some extra overhead of run-time translation the
  first time an instruction is encountered.

38
Hardware-assisted Virtualization

• AMD-V and Intel VT are architecture extensions to
  support virtualization.
• New execution modes
• Allows guest OS to run in execution ring 0 and
  VMM in yet a higher privileged mode
• Flags to indicate if running in this mode
• Essentially, the trap and emulate mode used in
  paravirtualization or binary translation is now
  done in hardware.
• Does away with need to modify guest OS is faster
  than binary translation.

39
Memory Virtualization

• VMM maintains a shadow page table for each
  virtual machine.
• When the guest OS makes an entry in its own page
  table, the VMM makes the same entry in the shadow
  table.
• Shadow page table points to actual page frame
• The hardware MMU uses the shadow page table when
  it translates virtual addresses.

40
Challenges

• Let the guest OS decide which of its pages to
  swap out
• VMwares ESX Server uses the concept of a balloon
  process, running inside the guest OS 1.
• When the VMM wants to swap out pages from a VM it
  notifies the balloon process to allocate more
  memory to itself.
• The guest OS must page out unused portions of
  other processes to its virtual disk.
• The VMM now knows which pages the guest OS thinks
  it can do without.

41
Other Virtual Memory Challenges

• To share or not to share pages across VM
  boundaries
• VMware tracks duplicate pages in different
  virtual machines stores only one copy of the
  actual page with pointers from the shadow page
  tables in sharing processes.
• Copy-on-write policy
• Xen focuses on total isolation of each virtual
  machine, which means no sharing

42
Summary Review (1)

• A virtual machine is a copy of a real machine
• Applications dont know if they are running on
  real or virtual hardware, other than having fewer
  resources.
• A virtual machine is isolated if several VMs
  execute on the same hardware they do not interact
  with each other directly or indirectly.
• The performance of a virtual machine should be
  about the same as that of the actual hardware.
• So most instructions should be directly executed
  by the hardware as opposed to being emulated.

43
Summary and Review (2)

• Process virtual machines (JVM) virtualize at a
  higher level, do not necessarily even correspond
  to real machines.
• System virtual machines virtualize at the level
  of the hardware-software interface
• Variations of classic system virtual machine
• Hosted (run on another operating system
• Emulation (provides virtual hardware and OS, as
  in Virtual PC) not really a virtual machine

44
Summary Review (3)

• Virtual Machine Monitor (hypervisor) runs on a
  bare machine, implements one or more virtual
  machines.
• The VMM allocates resources and controls resource
  sharing among all VMs
• Operation
• Each VM runs a guest OS
• VMM runs in kernel mode
• Guest OS and applications run in user mode
• Privileged instructions trap to the VMM
• Hypercalls (the VMM equivalent of system calls)
  may be used by a guest OS to request service from
  the VMM

45
Summary Review (4)

• Benefits of VM technology for non-hosted VMs
• Isolation and security
• Multiple servers on a single machine
• Encapsulation of an entire environment OS and
  application for the purpose of
• Migration
• Checkpointing
• Supporting system maintenance
• Running several OSs concurrently
• Older versions, experimental systems, Linux
  Windows,
• For hosted VMs, the major advantage is the
  ability to run two or more OSs at once

46
Appendix Examples

• Xen
• Denali
• Hardware Virtual Machines

47
Xen Intro

• Claim virtualization is better than
  multi-tasking as a way to share hardware.
• CPU requests, memory demand, disk accesses, other
  resource needs of one process impact the
  performance of other processes
• Xen solution multiplex resources at the OS level
  instead of the process level.

48
Domain 0 guest has privileged access to the Xen
hypervisor and can be used by the system
administrator to manage the system. Separation
of powers Xen only has to worry about
multiplexing hardware to multiple guests
Domain 0 Guest
Application Domain U Guest OS2
Application Domain U Guest OS3
VM1
VM2
VM3
Xen Hardware layer
Xen implementation of VMM
49
Xen Design Principles

• Virtualize all architecture features that are
  required by standard binary interfaces.
• To support existing applications without
  modification
• Support multi-application guest operating systems
• Use paravirtualization to get improved
  performance and resource isolation

50
Xen HVM (Hardware Virtual Machine)

• Some versions of Xen are designed to run on Intel
  VT and AMD-V chips with special virtualizing
  hardware.
• Able to run un-modified (no para-virtualization)
  operating systems. This implementation is known
  as a hardware virtual machine.
• Windows requires an HVM environment Linux,
  Solaris, and BSD systems dont.

51
Xen Memory Management

• Unlike VMWare and Denali, Xen expects the guest
  OSs to manage their own hardware page tables.
• To support this, each VM receives a fixed
  allocation of page frames which it can use as it
  wishes.
• New page tables must be registered with Xen and
  updates must be validated by Xen.
• Make the page table write protected.

52
Xen CPU Management

• Xen is designed for the X86 architecture which
  supports 4 rings, or privilege levels.
• Traditional OSs execute in ring 0 (most
  privileged) and applications in ring 3 (least)
• Xen executes in ring 0 (only level that can
  execute privileged instructions)
• Guest OS runs in ring 1, which isolates it from
  applications.
• Note since this paper was written there have
  been some modifications to X86 to better support
  virtualization.

53
Xen CPU Management

• Privileged instructions must be validated (is it
  OK?) and executed by Xen
• Exceptions (page faults, system calls, other
  traps to OS) are handled as much as possible by
  the guest OS.
• Exception handlers are registered validated
  with Xen
• System calls stop at the guest OS Xen is
  involved only if the OS executes a privileged
  instruction.

54
Denali Isolation Kernel

• Authors define Denali as a small-kernel operating
  system with similarities to microkernels and
  exokernels
• Once thought to be inefficient, modern hardware
  has improved performance of this kernel
  architecture
• They expected Denali to support multiple (up to
  10,000) untrusted applications that are virtually
  independent.

55
Isolation Kernel Design Principles

• Expose low-level resources rather than high-level
  abstractions for greater security
• Avoid layer-below attacks
• Prevent direct sharing by exposing only private,
  virtualized namespaces
• Keeps one VM from even naming the resources of
  another VM, let alone modifying them. 4

56
Isolation Kernel Design Principles

• Design for scalability
• Be able to support a work load that has a few
  popular services and many that are accessed
  infrequently.
• Modify the virtualized architecture for
  simplicity, scale and performance.
• Paravirtualization for reasons other than
  necessity.
• They do not believe isolation depends on
  providing an exact copy of hardware so they
  provide a hardware version that is modified to be
  more efficient and secure.

57
Zipfs Law

• Given a table that ranks something on the basis
  of its frequency of occurrence, Zipfs law states
  that the most frequent item occurs about twice as
  often as the next most frequent item, which in
  turn occurs twice as often as the next item, and
  so on.
• Zipf made this observation about words in a
  natural language. Here, were talking about
  accesses to various web services.

58
Statistically Multiplexing Services

• Studies showed that the popularity of most
  network services (server requests, document
  searches, etc) followed a Zipfian distribution.
• Implications
• Most requests go to a small number of services
• Most services arent popular, but the total
  number of requests for unpopular services is
  non-trivial
• With isolation it can be safe and efficient to
  run hundreds or even thousands of services
  concurrently on a single platform.

59
Proof-of-concept

• Denali is the virtualized architecture
• Yakima a VMM which was designed to run in ring 0
  on x86 hardware.
• Ilwaco a simple prototype guest OS which
  provides a full set of abstractions to its
  applications while hiding the Denali architecture
• Reasonable performance in tests
• 1.4 µsec to 9 µsec context switch time,
  depending on number of VMs
• End-to-end run times of network apps were
  comparable to those of a traditional operating
  system.

60
Conclusion

• The Denali research project terminated in the
  mid-2000s.
• The Denali research group was right in supposing
  that virtual machine technology would be most
  useful today to enable efficient use of server
  hardware.
• Multi-core computing the MPP of the future? How
  useful will VMM concepts be?