Virtual Workspaces State of the Art and Current Directions

1
Virtual WorkspacesState of the Art and Current
Directions
Borja Sotomayor University of Chicago (Dept. of
CS)? borja_at_cs.uchicago.edu
2
Index

• Virtual Workspaces
• Use Cases
• GT4 Workspace Service
• Current Directions

3
Index

• Virtual Workspaces
• Use Cases
• GT4 Workspace Service
• Current Directions

4
What is a workspace?

• What do I do if I want to run something in the
  Grid?
• We have to encapsulate our work in a job
  abstraction.
• This abstraction imposes a lot of restrictions on
  the resource consumer.
• For example Resource consumers can specify what
  libraries their jobs require, but they (usually)
  can't have those libraries dynamically installed
  when the job is submitted. If the library is not
  already available in the resources, they
  generally have to make a prior arrangement with
  resource providers to make sure that software is
  installed.
• However, these restrictions are usually perfectly
  acceptable.
• But... what if they're not?

5
What is a workspace?

• In some cases, resource consumers need execution
  environments to be dynamically deployed on remote
  resources.
• The virtual workspace is an abstraction for such
  an execution environment.

Ideal environment is deployed and can be managed
Dream up an ideal environment for your jobs (or
anything else!)?
6
What is a workspace?

• A virtual workspace is an execution environment
  that can be deployed dynamically and securely on
  the Grid.
• Dynamic Started and stopped on an on-demand
  basis.
• Secure Environment encapsulated inside the
  workspace and the user submitting the workspace
  are both trustworthy.
• Of course, this is not an entirely new idea... It
  is possible to create custom execution
  environments by
• Dynamically setting up cluster nodes
• CoD http//www.cs.duke.edu/nicl/cod/
• bcfg http//trac.mcs.anl.gov/projects/bcfg2/
• Providing access to existing installation
• Dynamic Accounts http//workspace.globus.org/da/
  
• Refining site configuration
• Pacman http//www.archlinux.org/pacman/

7
What is a workspace?

• Main two aspects of workspaces
• Environment definition Users get exactly the
  (software) environment they need.
• Resource allocation Provision and guarantee all
  the resources the workspace needs to function
  correctly (CPU, memory, disk, bandwidth,
  availability), allowing for dynamic renegotiation
  to reflect changing requirements and conditions
• Existing implementations either don't provide
  both, or...
• Quality of life Setting up a new software
  environment takes a long time, and still doesn't
  give the resource consumer full control.
• Quality of service Little or no enforcement.

Quality of Life
Quality of Service
8
What is a workspace?
What magic spell can we use? Virtual Machines
are a promising vehicle to achieve higher
quality workspaces
9
Virtual Machines

• VM technology allows multiple virtual machines to
  run on a single physical machine.

App
App
App
App
App
Xen
Guest OS (Linux)?
Guest OS (NetBSD)?
Guest OS (Windows)?
VMWare
UML
Virtual Machine Monitor (VMM) / Hypervisor
Denali
Hardware
etc.
Performance Paravirtualization (e.g. Xen) is
very close to raw physical performance
10
Virtual Machines andGrid Computing

• VMs have several qualities that make them an
  appealing technology in Grid systems
• Security and isolation
• Customization of execution environment
• Resource control
• Site independence
• Workspaces can be encapsulated in VM images, and
  dynamically deployed on VMM-enabled sites.

11
Index

• Virtual Workspaces
• Use Cases
• GT4 Workspace Service
• Current Directions

12
Virtual labs

• A CS department wants to teach a course on
  Parallel Programming.
• Unfortunately, they do not have a computing
  cluster the students can use. And, even if they
  do, the cluster admin won't give the students
  root access.
• Solution Deploy a virtual workspace that
  provides the students with a ready-to-use cluster.

13
Event-driven applications

• Some applications need large amounts of
  computational power the instant a specific event
  arrives.
• Science comes at you fast!
• Current solutions include job preemption or
  allowing urgent requests to skip to the front of
  the queue.
• Solution VMs allow seamless suspend/resume of
  execution environments. An urgent environment
  can be placed on standby, activated when the
  event arrives, and all other VMs can be paused.

Less draconian example Fusion experiments
14
Strict software requirements

• Resource consumers can have strict software
  requirements
• Jobs that will only run with a specific version
  of a library.
• Legacy software.
• Special access privileges
• Managing multiple software requirements can be
  problematic for the resource provider
• Conflicting software requirements.
• Switching between software environments can be
  costly (e.g., reimaging cluster nodes)?
• Avoid malicious use of resources.
• Solution Encapsulate desired software
  environment inside a VM.

15
Index

• Virtual Workspaces
• Use Cases
• GT4 Workspace Service
• Current Directions

16
GT4 Workspace Service

• The GT4 Virtual Workspace Service (VWS) is a
  VM-based workspace implementation.
• GT4 WSRF frontend
• Xen-based, but other VMMs could potentially be
  used.
• http//workspace.globus.org/

17
GT4 Workspace ServiceArchitecture
The workspace service has a WSRF frontend that
allows users to deploy and manage virtual
workspaces
Pool node
Pool node
Pool node
VWS Node
Workspace Service
The VWS manages a set of nodes (typically a
cluster). This is called the node pool.
Pool node
Pool node
Pool node
Each node must have a VMM (Xen)? installed, along
with the workspace backend (software that
manages individual nodes)?
Pool node
Pool node
Pool node
Image Node
Pool node
Pool node
Pool node
VM images are stored in a separate node.
Resource Provider
18
GT4 Workspace ServiceTypical Deployment
Users can interact directly with their workspaces
the same way the would with a physical machine.
Pool node
Pool node
Pool node
VWS Node
Workspace Service
Pool node
Pool node
Pool node
Pool node
Pool node
Pool node
Image Node
Pool node
Pool node
Pool node
Resource Provider
19
GT4 Workspace Service

• Workspace Service is a Globus incubator project.
• http//workspace.globus.org/
• Current release TP1.2.3 (04/20/07)?
• Provides the functionality described in the
  previous slides (managing a pool of nodes, and
  deploying single-machine workspaces to those
  nodes)?
• New in this release
• Multiple partitions per VM (with some
  restrictions)?
• HTTP staging
• Improved documentation
• Multiple bug fixes

20
Index

• Virtual Workspaces
• Use Cases
• GT4 Workspace Service
• Current Directions

21
Workspaces Batch Scheduling (I)?

• Workspaces allow us to dynamically deploy
  execution environments on a grid.
• But... how do they fit into the model of
  batch-oriented computation? The job is the most
  widely used execution abstraction on grids.
• In other words Will it play in Peoria?

Workspace Service
Workspace- aware LRM
LRM

• Option 1 Use a workspace-aware LRM.
• This would be ideal, but there is no such thing.
  There are plenty of commercial data center
  solutions, but they focus on resource leasing,
  not batch systems.
• Option 2 Use workspaces to deploy virtual
  compute clusters.
• Could be a valid option in certain cases (e.g.,
  gliding in your own scheduler). However, cluster
  admins have to give up their LRM, and the
  responsibility of providing an LRM rests on the
  job submitter.
• Option 3 Have LRMs and workspaces as separate
  components, but working together.

22
Workspaces Batch Scheduling (II)?

• We want the LRM to be able to create a workspace
  for jobs requiring a specific software
  environment.
• There is a naïve way of doing this Set the job
  prolog to start VM and the job epilog to stop
  VM. This adds overhead which can be excessive if
  not adequately managed 1
• Questions we want to answer
• How can we efficiently combine resource leasing
  and batch scheduling?
• Can we use the suspend/resume functionality of
  VMs to improve cluster utilization, despite the
  overhead of using VMs? (partially addressed in
  1)?
• How can we do this in such a way that cluster
  admins will be happy?

1 A Resource Management Model for VM-Based
Virtual Workspaces, Sotomayor, Borja.
Masters paper, University of Chicago
23
Configuration Management

• Virtual workspaces without a collection of VM
  images to choose from is like iTunes without the
  music.
• However, VM images cannot be created just once
  and then used out-of-the-box.
• VM images need to be configured to match the
  configuration of the site they are being deployed
  to (IP address, certificates, software in the
  image that depends on that information, ...)?
• Site admins need to establishing that a VM image
  is trustworthy.
• VM images are large, and can use up valuable
  storage resources.
• We need mechanisms that address these concerns.
  Otherwise, only site-specific VM images will be
  deployable in practice.
• Bradshaw et al. 2 discuss requirements and
  services (including Bcfg2 and Workspace Service)
  necessary to ensure scalable management and
  deployment of appliances (implemented as VM
  images)?

2 A Scalable Approach To Deploying And Managing
Appliances, Bradshaw, R., N. Desai, T.
Freeman, K. Keahey. TeraGrid 2007
24
STAR (I)?

• STAR is an experiment at the Relativistic Heavy
  Ion Collider (RHIC) at the Brookhaven National
  Laboratory.
• Their software is an example of an application
  that requires a very specific software
  environment to work correctly, and which cannot
  easily be deployed on Grid sites.
• Doug Olson (LBL)
• tends to push the boundaries on what will
  actually compile
• using rarely used features of the language
• even just validating a new platform is a big job
  even when it all compiles.
• The STAR offline analysis software is about 1.3M
  lines of code, 2/3 C, a bit under 1/3 Fortran,
  and a bit of C.

25
STAR (II)?

• rBuilder to the rescue
• rPath was founded by ex-RedHat luminaries
• Software Appliances (and not just VMs)?
• http//www.rpath.com/rbuilder/
• Stu Gott, Ken Vandine, and Marty Wesley worked
  with OSG and Doug Olson to produce a STAR
  appliance with rBuilder
• STAR image successfully deployed on Teraport Xen
  nodes.
• We have also experimented with the AliEn software
  (from the ALICE experiment)?
• http//www.rpath.org/rbuilder/project/alien/
• http//alien.cern.ch/twiki/bin/view/AliEn/Home

26
GridWay Workspaces (I)?

• The GridWay Metascheduler (http//www.gridway.org/
  ) is a very successful dev.globus project (first
  to escalate to a full project)?
• The GridWay team is interested in using
  workspaces for deployment of Virtual
  Infrastructures across sites.
• Dynamic deployment and reallocation of VMs
  between different administration domains.
• GridHypervisor Project http//www.gridhypervisor.
  org/
• We are collaborating with the GridWay team,
  although the lack of common funding is an
  impediment.

27
GridWay Workspaces (II)?
Users
Service Overlay
Applications
VM Request
Virtual Infrastructure
Middleware
Physical Infrastructure
Grid Site
Management of Virtual Machines in Grid
Infrastructures, Rubén S. Montero. OGF20.
28
GT4 Workspace Service

• On the implementation side, we are working on
  adding the following to the Workspace Service in
  the next releases
• Support for virtual clusters
• EC2 backend
• Workspace Marketplace http//workspace.globus.org
  /vm/marketplace.html
• Improved usability and documentation

29
Questions?
Borja Sotomayor University of Chicago (Dept. of
CS)? borja_at_cs.uchicago.edu