BOINC Case Study: LHChome

1
BOINC Case StudyLHC_at_home

• Ben Segal / CERN
• b.segal_at_cern.ch
• with
• Predrag Buncic / CERN
• Daniel Lombrana Gonzalez / Univ. Extremadura
• David Weir / Imperial College
• ASIA_at_home Workshop, Academica Sinica
• Taiwan, April 16, 2009

2
CERN
Mont Blanc, 4810 m
Geneva airport
CERN
LHC tunnel
3
The LHC
4
LHC_at_home

• Calculates stability of proton orbits in CERNs
  new LHC accelerator
• System is nonlinear and unstable so numerically
  very sensitive. Hard to get identical results on
  all platforms
• About 40 000 users, 70 000 PCs over 1500
  CPU years of processing
• Objectives extra CPU power and raising public
  awareness of CERN and the LHC - both successfully
  achieved.
• Started as an outreach project for CERN 50th
  Anniversary 2004 used for Year of Physics
  (Einstein Year) 2005

5
SixTrack program
SixTrack is a Fortran program by F. Schmidt,
based on DESY program SixTrack simulates 60
particles for 100k-1M LHC orbits Can include
measured magnet parameters, beam-beam
interactions LHC_at_home revealed reproducibility
issues, solved by E. McIntosh
Phase space images of a particle for a stable
orbit (left) and unstable chaotic orbit (right).
6
gt3000 CPU-yearsgt60k volunteers
7
Server managed by QMUL
Cool screensaver
Message boards
Credit for processing
8
LHC_at_home future plans
Sixtrack for LHC upgrade studies (W. Herr,
CERN) Sixtrack for other accelerators (Yun Luo et
al., Brookhaven National Lab) Garfield for the
detailed simulation of gases in detectors (R.
Veenhof, CERN)  Rivet, Jetweb Monte Carlo event
generator, validator and web archiver (J.
Butterworth. UCL) ATLFAST and ATHENA event
simulation for ATLAS using virtualization (B.
Segal, CERN) Volunteer thinking for LHC data
analysis (discussions with Jim Virdee, John
Ellis)  
9
(No Transcript)
10
(No Transcript)
11
(No Transcript)
12
BOINC Virtualization

• Work began at CERN in 2006 (w. Daniel Lombrana
  Gonzales
• 
  others)
• Chose VMware Player over (k)QEMU, etc.
• Proved feasibility and built a working prototype
• Continued project at Univ. Extremadura
• Work continued at CERN in 2007 (w. David Weir)
• Succeeded to build a VM for full ATLAS Athena
  environment
• Reduced size of VM image by mounting runtime
  libraries over AFS

13
BOINC Virtualization

• Work accomplished at CERN in 2008 (w. Pedrag
  Buncic)
• Established collaboration with the CernVM Project
• General interface to all CERN physics software
• Size of VM images now optimized and cached
  locally
• Able to build VMs for full ATLAS, LHCb, ALICE and
  CMS environments and tested a VM with full Athena
  environment
• Included an interface to physics groups
  production chain (tested with ATLAS PanDA job
  management)

14
BOINC Virtualization

• Work remaining at CERN in 2009 (in progress)
• Improve Wrappers handling of VMware VMs
• Cleanly Start, Stop, Suspend, Restart guest VMs
  from host OS (using VIX API available for VMware
  Server)
• Monitor host OS resources of guest VMs (for
  BOINC credit)
• OSs to include Linux, Windows, possibly MacOS
• Extend to Sun VirtualBox VMs with their API
• Possibly extend to kQEMU also

15
CernVM - Virtual Machine for LHC Experiments
Predrag Buncic (CERN/PH-SFT)

16
CernVM Background

• Over the past couple of years, the industry is
  redefining the meaning of some familiar computing
  terms
• Shift from glorious ideas of a large public
  infrastructure and common middleware towards
  end-to-end custom solutions and private corporate
  grids
• New buzzwords
• Amazon Elastic Computing Cloud (EC2)
• Breakthrough in industry approach to computing
• Everything is for rent (CPU, Storage, Network,
  Accounting)
• Blue Cloud (IBM) is coming
• Software as a Service (SaaS)
• Google App Engine
• Virtual Software Appliances and JeOS
• In all these cases, virtualization is emerging as
  a key enabling technology, and is supported by
  computer manufacturers
• Multiple cores
• Hardware virtualization (Intel VT, AMD-V)

17
Motivation

• Software _at_ LHC Experiment(s)
• Millions of lines of code
• Complicated software installation/update/configura
  tion procedure, different from experiment to
  experiment
• Only a tiny portion of it is really used at
  runtime in most cases
• Often incompatible or lagging behind OS versions
  on desktop/laptop
• Multi core CPUs with hardware support for
  virtualization
• Making laptop/desktop ever more powerful and
  underutilised
• Using virtualization and extra cores to get extra
  comfort
• Zero effort to install, maintain and keep up to
  date the experiment software
• Reduce the cost of software development by
  reducing the number of compiler-platform
  combinations
• Decouple application lifecycle from evolution of
  system infrastructure

18
How do we want to do this?

• Build a thin Virtual Software Appliance for use
  by the LHC experiments
• This appliance should
• provide a complete, portable and easy to
  configure user environment for developing and
  running LHC data analysis locally and on the Grid
• be independent of physical software and hardware
  platforms (Linux, Windows, MacOS)
• This should minimize the number of platforms
  (compiler-OS combinations) on which experiment
  software needs to be supported and tested, thus
  reducing the overall cost of LHC software
  maintenance.
• All this is to be done
• in collaboration with the LHC experiments and
  OpenLab
• By reusing existing solutions where possible

19
Key Building Blocks

• rBuilder from rPath (www.rpath.org)
• A tool to build VM images for various
  virtualization platforms
• rPath Linux 1
• Slim Linux OS binary compatible with
• Red Hat / SLC4
• rAA - rPath Linux Appliance Agent
• Web user interface
• XMLRPC API
• Can be fully customized and extended by means of
  plugins (401)
• CVMFS - CernVM file system
• Read-only file system optimized for software
  distribution
• Aggressive caching
• Operational in offline mode
• For as long as you stay within the cache

• Build types
• Installable CD/DVD
• Stub Image
• Raw File System Image
• Netboot Image
• Compressed Tar File
• Demo CD/DVD (Live CD/DVD)
• Raw Hard Disk Image
• VMware Virtual Appliance
• VMware ESX Server Virtual Appliance
• Microsoft VHD Virtual Appliance
• Xen Enterprise Virtual Appliance
• Virtual Iron Virtual Appliance
• Parallels Virtual Appliance
• Amazon Machine Image
• Update CD/DVD
• Appliance Installable ISO
• Sun Virtual Box Image

20
Thin Software Appliance
H T T P D
LAN/WAN (HTTP)
Software Repository
Cache
10 GB
1 GB
0.1 GB
21
CernVM File System
On same host
On File Server
/opt/lcg -gt /chirp/localhost/opt/lcg
/opt/lcg -gt /grow/host/opt/lcg
App
CernVM Fuse
open(/opt/lcg)
!Cache
Kernel
Cache
NFS
LFS
FUSE
22
Bridging Grids Clouds

• BOINC
• Open-source software for Volunteer Computing and
  Grid computing
• CernVM is being extended to support BOINC client
• http//boinc.berkeley.edu/
• CernVM CoPilot development
• Based on BOINC, LHC_at_home experience and CernVM
  image
• Image size is of utmost importance to motivate
  volunteers
• Can be easily adapted to Pilot Job frameworks
  (AliEn,Dirac, PanDA)
• or Condor Worker, or proofd..
• Aims to demonstrate running of ATLAS simulation
  using BOINC infrastructure and PanDA

23
CernVM CoPilot
AliEn/DIRAC/PanDA
0. Send host JDL (free disk space, free memory,
available packages)
1. Append framework- specific information and
request a job
2. Send user job JDL from Task Queue
3. Send input files and commands for
execution (packages are already there)
4. When the job is done send back the output
files (and the result of validation)
5. Register output files
24
Building the CernVM community

• Mailing lists
• cernvm-talk_at_cern.ch (Open list to discuss about
  design, user experience and related issues with
  the CernVM project)
• cernvm.support_at_cern.ch (End-user support for the
  CernVM project)
• Savannah Portal
• Please submit bugs and feature requests to
  Savannah at
• http//savannah.cern.ch/projects/cernvm
• Reference Web site(s)
• http//cernvm.cern.ch
• http//rbuilder.cern.ch