US-CMS Core Application Software Progress and Activities

1
US-CMS Core Application Software Progress and
Activities

• Ian Fisk
• DOE/NSF Status Report
• June 21, 2002

2
Outline

• Quick Introduction to the Core Application
  Software Project (CAS)
• Scope of the project
• Division of labor
• News from CMS and CERN
• Status
• Activities
• Progress
• Problems
• Summary

3
Introduction to CAS

• CAS is the US-CMS Core Application Software
  Project. We are involved in 4 main areas
• WBS 2.1 CMS Software Architecture
• Core Framework Development
• Sub-System Architecture Development
• CAFÉ (CMS Architecture Forum and Evaluation)
• WBS 2.2 IGUANA
• Graphical User Interfaces
• Visualization
• Data Browsing, plotting, fitting
• WBS 2.3 Distributed Data Management and
  Processing
• Evaluation,Testing, Integration of Grid Tools
• Distributed Process and Database Management
• System Services and Load Balancing
• Development of Production Tools
• Development of Distributed Production and
  Analysis Prototypes
• System Simulation and System Scalability
  Development
• WBS 2.4 Support

4
Inside International CMS

• CPT is a combination of Computing, Physics,
  and Trigger DAQ. Computing had been divided into
  7 sub-projects. There are 4 cross project
  groups to handle interactions between projects.
  Small reorganization this spring

CCS Core Computing Software
PRS Physics Reconstruction and Selection
TriDAS Online Software
1. Computing Centres
2. General CMS Computing Services
9. Tracker / b-tau
3. Architecture, Frameworks / Toolkits
7. Online Filter Software Framework
10. E-gamma / ECAL
8. Online Farms
4. Software Users and Developers Environment
11. Jets, Etmiss/HCAL
5. Software Process and Quality
CAS Work
12. Muons
6. Production Processing Data Management
7. Grid Systems
RPROM (Reconstruction Project Management)
SPROM (Simulation Project Management)
CPROM (Calibration Project Management)to be
created
GPI (Group for Process Improvement)recently
created
5
Inside International CMS

• CPT is a combination of Computing, Physics,
  and Trigger DAQ. Computing had been divided into
  7 sub-projects. There are 4 cross project
  groups to handle interactions between projects.
  Small reorganization this spring

CCS Core Computing Software
PRS Physics Reconstruction and Selection
TriDAS Online Software
1. Computing Centres
2. Architecture, Frameworks / Toolkits
9. Tracker / b-tau
3. Computing and Software Infrastructure
7. Online Filter Software Framework
10. E-gamma / ECAL
4. Production Processing Data Management
8. Online Farms
11. Jets, Etmiss/HCAL
5. Grid Systems
CAS Work
12. Muons
CMS Librarian
RPROM (Reconstruction Project Management)
SPROM (Simulation Project Management)
CPROM (Calibration Project Management)to be
created
GPI (Group for Process Improvement)recently
created
6
Introduction to CAS

• CAS Currently Employs 11 On-Project Developers at
  5 Institutions, which total 9 FTEs worth of
  effort.
• Michael Case UC Davis 2.1 75 2.4 25
• Greg Graham Fermilab 2.3 30
• Iosif Legrand Caltech (CERN) 2.3 75 2.4 25
• Vladimir Litvin Caltech 2.1 75 2.4 25
• Ianna Osborne Northeastern (CERN) 2.2 75 2.4
  25
• Shahzad Muzzaffar Northeastern (CERN) 2.2
  75 2.4 25
• Natalia Ratnikova Fermilab 2.4 50
• Bill Tanenbaum Fermilab 2.1 75 2.4 25
• Lassi Tuura Northeastern (CERN) 2.1 75 2.4 25
• Hans Wenzel Fermilab 2.1 20
• Tony Wildish Princeton (CERN) 2.3 50 2.4 50
• WBS 2.1 2.25
• WBS 2.2 1.50
• WBS 2.3 2.25
• WBS 2.4 3.0
• Totals 9.0
  

7
CAS engineer contributions so far

• Michael Case UC Davis
• Detector Description Database Development
• End-Cap Muon Parameter Storage
• Greg Graham Fermilab
• Development of IMPALA and MC_RunJob
• Distributed Production Prototypes
• FNAL Production Support
• Iosif Legrand Caltech
• MONARC Simulations
• Monitoring Tools
• Vladimir Litvin Caltech
• Production Support
• Calorimetry Reconstruction Framework
• Ianna Osborne Northeastern
• Deployment of IGUANA-based event display for ORCA
  associated PRS training
• SCRAM / external software support for all CMS
  projects
• Software tutorials

• Natalia Ratnikova Fermilab
• Configuration Tools Support and Development
• Software Distribution Tools (DAR)
• Bill Tanenbaum
• COBRA and Persistency Development
• Lassi Tuura Northeastern
• New IGUANA analysis architecture and associated
  prototype code
• Dependency Analysis Program written and deployed
• OSCAR visualization for developers
• Hans Wenzel Fermilab
• OSCAR Librarian
• Tony Wildish Princeton
• Database Administration Tools
• Distributed Computing Prototypes
• Production Management
• Shahzad Muzaffar

8
Highlights Since the Baseline Review

• Progress of decoupling the framework from
  Objectivity and adopting a new persistency
  baseline
• Release of IGUANA version 3.0
• Improved features requested by PRS groups for
  evaluating and debugging reconstruction
  algorithms
• OSCAR and GEANT4 visualization
• Production scaled up to 300 machines at a single
  site, 13 fully qualified production centers
• Successful production run for DAQ TDR
• 20TB of simulation were completed on-schedule
  using a globally distributed production
  facilities
• IMPALA and DAR required for use in production
  were
• Deployment of monitoring prototype at CERN, FNAL,
  and US-Tier2 centers
• Allows monitoring of facility status remotely.

9
News from CERN

• LHC Grid Computing Project Launching Workshop was
  the second week of March
• Should result in an influx of developers,
  especially in the Applications area
• Countries unable to send software engineers to
  the experiments directly have been able to make
  positions for the LCG
• CMS is prepared to make almost all of our
  development projects common projects with the LCG
• Everything is on the table, but the table is not
  empty
• Need to determine how many developers we are
  willing to devote to the common LCG Projects
• Need to balance doing our part and ensuring the
  success of the LCG, while hedging our bets and
  leaving enough manpower to complete integration,
  deployment, and testing.
• Current proposal for the developers in CMS AFT is
  around 50.
• The majority of CAS engineers are working with in
  projects within ATF
• Quite a few proposals for common application
  projects
• All are interesting, some are more critical than
  others (really need success for the new
  persistency baseline)
• Ill try to point out project were US-CMS effort
  is likely to be directly affected

10
The LCG

• The LCG is broken into 4 work areas
• Applications (Majority of new people are
  expected in this area)
• Facilities (Very few people here, but money for
  hardware at CERN)
• Grid Technology (Expected from Grid Projects)
• Grid Deployment
• Proposals for common projects are made by the
  managers of the 4 areas
• An RTAG is formed (Requirements Technical
  Assessment Group)
• Usually 4-6 expert representatives from
  experiments, IT, others
• Reports are finished within two months
• Depending on the outcome, common projects are
  organized
• Experiments pledge people to work
• An experiment agrees to host the project

11
WBS 2.1 Architecture

• Core Architecture Development
• At the time of the November baseline review. CMS
  was considering that it would switch away from
  Objectivity for the persistency mechanism
• The long term economic viability was in doubt
• Scalability of complex centralized services was
  questioned
• To start the effort CMS proposed a hybrid
  persistency solution based on ROOT/IO and a
  relational database
• ROOT/IO formal description document was created
  at FNAL
• This has evolved into the first project of the
  LCG
• First RTAG has met
• Interfaces should be defined by September and a
  basic prototype is expected for the fall
• In order use the new solution. The current CMS
  Framework COBRA has to be decoupled from
  Objectivity.
• Bill Tanenbaum has been working on the prototype
• Objectivity is removed and the interface to
  persistency is being made generic
• Possibility of a functional prototype for testing
  by the end of summer

12
2.1.2 CMS Subsystem Development

• US-CMS is currently involved in 4 areas subsystem
  areas
• 2.1.2.1 Detector Description Database
• 2.1.2.2 OSCAR GEANT4 Simulation
• 2.1.2.3 Reconstruction Framework
• 2.1.2.4 Analysis Sub-Architecture
• We have chosen to delay
• 2.1.2.5 Production Framework Development
• Until the persistency solution is more stable

13
2.1.2.1 DDD

• CAS Engineer Michael Case has been participating
  in the development of the Detector Description
  Database. A consolidated store of detector
  information used by CMS software clients. Needed
  for consistent geometry input to CMS Software
  Packages
• Tools were developed for translating geometry
  information directly from the CMS GEANT3 storage
  format (tz) and from the GEANT output.

• Functional prototype released Dec. 2001
• Basic Geometry and Materials
• Basic Core Functionality
• XML Schema
• Fully Functional Prototype released April 2002
• All CMS Required Solids
• All CMS positioning parameters
• Numbering Scheme
• Prototype of XML DDD editor

14
Progress of the DetectorDescriptionDatabase

• When last we met, the DDD was a concept. Now the
  entire detector is stored up to the geometry
  version in cmsim122.
• The DDD can be read from OSCAR and for validation
  the tracking efficiency can be well reproduced
• IGUANA has been modified to use the overlap
  detection tool. Problems in the detector
  geometry can be immediately visualized
• Additional CMS core software packages will soon
  use the DDD
• At the CMS Software tutorial in San Diego,
  participants went through the exercise of
  calculating the material budget in the tracking
  and modified the DDD to see the effect.
• There is already a strong interest
• in making a common LCG DDD
• effort. Workshop in July at CERN

15
OSCAR and Reconstruction

• 2.1.2.2 OSCAR (Hans Wenzel)
• Last year OSCAR could not do full detector
  simulation
• This year several thousand events can be
  simulated
• Big infusion of people. New coordinator, new
  developers, new librarian who has handled release
  and configuration
• Fast Development
• Can now read geometry and material information
  from the DDD
• Detailed Interactive display program released to
  users
• 2.1.2.3 Reconstruction Framework (Vladimir
  Litvin)
• Currently working on a code reorganization to
  improve indexing, navigation and speed.
• Release of improved code expected by the summer
• Working on changing the information stored
  persistently for future on-line tests.

16
2.1.2.4 Analysis Architecture

• New Analysis Architecture released by Lassi Tuura
  last October
• Relying on a very small very flexible kernel
• Uniform architecture and interfaces for multitude
  of tools
• Interoperable components/plug-ins
• Is currently used in the OSCAR visualization
  program, ORCA visualization program is being
  ported to it

17
WBS 2.2 IGUANA

• The Interactive Graphical User Analysis Project
  (IGUANA) addresses interactive visualization
  software needs for CMS
• This year
• IGUANA-based event display deployed and supported
  for PRS work in ORCA context
• IGUANA-based GEANT4 interactive visualization
  system deployed and supported in OSCAR context
• Progress Throughout the Year
• October 2001 IGUANA completed a review of
  baseline GUI tools
• October 2001 Release of new analysis
  architecture
• Creation of a prototype OSCAR visualization
  package based on it
• November 2001 Analysis Workshop Organized at CPT
  Week
• December 2001 Prioritized list of improvements
  and changes from PRS Groups
• May 2002 Release of IGUANA 3.0
• Improved ORCA and OSCAR visualization

18
ORCA Visualisation with IGUANA
19
ORCA Visualisation

• Based on generic IGUANA toolkit
• with CMS specific extensions for Detector
  Geometry
• Geant3 detector geometry
• Reconstruction geometry for the Tracker
• and Event
• Muon DT, CSC, and RPC sim hits DT and CSC
  track segments, CSC rec hits reconstructed and
  simulated tracks
• Tracker-Bt simulated and reconstructed tracks,
  measurements with directions, sim hits
• ECAL-Eg simulated and reconstructed hits
• HCAL-JetMEt digits, jets.

20
Application Track Reconstruction
Zoom
21
OSCAR (GEANT4) Visualisationusing IGUANA
IGUANA Viewer displaying OpenInventor scene
Control of arbitrary GEANT 4 tree
Correlated Picking
22
OSCAR Visualization Overlap Detection

• Integration of the detector overlap tool (Martin
  Liendl)
• Extending the scope of the configuration wizard

Example extension (a trivial wizard) Queried
from plug-in database, located on request and
bound to IGUANA G4 Run Manager
23
IGUANA Plans

• IGUANA Developers currently porting ORCA
  Visualization to new analysis architecture
• Expected Release soon
• Working on taking geometry information from the
  DDD prototype
• Useful for validating and debugging the DDD
• Schedule is to have a baseline visualization and
  data browsing tools synchronized with the
  development of the new persistency baseline.
• Interest has been expressed in using IGUANA and
  performing development within the context of the
  LCG.

24
WBS 2.3 Distributed Data Management and
Processing (DDMP)

• DDMP will provide tools for the CMS computing
  model, which is globally distributed in nature.
  Working together with the Grid projects DDMP will
  deliver functionality in the following areas.
• 2.3.1 Grid Planning and Interaction
• 2.3.2 Computing Complexity Progression
• 2.3.3 Distributed Process Management (Primarily
  off-project)
• 2.3.4 Distributed Database Management (Primarily
  off-project)
• 2.3.5 Load Balancing (Mixture)
• 2.3.6 Production Tools
• 2.3.7 Distributed Production Prototyping
  (Mixture)
• 2.3.8 Distributed Analysis Prototyping (Mixture)
• 2.3.9 System Simulation
• Tasks involve effort with Grid Projects and CMS
  specific development

25
Success of 2002 Production

• This spring CMS performed the simulated event
  production required to complete the DAQ TDR.
• 6M events
• Produced at 20 sites
• Production ran very
• smoothly
• US-CMS Contributed
• IMPALA
• Greg Graham
• Job Specification
• DAR
• Natalia Ratnikova
• Binary Release
• Both made a huge
• difference in ease
• of production

1.4 seconds per event, 2 months
Digitization at 1034
26
2.3.6 MC_runjob implementation

• A replacement for some of the IMPALA
  functionality, which is hitting the limit of its
  complexity.
• Being prepared for CMS summer production
• Allows specification and chaining of executables
• Allows production jobs to be templated and
  reduces the manual configuration of production
  managers
• Has a GUI
• PRS requests for several million event samples,
  which will be run from generation through
  simulation, reconstruction and analysis in a
  single job

27
2.3.2 System Complexity Progression

• CMS has an aggressive ramp up of computing
  complexity to reach a full sized system.
• Target dates are being reconsidered with new
  estimated start of the LHC
• Tony Wildish reached 300 CPU boxes used in
  production at a single site
• Changing to the new persistency baseline will
  likely slow the progress in the coming year.
• A lot of the central services we rely on with
  Objectivity will have to be duplicated for the
  new solution
• Along with the effort to make use of distributed
  computing resources, there is considerable effort
  needed to use large amounts of local resources
  when there are central services.

28
2.3.1 Interfacing with Grid Projects

• To better facilitate our interactions with the
  Grid Projects. CMS created a Grid Implementation
  Plan
• Will be released each year
• Summaries current activities and plans for the
  coming year
• This was the first year and has a lot of summary.
  Hopefully will define more CMS requirements in
  the future.

29
2.3.7 Distributed Production Prototypes

• PPDG Developed MOP System
• Relies on GDMP for file replication
• Globus GRAM for authentication
• Condor-G and local queuing systems for Job
  Scheduling
• IMPALA for Job Specification
• CMS assigned 200k events of
• official production to test the MOP
• system
• Integrate better with Grid
• Developers
• Leverage additional resources
• Really stressed the MOP system
• Exposed a number of problems
• in the underlying grid
• components

30
Virtual Data Grid System (Production)
Local Tracking DB
Concrete Planner/ WP1
Abstract Planner
Wrapper Scripts
Object-ivity Metadata Catalog
31
2.3.6 Monitoring

• One of the outstanding production issues is the
  need to monitor distributed computing centers
• Iosif Legrand has been working on a prototype
  which was deployed at CERN, FNAL, and US-Tier2s

32
2.4 Support

• US-CMS Organized a Software Tutorial May 1-4 at
  the Tier2 center in
• San Diego.
• 40 participants attended
• Software Tutorials now have a lot to cover
• Basics of CMS reconstruction software and
  interactions with the database
• ORCA
• OSCAR
• Each PRS group now has fairly advanced
  reconstruction framework
• Visualization
• CMS Production and production tools
• We included a few sessions on interactions with
  grid services
• It was very well received

33
Outlook and Schedule

• Since the December review the accelerator
  schedule has officially slipped to physics start
  in 2007
• The slip was not unexpected at the December
  review.
• Stretching higher level CMS milestones

34
Schedule and Outlook

• CMS is in a good position to gain from the LCG
• We are sufficiently advanced in a variety of
  areas to have our solutions pursued and developed
  in common
• We have functional prototypes, which give us
  insight into what we think is required from the
  final system.
• LCG will contribute considerable additional
  manpower to LHC Applications
• At the same time, CMS will get solutions out of
  the LCG in proportion to what were willing to
  contribute.
• LCG will have plans and schedules of its own
• Complicates planning for US-CMS
• US-CMS CAS WBS needs work to reflect the
  stretching CMS milestones and the addition of the
  LCG
• Hopefully LCG applications planning will begin to
  firm over the summer
• At the moment there are a variety of potential
  projects but no clear schedule or priority

35
Conclusions

• It has been a productive 6 months
• Concrete progress on new persistency baseline
• Fully functional DDD
• Release of IGUANA 3.0 for users
• The 2002 spring production a big success
• Progress in complexity and monitoring
• Developing a base of users in the US through
  tutorials
• US Engineers contributing well in a variety of
  areas
• US engineers are contributing to many of the CMS
  development areas
• Many of the CMS development areas are being
  examined for adoption by the LCG applications
  area
• Some percentage of our people may be expected to
  contribute to projects managed by the LCG.
• The LCG is an exciting project
• Going to require effort to make sure we get the
  most out of it.