CARF Functional Prototype

1
CARFFunctional Prototype
Software framework, services and persistency in
high level trigger, reconstruction and analysis

• Vincenzo Innocente
• CERN/EP/CMC

2
CMS Experiment-Data Analysis
Quasi-online Reconstruction
Environmental data
Detector Control
Online Monitoring
store
Request part of event
Store rec-Obj
Request part of event
Event Filter Object Formatter
Request part of event
store

Persistent Object Store Manager
Object Database Management System
Store rec-Obj and calibrations
store
Request part of event
Data Quality Calibrations Group Analysis
Simulation G3or G4
User Analysis on demand
3
CARFCMS Analysis Reconstruction Framework
Physics modules
Specific Framework
Reconstruction Algorithms
Data Monitoring
Event Filter
Physics Analysis
Generic Application Framework
Calibration Objects
Event Objects
Configuration Objects
CMS adapters and extensions
Utility Toolkit
ODBMS
C standard library Extension toolkit
Geant3/4
CLHEP
Paw Replacement
4
CMS Software RD

• 95-96 RD41 --- OO Detector Reconstruction
• Detector model, Local hit cache, Pattern
  recognition
• 95-97 RD45 --- OO Event Model (persistent)
• Event structure, Raw data, Reconstructed objects
• 95-97 RD45 --- Calibration Database
• Time dependent data, Versioning, Experience with
  Objectivity/DB
• 96-98 Program Architecture
• Implicit invocation, Event dispatching,
  Reconstruction on demand
• 97-98 Test-Beam (H2, X5)
• OO DAQ, Online filtering, ODB population,
  Interactive analysis
• 99-00 Test-Beam ORCA Production
• Event-Collections, Concurrent jobs,
  Multi-threading, RT dynamic loading
• Production management, MSS interface, Data
  import-export
• User databases, User event-collections
• Event visualization, Interactive analysis

5
CARF Components

• Basic Utility ToolKit
• Once was CERNLIB
• Today a set of classes that extends the standard
  C and C libraries
• Basic Mechanisms
• Sets of collaborating classes, implementation of
  the most popular patterns
• Base of Object Oriented Programming
• Basic Persistency Support
• Objectivity Wrappers
• Generic Persistent Classes
• User Interface
• Links keywords to objects
• Inputs from ascii file, database objects,
  interactive shell

6
CARF Components

• Persistent Object Model
• Event Catalog, Configuration and MetaData
• Simulated Event
• RawEvent (SimHits and Digis)
• Reconstructed Objects

7
CARF Components

• Framework
• Package Initializer
• Event Reader
• Configuration management
• DataBase Populator
• API (for developers)
• Simulated-Event Source
• SimHit Formatter
• SimHit Loader
• Reconstruction Detector
• Reconstruction Unit
• API (for users)
• Selectors
• Event Observers (Analyzers, Filters, etc)
• RecObj Collection

8
Functionalities

• Basic Application
• Iterates over an input Event Collection
• Selects events based on MetaData
• Today based just on event- and run-number and
  data availability
• Tomorrow will use user-defined Annotations
• Dispatch Events to Observers
• Standard and/or user module to
• Analyze and classify the event
• produce or update reconstructed event objects
• anti-select events based on reconstruction
  information
• Force production of persistent objects if
  required
• Produce an output Event Collection
• Any combination of shallow and deep copy of any
  part of the event is in principle allowed
• In practice few predefined options

9
Functionalities

• Simulation Application
• Back-end to Detector Simulation to provide
  persistency support for simulated tracks and hits
• Standard CMSIM converts from FZ to CARF/ORCA
• new G4 prototype converts from G4 to CARF/ORCA

10
Functionalities

• SimReader Application
• Reads output of Simulation and perform
  digitization
• Reads pile-up pseudo-randomly from a large event
  collection
• Can perform reconstruction if required
• Force digitization for all registered detectors

11
Functionalities

• Reconstruction Application
• Read Output of a previous Digitization or
  Reconstruction
• will re-digitize and re-reconstruct any detector
  declared obsolete
• will digitize and reconstruct any newly
  registered detector and reconstruction-unit

12
Quality

• Performance
• Framework performance not yet critical
• Some problems in opportunistic MetaData update
  from several concurrent jobs
• Reliability
• All components (CARF, Objy servers) seem to be
  reliable for single-user applications (some
  problems from HPSS)
• Still some problems in massive production (above
  150 concurrent jobs)
• Code quality still at prototype level
• Major coding guide-lines were not correctly
  identified or overlooked
• too much use of inline code
• too many compilation- and run-time dependencies
• user API too complex
• Requirements and priorities should be redefined

13
Impact of Strategic choices

• CMS has choose a non-traditional approach in
  several strategic areas
• Object-Oriented Programming in C
• Object Database Management System
• Plug-in
• Implicit invocation
• Multi-thread
• Opportunistic Database access
• All these technologies have proven to work well
  and fulfill the requirements for the framework
  and in several prototype
• We should now evaluate their impact on non-core
  software and on users code.

14
Short Term Plan

• Basic ToolKit
• Integrate with IGUANA, FAMOS and OSCAR equivalent
• Integrate with ANAPHE?
• Basic Persistency
• Extend to other projects
• integrate with HEPODBMS?
• Persistent Data Model
• Extend to other projects
• Prototype realistic RecObjs
• Investigate Generic User-data such as
• Annotations
• tuples
• Answer to the question
• What new kind of objects should be stored
  during physics analysis?

15
Outlook

• Most of the functionalities identified in the CTP
  and in later more detailed requests (HLT for
  instance) have been implemented
• Performance and Reliability not yet at production
  level
• Code quality still at prototype level
• Requirements and priorities should be redefined
  in this area
• Non-traditional technologies seem working well
  for core-software
• Impact on non-core software should be evaluated.