CRAB status and next evolution

1
CRAB status and next evolution

• Daniele Spiga
• University INFN Perugia
• On behalf of CRAB Team

2
OutLine

• Introduction
• Today status
• CrabServer evolution
• Motivations
• Implementation
• First results
• Schedule
• Conclusions plans

3
CRAB
CMS remote analysis builder Allow users to run
analysis jobs on the grid

• The aim of crab is to open the grid to the
  masses CMS user in addition to the development
  of his own analysis code must only specify few
  things to CRAB then it take care to
• Interface with user working environment
  (SCRAM-CMSSW)
• Interface with the Data Management (search where
  requests data are)
• Interact with the grid middleware performing
• job submission
• job status monitoring on demand
• Manage the produced output (copy back, into
  Storage Element, castor)
• Prepare all the stuff to run on remote WN (local
  environment configuration)

Other functionalities kill, resubmit,
postMortem.
4
Today Status
Release in production CRAB_1_4_2
Fully working with CMSSW, DataManagment
(dbs/dls), gLite(bulk submission)
Release in preparation CRAB_1_4_3 ?
support submission from scl4 to scl
CRAB_1_5_0 ? news support BOSS API usage
improve grid interaction (submission, check
status, getoutput)
G.Codispolti

• Work In Progess
• Integration with DBS2
• Publication of CRAB produced data to local scope
  DBS/DLS

F.Fanzago, S.Lacaprara
5
CrabServer Evolution
Actual Job flow CRAB working on UI
6
Implementation strategy
re-use as much as possible all what was done
during these years!!

• Client side
• same CRAB interface, same command line and
  same installation procedure
• Server side
• Adopt ProdAgent general architecture, adding
  all needed stuff

7
Client
User must just properly configure CRAB and use it
as usual
crab.cfg
New configuration keys
crab create same as crab standalone
crab submit submit crab project to the
server and NOT to the grid crab status
show a summary (prepared by the server) of the
status of the whole task crab -getoutput
retrieve from the server the output
The client ship to the server the user proxy too
8
Server
The server adopt the general PA
architecture Components are implemented as
independent agents, communication is performed
through an asynchronous and persistent message
service Plus a grid-Ftp server
DropBoxGuardian Check the dropBox for new
CRAB user task to be managed. ProxyTarAssociator A
ssociate the task to the right user proxy
CrabWorker submit jobs to the grid (lcg/osg) in
the CRAB style JobTraking Track every single job
(via BOSS) ErrorHandler Perform a basic error
handler JobSubmitter Resubmit single job if
needed TaskTracking Keep general
informations about all tasks under execution.
Notification Notify the user when the
task is ready via e-mail
D.Spiga F.Farina M.Cinquilli W.Bacchi A.Dorigo C.K
avka M.Corvo
New components re-used components
9
First results
Two days ago the server components were
integrated and started working together with
success!!

• First client version ready
• available _at_ /afs/cern.ch/cms/ccs/wm/scripts/Cr
  ab/
• submit the task to the server
• retrieve the output
• (task status check on going)
• Ship user proxy to the server in secure way
• Server installed and set up at cern
  crabdev1.cern.ch
• Submit tasks to the grid for the user
• Track the jobs and resubmit if needed
• Retrieve the produced output from the grid
• Prepare the finished task
• notify the user

10
Next few days Schedule
End of February

• Commit code on CVS
• discussion with CMS management on going
• Finalize the first prototype for O(4) users (beta
  tester)
• continue tests to check all works fine
• Setup a server _at_ CNAF
• replicate the installation _at_ cnaf
  (discussion with D.Bonacorsi)

11
Medium term Plans
June 07

• Wait for the feedback from the beta tester
• Add support for the other needed functionalities
  not yet implemented in the basic workflow
• active interaction with DM (DBS/DLS publication)
• status shown also via web page available on
  server
• killing of jobs/task by user
• server disk space management
• test on server functionalities and scalability

12
Summary Conclusions
CRAB stand-Alone

• Ready for sl4 transition
• Under development the support for publication on
  DBS/DLS of user output
• Used also with real data
• (TAC data see Nicolas talk)

Users support, still same problem which CMS
user support role? Just ask/remember us to answer
the users?
13
Summary Conclusions
CRAB Server

• Basic workflow is ready
• First server for beta tester almost ready
• Developed code to be commited on cvs
• Preliminary twiki page available
• https//twiki.cern.ch/twiki/bin/view/CMS/
  CrabServer
• A lot of work to have a Server-Client
  functionalities available to users for june07!!!