Jean-Yves Nief, CC-IN2P3

1
BaBar data distribution using the Storage
Resource Broker (SRB)
Jean-Yves Nief, CC-IN2P3 Wilko Kroeger,
SCCS/SLAC Adil Hasan, CCLRC/RAL
HEPiX, SLAC October 11th 13th, 2005
2
BaBar overview.

• Large amount of data produced by BaBar gt 800 TB.
• Using the collaboration Computing Centres
• Data (re)processing (prompt reco, skimming).
• Simulation production.
• Data analysis ? allow foreign collaborators to
  access data.
• 6 Tier A
• SLAC stores all the data simulation central
  repository.
• Computing tasks divided among all the Tier A
  sites.

3
BaBar data organization.
SLAC CC-IN2P3 Padova RAL CNAF GridKa
Data processing
Data analysis
simulation production (20 sites involved)
? Lots of data movement
4
Requirements.

• Short turn-around between data production and
  analysis.
• Data must be transfered efficiently and quickly
  to Computing Centres.
• Data transfer tools must be
• Reliable.
• Robust.
• Automated.
• Scalable.
• Easy to use.

5
SLAC CC-IN2P3.

• First analysis centres opened to the BaBar
  collaboration.
• Both held complete copies of data (Objectivity).
• Now only SLAC old a complete copy of the data.
• Natural candidates for testing and deployment of
  grid middleware.
• SRB good candidate for data distribution.

6
Whats SRB ?

• Storage Resource Broker developed by SDSC (San
  Diego).
• Provides an uniform interface to heterogeneous
  storage system (disk, tape, databases) for data
  distributed in multiple sites.
• Who is using SRB ?
• Biology, biomedical applications (e.g BIRN).
• Astrophysics, Earth Sciences (e.g NASA).
• Digital libraries (e.g NARA).
• Used world wide USA, Europe, Asia, Australia.

7
Some SRB features.

• Files organized in a logical space (directories,
  subdirectories ) logical view totally decoupled
  from the physical location of the files.
• Replica management.
• Search for files based on their attributes
  (metadata).
• Fine granularity for SRB usersACLs.
• Interfaced with various Mass Storage System
• Access to HPSS transparent to users.
• Small files management.
• And many more

8
SRB architecture.

• 1 zone
• 1 SRB/MetaCatalog server contains list of files,
  physical resources, users registered.
• several SRB servers to access the data at their
  physical location.

(3)
Site 1
test1.txt
Application (asking for test1.txt)
9
SRB BaBar architecture (2001-2004).
1 Zone (SLAC)
client
CC-IN2P3 (Lyon)
SLAC (Stanford, CA)
10
Extra details.

• Hardware
• SUN servers (Solaris 5.8, 5.9) NetraT 1405,
  V440.
• Software
• Oracle 8, 9 for the SLAC MCAT (530,000 files
  registered).
• Multithreaded client application (Python).
• Advantages
• All the 3 steps controlled by the client
  application in // ? no latency due to prestaging.
• Small amount of cache needed on both sides (a few
  GBs).
• Drawbacks
• No registration into the MCAT of the files stored
  in Lyon.
• Performance
• Up to 3 TB/day from tape to tape.

11
ESNET Traffic with one server on both sides
(April 2004).
SLAC (US) ? IN2P3 (FR)
1 Terabyte/day
Fermilab (US) ? CERN
SLAC (US) ? INFN Padva (IT)
Fermilab (US) ? U. Chicago (US)
U. Toronto (CA) ? Fermilab (US)
Helmholtz-Karlsruhe (DE)? SLAC (US)
CEBAF (US) ? IN2P3 (FR)
INFN Padva (IT) ? SLAC (US)
Fermilab (US) ? JANET (UK)
SLAC (US) ? JANET (UK)
DOE Lab ? DOE Lab
Argonne (US) ? Level3 (US)
DOE Lab ? DOE Lab
Fermilab (US) ? INFN Padva (IT)
Argonne ? SURFnet (NL)
IN2P3 (FR) ? SLAC (US)
12
SRB BaBar architecture (since 2005).
2 Zones (SLAC Lyon)
SRB
CC-IN2P3 (Lyon)
SLAC (Stanford, CA)
13
Extra details.

• Hardware
• Still the same.
• Software
• Oracle 10g for the SLAC MCAT.
• Oracle 9i for the Lyon MCAT (migration to 10g
  foreseen).
• MCATs synchronization only users and physical
  resources.
• Comparison of the MCATs contents to transfer the
  data.
• Advantage
• External client can pick up data from SLAC or
  Lyon without interacting with the other site.

14
Overall assessment.

• A lot of time saved for developping applications
  thanks to the SRB.
• Transparent access to data
• Very useful in an hybrid environment (disk,
  tape).
• Easy to scale the service (adding new servers).
• Not dependent of physical locations changes in
  the application.
• Fully automated procedure.
• Easy for SLAC to recover corrupted data.
• 132 TB (232,000 files) shipped to Lyon.
• Up to 3 TB /day from tape to tape (minimum
  latency).

15
Future plans.

• Possibly include RAL as part of the system
• Would reduce the network activity on the SLAC
  ESNET.
• Some tests
• Assessment of web services technologies.

16
Summary and outlook.

• SRB used by BaBar for more than 3 years.
• Data shipment fully automated.
• Lightweight development to make it work.
• Easy administration.
• More servers on both side
• Increase peak transfer rate to 5 TB/day.
• Hundreds of TB foreseen.
• Scalability the sky is the limit well and the
  money too
• Aiming at including RAL.

17
Never forget

• Cannot be achieved without the work of
• The network and system admins.
• The database admins.
• Many thanks to them!