AliEn uses bbFTP for the file transfers. Every FTD runs a server, and all the others FTD can connect and authenticate to it using certificates.

1
AliEn Resource Brokers
Predrag Buncic, Andreas Peters, Pablo Saiz for
the ALICE Collaboration
AliEn (ALICE Environment) is a lightweight GRID
framework developed by the Alice Collaboration.
When the experiment starts running, it will
collect data at at a rate of 2 PB per year,
producing more than 10 9 files per year. All
these files, including all simulated events
generated during the preparation phase of the
experiment, must be accounted and reliably
tracked in the GRID environment. The backbone of
AliEn is a distributed file catalogue, which
associates universal logical file name to
physical file names for each dataset and provides
transparent access to datasets independently of
physical location. The file replication and
transport is carried out under the control of the
File Transport Broker. In addition, the file
catalogue maintains information about every job
running in the system. The jobs are distributed
by the Job Resource Broker that is implemented
using a simplified pull (as opposed to
traditional push) architecture. This poster
describes the Job and File Transport Resource
Brokers and shows that a similar architecture can
be applied to solve both problems.
FOR MORE INFORMATION http//alien.cern.ch
AliEn Computing model The AliEn Computing model
is made of a central server, which keeps track of
all the jobs that have to be executed in the
system. Each site has at least one
ClusterMonitor. This service will ask the Broker
for jobs to execute (taking into account the
specifications of the site, like packages
installed or close storage elements), and will
send them to the Computing Elements. They will
send them to batch queues (like LSF or PBS), and
register the output in the system. There is as
well an optimizer, that checks the jobs waiting
to be executed. The optimizer is in charge of
modifying the waiting jobs. For example,
replicating the data that some jobs require in
sites with more computer power.
GP CALMA
AliEn Services AliEn has been implemented
having the idea of web services in mind. Most
of the services are built using SOAP (Simple
Object Access Protocol), which allows easy
communication among processes in a distributed
environment.
AliEn Transfer model The main component of the
transfer model is, like in the case of the
computing model, a database with all the
transfers that have to be done. A server inserts
new entries in the database, and an optimizer
checks the entries and sets the priorities for
the transfers. Each site runs a FTD (File
Transfer Daemon). Each FTD contacts the Broker
every minute and requests transfers.
ClassAds Classified Advertisements (classads) are
the lingua franca of Condor. They are used for
describing jobs, workstations, and other
resources. Classads are exchanged by Condor
processes to schedule jobs. AliEn uses classads
for the definition of jobs, machines and file
transfers. Two examples of classads are shown
below.
The transfers have to be made in several steps
first, the source FTD has to bring a copy from
the local mass storage system into a scratch
directory (this step is not necessary if bbFTP
can access the storage directly) then the remote
FTD can fetch the file and put it into its own
storage finally, the source FTD deletes the
scratch copy of the file.
Requirements ( other.Type "machine"
)(member(other.Packages,"AliRoot") ) Packages
"AliRoot" Arguments "--round 2002-02 --run
071 --event 269 --version v3.08.02 -grun GF"
Executable "/Alice/bin/AliRoot.sh" InputFile
LF/alice/simulation/2002-02/v3.08.02/0
0071/Config.C", "LF/alice/simulation/2002
-02/v3.08.02/00071/grun.C" Type "Job"
Requirements ( other.Type "Job" ) Packages
"ROOT3.03.07", "ROOT",
"AliRoot3.08.Rev.03", "AliRoot",
"AliRoot3.08.Rev.04" CloseSE
"AliceCERNCastor", "AliceCERNFile",
"AliceCERNscratch",
"AliceCERNpcepaip19" CE
"AliceCERNLXSHARE" Host "lxplus051.cern.ch
GridPartitions Production

W
E L C O M E to R O O T
Version 3.03/09
3 December 2002
You are welcome to visit our
Web site http//root.cern.ch

Comp
iled for linux with thread support. CINT/ROOT
C/C Interpreter version 5.15.61, Oct 6
2002 Type ? for help. Commands must be C
statements. Enclose multiple statements between
. root 0newanalysis-gtSubmit()
Job splitting AliEn also offers the possibility
of splitting jobs that require data spread over
several sites. Each site will compute their local
available data, and then the output of all the
jobs will be merged, giving back the output as if
it had run in only one place. The user should not
care at all where the job is executed.
Query for Input Data
?
Analysis Macro
MSS
MSS
Brokers The AliEn Brokers have a list with all
the classads of the tasks that have to be done,
and a priority for each task. Whenever a new
resource is free, it will send its classads to
the corresponding broker, The Broker will do the
matching with the highest priority task that
matches all the requirements. If no
CE
CE
MSS
CE
MSS
MSS
CE
CE
merged Trees Histograms
Job status A job in AliEn goes through the stages
shown in the diagram. First WAITING then a CE
picks it up and it is ASSIGNED QUEUED when it
is submitted to the local batch system RUNNING
when the job starts to execute on the worker
node, and finally DONE when it finishes AliEn
also provides a method to validate jobs. If a job
has to be validated, a new process for the
validation will start when the job finishes.
Then, the status of the job will be VALIDATED if
it passes the validation procedure, otherwise, it
will be marked as FAILED

• AliEn uses bbFTP for the file transfers. Every
  FTD runs a server, and all the others FTD can
  connect and authenticate to it using
  certificates.
• bbFTP implements its own transfer protocol which
  is optimized for large files (larger than 2GB).
  Its main features are
• SSH and Certificate authentication modules
• Big windows as defined in RFC1323
• Multi-stream transfer
• On-the-fly data compression
• Automatic retry

tasks are found, the resource will wait for a
while and request a new task

• Customizable time-outs
• Transfer simulation
• RFIO interface