Procedures%20on%20how%20to%20enter%20the%20GRID%20%20Christos%20Papachristos%20([e-mail])%20Site%20Manager%20of%20the%20HG-05-FORTH%20and%20GR-04-FORTH-ICS%20nodes%20Distributed%20Computing%20Systems%20Lab

1
Procedures on howto enter the GRID Christos
Papachristos(cpapachr_at_ics.forth.gr)Site
Manager of theHG-05-FORTH and GR-04-FORTH-ICS
nodesDistributed Computing Systems Lab
2
Answers provided to

• Who can apply for registration?
• What is the registration procedure?
• Who should be contacted?
• What are the Virtual Organizations?
• What tools do I need and where can I find them?
• How do I submit jobs to the Grid?
• What is the procedure of submitting jobs?
• Tutorials on job submission.
• Failed jobs. Who should be contacted?
• Where can I see the status of the Grid?
• What does the infrastructure look like?
• Is there any user support?
• ..and some other Grid related issues.

3
Who can apply for registration?

• The whole Greek research community
• Universities
• Technical Education Centers
• Research Centers
• The Eligible (global) Grid User Community is
  defined to be people from 
• the academic research and education community
• public funded bodies, institutes and research
  laboratories working in areas related to research
  and education
• other bodies and organisations, such as
  commercial companies, who are collaborating on
  academic research and education with the above
  institutes.
• EGEE Access Users Policy (AUP)

4
Registration procedure

• Register yourself as a HellasGRID user
• Get an account to a User Interface node (UI)
• Acquire a digital certificate from your
  accredited National Certification Authority (CA)
• Join an existing Virtual Organization (VO)

5
HellasGRID registration. Step 1

• The first step in order to enroll is to fill the
  User Registration form at the HellasGrid
  websitehttp//www.hellasgrid.gr/user-request
• This will send a notification to the User Support
  Team which will get in touch with you in order to
  clarify your needs and help you with your first
  steps in joining the EGEE Grid infrastructure.

6
Get a UI account. Step 2

• In order to become a user of the EGEE Grid
  Infrastructure you should
• Gain access to a User Interface (UI) machine
  which hosts the required client software for
  developing and running applications in the EGEE
  Grid
• A UI is a Linux box having the appropriate client
  software that will help you send and execute your
  applications in the Grid
• The choices on how to obtain a UI account are
• Set up your own UI node
• Obtain access to a local UI, already installed in
  your organization
• hellasgrid-ui.ics.forth.gr. Send an email at
• cpapachr, hargikas_at_ics.forth.gr
• Request for a UI account in the GRNET node

7
Get a UI account. Step 2.1

• In practice everyone can install and setup a UI
  with the required EGEE tools following the
  instructions in the LCG-2 Installation Guide.
• Also the Italian SA1 EGEE Activity has made
  available a package called UIPnP which can be
  installed in any Linux machine and turn it into a
  User Interface (you can even install it as a
  regular non-root user). Note though that it is
  pre-configured for accessing the Italian EGEE
  sites and some manual configuration steps are
  required at the moment in order to be useful for
  EGEE-SEE users (for instructions see How to
  customize UIPnP for SEE-VO).

8
Acquire a Digital Certificate. Step 3

• In order to access the Grid you need a Digital
  Certificate.
• First of all a fax has to be sent to the National
  Certification Authority (fax [phone]) with
  names and IDs.
• Then
• Login to your UI account and execute the
  following
• grid-cert-request
• The command will produce a .globus directory with
  3 files .globus/usercert.pem size0.globus/user
  cert_request.pem sizeltgt0.globus/userkey.pem
  sizeltgt0
• The user-cert-request.pem file has to be emailed
  to the HellasGrid-CA (hellasgrid-ca_at_physics.auth.g
  r).

9
Acquire a Digital Certificate. Step 3.1

• After you receive your signed certificate
• Overwrite the usercert.pem file with the signed
  certificate (e.g cp
  cpapachr_at_ics.forth.gr.pem /home/cpapachr/.globus/u
  sercert.pem
• Import your certificate to your email and web
  browsers
• Transform the certificate to pkcs12 format by
  executing the following command in the UI
  openssl pkcs12 -in
  usercert.pem -inkey userkey.pem -export -out
  certkey.p12
• Follow the instructions to import the certificate
  to the browser http//lcg.web.cern.ch/LCG/users/re
  gistration/load-cert.html
• Return a signed email to the HellasGRID CA
  accepting the certificate.

10
Join an existing VO. Step 4

• VOVirtual Organization. A group of scientists
  with common research interests.
• Most popular VOs can be found here,
• https//lcg-registrar.cern.ch/virtual_organization
  .html
• Most VOs are those of CERN dealing with
  experiments based on data that will be produced
  from the LHC (Large Hadron Collider) which will
  be fully operational in 2007.
• In our region we have the SEE VO which is
  supported by 27 sites.
• http//mon.egee-see.org

11
Our SEE VO

• In order to make life easier for the regional
  users, speed up and simplify the process of new
  application induction, SEE has established its
  own VO called SEE-VO.
• This VO will be the most adequate for SEE users
  that do not fit in any of the existing VOs.
• In order to request acceptance to the SEE-VO as a
  user you have to fill the following request form
  https//www.grid.auth.gr/services/voms/SEE/request
  .php
• Please note that this page has to be visited
  using the browser on which you have already
  loaded your digital certificate otherwise the
  process cannot be completed.

12
Brief summary of the procedure
HOME/.globus/usercert-request.pem HOME/.globus/u
serkey.pem HOME/.globus/usercert.pem
Start using the GRID
CA
UI
grid-cert-request
Generate pkcs12 file
Import pkcs12 file into mailer/browser
Public key is signed and the Certificate is issued
13
Ready to use the infrastructure!
14
A simplified overview of a job workflow
Resource Broker
Author. Authen.
15
How do I submit jobs to the Grid?

• Create a proxy certificate
• Write your own .jdl file (JDLJob Description
  Language file)
• Submit your job
• Retrieve the status of the job
• Get the output of the job

16
Step 1- Creating a proxy certificate
cpapachr_at_hellasgrid-ui cpapachr
grid-proxy-init -debug -verify User Cert File
/home/cpapachr/.globus/usercert.pem User Key
File /home/cpapachr/.globus/userkey.pem Trusted
CA Cert Dir /etc/grid-security/certificates Outpu
t File /tmp/x509up_u0 Your identity
/CGR/OHellasGrid/OUics.forth.gr/CNChristos
Papachristos Enter GRID pass phrase for this
identity Creating proxy .. ......
Done Proxy Verify OK Your proxy is
valid until Wed Jun 14 024917
2006 cpapachr_at_hellasgrid-ui cpapachr
17
Step 2 Creating the JDL file

• test.jdl
• test.sh

Executable "test.sh" StdOutput
"test.out" StdError "test.err" InputSandb
ox "./test.sh" OutputSandbox
"test.out","test.err" Requirements
other.GlueCEUniqueID"grid001.ics.forth.gr2119/j
obmanager-lcgpbs-dteam" RetryCount 0
!/bin/bash echo Hostname hostname echo
Working Directory pwd echo Running As
id echo Executed in date echo Enviroment
Variables env
18
Step 3 Submitting the job

• cpapachr_at_hellasgrid-ui cpapachr edg-job-submit
  --vo dteam test.jdl
• Selected Virtual Organisation name (from --vo
  option) dteam
• Connecting to host rb.isabella.grnet.gr, port
  7772
• Logging to host rb.isabella.grnet.gr, port 9002
• 
  
• JOB SUBMIT OUTCOME
• The job has been successfully submitted to the
  Network Server.
• Use edg-job-status command to check job current
  status. Your job identifier (edg_jobId) is
• - https//rb.isabella.grnet.gr9000/GgfNDFsgB0L3b
  tK_2CRfKQ
• Keep this URL for further reference to the job
  already submitted
• 
  
• cpapachr_at_hellasgrid-ui cpapachr

19
Step 4 Querying the job status

• cpapachr_at_hellasgrid-ui cpapachr edg-job-status
  https//rb.isabella.grnet.gr9000/GgfNDFsgB0L3btK_
  2CRfKQ
• 
  
• BOOKKEEPING INFORMATION
• Status info for the Job https//rb.isabella.grne
  t.gr9000/GgfNDFsgB0L3btK_2CRfKQ
• Current Status Scheduled
• Status Reason Job successfully submitted to
  Globus
• Destination grid001.ics.forth.gr2119/jobm
  anager-lcgpbs-dteam
• reached on Tue Jun 13 155426 2006
• Finally after a couple of minutes
• cpapachr_at_hellasgrid-ui cpapachr edg-job-status
  https//rb.isabella.grnet.gr9000/GgfNDFsgB0L3btK_
  2CRfKQ
• 
  
• BOOKKEEPING INFORMATION
• Status info for the Job https//rb.isabella.grne
  t.gr9000/GgfNDFsgB0L3btK_2CRfKQ
• Current Status Done (Success)
• Exit code 0
• Status Reason Job terminated successfully

20
Step 5 Retrieving the results

• cpapachr_at_hellasgrid-ui cpapachr
  edg-job-get-output https//rb.isabella.grnet.gr90
  00/GgfNDFsgB0L3btK_2CRfKQ
• Retrieving files from host rb.isabella.grnet.gr
  ( for https//rb.isabella.grnet.gr9000/GgfNDFsgB0
  L3btK_2CRfKQ )
• 
  
• JOB GET OUTPUT OUTCOME
• Output sandbox files for the job
• - https//rb.isabella.grnet.gr9000/GgfNDFsgB0L3b
  tK_2CRfKQ
• have been successfully retrieved and stored in
  the directory
• /tmp/jobOutput/cpapachr_GgfNDFsgB0L3btK_2CRfKQ
• 
  

21
Tutorials on job submission

• The LCG-2 User Guide describing almost everything
• https//edms.cern.ch/file/454439//LCG-2-UserGuide.
  html
• A small tutorial with information on JDL files
  and job submission
• https//edms.cern.ch/file/498081//UserScenario2.pd
  f
• There is also MPI support (MPICH v1.2.5). A test
  example can be found here
• http//quattor.web.lal.in2p3.fr/packages/mpi/

22
Failed job submissions

• Who do I contact if my jobs fail?
• GGUS system is the main user support framework.
• https//gus.fzk.de/pages/home.php
• Contact via email as well
• ltMyVOgt-user-support_at_ggus.org (ltMyVOgtatlas,
  biomed, ...)
• helpdesk_at_ggus.org

23
Monitoring the Grid

• Where can I see the status of the Grid?
• The Grid Operations Centre (GOC) maintains a map
  with the overall state of the grid and links to
  various monitoring pages.
• http//goc.grid-support.ac.uk/gridsite/monitoring/
  
• http//goc02.grid-support.ac.uk/googlemaps/lcg.htm
  l
• Our ROC also keeps track of the regional Grid
  resources of the SEE
• http//goc.grid.sinica.edu.tw/gstat/SouthEasternEu
  rope.html
• http//mon.egee-see.org/gridice/site/site.php

24
Monitoring services to check

• GridICE monitoring server
• http//mon.egee-see.org
• General status of the Grid
• http//goc.grid.sinica.edu.tw/gstat/
• Network latency monitoring
• http//mon.egee-see.org/cgi-bin/smokeping.cgi

25
A closer look at the infrastructure

• The HG-05-FORTH site

26
HG-05-FORTH

• Software
• 1 CE
• 1 SE (SE_dpm_mysql)
• 1 MON BOX
• 1 Quattor
• 60 WNs
• GPFS on SE and quattor
• (storage)
• All running
• LCG middleware v. 2_7_0
• Operating System SL3.05

• Hardware
• 60 nodes with dual Intel xeon processors
• 3.4GHz each
• SATA disk 80GB
• 4 nodes with dual Intel xeon processors
• 3.4GHz each
• 2 SCSI disks RAID (80GB each)
• 4.2TB of storage (300GBx12disks)
• 6 CISCO 3750 switches

27
User Support Team

• Operated by NCSR Demokritos - Institute of
  Nuclear Physics (http//grid.inp.demokritos.gr/)
• Responsibilities To guide new users in their
  first steps in the Grid and provide continuous
  support in their activities.
• Email (egee-user-support_at_inp.demokritos.gr)
• Wiki (http//wiki.egee-see.org/index.php/Users)
• Mailing list (user-info_at_grnet.gr) Includes all
  registered grid users
• Newsletter Published every 2 months in Greek.
  Provides local and EGEE-wide news update
  (http//grid.inp.demokritos.gr/indexegeenewsletter
  .htm)

28
For urgent matters

• The site managers can be contacted.
• cpapachr, hargikas_at_ics,forth.gr

29
MPI example MPItest.c

• include "mpi.h"
• include ltstdio.hgt
• int main(int argc, char argv)
• int numprocs / Number of processors /
• int procnum / Processor number /
• / Initialize MPI /
• MPI_Init(argc, argv)
• / Find this processor number /
• MPI_Comm_rank(MPI_COMM_WORLD, procnum)
• / Find the number of processors /
• MPI_Comm_size(MPI_COMM_WORLD, numprocs)
• printf ("Hello world! from processor d out of
  d\n", procnum, numprocs)
• / Shut down MPI /
• MPI_Finalize() return 0

30
MPI example MPItest.jdl

• Type "Job"
• JobType "MPICH"
• NodeNumber 10
• Executable "MPItest.sh"
• Arguments "MPItest"
• StdOutput "test.out"
• StdError "test.err"
• InputSandbox "MPItest.sh","MPItest.c"
• OutputSandbox "test.err","test.out","mpiexec.ou
  t"
• Requirements (other.GlueCEInfoLRMSType
  "PBS") RegExp("grid..lal.in2p3.fr.",other.Glu
  eCEUniqueID)

31
MPI example MPItest.sh

• !/bin/sh -x
• the binary to execute
• EXE1
• echo "
• echo "Running on HOSTNAME"
• echo "As " whoami
• echo "
• echo "
• echo "Compiling binary EXE"
• echo mpicc -o EXE EXE.c
• mpicc -o EXE EXE.c
• echo "
• if "xPBS_NODEFILE" ! "x" then
• echo "PBS Nodefile PBS_NODEFILE"
• HOST_NODEFILEPBS_NODEFILE
• fi
• if "xLSB_HOSTS" ! "x" then
• echo "LSF Hosts LSB_HOSTS"
• HOST_NODEFILEpwd/lsf_nodefile.

echo " CPU_NEE
DEDcat HOST_NODEFILE wc -l echo "Node
count CPU_NEEDED" echo "Nodes in
HOST_NODEFILE " cat HOST_NODEFILE echo
" echo
" CPU_NEEDED
cat HOST_NODEFILE wc -l echo "Checking ssh
for each node" NODEScat HOST_NODEFILE for
host in NODES do echo "Checking host..."
ssh host hostname done echo "
echo "
echo "Executing EXE with mpiexec"
chmod 755 EXE mpiexec pwd/EXE gt mpiexec.out
2gt1 echo "
echo " echo
"Executing EXE with mpirun" chmod 755 EXE
mpirun -np CPU_NEEDED -machinefile
HOST_NODEFILE pwd/EXE echo
"