HandsOn Session: Computing Element CE and site BDII Installation and Configuration

1
Hands-On SessionComputing Element (CE) and site
BDII Installation and Configuration

• Regional SEE-GRID-SCI Training for Site
  Administrators
• Institute of Physics Belgrade
• March 5-6, 2009

Dusan Vudragovic Institute of Physics Serbia dusan
_at_scl.rs
The SEE-GRID-SCI initiative is co-funded by the
European Commission under the FP7 Research
Infrastructures contract no. 211338
2
Overview
Information system
A CE is a grid batch queuewith a grid gate
front-end
Job request
LB
Logging
Loc. Info system
Gatekeeper
AA
Grid gate node
Local resource management systemCondor / PBS /
LSF master
Homogeneous set of worker nodes (WNs)
3
OS installation configuration

• Newest Scientific Linux series 4 (currently 4.7)
  should be installed (not SL5)
• Only 32-bit distribution is supported by lcg-CE
  and BDII_site so far
• We have chosen to install all base packages from
  5 SL4.7 CDs, then remove unnecessary
• Packages with great chances not to be used should
  be removed to speed up future software updates,
  i.e. openoffice.org
• Remove all LAM and OPENMPI packages, we'll be
  using MPICH
• Remove java-1.4.2-sun-compat package!
• Virtual environment is a possible solution

4
Further OS tuning

• Adjust services/daemons started at the boot time
• it is recommended to change the default runlevel
  to 3 in /etc/inittab
• disable yum auto-update, since this may bring
  trouble when new gLite updates appear
• If you install MPI_CE, it is suggested to disable
  SELINUX by replacing SELINUXenforcing with
  line SELINUXdisabled in the file
  /etc/selinux/config
• Configure NTP service
• Example of configuration file /etc/ntp.conf can
  be found on http//glite.phy.bg.ac.yu/GLITE-3/ntp.
  conf
• touch /etc/ntp.drift /etc/ntp.drift.TEMP
• chown ntp.ntp /etc/ntp.drift /etc/ntp.drift.TEMP
• chkconfig ntpd on

5
Repository adjustment

• DAG repo should be enabled by changing
  "enabled0" into "enabled1" in
  /etc/yum.repos.d/dag.repo
• Base SL repos must be PROTECTED, not allowing DAG
  packages to replace them! Add line protect1 to
  /etc/yum.repos.d/sl.repo and /etc/yum.repos.d/sl-e
  rrata.repo
• Following new files must be created in
  /etc/yum.repos.d
• lcg-ca.repo (Certification authorities packs)
• glite.repo (all gLite packages)
• jpackage5.0.repo (java stuff)
• contents of these files follow.....

6
Repository adjustment

• Local repository at SCL has been available since
  November 2008. Configuration files for the
  majority of repos can be found at
  http//rpm.scl.rs/yum.conf/

7
File system import/export

• Application software filesystem
• All WNs must have shared application software
  filesystem where VO SGMs (software grid managers)
  will install VO-specific software.
• If it's supposed to be located on CE itself,
  following (or similar) line must be appended to
  /etc/exports/opt/exp_soft 147.91.12.0/255.255.255
  .0(rw,sync,no_root_squash)
• If you want to map application software
  filesystem from other node (usually SE), append
  this line to /etc/fstabse.csk.kg.ac.yu/opt/exp_
  soft /opt/exp_soft nfs hard,intr,nodev,nosuid,tcp,
  timeo15 0 0Do not forget to create
  /opt/exp_soft directory!
• Shared /home filesystem
• In order to provide appropriate MPI support,
  entire /home must be shared among WNs.
• Procedure is equal to procedure for app. soft.
  filesystem

8
gLite software installation

• Valid host certificate must be present at
  /etc/grid-security
• gLite software binaries, libraries and other
  stuff are organized using meta-package paradigm.
  In order to install necessary packages for
  lcg-CE/BDII node with MPI support, following
  packages must be installed
• glite-BDII
• lcg-CE
• glite-TORQUE_server
• glite-TORQUE_utils
• glite-MPI_utils
• Due to temporary packaging inconsistency in
  glite-MPI_utils described in link, YUM command
  line must be
• yum install lcg-CE glite-BDII glite-TORQUE_server
  glite-TORQUE_utils glite-MPI_utils
  torque-2.1.9-4cri.slc4 maui-client-3.2.6p19_20.sna
  p.1182974819-4.slc4 maui-server-3.2.6p19_20.snap.1
  182974819-4.slc4 maui-3.2.6p19_20.snap.1182974819-
  4.slc4 torque-server-2.1.9-4cri.slc4
  torque-client-2.1.9-4cri.slc4

9
SSH configuration

• SSH must allow hostbased authentication between
  CE and WNs, as well as among WNs each other
• This is especially important if grid site
  supports MPI
• Helper script available in gLite can be found
  at/opt/edg/sbin/edg-pbs-knownhosts
• Script configuration can be adjusted
  in/opt/edg/etc/edg-pbs-knownhosts.conf
• Put all relevant FQDNs into /etc/ssh/shosts.equiv
• This is standard procedure for hostbased SSH
• Identical procedure applies to all WNs

10
gLite configuration

• All grid sevices must be configured properly
  using YAIM tool. Official info available at
  https//twiki.cern.ch/twiki/bin/view/LCG/YaimGuid
  e400
• Templates for input YAIM files can be taken
  fromhttps//viewvc.scl.rs/viewvc/yaim/trunk/?root
  seegrid
• Since YAIM is mainly a set of bash scripts,
  bash-like syntax must be used in input files
• Required input files are
• site-info.def
• users.conf
• wn-list.conf
• groups.conf
• directory vo.d with one file per VO
• YAIM config. files must not be readable for
  users!

11
gLite configuration

• site-info.def
• Main configuration input source
• Contains proper paths to all other configuation
  files
• users.conf
• Defines UNIX pool users for each Virtual
  Organization
• Helpful script at http//glite.phy.bg.ac.yu/GLITE-
  3/generate-pool-accounts-AEGIS-v4
• Example ./generate-pool-accounts-AEGIS-v4
  seegrid 20000 seegrid 2000 200 10 10 gtgt
  users.conf
• groups.conf
• Defines groups per VO, template can be employed
  as is.
• wn-list.conf
• Simple list of FQDNs of available Worker Nodes
• vo.d/
• Directory containing a file per each supported
  VO.

12
gLite configuration

• Following http//wiki.egee-see.org/index.php/SEE-G
  RID_MPI_Admin_Guide ,/opt/globus/setup/globus/pbs
  .inshould be replaced withhttp//cyclops.phy.bg.
  ac.yu/mpi/pbs.in before YAIM invocation in order
  to force WN to use local scratch instead of
  shared /home for single CPU jobs
• YAIM invocation command for lcg-CE/BDII_site
  combination with MPI support has to
  be/opt/glite/yaim/bin/yaim -c -s
  /path/to/site-info.def -n MPI_CE -n lcg-CE -n
  TORQUE_server -n TORQUE_utils -n BDII_site
• Note that MPI_CE has to be first in the line
• In case that YAIM returns an error anywhere in
  the procedure, check data in site-info.def and
  other input files and restart YAIM

13
MAUI post-configuration steps

• Verify that /var/spool/maui/maui.cfg contains the
  following lineADMIN3 edginfo rgma edguser
  tomcat
• Reserve a node for SAM test jobs in
  MAUIQOSCFGqossam MAXPROC1 PRIORITY100000GRO
  UPCFGprdseegrid QDEFqossam PRIORITY100000GROU
  PCFGsgmseegrid QDEFqossam PRIORITY100000SRCF
  Gsamreservation TASKCOUNT1RESOURCESPROCS1SR
  CFGsamreservation PERIODINFINITYSRCFGsamreser
  vation GROUPLISTprdseegrid,sgmseegridSRCFGsamr
  eservation HOSTLISTrti18.etf.bg.ac.yuSRCFGsamr
  eservation QOSLISTqossam
• If maui.cfg is modified, restart
  it/etc/init.d/maui restart

14
VO support

• SEEGRID VO
• Install latest seegrid RPM available
  athttp//www.irb.hr/users/vvidic/seegrid/
• AEGIS VO
• Put http//voms.phy.bg.ac.yu/voms.phy.bg.ac.yu.11
  9into /etc/grid-security/vomsdir
• ........

15
Testing configured system

• Verify local batching system
• qmgr -c "print server"
• Test if site properly provides info using GSTAT
  tool athttp//egee017.cnaf.infn.it/gstat/seegrid/
  
• GSTAT places delay of 15mins, but up-to-date
  info can be obtained using simple ldap client
  ldapsearch -x -H ldap//ltSITE_BDII_FQDNgt2170 -b
  mds-vo-nameltSITE-NAMEgt,ogrid
• Useful CE and SE info
• lcg-infosites --vo seegrid ce
• lcg-infosites --vo seegrid se

16
Helpful links

• http//wiki.egee-see.org/index.php/SG_GLITE-3_Guid
  e
• http//wiki.egee-see.org/index.php/SL4_WN_glite-3.
  1
• http//wiki.egee-see.org/index.php/SEE-GRID_MPI_Ad
  min_Guide
• https//twiki.cern.ch/twiki/bin/view/EGEE/GLite31J
  Package
• https//twiki.cern.ch/twiki/bin/view/LCG/YaimGuide
  400