CrossGrid Testbed Status

1
CrossGrid Testbed Status

• Jorge Gomes jorge_at_lip.pt
• LIP Computer Centre / X WP4

2
TESTBED EVOLUTION AND STATUS
3
X testbeds evolution
Development
testbed
Production
testbed
EDG 1.4.x
EDG 1.2.2/3
Production
testbed
Initial
testbed
EDG 1.4.x
EDG 1.2.2/3
Validation
testbed
Validation
testbed
EDG 1.4.8
Future
EDG 1.4.x
Development
testbed
Future
EDG 2.x

• Production testbed
• Used to run application prototypes.
• Validation testbed
• Test new production middleware releases after
  integration by the WP4 ITEAM and before
  deployment.
• Development testbed
• Development of middleware and tools and follow
  EDG.

4
Testbed resources

• CrossGrid has now 17 sites
• 12 are fully available.
• 3 have MDS related problems.
• 1 is not available to the X VO.
• 1 is still running EDG 1.2.x.

5
Site status
6
PRODUCTION TESTBED
7
The production Resource Broker
Amsterdam
Nikosia
Central site
Any
X
Athens
PROVAL
Lisbon/FCCN
Remote site
Krakow
5
Lisbon
2
Any
Resource
CrossGrid
JSS
broker
Karlsruhe
User
Interface
Santander
Valencia
Valencia
4
1
Bratislava
Bratislava
Information Index database
JDL Job
Poznan
Poznan
request
GRAM
Barcelona
rb01.lip.pt
snapshot
Warsaw
3
ii01.lip.pt
MDS
Santiago
Information Index
Thessaloniki
Dublin
8
The production Resource Broker (2)
9
Central production services
The CrossGrid production central services are
located in Lisbon and maintained by LIP.
Two sites are used LIP and FCCN



Central services
RB
FCCN NOC
II

LIP CC



Central services
RC
VO


Monitoring
10
Central production services (2)

• Resource Broker (rb01.lip.pt)
• Matchmaking and load balancing scheduler.
• Performs load sharing across X sites.
• Information Index (ii01.lip.pt)
• MDS/LDAP index of testbed resources CE SE.
• Certificate Proxy server (lngrid07.lip.pt)
• Short lived certificates for long lived
  processes.
• Used by the applications portal and by the RB.
• Virtual Organizations server (grid-vo.lip.pt)
• Database for user authentication.
• Is used to build the authorization databases of
  all X sites.
• Replica Catalogue (rc01.lip.pt)
• Database for physical replica file location.
• Central service to find the location of files in
  SEs.
• Network and Grid Monitoring (mapcenter.lip.pt)
• Early detection of problems in the X testbed.

11
Central production services (3)

• The OLD production services running 1.2.x have
  been removed.

X
12
The production Replica Catalogue

• The production RC
• This is basically an LDAP server.
• Hosted at rc01.lip.pt port 9980.
• Is used by the RB and RM.

URL ldap//rc01.lip.pt9980/ lccg,rcCG Replica
Catalogue,dcrc01,dclip,dcpt
13
Major production problems

• MDS
• Hanged servers (CE/SE) are still capable of
  blocking the II -gt RB.
• Information in the DBII is not purged.
• The MDS tree is too slow.
• Information provided by MDS might not be
  accurate.
• RB
• Stops scheduling jobs and needs to be restarted
  frequently.
• The LB stops answering queries and needs to be
  restarted frequently.
• VO
• Users still cant be in more than one VO without
  problems.

14
VALIDATION TESTBED
15
Validation sites status

• The validation testbed is used to test and
  validate software distributions before their
  deployment in the testbed and after the WP4
  integration.
• Currently is composed of resources from the two
  main sites.
• Some of the test and validation testbed
  resources are not permanently allocated.
• Some resources and even sites are allocated to
  tests on demand.

16
The validation parallel Resource Broker
CONFIGURED CE
Lisbon
Central site
Any
X
Lisbon/FCCN
Remote site
CONFIGURED CE
5
MODIFIED
2
MODIFIED
Resource
CrossGrid
Valencia
Valencia
JSS
broker
User
Interface
CONFIGURED CE
4
1
Barcelona
Information Index database
JDL Job
request
GRAM
lngrid03.lip.pt
3
MDS
ii01.lip.pt
Information Index
17
Central validation services
The CrossGrid validation central services are
located in Lisbon and maintained by LIP.
MyProxy RB RC VO UI Monitoring II
Certification proxy Resource broker Replica
catalogue Virtual organisation server User
interface Grid monitoring Information Index
I I
MyProxy


RB
Central services

RC
VO


Monitoring
18
DEVELOPMENT TESTBED
19
Development sites status

• There are two kinds of development sites
• Running 1.4.x
• Running 2.x
• Some of the development testbed resources are
  not permanently allocated.

20
Development sites status (2)

• EDG 2.x has significant changes from 1.4.x
• RH 7.3 LCFGng.
• VDT distribution of Globus.
• Replica location service.
• R-GMA replacing MDS (partially).
• Glue schema.

21
SUPPORT SERVICES
22
Virtual Organizations

• CrossGrid has its own VO server
• The VO server is used to build the authorization
  databases of the X testbed systems.
• Currently is an LDAP server.
• Hosted at grid-vo.lip.pt port 9990.
• CrossGrid users can send their VO membership
  requests to vo.admin_at_lip.pt
• 74 users are registered in the crossgrid VO.

23
Certification Authorities

• Five new CAs were created and are now recognized
  by CrossGrid.
• All CAs are operational issuing certs and CRLs.
• All CAs are recognized by DataGrid.

24
Helpdesk

• The CrossGrid helpdesk application is READY.
  PLEASE USE IT !

http//cg1.ific.uv.es/hlpdesk/
25
How to get support

• The CrossGrid helpdesk application
• http//cg1.ific.uv.es/hlpdesk/
• The CrossGrid testbed web pages
• http//grid.ifca.unican.es/crossgrid/wp4
• http//www.lip.pt/computing/cg-pro (EDG 1.4.x)
• http//www.lip.pt/computing/cg-tv (EDG
  1.4.x)
• http//www.lip.pt/computing/cg-dev (EDG 2.0)
• Installation manual
• http//cgi.di.uoa.gr/xgrid/cgfiles/Guide_2.0b1.pd
  f
• Email
• crossgrid-wp4-support_at_lists.cesga.es
• grid.support_at_lip.pt

26
Production testbed web site
http//www.lip.pt/computing/cg-pro
27
Testbed monitoring
Mapcenter grid monitoring framework. Mapcenter
was developed by DataGrid and adapted to
CrossGrid by LIP. Enhancements implemented
by LIP in cooperation with DataGrid.
http//mapcenter.lip.pt
28
Production RB statistics
Graphics with RB statistics available
from Mapcenter
29
Production RB statistics (2)

• Most of the failures are related with
• Authorization
• Matchmaking
• Testbed sites

30
Testbed CE statistics
Graphics with CE statistics available
from Mapcenter
31
Testbed CEs statistics (2)

• Production gatekeepers statistics for May of 2003

32
Testbed CEs statistics (3)

• Production gatekeepers statistics for June of 2003

33
Testbed downtime statistics
34
Host check tool
Host Check grid host checker. Host Check was
developed by LIP to support the CrossGrid testbed
deployment. Host Check produces a detailed
report for each testbed CE and SE.
http//www.lip.pt/computing/cg-services/site_check
35
Host check tool (2)
Host Check grid host checker. Host Check was
developed to support the CrossGrid testbed
deployment. Host Check produces a detailed
report for each testbed CE and SE.
http//www.lip.pt/computing/cg-services/site_check
36
Host check tool (3)
Host Check Testbed summary. Produces a summary
of the status of all CEs and SEs. Only major
errors are shown.
http//www.lip.pt/computing/cg-pro/testbed-sites.h
tm
37
CRL check
CRL issuance verifier. Was developed by to
support the CrossGrid testbed. Verifies whether
the CRLs are valid and warns CA administrators.
http//www.lip.pt/computing/cg-services/tests/chec
k_ssl
38
FINALLY
39
GGF Grid School

• The GGF Grid School took place in mid July.
• X has joined the EDG dissemination testbed
  (GriDis) for the GGF Grid Summer School
  tutorials.
• X has participated with four sites
• FZK
• CESGA
• UoA
• FUW

40
Future

• Test and deploy this release of CrossGrid
  middleware.
• Follow the test and validation procedure.
• Define clearly the development testbed.
• Sites and resources.
• Versions 1.4.x / 2.x.
• Follow the EDG 2.x developments.
• Follow the tag releases.
• Prepare for site validation and deployment in EDG
  2.x (tests, web site, docs).
• Study the usage of QoS in CrossGrid.
• Create a QoS test infrastructure.

41
E N D
LIP
FKZ
IFIC
IFCA