Globus Grid Tutorial Part 3: Information Services - PowerPoint PPT Presentation

1 / 37
About This Presentation
Title:

Globus Grid Tutorial Part 3: Information Services

Description:

Learn how to use the Globus Metacomputing Directory Service to locate and ... Lightweight Directory Access Protocol. Stripped down version of X.500 DAP protocol ... – PowerPoint PPT presentation

Number of Views:66
Avg rating:3.0/5.0
Slides: 38
Provided by: szt4
Category:

less

Transcript and Presenter's Notes

Title: Globus Grid Tutorial Part 3: Information Services


1
Globus Grid TutorialPart 3Information Services
2
Goals of this Tutorial
  • Learn how to use the Globus Metacomputing
    Directory Service to locate and determine
    characteristics of resources
  • Locate resources
  • Where are resources with required architecture,
    installed software, available capacity, network
    bandwidth, etc.?
  • Determine resource characteristics
  • What are the physical characteristics,
    connectivity, capabilities of a resource?

3
The Need for Information
  • System information is critical to operation of
    the grid and construction of applications
  • How does an application determine what resources
    are available?
  • What is the state of the computational grid?
  • How can we optimize an application based on
    configuration of the underlying system?
  • We need a general information infrastructure to
    answer these questions

4
Using Information forResource Brokering
Info service location selection
Metacomputing Directory Service
Resource Broker
What computers? What speed? When available?
20 Mb/sec
GRAM
Globus Resource Allocation Managers
50 processors storage from 1020 to 1040 pm
Fork LSF EASYLL Condor etc.
5
Examples of Useful Information
  • Characteristics of a compute resource
  • IP address, software available, system
    administrator, networks connected to, OS version,
    load
  • Characteristics of a network
  • Bandwidth and latency, protocols, logical
    topology
  • Characteristics of the Globus infrastructure
  • Hosts, resource managers

6
Information Infrastructure
  • Provide access to static and dynamic information
    regarding system components
  • A basis for configuration and adaptation in
    heterogeneous, dynamic environments
  • Requirements and characteristics
  • Uniform, flexible access to information
  • Scalable, efficient access to dynamic data
  • Access to multiple information sources
  • Decentralized maintenance

7
The GlobusMetacomputing Directory Service
  • Store information in a distributed directory
  • Directory stored in collection of servers
  • Directory can be updated by
  • Globus system
  • Other information providers and tools
  • Applications (i.e., users)
  • Information dynamically available to
  • Tools
  • Applications

8
Directory Service Functions
  • White Pages
  • Look up the IP number, amount of memory, and so
    forth, associated with a particular machine
  • Yellow Pages
  • Find all the computers of a particular class or
    with a particular property
  • Tuned for high-volume lookup or search
  • Temporary inconsistencies are considered okay

9
The Metacomputing Directory Service
Application
  • Standard interface and protocol
  • Lightweight Directory Access Protocol (LDAP)
  • Add gateways and pragmas
  • Globus specific data model
  • Host-centric representation
  • Data discovery update

Middleware
LDAP API represent.
Data model
MDS Client Library

SNMP
NWS
NIS
LDAP
10
MDS Components
  • Standard LDAP server
  • Netscape Directory Server v4
  • Tools for populating maintaining MDS
  • Integrated with Globus server release, not of
    concern to most Globus users
  • Discover static info, update dynamic info
  • APIs for accessing updating MDS contents
  • C, Java, PERL (LDAP API, JNDI)
  • Various tools for manipulating MDS contents
  • Command line tools, Shell scripts GUIs

11
LDAP Directory Service
  • Lightweight Directory Access Protocol
  • Stripped down version of X.500 DAP protocol
  • Supports distributed storage/access (referrals)
  • Supports replication
  • Becoming de facto standard
  • Defines
  • Network protocol for accessing directory contents
  • Information model defining form of information
  • Namespace defining how information is referenced
    and organized

12
Directory Structure
  • Directory contents
  • What information is stored in directory
  • Group related information into entries
  • Directory organization
  • Objects organized into tree structure
  • Position of object in tree uniquely names entry

13
Entries and Object Classes
  • Directory entry consists of related set of
    attributes and values
  • Entries are typed by their Object Class
  • Object class specifications define
  • Required and optional attributes
  • Attribute types
  • Single inheritance on object classes

14
Sample Object Classes
  • Compute Resources
  • Operating System
  • Memory Hierarchy
  • Health and Status
  • Network Interfaces
  • IP address
  • Interface types
  • Performance Data
  • Schedule Jobs
  • CPU Loads
  • Network Traffic
  • Resource Managers
  • Contact strings
  • Scheduled jobs
  • Free nodes
  • Software
  • Configuration
  • Version Control
  • Contact information
  • Organizations
  • People

15
Object Class Specification
GlobusHost OBJECT CLASS SUBCLASS OF
GlobusPhysicalResource RDN hn (hostName)
CHILD OF
organization
MUST CONTAIN type cis,
vendor cis, model cis,
ostype cis, osversion cis
MAY CONTAIN
networkNode dn, totalMemory
cis, totalSwap cis,
. . .
GlobusPhysicalResource OBJECT CLASS SUBCLASS OF
GlobusTop RDN rn (resourceName) MUST
CONTAIN imageObject dn
MAY CONTAIN manager dn,
provider dn, technician dn,
description cis, documentation cis

16
The Directory Information Tree
  • Directory entries organized into tree
  • Called Directory Information Tree (DIT)
  • Subtrees can be distributed or replicated
  • Position in tree uniquely names entry
  • Each object is uniquely determined by its
    distinguished name (DN)
  • List of unique attribute names and values along
    path from root of DIT to object, e.g.
  • lthnsp2.sdsc.edu, ouSDSC, oNPACI, oGlobus,
    cUSgt

17
MDS Data Model
Physical Structure
Directory Information Tree
cUS
oglobus
IBM SP
sunny
WAN
LAN
oUSC
oANL
nnWAN
hot
LAN
dark
cold


ouMCS
ouISI
nnMCS-lan
Steve
Ian
Steve
Carl
Gregor
Warren
cnCarl
nnSP-switch

ANL/MCS
USC/ISI
nnSP-ether
cnSteve
cnIan
Distinguished Name
hncold.mcs
cnGregor
lthn dark.mcs.anl.gov, ou MCS, o Argonne
National Laboratory, o Globus, c USgt
hndark.mcs
cnSteve
hnsp1.mcs.anl.gov
cnWarren

hnspN.mcs.anl.gov
18
MDS Tools
  • Web-based browsers and displays
  • MDS Object Class Browser
  • MDS Explorer
  • Various APIs and search tools
  • Translators from Globus Object Definition
    Language
  • Commented LDIF
  • LDAP schema definition language
  • HTML

19
MDS Object Class Browser
20
(No Transcript)
21
MDS Access/Update Commands
  • LDAP defines a set of standard commands
  • ldapsearch, ldapmodify, ldapdelete, etc.
  • We also define MDS-specific commands
  • grid-info-search, grid-info-create,
    grid-info-update, grid-info-remove
  • Routines to ensure data consistency and to
    insert metadata
  • APIs are defined for C, Java, etc.
  • ldap_search_s, ldap_modify_s, .

22
Searching an LDAP Database
  • grid-info-search options filter attributes
  • Default grid-info-search options
  • -h mds.globus.org MDS server
  • -p 389 MDS port
  • -b oGlobus, cUS search start point
  • -T 30 LDAP query timeout
  • -s sub scope subtree
    alternatives
  • base lookup this entry
  • one lookup immediate children

23
grid-info-search Filter
  • Filters allow selection of object based on
    relational operators (, ,lt, gt)
  • grid-info-search cputype
  • Compound filters can be construct with Boolean
    operations (, , !)
  • grid-info-search ((cputype)(cpuload1lt1.0))
  • grid-info-search ((hnsdsc.edu)(latencylt10))
  • Hints
  • white space is significant
  • use -L for LDIF format

required
24
Example 1 grid-info-search (mnsdsc.edu)
cnsp097.sdsc.edu-loadleveler, ouSDSC, oNPACI,
oGlobus, cUS objectclassGlobusResourceManager c
nsp097.sdsc.edu-loadleveler mnsp097.sdsc.edu-loa
dleveler hnsp097.sdsc.edu contactsp097.sdsc.edu
754/CUS/OGlobus/ONPACI/OUSDSC/ \
CNsp097.sdsc.edu-loadleveler apiversion0.1 apity
peloadleveler managedresourceshnsp097.sdsc.edu,
ouSDSC, oNPACI, \ oGlobus,
cUS gramversion1.53 gramversiondate1998/06/1921
1723 gramsecurityssleay freenodes49 totalnodes
117 lastupdateSat Jun 27 182334 GMT
1998 scheduledjobnone
25
Example 2grid-info-search -L (swGlobus)
dn swGlobus, hnhost.domain.edu, oMy
Organization, oGlobus, cUS sw
Globus objectclass GlobusSoftware releasemajor
1 releaseminor 1 releasepatch 0 configuration
--prefix/cpc/globus/service/sun4-sparc2 \
--with-ssl-path/cpc/ssl/sun4-sparc2
configuration --prefix/cpc/globus/tools/sun4-sp
arc2 \ --with-ssl-path/cpc/ssl/sun4-sparc2
installprefix /cpc/globus/ serviceprefix
/cpc/globus/service toolsprefix
/cpc/globus/tools deployprefix
/opt/globus/ userid globus objectname
swGlobus, hnhost.domain.edu, oMy Organization,
\ oGlobus, cUS lastupdate Wed Jan 20,
060613 GMT 1999
26
grid-info-search Attributes
  • Projections on the data objects are performed by
    supplying a list of attributes
  • Retrieve the DN for all objects in the MDS
  • grid-info-search -L (objectclass) dn

dn oGlobus, cUS dn oUniversity of Southern
California, oGlobus, cUS dn ouMCS, oArgonne
National Laboratory, oGlobus, cUS dn oNASA
Ames, oGlobus, cUS dn ouSDSC, oNPACI,
oGlobus, cUS
27
Multiple Attribute Selection
  • grid-info-search -L (objectclass) dn hn
  • Returns the distinguished name (dn) and
    hostname (hn) of all objects
  • Objects without hn fields are still listed
  • DNs are always listed

dn oCalTech, oGlobus, cUS dn
hnneptune.cacr.caltech.edu, oCalTech, oGlobus,
cUS hn neptune.isi.edu dn ouSDSC, oNPACI,
oGlobus, cUS dn oUniversity of Southern
California, oGlobus, cUS

28
Efficient Lookups
  • Naïve searches retrieve lots of information
  • grid-info-search (objectclass)
    grep contact
  • Whenever possible, reduce search domain
  • Specify a search base (-b)
  • grid-info-search -b oNASA Ames,
    oGlobus, CUS
  • Specify the scope of the search (-s)
  • options base, one, and sub
  • Narrow the amount of objects selected, i.e., use
    specialized filters
  • Specify which attributes you want

29
Authentication to MDS
  • Authentication is required to perform certain
    operations (e.g., write operations)
  • Each site has a Directory Manager
  • cnDirectory Manager, oOrganization, cUS
  • Users registered with the MDS have a DN
  • cnJane Doe, oOrganization, cUS
  • grid-info-search -D cnJane Doe,
    oOrganization, cUS \
  • -w ltpasswdgt

30
Updating MDS Entries
  • To update an existing database entry
  • grid-info-modify -D "cnDirectory Manager,
    oGlobus, cUS \ -w
    ltPASSWORDgt -f sample.ldif
  • dn hntuva.mcs.anl.gov, ouMCS, oArgonne
    National Laboratory, oGlobus, cUS
  • cpuload1 3.11
  • cpuload5 2.64
  • cpuload15 2.57
  • Metadata is inserted by grid-info-update
  • grid-info-update -D "cnDirectory Manager,
    oGlobus, cUS \ -w
    ltPASSWORDgt -f sample.cldif

31
Example Discovering CPU Load
  • Retrieve CPU load fields of compute resources
  • grid-info-search -L (objectclassGlobusComputeRes
    ource) \
  • dn cpuload1 cpuload5 cpuload15

dn hnlemon.mcs.anl.gov, ouMCS, oArgonne
National Laboratory, oGlobus, cUS cpuload1
0.48 cpuload5 0.20 cpuload15 0.03 dn
hntuva.mcs.anl.gov, ouMCS, oArgonne National
Laboratory, oGlobus, cUS cpuload1
3.11 cpuload5 2.64 cpuload15 2.57
32
Exercise 3Extended Exercises
  • Explore the MDS via
  • MDS object code Browser
  • http//www.globus.org/mds
  • MDS Navigator
  • http//www.globus.org/testbeds
  • Globus View
  • http//www.globus.org/testbeds
  • Write a set of MDS/LDAP queries
  • grid-info-search hnisi.edu

33
MDS Search Exercises
  • 1. List all hosts that have a lsf jobmanager
    type.
  • 2. List all hosts that have a CPU load greater
    than 5.
  • 3. List all hosts with 16 or more CPUs.
  • 4. Determine the current versions of Globus
    installed at Argonne.
  • 5. Determine the deployment directory on
    flash.isi.edu.
  • 6. Determine the bandwidth between
    pitcairn.mcs.anl.gov and bolas.isi.edu.

34
MDS Search Answers (1)
  • 1. List all hosts that have a lsf gatekeeper.
  • grid-info-search apitypelsf hostname
  • 2. List all hosts that have a CPU load greater
    than 5.
  • grid-info-search cpuload1gt5 hostname
  • 3. List all hosts with 16 or more CPUs.
  • grid-info-search cpucountgt16 hostname
  • 4. Determine the current versions of Globus
    installed at Argonne.
  • grid-info-search \ -b oArgonne
    National Laboratory, oGlobus, cUS \
    swGlobus releasemajor releaseminor
    releasepatch

35
MDS Search Answers (2)
  • 5. Determine the deployment directory on
    flash.isi.edu.
  • grid-info-search \ ((objectclassResou
    rceManager)(hnflash.isi.edu)) \ deploydir
  • 6. Determine the bandwidth between
    pitcairn.mcs.anl.gov and bolas.isi.edu.
  • grid-info-search \ (((sourcehostnamepit
    cairn.mcs.anl.gov) \
    (destinationhostnamebolas.isi.edu)) \
    ((sourcehostnamebolas.isi.edu) \
    (destinationhostnamepitcairn.mcs.anl.gov)))
    bandwidth

36
Summary
  • MDS provides the information needed to perform
    dynamic resource discovery and configuration
  • Critical component of resource brokers
  • MDS is base on existing directory service
    standards (LDAPv3)

37
Changes from 1.0 to 1.1
  • Tools are renamed
  • globus-mds-search,add,delete,modify became
  • grid-info-search,add,delete,modify
  • Configurable LDAP search timeout (-T)
Write a Comment
User Comments (0)
About PowerShow.com