Grid Programming in - PowerPoint PPT Presentation

1 / 69
About This Presentation
Title:

Grid Programming in

Description:

Drag and drop, copy-and-paste interface. Named sessions. Applet support. 28. Features ... Bang. A test to stderr. 3. 2. 1. Bang. 34. C Job Submission Example ... – PowerPoint PPT presentation

Number of Views:86
Avg rating:3.0/5.0
Slides: 70
Provided by: keithrj
Category:
Tags: and | bang | copy | grid | paste | programming

less

Transcript and Presenter's Notes

Title: Grid Programming in


1
  • Grid Programming in
  • Java Python
  • Prepared by
  • Gregor von Laszewski
  • Argonne National Laboratory
  • gregor_at_mcs.anl.gov
  • Keith R. Jackson
  • Lawrence Berkeley National Laboratory
  • krjackson_at_lbl.gov

2
Chapter Outline
  • Introduction to Grids
  • What is a Grid?
  • What is the Globus Toolkit?
  • What is a Commodity Grid Kit?
  • Using and Programming Grids with the JavaTM and
    Python CoG Kits
  • Secure access to remote resources
  • Remote job submission and monitoring
  • Distributed grid information management and
    remote data access
  • Graphical component to access a Grid
  • Conclusion for this part of the presentation
  • Exercises

3
Integration of High-end Resources
Supercomputers
Software catalogs
Sensor nets
Colleagues
Data archives
On-demand creation of powerful virtual computing
systems
4
Grid Computing and Existing Technologies
  • Commonalities between Grid computing and major
    industrial thrusts
  • Business-to-business, peer-to-peer, application
    service providers, storage service providers,
    distributed computing, Internet computing
  • Differences between Grid computing and existing
    technologies
  • Complicated requirements Run program X at site
    Y subject to community policy P, providing access
    to data at Z according to policy Q
  • High performance unique demands of advanced
    high-performance systems

5
What challenges do we have to solve?
  • Authenticate once
  • Specify simulation (code, resources, etc.)
  • Locate resources
  • Negotiate authorization, acceptable use, etc.
  • Acquire resources
  • Initiate computation
  • Steer computation
  • Access remote datasets
  • Collaborate on results
  • Account for usage

Domain 1
Domain 2
6
An Example ApplicationAn Advanced Scientific
Instrument
Virtual Reality Cave
Advanced Photon Source
Scientist
Avatar
Supercomputer
Electronic Library and Databases
Computing Portal Clients
7
The Globus Toolkit
  • The Globus Toolkit provides a range of basic Grid
    Services
  • Security, information, fault detection,
    communication, resource management
  • These services are simple and orthogonal
  • Independently use mix and match
  • Programming model independence
  • For each service there is generally a
    well-defined API
  • Standards are used extensively
  • E.g. LDAP, GSS-API, X.509

8
Globus Approach
  • A toolkit and collection of services addressing
    key technical problems
  • Modular bag of services model
  • Not a vertically integrated solution
  • General infrastructure tools (aka middleware)
    that can be applied to many application domains
  • Interdomain issues, rather than clustering
  • Integration of intradomain solutions
  • Distinction between local and global services

9
Globus Hourglass
  • Focus on architecture issues
  • Propose set of core services as basic
    infrastructure
  • Use to construct high-level, domain-specific
    solutions
  • Design principles
  • Keep participation cost low
  • Enable local control
  • Support adaptation
  • IP hourglass model

A p p l i c a t i o n s
Diverse global services
Core Globus services
Local OS
10
Layered Grid Architecture
Applica- tion
Application
Internet Protocol Architecture
Collective
Resource
Connectivity
Transport
Internet
Link
Fabric
11
Production Grids Testbeds
GUSTO Testbed
The Alliance National Technology Grid
NASAs Information Power Grid
12
Key Grid Services
  • Resource Discovery
  • List all of the solaris machines with at least 16
    processors and 2 Gigs of memory
  • Resource Acquisition
  • Submit my job to this machine
  • Data Management/Movement
  • Locate, or create, a replica of this data set
  • Move this data set to this host
  • Security
  • Authentication
  • Authorization

13
Grid Security Infrastructure
  • Standards based infrastructure that provides
  • Single sign-on
  • Authentication data integrity/confidentiality
  • Delegation
  • Based on
  • X.509 Proxy certificates (draft ietf pkix
    standard)
  • TLS
  • GSS-API

14
Delegation in GSI
Client
GSI Proxy
3) Submit Job
5) Retrieve Data Set
Compute Resource
Tertiary Storage
6) Run computation
15
GridFTP
  • Implements many of the optional RFCs for the ftp
    protocol.
  • Authentication using the Grid Security
    Infrastructure through the GSS-API
  • Restart markers
  • Third-party transfers
  • Adds some new extensions to support
    high-performance transfers.
  • Parallel streams
  • Striped servers
  • TCP Buffer Tuning

16
Definition CoG Kit
  • A Commodity Grid (CoG) Kit defines and implements
    a set of general components that map Grid
    functionality into a commodity environment.
  • Web/CGI, Java, Python, CORBA, DCOM, ....
  • CoG Kits help us build applications, Problem
    Solving Environments, and Portals.
  • CoG Kits take the good things from framework
    Grid

17
To further elaborate
  • Computational Grids
  • Globus Infrastructure
  • Commodity Technologies
  • CORBA, Java, Python, Perl, Web technologies, ...
  • Science Application
  • Computing Portal
  • Problem Solving Environments
  • Workbenches




18
Motivation Java Python CoG Kits
  • Use and leverage existing technologies for Grid
    programming
  • - The capabilities of the framework onto which
    Grid Services are mapped can be exploited
  • Objects, Events, Exceptions, JNDITM, ...
  • - Objects like jobs/tasks can be defined.
  • - XML support is provided.
  • - GUI's, ...., IDE's can be used (Forte, BOA
    Constructor)
  • Maximize software flexibility, extensibility, and
    reusability
  • Provide foundations for application developer
    teams that are familiar to develop applications
    in this framework
  • - Reduce development and maintenance cost
  • Use as glue for many technologies
  • Python is well suited to tying together many
    different languages/technologies.

19
What is the Java CoG Kit ?
  • The Java CoG Kit provides a mapping between Java
    and the Globus Toolkit. It extends the use of
    Globus by enabling to access advanced Java
    features such as events and objects for Grid
    programming.
  • The Java CoG Kit is implemented in pure Java. It
    speaks the Grid protocols.
  • It is not a wrapper of the C Globus Toolkit
  • This allows integration within applets.
  • Mostly client side support

20
What is the Python CoG Kit?
  • Similarly the Python CoG Kit provides a mapping
    between Python and the Globus Toolkit. It extends
    the use of Globus by enabling to access advanced
    Python features such as events and objects for
    Grid programming.
  • The Python CoG Kit is implemented as a series of
    Python extension modules that wrap the Globus C
    code.
  • Uses SWIG (http//www.swig.org) to help generate
    the interfaces.

21
Status Java CoG Kit
  • Modified core Globus components (Protocols)
  • Basic services are provided accessing
  • Security
    (GSI)
  • Remote job submission and monitoring (GRAM)
  • Quality of service
    (GARA)
  • Remote Data Access (GSIFTP)
  • Information Service Access (MDS)
  • Certificate store
    (myProxy)
  • Current 100 client side components includes
  • Reusable Grid GUI components

22
Status Python CoG Kit
  • Basic services are provided accessing
  • Security
    (security)
  • Remote job submission and monitoring
    (gramClient)
  • Gass Server (gassServerEZ)
  • Secure high-performance network IO (io)
  • Protocol independent data transfers (gassCopy)
  • High performance Grid FTP transfers (ftpClient)
  • Support for building Grid FTP servers (ftpControl)
  • Remote file IO (gassFile)
  • Replica Catalog (replicaCatalog)
  • Replica Management (replicaManagement)
  • GSI enabled SOAP (GSISOAP)

23
Python CoG Kit basics
  • Everything is contained in the pyGlobus package.
  • The low-level C-wrappers are all named modulec,
    e.g., ftpClientc, gramClientc
  • Shouldnt need to access these directly, instead
    use the Python wrappers (ftpClient, gramClient)
  • Exceptions are used to handle error conditions
  • pyGlobus.util.GlobusException is the base class
    for all pyGlobus exceptions. It extends the base
    Python exceptions.Exception class
  • The Python wrappers classes manage the underlying
    memory

24
Building the Python CoG Kit
  • Uses standard GNU tools (autoconf, make)
  • Download the latest source from
    http//www-itg.lbl.gov/Grid/projects/pyGlobus/
  • Tar zxvf pyGlobus-VERSION.tar.gz
  • cd pyGlobus mkdir build cd build
  • ../configure
  • Use with-flavorltGlobus flavor stringgt to
    override the default flavor
  • Use with-pythonltpath to pythongt to override the
    default python installation
  • make make install
  • Installs into your site-packages directory (No
    need to edit PYTHONPATH)

25
Where do we go from here?
  • On which computers can I perform my task?
  • Use a Grid information service (MDS) to answer
    this.
  • Use the LDAP browser as a good example of how to
    develop a Grid-based Portal to a directory based
    information service.
  • How can I execute a job on a remote machine?
  • Use the job submission API or GUI
  • How can I access data on a remote machine?
  • Use Grid FTP.
  • Use the protocol independent gassCopy module.
  • Remote file IO using the gassFile module.
  • Secure high-performance network IO.

26
LDAP Browser and Editor
  • User-friendly Windows Explorer-like interface to
    LDAP directories
  • It allows browsing and modification of
    information
  • It is written entirely in Java
  • Uses JFC (SwingSet) for the GUI
  • Uses JNDI class library for LDAP access
  • Connects to LDAP v2 and v3 servers.
  • Note v3 standard is defined as we speak
  • Recognition
  • More than 1600 commercial user licenses
  • Many users in academia (license for free)
  • Winner of the Best Student Application award in
    the Novell Developers' Contest
  • Jars top 25

27
Features
  • Browsing, editing, searching
  • Export
  • Binary attributes, customizable attribute editors
  • Object templates
  • LDAP v3 aware, SSL support
  • Drag and drop, copy-and-paste interface
  • Named sessions
  • Applet support

28
Features
  • Browsing, editing, searching
  • Export
  • Binary attributes, customizable attribute editors
  • Object templates
  • LDAP v3 aware, SSL support
  • Drag and drop, copy-and-paste interface
  • Named sessions
  • Applet support

29
Features
  • Browsing, editing, searching
  • Export
  • Binary attributes, customizable attribute editors
  • Object templates
  • LDAP v3 aware, SSL support
  • Drag and drop, copy-and-paste interface
  • Named sessions
  • Applet support

30
Screenshot
31
Tiny Information Query Program
  • MDS mds new MDS ("mds.globus.org", 389,
    "oGrid")
  • MDSresult result mds.search
  • ("(objectclassGridComputeResource)
  • (freenodes64))",
  • "contact freenodes totalnodes")
  • String contact result.get("contact")
  • System.out.println(result.print())
  • This can also be done with JNDI or Netscape SDK
  • We have this layer for portability

32
Remote Job Submission
  • Access to remote resources is an elementary Grid
    problem
  • Supports delegation to the remote resource

33
Gram Submission Demo
Hello World 1 2 3 Bang
A test to stderr 3 2 1 Bang
34
C Job Submission Example
callback_func(void user_arg, char job_contact,
int state, int errorcode)
globus_i_globusrun_gram_monitor_t monitor
monitor (globus_i_globusrun_gram_monitor_t )
user_arg globus_mutex_lock(monitor-gtmutex)
monitor-gtjob_state state
switch(state) case GLOBUS_GRAM_PROTOCOL_
JOB_STATE_PENDING globus_i_globusrun_gram_m
onitor_t monitor monitor
(globus_i_globusrun_gram_monitor_t ) user_arg
globus_mutex_lock(monitor-gtmutex)
monitor-gtjob_state state switch(state)
case GLOBUS_GRAM_PROTOCOL_JOB_STATE_FAILED
if(monitor-gtverbose)
globus_libc_printf("GLOBUS_GRAM_PROTOCOL_JOB_STATE
_FAILED\n") monitor-gtdone
GLOBUS_TRUE break case
GLOBUS_GRAM_PROTOCOL_JOB_STATE_DONE
if(monitor-gtverbose)
globus_libc_printf("GLOBUS_GRAM_PROTOCOL_JOB_STATE
_DONE\n") monitor-gtdone
GLOBUS_TRUE break
globus_cond_signal(monitor-gtcond)
globus_mutex_unlock(monitor-gtmutex)
35
C Job Submission Example (cont.)
globus_l_globusrun_gramrun(char request_string,
unsigned long options, char rm_contact)
char callback_contact GLOBUS_NULL char
job_contact GLOBUS_NULL
globus_i_globusrun_gram_monitor_t monitor
int err monitor.done GLOBUS_FALSE
monitor.verboseverbose globus_mutex_init(mo
nitor.mutex, GLOBUS_NULL) globus_cond_init(m
onitor.cond, GLOBUS_NULL) err
globus_module_activate(GLOBUS_GRAM_CLIENT_MODULE)
if(err ! GLOBUS_SUCCESS) err
globus_gram_client_callback_allow(
globus_l_globusrun_gram_callback_func,
(void ) monitor,
callback_contact) if(err !
GLOBUS_SUCCESS) err
globus_gram_client_job_request(rm_contact,
request_string, GLOBUS_GRAM_PROTOCOL_JOB_STAT
E_ALL, callback_contact,
job_contact) if(err ! GLOBUS_SUCCESS)
globus_mutex_lock(monitor.mutex)
while(!monitor.done) globus_cond_wait(m
onitor.cond, monitor.mutex)
globus_mutex_unlock(monitor.mutex)
globus_gram_client_callback_disallow(callback_cont
act) globus_free(callback_contact)
globus_mutex_destroy(monitor.mutex)
globus_cond_destroy(monitor.cond)
36
Gram
  • Creating a job
  • GramJob job new GramJob ("(executable/bi
    n/sleep)(arguments15)")
  • Listening to state changes via Listeners
  • job.addListener( new GramJobListener()
    public void statusChanged(GramJob job)
    System.out.println(Job
    job.getIDAsString() " " Status
    " job.getStatusAsString()) )

37
gramClient
  • from pyGlobus import gramClient
  • from threading import
  • cond 0
  • condV Condition(Lock())
  • try
  • gramClient GramClient()
  • callbackContact gramClient.set_callback(fun
    c, condV)
  • jobContact gramClient.submit_request(rm,rsl,
  • gramClient.JOB_STATE_ALL,
    callbackContact)
  • condV.acquire()
  • while cond 0
  • condV.wait()
  • condV.release()
  • except GramClientException, ex
  • print ex

38
gramClient Job State Callback
  • def func(cv, contact, state, error)
  • global cond
  • if state gramClient.JOB_STATE_FAILED
  • print "Job failed"
  • cv.acquire()
  • cond 1
  • cv.notify()
  • cv.release()
  • elif state
  • gramClient.JOB_STATE_DONE
  • print "Job is done"
  • cv.acquire()
  • cond 1
  • cv.notify()
  • cv.release()

39
Redirecting stdout with gramClient
  • from pyGlobus import gassServerEZ
  • opts gassServerEZ.STDOUT_ENABLE
  • server gassServerEZ.GassServerEZ(opts)
  • url server.getURL()
  • rsl "(executable/bin/sleep)(arguments15)
  • (stdouts/dev/stdout)
    url
  • (normal job submission from here)

40
Remote Data Transfer
  • Directly use the Grid FTP protocol to transfer
    the data.
  • Use the gassCopy module for protocol independent
    transfers. It currently supports the ftp, gsiftp,
    http, and https protocols.

41
Java UrlCopy (protocol independent file transfer)
import org.globus.io.urlcopy.
UrlCopy c new UrlCopy()
c.setSourceUrl(from) c.setDestinationUrl(to
) c.setUseThirdPartyCopy(true) // hint to
enable thridparty transfer // register a
transfer listener.... c.setListener(new
UrlCopyListener() public void transfer(int
total, int current)
System.out.println(total " " current)
public void transferError(Exception e)
System.out.println("transfer failed "
e.getMessage()) ) c.copy() //
this starts the copy
42
GSI FTPClient (not GridFTP)
import org.globus.io.ftp. GSIFTPClient ftp
new GSIFTPClient(host, port) ftp.authenticate()
ftp.setType(FTPClient.BINARY) // we have
convenient calls like ftp.makeDir(dir)
ftp.deleteDir(dir3) // the listener for the
transfers.... listener new TransferProgressList
ener public void transfer(int total, int
current, String from, String to)
System.out.println(total " " current " "
from " " to) public void
transferError(String from, String to, Exception
e) System.out.println("transfer failed "
from " " to)
43
Simple Examples
  • Retrieve a Makefile from a remote site and save
    it to Makefile.bak
  • File dst new File (Makefile.bak)
  • ftp.get (Makefile, dst, listener)
  • Retrieve all files in remote directory to a
    directory called Destination
  • File destinationDir newFile(Destination)
  • ftp.Get(, destinationDir, true, listener)
  • Disconnect
  • ftp.disconnect()
  • Examples in source code apis/examples, UrlCopy,
    FTPClient

44
ftpClient
  • Use Grid FTP to transfer a file.
  • from pyGlobus import ftpClient
  • from pyGlobus.util import Buffer
  • handleAttr ftpClient.HandleAttr()
  • opAttr ftpClient.OperationAttr()
  • marker ftpClient.RestartMarker()
  • ftpClnt ftpClient.FtpClient(handleAttr)
  • ftpClnt.get(url, opAttr, marker, done_func,
    condV)
  • buf Buffer(641024)
  • handle ftpClnt.register_read(buf, data_func, 0)
  • def data_func(cv, handle, buffer, bufHandle,
    bufLen, offset, eof, error)
  • g_dest.write(buffer)
  • if not eof
  • try
  • handle g_ftpClient.register_read(g_b
    uffer, data_func, 0)
  • except Exception, e

45
Performance Options in ftpClient
  • Setting tcpbuffer size
  • from pyGlobus import ftpControl
  • battr ftpControl.TcpBuffer()
  • battr.set_fixed(641024)
  • Or
  • battr.set_automatic(161024, 81024, 641024)
  • opAttr.set_tcp_buffer(battr)
  • Setting parallelism
  • para ftpControl.Parallelism()
  • para.set_mode(ftpControl.PARALLELISM_FIXED)
  • para.set_size(3)
  • opAttr.set_parallelism(para)

46
Using ftpClient plugins
  • Debug Plugin
  • from pyGlobus import ftpClient
  • f open(/tmp/out, w)
  • debugPlugin ftpClient.DebugPlugin(f, foobar)
  • handleAttr.add_plugin(debugPlugin)
  • RestartMarker Plugin
  • restartPlugin ftpClient.RestartMarkerPlugin(begi
    nCB,
  • markerCB,
    doneCB, None)

47
gassCopy
  • Provides a protocol independent API to transfer
    remote files.
  • from pyGlobus import gassCopy
  • from pyGlobus import ftpClient
  • srcAttr gassCopy.Attr()
  • handleAttr gassCopy.HandleAttr()
  • destAttr gassCopy.Attr()
  • ftpSrcAttr ftpClient.OperationAttr()
  • ftpDestAttr ftpClient.OperationAttr()
  • srcAttr.set_ftp(ftpSrcAttr)
  • destAttr.set_ftp(ftpDestAttr)
  • copy gassCopy.GassCopy(handleAttr)
  • copy.copy_url_to_url(srcUrl, srcAttr, destUrl,
    destAttr)

48
Remote File IO
  • Supports reading and writing remote files from
    http, https, ftp, gsiftp servers.
  • Can be opened either as normal Python file
    objects or as int file descriptors suitable for
    use with the os module.
  • from pyGlobus import gassFile
  • gassFile gassFile.GassFile()
  • f gassFile.fopen(gsiftp//foo.bar.com/file,
    r)
  • lines f.readlines()
  • gassFile.fclose(f)

49
Secure High-Performance IO.
  • Uses the Grid Security Infrastructure to provide
    authentication.
  • Provides access to the underlying network
    parameters for tuning performance.

50
TCP Server example
  • attr NetIOAttr.TCPIOAttr()
  • attr.set_authentication_mode(
    io.GLOBUS_IO_SECURE_AUTHENTICATION_MODE_GSS_API)
  • authData AuthData.AuthData()
  • authData.set_callback(auth_callback, None)
  • attr.set_authorization_mode(
    io.GLOBUS_IO_SECURE_AUTHORIZATION_MODE_CALLBACK,
    authData)
  • attr.set_channel_mode( io.GLOBUS_IO_S
    ECURE_CHANNEL_MODE_GSI_WRAP)
  • soc GSITCPSocket.GSITCPSocket()
  • port soc.create_listener(attr)
  • soc.listen()
  • childSoc soc.accept(attr)
  • buf Buffer.Buffer(size)
  • bytesRead childSoc.read(buf, size, size)

We will develop a similar example for Java, It is
already available as part of the GASS server.
51
TCP Client example
  • attr NetIOAttr.TCPIOAttr()
  • attr.set_authentication_mode(
    io.GLOBUS_IO_SECURE_AUTHENTICATION_MODE_GSS_AP)
  • authData AuthData.AuthData()
  • attr.set_authorization_mode( io.GLOBUS_I
    O_SECURE_AUTHORIZATION_MODE_SELF, authData)
  • attr.set_channel_mode(
    io.GLOBUS_IO_SECURE_CHANNEL_MODE_GSI_WRAP)
  • soc GSITCPSocket.GSITCPSocket()
  • soc.connect(host, port, attr)
  • nBytes soc.write(str, len(str))

52
Web Services in Grids
  • Provides a uniform, lightweight, standards based,
    framework for
  • Service description (WSDL)
  • Discovery (UDDI / WSIL)
  • Invocation (SOAP)
  • Allows the use of standard tools
  • Separates the interface from implementation
    details
  • Including separating service definition from
    protocol bindings
  • Language/Programming model agnostic
  • Reduces the cost of building scientific
    applications
  • Uses SOAP over HTTP over the Grid Security
    Infrastructure
  • Similar to HTTPS, but supports delegation

53
GSISOAP
  • Built on SOAP.py, looking at switching to ZSI
  • GSISOAP client example
  • from pyGlobus import GSISOAP, ioc
  • proxy GSISOAP.SOAPProxy(https//host.lbl.gov80
    81, namespaceurngtg-Echo)
  • proxy.channel_mode ioc.GLOBUS_IO_SECURE_CHANNEL_
    MODE
  • proxy.delegation_mode
  • ioc.GLOBUS_IO_SECURE_DELEGATION_MODE_NONE
  • print proxy.echo(spam, spam, spam, eggs, and
    spam)

54
GSISOAP (cont.)
  • GSISOAP server example
  • from pyGlobus import GSISOAP, ioc
  • def echo(s, _SOAPContext)
  • cred _SOAPContext.delegated_cred
  • Do something useful with cred here
  • return s
  • server GSISOAP.SOAPServer(host.lbl.gov, 8081)
  • server.channel_mode ioc.GLOBUS_IO_SECURE_CHANNEL
    _MODE_GSI_WRAP
  • server.delegation_mode ioc.GLOBUS_IO_SECURE_DELE
    GATION_MODE_FULL_PROXY
  • server.registerFunction(SOAP.MethodSig(echo,
    keywords0, context1), urngtg-Echo)
  • server.serve_forever()

55
Reliable File Transfer Service
  • A Grid Web Service that
  • Provides a simple interface to reliably transfer
    data between GridFTP servers
  • Supports reliable transfers in the face of
    network outages, machine crashes, and other
    faults
  • Supports the automated tuning of network
    parameters to optimize bandwidth

56
RFT Usage
Client
WSDL Repository (UDDI?, WSIL?)
GSI Proxy
Reliable File Transfer Service
GridFTP Server
GridFTP Server
57
Fault Recovery
Reliable File Transfer Service
GridFTP Server
GridFTP Server
58
Fault Recovery
Reliable File Transfer Service
GridFTP Server
GridFTP Server
59
Fault Recovery (cont.)
Machine crashes
Reliable File Transfer Service
DB
GridFTP Server
GridFTP Server
60
Fault Recovery (cont.)
Reliable File Transfer Service
DB
After machine reboot, recover incomplete
transfers from the DB
GridFTP Server
GridFTP Server
61
RFT WSDL
ltdefinitions name"FileTransferService namespace
stuff gt ltmessage name "submitTransferRequest
"gt ltpart name "srcUrl" type
"xsdstring"/gt ltpart name "destUrl"
type "xsdstring"/gt ltpart name
"priority" type "xsdint"/gt lt/messagegt
ltmessage name "submitTransferResponse"gt
ltpart name "return" type "xsdstring"/gt
lt/messagegt ltmessage name "checkStatusRequest"gt
ltpart name "handle" type
"xsdstring"/gt lt/messagegt ltmessage name
"checkStatusResponse"gt ltpart name
"return" type "xsdstring"/gt lt/messagegt
62
RFT Internals
Client
GSISOAP Interface
Transfer Queue
Persistent DB
Manager Thread
Transfer Threads
Optimization Module
GridFTP Server
GridFTP Server
63
RFT Usage Example
from rftClient import handle srcUrl
gsiftp//host.lbl.gov/data/climate_model_1234 de
stUrl gsiftp//lorenz.ncar.edu/tmp/climate_mode
l_1234 try handle submit_transfer(srcUrl,
destUrl) except TransferRequestException, ex
print ex try status check_transfer_status
(handle) Except TransferRequestException, ex
print ex print status
64
Many more CoGs
  • Perl
  • JSP
  • CORBA
  • .
  • GUIs
  • Applications

65
Java CoG Requirements
  • JDK 1.2
  • Security
  • Needs security package such as IAIK
  • We are able to replace security packages based on
    a couple of common functions with public domain
    package.
  • We hope that these package will reach the
    maturity of iaik

66
Python CoG Requirements
  • Python 2.0
  • The GT2-beta (or better) release of Globus.
  • Support for dynamic libraries.
  • Currently pyGlobus is tested on
  • Linux
  • Solaris
  • Should compile
  • Will support win32 when Globus is ported

67
Review
  • Modifications to core Globus
  • Http based protocols
  • APIs are not sufficient we need services
  • Components useful to enhance grid infrastructure
  • JavaBeansTM can be used to integrate with COTS.
  • Java and Python have a rich set of libraries
    (network, ldap, ...).
  • Grid programming in Java and Python is easier!
  • Java uses event model - not callbacks
  • GUIs do not have everything we need scripting!

68
References
  • Java CoG Kit
  • http//www.globus.org/cog.
  • Python CoG Kit
  • http//www-itg.lbl.gov/Grid/projects/pyGlobus
  • Globus
  • http//www.globus.org
  • SWIG
  • http//www.swig.org
  • krjackson_at_lbl.gov
  • gregor_at_mcs.anl.gov

69
Acknowledgements
  • This research is supported by the Mathematical,
    Information, and Computational Science Division
    subprogram of the Office of Advanced Scientific
    Computing Research, U.S. Department of Energy,
    under Contract W-31-109-Eng-38 and under Contract
    DE-AC03-76SF00098 with the University of
    California. Globus research and development is
    supported by DARPA, DOE, and NSF.
Write a Comment
User Comments (0)
About PowerShow.com