Workflow%20 %20Globus%20Toolkit%20Update%20September%208,%202004

1
Workflow Globus Toolkit UpdateSeptember 8, 2004

• Allan Doyle, NASA
• adoyle_at_intl-interfaces.com

2
Agenda

• Workflow activity
• Globus Toolkit status
• CA Activity (Yonsook)

3
Agenda

• Workflow activity
• Globus Toolkit status
• CA Activity (Yonsook)

4
Workflow Activity

• Began in late 2003
• Goal was to be able to chain processing among
  participants in a single workflow
• Held a series of teleconferences
• http//grid-tech.ceos.org/gridwiki/WorkflowTigerTe
  am
• (Ask for username/password)

5
Workflow activity - 1

• First, we learned about workflow and about CEOS
  Grid Participants workflow tools

6
Workflow overview

• The next slides are taken from Tech Team members
  presentations.
• Annotations that were added for this presentation
  look like this

Annotation box
7
What is a Workflow?
Dutch Space has renamed their workflow tools to
GridAssist
8
Workflow Levels
This material was redrawn to enhance visibility,
its from Ewa Deelmans workflow presentation
Application Domain
FFT
Application Component Selection
Specify a Different Workflow
Abstract Workflow
Resource Selection Data Replica
Selection Transformation Instance Selection
copy host1//home/filea to host2//u1/file1
Concrete Workflow
Pick Different Resources
Data Transfer
/usr/local/bin/fft /u1/file1
Retry
Execution Environment
From Pegasus presentation by Ewa Deelman, ISI
9
Typical Workflow Environment
10
Workflow example
This is a domain (or application level) workflow
11
Workflow detailed example
This is an abstract workflow
12
GMU Virtual Product Concept
Virtual data concept requires workflow tools in
order to be built
13
Workflow progression
Domain
Abstract
Concrete
Different levels of workflow
14
Workflow tools

• DutchSpace - Grid Assist
• Developed in-house, workflow engine and client
  tools. (Runs on Globus Toolkit 2.x)
• NCSA (UAH partner) - Open GCE Runtime Engine
• Based on Apache Ant (not Globus dependent)
• ISI - Pegasus
• Developed as part of Earth Systems Grid work
  w/Argonne (Runs on Globus 2.x, 3.x?)
• GMU - GeoTree
• Based on BPEL (Business Process Execution
  Language) (Runs on Globus 3.2.x)

15
Workflow Activity - 2

• Then we investigated workflow interoperability.
• The plan was to
• Ask participants for use cases
• Decompose use cases into workflow patterns

16
Use case descriptions

• Identifier
• Description
• Actors
• Goal
• Preconditions
• Assumptions
• Frequency

• Basic Course
• Alternate Courses
• Included Use Cases
• Extended Use Cases
• Issues
• Decisions

Workflows can be derived from these descriptions
of actions
17
Workflow Patterns

• http//tmitwww.tm.tue.nl/research/patterns/
• Provides a good explanation and definition of 21
  workflow patterns.
• Next slide shows some of the simple ones as
  examples.

18
Basic workflow patterns
19
Use case analysis

• Use cases were received from
• USGS
• UAH/NOMADS
• Dutch Space
• USGS and UAH/NOMADS use cases were refined via
  teleconference email
• All are available on Workflow web page

20
Initial Findings

• Workflow tools are a necessary component!
• Grid applications are partioned into work units
  that run on many machines.
• Runtime error handling is a vital part of
  workflow engine capability
• Grid workflows tools are complex
• Tools take a long time to learn
• Grid workflows support complex processing tasks
• Many (30) different Grid workflow systems exist
• Interoperability among workflow tools is
  currently nonexistent

21
Use case findings - 1Based on the USGS and
NOMADS/UAH Use Cases

• Found that use cases tended to have many
  hands-on steps where scientists would make
  choices
• This is one of the key areas the Grid workflow
  community has identified as needing work.
• Use of formal workflow tools may be more work
  than its worth for the use cases
• Application projects are using the Grid mainly
  for data transfer, not computing
• Workflows include some automation areas
  (reprojection, reformatting, subsetting).

22
Use case findings - 2Based on the Dutch Space
Application description

• Dutch Space application is well suited for formal
  tool
• Dutch Space has implemented their own tool in
  response to their need and the (then) lack of
  existing tools.
• OMI application has long processing chain (c.f.
  earlier slides)

23
Workflow Conclusions

• Workflow community is starting to talk about
  interoperable workflows
• http//www.extreme.indiana.edu/groc/ggf10-ww/inde
  x.html
• Perhaps cross-workflow interoperability is not an
  immediate need
• A single workflow tool can be used to control
  remote job execution without needing to
  interoperate with other workflow tools.
• CEOS Grid activity should wait until Application
  Projects have a clear need for cross-tool
  workflow interoperability before doing further
  work.

24
Agenda

• Workflow activity
• Globus Toolkit status
• CA Activity (Yonsook)

25
Globus Toolkit (GT) Update

• Current GT version is 3.2
• Most CEOS Globus users are still using 2.4
• GT version 4.0 release is scheduled for January
  31, 2005
• http//www-unix.globus.org/toolkit/GT4Facts/
• GT4 will be backwards compatible with GT3.2
• Next 5 slides taken from the presentation given
  in Tromso, also available at
• http//grid-tech.ceos.org/gridwiki/GlobusVersions
  

26
(No Transcript)
27
(No Transcript)
28
(No Transcript)
29
(No Transcript)
30
(No Transcript)
31
GT 4.0 schedule and content

• Next two slides have material from
• http//www-unix.globus.org/toolkit/GT4Facts/

32
GT4 Release Schedule
33
GT4 Components
34
Globus Toolkit Recommendations

• If you are using
• GT 2.4
• Migrate to 4.0
• Dont go to 3.2 as intermediate step
• GT 3.2
• Migrate to 4.0
• 4.0 will be backwards compatible

35
Agenda

• Workflow activity
• Globus Toolkit status
• CA Activity (Yonsook)

36
CA Activity

• Following earlier work on Certificate Authorities
  (CA), NASA IPG (Information Power Grid)
  volunteered to provide CA services for CEOS Grid.
• NASA IPG funding has been cut
• CEOS Grid cannot rely on IPG CA anymore
• We decided to establish a CEOS CA on a NASA
  machine at GSFC

37
CA Activity - cont.

• Ananth Rao has installed a test CA and has
  produced a Certificat Request tool similar to the
  one provided by IPG
• This tool will undergo further testing after
  WGISS-18
• Once we are satisfied with it, we will provide it
  to the CEOS Grid