Scientific workflow support in the VL-e PoC environment

1
Scientific workflow support in the VL-e PoC
environment

• Zhiming Zhao
• The VL-e workflow team

2
Outline

• Scientific workflow and e-Science
• Workflow systems in the VL-e PoC environment
• Recommendations

3
Scientific Workflow Management Systems in an
e-Science environment
Domain specific Applications

• Functionalities
• Automating experiment routines
• Rapid prototyping of experimental computing
  systems
• Hiding integration details between resources
• Managing experiment lifecycle
• Cross different layers of middleware for
  managing
• Data
• Computing
• Information
• Knowledge

Knowledge
Information
e-Science environment
Computing tasks
Data management
Generic Grid middleware
Grid infrastructure
4
Workflow systems in PoC

• Four systems
• Kepler, Taverna, Triana, and VLAM
• Why these systems?
• The recommendation is based on
• Analyzing requirement on application use cases
• Reviewing existing workflow systems
• We learned
• None of the existing systems support all
  requirements of the VL-e use cases
• These three systems together cover most of the
  requirements,
• They provide friendly interface

5
Kepler

• Background
• The Kepler project is a cross-project
  collaboration among SEEK, SPA, Ptolemy II, GEON,
  etc.
• Current version is 1.0 alpha 8
• Java based, open source
• Usability
• Friendly GUI
• Web service compliant. It also has specialized
  architecture for resources and controller (called
  Actor and Director)
• Grid support
• Inherits a powerful and matured framework from
  PTOLOMYII. Provides a rich library of actors. It
  has nice GUI, and provides Nimrod support.
• Limited support for job farming
• Alpha version, not stable, not enough
  documentation for developing new modules.

6
Taverna

• Background
• From myGrid, Sculf workbench V1.2
• Has a big Bio- user community
• Resources are WS based
• Usability
• It has a GUI (not very friendly)
• Enactment and runtime control (service
  orchestration)
• The support for developing WS resources is not
  included
• Web service based meta programming environment,
  has a big Bio-informatics user community.
• Limited GUI Support, not enough documentation for
  developing new modules.
• data flow based, support limited control
• limited scheduling strategies for parameter
  sweeping and job farming.

7
Triana

• Background
• Open source, in Grid lab project, at Cardiff
  university
• Version 3.2
• Usability
• Interfaces to invoke web services and use Grid
  resources (GAP, GAT) deployment of a workflow as
  a web service rich library of processing
  modules nice GUI
• Limited documentation for developers instability
  of some features.

8
VLAM

• Background
• The installed version in Marix (part of the PoC)
• Usability
• GUI
• PFT and Topology
• Grid and WS support
• Limit flexibility of GUI
• The PFT part is not mature
• Limited support for job farming

9
Recommendations

• Web/Grid service compliant interface for resource
  invocation and integration
• Follow software engineering disciplines

10
Direction

• Short term solution
• Use these workflow systems in the use cases
• Learn lessons
• Abstract useful features
• Long term solution
• A new design will be made