Independent Software Vendor (ISV) Remote Computing Primer - PowerPoint PPT Presentation

Loading...

PPT – Independent Software Vendor (ISV) Remote Computing Primer PowerPoint presentation | free to download - id: 7adc3a-OWU1N



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Independent Software Vendor (ISV) Remote Computing Primer

Description:

Independent Software Vendor (ISV) Remote Computing Primer Steven Newhouse The Problem . Growth of compute oriented applications Research applications with source ... – PowerPoint PPT presentation

Number of Views:41
Avg rating:3.0/5.0
Slides: 19
Provided by: JoelR169
Learn more at: http://ogfweb.ogf.org
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Independent Software Vendor (ISV) Remote Computing Primer


1
Independent Software Vendor (ISV) Remote
Computing Primer
  • Steven Newhouse

2
The Problem.
  • Growth of compute oriented applications
  • Research applications with source available
  • Commercial Applications from ISVs have no source
  • ISV Independent Software Vendors
  • Applications becoming bound by the desktop
  • Multi-core pushing out this boundary
  • But ensemble/parameter sweeps increases demand
  • Exploit resources provided within the enterprise
  • Growing adoption of HPC into ISV applications
  • How to deal with different schedulers, clusters
    OS?

3
The Environment
4
Goals
  • How can standards enable access to these
    distributed resources?
  • What scenarios need to be supported?
  • Deal with realistic network environments
  • Firewalls and NATed networks
  • Move from desktop to mobile clients
  • Clients are not always connected to the network
  • Clients will not have files on shared network
  • Embrace different scheduler interfaces

5
Specification Toolbox
  • WS-Addressing
  • Security specifications and profiles
  • Job Submission Description Language (JSDL)
  • JSDL Single Process Multiple Data Application
    Extension
  • JSDL Parameter Sweep Extension
  • Basic Execution Service (BES)
  • HPCP-Application Extension
  • HPC Basic Profile (HPCBP)
  • File Staging Extension to the HPC Basic Profile
  • ByteIO
  • GLUE (Resource Description)
  • Distributed Resource Management Application API
    (DRMAA)
  • Resource Namespace Service (RNS)

6
Core Specifications
  • WS-Addressing
  • Encapsulates service in Endpoint Reference (EPR)
    in XML
  • From a client perspective
  • Security specifications and profiles
  • Builds on core standard WS-Security
    infrastructure
  • WS-Secure Addressing (GFD 131)
  • Profile on WS-Addressing and WS-Security Policy
  • Embeds into EPR how to access a service
  • WS-Secure Communication (GFD 132)
  • Profile on mechanisms to enable easier
    interoperability

7
Job Submission Description Language
  • Core JSDL specification (GFD 56 ? 136) has many
    implementations
  • Describes what you want to run (the job)
  • Describes what you need to run it on (the
    resource)
  • Extensions
  • JSDL Single Process Multiple Data (SPMD)
    Application Extension (GFD 115)
  • JSDL Parameter Sweep Extension
  • Now entering public comment
  • HPCP-Application Extension (GFD 111)
  • Supported in HPC Basic Profile

8
Running jobs
  • Basic Execution Service (BES) (GFD 108)
  • Provides an interface to submit, monitor manage
    an activity
  • HPC Basic Profile (HPCBP) (GFD 114)
  • Specialises BES ( other specifications) for HPC
  • File Staging Extension to the HPC Basic Profile
    (GFD 135)
  • Profiles JSDL to allow transfers in out of a
    cluster

9
Other OGF Specifications
  • ByteIO (GFD 87)
  • Access through a web service to a file
    abstraction
  • Support for random access streaming patterns
  • GLUE (Finishing public comment)
  • Information model for grids
  • Focus on services resources for compute
    storeage
  • Expose virtual organization usage controls
  • Distributed Resource Management Application API
    (DRMAA) (GFD 22)
  • Client API to enable access to DRMs
  • Resource Namespace Service (RNS) (GFD 101)
  • Naming of resources in a hierarchical namespace

10
  • WOW

11
Application Scenarios
  1. Run a job
  2. Run a job through a standard API
  3. Run a job through a web service
  4. Run a job through a web service with file staging
  5. Run a job through a web service and have
    bi-directional interaction with it

12
Run a job
  • The current state of the art
  • Application calls an external script
  • Script invokes a local job submission command
  • Requires
  • Installation of local scheduler library
  • Customization of scripts to access local scheduler

13
Run a job using a standard API
  • Application internally calls a standard API
  • DRMAA specification with multiple implementation
  • Within DRMAA invoke scheduler specific plugin
  • Requires
  • Installation of local scheduler library
  • Installation of DRMAA plugin to call the library

Application
DRMAA Interface
DRMAA plug-in
Scheduler
14
Run a job through a web service
  • HPCBP client embedded in the application
  • Formulates the job using JSDL HPCP Application
  • HPCBP service used to access the cluster
  • Available for many schedulers
  • Requires
  • Installation of HPCBP service on the cluster
  • Integration of the HPCBP client in the
    application
  • Note Cross-platform interoperability
    demonstrated at SC06 SC07

15
Extension File Staging
  • Use the HPCBP File Staging specification
  • Can include ftps, ftp, scp sftp, and other
    protocols
  • By using an intermediary file store the client
    can disconnect reconnect later
  • Client copies the files to the intermediary store
  • Client then submits the job to the cluster
  • The cluster reads/writes files to the
    intermediary store
  • Once the job is complete the files are retrieved
  • Requires
  • Support of standard file protocols on client,
    cluster store

16
Extend Run-time interaction
  • Use ByteIO in streaming mode to
  • Send control requests from the client to the
    application
  • Return data from the application to the client
  • ByteIO service needs to be accessible from the
    client and the clusters compute nodes
  • E.g. the clusters head nodes
  • Requires
  • ByteIO service and integration into application
    client

17
Summary
  • Core set of established interoperable specs.
  • Demonstrated interoperability
  • Commercial and open-source implementations
  • Toolbox of specifications enables applications
  • Allows ISVs to build applications that
  • Can connect to any standard infrastructure
  • Run from mobile clients in network environments

18
Acknowledgements
  • Use Case Workshop participants
  • University of Virginia, March 2008
  • Core scenario Narfi Stefansson, Mathworks
  • Vigorous discussion over the two days
  • Further contribution feedback at OGF 23
  • And on mailing list and OGSA calls
  • Resulting document is now being published
  • Editors Steven Newhouse Andrew Grimshaw
About PowerShow.com