GridSpheres Grid Portlets A Grid Portal Development Framework - PowerPoint PPT Presentation

Loading...

PPT – GridSpheres Grid Portlets A Grid Portal Development Framework PowerPoint presentation | free to download - id: 25291-N2VmZ



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

GridSpheres Grid Portlets A Grid Portal Development Framework

Description:

Grid Portlets is still in prototype development mode. ... Mike in Brazil ;-) but plans on continuing to enhance grid portlets. 2004. 35 ... – PowerPoint PPT presentation

Number of Views:219
Avg rating:3.0/5.0
Slides: 44
Provided by: hartmut3
Learn more at: http://www.nesc.ac.uk
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: GridSpheres Grid Portlets A Grid Portal Development Framework


1
GridSpheres Grid PortletsA Grid Portal
Development Framework
  • Jason Novotny
  • Lawrence Berkeley National Laboratory
  • Jason Novotny, Michael Russell, Oliver Wehrens
  • The GridSphere Project
  • www.gridsphere.org

2
The Purpose Of This Talk
  • The purpose of this talk is to provide a brief
    overview of the Grid Portlets web application.
  • We will describe the basic ideas behind Grid
    Portlets, what functionality it currently
    provides and development plans

3
GridSpheres Grid Portlets
  • The GridSphere portlet container is designed to
    be web application independent. Indeed, one of
    the key advantages of the Portlet API is the
    reuse of web applications.
  • Thus, the GridSphere portlet container does not
    contain any support for using Grid technologies.
  • Instead, GridSpheres Grid related functionality
    is contained in a web application we call Grid
    Portlets.
  • Grid Portlets, together with the GridSphere
    portlet container, offers a generic Grid portal
    and can be used to develop application-specific
    Grid portal applications.

4
The GridLab Project
  • Funded by the EU (5 M), January 2002 December
    2004
  • Application and Testbed oriented
  • Cactus Code, Triana Workflow, all the other
    applications that want to be Grid-enabled
  • Main goal to develop a Grid Application Toolkit
    (GAT) and set of grid services and tools...
  • resource management (GRMS),
  • data management,
  • monitoring,
  • adaptive components,
  • mobile user support,
  • security services,
  • portals,
  • ... and test them on a real testbed with real
    applications

5
GridLab Is An Architecture
6
And A Global Effort
  • PSNC (Poznan) - coordination
  • AEI (Potsdam)
  • ZIB (Berlin)
  • Univ. of Lecce
  • Cardiff University
  • Vrije Univ. (Amsterdam)
  • SZTAKI (Budapest)
  • Masaryk Univ. (Brno)
  • NTUA (Athens)
  • Sun Microsystems
  • Compaq (HP)
  • ANL (Chicago, I. Foster)
  • ISI (LA, C.Kesselman)
  • UoWisconsin (M. Livny)
  • collaborating with
  • Users!
  • EU Astrophysics Network,
  • DFN TiKSL/GriKSL
  • NSF ASC Project
  • other Grid projects
  • Globus, Condor,
  • GrADS,
  • PROGRESS,
  • GriPhyn/iVDGL,
  • Most of the other European Grid Projects
    (GRIDSTART)
  • GWEN

7
GridLab Testbed Snapshot
8
GridLab Goals
  • Get Computational Scientists using the Grid and
    Grid services for real, everyday, production work
    (AEI Relativists, EU Network, Grav Wave Data
    Analysis, Cactus User Community), all the other
    potential grid apps
  • Make it easier for applications to make flexible,
    efficient, robust, use of the resources available
    to their virtual organizations
  • Dream up, prototype, and test new application
    scenarios which make adaptive, dynamic, wild, and
    futuristic uses of resources.

9
What Do Our Users Need?
  • Application oriented environment
  • Flexible, easy-to-use, simple interfaces
  • Efficient and effective use of resources
  • Robustness, fail-safety, adapability
  • The ability to work in distributed teams
  • Support for mobile working environments

10
What Do Our Users Want?
  • Larger computational resources
  • Memory/CPU
  • Faster throughput
  • Cleverer scheduling, configurable scheduling,
    co-scheduling, exploitation of un-used cycles
  • Easier use of resources
  • Portals, grid application frameworks, information
    services, mobile devices
  • Remote interaction with simulations and data
  • Notification, steering, visualization, data
    management
  • Collaborative tools
  • Notification, visualization, video conferencing,
    portals
  • Dynamic applications, New scenarios
  • Grid application frameworks connecting to
    services

11
Many Application Scenarios!
  • Dynamic Staging
  • move to faster/cheaper/bigger machine
  • Multiple Universe
  • create clone to investigate steered parameter
  • Automatic Convergence Testing
  • from initial data or initiated during simulation
  • Look Ahead
  • spawn off and run coarser resolution to predict
    likely future
  • Spawn Independent/Asynchronous Tasks
  • send to cheaper machine, main simulation carries
    on
  • Application Profiling
  • best machine/queue
  • choose resolution parameters based on queue
  • Dynamic Load Balancing
  • inhomogeneous loads
  • multiple grids
  • Portal
  • User/virtual organisation interface to the grid.
  • Intelligent Parameter Surveys
  • farm out to different machines
  • Make use of
  • Running with management tools such as Condor,
    Entropia, etc.
  • Scripting thorns (management, launching new jobs,
    etc)
  • Dynamic use of eg MDS for finding available
    resources

12
Where Grid Portals Fit In
  • GridLab needs an effective way to deliver its
    tools and services to its target users.
  • We want to avoid having to develop new
    client-side technologies and methods for
    deploying them.
  • Given the ubiquitous Web browser and the
    evolution of Grid standards and technologies, a
    Grid portal provides an obvious solution to this
    problem.

13
Developing Grid Portals
  • Grid web application development still remains a
    tedious task with little in the way of reusable
    components, forcing developers to constantly
    re-invent the wheel.
  • Often difficult and hard to maintain glue code
    must be written connecting the portal to Grid
    services, due to lack of/evolving standards.
  • Lack of real usability has made it difficult to
    test and evaluate user interfaces.
  • A portal is only as good as the underlying
    deployed infrastructure. portal development
    often involves debugging underlying middleware

14
What is a Grid Portal?
  • A portal is a web based application that
    commonly provides personalization, single sign
    on, content aggregation from different sources
    and hosts the presentation layer of Information
    Systems(JSR 168)
  • Grid Portals build upon the familiar Web portal
    model, such as Yahoo or Amazon, to deliver the
    benefits of Grid computing to virtual communities
    of users, providing a single access point to Grid
    services and resources.

15
Early Grid Portal Projects
  • Grid-Port
  • Perl based framework developed by Mary Thomas and
    Steve Mock at San-Diego Supercomputing Center
    (SDSC)
  • Grid Portal Development Toolkit (GPDK)
  • Developed by Jason Novotny at Lawrence Berkeley
    National Laboratories (LBNL)
  • Astrophysics Simulation Collaboratory (ASC)
  • Developed by Michael Russell at University of
    Chicago

16
Grid Portals Work Package
  • Within the GridLab Project, the Grid Portals Work
    Package (WP4) is responsible for developing a
    Grid portal for the GridLab Project.
  • The GridSphere Project was created by WP4 as it
    saw a need in the general community for a Grid
    portal framework.
  • WP4 is both developing and deploying GridSphere
    to serve target user communities in the GridLab
    Virtual Organization.

17
GridSphere 2.0
18
Personalized Environment
19
Single Sign-On Capabilities
20
Submit Jobs
21
Perform File Transfers
22
Manage Resources
23
Grid Portal Design Requirements
  • We require a portal that is generic and not tied
    to any particular set of Grid technologies. The
    GridLab Testbed, for example, is Globus 2.X based
    although integartion with OGSA well under way
  • Develop a set of portlets and portlet services
    that provide a higher level model of Grid
    concepts.The Grid portlet services should provide
    a generic API to which the Grid portlets are
    written.

24
Grid Portlets API
  • Our Grid Portlets API attempts to provide a
    high-level model of Grid concepts that
  • Abstracts the developer from particular Grid
    technologies and infrastructure.
  • Is extensible, can build upon its concepts to
    provide more complex services, resources and
    tasks.
  • Supports multiple implementations of the API.
  • Configurable at run-time via administrative
    interfaces
  • The Grid Portlets API builds from basic concepts
    to more advanced ones

25
Basic Concepts
  • Users
  • Users should get a personalized a view of the
    Grid based upon their access rights to resources
    and their preferences.
  • Credentials
  • Credentials are required to authenticate users to
    resources and authorize their rights to utilize
    them.
  • Resources
  • Resources are anything that can be utilized (by
    users, portlet services and other resources).
  • Tasks
  • Many operations are quite naturally modeled as
    tasks. Workflow is not provided yet, but is
    desired in future releases.

26
More Advanced Concepts
  • Jobs
  • Job submission to LSF, PBS, etc
  • Job migration from one resource to another
  • Files Related Tasks
  • Physical file browsing (list, transfer, etc.)
  • Logical and replica file browsing
  • Information Resources
  • LDAP/MDS information providers
  • Web service based providers
  • Other types

27
Simple Resource Diagram
28
Hardware Resource Model
  • Hardware resources are utilized for performing
    computations, storing files, etc.
  • Hardware resources contain
  • Service resources (such as a Grid FTP resource)
  • Software resources (such as Globus client tools)
  • Hardware accounts (allowing people to login and
    perform operations in a shell).

29
Job Resource Model
  • Job resources (such as Globus Gatekeepers) are
    service resources that enable the remote clients
    to submit jobs on computing resources.
  • Job resources provide access to job schedulers
    (such as PBS).
  • Job schedulers contain queues with different
    resource capabilities and usage policies.

30
Some Job Resources
  • In Grid Portlets, weve modeled two types of
    service resources as job resources, both of which
    have different capabilities for supporting job
    submission.
  • GTX Gatekeepers provide the ability to submit
    jobs directly to job managers running on hardware
    resources using the GRAM protocol.
  • GridLab Resource Manager Services (GRMS)s are web
    services that provide resource brokering
    capabilities and support job submission with GRAM
    as well as Condor.

31
Simple Task Diagram
32
How Portlets Are Developed
  • In Grid Portlets, the presentation layer is made
    up of portlets which contain reusable UI
    components.
  • The business logic layer is implemented as a
    collection of portlet services which provide some
    high-level capabilities and make use of external
    resources.

33
Job Submission Portlet
  • For example, the job submission portlet uses a
    number of UI components, including file browser
    components (i.e. for specifying locations of an
    executable).
  • These JSP components utilize a number of portlet
    services which provide access to job resources,
    file resources and so on

34
Current Status
  • Grid Portlets is still in prototype development
    mode.
  • Most of what is their works, but API is still
    evolving.
  • Documentation is still poor Mike promises to
    improve it
  • Source code is available via anonymous CVS.
  • Will move towards a regular development and
    release cycle in the coming months.
  • Mike in Brazil -) but plans on continuing to
    enhance grid portlets

35
Current Functionality
  • Credential Retrieval Management
  • Support for retrieval storing with MyProxy
  • Resource Registry Management
  • Resources can be described in XML and online
  • Resource Information Provider
  • Resource discovery and update with MDS2
  • Job Submission Migration
  • GRAM job submission with Java CoG 1.1
  • GRMS 1.9 job submission and migration
  • File Browsing Transferring
  • Support for GridFTP with Java CoG 1.1

36
Whats Next?
  • Enhancements to UI component model These
    enhancements may be incorporated into GridSphere
    portlet container, but we are still reviewing
    JSF.
  • Many enhancements planned to existing portlets,
    including better integration among portlets and
    reuse of UI components.
  • Progress already underway to make JSR 168
    compliant
  • Please submit feature requests and additions to
    bugtracker and mail list

37
Planned Development
  • Support for logical files replica management
  • Support for GridLab Data Management Services
  • Support for Globus Replica Catalogue
  • Enhancements to information gathering
  • Portlets for managing use of information
    providers
  • More resource information providers
  • Support for GridLab Adaptive Information Service
  • Support for Network Weather Service

38
How To Obtain Grid Portlets
  • Grid Portlets can be obtained from CVS.
  • Required Software
  • GridSphere 2.0.2 or higher.
  • Tomcat 4.1.29 or higher (Tomcat 5.X supported)
  • OGSA 3.0.2 (Plan to move OGSA 3.2)
  • MyProxy

cvs -d pserveranonymous_at_portal.aei.mpg.de/home/
repository co gridportlets
39
Configuring Grid Portlets
  • Configuration amounts to specifying what
    resources Grid Portlets makes available to users.
  • The first thing you should do is specify what
    MyProxy resource your portal will use for
    retrieving / storing Grid credentials.
  • If want to use our MDS2 information provider,
    then you should specify what GIIS resource you
    want to use.
  • Otherwise, you should make sure to specify the
    hardware and service resources that make up your
    Grid

40
How To Specify Resources
  • There are 2 ways to add resources to Grid
    Portlets
  • Online with the Resource Registry Portlet
  • Offline with Resources.xml

41
How To Get Involved
  • Join one of gridsphere mail lists. Ask questions,
    provide feedback / comments.
  • Submit bugs and feature requests to
    http//bugs.gridsphere.org.
  • Because we are a small group, getting involved as
    a developer means demonstrating some knowledge /
    commitment to the project. Best way is to email
    ideas / patches to the list. If youre loud
    enough, well probably give you commit access and
    a few tasks!

42
Projects Using Grid Portlets
  • AEI Numerical Relativity Portal
  • CCT Numerical Relativity Portal
  • NRC Portal
  • Unversity of Melborne, Australia
  • SDSC

43
Conclusion
  • Grid Portlets contains the Grid functionality in
    the GridSphere Project. It provides a framework
    for developing Grid portals.
  • It is still in prototype development mode, but we
    are already using it in the GridLab Project and
    in several other projects with which we are
    affiliated.
  • We will make announcements on our website at
    www.gridsphere.org as the status of Grid Portlets
    changes.
  • Feel free to try it out and send us comments.
About PowerShow.com