Building Grid Enabled Portals Using GridPort 3 - PowerPoint PPT Presentation

1 / 153
About This Presentation
Title:

Building Grid Enabled Portals Using GridPort 3

Description:

GT3 source download in the scheduler directory: http://www-unix.globus.org/toolkit ... Access from different types of devices (desktop, laptop, PDA, cell phone) ... – PowerPoint PPT presentation

Number of Views:207
Avg rating:3.0/5.0
Slides: 154
Provided by: josh204
Category:

less

Transcript and Presenter's Notes

Title: Building Grid Enabled Portals Using GridPort 3


1
Building Grid Enabled Portals Using GridPort 3
  • TACC Technology Transfer
  • October 25, 2004
  • Eric Roberts
  • Maytal Dahan
  • Akhil Seth
  • Tomislav Urban
  • David Walling

2
Goals
  • This training class is intended for current and
    future GridPort 3 users
  • GridPort 3.0.2 is the current release
  • Participants should leave knowing how to build
    and install GridPort3 and use it to develop
    grid-enabled portals
  • Required Skills
  • Basic Unix and Java Programming Skills
  • Basic Understanding of Grid Tools such as Globus

3
Agenda
  • 900 1000 AM
  • What is the Grid and Grid Computing?
  • Globus Toolkit
  • GridPort 3
  • 1015 Noon
  • GridPort Components Architecture, Informational
    Services, Interactive Services
  • Portal development with GridPort
  • 100 200 PM
  • Lab Installing, Configuring, and Testing
    GridPort
  • Lab Informational Administration Client
  • 215 400 PM
  • Lab Build and Deploy a Portlet Interface
  • Training Evaluation

4
What is Grid and Grid Computing?
5
What is Grid and Grid Computing?
  • There are several competing definitions for The
    Grid and Grid computing
  • These definitions tend to focus on
  • Implementation of Distributed computing
  • A common set of interfaces, tools and APIs
  • Some stress the inter-institutional aspect of
    grids and Virtual Organizations
  • The Virtualization of Resources abstraction of
    resources

6
What is Grid and Grid Computing?
  • Grid computing promises a standard, complete
    set of distributed computing capabilities
  • There is a lot of hype around grid computing
  • Traditional users need to get work done now!
  • Some CS researchers see it as a fad
  • But there is real-world value!
  • In e-science and e-business

7
What is Grid and Grid Computing?
  • Grid computing must provide basic functions
  • resource discovery and information collection
    publishing
  • data management on and between resources
  • process management on and between resources
  • common security mechanism underlying the above
  • process and session recording/accounting
  • Current grid computing tools such as Globus
    provide most of the above at some level
  • The current capabilities are incomplete
  • New web service based-standard will help current
    tools become interoperable.

8
Globus Toolkit
9
What is Globus Toolkit 3 (GT3) ?
  • Open source toolkit for building grids
  • Low-level Middleware and grid services
  • Migrating from OGSI to WSRF
  • Current Globus Toolkit implements the Open Grid
    Services Infrastructure (OGSI)
  • Set of WSDL specifications defining standard
    interfaces, behaviors, and schema for grid
    computing
  • GT3 is an implementation of OGSI (grid services)
  • This is being replaced by Web Services Resource
    Framework (WSRF)
  • Similar functionality to OGSI but adhering to Web
    services standards
  • Includes Globus 2 services (pre-OGSI)

10
GT3 Components
  • 4 Major components to GT3
  • Security (GSI)
  • Data management (Grid FTP, RFT)
  • Resource Management (GRAM, MMJFS)
  • Information Services (Index Services)
  • More information can be found at
    http//www.globus.org

11
Security
  • Globus uses Grid Security Infrastructure (GSI)
  • GSI Authentication has the following components
  • Certificate Authority (CA)
  • Certificate
  • Private Key
  • Proxy
  • http//www.globus.org/security/

12
Security Certificate Authority
  • Institution or 3rd party organization that grants
    certificates
  • The role of the CA in this process is to
    guarantee that the individual granted the unique
    certificate is, in fact, who he or she claims to
    be.
  • Every institution or virtual organization (VO)
    would have their own CA to create certificates
    for their users.
  • TACC has a CA where users can get certificates
    and create portal accounts for the TACC User
    Portal

13
Security Certificate and Key
  • Grid certificate is like a passport
  • Allows single sign on authentication
  • Certificate /.globus/usercert.pem
  • Key /.globus/userkey.pem
  • Globus uses these certificates to authenticate
    users

14
Security Proxies
  • Proxy gives you single sign on capabilities for
    12 hours by default (i.e. the proxy lifetime)
  • Log on to any grid resource for which you have
    authorization
  • Users have a Distinguished Name (DN) which is
    their certificate identity
  • The DN must be in a Globus grid-mapfile on the
    remote resources users want to access
  • Maps a users DN to a system account on a grid
    resource

15
Data Management
  • GridFTP
  • GridFTP is a high-performance, secure, reliable
    data transfer protocol optimized for
    high-bandwidth wide-area networks.
  • GSI Security on control channels and data
    channels
  • Built on FTP
  • Put, Get, Third Party Transfer
  • Currently available through GridPort
  • Reliable File Transfer (RFT) is a grid service
    implementation of GridFTP.
  • Not currently available through GridPort

16
Resource Management
  • Master Managed Job Factory Service (MMJFS)
  • Job Submission
  • Batch Job submission submit a job to a local
    scheduler and return a job handle
  • Job Handles URLs known as Grid Service Handles
    (GSH)
  • Interactive Job Submission execute a command
    and wait for results
  • Job Status query a jobs status using the GSH

17
Information Services
  • Data is gathered using informational grid
    services
  • GT3 uses Index Services to aggregate data from
    informational grid services
  • Individual grid services can register with an
    Index Service
  • Index Services can be arranged hierarchically

18
Bridging Globus and GridPort
  • GridPort 3 wraps most of the Globus components
    mentioned in order to provide a consistent and
    simple interface for rapid client development
  • GridPort 3 adds some of its own services that
    aggregate and/or enhance core Globus functionality

19
GridPort 3
20
GridPort
  • High-Level middleware introduced between the
    low-level grid services such as Globus and the
    interface layer such as portals.
  • This layer can fundamentally transform the ease
    and speed with which user-interface developers
    can bridge the gap between end-users and a grid.

21
Software Stack
22
GridPort 3 Status
  • Current Release GridPort 3.0.2
  • uses GT 3.0.2
  • GridPort 3.1 release scheduled for SC04
  • GT 3.2.1 compatibility
  • Additional functionality - SRB, Advanced File
    Transfer, and more
  • GridPort 3 is included in NMI GRIDS-R5 and
    NPACKage 1.x
  • http//www.gridport.net - download, project
    description, user guide, mailing list, etc.

23
GridPort-Globus Component Mapping
  • Security
  • GridPort repository and MyProxy support
  • Data Management
  • File Management
  • Resource Management
  • Single Resource
  • Batch Job Submission
  • Command Execution
  • Grid
  • Meta-scheduler Batch Job Submission
  • Information Services
  • GridPort Information Repository (GPIR)
  • Aggregated services
  • Job Sequencer

24
Globus Compatibility
  • Currently planning support for GT 3.2.1 in the
    GridPort 3.1 release
  • Will evolve to WSRF after GT4 is released
    February 05

25
GridPort Components Informational Services,
Interactive Services
26
Informational Services GPIR
  • Clients, such as portals, will frequently need
    information regarding the state of a grid
  • GridPort Information Repository (GPIR) is used to
    store and retrieve information about the grid
  • GPIR is a relational database (PostgreSQL) based
    system
  • Query and Ingestion functionality is accessible
    via a Web Services interface

27
GPIR
  • Stores two broad types of data
  • Dynamic grid status data
  • Load, Node, Resource Status, Jobs, NWS, MOTD
  • Human-centric data
  • Virtual organizations (VO), Resource data,
    Contacts, Sites, Institutions, Departments

28
GPIR Interfaces
  • Web Services
  • Query
  • Allows WS-based queries of grid data
  • Reads from the database
  • Ingester
  • WS-based input of dynamic grid data (e.g. Job
    Data)
  • Writes to the database
  • Administration Client
  • Web-based GUI for the maintenance of
    human-centric grid data

29
Getting Information into GPIR
  • In order to take advantage of GPIR, it must be
    populated with grid data
  • Ingester Web Service
  • Dynamic grid data such as machine load will be
    populated via the Ingester web service
  • The Ingester is written to via Information
    Providers that run on or poll the resources
    themselves
  • Administration Client
  • For infrequently changing data such as resource
    names, use the web-based GPIR Admin client

30
Administration Client
  • Go to the web client and edit data directly

31
Information Providers and the Ingester WS
  • Sends XML data (job, node, motd, etc.) to GPIR
    web services
  • Run as cron jobs at the desired periodicity
  • Sample information providers are included with
    the GridPort download in the examples directory
  • The IP address of any provider writing to the
    Ingester Web Service must be registered with GPIR
    (called a client) for a given resource in order
    to provide data for that resource
  • This registration can be achieved through the
    administration client

32
GPIR Web Services
Information Providers
Resources
Clients
dB
Portals
Perl Client
Portlets
edu.tacc. gridport.gpir
Java Client
Ingester WS
Query WS
MDS
GPIR PostgreSQL
Other Middleware
OGSA (Future)
Web Scraping
SOAP-XML
HTTP
Other
JDBC
33
GPIR
  • GridPort comes with sample providers to populate
    GPIR with dynamic data
  • Sample job, node, load providers for LSF, PBS,
    LoadLeveler
  • Located in the gridport3/examples/providers
  • Also sample with Ingester client written in Perl
    but clients can be written in any language that
    has web services support

34
Interactive Services
  • Authentication
  • Using the GridPort repository
  • Using MyProxy
  • File Management
  • File Listing
  • File Transfer
  • Resource Management
  • Single Resource
  • Batch Job Submission
  • Command Execution
  • Grid
  • Meta-scheduler Batch Job Submission using CSF
  • Sequencer

35
Interactive Services - Authentication
  • What does logging in mean in GridPort?
  • GridPort can get a proxy to authenticate the user
    during a session
  • GridPort creates a session for the user
  • 2 ways to authenticate a user in GridPort
  • GridPort Repository
  • MyProxy

36
Interactive Services - Authentication
  • Using the GridPort Repository
  • Repository has 2 main tasks
  • Store a users credentials
  • These credential files are stored according to
    GridPort username.
  • Example ltusernamegt_cert.pem and
    ltusernamegt_key.pem
  • User logs in to GridPort with username and
    private key password
  • Maintain state of a user
  • When a user logs in a session ID is created
  • The proxy for the session is named by the session
    ID
  • A session file for maintaining state is created
  • The session file is named with the session ID

37
Interactive Services - Authentication
  • Using MyProxy (http//grid.ncsa.uiuc.edu/myproxy/)
  • MyProxy is an online credential repository
  • User stores a proxy on a MyProxy server for a 7
    day period
  • User can log in to GridPort to by retrieving
    credentials stored in MyProxy
  • It is up to the user to store credentials in
    MyProxy
  • This is for more grid savvy users who dont want
    to give GridPort a copy of their credentials
  • What happens when a user logs in to GridPort
    with MyProxy?
  • GridPort checks out credentials from MyProxy
  • Generates a session file for this user

38
Sample Authentication Code
  • Using the GridPort Repository
  • Authentication authObj new Authentication()
  • String sessionId authObj.gsiLogin(username,
    passphrase)
  • GSSCredential cred authObj.getCredential()
  • Using Myproxy
  • Authentication authObj new Authentication()
  • String sessionId authObj.myproxyLogin(myproxyhos
    t,myproxyusername, myproxypassphrase)
  • GSSCredential cred authObj.getCredential()

39
Interactive Services File Management
  • File Listing
  • Used to list the files in a users home directory
    or any subdirectory
  • File Transfer
  • User can execute a GridFTP put transfers a file
    from the GridPort server to a remote resource
  • User can execute a GridFTP get transfers a file
    from a remote resource to the GridPort server
  • User can execute a GridFTP third party transfer
    transfers a file between two remote resources

40
Sample File Management Code
  • // Previous Login
  • Authentication authen new Authentication()
  • authen.gsiLogin(username, passphrase)
  • credential authen.getCredential()
  • ...
  • // File Listing
  • FileListing listing new FileListing(resource.ta
    cc.utexas.edu, credential)
  • FileInfo files1 listing.listFiles()
  • listing.closeConnection()
  • // File Put
  • FileTransfer.put(resource.tacc.utexas.edu,
    /tmp/localGridPortPutTest, /tmp/GridPortPutTest
    , credential)
  • // File Get
  • FileTransfer.get(resource.tacc.utexas.edu,
    /tmp/GridPortGetTest, /tmp/localGridPortGetTest
    , credential)
  • // Third Party file Transfer
  • FileTransfer.thirdPartyTransfer(resource1.tacc.ut
    exas.edu,
    /tmp/GridPortThirdPartyTest1,

    resource2.tacc.utexas.edu,

    /tmp/GridPortThirdPartyTest2,

41
Interactive Services Single Resource Management
  • Command execution
  • Allows a user to execute a command on a remote
    resource
  • Returns results immediately
  • Sample Command Execution Code
  • String command /bin/ls
  • String args -l
  • String serviceUrl SERVICEURL
  • Command Execution cu new CommandExecution(creden
    tial, serviceUrl)
  • cu.setExecutable(command)
  • cu.setArguments(args)
  • cu.execute()
  • String stdErr cu.getStderr()
  • String stdOut cu.getStdout()

42
Interactive Services Single Resource Management
  • Batch job submission
  • Allows a user to submit a batch job to a
    scheduler on a remote resource
  • Returns a job handle which can be used to obtain
  • Job status
  • Job results

43
Batch Job Submission Sample Code
  • // instantiate GridPort batch job object
  • BatchJob job new BatchJob(credential)
  • // set the required parameters for the batch job
    submission
  • job.setArguments(hello)
  • job.setStdOut(/home/user/testjob.out)
  • job.setStdError(/home/user/testjob.err)
  • job.setDirectory(/home/user)
  • String serviceUrl ltURL_TO_REMOTE_MMJFSgt
  • String executable /bin/echo
  • // submit the batch job and capture the returned
    job handle
  • String jobhandle job.submit(serviceUrl,
    executable)
  • // create a new BatchJob object using the job
    handle
  • // that was obtained during submission of the job
  • submittedJob new BatchJob(credential,
    jobHandle)
  • // get the status of the job
  • String status submittedJob.getStatus()

44
Interactive Services Grid Resource Management
  • Grid Resource Batch Job Submission
  • Platform Computing developed an Open Source
    Community Scheduler Framework (CSF)
  • CSF is a meta-scheduling framework implemented as
    an OGSI grid service which schedules jobs across
    a grid
  • The CSF meta-scheduler schedules the job on a
    particular resource based on a scheduling
    plug-in.
  • How to obtain CSF
  • From Platform Computing http//www.platform.com/pr
    oducts/globus/
  • GT3 source download in the scheduler directory
    http//www-unix.globus.org/toolkit
  • GridPort can submit batch jobs to CSF which will
    pick an appropriate resource and submit the job

45
Sample Grid Job Submission Code
  • // create a new CSF Job passing in a credential
  • CsfJob csfjob new CsfJob(credential)
  • String jobHandle csfjob.submit(serviceUrl,
    queue, executable, arguments, directory, count,
    stdin, stdOut, stdErr)
  • //Get Job Status
  • CSfJob submittedJob new CsfJob(credential,
    jobHandle)
  • String stat submittedJob.getStatus()
  • //Cancel a job submitted to CS
  • CsfJob submittedJob new CsfJob(this.cred,
    this.jobHandle)
  • boolean jobCancel cu.jobCancel()
  • submittedJob.destroyJobService()

46
Interactive Services - Sequencer
  • Allows a user to construct a sequence of steps
  • File transfer
  • CSF job submission
  • File transfer from the host CSF chose (if this
    step occurs directly after a CSF job submission)
  • Interface exposed as a web service
  • Uses a Sequencer session to maintain state and
    authentication of the sequence even after the
    user has logged out of the portal
  • Tentative plans to add batch job submission and
    command execution
  • This requires a working instance of GPIR

47
Sample Job Sequencer Code
  • //Create a new sequence passing in the sessionID
  • SequenceBuilder sb new SequenceBuilder(sessionID
    )
  • //Add a file transfer step to the sequence
  • this.sb.addGridFtpStep(transferFrom, transferTo,
    sourcefile, destfile)
  • //Add CSF Step to the sequence
  • this.sb.addCsfStep(factory, queue, executable,
    args, dir, count, stdIn, stdOut, stdError)
  • //Transfer CSF output file to another host
  • this.sb.addCsfFtpStep(toHost, csfOutputFile,
  • destFile )
  • //Now we have built the sequence submit it
  • sequenceID sq.submit()
  • //Get status of sequence
  • String status sq.getStatus()

48
GridPort Architecture
49
Interfaces to GridPort Portals and More
50
What is an Interface?
  • Definition
  • The point of interaction or communication
    between a computer and any other entity, such as
    a printer or human operator.
  • GridPort has many different types of interfaces
  • API
  • Portal
  • Web Service

51
Why are GridPort Interfaces Useful?
  • Need a method for exploiting the functionalities
    of GridPort
  • Can have multiple types of interfaces to access
    the same functionality but in different ways
  • A well designed interface should make complex
    tasks easier to understand and execute

52
Who Uses the GridPort Interfaces?
  • Users of the GridPort interfaces include
  • API Application or User Interface developer
  • Portal Users
  • Web Service Applications

53
API
  • Application Programming Interface
  • Users of GridPort developing applications require
    a well-defined interface that they can program to

54
Portal
  • Easily accessible by anyone with a Web browser
  • Low resource overhead for the user
  • Access from different types of devices (desktop,
    laptop, PDA, cell phone)
  • Centrally managed by portal development team

55
Web Service
  • Web services allow distributed applications to
    communicate with each other in a platform- and
    programming language-independent manner.
  • Web services and clients must both speak SOAP
    protocol

56
Portal Development with GridPort 3
57
Objectives
  • Discuss what a portal is and why they are useful
  • Look at the Jetspeed portal framework in detail
  • Discuss the demo portal that is packaged with
    GridPort 3.0
  • Build and deploy a grid portlet

58
Portal Overview
59
What is a Portal?
  • A portal is a gateway to a set of distributed
    services that can be accessed from a Web browser
  • A portal provides a consistent environment for
    accessing services

60
How does a portal relate to GridPort?
  • A portal is one example of an interface
    application that uses GridPort.
  • Exposes services through a Web-based interface.
  • A GridPort based portal is grid-enabled in that
    it can communicate with other grid middleware

61
Grid-enabled portals
  • GridPorts Role
  • Grid Middleware aids developers by easing use of
    low level grid tools
  • Simplifies portal development which
  • lowers the barrier of entry for grid portal
    developers
  • Allows them to easily build portals and
    applications that access grid resources

62
Jetspeed A Portal Framework
63
What is Jetspeed?
  • Open-source enterprise portal framework that uses
    Java and XML.
  • It provides basic portal capabilities such as
  • Security
  • Customization
  • Localization
  • Personalization
  • Jetspeed uses a technology called portlets
  • We will be using Jetspeed version 1.x for this
    tutorial

64
Portal Architecture
65
Portal Role
  • The role of the portal framework is to
  • aggregate the dynamic content produced by the
    portlets
  • organize that content into some layout
  • ultimate control over how content is displayed
  • renders the complete portal page

66
What is a portlet?
  • A portlet is a container managed Web component
    that processes client requests and produces
    dynamic content
  • Unlike servlets, portlets do not have direct
    interaction with Web clients.
  • Web clients interact with a portal through a
    request/response mechanism enforced by a portlet
    container which also manages the lifecycle of the
    portlets.
  • Generally, portlets have a clean separation of
    content from presentation which is managed by one
    or more Java classes that contain the application
    logic

67
Portlet Modes
  • There are three basic portlet modes
  • View
  • Normal user interaction with the portlet
  • Edit
  • Customize behavior of the portlet
  • Help
  • Help information about the portlet

68
Portlet Window States
  • There are three basic window states classified by
    the portlets real estate within the portal page
  • Normal
  • Portlet is sharing space with other portlets on
    the portal page
  • Maximized
  • Portlet may use more space than when in Normal
    state and may display more detailed content
  • Minimized
  • Portlet may use less space than when in Normal
    state and may display less or no content

69
General Portal Page Layout
70
Jetspeed Customizer
  • Tool for manipulating layout from within the
    portal
  • Provides the capability to
  • Add, remove and move portlets in a particular
    pane
  • Apply different skins to each pane
  • Rename a pane
  • Add, remove and move panes

71
(No Transcript)
72
Portal Registries
  • Portal entities defined in the Portal Registry
  • portlets
  • controllers (layout portlets)
  • controls (portlet decorators)
  • skins
  • media-types (HTML, WML)
  • Security Constraints
  • Registry entries are defined in XML file(s) with
    a .xreg extension and reside in the
    WEB-INF/conf directory

73
Portlet Registry Types
  • Instance
  • Contains all information to instantiate a portlet
  • Abstract
  • Cannot instantiate
  • missing required information
  • useful for abstracting properties for groups of
    similar portlets
  • Ref
  • References another registry entry (of any type)
  • cascade references until abstract or instance
    entry is encountered
  • overrides all parameters in entry and ATTEMPT to
    instantiate the portlet
  • From Jetspeed Tutorial

74
Registry Example Portlet
lt?xml version"1.0" encoding"UTF-8"?gt ltregistrygt
ltportlet-entry name"CommandExecution"
hidden"false" type"ref"
parent"Velocity" application"false"gt
ltmeta-infogt lttitlegtCommand
Executionlt/titlegt ltdescriptiongtAllows
a user to execute a command on a remote
resource.lt/descriptiongt lt/meta-infogt
ltclassnamegtorg.apache.jetspeed.portal.portlets.
VelocityPortletlt/classnamegt ltparameter
name"template" value"CommandExecution"
hidden"true" cachedOnName"true"
cachedOnValue"true"/gt ltparameter
name"action" value"gp3x.CommandExecutionAction"
hidden"true" cachedOnName"true"
cachedOnValue"true"/gt ltmedia-type
ref"html"/gt lturl cachedOnURL"true"/gt
lt/portlet-entrygt lt/registrygt
75
Skins
  • Skins are a way of changing the look and feel of
    the entire portal using Cascading Style Sheets
  • Skin settings are particular to each user
  • Examples of different skins

76
(No Transcript)
77
Registry Example Skin
lt?xml version"1.0" encoding"UTF-8"?gt ltregistrygt
ltskin-entry nameorange-grey"
hidden"false"gt ltproperty
name"text-color" value"000000"
hidden"false"/gt ltproperty
name"background-color" value"ffffff"
hidden"false"/gt ltproperty
name"title-text-color" value"000000"
hidden"false"/gt ltproperty
name"title-background-color" value"eeeeee"
hidden"false"/gt ltproperty
name"title-style-class" value"TitleStyleClass"
hidden"false"/gt ltproperty
name"highlight-text-color" value"ffffff"
hidden"false"/gt ltproperty
name"highlight-background-color" value"000000"
hidden"false"/gt lt/skin-entrygt lt/registrygt
78
Portal Layout Structure PSML
  • Layout in Jetspeed is specified using the Portlet
    Structure Markup Language (PSML) written in XML
  • PSML files are used to specify different layouts
    for different users, groups and roles that the
    portal supports
  • Static PSML files are loaded into a DB at portal
    runtime
  • Jetspeed Customizer manipulates PSML structure in
    DB
  • Upon portal shutdown PSML is serialized back to
    static PSML

79
PSML and Registry Entries
  • The following types of registry entries can be
    referenced in a PSML file
  • portlets,
  • controllers (layouts),
  • controls (decorators),
  • skins,
  • media-types
  • security

80
PSML Example Referencing Registry Entries
lt?xml version"1.0" encoding"UTF-8"?gt ltportlets
id"100"gt ltsecurity-ref parent"default"/gt
ltcontroller name"CardPortletController"gt
ltparameter name"parameter" value"pane"/gt
lt/controllergt ltcontrol name"TabControl"/gt
ltskin nameorange-grey"/gt ltportlets
id"301"gt ltsecurity-ref
parent"default"/gt ltcontroller
name"ColumnController"/gt ltmetainfogt
lttitlegtCommand Executionlt/titlegt
lt/metainfogt ltentry id"3011"
parentCommandExecution"/gt lt/portletsgt
lt/portletsgt
81
Profile Requests
  • Mapping of requests to template and PSML
    resources
  • Fallback algorithm used to locate the appropriate
    templates and PSML
  • Algorithm searches from more specific to more
    general
  • Country code
  • Language
  • Media type
  • User type
  • Can localize your portal for many languages
  • Appropriate language will display based on origin
    of the request

82
Templates
  • Using templates allows portlets to inject
    dynamic data into template interfaces which are
    then rendered into their final state
  • Jetspeed supports Velocity and JavaServer Page
    (JSP) template engines
  • Velocity templates do not allow as much freedom
    for putting application logic into the actual
    template code like JSP (which is a bad thing ?
    from a design perspective)
  • We will be building a Velocity based portlet in
    this tutorial

83
Built-in Portlet types
  • There are many pre-defined portlet types that are
    available with Jetspeed
  • AbstractPortlet (all other portlets extend this
    basic portlet type)
  • Velocity uses Velocity templates for displaying
    dynamic content
  • CustomizerVelocity like a Velocity portlet with
    customization support built-in
  • JSP uses JSP pages for displaying dynamic
    content
  • WebPagePortlet formats and displays content
    from and existing web page
  • HTML displays an HTML page
  • WML displays a WML page (e.g. on a handheld
    device)
  • RSS formats and displays a Rich Site Summary
    (RSS) feed
  • XSL formats and displays content that has been
    transformed using an XSL template
  • LinkManager manage links

84
Extending Built-in Portlet Types
  • Define your own portlets by extending a basic
    portlet types
  • Can only extend one basic portlet type (i.e.
    single inheritance)
  • Specify instance, abstract, or reference portlet
    types in portlet registry

85
Security Model
  • Users
  • Users of the portal
  • Permissions
  • Permissions to use portlet modes and states
  • Groups
  • Groups of users who generally have the same needs
    for using the portal
  • Roles
  • A user can have multiple roles which controls
    access to portal resources (e.g. user, admin)

86
Portal Services - Turbine
  • Built-in services
  • Jetspeed is built on top of the Turbine Servlet
    framework
  • Turbine provides
  • Security (login)
  • Presentation (velocity JSP)
  • Persistence (Database)
  • HTML Form Validation
  • Logging
  • Services (Fulcrum)
  • Caching
  • Scheduling
  • Services are available for use inside of
    portlets.
  • See Turbine documentation for defining your own
    services

87
Future Directions Portals
  • Sun has defined a standard Portlet API through
    Java Specification Request (JSR) 168 which
    defines the contract between portlets and the
    portlet container
  • We will be embracing this standard and will
    eventually re-implement our portlets using JSR168
    technology
  • JSR168 is still in its infancy so we expect a
    timeline of about 4-6 months _at least_

88
Future Directions Interfaces to GridPort
  • GridPort could be used for interfaces other than
    portals.
  • We are currently talking about Java Swing
    application interfaces, Command Line interfaces
    and more Web service interfaces

89
References
  • Jetspeed Portal Framework
  • http//portals.apache.org/jetspeed-1/
  • Turbine Web Application Framework
  • http//jakarta.apache.org/turbine/
  • JSR 168 Portlet Specification
  • http//www.jcp.org/en/jsr/detail?id168
  • Pluto Project
  • http//portals.apache.org/pluto/

90
Lab Section
91
Goal of the Lab
  • Install and Configure GridPort including GPIR
    administration client and a GridPort demo portal
  • GridPort web site has an installation guide that
    can walk you through the GridPort installation
    http//www.gridport.net
  • Slides from the training class will be available
    on the web side under publications presentations

92
Installing GridPort
93
Laptop Instructions
  • Login to Laptops
  • Username training
  • Passphrase taccguest
  • Accounts will only be active today
  • Start a New Terminal
  • right click on desktop and click on New
    Terminal
  • Disable Terminal Sound
  • On the terminal window go to Edit -gt Profiles
    Menu
  • Edit the Default Profile
  • Uncheck the Terminal Bell Checkbox on the General
    Tab page
  • To save hit Close on all the subsequent windows
  • Change directory to /usr/local/src/gridport3-train
    ing-software
  • Navigate to new terminal window
  • cd /usr/local/src/gridport3-training-software
  • This directory has all the required tar balls
    from the appropriate web sites.

94
Installing Required Tools
  • Java J2SE 1.4.1 or higher
  • http//java.sun.com/downloads/index.html
  • Already Installed run which java to see the
    location
  • Apache Ant 1.5 or higher
  • http//ant.apache.org/bindownload.cgi
  • Already Installed run which ant to see the
    location
  • Globus 3.0.2 must be installed on the resources
    you want to interact with.
  • http//www-unix.globus.org/toolkit
  • Platform CSF must be installed on a resource and
    configured to submit jobs to your resources.
  • http//www.platform.com/products/globus/

95
Required Tools
  • For GPIR and Job Sequencer
  • PostgreSQL 7.3
  • http//www.postgresql.org/
  • PostgreSQL JDBC Driver
  • http//jdbc.postgresql.org/
  • JBOSS 3.2.x with Tomcat 5.x
  • http//jboss.org
  • For Demo Portal
  • Apache Tomcat 4.1.x
  • http//jakarta.apache.org/site/binindex.cgi

96
Users during Install Process
  • Main users
  • GridPort User This is the user who will act as
    the main gridport user. We recommend this user
    have limited access and that permissions are made
    correctly. For our lab the gridport user will be
    your lab user training.
  • Root User This is required to install some of
    the prerequisite tools
  • Postgres User This is a database user. It runs
    the database and database related tasks.

97
Installing GridPort
  • Untar the GridPort Binary Install in /usr/local
  • cd /usr/local
  • tar xvfz /usr/local/src/gridport3-training-softw
    are/gridport3.src.tar.gz
  • This will create a gridport3 directory in
    /usr/local
  • GridPort directories
  • as contains application server files for GPIR
  • config contains GridPort configuration
    information
  • db contains database related information
  • examples Demo portal, GPIR schemas, and
    providers
  • lib contains all the jar files needed for
    GridPort
  • schema schema files that globus uses

98
Installing GridPort
  • Setup GridPort environment
  • cd gridport3
  • Use vi or your most comfortable text editor to
    create a gridport3.env file
  • vi gridport3.env
  • export GP_HOME/usr/local/gridport3
  • export JAVA_HOME/usr/java/j2sdk1.4.2_04
  • export ANT_HOME/usr/local/apache/apache-ant-1.6.1
  • export PATHPATHANT_HOME/bin
  • source gridport3.env
  • Note This is for bash shell modify to handle
    your environment

99
Installing GPIR PostgreSQL setup
  • Installing PostgreSQL
  • We have already untarred and compiled postgreSQL
    source to save time. If we didnt you would
    normally untar the postgres sql directory and
    compile it.
  • Note PostgreSQL doesnt need to be installed on
    the same machine, it can be installed on a DB
    machine

100
Installing GPIR PostgreSQL setup
  • PostgreSQL
  • cd /usr/local/src/gridport3-training-software/po
    stgresql-7.4.1
  • ./configure ALREADY DONE
  • gmake ALREADY DONE
  • su
  • gmake install
  • /usr/sbin/adduser postgres
  • mkdir /usr/local/pgsql/data
  • chown postgres /usr/local/pgsql/data
  • su - postgres
  • /usr/local/pgsql/bin/initdb -D
    /usr/local/pgsql/data
  • /usr/local/pgsql/bin/postmaster -D
    /usr/local/pgsql/data gtlogfile 2gt1
  • /usr/local/pgsql/bin/createdb test
  • /usr/local/pgsql/bin/psql test
  • (Type \q to exit the database)
  • exit
  • exit (return to training user)

101
Installing GPIR PostgreSQL setup
  • Setup PostgreSQL Environment
  • Open gridport3.env in the GridPort home directory
  • cd /usr/local/gridport3
  • vi gridport3.env
  • Append the following
  • export PGSQL_HOME/usr/local/pgsql
  • export PGDATAPGSQL_HOME/data
  • source gridport3.env

102
Installing GPIR PostgreSQL setup
  • Create GPIR tables and User (pgsql user)
  • su
  • su postgres
  • cd PGSQL_HOME/bin
  • ./createdb GPIR
  • ./psql GPIR
  • GPIR CREATE USER gridportdbuser WITH
  • GPIR PASSWORD gridportdbpass
  • GPIR \q

103
Installing GPIR PostgreSQL setup
  • Setup Connections (pgsql user)
  • Open the server to TCP/IP connections
  • gt vi PGDATA/postgresql.conf
  • Line 30 tcpip_socket false
  • Uncomment and change value to true. Line should
    look like this
  • tcpip_socket true

104
Installing GPIR PostgreSQL setup
  • Setup IP Connection (pgsql user)
  • NOTE Since we are using the DB locally we dont
    need this step. If DB is on another machine you
    will need this entry!
  • gt vi PGDATA/pg_hba.conf
  • Add entry to the bottom
  • host GPIR gridportdbuser ltIPADDRgt ltIPMASKgt trust

105
Installing GPIR PostgreSQL setup
  • Restart PostgreSQL (postgres user)
  • PGSQL_HOME/bin/pg_ctl stop m fast
  • PGSQL_HOME/bin/pg_ctl start
  • (hit enter)
  • exit
  • exit

106
Installing GPIR JBoss setup
  • Install JBoss
  • Untar JBoss
  • cd /usr/local
  • tar xvfz src/gridport3-training-software/jboss-3
    .2.4.tar.gz
  • This will create a jboss-3.2.4 directory in
    /usr/local
  • This has 3 server configurations all, minimal,
    default we will use default
  • Everything here done as user training

107
Installing GPIR JBoss setup
  • Setup environment
  • Add the following to
  • cd GP_HOME
  • vi GP_HOME/gridport3.env
  • Append the following
  • export JBOSS_HOME/usr/local/jboss-3.2.4
  • export JAVA_OPTS-Xms256m -Xmx256m
  • (the java opts isnt necessary but we found it to
    be useful)
  • source gridport3.env

108
Tomcat Configuration and Installation
  • Install Tomcat
  • cd /usr/local
  • unzip src/gridport3-training-software/jakarta-to
    mcat-4.1.30.zip
  • This will create a /usr/local/jakarta-tomcat-4.1.3
    0
  • Setup Environment
  • cd GP_HOME
  • vi gridport3.env
  • Append the following
  • export CATALINA_HOME/usr/local/jakarta-tomcat-4.1
    .30
  • source gridport3.env

109
Tomcat Configuration and Installation
  • Copy GP_HOME/lib/xml/xalan.jar to
    JAVA_HOME/jre/lib/endorsed
  • Note You may have to create the endorsed
    directory. If you installed Java as root you may
    need to become root to copy to the Java endorsed
    directory using the su command.
  • gt su (optional)
  • gt mkdir JAVA_HOME/jre/lib/endorsed
  • gt cp GP_HOME/lib/xml/xalan.jar
    JAVA_HOME/jre/lib/endorsed

110
Configuring GridPort
111
Authentication Configuration
  • Setup /etc/grid-security as root
  • su
  • cd /etc
  • tar xvf /usr/local/src/gridport3-training-softwa
    re/grid-security.tar
  • This will create an /etc/grid-security directory
    with the following
  • certificates/
  • 9a1da9f9.0
  • 9a1da9f9.signing_policy
  • exit (return to training user)

112
Modify GridPort Properties
  • Open GP_HOME/build.properties
  • gtvi GP_HOME/build.properties
  • Modify the following entries
  • jboss.location/usr/local/jboss-3.2.4
  • jdbc.driver.location/usr/local/src/gridport3-trai
    ning-software/pg74.1jdbc3.jar
  • gridport.db.passphrasegridportdbpass
  • Gridport.adminClient.usernamegridportadmin
  • Gridport .adminClient.passphrasegridportadmin

113
Modify GridPort Properties
  • gridport.repository.location/usr/local/GridPortRe
    pository
  • catalina.home/usr/local/jakarta-tomcat-4.1.30
  • csf.factory.urlhttp//laredo.tacc.utexas.edu1508
    0/ogsa/services/metascheduler/JobFactoryService
  • csf.queuenormal
  • csf.hostbuda.tacc.utexas.edu
  • file.management.machinesbuda.tacc.utexas.edu,blan
    co.tacc.utexas.edu,bandera.tacc.utexas.edu
  • portal.adminemailaddress

114
Install GridPort
  • Run the GridPort Install Target
  • This installs the GridPort tables in GPIR,
    configures and installs the GPIR administration
    client, and deploys the demo portal to tomcat
  • gt cd GP_HOME
  • gt ant install.all

115
Start JBoss
  • Startup JBoss from the GridPort home directory.
    This will run JBoss, create a log file in your
    home directory and run the process in the
    background.
  • Note You MUST start JBoss from the GridPort
    installation directory so that GridPort will
    correctly access configuration information!
  • gt cd GP_HOME
  • gt /bin/sh JBOSS_HOME/bin/run.sh gt
    /jboss-logfile 2gt1
  • Bring up the administration client and the GPIR
    web services to ensure they are running.
  • GPIR web services http//localhost8080/gridport/
    webservices
  • GPIR Administration client http//localhost8080/
    gridport

116
Startup Tomcat
  • Start Tomcat from the GridPort home directory
  • gt cd GP_HOME
  • gt chmod 744 CATALINA_HOME/bin/
  • gt /bin/sh CATALINA_HOME/bin/startup.sh
  • Ensure that Tomcat is up by making sure the
    demonstration portal comes up.
  • Open a browser to http//localhost9080/portal
  • There are two log files that are important when
    the portal is not behaving as expected. The
    first is for the Tomcat web server log located in
    CATALINA_HOME/logs/catalina.out
  • The second log file is for Jetspeed and is
    located in
  • CATALINA_HOME/webapps/portal/WEB-INF/log/jetspeed
    .log

117
Authentication Configuration
  • To create a GridPort account put sample
    certificate and key in stored credentials
    directory
  • cp /.globus/usercert.pem /usr/local/GridPortRep
    ository/storedCredentials/training_cert.pem
  • cp .globus/userkey.pem /usr/local/GridPortRepos
    itory/storedCredentials/training_key.pem

118
Authentication Configuration
  • Setup Myproxy
  • For simplicity we have already setup Myproxy.
  • Myproxy Host pecos.tacc.utexas.edu
  • Username training
  • Password trainingpass

119
Testing GridPort
120
Command Line Testing
  • GridPort uses JUnit for testing
  • Setup ant to use JUnit
  • su
  • cp GP_HOME/lib/common/junit.jar ANT_HOME/lib
  • exit
  • Tests are run and they create XML and HTML result
    files in GP_HOME/test/reports directory
  • Need to add parameters to tests.properties for
    the commands to test

121
Add localhost as GPIR client
  • The IP address of the machine sending data to
    GPIR must be registered with GPIR
  • Go to http//localhost8080/gridport and log in
  • Go to Admin -gt Client -gt Add
  • Name GP Training Client
  • IP Address 127.0.0.1
  • Hostname localhost
  • Location Training Class
  • Description Training class client
  • Scripts GP sample Install
  • Select all the resources
  • Hit Submit

122
Command Line Testing
  • Copy Tests.properties from /usr/local/src/gridport
    3-training-software/ to GP_HOME
  • Open tests.properties
  • List of Tests
  • Configuration
  • Authentication
  • File Listing
  • File Transfer
  • Batch Job
  • CSF Job
  • Command Execution
  • Job Builder
  • Ingester Client
  • Query Client

123
Command Line Testing
  • Run the command line tests
  • gt ant test.all
  • Results get stored in XML and HTML in the
    tests/reports directory

124
GPIR Administration Client
125
Testing Administration Client
  • It is important to test GPIR, JBoss, and the
    Administration Client.
  • Testing the Administration Client
  • Go to http//localhost8080/gridport
  • Log in to the Administration Client and make sure
    it is working. It should have a sample VO in it.

126
Using the GPIR Administration Client
  • Assuming that JBoss is running, the admin client
    will be at http//localhost8080/gridport
  • Log in using the username and password you
    created during the JBoss install
  • We are going to
  • Create a Virtual Organization
  • Create a Resource and add it to the virtual
    organization

127
Creating a Virtual Organization (VO)
  • Click on the sign to the left of the Admin
    link in the left-hand navigation menu
  • In the list under Admin click on VOs. This
    should display a list of VOs in the main part of
    the screen. You will probably have only a single
    sample VO listed.
  • Click on the add link at the bottom of the VOs
    list.
  • Fill in the add vo form (at least a name is
    required) and click on the submit link.
  • You should now have the VO that you just added in
    the list of VOs

128
Creating a new Resource
  • List the compute resources by clicking on the
    sign to the left of the Resources link under
    the Admin list and then clicking on Compute.
  • The add resource form is fairly extensive.
    Fill out the name, hostname, and ip address
    fields.
  • The vos checkboxes allow you to add the
    resource you are creating to one or more virtual
    organizations.
  • The clients checkboxes allow you to designate
    which clients will be allowed to update data for
    the resource you are creating via the GPIR
    Ingester web service
  • Click on submit when you are done.

129
Admin Client Demo
  • We have now added a new resource and associated
    it with the VO that we created earlier.
  • All of the lists available in GPIR, such as
    Institutions, Contacts, and Sites, are editable
    in a similar manner.
  • This provides a simple way to maintain your grid
    information.

130
Admin Client Questions?
  • Play with adding and editing other entities such
    as manufacturers and models, etc.
  • Questions? Problems?

131
Demo Portal
132
Portal Lab
  • Build and deploy a grid portlet
  • Add and configure the portlet in the portal layout

133
Create a New Portal Account
  • cd to GP_HOME
  • Start Tomcat and pull up http//localhost9080/por
    tal
  • Click the Create New Account link
  • Fill in the form with your user information
  • Username is training
  • Password is the grid credential passphrase for
    the training certificate
  • First Name, Last Name, Email Address
  • Press the Create New Account button
  • You should now be logged into the portal
  • Log out of the portal by clicking the Logout
    link
  • Log back into the portal with your username and
    password to verify
  • Whether creating a new account or just logging in
    you are authenticating with GridPort and a proxy
    credential is created on your behalf and stored
    in your session
  • Logout of the portal.

134
Portal Authentication
  • Since most grid tasks involving Globus Toolkit
    use GSI authentication the most important thing
    we need in order to interact with Globus is a
    proxy credential.
  • In the demo portal a proxy credential is created
    upon login and stored in the users session so
    that it is available at any time to any portlet
    application within the portal for that user.
  • The GridPort demo portal provides just one
    example of how to put a proxy credential into a
    users sessions. You may have different needs
    for authenticating users.

135
Building a Portlet
  • Building a portlet is simple. Building a
    grid-enabled portlet is more difficult. However,
    with the GridPort API at our disposal it makes
    life easier.
  • This part of the tutorial will show you how to
    build a grid-enabled portlet within the GridPort
    demo portal
  • Given a hostname our grid portlet will perform a
    simple command execution on a remote host to
    print some version information about that
    resource.

136
Generate a Template Portlet Application
  • GridPort comes with a utility for creating and
    deploying Velocity-based portlets.
  • Create a new portlet using the following Ant
    target
  • ant new-portlet
  • Name the portlet MyGridPortlet and give it a
    description
  • This will create the necessary portlet directory
    structure in GP_HOME/examples/portal/portlets
  • We will be editing the following files
  • GridHelloWorld.xreg (configuration)
  • GridHelloWorldAction.java (action)
  • GridHelloWorld.vm (template)
  • Add MyGridPortlet/lib directory

137
A Word About Velocity Portlets
  • All Velocity portlet actions should extend the
    VelocityPortletAction class
  • The following methods are available to override.
  • buildNormalContext (view mode)
  • buildConfigureContext (edit mode)
  • buildMaximizedContext (maximized state)
  • What about buildMinimizedContext?
  • A portlet in a minimized state doesnt get
    rendered so there is no sense in providing code
    for it.

138
Velocity Context and Turbine RunData
  • The Velocity Context is the context object used
    to share information between the template and the
    action
  • Populate the context in the action with key-value
    pairs and access them in the template using a
    ltkeygt syntax
  • The Turbine RunData contains portal runtime
    information
  • E.g. session information
  • A Velocity portlet uses both of these as well
    see

139
Registry Configuration
lt?xml version"1.0" encoding"UTF-8"?gt ltregistrygt
ltportlet-entry nameMyGridPortlet"
hidden"false" type"ref" parent"Velocity"
application"false"gt ltmeta-infogt
lttitlegtMyGridPortletlt/titlegt
ltdescriptiongtdemo grid portletlt/descriptiongt
lt/meta-infogt ltclassnamegtorg.apache.jetspeed.po
rtal.portlets.VelocityPortlet lt/classnamegt
ltparameter name"template" valueMyGridPortlet
" hidden"true"/gt ltparameter name"action"
valueMyGridPortletAction" hidden"true"/gt
ltmedia-type ref"html"/gt ltcategorygtMyPortletslt
/categorygt lt/portlet-entrygt lt/registrygt
MyGridPortlet.xreg
140
buildNormalContext get the users name
open GP_HOME/examples/portal/portlets/MyGridPortl
et/ \ \src/org/apache/jetspeed/modules/actions/Gr
idHelloWorldAction.java --------------------------
------------------------------------ // Add this
import to existing imports import
org.apache.turbine.om.security.User ... // Add
these methods to the class body protected void
buildNormalContext(VelocityPortlet
velocityPortlet, Context context, RunData
runData) context.put("full-name"
,getName(runData)) private String
getName(RunData runData) User user
runData.getUser() String fullName
user.getFirstName() user.getLastName() r
eturn fullName
MyGridPortletAction.java
141
Print the users name
open GP_HOME/examples/portal/portlets/MyGridPortl
et/ \ /templates/GridHelloWorld.vm -------------
-------------------------------------------------
GridHelloWorld.vm Hello ltbgtfull-namelt/bgt ltbrgt
ltbrgt
MyGridPortlet.vm
142
Deploy
Stop Tomcat CATALINA_HOME/bin/shutdown.sh Start
Tomcat CATALINA_HOME/bin/startup.sh
  • Shutdown Tomcat
  • cd GP_HOME
  • Run the following Ant target to compile and
    deploy your portlet web application
  • ant deploy-portlet -Dportlet.nameMyGridPortlet
  • Start Tomcat
  • Point your browser to http//localhost9080/portal
  • IMPORTANT Always logout of the portal before you
    shutdown Tomcat

143
Add the portlet to your layout
  • Click on the edit (pencil) icon in the upper
    right-hand part of the portal page to enter the
    Jetspeed Customizer
  • Click the Add Pane button
  • Give your new pane a name My Grid Portlets
  • Click on the Apply button
  • Click on My Grid Portlets link in the list of
    panes
  • Click the Add Portlet button
  • File portlets by category MyPortlets
  • Click the check box next to the MyGridPortlet
    portlet and click Apply
  • Click Save and Apply
  • Click Apply
  • Click on the tab named My Grid Portlets

144
Making the GridPort call in the Action
// Add these imports to existing imports import
org.ietf.jgss.GSSCredential import
org.ietf.jgss.GSSException import
edu.tacc.gridport.job.CommandExecution import
edu.ta
Write a Comment
User Comments (0)
About PowerShow.com