VisPortal Project developers experience

1
VisPortal Projectdevelopers experience

• C.E.Siegerist, J. Shalf, E.W. Bethel
• NERSC/LBNL Visualization Group
• T.J. Jankun-Kelley, O. Kreylos, K.L. Ma
• CIPIC/UC Davis

2
Overview

• Grid Introduction
• Portal Architecture/Software components.
• Visportal Services/Example Pages
• Users perspective
• Conclusions/Future/Caveats
• References

3
The Grid

• The Grid is an abstraction that allows
  transparent use of distributed computing
  resources.
• The problem that underlies the Grid concept is
  coordinated resource sharing and problem solving
  in dynamic, multi-institutional virtual
  organizations.

4
VisPortal Project

• The LBNL/NERSC VisPortal effort explores ways to
  deliver Grid-based advanced visualization and
  data analysis capabilities through a Web Portal
  interface.
• Using standard Globus Grid middleware and off the
  shelf web automation, the VisPortal hides the
  underlying complexity of resource selection and
  distributed application management.

5
VisPortal Services

• Motivation simplify remote visualization, offer
  point an click solutions for complex distributed
  applications
• File Management
• Vis Applications Visapult, AMR Renderer, Web
  VisSheet.
• One button Mpeg

6
Globus

• The Globus Project provides software tools
  (Globus toolkit) that make it easier to build
  computational grids and grid-based applications.

7
Globusrun

Example of a file conversion RSL script
globusrun r escher.nersc.gov f convert.rsl -w
(executableGLOBUSRUN_GASS_URL
/home/cristina/convertscript)
(enviroment(GLOBUSRUN_GASS_URL
(GLOBUS_GASS_URL))
8

• !/sbin/sh
• GLOBUS_LOCATION/usr/local/globus
• export GLOBUS_LOCATION
• PATHGLOBUS_LOCATION/binGLOBUS_LOCATION/sbin
• export PATH
• LD_LIBRARY_PATHGLOBUS_LOCATION/lib
• export LD_LIBRARY_PATH
• IMAGE_MAGICK/usr/freeware/bin
• export IMAGE_MAGICK
• globus-gass-server -t -u -r -w -c gt
  mytmpgassfile
• sleep 3
• set remote_gass_server_urlcat mytmpgassfile
• rm mytmpgassfile
• GLOBUS_LOCATION/bin/globus-url-copy
• GLOBUSRUN_GASS_URL/home/cristina/file.jpg
  file/tmp/file.jpg
• IMAGE_MAGICK/convert /tmp/file.jpg /tmp/file.bmp
• GLOBUS_LOCATION/bin/globus-url-copy
  file/tmp/file.bmp
• GLOBUSRUN_GASS_URL/home/cristina/file.bmp

9
GPDK

• The Grid Portal Development Kit (GPDK) is a set
  of modular, reusable components for accessing
  Grid services in the form of Java Beans. It
  derives most of its functionality from the Globus
  Java Commodity Grid (Java CoG) toolkit.
• Other toolkits GridPort, GridSphere

10
Portal Architecture
11
(No Transcript)
12
Internal Architecture
13
Software Components

• openssl 0.9.6h
• globus 2.x
• java 1.4.x
• MyProxy 1.0
• log4j_1.2.7

• Apache 1.3.27
• Tomcat 3.3.1
• gpdk cvs version
• Java CoG v1.0a
• mod_jk 1.3
• mod_ssl 2.8.12

14
GPDKDemo Portal
15
Login Page

• Myproxy username and password of delegated
  credential
• Local load from credential file (grid-proxy-init)

16
Update usersresourcesjob historyemail address
Profile
17
File Mgmt.

• Foreground transfers
• Background transfers
• Globus File Yanker(Shreyas Cholia)

18

Launch Applications Visapult
19

Launch Applications AMR Renderer
20

Visportal WebSheet
21

MPEG Generator

• Example of a user driven service the user
  connects to our Onyx to run an IDL batch job to
  produce a movie with data that he has in an IBM
  SP
• The portal provides an interface to select the
  data file and the mpeg parameters. It transfers
  the file, launches the IDL job and sends an email
  to the user with a URL to retrieve the movie.

22

• The portal is running (most of the time)
• https//
• frost.lbl.gov/visportal/servlet/visportal

23
Users perspective

• Accounts in all the resources.
• Request a user certificate http//www.doegrids.or
  g/pages/cert-request.html
• Be in the grid-map files of the resources
  (automated through NIM).
• Create a proxy certificate and login.

24
Challenges

• Difficult GUI Implementation
• Error Recovery
• Support
• Middleware is a moving target
• Resource management

25
Future

• Deploying a subset of services to NERSC users.
• Implementing specific user workflows.
• Upgrade to OGSA/GT3.0
• Explore Gridlab/Gridsphere, portlets.

26

• Soon
• visportal.nersc.gov

27
References

• The anatomy of the Grid Enabling Scalable
  Virtual Organizations, Intl. J. Supercomputer
  Applications, 2001.
• The Grid Portal Development Kit, J.Novotny,
  Cuncurrency Pract. Exper. 2000 001-7
• How the Grid will affect the Architecure of
  Future Visualization Systems, J.Shalf and W.
  Bethel, http//www-vis.lbl.gov/Publications
• Building a Portal Using GPDK A Developers
  Tutorial, J. Novotny, http//doesciencegrid.org//p
  ublic/events/GPDW/slides/gpdk-dev.pdf
• Globus, A Metacomputing Infrastructure Toolkit.
  I. Foster, C. Kessselman, Internation Journal of
  Supercomputing applications, 1997.
• Deploying Web-based Visual Exploratin Tools on
  the Grid. T.J.Jankun-Kelly, O. Kreylos, J. Shalf,
  K-L. Ma, B. Hammann, K. Joy, E.W. Bethel. IEEE
  Computer Graphics and Applications, march/april
  2003.
• VisPortal, Deploying grid-enabled visualization
  tools through a web-portal interface.
  http//www-library.lbl.gov/library/docs/LBNL/529/4
  0/PDF/LBNL-52940.pdf
• Globus www.globus.org
• GPDK www.doesciencegrid.org