Challenges and Opportunities in Grid Middleware

1
Challenges and Opportunities in Grid Middleware
Oxford e-Science TAG Meeting January 21 2003

• PTLIU Laboratory for Community Grids
• Geoffrey Fox
• Computer Science, Informatics, Physics
• Indiana University, Bloomington IN 47404
• http//grids.ucs.indiana.edu/ptliupages

gcf_at_indiana.edu
2
Fox Comments on UK e-Science

• Excellent Program with coordinated and
  application oriented approach
• Industry involvement, OGSA, Data Grid, GGF
  strategy good
• Regional/National centers good idea
• Quality appears generally high and competitive
  with international standards
• Some risk from hype associated with both Grid
  and e-Science leading to unclear technical and
  political situation
• Both words do not have general agreement
• Suggest defining some safe metrics of success
• Education Grids are an untapped opportunity

3
Why discuss Grid Software?

• There is intellectual opportunity in several
  areas where there are application requirements
  and interesting computer science research issues
• Workflow, federation
• There is a need to deploy ROBUST Grids across the
  UK (World) to enable cross-country or global
  e-Science
• Thats where we can ask about level of effort
  needed in current activities and if UK (Europe,
  USA) should enhance existing projects or start
  new synergistic activities
• I happen to think there are omissions in current
  projects which are nontrivial to retrofit -- for
  example
• Need for good messaging infrastructure
• Rich Security model supporting different practice
• Peer-to-peer capabilities

4
Further Suggestions

• Grid architecture could be addressed in a broader
  fashion than just the OSGA (core) level
• Add coordination in areas like portals (e.g. use
  portlets and develop application web services,
  and participate in GCE Research Group)
• Although there are separate programming and
  Grid Computing Environment groups in GGF, the
  focus of both can be considered as programming
  the Grid and hence obviously very important
• Workflow is just the programming of Web
  Services and likely there will be a few distinct
  approaches not just XML-based BPEL4WS from
  industry
• Benchmarks and an international testbed place
  to do testing of functionality and performance
  seem to be essential

5
Grids and e-Science

• There could be more discussion of what is Grid
  Software, what is Grid Software Development
  Model What do we really need as well as what
  are Grid interface standards
• What will Globus, IBM, Avaki, Sun, Platform,
  Fujitsu, HP, Oracle .. really deliver, will they
  interoperate and when will it happen?
• Is it true and does it matter that Complete
  Globus 3 is an order of magnitude more complex
  than Globus 2?
• IBM aimed at Enterprise and too complicated for
  most real people?
• Should one have N Complete Grid systems or an
  architecture allowing one to compose your Grid
  with components from N different providers?
• One must expect different parts of e-Science
  community to select different Grid providers

Legion was always object-based (nearer Web
Services than Globus) but Globus wasmore
effective in practice. Maybe robust Avaki Grid
is just what you want? Could you switch from
Globus to Avaki or more realistically
interoperate betweenAvaki and Globus? What
happens if some organization (e.g. industry)
demands better security How do you provide?
6
P2P Networks and Grids

• Grids manage and share asynchronous resources in
  a rather centralized fashion
• Peer-to-peer networks are just like Grids with
  different implementations of services like
  registration and look-up
• P2P Networks like Grids can be built from Web
  Services
• As typical network delay (gt100ms) large, one can
  buid very sophisticated logic into messaging
  environment
• Publish-subscribe, adaptive routing, QoS,
  Security .
• P2P Grids naturally federate and support concept
  of Personal/Private Grid proposed by US LHC
  experimentalists

7
Peer to Peer Grid
Peers
Service FacingWeb Service Interfaces
Peers
User FacingWeb Service Interfaces
Peer to Peer Grid of Resources supported by
internal MessageGrid of message or event
brokers Events are just Time-stamped messages
8
Users
Portal such as Jetspeed
Hosting Environment
Hosting Environment
GridComputing or ProgrammingEnvironments
Generic Application Services
Web Services
CoreGrid
Core System Services
Resource Grid Services
e.g. DAI compliantdatabase
Resources
9
Federation/Interoperability Problem?

• Have a collection of Web Services running in
  Grids defined by different suppliers?
• Interoperability particular application Web
  Service of supplier X can utilize core service
  of supplier Y
• Federation core service of supplier X can be
  integrated with core service of supplier Y to
  provide a integration/amalgam that is also a
  realization of core service

10
OGSI Interface Implications

• OGSI defines the features that all Web Services
  must have to be called Grid Services
• Using Web Services is equivalent roughly (in
  old-style computing lingo) to agreeing on a
  common (object-oriented) language
• It is equivalent to choosing C or Java or more
  precisely agreeing to use CORBA IDL or Java to
  define external interfaces (method calls) without
  prejudice as to implementation language
• For example, having a Grid Information Port is
  VERY roughly analogous to agreeing that all
  programs have a standard input and a standard
  output familiar from UNIX
• Agreeing that Grid Information Ports should
  communicate XML records (SDE Service Data
  Element) is roughly equivalent to saying standard
  output will use Fortan namelist syntax
• So this is not yet a complete interoperability or
  federation specification

11
The Grid Onion

• Like most systems, Grids can be thought of
  hierarchically with a set of core services
  surrounded by other (Web) services with
  increasing particularity
• The outer levels are likely to be easiest to
  agree on with lasting specifications (XML
  metadata and Web Service specifications)
• Portals are in my opinionan area where
  investmentwill have lasting impact asat edge of
  architecture
• Wise UK investment in OGSA-DAI similarly lasting
• A lot of meta-data specificationactivities can
  proceed irrespectiveof inner uncertainity

12
Possible Project InterGrids

• New open-source open-process effort developing
  Grid software including both the core and
  outer-levels of the onion
• Key underlying principles
• Support small Grids (university departments,
  Private Grids) with easy deployment (e.g. allow
  EJB, Oracle etc. but do not require)
• Support composability (P2P) and federation with
  itself and other Grids
• Mitigate risks in e-Science
• Allow innovative computer science research
• Use existing (Avaki, Globus SRB ICENI ..) bits
  and pieces if possible encourage such projects
  to produce modules useable in other Grid systems
  like InterGrids
• Good test/benchmark/tutorial material and good
  support

13
Time Scale and Plan for InterGrids I

• InterGrids has three distinct components that are
  synergistic and have some natural overlap
• FederatedGrids enabling federation of multiple
  OGSA or FGSA compatible Grids
• GridLite enabling light weight and personal Grids
  that can federate
• GridTestbed International compliance,
  Performance and 24by7 robustness test environment
• Now Set up requirements/planning team Solicit
  interest in project gather comments broadly
• Report March 2003 allowing call for participants
• Around April/June 2003 Initialize project and
  define initial activities, InterGrids
  architecture and software-stack Identify
  external software likely to be used

14
Time Scale and Plan for InterGrids II

• May 2003 Define initial FGSA Federatable Grid
  Service Architecture characteristics of Grids
  that can be federated in InterGrids
• (Obviously differences between OGSA and FGSA
  should be minimal and other GGF standards like
  GridFTP and OGSA-DAIS must be supported in FGSA)
• Around August 2003 Start software development
  and development of test applications/environment
• Around October 2003 Set-up testing environment
  for InterGrids and other Grid Systems
• Around mid-2004 Initial (tested) deliverables
• Continually go to GGF, test and gather
  requirements
• At or Before mid-2006, deliver full tested
  software with documentation and defined
  interfaces
• Include full portal and application Service
  generation kit.