The%20Grid%20Needs%20You.%20Enlist%20Now!

1
The Grid Needs You. Enlist Now!

• Professor Carole Goble
• University of Manchester, UK, carole_at_cs.man.ac.uk
• Co-director e-Science North West UK regional
  centre
• Director myGrid UK e-Science pilot project
• Co-chair Global Grid Forum Semantic Grid Research
  Group

2
The Grid Needs You. Enlist Now!

• The what and why of the Grid.
• Services, data and semantics and the Grid.
• Getting involved a call to arms.

3
The take home

• The Grid is the next big thing and it isnt
  just big computers and fat pipes.
• The Grid is actually the latest attempt at
  distributed computing
• If you arent involved yet maybe its because you
  dont think its relevant, or its done already or
  you havent anything to offer
• You are most likely wrong
• If you are already into the Grid this is a ra
  ra exercise ?

4
Origins of the Grid

• The Grid Blueprint for a New Computing
  Infrastructure
• Edited by Ian Foster and Carl Kesselman
• July 1998, 701 pages.
• a proposed distributed computing infrastructure
  for advanced science and engineering
• pervasive and dependable

5
What is the Grid?

• Computational power as a utility
• Securely and transparently sharing supercomputing
  resources on demand.
• Fast pig iron with fat pipes for cycle intensive
  scientific problems
• Large scale data access and transportation
• Making the most of what you have got

6
Why do it now?

• Enormous quantities of data Petabytes
• For an increasing number of communities, gating
  step is not collection but analysis
• Ubiquitous Internet 100 million hosts
• Collaboration resource sharing the norm
• Ultra-high-speed networks 10 Gb/s
• Global optical networks
• Huge quantities of computing 100 Top/s
• Moores law gives us all supercomputers

7
Isnt this just high performance computing for
high energy physicists?
8
What is the Grid for?

• Global e-Science
• Large-scale science and engineering are done
  through the interaction of people, heterogeneous
  computing resources, information systems, and
  instruments, all of which are geographically and
  organizationally dispersed.
• The motivation for Grids is to facilitate the
  routine interactions of these resources in order
  to support large-scale science and engineering.
• KEYWORDS
• Collaboration, Democratization, Speculation

Bill Johnston, NASA July 01
9
Global Collaborative Knowledge Communities
Slide courtesy of Ian Foster
10
Global Knowledge Communities

• Teams organised around common goals
• Communities Virtual organisations
• Overlapping memberships, resources and activities
• Essential diversity is a strength challenge
• membership capabilities
• Geographic and political distribution
• No location/organisation/country possesses all
  required skills and resources
• Dynamic adapt as a function of their situation
• Adjust membership, reallocate responsibilities,
  renegotiate resources

Slide derived from Ian Fosters SSDBM 03 keynote
11
The Grid Opportunity

• flexible, secure, coordinated resource sharing
  among dynamic collections of individuals,
  institutions, and resources - what we refer to as
  virtual organizations."

KEYWORD VIRTUALISATION
The Anatomy of the Grid Enabling Scalable
Virtual Organizations Foster, Kesselman, Tueke
12
Why Grids?

• A biochemist exploits 10,000 computers to screen
  100,000 compounds in an hour
• A biologist combines a range of diverse and
  distributed resources (databases, tools,
  instruments) to answer complex questions
• 1,000 physicists worldwide pool resources for
  petaop analyses of petabytes of data
• Civil engineers collaborate to design, execute,
  analyze shake table experiments
• Climate scientists visualize, annotate, analyze
  terabyte simulation datasets
• An emergency response team couples real time
  data, weather model, population data
• A multidisciplinary analysis in aerospace couples
  code and data in four companies

Slide courtesy of Steve Tuecke
13
Telemicroscopy

• Sharing of UHVEM(Ultra High Voltage Electron
  Microscopy) in Osaka University with NCMIR
  (National Center for Microscopy and Imaging
  Research)
• 3 Million electron volts the most powerful
  microscopy facility
• KEYWORDS SHARING SCARCE RESOURCES ON DEMAND

14
Smallpox Grid

• United Devices, IBM, Oxford University, Accelrys
• Analysis of 35 million drug compounds against
  nine smallpox proteins to try to find a way to
  stop the replication of the virus.
• Volunteers from over 190 countries donated their
  spare CPU power at www.grid.org, the world's
  largest public computing resource
• Contributed over 39,000 years of computing time
  in less than six months.

September 30, 2003 delivered the results of the
Smallpox Research Grid project to representatives
from the United States Department of Defense in
an event hosted by the British Embassy in
Washington, D.C.
15
Digital
Digital
2,000,000 - Screened every Year 120,000 -
Recalled for Assessment 10,000 - Cancers 1,250 -
Lives Saved
230 - Radiologists (Double Reading) 50 -
Workload Increase
16
http//www.nbirn.net/
17
RealityGrid http//www.realitygrid.org
Scientist remotely steers calculation from
laptop Visualization and computation use
supercomputers accessed via Grid.
X-ray microtomography produces 3D X-ray
attenuation maps of specimens at a microscopic
level

• Closely coupling computation and experiment
  to speed up scientific discovery.
• Simulation, visualization and data gathering
  coupled

18
Collaboration

• Interactive environments and virtual presence
  integrated with Grid middleware

• SARS Combat Grid, Taiwan
• Emergency Access Grids
• Integration of patient data and models of
  dissemination

http//www.accessgrid.org
19
Access Grid
20
Foundation for e-Science
sensor nets
Diagram derived from Ian Fosters slide
21
Butterfly.net

• Fully-distributed server technology pioneering
  the use of open grid computing protocols in
  large-scale immersive game networks that support
  unlimited numbers of players and require the most
  demanding levels of service.

22
More commercial examples

• Novartis Pharmaceuticals accelerate lead
  identification and profiling to increase relevant
  targets in drug discovery, screening applications
  that were previously considered CPU constrained.
• Nippon Life Insurance improve the performance of
  Financial Risk Management Applications customer
  project in applying Grid technology for this
  application. Reduced processing time for
  financial risk calculation from around 10 hours
  to about 49 minutes a 12-fold increase in
  speed. Can run more complex scenarios to reduce
  risk exposure

23
Global Grid Forumhttp//www.ggf.org

• Standards body for Grid Computing
• Over 2000 members
• All the vendors
• 44 WGs and RGs
• Three meetings per annum
• 1000 attendees at plenary meetings
• 400 at working meetings
• GGF10 Frankfurt, March 2004

24
Investment

• UK Government invested 240 million into
  e-Science and Grid related research
• EU invested 351million in FP5 and FP6
• USA invested lots!
• IBM invested 10-20 RD budget in Grid Computing
• 1.5million per annum on GridFTP alone
• Japan and China invested in Grids
• Practically every EU member has a Grid programme.

25
The Grid means what I say it means

• The Grid the vision of forming federations
• A Grid - A virtual organisation of resources
• Machines computational grid
• Geography a UK Grid
• A field Mouse Genome Grid
• A (temporary) problem protein folding
  simulation
• No one grid lots of interoperating Grids
• Grid middleware infrastructure specification
• Services stacks, policies, protocols, standards,
  APIs
• Reference implementations
• Globus, Condor, Unicore, Sun Grid Engine, Avaki,
  United Devices...
• Grid tools
• Portals, heartbeat monitors etc
• E-Science application of all the above for the
  benefit of Science

26
The Grid is forming federations

• Infrastructure middleware for establishing,
  managing, and evolving multi-organizational
  federations
• Dynamic, autonomous, domain independent
• On-demand, ubiquitous access to computing, data,
  and services
• Mechanisms for resource virtualization workflow
  management within federations
• New capabilities constructed dynamically and
  transparently from distributed services
• Service-oriented, virtualization

27
when the federations are

• Dynamic and volatile. A consortium of services
  (databases, sensors, compute servers)
  participating in a complex analysis may be
  switched in and out they become available or
  cease to be available
• Ad-hoc. Service consortia have no central
  location, no central control, and no existing
  trust relationships
• Large Hundreds of services could be orchestrated
  at any time
• Potentially long-lived. A simulation could take
  weeks.
• HOLD THESE THOUGHTS!

28
Grid Computing characteristics

• Implement One from Many
• Virtualization at every layer of the computing
  stack
• Provisioning of work and resources based on
  policies and dynamic requirements
• Pooling of resources to increase utilization
• Manage Many as One
• Self-adaptive software that largely tunes and
  fixes itself
• Unified management and provisioning

29
which gives some challenges!

• Dynamic formation and management of virtual
  organizations
• Online negotiation of access to services who,
  what, why, when, how
• Configuration of applications and systems able to
  deliver multiple qualities of service
• Autonomic management of distributed
  infrastructures, services, and applications
• Management of distributed state as a fundamental
  issue

30
myGrid http//www.mygrid.org.uk

• Knowledge-driven middleware for data
  intensive ad hoc in silico experiments in biology

• Straightforward discovery, interoperation,
  deployment sharing of services
• Service-oriented architecture
• Semantic based discovery of workflows and
  workflow composition
• Integration and Information
• Workflow Distributed DB Queries
• Experimentation
• Provenance, propagating change, personalisation

31
Three legacy views

• Grid middleware is a bag of low level protocols
• The Grid is about compute cycle stealing
• The Grid is about plumbing and has nothing to do
  with semantics

32
Three legacy views

• Grid middleware is a bag of low level protocols
• The Grid is about compute cycle stealing
• The Grid is about plumbing and has nothing to do
  with semantics
• This was once true. Some still hold this view
  (notably US programme managers)
• It is not the view of the Grid visionaries or the
  Grid policy makers outside the US.

33
Three legacy views

• Grid middleware is a bag of low level protocols
• The Grid is about compute cycle stealing
• The Grid is about plumbing and has nothing to do
  with semantics
• This was once true. Some still hold this view
  (notably US programme managers)
• It is not the view of the Grid visionaries or the
  Grid policy makers outside the US.

34
Grid Evolution1st generation

• Computationally intensive
• File access/transfer
• Bag of various heterogeneous protocols
  toolkits
• Monolithic design
• Recognises internet, ignores Web
• Academic teams

Increased functionality, standardization
Legion, Condor, Unicore
Custom solutions
Time
(based on Foster GGF7 Plenary)
35
Grid Evolution2nd Generation

• Data intensive -gt knowledge intensive
• Open services-based architecture
• Recognises Web services
• Global Grid Forum
• Industry participation

App-specific Services
Increased functionality, standardization
Custom solutions
Time
(based on Foster GGF7 Plenary)
36
Open Grid Services Architectureongoing since
early 2002
Specific services drug discovery pipeline
Standard services agreement, data access and
integration, workflow, security, policy
Standard interfaces and behaviours for
distributed systems naming, service state,
lifetime management, notification
Standard mechanisms for describing and invoking
services WSDL, SOAP, WS-Security etc
(Graphic courtesy of Savas Parastatidis )
37
OGSI Standard Web Services Interfaces
Behaviours

• Naming and bindings (basis for virtualization)
• Every service instance has a unique name (Grid
  Service Handle) from which can discover supported
  bindings which are volatile (Grid Service
  Reference)
• Two tiered naming scheme to cope with service
  migration and failover
• Lifecycle (basis for fault resilient state
  management)
• Service instances created by factories
• Destroyed explicitly or via soft state
• Information model (basis for monitoring
  discovery)
• Service data (attributes) associated with GS
  instances (SDEs)
• Operations for querying (introspecting) and
  setting this info
• Asynchronous notification of changes to service
  data
• Service Groups (basis for registries collective
  services)
• Group membership rules membership management
• Base Fault type

All sound kind of familiar?
38
OGSI

• Lifetime management
• Explicit destruction
• Soft-state lifetime

Data access
Implementation
Hosting environment/runtime (C, J2EE, .NET, )
(Slide courtesy of Ian Foster)
39
OGSI
Authentication authorization are applied to all
requests
Service factory
Service requestor (e.g. user application)
Service registry
Interactions standardized using WSDL
(Slide courtesy of Ian Foster)
40
OGSI and Handle Resolution

• Grid Service Handle (GSH)
• Permanent network pointer to a Grid service
• URI scheme indicates resolution mechanism
• Grid Service Reference (GSR)
• Network endpoint info to access the service
• Binding-specific (for SOAP, GSRWSDL doc)
• HandleResolverfindByHandle
• Service portType to resolve GSH gt GSR
• Service Locator structure
• Includes service GSHs, GSRs and portTypes
• Factory/Find communicate Locators

(Slide courtesy of Ian Foster)
41
GSHgtGSR Resolution

• Transparent service migration
• Move service state to different hosting
  environment
• Update GSR with new network endpoint info
• Update GSHGSR binding in HandleResolver
• After access error or GSR-expiration, new GSR
  obtained through GSH lookup
• (Re-)Activation of dormant service
• Transparent fail-over
• Load balancing
• Mobile services
• Files, database result sets, data fragments,
  agreements, etc.

(Slide courtesy of Ian Foster)
42
Service Migration
HandleResolver
Requester
GSH
GSR
GSH
GSR
ltwsdlgt
hdl1.2/abc
...
...
Service Locator
hdl1.2/abc
ltwsdlgt
...
...
Service
Hosting Environment B
Hosting Environment A
(Slide courtesy of Ian Foster)
43
Sound familiar?
Web Services Loose coupled, stateless, persistent

• Layering a component-based distributed object
  model over a web service framework
• Early OGSI implementations
• Globus Toolkit 3
• OGSI.NET
• OGSILite
• Unicore

Grid Services Robust naming, stateful, lifetime
management
CORBA Tightly coupled, naming, stateful,
lifetime management
44
OGSI Status and Issues
https//forge.gridforum.org/projects/ogsi-wg

• OGSI version 1.0 in GGF proposed recommendation
• Issue compliance to Web Service Standards
• GWSDL changes WSDL 1.1 by extending portType
  syntax to define a Service Data Element.
• Why not use WS standards for state management
  idioms e.g. WS-Context/Coordination?
• By eliminating a new mandatory infrastructure
  (OGSI), can use conventional tooling.
• But it needs to meet the requirements of Grid

(Graphic courtesy of Savas Parastatidis)
45
300 pound gorillas

• If you want to use standards then you have to use
  them or work with them
• W3C and OASIS are big gorillas
• E.g. GSH/GSR, Handle.net, Life Science Identifer
  and WS-address

46
Open Grid Service Architecturewhere are the
OGSI services

• Technical specifications
• Open Grid Services Infrastructure is almost
  complete
• Security, data access, Java binding, common
  resource models, etc., etc., in the pipeline
• Implementations and compliant products
• OGSA-based Globus Toolkit v3, OGSALite,
  OGSA.NET,
• IBM, Avaki, Platform, Sun, NEC, Oracle,
• Rich set of service definitions implementations
• Starting on OGSI-compliant services
• OGSA Use Cases
• https//forge.gridforum.org/projects/ogsa-wg
• OGSA-Data Access and Integration

47
Grid Applications On The MoveThe rise of the
Information Grid
Large scale data Large number of
machines Computationally intensive Simple
semantics Small homogeneous communities
Smaller scale data Data intensive Complex
heterogeneous applications Complex
semantics Large diverse communities
High Energy Physics
Functional Genomics Oceanography Biodiversity Eart
h Science Neuroscience
48
OGSA roadmap
(Slide courtesy of Hiro Kishimoto)
Use cases
Commercial Data Center
Analyze Extract
Requirements (Functions)
OGSA-WG
Data Sharing
Evaluate Prioritize
Mechanism (Services)
Data Management
Dispatch
OGSA-DAI
interface
Existing or new WGs
DAIS-WG
49
Data-intensive integrationwhat the e-scientist
REALLY wants

• Scientists do data integration
• Actually they do application and model
  integration too!
• Cooperative information systems
• Workflows
• Data virtualisation

50
Integrating Across Biological Systems
51
From WIT to Gwiz to Systems Biology(N. Maltsev
et al., Argonne)

Delivarables

Data sources

Whole genome

ORF identification

Genome Features

Un
-
annotated genomes

Analysis

Annotated Genome Maps

Genome Annotations

Genomes Comparisons

from public databases

Visualization

Experimental Results

Sequence Analysis
results

Delivarables

Gene Functions predictions

Domain analys
is

Sequence Analysis

Motif analysis


Evolutionary sequence
analysis

Gene Functions Predictions

Data sources

Assignments

Metabolic data from
public databases (EMP,
KEGG, EcoCyc, Brenda,
Gene Networks
Delivarables

etc)

Predictions of Regulation

Analysis

Regulatory data from
Predictions of New pathways

(Regulatory and
public databases
Functions of Hypotheticals

Metabolic)

(RegulonDB, Sentra, etc)

Networks Comparisons

Evolutionary Analysis


Experimental Results

Networks Analysis results

Gene Networks Reconstructions

Metabolic Flux Analysis

(Annotated s
toichiometric Matrices)

Data sources

Delivarables

Metabolic

Prediction of Dynamic
Enzymatic and enzyme
Behavior

kinetic data from EMP

Predictions of Phenotypes

Simulation

Experimental Results

Predictions of Gene
Networks Analysis results

Networks Architecture

Phenotypes Predictions

Levels ofRepresentation
Metabolic


Engineering
52
Types of Information
ID MURA_BACSU STANDARD PRT 429
AA. DE PROBABLE UDP-N-ACETYLGLUCOSAMINE
1-CARBOXYVINYLTRANSFERASE DE (EC 2.5.1.7)
(ENOYLPYRUVATE TRANSFERASE) (UDP-N-ACETYLGLUCOSAMI
NE DE ENOLPYRUVYL TRANSFERASE) (EPT). GN MURA
OR MURZ. OS BACILLUS SUBTILIS. OC BACTERIA
FIRMICUTES BACILLUS/CLOSTRIDIUM GROUP
BACILLACEAE OC BACILLUS. KW PEPTIDOGLYCAN
SYNTHESIS CELL WALL TRANSFERASE. FT ACT_SITE
116 116 BINDS PEP (BY SIMILARITY). FT
CONFLICT 374 374 S -gt A (IN REF.
3). SQ SEQUENCE 429 AA 46016 MW 02018C5C
CRC32 MEKLNIAGGD SLNGTVHISG AKNSAVALIP
ATILANSEVT IEGLPEISDI ETLRDLLKEI GGNVHFENGE
MVVDPTSMIS MPLPNGKVKK LRASYYLMGA MLGRFKQAVI
GLPGGCHLGP RPIDQHIKGF EALGAEVTNE QGAIYLRAER
LRGARIYLDV VSVGATINIM LAAVLAEGKT IIENAAKEPE
IIDVATLLTS MGAKIKGAGT NVIRIDGVKE LHGCKHTIIP
DRIEAGTFMI
53
Data on the Grid pre OGSA

• Chiefly files!
• LDAP as a query language
• No RDBMS access from Globus 1.1
• MDS and MCAT catalogs
• Honorable exception
• Storage Resource Broker
• Support data-intensive applications that
  manipulate very large data sets by building upon
  object-relational database technology and
  archival storage technology

54
OGSA-Data Access and IntegrationGGF OGSA-DAIS WG

• Data Grid applications benefit from many lower
  level services
• Data movement.
• Data Replication.
• Data Virtualisation
• Database access and integration.
• Work underway on designing, developing and
  standardising many core Grid Data Management
  services.
• Designing services in a dynamic and heterogeneous
  environment is non-trivial,
• Plenty to be done!!

Clever semantic integration stuff here
OGSA-DAI Distributed Query
OGSA-DAI Basic Services
Data Grid Infrastructure Location, Delivery,
Replication
Resource Grid Infrastructure OGSA
Database, Communication, OS Technology
55
Infrastructure Architecture

(Slide Courtesy Malcolm Atkinson, UK National
e-Science Centre
56
OGSA-DAIS, OGSA-DAIS, OGSA-DAIT

• Part of Globus Toolkit 3
• Data can be XML, RDBMS and ODBMS
• UK dominance

DB2
Oracle 10g
57
Data Access Integration Services
Slide Courtesy Malcolm
Atkinson, UK eScience Center

58
Any database challenges?

• Data Virtualisation
• Enable the user to view the output of a
  computation as an answer to a query.
• User defines the what rather than the how.
• Planners map query to an execution plan (eager,
  lazy and just in time).
• Workflow manager executes plan.
• Schedulers manage tasks.

• Performance
• Scalability
• Unpredictablility
• Meta-data-driven access
• From registries
• Federation
• DQP
• Workflows
• Dynamic provisioning for meeting quality of
  service

Terabytes of data to ship around Very long lived
workflows Services disappear under your feet!
59
Virtual Data Concept

• Capture and manage information about
  relationships among
• Data (of widely varying representations)
• Programs ( their execution needs)
• Computations ( execution environments)
• Apply this information to, e.g.
• Discovery Data and program discovery
• Workflow for organizing, locating, specifying,
  requesting data
• Explanation provenance
• Planning and scheduling

Search for WW decays of the Higgs Boson for
which only stable, final state particles are
recorded?
Workflow by Rick Cavanaugh and Dimitri Bourilkov,
University of Florida
60
Federation, Federation, Federation

• Data integration the derivation of new data
  from old, via coordinated computation(s)
• May be computationally demanding
• Science as Workflow
• Build workflows
• Share and reuse workflows
• Explain workflows
• Schedule workflows

Terabytes of data to ship around Very long lived
workflows Services disappear under your feet!
61
Grid intelligence semantics

• A gap between grid computing endeavours and the
  vision of Grid computing
• To support the full richness of the grid
  computing vision we need to explicitly assert
  explicitly use semantics (knowledge) throughout
  the Grid software stack
• The Grid has always had lots of semantics
  embedded in Schema and Directory services, and
  used by schedulers and brokers
• Globus MDS2 -gt Globus Information Service
• Condor ClassAds

62
Semantic Grid http//www.semanticgrid.org

• Semantic Web Services -gt Semantic Grid Services
• GGF SEM-GRD RG bringing semantic web technologies
  and techniques to the Grid
• Ontologies RDF

63
Grids are driven by metadata

• The semantics might be buried but they are there
  nonetheless!
• Grid Applications
• Operational know-how of the domain.
• a query or workflow the annotation of results,
  parameters, personal notes, provenance data
  describing sources and derivation paths of
  information, etc
• Knowledge about the domain its data and its
  processes

64
A Multi-Hierarchical Rock Classification
Ontology (GSC)
Slide courtesy of Bertram Ludascher
Genesis
Fabric
Composition
Texture
65
Grids are driven by metadatathe semantics might
be buried but they are there nonetheless!

• Grid infrastructure
• the classification of computational and data
  resources, performance metrics, job control
  schema integration, workflow descriptions,
  resource brokering, resource scheduling, service
  state, event notification topics, typing service
  inputs and outputs, provenance trails access
  rights to databases, personal profiles and
  security groupings charging infrastructure
• problem solving selection and intelligent
  portals

Managing and operating a Grid intelligently
requires the interpretation of knowledge about
the state and properties of Grid components, and
their configurations for solving problems
Knowledge permeates the Grid Data
elements Service descriptions (service data
elements) Protocols (e.g. policy, provisioning)
66
Semantics in myGrid http//www.mygrid.org.uk
Workflow construction
Semantic mark up of results and logs
Service discovery
Workflow discovery
67
Pegasus planning environment for LIGO Pulsar
search
Slide courtesy of Jim Blyth
68
Grid Interoperability ProjectInteroperable
Resource Broker
NJS
Resource Discovery Service
Diagram Of Broker Architecture
Delegates resource check
Broker
Other Brokers
Unicore Broker
Globus Broker
Delegates translation
Uses to drive MDS search
Lookup resources
Translator
Uses to Drive MDS Search
IDB
Filter
Ontology engine
Hierarchical Grid Search
Nodal Grid Search
Filter
Resource Discovery Service
Slide courtesy of John Brooke
69
Semantics for integration and scientific workflows

• Semantic registration of data sets
• How to employ semantic information in data
  discovery, workflow discovery, service discovery,
  data binding, query and workflow planning and
  execution
• Semantic matchmaking of grid resources to satisfy
  requirements of application components in
  workflows, and indeed substituting whole
  workflows
• Intelligent reasoners for grid computing
  (semantic matchmakers, planners, resource
  brokers, etc.) that exploit knowledge of
  scientific applications as well as grid
  resources
• Scientific workflow design and execution
• Scientific workflow lifecycle methodology
  (authoring, publishing, discovering,
  personalising, enacting, validating, modifying of
  workflows)

The list goes on.
70
Semantic Grid
Web Services
Grid services
Semantic Web Services
Grid
Semantic Web
Semantic Grid
Semantics for the Grid
Grid-ware Semantic Services
71
An attempt at a context picture
72
Reality Check!

• Official production request of the CMS
  collaboration of 1,200,000 Monte Carlo simulation
  data with Grid resources.
• We encountered many problems during the run, and
  fixed many of them, including integration issues
  arising from the integration of legacy CMS
  software tools with Grid tools, bottlenecks
  arising from operating system limitations, and
  bugs in both the grid middleware and application
  software.
• Every component of the software contributed to
  the overall "problem count" in some way.
  However, we found that with the current level of
  functionality, we were able to operate the US-CMS
  Grid with 1.0 FTE effort during quiescent times
  over and above normal system administration and
  up to 2.5 FTE during crises.

The Grid in Action Notes from the Front G.
Graham, R. Cavanaugh, P. Couvares, A. DeSmet, M.
Livny, 2003
73
Goal
B e n e f i t s
Effort
Slide courtesy of Miron Livny
74
Ok, whats the reality?

• The Grid is in the same state as the Web was 10
  years ago
• Few production grids and not many killer demos -
  something you couldnt have done before.
• Middleware hard to use and incomplete (and
  certainly not invisible!)
• OGSA in its infancy.
• Varying degrees of maturity, but people use it
  anyway!
• Deployment, research, development, applications
  and standardisation all happening together
• Danger of half-baked solutions, premature
  standardisation, a Grid Winter

The Invisible Grid? Come back in 10 years
Pioneering spirit! Its the Wild West!! Its all
very exciting and rather daunting
75
Are you involved in Grid?

• There is hardly a paper at OTM that isnt
  relevant.
• But participation in Grid is largely from the
  Grid Community
• When the database people came to town they rocked
  it!
• But there are not so many that take part, and
  its the vendors that dominate though there are
  many research problems to overcome.
• Reinvention, muddle, confusion ensues.
• Why arent you involved?

76
Why you should be involved in Grid

• Established communities can be hard work to get
  involved in the latest thing
• DCOM, CORBA, WSwe have seen it all before!
• So your history is valuable. And its not just
  rehashing your history either (crossing out
  agents and crayoning in grid aint gonna work!)
• An amazing, open and active community.
• With tons of real applications and users who
  really need this stuff.
• GridPP had better work!!
• Some substantial industry and government backing.

77
Between community travellers Pioneers on tour!

• SSDBM2003
• ISWC2002
• WWW2002
• VLDB2003
• OTM2003
• AIMA2003

The Web
The Semantic Web
The Grid
WWW2002 Waikiki, Hawaii
78
Grid Middleware On The Move
Data and Information Grids
Semantic Grids
Second Generation Grid Computing
Open Service Architecture
79
The Grid Needs You! Enlist Now!

• http//www.ggf.org

The Grid
Now with added services architecture, data
management and semantics!!