The Grid and the Future of Business

1
The Gridand the Future of Business

• Ian Foster
• Mathematics and Computer Science Division
• Argonne National Laboratory
• and
• Department of Computer Science
• The University of Chicago
• http//www.mcs.anl.gov/foster

2
Grid Computing
3
Abstract

• By enabling the integration of services and
  resources within and across enterprises, the Grid
  promises to enable new ways of doing business
  powered by on-demand access to computing,
  seamless access to corporate data, and dynamic
  outsourcing to service providers. In this
  visionary keynote, the founder of Grid computing
  reviews the origins of Grid computing in
  e-science, discusses its likely impact in
  e-business and the commercial marketplace, and
  describes progress towards the integration of
  Grid technologies and Web services within the
  Open Grid Services Architecture.

4
Technical Universe, Circa 1890

• Ubiquitous communication infrastructure
• The telegraph
• Local power generation
• Electric power generators serve at most city
  blocks

5
Technical Universe, Circa 2000

• Ubiquitous comms infrastructure
• Internet, email, Web
• Ubiquitous power distribution
• (Reliable?) standard access
• Tremendous variety of devices
• Local computing
• Most computing and storage on internal
  enterprise computers

6
Exponentials (and Coefficients)

• Network vs. computer performance
• Computer speed doubles every 18 months
• Network speed doubles every 9 months
• Difference order of magnitude per 5 years
• 1986 to 2000
• Computers x 500
• Networks x 340,000
• 2001 to 2010
• Computers x 60
• Networks x 4000

Scientific American (Jan-2001)
7
Therefore A Computing Grid

• On-demand, ubiquitous access to computing, data,
  and services
• New capabilities constructed dynamically and
  transparently from distributed services

When the network is as fast as the computer's
internal links, the machine disintegrates across
the net into a set of special purpose
appliances (George Gilder)
8
My Presentation

• The emergence of the Grid concept
• Origins in eScience, and the Globus Toolkit
• Grids and e-business
• Opportunities requirements
• Technology convergence
• Open Grid Services Architecture
• Summary

9
My Presentation

• The emergence of the Grid concept
• Origins in eScience, and the Globus Toolkit
• Grids and e-business
• Opportunities requirements
• Technology convergence
• Open Grid Services Architecture
• Summary

10
E-Science The Original Grid Driver

• Pre-electronic science
• Theorize /or experiment, in small teams
• Post-electronic science
• Construct and mine very large databases
• Develop computer simulations analyses
• Access specialized devices remotely
• Exchange information within distributed
  multidisciplinary teams
• Need to manage dynamic, distributed
  infrastructures, services, and applications

11
And Thus The Grid

• Resource sharing coordinated problem solving
  in dynamic, multi-institutional virtual
  organizations

12
The GridA Brief History

• Early 90s
• Gigabit testbeds, metacomputing
• Mid to late 90s
• Early experiments (e.g., I-WAY), academic
  software projects (e.g., Globus), applications
• 2002
• Dozens of application communities projects
• Significant technology base (Globus ToolkitTM)
• Global Grid Forum 500 people, 20 countries

13
Sloan Digital Sky Survey Analysis
14
Sloan Digital Sky Survey Analysis
Size distribution of galaxy clusters?
15
A Large Virtual Organization CERNs Large Hadron
Collider

• 1800 Physicists, 150 Institutes, 32 Countries
• 100 PB of data by 2010 50,000 CPUs?

16
Data Grids for High Energy Physics
17
Grids at NASA Aviation Safety
Wing Models

• Lift Capabilities
• Drag Capabilities
• Responsiveness

Stabilizer Models
Airframe Models

• Deflection capabilities
• Responsiveness

Crew Capabilities - accuracy - perception -
stamina - re-action times - SOPs
Engine Models

• Braking performance
• Steering capabilities
• Traction
• Dampening capabilities

• Thrust performance
• Reverse Thrust performance
• Responsiveness
• Fuel Consumption

Landing Gear Models
18
Life Sciences Telemicroscopy
DATA ACQUISITION
PROCESSING,ANALYSIS
ADVANCEDVISUALIZATION
NETWORK
COMPUTATIONALRESOURCES
IMAGING INSTRUMENTS
LARGE DATABASES
19
Underlying Technical Requirements

• Dynamic formation and management of virtual
  organizations
• Online negotiation of access to services who,
  what, why, when, how
• Configuration of applications and systems able to
  deliver multiple qualities of service
• Autonomic management of distributed
  infrastructures, services, and applications

20
State of the ArtGlobus ToolkitTM (since 1996)

• Small, standards-based set of protocols
• Authentication, delegation resource discovery
  reliable invocation etc.
• Information-centric design
• Data models publication, discovery protocols
• Open source implementation community
• With commercial support
• Enabler of services and applications

21
Grid Projects in eScience
22
My Presentation

• The emergence of the Grid concept
• Origins in eScience, and the Globus Toolkit
• Grids and e-business
• Opportunities requirements
• Technology convergence
• Open Grid Services Architecture
• Summary

23
And What About Business?

• Fragmentation of enterprise infrastructure
• Specialized platforms -gt commodity servers
• Intelligence embedded in networks
• The rise of the eUtility (IBM, HP, )
• Outsourcing, economies of scale
• Business-to-business computing
• Especially complex virtual organizations
• Ever more challenging QoS requirements
• Green-screen -gt ubiquitious web presence

24
Todays EnterpriseComputing Environment
25
The Business Opportunity

• On-demand computing, storage, services
• Significant savings due to reduced build-out,
  economies of scale, reduced admin costs
• Greater flexibility gt greater productivity
• Entirely new applications and services
• Based on high-speed resource integration
• Solution to enterprise computing crisis
• Render distributed infrastructures manageable

26
Grids and Industry Early Examples
Entropia Distributed computing (BMS, Novartis, )
27
Realizing the PromiseRequires Significant
Innovation

• Automation of infrastructure operation to achieve
  economies of scale
• Management and component models for distributed
  service provisioning
• New applications and tools powered by distributed
  services and resources
• Business and service models to support
  specialization of function

28
My Presentation

• The emergence of the Grid concept
• Origins in eScience, and the Globus Toolkit
• Grids and e-business
• Opportunities requirements
• Technology convergence
• Open Grid Services Architecture
• Summary

29
Grid EvolutionOpen Grid Services Architecture

• Refactor Globus protocol suite to enable common
  base and expose key capabilities
• Service orientation to virtualize resources and
  unify resources/services/information
• Embrace key Web services technologies standard
  IDL, leverage commercial efforts
• Result standard interfaces behaviors for
  distributed system management

30
Transient Service Instances

• Web services address discovery invocation of
  persistent services
• Interface to persistent state of entire
  enterprise
• In Grids, must also support transient service
  instances, created/destroyed dynamically
• Interfaces to the states of distributed
  activities
• E.g. workflow, video conf., dist. data analysis
• Significant implications for how services are
  managed, named, discovered, and used

31
Open Grid Services Architecture

• Defines fundamental (WSDL) interfaces and
  behaviors that define a Grid Service
• Required optional interfaces WS profile
• Defines WSDL extensibility elements
• E.g., serviceType (a group of portTypes)
• Open source Globus Toolkit 3.0
• Leverage GT experience, code, community
• And also commercial implementations

32
The Grid Service Interfaces/Behaviors Service
Data
Service data element
Service data element
Service data element
Binding properties - Reliable invocation -
Authentication
Implementation
Hosting environment/runtime (C, J2EE, .NET, )
33
Example Database Service

• DBaccess Grid service supports at least two
  portTypes
• GridService
• DBaccess
• Each has service data
• GridService basic introspection, lifetime,
• DBaccess database type, current load, ,
• Maybe other portTypes as well
• E.g., NotificationSource (SDE subscribers)

Grid Service
DBaccess
Name, lifetime, etc.
DB info
34
Data Mining for Bioinformatics
Community Registry
Mining Factory
Database Service
BioDB 1
Compute Service Provider
User Application
. . .
. . .
I want to create a personal database containing
data on e.coli metabolism
Database Service
Database Factory
BioDB n
Storage Service Provider
35
Data Mining for Bioinformatics
Find me a data mining service, and somewhere to
store data
Community Registry
Mining Factory
Database Service
BioDB 1
Compute Service Provider
User Application
. . .
. . .
Database Service
Database Factory
BioDB n
Storage Service Provider
36
Data Mining for Bioinformatics
Community Registry
Mining Factory
Database Service
Handles for Mining and Database factories
BioDB 1
Compute Service Provider
User Application
. . .
. . .
Database Service
Database Factory
BioDB n
Storage Service Provider
37
Data Mining for Bioinformatics
Community Registry
Mining Factory
Database Service
Create a data mining service with initial
lifetime 10
BioDB 1
Compute Service Provider
User Application
. . .
. . .
Create a database with initial lifetime 1000
Database Service
Database Factory
BioDB n
Storage Service Provider
38
Data Mining for Bioinformatics
Community Registry
Mining Factory
Database Service
Create a data mining service with initial
lifetime 10
BioDB 1
Miner
Compute Service Provider
User Application
. . .
. . .
Create a database with initial lifetime 1000
Database Service
Database Factory
BioDB n
Database
Storage Service Provider
39
Data Mining for Bioinformatics
Community Registry
Mining Factory
Database Service
Query
BioDB 1
Miner
Compute Service Provider
User Application
. . .
. . .
Query
Database Service
Database Factory
BioDB n
Database
Storage Service Provider
40
Data Mining for Bioinformatics
Community Registry
Mining Factory
Database Service
Query
BioDB 1
Miner
Keepalive
Compute Service Provider
User Application
. . .
. . .
Query
Database Service
Database Factory
Keepalive
BioDB n
Database
Storage Service Provider
41
Data Mining for Bioinformatics
Community Registry
Mining Factory
Database Service
BioDB 1
Miner
Keepalive
Compute Service Provider
User Application
. . .
. . .
Results
Database Service
Database Factory
Keepalive
Results
BioDB n
Database
Storage Service Provider
42
Data Mining for Bioinformatics
Community Registry
Mining Factory
Database Service
BioDB 1
Miner
Compute Service Provider
User Application
. . .
. . .
Database Service
Database Factory
Keepalive
BioDB n
Database
Storage Service Provider
43
Data Mining for Bioinformatics
Community Registry
Mining Factory
Database Service
BioDB 1
Compute Service Provider
User Application
. . .
. . .
Database Service
Database Factory
Keepalive
BioDB n
Database
Storage Service Provider
44
GT3 OGSA-Based Globus Toolkit

• GT3 Core
• Grid service interfaces
• Reference impln of evolving standard
• Multiple hosting envsJava/J2EE, C, C/.NET?
• GT3 Base Services
• Globus capabilities
• Many other services

Other Grid
GT3
Services
Data
Services
GT3 Base Services
GT3 Core
45
OGSA Current Status

• Grid service specification other documents
  moving forward in GGF
• www.gridforum.org/ogsi-wg
• Globus Project on track for open source
  OGSA-based GT3 release end of 2002
• www.globus.org/ogsa
• IBM committed to various OGSA-compliant software
  releases (e.g., WebSphere)
• Other industrial efforts underway

46
My Presentation

• The emergence of the Grid concept
• Origins in eScience, and the Globus Toolkit
• Grids and e-business
• Opportunities requirements
• Technology convergence
• Open Grid Services Architecture
• Summary

47
Summary The Grid

• Resource sharing coordinated problem solving in
  dynamic, multi-institutional virtual
  organizations
• On-demand, ubiquitous access to computing, data,
  and services
• New capabilities constructed dynamically and
  transparently from distributed services
• Evolved to be dominant eScience, now
  transitioning to industry (think Web in 1994)

48
Open Grid Services Architecture

• Open Grid Services Architecture represents next
  step in Grid evolution
• Service orientation enables unified treatment of
  resources, data, and services
• Standard interfaces and behaviors (the Grid
  service) for managing distributed state
• Open source Globus Toolkit implementation (and
  numerous commercial value adds)

49
For More Information

• Grid Book
• www.mkp.com/grids
• Survey articles
• www.mcs.anl.gov/foster
• The Globus Project
• www.globus.org
• Global Grid Forum
• www.gridforum.org
• Edinburgh, July 22-24
• Chicago, Oct 15-17