Grids: How do we define success

1
Grids How do we define success?

• Vicky White, Fermilab
• April 27, 2004
• ISGC-2005 , Taiwan

2
Views of a Straddler

• Verb to straddle   
• To stand or sit with a leg on each side of
  bestride straddle a horse.
• To be on both sides of extend over or across a
  car straddling the centerline. .

3
Hopefully not a Straggler

• Strag"gler1. One who straggles, or departs from
  the direct or proper course, or from the company
  to which he belongs one who rambles without any
  settled direction.

4
What is a Grid?

• So many definitions e.g.
• Coordinated resource sharing and problem solving
  in dynamic, multi-institutional virtual
  organizations. Ian Foster
• So many analogies but most are not very useful
• Power Grid
• The next big thing after the WWW ?
• Seamless virtual environment
• So many technologies and types of Grid
• Globus almost a standard but not universal
• Data Grid, Computational Grid, Campus Grid,
  National Grid, Climate Grid, and so on.
• So much hype and such high expectations !
• Confusion with
• Public Resource Computing e.g. SETI_at_home
• Plain old distributed computing, commercial
  enterprise computing

5
Global GridCommunity
6
The (Dubious) Power Grid Analogy

• Do we ship work to the power source?

On-demand access to and integration of resources
services, regardless of location
Source Ian Foster
7
Grid Vision
Federico.Carminati_at_cern.ch
8
Grid Vision

• Useful, able to support/promote new science
• Usable (accessible, robust, easy-to-use)
• High-capacity (rich in resources)
• High capability (rich in options)
• Evolutionary (able to adapt to new technologies
  and uses)
• Persistent (usable by community in the long term)
  
• Stable (usable by community in the short term)
• Scalable (growth must be a part of the design)
• Integrative (promoting end-to-end solutions)

Source Fran Berman, SDSC
9
Grid Vision CyberInfrastructure?
10
Why do we want to use Grids for Science?

• Cost ?
• Reliability ?
• Ease of use ?
• Scalability with growing datasets ?
• Attracts funding?
• ?

11
Large Hadron Collider(LHC) at CERN
12
Is Grid really about costs and technology?

• The way to get massive computing cycles?
• The CMS detector alone cost several hundred
  million CHF and the LHC Accelerator several
  billion CHF
• The total estimated cost of the Tier0 and Tier 1
  center at CERN for LHC is in the tens of millions
  of CHF
• Total estimated cost for computing might be 3
  times this
• Power and cooling for all this is a problem but.
• The only way to store multi-petabyte datasets ?
• Today at Fermilab we have robotic storage that
  houses 2.5 PB of data
• Potential for 13 PB with todays tape technology
  and probably at least double that by 2008
• The way to reasonably share the costs ?

13
Is Grid about scavanging unused computing cycles?

• NOT Public Resource Computing
• seti_at_home
• Einstein_at_home
• folding_at_home
• LHC_at_home
• You-name-it_at_home

14
Why do companies jump on Grid

• To make money
• To control markets
• To leverage open source into market leadership
• For Science and Computer Science these are not
  our goals or the ways in which we determine
  success.

15
The Value of Grid ComputingIBM Perspective

Increased Efficiency
Higher Quality of Service
Increased Productivity ROI
Reduced Complexity Cost
Improved Resiliency
16
Grids HP Perspective
computing utility or GRID
virtual data center
value
programmable data center
grid-enabled systems
UDC
Tru64, HP-UX, Linux
clusters
Open VMS clusters, TruCluster, MC ServiceGuard
today
shared, traded resources
17
Grid - its really about collaboration!

• Its about sharing and building a vision for the
  future
• And its about getting connected
• Its about the democratization of science

18
Cathedral and Bazaar

• Linux overturned much of what I thought I knew.
  I had been preaching the Unix gospel of small
  tools, rapid prototyping and evolutionary
  programming for years. But I also believed there
  was a certain critical complexity above which a
  more centralized, a priori approach was required.
  I believed that the most important software
  (operating systems and really large tools like
  Emacs) needed to be built like cathedrals,
  carefully crafted by individual wizards or small
  bands of mages working in splendid isolation,
  with no beta to be released before its time.
• The Linux community seems to resemble a great
  babbling bazaar of different agendas and
  approaches.. out of which a coherent and stable
  system could seemingly emerge only by a
  succession of miracles
• Eric S. Raymond, The Cathedral and the
  Bazaar

19
The Grid Bazaar

• Many Grids and Grid environments
• An evolutionary process
• An environment for innovation and prototyping
  ways of achieving the grand vision
• A place where seemingly miracles are needed to
  make it all work together
• But really it is smart people and the will to do
  so
• Interoperability of seemingly divergent efforts
  will become the norm
• Standards must only come later and not suppress
  the possibility of achieving the grander goals
  and visions

20
For Success Define a Destination?
or look for a pot of gold?
To travel hopefully is a better thing than to
arrive. Robert Louis Stevenson
21
Borromean Rings
These rings have the interesting feature that,
while no two rings are linked, the set of rings
is linked. Remove any one ring, and the system
falls apart.
Computer and Information Science
Behavioral and Organizational Sciences, (Policy,
Government, Economics)
Scientific Research and Education
22
Collaboration

• EMSL Collaboratory
• The logo of the Collaboratory Project at the
  Environmental Molecular Sciences Laboratory uses
  the rings to symbolise collaboration between the
  researchers. Their website expains
• The Borromean Rings are three symmetric loops.
  Although the rings are interwoven and cannot be
  pulled apart, no two rings are interlinked -
  removing (breaking) one allows the other two to
  slide apart. This is the nature of collaborative
  work as well - people working together create
  something that is more than the simple sum of
  their individual efforts. This synergism is the
  principle behind the Collaboratory concept.

23
Pasteurs Quadrant
Creation of knowledge basic, curiosity-driven
research
Application of knowledge
Classic Linear Research Model
Pasteurs Quadrant Research Model

Pasteur
Yes
Bohr
Focus on New Knowledge Creation?
No
Edison
No
Yes
Focus on Application?
Source Dan Atkins
24
Many possible Grid Quadrants

• Information Technology v. Science Applications
• Local economic, network and IT development v.
  Global Science collaborations and projects
• Physics v. Biology
• Earth Observation v. ?
• Education v. Scientific Research

25
Fermilab and Collaboration

• Active experiments from the 2003 Research
  Program Workbook
• Of 213 Institutions involved 114 of them are
  non-US
• Of 1916 physicists, 753 are non-US
• Of 699 students, 234 are non-US

26
Taiwan-Fermilab Collaboration on physics

• Incredibly valuable contributions to the CDF
  experiment over many years
• Now we travel the Grid road together also

27
Open Science Grid a Collaboration
28
Working towards the Grid vision- a building
process, not a deployment activity

• How many physicists does it take to change a
  light bulb
• Answer One to hold the lightbulb, but more than
  100 to turn the building
• How many physicists does it take to BUILD a Grid
• Two typical types of answer
• (1) None, because they all know that someone else
  is deploying it for them
• (2) Three times as many as it would take to get
  their work done without using a Grid

29
Grids Metrics for Success

• How many jobs are run ?
• With what level of errors?
• How much data is stored/moved/read/written?
• How many people are using a specific Grid ?
• How many press releases are there about a Grid?
• How much money do governments allocate to a Grid?
• How easy to use is a Grid?
• How much more science gets done from use of a
  Grid?
• How effectively are resources being shared and
  well used?

30
Grids Metrics for Success

• All of the previous metrics are great indicators,
  but not the real measure for success
• Too many spins and interpretations

31
Grid Interoperable set of cyber infrastructures
(Grids)
A few problems still to solve

• Economy
• Payment
• Value of contributions
• Currencies
• Policy
• Who can use?
• Who is responsible?
• Who can decide?
• Security (of course)

32
If we can work together

• To make a contribution to all of the following
• Science and Scientific Research
• Society and its prosperity and harmony
• Education of the next generation
• Creating a dynamic collaborative environment for
  problem solving and innovation
• Then we will have succeeded

33
Contributing and Communicating

• If we can in some small way contribute to peace
  and prosperity in the world
• If we can help to close the digital divide and
  make contributing to science and technology and
  innovation a more democratic process (between the
  haves and the have-nots)
• If we can speak to our governments and have
  them better understand the Grid Quadrants and
  the Borromean rings I spoke of earlier
• Then we will have succeeded

34
Success on a Worldwide scale

• If we can bring together people from all over the
  world (whether they be physicists, biologists,
  computer scientists, climate researchers or .)
  and they
• Want to be part of building the cyber
  infrastructure or Grid environments or
  e-science environments for the future
• Actively participate
• Get benefit from the collaboration
• Then we will be succeeding

35
Success on a Worldwide scale

• The work will get done
• Scalability issues will get addressed
• Robustness will come
• Interoperability will be made to work
• Even more diversity than we see today
• Because people working together can and will make
  things work if they are involved and getting
  benefit for themselves, their institutions, their
  science, their nation in a broad way

36
Success is when we clearly communicate

• To each other
• To governments and funders
• To the public
• To our communities
• And the message we give is clear and positive

37
The Message

• Collaboration is essential on a worldwide scale
• To carry out many of the ambitious endeavors we
  want to undertake such as LHC and LHC
  experiments and after that the International
  Linear Collider and of course many other large
  international science endeavors in fields other
  than physics
• Diversity is the norm and healthy
• Networks are fundamental and high speed
  connectivity is essential
• Data is pervasive and ever growing and we need to
  analyze it, organize it, mine it, catalog it,
  curate it, understand it for all purposes ranging
  over
• Wallmart inventory tracking
• Climate and Earthquake predictions
• Finding the Higgs? And understanding the
  fundamental nature of matter
• Mapping the human genome
• Fighting cancer
• And much more..

38
Success

• When Grid the vision and the way of working -
  helps us get this message out
• Without finger pointing
• Without complaining about technology and its
  imperfections it will be gone before its
  perfect
• With balance between what must work today and
  what might help us tomorrow

39
ISGC-2005

• This workshop is one of the very signs of success
  
• The program is really excellent and diverse with
  strong participation from Asia-Pacific, Europe
  and the Americas
• Lets shut down our email and listen to the
  speakers and hear how their solutions to problems
  (in the small) fit in and each play a part in
  building a Grid environment (in the large)
• Lets communicate by listening

40
Thank you

• To Academia Sinica for hosting us and to Simon
  Lin and all of his staff for organizing this
  workshop
• To all of you who have been and will be
  collaborators on both Science and Grid projects
• Thank you for your energy and willingness to work
  together despite financial and cultural pressures
  that make collaboration hard
• To all of you for listening