Collaboration and Cyberinfrastructure

1
Collaboration and Cyberinfrastructure
NSF Symposium on Knowledge Environments for
Science

• Dr. Francine Berman
• Director, NPACI and SDSC
• Professor, Department of Computer Science and
  Engineering, UC San Diego

2
Outline of Presentation

• How we collaborate today
• Collaboration and cyberinfrastructure -- More
  than technical challenges
• Changing the rules to promote collaboration in
  cyberinfrastructure

3
How we collaborate today
4
Collaboration enables science

• Science as a team sport
• Grand Challenge Projects
• Formal program for multi-disciplinary
  collaborations to attack largest-scale problems
• Community Databases
• Broad collaboration to build key resources for
  science

NationalVirtualObservatory
5
Collaboration creates new science

• SETI_at_home
• 3.8M users in 226 countries
• 1200 CPU years/day
• 38 TF sustained (Japanese Earth Simulator is 40
  TF peak)
• 1.7 ZETAFLOP over the last 3 years (1021,
  beyond peta and exa )

6
Collaboration increasingly prevalent for
non-scientists

• Sharing as a default mode of interaction
• The Internet
• Globalization
• Open source
• Napster

• Distributed Entertainment Everquest
• 45 distributed clusters supporting 430,000
  players
• Real-time interaction and collaboration,
  individualized database management, back channel
  communication between players
• Game masters collaborate with players for
  real-time game management

7
Collaboration is fundamental for the next
generation of science advances
Cyberinfrastructure
BIRN
NSF PACI
Grid Computing provides a framework for
collaboration which both enables science and
creates new science
8
Collaboration andCyberinfrastructure
9
More than technical challenges

• Collaboration is a fundamental paradigm for
  cyberinfrastructure
• Technical support for collaboration in
  cyberinfrastructure is not enough
• Cyberinfrastructure program must be structured
  to promote and incentivize collaboration

Collaboration is central to cyberinfrastructure
10
Over the last two decades, NSF has played a
leading role in Information Infrastructure

• Supercomputer Centers program brought high-end
  computation to the open scientific community
• PACI partnerships focusing on collaboration and
  critical infrastructure development driven by
  science needs
• Perhaps the largest worked example of
  collaboration to date
• Cyberinfrastructure must be designed to meet the
  hardware, software, and human infrastructure
  needs of the next generation of science and
  technology
• Will need to take collaboration and partnership
  to new levels

11
Cyberinfrastructure will bring new challenges
Supercomputing
Cyber-computing
Aggregate power,storage, bandwidthEnd-to-end
performance, ease of use Principal foci HW,SW
Power (FLOPS), size of run, length Principal
Focus HW
Conventional measures of evaluation
Coordinating SW, interfaces, distributed use and
access
OS, compilers, Schedulers for supercomputers
Focus of SW development
In addition, data-oriented users, Grid users,
users of remote instruments, DBs
Largely compute- intensive users
User base
12
Cyberinfrastructure will bring new challenges
13
New rules will be needed to ensure the success of
cyberinfrastructure

• Program which promoted success for supercomputer
  centers will not incentivize for success for
  cyber-centers
• Notion of partnership going forward must be more
  coordinated than original PACI program
• Cyberinfrastructure requires long-term commitment

• Cyberinfrastructure program must incentivize for
  real cooperation
• Program must incentivize for serious SW
  infrastructure development
• Persistent, stable infrastructure, not just
  demos
• Program must have large-scale resources
• Compute, data, networks, viz
• Grid computing is not a replacement for
  investment in high-end resources.
• Program must recognize the importance of human
  infrastructure
• Scope/goals/budget should account for human
  infrastructure needed for success

14
New metrics will be needed to ensure the success
of cyberinfrastructure

• What will it mean for cyberinfrastructure to be
  successful?
• What metrics will promote long-term success?
• What metrics will incentivize cooperation?
• What will it mean for software to be usable?
• How will we know if were serving the needs of
  the disciplinary communities?
• New metrics will be needed to ensure that
  cyberinfrastructure is successful

15
Changing the rules to promote collaboration in
cyberinfrastructure
16
Hierarchy and Collaboration

• The U.S. Transportation Model
• Hubs provide full-service, large-scale facilities
• More than one hub amortizes traffic and provides
  geographical distribution
• New hubs not added without considerable thought,
  planning, experience
• Regional airports provide end-to-end service
  through hubs

• All sites must collaborate to ensure success for
  travelers

17
Food for thought -- A Collaborative Model of
Cyberinfrastructure

• Tier 1 sites provide leadership, full
  functionality
• Sites have unique large-scale facilities, large
  expert staff
• Tier 1 sites natural venue for SW development,
  last help desk, etc.
• Sites must cooperate for best success
• End-to-end performance will be dependent on
  coordination
• Model must support evolution and growth
• New sites will require new funding
• New Tier 1 sites should start as Tier 2 sites to
  demonstrate readiness
• Model must promote persistence and stability
• Stringent review, long-term planningrather than
  re-competitions and incremental short-term plans

• Tier 1 sites provide high-end facilities, SW, HW,
  infrastructure, and science leadership
• Tier 2 sites provide value added (e.g. data
  center, ocean center), coordinate with Tier 1
  sites for end-to-end solutions

18
Cyberinfrastructure is an NSF-wide activity

• Collaboration among NSF Directorates is critical
• Needs of disciplinary communities should drive
  research, development and procurement agendas
• Building a successful cyberinfrastructure will
  require an unprecedented level of community
  collaboration, including
• Researchers (to address cyberinfrastructure-specif
  ic disciplinary and computer science problems)
• Technologists (to provide stable, evolutionary
  infrastructure, not demos)
• Funding agencies (to develop programs which
  promote collaboration and coordination, to
  provide long-term agency advocacy)
• Developing stable, usable cyberinfrastructure is
  hard we will need to build on current successes
  to retain leadership
• Dont reinvent the wheel -- Communities should
  leverage core infrastructure, allowing new
  efforts to focus on new functionalities

19
Thank You

• berman_at_sdsc.eduSan Diego Supercomputer Center
• National Partnership for Advanced Computational
  Infrastructure
• www.sdsc.edu, www.npaci.edu