From NCSA to the Alliance - Computer Science interacting with Computational Science

1
From NCSA to the Alliance - Computer Science
interacting with Computational Science

• Invited Talk to UIUC Computer Science Grad
  Students
• January 29, 1998

2
The Alliance Emerges From the NSF Supercomputing
Center

• National Center for Supercomputing Applications
• Unit of the University of Illinois at
  Urbana-Champaign
• Founded 1985
• Becomes the Alliance Leading Edge Center
• Information and Computational Infrastructure
  Arises
• NSFnet Develops from Centers Backbone
• Macs and PCs Hook in Using NCSA Telnet
• Web Browsing Grows out of NCSA Mosaic
• Partnerships in Advanced Computational
  Infrastructure
• New NSF Program
• Two Winners Alliance and NPACI

3
The Alliance is Prototyping the National
Technology Grid

• Leading Edge Centers
• Supernodes of the Grid
• Enabling Technology Teams
• Architects of the Grid
• Applications Technologies Teams
• Specifications for the Grid
• Education, Outreach, and Training Teams
• Content for the Grid
• Partners for Advanced Computational Services
• Support for the Grid
• Industrial Partners and Strategic Vendors
• Technology Transfer for the Grid

4
The Alliance National Technology Grid -
Prototyping the 21st Century Infrastructure
5
NCSA Industrial Partners

• Allstate Insurance Co.
• Boeing Company
• Caterpillar Inc.
• Eastman Kodak Co.
• Eli Lilly and Company
• FMC Corporation
• Ford Motor Company
• J. P. Morgan

• Motorola, Inc.
• Phillips Petroleum Co.
• SABRE Group, Inc.
• Schlumberger
• Sears, Roebuck Co.
• Shell Oil Company
• Tribune Company

6
NCSA Strategic Vendor Partners
Hardware
Software

• Ameritech
• Compaq Computer
• Cisco
• Dell
• EMC
• Hewlett-Packard
• MCI
• Silicon Graphics/Cray

• Computer Associates
• IBM
• Microsoft
• Oracle
• Platform Computing
• Sun Microsystems
• Sybase

7
DCS Faculty Working With the Alliance

• Reed - I/O, Performance Analysis, VR
• Chien - NT Clusters
• Torrellas - Architecture and Apps. Analysis
• Nahrstedt - Networks, QoS, Multimedia
• Alliance Funds DCS for These Areas

8
Alliance Enabling Technologies Teams - Faculty
Leads

• Parallel Computing (16)
• Ken Kennedy, Rice U
• Greg McRae, MIT
• Distributed Computing (15)
• Rick Stevens, Argonne
• Paul Woodward, U Minnesota
• Data and Collaboration (14)
• Dan Reed, UIUC
• Roscoe Giles, Boston U

9
FY 98 Parallel ComputingET Team Goals

• Development
• Basic FFT Routines
• Adaptive Grid Tutorials
• ADIC
• CAVE Library on DSM
• HPC Library
• Experimental HPF Compiler
• F77 F90 Perfomance Tools
• Version 1 NT Cluster Software Distribution
• Treadmarks on NT Clusters

• Current Deployment
• Enabling Libraries
• Dense linear algebra
• Parallel MPI
• SQP
• Portable Extensible Toolkits for Sci.Computing
• Distributed Adaptive Grid Hierarchy
• HPF application and compiler studies
• HPF, C, and I/O performance tools

10
FY 98 Distributed Computing ET Team Goals

• Current Activity
• Initial Team in Place
• ANL/NCSA staff
• Support from PACS
• Strategy Evolving
• Leverage of existing effort
• Model after Internet Engineering Task Force
• Formed Working Groups
• Security
• Networking
• Distributed Queuing
• Several others

• Development
• Tele-immersion Driver
• Multi-flow Network QoS Requirements Outlined
• ANL Testbed to Connect EVL, NCSA
• Globus Testbed In Place
• ANL, ISI, NCSA, Indiana
• Visual Supercomputing Testbeds
• Utah, EVL, ANL, NCSA, UMinn.
• Object Technology
• Alliance-wide Strategy

11
FY 98 Data and Collaboration ET Team Goals

• Collaborative Tool Deployments
• Decision Support for Teams
• Univ of Michigan
• Synchronous, web-integrated
• Tango (Syracuse)
• Synch/Async framework
• Habanero / Isaac (NCSA)
• Team Plans to use tools itself, with Various AT
  teams

• Scalable I/O Storage
• Hierarchical Storage
• Multiple Platform Evaluation (HPSS, DMF, Unitree)
• NCSA/PSC User Data Migration
• Large-scale RAID Testbeds
• NCSA 4TB EMC RAID
• Minnesota Fibre-channel Ciprico RAID
• Scalable I/O Initiative
• Performance Tools
• Parallel I/O

12
How to Find out More About the Alliance
See also http//alliance.ncsa.uiuc.edu
13
NSF vBNS and PACI - Mutually Interdependent
NCSA Alliance
NPACI
Both NCSA Alliance and NPACI
Other High Performance Connection sites
Current vBNS Backbone sites
14
NSF Funds New Grant to Support National
Distributed Applications Support Team

• NCSA Award for FY98-2000 for 2M
• Start Date November 1, 1997
• Follow-on to National Laboratory for Applied
  Network Research (NLANR)
• Establish vBNS Support Center at NCSA
• Distributed Applications Help Desk
• Training and Workshops for Distributed
  Application Developers and Users
• Repository for Application User and Developer
  Resources, Analyses, and Lessons Learned

http//access.ncsa.uiuc.edu/NewsBriefs/1.21.98DAST
.html
15
The Illinois Century Network Illinois Board of
Higher Ed. Technology Task Force

• Proposal calls for the creation of a high-speed
  state backbone 155-622 Mbps. (Internet 2 Type
  speeds)
• Higher education institutions would connect at
  45-155Mbps

16
International Connections Through STAR TAP
17
NCSAs Three Major Technology Foci

• Scalable Computing
• Clustered Computing Architectures
• Evolution to NT/Intel
• Knowledge Management
• Intranet Use and Development
• Automated Discovery Learning
• Virtual Environments
• Desktop to CAVE
• Tele-immersion

18
TOP500 Systems by Vendor
TOP500 Reports http//www.netlib.org/benchmark/t
op500.html
19
Three Architectures for High End Computing

• Clusters of DSM
• SGI Origin2000 UNIX DSMs
• HP Exemplar UNIX DSM
• Clusters of SMPs
• SGI Power Challenge UNIX SMPs
• HP and Compaq NT/Intel SMPs
• Networks of Computers
• UWisc. Condor for UNIX Workstations
• NCSA Symbio for NT Desktops

NCSA FY98
20
NCSA Combines Shared Memory Programming with
Massive Parallelism
Future Upgrade Under Negotiation with NSF
21
Worlds Largest Unclassified SGI/Cray Origin -
NCSAs Cluster of DSMs
Origin Array
Processors
128
4x64
4x32
Power Challenge Array
10x16
128 Processors Equals a 512 Processor CM-5
22
The Road to Merced
23
Alliance NT Cluster Approaches

• Fast Messaging
• Andrew Chien, UIUC DCS
• High Performance Network Backplane on SMPs
• Support for MPI-FM, MPI-2 put/get, global arrays
• NCSA Symbio
• Briand Sanderson, UIUC NCSA
• Parallel Distributed Computing Environment
• DCOM / COM based over Desktops and Servers
• Treadmarks
• Willy Zwaenepol, Rice CRPC
• Software DSM over NT Cluster

24
NCSA Knowledge Management Workspaces
Distributed Object Technology
Object and Relational Databases
Simulation Engine
Optimization
Agents ORBs (CORBA / ActiveX ) Scripting JavaBean
s / Enterprise Objects Java / Smalltalk
Collections
Optimization Tools
Data Warehouses
Knowledge Discovery and Visualization
Analysis
CAVE Devices
SGI Mineset
Collaborations (Habanero, Tango)
VRML Browser
AVS, VDI
Application Specific Browser
Automated Discovery
25
Unicenter TNG on NCSA LAN - Drilling Down to
Agent View
Source CA and Cameron Ninham, NCSA
26
Java Agents For Knowledge IntegrationOver
Distributed Databases
WWW User Interface
User Request
Result
Data Query Reporting
Application
OLAP
Knowledge Discovery
Facilitator
Agent
Agent
Agent
Munich
Champaign
Agent
Subspace
Sidney
Tokyo
Distributed Space
27
How Application Teams Drive the Grid

• Cosmology
• Metacomputing
• Environmental Hydrology
• Immersive Collaboration
• Chemical Engineering
• Virtual Prototyping
• Bioinformatics
• Distributed Data
• Nanomaterials
• Remote Microengineering
• Scientific Instruments
• Virtual Observatories

28
How Application Teams Drive the Grid

• Cosmology
• Metacomputing
• Environmental Hydrology
• Immersive Collaboration
• Chemical Engineering
• Virtual Prototyping
• Bioinformatics
• Distributed Data
• Nanomaterials
• Remote Microengineering
• Scientific Instruments
• Virtual Observatories

29
Collaborative Virtual Environment -Environmental
Modeling
Coupling Chesapeake Bay Simulations and Databases
ImmersaDesks
DREN
SGI Onyx (CEWES) Vicksburg, MS
vBNS
SGI Onyx (NCSA)
SGI Onyx (U. Wisc)
SGI Onyx (Old Dominion)
Integrated M-Bone Videoteleconferencing
John Shalf,Polly Baker NCSA Mike Stephens and
Carl Cerco, CEWES
30
Using NCSAs Virtual Director in CAVE5D to
Create a Digital Video Output
Donna Cox, Robert Patterson, Stuart Levy,
NCSAVirtual Director Team
31
Using CAVE5D with NCSAs Virtual Director to
Analyze Chesapeake Bay Simulations
Alliance Environmental Hydrology Applications
Team Glen Wheless and Cathy Lascara, Center for
Coastal Physical Oceanography, Old Dominion
University
Fish Larvae at Mouth of the Bay 15 Day
Period Salinity (Red-High, Yellow-Low)
Donna Cox, Robert Patterson, Stuart Levy,
NCSAVirtual Director Team
32
Alliance Researchers Using a Digital
VideoComputational Infrastructure
CAVE Virtual Director
Habanero Teams
Desktop Video Teleconferencing
Internet, vBNS
Individual Desktops
Create Digital Video Animation Concurrently with
Supercomputing
Digital Video Server
33
Putting a Window in the Supercomputer Oven-
Coupling the vBNS to Scalable Computing
Evolution of a Red Giant with White Dwarf Core
Surface View
Interior View
Data Moved From NCSA over vBNS to U
Minnesota- Visualization at SC97 While Week Long
Simulation Runs at NCSA vBNS Gives 100-Fold
Thruput Increase Over Commercial Internet!
Porter, Anderson, Habermann, Ruwart, Woodward ,
LCSE,Nov. 1997
34
Proposed National Analysis and Visualization
Facility
Joint Project of NCSA / LCSE / EVL
6400x4800 Pixel Display IMAX Film
Resolution Interactive VR
Silicon Graphics Reality Monster


Design by Laboratory for Computational Science
Engineering, Univ of Minnesota
35
Alliance National Technology GridWorkshop,
Collaboration, and Training Facilities
Powered by Silicon Graphics Linked by the NSF vBNS
36
Caterpillars Distributed Virtual Reality
Data courtesy of Valerie Lehner, NCSA, 1996
37
Tele-Immersion Networking Requirements
Source Tom DeFanti (UIC), Rick Stevens (ANL)
38
Bringing Scientific / Information Visualization
Virtual Environments to the NT Desktop

• Early Partner with SGI NT Supergraphics
• Port CAVE to NT
• Work with Alias/Wavefront on NT
• VRML and Java3D
• Compatible with CAVE Libraries
• Develop Specific Applets
• Integration with Collaborative Environments
• Work with AVS, VDI, and other Third Party
• Partner with Graphics Team at Microsoft

39
Alliance Visualization Development and
Deployment Partners
40
Graduate Research Assistants at NCSA

• 23 Computer Science
• 7 Engineering (other departments)
• 9 LAS
• 2 Commerce MBA
• 2 Education
• 1 MBA/CS combined
• 1 Labor and Industrial Relations
• 1 Ill. State Geological Survey

41
Undergraduate Internship Program

• Pilot program, spring 1998
• Sophomores and Juniors eligible
• Faculty nominations of students to NCSA
• by February 16
• Students work with NCSA in areas of interest
• Up to 20 hours per week for 20 weeks
• Email Internship_at_ncsa.uiuc.edu

42
Undergraduates Employed at NCSA

• Computer Science - 16
• Engineering (other departments) - 16
• Liberal Arts Sciences - 9
• Fine and Applied Arts - 4
• Commerce and Business Admin - 3
• College of Communications - 1
• ACES (Agriculture-Coop Extension) - 1
• Applied Life Studies - 1

43
Key Areas Where NCSA has Openings

• vBNS Distributed Applications Support Team
• Multimedia - Audio and Video Serving
• Artificial Intelligence and Data Mining
• NT Clusters and Symbio
• Java and Application Teams
• Visualization, VR and Tele-Immersion
• Collaboration and Distance Learning Modalities
• Contact Janet Payne, NCSA Recruiter
• jlp_at_ncsa.uiuc.edu, 217-265-0619