Millennium Overview and Status

1
Millennium Overview and Status

• David Culler and Jim Demmel
• Computer Science Division
• culler,demmel_at_cs.berkeley.edu
• http//www.millennium.berkeley.edu

2
MM Proposed Cluster of Clusters
Business
SIMS
BMRC
Chemistry
C.S.
E.E.
Gigabit Ethernet
Astro
NERSC
M.E.
Physics
N.E.
Math
IEOR
Transport
Economy
C. E.
MSME
3
Physical Connectivity
4
Associated Commitments

• Intel provides 6 M in equipment
• Sun provides all Solaris x86 software
• Microsoft provides all NT software
• Campus provides staff to support the core
  infrastructure, networking
• CS will try to raise funds for the network and
  campus cluster infrastructure
• Departments provide 1/2 of system admin for own
  side and 20 K for group cluster
• racks, network, software, ...

5
Where are we?

• Evolving Vision gt SimMillennium
• Hardware deployment
• Software availability
• Cluster Environment
• Grants
• Networking

6
The Vision

• To work, think, and study in a computationally
  rich environment with deep information stores and
  powerful services
• test ideas through simulation
• explore and investigate data and information
• share, manipulate, and interact through natural
  actions
• Organized in a manner consistent with the
  University setting

7
SimMillennium Project Goals

• Enable major advances in Computational Science
  and Engineering
• Simulation, Modeling, and Information Processing
  becoming ubiquitous
• Explore novel design techniques for large,
  complex systems
• Fundamental Computer Science problems ahead are
  problems of scale
• Develop fundamentally better ways of
  assimilating and interacting with large volumes
  of information
• and with each other
• Explore emerging technologies
• networking, OS, devices

8
Components of the Effort

• Community
• Cluster-based Resources
• Connectivity
• User Interaction
• Computational Economics

9
Component 0 Community

• An inter-disciplinary community with common
  interests and shared view of the future
• strong momentum in computational science and
  engineering
• Members of 17 campus units and NERSC in Intel
  Millennium
• Need and commitment required for participation
• Key subset represented in SimMillennium proposal

Strong Progress with Pacific Earthquake
Engineering Research Center New Thrust NSF
Science and Technology Center for the Study of
Turbulence in Geophysical and Astrophysical Flows
10
Component 1 Resources (Millennium)

• An environment with vast cluster-based computing
  power and storage (CLUMPS) behind a personal
  3D desktop

Campus Cluster
11
Resource Component Support

• Computers via Intel Technology 2000 grant
• 200 NT desktops
• 16 department 4-way SMPs
• 8 5x4 Group Clusters,
• 1 100x4 Campus Cluster
• PPro gt Pentium II gt Merced
• Additional storage via IBM SUR grant
• 0.5 TB this year gt 4 TB
• NT tools via Microsoft grant
• Solaris x86 tools via SMCC grant
• Campus provides Technical staff
• Research provides the prog. and system support

200 Gflop/s 150 GB memory 8 TB disk
12
NSF investment Cluster Network

• Transforms large collection of individual
  resources into a powerful system
• can be focus on a problem
• High Bandwidth
• scales with the number of processors (Gb/s per
  proc)
• Low Latency
• Low Overhead
• Low Cost
• Simple and Flexible
• Almost no errors
• Low Risk
• Today Myrinet

13
Cluster Research Agenda

• Applications grow into resources
• huge range of needs
• require Algorithmic Innovation, Prog. Tools,
  Performance
• Dealing Deep Memory Hierarchy
• New numerical algorithms on CLUMPs
• New compiler techniques for parallel object
  language
• Fast Multi-protocol Communication
• Global system at large scale
• Unix vs. NT, single system image vs. objects
• Exciting technology turnover
• VIA, SANs, Gigabit Ethernet

14
Component 2 Connectivity

• Create a richly interconnected pool of resources
  owned by members of the community
• Enable transportation of huge data sets and
  computation
• Enable remote visualization and collaboration
• Enable extensive sharing of resources
• Expand networking technology

CS Cluster
EE Cluster
CE Cluster
ME Cluster
Astro/Phys Cluster
xport Cluster
BIO Cluster
Econ/Math Cluster
Campus Cluster
15
NSF Investment Inter-cluster network

• Gigabit Ethernet connecting group clusters and
  campus cluster
• Bay Networks provides 70 discount
• Campus provides fiber plant, maintenance, and
  staff

16
Inter-Cluster Research Agenda

• Vastly expands the scope of systems challenge
• integrate well-connected resources according
  application needs, rather than physical packaging
• resource allocation, management, and
  administration
• Network bandwidth matches display BW
• Protocols and run-time sys. for visualization,
  media transport, interaction, and collaboration.
• Community can share non-trivial resources while
  preserving sense of ownership
• Bandwidth translates into efficiency of exchange
• Data can be anywhere
• Important networking technology in its own right.
• Emulate networks on Internet Scale!

17
Component 3 User Interaction

• High-quality 3D graphics emerging on
  cost-effective platforms
• desktops and dedicated cluster nodes
• NERSC team provides modern scientific
  visualization support
• Gigabit network allows this to be remote.
• New displays create workbench environment where
  large volumes of information can be viewed and
  manipulated.
• Trackers and Haptic interfaces greatly enhance
  degrees of user input
• 3D capture

18
NSF Investment UI Technology

• Two Projection Table
• large field of view in horizontal (or vertical)
  orientation
• Phantom Haptic Interface
• 3D force feedback
• Motion Tracker
• untethered position
• 3D Shutter Glasses
• low cost visualization

19
User Interaction Research Agenda

• Expand access to 3D visualization
• Explore any data anywhere
• Ease development
• Develop lab-bench metaphor for Viz
• two hands, physical icons
• Fast prototyping and exchange through Informal
  Interfaces
• sketching
• Dealing with large volumes of information
• lenses, brushing and linking
• 3D collaboration and interaction

20
Component 4 Computational Economy

• How is this vast, integrated pool of resources
  managed?
• Traditional system approach empower global OS to
  provide optimal allocation to blind
  applications
• predefined metric, tuned to fixed workload
• ignores the inherent adaptation of demand
• Computer Center
• charge gt director-to-user feedback according to
  cost
• Economic view decentralized allocation according
  to perceived value
• pricing gt user-to-user feedback
• compatible niches,sense of control, cooperation
• idea has been around, why now?

21
Research Agenda

• Natural fit to academic structure
• members want control over own resources, and each
  has varying needs that far exceed dedicated
  resources
• incentive for maintaining resources up to par
• Address partial or delayed information, component
  failure, and user satisfaction from the start
• Framework for elevating design from resources to
  services
• Rich body of theory, little empirical validation
• experts in several parts of the community
• New paradigm for algorithms perf. Analysis
• Complex, large-scale systems

22
Basic Approach

• Desktop an active agent conducting automated
  negotiation for resources
• Servers provide resources to highest bidders
• monitor usage and enforce limits within remote
  execution environment
• placement based on economic advantage
• Higher level system functions are self-supporting
• resource availability, brokering, directories
• Useful applications packaged as services
• may charge more than resources cost

23
NSF Investment Staff Support

• Provide enabling technology and let it evolve
• monitoring, enforcement
• exchange
• negotiation tools
• Integrate it into users enviroment
• Tools and measurements to determine effectiveness

24
Integrated Research Agenda

• Advance the State of Computational Science and
  Engineering
• immerse a community in a computationally rich
  environment with the right tools algorithms,
  programming system support
• Path to exploiting novel techniques and
  technology
• Explore design techniques for robust large-scale
  distributed systems
• economic (or ecologic) approach
• Explore new ways of interacting with information
• large paste-ups, two hands, sketching, 3D
  collaboration
• Investigate new technology
• SMP nodes, gigabit Ethernet, SANs, VIA
• NT, dCOM, Java beans, directory services
• workbench displays, 3D icons, haptics, position
  sensors

25
Perspective

• Highly leveraged investment in a large scale
  infrastructure for studying problems of scale
• Deep commitment across the campus
• Sense of ownership and participation
• Rich research agenda

26
Nuts and Bolts
27
Current Environment

• All projects have servers and several desktops
• last few in current shipment
• millennium.berkeley.edu domain established
• solaris/x86 shared servers with 60GB disks
• www.millennium.berkeley.edu, MM.millennium.berkel
  ey.edu
• imap, ...
• solaris/x86 sww served from CS servers
• NT domain server with 60 GB disks
• MMNT.millennium.berkeley.edu
• exchange, file, ...
• First cluster ready for use
• NSF SimMillennium grant to cover network
• Two of three Millennium staff hired

28
Hardware Deployment (per Q2_98)
29
Basic Software Tools

• Standard Unix Tools /usr/sww/bin
• 484 packages
• GNU Tools
• gcc, g (version 2.8.1), gdb, g...
• Sun ProWorks
• workshop development environment
• C, C, F77
• Debugger
• Sun Performance Library /usr/sww/lib
• Sun Math Library
• NT dialtone

30
Cluster Developments

• 4 x 4 PentiumPro Cluster ready
• Full NOW environment
• Glunix, MPI, Active Messages, Split-C
• titanium protoype
• Transfering to Cory to debug deployment
• CS / Civil Eng. Consortium has been shaking it
  out
• petc graph partitioner, finite element classes,
• ScaLapack (???)
• Have parts for three 8x2 Pentium II clusters
• AstroPhysics
• Soda Solaris x86 to be shipped out
• Soda NT cluster

31
Grants

• NSF CISE Research Infrastructure grant
• cluster networking
• inter-cluster networking (campus mgmt)
• computational economy staff support
• visualization
• devoted to CS research with disciplinary applns
• Bay Networks 70 discount up to 5 M
• gigabit networks
• IBM SUR disk towers
• IBM CS extension to PDA access
• NERSC/LBL - DOE2000 Initiative
• gt NSF Science and Tech. Centers in process

32
Networking

• Campus using Millennium to drive planning
• CNS is working closely with us.
• Testing gigabit ethernet
• Campbell, Evans, Davis moving forward
• CS will be able to cover the intracluster network
  (Myrinet) and gigabit switch in the group
  clusters
• frees up some of the committed resources

33
Going forward

• Roll out group clusters over next few quarters
• Roll out gigabit interconnect
• Build up the cluster programming tools
• Intel Merced pushed out
• campus cluster delayed
• utilize NOW as an alternative
• combine with NPACI infrastructure
• Exploring cluster technology
• VIA
• Synfinity, ServerNet, Gigabit Ethernet
• Exploring NT

34
A New Tier to Millennium
Wireless Infrastructure
Future Devices
PDAs
Cell Phones