Transition in Campus CyberInfrastructure: Community Clusters, Storage and CoLoc

1
Transition in Campus CyberInfrastructure
Community Clusters, Storage and Co-Loc
Presented by Dwight McKay, Director of Systems
Engineering ITaP Rosen Center for Advanced
Computing Purdue University
2
Outline

• Introduction
• Community Clusters
• Summarize model and issues

3
Introduction
Community Clusters
Summary
4
Introduction
Community Clusters
Summary
5
Rosen Center for Advanced Computing
Introduction
Community Clusters
Summary
6
Rosen Center for Advanced Computing
Introduction
Community Clusters
Summary
7
Transition in Research Computing Support
Introduction
Community Clusters
Summary
8
Transition in Research Computing Support

• Change in Direction
• From central purchase to researcher / project
  purchase
• From central shared facility to resource or
  service provider
• From service desk to partner

Introduction
Community Clusters
Summary
9
Transition Implications

• Paying Customers
• Higher Expectations
• Service Support
• Formal Agreements
• Cultural Change

Introduction
Community Clusters
Summary
10
Rosen Center for Advanced Computing
Research
User Support
Infrastructure
Introduction
Community Clusters
Summary
11
Rosen Center for Advanced Computing
Project
Project
Introduction
Community Clusters
Summary
12
Rosen Center for Advanced Computing
Project
Introduction
Community Clusters
Summary
13
Transition Implications

• New Business Models Needed
• HW/SW, infrastructure, support?
• Unpredictable Demand Funding
• Planning for power/space/cooling?
• Non-paying Users

Introduction
Community Clusters
Summary
14
Community Clusters

• Condominium Model
• Purchase Computation by the node
• Nodes come with bundle of services
• Purchase Storage by the TeraByte
• Storage comes with bundle of services
• Custom Arrangements for Specific Projects

Introduction
Community Clusters
Summary
15
Community Clusters

• Services

• Disk Storage
• Archival Storage
• On-Call Support
• Disaster Recovery
• Security

• HW Installation
• HW Maintenance
• Facilities Support
• Network Connection
• Infiniband Connnection
• OS installation and management

Introduction
Community Clusters
Summary
16
Community Clusters

• Services

• Disk Storage
• Archival Storage
• On-Call Support
• Disaster Recovery
• Security

• HW Installation
• HW Maintenance
• Facilities Support
• Network Connection
• Infiniband Connnection
• OS installation and management

Node Bundle
Introduction
Community Clusters
Summary
17
Community Clusters

• Tiered Cycle Allocation
• Owners guaranteed specific share
• Owners given first pick of idle cycles
• Owners agree to harvesting remaining idle cycles
• Use the whole buffalo. -- Brad Bird

Introduction
Community Clusters
Summary
18
Community Clusters

• Tiered Cycle Allocation

Owner
Introduction
Community Clusters
Summary
19
Community Clusters

• Tiered Cycle Allocation

Owner
Pre-empt
Introduction
Community Clusters
Summary
20
Community Clusters

• Tiered Cycle Allocation

Owner
Cycle Harvesting
Pre-empt
Introduction
Community Clusters
Summary
21
Community Clusters

• Challenges
• Business model that spans system generations
  recovers shared infrastructure costs
• Cluster Heterogeneity
• Multiple communities sharing one cluster
  (TeraGrid, OSG, NWICG)
• Other architectures special needs?

Introduction
Community Clusters
Summary
22
Community Clusters

• Storage
• Three Primary Tiers
• Fast for scratch files
• Commodity for home directories
• Archival for longer term storage
• Custom Storage for Specific Projects

Introduction
Community Clusters
Summary
23
Community Clusters

• Storage Challenges
• Business model that spans lifetime of data /
  researcher?
• Media Purchase vs. Space Rental?
• Data Retention Policy

Introduction
Community Clusters
Summary
24
Community Clusters

• Connectivity Challenges
• Multiple classes of network are needed
• Direct Routes
• Data Center to Key Research WANs
• Data Center to Key Research Labs

Introduction
Community Clusters
Summary
25
Introduction
Community Clusters
Summary
26
Community Clusters

• Four Fold Network
• Commodity
• Secure
• Research
• High Performance / Large Data
• Network Research
• Low Latency

Introduction
Community Clusters
Summary
27
Summary

• Transition to resource provider / partner /
  service
• Architecture
• Computation -gt Community Clusters
• Storage -gt 3 Tiers
• Connectivity -gt Direct Data Center connections to
  Research WAN and Lab Access
• Challenges
• Customer expectations and implications
• Serving up storage and other architectures by
  the slice
• Recovering ALL the costs, especially support

Introduction
Community Clusters
Summary