The OptIPuter: Enabling Campus, State, National, and Planetary-Scale LambdaGrids

1
"The OptIPuter Enabling Campus, State, National,
and Planetary-Scale LambdaGrids"

• LONI Forum
• Baton Rouge, LA
• September 2, 2004

Dr. Larry Smarr Director, California Institute
for Telecommunications and Information
Technologies Harry E. Gruber Professor, Dept. of
Computer Science and Engineering Jacobs School of
Engineering, UCSD
2
OptIPuter Application Motivation Interactive
Analysis of Remote Multi-Gigabyte Data Objects

• Hundreds Of Megapixels 2-D Images
• Satellite Imaging and Remote Sensing
• Telescopes or Microscopy
• GigaZone 3-D (1k x 1k x 1k) Objects
• Supercomputer Simulations
• Seismic or Medical Imaging
• Goal Remote Interactive Analysis and
  Visualization
• Virtually Impossible Today
• New Technology of Dedicated Lightpaths
• Bring These Capabilities to Laboratory
  Researchers

3
Cosmic Simulator with a Billion Zone and
Gigaparticle Resolution
Source Mike Norman, UCSD

• 5123 AMR or 10243 Unigrid
• 8-64 Times Mass Resolution
• Can Simulate First Galaxies
• One Gigazone Run
• Output 10 TeraByte
• Snapshot is 100 GB
• Must Visually Analyze

10243 Unigrid
Compare with Sloan Survey
SDSC Blue Horizon (2004)
4
OptIPuter Driver On-Line Microscopes
CreatingVery Large Biological Montage Images
IBM 9M Pixels

• 2-Photon Laser Confocal Microscope
• High Speed On-line Capability
• Montage Image Sizes Exceed 16x Highest Resolution
  Monitors
• 150 Million Pixels!
• Use Graphics Cluster with Multiple GigEs to Drive
  Tiled Displays

Source David Lee, NCMIR, UCSD
5
Earth System Enterprise-Data Lives in
Distributed Active Archive Centers (DAAC)
EOS Aura Satellite Has Been Launched Challenge is
How to Evolve to New Technologies
6
However, Average Throughput of NASA Data Products
to End User is Only 50-100 Megabits/s
Tested from GSFC-ICESAT August 2004
http//ensight.eos.nasa.gov/Missions/icesat/index.
shtml
7
High Resolution Aerial Photography Generates
Images With 10,000 Times More Data than Landsat7
Shane DeGross, Telesis
USGS
8
States are Acquiring Their Own Dark Fiber
Networks -- Illinoiss I-WIRE and Indianas
I-LIGHT
Today Nearly Two Dozen States are Creating Dark
Fiber NetworksIncluding the Most Advanced LONI!
Source Charlie Catlett, ANL
9
Regional Network Evolution

• The Fundamental Nature of Regional Networking is
  Rapidly Changing
• The GigaPoP Model Based on Provisioned,
  High-Capacity Services Steadily Is Being Replaced
  
• On The Metro and Regional Scales
• A Model of Facility-Based Networking Built with
  Owned Assets has Emerged
• Regional Optical Networks (RONs)
• Notably, this Change Increases the Importance of
  Regional Networks in the Traditional Three-Level
  Hierarchy of U.S. RE Advanced Networking

Source Steve Corbató, Internet2
10
Leading Emerging Regional Optical Networks

• New York New England Region (NEREN)
• North Carolina (NC LambdaRail)
• Ohio (Third Frontier Network)
• Oklahoma (OneNet)
• Oregon
• Pacific Northwest (Lariat Supported by NIH)
• Rhode Island (OSHEAN)
• SURA Crossroads Southeastern U.S.)
• Texas (LEARN)
• Utah
• Virginia (MATP)
• Wisconsin (WEROC)

• California (CALREN)
• Colorado (FRGP/BRAN)
• Connecticut (Conn. Education Network)
• Florida (Florida LambdaRail)
• Georgia (Southern Light Rail)
• Indiana (I-LIGHT)
• Illinois (I-WIRE)
• Louisiana (LONI)
• Maryland, D.C. northern Virginia (MAX)
• Michigan (MiLR)
• Minnesota

Source Steve Corbató, Internet2
11
NLR Will Provide an Experimental Network
Infrastructure for U.S. Scientists Researchers
National LambdaRail Partnership Serves Very
High-End Experimental and Research Applications
4 x 10Gb Wavelengths Initially Capable of 40 x
10Gb wavelengths at Buildout
Starting Fall 2004
12
Dark Fiber Gauging Community-Wide Progress

• Aggregate Dark Fiber Assets Acquired by/for U.S.
  RE Optical Initiatives
• CENIC (for CalREN NLR)
  6,200 (Route-miles)
• FiberCo (via Level 3 for NLR RONs)
  5,650
• SURA (via ATT)
  6,000
• Plus 2,000 Route-Miles for Research
• NLR Phase 2 (WilTel Qwest)
  4,000
• OARnet
  1,600
• ORNL (via Qwest)
  900
• Other Projects (IN,IL,MI,OR, )
  1,500
• Total (conservative estimate)
  25,850
• Over 55 of these Assets are in Regionals
• Remainder held by NLR (11,250 Route-Miles)

Source Steve Corbató, Internet2
13
Global Lambda Integrated Facility (GLIF)Creates
Metacomputers on the Scale of Planet Earth
Stockholm NorthernLight
New York MANLAN
10 Gbit/s
IEEAF 10 Gbit/s
10 Gbit/s
2.5 Gbit/s
2.5 Gbit/s
10 Gbit/s
SURFnet 10 Gbit/s
CAnet4
Amsterdam NetherLight
Dwingeloo ASTRON/JIVE
Chicago StarLight
Tokyo WIDE
10 Gbit/s
IEEAF 10 Gbit/s
DWDM SURFnet
NSF 10 Gbit/s
10 Gbit/s
10 Gbit/s
2.5 Gbit/s
Tokyo APAN
SURFnet 10 Gbit/s
2.5 Gbit/s
10 Gbit/s
Prague CzechLight
Geneva CERN
London UKLight
www.glif.is Created in Reykjavik, Iceland 2003
Source Kees Neggers, SURFnet
14
The OptIPuter Project Removing Bandwidth as an
Obstacle In Data Intensive Sciences

• NSF Large Information Technology Research
  Proposal
• Cal-(IT)2 and UIC Lead CampusesLarry Smarr PI
• USC, SDSU, NW, Texas AM, Univ. Amsterdam
  Partnering Campuses
• Industrial Partners
• IBM, Sun, Telcordia/SAIC, Chiaro Networks,
  Calient, Glimmerglass
• 13.5 Million Over Five Years
• Optical IP Streams From Lab Clusters to Large
  Data Objects

NIH Biomedical Informatics
NSF EarthScope and ORION
Research Network
http//ncmir.ucsd.edu/gallery.html
siovizcenter.ucsd.edu/library/gallery/shoot1/index
.shtml
15
OptIPuter Middleware Architecture for
Distributed Virtual Computers
OptIPuter Applications
Visualization
DVC 1
DVC 2
DVC 3
Layer 5 SABUL, RBUDP, Fast, GTP
Real-Time Objects
Security Models
Data Services DWTP
Higher Level Grid Services
Grid and Web Middleware (Globus/OGSA/WebServices
/J2EE)
Layer 4 XCP
Node Operating Systems
l-configuration, Net Management
Physical Resources
From Grids to LambdaGrids
Source Andrew Chien, UCSD OptIPuter Software
Systems Architect
16
Creating a Model for a Campus LambdaGrid
17
OMNInetThe Metro Area OOO Tesbed
NTON
10 Gb Lambdas
18
10GE OptIPuter CAVEWAVEWill Help Launch the
National LambdaRail
EVL
Source Tom DeFanti, OptIPuter co-PI
19
OptIPuter Scalable Visualization SystemsEnd
User Clusters Connected With Gigabit Flows
Earth Sciences are an OptIPuter Driver Major NASA
Goddard Visit This Week
20 Megapixels
3 Megapixels
Linux Cluster with Nvidia Graphics Cards
20
High Bandwidth Optical Fibers Will Enable High
Definition Global Virtual Teaming
UC Irvine
UC San Diego
In 2005 Cal-(IT)2 will Link Its Two Buildings
Creating a Collaboration Laboratory
21
Ultra-Resolution OptIPuter Displays Utilizing
Photonic Multicasting --Scaling to 100 Million
Pixels
UIC-EVL
GlimmerglassSwitch Used to Multicast and Direct
TeraVision Stream from One Tile to Another on the
Geowall-2
Driven by Linux Graphics Clusters
Glimmerglass Switch
30 Megapixel High-Resolution Visualizations 1
Gigapixel at 30fps 30Gb/s Bandwidth
22
Currently Developing OptIPuter Software to
Coherently Drive 100 MegaPixel Displays

• 55-Panel Display
• 100 Megapixel
• 11x5 21 LCDs
• Developed at EVL
• Driven by 30 Dual-Opterons (64-bit)
• 60 TB Disk
• 30 10GE interfaces
• Linked to OptIPuter
• Complementary to NASA ARC HyperWall

23
Special issue of Communications of the ACM
(CACM)Blueprint for the Future of
High-Performance Networking

• Introduction
• Maxine Brown (guest editor)
• TransLight A Global-scale LambdaGrid for
  e-Science
• Tom DeFanti, Cees de Laat, Joe Mambretti, Kees
  Neggers, Bill St. Arnaud
• Transport Protocols for High Performance
• Aaron Falk, Ted Faber, Joseph Bannister, Andrew
  Chien, Bob Grossman, Jason Leigh
• Data Integration in a Bandwidth-Rich World
• Ian Foster, Robert Grossman
• The OptIPuter
• Larry Smarr, Andrew Chien, Tom DeFanti, Jason
  Leigh, Philip Papadopoulos
• Data-Intensive e-Science Frontier Research
• Harvey Newman, Mark Ellisman, John Orcutt

www.acm.org/cacm