Planning

1
Planning

• Hui Li
• Sep 13, 2004

2
Why another Grid?

• Grid is
• Learn the latest Grid technologies by building a
  Campus Grid
• A system that works
• Harnessing the idle cycles of desktops
• Intra and inter faculty resource sharing
• Have fun

HOT
3
The Evolution of Species - Sharing
Resource Sharing P2P, Web Services, Grids
Information Sharing WWW, HTTP, HTML
Communications and Data Sharing Email, ftp,
telnet, TCP/IP
Sharing
Networking ARPANET
4
Levels of Resource Sharing
Cluster Grid
Campus Grid
Global Grid
5
Enabling Technologies

• The key is middleware virtualizing and
  coordinating the underlying resources
• Open source

6
Campus Grid Worldwide

• Nanyang Technological University (NTU)
• University of Houston
• University of Wisconsin at Madison
• Virginia Tech (VT)
• Tokyo Institute of Technology (TiTech)
• University of Texas at Austin

7
NTU Campus Grid

• Partners CSV, BIRC, PDCC, APSTC, SUN
• Globus, SGE, Ganglia, MapCenter, ClearningHouse
  portal

8
University of Houston

• Partners CS, HPC group
• EZ-Grid (resource brokering)

9
University of Wisconsin at Madison

• Condor, Virtual Data Toolkit (VDT)

10
Virginia Tech

• Teracluster 1,100 dual processor 2Ghz PowerMac
  G5.

11
Tokyo Institute of Technology
12
UT/TACC Grid

• Alliance with IBM to build the largest Campus
  Grid in US
• Support for a large community of scientific
  researchers (Engineering, Biology, Physics,
  Chemistry, Computer Science, Geosciences, Arts,
  Business, etc)
• Middleware GridPort

13
Summary of Campus Grid Projects

• Have a couple of successful cases but still in
  the early stage of development
• Driven by the growing need for computing power of
  campus researchers
• Driven and enabled by the emerging Grid research
  and technologies
• Funding and industrial support

14
Our Situation

• DAS cluster in Leiden (full control)
• 20 Pentium Pro 200MHz nodes
• 64MB RAM
• 2.5 GB local disk
• Fast Ethernet interface

• Utilizing the existing IT infrastructure AMAP
• Harnessing idle cycles
• Demonstrate that a group of students can
  actually build a functional system

15
Small Fishs Approach

• Planning (first phase)
• Building the testbed
• Deploying middleware
• Developing high-level services
• Proof-of-concept applications
• Building and sharing expertise
• Prepare going campus-wide

16
Big Picture
17
Project 1 Fabric Management - Setting Up the
Testbed

• Networking (DHCP, NFS, NTP, DNS, etc)
• Linux Automatic Installation Infrastructure
• Middleware selection, installation and
  configuration (VDT)
• Security Infrastructure (NIKHEF CA)
• Testing and verification
• User Support web site, documentation, mailing
  lists, trouble ticket system (Bugzilla)

18
Virtual Data Toolkit (VDT)

• Globus
• Condor/Condor-G
• Virtual Data System (Chimera, etc)
• MyProxy
• MonaLisa
• Netlogger
• Various patches, packages, and utilities

19
Project 2 Stealing the Idle Cycles
20
Project 2 Stealing the Idle Cycles

• Condor open source, widely deployed,
  multi-platform support, etc
• Setup a linux Condor pool, several Windoz XP/NT
  machines with Condor packages.
• Goal Investigate the feasibility, policy, and
  configuration of desktop computing with Condor.

21
Project 3 LUCGrid Portal

• A Grid portal is a web server that provides an
  interface to Grid services, allowing users to
  submit jobs, transfer files, and query Grid
  information services from a standard web browser.
• Tools to facilitate building portals
• OGCE (Open Grid Computing Environment)
• Goal
• Setting up a portal server (lucgrid.liacs.nl)
• Integrating with other Grid components

22
Project Integration
Data Management Component (Proj. 7)
23
Project 4 Resource Brokering and Workload
Management
24
Project 4 Resource Brokering and Workload
Management

• Deployment
• Job submission interfaces to portals
• Interfaces to local resource management systems
  (PBS, Condor, etc)
• Interfaces to Information Services
• Development
• Simple brokering services

25
Project 5 Information Services and monitoring
26
Project 5 Information Services and monitoring

• Cluster monitoring using NWS, Ganglia, MapCenter,
  and MonaLisa
• Interfaces to Portals
• Interfaces to RM components
• Interfaces to DM components
• (If interested) Pull-based on-demand information
  framework (Surfer, etc)

27
Project 6 Security and VO Management

• Help with authentication infrastructure in
  Project 1 (fabric management)
• MyProxy - integration with portal
• VO (Virtual Organization) Management
• Setting up multiple VOs
• Authorization and Manage VOs
• VOMS software

28
Project 7 Data Management

• Interfaces to portals
• Data transfer using GridFTP
• Network monitoring with Netlogger
• Virtual Data System
• Replica location and selection service
• Demo Transfer large files between two sites
  through the portal

29
Project 8 Applications (Proof-of-Concept)

• Adapting applications to run on a Grid
• Potential application domains Bioinformatics
  (BLAST), Medicine, Physics, Computational
  Sciences, etc
• Goals
• Demo applications through the portal
• Documentation (quick-start guide, users guide)
• Communications with other faculties

30
Project Summary

• A functional Grid testbed with software stack,
  portal interfaces, demo applications, and user
  support
• Final report
• Detailed documentation of what you have done
• Submission to portal for dissemination
• Research projects and opportunities

31
Project Topics

• System - LUCGrid (FCFS, 7)
• Fabric Management (1)
• LUCGrid Portal (1)
• Desktop Computing (1)
• Resource Management (1)
• Information Services and Monitoring (1)
• Security (1)
• Data Management (1)
• Applications (Flexible, e.g. Bioinformatics,
  Finance, CS)
• Others
• OGSA and WSRF (Web Services)
• P2P
• Semantic Web

32
Project Organization (System Track)

• Coordinators - Hui and Lex, Advisor - David
• Nov 8 -
• Planning and starting LUCGrid project
• Nov 15, 22 and 29 -
• Deployment by individual projects (coordinated
  by Hui, fabric, security and portal students)
• Dec 6 - Integration meeting and retrospect
• Final report and demo

33
Project Organization (Application and others)

• Same as previous years
• Implement an grid-enabled application that runs
  on multiple clusters (DAS-2)
• Flexible in area selection
• For other possibilities (Web Services, P2P, etc),
  discuss with us

34
Presentation Topics

• Sep 27 -
• Fabric Management / Resource Management
• Oct 11 -
• Information Services / Security
• Oct 18 -
• Data Management / Networking
• Oct 25 -
• Grid Portals / Applications (User Cases)
• Nov 1 -
• OGSA and WSRF / P2P Computing

35
Presentation Organizations

• Two topics every time
• Two students every topic (45 minutes)
• Prepare one presentation together
• 1st student intro, architecture (20 min)
• 2nd student software, system (20 min)
• Questions and discussions (5 min, every
  participant is expected to ask one non-trivial
  question)

36
Schedule

• Sep 27 -
• Fabric Management (2)
• Resource Management (2)
• Oct 11 -
• Information Services (2)
• Security (2)

37
A taste of VM Technology

• VMware
• Virtual PC (Microsoft)
• User Mode Linux (UML)

38
Evaluation

• VMware (v)
• Pro performance, networking, multi-platform
• Con memory consumption
• Virtual PC
• Pro memory consumption
• Con networking, performance, uni-platform
• User Mode Linux
• Open source

39
VMware
40
Virtual PC
41
User Mode Linux
42
http//www.liacs.nl/hli/professional/courses/grid
2004/lucgrid.htm