dev'globus and the Incubator Process

1
dev.globus and the Incubator Process

• Jennifer M. Schopf
• Argonne National Lab
• National eScience Centre
• http//www.mcs.anl.gov/jms/Talks/

2
Outline

• Jennifer Schopf
• dev.globus and the incubator process
• Kate Keahey
• Virtual Workspace Project and Dynamic Accounts
  Project
• Ignacio M. Llorente
• GridWay MetaScheduler
• Von Welch
• GridShib
• Questions

3
The Globus Commitment to Open Source

• Globus was first established as a project in 1996
• Always under an open source license-
• Currently Apache License version 2 (BSD-style)

4
Why is Globus Software Open Source?

• To allow for inspection
• For consideration in standardization processes
• To encourage adoption
• In pursuit of ubiquity and interoperability
• To encourage contributions
• Harness the expertise of the community

5
Open Contribution

• But distributing code under an open source
  license does not guarantee open development!
• Open development requires open processes
• So we have created dev.globus to facilitate
  contributions
• Globus Development Environment
• http//dev.globus.org/

6
Governance Model

• Based on Apache Jakarta
• Individual development efforts organized as
  projects
• Consensus-based decision making
• Control over each project in the hands of its
  most active and respected contributors
  (committers)
• Globus Management Committee (GMC) providing
  overall guidance and conflict resolution

7
Common Infrastructure

• Code repositories (CVS, SVN)
• Mailing lists
• -dev, -user, -announce, -commit for every
  project
• Issue tracking (bugzilla)
• Including roadmap info for future development
• Wikis
• Known interactions for people accessing your
  project

8
Sample

• http//dev.globus.org/wiki/GRAM

9
(No Transcript)
10
(No Transcript)
11
(No Transcript)
12
(No Transcript)
13
Technology Projects

• Common runtime projects
• C Core Utilities, C WS Core, CoG jglobus, Core WS
  Schema, Java WS Core, Python Core, XIO
• Data projects
• Data Replication, GridFTP, OGSA-DAI, Reliable
  File Transfer, Replica Location
• Execution projects
• GRAM, MPICH-G2
• Information services projects
• MDS4
• Security Projects
• C Security, CAS/SAML Utilities, Delegation
  Service, GSI-OpenSSH, MyProxy

14
Non-Technology Projects

• Distribution Projects
• Globus Toolkit Distribution
• Process was used for April 4.0.2 4.0.3 releases
• Documentation Projects
• GT Release Manuals
• Incubation Projects
• Incubation management project
• And any new projects wanting to join

15
Globus Toolkit Distribution Project

• Creates the official Globus Toolkit distributions
  
• Integrates a select group of Globus technologies
• Users include members of the eScience community
  and system integrators around the world

16
GT Release Manual Project

• Provides formalized documentation for the
  components in the official GT distribution
• Goal to provide comprehensive information
• Clear concise manner
• In language that is not too technical or obscure
• Specific guides for system administrators,
  developers and users
• Public Interface Guides and Quality Profile
  Guides
• Documentation (starting with GT4.0) is written in
  DocBook XML and output in HTML and PDF

17
Incubator Process in dev.globus

• Entry point for new Globus projects
• Incubator Management Project (IMP)
• Oversees incubator process form first contact to
  becoming a Globus project
• Quarterly reviews of current projects
• Process being debugged by Incubator Pioneers
• http//dev.globus.org/wiki/Incubator/
  Incubator_Process

18
Incubator Process (1 of 3)

• Project proposes itself as a Candidate
• A proposed name for the project
• A proposed project chair, with contact info
• A list of the proposed committers for the
  project
• An overview of the aims of the project
• An overview of any current user base or user
  community, if applicable
• An overview of how the project relates to other
  parts of Globus
• A summary of why the project would enhance and
  benefit Globus.

19
Incubator Process (2 of 3)

• IMP meet, discuss, and accept project as a
  ProtoProject
• ProtoProject now part of the Incubator framework
• Get assigned a Mentor to help
• Member of IMP
• Bridge between Globus and new ProtoProject
• Opportunity to get up to speed on Globus
  Development process

20
Incubator Process (3 of 3)

• Quarterly reviews by IMP determine
• Stay a ProtoProject
• Retire
• Escalate to a full Globus project
• Escalation when ProtoProject passes checklist
• Legal
• Meritocracy
• Alignment/Synergy
• Infrastructure

21
Current Incubator Projectsdev.globus.org/wiki/Wel
come Incubator_Projects

• Dynamic Accounts
• GridShib
• GridWay
• gt-hs
• Metrics
• MEDICUS

• OGCE
• PURSE
• Servmark
• Cog Workflow
• Virtual Workspaces

22
Current Incubator Projectsdev.globus.org/wiki/Wel
come Incubator_Projects

• Dynamic Accounts
• GridShib
• GridWay
• gt-hs
• Metrics
• MEDICUS

• OGCE
• PURSE
• Servmark
• Cog Workflow
• Virtual Workspaces

23
GridWay MetaScheduler

• Ignacio M. Llorente, Universidad Complutense
  Madrid
• Automatically perform job scheduling steps
• Provide the runtime mechanisms needed for
  dynamically adapting execution
• Runs over pre-WS GRAM, WS-GRAM, MDS2, MDS4
• Basic scheduling, flexible policies

24
Virtual Workspace Project

• Kate Keahey, ANL
• Virtual Workspace
• Abstraction of an execution environment
• Dynamically available to authorized clients
• Abstraction captures
• Resource quota for execution environment on
  deployment
• CPU or memory share)
• Software configuration aspects of the environment
  
• OS installation or provided services
• Workspace Service allows a Grid client to
  dynamically deploy and manage workspaces

25
Dynamic Accounts

• Kate Keahey, project chair
• Additional work has been done to provide basic
  services for creating dynamic accounts
• Service to allow a Grid client to dynamically
  assign Unix accounts on a remote resource
• Based on PKI credentials and the authorization
  information they carry

26
GridShib

• Von Welch, NCSA
• Allows the use of Shibboleth-transported
  attributes for authorization in GT4 deployments
• And, more generally, SAML support
• 2 year project started December 1, 2004
• Beta software released September 16, 2005

27
Globus Toolkit Handle SystemIntegration

• Frank Seibenelist, ANL
• The Handle System
• CNRI (http//www.handle.net)
• General-purpose global name service
• Secure name resolution over the internet
• The Handle System-GT Integration Project
• Leverage the Handle System for identifier and
  resolution services
• Tight integration with GT4s Web services
  protocols

28
Metrics

• Lee Liming, ANL
• Measure the use of Globus software
• Quantity issues
• how much is the software being used
• by how many people
• how those people are distributed
• Quality issues
• how the software is being used
• how useful it has been
• what the results of that use have been

29
OGCE

• Marlon Pierce, U Indiana
• Builds computational Grid and science gateway Web
  ports, Grid client tools, and supporting software
• Provides client environments for the Globus
  Toolkit 4, Storage Resource Broker, and Condor
• Allows for the integration of collaboration tools
  and services, such as those provided by the Sakai
  project.

30
ServMark

• Catalin Dumitrescu, University Chicago
• Set of performance evaluation tools for Globus
  and Grid software
• DiPerF
• Ability to test services in a distributed and
  scalable way
• GrenchMark
• Allow users to generate and run dynamic test
  workloads with complex structure.

31
MEDICUS

• Stephan Erberich, Childrens Hospital LA
• Medical Imaging and Computing for Unified
  Information Sharing
• Federate medical imaging and computing resources
  for clinical and research applications.

32
PURSe

• Rachana Ananthakrishnan
• Portal-based User Registration Service (PURSe)
• Provides a set of tools for automating user
  registration and credential management,
  especially for portal-based systems
• Set of customizable components to manage the full
  lifecycle of credential management

33
And youll hear about some others next!
34
How Can You Contribute?

• Create a new project
• Do you have a project youd like to contribute?
• Does your software solve a problem you think the
  Globus community would be interested in?
• Contact incubator-committers_at_globus.org for more
  information about how your project can become
  part of dev.globus, and you can help grow the
  Globus Community

35
Contribute to an Existing Project

• Contribute code, documentation, design ideas, and
  feature requests
• Joining the mailing lists
• -dev, -user, -announce for each project
• See the project wiki page at dev.globus.org
• Chime in at any time
• Regular contributors can become committers, with
  a role in defining project directions.

36
How Can You Contribute?

• Ask and answer questions
• Each project has a set of mailing lists set up
  for users and developers
• See the project wiki pages at dev.globus.org for
  additional information.
• Track progress
• All dev.globus projects have a roadmap in
  bugzilla where feature requests and bugs can be
  entered, commented on, and tracked by anyone in
  the community.
• For more information on contributing see the
  links at
• dev.globus.org/wiki/How_to_contribute

37
(No Transcript)
38
Some Useful Links

• Incubator Management Project (IMP)
• http//dev.globus.org/wiki/Incubator/Incubator_Man
  agement
• Incubation Process
• http//dev.globus.org/wiki/Incubator/Incubator_Pro
  cess
• Current incubator projects
• http//dev.globus.org/wiki/WelcomeIncubator_Proje
  cts
• Basics on Dev.globus
• http//dev.globus.org/wiki/Guidelines

39
Thanks To

• IMP Committers
• Charles Bacon
• Ann Chervenak
• Lisa Childers
• Neil Chue Hong
• Ian Foster
• Jen Schopf (chair)

• Infrastructure Support
• Charles Bacon
• Cris Williams
• Apache, for their process

40
For More Information

• Jennifer M. Schopf
• jms_at_mcs.anl.gov
• www.mcs.anl.gov/jms
• This talk available online shortly
• Support from NSF, DOE, NeSC/JISC

41
Outline

• Jennifer Schopf
• dev.globus and the incubator process
• Kate Keahey
• Virtual Workspace Project and Dynamic Accounts
  Project
• Ignacio M. Llorente
• GridWay MetaScheduler
• Von Welch
• GridShib
• Questions