OSG Integration - PowerPoint PPT Presentation


PPT – OSG Integration PowerPoint presentation | free to download - id: 46a60f-MmQ4Y


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation

OSG Integration


Welcome to CW 2007!!! The Condor Project (Established 85) Distributed Computing research performed by a team of ~40 faculty, full time staff and students who face ... – PowerPoint PPT presentation

Number of Views:39
Avg rating:3.0/5.0
Slides: 30
Provided by: R318
Tags: osg | integration | lisa | mona


Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: OSG Integration

Welcome to CW 2007!!!
The Condor Project (Established 85)
  • Distributed Computing research performed by a
    team of 40 faculty, full time staff and students
  • face software/middleware engineering challenges
    in a UNIX/Linux/Windows/OS X environment,
  • involved in national and international
  • interact with users in academia and industry,
  • maintain and support a distributed production
    environment (more than 4000 CPUs at UW),
  • and educate and train students.

  • Since the early days of mankind the primary
    motivation for the establishment of communities
    has been the idea that by being part of an
    organized group the capabilities of an individual
    are improved. The great progress in the area of
    inter-computer communication led to the
    development of means by which stand-alone
    processing sub-systems can be integrated into
    multi-computer communities.

Miron Livny, Study of Load Balancing Algorithms
for Decentralized Distributed Processing
Systems., Ph.D thesis, July 1983.
A good year for the principals and concepts we
pioneered and the technologies that implement
In August 2006 the UW Academic Planning Committee
approved the Center for High Throughput Computing
(CHTC). The LS College created to staff
positions for the center
Main Threads of Activities
  • Distributed Computing Research develop and
    evaluate new concepts, frameworks and
  • Keep Condor flight worthy and support our users
  • The Open Science Grid (OSG) build and operate a
    national High Throughput Computing infrastructure
  • The Grid Laboratory Of Wisconsin (GLOW) build,
    maintain and operate a distributed computing and
    storage infrastructure on the UW campus The NSF
    Middleware Initiative
  • Develop, build and operate a national Build and
    Test facility powered by Metronome

Later today Incorporating VM
technologies (Condor VMs are now called
slots) and improving support for parallel
Downloads per month
Downloads per month
Software Development for Cyberinfrastructure (NSF
07-503) Posted October 11, 2006
  • All awards are required to use NMI Build and Test
    services, or an NSF designated alternative, to
    support their software development and testing. 
    Details of the NMI Build and Test facility can be
    found at http//nmi.cs.wisc.edu/.

Later today Working with RedHat on integrating
Condor into Linux
Miron Livny and Michael Litzkow, "Making
Workstations a Friendly Environment for Batch
Jobs", Third IEEE Workshop on Workstation
Operating Systems, April 1992, Key Biscayne,
Florida. http//www.cs.wisc.edu/condor/publicatio
  • 06/27/97 This month, NCSA's (National
    Center for Supercomputing Applications) Advanced
    Computing Group (ACG) will begin testing Condor,
    a software system developed at the University of
    Wisconsin that promises to expand computing
    capabilities through efficient capture of cycles
    on idle machines. The software, operating within
    an HTC (High Throughput Computing) rather than a
    traditional HPC (High Performance Computing)
    paradigm, organizes machines into clusters,
    called pools, or collections of clusters called
    flocks, that can exchange resources. Condor then
    hunts for idle workstations to run jobs. When the
    owner resumes computing, Condor migrates the job
    to another machine. To learn more about recent
    Condor developments, HPCwire interviewed Miron
    Livny, professor of Computer Science, University
    of Wisconsin at Madison and principal
    investigator for the Condor Project.

Why HTC?
  • For many experimental scientists, scientific
    progress and quality of research are strongly
    linked to computing throughput. In other words,
    they are less concerned about instantaneous
    computing power. Instead, what matters to them is
    the amount of computing they can harness over a
    month or a year --- they measure computing power
    in units of scenarios per day, wind patterns per
    week, instructions sets per month, or crystal
    configurations per year.

High Throughput Computing is a 24-7-365 activity
FLOPY ? (606024752)FLOPS
The Grid Blueprint for a New Computing
Infrastructure Edited by Ian Foster and Carl
Kesselman July 1998, 701 pages.
The grid promises to fundamentally change the way
we think about and use computing. This
infrastructure will connect multiple regional and
national computational grids, creating a
universal source of pervasive and dependable
computing power that supports dramatically new
classes of applications. The Grid provides a
clear vision of what computational grids are, why
we need them, who will use them, and how they
will be programmed.
  • We claim that these mechanisms, although
    originally developed in the context of a cluster
    of workstations, are also applicable to
    computational grids. In addition to the required
    flexibility of services in these grids, a very
    important concern is that the system be robust
    enough to run in production mode continuously
    even in the face of component failures.

Miron Livny Rajesh Raman, "High Throughput
Resource Management", in The Grid Blueprint for
a New Computing Infrastructure.
Later today Working with IBM on supporting HTC
on the Blue Gene
Taking HTC to the National Level
The Open Science Grid (OSG) Taking HTC to
the National Level
Miron Livny OSG PI and Facility
Coordinator University of Wisconsin-Madison
The OSG vision
  • Transform processing and data intensive science
    through a cross-domain self-managed national
    distributed cyber-infrastructure that brings
    together campus and community infrastructure and
    facilitating the needs of Virtual Organizations
    at all scales

OSG Principles
  • Characteristics -
  • Provide guaranteed and opportunistic access to
    shared resources.
  • Operate a heterogeneous environment both in
    services available at any site and for any VO,
    and multiple implementations behind common
  • Interface to Campus and Regional Grids.
  • Federate with other national/international Grids.
  • Support multiple software releases at any one
  • Drivers -
  • Delivery to the schedule, capacity and capability
    of LHC and LIGO
  • Contributions to/from and collaboration with the
    US ATLAS, US CMS, LIGO software and computing
  • Support for/collaboration with other
    physics/non-physics communities.
  • Partnerships with other Grids - especially EGEE
    and TeraGrid.
  • Evolution by deployment of externally developed
    new services and technologies.

Tomorrow Building Campus Grids with Condor
Grid of Grids - from Local to Global
Who are you?
  • A resource can be accessed by a user via the
    campus, community or national grid.
  • A user can access a resource with a campus,
    community or national grid identity.

Tomorrow Just in time scheduling with Condor
glide-ins (scheduling overlays)
OSG challenges
  • Develop the organizational and management
    structure of a consortium that drives such a
    Cyber Infrastructure
  • Develop the organizational and management
    structure for the project that builds, operates
    and evolves such Cyber Infrastructure
  • Maintain and evolve a software stack capable of
    offering powerful and dependable capabilities
    that meet the science objectives of the NSF and
    DOE scientific communities
  • Operate and evolve a dependable and well managed
    distributed facility

The OSG Project
  • Co-funded by DOE and NSF at an annual rate of
    6M for 5 years starting FY-07.
  • 15 institutions involved 4 DOE Labs and 11
  • Currently main stakeholders are from physics - US
    LHC experiments, LIGO, STAR  experiment, the
    Tevatron Run II and Astrophysics experiments
  • A mix of DOE-Lab and campus resources
  • Active engagement effort to add new domains and
    resource providers to the OSG consortium

Security Workflows Fire-walls Scalability Scheduli
Thank you for building such
a wonderful community
About PowerShow.com