Globe

1
(No Transcript)
2
Globus

• Grid Middleware Toolkit
• Otto Sievert
• CSE 225
• 8 June, 2000

3
Globe

• and other European Grid Activities
• Otto Sievert
• CSE 225
• 8 June, 2000

4
EGRID

• European Grid Community
• Collaborative community, not a standards group
• Commercial and Academic Interests
• www.egrid.org

5
European Tour

• Netherlands
• Germany
• Poland
• Italy
• Sweden

6
(No Transcript)
7
(No Transcript)
8
(No Transcript)
9
(No Transcript)
10
Grid Theme 1

• Be very (very) careful when choosing a project
  name.

11
Amsterdam Globe

• Vrije Universiteit
• Maarten van Steen
• Andrew Tanenbaum
• Middleware to facilitate
• large-scale distributed applications
• Web focus
• object-based coherency

12
Globe Uniqueness

• Too much data, too little resources (bandwidth,
  etc.)
• Caching helps
• Data Coherency integral to Cache Policy
• Release constraint of a single replication/
  distribution policy for all objects
• example web pages

13
Globe Object

• Physically Distributed
• Replicated
• Distribution Policy

14
Globe Local Object

• 4 subobjects (minimum)
• Modularity

15
Globe Binding

• 1. Name server
• 2. Object handle
• 3. Location service
• 4. Contact points
• 5. Choose point
• 6. Repository
• 7. Protocol
• 8. Bind!

16
Legion Binding

• Two-stage name resolution
• Binding agent
• No implementation repository

17
Autopilot Binding

• 1. Sensor registers with the sole manager
• 2. AP client requests sensors from the manager
• 3. Manager returns available sensors
• 4. Client and sensor communicate directly

18
Globe Claim Remote Object Model Lacks Replication

• Globe
• objects can be replicated
• still maintain one state
• allows complex coherency policies

• Legion
• in theory, supports replication
• replicated state
• allows some but not all coherency policies
• in practice is not allowed

19
Globe Architecture

• Why all these servers?
• separate naming from locating
• allow extensible binding protocols (?)

20
Grid Theme 2 Communication

• Centralized
• NWS
• Globus MDS
• Simple Management
• Single Point of Failure

• Distributed
• NetSolve
• Frans Sensor Net
• Complex Management
• Scalable
• Mixed
• Legion
• Globe
• Autopilot

21
Globe Implementation Example

• Set of HTML/image/ Java files
• One semantics subobject
• browsers arent that extensible, so use gateway

gateway
http//globe.foo.edu8989/dir/file.html
browser
22
Globe Example (contd)

• Replication Policies
• Object-based
• Permanent store
• Object-initiated store
• Client-based
• Client-initiated store
• How is this any better than what we have now?

23
Globe Live Demo ...
24
Globe Location Service

• Scalability Questions
• Tree Heirarchy
• again Legion-like in its worst-case behavior
• Typical soln assumes mobile client
• Globe soln assumes mobile software

25
Does This Work?

• Single experiment - 5 wk. web server trace
• compare
• no caching
• various complex replication/coherency policies
• automatic adaptive policy
• Results
• (essentially) any replication scheme wins big
• individual object adaptivity didnt perform well

26
Globe Conclusion

• Explicit coherency is clearly a Good Thing
• Security?
• Representative implementation?

27
Germany Cactus

• Albert Einstein Institute, Potsdam
• Thomas Radke
• Ed Seidel
• Distributed Astrophysics
• Software Engineering
• NCSA hot code

28
Cactus

• Separate CS from disciplinary science
• Flesh CS architecture
• Thorns science modules
• 2-stage compilation
• encapsulation
• modularity
• reuse

29
Cactus Compilation

• Two stage
• Permanently bind thorns Perl
• Compile binary C/F77
• Efficient
• Dont carry unneeded thorn info

30
Grid Theme 3 Applications

• Numerical, or Non
• Computation vs. Specialization
• Performance Measures
• Execution Time
• Scale
• Efficiency
• Distribution

31
Grid Theme 4 Transparency

• Ease of Use vs. High Performance
• As system becomes opaque, EoU increases
• As system becomes opaque, Perf decreases
• Where is the balance?

32
Germany Gridware

• 1999 San Jose-based merger of two companies
  Genias GmbH and Chord (U.S.)
• CoDINE
• Compute farm load balancing system
• Recently adopted by Sun
• PaTENT
• WinNT MPI

33
Grid Theme 5 Commoditization

• Reuse is strong in the Grid
• Resources (Beowulf)
• Middleware (Globus, PaTENT)
• Applications (Cactus)
• Industry is influential
• Largest grid apps in use today are commercial
• Grid-ernet is profitable

34
To This Point ...
35
Germany Unicore

• UNIform Computer Resources (German SC access)
• Goal is to provide uniform access to high
  performance computers - painful to learn
• OS details
• Data storage conventions
• Administration policies
• 3 phase project
• I self-contained jobs
• II remote data access
• III concurrent remote execution

36
Unicore (contd)

• How is this done?
• Web (Java) user interface
• X.509 authentication
• Network Job Supervisor
• interprets Abstract Job Objects
• manages jobs and data
• interfaces with local batch systems (like
  LoadLeveler and CoDINE)
• vs. Globus?

37
Poland POL-34

• National Grid
• Very like the system used by Unicore, a
  collection of widely-distributed parallel
  computers
• Tree-connected ATM network

38
POL-34

• Yellow 2 Mb/s
• Red 34 Mb/s
• Cyan 155 Mb/s
• Single administrative domain via Virtual Users
  (skirting the grid issue)
• Use Load Sharing Facility (LSF)

39
Italy SARA

• University of Lecce, Italy Giovanni Aloisio (with
  Roy Williams of Caltech)
• Synthetic Aperature Radar Atlas
• Distributed data-intensive app
• Alan Su and the UCSD AppLeS group is involved

40
SARA Architecture

• The goal easy, fast, efficient retrieval and
  processing of SAR image data
• Issues
• data is distributed, stored in tracks
• complex hierarchical system
• Prototypical grid app

41
Sweden Computational Steering

• Parallelldatorcentrum Royal Institute of
  Technology Per Oster
• Using Globus and the Visualization Toolkit (VTK)
  to steer a single CFD code.
• Little data available
• Eclipsed by Autopilot

42
Grid Theme 6 Heterogeneity

• Some attempt to hide it
• Globus, CORBA, Java/Jini
• Some take advantage of it
• Netsolve, Ninf
• Some characterize and manage it
• AppLeS (SARA), NWS

43
To This Point ...
44
Conclusion

• Explored Globe, Cactus, and other EuroGrid
  favorites in the context of
• Communication architectures
• Grid application characteristics
• Grid transparency
• Commodity computing influence
• Grid heterogeneity

45
Network Weather Service (NWS)

• Rich Wolski, U. Tenn.
• Monitors and Predicts Grid Resources
• network latency, bandwidth
• CPU load, avail. Mem.
• Central NWS data server
• nws.npaci.edu/NWS