jGMA: A lightweight implementation of the Grid Monitoring Architecture

1
jGMA A lightweight implementation of the Grid
Monitoring Architecture

• Mat Grove
• Distributed Systems Group
• University of Portsmouth
• http//dsg.port.ac.uk/mjeg/

2
Outline

• Motivation.
• An overview of GMA.
• Implementation jGMA.
• jGMA performance.
• Summary and future work.

3
Motivation

• GridRM is
• A generic resource monitoring system basically
  a framework for gathering data from heterogeneous
  data sources.
• A scalable means of monitoring resources
  distributed over the Grid.
• Written in Java, using commons standards
• Agent API, Servlets, XML, etc.
• GridRM needed a standards-based transport layer
  to bind together its sites.
• A GMA-based transport appears best candidate.

4
An Overview of the Grid Monitoring Architecture

• The GGF GMA is the recommended means for
  monitoring within the Grid.
• The GMA document (2001) recommends
• A publish, subscribe and bind architecture.
• An implementation that is
• Secure and scalable.
• Has a low latency and high data rate.
• Imposes minimal impact.
• GMA compliancy
• Approximately 20 features.
• GGF document is only a guide.

5
GMA Components

• Registry

• Register/Publish

• Register/Search

• Bind

• Producer

• Consumer

• Events

6
Some GMA Implementations

• Embedded implementations
• For example, Globus MDS
• MDS3 planned to be standalone system.
• Standalone implementations
• Berkeley National Lab pyGMA.
• Datagrid R-GMA.
• None of these meet systems meet GridRM
  requirements
• Compatible API (Java).
• Minimal dependencies.
• GMA compliant.
• Led us to develop jGMA.

7
jGMA Components

• jGMA has the following components
• Registry.
• Producer, Consumer (shared code-base).
• PC Servlet.
• jGMA features
• Small API (lt20 calls).
• No dependencies.

8
jGMA Implementation
9
jGMA Performance

• Optimising LAN-based performance
• In the first instance point-to-point
  communication
• Need both blocking and non-blocking I/O.
• Interested in latency and throughput.
• Scalability of jGMA producer(s)/consumer(s)
• Single consumer/producer multiple producers or
  consumers.
• Some testing issues
• Java timer resolution
• Java only has clock with millisecond resolution,
  JNI native clock is required.
• Configuration of the JVM.

10
jGMA Performance (Cont)

• Optimise jGMA I/O
• Initial results were not satisfactory
• Moved from RMI to Sockets.
• Improve performance by
• Minimising the number of times data is copied.
• Moved from Java objects to byte arrays.
• Improved message queue handling.

11
Test Configuration

• Configuration
• One Consumer and One Producer.
• Same machine (Localhost) and Fast Ethernet.
• Vary message sizes.
• Standard Ping Pong test.
• Designed to measure
• Message latency of jGMA I/O.
• Maximum throughput.

12
Message Latency Results

• Time (Microseconds)

256k

• Message Length (Bytes)

13
Message Throughput Results

• Bandwidth (Mbytes/S)

• Message Length (Bytes)

14
Scalability Test Configuration

• Configuration
• One Consumer multiple Producers
• Could be other way around and get same results.
• Over Fast Ethernet.
• Selected Message Sizes appropriate for GridRM.
• Designed to measure
• Scalability of Consumer/Producer setup.
• Maximum Consumer/Producer message consumption.
• Producer/Consumer stability under extreme loading.

15
Scalability Results

• Messages Per Second

• Number Of Producers

16
Summary and Future Work

• Summary
• jGMA is a reference implementation of GMA in
  Java.
• jGMA is a simple, scalable and robust system.
• Our early tests show that jGMA has good
  performance.
• Future developments
• Optimisation of I/O for WAN communications
• HTTP/Objects
• Implement a persistent registry
• File.
• Xindice.
• Deployment with GridRM.

17
Acknowledgements

• University of Portsmouth,
• Member of the DSG
• Mark Baker,
• Hong Ong,
• Garry Smith,
• Mr Boe-akis,
• Aamir Shafi.

18
Information

• Project Web page
• http//dsg.port.ac.uk/projects/jGMA/
• The DSG Web page
• http//dsg.port.ac.uk/
• GridRM
• http//gridrm.org/
• Related links
• GGF http//www.gridforum.org/
• R-GMA http//www.r-gma.org/