Tycho: A General Purpose Virtual Registry and Asynchronous Messaging System

1
Tycho A General Purpose Virtual Registry and
Asynchronous Messaging System
Matthew Grove ACET Invited TalkFebruary 2006
2
Outline

• Introduction and motivation,
• The architecture of Tycho,
• Implementation details,
• Benchmarking,
• Developing services for online games,
• Summary,
• Demo.

3
Introduction

• Tycho is a reference implementation of a combined
  extensible wide-area messaging framework with a
  built in distributed registry.
• Tycho follows the publish-subscribe paradigm.
• It provides services to allow produces and
  consumers to discover each other using a Virtual
  Registry (VR) made up of a network of mediators
  and then aid communication over both LAN and WAN.
• Tycho aims to simplify and speed application
  development by freeing developers from the need
  to use a combination of middleware to provide
  service discovery and messaging.

4
Design Philosophy

• Reuse existing software components, if possible,
  rather than reinvent functionality.
• Try to make use of existing infrastructure.
• Make Tycho simple to install, configure and use.
• Provide a basic release with a choice to extend
  functionality with a further more sophisticated
  component.

5
The Two Parts of Tycho

• Message passing
• Provides asynchronous communications between
  Consumers, Producers and Mediators.
• Virtual Registry
• Boot-strapping Allow Tycho mediators to discover
  each other and form the VR with minimal
  hardwiring.
• Communications Efficient routing of queries
  between Tycho Virtual Registries.
• Caching Keeping a temporary local copy of some
  information to reduce the amount of
  communications between peers.

6
Tychos Architecture
7
Tycho Implementation

• Java was used for portability and
  interoperability with other distributed systems,
  plus rapid development.
• The message passing services were implemented
  first as some of their functionality is used by
  the VR
• An initial prototype was developed over a few
  days using Java RMI.
• This has been iteratively optimised and
  redeveloped to reduce latency and memory use,
  currently we use byte arrays for all data and
  Java sockets for communications.

8
The Tycho Library and API

• We provide a choice of implementations for each
  core service.

9
Tycho Core Services

• Transport handler, allows different protocols to
  be used for communications (HTTP, Sockets, IRC).
• Local store, for a mediator and VR information
  (JDBC, Java simple store).
• Boot service, used by the VR within a mediator to
  locate and join the rest of the VR (HTTP, IRC).
• Query Parser and result annotator, to support
  different query and markup languages (SQL, LDIF).

10
Example Tycho Setup
11
Tycho Benchmarks

• We performed two rounds of benchmarking
• Message Passing These tests measured the
  performance of inter-client and inter-mediator
  communications for Tycho and NaradaBrokering.
• Virtual Registry These tests measured the
  performance of the VR (query routing and
  processing) in a range of different
  configurations

12
Message Passing Tests (A)

• We measured the latency of communication for LAN
  and simulated WAN messaging between two clients
  varying message size (ping-pong tests).
• Then we measured the affect on scalability of
  message latency using many producers publishing
  to a single consumer and single consumer
  consuming from multiple producers.
• We used the DSG cluster to run the tests.

13
Ping-Pong - Latency
Latency vs Message Length
14
Message Passing Tests - Summary

• Tycho achieves comparable or lower latency than
  NaradaBrokering in all of these tests.
• The performance of both systems does not drop off
  after a producer or consumer has become
  saturated.
• NaradaBrokering needs the JVM heap size to be
  increased as the number of clients increases (due
  to internal buffers)
• Tycho used the default heap for all of the tests.

15
Virtual Registry Tests (B)

• We used 4 different configurations to measure the
  performance of the Tycho VR.
• For each test we varied one or more aspect of the
  configuration and measured the effect on client
  query latency
• Number of records in a mediator (100,000
  records),
• Number of simultaneous client queries (1000
  clients),
• Numbers of mediators in the VR (1000 mediators),
• Different VR-interconnects between mediators.
• In order to make some comparisons between Tycho
  and R-GMA, MDS4, MDS2 and Hawkeye we attempted to
  reproduce tests used in a related paper.

16
Virtual Registry Tests - Summary

• The Java simple store performs best for small
  numbers of records, but HSQLDB and MySQL scale
  better with registry size.
• The response time increases linearly with number
  of clients.
• For routing queries in the VR, IRC has better
  performance than HTTP.
• As response size increases HTTP performs much
  better than IRC, this is due to the IRC code
  having to send multiple messages.

17
Comparison Tests

• Aims to compare Tychos VR to R-GMA, MDS4, MDS2
  and Hawkeye.
• The tests measure the impact of increasing the
  number of clients simultaneously querying a
  registry containing 10 records.
• The set up was similar to a benchmark we had
  already performed but we altered some things to
  make it as close a match as possible
• We used 5 compute nodes instead of 8.
• Each system has different sized responses which
  we matched by adding padding to Tycho registry
  information.
• The main difference is CPU (2.8 Ghz vs 1.1Ghz).

18
Comparison Tests (2K)
Response Time vs Number of Clients (2 Kbyte
Responses)
19
Comparison Tests (10K)
Response Time vs Number of Clients (10 Kbyte
Responses)
20
Comparison Tests (19K)
Response Time vs Number of Clients (19 Kbyte
Responses)
21
VR Comparison Tests Summary

• Tycho shows linear increase in response times
  regardless of message size and number of clients.
• Tycho had lower response times than R-GMA, MDS4
  and MDS2 without caching.
• MDS2 with caching is basically OpenLDAP - which
  is mature and performs well.
• Hawkeye performs better than Tycho for less than
  100 clients, but it does not scale well with
  larger number of clients.

22
Tycho Core Future Work

• Some performance improvements
• Caching of local mediator queries to reduce
  response times,
• The Simple store could be improved by making it
  more intelligent,
• We could use a hybrid VR-interconnect to use IRC
  for query routing and HTTP for transporting large
  responses.
• More functionality can be added to provide more
  advanced services
• HTTPS and SSL Sockets for secure communications,
• Support for transactions.

23
Tycho Applications

• We feel strongly that it is necessary to develop
  real world applications to further validate the
  implementation.
• A number of increasingly sophisticated
  applications have been developed and implemented
• Publishing and discovering Web-cams,
• Remote resource discovery for the VOTechBroker
  project,
• Binding components of the Slogger project
  together to publish system logs for analysis
  using Semantic Web technologies.
• An area that will stress and probably push any
  message-oriented middleware to its limits is
  online gaming.

24
Distributed Gaming

• Online distributed gaming has become increasingly
  popular with the wide spread uptake of broadband
  in homes.
• Games publishers have each tried to provide an
  infrastructure to support their online games.
• There is an opportunity to develop a standard set
  of services, based on a completed Tycho
  implementation, to support these games.

25
Distributed Gaming Services

• Discovery The player requires a scalable service
  to index large numbers of game servers and allow
  searches based on user-defined parameters such as
  the current map being played.
• Content distribution Scalable infrastructure to
  provide new or updated content from the content
  provider to the players.
• Single sign-on Providing a unique identity for
  each player makes global high score tables and
  other statistic tracking possible
• User profiles (key bindings) could be stored
  server side to allow roaming.

26
Summary

• The initial reference implementation has been
  completed.
• Both the message-passing code and VR have been
  benchmarked and appears to perform well.
• The focus now is to review the current state of
  online game services and select a game framework
  to enhance using Tycho-based services.

27
Webcam Browser Demo

• http//dsg.port.ac.uk/projects/tycho/demos/web/

28
Links

• Project Web page
• http//dsg.port.ac.uk/projects/tycho/
• The DSG Web page
• http//dsg.port.ac.uk/