Summer 2002 MDS 2 Performance Evaluation Xuehai Zhang Dept of CS The University of Chicago 82902

1
Summer 2002MDS 2 Performance EvaluationXuehai
ZhangDept of CSThe University of
Chicago8/29/02
2
Outline

• Motivation
• Performance topics and metrics
• Experiments setting up
• Performance results and analysis
• MDS re-inspection
• Conclusion and future work

3
Project Motivation

• Performance awareness is important to a GRID
• Why exploring MDS 2?
• Another candidate R-GMA
• To give leverage to system improvement
• Few related work has been done

4
Overview of MDS Architecture

• Key Components
• Information Provider, GRIS, GIIS

IP
IP
Resource B
Resource A
IP
IP
IP
User 1
IP
Users 1 and 2 request infodirectly from
resources.
User 2
User 3 uses GIIS for searching collective
information.
GIIS requests information from GRIS services as
needed.
User 3
GIIS Cache contains info from A and B
VO 1
5

Interesting Performance Topics

• Capacity
• The limit of concurrent users a GRIS can support
  (absolute performance based)
• The limit of concurrent users a GIIS can support
  w/ different caching mechanisms, etc.
• Scalability
• The limit of information providers a GRIS can
  support
• The limit of GRIS a GIIS can support, etc.
• Others
• The effect of underlying hardware on the
  GRIS/GIIS performance
• The effect of using different query techniques
  (JNDI Grid-info-search) on the GRIS/GIIS
  performance,etc.

6

Performance Metrics

• Overheads
• load1, load5, CPU-user, CPU-system, etc
• Users
• The MDS clients and the issuers of MDS queries
• Query Interval
• The pause time between two continuous queries
  from one user
• Response Time
• The total time required for a MDS query operation
• Throughput
• The average number of queries served per second
  by a GRIS or a GIIS

7
Experiments

• Will explain in the latter slides

8
Experiment Setting Up

• Testbeds
• lucky nodes _at_ ANL
• edelweis.cs.northwestern.edu
• Gt2.0 deployment
• Ganglia
• Server setting up
• Use software-based simulation
• limitation
• Client codes
• C code Grid-info-search script
• Java code JNDI api

9
The Performance of GRIS

• Global configuration
• GRIS server
• Lucky7.mcs.anl.gov2235
• Client machines simulating users
• Up to 20 Linux boxes at Univ of Chicago
• Up to 50 processes per machine

10
GRIS Experiment1 GRIS performance vs.
Concurrent Users

• Goal
• Experimental Configuration
• GRIS server
• default MDS configuration
• User(Client)s query
• based on Grid-info-search
• worst-case search
• Query interval
• includes 0.5 sec, 2 sec, 5 sec, 10 sec, 20 sec

11
GRIS Experiment1 (contd)

• Throughput Results

12
GRIS Experiment1 (contd)

• Response Time Results

13
GRIS Experiment1 (contd)

• Load1 Results

14
GRIS Experiment1 (contd)

• CPU-User Results

15
GRIS Experiment1 (contd)

• CPU-System Results

16
GRIS Experiment1 Result Analysis

• GRIS has a fair performance to support a large
  number of users
• Throughput Response time increase with more
  users
• Threshold constraint
• Why?
• Caching mechanism helps
• How is without-caching?
• Better performance in real case
• Could be worse if support more IP.

17
GRIS Experiment2 GRIS performance vs.
Information Providers

• Goal
• Experimental Configuration
• GRIS server
• default MDS configuration
• User(Client)s query
• based on Grid-info-search
• worst-case search
• Information Providers
• new memory info provider
• include default , 10, 50, 80
• Users up to 500 w/ query interval 0.5 sec

18
GRIS Experiment2 (contd)

• Throughput Results

19
GRIS Experiment2 (contd)

• Response Time Results

20
GRIS Experiment2 (contd)

• Load1 Results

21
GRIS Experiment2 Result Analysis

• GRIS NOT performs well with a large number of
  registered information providers
• Response time increases dramatically with more
  information providers
• Ideal information provider
• No larger than a number in 20, 60
• Better performance in real case
• Could be worse with realtime IP.

22
GRIS Experiment3 GRIS performance vs.
Underlying Hardware

• Goal
• Experimental Configuration
• GRIS servers
• an extra GRIS at edelweis.cs.northwestern.edu2235
  
• both use default MDS configuration
• User(Client)s query (same as 1,2)
• Users up to 900 w/ query intervals 0.5 sec
• Hardware difference How

23
GRIS Experiment3 (contd)

• Overhead Results

24
GRIS Experiment3 (contd)

• CPU Performance Results

25
GRIS Experiment3 (contd)

• Throughput Response Time Results
• Sorry, they are missing

26
GRIS Experiment3 Result Analysis

• The performance of GRIS closely depends on the
  underlying hardware
• A server with more advanced hardware buys better
  performance of the hosted GRIS
• CPU capability
• For an in-memory directory, the CPU is a
  significant bottleneck. Using dual processors
  improves performance by 40.
• from Measurement and Analysis of LDAP
  Performance by X. Wang, H. Schulzrine, D.
  Kandlur, D. Verma

27
GRIS Experiment4 GRIS performance vs. Query
Techniques

• Goal
• Experimental Configuration
• GRIS servers
• use default MDS configuration
• User(Client)s query techniques
• use JNDI api
• use Grid-info-search
• Users up to 700 w/ query intervals 0.5 sec

28
GRIS Experiment4 (contd)

• Throughput Results

29
GRIS Experiment4 (contd)

• Response Time Results

30
GRIS Experiment4 Result Analysis

• GRIS performs differently towards the queries
  sent by JNDI and Grid-info-search
• The difference gets slighter when users increases
• The overhead brought by ldapsearch is bigger

31
The Performance of GIIS

• Global configuration
• GIIS server
• lucky0.mcs.anl.gov2235
• registered GRIS servers the rest of lucky nodes
• Client machines simulating users
• Up to 20 Linux boxes at Univ of Chicago
• Up to 50 processes per machine

32
GIIS Experiment1 GIIS performance vs. Caching
Configuration

• Goal
• Experimental Configuration
• Two Caching Scenarios
• GRIS data always in cache
• GRIS data always not in cache
• GIIS server (adopt the above two cases)
• GRIS server lucky7.mcs.anl.gov2235
• User(Client)s query (same as GRIS Exp. 1,2)
• Users up to 500 w/ query intervals 2 sec

33
GIIS Experiment1 (contd)

• Throughput Results

34
GIIS Experiment1 (contd)

• Response Time Results

35
GIIS Experiment1 (contd)

• Load1 Results

36
GIIS Experiment1 (contd)

• CPU-User Results

37
GIIS Experiment1 Result Analysis

• GIIS could not support large number of users
  without caching
• Response time goes over 1 min with 150 users
  without caching
• New problem will caching hurt data freshness?
• Need we get rid of the data provider role of
  GIIS?

38
GIIS Experiment2 GIIS performance vs.
Registered GRIS

• Goal
• Experimental Configuration
• GIIS server
• Use MDS default configuration
• GRIS servers
• Simulated by the rest of lucky nodes
• Up to 40 GRIS processes per node
• User(Client)s query (same as GIIS Exp. 1)
• Users up to 80 w/ query intervals 0.5 sec

39
GIIS Experiment2 (contd)

• Throughput Results

40
GIIS Experiment2 (contd)

• Response Time Results

41
GIIS Experiment2 Result Analysis

• GIIS could not support large number of GRIS
• Ideal number of registered GRIS for a GIIS should
  be less than several tens
• Directly affect the size of virtual organization

42
MDS Retrospection

• An Old Story Pull or Push or both?
• Push model overcomes Pull model
• Need to investigate R-GMA
• Scalability A Topic Inevitable
• Maximum concurrent connections supported by
  OpenLDAP
• Need we get rid of the data provider role of
  GIIS?
• Limitation of information providers supported by
  GRIS and GRIS supported by GIIS

43
Conclusion

• MDS 2 performance analysis and evaluation
• Based on experiments
• Relatively accurate, but achieve goals
• MDS 2 performance quality is cased-based
• The potential to improve MDS

44
Future Work

• More MDS 2 experiments
• To study the effect of the security factor on the
  performance of GRIS/GIIS
• To study the effect of wide-area networking on
  the performance of GIIS and compare with the
  local-networking case.
• To study the MDS performance in a multiple
  hierarchical environment, like 3 levels.
• The performance evaluation of R-GMA and the
  comparison with MDS 2

45
Info

• Email hai_at_cs.uchicago.edu
• Homepage people.cs.uchicago.edu/hai
• Advisor Dr. Jennifer Schopf

46
Hardware Difference
Back