EAS308 Application Server Benchmarks And Sizing

1
EAS308 Application Server Benchmarks And Sizing
Prashanth Ponnachath Software Engineerpponnach_at_sy
base.com / 925 236 6286 08/05/2003
2
Session Objectives

• Benchmarks
• What is a benchmark and why perform it
• How to create a credible App Server benchmark
• Faulty benchmarks practiced in the industry
• Examples of well designed credible benchmarks and
  their metrics

3
Session Objectives

• Sizing
• Build and understand a sizing model that can be
  used on any application server
• Learn techniques for improving performance in an
  application server environment

4
Definitions

• Benchmark
• Entities that perform or claim to perform similar
  functions are compared
• A standard by which something can be measured or
  judged
• Measure (a rival's product) according to
  specified standards

5
Definitions

• Application Server
• Container that provides middleware services to
  server-side components
• Examples Enterprise JavaBeans .Net, Power
  Builder, and CORBA Common Object Request Broker
  Architecture

6
Calling Life Cycle

• Typical E Business Application

Business Logic
Presentation
Data
7
Definitions

• Application Server
• Runtime environment for the deployed components
• Business logic in a typical 3 tier architectural
  model
• Perform complex database transactions
• Support Web Services

8
Definitions

• Web Server

9
Definitions

• Web Server
• Delivers Web pages or HTML pages to clients
• HTTP protocol is used
• Some Web Servers handle dynamic content (JSP,
  ASP, Servlets, PowerDynamo)

10
7 Characteristics Of Highly Effective Benchmarks

• Understand core functionalities of the product
• Design workload to evaluate core functionality
• Performance workload must simulate real world
• Neutrality Fidelity to real-world business
• Openess Should work on all entities being tested
• Proper metrics should be designed
• Represent results in a clear and concise manner

11
Need For Credible App Server Benchmarks

• Many Application Sever Vendors in the market
• Many Vendor benchmarks that are misleading
• App server investment is crucial. An App Server
  that is 25 better requires 25 less hardware
• Knowing scalability ahead of time is crucial

12
Core functionalities of App Server

• High availability
• -How many simultaneous users?
• Handle complex server side transactions
• - Entity beans, Session Beans
• Handle database transactions

13
Stress Tests And Performance Tests

• Stress Test
• Maximum number of users an server can handle
  Example Online book store can support 2000 users
  
• Depends on hardware and software used
• Performance Test
• Specific metrics measured for a finite load
• Response time throughput are examples

14
Flawed Benchmark

• Testing dynamic content without component access

15
Flawed Benchmark

• Dynamic content without component access
  (continued)
• Testing only partial functionality
• Test application has no server side components
  like EJBs
• Written in JSP
• Web server and the servlet engine being tested

16
Flawed Benchmark

• Dynamic content without component access
  (Continued)
• Analogous to comparing cars by testing battery
  life
• Testing web server and servlet engine
• Application Server investment is a waste
• Web server with servlet engine is sufficient

17
Flawed Benchmark

• Testing a static HTML page

18
Flawed Benchmark

• Testing a static HTML page (continued)
• Test application consists of static html pages
• Stress test by sending http requests to
  application server
• Only the Web Server is tested

19
Flawed Benchmark

• Testing a static HTML page (continued)
• Comparing cars by testing the attached music
  systems
• Not testing engine performance, transmission etc
• Application Server investment is a waste
• Web Server would serve the purpose

20
Credible App Server Benchmarks
21
Credible Benchmarks

• Test application requirements
• Full-fledged e-business application that includes
  EJBs, servlets and JSPs
• Work on all the application servers and on all
  versions
• Simulate a real world environment

22
Credible Benchmarks

• Requirements
• Test all EJBs , servlets and JSPs deployed
• Perform complex database transactions
• Test the object caching, JSP caching and
  connection pooling

23
Examples Of Credible Benchmarks

• SPECjAppServer benchmark
• Developed by the Standard Performance Evaluation
  Corp
• New industry-standard benchmark
• Measures the scalability and performance of
  application servers.

24
Examples Of Credible Benchmarks

• SPECjAppServer benchmark (continued)
• SPECjAppServer2001 tests the EJB container in a
  J2EE 1.2 server
• J2EE application modeling real world scenario
• Workload models a manufacturing, supply-chain
  management, and order/inventory environment

25
SPECjAppServer Workload

• Business Model A Real World Business Model

26
SPECjAppServer Workload

• Application Architecture Full Fledged E
  Business Application

27
Examples Of Credible Benchmarks

• Performance results influenced primarily by
• Hardware configuration
• J2EE application software The test application
  containing server side components
• Application Server and Database Software

28
Examples Of Credible Benchmarks

• SPECjAppServer benchmark metrics
• BOPS (business operations per second)
• -No of customer order transactions No of
  manufacturing work orders / measurement period in
  seconds
• /BOPS
• The price/performance of the system under test
  (including hardware, software, and support)

29
Examples Of Credible Benchmarks

• SPECjAppServer benchmark results
• EAServer achieved record-breaking price and
  price/performance results
• EAServer achieved a record 202.12 BOPS_at_DualNode
• EAServer achieved the best overall
  price/performance, 332.73/BOPS_at_DualNode

30
Examples Of Credible Benchmarks

• Nile.com Benchmark
• Developed by Doculabs
• Bookstore called Nile.com (like amazon.com)
• Written with EJBs, JSPs, and Servlets
• Million rows of customers and books

31
Examples Of Credible Benchmarks

• Nile.com Benchmark Metrics Throughput

32
Sizing

• Sizing

33
What We Need to Know For Sizing

• Looking for Peak Load
• Determine the number of users at Peak Load
• Determine how many hits will occur
• Size for sudden bursts of activity

34
Pre-Sizing Chart

• Estimates in the planning stage

35
Determining Peak Load

• Calls from users do NOT come in exactly spaced
• Studies have shown that chaos flow patterns
  apply to calls from web users on average

36
Calling Life Cycle

• Basic Calling Cycle

App Server Web Server
37
Calling Life Cycle - Time View

• Calling Cycle in Time

Time
38
Calling Life Cycle Origination

• Client initiates a call

Time
39
Calling Life Cycle Call Received

• Call is received by server

Time
40
Calling Life Cycle Call Received

• Server Begins Processing

Time
41
Calling Life Cycle Processing Begins

• Web Application Starts Working

Web App Processing
Time
42
Calling Life Cycle NVO Called

• NVO is called by Web Application

NVO Pre-DB Processing
Time
43
Calling Life Cycle - DB

• Communication with the DB is established

Database Call
Time
44
Calling Life Cycle Data Processing

• Data is returned and processing begins

NVO Processing
Time
45
Calling Life Cycle Display Processing

• When the data gets back to the JSP, display
  processing starts

JSP Processing
Time
46
Calling Life Cycle Return Data to Client

• And finally, the data is returned to the client.

Time
47
Deciphering The Timing Data

• Lets break out what we need

1. Remove all the network lag time
Time
48
Deciphering The Timing Data (2)

• Lets break out what we need

1. Remove all the network lag time
2. Remove the Database time
Time
49
Deciphering The Timing Data (3)

• What is left is actual CPU time.

T1jsp(ms)
T2jsp(ms)
T1nvo(ms)
T2nvo(ms)
Time
50
More Formulas

• Which generate in to formulas..

T1jsp(ms) T2jsp(ms) Totaljsp(ms)
T1nvo(ms) T2nvo(ms) Totalnvo(ms)
Totaljsp Totalnvo Totaloverall
51
Sizing the Timing to the CPUs

• Unfortunately, total time does not equal real
  time
• Overhead by the Operating System
• Overhead by EAServer
• Small Fluctuations in timing by tools
• Network Response times
• Context Switches

52
Timing Chart for Intel CPUs
53
Examples of Using the Timing Chart

• Example
• If we had a process, and we had timed it as
  taking .085 ms for JSP and .166 ms for NVO
• .085 ms .166 ms .251 ms Total Time
• On a single CPU machine, that would be
• .86 ms total CPU / (.251 ms/call)
  3.42 calls per actual second

54
Tips for Reducing Response Times

• Database Connection Pools
• Reduce the number of rows returned
• Accessing Resources
• Accessing the pool itself
• Allowing the pool to self test(known as sanity
  checking)
• Database CPU is expensive
• Stored Procedures can reduce overhead

55
Tips for Reducing Response Times

• Establishing Connections to Other Resources
• Home Object can be stored
• DataStores can be pre-read from the disk, and
  loaded with data when necessary
• Connections can (sometimes) be kept and re-used,
  rather than re-requesting them

56
Tips for Reducing Response Times

• Use Good Programming Techniques
• Only go to the Expensive database once!
• Keep data cached that can be stored
• Watch for logic that is always being run that is
  not necessary
• Good Programming Flow Improves Performance Timing

57
SDN Presents CodeXchange
A New Benefit for Sybase Developers

• Forum for exchanging samples, tools, scripts,
  etc.
• New features enable community collaboration
• Download samples created by Sybase or external
  users
• Leverage contributions of others to extend Sybase
  products
• Contribute code or start your own collaborative /
  open source project with input from other product
  experts
• Any SDN member can participate
• Log in using your MySybase account via SDN
• www.codexchange.sybase.com
• Or via SDN at www.sybase.com/developer
• SDN CodeXchange at TechWave
• Visit SDN in the Technology Boardwalk
• Learn about CodeXchange in the Sybase Booth
  Theater

58
Questions.