TDC 369 TDC 432 - PowerPoint PPT Presentation

1 / 61
About This Presentation
Title:

TDC 369 TDC 432

Description:

How many servers does Amazon.com need to provide average search times of less than 10 seconds? ... 3 disk accesses avg. 50 ms. per access. AnyCo Web Server ... – PowerPoint PPT presentation

Number of Views:151
Avg rating:3.0/5.0
Slides: 62
Provided by: gregbr
Category:
Tags: tdc | avg | com | free | help | math

less

Transcript and Presenter's Notes

Title: TDC 369 TDC 432


1
TDC 369 / TDC 432
  • Spring 2003
  • Greg Brewster
  • DePaul University

2
Agenda
  • Course Overview
  • What is performance analysis?
  • Methods for performance analysis
  • Measurements
  • Simulation
  • Analysis

3
Course Goals
  • Study analytical tools for performance analysis
    of web-based computer networks and systems.

4
Course Text
  • Required Capacity Planning for Web Services
    Metrics, Models and Methods, Menasce Almeida,
    Prentice-Hall, 2002.
  • Excellent coverage of performance analysis issues
    for web services and web servers
  • For more math Queueing Networks and Markov
    Chains Modeling and Performance Evaluation with
    Computer Science Applications, Bolch, Greiner, de
    Meer and Trivedi, Wiley, 1998.
  • A good reference book covering mathematics and
    analysis aspects of the course in more detail

5
Office Hours and Web Site
  • Course Web Site
  • http//dlweb.cs.depaul.edu
  • Class Times MW 1150 AM 120 PM
  • Ill be available in my office for 1.5 hours
    after each class
  • Office Hours MW 130 300 PM

6
Course Web Site and Handouts
  • Class presentation handouts (PPT slides) will be
    available on the web site
  • Monday class slides on web site by 300 PM on
    Friday
  • Wednesday class slides on web site by 800 AM on
    Tuesday
  • It is your responsibility to print out copies of
    PPT slides for your own note-taking before coming
    to each class session.

7
Performance Analysis
  • Our goal is to determine values for performance
    attributes of systems, such as
  • Delay (Response Time)
  • Elapsed time between two events
  • Throughput
  • Events per second sustained by a system
  • Utilization
  • Fraction of time a system resource is busy
  • Reliability
  • Fraction of time a system resource is available

8
Example Web Server Performance
  • Response Time
  • What is the delay between the time a user clicks
    on a hyperlink and the time the web page is
    displayed?
  • Throughput
  • How many web page accesses per second can the
    server support?
  • Utilization
  • What fraction of time is the server busy?
  • Reliability
  • How often is the server down?

9
Applications of Performance Analysis Results
  • Capacity Planning
  • Predicting the resources needed to provide some
    Quality of Service in an existing system
  • Systems Design
  • Evaluate design tradeoffs for components of a
    system to be constructed
  • Systems Research
  • Quantify the value of proposed new protocols, etc.

10
Sample Applications
  • How many servers does Amazon.com need to provide
    average search times of less than 10 seconds?
  • How much faster would DePauls web access be if
    we doubled the bandwidth to our ISP?
  • Can performance of the TCP protocol be improved?

11
Capacity Planning
  • Capacity Planning Question Given current system
    performance and demand, how much can the demand
    increase before
  • the system becomes saturated
  • Saturation occurs when some vital system resource
    nears exhaustion example network utilization gt
    95.
  • or some critical system service level is exceeded
  • Example user response time exceeds 5 seconds

12
Capacity Planning Example
  • AnyCo runs an Intranet server which currently
    handles about 60 requests per second and gives
    acceptable performance.
  • Management would like to hire more employees,
    which will increase requests/sec.
  • Key Service Level Server response time must be
    kept under 8 seconds

13
AnyCo Reponse Time Analysis
14
Response Time Results
  • AnyCo finds that the web server, as currently
    configured, can handle up to about 150 requests
    per second before response times become
    unacceptable.
  • Most response time graphs have a knee value for
    the graph, where response time starts to increase
    rapidly with increasing load
  • For this example, the knee of the curve occurs at
    around 130 requests/second.

15
Server CPU Utilization Analysis
16
Utilization Results
  • CPU utilization defines the fraction of the time
    that the CPU is busy processing requests.
  • AnyCo finds that the web CPU utilization
    increases linearly with requests up to about 190
    requests per second, when it reaches maximum
    utilization (100).
  • This suggests that CPU utilization represents a
    bottleneck resource for this system and that
    increasing the server CPU speed (or installing
    multiple processors or multiple servers) would be
    necessary to increase system capacity beyond 190
    requests per second.

17
AnyCo Throughput Analysis
18
Throughput Results
  • Results show that all requests are processed
    (that is, throughput offered load) up to around
    185 requests per second.
  • Past 185 requests per second, the system starts
    to drop requests (throughput lt offered load),
    indicating that the system capacity has been
    exceeded.

19
Performance Analysis Approaches
  • Systems Measurement
  • Simulation
  • Analytical Modeling

20
Comparison
From Tools for Designing Data Networks, Al
Edlund, Data Communications Review, Dec. 1999.
21
Systems Measurement
  • Idea Directly measure the performance of an
    existing physical system
  • Tools
  • Hardware analyzers
  • Network sniffers
  • Metered software

22
Systems Measurement
  • Advantage
  • Highly accurate performance results
  • Disadvantages
  • System must exist and be measureable
  • Cannot measure proposed systems or modifications
    to existing systems
  • Measurement may interfere with system operation
  • Generalizeable? Results may apply only to
    particular time period measured

23
Measurements Marketplace
  • Measurement Tools
  • Traffic generators, Sniffers, probes, etc.
  • Network General, Computer Associates, Microsoft,
    etc
  • Measurement Services
  • Companies will generate traffic to test your web
    site
  • Gomez, Keynote Systems, Matrix NetSystems

24
Simulation
  • Idea Utilize simulation software that calculates
    progressive system states and measures
    resulting performance values
  • Tools
  • Simulation packages, e.g. COMNET
  • Custom software

25
Simulation
  • Advantages
  • Can measure performance of systems that do not
    yet exist
  • Accurate results if specifications known
  • Disadvantages
  • Software is complex to write and debug
  • Simplifying assumptions are typically required
  • Simulation must be run for thousands or millions
    of iterations for accurate results, which may
    require hours of simulation execution time.

26
Simulation Marketplace
  • Simulation Packages
  • NS (network simulator)
  • Free software from Berkeley
  • OPNET
  • www.opnet.com
  • COMNET
  • www.compuware.com

27
Analytical Models
  • Idea Represent system performance via
    mathematical model that can be solved directly
  • Tools
  • Probability Theory
  • Queueing Theory
  • Model solution packages, e.g. GreatSPN, PMVA
  • Mathematical software, e.g. MatLab or Mathematica

28
Analytical Models
  • Advantages
  • Can analyze systems that do not yet exist
  • Results are derived much more quickly compared
    with simulation
  • Disadvantages
  • May be difficult to determine appropriate models
    to use
  • Simplifying assumptions are used (more than
    simulation) to ensure that solution is
    mathematically feasible

29
Analysis Marketplace
  • Special purpose software packages exist
  • MMNK software for queue delay analysis
  • MVA software for queueing network analysis
  • Petri Net software packages
  • Markov chain solvers
  • Once a model is understood, some modeling and
    analysis can be done with basic tools, such as
  • Excel
  • Mathematica
  • Companies will sell you analysis services
  • NetForeCast.com and others

30
Example AnyCo Web Server
  • AnyCo runs a web server and would like to
    support up to 100 users performing transactions
    on this server at any one time
  • Key criterion User response time must be less
    than 8 seconds

31
AnyCo Web Server Example
32
AnyCo Web Server
  • AnyCo workload characterization
  • Each active user reads a web page for an average
    of 5 seconds before clicking a new hyperlink
  • HTTP Request message length averages 200 bytes
  • HTTP Responses (web pages) average 40,000 bytes
  • Each request requires
  • 80 ms. of Web server CPU time
  • 3 disk accesses avg. 50 ms. per access

33
AnyCo Web Server Systems Measurement Approach
  • AnyCo sets up test terminals that generate
    transactions on the web server.
  • They generate traffic equivalent to 1, 10, 20,
    30, , 100 users using test terminals and measure
    response times.
  • Each set of measurements is repeated many times
    to ensure a statistically significant result.

34
AnyCo Measurements
35
AnyCo Web Server Example Systems Measurement
Approach
  • Result Response time exceeds 8 seconds for more
    than 60 users
  • Follow-Up
  • Look for causes of unacceptable performance
  • Detailed server code measurements
  • Network utilization measurements
  • Problem How to predict impact of any system
    modifications?

36
AnyCo Web Server Systems Measurement Approach
  • Effort and Expense are High
  • Test measurement software must be purchased and
    configured on test workstations
  • Server must be off-line and available for
    measurement
  • Network Sniffer or probe must be installed to
    capture performance information

37
AnyCo Web Server Simulation Approach
  • AnyCo creates a detailed simulation of their web
    server system.
  • Key system parameters are used as inputs to the
    simulation
  • Data packet sizes, overheads
  • Router delays,
  • CPU time required for various operations
  • Setting up / writing simulation may take many
    hours or days depending on software used and
    complexity of simulation.

38
AnyCo Web Server Simulation Approach
  • Assumptions
  • Delay components
  • ISP router, T1 access line, AnyCo router,
    Ethernet, Web Server CPU time, Web Server disk
    access times, Ethernet, AnyCo router, T1 access
    line, ISP router.
  • Since we do not know details of user access to
    the Internet, we do not include delay for
    transporting data over user access lines
  • Router processing times negligible (only include
    data transmission times)
  • Ethernet transmission times negligible

39
AnyCo Web Server Simulation Approach
  • Simulation is now run
  • Each point on performance graph requires one
    simulation run.
  • Simulation run must iterate thousands or millions
    of times to get a statistically valid result.
  • Each simulation run may typically take 10 minutes
    to one hour to run.

40
AnyCo Web Server Simulation Model
Example Graphical model, simulation and results
analysis using COMNET software.
41
AnyCo Web Server Simulation Approach
  • Model Accuracy How accurately does this
    simulation model capture the behavior of the real
    system??
  • Must be addressed via model validation study
    where simulation results are compared against
    measured system results.
  • Follow-Up After examining simulation results, we
    can modify parameters and do what-if analysis for
    system changes.

42
AnyCo Web Server Analysis Approach
  • AnyCo sets up a simple analytical model of their
    web server system.
  • Key system parameters are used as inputs to the
    simulation
  • Data packet sizes, overheads
  • Router delays,
  • CPU time required for various operations
  • Model can typically be set up quickly, as it only
    captures the few essential performance attributes
    of the system

43
AnyCo Web Server Analysis Approach
  • Assumptions
  • Delay components
  • ISP router, T1 access line, AnyCo router,
    Ethernet, Web Server CPU time, Web Server disk
    access times, Ethernet, AnyCo router, T1 access
    line, ISP router.
  • Since we do not know details of user access to
    the Internet, we do not include delay for
    transporting data over user access lines
  • Router processing times negligible (only include
    data transmission times)
  • Ethernet transmission times negligible
  • There are some restrictions on distributions of
    service times and request routing (details later
    in course)

44
AnyCo Web Server Analytical Approach
  • Analytical Solution package is run
  • Each point on performance graph requires one
    solution of an analytical model.
  • Each analytical model solution is exact (just 1
    iteration) and takes about 0.1 second.
  • All graph data generated in less than 1 second!!

45
Analytical Models
  • The fundamental building block for many
    analytical models is Queueing Theory

46
Queueing Theory
  • Queueing Theory deals with the behavior of jobs
    arriving to queues connected to servers.

47
Queue Operations
  • Jobs arrive according an arrival process, A(t).
  • Arriving jobs are placed in queue
  • Server provides service to first job in queue,
    other jobs wait their turn.
  • Service time required by each job determined by
    service process, S(t)
  • Each job leaves queue after receiving service.

48
AnyCo Web Server Example
49
AnyCo Queueing Network Model
50
AnyCo Web Server Queueing Network Modelmore
details later in course!!
  • Each time a user clicks a hyperlink
  • Request message sent by ISP router across T1
    (0.001 s) and across Ethernet
  • Web server processes request using 3 disk
    accesses (0.05 s. each) interspersed with 4 CPU
    time slots (0.02 s. each 0.08 s.)
  • Response message sent across Ethernet and by
    AnyCo router across T1 (0.02 s)
  • User examines result for average of 5 sec before
    making next request.

51
AnyCo Web Server Sample of PMVA analysis input
file
  • network
  • servers
  • users is li 5.0000,
  • isprtr fcfs li 0.0010,
  • corprtr fcfs li 0.2080,
  • servercpu ps li 0.0200,
  • serverdisk fcfs li 0.0500
  • routing
  • users gt isprtr gt servercpu,
  • servercpu gt serverdisk (0.7500) corprtr
    (0.2500),
  • serverdisk gt servercpu,
  • corprtr gt users
  • end

52
AnyCo Analytical Model Results
53
AnyCo Analytical Model Results
54
AnyCo Web Server Analytical Approach
  • Result of Analysis The ISP Access Line is
    overloaded!!
  • Solution Increase Bandwidth on Access Line
  • Current ISP Access T1 (1.536 Mbps)
  • New Model Dual T1 (3.072 Mbps)

55
AnyCo Analytical Model Results
56
AnyCo Web Server Analytical Approach
  • Modified System (with Dual T1 Access Line)
  • Performance is greatly enhanced.
  • System can now handle up to 85 users before
    exceeding 8 second response time

57
AnyCo Analytical Model Results
58
AnyCo Web Server Analytical Approach
  • Result of Analysis
  • ISP Access Line is no longer a bottleneck
  • At high loads, the Web Server Disk is overloaded
  • Possible Solution Upgrade Server Disk drive
  • Current Disk 50 ms. per disk access
  • Upgraded Disk 30 ms. per disk access

59
AnyCo Analytical Model Results
60
AnyCo Web Server Analytical Approach
  • Modified System
  • Performance is greatly enhanced.
  • System can now easily handle the required 100
    users with calculated average user response time
    of around 4 seconds.
  • Recommendations to AnyCo
  • Increase bandwidth on ISP access line
  • Upgrade server disk drive

61
AnyCo Web Server Analytical Approach
  • Key Question How accurately does this simple
    analytical model capture the behavior of the real
    web server system??
  • Must be addressed via model validation study
    where analytical model results are compared
    against measured system results.
  • Note even if model is not perfectly accurate, it
    can still help to predict performance trends.
Write a Comment
User Comments (0)
About PowerShow.com