the ebusiness management company

1

• the e-business management company

SPIN March 21, 2000
2
Program Mission
eConfidence

Unmatched expertise to deliver e-business
reliability solutions
3
The Risk Site Availability and Performance
Threatened by Constant Change

• New Functionality
• Web Front-end
• Web and App Server Logic
• Front- and Back-office Systems
• Inter-business Systems
• Business Conditions
• Traffic Spikes from Planned Promotional/Seasonal
  Activities or Unplanned Market Conditions

• Technology
• Browser Versions
• Web/App Server Versions
• Load Balancing Introduction/Changes
• Middleware Versions
• Security Tightening
• ERP Versions
• DBMS Versions
• Front-office Development Tool Versions
• OS Version Changes
• New Hardware

4
The Result

• Delayed Rollout of Valuable Improvements
• Loss of Revenue
• Loss of Credibility
• Business Consequences of Poor Site Availability/
  Performance
• Loss of Market Share
• Missed/Delayed Cost Savings
• Sanctions from Regulators

5
e-Business Reliability Testing Needs
Web Clients
. . .
Observing
CORBA
EJB
Monitoring
RMI
Tuxedo
Metering
. . .
Performance
6
The Solution Run your e-business with
eConfidence

• Regression Tools
• Configuration of Functional Testing for Browsers
• Configuration of Functional Testing for Browser
  UI Components
• Creation of Test Harness/Test Suite
• Implementation of Transactions
• Performance Tools
• Customization and Configuration of Load Testing
  Solution and interaction with Web Server
• Implementation of Load Tests to Determine Site
  Capacity and Performance Degradation Points

7
The Solution Run your e-business with
eConfidence

• Benefits of Test Automation
• Cost Savings
• Time to Market Savings
• Repeatability
• Testing Breadth and Depth

8
Real World Solutions
To be successful, you need

• A modular approach to component and functional
  testing - connect the dots
• Tools that meet the diverse demands of a mix of
  technologies for web and client/server
  applications
• Support for test planning and management - too
  many chefs in the kitchen?

9
Functional Testing ToolsKey Feature - Support
for a Mix of Technologies
Feature - support for all of the major
platforms, browsers, and development
environments. Benefit - use one testing tool for
functional and regression testing
of many components
Windows NT Windows 2000 Windows 95,
98 UNIX 3270/5250
Internet Explorer Navigator
C/C Java Visual Basic ActiveX
10
Functional Testing ToolsKey Feature - Automated
Testing
Feature - automated testing Benefit - maximize
productivity by letting your testing tool do
the work for you

• Choose recording, scripting, or both to create
  tests
• Reuse testcases with a data driven approach
• Run unattended tests overnight using a recovery
  system and still get meaningful results
• Schedule and Drive your scripts from a central
  point of control, even on different platforms

11
Functional Testing ToolsKey Feature - Test
Planning and Management
Feature - test planning and management Benefit -
plan, coordinate, control and report on
group testing efforts

• Centralize and modularize your test plan with a
  master plan and sub-plans
• Include manual tests for progress and completion
  reports
• Query your plan to flag and run only those
  testcases that meet specified criteria
• Prepare reports on testing progress, including
  graphs

12
Functional TestingKey Feature - Direct Database
Verification
Feature - direct database verification Benefit -
test the complexity of your database without
relying on the user interface

• Begin database and black box testing before the
  user interface is ready
• Validate database components that have no UI,
  such as stored procedures, Remote Procedure
  Calls, triggers, and data replication
• Connect to any ODBC compliant database
• Connect to a local database host or to a
  networked server database

13
Functional Testing
Feature - The more the tool can do, the
better Benefit - Spend less time managing tools,
more time interpreting results

• Repository for storing and managing results
• Defect tracking integration or communication
• Link checking
• Environment should be user friendly

14
The Technical Challenge of E-BusinessHow To Load
Test This?
Simulate Impact of Multi-Network Components
Simulate Traffic of Client/Server Components
Simulate Impact of Web Components
Simulate Integration of Packaged Applications
Traffic
Load Testing Tool
15
Load Testing Feature - Multiple
Protocol/Environment Testing
Feature - Support Multiple Protocols and
Environments Benefit - Enables testing web,
client- server, database, and ERP systems

• Test the integration of multiple applications on
  one common resource (i.e. Database)
• Support all major Web Protocols (HTTP, HTTPS,
  FTP, POP3, SMTP, SOCKS, LDAP, TCP)
• Support Database Load-Testing for ODBC and
  popular Databases (Oracle, SQL Server, etc.)
• Support Popular Middleware Systems and Protocols
  (CORBA, Tuxedo, MQ Series)

16
Load TestingKey Feature - Results Reporting
Feature - Allow you to extract from your load
testing accurate, useful information Benefit -
Use this information to understand current
scalability and plan for future needs
Display your results in multiple formats and
compare run-to-run results.
Display only the system and results information
you want
17
Load Testing Key Feature - Ease of Use
Feature - Ease of Use Benefit - Comprehensive
creation, execution and customization
settings are available

• Record tests quickly and easily
• Scripting language allows customization
• Monitor script, system and results information in
  real-time
• Setup tests that make use of error limits, custom
  timeout settings and external data
• Customize users to emulate real-world settings
  of connection speeds, browser-types and think
  times

18
Load Testing Key Feature - Realistic load
generation
Feature - Load generation as real as
possible Benefit - Accurate loads create accurate
statistics for intelligent analysis and
conclusions

• Create multi-user/multi-transaction load tests
• Include IP address simulation
• Modem speed simulation
• Replay secure traffic
• Real simulation of browser cache
• Full integration of server side statistics and
  custom statistics

19
eConfidence in Production

• Reliability Plan Specific to Business and
  Technology Plans for the Site
• Metrics for the Site's Current Functionality and
  Performance
• Implementation of Functionality and Performance
  Tests Specified in the Reliability Plan
• Periodic reports on reliability and change
  metrics
• Recommendations for Ongoing Production Reliability

20
eConfidence in Production

• Baseline and Plan Development
• Establish a health status on the sites current
  functionality, uptime and performance
• How much of the site currently works
• Response times for typical activity and
  transactions
• Site performance break points
• Develop a specific reliability plan
• Reports baseline findings
• Identify risks of your business and technology
  plans
• Point-by-point plans for addressing risk

21
eConfidence in Production

• Reliability Plan Implementation
• Develop and run scenarios
• Diagnose findings
• Share results with technology owners and users
• Monthly/quarterly reports on reliability and
  change metrics

22
Market Drivers
Significant investments in e-Business applications
demand results

• The best website in your space, driven by market
  conditions and technology
• Reliability across multi-tiered, technologically
  complex system
• Maximum return on your QA investment

23
24 Hour Site Monitor
What does this tell me about the overall
performance of my web site?
24
24 Hour Site Monitor With Competitors
How does my web site compare to other sites?
25
Transaction Comparison
How was the time spent?
26
Where To Now? Testing...
How do I get accurate results?

• Browser Simulation
• Threads, Browser Specific Content (DHTML,
  ActiveX, etc.)
• Connection Speed Simulation
• Business Users (T1) versus Home Users (Modems,
  ISDN)
• Workload Type
• Steady-State Performance Metrics
• Increasing Workload Server Scalability, Breaking
  Point
• Multiple Machines/Multiple Addresses Per Machine
• Distributed Testing, IP Address Simulation
• Randomization Of Data
• Uniform, Individual, Binomial Distributions
• In Script or From File

27
How do I find problems with my site?
Errors as number of users crosses over 3200
users
error rate
28
Server Busy Time
How is response time affected by number of users?
29
Another View Of Response Time
What does this tell us about our servers
scalability?
30
Run Some Tests, Make Some Changes...
How did tuning my server affect its performance?
31
Real World Solutions
To be successful, you need

• To ensure your applications scalability -
  Downtime and Limited Capacity mean lost business
• Support for the e-Business development model -
  You MUST validate that newly introduced
  applications DO NOT create bottlenecks for
  existing applications
• Tools which meet the diverse demands of varied
  users responsible for Web application development
  and deployment

32
Summary Run your e-business with eConfidence

• Faster site improvement and change adoption with
  minimized risk
• Improved decision-making on sites ability to
  meet business plans
• Ingrained reliability awareness and methodology
• Improved team skills and intra-team processes
• Focus on business goals
• Time/Cost savings