Test Automation ROI - PowerPoint PPT Presentation

Loading...

PPT – Test Automation ROI PowerPoint presentation | free to download - id: 43f473-OTA1Y



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Test Automation ROI

Description:

Test Automation ROI Webinar Presented by Dion Johnson – PowerPoint PPT presentation

Number of Views:1008
Avg rating:3.0/5.0
Slides: 28
Provided by: DJ160
Learn more at: http://www.stickyminds.com
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Test Automation ROI


1
Test Automation ROI
  • Webinar
  • Presented by Dion Johnson

2
Introduction Importance of ROI
  • ROI is as important if not more important than
    technical automation topics
  • Test Automation is an investment

3
Agenda
  • ROI Challenge
  • ROI Definition
  • ROI Factors
  • Calculating ROI
  • Improving ROI
  • Conclusion

4
ROI Challenge
UNREALISTIC EXPECTATIONS
  • 100 Automation
  • Immediate Payback
  • No Automation Analysis

5
ROI Dilemma
  • Continued automation justification
  • Difficult to meet unrealistic expectations
  • You MUST Redefine expectations in terms of true
    test automation benefits

Increased Cost Savings
Increased Efficiency
Increased Quality
6
ROI Definition
  • ROI Return on Investment
  • The calculation for ROI is the return from an
    action divided by the cost of that action, where
    the return is equal to the cost subtracted from
    the gain

ROI Gains Investment Costs
ROI Investment Costs
7
ROI Gains
  • Gains are equal to the cost of the old process
    that we are eliminating
  • MANUAL

8
ROI Investment
  • Investment is equal to the cost of the new
    process that we are adding
  • AUTOMATION

9
Calculating ROI Factors
  • General Testing Factors

1
2
3
Total of Functional Test Cases
Tester Hourly Rate
Expected of Runs
10
Calculating ROI Factors
  • Manual Testing Factor

1
Manual Test Execution/ Analysis Time
11
Calculating ROI - Factors
  • Automation Testing Factors
  • Tool Costs
  • Software
  • License
  • Execution Machines
  • of Tests to Automate
  • Automated Development/Debugging Time (average per
    test)
  • Automated Test Execution Time (average per test)
  • Automated Test Analysis Time
  • Automated Test Maintenance Time

Fixed
Variable
12
Calculating ROI Estimating Automation
  • If ROI Calculation is required prior to beginning
    automation, the following variable factors must
    be estimated
  • Automated Development/Debugging Time (average per
    test)
  • Automated Test Execution Time (average per test)
  • Automated Test Analysis Time
  • Automated Test Maintenance Time

13
I have to Estimate?
But HOW?
14
Calculating ROI Estimating Automation
  • Automated Development/Debugging Time

3 to 10 times manual execution time
10
3
Simple Tests/Structured Framework
Complex Tests/Unstructured Framework
15
Calculating ROI Estimating Automation
  • Automated Test Execution Time

1/5 of the manual execution time
16
Calculating ROI Estimating Automation
  • Automated Test Analysis Time

¼ of the automated test execution time
17
Calculating ROI Estimating Automation
  • Automated Test Maintenance Time

4 to 8 hours per run/build
18
Calculating ROI
Simple
Efficiency
Risk
19
Calculating ROI Simple ROI
Identifies savings related to
  • Gain (Manual Calculation)
  • Manual Test Execution Time Number of Test to Be
    Automated of Test Runs Tester Hourly Rate
  • Investment (Fixed Variable Calculation)
  • S of Fixed Automation Costs (Automated Test
    Development Time Automation Test Execution Time
    of Test Runs Automated Run Analysis Time
    of Test Runs Maintenance Time of Test
    Runs) Tester Hourly Rate

- Note Automation Execution Time doesnt have to
be included if using dedicated machines
20
Calculating ROI Simple ROI
  • PROS
  • are easy to communicate to upper management
  • CONS
  • May be oversimplified and misleading
  • Asserts that automated tests completely replace
    manual tests
  • Asserts that project budget will decrease as
    opposed to just being reallocated
  • Assumes full regression is done each build

21
Calculating ROI Efficiency ROI
Identifies savings related to Time
  • Gain (Total Manual Time Calculation)
  • Manual Test Execution Time Total Number of
    Functional Tests of Test Runs
  • Investment (Variable Auto Time Calculation
    Remaining Manual Time Calculation)
  • (Automated Test Development Time Automation
    Test Execution Time of Test Runs Automated
    Run Analysis Time of Test Runs Maintenance
    Time of Test Runs) Time For Executing
    Remaining Manual Tests

22
Calculating ROI Efficiency ROI
  • PROS
  • Removes focus from misleading figures
  • CONS
  • May still be oversimplified and misleading
  • Asserts that automated tests completely replace
    manual tests
  • Asserts that project budget will decrease as
    opposed to just being reallocated
  • Assumes full regression is done each build
    Doesnt address increased coverage as a return

23
Calculating ROI Risk Reduction ROI
Identifies savings related to Quality
  • Gain (Potential Loss Calculation)
  • Amount of money lost if production defect occurs
  • Investment (Fixed Variable Calculation)
  • S of Fixed Automation Costs (Automated Test
    Development Time Automation Test Execution Time
    of Test Runs Automated Run Analysis Time
    of Test Runs Maintenance Time of Test
    Runs) Tester Hourly Rate

Risk
- Note Automation Execution Time (shown in bold)
doesnt have to be included if using dedicated
machines
24
Calculating ROI Risk Reduction ROI
  • PROS
  • Eliminates some of the improper comparisons
    between manual and automated tests
  • Addresses advantage of test coverage
  • CONS
  • Guessing the Gain is error prone
  • Relies heavily on subjective information
  • Requires a degree of risk analysis that is not
    typical in real-world projects
  • Doesnt show any of the benefits automation has
    over manual test execution
  • Gives the impression that automation will cure
    poor testing processes

25
Improving ROI
Improvement
Result
Put more time into developing a framework and repeatable automation process Reduced automated test development and maintenance
Increase understanding of tool features and how they may be used effectively Reduced automated test development and maintenance
Increase understanding of application functionality and dynamic object behaviors Reduced automated test maintenance
26
Improving ROI
Improvement
Result
Increase Modularity Reduced automated test maintenance
Exception handling Reduced automated test analysis
Improved reporting mechanisms Reduced automated test analysis
Greater focus on process automation (set-up procedures, data manipulation, test analysis and reporting) Reduces time spent on processes associated with Functional Testing
27
Conclusion
  • There are three prominent ROI calculations
  • Simple
  • Efficiency
  • Risk
  • Each has Pros and Cons
  • Use ROI calculations for assessing automation
    expectations and returns
About PowerShow.com