Automated Software Testing - PowerPoint PPT Presentation

About This Presentation
Title:

Automated Software Testing

Description:

Title: Recommending a Strategy Last modified by: Elfriede Created Date: 6/2/1995 10:12:36 PM Document presentation format: Letter Paper (8.5x11 in) – PowerPoint PPT presentation

Number of Views:66
Avg rating:3.0/5.0
Slides: 41
Provided by: csGmuEdu56
Learn more at: https://cs.gmu.edu
Category:

less

Transcript and Presenter's Notes

Title: Automated Software Testing


1
Automated Software Testing
Welcome
2
Automated Software Testing
  • Elfriede Dustin
  • Latest book Implementing Automated Software
    Testing Mar 2009
  • Book The Art of Software Security Testing (Dec
    2006)
  • Book SAP Testing (Spring 2007)
  • Author of book Effective Software Testing (Dec
    2002)
  • Books Automated Software Testing and Quality
    Web Systems
  • www.idtus.com

3
Who is IDT?
  • IDT specializes in the design, development, and
    implementation of Automated Software Testing
    solutions
  • Implement turn key automated test suites
  • Training and pilot project implementation with
    companies
  • Research Development related to Automated
    Testing Technologies

4
Challenge The Software Landscape
Deliver More Capability, Faster and Cheaper
  • Innovative Software Technologies
  • Increased System Complexity
  • More Software to Test
  • Less Time
  • Fewer People

Our Ability to Test these Technologies has Not
Kept Pace with Our Ability to Create Them
5
More Time Than Ever is Spent on Software Testing
1990 Beizer half the labor expended to
develop a working program is typically spent on
testing activities. 2002 Hailpern and
Santhanam "... debugging, testing, and
verification activities can easily range from 50
to 75 percent of the total development cost. "
2008 Redmond Developer News Industry
analysts estimate that developers spend only
about 20 percent of their time designing and
coding ... the bulk of their time is spent on
resolving application problems"
6
The Opportunity
Utilize Automated Testing Strategies and
Technologies to Improve Productivity and Quality

Increased Test Coverage
Automate manual tests that consume the most
resources
Reduction in Test Days
Manual
Automated
Manual vs. Automated Test Coverage
7
Who Can Benefit?
Organizations and Projects Responsible for
Software Testing of Complex Systems
  • Regression testing
  • Manpower Intensive
  • Time Consuming
  • Expanded test coverage
  • Analysis for large quantities of test data
  • Performance and endurance testing
  • Compliance testing

8
Definition of Automated Software Testing
  • Application and implementation of software
    technology throughout the entire Software Testing
    Life Cycle (STL) with the goal to improve STL
    efficiencies and effectiveness

9
Why Automate?
  • Reduces the time and cost of software testing
  • Improves software quality
  • Enhances manual testing efforts via increased
    testing coverage and replaces the manually
    mundane and labor-intensive tasks
  • Does what manual testing can hardly accomplish,
    such as memory leak detection under specific
    conditions, concurrency testing, performance
    testing and more

10
Need for automated testing of mission critical
systems - Requirements
  • Cant be intrusive to system under test
  • Needs to be OS independent
  • Needs to be able to handle display and
    non-display centric automation
  • Needs to be able to handle multi-computer
    environment
  • Non-developers should be able to use the tool

11
Need for automated testing of mission critical
systems - Requirements
  • Cant be intrusive to system under test

IDTs answer Use of VNC Technology and Image
based automation
12
Need for automated testing of mission critical
systems - Requirements
  • Needs to be OS independent

IDTs answer Eclipse and plugin based, Java based
13
Need for automated testing of mission critical
systems - Requirements
  • Needs to be able to handle display and
    non-display centric automation

IDTs answer Capture/Playback Display based
automation and message based automated testing,
handling various protocols
14
Need for automated testing of mission critical
systems - Requirements
  • Needs to be able to handle multi-computer
    environment

IDTs answer Implementation handles
heterogeneous environments, connecting multiple
computers
15
Need for automated testing of mission critical
systems - Requirements
  • Non-developers should be able to use the tool

IDTs answer Keyword driven implementation
16
ATRT Addresses Unique Challenges of Mission
Critical Systems
Current Industry provided automated testing tools focus on IDTs unique ATRT solutions address
Web based applications Thick client, diverse applications and protocols
Windows Based Cross-platform (Linux, all flavors, Windows, etc.)
GUI Applications with Single Consoles Displays and message based testing, handling various protocols across many computers and consoles
Intrusive to System Under Test (SUT) Non-intrusive to SUT Uses VNC technology
17
ATRT Overview
  • Test Organization
  • Projects
  • Test Cases
  • Test Steps
  • Substeps .
  • Currently support 2 types of Test Cases
  • Display / GUI
  • Message

18
ATRT Display Automation Capture / Record
Playback
  • Enables automation of actions of the test
    engineer
  • During testing, the engineer uses the keyboard,
    mouse, touch panels, etc to perform actions
  • Testing tool captures actions and information
    from the screen, which are baselined in an
    automated test script
  • Input actions and expected results become the
    baseline
  • During test playback, compare latest outputs with
    baselined results.
  • Utilizes VNC technology to remotely connect to
    system under test

19
ATRT - Capture/Playback
  • ATRT Display Automation uses a keyword driven
    approach
  • User selects/clicks on a selection/image and a
    related keyword
  • ATRT generates relevant code behind the scene
  • User does not need to be a software
    developer/coder
  • Example Keywords used
  • Left-click Right-click Wait for Enter text .

20
Capture/Playback Illustration
  • Test Step
  • Open File

21
ATRT Capture Playback Illustration using
Keywords
  • ATRT Capture Action
  • Select File
  • Left Click

22
Capture / Record Playback ATRT generated
code based on Keywords
  • Keyword Left-click generates this code behind
    the scenes
  • try       put IDTImageRetryFunction("1", "1",
    "Image_1.tiff", "Click", "Image Image_1.tiff was
    found", "Image Image_1.tiff was not found")
  •  if ImageFound ("Image_1.tiff") then  put
    IdtLogLocal("1", "1", SUB_STEP_PASS, "Image
    Image_1.tiff was found")  Click
    FoundImageLocation() else                 wait
    3                if ImageFound ("Image_1.tiff")
    then          put IdtLogLocal("1", "1",
    SUB_STEP_PASS, "Image Image_1.tiff was found on
    2nd try")          Click FoundImageLocation()   
           else                        wait
    3                        if ImageFound
    ("Image_1.tiff") then                  put
    IdtLogLocal("1", "1", SUB_STEP_PASS, "Image
    Image_1.tiff was found on 3rd try")              
        Click FoundImageLocation()                 
    else
  •                   put IdtLogLocal("1", "1",
    SUB_STEP_FAIL, "Image Image_1.tiff was not
    found")                               
    CaptureScreen "1.1.error"                         
                                         put
    sendError("1.1.error.tiff")                      
                                       end
    if                  end if end if  
  • catch    put IDTException("1", "1")end try

23
File - Open
24
Image Comparison
  • Image results are compared to the baseline image
    previously captured
  • Eggplant is the COTS product we currently use for
    the image comparison
  • Features include
  • Image library
  • Wait for Image
  • Debug
  • Optical Character Recognition (OCR)

25
Other Features
  • Allow for Manual Steps
  • Real-time Status
  • Reporting
  • Summary
  • Detailed

26
  • Demo

27
Automated Test Strategy Components
ROI
Metrics
Design
Process
Strategy
Business Case
28
Business Case
  • The Business Case is the summation of the work of
    the work performed in the Automated Test
    Strategy.
  • It will layout each of the steps outlined in the
    following slides.
  • Begins with a current process assessment
  • Identify the keys to successfully implementing a
    program that will reduce risk and vulnerability.

29
Current Process Assessment
  • The Current Process Assessment reviews your
    current testing strategy to identify
  • Testing Coverage
  • Testing Coverage Requirements
  • Testing Limitations
  • Potential Risks

Testing Coverage Today
Testing Coverage Needed
30
Strategy
  • The Strategy section defines the plan on how to
    most cost effectively apply resources to
    attacking the problem
  • Not all tests can and should be done using
    automation.
  • This step will identify the best candidates and
    the logical order in which they should be
    performed.

Start w/ these
Importance
Required Executions
31
Process
  • The process definition will outline what it will
    take to setup and execute automated testing.
  • This definition will include
  • Hardware Requirements
  • Recommended Software
  • Integration Strategy and Requirements
  • The process will also identify expected costs and
    effort to setup the proper environment.

IDT automation techniques allow for the testing
of the system without the installation of any
software on the actual system thus allowing for a
more accurate testing environment
32
Design
  • The Design step identifies the people and
    procedures to perform the specified tasks.
  • A key part of this is to identify the people
    necessary, by skill type, in order to ensure a
    successful program.
  • Once the people have been identified, you design
    how they will work to accomplish the mission.

It is a mistake to think that software developers
alone can implement a successful automated
testing program
33
Metrics
  • Specific Metrics will be designed to be able to
    measure the overall success of the program.
  • Initial values for these Metrics will be gathered
    to define the baseline from the current process.
  • Example Metrics include
  • Person Hours Per Test
  • Tests Per Iteration
  • Coverage Percentage

34
ROI
  • The last step is to develop a potential Return On
    Investment from using Automated Test
  • Combines the Costs Identified in the Strategy,
    Process and Design Steps with the savings from
    both the automation and the risk and
    vulnerability reduction.

35
State of Testing
  • Business Drives Software Development and Testing
  • Software Development and Testing Drives Business
  • The best business ideas cannot be implemented on
    time or of high quality, if the software and
    testing teams dont deliver.

36
Software Testing Perceived vs Actual Quality
  • 10 defects that occur very frequently and impact
    critical functionality would usually be perceived
    by an end user as poor quality even if the defect
    density was very low
  • 100 defects that occur very infrequently and have
    almost no impact on operations would usually be
    perceived by an end user as good quality even if
    the defect density was high
  • Usage-Based Testing exploits the concept of
    perceived quality and thus yields higher
    perceived quality

37
Perceived vs Actual Quality
  • Concept of Perceived/Expected vs Actual Quality
  • Difference between growth and stagnation/failure/l
    ess success
  • aol.com vs CompuServe.com
  • Google.com vs Yahoo.com
  • Amazon.com vs Otherbookseller.com
  • Etc.

38
Perceived vs Actual Quality
  • Concept of Perceived/Expected vs Actual Quality
  • Difference between growth and stagnation/failure
  • Amazon.com vs Otherbookseller.com
  • Etc.
  • Competition is only a few clicks away

39
ABOK Certification
What is the ABOK?
  • Tool neutral test automation skill set
  • Reflects widely accepted industry practices
  • Tool for assessing test automation proficiency
  • Tool for identifying a track for test automator
    growth and development

A utomation B ody O f K nowledge
ABOK
40
Automated Software Testing
.. Changing How
Testing Is Done QA
Elfriede Dustin Email edustin_at_idtus.com 4401
Wilson Blvd, Suite 810 Arlington, VA 22203 Office
Phone (703) 725-3051 www.idtus.com
Write a Comment
User Comments (0)
About PowerShow.com