Susan Windsor - PowerPoint PPT Presentation

1 / 53
About This Presentation
Title:

Susan Windsor

Description:

Global development, with large projects having multi site, multi geography and ... Migration from current banking platform IBIS to Temenos Globus G13 ... – PowerPoint PPT presentation

Number of Views:72
Avg rating:3.0/5.0
Slides: 54
Provided by: IBMU373
Category:
Tags: ibis | susan | windsor

less

Transcript and Presenter's Notes

Title: Susan Windsor


1
Strategic Direction for Functional Test
Automation SoftTest 2005
  • Susan Windsor
  • Insight Through Intelligence
  • WMHL Consulting Limited, MD

Title slide
2
AGENDA
  • Future for Functional Testing
  • History of Functional Automation
  • Strategic Direction
  • Automation Frameworks in Action
  • Impact upon You?

3
Future for Functional Testing
4
Increased Demand..?
  • Global development, with large projects having
    multi site, multi geography and multi suppliers
    to contend with.
  • Corporate and regulatory requirements growing all
    the time
  • Business demands new applications, faster and
    cheaper to obtain competitive advantage
  • Closer alignment of IT and business to repair
    lack of business confidence

5
..Or, increased pressures?
  • Automation frameworks and greater business
    knowledge requirements, mean reduced numbers of
    traditional functional testers
  • Agile development methods mean developers
    undertake more unit and component testing
  • Growth of outsourced testing to different
    geographies, leading to greater competition for
    the roles

Is the role of the functional tester (who is
neither technical or business specialist) dead?
6
History of Functional Automation
7
Record and Playback
  • all you need to do.!
  • Easy to record the scripts
  • Extremely fragile
  • Expensive to maintain
  • On average, each test run requires at least 50
    of the scripts to be recorded again

Please tell me no one does this anymore!!!
8
Scripting
  • Use the scripting language to write scripts that
    do what you want
  • Build in as much robustness as possible
  • But, youre building an application to test an
    application, which also needs testing!
  • Maintenance costs can be very high
  • Needs programmer skills
  • Cost of implementation not affordable at project
    level

9
Table Driven
  • Like scripting but more flexible and greater
    re-use
  • Remove items from the script that change, e.g.
    data
  • Reduces maintenance costs a bit
  • Implementation costs about the same
  • Still requires specialist skills to implement

10
Functional Test Automation is Broken!
  • Focus on technology rather than business needs
  • 80 of functional testing still manual
  • 60 to 70 of automation tools used for
    non-functional testing
  • Typically, traditional functional automation
    stops at 100 scripts, regardless of test coverage
    requirement
  • Critical factors cost of implementation and
    maintenance, skills required and asset sharing
    over different technologies

11
Strategic Direction
12
Automation Frameworks
  • Wouldnt it be good if..
  • Tests could be documented in common format,
    regardless of whether they are manual or
    automated
  • The format for the tests resulted in quicker
    preparation time than traditional manual tests
  • Both developers and business testers could
    understand and use the same test format
  • Script maintenance was no longer a requirement
  • Tests could use any of the test execution tools
    required by the underlying technology

Faster planning, faster execution, and far less
technical skill required
13
Business Analysts already using them, and use
will grow
  • Home grown frameworks built within organisations
    to meet business demands
  • Niche suppliers providing frameworks try a
    Google search 512,000 hits this week
  • Seen a handful that appear mature
  • Latest review by Paul Herzlich (OVUM analyst)
  • Market Leaders such as Mercury developing
    Business Process Tester (BPT)

This is the industry direction now
14
Automation Frameworks in Action
15
The project
  • Lloyds TSB Offshore in Channel Islands
  • Migration from current banking platform IBIS to
    Temenos Globus G13
  • Tight deadlines due to EU Savings Tax Directive
    coming into force on 1st July 2005
  • Business processes required additional Globus
    modules
  • Multiple releases leading up to deadline required
    substantial testing
  • SQS-UK contacted to provide consultancy

16
The Team
  • SQS-UK recognised the requirement for test
    automation
  • Joint team from Odin Technology and SQS-UK
    assembled
  • Working alongside Lloyds TSB staff
  • Odin Technology provided testing technology
  • SQS-UK provided implementation resource and
    testing know-how

17
Project Approach
  • Split project roles for Automation Technicians
    and Testers
  • Three phase technical approach
  • Tool Evaluation Technical Feasibility
  • Technical Implementation
  • Support
  • Test Analysis and Design throughout

18
Project Phase 1Tool Evaluation and Technical
Feasibility
19
Tool Evaluation
  • Automation isnt always straightforward!
  • GUI Tools dont work out of the box for all
    environments
  • They will almost certainly require some custom
    functionality to interact with the SUT
  • The Evaluation process
  • Assess what custom functionality is required
  • Assess what involvement is required from an
    Automation Technician to provide the custom
    functionality
  • Assess if there are any other efficiency
    improvements that can be made through application
    of technology
  • This provides the scope of technical
    implementation phase

20
Project Phase 2 What would normally happen...
21
Typical Automation Project
  • Take existing Manual Scripts
  • Rework by hand into automated scripts function
    libraries
  • The bulk of the work
  • Building the Object Map
  • Providing the Mapping between the business terms
    used to describe the SUT Interface and the test
    tools requirements
  • Coding Test Scripts
  • Providing the instructions and associated data
    for the test tool to interact with the SUT and
    perform testing

22
Object Mapping
  • Providing the Mapping between the business terms
    used to describe the SUT Interface and the test
    tools requirements

23
Example
Manual Tester
The Login Window
Test Tool
class MSWDialog label Login.
24
Example 2
Manual Tester
Username
Test Tool
class HTMLEdit id Lgn_Uid
25
Object Mapping in a typical Automation Tool
  • Usually implemented under different names in
    different tools
  • GUI Map
  • Object Repository
  • Object Map
  • Mapping an Object
  • Start the Object recognition tool
  • Point at the Object with the Mouse Pointer
  • Select the Object
  • Give the Object a meaningful business name
  • Make manual amendments to the properties if
    required

26
Test Script Creation
  • Providing the instructions and associated data
    for the test tool to interact with the SUT and
    perform testing

27
Scripting Approach
  • Use the Tools Scripting Language to construct
    business processes

28
(No Transcript)
29
Project Phase 2 What was done differently
  • Taught an old dog some new tricks

30
Test Design Model
  • A simple structured way of defining tests that is
    independent of the interface and the means of
    execution

31
Features of the Test Design Model
  • Easy to design tests without technical knowledge
  • Well structured
  • Can be used to describe tests for any interface
  • Graphical User Interfaces (GUI)
  • Non-UI Components (WebServices, APIs etc.)
  • Independent of Execution Mechanism
  • Manual testing
  • Automated testing
  • Not theory - proven 7 year history of
    implementations
  • In use in UK and Europe at many organisations for
    defining testing

32
Test Design Model
33
Test Model GUI Example
34
A test step
  • Object Action Data
  • Action
  • SET enter data, navigation
  • GET retrieve data
  • VAL compare expected/actual data

35
How tests are designed
  • Using Microsoft Excel
  • Business processes are modelled
  • Sequences of steps and data are built up in
    Sub-Test Tables
  • Sequences of Sub-Tests model business processes

36
Interpreting the Model
  • There are products that can process the model
  • To generate Test Automation Scripts
  • For UI Testing using GUI Automation tools
  • For Non-UI Testing using Harnesses
  • To generate documentation for manual testing

37
Interpreting the Model
38
Globus A Self Descriptive Application
39
What is a Self-Descriptive Application?
  • An application that, through an automated
    process, can provide a description of its
    interfaces

40
The Globus User Interface How it works
41
How Globus Describes its Interface
42
A Sample of Globus Information
APP CUSTOMER_INST FIELD CUST_SNAME TYPE INPUT
LABEL Surname LENGTH 20 SEQ 1 FIELD CUST_GEN T
YPE OPTION LABEL Gender LENGTH 1 SEQ 7
43
How this Information was used
  • The Description gives us 3 things
  • The business names used for the objects
  • The properties the test tool required for object
    identification
  • The sequence of the objects in the window
  • This could provide us with two things
    automatically
  • The Object Map
  • The sequence of steps for our TDM sub-tests

44
More on Self-Descriptive Apps
  • Requirement on development to name objects in
    business terms
  • Microsoft .NET Applications
  • Windows Forms and Web Applications can provide us
    with
  • Object Maps
  • Sub-tests for the Test Design Model
  • This can drastically reduce the cost of
    automation

45
Overall - The Test System Design
46
Project Phase 2Implementation
47
The Implementation
  • Technical Implementation
  • 9 days of Test Tool Technician
  • Using Automatic Interface Description in Globus
  • 1800 Objects Mapped automatically
  • 90 Sub-test components generated
  • 30-75 steps per Sub-test
  • Time to Generate 42 seconds

48
What was left for the Testers?
  • Test Design!
  • Designing scenarios for testing the application
  • Providing Test Data
  • Sequencing Sub-Tests to form business processes

49
Designing the Tests
  • 4 Testers Using the Test Design Model
  • No previous automation experience
  • 40 Days Elapsed time
  • 1012 Complex Banking Scenarios Automated

50
Raw Statistics Test Execution
  • Single Script
  • Automated 80secs
  • Manual 480secs
  • Automated is 6x Faster
  • Full Regression Pack
  • Automated 22 Hrs (1 Workstation)
  • 5.5 Hrs (4 Workstations)
  • Manual 132 Hrs
  • 22 Man/Days (using 6hrs productivity per
    day)
  • 22 Man/Days Testing in 5.5 Hrs.

51
Impact Upon You
52
Recommendations
  • Find out your organisations direction
  • Decide upon your own future
  • Increased management skills
  • Increased non-functional skills
  • Increased business skills
  • Ensure you understand how Automation Frameworks
    support testing so you have a view when asked,
    because you will be.!

53
Thank You
  • Susan Windsor
  • WMHL Consulting Limited, MD
  • susan_at_wmhl.co.uk
  • www.wmhl.co.uk

Case Study provided by Odin Technology Ltd
www.odin.co.uk
Closing slide
Write a Comment
User Comments (0)
About PowerShow.com