Test Case Modeling and Standardization for Service Oriented Computing

1
Test Case Modeling and Standardization for
Service Oriented Computing

• By
• Jay Elston and Dan Marstein
• October, 2007

2
Contents

• Objective
• Problem Statement
• SOC Testing Challenges
• Work To Date
• Current Test Case Representation
• SOC Testing Framework

3
Objectives

• Explain why it is desirable to standardize test
  cases.
• Discuss some of the SOC testing challenges as
  they relate to standardization
• Present an approach to developing such a standard
• Survey work towards modeling and standardizing
  test cases
• Present a web service testing framework

4
Problem Statement

• Collaborative testing is seen as one of the keys
  to providing reliable, trustworthy web services
• In order to enable the sharing of test cases
• Standards are needed
• What should this standard look like?
• What information is needed in the test cases?
• What are the options for publishing test cases?

5
Problem Statement

• This project will develop a test case model for
  service oriented computing that will
• Permit the publication and sharing of test cases
  for web services
• Provide information for the various phases of
  testing
• Simulation
• Integration Testing
• Hybrid Testing
• Work within the context of a SOC Testing
  Framework

6
Approach

• Existing work
• Test Case Languages
• Test Case Representation in commercial tools
• Web Service Testing
• Define SOC Testing Challenges and Requirements
• Present SOC Testing Framework
• Define the Elements of a Test Case derive a
  model
• Prototype the model using different service
  architectures and testing types

7
Work To Date

• Survey of existing work in progress
• Some models have been selected

8
Survey of Existing Work

• TSL (Test Specification Language)
• TTCN
• Tree and Tabular Combined Notation
• part of ISO/IEC 9646
• Used for protocol testing
• eTest-ML (Event-driven Testing Model Language -
  http//www.oasis-open.org/committees/download.php/
  20693/eTestML-draft-07.pdf)
• WSDLTest

9
SOC Testing Challenges

• Collaborative Testing anyone can publish test
  cases
• Different types and phases of service testing
• Simulation Test
• Requires service and application simulation
• Hybrid testing
• Requires service simulation with some level of
  fidelity
• Run time testing
• Not always possible

10
SOC Architectural Patterns and Testing Challenges

• Collaborative
• Example Store Front
• Sensor, Control, Actuator
• Example Robot
• Grid
• Example Optimization Problems

11
SOC Architectural patterns

• Application and Composite services have
  architectures
• Architecture refers to
• the way that the services are glued together
• where services are hosted
• how they are orchestrated and choreographed
• their environment is
• Examples of SOC Architectural Patterns
• N-Tiered (client/server)
• Actuator and Control
• Grid
• Query Analysis

12
N-Tier (client/server) Architectures
Service Host
Environment
Application
Service Host

• Example
• Online Store
• Testing Challenges
• Workflow
• Orchestration Choreography
• Policy
• Unplanned changes to atomic services
• Simulation (environment and external services)
• Characteristics
• Long Lifetime
• Change driven by consumer, services, and
  environment
• State per session only

13
Actuator/Control Architecture
Service Host
Sensors and Devices
Application
Environment

• Examples
• Robot Control
• Manufacturing Line Control
• Smart home
• Testing Challenges
• Timing
• Complex Environment
• Characteristics
• Long Lifetime
• Planned changes to atomic services
• Change driven by consumer, and environment
• Persistent State

14
Grid Architecture
Service Host
Application
Environment
Service Host

• Example
• Climate Modeling
• Testing Challenges
• Managing services dynamically
• Characteristics
• Stable Application
• Known, controlled environment

Service Host
Service Host
Service Host
Service Host
15
SOC Testing
16
SOC Testing Frameworks

• WebStrar (ASU)
• Framework proposal exists
• Eclipse Web Service Test Framework
• One page design document exists, but not much
  else
• Other Web Service Test Frameworks focus on single
  web services
• WebInject (http//www.webinject.org/)
• Tests one service
• IBM Rational Functional Test
• Generate WSDL test cases for a specific web
  service

17
Webstrar Infrastructure
Service Providers submit WS test cases
test case oracle
Model checking
Test case generators
Test case database
WS test master
Access Data - Reliability - Ranks
WSDL OWL-S DAML-S
Reliability Models
Test case ranking
Oracle updates
Reliability database
Test case validation
Service composition
Trustworthy WS repository
WS ranking
WS directory
Composite WS
Service Requestors / Clients
17
CSE 565 -- Test Case Modeling
18
Test Framework Considerations

• Be capable of running tests in various test
  phases
• Simulation only
• Integeration
• Hybrid
• Use (and reuse) published test cases
• Maintain records
• Which service have been tested
• Which tests have been performed
• Web Service Reliability
• Model
• Reliability
• Environment

19
Proposed Testing Framework
20
Proposed SOC Testing Framework Features
(preliminary)

• Sets up test harness for test runs
• Interface to application or system under test
• Creates agents for dependent services
• Runs test cases
• Performs environment modeling
• Provide simulation inputs to simulators
• Records test results
• Updates Reliability Statistics

21
Test Case Elements(preliminary)

• Administrative
• ID
• Version
• Source (Author, Company, etc.)
• Associative (Ontological)
• Associated Service(s)
• Test Phase
• Associated Simulations
• Functional
• Inputs, Outputs, Preconditions, Effects
• Environmental Parameters
• Simulation Parameters
• Other
• Expected test time