Graphical Testing With UML The UML Testing Profile

1
Graphical Testing With UMLThe UML Testing Profile

• Eric Samuelsson
• eric.samuelsson_at_telelogic.com

2
Agenda

• Background
• Key concepts
• Advanced concepts
• Summary

3
Background

• Test technologies are not on par with modern
  design technologies such as model-driven
  architectures (MDA)
• Theres a need for a widespread graphical test
  language
• OMG realized this and issued and RFP for a
  graphical testing profile to UML
• Using the UML ensures widespread use across a
  wide range of application areas
• Numerous tool vendors support UML
• UML is a proven de facto standard
• But UML is focused on definition of system
  structure and behavior and has no built-in
  testing constructs...

4
UML Testing Profile RFP

• Issued by OMG in July 2001
• LOIs submitted by
• Telelogic, Rational, SofTEAM, FOKUS, Ericsson,
  Motorola, IBM
• One joint submission
• Initial submission made in April 2002
• Revised initial submission made in June 2002
• Terminology changes
• Elaboration of Test Data
• Final submission in January 2003

5
Requirements and restrictions

• Define structural and behavioral test concepts
• Conformance testing only
• Black-box functional testing
• Based upon the UML meta-model
• UML 2.0 as proposed by the U2 Partners
• Inter-operation with existing testing
  technologies
• TTCN 3
• Junit
• ...

6
Current proposal

• A graphical testing language based the current
  UML 2 metamodel
• Reusing as many UML concepts as possible
• Recognition important for UML users
• No need to re-invent the wheel
• Designed with TTCN 3 in mind
• Semantics basically follows TTCN 3 semantics
• But caters for other technologies like JUnit as
  well

7
Current proposal

• Defined as a UML profile
• Will be recognized by UML users
• Basically a number of stereotypes
• Easy to implement for tool vendors
• Models will be interchangeable
• Any tool implementing XMI will be able to
  exchange models

profile Testing Profile
UML
import
stereotype TestCase
metaclass BehavioralFeature
8
Key Concepts

• Test Architecture
• Test Context, Test Configuration
• Test Component, SUT
• Arbiter
• Test Behavior
• Test Case
• Verdict, Validation Action
• Defaults
• Test Data
• Logical partitions, Pattern matching
• Test Deployment

9
Test Objective

• An informal description of what to test in one
  test case
• Commonly expressed using natural language
• Can also be expressed using other diagrams, such
  asuse case diagrams and sequence diagrams

Verify that if a user inserts and authorizes a
valid card correctly, he is able to withdraw
money if he has sufficient funds.
Withdrawal
Client
Client
ltltincludegtgt
InvalidPin
Tester
10
Test Case

• A specification of one test, including
• Test objective
• Test behavior
• Defined as a BehavioralFeature in UML
• Its method is the behavior of the test case

testCase validWithdrawal() Verdict
testContext ATMContext
testCase validWithdrawal()
Verdict invalidPIN() Verdict -authorizeCard()
Verdict
11
Test Behavior

• Describes the expected behavior of a test case
• Defined using any type of UML behavior
• Interactions, statemachines and activities
• Facilitates specification of arbitrarily complex
  behavior

Idle
WaitCode
12
Test Architecture

• A complete specification of all types used in the
  tests
• An ordinary UML package
• Contains Test Contexts, Test Components, Ports,
  Interfaces, etc.

testArchitecture ATMTest
13
Test Architecture Example
testArchitecture ATMTest
IAccount
testContext ATMContext
testComponent HWEmulator
bePort
IScreen, ICardMachine, IMoneyBox
-pinOk Boolean -enteredPIN String -message
String
testCase validWithdrawal()
Verdict invalidPIN() Verdict -authorizeCard()
Verdict
testComponent BankEmulator
hwCom
IATM
accounts

Account
ATMCardData
balance Integer number String

pinCode Integer number String
cards
credit(a Integer) debit(a Integer)
isPinCorrect(c Integer) Boolean
14
Test Component

• Implements (parts of ) the behavior of one or
  more test cases
• Each test case is normally executed by several
  components
• Ordinary classes or components

testComponent HWEmulator
-pinOk Boolean -enteredPIN String -message
String
IScreen, ICardMachine, IMoneyBox
hwCom
IATM
-currentCard
0..1
ATMCardData
pinCode Integer number String
isPinCorrect(c Integer) Boolean
15
Test Context

• A classifier that contains
• a number of test cases and their behavior and
  objectives
• default behavior used by the test cases
• configuration of test components
• test control behavior
• Can be a class, a component or a collaboration

testContext ATMContext
testCase validWithdrawal()
Verdict invalidPIN() Verdict -authorizeCard()
Verdict
16
Test Configuration

• A specification of the structural aspects of test
  cases
• Defines the parts participating in the test case
  and their interconnections
• Test Components, SUT, Ports, Interfaces and
  Connectors
• Defined as the internal structure of the Test
  Context

sut atm BankATM
be BankEmulator
bePort
netCom
atmPort
hwCom
accnts Account 0..
hwe HWEmulator
17
Test Control

• Test execution can be controlled is two ways
• By defining the classifier behavior of a test
  context
• Or by invoking the test cases from outside of the
  context

18
Test Control by ClassifierBehavior
sd ATMContext
invalidPIN()
testContext ATMContext
testCase validWithdrawal()
Verdict invalidPIN() Verdict -authorizeCard()
Verdict
verdict fail
verdict pass
validWithdrawal()
19
Test Control by Invocation
ad myOp
testContext MyTestContext
verdict context.TC1()
MyClass
-context
-verdict Verdict
testCase TC1() Verdict TC2() Verdict
-TC3() Verdict

myOp()
else
verdict pass
verdict context.TC2()
20
Test Data

• Parameters, default values, return values are
  reused from UML without changes
• Logical partitions
• Pattern matching
• TDB

21
Test Deployment

• Test Deployment is important, but there is
  probably no difference between deploying test
  systems compared to ordinary UML systems.
• We will try to reuse the existing UML deployment
  concepts without changes
• Under investigation

22
Advanced Concepts

• Test Evaluation
• Verdict
• Validation action
• Arbiter
• Default behavior
• Structural and behavioral reuse

23
Test Evaluation

• A number of concepts are used to evaluate test
  results
• Verdicts
• Validation actions
• Arbiter

24
Verdicts and Validation Actions

• Each test component is associated with a verdict
• Validation Actions are used to set the verdict
• A Validation Action sets the verdict of a test
  component and reports the verdict to the Arbiter

sut
Client
ATM
storeCardData(cardData)
display(Enter PIN)
isPinCorrect(pin)
true
false
setverdict(fail)
25
Arbiter

• A test component that knows how to evaluate the
  test behavior
• Separates test evaluation from test case
  specification
• One behavior can yield different results
• Each test context has a default arbiter
• Implicit connections to all test components
• TTCN 3 semantics
• Advanced arbiters can be defined
• Caters for more advanced semantics

arbiter myArbiter
setVerdict(v Verdict)
26
Default behavior

• Defines behavior for unexpected observations
• Can be expressed using any UML behavior
• Can be applied to several different levels
• Test component
• Message reception
• Alternative statement
• ...

sd authorizeCard
sut
Client
ATM
storeCardData(cardData)
display(Enter PIN)
isPinCorrect(pin)
true
27
sd enterPinDefault
self
getStatus
statusOk
setverdict(pass)
Default expressed as interaction...
display(Connection lost)
setverdict(inconc)
ejectCard()
setverdict(fail)
28
enterPinDefault
getStatus / statusOk, setverdict(pass)
...and as statemachine

ejectCard / setverdict(fail)
display(Connection lost) / setverdict(inconc)
The representations are semantically equivalent
29
Meta Default

• Each test component has a meta default to cater
  for differences in semantics
• Can be redefined

MetaDefault

/ defer
30
Reuse of Test Structure and Behavior
31
testContext ATMContext
sut atm BankATM
be BankEmulator
accnts Account 0..
bePort
netCom
atmPort
ATM
hwCom
hwe HWEmulator
ac AuthorizationContext
Client
32
sd ValidWithdrawal
sut
hwe
be
atm
netCom
atmPort
ac.authorizeCard()
selectOperation(withdrawal)
true
withdraw(amount)
findAccount(cardData)
account
debitAccount(account, amount)
true
deliverMoney(amount)
display(Take cash)
true
33
Summary

• The testing profile enables complete graphical
  specification of tests for conformance testing of
  UML models
• Defined as a UML profile
• Reuses existing UML concepts whenever possible
• Designed for inter-operation with TTCN-3 and
  other technologies
• Due date
• January 6th, 2003
• More information
• Testing Profile Consortium
• OMGs Testing Profile page