Michael Ebner

1
TTCN-3 Test Case Generationfrom Message Sequence
Charts

• Michael Ebner
• Institute for Informatics University of
  Göttingen Germany

ISSRE2004 - WITUL
2
Overview

• Concept
• Mapping of MSC to TTCN-3
• Related Concepts
• Graphical Format of TTCN-3 (GFT)
• UML Testing Profile (UTP)
• Comparison
• Summary

3
Unified Modeling Language (UML)
4
UML-based Test Specification
UML
map
generate
5
Overview

• Concept
• Mapping of MSC to TTCN-3
• Related Concepts
• Graphical Format of TTCN-3 (GFT)
• UML Testing Profile (UTP)
• Comparison
• Summary

6
MSC Test Purpose Description
7
Message Descriptions

• Matching mechanisms are only allowed for messages
  to be received from the SUT.

8
Mapping I

• MSC
• Basic MSC
• HMSC
• instance axis
• note, comment, text comment
• chart name
• instance kind
• flow control
• instance creation

• TTCN-3
• Testcase(s)
• Testcase calls (control part)
• represents a port (PCO)
• comment(ttcn-3,begin,position)
• Testcase name
• component name or SUT
• call or getreply procedure
• create component

9
Mapping II

• MSC
• Message
• Flow control
• (Synchronise) Condition
• Timer
• Action
• Coregion
• Alternative, Option,and Exception
• Loop
• MSC reference

• TTCN-3
• Send or receive message
• call or getreply procedure
• Synchronising
• Timer (start, stop, timeout)
• TTCN-3 Statements
• Interleave
• Alternative (incl. sync.)
• For or While loop (incl. sync.)
• Function call

10
Example I Mapping of MSC
testcase InresRTexample() runs on inres var
float sendTime1-1.0 var integer
iterator10 for (iterator10 iterator1lt100
iterator1iterator11) if
(sendTime1-1.0) sendTime1self.now0.01
else resume(sendTime1)
sendTime1sendTime10.01
log(TimestampTypeself.now,"IDATreq1")
ISAP.send(IDATreq"data") MSAP.receive(MDATin
dDT,number,"data") log(TimestampTypeself.n
ow,"MDATind2") MSAP.send(MDATreqAK,number)
setverdict(pass) stop
11
Example II Mapping of MSC

• testcase InlineAlt()runs on syminres
• ISAP1.send( ICONreq )
• ISAP1.send( IDATreq )
• ISAP2.receive( ICONind )
• alt
• ISAP2.receive( IDATind )
• ISAP1.receive( IDISind )
• ISAP1.send( ICONreq )
• ISAP1.send( IDATreq )
• ISAP2.receive( ICONind )
• ISAP2.receive( IDATind )
• ISAP2.send( IDISreq )
• ISAP1.receive( IDISind )

msc
InlineAlt
SUT
ISAP1
syminres
ISAP2
ICONind
alt
IDATind
IDISind
ICONreq
ICONind
IDATind
IDISreq
IDISind
12
Overview

• Concept
• Mapping of MSC to TTCN-3
• Related Concepts
• Graphical Format of TTCN-3 (GFT)
• UML Testing Profile (UTP)
• Comparison
• Summary

13
Overall Picture of TTCN-3
ASN.1 Types Values
TTCN-3 Core Language
Tabular Format
IDL Interfaces Types
IDL Interfaces Types
Graphical Format
TTCN-3 User
XML Types
Other Types Valuesn
Presentation formatn
14
GFT I

• Provides a test specific profile forMSC-2000
• Defines generic extensions
• Value returning GFT References
• Hyper GFT
• Hybrid GFT
• Defines test specific extensions, e.g.
• Port instances
• Special Messages for instance handling
• Type and Template definition for messages
• Trigger and check message
• Verdicts as special conditions
• Defaults

15
GFT II

• Does not use
• Gates
• General Ordering
• Time Concepts
• MSC Reference Expression
• Instance Decomposition
• Assumed data types are TTCN-3 data types

16
TIMEDGFT-Example
testcase InresRTexample() runs on inres var
float sendTime1-1.0 var integer
iterator10 for (iterator10 iterator1lt100
iterator1iterator11) if
(sendTime1-1.0) sendTime1self.now0.01
else resume(sendTime1)
sendTime1sendTime10.01
log(TimestampTypeself.now,"IDATreq1")
ISAP.send(IDATreq"data") MSAP.receive(MDATin
dDT,number,"data") log(TimestampTypeself.n
ow,"MDATind2") MSAP.send(MDATreqAK,number)
setverdict(pass) stop
17
Overview

• Concept
• Mapping of MSC to TTCN-3
• Related Concepts
• Graphical Format of TTCN-3 (GFT)
• UML Testing Profile (UTP)
• Comparison
• Summary

18
The UML 2.0 Testing Profile (UTP)

• Profile UML extension mechanism
• Initiated late 2001
• Make UML applicable for SW testing by enhancing
  UML with test concepts
• Enhances reuse of design model
• Adopted by OMG in June 2003 and since April 2004
  standardized
• Allow automated test generation with UML

19
UTP Concepts
Architecture concepts Behaviour concepts Dataconcepts TimeConcepts
SUT Test objective Wildcards Timer
Test components Test case Data partition Time zone
Test suite Defaults Coding rules
Test configuration Verdicts
Test control
Arbiter
Utility part

• UML 2.0 based meta-model

20
sd BT-Roaming
Slave BTRoaming
Slave Application
Location Server
Slave BT-HW
21
UTP Outlook

• Upcoming tool support
• IBM/Rational Eclipse Project Hyades
• Telelogic Tau G2
• Microsoft Visual Studio
• Daimler/FOKUS/Nokia/TestingTech ITEA-Project
  TT-Medal

22
Overview

• Concept
• Mapping of MSC to TTCN-3
• Related Concepts
• Graphical Format of TTCN-3 (GFT)
• UML Testing Profile (UTP)
• Comparison
• Summary

23
Comparison I

• all three concepts were based on MSC
• support of graphical test development
• differ in deepness of test features
• MSC
• GFT
• UTP

24
Comparison II

• Test Case Generation from MSCs
• centred on test purposes
• is focussed on manual, scenario testing
• abstracts from details
• no support of all TTCN-3 features
• direct mapping to test case templates
• requires less knowledge about TTCN-3
• timedTTCN-3 support
• GFT
• MSC based representation format for TTCN-3
• has to present all TTCN-3 features
• timedTTCN-3 support
• not desired as graphical specification language

25
Comparison III

• UTP
• representation and specification of all aspects
• independent of TTCN-3
• no complete mapping to TTCN-3
• enhanced testing concepts
• better integration to UML
• meta-model for testing
• wide application area (mapping to JUnit and
  TTCN-3)

26
Summary

• MSC as powerful means for test purpose and test
  case specifications
• MSC like presentation formats
• Test case generation from MSC test purposes

27
TTCN-3 Test Case Generationfrom Message Sequence
Charts

• Michael Ebner
• Institute for Informatics University of
  Göttingen Germany

WITUL 2004 (IEEE Conference ISSRE)
28
END
29
Backup
30
Autolink - Process
Test Purpose Specification
Test Case Generation
Test SuiteProduction
31
Example III Mapping of MSC

• testcase NamingServiceExample()runs on Naming
• Client1.call( bindingContext )
• Client1.call( bind )
• Client2.call( resolve )
• alt
• Client1.catch(InvalidName)
• Client1.call( bind )
• Client2.catch(NotFound)
• Client2.call( resolve )
• else
• Client1.call( unbind )

msc
NamingServiceExample
SUT
Client1
Naming
Client2
bindContext
resolve
InvalidName
NotFound
bind
resolve
unbind
32
GFT Example timedTTCN-3
Test behaviour
mapping
TTCN-3
TIMEDGFT
TIMED
Core notation
Presentation format
33
Test Purpose Example
34
Generated Testcase
testcase InresRTexample() runs on inres var
float sendTime1-1.0 var integer
iterator10 for (iterator10 iterator1lt100
iterator1iterator11) if
(sendTime1-1.0) sendTime1self.now0.01
else resume(sendTime1)
sendTime1sendTime10.01
log(TimestampTypeself.now,"IDATreq1")
ISAP.send(IDATreq"data") MSAP.receive(MDATin
dDT,number,"data") log(TimestampTypeself.n
ow,"MDATind2") MSAP.send(MDATreqAK,number)
setverdict(pass) stop
35
sd BT-Roaming
Slave BTRoaming
Slave Application
Location Server
con_request
con_accept
con_confirm
con_confirm
connected(Slave)
ref
RList makeList (Slave)
roamingList(RList,Slave)
roamingList(RList)
data
get_link_quality
quality(good)
0.5s ..0.5s
data
get_link_quality
quality(bad)
ref
newMaster Search_NewMaster()
36
Use of SDL and MSC
Specification
Ideas
modelled by
Interpreted as
used to create
Customer
SDL-2000 MSC-2000
modelled by
Design
Design
Interpreted as
used to produce
Engineers
made as
Software
Implementation
executes as
37
UML-based Test Specification I
UML
map
generate
ooTTCN-3
TTCN-3
38
Graphical Format for TTCN-3 (GFT)

• Provides a test specific profile forMSC-2000
• Uses a subset of MSC-2000
• Defines test specific extensions
• Defines general extensions
• Formerly called Test Sequence Charts (TSC)
• Supports different forms to represent test cases

39
UTP Methodology Wrap-Up

• Test preparations
• Define a test package with a test suite.
• Determine SUT and import its implementation.
• Determine test objectives.
• Test configuration
• Assign roles SUT, test components, utility part.
• Attach stereotypes.
• Test cases
• Group instances.
• Attach stereotypes.
• Define re-usable test fragments to functions.
• Assign verdicts.
• Specify timers.
• Specify default behaviours

40
Use of Inline Expressions and HMSCs

• Problem
• Several alternative test sequences referred to in
  one diagram.
• Two possibilities
• All alternatives are put in one test case.
• For each alternative a separate test case is
  generated.
• Strategy
• Alternatives described in form of HMSCs are
  translated into separate test cases.
• Alternatives described by inline expressions are
  put into the same test case.

41
ToC der Dissertation

• Introduction
• A General Metamodel For Testing
• Graphical Test Development (with case study)
• With TTCN-3
• With U2TP (UML Based Metamodel)
• Relationship and Mapping Between TTCN-3 and U2TP
• Graphical Realtime Testing with TTCN-3 and U2TP
  (with case study)
• Conclusion

42
UTP Outlook

• FTF submission April 2004
• Upcoming tool support
• IBM/Rational Eclipse Project Hyades
• Telelogic Tau G2
• Microsoft Visual Studio
• Daimler/FOKUS/Nokia/TestingTech ITEA-Project
  TT-Medal
• Automated test generation

43
ToC der Dissertation

• Introduction
• A General Metamodel For Testing
• Graphical Test Development (with case study)
• With TTCN-3
• With U2TP (UML Based Metamodel)
• Relationship and Mapping Between TTCN-3 and U2TP
• Graphical Realtime Testing with TTCN-3 and U2TP
  (with case study)
• Conclusion

44
Unified Modeling Language (UML)
45
... und Testen
Entwickler
Integrator
System-integrator
46
(No Transcript)
47
TTCN-3 Übersicht
TTCN-3 Kern-sprache
ASN.1 Typen Werte
Tabellarisches Format
Graphisches Format
TTCN-3 Benutzer
Andere Typen Werten
Präsentations- formatn
48
MSC and TTCN (1)
System specification
MSC
MSC-based Presentation Format for TTCN-3
Test purpose
Abstract test case
TTCN-3
Executable test case
Test results
Assessment
49
MSC and TTCN (2)
1. Generation
TTCN-3 ATS
MSC- 2000
2. Presentation
Compilation
Compilation
Tracing
TTCN-3 ETS
MSC Simulator
Validation
50
Information in MSC Test Purposes

• Descriptions of messages to be sent toand
  received from the SUT
• Synchronization information required dueto the
  partial order semantics of MSC
• Structure information

51
MSC Objectives

• is a scenario language
• supports complete and incomplete specifications
• is a graphical language
• is widely applicable
• can be used throughout the engineering process
• supports structured design
• is often used in conjunction with other methods
  and languages

52
The Tutorial Example

• INRES - Initiator-Responder protocol
• an abridged version of the Abracadabra protocol
  used for academic studies and illustrative
  purposes.
• a connection-oriented, asymmetric communication
  protocol
• a medium with uncorrupted data, loss of data,
  misordering

53
INRES Structure
User
ICONreq, IDATreq, IDISreq, ICONconf, IDISind
ISAP1
Initiator
Responder
MDATreq, MDATind
MSAP2
Medium
SUT (System under Test) Initiator and Medium
54
Objectives (3)
MSAP2
?
port
MSAP2type
55
MSC Test Purpose Development View
56
Objektorientierung für TTCN-3

• Kein Objekttyp und Portinstanzen können nicht
  übertragen werden
• Objektreferenzen in IUT nicht handhabbar
• Einführung von Klassen
• Erweiterung vorhandener Konzepte
• Vererbung von Templates
• Hierarchie von Komponenten

57
Autolink - INAP CS-2 Testzweck
58
Zusammenfassung und AusblickAusblick

• Testing and Test Control Notation 3 (TTCN-3)
• Universelle Testbeschreibungssprache
• Standardisierung 2001 durch ETSI
• Entwicklung eines TTCN-3-Syntaxcheckers
• Verbesserte Modellierung der Testumgebung
• Praxis Standard-Werte und benutzerdefinierte
  Signale
• Ineffektive Suche, Nachbearbeitung von Testfällen
  
• Lösung Symbolische Ausführung
• Entwicklung eines Prototypen

59
AusblickTTCN-3 Beispiel
60
Weitere ArbeitenTesting and Test Control
Notation 3

• Erweiterung der Anwendungsgebiete
• Testarten Interoperabilitätstest,
  Performanztests, ...
• Architekturen CORBA, APIs, ...
• Moderne, allgemeine Sprachkonzepte
• C/C-ähnliche Notation
• keine OSI-spezifischen Elemente
• Erweiterte Kommunikationskonzepte
• Dynamische, verteilte Testarchitekturen
• Standardisierung 2001 durch ETSI
• Entwicklung des ersten freien TTCN-3-Syntaxchecker
  s

61
ToC der Dissertation

• Introduction
• A General Metamodel For Testing
• Graphical Test Development (with case study)
• With TTCN-3
• With U2TP (UML Based Metamodel)
• Relationship and Mapping Between TTCN-3 and U2TP
• Graphical Realtime Testing with TTCN-3 and U2TP
  (with case study)
• Conclusion

62
How to Test?
System implementation
System specification
identify
Test purpose
specify
Abstract test case
implement
Executable test case
execute
Test results
evaluate
Assessment
63
MSC and TTCN (1)
System specification
MSC
MSC-based Presentation Format for TTCN-3
Test purpose
Abstract test case
TTCN-3
Executable test case
Test results
Assessment
64
MSC and TTCN (2)
1. Generation
TTCN-3 ATS
MSC- 2000
2. Presentation
Compilation
Compilation
Tracing
TTCN-3 ETS
MSC Simulator
Validation
65
Basic MSC in a Nutshell
MSC diagram
Instance
msc
INRES_Example1
Message to the environment
User
MSC heading
Idle
ICONreq
MDATind
Condition
MDATreq
Instance end
ICONconf
Output event
Connected
Input event
66
Method Calls
Method call at a gate
Method area
Suspension area
Call on suspension
Reply at a gate
67
HMSC
HMSC Start
Loop
msc
INRES_Behaviour
Idle
MSC Reference
Connection Point
Connecting
Condition
DataTransfer
Alternative
Disconnecting
68
New challenges for TTCN (1)

• Corba and Internet Testing
• Dynamic configurations
• Addressing entities inside the SUT
• Synchronous communication
• Real-time- and performance- Testing
• Handling of load generators
• Possibility to repeat test runs
• New and improved design techniques
• Smooth interworking with SDL'2000, MSC'2000,
  ASN.1'99, UML and IDL
• Adaptation to different application areas

69
New challenges for TTCN (2)

• Module and Group concept
• New data types
• Synchronous communication
• Less OSI concepts and terms
• Introduction of templates
• Textual base

70
TTCN-3 Test Architecture (2)
Test system
Connected Ports
IN
OUT
MTC
PTC
OUT
IN
IN
OUT
Mapped Ports
Abstract Test System Interface
IN
OUT
Real Test System Interface
System Under Test (SUT)
71
TTCN-3 Example
72
The Development of MSC
73
Object Orientation

• MSC documents define instance kinds
• instance kinds are types suited for object
  orientation
• Inheritance
• inheritance of instance kinds means inheriting
  all contained instances and MSCs
• Virtuality
• Virtual MSCs means that MSCs may be redefined in
  specialized instance kinds
• Close correspondance with object orientation in
  SDL and other object-oriented languages such as
  Java

74
Data in MSC-2000

• MSC has no data language of its own!
• MSC can make use of data languages such that
• fragments of your favorite (data) language can be
  used (C, C, SDL, Java, TTCN-3, ... )
• MSC can be parsed without knowing the details of
  the chosen data language
• the data language strings have no unnecessary
  extra delimiters
• Mappings to data languages have not yet been made
  
• up to any user
• could be standardized in the future

75
Use of Data
msc Transmission (isdu ISDUType seq_no, try
Integer)
Initiator
Responder
Medium

• Static variables have MSCs as homes (cannot
  change binding within the MSC)

seq_no 1-seq_no
MDATreq(DT(isdu,seq_no)dt)
MDATind(dt rec_dt)
Check_Data(rec_dt)
MDATreq(noACK(rec_no))

• Binding
• Left-hand or right-hand
• in actions
• with messages
• with MSC references

MDATind(rec_noACK(rec_no))
when (seq_no ! rec_no) and (try lt4)
Transmission(isdu,seq_no,try1)
76
Method Calls

• Method Calls
• Blocking (with reply)
• Non-Blocking
• Method calls can be super-imposed
• Used to describe the control flow between
  instances

77
Summary Outlook (2)

• MSC as powerful means for test purpose and test
  case specifications
• in particular, extensions of MSC-2000 such as
  data support and method calls are used
• Test case generation from MSC test purposes
• direct mapping to test case templates
• MSC like presentation format for TTCN-3

78
Enjoy your lunchandhave a nice trip home! ?
79
The UML 2.0 Testing Profile

• Profile UML extension mechanism
• Initiated late 2001
• Make UML applicable for SW testing
• Adopted by OMG in June 2003
• At time it is revised by the Finalization Task
  Force (FTF)

80
Outlook

• FTF submission April 2004
• Upcoming tool support
• IBM/Rational Eclipse Project Hyades
• Telelogic Tau G2
• Microsoft Visual Studio
• Daimler/FOKUS/Nokia/TestingTech ITEA-Project
  TT-Medal
• Automated test generation

81
Outline Introduction
Test behaviour
TTCN-3
GFT
mapping
TIMED
Core notation
Presentation format
82
TIMEDTTCN-3TTCN-3Time Extensions

• Local clock
• read (self.now), wait (resume)
• Logfile
• log, sort, retrieve of structured data
• Timezones
• specification of clock synchronised test
  components
• Conf-Verdict
• pass?conf?inconc?fail

83
Test Case Generation
Test behaviour
mapping
TTCN-3
TIMEDGFT
TIMED
Core notation
Presentation format
84
Generating TIMEDTTCN-3 from MSC

• Pure functional test cases Proven approach and
  implemented by TTCN-2 tools, e.g.
• Autolink (Telelogic)
• ptk (Motorola)
• New generating TIMEDTTCN-3 test cases.
• MSC allows to attach time constraints to events
• Absolute time constraints,
• Relative time constraints,
• Time measurements.

85
Test Purpose Example
86
Generated Testcase
testcase InresRTexample() runs on inres var
float sendTime1-1.0 var integer
iterator10 for (iterator10 iterator1lt100
iterator1iterator11) if
(sendTime1-1.0) sendTime1self.now0.01
else resume(sendTime1)
sendTime1sendTime10.01
log(TimestampTypeself.now,"IDATreq1")
ISAP.send(IDATreq"data") MSAP.receive(MDATin
dDT,number,"data") log(TimestampTypeself.n
ow,"MDATind2") MSAP.send(MDATreqAK,number)
setverdict(pass) stop
87
Graphical Test Case Specification
Test behaviour
Test purpose
TTCN-3
GFT
mapping
MSC
generate
TIMED
Core notation
Presentation format
88
TIMEDGFT-Example
testcase InresRTexample() runs on inres var
float sendTime1-1.0 var integer
iterator10 for (iterator10 iterator1lt100
iterator1iterator11) if
(sendTime1-1.0) sendTime1self.now0.01
else resume(sendTime1)
sendTime1sendTime10.01
log(TimestampTypeself.now,"IDATreq1")
ISAP.send(IDATreq"data") MSAP.receive(MDATin
dDT,number,"data") log(TimestampTypeself.n
ow,"MDATind2") MSAP.send(MDATreqAK,number)
setverdict(pass) stop
89
TIMEDGFT

• Overview New GFT symbols and mapping
  TIMEDTTCN-3 ?TIMEDGFT

TIMEDTTCN-3 TIMEDGFT
90
TIMEDGFT (continued)

• TIMEDTTCN-3 TIMEDGFT