The Use of TTCN-3 for Software Testing

1
The Use of TTCN-3for Software Testing

• Ina Schieferdecker
• FOKUS, Berlin
• schieferdecker_at_fokus.fhg.de

2
Content

• Overview on TTCN-3
• TTCN-3 applied to software
• TTCN-3 and XML
• Test Execution

3
Content

• Overview on TTCN-3
• TTCN-3 applied to software
• TTCN-3 and XML
• Test Execution

4
Introduction

• TTCN-3 is the Testing and Test Control Notation
• The new standardised test specification and test
  implementation language
• Developed from 1999 2002 at the European
  Telecommunications Standards Institute (ETSI).
• Developed based on experiences from previous TTCN
  editions
• Removal of OSI specific concepts Improvement of
  concepts Introduction of new concepts.
• Applicable for all kinds of black-box testing for
  reactive and distributed systems, e.g.,
• Telecom systems (ISDN, ATM, GSM, UMTS) Internet
  (IP, IP based protocols and applications)
  Software systems (Java, XML) Middleware
  platforms and component-based systems (CORBA,
  .Net, EJB).

5
Overview on TTCN-3
TTCN-3 Core Language
ASN.1 Types Values
Tabular Format
testcase myTestcase () runs on MTCType system
TSIType mydefault activate (OtherwiseFail)
verdict.set(pass) connect(PTC_ISAP1CP_ISAP
1,mtcCP_ISAP1) map(PTC_ISAP1ISAP1,
systemTSI_ISAP1) PTC_ISAP1.start(func_PTC_IS
AP1()) PTC_MSAP2.start(func_PTC_MSAP2()) Synch
ronization() all component.done log(Correct
Termination)
Graphical Format
Presentation Format n
Other Types Values n
6
TTCN-3 Based Black-Box Testing

• Assignmentof aTest Verdict

7
Component-Based Test System
TTCN-3 Test Case
8
Main Elements of TTCN-3

• Module covers declarations and control
• Templates (test data description) and matching
  mechanisms (pattern matching)
• Test configurations
• Formally defined interfaces to the SUT
• Dynamic creation of test component
• Concurrency to describe distributed test setups
• Test cases
• Small (complete) separate compilable programs
• Share (type and data) information
• Test verdicts

9
Content

• Overview on TTCN-3
• TTCN-3 applied to software
• TTCN-3 and XML
• Test Execution

10
TTCN-3 and Software Testing

• Message-Based Software
• Such as Protocols, Messaging Services, Web site
  and Portals, ...
• Example technology is XML

• Signature-Based Software
• Such as Client-Server, Peer-to-Peer,
  Component-based Systems, ....
• Example technology is IDL

• Reuse of XML/IDL/... data within TTCN-3
• Define a mapping from XML/IDL/... to TTCN-3

11
TTCN-3 and Software Testing
TTCN-3 Core Language
ASN.1 Types Values
Tabular Format
Graphical Format
Presentation Format n
Other Types Values n
12
Content

• Overview on TTCN-3
• TTCN-3 applied to software
• TTCN-3 and XML
• Test Execution

13
XML in Distributed Applications

• Increasing number of distributed applications use
  XML for
• Description of messages in datacom protocols
• Exchange format between software components
• Data description in Web-applications
• etc.
• XML
• is a structured method for putting data into a
  textual presentation by marking up data
• can have attributes that describe additional
  information
• describes both attributes and content
• is intuitive and self describing

14
Mapping XML to TTCN-3

• Idea
• Map element tags and attributes to TTCN-3 fields
  
• Different grammar definitions and mappings
• Schemas
• Embedded approach
• Flat-Catalog approach
• Named Type approach
• DTDs

15
Example The Dinosaur Database
Example
16
The Request Interface
URL http//www.testingtech.de/TTCN-3_Example/dinol
ist.xml
17
Principal Approach
18
Structured Type Defintions
Example
lt!ELEMENT dinolist (dinosaur)gt lt!ELEMENT
dinosaur (name, len, mass, time,
place)gt lt!ELEMENT name (PCDATA)gt lt!ELEMENT len
(PCDATA)gt lt!ELEMENT mass (PCDATA)gt lt!ELEMENT
time (PCDATA)gt lt!ELEMENT place (PCDATA)gt
XML DTD

• type set of dinosaur dinolist
• type record dinosaur
• charstring name,
• charstring len,
• charstring mass,
• charstring time,
• charstring place

Set of Type Definition
Record Type Definition
Field Definition
19
Test Data Definitions
Example

• template dinolist DinoList
• ?, ?, Brachiosaurus, ?, ?, ?, ?
• template dinosaur Brachiosaurus
• name "Brachiosaurus",
• len ?,
• mass ?,
• time ?,
• place ?

Any Value
20
Test Port
Example
Port Definition

• / communication port type definition /
• type port httpTestPortType message
• out url
• in dinolist

21
Test Components
Example
Component Definition

• / component type definitions /
• type component httpTestComponent
• port httpTestPortType httpPort
• timer localTimer 3.0
• type component httpTestSystemComponent
• port httpTestPortType httpTestSystemPort

Local Port
Local Timer
22
Test Behavior
Example
Sending a message
Starting the timer

• httpPort.send(requestURL)
• localTimer.start
• alt
• httpPort.receive(DinoList)
• localTimer.stop
• setverdict(pass)
• httpPort.receive
• localTimer.stop
• setverdict(fail)
• localTimer.timeout
• setverdict(fail)

Alternative reactions
The expected response
An unexpected response
A timeout
23
Simplification Altstep
Example
Test Component Type

• altstep DinoList_Default_1()
• runs on httpTestComponent
• httpPort.receive
• localTimer.stop
• setverdict(fail)
• localTimer.timeout
• setverdict(fail)

Handling of unexpected response
Handling of timeouts
24
Simplification Use of the Altstep
Example
Default activation

• ...
• activate(DinoList_Default_1())
• httpPort.send(requestURL)
• localTimer.start
• httpPort.receive(DinoList)
• localTimer.stop
• setverdict(pass)

The expected response is given here only, All
other cases are handled by the default
25
A Test Case
Example
Test Case Definition

• testcase DinoList_Test_1()
• runs on httpTestComponent
• system httpTestSystemComponent
• map(mtchttpPort, systemhttpTestSystemPort)
• activate(DinoList_Default_1())
• httpPort.send(requestURL)
• localTimer.start
• httpPort.receive(DinoList)
• localTimer.stop
• setverdict(pass)

MTC Type
TSI Type
Mapping the Ports
26
Module
Example
Module Name

• module dinolistTest
• type record url ...
• type set of dinosaur dinolist
• type record dinosaur ...
• template url requestURL ...
• template dinolist DinoList ...
• template dinosaur Brachiosaurus ...
• type port httpTestPortType message ...
• type component httpTestComponent ...
• type component httpTestSystemComponent ...
• altstep DinoList_Default_1() runs on
  httpTestComponent ...
• testcase DinoList_Test_1() runs on
  httpTestComponent
• system httpTestSystemComponent ...

The Complete Module
27
Graphical Format
map(mtchttpPort,systemhttpTestSystemPort)
28
Content

• Overview on TTCN-3
• TTCN-3 applied to software
• TTCN-3 and XML
• Test Execution

29
Test Execution
Abstract Test System Interface
30
TTCN-3 Execution

• Generic XML adaptor that adheres to the mapping

Test System User
Test System
TM Management
TE
CD CoDec
CH ComponentHandling
SA Communication
PA Timer
System Under Test (SUT)
31
TTCN-3 Runtime Interface

• Adaptation to the SUT

32
TTCN-3 Control Interfaces

• Adaptation to the test platform/test device

33
Summary of TTCN-3

• New version of the only standardized test
  notation
• Modernization Programming-like test
  specification with flexible data support and
  various representation formats
• Wider scope of application
• applicable to many kinds of test applications not
  just conformance (development, system,
  integration, interoperability, scalability )
• applicable in the datacom domain
• Harmonization
• first choice for test specifiers, implementors
  and users both for standardized test suites and
• as a generic solution in industrial software
  development
• Was successfully shown to be applicable to
  testing of IDL and XML interfaces, Java and C
  classes,
• Tools are available

34
Thank You.Questions?
35
The TTCN-3 Set of Standards

• ETSI ES 201 873-1 TTCN-3 Core Notation
• ETSI ES 201 873-2 TTCN-3 Tabular Presentation
  Format
• ETSI ES 201 873-3 TTCN-3 Graphical Presentation
  Format
• ETSI ES 201 873-4 TTCN-3 TTCN-3 Semantics
• ETSI ES 201 873-5 The TTCN-3 Runtime Interface
• ETSI ES 201 873-6 The TTCN-3 Control Interfaces