Title: Qualitative and Quantitative Analysis of Software Systems: a Modelbased View on Integrated Analysis
1Qualitative and Quantitative Analysis of Software
Systemsa Model-based View on Integrated
Analysis
Ed Brinksma University of Twente,
Netherlands ISSTA/Wosp Rome, July 24th, 2002
2MOTIVATION
Can the qualitative and quantitative aspects
of reactive systems be modelled and analysed
within one compositional framework?
Central Issue
- increasing importance of quantitative behaviour
- need for integrated design disciplines
- cross-fertilization
- theory of approximate correctness
3Contents
- Analysis, validation, and modelling
- Design, composition, and algebra
- Model-based test generation
- Integrated performance analysis
- Conclusions perspective
4Modelling
- science vs. engineering
- software programs as models
- models of software systems
- the role of experimentation
5The Empirical Cycle
6Methodology of Science
(Vienna Circle Carnap et al.)
- a normative procedure
- when is a theory is acceptable?
- models are refined by iteration
- models play a descriptive role
7Engineering
- models (specs) are prescriptive
- deals with artefacts
8The Design Cycle
9Methodology of Engineering
- design cycle is normative, a posteriori
-
- artefact is refined by iteration
- can employ theory of underlying
- empirical cycles
10The formal nature of software
11Programming as mathematics
(Dijkstra, Hoare et al.)
- programs are formal models of their realizations.
- program derivation and verification are
mathematical activities, subject to proofs - validation can be achieved through reliable
compilation/interpretation
12Limits of the Mathematical Paradigm
combinatorial explosion
large incomplete
incomplete
unreliable
13System Modelling
specification fragment
14System Modelling
- Abstract models of software
- Interpolation specs and programs
- Basis for verification
- Basis for validation
like traditional engineering models
15System Model Validation
- Top-down specification refinement
- Bottom-up implementation abstraction
- Formally by proof or by construction
- Empirically by experimentation
16Empirical Validation
analyse
17Contents
- Analysis, validation, and modelling
- Design, composition, and algebra
- Model-based test generation
- Integrated performance analysis
- Conclusions perspective
18Design and Complexity
- Divide and conquer distribute functionality
- Hierarchical design different abstraction levels
Modelling formalism must support compositionality
and abstraction
19Navigating the Design Graph
Modelling formalism must support model
transformation
Abstraction level
Alternatives
20Process Algebra
- abstract process modelling
- compositionality
- abstraction
- transformation laws
- operational interpretation
21Basic Process Algebraic Operators
- inaction 0
- action-prefix a.B , ?.B
- choice B C , ?I Bi
- composition B AC
- hiding B \ A
- definition p def B
- application p
22A very basic example
in
out
- A simple one place buffer
Buf
Buf def in . out . Buf
- Two instances of this buffer
mid
in
out
Buf
hide mid in mid
mid
23A very basic example II
Buf2 def in.Half Half def in.Full out.Buf2
Full def out.Half
24Equivalence
Two ways to represent a two place buffer
examples for the need to study equivalences
25Equivalence
- Process algebraic equivalences are based on
different answers to the question - What is an observable part of the behaviour of a
process ? - Various notions have been studied (see van
Glabbeek) - trace equivalences
- testing equivalences
- bisimulation equivalences
26Algebraic Laws
Equivalences (congruences) induce algebraic laws
B C C B (B C) D B (C D) B 0
B B B B
B A C C A B (B A C) A D B A (C
A D)
27Expansion Laws
Parallelism can be removed step by step
Let B ?k ak .Bk and C ?l cl .Cl
B A C ?ak .(Bk A C ) ak? A
?cl .(B A Cl ) cl? A ?d .(Bk
A Cl ) d ak cl ?A
Example a.0? c.0 a.c.0 c.a.0
28Expansion Laws
These laws are crucial for the success of process
algebraic modelling
- move between abstraction levels
- (de)compose functionality
- stepwise simulation
29Contents
- Analysis, validation, and modelling
- Design, composition, and algebra
- Model-based test generation
- Integrated performance analysis
- Conclusions perspective
30Testing With Formal Methods
test generator
specification
tests
implements
test executor
verdict (pass/fail)
implementation
- Advantages formal approach
- algorithmic generation of sound tests
- formal validation of tests
31Correctness Relations
- synchronous communication
- testing preorder (De Nicola Hennessy, 1984)
- conformance preorder (Brinksma 1987)
- refusal testing (Phillips 1987, Langerak 1990)
- asynchronous communication
- queue-based testing (Brinksma Tretmans 1992)
- I/O-based testing (Phalippou,Tretmans,Segala
1993) - repetitive quiescence (Tretmans 1996)
- multiple I/O (Heerink Tretmans 1997)
32Asynchronous Test Contexts
implementation under test
test engine
- operating system
- test method
- communication buffers
( Env)/A
compositionality
33Formal Correctness
- Intuition
- I ioco-conforms to S iff
- if I produces output x after trace ?, then S can
produce x after ? - if I cannot produce any output after trace ?,
then S cannot produce any output after ?
(quiescence)
I ioco S def ?? ? Straces(s) out(I after ?)
? out(S after ?)
Example equation solver for y2x
ioco
34Formal Test Generation
Nondeterministic algorithm Generate a test case
t(S) from a transition system model with S a set
of states (initially S s0)
Apply the following steps recursively,
nondeterministically
35Test Generation Example
Test
Note to cope with non deterministic behaviour,
tests are not linear traces, but trees
36Validity of Test Generation
For every set of tests T generated with the
algorithm
Soundness generated test will never fail with
ioco-correct implementation I ioco S
implies I passes T
37Test Generation Tools for ioco
- TVEDA (CNET - France Telecom)
- derives TTCN tests from SDL specification
- developed from practical experiences
- TGV (IRISA - Rennes)
- derives tests in TTCN from LOTOS or SDL
- uses test purposes
- TorX (Côte de Resyste)
- on-the-fly test generation and execution
- uses LOTOS and Promela
38TorX Tool Architecture
concentrate on on-the-fly testing
39TorX ApplicationHighway Tolling System
40Results
- Test results
- Errors found during model validation (design
error) and during testing (coding error) - Automated testing
- beneficial high volume and reliability
- very flexible adaptation and many
configurations - Real-time
- How to cope with real time constraints ?
- Efficient computation for on-the-fly testing ?
- Lack of theory quiescence vs. time-out
41Contents
- Analysis, validation, and modelling
- Design, composition, and algebra
- Model-based test generation
- Integrated performance analysis
- Conclusions perspective
42 Markovian Process Algebra
- basic idea
- incorporate delays that follow
exponential
distributions into process algebra - MEMORYLESS
- Two distinct approaches
- associate delays to actions
- TIPP, PEPA, EMPA, ...
- introduce delays as orthogonal entities
- IMC (also MLOTOS)
43Interactive Markov Chains
- inaction 0
- prefix a . B , ? . B
- choice B C or ?I Bi
- composition B AC
- hiding B \ A or hide A in B
- definition p def B
- application p
- inaction 0
- prefix a . B , ? . B
- choice B C or ?I Bi
- composition B AC
- hiding B \ A or hide A in B
- definition p def B
- application p
(?). B ,
supports phase type distributions
44Algebraic Laws for IMC
- B C C B
- (B C) D B (C D)
- B 0 B
- a.B a.B a.B
These are the algebraic laws for strong Markovian
bisimulation, a straightforward combination of
strong bisimulation and lumpability.
45Interleaving Law
The interleaving law holds for IMC
This does not hold for general distributions!
46 Queuing Systems in IMC
hide enter,serve in CUSTOMER enter
QUEUE(0) serve SERVER
- arriving customers
- queue
- service clerk
CUSTOMER def (? ). enter . CUSTOMER
QUEUE(i) def ilt6-gt enter. QUEUE(i1)
igt0-gt serve. QUEUE(i-1)
SERVER def serve . (? ) . SERVER
47 Queuing Systems in IMC
hide enter,serve in CUSTOMER enter
QUEUE(0) serve SERVER
?
48 Queuing Systems in IMC
hide enter,serve in CUSTOMER enter
QUEUE(0) serve SERVER
49A telephony system
- Original specification developed by P. Ernberg
(SICS), further studied in the French/Canadian
Eucalyptus project more than 1500 lines of
LOTOS.
- Extensively verified using state-of-the-art
techniques - model checking
- equivalence checking
50Performance analysis of the telephony system
using a dedicated operator, time constraints
- Takes the original
specification without changes. - Stochastic delays are incorporated
- in a compositional way, i.e. as additional
constraints
imposed on
the specification. - exponential, Erlang and phase-type
distributions. - Weak bisimulation is used to factor out
nondeterminism. - State space gt107 leads to a Markov Chain
of 720 states
with a highly irregular structure.
51Time constraints
- A particular phone
- The time it takes to pick up the phone
- The phone with time constraints
?
?
52Analysis results
- 14 different time constraints incorporated.
- Compositional minimisation to avoid state space
explosion. - Here two subscribers phoning each other.
53Tools used
- CAESAR/ALDEBARAN
- original specification,
- first minimisation steps.
- TIPPtool
- time constraints,
- final minimisations,
- numerical analysis.
54Contents
- Analysis, validation, and modelling
- Design, composition, and algebra
- Model-based test generation
- Integrated performance analysis
- Conclusions perspective
55Conclusions
- software engineering analysis need modelling,
like in traditional engineering - modelling formalisms must support composition,
abstraction transformation - process algebra provides theoretical framework
tool support (expansion laws) - integrated quantitative qualitative modelling
analysis by embedding models in specialized
contexts (extending theory where needed)
56Analytic Contexts
model
57Perspectives
- Extending specialized contexts
- Existing testing, soft hard real-time
- Future dependability, security
- Extending scope of analysis techniques
- Model checking CTMCs (ETMCC tool)
- Non-Markovian analysis
- Hard real-time testing (STRESS TorX Uppaal)
- Soft real-time testing
- Integrated modelling and tool support
- MoDest language and tools project
-