A Journey Through Model Checking Land

1
A Journey Through Model Checking Land

• Enrico Tronci
• Dipartimento di Informatica, Universita di Roma
  La Sapienza,
• Via Salaria 113, 00198 Roma, Italy
• Tronci_at_dsi.uniroma1.it
  http//www.dsi.uniroma1.it/tronci

2
Contents

• Goals
• Symbolic Model Checking
• Explicit Model Checking
• Hybrid Systems

3
Goal

• Automatic Analysis of Concurrent Systems

Example
Given an undesired state BAD (e.g. an error
state) we want to know under which conditions, if
any, our system can reach BAD.
4
HOW
System Model Param. Ranges Disturbances
Reqs (undesired/desired states)
Init States
Model Checker
Yes I.e. no sequence of events (states) can
possibly lead to an undesired state.
Counterexample I.e. sequence of events (states)
leading to undesired state.
5
Example (Simulation 1)
x(t 1) if x(t) lt 3 then x(t) u(t) else
x(t) u(t), u(t) 1, 2. x(0)
0
2
1
3
0
1
1
1
1
1
2
2
2
4
2
Sim length 10 1, 2, 1, 2, 1, 1, 2, 2, 2, 1
2
Spec x(t) lt 5. I.e. no state with x(t) gt 5 is
reachable.
Spec does not fail on this run
6
Example (Simulation 2)
x(t 1) if x(t) lt 3 then x(t) u(t) else
x(t) u(t), u(t) 1, 2. x(0)
0
1
3
2
2
0
1
5
1
1
2
4
2
Sim length 6 1, 2, 1, 2, 1, 2
Spec x(t) lt 5. I.e. no state with x(t) gt 5 is
reachable.
Spec FAIL
7
Example (Model Checking)
x(t 1) if x(t) lt 3 then x(t) u(t) else
x(t) u(t), u(t) 1, 2. x(0)
0
2
1
3
2
2
0
1
5
1
1
1
1
2
1
2
2
4
2
Spec x(t) lt 5. I.e. no state with x(t) gt 5 is
reachable.
Spec FAIL Spec ok if u(t) 0, 1.
8
Remark

• MC and Simulation have different, complementary
  goals.
• MC from the system model AND state X produces a
  sequence of stimuli (events) j, if any, leading
  to state X. (Obs. State Explosion)
• Simulation from the system model AND a sequence
  of stimuli (events) j shows where j leads (in j
  steps). (Obs. False negatives).

.
9
A Larger System
x(t 1) case x(t) 2 u(t) when x(t)
y(t) gt 4 x(t) 1 u(t) when x(t) y(t)
4 x(t) u(t) when x(t) y(t) 3
x(t) 1 u(t) when x(t) y(t) 2 x(t)
2 u(t) when x(t) y(t) lt 2 esac
y(t 1) u(t) u(t) -1, 0, 1
x,y
1,-1
2,-1
3,-1
-1
0
0,0
2,0
3,0
4,0
1
3,1
4,1
5,1
10
Constraining Disturbances
x(t 1) case x(t) 2 u(t) when x(t)
y(t) gt 4 x(t) 1 u(t) when x(t) y(t)
4 x(t) u(t) when x(t) y(t) 3
x(t) 1 u(t) when x(t) y(t) 2 x(t)
2 u(t) when x(t) y(t) lt 2 esac
y(t 1) u(t) u(t) -1, 0, 1 u(t) y(t) lt 1
x,y
1,-1
2,-1
3,-1
-1
Adding onstraints on disturbances we can control
nondeterminism in the model thus blending
simulation and model checking. This delay state
explosion but may lead to false negatives.
0
0,0
2,0
3,0
4,0
1
3,1
4,1
11
No Disturbances
x(t 1) case x(t) 2 u(t) when x(t)
y(t) gt 4 x(t) 1 u(t) when x(t) y(t)
4 x(t) u(t) when x(t) y(t) 3
x(t) 1 u(t) when x(t) y(t) 2 x(t)
2 u(t) when x(t) y(t) lt 2 esac
y(t 1) u(t) u(t) 0
0
0,0
2,0
3,0
12
MC Techniques

• Implicit (or Symbolic).
• Functions are used to represent set of states.
• Use functionals to represent transition
  relations, initial states, etc.
• Function representation MUST be compact. E.g.
  OBDDs.
• E.g. SMV (OBDDs), HyTech.
• Typically works well for hardware like systems
  (syncronous systems)
• Explicit.
• States are represented explicitely.
• Hash Table Based.
• E.g. Murphi, SPIN.
• Typically works well for software like systems
  (interleaving).
• E.g. protocols.

13
?-Calculus

• Formulas
• R(z1, zn), where R is an n-ary relational term
  and z1, zn are individual variables not free in
  R.
• ?f, f ? g, ?zf, where f and g are formulas and
  z is an individual variable.
• Relational terms of arity n
• P, where P is an n-ary relational variable.
• ?z1 znf, where f is a formula and z are
  distinct individual variables.
• ?PR, (least fix point) where P is an n-ary
  relational variable and R is an n-ary relational
  term that is formally monotone w.r.t. P.

Greatest fix point ?PR stands for ?PRltP
(P)gt
14
Example 1
N(x, y) transition relation I(x) initial
states Reachable states ?R?xI(x) ? ?zR(z) ?
N(z, x) Equation R(x) I(x) ? ?zR(z) ?
N(z, x) Spec (all reachable states satisfy P)
Idea (all states reachable from s satisfy P)
Q(s) P(s) ? ?N(s, t) ? Q(t) ?-formula
?Q?sP(s) ? ?N(s, t) ? Q(t) In fact AG P
P ? AX AG P
15
Example 2
Ni(x1, x2, xi) transition relation process i
(i 1, 2). Ii(xi) initial states process i (i
1, 2). Sync system N(x1, x2, x1, x2)
(N1(x1, x2, x1) ? N2(x1, x2, x2)) I(x1, x2)
(I1(x1) ? I2(x2)) x ? x1, x2 Reachable states
?R?xI(x) ? ?zR(z) ? N(z, x) Spec (all
reachable states satisfy P) Idea (all states
reachable from s satisfy P) Q(s) P(s) ? ?N(s,
t) ? Q(t) ?-formula ?Q?sP(s) ? ?N(s, t) ?
Q(t)
16
Example 3
Ni(x1, x2, xi) transition relation process i
(i 1, 2). Ii(xi) initial states process i (i
1, 2). Async system N(x1, x2, x1, x2)
(N1(x1, x2, x1) ? (x2 x2)) ? (N2(x1, x2,
x2) ? (x1 x1)) I(x1, x2) (I1(x1) ?
I2(x2)) x ? x1, x2 Reachable states
?R?xI(x) ? ?zR(z) ? N(z, x) Spec (all
reachable states satisfy P) Idea (all states
reachable from s satisfy P) Q(s) P(s) ? ?N(s,
t) ? Q(t) ?-formula ?Q?sP(s) ? ?N(s, t) ?
Q(t)
17
Models 1
M ltD, IR, IDgt, where D non-empty set called
the domain of the model IR relational variable
interpretation, IR(P) ? (Dn ? 0, 1) ID
individual variable interpretation, ID(y) ? D.
ID set of IDs IR set of IRs D maps
formulas to (IR ? (ID ? 0, 1)), n-ary
relational terms to (IR ? (ID ? (Dn ? 0, 1)))
18
Models 2

• N-ary relationl terms
• D(R(z1, zn))(IR)(ID) 1 iff ltID(z1), ID(zn)gt
  ? D(R)(IR)(ID)
• Formulas
• D(?f)(IR)(ID) ? D(f)(IR)(ID)
• D(f ? g)(IR)(ID) D(f)(IR)(ID) ? D(g)(IR)(ID)
• D(?zf )(IR)(ID) ?e?DD(f)(IR)(IDltz egt)
• Relational terms
• D(P)(IR)(ID) IR(P)
• D(?z1 znf)(IR)(ID)
• lte1, engt ? Dn
  D(f)(IR)(IDltz1 e1, zn engt)
• D(?PR)(IR)(ID) Z, where Z ? Dn is the least
  fix point solution of the equation Z D(P)(IR
  ltP Zgt)(ID)
• M ? f stands for D(f)(IR)(ID) 1

19
Model Checking Problem
Given M ltD, IR, IDgtand a formula f check if M
? f holds.
M ltD, IR, IDgt defines the system to be verified
whereas f defines a requirement for the
system. For us D 0, 1 (finite state
systems). Thus IR(R) is a boolean function. We
use OBDDs to efficiently represent and manipulate
boolean functions. Indeed OBDDs represent
boolean expressions. Thus to represent function
?z1 zn. f(z1, zn) we use expression f(z1,
zn). That is we must choose a set of (free)
variables for boolean expressions.
20
OBDD based MC Algorithm
Given M ltD, IR, IDgt and f compute D(f) using
OBDDs.
An n-ary relational variable is represented with
an n-ary boolean expression on fresh vars d1,
dn.
OBDDf(f, IR) returns an OBDD s.t. OBDDf(f,
IR)(ID) 1 iff D(f)(IR)(ID) 1 OBDD_ATOM(v)
1 iff v 1
21
OBDDf
Function OBDDf(f formula, IR rel-interp)
OBDD Begin case f is an individual variable
return OBDD_ATOM(f) f is of the form (f1 ?
f2) return OBDD_AND(OBDDf(f1, IR), OBDDf(f2,
IR)) f is of the form (? f1) return
OBDD_NOT(OBDDf(f1, IR)) f is of the form ?zf1
return OBDD_EXISTS(z, OBDDf(f1, IR)) f is
of the form R(x1, xn) return OBDDR(R,
IR)ltd1 x1, dn xngt esac End
22
OBDDR
Function OBDDR(R rel-term, IR rel-interp)
OBDD Begin case R is a relational variable
return IR(f) R is of the form ?z1 znf
return OBDDf(f, IR)ltz1 d1, zn dngt
R is of the form ?PQ return
FIXED_POINT(P, Q, IR, FALSEOBDD) esac End
23
FIXED_POINT
Function FIXED_POINT(P rel-var, R rel-term, IR
rel-interp, Z OBDD)
OBDD Begin do Z Z Z
OBDDR(R, IRltP Zgt) while (Z ! Z)
return(Z) End
24
Idea
Let I be the (caracteristic function for the) set
of initial states and let N be the transition
relation for our sysytem. The (caracteristic
function for the) set of reachable states R can
be defined as follows ?R?xI(x) ? ?zR(z) ?
N(z, x). R can be computed as
follows R(0)(x) 0 R(k 1)(x) I(x) ?
?zR(k)(z) ? N(z, x) When R(k 1)(x) R(k
)(x) the least fix point has been reached
and R(x) R(k )(x).
25
CTL
A set of atomic propositions.

• Syntax of CTL formulas
• Every p?A is a CTL formula
• If f, g are CTL formulas, then so are ?f, f ? g,
  EX f, Ef U g, EG f.

26
Models of CTL

• M ltA, S, L, N, S0gt, where
• A set of atomic propositions
• S finite set of states
• L S ? 2A (state labeling)
• N ? S x S total transition relation
• S0 ? S set of initial states.

Paths

• A path ? is an infinite sequence of states s0,
  s1, s.t. N(si, si 1) for every i.
• ?(i) si.
• Path(s) ? ?(0) s.

27
CTL Semantics

• M, s ? p iff p ? L(s)
• M, s ? ?f iff not M, s ? ?f
• M, s ? f ? g iff (M, s ? f and M, s ? g)
• M, s ? EX f iff ?t?S s.t. N(s, t) and M, t ? f
• M, s ? Ef U g iff
• ? ? ? Path(s) ? i?0 s.t. M, ?(i) ?
  f and ?j lt i M, ?(j) ? f .
• M, s ? EG f iff ? ? ? Path(s) ?i?0 M, ?(i) ? f

28
From CTL to ?-calculus
CTL M ltA, S, L, N, S0gt ? -calculus M ltD,
IR, IDgt
IR defines N and S0. IR(p)(s) 1 iff p ?
L(s). CTL formula f seen as abbreviation of ?
-calculus relational term R. Then f is true at
state s iff R(s) 1.

• p ?sp(s)
• ?f ?s?f(s)
• f ? g ?sf(s) ? g(s)
• EX f ?s?tN(s, t) ? f(t)
• EG f ?Q?sf(s) ? ?tN(s, t) ? Q(t).
• In fact EG f is the greatest fixpoint of the
  equation EG f f ? EX EG f. Thus
• EG f ?Qf ? EX Q ?Q?sf(s) ? ?tN(s, t) ?
  Q(t).
• Ef U g ?Q?sg(s) ? (f(s) ? ?tN(s, t) ?
  Q(t)).
• In fact Ef U g g ? (f ? EX Ef U g) ?Qg
  ? (f ? EX Q)
• ?Q?sg(s) ? (f(s) ? ?tN(s, t) ? Q(t)).
• f true iff f true on all initial states f ?s
  S0(s) ? f(s)

29
OBDDs 1
?x1
x1 ? x2
x1 ? x2
x2
1
1
1
0
0
0
1
2
1
1
0
1
1
0
30
OBDDs 2
OBDDs represent f(x1, xn) in a canonical way
once an ordering on the variable x1, xn is
given. Equality test O(1). If_then_else(F, G,
H) computable in O(max(F, G, H)). OBDDs
often compact on boolean functions occurring in
practice (isotropic boolean functions).
31
Explicit Model Checking
States represented explicitely rather than
symbolically. State space explored with a search
(BF or DF). Hash table used to store visited
states. E.g. Murphi, SPIN.
32
Murphi Mutex OK
Rule "t2c move 1"(statei t2c) (state1-i
nc) gt begin statei cr endRule "t2c
move 2"(statei t2c) (state1-i t2c)
(turn i)gt begin statei cr endRule
"cr moves"(statei cr) gt begin statei
ncendRule "turn 1"(state0 nc) (state1
t2c)gt begin turn 1 endRule "turn
0"(state1 nc) (state0 t2c)gt begin
turn 0 endEnd -- rulesetInvariant(state
0 ! cr) (state1 ! cr)
CONSTN 1TYPE state_type enumnc, cr,
t2cVARstate array 0..N of state_typei
0..N -- proc indexturn 0..N-- Define
initial stateStartstateBeginstate0
ncstate1 ncturn 0i
0End Ruleset i 0..N DoRule "nc moves
(statei nc) gt begin statei t2c end
33
Murphi Output on Mutex OK
Protocol mutex.okAlgorithm Verification by
breadth first search. with symmetry algorithm 3
-- Heuristic Small Memory Normalization with
permutation trial limit 10.Memory usage The
size of each state is 32 bits (rounded up to 4
bytes). The memory allocated for the hash
table and state queue is 8 Mbytes. With two
words of overhead per state, the maximum size
of the state space is 998651 states. Use
option "-k" or "-m" to increase this, if
necessary. Capacity in queue for breadth-first
search 99865 states. Change the constant
gPercentActiveStates in mu_prolog.inc to
increase this, if necessary.

Status No error found.State Space
Explored 16 states, 34 rules fired in 0.10s.
34
Murphi Mutex OKOK
CONSTN 1TYPE state_type enumini, nc,
cr, t2cVARstate array 0..N of
state_typei 0..N -- proc indexturn
0..N-- Define initial stateStartstate Begin
state0 ini state1 ini turn
0EndRuleset i 0..N DoRule "ini (statei
ini) gt begin statei nc endRule "ini
(statei ini) gt begin statei t2c
endRule "ini"(statei ini) (state1 - i
! cr) gt begin statei cr endend --
ruleset
Ruleset i 0..N DoRule "nc moves (statei
nc) gt begin statei t2c endRule "t2c
move 1"(statei t2c) (state1-i nc) gt
begin statei cr endRule "t2c move
2"(statei t2c) (state1-i t2c) (turn
i) gt Begin statei cr endRule "cr
moves (statei cr) gt begin statei nc
endRule "turn 1"(state0 nc) (state1
t2c)gt begin turn 1 endRule "turn
0"(state1 nc) (state0 t2c)gt begin
turn 0 endEnd -- rulesetInvariant(state
0 ! cr) (state1 ! cr)
35
Murphi Output for Mutex OKOK
Protocol mutex.okokAlgorithm Verification by
breadth first search. with symmetry algorithm 3
-- Heuristic Small Memory Normalization with
permutation trial limit 10.Memory usage The
size of each state is 32 bits (rounded up to 4
bytes). The memory allocated for the hash
table and state queue is 8 Mbytes. With two
words of overhead per state, the maximum size
of the state space is 998651 states. Use
option "-k" or "-m" to increase this, if
necessary. Capacity in queue for breadth-first
search 99865 states. Change the constant
gPercentActiveStates in mu_prolog.inc to
increase this, if necessary.

Status No error found.State Space
Explored 23 states, 60 rules fired in 0.10s.
36
Murphi Mutex FAIL
Rule "t2c move 1"(statei t2c) (state1-i
nc) gt begin statei cr endRule "t2c
move 2"(statei t2c) -- (state1-i t2c)
-- OMITTED (turn i)gt begin statei cr
endRule "cr moves"(statei cr) gt begin
statei ncendRule "turn 1"(state0 nc)
(state1 t2c)gt begin turn 1 endRule
"turn 0"(state1 nc) (state0 t2c)gt
begin turn 0 endEnd -- rulesetInvariant
(state0 ! cr) (state1 ! cr)
CONSTN 1TYPE state_type enumnc, cr,
t2cVARstate array 0..N of state_typei
0..N -- proc indexturn 0..N-- Define
initial stateStartstateBeginstate0
ncstate1 ncturn 0i
0End Ruleset i 0..N DoRule "nc moves
(statei nc) gt begin statei t2c end
37
Murphi Output on Mutex Fail
Protocol mutex.failAlgorithm Verification by
breadth first search. with symmetry algorithm 3
-- Heuristic Small Memory Normalization with
permutation trial limit 10.Memory usage The
size of each state is 32 bits (rounded up to 4
bytes). The memory allocated for the hash
table and state queue is 8 Mbytes. With two
words of overhead per state, the maximum size
of the state space is 952381 states. Use
option "-k" or "-m" to increase this, if
necessary. Capacity in queue for breadth-first
search 95238 states. Change the constant
gPercentActiveStates in mu_prolog.inc to
increase this, if necessary.The following is
the error trace for the error Invariant
"Invariant 0" failed.
38
Mutex Fail Error Trace
Startstate Startstate 0 fired.state0ncstate1
nci0turn0---------- Rule nc moves, i1
fired.state1t2c---------- Rule t2c move 1,
i1 fired.state1cr----------
Rule nc moves, i0 fired.state0t2c----------
Rule t2c move 2, i0 fired.The last state of
the trace (in full) isstate0crstate1cri
0turn0---------- End of the error trace.

Result Invariant
"Invariant 0" failed.State Space Explored 14
states, 27 rules fired in 0.10s
39
Explicit State Space Exploration

• From the system definition we get (as for the
  symbolic case) the following funcionts
• Next(s) returning the set of successors of state
  s
• Start() returning the set of initial states
• Inv(s) returning true iff state s satisfies the
  invariants.

With such functions we get define a State Space
Exploration function. E.g. we can use a BFS or a
DFS.
40
BFS
Hash_Table T Queue Q bfs() for each
startstate s insert(T, s) enqueue(Q, s)
while (Q is not empty) s
dequeue(Q) check invariants for
s for all s in Next(s)
if (s is not in T) / fresh state
/ insert(T, s)
enqueue(Q, s)
41
Hash Compaction
States may take hudred of bytes. To save on RAM
we can store in T just state signatures h(s).
Usually a state signature takes 5 bytes or so. It
can be proved that the omission probability is
very low.
Hash_Table T Queue Q bfs() for each
startstate s insert(T, h(s)) enqueue(Q, s)
while (Q is not empty) s
dequeue(Q) check invariants for
s for all s in Next(s)
if (h(s) is not in T) / fresh
state / insert(T,
h(s)) enqueue(Q, s)
42
Caching
To save even more RAM we can forget some of the
state signatures in T We can forget about 50 of
the states in T and still get termination. This
means that we are using a cache rather than a
hash table. This work because protocol
transitions are local.
Cache T / Upon a cache collision we
overwrite / Queue Q / Q can be also on disk
here / bfs() for each startstate s
insert(T, h(s)) enqueue(Q, s) while (Q is
not empty or collision rate too high)
s dequeue(Q) check
invariants for s for all s in
Next(s) if (h(s) is not
in T) / fresh state /
insert(T, h(s)) enqueue(Q, s)