Bounded Probabilistic Model Checking with the Murphi Verifier

1
Bounded Probabilistic Model Checking with the
Murphi Verifier

• Enrico Tronci
• Computer Science Department, University of Rome
  La Sapienza, Via Salaria 113,
• 00198 Roma, Italy, tronci_at_di.uniroma1.it,
  http//www.dsi.uniroma1.it/tronci
• Joint work with
• G. Della Penna, B. Intrigila, I. Melatti, M.
  Venturini Zilli

FMCAD 2004 Formal Methods in Computer-Aided
Design, Austin, Texas, USA, November 14-17, 2004
2
Goal

• Given
• a stochastic system (e.g. a randomized protocol)
  M and
• a property ? for M,
• check automatically that M satisfies ?.

For example, a Probabilistic Model Checker can be
used to verify properties like The probability
that a message is not delivered after 0.1 seconds
is less than 0.8
3
Example
Low System (e.g. public info)
High System (e.g. private info)
NRL PUMP
Statistically modulated ACK
ACK
Buffer
LS
HS
Data
Data
The NRL Pump is a special purpose-device that
forwards data from a low (security) level system
LS to a high (security) level system HS, but not
conversely. Idea LS ACK delay is
probabilistically based on a moving average of HS
ACK delays.
NRL Pump (Probabilistic) Properties
LS
HS
Ready to send
Ready to receive
Minimize information flow from HS to LS.
Read Data
Got ACK
Send Data
Send ACK
Enforce reasonable performances, i.e.
ltaverage ACK delay as seen from LSgt ltaverage
HS ACK delaygt
Waiting ACK
Done
4
Probabilistic Model Checking

• Probabilistic Model Checking (PMC) typically
  uses
• Markov Chains to represent the system M to be
  verified
• A probabilistic logic (e.g. PCTL) to represent
  the property ? to be verified.

0.6
? the probability of reaching (from the initial
state 1) state 3 is less than 0.2 (TRUE). PCTL
tt U (s 3) ? 0.2
M
1
0.4
0.5

• the probability of reaching (from the initial
  state 1) state 3 is less than 0.1 (FALSE).
• PCTL tt U (s 3) ? 0.1

2
3
0.4
1.0
0.1
In fact, the probability of reaching state 3 is
exactly 0.16.
5
Bounded Probabilistic Model Checking

• Bounded Probabilistic Model Checking (BPMC)
  uses
• Markov Chains to represent the system M to be
  verified
• A probabilistic logic with bounded Untils (e.g.
  BPCTL) to represent the property ? to be verified.

As for BMC, BPMC is less expressive (and may be
more efficient) than PMC.
0.6
M
? the probability of reaching state 3 in at most
2 steps is less than 0.2 (TRUE).
PCTL tt U?2 (s 3)?0.2
1
0.4
? the probability of reaching state 3 in at most
2 steps is at least than 0.1 (TRUE).
PCTL tt U?2 (s 3)? 0.1
0.5
2
3

• the probability of reaching state 3 in at most
  1 step is at least 0.1 (FALSE).
  PCTL tt U?1 (s 3)? 0.1

0.4
1.0
0.1
In fact, the probability of reaching state 3 in
at most 2 steps is exactly 0.16, whereas state 3
is not reachable in 1 step.
6
State of the Art
Not surprising the main obstruction to
probabilistic model checking is
State Explosion
Roughly speaking there are 4 approaches

• Linear Algebra and Sparse Matrices, e.g. as in
  ETMCC
• Symbolic, OBDD based, e.g. PRISM
• Symbolic, MTBDD(Multi Terminal Binary Decision
  Diagrams) based, e.g. ProbVerus
• Hybrid, using MTBDDs allowing reals in 0, 1 on
  terminal nodes to represent the transition matrix
  and linear algebra algorithms to carry out the
  computation (PRISM).

Typically, PRISM hybrid approach is faster than a
probabilistic model checker based only on MTBDDs
and avoids the state explosion of a probabilistic
model checker based only on sparse matrices.
Since PRISM implements all of the above
approaches we will compare our results with PRISM
best performances.
7
Hybrid Systems Analysis
On (Discrete Time) Hybrid Systems we run quite
soon into state explosion using ANY of the
previously mentioned approaches (i.e. using
PRISM).

• Intuitively, here is why
• the number of states is too large to be handled
  using a linear algebra approach
• the complex math computations entailed by the
  system dynamics makes life very hard for OBDDs or
  MTBDDs.

8
Finite Horizon Probabilistic Murphi (FHP-Murphi)

• We have shown (CHARME03) that an explicit
  approach can overcome such limitations. Namely,
  we have extended the Murphi verifier as follows
  (FHP-Murphi)
• Added finite precision real numbers and
  probabilities
• Replaced the set of the initial states with a
  distribution of initial states
• Labeled Murphi rules with probabilities, so as
  to define Markov Chains
• Replaced Murphi deterministic invariant with a
  BPCTL formula of form tt U?k p?? stating that
  that the probability of reaching an error state
  in k steps is less than a given threshold ?.

• As for FHP-Murphi performances
• when PRISM can handle the verification task
  PRISM is faster than FHP-Murphi,
• however there are many verification tasks
  (typically those involving hybrid systems)
  feasible with FHP-Murphi and out of reach for
  PRISM because of state explosion.

9
Nested Untils Are Interesting
FHP-Murphi can only handle finite horizon safety
properties, that is BPCTL formuals of form tt
U?k p??.
However there are many interesting properties
that are not definable using formulas of form tt
U?k p??, and thus cannot be handled by
FHP-Murphi.
For example, robustness and reliability
properties are very useful when carrying out
requirements analysis. Typically such properties
have the following form
The probability of getting into an undesired
system state and staying there for a long time,
is low.
More formally There is a low probability of
reaching (within k steps) an undesired state s,
such that there is not an high probability of
reaching (within h steps) a non-undesired state
from s.
Example if a (probabilistic dining) philosopher
has not eaten since a long time (undesired
state), then he will soon eat, thus avoiding
perishing for starvation.
We need nested Untils to define reliability
properties (? defines undesired states)
tt U?k (? ? ?tt U?h ? ?? 1)?0
10
Our Goal
Using nested untils it is possible to express
interesting properties (e.g. robustness,
reliability). Moreover such properties are
particularly important for hybrid systems.
Thus it is worthwhile extending FHP-Murphi to
full BPCTL.
Remark 1. Of course PRISM can handle full BPCTL,
however when handling hybrid systems it dies
quite soon because of state explosion.
Remark 2. Of course explicit PCTL (no bound on
path lengths) would also be interesting, however
it is computationally harder than BPCTL.
Here we will show how FHP-Murphi can be extended
so as to handle full BPCTL.
11
FHP-Murphi Verification Algorithm
No-nested-untils FHP-Murphi uses a BF visit to
carry out its computations.
However, a BF visit is not effective when dealing
with nested untils.
Example. Suppose we want to check if s F
holds, where F tt U?k ??? , ?
tt U?h ??? and ? is an atomic
proposition.
.
Using the no-nested-untils FHP-Murphi algorithm
this requires to check that for all t reachable
from s in at most k steps it holds t ?. Since
? tt U?h ??? , this means that for all t
reachable from s in at most k steps we need to
start a (nested) BF visit to check if t tt
U?h ??? holds.
We will show how, using a Depth First (DF) visit
and a cache we can avoid most of these nested
visits.
12
Easy Case No Nested Untils
1
s8
1
s6
s5
1/3
1
1/2
s4 ?
1
1/3
s2
s0
1/2
s3
1/3
1
s1 ?
1
s7 ?
1
We want to verify if s0 tt U?2 ??0.5 .
Atomic formula ? holds in states s1 , s4 , s7.
After visitig s0, the probability of reaching a
state in which ? holds is 0
13
Easy Case No Nested Untils
1
s8
1
s6 ?
s5
1/3
1
1/2
s4 ?
1
1/3
s2
s0
1/2
s3
1/3
1
s1 ?
1
s7
1
We want to verify if s0 tt U?2 ??0.5
holds. Atomic formula ? holds in states s1 , s4
, s6.
After visitig s1 the probability of reaching a
state in which ? holds is (1/3)1 (1/3).
14
Easy Case No Nested Untils
1
s8
1
s6 ?
s5
1/3
1
1/2
s4 ?
1
1/3
s2
s0
1/2
s3
1/3
1
s1 ?
1
s7
1
We want to verify if s0 tt U?2 ??0.5
holds. Atomic formula ? holds in states s1 , s4
, s6.
After visitig s2 the probability of reaching a
state in which ? holds is (1/3) (1/3)
15
Easy Case No Nested Untils
1
s8
1
s6 ?
s5
1/3
1
1/2
s4 ?
1
1/3
s2
s0
1/2
s3
1/3
1
s1 ?
1
s7
1
We want to verify if s0 tt U?2 ??0.5
holds. Atomic formula ? holds in states s1 , s4
, s6.
After visitig s4 the probability of reaching a
state in which ? holds is (1/3)
(1/3)(1/2)1
16
Easy Case No Nested Untils
1
s8
1
S6 ?
s5
1/3
1
1/2
s4 ?
1
1/3
s2
s0
1/2
s3
1/3
1
s1 ?
1
s7
1
We want to verify if s0 tt U?2 ??0.5
holds. Atomic formula ? holds in states s1 , s4
, s6.
After visitig s3 the probability of reaching a
state in which ? holds is (1/3)
(1/3)(1/2)1 (1/2)0
17
Easy Case No Nested Untils
1
s8
1
s6 ?
s5
1/3
1
1/2
s4 ?
1
1/3
s2
s0
1/2
s3
1/3
1
s1 ?
1
s7
1
We want to verify if s0 tt U?2 ??0.5
holds. Atomic formula ? holds in states s1 , s4
, s6.
After visitig s5 the probability of reaching a
state in which ? holds is (1/3) (1/3)(1/2)1
(1/2)0 (1/3)
18
Easy Case No Nested Untils
1
s8
1
s6 ?
s5
1/3
1
1/2
s4 ?
1
1/3
s2
s0
1/2
s3
1/3
1
s1 ?
1
s7
1
We want to verify if s0 tt U?2 ??0.5
holds. Atomic formula ? holds in states s1 , s4
, s6.
After visitig s5 the probability of reaching a
state in which ? holds is (1/3) (1/3)(1/2)1
(1/2)0 (1/3)1
19
Easy Case No Nested Untils
1
s8
1
s6 ?
s5
1/3
1
1/2
s4 ?
1
1/3
s2
s0
1/2
Never reached, since at distance 3 from s0.
s3
1/3
1
s1 ?
1
s7
Never reached, since ? holds in s1.
1
We want to verify if s0 tt U?2 ??0.5
holds. Atomic formula ? holds in states s1 , s4
, s6.
Finally, the probability of reaching a state in
which ? holds is (1/3) (1/3)(1/2)1 (1/2)0
(1/3)1 (1/3) (1/6) (1/3) (5/6) ? 0.5.
Thus s0 tt U?2 ??0.5 holds.
20
Nested Untils (Use Cache)
CACHE
1/2
s5

1/2
1/2
. . .
s0
1
s4
s3
..
1/2
s1
1/2
1
s2
1/2
We want to verify if s0 F holds, where
F ? ? U?k ??0.4 , ? ? tt U?h ??0.9 ,
? atomic formula.
Notation P(s, ? U?k ?) the probability that
(? U?k ?) holds on a path starting from s.
21
Nested Untils (Use Cache)
CACHE
1/2
s5
b3 P(s3, tt U?h ?) p3 P(s3, ? U?k-3 ?)
1/2
1/2
. . .
s0
1
s4
s3
..
1/2
s1
1/2
1
s2
1/2
F ? ? U?k ??0.4 , ? ? tt U?h ??0.9 , ?
atomic formula.
DF visit of s3 completed. We first compute P(s3,
tt U?h ?) and then P(s3, ? U?k-3 ?). Both
probabilities are stored in cache.
22
Nested Untils (Use Cache)
CACHE
1/2
s5
b3 P(s3, tt U?h ?) p3 P(s3, ? U?k-3 ?) p2
P(s2, ? U?k-2 ?)
1/2
1/2
. . .
s0
1
s4
s3
..
1/2
s1
1/2
1
s2
1/2
F ? ? U?k ??0.4 , ? ? tt U?h ??0.9 , ?
atomic formula.
DF visit of s2 completed. P(s2, ? U?k-2 ?)
1p3 computed and stored in cache.
Remark No need to revisit s3, we can use p3 in
cache.
23
Nested Untils (Use Cache)
CACHE
1/2
s5
b3 P(s3, tt U?h ?) p3 P(s3, ? U?k-3
?) p2 P(s2, ? U?k-2 ?)
1/2
1/2
. . .
s0
1
s4
s3
..
1/2
s1
1/2
1
s2
1/2
F ? ? U?k ??0.4 , ? ? tt U?h ??0.9 , ?
atomic formula.
Visiting s1. P(s1, ? U?k-1 ?) (1/2) P(s2, ?
U?k-2 ?) (1/2) P(s4, ? U?k-2 ?) (1/2)p2
Remark No need to revisit s2, we can use p2 in
cache. We need P(s4, ? U?k-2 ?).
24
Nested Untils (Use Cache)
CACHE
1/2
s5
b3 P(s3, tt U?h ?) p3 P(s3, ? U?k-3 ?) p2
P(s2, ? U?k-2 ?) p4 P(s4, ? U?k-2 ?)
1/2
1/2
. . .
s0
1
s4
s3
..
1/2
s1
1/2
1
s2
1/2
F ? ? U?k ??0.4 , ? ? tt U?h ??0.9 , ?
atomic formula.
Completed DF visit of s4. P(s4, ? U?k-2 ?)
1 P(s3, ? U?k-3 ?) 1p3.
Remark No need to revisit s3, we can use p3 in
cache.
25
Nested Untils (Use Cache)
CACHE
1/2
b3 P(s3, tt U?h ?) p3 P(s3, ? U?k-3 ?) b3
P(s3, tt U?h ?) p2 P(s2, ? U?k-2 ?) p4
P(s4, ? U?k-2 ?) p1 P(s1, ? U?k-1 ?)
s5
1/2
1/2
. . .
s0
1
s4
s3
..
1/2
s1
1/2
1
s2
1/2
F ? ? U?k ??0.4 , ? ? tt U?h ??0.9 , ?
atomic formula.
Completed DF visit of s1. P(s1, ? U?k-1
?) (1/2) P(s2, ? U?k-2 ?) (1/2) P(s4, ?
U?k-2 ?) (1/2)p2 (1/2)p4.
Remark No need to revisit s2, s4, we can use p2,
p4 in cache.
26
Nested Untils (Use Cache)
CACHE
1/2
s5
b3 P(s3, tt U?h ?) p3 P(s3, ? U?k-3 ?) p2
P(s2, ? U?k-2 ?) p4 P(s4, ? U?k-2 ?) p1
P(s1, ? U?k-1 ?)
1/2
1/2
. . .
s0
1
s4
s3
..
1/2
s1
1/2
1
s2
1/2
F ? ? U?k ??0.4 , ? ? tt U?h ??0.9 , ?
atomic formula.
Visiting s0. P(s0, ? U?k ?) (1/2) P(s1,
? U?k-1 ?) (1/2) P(s5, ? U?k-1 ?) (1/2)p1
.
Remark No need to revisit s1, we can use p1 in
cache. We need P(s5, ? U?k-1 ?).
27
Nested Untils (Use Cache)
CACHE
1/2
s5
b3 P(s3, tt U?h ?) p3 P(s3, ? U?k-3
?) p2 P(s2, ? U?k-2 ?) p4 P(s4, ? U?k-2
?) p1 P(s1, ? U?k-1 ?)
1/2
1/2
. . .
s0
1
s4
s3
..
1/2
s1
1/2
1
s2
1/2
F ? ? U?k ??0.4 , ? ? tt U?h ??0.9 , ?
atomic formula.
Visiting s5. P(s5, ? U?k-1 ?) (1/2)
P(s4, ? U?k-2 ?) (1/2) P(s3, ? U?k-2 ?)
(1/2)p4 .
Remark No need to revisit s4, we can use p4 in
cache. We need P(s3, ? U?k-2 ?) (not in cache)
28
Nested Untils (Use Cache)
CACHE
1/2
s5
b3 P(s3, tt U?h ?) p3 P(s3, ? U?k-3
?) p2 P(s2, ? U?k-2 ?) p4 P(s4, ? U?k-2
?) p1 P(s1, ? U?k-1 ?) q3 P(s3, ? U?k-2
?)
1/2
1/2
. . .
s0
1
s4
s3
..
1/2
s1
1/2
1
s2
1/2
F ? ? U?k ??0.4 , ? ? tt U?h ??0.9 , ?
atomic formula.
Visit s3 and compute P(s3, ? U?k-2 ?).
Remark This computation may use P(s3, tt U?h ?)
(b3) in cache.
29
Nested Untils (Use Cache)
CACHE
1/2
s5
b3 P(s3, tt U?h ?) p3 P(s3, ? U?k-3
?) p2 P(s2, ? U?k-2 ?) p4 P(s4, ? U?k-2
?) p1 P(s1, ? U?k-1 ?) q3 P(s3, ? U?k-2
?) p5 P(s5, ? U?k-1 ?)
1/2
1/2
. . .
s0
1
s4
s3
..
1/2
s1
1/2
1
s2
1/2
F ? ? U?k ??0.4 , ? ? tt U?h ??0.9 , ?
atomic formula.
Completed DF visit of s5. P(s5, ? U?k-1 ?)
(1/2) P(s4, ? U?k-2 ?) (1/2) P(s3, ?
U?k-2 ?) (1/2)p4 (1/2)q3.
Remark No need to revisit s4, s3, we use p4, q3
in cache.
30
Nested Untils (Use Cache)
CACHE
b3 P(s3, tt U?h ?) p3 P(s3, ? U?k-3 ?) p2
P(s2, ? U?k-2 ?) p4 P(s4, ? U?k-2 ?) p1
P(s1, ? U?k-1 ?) p6 P(s6, ? U?k-2 ?) q3
P(s3, ? U?k-2 ?) p5 P(s5, ? U?k-1 ?) p0
P(s0, ? U?k ?)
1/2
s5
1/2
1/2
. . .
s0
1
s4
s3
..
1/2
s1
1/2
1
s2
1/2
F ? ? U?k ??0.4 , ? ? tt U?h ??0.9 ,
? atomic formula.
Completed DF visit of s0. P(s0, ? U?k
?) (1/2) P(s1, ? U?k-1 ?) (1/2) P(s5,
? U?k-1 ?) (1/2)p1 (1/2)p5.
Remark 1. No need to revisit s1, s5, we use p1,
p5 in cache.
Remark 2. The sketched procedure is carried out
on all U-subformulas of the BPCTL formula to be
verified.
31
Experimental Results

• To assess effectiveness of our approach we run
  experiments with
• Randomized protocols
• Hybrid Systems

32
Randomized Protocols

• For such experiments we used
• PZ Pnueli-Zuck probabilistic dining
  philosophers protocol, as found in PRISM
  distribution
• MPZ PZ modified so as to meet
  quality-of-service requirements
• LR Lehmann-Rabin probabilistic dining
  philosophers protocol
• MLR LR modified so as to meet
  quality-of-service requirements.

Property When an undesired state s is reached,
then the system almost always reaches, from s and
in a few steps, a non-error state. More
formally There is a low probability that we
reach in d steps an undesired state s, and there
is not a high probability of reaching in k steps
from s, a non-error state. BPCTL form tt
U?k (?und ? ?tt U?h ??err? 1)?0 (PZ, LR,
MPZ, MLR).
All experiments run on a Linux 500 MHz INTEL
Pentium III with 2GB RAM.
FHP-Murphi options -500m (use 500 MB RAM) PRISM
options default
33
Randomized Protocols Results
NPHIL Result Murphi MB PRISM MB Murphi T (s) PRISM T (s)
5 False 500 1.70 3411 1.318
6 False 500 14.30 gt 300,000 12.60
PZ
NPHIL Result Murphi MB PRISM MB Murphi T (s) PRISM T (s)
3 True 500 1.42 1792 2.018
4 True 500 23.56 142,337 103.414
LR
MPZ
WAIT NPHIL Result Murphi MB PRISM MB Murphi T (s) PRISM T (s)
4 5 False 500 917 12,838 1197
5 5 False 500 N/A 12,737 N/A
WAIT NPHIL Result Murphi MB PRISM MB Murphi T (s) PRISM T (s)
4 3 True 500 70 5006 536
3 4 True 500 N/A 111,481 N/A
MLR
PRISM better on PZ, LR. FHP-Murphi better on MPZ,
MLR
34
Hybrid Systems
We experimented with verification of a turbogas
control system (HSCC03), given a probability
distribution on the user demand. Because of state
explosion, this verification is feasible only
with FHP-Murphi (i.e. PRISM runs out of memory).
Property If the system reaches an undesired
state, then with high probability it is able to
return to a non-undesired state. More
formally There is a low probability that we
reach an undesired state s, such that there is
not a high probability of reaching a
non-undesired state from s. BPCTL form tt
U?k (? ? ?tt U?h ??? 1)?0 (? defines
undesired states).
All experiments run on a Linux 500 MHz INTEL
Pentium III with 2GB RAM. FHP-Murphi options
-500m (use 500 MB RAM)
35
Gas Turbine System
Disturbances electric users, param. var, etc
Settings
Fuel Valve Opening FG102
Controller
Gas Turbine (Turbogas)
Vrot, Texh, Pel, Pmc
Vrot Turbine Rotation speed Texh Exhaust smokes
Temperature Pel Generated Electric Power Pmc
Compressor Pressure
36
Hybrid System Results
MAX_D_U Visited States Rules Fired Horizon (k) CPU T (s) Probability
35 1.159e05 3.477e05 800 3.702e03 4.105e-03
45 4.098e04 1.229e05 700 1.314e03 1.793e-02
50 4.067e04 1.220e05 700 1.309e03 3.825e-02
Probability P(q, tt U?k (? ? ?tt U?h ???
1)), q initial state, h (k/100).
MAX_D_U Max speed of variation of user demand
(KW/sec).
37
Conclusions
Performances When PRISM terminates it is faster
than FHP-Murphi. However, PRISM is not suitable
for verification tasks involving (complex) math
computations. Examples of such tasks are
analysis of hybrid systems, analysis of
randomized protocols, performance analysis, etc.
FHP-Murphi can handle such verification tasks
when the horizon is not too large.
Expressivness PRISM can handle BPCTL formulas
and much more. Hard to model hybrid systems using
PRISM. FHP-Murphi only handles BPCTL formulas.
Hybrid systems are easily modeled using
FHP-Murphi (since all C functions on real number
available).
Moral If you have to handle (complex) math use
FHP-Murphi, else use PRISM.
38
Future Work

• Infinite Horizon
• Continuous Markov Chains
• ..