Probabilistic CEGAR* Bj

1
Probabilistic CEGARBjörn Wachter
To appear in CAV

• Joint work with Holger Hermanns, Lijun Zhang

Supported by
Uni Saar
AVACS
TexPoint fonts used in EMF. Read the TexPoint
manual before you delete this box. AAAAAAAAAA
2
Introducing

• Probabilistic Model Checking
• CEGAR (counterexample-guided abstraction
  refinement)
• PASS does CEGAR for probabilistic models

1
3
PRISM PASS

• PRISM
• Very popular probabilistic model checker
• Finite-state
• PASS
• Supports PRISM models
• handles infinite-state as well
• Under the Hood
• Predicate abstraction
• SMT
• Interpolation

4
Comparison to PRISM

• Network protocols
• Wireless LAN, CSMA
• Bounded Retransmission
• Sliding Window

PRISM vs PASS
Model () State reduction Speed-up
WLAN (3) WLAN (1) 16x-152x ? 1,3x-7x TO-gt311s
CSMA (4) 41x-248x 1x-2x
BRP (3) 1x 1/2x - 1/3x
5
Overview

• Basics
• Paths, Markov Chains, MDPs
• Counterexamples
• Probabilistic Programs
• Predicate Abstraction
• Abstraction Refinement
• Abstract Counterexamples
• Path Analysis
• Strongest Evidence
• CEGAR algorithm
• Experimental Results
• Conclusion

Probabilistic Reachability Problem
Program
e
6
Paths, MCs, MDPs

• Weighted
• Path
• Markov
• Chain
• non-determinism

1/3
2/3
1/3
1/3
2/3
1/3
1/3
1/3
7
Paths, MCs, MDPs

• Weighted
• Path
• Markov
• Chain
• Markov
• Decision
• Process

1/3
2/3
1/3
1/3
2/3
1/3
1/3
1/3
8
Adversary

• Adversary resolves transition non-determinism

1/3
2/3
1/3
1/3
1/3
9
Probabilistic Reachability

• Probability to get from green to red
• Weighted
• Path
• Markov
• Chain
• Markov
• Decision
• Process

1/3
2/3
1/3
1/3
2/3
1/3
1/3
1/3
1/3
1
2/3
1/3
1/3
1/3
1/2
1/2
10
Probabilistic Programs

• Guarded command language à la PRISM
• Variables integer, real, bool
• Non-determinism interleaving
• Example
• Program (variables, commands, initial condition)

x1
Labels for CEX Analysis
11
Predicate Abstraction

• Predicates partition the state space
• are boolean expressions
• xgt0, xlty, x y 3 (variables x,y)
• ? Abstract MDP
• Probabilistic may-transitions
• Similar to Blast, SLAM, Magic
• See our Qest07 paper
• Abstraction guarantees upper bound

Probability
1
Abstract MDP
actual
0
12
May Transitions

• Hier ists noch nicht verständlich genug!
• Besseres Beispiel wo abs. trans lt conc. trans

abstract
concrete
13
CEGAR Loop
abstract
check
Probability
p
?
CEX
refine
Low enough
Real CEX
14
Counterexamples (CEX)

• Resolution of non-determinism
• initial state
• adversary
• induces a Markov chain
• Counterexample
• Resolution of non-det
• such that probability threshold exceeded
• Example
• CEX for

Witness of Reachability probability in MDP
1/3
2/3
1/3
1/3
1/3
15
Counterexample Analysis Idea

• Idea
• Enumerate paths of Markov chain
• Sort paths by probability Han\Katoen2007
• visit paths with highest measure first
• Realizable Spurious

Path 1
Path 2
Path 3
Path 4

Path 1
Path 2
Path 3
Path 4

Probability of Abstract CEX / Markov Chain
How much MEASURE is REALIZABLE? More than p?
16
Path Analysis
Logic (SMT)

• Abstract path Two cases
• Realizable if theres a corresponding concrete
  path
• Spurious no corresponding path
• Splitter predicate exists iff path spurious
• Interpolation predicate from unsatisfiable path
  formula

Path formula
SAT
UNSAT
u
u
u
Reachable with prefix
u
u
Can do postfix
u
17
Path Analysis
Logic (SMT)

• Abstract path Two cases
• Realizable if theres a corresponding concrete
  path
• Spurious no corresponding path
• Splitter predicate (interpolant)

Path formula
SAT
UNSAT
u
u
u
u
u
u
0
x1
x0
xgt1
X 10
18
Example
Probability
Upper 1.0
0.8
0.2
0
concrete
abstract
0.8
0.5
0.5
19
Example(cont) after refinement
Probability
Upper 0.4
0.4
0
Concrete
abstract
0.8
0.5
20
Example 2
Upper 1.0
0.8
0.2
0
concrete
abstract
1.0
0.8
21
Example 2
Probability

• Find Maximal Combination by MAX-SMT (? paper)

Upper 1.0
0.8
0.8
0.2
0.8
0
concrete
abstract
0.2
1.0
0.8
Maximum
22
CEX AnalysisSemi decision procedure

• Problem in general undecidable
• Too many spurious paths ? abort counterexample
  analysis
• Output collection of predicates
• Enough realizable probability

Path 1
Path 2
Path 3
Path 4

Path 1
Path 2
Path 3
Path 4

gt C
Limit of spurious paths to enforce termination
Path 1
Path 2
Path 3
Path 4

Path 1
Path 2
Path 3
Path 4

Can take many paths To obtain enough
realizable probability
0
23
Related Work

• Probabilistic Counterexamples
• however not in the context of abstraction
• Hermanns/Aljazzar (FORMATS05) , Han/Katoen
  (TACAS07)
• Abstraction Refinement for Prob. Finite-state
  Models
• CEGAR for stochastic games, Chatterjee et al
  (UAI05)
• Not based on counterexamples
• DArgenio (Papm-Probmiv02), Fecher al
  (SPIN06) simulation
• Magnifying-lens, de Alfaro et al (CAV07)
  probability values

24
Conclusion Future Work

• Abstraction refinement
• Counterexamples Markov Chains
• Markov Chains have cycles
• Model Checking Infinite-state Probabilistic
  Models
• Speed-up for huge finite-state models
• Future Work
• Better Lower bounds

25
References

• Tool website
• http//depend.cs.uni-sb.de/pass
• Literature
• Our work
• Hermanns, Wachter, Zhang Probabilistic CEGAR
  (CAV08)
• Wachter, Zhang, Hermanns MC Modulo Theories
  (Qest07)
• Counterexamples
• Hermanns, Aljazar CEX for timed prob
  reachability, FORMATS05
• Han, Katoen CEX in probabilistic model checking,
  TACAS07
• Probabilistic Abstraction Refinement
• De Alfaro, Magnifying-lens abstraction for MDPs,
  CAV07
• Chatterjee, Henzinger, Majumdar CEX-guided
  planning, UAI05

26
Questions?
27

• Is Counterexample analysis problem undecidable?
• Semi-decision algorithm ? heuristics
• If we only need finiteley many paths
• ? decidable if logic is
• If we need infinitely many
• ? undecidable