Symbolic Algorithms for Verification and Control

1
Symbolic Algorithms forVerification and Control

• Rupak Majumdar (UC Berkeley)
• Joint work with
• Thomas A. Henzinger, Luca de Alfaro

2
Symbolic Model Checking

• Model Checking problem Given system M, and
  specification f, does M ² f ?
• Symbolic Model Checking
• Represent sets of states as constraints
• Algorithm manipulates sets of states
• Pre, Post, boolean operations
• Efficient BDD based methods

3
Symbolic Approach to Verification and Control

• Abstract symbolic algorithms
• No data structure considerations
• Termination of model checking algorithms
• Relationship between verification and control
  algorithms
• General class of structures

4
Outline

• Logics, equivalences, symbolic algorithms
• Classify symbolic transition systems
• Classify symbolic games
• Relate algorithms for verification and control
• Probabilistic games

5
Symbolic Algorithms forTransition Systems

• Model reactive systems as Labeled transition
  systems
• S Set of states (possibly infinite)
• ? Set of actions
• d S X ? ? S Successor function

6
Lifted Transition Systems

• Manipulate sets of states
• S Set of states, ? Set of actions
• Post 2S X ? ? 2S Successor function
• Post(S) s9 s2 S9 a 2S. d(s,a)s
• Pre 2S X ? ? 2S Predecessor function
• Pre(S) s9 a 2S. d(s,a)2 S

7
Symbolic Transition Systems

• S, ?, Pre, Post, ?
• Set of regions RR1,R2,, Ri?S
• ? ?R
• Pre, Post R X ??R
• ?,?,\ RXR?R
• ? RXR ? T,F

Computable
Symbolic semi-algorithm Start with regions in ?
and compute new regions using the operations above
8
Example Rectangular Hybrid Automata

• Polyhedral hybrid systems ACH
• Rectangular automata HKPV
• Timed automata AD94
• Symbolic representation
• Regions formulas in (R, )
• Pre and Post Quantifier elimination

9
Verification Questions

• Q1 Reachability
• Is an unsafe state reachable? EF unsafe
• Q2 Linear Temporal Logic (regular properties)
• Is progress being made? E(GF fair ? F goal)
• Q3 ½ Branching temporal logic(ECTL,ACTL)
• Nested reachability EF (unsafe ? EF err1 ? EF
  err2)
• Q4 Branching temporal logic (CTL)
• Is progress possible? AG(tick - EXEF tick)

10
Q1 Reachability EF

• Is there a trajectory to an unsafe state?

R final loop if R ? init?? then yes if
Pre(R) ? R then no R R ? Pre(R) end
. . .
init
final
final ?Pre(final)
Similar algorithm by iterating Posts
11
Algorithms m Calculus

• We encode symbolic algorithms as m-calculus
  formulas
• f p p x f1 Ç f2 f1 Æ f2 Pre(f)
• m x. f n x. f
• Expressive logic
• Can be implemented directly
• For example,
• EF p m x. p Ç Pre(x)

12
Q2 LTL Model Checking

• Example Repeated Reachability EGF
• Can a set of states be reached infinitely often?
• EGF final n y m x. (Pre(x) Ç (final Æ Pre(y)))

init
final
R
. . . .
Operations Pre,?, ? with observables
R2 EXEF R1
R1 EXEF final
13
Q3 ECTL Model Checking

• ECTL nested reachability
• EF(goal1 /\ EF(goal2) /\ EF(goal3))
• Operations Pre, ?, ?

EF (goal1 /\ EF goal2 /\ EF goal3)
EF goal3
EF goal2
goal1 /\ EF goal2 /\ EF goal3
14
Q4 CTL Model Checking

• CTL can all trajectories from init to goal1 be
  extended to goal2?
• AG(goal1 - EF goal2) EF (goal1 /\ EF goal2)
• Operations Pre, ?, ?, \

EF (goal1 /\ EF goal2)
EF goal2
15
Three Specification Logics

• L1 CTL (or, m calculus)
• L2 ECTL or ACTL
• L3 LTL

16
Three Symbolic Semi-Algorithms

• A1 Close ? under pre, ?, ?, \
• A2 Close ? under pre, ?, ?
• A3 Close ? under pre, ?, ?obs
• (intersection with observables)

P0 ? for i 1,2,3, Pi Pi-1 ? pre(R) R
? Pi-1 ? R1 ? R2
R1,R2 ? Pi-1 ? R1 ? R2 R1,R2
? Pi-1 ? R1 \ R2 R1,R2 ?
Pi-1 until Pi Pi-1
17
Three State Equivalences

• E1 Bisimilarity
• E2 Similarity (mutual simulation)
• E3 Trace Equivalence

18
Similarity

• Similarity moves can be matched
• Bisimilarity Symmetric similarity
• Trace equivalence same languages

?
?
19
Three Categories
Symbolic algorithms
State equivalences
Logics
L1 CTL L2 ECTL L3 LTL
A1 PreBoolean A2 Pre Positive
Boolean A3 Pre Positive Boolean
with ? only with observables
E1 Bisimilarity E2 Similarity E3 Trace
equivalence
20
Ai Symbolic semi-algorithm
Li State Logic
Model-checks
i 1,2,3
computes
induces
Ei State Equivalence
All regions definable by Li are generated by Ai
If Ai terminates, then symbolic model checking of
Li terminates
21
Ai Symbolic semi-algorithm
Li State Logic
Model-checks
i 1,2,3
computes
induces
Ei State Equivalence
States s and t are Ei equivalent iff for all
regions R generated by Ai, s?R iff t?R
Ai terminates iff Ei has finite index
22
Ai Symbolic semi-algorithm
Li State Logic
Model-checks
i 1,2,3
computes
induces
Ei State Equivalence
States s and t are Ei equivalent iff for all
formulas ? of Li, s satisfies ? iff t satisfies ?
If Ei has finite index, then Li can be model
checked on a finite quotient
23
Classification of systems STACS00

• STS1
• A1 terminates, finite bisimilarity, can model
  check CTL
• Ex Timed automata AD94,HNSY94
• STS2
• A2 terminates, finite similarity, can model check
  ECTL
• Ex 2D rectangular automata HHK95
• STS3
• A3 terminates, finite trace equivalence, can
  model check LTL
• Ex initialized rectangular automata HKPV98,
  STACS00

24
Why is this good?

• Useful in proving termination on specific models
• Gives symbolic algorithms at the same time
• Clarifies proofs for other algorithms
• Counterexample driven refinement for m-calculus
  CONCUR02
• Recipe for engineering model checkers at a high
  level independent of data structures
• BLAST model checker for C POPL02,CAV02

25
Classification of Games
26
Open SystemsGames on Components

• Transition systems are good models for closed
  systems
• Like to model interactions between components
• Games as models of interaction
• Example
• Plant Control

27
Control and Verification

• Can some component ensure a behavior no matter
  how the other components behave?
• Verification 9 Y
• Does there exist a run satisfying Y?
• Control h 1 i Y
• Does player 1 have a strategy to enforce Y on all
  outcomes?

28
Concurrent Games

• Two players
• Set of states S
• Finite set of actions S
• Transition function dSX S XS?S
• Controllable predecessor operation Cpre
• Cpre(S) s 9 a8 b. d(s,a,b)2 S

29
Example Rectangular Games

• Generalization of hybrid automata to games
  MPS95,AMPS98,CONCUR99
• Components (players) are explicit in the model
• Suitable for modeling hybrid control problems

30
Control Questions

• Q1 Controllability
• Can player 1 force the game to goal? F goal
• Q2 Linear Temporal Logic (regular properties)
• Omega regular games (GF fair ? F goal)
• Q3 ½ alternating temporal logic (1ATL,2ATL)
• Nested controllability
• F (unsafe ? F err1 ? F err2)
• Q4 Alternating temporal logic
• Nested boolean combinations of games
• G(tick - F tick)

31
Three Specification Logics

• GL1 ATL (or, alternating m calculus)
• GL2 1-ATL or 2-ATL
• GL3 ALTL

32
Three Symbolic Semi-Algorithms

• GA1 Close ? under Cpre, ?, ?, \
• GA2 Close ? under Cpre, ?, ?
• GA3 Close ? under Cpre, ?, ?obs
• (intersection with observables)

P0 ? for i 1,2,3, Pi Pi-1 ? Cpre(R)
R ? Pi-1 ? R1 ? R2
R1,R2 ? Pi-1 ? R1 ? R2 R1,R2
? Pi-1 ? R1 \ R2 R1,R2 ?
Pi-1 until Pi Pi-1
33
Three State Equivalences

• GE1 Alternating Bisimilarity AHKV
• GE2 Alternating Similarity (mutual simulation)
• GE3 Alternating Trace Equivalence

34
Alternating similarity

• Similarity moves can be matched
• Alternating (or game) similarity strategies can
  be matched

?
?
35
Alternating similarity

• Alternating (or game) similarity strategies can
  be matched

?
?
?
?
For example, if player 1 can force the game to
purple on the left, she can also force it to
purple on the right
36
Three Categories
Symbolic algorithms
Game equivalences
Logics
GL1 ATL GL2 1-ATL GL3 A-LTL
GA1 CpreBoolean GA2 Cpre Positive
Boolean GA3 Cpre Positive Boolean
with ? only with observables
GE1 Game bisimilarity GE2 Game
similarity GE3 Game trace
equivalence
37
GAi Symbolic semi-algorithm
GLi State Logic
Model-checks
i 1,2,3
computes
induces
GEi State Equivalence
All regions definable by GLi are generated by GAi
If GAi terminates, then symbolic model checking
of GLi terminates
38
GAi Symbolic semi-algorithm
GLi State Logic
Model-checks
i 1,2,3
computes
induces
GEi State Equivalence
States s and t are GEi equivalent iff for all
regions R generated by GAi, s?R iff t?R
GAi terminates iff GEi has finite index
39
GAi Symbolic semi-algorithm
GLi State Logic
Model-checks
i 1,2,3
computes
induces
GEi State Equivalence
States s and t are GEi equivalent iff for all
formulas ? of GLi, s satisfies ? iff t satisfies ?
If GEi has finite index, then GLi can be model
checked on a finite quotient
40
Classification of games CONCUR01

• GS1
• GA1 terminates, finite game bisimilarity, can
  model check ATL
• Ex Timed games MPS95
• GS2
• GA2 terminates, finite game similarity, can model
  check ATL
• Ex 2D rectangular games CONCUR99,CONCUR01
• GS3
• GA3 terminates, finite game trace equivalence,
  can solve LTL control
• Ex initialized rectangular games
  CONCUR99,CONCUR01

41
Control
Verification
Transition System
Game
Property ? Is there a run satisfying ??
Property ? How can we ensure ??
?
Algorithm to verify ?
Algorithm to control for ?
What is the relationship between these algorithms?
42
Algorithms for Verification and Control

• Consider only LTL verification and control
• Let f(Pre) be a m-calculus formula solving 9Y.
• When does f(Cpre/Pre) solve h1i Y?
• The actual algorithms for transition structures
  and games may be different

43
Co-Büchi PropertyEventually Always p
a
a
b
c
s3
s2
s1
Verification EFG s1, s3
s1, s2, s3
Verification Algorithm m X. Pre(X) Ç (n Y. s1,
s3 Æ Pre(Y) )
So EFG p m X. Pre(X) Ç (n Y. p Æ Pre(Y))
44
Co-Büchi Games
a,a
a,a
2
a,a
c,a
s3
s2
s1
Verification EFG s1, s3
s1, s2, s3
So EFG p m X. Pre(X) Ç (n Y. p Æ Pre(Y))
Control h 1 iFG s1, s3
s1, s2, s3
Control Algorithm m X. Cpre(X) Ç (n Y. p Æ
Cpre(Y) ) ??
NO h1iFG p ? m X. Cpre(X) Ç (n Y. p Æ Cpre(Y))
45
Dual Verification 9Y, 8Y
Verification problem 9Y
Is there a run satisfying Y?
Equivalent to h1i Y if player 2 has no choice
Solved using pre operator Pre (S) s 9
a2S.d(s,a) 2 S
Dual verification problem 8Y
Do all runs satisfy Y?
Equivalent to h2i Y if player 1 has no choice
Solved using 8 pre operator 8 Pre (S) s 8
a2S.d (s, a) 2 S
46
Extremal Model Theorem LICS01
For an LTL formula Y, f(Cpre) solves h 1i Y
iff f(Pre) solves 9Y, and f(8 Pre) solves 8Y

• The verification questions are extreme cases
  of a game where one of the players has no
  choice of moves
• An algorithm that solves the extreme games
  correctly also solves all games in between

47
Extremal Model Theorem
For an LTL formula Y, f(Cpre) solves h 1i Y
iff f(Pre) solves 9Y, and f(8 Pre) solves 8Y

• Sketch of Proof
• Towards a contradiction, suppose f() solves the
  verification questions, but not the control
  question.
• Fix a winning strategy of player 1 or 2
  depending on the direction of error. Show that
  f() cannot be correct on the resulting
  verification problem

48
Co-Büchi Games
a
a
b
c
s3
s2
s1
Verification AFG s1, s3
s1, s2, s3
Verification Algorithm m X. 8 Pre(X) Ç (n Y. p Æ
8 Pre(Y) ) ?
NO m X. (8 Pre(X) Ç (n Y. p Æ 8 Pre(Y) )) s2,
s3
49
Co-Büchi Games
a,a
a,a
a,a
c,a
s3
s2
s1
h1iFG p m X. n Y. (Cpre(X) Ç (p Æ Cpre(Y))
s1,s2, s3
50
LTL Verification

• Standard LTL - m-calculus compilations
  EL86,Dam94 convertLTL - nondet w-automaton -
  m calculus
• In particular, resulting formula for co-Büchi
  formulas does not work for games
• Question How do we find formulas that solve
  games (hence work in both cases)?

51
Solving LTL Games

• Construction goes through deterministic
  Rabin-chain (parity) automata
• Given a game G and a formula Y
• Solve a related game on a product structure with
  Rabin-chain winning condition using the EJ91
  algorithm
• From this, construct a m-calculus algorithm
  solving the original game
• 2EXPTIME algorithm

52
Solving LTL Games

• This also gives a symbolic algorithm for LTL
  control
• Moreover, the winning strategy can be synthesized
  symbolically CONCUR01

53
Extending Symbolic Algorithms to Probabilistic
Games
54
Concurrent Randomized Games
01 10
01 10
Iterated matching pennies
00 11
00 11
Probability to win with deterministic strategies
is 0
Player 1 has a randomized strategy to win with
probability 1/2
Quantitative winning!
55
Concurrent Games

• Two players
• Finite set of states S
• Finite set of actions S
• Probabilistic transition function
• d(s, a1, a2)(t) Pr t s, a1, a2

56
Winning Conditions w-regular sets
Safety
Reachability
B
Always in B
Reach B
B
Büchi
coBüchi
Visit B infinitely often
Eventually forever B
B
B
1
2
3
0
Rabin chain
The highest index visited infinitely often is even
57
Winning Conditions

• Value of a game is the maximal probability of
  ensuring the outcome is in Y
• h 1 iY(s) supx 1infx 2 Prsx 1x 2 Y

58
Boolean vs Quantitative
59
One-Step Game Ppre

• Regions are functions f S ! 0,1
• Maximal expectation of ensuring f(Q)
• Define the value
• Ppre (f) (s) supx 1infx 2ESf(Q)
• Equivalent to zero sum games
• Value and optimal strategies exist

60
One-Step Game

• Monotone and continuous
• Equivalent to zero-sum matrix games
• Value and optimal randomized strategies exist for
  both players vonNeumann
• Can be computed by linear programming

61
Reachability

• Maximal probability of reaching a set U of states
• Can be reduced to positive stochastic games
• Algorithm
• X0 0 Xn1 max(U, Ppre(Xn))
• X lim Xn
• Correctness is by induction on the n-step game

62
Reachability Example
01 10
01 10
S3
00 11
00 11
S1
S2
S4
Computing the least fixed point solution m x.
max (s4, Ppre(x))
63
Conjecture

• For reachability, f Ppre / Cpre gave
  corresponding algorithm for concurrent games
• Conjecture that the same holds for all properties
  of interest

64
Quantitative m calculus

• f p x fÇf fÆf Pre(f) m x.f n x.f

Normal m calculus
Quantitative m calculus
65
Proof Strategy
Strategy for Player 1 that ensures f - e
Proving h 1 iY ? f
Objective Y
Syntactically negate f
negate Y
Strategy for Player 2 that ensures f - e
Proving h 1 iY? f
Objective Y
66
Winning Conditions w-regular sets
Safety
Reachability
B
Always in B
Reach B
B
Büchi
coBüchi
Visit B infinitely often
Eventually forever B
B
B
1
2
3
0
Rabin chain
The highest index visited infinitely often is even
self dual
67
Safety

• Maximal probability of staying forever in a set U
  of states
• m-calculus algorithm n x. UÆ Ppre(x)
• Complement of the reachability formula
• (m x. UÇ Ppre(x)) n x. U Æ Ppre(x)
• Iterative approximation
• X0 1 Xi1 U Æ Ppre(Xi)

68
Safety

• Let w U Æ Ppre (w)
• Strategy While in U, play to maximize the
  probability of going to w in one step
• Define a random process (submartingale)
• Show that the nth stage of the random process
  bounds the max probability of staying in U for n
  steps
• Finally, show that the limit of the process as n!
  1 converges to the value of the safety game

69
Safety Proof

• Let w n x. U Æ Ppre(x)
• Consider the following strategy p1 of player 1
• s2 U play optimally in Ppre(w)(s)
• sÏ U play arbitrary
• Fix a state t and a strategy p2 of player 2

70
Safety Proof

• Define the process Hn as Hn w(Qn)
• For s2 U, we have w(s) Ppre(w)(s)
• From definition of p1 get for n 0
• Et Hn1 H0 Hn Hn
• So Et Hn H0 w(t)
• But Et Hn is bounded above by the event of
  staying in U for at least n steps
• Now take the limit as n! 1

71
Reachability and Safety

• For reachability optimal strategies may not
  exist, memoryless e-optimal strategies exist
• For safety memoryless optimal strategies exist
• Strategies may require randomization

72
Büchi and co-Büchi Games

• Büchi Maximal probability of visiting a set U
  infinitely often
• coBüchi Maximal probability of eventually
  always staying in a set U

n y. m x. (( U Æ Ppre(x)) Ç (U Æ Ppre(y)))
m x. n y. (( U Æ Ppre(x)) Ç (U Æ Ppre(y)))
73
Büchi Games

• Strategy construction uses arguments similar to
  the safety case
• Reach U, then reach the U again
• Optimal strategies may not exist
• e-optimal strategies for Büchi games may require
  infinite memory

74
Rabin-chain games

• Winning condition
• Let C S ! 0, , N-1 be a coloring of the
  states
• A trace satisfies the Rabin-chain condition if
  the maximum color appearing infinitely often is
  even.
• All LTL games can be reduced to a Rabin-chain
  game on a product structure

75
Rabin-chain games

• m calculus algorithm
• lN-1 m x1 n x0. Çi0N-1 (Ci Æ Ppre (xi))
• The classical algorithm EJ91 for boolean
  turn-based game has an identical syntactic form
• But the proof is different

76
Rabin-chain games

• Infinite memory e-optimal strategies exist
• Turn-based Rabin-chain games have deterministic
  and optimal winning strategies

77
Reachability
a,b
a,b
s
t
u
Reach u (t) (-32p 5)/5
78
Open Problems

• Complexity for quantitative concurrent games?
• Games can have irrational values
• Or approximation schemes based on
  value-iteration?
• Future work Discounted games, discounted
  equivalences

79
Open Problems

• Variants of Closure algorithms
• More general theorems on termination
• E.g., in software model checking
• Engineering Issues

80
Acknowledgments

• Ben Horowitz, Ranjit Jhala, Freddy Mang, George
  Necula, Jean-Francois Raskin, Greg Sutre, Wes
  Weimer