Metrics for real time probabilistic processes

1
Metrics for real time probabilistic processes

• Radha Jagadeesan, DePaul University
• Vineet Gupta, Google Inc
• Prakash Panangaden, McGill University
• Josee Desharnais, Univ Laval

2
Outline of talk

• Models for real-time probabilistic processes
• Approximate reasoning for real-time probabilistic
  processes

3
Discrete Time Probabilistic processes

• Labelled Markov Processes

For each state s For each label a K(s, a,
U) Each state labelled with propositional
information
0.3
0.5
0.2
4
Discrete Time Probabilistic processes

• Markov Decision Processes

For each state s For each label a K(s, a,
U) Each state labelled with numerical rewards
0.3
0.5
0.2
5
Discrete time probabilistic proceses

• nondeterminism label does not determine
  probability distribution uniquely.

6
Real-time probabilistic processes

• Add clocks to Markov processes Each clock
  runs down at fixed rate r c(t)
  c(0) r t Different
  clocks can have different rates
• Generalized SemiMarkov Processes Probabilistic
  multi-rate timed automata

7
Generalized semi-Markov processes.
Each state labelled with propositional
Information Each state has a set of clocks
associated with it.
c,d
s
c
t
u
d,e
8
Generalized semi-Markov processes.
Evolution determined by generalized states
ltstate, clock-valuationgt lts,c2,
d1gtTransition enabled when a clockbecomes
zero
c,d
s
c
t
u
d,e
9
Generalized semi-Markov processes.
lts,c2, d1gt Transition enabled in 1
time unit lts,c0.5,d1gt Transition enabled in
0.5 time unit
c,d
s
c
t
u
d,e
Clock c
Clock d
10
Generalized semi-Markov processes.
Transition determines a. Probability
distribution on next states b.
Probability distribution on clock values
for new clocks
c,d
s
0.2
0.8
c
t
u
d,e
Clock c
Clock d
11
Generalized semi Markov proceses

• If distributions are continuous and states are
  finite Zeno traces have measure 0
• Continuity results. If stochastic processes
  from lts, gt converge to the stochastic process
  at lts, gt

12
Equational reasoning

• Establishing equality Coinduction
• Distinguishing states Modal logics
• Equational and logical views coincide
• Compositional reasoning bisimulation is a
  congruence

13
Labelled Markov Processes PCTL Bisimulation Larsen-Skou, Desharnais-Panangaden-Edalat
Markov Decision Processes Bisimulation Givan-Dean-Grieg
Labelled Concurrent Markov Chains PCTL Hansson-Johnsson
Labelled Concurrent Markov chains (with tau) PCTL Desharnais-Gupta-Jagadeesan-Panangaden Weak bisimulation Philippou-Lee-Sokolsky, Lynch-Segala
14
With continuous time
Continuous time Markov chains CSL Aziz-Balarin-Brayton-Sanwal-Singhal-S.Vincentelli Bisimulation,Lumpability Hillston, Baier-Katoen-Hermanns
Generalized Semi-Markov processesStochastic hybrid systems CSL Bisimulation????? Composition?????
15
Alas!
16
Instability of exact equivalence
Vs
Vs
17
Problem!

• Numbers viewed as coming with an error
  estimate. (eg) Stochastic noise as
  abstraction Statistical methods for
  estimating numbers

18
Problem!

• Numbers viewed as coming with an error estimate.
• Reasoning in continuous time and continuous space
  is often via discrete approximations. eg.
  Monte-Carlo methods to approximate probability
  distributions by a sample.

19
Idea Equivalence metrics

• Jou-Smolka, Lincoln-Scedrov-Mitchell-Mitchell
• Replace equality of processes by
  (pseudo)metric distances between processes
• Quantitative measurement of the distinction
  between processes.

20
Criteria on approximate reasoning

• Soundness
• Usability
• Robustness

21
Criteria on metrics for approximate reasoning

• Soundness
• Stability of distance under temporal evolution
  Nearby states stay close ' through temporal
  evolution.

22
Usability criteria on metrics

• Establishing closeness of states Coinduction.
• Distinguishing states Real-valued modal logics.
• Equational and logical views coincide Metrics
  yield same distances as real-valued modal logics.

23
Robustness criterion on approximate reasoning

• The actual numerical values of the metrics
  should not matter --- upto uniformities.

24
Uniformities (same)
m(x,y) 2x sinx -2y siny
m(x,y) x-y
25
Uniformities (different)
m(x,y) x-y
26
Our results
27
Our results

• For Discrete time models Labelled Markov
  processes Labelled Concurrent Markov
  chains Markov decision
  processes
• For continuous time Generalized
  semi-Markov processes

28
Results for discrete time models
Bisimulation Metrics
Logic (P)CTL() Real-valued modal logic
Compositionality Congruence Non-expansivity
Proofs Coinduction Coinduction
29
Results for continuous time models
Bisimulation Metrics
Logic CSL Real-valued modal logic
Compositionality ??? ???
Proofs Coinduction Coinduction
30
Metrics for discrete time probablistic processes
31
Bisimulation

• Fix a Markov chain. Define monotone F on
  equivalence relations

32
Defining metric An attempt

• Define functional F on metrics.

33
Metrics on probability measures

• Wasserstein-Kantorovich
• A way to lift distances from states to a
  distances on distributions of states.

34
Metrics on probability measures
35
Metrics on probability measures
36
Example 1 Metrics on probability measures
Unit measure concentrated at x
Unit measure concentrated at y
m(x,y)
x
y
37
Example 1 Metrics on probability measures
Unit measure concentrated at x
Unit measure concentrated at y
m(x,y)
x
y
38
Example 2 Metrics on probability measures
39
Example 2 Metrics on probability measures
THEN
40
Lattice of (pseudo)metrics
41
Defining metric coinductively

• Define functional F on metrics

Desired metric is maximum fixed point of F
42
Real-valued modal logic
43
Real-valued modal logic
Tests
44
Real-valued modal logic (Boolean)
q
q
45
Real-valued modal logic
46
Results

• Modal-logic yields the same distance as the
  coinductive definition
• However, not upto uniformities since glbs in
  lattice of uniformities is not determined by glbs
  in lattice of pseudometrics.

47
Variant definition that works upto uniformities

• Fix clt1. Define functional F on metrics

Desired metric is maximum fixed point of F
48
Reasoning upto uniformities

• For all clt1, get same uniformity
• see Breugel/Mislove/Ouaknine/Worrell
• Variant of earlier real-valued modal logic
  incorporating discount factor c characterizes the
  metrics

49
Metrics for real-time probabilistic processes
50
Generalized semi-Markov processes.
Evolution determined by generalized states
ltstate, clock-valuationgt Set of
generalized states
c,d
s
c
t
u
d,e
Clock c
Clock d
51
Generalized semi-Markov processes.

Path Traces((s,c)) Probability distribution
on a set of paths.
c,d
s
c
t
u
d,e
Clock c
Clock d
52
Accomodating discontinuities cadlag functions

• (M,m) a pseudometric space. cadlag if

53
Countably many jumps, in general
54
Defining metric An attempt

• Define functional F on metrics. (c lt1)

traces((s,c)), traces((t,d)) are distributions on
sets of cadlag functions. What is a metric on
cadlag functions???
55
Metrics on cadlag functions
x
y
are at distance 1 for unequal x,y
Not separable!
56
Skorohod metrics (J2)

• (M,m) a pseudometric space. f,g cadlag with range
  M.
• Graph(f) (t,f(t)) t \in R

57
Skorohod J2 metric Hausdorff distance between
graphs of f,g
g
f
(t,f(t))
f(t) g(t)
t
58
Skorohod J2 metric

• (M,m) a pseudometric space. f,g cadlag

59
Examples of convergence to
60
Example of convergence
1/2
61
Example of convergence
1/2
62
Examples of convergence
1/2
63
Examples of convergence
1/2
64
Examples of non-convergence
Jumps are detected!
65
Non-convergence
66
Non-convergence
67
Non-convergence
68
Non-convergence
69
Summary of Skorohod J2

• A separable metric space on cadlag functions

70
Defining metric coinductively

• Define functional on 1-bounded pseudometrics (c
  lt1)

a. s, t agree on all propositionsb.
Desired metric maximum fixpoint of F
71
Real-valued modal logic
72
Real-valued modal logic
73
Real-valued modal logic
h Lipschitz operator on unit interval
74
Real-valued modal logic
75
Real-valued modal logic
Base case for path formulas??
76
Base case for path formulas
First attempt
Evaluate state formula F on state at time t
Problem Not smooth enough wrt time since paths
have discontinuities
77
Base case for path formulas
Next attempt
Time-smooth evaluation of state formula F
at time t on path
Upper Lipschitz approximation to
evaluated at t
78
Real-valued modal logic
79
Non-convergence
80
Illustrating Non-convergence
1/2
1/2
81
Results

• For each clt1, modal-logic yields the same
  distance as the coinductive definition
• All clt1 yield the same uniformity. In this case,
  construction can be carried out in lattice of
  uniformities.

82
Proof steps

• Continuity theorems (Whitt) of GSMPs yield
  separable basis
• Finite separability arguments yield closure
  ordinal of functional F is omega.
• Duality theory of LP for calculating metric
  distances

83
Results

• Approximating quantitative observablesExpectati
  ons of continuous functions are continuous
• Continuous mapping theorems for establishing
  continuity of quantitative observables

84
Summary

• Approximate reasoning for real-time probabilistic
  processes

85
Results for discrete time models
Bisimulation Metrics
Logic (P)CTL() Real-valued modal logic
Compositionality Congruence Non-expansivity
Proofs Coinduction Coinduction
86
Results for continuous time models
Bisimulation Metrics
Logic CSL Real-valued modal logic
Compositionality ??? ???
Proofs Coinduction Coinduction
87
Questions?
88
Real-valued modal logic