Reachability Analysis for Discrete Time Stochastic Hybrid Systems

1
Reachability Analysis for Discrete Time
Stochastic Hybrid Systems

• Saurabh Amin Alessandro Abate Shankar Sastry
• UCBerkeley

• Introduction
• Stochastic hybrid systems (SHS) can model
  uncertain dynamics and stochastic interactions
  that arise in many systems. An important problem
  in SHS theory is that of probabilistic
  reachability
• Probabilistic Reachability Problem
• What is the probability with which the system can
  reach a set during some time horizon?
• (If possible), select a control input to ensure
  that the system remains outside the set with
  sufficiently high probability
• When the set is unsafe, the problem becomes a
  quantitative safety verification problem. In this
  case, find the maximal safe sets corresponding to
  different safety levels
• Discrete Time Stochastic Hybrid System (DTSHS)
• DTSHS definition
• Where is the set of modes, the map
  defines the dimension of the continuous state
  space of these modes, and are the
  transition and reset control spaces, and ,
  , and are continuous, discrete, and reset
  stochastic kernels respectively
• DTSHS as controlled Markov process
• Motivational example
• Thermostat

• Computational Results
• Maximal probabilistic safe Sets for different
  safety levels

Dynamics of thermostat system
Executions generated by optimal policy for the
three safe sets
Optimal control policy for three different safe
sets
Continuous dynamics
Continuous and reset transition kernels
Off On
Off On
Discrete transition kernel
Switch action
Dont switch action
Stochastic Reachability
,
Consider Markov polices
Assume complete observability and finite time
horizon
Reach probability is the probability that the
execution of associated with policy
and initial distribution will enter set
during time
Probabilistic safe set is the set that guarantees
safety probability

for policy
Backward Reachability Computations
Since
we have
For computing reach probability for fixed Markov
policy ,
State space
define the functions by

Control space
Controlled transition kernel
which is
if
Then,
and
if
For initial mode OFF
For initial mode ON
and so,
Maximal Probabilistic Safe Set Computation
For safety level the maximal safe set
Dynamic programming recursion
Define the functions by

and so,
Then,
Existence of optimal policy is shown and