Two Worlds: Abstractions in the Continuous World

1
Two WorldsAbstractions in the Continuous World

• Rupak Majumdar
• Max Planck Institute for Software Systems

2
Cyber-Physical Systems

• Software Controlled interactions
• with the physical world
• 2. Safety Critical
• Software a major component
• Boeing 747 50ECUs, 4M LOC
• ETCS Kernel 0.5MLOC
• Lexus 2006 100 CPUs, 7M LOC
• BMW 70-100CPUs, 100M LOC!

3
Cyber-Physical Systems

• Software Controlled interactions
• with the physical world
• 2. Safety Critical
• 3. Software is the hard part
• - Expensive, brittle
• - Low productivity, High QA cost
• - Major part of development cost

4
Control System Development
against system performance spec
Validate
Combine
Plant Model x Ax Bu
Environment spec
Controller Model u Kx
Control Software spec
Virtual World
Real World
Plant (Hardware)
Environment impl
Controller (SoftwareHardware)
Control impl
Combine
Validate
5
Formal Methods Challenges

• Verification
• How can we ensure a system meets its
  specifications?
• Synthesis
• How can we automatically construct controllers
  for temporal requirements?
• Abstraction and Robustness
• When are two systems close? When is a system
  robust?

6
This Talk FM in the Control World

• - Proof techniques for verification
• Epsilon-bisimulations and reactive synthesis
• Input-output robustness
• End-to-end arguments

7
Disclaimer

• Tutorial introduction to the field

8
Continuous Dynamical Systems

• Specification
• Stability Under the action of the controller,
  the dynamics converges to the origin

• f Dynamics
• u Input from the controller
• assume f is nice
• Trajectory Solution of the differential equation

9
Hybrid Dynamical Systems

Discrete constraint - Control task can only run
once every k cycles - The system must reach a
sequence of setpoints while avoiding bad
states - LTL specification
10
Verification Question

• Given a controller that claims to
• Stabilize the system
• Satisfy additional discrete constraints
• Check the controller works correctly

11
Synthesis Question

• Synthesize a controller that
• Stabilizes the system
• Satisfies additional discrete constraints

12
Formal Methods Perspective

• Verification
• Safety ? Inductive invariants
• Liveness ? Ranking functions
• Synthesis
• Controller design ? Reactive synthesis
• Q How do we apply these techniques to the
  continuous world?

13
Verification
14
Commonalities

• Formal Methods

• Control Theory

• Safety Show that program stays in safe states
• Liveness Show that program eventually terminates
• -Techniques (Discrete) Logic

• Safety Show that system stays in safe states
• Stability Show that system eventually goes to
  setpoint
• Techniques Real Analysis

15
Model
Problem Ensure no trajectory from Init reaches
Bad
16
Barriers B(x)
PrajnaJadbabaie04
Init
Bad
The dynamics pushes the state back at the
boundary of the barrier
17
Reachability
Target
18
Lyapunov functions L(x)
LyapunovB.C.
The dynamics pushes the state down along the
level sets of L(x)
19
Commonalities

• Formal Methods

• Control Theory

• Safety Show that system stays in safe states
• Barrier certificates
• Stability Show that system eventually goes to
  setpoint
• Lyapunov functions
• -Techniques Real Analysis
• Constraints?

• Safety Show that program stays in safe states
• Inductive invariants
• Liveness Show that program eventually terminates
• Rank functions
• Techniques (Discrete) Logic
• Horn clauses

20
Barriers/LF to Constraints
21
Constraints Polynomials

• Assume f(x) is a polynomial
• Fix polynomial template for B
• ? Polynomial constraints

22
Aside Sum of Squares

• Want to show
• p(x) 0
• Look for polynomials p1(x), , pk(x) s.t.
• p(x) p1(x)2 pk(x)2
• Sufficient but not necessary
• ? But search for sum of squares polynomials
  reduces to convex optimization (semi-definite
  programming)

23
Not just Safety/Reachability

• Horn clause formulations carry over
• - LTL, CTL, ATL DimitrovaM
• Idea for LTL
• Convert to parity conditions
• Certificate Sequence of functions V0,,Vk
• - even i ? barrier
• - odd i ? Lyapunov function that exits this
  color

24
Formal Methods Challenge

• Design numerically stable and scalable decision
  procedures for polynomial arithmetic
• Connect the search for barriers and Lyapunov
  functions to abstraction-refinement techniques

25
Synthesis
26
Controller Synthesis for LTL
Continuous system
Abstraction
?
Control input u
Reactive synthesis
Refinement
Discrete controller
27
e-Bisimulation
GirardPappas07,Tabuada
(x,y)?R means that every trajectory starting from
x is matched up to e by a trajectory from y and
vice versa
28
Controller Synthesis for LTL
Continuous system
Abstraction
Control input u
Reactive synthesis
Refinement
Discrete controller
When do finite bisimulations exist?
29
Incremental Stability
Angeli02

• Trajectories converge to each other as time
  progresses
• Incremental asymptotic stability (AS)
• x(t, x0, u) - y(t, y0, u) ß ( x0 y0
  , t)
• for all u
• Incremental input-to-state stability (ISS)
• x(t, x0, u) - y(t, y0, v) ß ( x0 y0
  , t)
• ?( u v )
• ß is KL, ? is K8

30
Incremental Stability, in Pictures

• Linear systems
• Asymptotic stability
• ( all eigenvalues have negative real part)
• ?
• incremental stability

31
Transition Systems

• Fix a sampling time t
• Transition system
• States Rn
• Labels Piecewise constant control inputs
• Transitions

32
Intuition

• Discretize state and input space
• Error accumulated due to discretization cancel
  out because of incremental stability

x
y
33
Finite Bisimilarity

• Fix an incremental ISS continuous system
• Fix precision e, sampling time t
• Theorem PolaGirardTabuada Can choose
  discretization parameters
• a (state discretization), b (input
  discretization)
• s.t. there is a finite e bisimulation

34
Extensions Stochastic Dynamics
ZamaniEfsahaniM.AbateLygeros

• Extend notions of incremental ISS to stochastic
  ones
• Finite epsilon-bisimulation (in the sense of
  expectations) exists for any compact set

35
Good News/Bad News

• Now discrete synthesis can be applied
• Tool Pessoa RoyM.Tabuada
• (coming up)
• Expensive procedure exponential in the dimension
  of the system

36
Example 1 Motion Planning
37
Example 1 Motion Planning
38
Example 1 Motion Planning
Abstraction 91035 states (585s) Control 155s
39
Example 2 DC Motor Speed Control
Spec
Abstraction 1M states, 150s, Controller found
in 4s
40
Formal Methods Challenges

• Better abstractions for bisimulations?
• - Using timed automata?
• (exponentially succinct representations)
• 2. Abstraction and refinement for control?

41
End-to-end Design
42
Control System Development
against system performance spec
Validate
Combine
Plant Model x Ax Bu
Environment spec
Controller Model u Kx
Control Software spec
Virtual World
Real World
Plant (Hardware)
Environment impl
Controller (SoftwareHardware)
Control impl
Combine
Validate
43
Controller Implementations

• Physical world and software implementations
  may not match up
• Resource constraints, finite precision,
  distributed computation
• Uncertainties in measurements/actuations
• How can we ensure that the implemented system
  correctly implements the controller?
• What does correctly mean?

44
Stability

• The physical plant converges to a desired
  behavior under the actions of the controller

Example In the steady state, the angular
velocity of a DC motor will be between 7.5 and
8.5 rad/s
Mathematical Model
Software Implementation
45
Stability
Example In the steady state, the angular
velocity of a DC motor will be between 7.5 and
8.5 rad/s
Mathematical Model
Software Implementation
Question What is the effect of implementation
error on system stability?
46
Effects of Implementation Error
?
Ideal, Mathematical Model
Implementation

• The software implementation introduces errors due
  to
• Limited precision arithmetic
• Quantization of sensing and actuation
• Computation times
• Can we bound the effect of error on the stability?

47
Bound on Errors

• Theorem AntaM.SahaTabuada10 If a is the L2 gain
  of a linear control system and b a bound on the
  implementation error, then
• ? a . b
• Separation of concerns
• Calculate L2 gain from the mathematical model
• Calculate implementation error from the code

48
Non-linear Systems

• System x f(x,u) Controller u k(x)
• Use an ISS Lyapunov function V, and the
  additional constraint from robust control theory
• ?V/?x . f(x,k(x)e) - ?V(x) s e
  

49
Intuition

• ?V/?x . f(x,k(x)e) - ?V(x) s e
  
• The Lyapunov function -?V(x) forces the dynamics
  down toward the origin
• The disturbance s e pushes the dynamics
  up away from the origin
• Guarantee When these balance out
• States for which V(x) s/? . e

50
Non-linear Systems Error Bounds

• Theorem AntaM.SahaTabuada10 If b is a bound
  on the implementation error, and s, ? as before
  for some Lyapunov function V, then ? s/? . b
• The value of s and ? can be found using Sum of
  Squares (SoS) optimization techniques

51
Error Sources

• Sampling errors Sampling a function at discrete
  points
• Quantization errors Finite precision arithmetic
• Assume that sampling errors are negligible (by
  sampling fast enough)
• Focus on quantization errors

52
Bounding the Error Finite Precision

• Only consider error due to finite precision
• Target fixed-point implementations
• Each real variable is implemented using n bits,
  with k bits for the fractional part

k
n
53
Fixed Point Arithmetic

• Can perform arithmetic operations on this
  representation (using bitshifts and arithmetic)

k1
k1
n
n


k1
k2
n
n
k1
n
54
(No Transcript)
55
Algorithm

• Given function y f(x), implementation y F(x)
• Set up optimization problem

Max y y the difference in
outputs Subject to x ? l,u,
the range of inputs x x e,
the precision of the representation y
f(x) the actual controller output
SP(F)(x,y) the computed
controller output
SP(F)(x,y) is a logical formula relating inputs x
and outputs y of function F
56
Computing SP Symbolic Execution

• Run the program with symbolic inputs
• Each execution maintains
• A symbolic store map program variables to
  symbolic expressions
• A path constraint that specifies constraints on
  inputs for the current path to be executed
• SP(F)
• Disjunction of path constraints along all paths

57
Implementation
Simulink Model

• Implementation of concolic execution with support
  for numerical operations
• Collect symbolic constraints and relate to
  control system parameters
• Model fixed-point arithmetic precisely

Real-Time Workshop
C code
CIL
Instrumented C code
Concolic Execution
YicesHySat
Symbolic constraints
58
Experiments
Example Error Bound ? Time
Vehicle steering 0.0163 0.0375 1min
Train-car 1.65E-4 0.0627 20m
Dc motor 0.0473 1.0889 2min
Jet engine 4E-3 0.0230 0.5s
Jet engine (400 LUT) 1.25 7.2348 18m
Jet engine (104 LUT) 0.33 1.91 103m
59
From Verification to Synthesis

• Verification Problem Given a controller, compute
  the bound ?
• Synthesis Problem Find a controller
  implementation for which the bound is minimized
• Search over
• - all implementations of a given controller
• - all stabilizing controllers for a fixed budget

?
60
Recent Directions in Synthesis I

• Given an arithmetic program, can you re-arrange
  the operations so that the implementation has
  minimal error? DarulovaSahaKuncakM.
• NP-hard just to find optimal permutation
• Stochastic local search (bounded path to
  convergence, but to local minimum)

61
Recent Directions in Synthesis II

• Given an implementation budget, find a
  stabilizing controller that has the best
  implementation
• M.SahaZamani12
• Modify the LQR-LQG control design procedure to
  additionally have an error term
• Non-convex problem ? Stochastic search
• Within 10 of LQR-optimal, 4X better bound

62
From Controllers to Tasks
63
From Controllers to Tasks
Control Design u f(x,t) Sampling time t
Virtual World Control Theory
Real World Real-Time OS
Task Control Computation Period t
Guarantee Controlled system has
required performance if control f is applied
every t units
64
Shared Resources
RTOS Scheduler
Shared CPU
65
Shared Resources and Scheduling
Virtual World Control Theory
Real World Real-Time OS
T1 wcet1 Period t1
Tk wcetk Period tk
T2 wcet2 Period t2
Tasks
Schedulable? Schedule tasks
66
Hard Real Time Scheduling

• Given tasks with worst case execution times and
  periods, is there a way to execute them so that
  all tasks finish executing before their periods?
• Key problem in real-time systems
• System schedulable ? Implement!
• System not schedulable ? Send back to designer
• Or Throw more resources at it!

67
Not-so-hard Real Time Scheduling

• Suppose we relax the scheduler
• In some rounds, the scheduler can decide not
  to execute a task
• - Easier problem
• - But what happens to the controlled system?
• If we ignore a control task too much, the system
  can become unstable

68
Factoring Drops into the Design

• Design a controller for the plant such that a
  fraction r of the control actions can be dropped
• Networked control design
• Plant and controller connected by network
• Network can drop packets
• Find controller to achieve certain performance

69
Relate Drop Rate to Stability

• Theorem BranickyPhillipsWang Consider the
  control system with packet loss
• xk1 A xk B (drop?uk-1uk)
• and controller uk K xk
• If e(A) gt 1 and e(A BK) lt 1, then the
  system is exponentially stable for all drop rates
  less than
• e(A) maximum eigenvalue of A

70
Relate Drop Rate to Performance

• Theorem M.SahaZamani11 A more technical
  condition relating drop rate with L8 to RMS gain
  (a common notion of performance)
• Moreover, can find optimal performance through
  convex optimization
• result for nonlinear systems using Lyapunov
  functions

71
Drop rate vs Performance

• Performance is
• Not monotonic with drop rate
• ? Increasing resources may
• not make system better
• Moral An end-to-end argument can give a better
  overall system performance, even with lower
  resources

72
Schedulability with Task Drops

• Given Task i WCET(i), period t(i), drop rate
  r(i)
• Find Schedule such that
• executions of Task i finish before the deadline,
  and
• the scheduler drops r(i) fraction of packets in
  the long run
• Separation of concerns this is a CS problem!

73
Static Scheduling

• Consider a static scheduler
• All slots pre-assigned to tasks
• Constraints on scheduler
• Chosen tasks must finish before deadlines
• Tasks should not be dropped more than drop rates
• Choice to the scheduler
• - Decide whether to drop task, map task to slot

74
Static Scheduling to SMT

• Can encode constraints as an SMT problem
• Hyperperiod lcm of periods of all tasks
• Boolean variable ti Slot allocated to task i
• Boolean variable si,j if task i is scheduled
  in round j
• If si,j 1, then wcet(i) slots in the jth
  period allocated to task i
• Fraction of slots in which task i is chosen r(i)

75
Inverted Pendulum Example
76
This Talk

• A. Two Examples
• Stability analysis for implementations
• Controller-Scheduler co-development
• B. Some future directions

77
Robustness in Discrete Control

• Observation
• Continuous controlled systems degrade gracefully
• What about discrete (automata theoretic)
  controller synthesis?

78
I/O Stability for Transducers I

• Setting
• Transducer f S ? ?
• Input cost function I S ? N
• Output cost function O ? ? N
• (Assume correct behaviors have O(s) 0)

79
I/O Stability for Transducers I

• Requirement 1 Output costs bounded by input
  costs
• Given ? gt 0, transducer f is ?-IO stable if for
  all strings s in
• O(f(s)) ? I(s)
• Intuitively The deviation cost in the output is
  bounded as a function of the cost in the input
• Similar notions in Tarraf, Bloem etal. and
  linear control systems

80
I/O Stability for Transducers I

• Requirement 1 Output costs bounded by input
  costs
• Given ? gt 0, transducer f is ?-IO stable if for
  all strings s in
• O(f(s)) ? I(s)
• Not enough the effect of an error in the input
  remains forever
• This is not an issue in linear control systems

81
I/O Stability for Transducers II

• Requirement 2 Output deviations fade over time
• Given ? gt 0, transducer f is (?, ?)-IO stable if
  for all strings s in
• O(f(s)) max ? I(s) ?(s-s s s
• Motivated by a notion of IO-stability for
  nonlinear systems Grune02

82
I/O Stability for Transducers III

• Given ? gt 0, transducer f is (?, ?)-IO stable if
  for all strings s in
• O(f(s)) max ? I(s) ?(s-s s s
• Satisfies requirements 1 and 2
• For cost functions given by automata, can verify
  in polynomial time

83
Some Future Directions

• What are other examples of combinations of
  control and program analysis?
• Connect continuous and discrete systems by
  abstraction (Related work Girard, Pappas,
  Tabuada)
• Is there an analog of robustness for discrete
  systems? (Related work Chatterjee, Doyen,
  Henzinger, Raskin, and co-workers)

84
Conclusion

• Abstraction Verification techniques
• from computer science
• can help
• build better systems that
• interact with the physical world

85
Thank You http//www.mpi-sws.org/rupak/