John Doyle

1
Robustness and Complexity
John Doyle Control and Dynamical
Systems BioEngineering Electrical
Engineering Caltech
2
Collaborators and contributors(partial list)

• Theory Parrilo, Carlson, Paganini,
  Papachristodoulo, Prajna, Goncalves, Fazel, Lall,
  DAndrea, Jadbabaie, many current and former
  students,
• Web/Internet Low, Willinger, Vinnicombe,Kelly,
  Zhu,Yu, Wang, Chandy, Effros,
• Biology Csete,Yi, Arkin, Simon, AfCS, Borisuk,
  Bolouri, Kitano, Kurata, Khammash, El-Samad,
  Gross, Endelman, Sauro, Hucka, Finney,
• Physics Mabuchi, Doherty, Barahona, Reynolds,
  Asimakapoulos,
• Turbulence Bamieh, Dahleh, Bobba, Gharib,
  Marsden,
• Engineering CAD Ortiz, Murray, Schroder,
  Burdick,
• Disturbance ecology Moritz, Carlson, Robert,
• Finance Martinez, Primbs, Yamada, Giannelli,

Caltech faculty
Other Caltech
Other
3
For more details
www.cds.caltech.edu/doyle www.aut.ee.ethz.ch/par
rilo
And thanks to Carla Gomes for helpful discussions.
4
Subthemes of this program

• Scalability of algorithms and protocols
• Large network and physical problems
• Decentralized, asynchronous, multiscale
• Computational complexity P/NP/coNP
• Approaches
• Duality
• Randomness
• Workshop II part of this program
• Workshop last week on Phase Transitions of
  Algorithmic Complexity

5
The Internet hourglass
IP
6
The Internet hourglass
Everything on IP
IP
From Hari Balakrishnan
7
Towards a theory of the Internet

• The well-known original design principles are a
  rudimentary theory of the Internet.
• This is a nearly pure robustness theory (little
  else is being optimized).
• Can we provide a deep, complete, and coherent
  theory of internetworking? (Like standard comms
  and controls.)
• If we cant say something systematic about the
  Internet protocols, were probably kidding
  ourselves about our ability to treat more complex
  problems.
• Nevertheless this is just a warm-up for a
  theory of ubiquitous embedded software,
  protocols, and networks for real-time control of
  everything, everywhere.

8
Network protocols.
Files
HTTP
TCP
IP
packets
packets
packets
packets
packets
packets
Routing Provisioning
9
Network protocols.
HTTP
TCP
Vertical decomposition Protocol Stack
IP
Routing Provisioning
10
Network protocols.
HTTP
TCP
IP
Horizontal decomposition Each level is
decentralized and asynchronous
Routing Provisioning
11

• Breaks standard communications and control
  theories.
• Coherent, complete theory is missing but
  possible. First cut nearly done.
• In what sense, if any, is this optimal?
• What needs to be done to fix it?

HTTP
TCP
Vertical decomposition
IP
Horizontal decomposition
Routing Provisioning
12
Key elements of new theory

• Primal/dual vertical and horizontal decomposition
  (Kelly et al, Low et al)
• Source coding into mice and elephants. (Appears
  to be universal but needs more study.)
• Congestion control for bandwidth utilization and
  minimal delay. Proofs use relaxations (but still
  handcrafted).
• How bad is short path (low delay for mice)
  routing for elephants in a well-provisioned
  network? Conjecture Not bad.
• Vertical and horizontal integration can be made
  nearly optimal in an asymptotic sense. (In what
  sense?)
• Lots of people here are working out details (the
  IPAM team!). Stay tuned.

13

• Networking protocols
• Multiscale physics
• Biological networks
• Business, finance, econ organization
• Unifying theoretical framework?

Vertical decomposition
Horizontal decomposition
14
Whats next?

• Scalable, integrated robustness analysis and
  software/protocol verification for hybrid control
  of nonlinear systems.
• New extensions to robust control using
  sum-of-squares and semidefinite programming
  (SOS/SDP) offers extraordinary promise.
• Already demonstrated on wide array of complex
  problems (controls, maxcut, quantum
  entanglement).
• Potentially deep connections between verification
  and robustness.
• Huge implications for biology and physics.
• Thats the good news.

15
Communications and computing
Store
Communicate
Compute
Communicate
Communicate
16
Store
Communicate
Compute
Communicate
Communicate
Act
Environment
17
Computation
Communication
Communication
Devices
Devices
Dynamical Systems
18

• From
• Software to/from human
• Human in the loop

• To
• Software to Software
• Full automation
• Integrated control, comms, computing
• Closer to physical substrate

Computation

• New capabilities robustness
• New fragilities vulnerabilities

Communication
Communication
Devices
Devices
Control
Dynamical Systems
19
Good new, bad news, good news

• Good Powerful new capabilities enabled by
  embedded, everywhere
• Bad Frightening new potentials for massive
  cascading failure events
• Good Need for new math tools for verifying
  robustness of embedded networking.
• Embedded Ubiquitous, sensing, actuating
• Networking Connected, distributed, asynchronous

DeborahWorld
20

• Until recently, there were no promising methods
  for addressing this full problem
• Even very special cases have had limited
  theoretical support for systematic verification
  of robustness
• Everything has changed!

• This represents an enormous change, the impact of
  which is not fully appreciated
• Robustness and verifiability of highly autonomous
  control systems with embedded software is the
  central challenge

Computation

• New capabilities robustness
• New fragilities vulnerabilities

Communication
Communication
Devices
Devices
Control
Dynamical Systems
21

• Breaks standard communications and control
  theories.
• Duality as a method for decomposition
• Distributed and asynchronous control
• Other applications
• Robustness analysis
• A posteriori error bounds for PDEs

HTTP
TCP
Vertical decomposition
IP
Horizontal decomposition
Routing Provisioning
22
Robust hybrid/nonlinear systems theory of
embedded networks?
Linear theory plus bounds, with scalable
algorithms.
Theory without scalable algorithms.
Hacking. (Scalable algorithms without theory.)
Theory with scalable algorithms?
Most research Not scalable, no theory.
23
Provably robust, scalable Internet protocols.
Robustness verification of embedded control
software/hardware.
Hacking.
Theory with scalable algorithms.
24
Key issues

• Robustness/Fragility Uncertainty in components,
  environment, and modeling, assumptions, and
  computational approximations
• Verifiability Short proofs of robustness
• Complexity Extreme, highly structured internal
  complexity is typically needed to produce
  verifiably robust behavior
• Scarce resources All tradeoffs are aggravated by
  efficiency and scarce resources

25
Robustness, evolvability/scalability,
verifiability
Robustness
Ideal performance

• Relative tonominal performance under ideal
  conditions, robust performance typically requires
• greater internal complexity
• some loss of nominal performance
• Tradeoffs between robustness, evolvability, and
  verifiability seem less severe (e.g. IP)

26
Robustness, evolvability/scalability,
verifiability
Ideal performance
Robustness
Evolvability
Verifiability

• That a system is not merely robust, but
  verifiably so, is an important engineering
  requirement and major research challenge
• There is much anecdotal evidence and some new
  theoretical support as well for the compatibility
  of robustness, evolvability, and verifiability
• Verifiability in forward engineering translates
  into comprehensibility in reverse engineering of
  biological systems
• This research direction may be good news for
  understanding complex biological processes

27
Computational complexity
?

• Assume you already know
• P/NP and NP complete
• SAT and 3-SAT
• but not necessarily
• NP vs coNP
• Duality and relaxations

28
Typically NP hard.
?
29
Typically coNP hard.

• Fundamental asymmetries
• Between P and NP
• Between NP and coNP

?

• More important problem.
• Short proofs may not exist.

Unless theyre the same
30
?
What makes a problem harder?
31
Easy to find solutions?
?
Satisfiable or feasible
32
?
Easy to find proofs?
Unsatisfiable or infeasible
33
?0
Complexity?
34
Example Satisfiability

• SAT Given a formula in propositional calculus,
  is there an assignment to its variables making it
  true?
• We consider clausal form, e.g.
• (a OR (NOT b) OR c) AND (b OR d) AND
  (b OR (NOT d) OR a)
• a, b, c, and d are Boolean (True/False)
  variables.
• Problem is NP-Complete. (Cook 1971)
• Shows surprising power of SAT for encoding
  computational problems.

35
Generating Hard Random Formulas

• Key Use fixed-clause-length model.
• (Mitchell, Selman, and Levesque 1992)
• Critical parameter ratio of the number of
  clauses to the
  number of variables.
• Hardest 3SAT problems at ratio 4.3

36
Hardness of 3SAT
4000
50 var
40 var
20 var
3000
DP Calls
Hard
2000
Easy
1000
Easy
0
2
3
4
5
6
7
8
Ratio of Clauses-to-Variables
37
1.0

• At low ratios
• few clauses (constraints)
• many assignments
• easily found
• At high ratios
• many clauses
• inconsistencies easily detected

50 sat
0.8
0.6
Probability
0.4
0.2
0.0
2
3
4
5
6
7
8
Ratio of Clauses-to-Variables
Mitchell, Selman, and Levesque 1991
The 4.3 Point
38

• Refer to as a
• SAT transition
• Complexity transition
• Is SAT transition either necessary or sufficient
  for complexity transition?
• Connections with phase transitions in statistical
  physics?
• Are transitions sharp in large size limit?

39
Theoretical Status Of Threshold

• Very challenging problem ...
• Current status
• 3SAT threshold lies between 3.45 and 4.6
• (Motwani et al. 1994, Achlioptas et al. 2001,
• Kirousis 2002, Broder and Suen 1993, Dubois
• 2000 Achlioptas and Beame 2001, Friedgut
  1997,
• etc.)
• Other problems better characterized (NPP)

40
SAT Phase transitions
?
?
Complexity
41
Quasigroups or Latin Squares
A quasigroup is an n-by-n matrix such that each
row and column is a permutation of the same n
colors
Quasigroup or Latin Square (Order 4)
32 preassignment
Gomes and Selman 96
42
Quasigroup with Holes (QWH)

• Given a full quasigroup, punch holes into it

• Always completable (satisfiable), so no SAT
  transition.
• Appears to have a complexity transition
  (easy-hard-easy).

43
SAT Phase transitions
?
?
Complexity
44
Lots of problems with statistical physics story.
45
Why may it be reasonable that math, algorithms,
and randomness are so effective?

• Robust systems are verifiably so?
• Do only robust systems persist as coherent,
  structured objects of study (universes, solar
  systems, planets, life forms, protocols, )?
• If so, then mostly robust (and verifiably so)
  systems are around for us to study.

46
Lattice models?
What can we do with lattices that will be easy to
understand, yet relevant to the real
computational complexity problems that we most
care about?

• Key abstractions
• Robustness/Fragility
• Verifiability
• Complexity

47
.2
.4
.6
.8
Density fraction of occupied sites (black)
Focus on horizontal paths.
48
Vertical paths in empty sites are allowed to
connect through corners or edges. (8 neighbors)
Horizontal paths connect only on edges. (4
neighbors.Ordinary square site percolation.)
Focus on horizontal paths.
Some (nonstandard) definitions
49
Critical phase transition at density .59
50
.2
.4
.6
.8
Density fraction of occupied sites (black)
Focus on horizontal paths.
51

• Robustness is provided by barriers in some state
  space. These prevent cascading failure events.
• Lattices offer a crude abstraction, in that paths
  can be thought of as barriers, with robustness to
  perturbations in the lattice.

• Verifiability complexity is measured in the
  length of the proof required to verify
  robustness.
• Lattices can offer a variety of crude
  abstractions to this as well. The length of
  minimal paths would be a simple measure of proof
  length.

52

• Very special features
• Dual and primal problems are essentially the
  same.
• There is no duality gap.

Caution potential source of confusion.
53
Barriers in 3d lattices are 2d cuts.
Barriers in 1d lattices are 0d cuts.
path fragments
barrier
In general, barriers are d-1 dimensional (dual)
cuts stopping 1-dim (primal) paths in a d-dim
lattice.
54
Critical phase transition at density .59
55

• Lattices offer pedagogically useful but
  potentially dangerously misleading
  simplifications, which are thus both strengths
  and weaknesses
• Internal complexity
• Computational complexity
• Duality

Focus on horizontal paths.
56

• Internal vs external complexity Real biology
  and technology uses extremely complex
  hierarchical organization in order to create
  robust and verifiably (simple) behavior.
  Lattices allow no distinction between complex
  organization and complex behavior. This can be
  very misleading.
• Computational complexity Most lattice
  computational problems are in P and thus easily
  explored, but fail to illustrate the P/NP
  asymmetry. We will rely on notions of complexity
  that are good analogies, but not precisely
  comparable.
• Duality Duality is greatly simplified and
  transparent. This makes exposition easy but hides
  the NP/coNP asymmetry which is central to the
  general problem.

57

• Lattices offer enormous (and potentially
  dangerous) simplifications
• Robustness problem existence of horizontal path
• Verification prove existence of horizontal
  path
• Complexity minimum horizontal path length (of
  proof)
• Model fragility minimum number of site changes
  to break all horizontal paths ( create a
  vertical path)

Focus on horizontal paths.
58
Note Im going to draw small lattices and rely
on your imagination for what large lattices would
look like.
59

• Alternative definition of complexity
• The computer is you, looking at the lattice
  and determining by inspection whether there is a
  path or not.
• This can be easy or hard, depending on the
  density.
• This is not exactly the same as minimal path
  length, but close enough for now.
• Do a very informal story, and then make it
  rigorous.

.2
.4
.6
.8
Density fraction of occupied sites (black)
60
No
Yes
Easy
Exist horizontal path?
Hard
For random lattices, there are 4 regimes, with
all combinations of Easy/Hard and Yes/No. The
hard cases correspond to lattices that are of
intermediate density, near the critical point.
Easy cases are either high or low densities,
which always correspond to Yes or No,
respectively.
61
No
Yes
No
Yes
Easy
Easy
Hard
Hard
It is much easier to see with all the clusters
colored. But thats cheating, because determining
the clusters is essentially the computational
problem.
62
The orthodox story
No
Yes
Easy
Hard problems are associated in some way with the
phase transition.
Hard
63
The counter-examples
Exactly the opposite of criticality
No
Yes

• Yes or no
• Easy or hard
• High or low density
• Robust or fragile (to perturbations)

Easy
Hard
64
The counter-examples
Exactly the opposite of criticality

• Yes or no
• Easy or hard
• Low or high density
• Robust or fragile (to perturbations)

16 different possible combinations
65
The counter-examples
Exactly the opposite of criticality
8

• Yes or no
• Easy or hard
• Low or high density
• Robust or fragile (to perturbations)

16 different possible combinations
66
Robust
Fragile
Robust
Fragile
Easy
Easy
Hard
Hard
Low Density (but connected)
High density
Hard implies fragile (well prove this later). So
only 6 of the 8 possibilities exist, and the
critical density is nothing special. We will
prove that these and only these implications hold.
67
Robust
Fragile
Robust
Fragile
Easy
Easy
Hard
Hard
Low Density
High density
Robust
Fragile
Random
Easy
Hard
68
Robust
Fragile
All interesting real world problems are in this
regime, with efficient, highly structured, rare
configurations, using scarce (limited) resources.

Easy
Hard
Low Density
Robust
Fragile
Random
Easy
Hard
69
Robust
Fragile
Easy
Impossible.
Hard
Low Density
Robust
Fragile
Improbable in random lattices.
Random
Easy
Hard
70
Robust
Fragile
Robust
Fragile
Easy
Easy
Hard
Hard
Low Density
High density
Proof tonite.
71
Random lattices are complex (and fragile) only
at critical phase transition.
Low Density
High density
Robust
Fragile
Easy
Hard
72
Definitions. Assume there is a connected
(horizontal) path of minimal length l .
n length of side r density l MinPath length
Occupied
Empty
MinPath
Typical minimal path
73
Definitions. Assume there is a connected path of
minimal length l .
n length of side r density l MinPath
length b MinCut barrier length
Typical minimal cut
Occupied
Empty
MinPath
Typical minimal path
b
74
Definitions. Assume there is a connected path of
minimal length l .
n length of side r density l MinPath
length b MinCut barrier length
Vertical path
b
75
n length of side r density l MinPath
length b MinCut barrier length
Assume a path exists. (Otherwise LF?.)
Necessarily ? ? 1/n, n2 ? l ? n and define
76
l MinPath length b MinCut barrier length
77
l MinPath length b MinCut barrier length
Proof (Vinnicombesushi) To provide robustness
to b changes, there must be at least b
independent paths, which by assumption have
minimum length l. Necessarily ? n2 ? lb, or ?
n/b ? l/n. Take log of both sides.
78
This is maximally tight in the sense that

• Lattices and paths can be
• Resources Scarce or rich
• Existence of path Yes or no
• Complexity Hard or easy
• Perturbations Fragile or robust

79

• Lattices and paths can be
• Existence Yes or no
• Resources Scarce or rich
• Perturbations Fragile or robust
• Complexity Hard or easy

Anything is possible, consistent with the theorem.
Well just consider the 8 cases with paths.
80
Fragile
Hard
Robust
Easy
Rich
Scarce
-Slog(?)
81
Fragile
Hard
Robust
Easy
Rich
Scarce
82
Fragile
Hard
Robust
Easy
Rich
Scarce
83
Hard
Fragile
Scarce
Rich
Easy
Robust
84
Easy
Occupied
Empty
MinPath
FS, L0
85
Easy
Most robust possible.
FS, L0
86
Easy and Fragile
Flog(n)gtS, L0
87
Hard
Fragile
FSL
Scarce
Rich
Easy
m
d
Robust
b
Occupied
Empty
MinPath
88
r density b MinCut barrier length l MinPath
length n length of side m of cells d
width of open regions
To construct asymptotically tight cases where ?n2
lb, consider the lattice below.
m
d
b
b
d
89
Now take limits
By constructing lattices as below, with ngtgtmgtgt1,
it is possible to find lattices such that any ?n2
? lb, with ?lt1 is achievable.
90
FSL
Hard
Fragile
Scarce
Rich
Easy
Robust
91
The Fragile Face
Hard
Fragile
Scarce
Rich
Easy
Robust
92
The Four Corners
Hard
Fragile
Scarce
Rich
Easy
Robust
93
FSL
Fragile
Most Fragile FgtgtS
Scarce
Most Robust FS
Easy
Robust
94
Random
Hard
Fragile
Scarce
Rich
Easy
Robust
95
Efficient and robust is far from random
96

• How general is this?
• Seems to hold in all theory where it has been
  investigated.
• Extensive literature on ill-conditioning in LPs
  and numerical linear algebra.
• Anecdotally, seems to capture essence of many
  complexity problems.
• Needs to be combine with laws constraining net
  system fragility.

97
Phase transitions
Complexity
98
Bad news and good news

• Bad news? Some hoped-for connections between
  phase transitions and complexity are not there.
• Good news? Ideas still interesting.
• Lots more really good news!
• The alternative is much richer and useful, and
  connects in interesting ways with phase
  transitions
• New algorithms, new mathematics, new practical
  applications,
• And deep implications for physics.

99
Physics and the edge of chaocritiplexity
Phase transitions

• Internet traffic and topology
• Biological and ecological networks
• Evolution and extinction
• Earthquakes and forest fires
• Finance and economics
• Social and political systems

?
Complexity
100
(No Transcript)
101
Physics and the edge of chaocritiplexity
Phase transitions

• Internet traffic and topology
• Biological and ecological networks
• Evolution and extinction
• Earthquakes and forest fires
• Finance and economics
• Social and political systems

?
Rich new unifying theory of complex control,
communication, and computing systems
Complexity
102
Physics and the edge of chaocritiplexity

• Ubiquity of power laws
• Coherent structures in shear flow turbulence
• Macro dissipation and irreversibility vs. micro
  reversibility.
• Quantum entanglement, measurement, and the
  QM/Classical transition
• Growing group of physicists and experimentalists
  are joining this effort (Carlson, Mabuchi,
  Doherty, Gharib,)

Rich new unifying theory of complex control,
communication, and computing systems
103
More powerful bounds for the co-NP side
Semialgebraic geometry convex optimization
(SDP)

• Polynomial time computation.
• Never worse than the standard.
• Exhausts co-NP.

104

• Polynomial functions NP-hard problem.

105
?0
Complexity?
106
Special case Scalar QP
107
Special case Scalar QP
108

• Polynomial functions NP-hard problem.
• A simple relaxation (Shor) find the minimum
  ?such that ?- F(x) is a sum of squares (SOS).
• Upper bound on the global maximum.
• Solvable using SDP, in polynomial time.
• A concise proof of nonnegativity.
• Surprisingly effective (Parrilo Sturmfels 2001).

109

• Exactly as in QP case, SAT phase transition
  does not imply complexity.
• SOS/SDP relaxations much faster than standard
  algebraic methods (QE,GB, etc.).
• Before SOS/SDP, might have conjectured that this
  was an example of phase transition induced
  complexity.
• SOS/SDP gives certified upper bound in polynomial
  time.
• If exact, can recover an optimal feasible point.
• Surprisingly effective
• In more than 10000 random problems, always the
  correct solution
• Bad examples do exist (otherwise NPco-NP), but
  rare.
• Variations of the Motzkin polynomial.
• Reductions of hard problems (e.g. NPP is nice)
• None could be found using random search

110
Sums of squares (SOS)
A sufficient condition for nonnegativity

• Convex condition (Shor, 1987)
• Efficiently checked using SDP (Parrilo). Write

where z is a vector of monomials. Expanding and
equating sides, obtain linear constraints among
the Qij. Finding a PSD Q subject to these
conditions is exactly a semidefinite program
(LMI).
111
Nested families of SOS (Parrilo)
exhaust co-NP
112

• Conjectures on why such a boring phase
  transition
• One polynomial is generically robust, therefore
  no complexity.
• QPs capture the essence of this.
• Can make up other phase transitions which
  create fragilities, and thus the possibility of
  complexity

?0
113
Search for counterexample
coNP
NP
Search for proof
114
Positivstellensatz
Search for counterexample
Search for proof

• Convex, but infinite dimensional.
• Efficient (P time) search subsets (relaxations)
  using SOS/SDP (Parrilo)
• Guaranteed to converge

115
Search for simple counterexample
Search for short proof
116
Special case LP
117
NPP
?
Fragile large changes in solution from small
changes in data
118
NPP
119
Random f
120
Very unlikely to be feasible. Contrast with
random polynomial.
121

• Complexity is caused by fragility
  (ill-conditioning).
• Another example Purely satisfiable QCP
• Phase transitions are, in general, unrelated to
  complexity
• Random scalar QP problems are generically robust
  (well-conditioned) and thus simple

122
Phase transitions
Complexity
123
More powerful bounds for the co-NP side
Semialgebraic geometry convex optimization
(SDP)

• Polynomial time computation.
• Never worse than the standard.
• Exhausts co-NP.

124
A key insight
Think of LMIs as quadratic forms, not as
matrices. LMIs quadratic forms, that are
positive definite.

• General forms , not necessarily quadratic.

• Instead of nonnegativity (NP-hard), use sum of
  squares.

SOS multivariable forms, that are sum of
squares.
125
Search for counterexample

• Models describe sets of possible (uncertain)
  behaviors intersected with sets of unacceptable
  behaviors (failures)
• Thus verification of robustness (of protocols,
  embedded, dynamics, etc) involves showing that a
  set is empty.
• Searching for an element x ?M is in NP, since
  checking whether a given x ?M is typically in P.
• Proving that M is empty is in coNP and there may
  not be short proofs.

Search for proof
126
Search for counterexample
Seach for proof

• Convex, but infinite dimensional.
• Efficient (P time) search subsets (relaxations)
  using SOS/SDP
• Guaranteed to converge

127
Search for simple counterexample
Search for short proof
128
Special case LP
129
Search for simple counterexample
Search for short proof
130
Search for simple counterexample
Failure to find short proof implies some relaxed
model is nonempty (which is bad).
Search for short proof
131
Sums of squares (SOS)
A sufficient condition for nonnegativity

• Convex condition (Shor, 1987)
• Efficiently checked using SDP (Parrilo). Write

where z is a vector of monomials. Expanding and
equating sides, obtain linear constraints among
the Qij. Finding a PSD Q subject to these
conditions is exactly a semidefinite program
(LMI).
132
Nested families of SOS (Parrilo)
exhaust co-NP
133
?0
134
A Few Applications

• Nonlinear dynamical systems
• Lyapunov function computation
• Bendixson-Dulac criterion
• Robust bifurcation analysis
• Continuous and combinatorial optimization
• Polynomial global optimization
• Graph problems e.G. Max cut
• Problems with mixed continuous/discrete vars.
• Hybrid???

Lets see some examples
135
Continuous Global Optimization

• Polynomial functions NP-hard problem.
• A simple relaxation (Shor) find the maximum
  ?such that f(x) ? is a sum of squares.
• Lower bound on the global optimum.
• Solvable using SDP, in polynomial time.
• A concise proof of nonnegativity.
• Surprisingly effective (Parrilo Sturmfels 2001).

136

• Much faster than exact algebraic methods (QE,GB,
  etc.).
• Provides a certified lower bound.
• If exact, can recover an optimal feasible point.
• Surprisingly effective
• In more than 10000 random problems, always the
  correct solution
• Bad examples do exist (otherwise NPco-NP), but
  rare.
• Variations of the Motzkin polynomial.
• Reductions of hard problems.
• None could be found using random search

137
More general framework

• A model co-NP problem
• Check emptiness of semialgebraic sets.
• Obtain LMI sufficient conditions.
• Can be made arbitrarily tight, with more
  computation.
• Polynomial time checkable certificates.

138
Semialgebraic Sets

• Semialgebraic finite number of polynomial
  equalities and inequalities.
• Continuous, discrete, or mixture of variables.
• Is a given semialgebraic set empty?
• Feasibility of polynomial equations NP-hard
• Search for bounded-complexity emptiness proofs,
  using SDP. (Parrilo 2000)

139
Positivstellensatz (Real Nullstellensatz)
if and only if

• Stengle, 1974
• Generalizes Hilberts Nullstellensatz and LP
  duality
• Infeasibility certificates of polynomial
  equations over the real field.
• Parrilo Bounded degree solutions computed via
  SDP!
• ? Nested family of polytime relaxations for
  quadratics, the first level is the S-procedure

140
Combinatorial optimization MAX CUT

• Given a graph
• Partition the nodes in two subsets

• To maximize the number of edges between the two
  subsets.

A mathematical formulation
Hard combinatorial problem (NP-complete).
Compute upper bounds using convex relaxations.
141
Standard semidefinite relaxation
Dual problems
This is just a first step. We can do better! The
new tools provide higher order relaxations.

• Tighter bounds are obtained.
• Never worse than the standard relaxation.
• In some cases (n-cycle, Petersen graph),
  provably better.
• Still polynomial time.

142
MAX CUT on the Petersen graph
The standard SDP upper bound 12.5 Second
relaxation bound 12. The improved bound is
exact. A corresponding coloring.
143
Finding Lyapunov functions

• Ubiquitous, fundamental problem
• Algorithmic LMI solution

Convex, but still NP hard.
Test using SOS and SDP.
After optimization coefficients of V.
A Lyapunov function V, that proves stability.
144
Finding Lyapunov functions

• Ubiquitous, fundamental problem
• Algorithmic LMI solution

Given
Propose
After optimization coefficients of V.
A Lyapunov function V, that proves stability.
145
Conclusion a certificate of global stability
146
Flow of f
(x) with the corresponding Lyapunov function
1
5
c1
c2.164
c10
0
-5
-10
-15
-10
-5
0
5
10
-10
-5
0
5
10
-10
-5
0
5
147
2
1.5
1
0.5
0
2
x
-0.5
-1
-1.5
-2
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
x
1
Global stability of a switching system using 4th
order MLFs defined in 6 equiangular partitions
148
(No Transcript)
149
DS applications Bendixson-Dulac

• In 2D rules out periodic orbits.
• Higher dimensional generalizations (Rantzer)
  provide
• Weaker stability criterion than Lyapunov
  (allowing a zero-measure set of divergent
  trajectories).
• Convexity for synthesis.
• How to search for ? ?

150
DS applications Bendixson-Dulac

• Restrict to polynomial (or rational) solutions,
  use SOS.
• As for Lyapunov, now a fully algorithmic
  procedure.

Given
Propose
After optimization
151
Conclusion a certificate of the inexistence of
periodic orbits
x ' y


2
2
y ' - x - y x
y


3
2
1
0
y
-1
-2
-3
-3
-2
-1
0
1
2
3
x
152
Conclusion a certificate of the inexistence of
periodic orbits
x ' y


saddle
y