John Doyle

1

• John Doyle
• Control and
• Dynamical Systems
• Caltech

Jean Carlson Physics UCSB
2
Complexity and robustness

• Complexity phenotype robust, yet fragile
• Complexity genotype internally complicated
• New theoretical framework HOT (Highly optimized
  tolerance)
• Applies to biological and technological systems
• Pre-technology simple tools
• Primitive technologies use simple strategies to
  build fragile machines from precision parts.
• Advanced technologies use complicated
  architectures to create robust systems from
  sloppy components
• but are also vulnerable to cascading failures

3
Robust, yet fragile phenotype

• Robust to large variations in environment and
  component parts (reliable, insensitive,
  resilient, evolvable, simple, scaleable,
  verifiable, ...)
• Fragile, often catastrophically so, to cascading
  failures events (sensitive, brittle,...)
• Cascading failures can be initiated by small
  perturbations (Cryptic mutations,viruses and
  other infectious agents, exotic species, )
• Greater pheno-complexity more extreme robust,
  yet fragile

4
Robust, yet fragile phenotype

• Cascading failures can be initiated by small
  perturbations (Cryptic mutations,viruses and
  other infectious agents, exotic species, )
• In many complex systems, the size of cascading
  failure events are often unrelated to the size of
  the initiating perturbations
• Fragility is interesting when it does not arise
  because of large perturbations, but catastrophic
  responses to small variations

5
Complicated genotype

• Robustness is achieved by building barriers to
  cascading failures
• This often requires complicated internal
  structure, hierarchies, self-dissimilarity,
  layers of feedback, signaling, regulation,
  computation, protocols, ...
• Greater geno-complexity more parts, more
  structure

6
Robustness of HOT systems
Fragile
Fragile (to unknown or rare perturbations)
Robust (to known and designed-for uncertainties)
Uncertainties
Robust
7
Robustness of HOT systems
Fragile
Humans
Chess
Meteors
Robust
8
Robustness of HOT systems
Fragile
Humans
Archaea
Chess
Meteors
Machines
Robust
9
Diseases of complexity
Fragile

• Cancer
• Epidemics
• Viral infections
• Auto-immune disease

Uncertainty
Robust
10
Biochemical Network E. Coli Metabolism
Regulatory Interactions
Supplies Materials Energy
Supplies Robustness
From Adam Arkin
from EcoCYC by Peter Karp
11
Biochemical Network E. Coli Metabolism
Complexity ? Robustness
Regulatory Interactions

• Reverse engineering such networks uses theory
  from
• Robust control
• Communications
• Computation
• Dynamical systems

Supplies Robustness
From Adam Arkin
from EcoCYC by Peter Karp
12
What about ?

• Not really about complexity
• These concepts themselves are robust, yet
  fragile
• Powerful in their niche
• Brittle (break easily) when moved or extended
• Some are relevant to biology and engineering
  systems
• Comfortably reductionist
• Remarkably useful in getting published

• Information entropy
• Fractals self-similarity
• Criticality and power laws
• Chaos
• Undecidability
• Fuzzy logic, neural nets, genetic algorithms
• Emergence
• Self-organization
• Complex adaptive systems
• New science of complexity

13
Criticality and power laws

• Tuning 1-2 parameters ? critical point
• In certain model systems (percolation, Ising, )
  power laws and universality iff at criticality.
• Physics power laws are suggestive of criticality
• Engineers/mathematicians have opposite
  interpretation
• Power laws arise from tuning and optimization.
• Criticality is a very rare and extreme special
  case.
• What if many parameters are optimized?
• Are evolution and engineering design different?
  How?
• Which perspective has greater explanatory power
  for power laws in natural and man-made systems?

14
6
Data compression (Huffman)
WWW files Mbytes (Crovella)
5
4
Cumulative
3
Frequency
Forest fires 1000 km2 (Malamud)
2
1
0
-1
-6
-5
-4
-3
-2
-1
0
1
2
Decimated data Log (base 10)
Size of events
15
Size of events x vs. frequency
log(Prob gt size)
log(rank)
log(size)
16
?1
1e3 samples from a known distribution
log10(P)
log10(x)
x integer
17
Cumulative Distributions
Slope -?
18
?1
Correct
Cumulative Distributions
?0
Noncumulative Densities
Wrong
?0
19
Data Model/Theory
6
DC
5
WWW
4
3
2
1
Forest fire
0
-1
-6
-5
-4
-3
-2
-1
0
1
2
20
SOC ? .15
21
Cumulative distributions
? .15
22
6
Web files
5
Codewords
4
Cumulative
3
Frequency
Fires
2
1
0
-1
-6
-5
-4
-3
-2
-1
0
1
2
Size of events
Log (base 10)
23
The HOT view of power laws

• The central limit theorem gives power laws as
  well as Gaussians
• Many other mechanisms (eg multiplication noise)
  yield power laws
• A model producing a power law is per se
  uninteresting
• A model should say much more, and lead to new
  experiments and improved designs, policies,
  therapies, treatments, etc.

24
The HOT view of power laws

• Engineers design (and evolution selects) for
  systems with certain typical properties
• Optimized for average (mean) behavior
• Optimizing the mean often (but not always) yields
  high variance and heavy tails
• Power laws arise from heavy tails when there is
  enough aggregate data
• One symptom of robust, yet fragile

25
Source coding for data compression
26
Shannon coding

• Ignore value of information, consider only
  surprise
• Compress average codeword length (over stochastic
  ensembles of source words rather than actual
  files)
• Constraint on codewords of unique decodability
• Equivalent to building barriers in a zero
  dimensional tree
• Optimal distribution (exponential) and optimal
  cost are

27
Data
6
DC
5
How well does the model predict the data?
4
3
2
1
0
-1
0
1
2
28
Data Model
6
DC
5
How well does the model predict the data?
4
3
Not surprising, because the file was compressed
using Shannon theory.
2
1
0
-1
0
1
2
Small discrepancy due to integer lengths.
29
Why is this a good model?

• Lots of models will reproduce an exponential
  distribution
• Shannon source coding lets us systematically
  produce optimal and easily decodable compressed
  files
• Fitting the data is necessary but far from
  sufficient for a good model

30
Web layout as generalized source coding

• Keep parts of Shannon abstraction
• Minimize downloaded file size
• Averaged over an ensemble of user access
• But add in feedback and topology, which
  completely breaks standard Shannon theory
• Logical and aesthetic structure determines
  topology
• Navigation involves dynamic user feedback
• Standard theory breaks, but can be extended
• Equivalent to building 0-dimensional barriers in
  a 1- dimensional tree of content

31
A toy website model( 1-d grid HOT design)
document
32
Optimize 0-dimensional cuts in a 1-dimensional
document
links files
33
More complete website models(Zhu, Yu, Effros)

• Necessary for web layout design
• Statistics consistent with simpler models
• Improved protocol design (TCP)
• Commercial implications still unclear

34
Generalized coding problems

• Optimizing d-1 dimensional cuts in d dimensional
  spaces
• To minimize average size of files
• Models of greatly varying detail all give a
  consistent story.
• Power laws have ? ? 1/d.
• Completely unlike criticality.

Data compression
Web
35
Theory results
d 0 data compression d 1 web layout
36
Data
6
DC
5
WWW
4
3
2
1
0
-1
-6
-5
-4
-3
-2
-1
0
1
2
37
Data Model/Theory
6
DC
5
WWW
4
3
2
1
0
-1
-6
-5
-4
-3
-2
-1
0
1
2
38
Typical web traffic
Heavy tailed web traffic
? gt 1.0
log(freq gt size)
p ? s-?
log(file size)
Is streamed out on the net.
Creating fractal Gaussian internet traffic
(Willinger,)
Web servers
39
Data Model/Theory
6
DC
5
WWW
4
3
What about Forest fires?
2
1
0
-1
-6
-5
-4
-3
-2
-1
0
1
2
40
Forest fires dynamics
Intensity Frequency Extent
41
Santa Monica Mountains
42
(No Transcript)
43
GIS fuel (vegetation) data
44
Models for Fuel Succession
45
1996 Calabasas Fire
Historical fire spread
Simulated fire spread
46
Critical percolation and SOC forest fire models

• SOC HOT have completely different
  characteristics.
• SOC vs HOT story is consistent across different
  models.
• Focus on generalized coding abstraction for
  HOT

47
A HOT forest fire abstraction
Fire suppression mechanisms must stop a 1-d front.
Optimal strategies must tradeoff resources with
risk.
48
Generalized coding problems

• Optimizing d-1 dimensional cuts in d dimensional
  spaces
• To minimize average size of files or fires,
  subject to resource constraint.
• Models of greatly varying detail all give a
  consistent story.

Data compression
Web
49
Theory
d 0 data compression d 1 web layout d
2 forest fires
50
Data
6
DC
5
WWW
4
3
FF
2
1
0
-1
-6
-5
-4
-3
-2
-1
0
1
2
51
Data Model/Theory
6
DC
5
WWW
4
3
FF
2
1
0
-1
-6
-5
-4
-3
-2
-1
0
1
2
52
Forest fires?
Fire suppression mechanisms must stop a 1-d front.
53
Forest fires?
Geography could make d lt2.
54
California geographyfurther irresponsible
speculation

• Rugged terrain, mountains, deserts
• Fractal dimension d ? 1?
• Dry Santa Ana winds drive large (? 1-d) fires

55
Data HOT Model/Theory
6
5
California brushfires
4
3
FF (national) d 2
2
1
0
-1
-6
-5
-4
-3
-2
-1
0
1
2
56
Data HOTSOC
6
5
4
3
2
1
0
-1
-6
-5
-4
-3
-2
-1
0
1
2
57
Critical/SOC exponents are way off
Data ? gt .5
SOC ? lt .15
58
Cumulative distributions
SOC ? .15
59
18 Sep 1998
Forest Fires An Example of Self-Organized
Critical Behavior Bruce D. Malamud, Gleb Morein,
Donald L. Turcotte
4 data sets
60
HOT FF d 2
2
10
1
10
0
10
-2
-1
0
1
2
3
4
10
10
10
10
10
10
10
Additional 3 data sets
61
(No Transcript)
62
Fires are compact regions of nontrivial area.
Fires 1930-1990
Fires 1991-1995
63
HOT
SOC and HOT have very different power laws.
d1
SOC
d1

• HOT ? decreases with dimension.
• SOC?? increases with dimension.

64

• HOT yields compact events of nontrivial size.
• SOC has infinitesimal, fractal events.

HOT
SOC
large
infinitesimal
size
65
SOC and HOT are extremely different.
HOT
SOC
66
SOC and HOT are extremely different.
HOT
SOC
67
Summary

• Power laws are ubiquitous, but not surprising
• HOT may be a unifying perspective for many
• Criticality SOC is an interesting and extreme
  special case
• but very rare in the lab, and even much rarer
  still outside it.
• Viewing a system as HOT is just the beginning.

68
The real work is

• New Internet protocol design
• Forest fire suppression, ecosystem management
• Analysis of biological regulatory networks
• Convergent networking protocols
• etc

69
Collaboratorsand contributors(partial list)

• Web/Internet Low, Effros, Zhu,Yu, Chandy,
  Willinger,
• Disturbance ecology Moritz, Morais, Zhou,
  Robert,
• Theory Lall, Parrilo, Paganini, Barahona,
  DAndrea,
• Biology Csete,Yi, Borisuk, Bolouri, Kitano,
  Kurata, Khammash, El-Samad,
• Alliance for Cellular Signaling Gilman, Simon,
  Sternberg, Arkin,
• Turbulence Bamieh, Dahleh, Gharib, Marsden,
  Bobba,
• Physics Mabuchi, Doherty, Marsden,
  Asimakapoulos,
• Engineering CAD Ortiz, Murray, Schroder,
  Burdick, Barr,
• Power systems Verghese, Lesieutre,
• Finance Primbs, Yamada, Giannelli,
• and casts of thousands