Towards new order

1
Towards new order

• Lasse Eriksson
• 12.3.2003

2
Towards new order - outline

• Introduction
• Self-Organized Criticality (SOC)
• Sandpile model
• Edge of Chaos (EOC)
• Two approaches
• Measuring Complexity
• Correlation distance
• Phase transition
• Highly Optimized Tolerance (HOT)
• Summary and conclusions

3
Introduction

• From Catastrophe to Chaos? 12
• Catastrophe theory studies and classifies
  phenomena characterized by sudden shifts in
  behavior arising from small changes in
  circumstances
• Originated by the French mathematician Rene Thom
  in the 1960s, catastrophe theory is a special
  branch of dynamical systems theory
• ... the big fashion was topological defects.
  Everybody was ... finding exotic systems to write
  papers about. It was, in the end, a reasonable
  thing to do. The next fashion, catastrophe
  theory, never became important for anything. -
  James P. Sethna (Cornell University, Ithaca) 11

4
Introduction

• We are moving from Chaos towards new Order
• From Chaotic to Complex systems
• What is the difference, just a new name? Is it
  really something new?
• Langton's famous egg diagram (EOC, 7)

5
Introduction 13

• Chaos vs. Complex
• Advances in the scientific study of chaos have
  been important motivators/roots of the modern
  study of complex systems
• Chaos deals with deterministic systems whose
  trajectories diverge exponentially over time
• Models of chaos generally describe the dynamics
  of one (or a few) variables which are real. Using
  these models some characteristic behaviors of
  their dynamics can be found
• Complex systems do not necessarily have these
  behaviors. Complex systems have many degrees of
  freedom many elements that are partially but not
  completely independent
• Complex behavior "high dimensional chaos

6
Introduction

• Chaos is concerned with a few parameters and the
  dynamics of their values, while the study of
  complex systems is concerned with both the
  structure and the dynamics of systems and their
  interaction with their environment
• Same kind of phenomena in catastrophe, chaos and
  complexity theory
• Stable states become unstable
• Sudden changes in systems behavior
• Critical points, edges...

7
Introduction 13

• Complexity is (the abstract notion of complexity
  has been captured in many different ways. Most,
  if not all of these, are related to each other
  and they fall into two classes of definitions)
• 1) ...the (minimal) length of a description of
  the system.
• 2) ...the (minimal) amount of time it takes to
  create the system
• The length of a description is measured in units
  of information. The former definition is closely
  related to Shannon information theory and
  algorithmic complexity, and the latter is related
  to computational complexity

8
Introduction

• Emergence is...
• 1) ...what parts of a system do together that
  they would not do by themselves collective
  behavior
• How behavior at a larger scale of the system
  arises from the detailed structure, behavior and
  relationships on a finer scale
• 2) ...what a system does by virtue of its
  relationship to its environment that it would not
  do by itself e.g. its function
• Emergence refers to all the properties that we
  assign to a system that are really properties of
  the relationship between a system and its
  environment
• 3) ...the act or process of becoming an emergent
  system

9
About fractals 1

• Many objects in nature are best described
  geometrically as fractals, with self-similar
  features on all length scales
• Mountain landscapes peaks of all sizes, from
  kilometres down to millimetres
• River networks streams of all sizes
• Earthquakes occur on structures of faults
  ranging from thousands of kilometres to
  centimetres
• Fractals are scale-free (you cant determine the
  size of a picture of a part of a fractal without
  a yardstick)
• How does nature produce fractals?

10
About fractals

• The origin of the fractals is a dynamical, not a
  geometrical problem
• Geometrical characterization of fractals has been
  widely examined but it would be more interesting
  to gain understanding of their dynamical origin
• Consider earthquakes
• Earthquakes last for a few seconds
• The fault formations in the crust of the earth
  are built up during some millions of years and
  the crust seems to be static if the observation
  period is a human lifetime

11
About fractals

• The laws of physics are local, but fractals are
  organized over large distances
• Large equilibrium systems operating near their
  ground state tend to be only locally correlated.
  Only at a critical point where continuous phase
  transition takes place are those systems fractal

12
Example damped spring

• Consider a damped spring as shown in the figure
• Lets model (traditionally!) and simulate the
  system
• mass m
• spring constant k
• damping coefficient B
• position x(t)

13
Example damped spring
14
Example damped spring
Zoom
15
Damped spring 1

• In theory, ideal periodic motion is well
  approximated by sine wave
• Oscillatory behavior with decreasing amplitude
  theoretically continues forever
• In real world, the motion would stop because of
  the imperfections such as dust
• Once the amplitude gets small enough, the emotion
  suddenly stops
• This generally occurs at the end of an
  oscillation where the velocity is smallest
• This is not the state of smallest energy!
• In a sense, the system is most likely to settle
  near a minimally stable state

16
Multiple Pendulums

• The same kind of behavior can be detected when
  analysing pendulums
• Consider coupled pendulums which all are in a
  minimally stable state
• The system is particularly sensitive to small
  perturbations which can avalanche through the
  system
• Small disturbances could grow and propagate
  through the system with little resistance despite
  the damping and other impediments

17
Multiple Pendulums

• Since energy is dissipated through the process,
  the energy must be replenished for avalanches to
  continue
• If Self-Organized Criticality i.e. SOC is
  considered, the interest is on the systems where
  energy is constantly supplied and eventually
  dissipated in the form of avalanches

18
Self-Organized Criticality 1,2

• Concept was introduced by Per Bak, Chao Tang, and
  Kurt Wiesenfeld in 1987
• SOC refers to tendency of large dissipative
  systems to drive themselves to a critical state
  with a wide range of length and time scales
• The dynamics in this state is intermittent with
  periods of inactivity separated by well defined
  bursts of activity or avalanches
• The critical state is an attractor for the
  dynamics
• The idea provides a unifying concept for
  large-scale behavior in systems with many degrees
  of freedom

19
Self-Organized Criticality

• SOC complements the concept chaos wherein
  simple systems with a small number of degrees of
  freedom can display quite complex behavior
• Large avalanches occur rather often (there is no
  exponential decay of avalanche sizes, which would
  result in a characteristic avalanche size), and
  there is a variety of power laws without cutoffs
  in various properties of the system
• The paradigm model for this type of behavior is
  the celebrated sandpile cellular automaton also
  known as the Bak-Tang-Wiesenfeld (BTW) model

20
Sandpile model 1,2

• Adding sand slowly to a flat pile will result
  only in some local rearrangement of particles
• The individual grains, or degrees of freedom, do
  not interact over large distances
• Continuing the process will result in the slope
  increasing to a critical value where an
  additional grain of sand gives rise to avalanches
  of any size, from a single grain falling up to
  the full size of the sand pile
• The pile can no longer be described in terms of
  local degrees of freedom, but only a holistic
  description in terms of one sandpile will do
• The distribution of avalanches follows a power law

21
Sandpile model

• If the slope were too steep one would obtain a
  large avalanche and a collapse to a flatter and
  more stable configuration
• If the slope were too shallow the new sand would
  just accumulate to make the pile steeper
• If the process is modified, for instance by using
  wet sand instead of dry sand, the pile will
  modify its slope during a transient period and
  return to a new critical state
• Consider snow screens if you build them to
  prevent avalanches, the snow pile will again
  respond by locally building up to steeper states,
  and large avalanches will resume

22
14
23
Simulation of the sandpile model 1

• 2D cellular automaton with N sites
• Integer variables zi on each site i represent the
  local sandpile height
• When height exceeds critical height zcr (here 3),
  then 1 grain is transferred from unstable site to
  each 4 neighboring site
• A toppling may initiate a chain reaction, where
  the total number of topplings is a measure of the
  size of an avalanche
• Figure after 49152 grains dropped on a single
  site (fractals?)

24
Simulation of the sandpile model

• To explore the SOC of the sandpile model, one can
  randomly add sand and have the system relax
• The result is unpredictable and one can only
  simulate the resulting avalanche to see the
  outcome
• State of a sandpile after adding pseudo-randomly
  a large amount of sand on a 286184 size lattice
• Figure open boundaries
• Heights 0 black, 1 red, 2 blue, 3 green

25
Simulation of the sandpile model

• Configuration seems random, but some subtle
  correlations exist (e.g. never do two black cells
  lie adjacent to each other, nor does any site
  have four black neighbors)
• Avalanche is triggered if a small amount of sand
  is added to a site near the center
• To follow the avalanche, a cyan color has been
  given to sites that have collapsed in the
  following figures

26
Simulation of the sandpile model
time
27
Sandpile model

• Figure shows a log-log plot of the distribution
  of the avalanche sizes s (number of topplings in
  an avalanche), P is the probability distribution

28
Sandpile model

• Because of the power law, the initial state was
  actually remarkably correlated although it seemed
  at first featureless
• For random distribution of zs (pile heights),
  one would expect the chain reaction of an
  avalanche to be either
• Subcritical (small avalanche)
• Supercritical (exploding avalanche with collapse
  of the entire system)
• Power law indicates that the reaction is
  precisely critical, i.e. the probability that the
  activity at some site branches into more than one
  active site, is balanced by the probability that
  the activity dies

29
Simulation of the sandpile model 3
Sandpile Java applet
30
The Edge of Chaos 4

• Christopher Langtons 1-D CA
• States alive, dead if a cell and its
  neighbors are dead, they will remain dead in the
  next generation
• Some CAs are boring since all cells either die
  after few generations or they quickly settle into
  simple repeating patterns
• These are highly ordered CAs
• The behavior is predictable
• Other CAs are boring because their behavior
  seems to be random
• These are chaotic CAs
• The behavior is unpredictable

31
The Edge of Chaos

• Some CAs show interesting (complex, lifelike)
  behavior
• These are near the border of between chaos and
  order
• If they were more ordered, they would be
  predictable
• If they were less ordered, they would be chaotic
• This boundary is called the Edge of Chaos
• Langton defined a simple number that can be used
  to help predict whether a given CA will fall in
  the ordered realm, in the chaotic realm, or near
  the boundary
• The number (0 ? l ? 1) can be computed from the
  rules of the CA. It is simply the fraction of
  rules in which the new state of the cell is living

32
The Edge of Chaos

• Remember that the number of rules (R) of a CA is
  determined by ,
• where K is the number of states and N is the
  number of neighbors
• If l 0, the cells will die immediately if l
  1, any cell with a living neighbor will live
  forever
• Values of l close to zero give CA's in the
  ordered realm and values near one give CA's in
  the chaotic realm. The edge of chaos is somewhere
  in between
• Value of l does not simply represent the edge of
  chaos. It is more complicated

33
The Edge of Chaos

• You can start with l 0 (death) and add randomly
  rules that lead to life instead of death gt l gt 0
• You get a sequence of CAs with values of l
  increasing from zero to one
• In the beginning, the CAs are highly ordered and
  in the end they are chaotic. Somewhere in
  between, at some critical value of l, there will
  be a transition from order to chaos
• It is near this transition that the most
  interesting CA's tend to be found, the ones that
  have the most complex behavior
• Critical value of l is not a universal constant

34
The Edge of Chaos
Edge of Chaos CA-applet
35
EOC another approach 5

• Consider domino blocks in a row (stable state,
  minimally stable?)
• Once the first block is nudged, an avalanche is
  started
• The system will become stable once all blocks are
  lying down
• The nudge is called perturbation and the duration
  of the avalanche is called transient
• The strength of the perturbation can be measured
  in terms of the effect it had i.e. the length of
  time the disturbance lasted (or the transient
  length) plus the permanent change that resulted
  (none in the domino case)
• Strength of perturbation is a measure of
  stability!

36
EOC another approach

• Examples of perturbation strength
• Buildings in earthquakes we require short
  transient length and return to the initial state
  (buildings are almost static)
• Air molecules they collide with each other
  continually, never settling down and never
  returning to exactly the same state (molecules
  are chaotic)
• For air molecules the transient length is
  infinite, whereas for our best building method it
  would be zero. How about in the middle?

37
EOC another approach

• A room full of people
• A sentence spoken may be ignored (zero
  transient), may start a chain of responses which
  die out and are forgotten by everyone (a short
  transient) or may be so interesting that the
  participants will repeat it later to friends who
  will pass it on to other people until it changes
  the world completely (an almost infinite
  transient - e.g. the Communist Manifesto by Karl
  Marx is still reverberating around the world
  after over 120 years)
• This kind of instability with order is called the
  Edge of Chaos, a system midway between stable and
  chaotic domains

38
EOC another approach

• EOC is characterised by a potential to develop
  structure over many different scales and is an
  often found feature of those complex systems
  whose parts have some freedom to behave
  independently
• The three responses in the room example could
  occur simultaneously, by affecting various group
  members differently

39
EOC another approach

• The idea of transients is not restricted in any
  way and it applies to different type of systems
• Social, inorganic, politic, psychological
• Possibility to measure totally different type of
  systems with the same measure
• It seems that we have a quantifiable concept that
  can apply to any kind of system. This is the
  essence of the complex systems approach, ideas
  that are universally applicable

40
Correlation Distance 5

• Correlation is a measure of how closely a certain
  state matches a neighbouring state, it can vary
  from 1 (identical) to -1 (opposite)
• For a solid we expect to have a high correlation
  between adjacent areas, but the correlation is
  also constant with distance
• For gases correlation should be zero, since there
  is no order within the gas because each molecule
  behaves independently. Again the distance isn't
  significant, zero should be found at all scales

41
Correlation Distance

• Each patch of gas or solid is statistically the
  same as the next. For this reason an alternative
  definition of transient length is often used for
  chaotic situations i.e. the number of cycles
  before statistical convergence has returned
• When we can no longer tell anything unusual has
  happened, the system has returned to the steady
  state or equilibrium
• Instant chaos would then be said to have a
  transient length of zero, the same as a static
  state - since no change is ever detectable. This
  form of the definition will be used from now on

42
Measuring Complex Systems 5

• For complex systems we should expect to find
  neither maximum correlation (nothing is
  happening) nor zero (too much happening), but
  correlations that vary with time and average
  around midway
• We would also expect to find strong short range
  correlations (local order) and weak long range
  ones
• E.g. the behavior of people is locally correlated
  but not globally
• Thus we have two measures of complexity
• Correlations varying with distance
• Long non-statistical transients

43
Phase Transitions 5,6

• Phase transition studies came about from the work
  begun by John von Neumann and carried on by
  Steven Wolfram in their research of cellular
  automata
• Consider what happens if we heat and cool systems
• At high temperatures systems are in gaseous state
  (chaotic)
• At low temperatures systems are in solid state
  (static)
• At some point between high and low temperatures
  the system changes its state between the two i.e.
  it makes a phase transition
• There are two kinds of phase transitions first
  order and second order

44
Phase Transitions

• First order we are familiar with when ice melts
  to water
• Molecules are forced by a rise in temperature to
  choose between order and chaos right at 0 C,
  this is a deterministic choice
• Second order phase transitions combine chaos and
  order
• There is a balance of ordered structures that
  fill up the phase space
• The liquid state is where complex behaviour can
  arise

45
Phase Transitions 7

• Schematic drawing of CA rule space indicating
  relative location of periodic, chaotic, and
  complex'' transition regimes

46
Phase Transitions

• Crossing over the lines (in the egg) produces a
  discrete jump between behaviors (first order
  phase transitions)
• It is also possible that the transition regime
  acts as a smooth transition between periodic and
  chaotic activity (like EOC experiments with l).
  This smooth change in dynamical behavior (smooth
  transition) is primarily second-order, also
  called a critical transition

47
Phase Transitions

• Schematic drawing of CA rule space showing the
  relationship between the Wolfram classes and the
  underlying phase-transition structure

48
Wolframs four classes 8

• Different cellular automata seem to settle down
  to
• A constant field (Class I)
• Isolated periodic structures (Class II)
• Uniformly chaotic fields (Class III)
• Isolated structures showing complicated internal
  behavior (Class IV)

49
Phase Transitions

• Phase transition feature allows us to control
  complexity by external forces
• Heating or perturbing system gt chaotic behavior
• Cooling or isolating system gt static behavior
• This is seen clearly in relation to brain
  temperature
• Low static, hypothermia
• Medium normal, organised behaviour
• High chaotic, fever

50
Highly Optimized Tolerance 9

• HOT is a mechanism that relates evolving
  structure to power laws in interconnected systems
• HOT systems arise, e.g. in biology and
  engineering where design and evolution create
  complex systems sharing common features
• High efficiency
• Performance
• Robustness to designed-for uncertainties
• Hypersensitivity to design flaws and
  unanticipated perturbations
• Nongeneric, specialized, structured
  configurations
• Power laws

51
Highly Optimized Tolerance

• Through design and evolution, HOT systems achieve
  rare structured states which are robust to
  perturbations they were designed to handle, yet
  fragile to unexpected perturbations and design
  flaws
• E.g. communication and transportation systems
• Systems are regularly modified to maintain high
  density, reliable throughput for increasing
  levels of user demand
• As the sophistication of the systems is
  increased, engineers encounter a series of
  tradeoffs between greater productivity and the
  possibility of the catastrophic failure
• Such robustness tradeoffs are central properties
  of the complex systems which arise in biology and
  engineering

52
Highly Optimized Tolerance 9,10

• Robustness tradeoffs also distinguish HOT states
  from the generic ensembles typically studied in
  statistical physics under the scenarios of the
  edge of chaos and self-organized criticality
• Complex systems are driven by design or evolution
  to high-performance states which are also
  tolerant to uncertainty in the environment and
  components
• This leads to specialized, modular, hierarchical
  structures, often with enormous hidden
  complexity with new sensitivities to unknown or
  neglected perturbations and design flaws

53
Highly Optimized Tolerance 10
Fragile

• Robust, yet fragile!

Robustness of HOT systems
Fragile (to unknown or rare perturbations)
Robust (to known and designed-for uncertainties)
Uncertainties
Robust
54
A simple spatial model of HOT

• Square site percolation or simplified forest
  fire model
• Carlson and Doyle,
• PRE, Aug. 1999

55
A simple spatial model of HOT
Assume one spark hits the lattice at a single
site
A spark that hits an empty site does nothing
56
A simple spatial model of HOT
A spark that hits a cluster causes loss of that
cluster
57
A simple spatial model of HOT
Yield the density after one spark
58
1
Y (avg.) yield
0.9
critical point
0.8
0.7
0.6
N100 (size of the lattice)
0.5
0.4
0.3
0.2
0.1
0
0
0.2
0.4
0.6
0.8
1
? density
59
(No Transcript)
60
Y
Fires dont matter.
Cold
?
61
Everything burns.
Y
Burned
?
62
Critical point
Y
?
63
Power laws
Criticality
cumulative frequency
cluster size
64
Edge-of-chaos, criticality, self-organized
criticality (EOC/SOC)

• Essential claims
• Nature is adequately described by generic
  configurations (with generic sensitivity)
• Interesting phenomena are at criticality (or
  near a bifurcation)

yield
density
65
Highly Optimized Tolerance (HOT)
critical
Cold
Burned
66
Why power laws?
Optimize Yield
Almost any distribution of sparks
Power law distribution of events
67
Probability distribution of sparks
High probability region
2.9529e-016
0.1902
5
10
15
20
25
30
2.8655e-011
4.4486e-026
5
10
15
20
25
30
68
Increasing Design Degrees of Freedom

• The goal is to optimize yield (push it towards
  the upper bound)
• This is done by increasing the design degrees of
  freedom (DDOF)
• Design parameter ? for a percolation forest fire
  model

DDOF1
69
Increasing Design Degrees of Freedom
DDOF 4
4 tunable densities Each region is
characterized by the ensemble of random
configurations at density ?i
70
Increasing Design Degrees of Freedom
DDOF 16
16 tunable densities
71
Design Degrees of Freedom Tunable Parameters
SOC 1 DDOF
72
Design Degrees of Freedom Tunable Parameters

• The HOT states specifically optimize yield in the
  presence of a constraint
• A HOT state corresponds to forest which is
  densely planted to maximize the timber yield,
  with firebreaks arranged to minimize the spread
  damage

Blue ? ?c Red ? 1
HOT many DDOF
73
HOT Many mechanisms
grid
evolved
DDOF
All produce

• High densities
• Modular structures reflecting external
  disturbance patterns
• Efficient barriers, limiting losses in cascading
  failure
• Power laws

74
Small events likely
Optimized grid
density 0.8496 yield 0.7752
1
0.9
High yields
0.8
0.7
0.6
grid
0.5
0.4
random
0.3
0.2
0.1
0
0
0.2
0.4
0.6
0.8
1
75
Increasing DDOF increases densities, increases
yields, decreases losses but increases sensitivity
Y
Robust, yet fragile
?
76
HOT summary

• It is possible to drive a system over the
  critical point, where SOC systems collapse. These
  overcritical states are called highly optimized
  tolerance states
• SOC is an interesting but extreme special case
• HOT may be a unifying perspective for many
  systems
• HOT states are both robust and fragile. They are
  ultimately sensitive for design flaws
• Complex systems in engineering and biology are
  dominated by robustness tradeoffs, which result
  in both high performance and new sensitivities to
  perturbations the system was not designed to
  handle

77
HOT summary

• The real work with HOT is in
• New Internet protocol design (optimizing the
  throughput of a network by operating in HOT
  state)
• Forest fire suppression, ecosystem management
• Analysis of biological regulatory networks
• Convergent networking protocols

78
Summary and Conclusions

• Catastrophe Chaos Complex
• Common features, different approaches
• Complexity adds dimensions to Chaos
• Lack of useful applications
• Self-Organized Criticality
• Refers to tendency of large dissipative systems
  to drive themselves to a critical state
• Coupled systems may collapse during an
  avalanche
• Edge of Chaos
• Balancing on the egde between periodic and
  chaotic behavior

79
Summary and Conclusions

• Parts of a system together with the environment
  make it all function
• Complex systems
• Structure is important in complex systems
• Between periodic (and static) and chaotic systems
• Order and structure to chaos
• Increasing the degrees of freedom
• HOT
• Optimizing the profit/yield/throughput of a
  complex system
• By design one can reduce the risk of catastrophes
• Yet fragile!

80
References

• 1Bunde, Havlin (Eds.) Fractals in Science
  1994
• 2http//cmth.phy.bnl.gov/maslov/soc.htm
• 3http//cmth.phy.bnl.gov/maslov/Sandpile.htm
• 4http//math.hws.edu/xJava/CA/EdgeOfChaos.html
• 5http//www.calresco.org/perturb.htm
• 6http//www.wfu.edu/petrejh4/PhaseTransition.h
  tm
• 7http//www.theory.org/complexity/cdpt/html/nod
  e5.html
• 8http//delta.cs.cinvestav.mx/mcintosh/newweb/
  what/node8.html

81
References

• 9Carlson, Doyle Highly Optimized Tolerance
  Robustness and Design in Complex Systems 1999
• 10Doyle HOT-intro Powerpoint presentation,
  John Doyle www-pages http//www.cds.caltech.edu/
  doyle/CmplxNets/
• 11http//www.lassp.cornell.edu/sethna/OrderPara
  meters/TopologicalDefects.html
• 12http//www.exploratorium.edu/complexity/CompL
  exicon/catastrophe.html
• 13http//necsi.org/guide/concepts/
• 14http//www.neci.nec.com/homepages/tang/sand/s
  and.html

82
References

• 15http//pespmc1.vub.ac.be/COMPLEXI.html
• 16http//pespmc1.vub.ac.be/CAS.html