Unit failure and circuit complexity or Why do longliving animals need larger brains

1
Unit failure and circuit complexityorWhy do
long-living animals need larger brains?

• Alexander G DimitrovCenter for Computational
  Biology andDepartment of Cell biology and
  NeuroscienceMontana State UniversityBozeman,
  Montana

2
Different nervous systems

• Mammals
• Many neurons
• Seem unreliable
• Seem imprecise (gt 5ms precision)
• Synaptic failure rate gt .75
• However! Loss of a few is irrelevant
• Insects
• Few neurons
• In general reliable
• In general precise (lt .1 ms precision)
• Synaptic failure rate lt .1
• However! Loss of a few leads to major deficiencies

3

An orientation-selective neuron in the cricket
cercal sensory system.
4
Olfactory system fan-out
x 1-5
x 25-50
Hildebrand Shepherd, 97
5
Computations with unreliable elements

• Option 1 produce better, more reliable and
  fault-tolerant building elements.
• Like in insects, modern computers, VLSI.
• Except that insects are much cheaper to build,
  and there are so many of them
• Option 2 produce circuits that can perform the
  computation even with unreliable elements.
• Like mammals, where the basic building blocks are
  numerous, and relatively cheap.
• Research in the 60-s, shadowing Shannons
  Information Theory (von Neumann, Cowan, Winograd,
  Shannon)

6
Fault-tolerant circuits.With failure-prone
elements

• Replicate system.
• Like insects (thousands of clones), or modern
  computer chips if one breaks, there are more
• Replicate units.
• Von Neumann replicate every element N-fold. A
  circuit with k elements expands to n N k. In
  his solution, N is large (5000-10000 fold
  increase). Better solutions with N 20-50.
• Important idea Part of the circuit performs
  noise correction essentially by fixed point
  dynamics.
• Expand circuit to compute in fault-tolerant
  manner
• Cowan expand k elements to n, increase (k,n) so
  that the computational capacity Rk/n remains
  constant. N n/k. Modest increase in the number
  of elements.
• Modify the signal representation, from k inputs
  (outputs) expand to n inputs (outputs). Perform
  the computation in the expanded space, with the
  expanded circuit. Perform noise correction in the
  expanded space!

7
Modifying the communication channel
Source coder channel decoder .

• Shannons formulation coder which expands the
  representation to protect from noise, decoder
  projects the representation back in the original
  space.
• New model coder expands the representation, then
  all computation and noise correction is performed
  in expanded space.

N?
Source coder channel computer
N?F
8
Model for noise-tolerant circuits
Extend to
A circuit with k binary units. A deterministic
computation maps X to Y. The circuit need not be
monolithic (i.e., can be a collection of several
circuits).
A circuit with ngtk binary units. A noisy
computation maps X to Y. There is a
deterministic mapping (coding) between (X,Y) and
(X,Y). The circuit is monolithic. All
computations in the original circuits could be
multiplexed.
9
Error correction in fault-tolerant circuits

• Treat in information-theoretic manner.
• Theorem (Cowan and Winograd, 63)All
  computational rates up to capacity are
  asymptotically achievable.
• Corollary The probability of error in a circuit
  for sufficiently large n is Pe?2-nI.

10
(No Transcript)
11
Properties size of Means that there are
about distinguishable sequences in X, which can
be coded by Y, with probability of error
.
12
Error correction in fault-tolerant circuits
details

• Consider a computation with n binary units
  (extended).
• Assume a probability of error ? for each unit.
• The typical noise set for each computation then
  has about 2nH(?) elements.
• The number of useful computations that can be
  performed is about 2nI(?), where I(?)
  H(input)-H(?) ? 1-H(e) at capacity.
• Hence, the proportion of units dedicated to error
  correction is about nH(?), the proportion
  dedicated to computation is kn(1- H(?)).
• The computational capacity is R k/n 1- H(?).

13
How to test whether a nervous system uses such
circuits?
14
Relation between circuit complexity and expected
lifespan

• The expected probability of error (malfunction)
  per circuit use is approximately Pe2-nI(?). This
  probability can be made arbitrarily small by
  increasing n.
• The expected number of errors for N uses is
  Ne?Npe.
• The number of circuit uses N is proportional to
  the lifespan T N ?T.
• Corollary The less you use your brain, the less
  likely it is to malfunction ?
• Conjecture Ne is the same for animals with
  different expected lifespans An animal will
  experience on the average ½ errors per expected
  lifespan.

15
Relation of circuit complexity to expected
lifetime

• For circuits with similar unit error rate,
  N12-In1 N22-In2. Hence, N1/N2 2I(n1-n2),
  or I(n1-n2) ln2 N1/N2
• Assuming constant rate of circuit use in related
  species, I(n1-n2) ln2 T1/T2
• Normalizing to a standard animal (no,To) n
  a b ln T a no - 1/I ln To b 1/I

16
Issues with comparing to data

• Direct approach unreliable measurements.
• Difficulty with finding homologous circuits in
  animals with radically different lifespan.
• Difficulty with determining the core size of
  the circuit the minimal set of elements which
  performs useful computations.
• Care should be taken not to include effects of
  brain size with various allometric quantities
  (body mass, diet, etc.)
• Try local signatures of fault tolerant circuits.

17
One extra step

• There is a relation between the number of
  contacts per unit and the number of units
  (complexity) in the circuit!
• Theorem (Winograd, 63)If t errors are
  corrected anywhere in a circuit, then the average
  number of inputs per module in the circut, s is
  bounded by s ? (2t1) R
• Corollary Since t ? 2nH(e) for large circuit
  size n, the average number of inputs s C
  2nH(e), C 2R ? n c d ln s c - ln R, d
  1/H(e)(has to be tested against direct
  measurements!)

18

• Combining those together yields a relation
  between the average number of contacts per
  circuit element in fault-tolerant circuits, and
  system longevity c d ln s a b ln T
  ? s A TB (scaling relation), with A(e,R),
  and B(e) H(e)/I(e)2
• For small e, B(e) ? e ln e, so larger exponent
  means more noise.

19
Comparison to available dataindirect measures

• Direct estimates of number of synapses per unit
  in a few species (review in Peters, 87) too
  variable!
• Data on volumes of gray matter for a number of
  mammals (Bush and Allman, 03).
• A relation between synapses/unit and volume of
  gray matter s Vgray1/3 (Tower 54, Abeles 91,
  Prothero 97, summarized by Changizi, 00).
• Data on longevity of monkeys (Allman et.al.,
  93).
• All these combined provide an estimate for number
  of synapses/unit as a function of longevity for
  monkeys.

20
Cerebellar data, monkeys
21
Neocortical data, monkeys
?
22
Signatures of fault-tolerant circuits ways to
decrease the cumulative error rate

• Lower activity (use units as little as you can
  get away with)
• Multiple connections
• Multiplexing of function (distributed function)
• Imprecise units can be used there is relatively
  little overhead once the circuit is in place.

23
An interesting conjecture

• At some point, there may be a tradeoff between
  the amount of resources dedicated to improving
  circuit elements versus the amount dedicated
  expanding the circuit.
• I.e., systems that need large circuits anyway,
  may use cheaper, more failure-prone elements!
  (number of elements proportional to log T, log
  1/e)

24
To do extend to more species. E.g. Zhang and
Sejnowski, 2000.Need longevity data! Test with
direct counts synapses/neuron.
25
Conclusions

• Neural systems need to compute with unreliable
  elements.
• The expected lifespan of the organism increases
  the precision demand on the system ( linearly
  with time).
• Short-living systems (like insects) may be able
  to get away with no failure protection other than
  cloning.
• For long-living systems this is not an option
  the probability for malfunction or death of an
  elements is essentially 1!

26
(No Transcript)