Analog Logic: Programmable, ContinuousTime, Analog Circuits for Statistical Signal Processing

1
Analog LogicProgrammable, Continuous-Time,
Analog Circuits for Statistical Signal
Processing

• Benjamin Vigoda, Ph.D.

MIT Media Laboratory
Physics and Media Group
NSF CCR-0122419
Benjamin Vigoda, September, 2003
2
Digital Scaling Limits

• Limits of Digital Scaling
• Speed of Light (clocking)
• Power consumption
• Heat dissipation
• Statistics of electrons
• Managing Complexity (10s millions of transistors
  and growing. Each must work perfectly.)

Benjamin Vigoda, September, 2003
3
Analog Scaling Limits

• When applicable analog is
• 5-10x faster digital
• 10-100x lower power than digital
• More Dynamic Range
• Limits of Analog
• Not modular
• Not programmable
• Limited scope of applications
• Hard to scale to new fabrication process
• Analog-to-Digital Converters scale much slower
  than Moores Law

Benjamin Vigoda, September, 2003
4
Statistical Inferencewith a Digital Signal
Processor (DSP)

• DSP converts analog signal to digital then
  applies model
• Want a smarter Analog-to-Digital Converter
• Applies model of the transmitted signal before
  making digital decisions
• A Dynamical System that Performs Inference

Benjamin Vigoda, September, 2003
5
Applications

• Ultra Wideband (UWB) and Pulse-based Radio
• High-speed interconnect
• DS/CDMA Spread Spectrum
• Low-power processors for statistical inference
• Sensor networks
• Interfaces
• Asynchronous Logic

Benjamin Vigoda, September, 2003
6
Nonlinear Dynamical Systemsfor Communications
Chaotic Communication

• Chaotic Entrainment

Circuit Implementation of Synchronized Chaos with
Applications to Communications. Cuomo and
Oppenheim. Physical Review Letters,
1993. Analysis and CMOS Implementation of a
Chaos-based Communication System. Mandal and
Banerjee. Submitted to IEEE Transactions on
Circuits and Systems I.
Benjamin Vigoda, September, 2003
7
Statistical Inference withFactor Graphs
Benjamin Vigoda, September, 2003
8
Factor Graphs Solve Many Problems
Figures from Circuit Generalized Belief
Propagation and Free Energy Minimization.
Jonathan Yedidia. Information Theory Workshop at
Mathematical Sciences Research Institute (MSRI),
March 2002
Benjamin Vigoda, September, 2003
9
Factor Graphsfor Engineering Complex
Computational Systems
I basically know of two principles for treating
complicated systems in simple ways modularity
and abstraction I believe that probability
theory implements these two principles in deep
and intriguing ways -- namely through
factorization and through averaging. Michael
I. Jordan, Massachusetts Institute of
Technology, 1997.
Benjamin Vigoda, September, 2003
10
Factor Graph Example soft inverter
Benjamin Vigoda, September, 2003
11
Factor Graph Example soft inverter
Benjamin Vigoda, September, 2003
12
Factor Graph Example soft-XOR
Benjamin Vigoda, September, 2003
13
Soft-Gates In General
Factor Graphs and the Sum-Product Algorithm.
Kschischang, Frey and Loeliger. IEEE Transactions
on Information Theory, 1998.
Benjamin Vigoda, September, 2003
14
Factor Graph Example Marginalization on
Tree(message passing metaphor)
Problem Find marginal probability p(z) Given
p(w) and p(y) 1. Find p(x) from p(w) using
soft-inverter 2. Find p(z) from p(x) and p(y)
using soft-XOR
Imagine that nodes are sending messages along the
edges
Benjamin Vigoda, September, 2003
15
Factor Graph Example Marginalization on
Tree(variable nodes multiply incoming messages )
Problem Find marginal probability p(z)
Given p(w), p(x), p(y) and p(w,x,y,z)
1. Find p(z) message from p(w) using
soft-inverter 2. Find p(z) message from p(x) and
p(y) using soft-XOR 3. Multiply p(z) messages
together 4. Normalize
Benjamin Vigoda, September, 2003
16
Factor Graphs Joint Marginals(Generalized
Belief Propagation)
Problem Find joint probability p(x,y) Given
p(z)
Constructing Free Energy Approximations and
Generalized Belief Propagation Algorithms.
Yedidia, Freeman and Weiss. IEEE Transactions on
Information Theory. 2002
Benjamin Vigoda, September, 2003
17
Factor Graphs Graphs with Cycles (Loopy)

• If a graph has loops, it may not settle to an
  answer. (Try damping)
• If it settles, answer may be approximate
  (wrong)
• But the solution will be a stationary point of
  the Bethe approximation to the free energy.

Stable fixed points of belief propagation are
minima of the Bethe free energy. Tom Heskes.
Neural Information Processing. 2002
Benjamin Vigoda, September, 2003
18
Ring Oscillator Synchronization with Factor
Graphs
Benjamin Vigoda, September, 2003
19
The Ring (Relaxation) Oscillator
Ring Oscillator Circuit
Ring Oscillator Factor Graph
Benjamin Vigoda, September, 2003
20
Receiver Ring Oscillator TrellisSoft-inverter
message
Benjamin Vigoda, September, 2003
21
Receiver Ring Oscillator TrellisMaximum-Likeliho
od Synchronization
Trellis section for ring oscillator receiver
Benjamin Vigoda, September, 2003
22
Receiver Ring Oscillator TrellisChannel Model
Message
Benjamin Vigoda, September, 2003
23
Receiver Ring Oscillator TrellisMessage from
state node
Benjamin Vigoda, September, 2003
24
Coupled Ring OscillatorsPerform
Maximum-Likelihood Synchronization
Transmit ring oscillator
Receiver ring oscillator rolled up version of
receiver ring oscillator trellis
Benjamin Vigoda, September, 2003
25
Ring Oscillator Movie
Benjamin Vigoda, September, 2003
26
LFSR Synchronization with Factor Graphs
Benjamin Vigoda, September, 2003
27
Linear Feedback Shift Register (LFSR)
(000111101011001) or 0
Benjamin Vigoda, September, 2003
28
LFSR Synchronization
Benjamin Vigoda, PhD, August, 2003
29
Applications of LFSR Synchronization

• LFSR Synchronization in
• Global Positioning Systems (GPS)
• DS/CDMA (cell phones, etc.)
• More general Phase Lock Loop (PLL)
• Clock distribution
• Multi-user lite
• Agile modulation

Benjamin Vigoda, September, 2003
30
LFSR Trellis Maximum-Likelihood but Exponential
Complexity
Benjamin Vigoda, September, 2003
31
Factor Graph for LFSR Trellis(forward-only)
Benjamin Vigoda, September, 2003
32
Shift Graph for LFSR State Estimation
Benjamin Vigoda, September, 2003
33
Shift Graph for LFSR State EstimationForward-Only
Message Passing
Benjamin Vigoda, September, 2003
34
Shift Graph for LFSR State Estimation
Benjamin Vigoda, September, 2003
35
Shift Graph for LFSR State EstimationA Single
Section
Benjamin Vigoda, September, 2003
36
Shift Graph for LFSR State Estimation The Noise
Lock Loop (NLL)
Benjamin Vigoda, September, 2003
37
The Noise Lock Loop
Transmitter LFSR factor graph
Receiver LFSR factor graph Noise Lock Loop
Low-Complexity LFSR Synchronization by
Forward-Only Message Passing, Vigoda, Dauwels,
Gershenfeld and Loeliger, submitted IEEE
Transactions on Information Theory, 2003
Benjamin Vigoda, September, 2003
38
NLL Results

• Nice distribution of synchronization times
• (15-bin, 2-tap, SNR 3dB)

Low-Complexity LFSR Synchronization by
Forward-Only Message Passing, Vigoda, Dauwels,
Gershenfeld and Loeliger, submitted IEEE
Transactions on Information Theory, 2003
Benjamin Vigoda, September, 2003
39
NLL Results

• Less noise shorter synchronization time
• Shorter LFSR sequence shorter synchronization
  time
• NLL synchronizes on some percentage of trials
• Trellis always synchronizes eventually
• With GBP we should be able to turn a knob
  between the Trellis and the NLL.

Low-Complexity LFSR Synchronization by
Forward-Only Message Passing, Vigoda, Dauwels,
Gershenfeld and Loeliger, submitted IEEE
Transactions on Information Theory, 2003
Benjamin Vigoda, September, 2003
40
Circuits
Benjamin Vigoda, September, 2003
41
Soft-Gate Circuits
Probability Propagation and Decoding in Analog
VLSI, Loeliger, Lustenberger, Helfenstein, and
Tarköy, IEEE International Symposium on
Information Theory, 1998
Benjamin Vigoda, September, 2003
42
Soft-XOR Circuit
Translinear circuits A proposed classification.
Barrie Gilbert. Electronics Letters, 1975.
Benjamin Vigoda, September, 2003
43
Variable Node Circuit
Benjamin Vigoda, PhD, August, 2003
44
Design FlowCompile MATLAB to Spice to Circuits
Waveform probing commands .probe .options
probefilename"D probesdbfile"D
probetopmodule"3 softxors" .SUBCKT softxor p_x0
p_x1 p_y0 p_y1 p_z0 p_z1 GND Vdd M4 N5 p_y1 N7
GND NMOS_49 W'48l' L'4l' AS'40ll'
AD'40ll' PS'24l' PD'24l' M1 M8 N1 p_y0 N7
GND NMOS_49 W'48l' L'4l' AS'40ll'
AD'40ll' PS'24l' PD'24l' M1 M7 N8 p_y1 N1
GND NMOS_49 W'48l' L'4l' AS'40ll'
AD'40ll' PS'24l' PD'24l' M1 M3 N8 p_y0 N5
GND NMOS_49 W'48l' L'4l' AS'40ll'
AD'40ll' PS'24l' PD'24l' M1 M9 GND p_x1
N1 GND NMOS_49 W'48l' L'4l' AS'40ll'
AD'40ll' PS'24l' PD'24l' M1 M6 N5 p_x0
GND GND NMOS_49 W'48l' L'4l' AS'40ll'
AD'40ll' PS'24l' PD'24l' M1 M12 p_z1 N7
Vdd N6 PMOS_49 W'48l' L'10l' AS'66ll'
AD'66ll' PS'60l' PD'60l' M1 M11 Vdd N7 N7
N2 PMOS_49 W'48l' L'10l' AS'66ll'
AD'66ll' PS'60l' PD'60l' M1 .ENDS Main
circuit 3 softxors .tran 1n 1600n .include
Mami_15.md .options abstol1e-15 .param l0.8u R1
N1 GND 50 TC0.0, 0.0 R2 N7 GND 50 TC0.0,
0.0
SPICE netlist written by Bayes2Gates (c) 2002
Ben Vigoda, MIT Media Lab Written on
28-Jun-2002
Xsoftxor_1 N5 N3 N4 N19 N11 N10 GND Vdd
softxor Xsoftxor_2 N11 N10 N9 N8 N7 N1 GND Vdd
softxor Xsoftxor_3 N2 N16 N15 N14 N9 N8 GND Vdd
softxor
MATLAB Bayes Net Toolbox
Spice Netlist
Benjamin Vigoda, September, 2003
45
Programmability Soft-Gate Arrays
Benjamin Vigoda, September, 2003
46
Programmable Soft-Gate Arrays The
Soft-Multiplexer
Benjamin Vigoda, September, 2003
47
RF Soft-Gates2 orders of magnitude less power

• 1GHz 8-Bit Digital Multiply-Accumulate (MAC)
• Several 1000 transistors
• Approx. .1uW per transistor
• ¼ 100uW
• 1GHz soft-xor multiply circuit with 8-bit
  resolution
• 255mV dynamic range
• 1mV input referred voltage noise
• 255 analog levels performs 8 bit multiply
• ¼ 1 to 10uW
• Can trade-off Speed, Power and Resolution

Benjamin Vigoda, September, 2003
48
Continuous-Time
Benjamin Vigoda, September, 2003
49
Continuous-Time Analog Memory ElementsChebyshev
Filter
Benjamin Vigoda, September, 2003
50
Noise Lock Loop Transmitter Simulation
Circuit signals in Spice
Amplitude
Chebyshev filter output
XOR output
time
Benjamin Vigoda, September, 2003
51
Noise Lock Loop Transmitter Circuit
Benjamin Vigoda, September, 2003
52
Noise Lock Loop Receiver Circuit
MATLAB simulation results for NLL
Amplitude
time
Benjamin Vigoda, September, 2003
53
Current Research
Benjamin Vigoda, September, 2003
54
Convolutional Coders

• Pulse-based radio (Ultra-Wide Band)
• Disciplined Chaotic Coding

Benjamin Vigoda, September, 2003
55
Continuous-Time Analog Memory Circuits
Soft-Gate Soliton Circuits
Benjamin Vigoda, PhD, August, 2003
56
Binary Counters

• Useful for Asynchronous Logic?

Benjamin Vigoda, September, 2003
57
Continuous-Time, Analog, Distributed Computation
Benjamin Vigoda, PhD, August, 2003
58
Continuous-Time, Analog, Distributed Computation

• Analog Circuits imply Continuous-Time
• Sharp transitions add glitches and noise to
  analog circuits
• Analog Circuits imply Distributed
• Delicate analog signals should not travel far
• Un-clocked implies Distributed
• Centralized computing without a clock is fragile
  (race-conditions) or costly (asynchronous logic)
• Peer-to-peer self-synchronization should be
  short-range (small delays) for stability
• Un-clocked implies Analog Circuit
• Analog circuits can locally self-synchronize
• Distributed implies Analog Circuits
• parallelism creates interconnect complexity
  compared to centralized bus architecture
• Analog can reduce the number of wires
• Distributed implies Un-clocked
• Clocks are costly (power, silicon area for clock
  distribution tree)
• Distributed processors should only synchronize
  to communicate
• What if it were feasible for each processor was
  like a user in a cell-phone network with
  peer-to-peer synchronization?

Benjamin Vigoda, PhD, August, 2003
59
Continuous-Time, Analog, Distributed Computation
Benjamin Vigoda, PhD, August, 2003
60
Fault Tolerant Architectures
Coding for Logical Operations. S. Winograd. IBM
J. Res. Develop., 6430--436, 1962.
Benjamin Vigoda, PhD, August, 2003
61
Factor Graphs for Probabilistic Ad-hoc Routing
Benjamin Vigoda, September, 2003
62
Summary

• Arbitrary waveform generators that synchronize
• Factor graphs make it easy
• Knob for quality of synchronization vs.
  computational complexity
• Design Flow
• Data and Constraints
• Factor graph (MATLAB)
• Compile to circuits
• Layout or Program Gate array
• Beyond Analog Vs. Digital
• Recover continuous degrees of freedom of devices
• Digital thresholds always, analog thresholds
  never, we threshold just enough
• Higher speed, lower power
• Robust scaling

Benjamin Vigoda, September, 2003
63
MAP and ML Estimation
Benjamin Vigoda, PhD, August, 2003
64
ML Decoding
Benjamin Vigoda, PhD, August, 2003
65
Statistical Inference with Factor Graphs A Code
Benjamin Vigoda, PhD, August, 2003
66
Statistical Inference with Factor Graphs Decoder
Benjamin Vigoda, PhD, August, 2003