Iterative Coding for Broadband Communications: New Trends in Theory and Practice

1
Iterative Coding for Broadband Communications
New Trends in Theory and Practice

• Amir H. Banihashemi
• Broadband Communications and Wireless Systems
  (BCWS) Centre
• Dept. of Systems Computer Engineering
• Carleton University

2
outline

• Iterative coding schemes and LDPC codes
• Min-sum algorithm and its modifications
  (Zarkeshvari, Zhao)
• New schedules for iterative decoding (Mao, Xiao)
• Normalized and offset belief propagation
  (Yazdani, Hemati)
• Majority-based algorithms (Zarrinkhat)
• Hybrid algorithms (Zarrinkhat, Xiao)
• Bootstrap decoding and reliability-based
  scheduling (Nouh)
• Iterative decoding in analog electronics and
  optics (Hemati)
• Dynamics of asynchronous continuous-time
  iterative decoding (Hemati)
• RC-LDPC codes in hybrid ARQ schemes (Yazdani)
• LDPC codes on channels with burst errors (Hong)

3
LDPC codes and iterative decoding

• Iterative coding schemes, such as turbo codes and
  LDPC codes, provide excellent performance/complexi
  ty tradeoff.
• Iterative decoding can be naturally described
  using graph representations (Tanner graph (TG)).
• For linear block codes

Check Nodes I II III
1 2 3 7 6
5 4
Variable Nodes
4
Iterative decoding algorithms

• There are a number of iterative message-passing
  decoding algorithms, each offering a particular
  tradeoff between error performance and decoding
  complexity.
• Best performing belief propagation (BP) or
  sum-product (SP). It converges to a posteriori
  probabilities (APP) for bits on a cycle-free
  graph.
• Less complex min-sum (MS), also referred to
  as max-sum, or max-product. It converges to
  Maximum-likelihood (ML) solution for codewords on
  a cycle-free graph.

5
Min-sum algorithm

• Min-sum can be considered as an approximation to
  BP in log-likelihood ratio domain.
• Advantages over BP
• - Simpler to implement
• - Doesnt require an estimate of noise power
• - More robust against quantization error
• Disadvantage Inferior error performance

6
Min-sum algorithm on BI-AWGN Channel

• Min-sum Initialization
• Check node step
• Variable node step
• Hard decision (at variable
• node s)

7
Min-sum algorithm

• Effects of clipping and quantization on MS at
  short block lengths are studied
• - clipping improves the performance,
• - 4 quantization bits provide performance
  close to or even better than that of unquantized
  MS (compared to 6 bits for BP in LLR domain).
• Simple modifications that can considerably
  improve MS performance are proposed
• - Modified MS can outperform BP!

8
Quantized MS (1268,456) irregular code
9
Quantized MS (273,191) and (8000,4000) regular
codes
10
Modified MS Algorithms (1268,456) code
11
Modified MS Algorithms (273,191) code
12
Modified MS Algorithms (8000,4000) code
13
Min-sum and its modifications concluding remarks

• With optimal clipping threshold, only 4 bits
  suffice to obtain near (or even better than)
  unquantized performance.
• Modifications to min-sum algorithm, which
  considerably improve the performance with small
  increase in complexity are proposed.
• In some cases, the modified min-sum algorithms,
  even in their quantized form, outperform belief
  propagation!
• This indicates that algorithms which are optimal
  on cycle-free graphs do not necessarily deliver
  the best performance on graphs with cycles.
• Min-sum with unconditional correction seems to be
  a very good choice for practical digital decoding
  of LDPC codes.

14
Message-passing schedules

• Motivation Given an LDPC code with a particular
  TG, a given channel model, and an iterative
  decoding algorithm, is there any space for
  performance improvement?
• Yes! with similar or even lower complexity!
• Main idea Schedule the message-passing on the TG
  according to the structure of the graph to
  minimize the sub-optimality of the decoder.
• Implementation Girth- and closed-walk-dependent
  schedules
• Node-based vs. Edge-based
• Unidirectional vs.
  Bidirectional
• Deterministic vs.
  Probabilistic

15
Message-passing schedules

• Different schedules provide different
  performance/complexity tradeoffs.
• In general, more complex schedules perform
  better.
• Edge-based and probabilistic schedules are more
  complex to implement compared to node-based and
  deterministic schedules, respectively.
  Bidirectional schedules are roughly twice as
  complex as the corresponding unidirectional ones.
• The performance/complexity tradeoff is not only a
  function of schedule and TG, but also depends on
  decoding algorithm and channel model.

16
Message-passing schedules

• Codes I. Regular (1200,600)
• II. Regular (8000,4000)
• III. Irregular (1268,456)
• IV. Irregular (3072,1024)
• Channel models BSC, AWGN, Rayleigh fading (with
  and without SI)
• Decoding algorithms Gallagers algorithm A, BP,
  MS

17
Edge-based vs. node-based schedule (GA for code I
over BSC)
18
Bidirectional vs. unidirectional schedule (BP for
code II over AWGN channel)
19
Deterministic vs. probabilistic schedule (BP for
code III over AWGN channel)
20
Scheduling for MS (code I over AWGN channel)
21
Scheduling on uncorrelated Rayleigh fading
channels (BP, code IV)
22
Message-passing schedules concluding remarks

• Different schedules provide different tradeoffs
  between error performance and decoding
  complexity.
• The tradeoff depends not only on the girth and
  closed-walk distributions of the TG, but also on
  the decoding algorithm and the channel model.
• In general, the new schedules outperform the
  conventional flooding schedule.

23
Reliability-based schedule (273,191) regular code
24
Reliability-based schedule (273,191) PG code
25
Normalized and offset BP

• Motivation Reliability of BP messages are
  overestimated on graphs with cycles.

26
Majority-based decoding algorithms

• Majority-based algorithms work based on a
  generalized majority-decision rule
• For the ensemble of (dv, dc)-regular graphs
  (dc gt dv 3), a majority based algorithm of
  order ?, 0 ? dv 1 ?dv / 2?, denoted by
  MB?, is defined by

27
Majority-based decoding algorithms

• They are particularly attractive for their
  remarkably simple implementation (per iteration).
  
• Both Gallagers algorithm A and standard majority
  decoding belong to this family.
• We investigate the dynamics of these algorithms
  using density evolution and compute the threshold
  values for regular LDPC codes decoded by these
  algorithms.
• It appears that many of these algorithms enjoy
  very fast convergence, and/or have better
  threshold values compared to Gallagers algorithm
  A.

28
Threshold values
29
Convergence speed
Number of iterations required to achieve an
average fraction of erroneous messages below
10-6. The channel parameter is 90 of the
smallest threshold value amongst different orders.
30
Majority-based decoding concluding remarks

• Many of the majority-based algorithms have a
  larger noise threshold and enjoy a much faster
  convergence compared to Gallagers algorithm A.
• Can be used in conjunction with soft decoding
  algorithms in hybrid platforms to achieve very
  good performance/complexity tradeoffs.

31
Hybrid algorithms

• Combining different iterative decoding algorithms
  with the aim of improving the performance/complexi
  ty tradeoff.
• Suppose that are N
  message-passing algorithms which can be used to
  decode over a given channel. Hybrid
  algorithm
• is defined by
• where
  and
  are
• probability mass functions at iteration
  for partitioning variable and check nodes into N
  partitions, respectively. The nodes in the i th
  partition process the messages according to
  
• Class I
• Class II

32
Hybrid algorithms

• Threshold values for some optimized hybrid
  algorithms

33
Hybrid algorithms concluding remarks

• Hybrid algorithms can provide large improvements
  in threshold and speed of convergence compared to
  their constituent algorithms.
• Class II (switch-type) algorithms have slightly
  better thresholds compared to class I
  (time-invariant) algorithms. The latter class
  however is far less sensitive to channel
  conditions and thus can be practically more
  attractive.
• The convergence region of many majority-based
  algorithms extends to , which
  indicates that these simple algorithms can take
  care of decreasing the error probability to zero
  given that a more powerful algorithm has
  sufficiently reduced it, already.
• Majority-based algorithms are good candidates for
  class II hybrid algorithms.

34
Iterative decoding in analog electronics

• Need for real computations and iterative nature
  of BP algorithm has motivated some very recent
  research on analog implementations (1999 2002).
• This is projected to improve the ratio of speed
  to power consumption by two orders of magnitude.
• Proposed implementations are based on either
  BiCMOS or subthreshold CMOS technologies.
• We show that min-sum algorithm can be implemented
  by full CMOS technology.
• Max winner-take-all (WTA) circuits with high
  swing, low voltage and very good accuracy have
  been designed.

35
Full CMOS min-sum analog iterative decoder

• current-mode circuits
• lower fabrication cost and/or simpler design
  compared to previously reported analog iterative
  decoders that are based on BiCMOS or
  sub-threshold CMOS technology.
• higher robustness in MS is favorable in
  mitigating the problems of mismatch and parameter
  variations due to the change of temperature in
  large analog integrated circuits.

36
Full CMOS min-sum analog iterative decoder

• modules with large number of inputs can be
  fabricated easily and simulations show that
  increasing the number of inputs does not increase
  the delay as much.
• Special circuits have also been designed for deep
  submicron technologies, where short channel
  effects degrade the performance of conventional
  circuits and low voltage power supplies are used.
• functionality of circuits has been tested by
  simulating the decoder based on TSMC 0.18 µm CMOS
  technology for (7,4) Hamming code.
• An MS decoder for a regular (32,24) code has been
  designed and
• submitted for fabrication.

37
Dynamics of asynchronous continuous-time
iterative decoding

• Iterative decoding with flooding schedule can be
  formulated as a fixed-point problem solved
  iteratively by successive substitution method.
• Analog asynchronous decoding can be approximated
  as the application of the well-known successive
  over relaxation (SOR) method for solving the
  fixed-point problem.
• Simulation results confirm that SOR, which is in
  general superior to the simpler successive
  substitution method, can considerably improve the
  performance of BP and MS for short codes.

38
Simulation results
39
Dynamics of analog iterative decoding concluding
remarks

• Implementation of iterative decoding algorithms
  in analog circuits not only increases the ratio
  of speed to power consumption compared to digital
  synchronous circuits, but also can provide a
  better performance.
• This work also suggests yet another framework for
  improving iterative decoding algorithms, in
  general, and belief propagation, in particular,
  on graphs with cycles.

40
RC-LDPC codes in hybrid ARQ schemes

• Type-II hybrid ARQ protocol
• Rate-compatible (RC) LDPC codes constructed by
  progressive edge growth (PEG) construction
• Linear-time encoding
• Design of puncturing and extending patterns

41
RC-LDPC codes in hybrid ARQ schemes
42
Wish I had more time!Thanks!