Lecture 3 Quantization in Signals and Systems

1
Lecture 3Quantization in Signals and Systems

• by
• Graham C. Goodwin
• University of Newcastle
• Australia

Presented at the Zaborszky Distinguished Lecture
Series December 3rd, 4th and 5th, 2007
2
Overview

• Topics to be covered include
• signal quantization,
• predictive and noise shaping quantizers,
• networked control,
• signal coding in networked control,
• channel capacity issues in networked control,
• applications in audio compression and control
  over communication channels.

3
Outline

1. Recall Quantization
2. Predictive and Noise Shaping Quantizers
3. Application to Audio Compression
4. Networked Control
5. Modelling Communication Link
6. Predictive and Noise Shaping Coding
7. Experimental Results
8. Conclusions

4
Recall Basic Idea of Samplingand Quantization
Quantization
t
t1
t3
t2
0
t4
Sampling
5
Quantization

• (Actually we saw some aspects of this in relation
  to coefficient quantization in lecture 2.)
• Here Fix the sampling pattern (say uniform for
  simplicity) and examine the quantization of the
  samples.
• Approaches
• Nonlinear quantization is an inherently
  nonlinear process.
• Linear approximate quantization errors as
  noise.
• To illustrate ideas we will follow route 2.
• (Generally gives design insights.)

6
Signal to Noise Ratio Model for Quantization
b 3 L 7

• b bit quantizer
• levels
• Assume quantization errors are
• white noise uniformly distributed
• We want small probability that signal amplitude
  exceeds
• the range of the quantizer. Assume variance of
  signal is ,
• then e.g. 4 s.d. rule says that
  .
• Hence

Q
range
Uniform Quantizer
7
Outline

1. Quantization
2. Predictive and Noise Shaping Quantizers
3. Application to Audio Compression
4. Networked Control
5. Modelling Communication Link
6. Predictive and Noise Shaping Coding
7. Experimental Results
8. Conclusions

8
Predictive and Noise Shaping Quantizers
(Quantization errors modeled as additive white
noise)
N
C
D
U
E
-
L1
L3
Quantizer
L2
Utilizing the power of feedback!
Note that the feedback loop is related to the
delta operator (lecture 2) since we subtract
what we already know before quantizing/approxi
mating.
(We will return to this approximation later!)
9
Focus on Frequency Weighted (W) Noise Power in D
where is the input signal spectrum
Use normalized transfer functions G(0) 1
10
Heuristic Explanation of the Optimal Design
N
C
D
U
E
-
L1
L3
W
Quantizer
L2

• Spectrum of C and characteristics of W are known.
• We have 3 filters to design.
• One degree of freedom removed by
  Perfect Reconstruction requirement
  i.e.,
• With remaining 2 degrees of freedom can (i) shape
  E to have minimal variance (prediction) and (ii)
  shape component of due to N to have minimal
  variance (noise shaping).

11

• Perfect Reconstruction Constraint
• Minimizing variance of E
• Minimize variance of WD due to N
• Solution

(Whitening Filter Predictive coding)
(Noise shaping)
12
Predictive Coder

• Choose W 1
• Optimal choices are

This solution corresponds to Minimum Variance
Control
13
Noise Shaping Quantizer(Sigma-delta)

• Add extra constraint L3 1
• Optimal Choices
• Then (Noise
  Shaping)
• (Achieved Performance)

14
The Role of Oversampling

• Say we choose L3 1 and W as ideal low pass
  filter

W
1
15
Insights from Feedback Theory

• is a sensitivity function.

We know from Bode integral that


(Water Bed Effect)
Thus making the sensitivity arbitrarily small in
some frequency range automatically means that it
will be arbitrarily large somewhere else!
16

• Indeed, this goes back to the early simplifying
  assumption that

In fact it should have been
and Noise Power in More Complex (but more
realistic) optimization problem.
It turns out to be convex!
17

• In summary we can design an optimal
• quantizer which
• minimizes the impact of quantization noise on the
  output, and
• takes account of the fact that quantization
  errors ultimately need themselves to be quantized
  due to the feed back structure.

18
Outline

1. Quantization
2. Predictive and Noise Shaping Quantizers
3. Application to Audio Compression
4. Networked Control
5. Modelling Communication Link
6. Predictive and Noise Shaping Coding
7. Experimental Results
8. Conclusions

19
Application Audio Compression
Original N0 N1 N2
44.1 kHz Bits 3
Stop
Elvis Presley
20
Other Insights From Control Theory

• (i) Bode integral

Spectrum of Errors due to Quantization
21
Outline

1. Quantization
2. Predictive and Noise Shaping Quantizers
3. Application to Audio Compression
4. Networked Control
5. Modelling Communication Link
6. Predictive and Noise Shaping Coding
7. Experimental Results
8. Conclusions

22
Network Control Systems

• In a Networked Control System (NCS) controller
  and plant are connected via a communication link.
• Therefore, signals transmitted
• Have to be quantized
• May be delayed
• May get lost
• The communication link constitutes a performance
  bottleneck.
• When designing NCSs the characteristics of the
  network should be accounted for to ensure
  acceptable performance levels.
• When comparing to traditional control loops, in
  NCSs there exist additional degrees of freedom
  to be designed.
• It is useful to investigate
• Architectural issues
• Signal coding methods

23
(a)
(b)

• Networked Control Problem

24

• Useful analog to think about

25
Nominal Control Design

• We will consider the situation where an LTI
  controller has already been designed for a SISO
  LTI plant model.
• We will refer to this design as the nominal
  design and we will assume that it gives
  satisfactory performance in a non-networked
  setting.
• We will show how to minimize the impact of the
  communication link on closed loop performance.

26
Design Relations
Disturbance
d



r
y
Reference

Plant Output
-
Controller
Plant

n
Noise

• The tracking error is given by
• where
• are the loop sensitivity functions.

27
Design Relationships

• In non-networked situation we have

• To achieve good reference following and
  disturbance attenuation, C(z) is typically chosen
  such that the open loop gain,
  is large at frequencies where and
  are significant.
• To handle measurement noise and plant model
  inaccuracies, the open loop gain should be
  reduced at appropriate frequencies.

28
Outline

1. Quantization
2. Predictive and Noise Shaping Quantizers
3. Application to Audio Compression
4. Networked Control
5. Modelling Communication Link
6. Predictive and Noise Shaping Coding
7. Experimental Results
8. Conclusions

29
The Communication Link

• The novel ingredient in an NCS, when compared to
  a traditional control loop, is the communication
  link.
• It constitutes a significant bottleneck in the
  achievable performance.
• From a design perspective, this opens the
  possibility of investigating

NCS Architectures
Where do I place the processing power?
Signal Coding
What information do I send?
30
Channel Model

• We will consider an additive Noise model

q
zero-mean stationary white noise with variance
w
v
Channel

• The channel has a given signal-to-noise ratio,
  say SNR
• The above characterization encompasses, e.g.,
• AWGN channels
• Bit-rate limited channels (networks), where
  transmitted signals are passed through an
  appropriately scaled memoryless quantizer.

31
Outline

1. Quantization
2. Predictive and Noise Shaping Quantizers
3. Application to Audio Compression
4. Networked Control
5. Modelling Communication Link
6. Predictive and Noise Shaping Coding
7. Experimental Results
8. Conclusions

32

• Recall Predictive and Noise shaping quantizer

w
v
x
Noise Shaping Quantizer

• Use this idea in Source Coding
• L1 L2 become part of source coder
• L3 becomes part of the source decoder
• Make transparent to
• nominal control loop
• (i.e., Perfect Reconstruction)

33
Illustration

• Channel in the Downlink

Communication Link
q
channel
y
r
w
v
u
-
-
Noise Shaping NCS Architecture
Constraint
34
Analysis

• Hence, variance of output error due to
  quantization errors is

(a)
However, from SNR model
(b)
Now
(c)
35

• From (a), (b), (c)

36

• Expression is essentially as for the Predictive
  and Noise Shaping Quantizer Design save that now
  the Weighting Function is determined by the
  Nominal Loop Sensitivity.
• Hence can readily determine optimal values of L1,
  L2 and L3 as before!

37
Special Case (Predictive Coding)(PCM)

• Fix L2 0

Then
38
Relationship to Channel Capacity Constraints

• The theory shows that for stability when
  deploying an AWGN channel, one needs

• On the other hand, the channel capacity of an
  AWGN channel is

• Therefore, if we redesign the controller, the
  smallest channel capacity consistent with
  stability is

where pi are the unstable poles of the plant.
39
Optimal Results 1 PCM Coder in Downlink

• Optimal performance for the down-link
  architecture
• The minimum loss function is given by

• The optimal encoder satisfies

where kD is any positive (fixed) real number.
40
Optimal Results 2 Up-Link NCS Architecture

• For alternative architecture where the
  communication system is located in the up-link,
  i.e., between plant output and controller input.

d



r
y

-
Controller
Plant

n
channel
Encoder
Decoder
Communication Link
41
Optimal Coding

• Proceeding as before, we can characterize optimal
  coders via
• where is the power spectral
  density of the signal

• The corresponding optimal loss function is

42
Special Case

• Internal Model Control
• Choose C such that
• Random Walk disturbances
• Then
• i.e., no need for coding in this special case.

43
Optimal Results 3Predictive and Noise Shaping
Coder in Downlink

• The optimal noise shaping parameters are given by
• where are generalized
  Blaschke products for
  and , respectively.

44

• The corresponding optimal loss function is given
  by

45
Some Observations

1. For PCM coding, if disturbances dominate
   (r 0), then up-link and down-link architectures
   give same optimal performance.
2. For PCM coding, if GC D then optimal coder
   for up-link case is unity (i.e., no need for
   coding).
3. If approximately constant as a
   function of frequency, then (i.e., PCM
   optimal), otherwise Predictive Noise Shaping
   Coding necessary to achieve optimal performance.

46
Outline

1. Quantization
2. Predictive and Noise Shaping Quantizers
3. Application to Audio Compression
4. Networked Control
5. Modelling Communication Link
6. Predictive and Noise Shaping Coding
7. Experimental Results
8. Conclusions

47
1. Simulated Example

• We consider a continuous time plant given by
  , sampled
  ever using a zero order hold at its
  input. The corresponding discrete time transfer
  function is

We will consider two different reference
signals, r1 and r2 with PSDs given by
For the control of G(z) we choose the PI
controller
48
The Case of r1

• In this case,
• is approximately constant for all . Then, the
  PCM based scheme should have a performance which
  is close to that of the noise shaping based
  scheme.

Tracking error sample variance as a function of
the channel bit-rate (r r1).
49
The Case of r2

• In this case,
  is far from being constant. Therefore, the noise
  shaping coder system outperforms PCM.

50

• Finally, you may wonder about the simplification
  made by approximating the channel quantization
  errors (a nonlinear phenomenon) by a SNR
  constrained noise source.
• The following figure compares the theoretical
  tracking error (using the noise model
  expressions) with the practical (empirical)
  errors.

51
Comparison between theoretical variance of the
racking error as given by Theorem 1 and
empirical tracking error sample variation with r
r2 and optimal noise shaping coding.
52
2. Laboratory Results
53
Details

• Communications
• IEEE 802.3 Ethernet
• TCP/IP protocol
• Process ACT
• 6 second sampling interval word length 2 bits
  bits/second.

54
Measured response when the channel is in the
down-link measured plant output (with respect
to the operating point dotted line) and plant
input (solid line).
55
Measured response when the channel is in the
up-link measured plant output (with respect to
the operating point dotted line) and plant
input (solid line).
56

• Table for the proposed loops.

non ideal down-link non ideal up-link
without disturbance 7.2 5.5
with disturbance 194 162
As predicted by the theory In the absence of
coder/decoder- better to put channel in up-link
(i.e., controller immediately before plant).
57
Outline

1. Quantization
2. Predictive and Noise Shaping Quantizers
3. Application to Audio Compression
4. Networked Control
5. Modelling Communication Link
6. Predictive and Noise Shaping Coding
7. Experimental Results
8. Conclusions

58
Conclusions

• This lecture has focused on quantization.
• Key Tool Predictive and Noise Shaping
  Quantizers widely used in Signal Processing and
  Telecommunication, and very recently in control
  and other areas e.g. Power Electronics (State of
  the Art).
• Applications to Audio Compression and Networked
  Control.
• Recent work includes extension to multivariable
  systems and co-design of controller and
  coder/decoder pairs.
• All results in this lecture can be given
  alternative interpretation via Information Theory
  (Mutual Information, Source Coding, Channel
  Coding).

59
A Final Observation

• Note that
• Multivariable sampling (lecture 1)
• Delta operator (lecture 2),
• Asymptotic sampling zero dynamics (lecture 2),
• Predictive/Noise Shaping Quantizers (lecture 3),
• Networked Control (lecture 3)
• are all examples of a common principle-
• Dont waste limited resources describing
  (storing, transmitting, calculating.) things
  that are either (i) already known or (ii)
  predetermined by a-priori knowledge regarding the
  signal or system.

60
(No Transcript)
61
References

• Quantization
• R.M. Gray and D.L. Neuhoff, Quantization, IEEE
  Transactions on Information
• Theory, Vol.44, No.6, pp.2325-2383, 1998.
• M. Fu and L. Xie, The sector bound approach to
  quantized feedback control,
• IEEE Transactions on Automatic Control, Vol.50,
  No.11, pp.1698-1711, 2005.
• A. Gersho and R.M. Gray, Vector Quantization and
  Signal Compression,
• Boston, MAKluwer Academic, 1992.
• Predictive and Noise Shaping Quantizers
• S.R. Norsworthy, R. Schreier and G.C. Temes,
  Eds, Delta-Sigma Data
• Converters Theory, Design and Simulation.
  Piscataway, NJ IEEE Press, 1997.
• S.K. Tewksbury and R.W. Hallock, Oversampled,
  linear predictive and
• noise-shaping coders of order Ngt1, IEEE
  Transactions on Circuits and Systems,
• Vol.25, No.7, pp.436-447, 1978.
• Audio Compression
• G.C. Goodwin, D.E. Quevedo and D. McGrath,
  Moving-horizon optimal quantizer for audio
  signals, Journal Audio Engineering Society,
  Vol.51, No.3, pp.138-149, 2003.
• D.E. Quevedo and G.C. Goodwin, Multistep
  optimal analog-to-digital
• conversion, IEEE Transactions on Circuits and
  Systems I, Vol.52, No.4, pp.503-515, 2005.

62
References

• Networked Control
• D. Hristu-Varsakelis and W. Levine (Eds),
  Handbook of Networked and Embedded Systems.
  BostonBirkhäuser 2005.
• Special Issue on networked control systems,
  IEEE Transactions on Automatic Control, Vol.49,
  No.9, 2004.
• H. Ishii and B.A. Francis, Limited Data Rate in
  Control Systems with Networks, Springer, 2002.
• N. Elia and S. Mitter, Stabilization of linear
  systems with limited information, IEEE
  Transactions on Automatic Control, Vol.46, No.9,
  pp.1384-1400, 2001.
• W.S. Wong and R.W. Brockett, Systems with
  finite communication bandwidth constraints II
  Stabilization with limited information feedback,
  IEEE Transactions on Automatic Control, Vol.44,
  No.5, pp.1049-1053, 1999.
• G. Nair and R. Evans, Stabilizability of
  stochastic linear systems with finite feedback
  data rates, SIAM Journal on Control and
  Optimization, Vol.43, No.2, pp.413-436, 2004.
• S. Tatikonda and S. Mitter, Control under
  communication constraints, IEEE Transactions on
  Automatic Control, Vol.49, No.7, pp.1056-1068,
  2004.

63
References

• Modelling Communication Links
• J.H. Braslavsky, R.H. Middleton and J.S.
  Freudenberg, Feedback stabilization over
  signal-to-noise ratio constrained channels, in
  Proceedings of the 2004 American Control
  Conference, Boston, USA, July 2004.
• D. Tse and P. Viswanath, Fundamentals of
  Wireless Communication, Cambridge University
  Press, 2005.
• Predictive and Noise Shaping Coding
• G.C. Goodwin, D.E. Quevedo and E.I. Silva,
  Architectures and coder design for networked
  control systems, to appear Automatica, 2007.
• E.I. Silva, G.C. Goodwin, D.E. Quevedo and M.S.
  Derpich, Optimal noise shaping for networked
  control systems, European Control Conference,
  Kos, Greece 2-5 July 2007.

64
Lecture 3Quantization in Signals and Systems

• by
• Graham C. Goodwin
• University of Newcastle
• Australia

Presented at the Zaborszky Distinguished Lecture
Series December 3rd, 4th and 5th, 2007