Lecture 1 Sampling of Signals

1
Lecture 1Sampling of Signals

• by
• Graham C. Goodwin
• University of Newcastle
• Australia

Lecture 1 Presented at the Zaborszky
Distinguished Lecture Series December 3rd, 4th
and 5th, 2007
2
Recall Basic Idea of Samplingand Quantization
Quantization
t
t1
t3
t2
0
t4
Sampling
3

• In this lecture we will ignore quantization
  issues and focus on the impact of different
  sampling patterns for scalar and multidimensional
  signals

4
Outline

• One Dimensional Sampling
• Multidimensional Sampling
• Sampling and Reciprocal Lattices
• Undersampled Signals
• Filter Banks
• Generalized Sampling Expansion (GSE)
• Recurrent Sampling
• Application Video Compression at Source
• Conclusions

5

• Sampling Assume amplitude quantization
  sufficiently fine to be negligible.
• Question Say we are given
• Under what conditions can we recover
  
• from the samples?

6
A Well Known Result (Shannons Reconstruction
Theorem for Uniform Sampling)

• Consider a scalar signal f(t) consisting of
  frequency components in the range .
  If this signal is sampled at period ,
  then the signal can be perfectly reconstructed
  from the samples using

7

• Proof Sampling produces folding

Low pass filter recovers original spectrum
Hence
or
8
A Simple (but surprising) Extension
Recurrent Sampling

• where

is a periodic sequence of integers i.e.,
Let
Note that the average sampling period is
e.g.
average 5
9
x
x
x
x
x
x
0
9
-1
10
19
20

• Non-uniform

x
x
x
x
x
0
5
10
15
20
Uniform
10

• Claim
• Provided the signal is bandlimited to
  where , then the signal can be
  perfectly reconstructed from the periodic
  sampling pattern.
• where average sampling period
• Proof
• We will defer the proof to later when we will
  use it as an illustration of Generalized Sampling
  Expansion (GSE) Theorem.

11
Outline

• One Dimensional Sampling
• Multidimensional Sampling
• Sampling and Reciprocal Lattices
• Undersampled Signals
• Filter Banks
• Generalized Sampling Expansion (GSE)
• Recurrent Sampling
• Application Video Compression at Source
• Conclusions

12
Multidimensional Signals

• Digital Photography

x1
x2
Digital Video
x2
x1
x3 (time)
13
Outline

• One Dimensional Sampling
• Multidimensional Sampling
• Sampling and Reciprocal Lattices
• Undersampled Signals
• Filter Banks
• Generalized Sampling Expansion (GSE)
• Recurrent Sampling
• Application Video Compression at Source
• Conclusions

14

• How should we define sampling for
  multi-dimensional signals?
• Utilize idea of Sampling Lattice
• Sampling Lattice

15

• Also, need multivariable frequency domain
• concepts.
• These are captured by two ideas
• Reciprocal Lattice
• Unit Cell

16
Reciprocal Lattice

• Unit Cell (Non-unique)

17
One Dimensional Example
x
x
x
x
x
0
-20
10
20
-10

• Sampling Lattice

18
Reciprocal Lattice and Unit Cell
Unit Cell
0
19
Multidimensional Example
x2
5 4 3 2 1
x1
1 2 3 4 5
-4 -3 -2 -1
-1 -2 -3 -4
20
Reciprocal Lattice and Unit Cell for Example
1/2 1/4
1/4 1/2 3/4 1
-1/4 -1/2 -3/4 -1
21
Outline

• One Dimensional Sampling
• Multidimensional Sampling
• Sampling and Reciprocal Lattices
• Undersampled Signals
• Filter Banks
• Generalized Sampling Expansion (GSE)
• Recurrent Sampling
• Application Video Compression at Source
• Conclusions

22

• We will be interested here in the situation
  where the Sampling Lattice is not a Nyquist
  Lattice for the signal (i.e., the signal cannot
  be perfectly reconstructed from the original
  pattern!)

Strategy We will generate other samples by
filtering or shifting operations on the
original pattern.
23

• Consider a bandlimited signal .
• Assume the D-dimension Fourier transform has
  finite support, S.
• Then for given D-dimensional lattice T, there
  always exists a finite set ,
  such that support

Heuristically The idea of Tiling the area of
interest in the frequency domain
24
One Dimensional Example

• Our one dimensional example continued.
• Sampling Lattice

Unit Cell
0
Bandlimited spectrum
Use
Support
25
Outline

• One Dimensional Sampling
• Multidimensional Sampling
• Sampling and Reciprocal Lattices
• Undersampled Signals
• Filter Banks
• Generalized Sampling Expansion (GSE)
• Recurrent Sampling
• Application Video Compression at Source
• Conclusions

26
Generation of Extra Samples

• Suppose now we generate a data set
• as shown in below

Q Channel Filter Bank
27
Outline

• One Dimensional Sampling
• Multidimensional Sampling
• Sampling and Reciprocal Lattices
• Undersampled Signals
• Filter Banks
• Generalized Sampling Expansion (GSE)
• Recurrent Sampling
• Application Video Compression at Source
• Conclusions

28

• Define

Let
be the solution (if it exists) of
for
29
Conditions for Perfect Reconstruction
GSE Theorem

• can be reconstructed from
• if and only if has full row rank for
  all in the Unit Cell
• where

30

• Proof
• Multiply both sides by where
  (the
• Reciprocal Lattice). Then sum over q
• Note that
  tiles the entire
• support S
• Thus,

from the Matrix identity that defines
31

• where we have used the fact that
• Since is the output of f(x) passing
  through ,
• then
• Hence, we finally have

32
Outline

• One Dimensional Sampling
• Multidimensional Sampling
• Sampling and Reciprocal Lattices
• Undersampled Signals
• Filter Banks
• Generalized Sampling Expansion (GSE)
• Recurrent Sampling
• Application Video Compression at Source
• Conclusions

33
Special Case Recurrent Sampling

• (where is implemented by a spatial shift
  )
• This amounts to the sampling pattern
• where w.l.o.g.
• Now, given the samples , our goal
  is to perfectly reconstruct

34

• Here , and
• Thus
• To apply the GSE Theorem we require

Nonsingular
35
Something to think about

• The GSE result depends on inversion of a
  particular matrix, H(w). Of course we have
  assumed here perfect representation of all
  coefficients. An interesting question is what
  happens when the representation is imperfect i.e.
  coefficients are represented with finite
  wordlength (i.e. they are quantized)
• We will not address this here but it is something
  to keep in mind.

36
Return to our one-dimensional example

• Recall that we had
• so that
• support
• Say we use recurrent sampling with

37
x
x
x
0
10
20
x
x
x
0
19
9
-1
x
x
x
x
x
x
9 10
19 20
0
-1
38
Condition for Perfect Reconstruction is

• nonsingular

Hence, the original signal can be recovered from
the sampling pattern given in the previous slide.
39
Summary

• We have seen that the well known Shannon
  reconstruction theorem can be extended in several
  directions e.g.
• Multidimensional signals
• Sampling on a lattice
• Recurrent sampling
• Given specific frequency domain distributions,
  these can be matched to appropriate sampling
  patterns.

40
Outline

• One Dimensional Sampling
• Multidimensional Sampling
• Sampling and Reciprocal Lattices
• Undersampled Signals
• Filter Banks
• Generalized Sampling Expansion (GSE)
• Recurrent Sampling
• Application Video Compression at Source
• Conclusions

41
Application Video Compression Source

• Introduction to video cameras
• Instead of tape, digital cameras use 2D sensor
  array (CCD or CMOS)

42
Image Sensor

• A 2D array of sensors replaces the traditional
  tape
• Each sensor records a 'point' of the continuous
  image
• The whole array records the continuous image at a
  particular time instant

43
2D Colours Sensor Array
Data transfer from array is sequential and has a
maximal rate of Q.
Based on http//www.dpreview.com/learn/
44
Current Technology

• Uniform 3D sampling
• a sequence of identical frames equally spaced in
  time

45
Video Bandwidth
depends on the frame rate
depends on spatial resolution of the frames
The volume of box depends on the capacity
pixel rate (frame rate) x (spatial resolution)
46
Hard Constraints

• Data recording on sensor

• Sensor array density
• - for spatial resolution

• Sensor exposure time
• - for frame rate

2. Data reading from sensor

• Data readout time
• - for pixel rate

47
BUT...
Generally Q ltlt RF Need R1lt R F1 lt
F s.t. R1F1 Q

• Compromise
• spatial resolution R1lt R
• temporal resolution F1 lt F

48
Actual Capacity (Data Readout)
49
Observation
50
The Spectrum of this Video Clip
uniform sampling - no compromise
uniform sampling - compromise in frame rate
uniform sampling - compromise in spatial
resolution
51
New Idea

• Idea is to deviate from constant resolutions in a
  recorded video clip. This means that sampling
  patterns within the video clip will not be
  uniform.
• Specifically, the idea is to have, within the
  recorded video clip, a combination of fast frames
  with low spatial resolution and slow frames with
  high spatial resolution.

52
Recurrent Non-Uniform Sampling
frame type A
frame type B
53
What Does it Buy?
54
Schematic Implementation
non-uniform data from the sensor
uniform high def. video
'compression at the source'
55
Recurrent Non-Uniform Sampling

• A special case of
• Generalized Sampling Expansion Theorem

56
Sampling Pattern
The resulting sampling pattern is given by
57
Frequency Domain
where
is the unit cell of the reciprocal lattice
58
Reciprocal Lattice
?t
?x
59
Apply the GSE Theorem
where is uniquely defined by H1H2()
? is a set of 2(LM)1 constraints
If exists, we can find the reconstruction
function
60
Reconstruction Scheme
?
I(x,t)
Î(x,t)
?2L1
H2L1
The sub-sampled frequency of each filter H is
61
Reconstruction functions
for r 2,3,,2L1
for r 2(L1),,2(LM)1
Multidimensional sinc like functions
62
Demo
Full resolution sequence
Reconstructed sequence
Temporal decimation
Spacial decimation
63
Outline

• One Dimensional Sampling
• Multidimensional Sampling
• Sampling and Reciprocal Lattices
• Undersampled Signals
• Filter Banks
• Generalized Sampling Expansion (GSE)
• Recurrent Sampling
• Application Video Compression at Source
• Conclusions

64
Conclusions

• Nonuniform sampling of scalar signals
• Nonuniform sampling of multidimensional signals
• Generalized sampling expansion
• Application to video compression
• A remaining problem is that of joint design of
  sampling schemes and quantization strategies to
  minimize error for a given bit rate

65
References

• One Dimensional Sampling
• A. Feuer and G.C. Goodwin, Sampling in Digital
  Signal Processing and Control. Birkhäuser, 1996.
• R.J. Marks II, Ed., Advanced Topics in Shannon
  Sampling and Interpolation Theory. New Your
  Springer-Verlag, 1993.
• Multidimensional Sampling
• W.K. Pratt, Digital Image Processing, 3rd ed
  John Wiley Sons, 2001.
• B.L. Evans, Designing commutative cascades of
  multidimensional upsamplers and downsamplers,
  IEEE Signal Process Letters, Vol4, No.11,
  pp.313-316, 1997.
• Sampling and Reciprocal Lattices, Undersampled
  Signals
• A.Feuer, G.C. Goodwin, Reconstruction of
  Multidimensional Bandlimited Signals for Uniform
  and Generalized Samples, IEEE Transactions on
  Signal Processing, Vol.53, No.11, 2005.
• A.K. Jain, Fundamentals of Digital Image
  Processing, Englewood Cliffs, NJ Prentice-Hall,
  1989.

66
References

• Filter Banks
• Y.C. Eldar and A.V. Oppenheim, Filterbank
  reconstruction of bandlimited signals from
  nonuniform and generalized samples, IEEE
  Transactions on Signal Processing, Vol.48, No.10,
  pp.2864-2875, 2000.
• P.P. Vaidyanathan, Multirate Systems and Filter
  Banks. Englewood Cliffs, NJ Prentice-Hall, 1993.
• H. Bölceskei, F. Hlawatsch and H.G. Feichtinger,
  Frame-theoretic analysis of oversampled filter
  banks, IEEE Transactions on Signal Processing,
  Vol.46, No.12, pp.3256-3268, 1998.
• M. Vetterli and J. Kovacevic, Wavelets and
  Subband Coding, Englewood Cliffs, NJ Prentice
  Hall, 1995.

67
References

• Generalized Sampling Expansions, Recurrent
  Sampling
• A. Papoulis, Generalized sampling expansion,
  IEEE Transaction on Circuits and Systems,
  Vol.CAS-24, No.11, pp.652-654, 1977.
• A. Feuer, On the necessity of Papoulis result
  for multidimensional (GSE), IEEE Signal
  Processing Letters, Vol.11, No.4, pp.420-422,
  2004.
• K.F.Cheung, A multidimensional extension of
  Papoulis generalized sampling expansion with
  application in minimum density sampling, in
  Advanced Topics in Shannon Sampling and
  Interpolation Theory, R.J. Marks II. Ed., New
  York Springer-Verlag, pp.86-119, 1993.
• Video Compression at Source
• E. Shechtman, Y. Caspi and M. Irani, Increasing
  space-time resolution in video, European
  Conference on Computer Vision (ECCV), 2002.
• N. Maor, A. Feuer and G.C. Goodwin, Compression
  at the source of digital video, To appear
  EURASIP Journal on Applied Signal Processing.

68
Lecture 1Sampling of Signals

• by
• Graham C. Goodwin
• University of Newcastle
• Australia

Lecture 1 Presented at the Zaborszky
Distinguished Lecture Series December 3rd, 4th
and 5th, 2007