Goals of Adaptive Signal Processing

1
Goals of Adaptive Signal Processing

• Design algorithms that learn from training data
• Algorithms must have good properties attain good
  solutions, simple to implement, converge quickly

2
Learning from Examples

• Systems or filters (tap delay line) consist of
  parameters or weights that are updated
• Learning algorithm receives training examples
  (observation data) and updates parameters of
  system
• Updates depend on specified criterion
• Learning can be batch or on-line

3
Tap Delay Line
x(n)
x(n-1)
x(n-2)
D
D
w0
w1
w2
X
X
X
S
y(n)
4
Tap Delay Line Parameters

• Inputs x(n)
• Delay units (two)
• Weights w ? R3
• Output y(n) w0 x(n) w1 x(n-1) w2 x(n-2)

5
Types of Learning Algorithms

• Supervised learning (learning with a teacher)
• learning algorithm receives inputs and desired
  outputs
• Unsupervised learning (no teacher)
• learning algorithm receives inputs
• Reinforcement learning (learning with a critic)
• learning algorithm receives inputs and an
  evaluation cost or penalty, possibly delayed

6
Supervised Learning
x(n)
x(n-1)
x(n-2)
D
D
w0
w1
w2
X
X
X
e(n)
S
y(n)
S
-
d(n)

7
Supervised Learning Parameters

• Inputs x(n)
• Outputs y(n)
• Weights w
• Desired Output d(n)
• Error signal e(n) d(n) y(n)

8
Adaptive Learning System (two phases)

• Training or learning phase (equivalent to write
  phase in conventional computer memory) weights
  are adjusted to meet certain desired criterion.
• Recall or test phase (equivalent to read phase in
  conventional computer memory) weights are fixed
  as system realizes some task.

9
How are weights updated?

• Iterative on-line algorithm weights of system
  are adjusted on-line as training data is
  received.
• w(k1) L(w(k),x(k),d(k)) for supervised
  learning where
• d(k) is desired output.
• Cost criterion common cost criterion
• Mean Squared Error for one output J(w) ?
  (y(k) d(k)) 2
• Goal is to find minimum J(w) over all possible w.
  We will consider stochastic gradient based and
  least squares methods.

10
Comments

• Most filters are linear time invariant filters,
  but when modifying weights they become nonlinear
  time varying systems
• Focus on supervised on-line iterative learning
  algorithms with squared error cost functions

11
Adaptive Signal Processing Problems

• System identification approximate unknown system
• Inverse modeling find a model for inverse system
  for unknown noisy plant, equalization
• Prediction predict value of a noisy random
  signal, time series (financial, biological)
• Interference cancellation cancel unknown
  interference, noise cancellation and adaptive
  beamforming

12
Applications

• Adaptive equalization remove intersymbol
  interference, (data transmission)
• Speech coding (linear predictive coding), speech
  analysis and synthesis
• Spectrum analysis estimate spectrum of signal in
  noise based on time series data
• Adaptive noise cancellation adaptive echo
  cancellers
• Adaptive beamforming signals arranged spatially,
  adaptive antenna arrays, cancel sidelobes