Audio and Speech Processing Topic3 Noise Reduction

1
Audio and Speech ProcessingTopic-3 Noise
Reduction

• Marc Moonen/Ann Spriet
• Dept. E.E./ESAT, K.U.Leuven
• marc.moonen_at_esat.kuleuven.be
• homes.esat.kuleuven.be/moonen/

2
Overview

• Spectral subtraction for single-micr. noise
  reduction
• Single-microphone noise reduction problem
• Spectral subtraction basics (spectral filtering)
• Features gain functions, implementation, musical
  noise,
• Iterative Wiener filter based on speech modeling
• Multi-channel Wiener filter for multi-micr. noise
  red.
• Multi-microphone noise reduction problem
• Multi-channel Wiener filter (spectralspatial
  filtering)
• Kalman filter based on speech noise modeling
• Kalman filters
• Kalman filters for noise reduction

3
Single-Microphone Noise Reduction Problem

• Microphone signal is
• Goal Estimate sk based on yk
• Applications
• Speech enhancement in conferencing, handsfree
  telephony, hearing aids,
• Digital audio restoration
• Will consider speech applications sk speech
  signal

desired signal contribution
noise contribution
4
Spectral Subtraction Methods Basics

• Signal chopped into frames (e.g. 10..20msec),
  for each frame a frequency domain representation
  is
• 
  (i-th frame)
• However, speech signal is an on/off signal, hence
  some frames have speech noise, i.e.
• some frames have noise only, i.e.
• A speech detection algorithm is needed to
  distinguish between these 2 types of frames
  (based on energy/dynamic range/statistical
  properties,)

5
Spectral Subtraction Methods Basics

• Definition ?(?) average amplitude of noise
  spectrum
• Assumption noise characteristics change slowly,
  hence estimate ?(?) by (long-time) averaging over
  (M) noise-only frames
• Estimate clean speech spectrum Si(?) (for each
  frame), using corrupted speech spectrum Yi(?)
  (for each frame, i.e. short-time estimate)
  estimated ?(?)
• based on gain function

6
Spectral Subtraction Gain Functions
Magnitude Subtraction
Spectral Subtraction
Wiener Estimation
Maximum Likelihood
Non-linear Estimation
Ephraim-Malah Suppr. Rule most frequently
used in practice
see next slide
7
Spectral Subtraction Gain Functions

• Ephraim-Malah Suppression Rule (EMSR)
• with

ignore formulas
modified Bessel functions
previous frame
8
Spectral Subtraction Gain Functions

• Example 1 Magnitude Subtraction
• Signal model
• Estimation of clean speech spectrum
• PS half-wave rectification

9
Spectral Subtraction Gain Functions

• Example 2 Wiener Estimation
• Goal find linear filter Gi(?) such that MSE
• 
  is minimized
• Solution
• Assume speech sk and noise nk are
  uncorrelated, then...
• PS half-wave rectification

10
Spectral Subtraction Implementation
Yn,i
yk

• Short-time Fourier Transform (uniform
  DFT-modulated analysis filter bank)
• 
  estimate for Y(?n ) at time i
  (i-th frame)
• Nnumber of frequency bins
  (channels) n0..N-1
• Mdownsampling factor
• Kframe length
  hk length-K analysis window (prototype
  filter)
• frames with 50...66 overlap (i.e. 2-, 3-fold
  oversampling, N2M..3M)
• subband processing
• synthesis bank matched to analysis bank (see
  DSP-II)

Short-time analysis
Short-time synthesis
Gain functions
11
Spectral Subtraction Musical Noise

• Audio demo car noise
• Artifact musical noise
• What?
• Short-time estimates of Yi(?) fluctuate
  randomly in noise-
• only frames, resulting in random gains Gi(?)
• statistical analysis shows that broadband noise
  is transformed into signal composed of
  short-lived tones with randomly distributed
  frequencies (musical noise)

12
Spectral Subtraction Musical Noise

• Solutions?
• Magnitude averaging replace Yi(?) in calculation
  of Gi(?) by a local average over frames
• EMSR (p7)
• augment Gi(?) with soft-decision VAD
• Gi(?) ? P(H1 Yi(?)). Gi(?)
  

13
Spectral Subtraction Iterative Wiener Filter

• Basic
• Wiener filtering based spectral
  subtraction (p9), with (improved) spectra
  estimation based on parametric models
• Procedure
• Estimate parameters of a speech model from noisy
  signal yk
• Using estimated speech parameters, perform noise
  reduction (e.g. Wiener estimation, p. 9)
• Re-estimate parameters of speech model from the
  speech signal estimate
• Iterate 2 3

14
Spectral Subtraction Iterative Wiener Filter
pulse train


all-pole filter
voiced
pitch period
speech signal
uk
x
white noise generator
unvoiced
frequency domain time
domain
linear prediction parameters
15
Spectral Subtraction Iterative Wiener Filter

• For each frame (vector) ym (iiteration
  nr.)
• Estimate and
• Construct Wiener Filter (p.9)
• with
• estimated during noise-only periods
• 3. Filter speech frame ym

repeat until some error criterion is satisfied
16
Overview

• Spectral subtraction for single-micr. noise
  reduction
• Single-microphone noise reduction problem
• Spectral subtraction basics (spectral filtering)
• Features gain functions, implementation, musical
  noise,
• Iterative Wiener filter based on speech modeling
• Multi-channel Wiener filter for multi-micr. noise
  red.
• Multi-microphone noise reduction problem
• Multi-channel Wiener filter (spectralspatial
  filtering)
• Kalman filter based on speech noise modeling
• Kalman filters
• Kalman filters for noise reduction

17
Multi-Microphone Noise Reduction Problem
speech source
?
(some) speech estimate
microphone signals
noise source(s)
speech part
noise part
18
Multi-Microphone Noise Reduction Problem
Will estimate speech part in microphone 1 ()
()
?
() Estimating sk is more difficult, would
include dereverberation (topic 6), etc..
() This is similar to single-microphone model
(p.3), where additional microphones
(m2..M) help to get a better estimate
19
Multi-Microphone Noise Reduction Problem

• Data model
• See Topic-2 on multi-path propagation, with q
  left out for conciseness.
• Hm(?) is complete transfer function from
  speech source position to m-the microphone

20
Multi-Channel Wiener Filter (MWF)

• Data model
• Will use linear filters to obtain speech estimate
  (as in Topic-2)
• Wiener filter (MMSE approach)
• Note that (unlike in DSP-II) desired
  response signal S1(w) is unknown here (!), hence
  solution will be unusual

21
Multi-Channel Wiener Filter (MWF)

• Wiener filter solution is (see DSP-II)
• All quantities can be computed !
• Special case of this is single-channel Wiener
  filter formulae on p.9 !

22
Multi-Channel Wiener Filter (MWF)

• MWF combines spatial filtering (as in Topic-2)
  with single-channel spectral filtering (as in
  Topic-3/single-channel noise reduction)
• if
• then

23
Multi-Channel Wiener Filter (MWF)

• then it can be shown that
• represents a
  spatial filtering ()
• Compare to formulae for superdirective
  delay-and-sum beamf. (Topic-2)
• Delay-and-sum beamf. maximizes array gain in
  white noise field
• Superdirective beamf. maximizes array gain in
  diffuse noise field
• MWF maximizes array gain in unknown (!) noise
  field.
• MWF is operated without invoking any prior
  knowledge (steering
• vector/noise field) ! (the secret is in the
  voice activity detection (explain))
• () Note that spatial filtering can
  improve SNR, spectral filtering never improves
  SNR
• (at one frequency)

24
Multi-Channel Wiener Filter (MWF)

• then it can be shown that (continued)
• represents an additional
  spectral post-filter
• i.e. single-channel Wiener estimate
  (p.9), applied to output signal
• of spatial filter
• 
  (prove
  it!)

25
Multi-Channel Wiener Filter Implementation

• Implementation with short-time Fourier
  transform see p.10
• Implementation with time-domain linear
  filtering

26
Multi-Channel Wiener Filter Implementation

• Solution is

• Implementation with time-domain linear
  filtering

27
Multi-Channel Wiener Filter Implementation

• Implementation with time-domain linear
  filtering
• Block algorithm
• For each block
• -Apply voice activity detection
• -Update correlation matrices
• -Recompute filter coefficients
• -Apply filters
• Cheaper algorithms with stochastic gradients
• Frequency domain algorithms
• Details omitted here (see literature)

28
Speech Distortion Weighted MWF

• SDW-MWF is MWF with additional tuning parameter
• Design criterion for can be re-written as
• i.e. speech distortionresidual noise is
  minimized

29
Speech Distortion Weighted MWF

• Design criterion may now be modified to trade-off
  noise reduction against speech distortion
• Then optimal solution is
• i.e. (rather) straightforward
  modification
• By increasing mu, more noise is reduced, at the
  expense of more speech distortion (which is
  acceptable to a certain level) .
• means all emphasis is on noise
  reduction, speech distortion is ignored ( and
  then ! )

30
Overview

• Spectral subtraction for single-micr. noise
  reduction
• Single-microphone noise reduction problem
• Spectral subtraction basics (spectral filtering)
• Features gain functions, implementation, musical
  noise,
• Iterative Wiener filter based on speech modeling
• Multi-channel Wiener filter for multi-micr. noise
  red.
• Multi-microphone noise reduction problem
• Multi-channel Wiener filter (spectralspatial
  filtering)
• Kalman filter based on speech noise modeling
• Kalman filters
• Kalman filters for noise reduction

31
Kalman Filter

• Given state space model of a discrete-time MIMO
  system
• with vk and wk mutually uncorrelated,
  white noises
• Then
• given A, B, C, D and input/output-observatio
  ns uk,yk, k1,2,... Kalman filter produces
  MMSE estimates of internal states xk, k1,2,...
  (Wiener filter for
  dynamic systems)

32
Kalman Filter

• Definition MMSE-estimate of xk
  using all
• available data
  up until time l
• FILTERING estimate
• PREDICTION estimate
• SMOOTHING estimate

33
Kalman Filter filtering and 1-step prediction

• Given together with error
  covariance matrix Pkk-1
• Then obtain and
  using uk, yk
• Step 1 Measurement Update
• Step 2 Time Update

Kalman Gain
34
Kalman Filter smoothing

• Estimate states x1, x2,, xN
• based on data uk, yk, k 1, 2, N
• How?
• 1. forward run apply previous equations for k
  1, 2, N
• Result estimates
• 2. backward run apply following equations for k
  N, N -1, 1
• Result (better) estimates
  

35
Kalman filter for Speech Enhancement

• Assume AR model of speech and noise
• Equivalent state-space model is
• 
  ymicrophone signal

uk, wk zero mean, unit variance,white noise
36
Kalman filter for Speech Enhancement

• with

37
Kalman filter for Speech Enhancement

• PS This was single-microphone case.
• How can this be extended to
  multi-microphone case ?
• Same A, x, v
• C?

38
Kalman filter for Speech Enhancement
Iterative algorithm
iterations
yk
split signal in frames
estimate parameters
Kalman Smoother or Kalman Filter
reconstruct signal

• Disadvantages iterative approach
• complexity
• delay

39
Kalman filter for Speech Enhancement

• iteration index time index (no
  iterations)

Sequential algorithm
D
State Estimator Kalman Filter
Parameters Estimator (Kalman Filter)
D
40
CONCLUSIONS

• Single-channel noise reduction
• Basic system is spectral subtraction
• Only spectral filtering, not easily extended to
  multi-channel case for additional spatial
  filtering
• Hence can only exploit differences in spectra
  between noise and speech signal
• noise reduction at expense of speech distortion
• achievable noise reduction may be limited
• Multi-channel noise reduction
• Basic system is MWF, possibly extended with
  speech distortion regularization
• Provides spectral spatial filtering (links
  with beamforming!)
• Kalman filtering
• alternative approach (though not easily applied
  in practice)