Adaptive Filters – Wiener Filter
Gerhard Schmidt
Christian-Albrechts-Universität zu Kiel Faculty of Engineering Institute of Electrical and Information Engineering Digital Signal Processing and System Theory
Contents of the Lecture
•Today Contents of the Lecture: Introduction and motivation Principle
of orthogonality Time-domain solution Frequency-domain solution Application example: noise suppression
Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter
Slide 2
Basics
•History and Assumptions Filter design by means of minimizing the squared error (according to Gauß) Independent development
1941: A. Kolmogoroff: Interpolation und Extrapolation von stationären zufälligen Folgen, Izv. Akad. Nauk SSSR Ser. Mat. 5, pp. 3 – 14, 1941 (in Russian)
1942: N. Wiener: The Extrapolation, Interpolation, and Smoothing of Stationary Time Series with Engineering Applications, J. Wiley, New York, USA, 1949 (originally published in 1942 as MIT Radiation Laboratory Report)
Assumptions / design criteria: Design of a filter that separates a desired signal optimally from additive noise
Both signals are described as stationary random processes Knowledge
about the statistical properties up to second order is necessary
Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter
Slide 3
Application Examples – Part 1
•Noise Suppression Application example:
Wiener filter
Speech Noise
(No echo components)
Model: Speech (desired signal)
+ Noise (undesired signal) The Wiener solution if often applied in a “block-based fashion”. Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter
Slide 4
Application Examples – Part 2
•Echo Cancellation Application example:
Model: Echo cancellation filter
+
+
+
+ Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter
The echo cancellation filter has to converge in an iterative manner (new = old + correction) towards the Wiener solution. Slide 5
Generic Structure
•Noise Reduction and System Identification Wiener filter
Error signal
Wiener filter
+
+
+
Linear system Generation of a desired signal Echo cancellation
Noise suppression
+
+
Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter
+
+
Slide 6
Literature Hints
•Books Main text: E. Hänsler / G. Schmidt: Acoustic Echo
and Noise Control – Chapter 5 (Wiener Filter), Wiley, 2004
Additional texts: E. Hänsler: Statistische Signale: Grundlagen
und Anwendungen – Chapter 8 (Optimalfilter nach Wiener und Kolmogoroff), Springer, 2001 (in German) M. S.Hayes: Statistical Digital Signal Processing and Modeling – Chapter 7 (Wiener Filtering), Wiley, 1996 S. Haykin: Adaptive Filter Theory – Chapter 2 (Wiener Filters), Prentice Hall, 2002
Noise suppression: U. Heute: Noise Suppression,
in E. Hänsler, G. Schmidt (eds.), Topics in Acoustic Echo and Noise Control, Springer, 2006
Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter
Slide 7
Principle of Orthogonality
•Derivation Derivation during the lecture …
Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter
Slide 8
Principle of Orthogonality
•A Deterministic Example Derivation during the lecture …
Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter
Slide 9
Wiener Solution
•Time-Domain Solution Derivation during the lecture …
Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter
Slide 10
Time-Domain Solution
•Example – Part 1 Desired signal: Sine wave with known frequency but with unknown phase, not correlated with noise
FIR filter of order 31, delayless estimation at filter output
+
Noise:
White noise with zero mean, not correlated with desired signal
Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter
Slide 11
Time-Domain Solution
•Example – Part 2 Wiener solution:
Desired signal and noise are not correlated and have zero mean:
Simplification according to the assumptions above:
Wiener solution (modified):
Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter
Slide 12
Time-Domain Solution
•Example – Part 3 Input signals: Excitation: sine wave Noise: white noise
Assumptions: Knowledge
of the mean values and of the autocorrelation functions of the desired and of the undesired signal Desired signal and noise are not correlated Desired signal and noise have zero mean 32 FIR coefficients should be used by the filter
Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter
Slide 13
Time-Domain Solution
•Example – Part 4 After a short initialization time the noise suppression
performs well (and does not introduce a delay!)
Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter
Slide 14
Error Surface
•Derivation – Part 1 Derivation during the lecture …
Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter
Slide 15
Error Surface
•Derivation – Part 2 Error surface for:
Properties: Unique minimum (no local minima) Error
surface depends on the correlation properties of the input signal
Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter
Slide 16
Frequency-Domain Solution
•Derivation Derivation during the lecture …
Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter
Slide 17
Applications
•Noise Suppression – Part 1 Frequency-domain Wiener solution (non-causal):
Desired signal = speech signal:
Desired signal and noise are orthogonal:
Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter
Slide 18
Applications
•Noise Suppression – Part 2 Frequency-domain solution:
Approximation using short-term estimations:
Practical approaches: Realization using a filterbank system (time-variant attenuation of subband signals)
Analysis filters with length of about 15 to 100 ms Frame-based processing with frame shifts between
1 and 20 ms The basic Wiener characteristic is usually „enriched“ with several extensions (overestimation, limitation of the attenuation, etc.)
Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter
Slide 19
Applications
•Noise Suppression – Part 3 Processing structure: Analysis filterbank
Synthesis filterbank
Input PSD estimation
Filter characteristic Noise PSD estimation
PSD = power spectral density Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter
Slide 20
Applications
•Noise Suppression – Part 4 Power spectral density estimation for the input signal:
Power spectral density estimation for the noise:
Estimation schemes using voice activity detection(VAD)
Tracking of minima of short-term power estimations
Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter
Slide 21
Applications
•Noise Suppression – Part 5 Schemes with voice activity detection:
Tracking of minima of the short-term power:
Constant slightly larger than 1 Bias correction
Constant slightly smaller than1
Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter
Slide 22
Applications
•Noise Suppression – Part 6 Problem: The short-term
power of the input signal usually fluctuates faster than the noise estimate – also during speech pauses. As a result the filter characteristic opens and closes in a randomized manner, with results in tonal residual noise (so-called musical noise).
Simple solution: By inserting a fixed overestimation
the randomized opening of the filter can be avoided. This comes, however, with a more aggressive attenuation characteristic that attenuates also parts of the speech signal.
Enhanced solutions: More
enhanced solutions will be presented in the lecture “Speech and Audio Processing – Audio Effects and Recognition” (offered next term by the “Digital Signal Processing and System Theory” team).
Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter
Slide 23
Applications
•Noise Suppression – Part 7 : Microphone signal : Output without overestimation : Output with 12 dB overestimation
Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter
Slide 24
Applications
•Noise Suppression – Part 8 Limiting the maximum attenuation: For
several application the original shape of the noise should be preserved (the noise should only be attenuated but not completely removed). This can be achieved by inserting a maximum attenuation:
In addition, this attenuation limits can be varied slowly over
time (slightly more attenuation during speech pauses, less
attenuation during speech activity).
Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter
Slide 25
Applications
•Noise Suppression – Part 9 : Microphone signal : Output without attenuation limit : Output with attenuation limit
Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter
Slide 26
Applications
•Noise Suppression – Part 10
Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter
Slide 27
Adaptive Filters – Wiener Filter
•Summary and Outlook This week: Introduction and motivation Principle of orthogonality Time-domain solution Frequency-domain solution Application example: noise suppression
Next week: Linear Prediction
Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter
Slide 28