Gerhard Schmidt

Christian-Albrechts-Universität zu Kiel Faculty of Engineering Institute of Electrical and Information Engineering Digital Signal Processing and System Theory

Contents of the Lecture

•Today Contents of the Lecture: Introduction and motivation Principle

of orthogonality Time-domain solution Frequency-domain solution Application example: noise suppression

Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter

Slide 2

Basics

•History and Assumptions Filter design by means of minimizing the squared error (according to Gauß) Independent development

1941: A. Kolmogoroff: Interpolation und Extrapolation von stationären zufälligen Folgen, Izv. Akad. Nauk SSSR Ser. Mat. 5, pp. 3 – 14, 1941 (in Russian)

1942: N. Wiener: The Extrapolation, Interpolation, and Smoothing of Stationary Time Series with Engineering Applications, J. Wiley, New York, USA, 1949 (originally published in 1942 as MIT Radiation Laboratory Report)

Assumptions / design criteria: Design of a filter that separates a desired signal optimally from additive noise

Both signals are described as stationary random processes Knowledge

about the statistical properties up to second order is necessary

Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter

Slide 3

Application Examples – Part 1

•Noise Suppression Application example:

Wiener filter

Speech Noise

(No echo components)

Model: Speech (desired signal)

+ Noise (undesired signal) The Wiener solution if often applied in a “block-based fashion”. Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter

Slide 4

Application Examples – Part 2

•Echo Cancellation Application example:

Model: Echo cancellation filter

+

+

+

+ Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter

The echo cancellation filter has to converge in an iterative manner (new = old + correction) towards the Wiener solution. Slide 5

Generic Structure

•Noise Reduction and System Identification Wiener filter

Error signal

Wiener filter

+

+

+

Linear system Generation of a desired signal Echo cancellation

Noise suppression

+

+

Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter

+

+

Slide 6

Literature Hints

•Books Main text: E. Hänsler / G. Schmidt: Acoustic Echo

and Noise Control – Chapter 5 (Wiener Filter), Wiley, 2004

Additional texts: E. Hänsler: Statistische Signale: Grundlagen

und Anwendungen – Chapter 8 (Optimalfilter nach Wiener und Kolmogoroff), Springer, 2001 (in German) M. S.Hayes: Statistical Digital Signal Processing and Modeling – Chapter 7 (Wiener Filtering), Wiley, 1996 S. Haykin: Adaptive Filter Theory – Chapter 2 (Wiener Filters), Prentice Hall, 2002

Noise suppression: U. Heute: Noise Suppression,

in E. Hänsler, G. Schmidt (eds.), Topics in Acoustic Echo and Noise Control, Springer, 2006

Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter

Slide 7

Principle of Orthogonality

•Derivation Derivation during the lecture …

Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter

Slide 8

Principle of Orthogonality

•A Deterministic Example Derivation during the lecture …

Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter

Slide 9

Wiener Solution

•Time-Domain Solution Derivation during the lecture …

Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter

Slide 10

Time-Domain Solution

•Example – Part 1 Desired signal: Sine wave with known frequency but with unknown phase, not correlated with noise

FIR filter of order 31, delayless estimation at filter output

+

Noise:

White noise with zero mean, not correlated with desired signal

Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter

Slide 11

Time-Domain Solution

•Example – Part 2 Wiener solution:

Desired signal and noise are not correlated and have zero mean:

Simplification according to the assumptions above:

Wiener solution (modified):

Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter

Slide 12

Time-Domain Solution

•Example – Part 3 Input signals: Excitation: sine wave Noise: white noise

Assumptions: Knowledge

of the mean values and of the autocorrelation functions of the desired and of the undesired signal Desired signal and noise are not correlated Desired signal and noise have zero mean 32 FIR coefficients should be used by the filter

Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter

Slide 13

Time-Domain Solution

•Example – Part 4 After a short initialization time the noise suppression

performs well (and does not introduce a delay!)

Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter

Slide 14

Error Surface

•Derivation – Part 1 Derivation during the lecture …

Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter

Slide 15

Error Surface

•Derivation – Part 2 Error surface for:

Properties: Unique minimum (no local minima) Error

surface depends on the correlation properties of the input signal

Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter

Slide 16

Frequency-Domain Solution

•Derivation Derivation during the lecture …

Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter

Slide 17

Applications

•Noise Suppression – Part 1 Frequency-domain Wiener solution (non-causal):

Desired signal = speech signal:

Desired signal and noise are orthogonal:

Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter

Slide 18

Applications

•Noise Suppression – Part 2 Frequency-domain solution:

Approximation using short-term estimations:

Practical approaches: Realization using a filterbank system (time-variant attenuation of subband signals)

Analysis filters with length of about 15 to 100 ms Frame-based processing with frame shifts between

1 and 20 ms The basic Wiener characteristic is usually „enriched“ with several extensions (overestimation, limitation of the attenuation, etc.)

Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter

Slide 19

Applications

•Noise Suppression – Part 3 Processing structure: Analysis filterbank

Synthesis filterbank

Input PSD estimation

Filter characteristic Noise PSD estimation

PSD = power spectral density Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter

Slide 20

Applications

•Noise Suppression – Part 4 Power spectral density estimation for the input signal:

Power spectral density estimation for the noise:

Estimation schemes using voice activity detection(VAD)

Tracking of minima of short-term power estimations

Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter

Slide 21

Applications

•Noise Suppression – Part 5 Schemes with voice activity detection:

Tracking of minima of the short-term power:

Constant slightly larger than 1 Bias correction

Constant slightly smaller than1

Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter

Slide 22

Applications

•Noise Suppression – Part 6 Problem: The short-term

power of the input signal usually fluctuates faster than the noise estimate – also during speech pauses. As a result the filter characteristic opens and closes in a randomized manner, with results in tonal residual noise (so-called musical noise).

Simple solution: By inserting a fixed overestimation

the randomized opening of the filter can be avoided. This comes, however, with a more aggressive attenuation characteristic that attenuates also parts of the speech signal.

Enhanced solutions: More

enhanced solutions will be presented in the lecture “Speech and Audio Processing – Audio Effects and Recognition” (offered next term by the “Digital Signal Processing and System Theory” team).

Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter

Slide 23

Applications

•Noise Suppression – Part 7 : Microphone signal : Output without overestimation : Output with 12 dB overestimation

Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter

Slide 24

Applications

•Noise Suppression – Part 8 Limiting the maximum attenuation: For

several application the original shape of the noise should be preserved (the noise should only be attenuated but not completely removed). This can be achieved by inserting a maximum attenuation:

In addition, this attenuation limits can be varied slowly over

time (slightly more attenuation during speech pauses, less

attenuation during speech activity).

Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter

Slide 25

Applications

•Noise Suppression – Part 9 : Microphone signal : Output without attenuation limit : Output with attenuation limit

Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter

Slide 26

Applications

•Noise Suppression – Part 10

Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter

Slide 27

Adaptive Filters – Wiener Filter

•Summary and Outlook This week: Introduction and motivation Principle of orthogonality Time-domain solution Frequency-domain solution Application example: noise suppression

Next week: Linear Prediction

Digital Signal Processing and System Theory | Adaptive Filters | Wiener Filter

Slide 28