Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control Slide 20 Maximum Convergence Speed –Part 1 Adaptation Control For t...

Gerhard Schmidt Christian-Albrechts-Universität zu Kiel Faculty of Engineering Electrical Engineering and Information Technology Digital Signal Processing and System Theory Slide 1

Contents of the Lecture Today: Adaptation Control:  Introduction and Motivation  Prediction of the System Distance  Optimum Control Parameters

 Estimation Schemes  Examples

Slide 2

Application Example – Echo Cancellation Basics Application example:

Objective: Remove those components in the microphone signal that originate from the remote communication partner!

x(n)

Model:

x(n)

b h(n)

x(n)

b h(n)

b d(n)

h(n)

Echo cancellation filter

d(n) s(n)

b(n)

y(n)

e(n)

+ s(n)

b d(n)

e(n)

d(n)

+ +

y(n)

+ b(n)

Slide 3

Application Example – Echo Cancellation Basic Approach Model: The loudspeaker-enclosure-microphone (LEM) system is modelled as a linear (only slowly changing) system with finite memory.

Approach: Cancelling acoustic echoes by means of an adaptive filter with coefficients, operating at a sample rate kHz. For the adaptation of the filter the NLMS algorithm should be used.

Advantages and disadvantages: + In contrast to former approaches (loss controls) simultaneous speech activity in both communication directions is possible now. + The NLMS algorithm is a robust and computationally efficient approach. _ Compared to former solutions more memory and a larger computational load are required. _ Stability can not be guaranteed.

Slide 4

Application Example – Echo Cancellation NLMS-Algorithm Computation of the error signal (output signal of the echo cancellation filter):

Recursive computation of the norm of the excitation signal vector

Slide 5

Application Example – Echo Cancellation Convergence Examples – Part 1 Convergence without background noise and without local speech signals

Excitation signal

Local signal

Microphone signal

Error signal

Microphone and error power

Time in seconds Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Slide 6

Application Example – Echo Cancellation Convergence Examples – Part 2 Convergence with background noise but without local speech signals

Excitation signal

Local signal

Microphone signal

Error signal

Microphone and error power

Time in seconds Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Slide 7

Application Example – Echo Cancellation Convergence Examples – Part 3 Convergence without background noise but with local speech signals

Excitation signal

Local signal

(step size = 1) Microphone signal

Error signal

Microphone and error power

Time in seconds Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Slide 8

Application Example – Echo Cancellation Convergence Examples – Part 4 Convergence without background noise but with local speech signals

Excitation signal

Local signal

(step size = 0.1) Microphone signal

Error signal

Microphone and error power

Time in seconds Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Slide 9

Adaptation Control Literature Basic texts:  E. Hänsler / G. Schmidt: Acoustic Echo

and Noise Control – Chapter 7 (Algorithms for Adaptive Filters), Wiley, 2004

 E. Hänsler / G. Schmidt: Acoustic Echo

and Noise Control – Chapter 13 (Control of Echo Cancellation Systems), Wiley, 2004

Further details:  S. Haykin: Adaptive Filter Theory – Chapter 6 (Normalized Least-Mean-Square Adaptive

Filters), Prentice Hall, 2002  C. Breining, A. Mader: Intelligent Control

Strategies for Hands-Free Telephones, in E. Hänsler, G. Schmidt, Topics on Acoustic Echo and Noise Control – Chapter 8, Springer, 2006

Slide 10

Adaptation Control Control Approaches – Part 1 Scalar control approach:

Step size

Regularization

Vector control approach:

Slide 11

Adaptation Control Control Approaches – Part 2 Example for a sparse impulse response

Impulse response of the system to be identified

For such systems a vector based control scheme can be advantageous. Example for a vector step size

Coefficient index i

Slide 12

Adaptation Control How do we go on … Problem (echo cancellation performance during „double talk“) Analysis of the average system distance (taking local signals into account)

Derivation of an optimal step size (using non-measurable signals) Estimation of the non-measurable signal components (leads to an implementable control scheme) Solution (robust echo cancellation due to step-size control) Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Slide 13

Adaptation Control Average System Distance – Part 1 Assumptions:  Adaptation using the NLMS algorithm (only step-size controlled) :

 White noise as excitation and (stationary) distortion:

x(n)  Statistical independence between filter vector

and excitation vector.

b h(n)

 Time-invariant system:

Definition of the average system distance:

h

s(n) b d(n)

e(n) Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

d(n)

+ +

y(n)

n(n)

+ b(n) Slide 14

Adaptation Control Average System Distance – Part 2 … Derivation during the lecture …

Slide 15

Adaptation Control Average System Distance – Part 3 Generic approach (control scheme with step size and regularization):

Result:

Contraction parameter

Expansion parameter

Slide 16

Adaptation Control Contraction and Expansion Parameters

Contraction parameter

:

 Range:  Desired:

as small as possible

 Determines

the speed of convergence without distortions

Expansion parameter

:

Opposite to each other – a common solution (optimization) Has to be found!

 Range:  Desired:

as small as possible

 Determines

the robustness against distortions

Slide 17

Adaptation Control Influence of the Control Parameters Values for the contraction and expansion parameters for the conditions:

Expansion parameter

Contraction parameter

  Step size

Regularization

Regularization

Step size

Step size

Step size

Regularization

Regularization

Slide 18

Adaptation Control True and Prediction System Distance Boundary conditions of the simulation:

Excitation

 Excitation: white noise  Distortion: white noise  SNR:

Distortion

30 dB System distance (Simulation) (Simulation) (Theory) (Theory)

Iterations Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Slide 19

Adaptation Control Maximum Convergence Speed – Part 1 For the special case without any distortions

and with optimal control parameters for that case

we get

Meaning that the average system distance can be reduced per adaptation step by a factor of . As a result adaptive filters with a lower amount of coefficients converge faster than long adaptive filters.

Slide 20

Adaptation Control Maximum Convergence Speed – Part 2 If we want to know how long it takes to improve the filter convergence by 10 dB, we can make the following ansatz:

As on the previous slide we assumed an undisturbed adaptation process. By applying the natural logarithm we obtain

By using the following approximations for

and

we get

This means: At maximum speed of convergence it takes about 2N iterations until the average system distance is reduced by 10 dB. Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Slide 21

Adaptation Control The „10 dB per 2N“ Rule Boundary conditions of the simulation:

Average system distance

 Excitation: white noise  Distortion: white noise  SNR: 30 dB  Step size: 1  Different filter lengths

(500 and 1000) Average system distance

Iterations Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Slide 22

Adaptation Control Prediction of the Steady-State Convergence – Part 1 Recursion of the average system distance:

For

and appropriately chosen control parameters we obtain:

Slide 23

Adaptation Control Prediction of the Steady-State Convergence – Part 2 By inserting the results from the previous slide we obtain:

For the adaptation without regularization we get:

Slide 24

Adaptation Control Optimal Step Size – Motivation Remarks:

Average system distance (only step-size control)

With a large step size one can achieve a fast initial convergence, but only a poor steady-state performance. Estimated speed of convergence

With a small step size a good steady-state performance can be obtained, but only a slow initial convergence.

Solution: Utilization of a timevariant step-size.

Iterations

Slide 25

Adaptation Control Optimal Step Size – Derivation … Derivation during the lecture …

Slide 26

Adaptation Control Optimal Step Size – Example Average system distance

Boundary conditions of the simulation:

Step size = 1 Step size = 0.5 Step size = 0.25 Time-variant step size

 Excitation: white noise  Distortion: white noise  SNR: 30 dB  Filter length: 1000

coefficients

Computation of the step size:

Time-variant step size

with

Iterations Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Slide 27

Adaptation Control Estimation of the Optimal Step Size Approximation for the optimal step size:

For white excitation we get:

Ansatz:

Short-term power of the excitation signal

Estimated system distance

Short-term power of the error signal

Slide 28

Adaptation Control Estimation Procedures for the Optimal Step Size – Part 1 First order IIR smoothing with different time constants for rising and falling signal edges:

Basic structure:

Different time constants are used to achieve smoothing on one hand but also being able to follow sudden signal increments quickly on the other hand.

Slide 29

Adaptation Control Estimation Procedures for the Optimal Step Size – Part 2 Boundary conditions of the simulation:

Microphone signal

 Excitation: speech  SNR: about 20 dB  ¯r = 0:007; ¯f = 0:002  Sample rate: 8 kHz Estimated short-term power

Time in seconds Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Slide 30

Adaptation Control Estimation Procedures for the Optimal Step Size – Part 3 Estimating the system distance:

Problem: The coefficients

are not known.

Solution: We extend the system by an artificial delay of we have

samples. For that part of the impulse response

for With these so-called delay coefficients we can extrapolate the system distance:

Slide 31

Adaptation Control Estimation Procedures for the Optimal Step Size – Part 4 Structure of the system distance estimation:

Slide 32

Adaptation Control Estimation Procedures for the Optimal Step Size – Part 5 „Error spreading property“ of the NLMS algorithm:

System error vector (magnitudes) after 0 iterations

System error vector (magnitudes) after 500 iterations

System error vector (magnitudes) after 2000 iterations

System error vector (magnitudes) after 4000 iterations

Coefficient index

Slide 33

Adaptation Control Estimation Procedures for the Optimal Step Size – Part 6 Boundary conditions of the simulation:  Excitation: speech  Distortion: speech

Excitation signal

Local speech signal

 SNR during single talk:

30 dB

Measured and estimated system distance

 Filter length:

Measured system distance Estimated system distance

1000 coefficients

Step size

Iterations

Slide 34

Application Example – Echo Cancellation Convergence Examples – Part 5 Excitation signal

Local signal

Microphone signal

Error signal

Microphone and error power

For Comparison: Fixed step size 1.0 Fixed step size 0.1

Time in seconds

Slide 35

Application Example – Echo Cancellation Convergence Examples – Part 6 Fixed step size (1.0)

Microphone signal

Fixed step size (0.1) Controlled step size Short-term microphone and error power

Time in seconds Digital Signal Processing and System Theory| Adaptive Filters | Adaptation Control

Slide 36