Adaptive Filters Linear Prediction

Digital Signal Processing and System Theory ... Statistical Digital Signal Processing and Modeling ... Estimation of the Autocorrealtion Function –Par...

0 downloads 159 Views 2MB Size
Adaptive Filters – Linear Prediction

Gerhard Schmidt Christian-Albrechts-Universität zu Kiel Faculty of Engineering Institute of Electrical and Information Engineering Digital Signal Processing and System Theory Slide 1

Contents of the Lecture Today:  Source-filter model for speech generation  Derivation

of linear prediction  Levinson-Durbin recursion  Application example

Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction

Slide 2

Linear Prediction Motivation

Human Speech Generation and Appropriate Modelling

Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction

Slide 3

Motivation Speech Production Filter part

Principle: Nasal cavity Pharynx cavity

Vocal cords

Mouth cavity



An airflow, coming from the lungs, excites the vocal cords for voiced excitation or causes a noise-like signal (opened vocal cords).

 The mouth, nasal, and pharynx

Lung volume

Source part

cavity are behaving like controllable resonators and only a few frequencies (called formant frequencies) are not attenuated.

Muscle force

Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction

Slide 4

Speech Production Source-Filter Model Vocal tract filter

Fundamental frequency Impulse generator Source part of the model

Noise generator

¾(n)

Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction

Filter part of the model

Slide 5

Literature Books Basic text:  E. Hänsler / G. Schmidt: Acoustic Echo

and Noise Control – Chapter 6 (Linear Prediction),

Wiley, 2004

Speech processing:  P. Vary, R. Martin: Digital Transmission of Speech Signals

– Chapter 2 (Models of Speech

Production and Hearing), Wiley 2006  J. R. Deller, J. H. l. Hansen, J. G. Proakis: Discrete-Time Processing of Speech Signals – Chapter 3 (Modeling Speech Production), IEEE Press, 2000

Further basics: E. Hänsler: Statistische Signale: Grundlagen und Anwendungen – Chapter 6 (Linearer Prädiktor), Springer, 2001 (in German)  M. S. Hayes: Statistical Digital Signal Processing and Modeling – Chapters 4 und 5 (Signal Modeling, The Levinson Recursion), Wiley, 1996 

Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction

Slide 6

Linear Prediction Basics

Basics of Linear Prediction

Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction

Slide 7

Linear Prediction Basic Approach Estimation of the current signal sample on the basis of the previous

samples:

Linear prediction filter

With: 

: estimation of



: predictor coefficients



: length / order of the predictor

Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction

Slide 8

Linear Prediction Optimization Criterion Optimization: Estimation of the filter coefficients

such that a cost function is optimized.

Cost function:

Structure:

Linear prediction filter Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction

Slide 9

Linear Prediction „Whitening“ Property Cost function:

 Strong frequency

components will be attenuated most (due to Perceval).  This leads to a spectral

„decoloring“ (whitening) of the signal.

Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction

Slide 10

Linear Prediction Inverse Filter Structure Properties:  The inverse predictor

error filter is an all-pole filter  The cascaded structure

FIR filter (sender)

All-pole filter (receiver)

Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction

- consisting of a predictor error filter and an inverse predictor error filter can be used for lossless data compression and for sending and receiving signals.

Slide 11

Linear Prediction Computing the Filter Coefficients Derivation during the lecture …

Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction

Slide 12

Linear Prediction Examples – Part 1 First example:  Input signal

: white noise with variance  Prediction order:  Prediction of the next sample:

(zero mean)

This leads to: , respectively

, what means the no prediction is possible or – to be precise – the best prediction is the mean of the input signal which is zero.

Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction

Slide 13

Linear Prediction Examples – Part 2 Second example:  Input signal

: speech, kHz

sampled at  Prediction order:  Prediction of the next sample:

Single optimization of the filter coefficients for the entire signal sequence New adjustment of the filter coefficients every 64 samples

Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction

Slide 14

Linear Prediction Estimation of the Autocorrealtion Function – Part 1 Problem: Ensemble averages are usually not known in most applications.

Solution: Estimation of the ensemble averages by temporal averaging (ergodicity assumed):

Assumption: is a representative signal of the underlying random process.

Estimation schemes: A few schemes for estimating an autocorrelation function exist. These scheme differ in the properties (such as unbiasedness or positive definiteness) that the resulting autocorrelation gets significantly.

Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction

Slide 15

Linear Prediction Estimation of the Autocorrealtion Function – Part 2 Example: „Autocorrelation method“: Computed according to:

Properties:  The estimation is biased, we achieve:  But we obtain:

 The resulting (estimated) autocorrelation matrix is positive definite.  The resulting (estimated) autocorrelation matrix has Toeplitz structure.

Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction

Slide 16

Linear Prediction Levinson-Durbin Recursion – Part 1 Problem: The solution of the equation system has – depending on how the autocorrelation matrix is estimated – a complexity proportional to or , respectively. In addition numerical problems can occour if the matrix is ill-conditioned.

Goal: A robust solution method that avoids direct inversion of the matrix

.

Solution Exploiting the Toeplitz structure of the matrix

:

 Recursion over the filter order  Combining forward and backward prediction

Literature:  J. Durbin: The Fitting of Time Series Models, Rev. Int. Stat. Inst., no. 28, pp. 233

- 244, 1960  N. Levinson: The Wiener RMS Error Criterion in Filter Design and Prediction, J. Math. Phys., no. 25, pp. 261 - 268, 1947 Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction

Slide 17

Linear Prediction Levinson-Durbin Recursion – Part 2 (Backward Prediction) Equation system of the forward prediction:

Changing the equation order:

Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction

Slide 18

Linear Prediction Levinson-Durbin Recursion – Part 3 (Backward Prediction) After rearranging the equations:

Changing the order of the elements on the right side:

Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction

Slide 19

Linear Prediction Levinson-Durbin Recursion – Part 4 (Backward Prediction) After changing the order of the elements on the right side:

Matrix-vector notation:

Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction

Slide 20

Linear Prediction Levinson-Durbin Recursion – Part 5 (Backward Prediction) Matrix-vector notation:

Due to symmetry of the autocorrelation function:

Backward prediction by N samples:

Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction

Slide 21

Linear Prediction Levinson-Durbin Recursion – Part 6 (Derivation of the Recursion) Derivation during the lecture …

Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction

Slide 22

Linear Prediction Levinson-Durbin Recursion – Part 7 (Basic Structure of Recursive Algorithms) Estimated signal using a prediction filter of length :

Inserting the recursion

:

Innovation

Forward predictor of length N-1

Additional sample

Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction

Backward predictor of length N-1 Slide 23

Linear Prediction Levinson-Durbin Recursion – Part 8 (Basic Structure of Recursive Algorithms) Structure that shows the recursion over the order: Backward predictor of lenght N-1

Forward predictor of length N-1 Forward predictor of length N

In short form:

New estimation = old estimation + weighting * (new sample – estimated new sample)

Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction

Slide 24

Linear Prediction Levinson-Durbin Recursion – Part 9 (Recursive Computation of the Error Power) Derivation during the lecture …

Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction

Slide 25

Linear Prediction Levinson-Durbin Recursion – Part 10 (Summary) Initialization  Predictor:  Error

power (optional):

Recursion:  Reflection coefficient:

 Forward

predictor:

 Backward predictor:  Error

power (optional):

Condition for termination:  Numerical problems:

If is true, use the coefficients of the previous recursion and fill the missing coefficients with zeros.

 Order:

If the desired filter order is reached, stop the recursion.

Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction

Slide 26

Linear Prediction Matlab Demo

Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction

Slide 27

Linear Prediction Matlab Demo – Input Signal and Estimated Signal

Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction

Slide 28

Linear Prediction Matlab Demo – Error Signals

Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction

Slide 29

Adaptive Filters – Linear Prediction Summary and Outlook

This week:  Source-filter model for speech generation  Derivation

of linear prediction  Levinson-Durbin recursion  Application example Next week:  Adaptation algorithms – part 1

Digital Signal Processing and System Theory| Adaptive Filters | Linear Prediction

Slide 30