Optimal Design of Adaptive Normalized Matched Filter For Large Antenna Arrays Abla Kammoun, Romain Couillet, Frédéric Pascal, Mohamed Slim-Alouini
To cite this version: Abla Kammoun, Romain Couillet, Frédéric Pascal, Mohamed Slim-Alouini. Optimal Design of Adaptive Normalized Matched Filter For Large Antenna Arrays. 2016 IEEE Statistical Signal Processing Workshop (SSP), Jun 2016, Palma de Majorca, Spain. �10.1109/ssp.2016.7551722�. �hal-01633446�
HAL Id: hal-01633446 https://hal.archives-ouvertes.fr/hal-01633446 Submitted on 7 Apr 2020
HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.
OPTIMAL ADAPTIVE NORMALIZED MATCHED FILTER FOR LARGE ANTENNA ARRAYS Abla Kammoun1 , Romain Couillet2 , Fr´ed´eric Pascal2 , Mohamed-Slim Alouini1 King Abdullah University of Science and Technology (KAUST), Saudi Arabia1 CentraleSup´elec, Universit´e Paris-Saclay, Gif-sur-Yvette, France2 ABSTRACT This paper focuses on the problem of detecting a target in the presence of a compound Gaussian clutter with unknown statistics. To this end, we focus on the design of the adaptive normalized matched filter (ANMF) detector which uses the regularized Tyler estimator (RTE) built from N -dimensional observations x1 , · · · , xn in order to estimate the clutter covariance matrix. The choice for the RTE is motivated by its possessing two major attributes: first its resilience to the presence of outliers, and second its regularization parameter that makes it more suitable to handle the scarcity in observations. In order to facilitate the design of the ANMF detector, we consider the regime in which n and N are both large. This allows us to derive closed-form expressions for the asymptotic false alarm and detection probabilities. Based on these expressions, we propose an asymptotically optimal setting for the regularization parameter of the RTE that maximizes the asymptotic detection probability while keeping the asymptotic false alarm probability below a certain threshold. Numerical results are provided in order to illustrate the gain of the proposed detector over a recently proposed setting of the regularization parameter. 1. INTRODUCTION We consider the problem of detecting a complex signal vector p corrupted by an additive noise as: y = αp + x where y ∈ CN N -dimensional received vector, x stands for the noise clutter and α is a complex scalar modeling the unknown target amplitude. The signal detection problem is phrased as the following binary hypothesis test: H1 : y = αp + x (1) H0 : y = x. In order to account for the impulsive nature of the noise, we assume that the probability distribution of x belongs to the class of compound Gaussian distributions, that is x satisfies: x=
√
1 2 τ CN w
where τ is a positive scalar random variable, referred to as the texture in the parlance of radar detection, CN is the covariance matrix of the clutter, and w is a N × 1 normal distributed vector independent of τ . Theoretically speaking, the scalar τ can follow any positive distribution. However, in order to reflect the impulsive character of the noise, it makes more sense to assume its being heavy-tailed distributed. Several distributions can be considered, among them
we distinguish the t-student distribution, the Generalized Gaussian distribution, and the K-distribution, to name a few [1]. The design of an appropriate statistic to the above hypothesis test in (1) depends on the amount of knowledge that is available to the detector. If the clutter is Gaussian and the covariance matrix CN is known up to a scale factor, while α is unknown, the Generalized Likelihood Ratio (GLRT) for the detection problem in (1) results in the following test statistic: TN
∗ −1 y C p N q = q −1 ∗ y CN y yp∗ C−1 N p
This statistic corresponds to the normalized matched filter detector (NMF). It has been derived independently by several works, leading to many other alternative appellations like the constant false alarm (CFAR) matched subspace detector (MSD) [2], or the linear Quadratic GLRT (GLRT-LQ) detector [3]. It is worth pointing out that the optimality of this statistic in regards of the GLR principle holds only when the clutter is Gaussian. In case of non-Gaussian clutters, the optimal test statistic depends on the distribution of the texture τ . However, the use of TN for non-Gaussian clutters have often been considered as a reasonable alternative, for two major reasons. First, the clutter distribution cannot be easily acquired in practice, and second, it has been established that TN approximates the optimal statistic when N tends to infinity and that regardless of the underlying texture distribution [4]. The only requirement that imposes the use of TN is the estimation CN up to a scale factor, which is not viewed as a stringent requirement, being frequently considered by radar detection applications. In order to acquire an estimate of CN , we assume the detector uses n independent and identically distributed signal free observation, x1 , · · · , xn . Based on these observations, which are in passing often called secondary data, several estimators of the scatter matrix can be considered. The most appropriate ones are those belonging to the class of robust-estimators, being well-recognized for their high resilience to the impulsive nature of the clutter. Of interest in this paper is the regularized Tyler estimator defined as the unique solution to: n X xi x∗i ˆ N (ρ) = (1 − ρ) 1 C + ρIN ˆ −1 (ρ)xi n i=1 1 x∗i C N N n where ρ ∈ max(0, 1 − N ), 1 . This estimator can be thought of as a hybrid-robust shrinkage estimator reminding Tyler’s M-estimator of scale and Ledoit-Wholf’s shrinkage estimator. It will be thus coined Regularized Tyler Estimator (RTE). Besides its robustness, the RTE has many interesting features. First, it is well suited to the cases in which cN , N is large while standard robust-scatter esn timates are ill-conditioned if not defined when N > n. Second, it
defines a whole class of robust-scatter estimators for different settings of the regularization parameter ρ. By varying ρ, one can move from the unbiased Tyler estimator (ρ = 0) to the identity matrix (ρ = 1) which represents a crude guess for the unknown covariˆ N (ρ) in TN , we obtain the ance matrix CN . Substituting CN by C following test statistic: ∗ ˆ −1 y CN (ρ)p b q TN (ρ) = q ˆ −1 (ρ)y p∗ C ˆ −1 (ρ)p y∗ C N N The detector corresponding to the use of TbN (ρ) as a decision rule is coined adaptive normalized matched filter detector (ANMF) in reference to the step of estimating the clutter covariance matrix. Upon replacing in TN the unknown covariance matrix by the RTE, the question of how should the regularization parameter ρ be selected naturally arises. While easy-to-compute settings have been proposed in recent works [5, 6], one of the major criticisms to these choices is that they are performed regardless of the application under consideration. A more appropriate choice to the application under study would be to select the values of ρ that maximize the detection probability while keeping fixed the false alarm probability to a certain threshold. To achieve this goal, we need to characterize the distribution of TbN (ρ) under Hypothesis H0 and H1 . Obviously, this task might be unachievable if N and n are considered as fixed. To overcome this issue, we relax this assumption and consider the asymptotic regime in which N and n grow to infinity with N → c ∈ (0, ∞). This will allow us to leverage the recent results n of [7] regarding the asymptotic behaviour of quadratic forms with kernel the RTE. 2. OPTIMAL DESIGN OF THE ANMF DETECTOR A crucial step towards the design of the ANMF detector is to analyze its corresponding false alarm and the detection probabilities. For tractability, this task is carried out under the following asymptotic regime: Assumption A-1. We assume the following growth rate of the system dimensions: N, n → ∞ with cN =
N → c ∈ (0, ∞) n
Additionally, we assume that the secondary data x1 , · · · , xn satisfy the following: 1 2 Assumption A-2. For i ∈ {1, · · · , n}, xi = CN wi with:
• w1 , · · · , wn are N × 1 independent standard Gaussian random vectors with zero-mean and covariance IN , • CN ∈ CN ×N is such that lim sup kCN k < ∞ and 1 trCN = 1, N • lim inf N
1 N
p∗ CN p > 0.
constant invariant with N , the probability of detection converges to 1. In order to avoid getting such trivial statements, we shall assume that Γ = √rN for some r > 0 and α = aN −1/2 for some fixed a > 0, while kpk2 = N . As will be discussed next, our derivations are mainly based on the recent results concerning the first order and second order behaviour of the RTE. Particularly, it has been established in [8], that the RTE exhibits the same behaviour as: ˆ N (ρ) = S
n 1 1 1−ρ 1 X 12 2 C wi wi∗ CN + ρIN γN (ρ) 1 − (1 − ρ)cN n i=1 N
where γN (ρ) is the unique solution to: 1=
1 trCN (γN (ρ)ρIN + (1 − ρ)CN )−1 . N
More specifically, for any κ > 0, define Rκ = κ + max 1, 1 − c−1 , 1 , and let a and b be any random or deterministic vectors with unit norm. Then, ˆ −1 a.s. ∗ ˆ −1 N 1− sup a∗ C N (ρ)b − a SN (ρ)b −→ 0. ρ∈Rκ
A major consequence of the above convergence is that quadratic ˆ N (ρ) have the same first order and second orforms with kernel C ˆ N (ρ). Hence, the der behaviour of quadratic forms associated with S false alarm and detection probabilities can be derived by substituting ˆ N (ρ) by S ˆ N (ρ), leading to the asymptotically equivain TbN (ρ), C e lent test statistic TN (ρ) given by: ∗ ˆ −1 y SN (ρ)p e q . TN (ρ) = q ˆ −1 (ρ)y p∗ S ˆ −1 (ρ)p y∗ S N N The interest of this result is that, contrary to TbN (ρ), the statistics of TeN (ρ) can be successfully derived using standard results from random matrix theory. In doing so, we can thus obtain asymptotic closed-form expressions for the false alarm and detection probabilities. These results are stated in the following theorems: Theorem 1 (False alarm probability, [7]). Let m(−ρ) be the unique solution to the following equation: −1 c(1 − ρ) −1 m(−ρ) = ρ + trCN (IN + (1 − ρ)m(−ρ)CN ) N Let RRTE = κ + max(0, 1 − c−1 ), 1 . As N, n → ∞ with cN → κ c ∈ (0, ∞), 2 − 2 r r bRTE 2σ (ρ) N,RTE sup P TN (ρ) > √ |H0 − e → 0, N ρ∈RRTE κ
1 N
Note that the normalization trCN = 1 is not a limiting constraint since TbN is invariant to the scaling of CN . We aim at providing asymptotic expressions approximating for the false alarm Theilatter write more forh and detection i probabilities. h mally as P TbN > Γ|H0 and P TbN > Γ|H1 . A close inspection to the behaviour of TbN reveals that for all Γ > 0 fixed, the probability of false alarm converges to zero. Similarly, if α > 0 is taken as a
where ρ˜ = ρ ρ +
1−ρ 1 γN (ρ) 1−(1−ρ)c
−1
and
p∗ CN Q2N (˜ ρ)p 1 ∗ 2 p QN (˜ ρ)p N1 trCN QN (˜ ρ) 1 × 1 − c(1 − ρ˜)2 m(−˜ ρ)2 N1 trC2N Q2N (˜ ρ)
2 σN,RTE (ρ) ,
with QN (˜ ρ) , (IN + (1 − ρ˜)m(−˜ ρ)CN )−1 and.
Theorem 2 (Detection probability). Consider the setting and notations of Theorem 1. As N, n → ∞ with cN → c ∈ (0, ∞), r RTE b sup P TN (ρ) > √ |H1 N ρ∈RRTE κ r → 0, −E Q1 gRTE (p), σN,RTE (ρ)
the covariance matrix CN which is unknown to the detector. Acquiring a consistent estimate of fRTE (ρ) and σN,RTE based on the ˆ N is thus mandatory. This is the goal of the following available C propositions. Proposition 3 (Proposition 1 in [7]). For ρ ∈ max( 0, 1 − c−1 ,1 . N Define, ˆ −2 (ρ)p p∗ C
where the expectation is taken over the distribution of τ , σN,RTE (ρ) has the same expression as in Theorem 1 and q 1 − c(1 − ρ˜)2 m(−˜ ρ) N1 trC2N Q2N (˜ ρ) p gRTE (p) = 2 ∗ p CN QN (˜ ρ)p r 2 × a |p∗ QN (˜ ρ)p| . Nτ and Q1 is the Marcum Q-function. According to Theorem 1 and Theorem 2, TbN (ρ) behaves differently depending √ on whether a signal is present or not. In particular, under H0 , N TbN (ρ) behaves like a Rayleigh distributed random variate with parameter σN,RTE (ρ) while it becomes wellapproximated under H1 by a Rice distributed random variable with parameters gRTE (p) and σN,RTE (ρ). We will now discuss the choice of the regularization parameter ρ and the threshold r. Given the application under consideration, it is sensible to set ρ and r in such a way to keep the asymptotic false alarm probability equal to a fixed value η while maximizing the asymptotic probability of detection. From Theorem 1, one can easily see that the values of r and ρ that provide an asymptotic false alarm probability equal to η should satisfy: p r = −2 log η. σN,RTE (ρ) From these choices, we have to take those values that maximize the asymptotic detection which is given, according to Theorem 2, by: r Q1 gRTE (p), . σN,RT E (ρ) The second argument of Q1 should be kept fixed in order to ensure the required asymptotic false alarm probability. As the Marcum-Q function is an increasing function of its first argument, the optimization of the detection probability boils down to considering the following values of ρ: ρ ∈ argmax {fRTE (ρ)}
(2)
where:
1 2 fRTE (ρ) , 2 gRTE (p) 2a ∗ Let ρRTE be among the values satisfying (2). The maximal asymptotic detection probability that can be obtained while satisfying an asymptotic false alarm probability equal to η is thus given by: √ r∗ Pd,RTE = Q1 2afRTE (ρ∗RTE ), (3) σN,RTE (ρ∗RTE ) where
1 − ρ ∗ ˆN p C−1 (ρ)p 1 2 N σ ˆN,RTE (ρ) = 2 (1 − cN + cN ρ) (1 − ρ) 2 2 and let σ ˆN,RTE (1) , limρ↑1 σ ˆN (ρ). Then, we have:
2 a.s. 2 σN,RTE (ρ) − σ ˆN,RTE (ρ) −→ 0.
sup ρ∈RRTE κ
Proposition 4. For ρ ∈ max 0, 1 − c−1 , 1 , let N 2 1 ˆ ˆ −1 tr C (ρ) − ρ fˆRTE (ρ) = p∗ C (ρ)p N N N ×
(1 − cN + cN ρ)2 ˆ −2 (ρ)p ˆ −1 (ρ)p − ρp∗ C p∗ C N N
and fˆRTE , limρ↑1 fˆRTE (ρ). Then, we have: a.s. sup fˆRTE (ρ) − fRTE (ρ) −→ 0. ρ∈RRTE κ
Since the results in Proposition 4 and Theorem 2 are uniform in ρ, we have the following corollary: Corollary 5. Let fˆRTE (ρ) be defined as in Proposition 4. Define ρˆ∗N as any value satisfying: n o ρˆ∗N ∈ arg max fˆRTE (ρ) . ρ∈RRTE κ
Then, for every r > 0, √ P N TN (ˆ ρ∗N ) > r|H1 n √ o a.s. − max P N TN (ρ) > r |H1 −→ 0. ρ∈RRTE κ
Gathering all the aforementioned results, it entails that an asymptotically optimal design procedure is given by the following steps: • First, set the regularization parameter to one of the values maximizing fˆRT E (ρ): n o ρˆ∗N ∈ arg max fˆRT E (ρ) ; E ρ∈RRT κ
• Second, set the threshold to rˆ p rˆ = σ ˆN,RT E (ˆ ρ∗N ) −2 log η where η is the required FAP.
∗ rRTE = σN,RTE (ρ∗RTE )
p −2 log η.
However, the optimization of fRTE (ρ) and the computation of σN,RTE are not possible in practice, since their expressions features
The above design strategy is asymptotically optimal in the sense that it should guarantee for N and n large enough close-to-optimal detection performances.
3. NUMERICAL RESULTS In this section, we present some simulations validating our theoretical derivations and comparing our selection procedure of the regularization parameter with the setting proposed in [5]. In all our simulations, we consider that the secondary data are drawn from K distribution with zero mean, covariance CN and shape ν. Additionally, we assume that the array steering vector is given by:
1
p = a(fd ) ⊗ a(fs ) Detection Probability
where fd and fs denote respectively the normalized target Doppler frequency and the target spatial frequency [9], and a(fd ) ∈ CNp and a(fs ) ∈ CNa are the temporal and spatial array steering vectors given by: aNp (fd ) k = exp (2π(k − 1)fd ) , k = 1, · · · , Np
0.8
[aNa (fs )]` = exp (2π(` − 1)fs ) , ` = 1, · · · , Na .
i=1
where Nφ represents the total number of scatterers, fdi and fsi their corresponding doppler and spatial frequencies and α is a normalizing factor and ANp (fdi , fsi ) = aNp (fdi ) ⊗ aNa (fsi ). We carry out Monte Carlo simulations using 50 000 runs in order to represent the Receiver Operating Characteristics (ROC). For each run, we generate the primary signal y = αp + x and the secondary data x1 , · · · , xn . This experiment is carried out when Na = 4, Np = 32, fs = 0.5 fd = 0.2, η = 4.5 and n = 128. Under these circumstances, we compare the detection performances of the proposed detector with the one using the regularization setting of Ollila in [5]. Fig. 1 reports the obtained results. The red curve corresponds to the asymptotic detection probability given by (3). We note the superiority of the proposed detector despite the appearing discrepancy between the empirical and theoretical results. This discrepancy is obviously due to the number of samples and of antennas being too small to ensure the accuracy of the asymptotic results. In order to confirm this, we conduct again the same experiment when n = 320 and N = 320 (Na = 10, Np = 32). As seen from Fig. 2, a better accuracy is achieved while the gain over the setting of Ollila is maintained. 4. CONCLUSION This paper addressed the design of the ANMF detector that uses the RTE as a replacement of the unknown covariance matrix. A major question that naturally arises in this case concerns the setting of the regularization parameter. One major bottleneck towards the determination of the appropriate parameter lies in the difficulty to clearly characterize the false alarm and detection probabilities. In order to deal with this issue, we consider the regime in which the number of secondary observations and their dimensions grow together to infinity. This allows us to simplify the design procedure since in this asymptotic regime, the false alarm and detection probabilities converge to deterministic quantities, which we compute using results from random matrix theory. Based on these results, we proposed to set the regularization parameter to the value that maximizes the asymptotic detection probability while keeping the asymptotic false alarm probability below a certain threshold.
Proposed design Theory Design using ρˆollila
0.4
0.2
1
2
3
4
False alarm probability
5 ·10−2
Fig. 1. ROC curves for non Gaussian clutters when N = 128 (Na = 10, Np = 25), n = 128, fs = 0.5, fd = 0.2, a = 0.3
1
0.8 Detection Probability
with Np and Na being the number of pulses and that of sensors. Then N = Np Na . We assume that the clutter covariance matrix CN is given by: Nφ X 2 ∗ CN = αIN + σ ANp (fdi , fsi )ANp (fdi , fsi )
0.6
0.6 Proposed design Theory Design using ρˆollila
0.4
0.2
0
1
2
3
False alarm probability
4
5 ·10−2
Fig. 2. ROC curves for non-Gaussian clutters when N = 320 (Na = 10, Np = 32), n = 320, fs = 0.5, fd = 0.2, a = 0.3
5. REFERENCES [1] E. Olilla, D. E. Tyler, V. Koivunen, and H. V. Poor, “Complex Elliptically Symmetric Distributions: Survey, New Results and Applications,” IEEE Trans. Signal Process., vol. 60, no. 11, Nov. 2012. [2] L. L. Scharf and B. Friedlander, “Matched Subspace Detectors,” IEEE Trans. Signal Process., vol. 42, no. 8, pp. 2146–2157, 1994. [3] F. Gini, “Sub-Optimum Coherent Radar Detection in a Mixture of K-Distributed and Gaussian Clutter,” IEEE Proceeedings in Radar, Sonar and Navigation, vol. 144, no. 1, Feb. 1997. [4] E. Conte, M. Lops, and G. Ricci, “Asymptotically Optimum Radar Detection in Compound-Gaussian Clutter,” IEEE Trans. Aerosp. Electron. Syst., pp. 617–625, Apr. 1995. [5] E. Ollila and D. E. Tyler, “Regularized M-estimators of Scatter Matrix,” IEEE Trans. Signal Process., vol. 62, no. 22, Nov. 2014. [6] Y. Chen, A. Wiesel, Y. C. Eldar, and A. O. Hero, “Shrinkage Algorithms for MMSE Covariance Estimation,” IEEE Trans. Signal Process., vol. 58, no. 10, Oct. 2010. [7] R. Couillet, A. Kammoun, and F. Pascal, “Second Order Statistics of Robust Estimators of Scatter. Application to GLRT Detection for Elliptical Signals,” Submitted to Journal of Multivariate Analysis, 2014, http://arxiv.org/abs/1410.0817. [8] R. Couillet and M. McKay, “Large Dimensional Analysis and Optimization of Robust Shrinkage Covariance Matrix Estimators,” Journal of Multivariate Analysis, vol. 31, pp. 99–120, 2014. [9] F. Bandiera, D. Orlando, and G. Ricci, Advanced Radar Detection Schemes under Mismatched Signal Models. New Yorl: Morgan and Claypool: Synthesis Lectures on Signal Processing N, 8, 2009.
